To Interact With This Interface, Just Smile

In this uncanny prototype, you grin to open a door. It’s a glimpse of how machine learning is learning to read our emotions.

The smile: It’s a universal, biological expression of happiness and joy. But what if your smile–or frown–was the basis for more than just how you interacted with your peers? What if certain elements of your environment required you to communicate with your facial expressions?


University of London masters student Freddie Hong is exploring how our faces–and specifically, our smiles–could serve as an interface. For a class on physical computing at Goldsmiths College, Hong built a connected door that only opens when you’re smiling,  thanks to facial recognition software. “The focus of this project was to provide the audience an experience of what it feels like to have your emotions read by the computer,” he tells Co.Design in an email.

[Image: courtesy Freddie Hong]
While developing the project and testing out the door, Hong noticed that being forced to smile made some users uncomfortable. But instead of tweaking the interface to make the smile more like a “button” that users could quickly “push,” he actually lengthened the time it took to read each user’s smile. This intentionally compounded the feeling of awkwardness–and drew attention to the fact that the machine is analyzing the user’s face.

The Smiling Door points to the next wave of emotive technology, which will use facial recognition software to analyze the difference between a smile, a grimace, and a frown. That’s something the startup Affectiva, an offshoot of MIT Media Lab that uses advanced facial recognition to track and analyze emotions, is already doing. Microsoft, too, has its own Emotion API that developers can use to add emotion-based context to their products. Embedding emotional awareness into our products seems like the logical next step for UX design–regardless of how invasive or manipulative it may be.

[Photo: courtesy Freddie Hong]
It’s a paradigm that Hong takes to an extreme with his prototype, a vision of the future where we’re being read and analyzed by the machines that surround us at every moment. Still, could there be an upside to having to fake a smile to get through the Smiling Door? Studies have shown that even a forced smile can reduce stress and even make you happier. If you had to smile on your way to the office each morning, maybe a long commute wouldn’t feel quite so horrendous.

But it would mean allowing our emotions to be manipulated by unthinking algorithms that were likely developed to make someone else money. Either way, smiling for the camera would take on an entirely new meaning.

About the author

Katharine Schwab is an associate editor at Co.Design based in New York who covers technology, design, and culture.