MIT Invents A Shapeshifting Display You Can Reach Through And Touch

The Tangible Media Group at MIT's Media Lab has unveiled a futuristic display made of atoms, not pixels.

We live in an age of touch-screen interfaces, but what will the UIs of the future look like? Will they continue to be made up of ghostly pixels, or will they be made of atoms that you can reach out and touch?

At the MIT Media Lab, the Tangible Media Group believes the future of computing is tactile. Unveiled today, the inFORM is MIT's new scrying pool for imagining the interfaces of tomorrow. Almost like a table of living clay, the inFORM is a surface that three-dimensionally changes shape, allowing users to not only interact with digital content in meatspace, but even hold hands with a person hundreds of miles away. And that's only the beginning.

Created by Daniel Leithinger and Sean Follmer and overseen by Professor Hiroshi Ishii, the technology behind the inFORM isn't that hard to understand. It's basically a fancy Pinscreen, one of those executive desk toys that allows you to create a rough 3-D model of an object by pressing it into a bed of flattened pins. With inFORM, each of those "pins" is connected to a motor controlled by a nearby laptop, which can not only move the pins to render digital content physically, but can also register real-life objects interacting with its surface thanks to the sensors of a hacked Microsoft Kinect.

To put it in the simplest terms, the inFORM is a self-aware computer monitor that doesn't just display light, but shape as well. Remotely, two people Skyping could physically interact by playing catch, for example, or manipulating an object together, or even slapping high five from across the planet. Another use is to physically manipulate purely digital objects. A 3-D model, for example, can be brought to life with the inFORM, and then manipulated with your hands to adjust, tweak, or even radically transform the digital blueprint.

But what really interests the Tangible Media Group is the transformable UIs of the future. As the world increasingly embraces touch screens, the pullable knobs, twisting dials, and pushable buttons that defined the interfaces of the past have become digital ghosts. The tactile is gone and the Tangible Media Group sees that as a huge problem.

"Right now, the things designers can create with graphics are more powerful and flexible than in hardware," Leithinger tells Co.Design. "The result is our gadgets have been consumed by the screen and become indistinguishable black rectangles with barely any physical controls. That's why BlackBerry is dying."

In other words, our devices have been designed to simulate affordances--the quality which allows an object to perform a function, such as a handle, a dial or a wheel--but not actually have them. Follmer says that's not the way it's supposed to be. "As humans, we have evolved to interact physically with our environments, but in the 21st century, we're missing out on all of this tactile sensation that is meant to guide us, limit us, and make us feel more connected," he says. "In the transition to purely digital interfaces, something profound has been lost."

The solution is programmable matter, and the inFORM is one possible interpretation of an interface that can transform itself to physically be whatever it needs to be. It's an interesting (and literal) analogue to skeuomorphism: while in the touch-screen age we have started rejecting interfaces that ape the look of real world affordances as "tacky" in favor of more pure digital UIs, the guys at the Tangible Media Group believe that interface of the future won't be skeuomorphic. They'll be supermorphic, growing the affordances they need on the fly.

Although the inFORM is primarily a sandbox for MIT to experiment with the tactile interfaces to come, it would be wrong to dismiss this project as mere spitballing. "We like to think of ourselves as imagining the futures, plural," Follmer says. "The inFORM is a look at one of them." But while the actual consumer implementation may very well differ, but both Follmer and Leithinger agree that tangible interfaces are coming.

"Ten years ago, we had people at Media Lab working on gestural interactions, and now they're everywhere, from the Microsoft Kinect to the Nintendo Wiimote," says Follmer. "Whatever it ends up looking like, the UI of the future won't be made of just pixels, but time and form as well. And that future is only five or ten years away. It's time for designers to start thinking about what that means now."

Add New Comment

90 Comments

  • Theart Korsten

    Incredible - was waiting for a face. like in the metal pin manual ones. But brilliant. Would it work with a face?

  • chris.soverow

    As with most leaps in technology in the past 100 years (e.g., the camcorder, VHS, the internet), this will only really take off once the porn industry is able to use it.

  • Debbie Cailler Walker

    Unlike the Digital Native, Katya, I am nearly 50 and LOVE this technology. I can't wait to see this apply in the future. Wouldn't it be a kick in the pants if it saved Katya's life someday. I love irony, and sarcasm, especially in the same sentence.

    It's a bit bulky at this stage, but like most technology, I'm sure hat it will shrink over time. I know that when I FaceTime with my 3 year old granddaughter we wish we could touch and talk. It would be very cool to have a board game attachment to our cell phones to play.

    Keep up the good work brains.

    Debbie

  • yeah people are lacking tactile sensation... why don't you just throw your smartphone out of the window and go pet your cat! Hate this modern technology( Im 27 years old btw, not 90

  • Aldous Huxley in Brave New World foresaw all this back in the 1950's. Edward Snowden is a prototype of his Noble Savage. Does anyone care about the courageous sacrifices he made? A few ACLU and Amnesty International do, but they are so few. In the end it will be the mob who will murder Snowden. Ortega's Mass Man.

  • Walter Oliver Neal

    Brilliant concept; but I can't help but wonder if the gist of the concept--the flatten pins--was not inspired by the Opening Spectacle at the 2012 Beijing Olympics.

  • jimmer

    A wonderful idea! It's easy to imagine how this could develop into something very interesting and usable.

    I made a portable media room to display an early gestural interface, in the form of the interactive video game "Bubbles" by Wolfgang Muench. Bubbles was written in 2000, a few years earlier than the work mentioned at MIT.

    I was commissioned by the newly emerging EMPAC of RPI in 2004 to design and build this as their first public art project. Here is a video of it installed at the Empire State Plaza in Albany, NY:

    http://icarusfurniture.com/pages/domemovie3.html

    Look at the Kinect and you can see how the idea has come a long way, and you can imagine how much further it will still go.

  • Love it! could someone build a version that has, instead of solid extrusions moving up and down, just a cube top for each pixel, so the cubes float in the air above the plane? combine the sam x-y grid layout with a second set of cubes that move down the negative "z" axis. (even below the virtual table top.) that allows floating shapes, rather than just the bas-relief shapes shown above. I've been trying to do that with physical sculptures for 20 years.

  • Mj Mj

    nice extended idea there. Do they have to be cubes? could it just be a plane, maybe one that could be tipped on an axis?