MIT Invents A Shapeshifting Display You Can Reach Through And Touch

The Tangible Media Group at MIT's Media Lab has unveiled a futuristic display made of atoms, not pixels.

We live in an age of touch-screen interfaces, but what will the UIs of the future look like? Will they continue to be made up of ghostly pixels, or will they be made of atoms that you can reach out and touch?

At the MIT Media Lab, the Tangible Media Group believes the future of computing is tactile. Unveiled today, the inFORM is MIT's new scrying pool for imagining the interfaces of tomorrow. Almost like a table of living clay, the inFORM is a surface that three-dimensionally changes shape, allowing users to not only interact with digital content in meatspace, but even hold hands with a person hundreds of miles away. And that's only the beginning.

Created by Daniel Leithinger and Sean Follmer and overseen by Professor Hiroshi Ishii, the technology behind the inFORM isn't that hard to understand. It's basically a fancy Pinscreen, one of those executive desk toys that allows you to create a rough 3-D model of an object by pressing it into a bed of flattened pins. With inFORM, each of those "pins" is connected to a motor controlled by a nearby laptop, which can not only move the pins to render digital content physically, but can also register real-life objects interacting with its surface thanks to the sensors of a hacked Microsoft Kinect.

To put it in the simplest terms, the inFORM is a self-aware computer monitor that doesn't just display light, but shape as well. Remotely, two people Skyping could physically interact by playing catch, for example, or manipulating an object together, or even slapping high five from across the planet. Another use is to physically manipulate purely digital objects. A 3-D model, for example, can be brought to life with the inFORM, and then manipulated with your hands to adjust, tweak, or even radically transform the digital blueprint.

But what really interests the Tangible Media Group is the transformable UIs of the future. As the world increasingly embraces touch screens, the pullable knobs, twisting dials, and pushable buttons that defined the interfaces of the past have become digital ghosts. The tactile is gone and the Tangible Media Group sees that as a huge problem.

"Right now, the things designers can create with graphics are more powerful and flexible than in hardware," Leithinger tells Co.Design. "The result is our gadgets have been consumed by the screen and become indistinguishable black rectangles with barely any physical controls. That's why BlackBerry is dying."

In other words, our devices have been designed to simulate affordances--the quality which allows an object to perform a function, such as a handle, a dial or a wheel--but not actually have them. Follmer says that's not the way it's supposed to be. "As humans, we have evolved to interact physically with our environments, but in the 21st century, we're missing out on all of this tactile sensation that is meant to guide us, limit us, and make us feel more connected," he says. "In the transition to purely digital interfaces, something profound has been lost."

The solution is programmable matter, and the inFORM is one possible interpretation of an interface that can transform itself to physically be whatever it needs to be. It's an interesting (and literal) analogue to skeuomorphism: while in the touch-screen age we have started rejecting interfaces that ape the look of real world affordances as "tacky" in favor of more pure digital UIs, the guys at the Tangible Media Group believe that interface of the future won't be skeuomorphic. They'll be supermorphic, growing the affordances they need on the fly.

Although the inFORM is primarily a sandbox for MIT to experiment with the tactile interfaces to come, it would be wrong to dismiss this project as mere spitballing. "We like to think of ourselves as imagining the futures, plural," Follmer says. "The inFORM is a look at one of them." But while the actual consumer implementation may very well differ, but both Follmer and Leithinger agree that tangible interfaces are coming.

"Ten years ago, we had people at Media Lab working on gestural interactions, and now they're everywhere, from the Microsoft Kinect to the Nintendo Wiimote," says Follmer. "Whatever it ends up looking like, the UI of the future won't be made of just pixels, but time and form as well. And that future is only five or ten years away. It's time for designers to start thinking about what that means now."

Add New Comment

76 Comments

  • Aldous Huxley in Brave New World foresaw all this back in the 1950's. Edward Snowden is a prototype of his Noble Savage. Does anyone care about the courageous sacrifices he made? A few ACLU and Amnesty International do, but they are so few. In the end it will be the mob who will murder Snowden. Ortega's Mass Man.

  • Walter Oliver Neal

    Brilliant concept; but I can't help but wonder if the gist of the concept--the flatten pins--was not inspired by the Opening Spectacle at the 2012 Beijing Olympics.

  • jimmer

    A wonderful idea! It's easy to imagine how this could develop into something very interesting and usable.

    I made a portable media room to display an early gestural interface, in the form of the interactive video game "Bubbles" by Wolfgang Muench. Bubbles was written in 2000, a few years earlier than the work mentioned at MIT.

    I was commissioned by the newly emerging EMPAC of RPI in 2004 to design and build this as their first public art project. Here is a video of it installed at the Empire State Plaza in Albany, NY:

    http://icarusfurniture.com/pages/domemovie3.html

    Look at the Kinect and you can see how the idea has come a long way, and you can imagine how much further it will still go.

  • Love it! could someone build a version that has, instead of solid extrusions moving up and down, just a cube top for each pixel, so the cubes float in the air above the plane? combine the sam x-y grid layout with a second set of cubes that move down the negative "z" axis. (even below the virtual table top.) that allows floating shapes, rather than just the bas-relief shapes shown above. I've been trying to do that with physical sculptures for 20 years.

  • Mj Mj

    nice extended idea there. Do they have to be cubes? could it just be a plane, maybe one that could be tipped on an axis?

  • James Simpson

    Finally, a way to handle a hot cup of coffee (though now we have to create a way to not drink it and still get the benefits). Or, now Superman can FINALLY play around with a ball made of Kryptonite.

  • There are certainly a lot of different elements at play here. For actually designing the hardware this looks like a pretty healthy mix of electrical and mechanical engineering. As it likely required custom boards, the electrical engineering likely would have leaned more towards work done by computer engineers, but this would depend on your university as how they classify degrees, some may offer a degree that is electrical and computer engineering as a single program. As far as computer science is concerned, new interfaces like this certainly open up new software possibilities, designing interfaces on new platforms like this could be aided by a focus on human computer interaction, animation/graphics and software engineering as part of a computer sciences degree.

  • Si yo estudio Ciencias de la Computacion puedo hacer eso o me conviene estudiar otra cosa como ingenieria de software . Por favor necesito su ayuda . Muchas gracias . Saludos desde latinoamerica .

  • Hola "estudiante", La creación tecnológica en nuestros días es mas complicada y avanzada que cualquier programa universitario, pero lo mas importante es quien eres tu? Puedes "ver" en 3 dimensiones?, te resultan fáciles las matemáticas?, tienes habilidades mecánicas?, cual es la especialidad de la universidad en la que puedes estudiar? Cuales son tus posibilidades económicas? Esta es una de las decisiones mas importantes de tu vida y tienes que ser realista y conectar contigo mismo y con tus condiciones, si un proyecto como este te fascina debes ser una persona creativa y el primer problema que debes resolver es como enfocar tu imaginación el camino, cada vez mas complejo, de tu realidad inmediata. Exitos.