Co.Design

Fresh From TED: A Mind-Blowing App That Could Remake Mobile Retail

Jinha Lee, an MIT Media Lab alum, gives Co.Design a first look at an invention that allows you to try on clothes virtually.

To be perfectly honest, Jinha Lee—the MIT Media Lab alumnus and TED Fellow who you may know best for inventing interactive floating pixels—didn’t even want to share this clip. He thought it might be too brief. But when we heard him describe the idea, we insisted. And we’re really glad we did.

What You Click Is What You Wear, which Lee developed alongside Daewung Kim, is a means to try on clothing virtually. It’s not intended to be a shopping platform, but rather a working prototype of what’s possible “when information lives in our space.”

What you’re actually looking at is an arm behind a transparent display, but the system could just as easily work as a HUD in something like Google Glass. The real-time simulation looks so accurate because the system tracks motion in two ways. A depth camera, like a Kinect, follows the general position of your body in space. Meanwhile, a phone, which the user holds, passes along gyroscope readings (the rotational position of your appendage).

“With just a camera, you can’t rotate your arm. The watch will never follow you,” Lee explains. “But if you’re holding a mobile phone, you can get the orientation.”

Bigger picture, the project is grounded in progressive philosophy. Of course we’ve seen some decent augmented-reality apps that use taped-on QR code markers. But when you consider the practical future of wearables, in which several discrete sensors will already be tracking various parts of your body at all times, it’s a natural evolution that some umbrella systems will be able to combine many of these measurements to produce simulations of incredible fidelity. (Lee points out that this idea of combining phone sensors with camera sensors seems to have been created by Microsoft Research’s Andy Wilson.)

But I think what makes this young project so fun is its dead-simple interaction model. Click a single button on your phone and, presto, there’s a watch on your arm. It’s like magic.

“The inspiration behind these projects started when I was young. Growing up, my mother was a paper artist doing origami,” he recounts. “So I witnessed this beautiful connection of information display and how physical information is made into a shape."

Then he pauses for a moment.

"I wish I said that on stage at TED," he adds.

See more here.

[Image: Phone via Shutterstock]

Add New Comment

4 Comments

  • siddmaini

    I am not impressed with this technology so far. As mentioned in a previous comment, how is one expected to really feel the device using this virtual technology? I do not see any kind of haptic or sensory feedback whatsoever. This technology is 'cool' at best unless somebody can find a way to integrate it with our lives and not the other way round by forcing us to or rather change to use it.

  • Eric Flood

    This is garbage..that watch looks horrible. It isn't always about how things look either..feel, weight, etc. 

    Virtual living isn't real living. I hope this doesn't take off. 

  • Guest-Eman

    Google glass and this would app would be completely feasible. Welcome to virtual living.