How many gadgets do you tote around at any given time? Your smartphone, of course. But also maybe a tablet or an e-reader or a fitness band or a second smartphone. (Or perhaps all of the above.) That’s a lot of stuff to carry. Worse, you waste time toggling between devices and navigating the quirks of each one. Imagine instead having just a single gadget that you deform physically into different shapes to suit your needs.
That's the kind of smartphone that researchers at Hasselt University iMinds in Belgium are building. Called “Paddle,” their prototype phone, is designed around engineering principles derived from the popular 3-D Rubik’s Magic Puzzle. It's a folding plate puzzle consisting of a loop of eight square shaped tiles that can be transformed in a variety of surprising ways. Studying the puzzle over a period of seven months enabled the Hasselt researchers to create a phone that could be quickly transformed into various shapes in a few simple steps.
"At the moment our Paddle prototype supports around 15 different shapes but this number increases every day as we are including more and more shapes of the original Rubik’s Magic puzzle," says Raf Ramakers, a PhD Student in Human Computer Interaction at the University. "When unfolding Paddle completely it is nearly the size of an iPad, but when folding it up, it can become smaller than an iPhone."
For instance, you might transform it into a book, whose pages you can leaf through to read. Need to scroll through a list? Not a problem. Simply turn your phone into a bracelet and roll through individual links to scroll through various list items. Want to go for a jog? Turn it into an armband. Instead of carrying around an arsenal of devices, you’d just need one.
Paddle is more than just a matter of convenience, though. Deformable phones like PaperPhone explore the idea of using bends and folds to physically control the device; a prototype used different types of bending gestures to make calls, navigate menus, and more. But the gestures don't correspond to real-world controls. That's where Paddle differs. Users can physically bend the phone in ways that resemble real-world behaviors, with input and output becoming virtually indistinguishable. Ideally, that enables a seamless interaction with the device; we no longer need to learn specific movements to control it.
"For example, when scrolling through elements with Paddle, the user just uses the ring form-factor," Ramakers explains, referring to a shape that resembles a prayer bead necklace; you hold it in your hands and just roll through it. "He is not even aware that he is scrolling through elements as in a traditional interface where the user explicitly interacts with a scrollbar," Ramakers says. "With Paddle it even becomes hard to think about physical-digital or input-output as separate components."
Paddle's current design makes use of two external components: an optical tracking system and a projector. The tracking system consists of eight infrared cameras that illuminate the room. The cameras capture light reflected off small infrared reflective markers (like the material used in reflectors on bikes). The system calculates the position of every reflective marker in 3-D by combining images from all the cameras. The user's fingers are also tracked to enable touch interaction on the device.
The projector maps the user interface onto the device and also distorts the device in response to the user's movements. The team plans to create a prototype that's entirely self-contained by replacing the external tracking system with tiny integrated displays like O-LED and E Ink displays that are sensitive to the user’s movements. They expect to have a working prototype in another 12 to 18 months. Adding electronics to it Ramakers says, won't limit Paddle's transformational capabilities.
Currently Paddle provides visual cues to the user to communicate how the device can be transformed, such as highlighting regions to show where fingers go or using arrows to indicate the folding or unfolding directions. The team is studying how quickly people remember transformations to see whether their muscle memory can take over after a while, allowing users to perform transformations as unconsciously as they might drive a car or play an instrument. Kind of like how speed cubers solve Rubik's puzzles in a matter of seconds.
Paddle, Ramakers says, is a first step in developing devices that address the "dexterity of the human hand”—devices that combine the flexibility of touch screens with the physical qualities that real-world controls provide. Like grabbing something roughly to move it about or grasping something more precisely for finer control.
"In the real world, we often rely on our motor learning skills, for example, when driving a car, cycling, eating with Chinese sticks or playing sports or instruments," he explains. "In contrast, the devices that we work with nowadays do not allow for building up skill on a motor control level. When taking into account the time we spend interacting with our phones nowadays, one would expect us all to be veritable virtuosos on our devices, but current interaction techniques don't allow for this. We think this is a missed opportunity."
Paddle could also provide a welcome relief to those who struggle with touch screens, such as the elderly. With Paddle, they could manipulate a single mobile device—one that they operate with natural movements—greatly reducing the learning curve.
The researchers, including Ramakers, Johannes Schöning, and Kris Luyten, will be presenting their findings on the benefits of physical controls over traditional touch based interfaces at the ACM CHI Conference on Human Factors in Computing Systems conference (CHI 2014) in Toronto, Canada in late April.