Let me see if I can write this post without referring to Minority Report — nope, not possible, since Imogen Heap’s musical gloves rely on the exact same idea as Tom Cruise’s image-remixing handgear. Having the gestural interface scheme applied to music, though, is a pretty neat live effect for a stage show — which Heap worked to great effect in a show at the TED conference in Edinburgh last month. Using gestures designed by her collaborator, Dr. Tom Mitchell of the University of West England at Bristol, Heap wove a digital melody out of thin air.
Sadly TED hasn’t released video of the performance (“They only recorded in mono and we need stereo for the full effect,” Mitchell tells Co.Design), but it’s easy to visualize how the gloves work. Motion sensors, gyroscopes, and accelerometers track Heap’s hand and finger positions in real time, streaming the data to a laptop which interprets the gestures as specific commands that it then applies as audio processing. The gestures themselves are cheekily intuitive: To sample a sound, Heap literally makes a grasping motion as if plucking its aural essence; to filter or dampen a sound, she clasps her hands together as if muffling a squealing cell phone.
Mitchell says that his system isn’t going to be available for anyone else but Heap anytime soon — it’s part of “a research project investigating methods for musical expression,” he told New Scientist. The rest of us working stiffs can make do with Toscanini, a scrappy open-sourced gestural interface device designed for dancers and musicians.