This Gizmo Lets You Draw A UI On Paper, Then Turns It Into A Touch Screen

You’ll just have to watch the video to appreciate the amazing possibilities of the SketchSynth, created by Billy Keyes.

This Gizmo Lets You Draw A UI On Paper, Then Turns It Into A Touch Screen

You know those huge multichannel mixers–the massive boards that audio engineers manage during concerts to control everything from sound to lights? It’s the sort of highly specialized hardware that the average person would never come into contact with, because why would they? But what if you could just draw it?


That’s the idea behind the SketchSynth, by Carnegie Mellon student Billy Keyes. It allows you to draw your own specialized piece of sound hardware–in this case, a MIDI board–on any random piece of paper.

“Ever since I was little, I’ve been fascinated by control panels,” Keyes explains on his blog. “In elementary school, my neighbor and I would spend our bus rides pretending to operate incredible imaginary machines with cardboard controllers we drew, cut, and taped the night before. Even now, I pause when I see desks covered in gauges, switches, knobs, and buttons, wondering what they all do.”

His approach is a compromise between boundless childhood imagination and human factors of practicality: He designed three distinct controls that anyone could draw. So users create buttons by drawing circles, toggles by drawing rectangles, and sliders by drawing an elongated I.

A simple webcam picks up the shapes and sends them to a computer, then, a projector actually lays extra data on top of the drawing, like virtual nubs to control the sliders. This approach allows the user to draw something simple and stagnant, while light can animate additional content wherever it may be needed. The camera is able to track the user’s hands on the controls, not through fancy IR-based 3-D models, but just by sensing the color green in human skin.

And while this demo is clearly pretty basic, this principle could easily scale, adding all sorts of complex music visualizers to a user’s basic control diagram. Imagine drawing a rectangle to place an equalizer, or a square with a border to create a waveform monitor. Especially if you swapped out the system’s projector with a touch screen, the sky would be the limit, no specialized hardware required.

The result would be a lot like the design-it-yourself Beatsurfing app, but with all the mass-approachable charm of Draw Something. It makes me realize, the iPad doesn’t need a stylus; it needs a Sharpie.

[Hat tip: Creative Applications]

About the author

Mark Wilson is a senior writer at Fast Company. He started, a simple way to give back every day.