This Gizmo Lets You Draw A UI On Paper, Then Turns It Into A Touch Screen

You’ll just have to watch the video to appreciate the amazing possibilities of the SketchSynth, created by Billy Keyes.

You know those huge multichannel mixers—the massive boards that audio engineers manage during concerts to control everything from sound to lights? It’s the sort of highly specialized hardware that the average person would never come into contact with, because why would they? But what if you could just draw it?

That’s the idea behind the SketchSynth, by Carnegie Mellon student Billy Keyes. It allows you to draw your own specialized piece of sound hardware—in this case, a MIDI board—on any random piece of paper.

"Ever since I was little, I’ve been fascinated by control panels," Keyes explains on his blog. "In elementary school, my neighbor and I would spend our bus rides pretending to operate incredible imaginary machines with cardboard controllers we drew, cut, and taped the night before. Even now, I pause when I see desks covered in gauges, switches, knobs, and buttons, wondering what they all do."

His approach is a compromise between boundless childhood imagination and human factors of practicality: He designed three distinct controls that anyone could draw. So users create buttons by drawing circles, toggles by drawing rectangles, and sliders by drawing an elongated I.

A simple webcam picks up the shapes and sends them to a computer, then, a projector actually lays extra data on top of the drawing, like virtual nubs to control the sliders. This approach allows the user to draw something simple and stagnant, while light can animate additional content wherever it may be needed. The camera is able to track the user’s hands on the controls, not through fancy IR-based 3-D models, but just by sensing the color green in human skin.

And while this demo is clearly pretty basic, this principle could easily scale, adding all sorts of complex music visualizers to a user’s basic control diagram. Imagine drawing a rectangle to place an equalizer, or a square with a border to create a waveform monitor. Especially if you swapped out the system’s projector with a touch screen, the sky would be the limit, no specialized hardware required.

The result would be a lot like the design-it-yourself Beatsurfing app, but with all the mass-approachable charm of Draw Something. It makes me realize, the iPad doesn’t need a stylus; it needs a Sharpie.

[Hat tip: Creative Applications]

Add New Comment


  • Philip Shook

    Think it would be a great idea to put your Twitter handle on your tweets.  So when we tweet an article your handle shows in the tweet.

  • mobileHolly

    This is beautiful. It's going to let designers' and coders' dreams reality so much faster if it works.

  • Steve

    I could see a feature being added to Beatsurfing to mimic this sort of creation: using the iPad's camera,one could take a picture of a hand-drawn interface, then build a Beatsurfing interface to match using image analysis.

  • gagarine

    I was a sound technician during my study to make money. I can tell you the complexity is not in the interface and I will definitely sketch it in illustrator or UI design tools so I can copy past!
    A mixer is hundred time the same track... so know I don't want to draw hundred time the same things. Also a good mixer is about alignment, color, millimeter so you know where is what.

    But again what is hard to do in a mixer table is *not* the interface, but the connection between the track. Witch digital mixer is different as you have different screen (the mixer, effect, ...) and you have to passe from one to an other.. so again paper doesn't work.