Co.Design

Microsoft Unleashes The Kinect And Promptly Misses The Point

A new chapter in the strange history of the Kinect.

This year was the last Microsoft keynote at CES and by most accounts the bright spot was the expansion of the Kinect line with a new Kinect for Windows. The new SDK and hardware is aimed at commercial development with a new "Near Mode" that takes a technology designed to recognize a body across the livingroom and calibrates it to also recognize objects within a foot or two.

Microsoft hinted at the new development platform last year with the release of a non commercial SDK and a site called Kinect Effect, showing the Kinect being used in a variety of non-Xbox situations like medical therapies. They released a corresponding video in October.

Note the fine print: "Depictions are visionary". A little while ago, we complained about Microsoft’s lifeless concept videos and watching the Kinect Effect video, I get that same feeling.

Kinect’s voice and gesture input is often referred to as a "natural user interface" but there are very few situations where humans have been able to control tools by waving our hands in the air. A key part of human manipulation of the world is the manual aspect. We anchor ourselves to our environments through our sense of touch. If you buy Bret Victor’s complaint that touchscreen interfaces are Pictures Under Glass then the Kinect is Waving at Pictures Off In the Distance.

Take the parts with the musicians. I’ve played cello and percussion--both activities depicted in the film. In my experience, tactile feedback is a crucial part of playing the instrument. With it, you can play with your eyes closed. Without it, you end up with Theremin 2.0 and the Theremin is not an easy interface to learn. Similarly, when the guy is controlling a robot to disarm the car bomb, it seems clear that he’d be better off with a haptic interface, so he can feel what kind of resistance the plug is giving.

That said, the part with the surgeon swiping in the air to look at X-rays makes a lot of sense, but that’s only because he needs to keep things sterile--he can’t touch anything with his hands.


[Microsoft’s previous vision of the future]

This is not to say that a lot of amazing Kinect-based applications won’t find their way to market. For all that individuals make their way through the world by grasping and holding things, we interact with one another socially through the Kinect’s two inputs--voice and touch. The Kinect offers a way to live in society with machines. Think HAL 9000 before he went crazy. There will be great applications, but I doubt very much that the successful ones will look anything like the video.

It’s worth remembering the road that brought us here. Writing for Wired, Tim Carmody gives an excellent overview. Back in 2010, Microsoft representatives were making vaguely threatening comments about Kinect hackers as Adafruit sponsored a $3,000 bounty on the creation of open source drivers for the device. Microsoft quickly reversed course and it was later revealed that one of the Kinect’s designers, Johnny Lee had secretly sponsored the bounty after he failed to convince Microsoft to open up their drivers. In effect, Microsoft has had to be dragged, kicking and screaming, into the future.

Add New Comment

16 Comments

  • Gorj

    Perhaps I need more light, or a spot light, but . . .
    The Kinect in my living room has a difficult time reading my gestures when I am wearing greys, dark or light.  My feet are displayed on the screen an inch or two under the surface of the carpet no matter what I wear.. It has a real hard time when I face away from the camera, or turn all the way around.
    I'm not sure it is going to be able to read my playing violin or cello, air or real.  The Kinect surely doesn't like it when I sit down.  When playing classical or fiddle tunes I don't hold my violin still in 3D space.  I move a lot to the beat and feel.  I have a feeling that there would be a programming nightmare for air fiddle.
    I used to have a couple of games that came with my Creative black parallel interface web cam.  One, the most fun, had us hopping up and down to make a robot fly through space, then flapping our arms to make the pterodactyl fly.  The other you were hitting a ball around in 3-space.  I had Win95 at the time.
    The concepts are out there.  Bringing them to reality is in the future.

  • RaphaelM

    That's a big discussion simulating nature or not.

    The mouse was a revolution to the KB based UX, yes, but it still completely unnatural. Anyone that has a macbook can see how touch based interface like the magic trackpad is light years away in usability.

    The thing about copying nature's design is that it had hundreds of millions of years of evolution to back up those choices, voice and touch are safe bets. Imagining some entirely new is incredibly hard and has an amazing potential to flop/backfire.

  • Bob Heddle

    Hi Tim - you wrote a few critical words about our concept videos, but i'm going with  your comment about "amazing Kinect-based applications".  That's what I'm excited about, there are some truly amazing things that are coming to life thanks to the creativity of the developer community out there.  Our "Kinect Effect" video is really a tip-of-the-hat to those people who saw the potential before we did, and a summary of all the inspiring ideas we have seen.

    I read the comments - a lot of enthusiasm out there, we love to passion and creativity!

    - Bob Heddle, Kinect for Window team at Microsoft.

  • Shun

    The author deleted my post. :( So sad... I just showed him this video by The Verge: http://www.theverge.com/2011/1...

    Seems to be he was embarrassed he did didn't research that video before posting. If you look at the videos of The Verge you'll see that all the stuff in the Future video are existing in their labs.

  • Jamesbedell

    It seems to me, that if there are amazing unimagined uses for Kinect technology it might be hard to put them in a commericial, they are, after all, unimagined. 

  • Simon Field

    The Kinect has amazing potential, even the promotional video makes that perfectly clear.

    The real joy is tooling about with one in your study and coming up with things to do with it, the same way that app developers for mobile devices start small while thinking big. 

    While Microsoft presents a somewhat sterile atmosphere in which man and machine are working in perfect harmony (where's the one you threw across the room in frustration because your code wasn't working?), the sort of amalgamation between digital media and human interaction is coming. 

    The same way that even 7-8 years ago people would have never dreamed that phones are what they are now, this kind of device will change how we interact with the world around us.

  • Whoops!

    Chris Munns - Hilarious, your so silly , these arent writers on here. Oh wait, his credit says he is. So sorry. 

    BTW Tim, I'm so happy riding my horse and buggy around town! 

    This by far the best comedy blog I know of. The comments are priceless.

  • Philipp Schaefer

    In case you missed it - watch this video from CES, filmed at PrimeSense (go to minute 2:20 http://techcrunch.com/2012/01/.... This is a breakthrough for gestural interfaces - a new paradigm - that actually is not a concept or a study, it's real and it happened already last week.Combining this with other input (like voice) will make the working and magic interface of the future.

  • Philipp Schaefer

    In case you missed it - watch this video from CES from PrimeSense (go to minute 2:20 http://techcrunch.com/2012/01/.... This is a breakthrough for gestural interfaces - a new paradigm - that actually is not a concept or a study, it's real and it happened already last week.
    Combining this with other input (like voice) will make the working and magic interface of the future.

  • Steve

    Mr Maly, get off the high horse and celebrate the positive which comes with this event.

  • CynicalMe

    Don't forget that before the mouse people were happy enough plodding away just using a keyboard in a command-line environment.  With the Microsoft mouse came Windows OS.  Kinect is a new way of interfacing with machines and therefore needs a redesigned OS to get the best use.

    The issue that it brings is speed (or a lack thereof).  Don't expect to write an email using the Kinect interface only as you wouldn't expect to write a email just using a mouse.

    We only currently have part of the future, Microsoft still needs to work on the rest.

  • jsphrs

    Also disappointed at the sensationalist title. Writer publishes and article and promptly misses the point.

  • Stephen Lobo

    Vaguely threatening comments? About cracking the Kinect and potentially removing safeguards on the device so that malicious crackers could take control of your device? That's what they were concerned about - go back and read what their statements said instead of participating in the internet echo chamber...

    On NPR they made it clear that the "hacking" they were referring to was about precisely that. Not the hacking where the open USB stream(which they intentionally left open) is interpreted for use by researchers. In case you haven't realized it yet, no one had to crack the Kinect to get access to that stream. They simply had to write their own drivers to do whatever they wanted.

    Kicking and screaming? Do you think that Microsoft employee would still be working there if it wasn't their intention to begin with? 

    This article sounds more like FUD and sensationalism when there is none. If you're talking about the lack of vision in the marketing campaign there may be a hint of truth to that. Blame the marketing division. But also recognize that the company has been pushing very hard to get people to develop new applications on the Kinect as well...

  • Chris Munns

    This article does not list the point that Microsoft misses. Seems as if Mr. Maly misses the point about writing an article (and watch this Tim) by not creating any argument about Microsoft missing a point.

  • Michael Discenza

    Yeah, I'm not sure what the issue is either.  Indeed, it did take too long for Microsoft to release the SDK for developers to start doing what they wanted to with the device and yes, you might be able to complain about the lack of tactile feedback- the Theremin issue, but gestural control seems to be right at home in the OR in the commercial.  That seems like an astounding application that would be very useful and hygienic, not just cool in a cheesy "Minority Report" kind of way.

  • 5monkeez

    Does your article have a page 2? It really doesn't drive home the argument  about Microsoft missing the point. If anything, Microsoft really understands the power of opening up Kinect to developers across the globe.