A 3-D Animated Human Created Without An Animator, Using Just Kinect And A Camera

New 3-D video technologies could redefine everything from Skype chats to Hollywood movies.

We live much of our lives in digital spaces, but our avatars are basic at best. Even full-motion HD video lacks the substance and nuance of a good old-fashioned analog encounter. But a new imaging system by two creative technologists James George and Jonathan Minard is redefining the talking head.

The duo, who work in the lab of famed artist and technologist Golan Levin, call their invention "virtual cinematography," and it’s powered by stock electronics: a Microsoft Kinect coupled with an SLR camera. The Kinect tracks a figure in 3-D space while the SLR provides a sizable upgrade to the Kinect’s own camera. Custom software maps the SLR image onto the Kinect model, creating a fully-textured, digitized human.

"You can’t take your eyes off of this digital phantasm," Jonathan Minard tells us. "The texture of the depth video is also unlike any previous medium. The physical form subtly writhes and oscillates as a result of ‘depth noise.’ The details are abstracted, and artifact laden, but the motion of the figure and audio are perfectly true to life. The 3-D figures embody both abstract and hyper-realistic elements. This ‘almost’ real quality keeps the viewer engaged."

We’re inclined to agree. The example video here, featuring new media artist Golan Levin simply answering some questions for a recent Reddit AMA, is nothing short of entrancing. Furthermore, the shots you see here are entirely malleable. Once the Kinect/SLR system has captured a subject, that subject is a true 3-D model, allowing completely new perspectives in post-production through their in-house RGBD Visualize software.

RGBD Visualize is much like the virtual camera technology that’s fueled the machinima culture behind games like Halo and Team Fortress 2, coupled with a touch of Final Cut Pro’s nonlinear editing. The difference here is the human element: the smallest quirks of movement coupled with the uniqueness of one’s own face and skin. It’s 3-D animation without the animator. "As the medium progresses, it’s only a matter of time before the cameras evolve to the point of recording all of the photons to be reassembled in a virtual 3-D space which will be indistinguishable from reality," says Minard.

Now take the idea one step further. Mix in a few Hollywood actors with the advanced AI we see in video games, and we could be talking about a whole new medium of storytelling—one that’s unpredictable, dynamic, and, above all else, strikingly human.

[Hat tip: NOTCOT]

Add New Comment

9 Comments

  • Cyle Ferguson

    I don't think it's fair to call this "3-D animation without the animator" when it's still recording live action in real time. Regardless, it is very impressive.

  • Daniel Kim

    Search for "Miku Miku Dance" and "Kinect" to get another approach. Using Kinect to capture motion data from the human subject, and the Miku Dance software to impose the motion data onto a 3D animated figure.

  • Jeffrey Bergier

    It is so uplifting to hear an optimist talk about design and technology. Far too often, news is bogged down by shortsighted thinking.

  • Stephanie Cooper

    Take it one step further...with a digital image of me - I could be portrayed either being somewhere, or doing something...that I never really did?! That's pretty frightening.

  • N0mad

    Reading is fundamental: "Once the Kinect/SLR system has captured a subject, that subject is a true 3-D model, allowing completely new perspectives in post-production through their in-house RGBD Visualize software."