This Algorithm Copies Your Artistic Style By Watching You Color

Plasticky 3-D renders are about to get a whole lot more life.

Your average 3-D rendered scene needs a lot of love to turn the glossy geometries of computer-drawn shapes into the sorts of rough, dirty, sticky objects we have in the real world. So much of what we see–even at the highest levels of Hollywood–is still too slick to recognize the human touch.


But a new research project called Stylit, by DCGI and Adobe Research, spotted by Prosthetic Knowledge, can take the very personal way you sketch and apply that aesthetic directly to a 3-D scene–letting you apply your personal style to the impersonal digital world.

Here’s how it works: the researchers set up a simple scene–a sphere sitting on a table–and printed it out at a very low contrast. Then, artists colored it in while a camera over the table tracked them. The artists could use any medium they like, since they’re just coloring on paper–marker, watercolor, colored pencil, acrylic paint, or ink. Meanwhile, an algorithm converted their color, shadow, and highlights onto a more complex 3-D figure on the screen. The neatest part? Artists could watch the style transfer be applied from paper to screen in real time, meaning they could adjust their own technique to improve the result.

The technology looks amazing, though it’s not without a few limitations. Right now, some fidelity can be lost in transit, making the final 3-D object murkier than intended. The lighting between the source object and final object needs to be fairly similar–you can’t just change the lighting in post production. And all of this means that Stylit is not quite ready to be applied to full 3-D animation yet.

However, Stylit is another stunning data point in a burgeoning trend: computers are learning to master our own, very personal, artistic techniques. And as that happens, the unique purview of humanity will be reproduced, remixed, and reimagined with a prolificness that no human could ever match.

All Images: via DCGI


About the author

Mark Wilson is a senior writer at Fast Company. He started, a simple way to give back every day.