This Neural Network Makes Faces From Scratch (And They’re Terrifying)

Google’s machine learning artist-in-residence trains a neural network to make portraits–and unleash demons on the Internet, apparently.

Mario Klingemann is the Machine Learning Artist in Residence at the Google Cultural Institute in Paris (the same arm of Google behind the new Louvre-busting museum app). But when he’s not tinkering with neural networks to figure out how they can be leveraged against Google’s database of several million scanned cultural artifacts, Klingemann’s training them on human faces… and posting the unsettling results online.


Trained on a database of over 20,000 faces organized by rough category (age, gender, pose, etc), Klingemann experiments use a convolutional neural network similar to Deep Dream. Except instead of just recursively reinterpreting the same picture with trippy results, Klingemann’s face maker generates ghoulish visages entirely from scratch, according to how computers interpret faces. And how do computers interpret faces? If Klingemann’s results are anything to go on, like Son of Sam going through Facebook on mescaline.

While these noseless, zipper-mouthed melt demons might suggest otherwise, it turns out that Klingemann’s neural network actually looks for the same things human brains do when trying to identify faces, focusing primarily on the eyes and mouth. Klingemann’s neural network also looks for the gradient of the skin, and has some remedial hair detection capabilities, judging from the different skin colors and hair smears boasted by his Deep Dream Cenobites.

So why is Klingemann messing with this nightmare fodder? “My ultimate goal is to create a machine that can produce ‘art’ entirely autonomously,” he says. “But I’m definitely not there yet, so right now, I’m in the phase where I build tools that produce serendipitous visuals that are different or ‘new’ in some way.”

While few people would argue that these neural network’s vision of human faces are pretty, Klingemann says he is pleased with them. Other researchers, he points out, have successfully used neural networks to create relatively realistic faces from scratch, using different data sets and types of code. But photorealism doesn’t really equal art, which is what Klingemann eventually wants to train neural networks to produce.

“My goal is not to create realistic portraits, but rather abstract versions of faces,” he tells me. “What I like about them is that they seem to come right out of the uncanny valley. They’re just all somehow on the edge.”