An AI Learned To Make Fireworks, And They’re Mesmerizing

Google artist-in-residence Mario Klingemann decided to train a neural network on his New Year’s Eve fireworks videos, with remarkable results.

What do you do post-New Year’s Eve when you’re an AI artist? You train a neural network to take your smartphone footage of fireworks and generate new ones.


At least, that’s what Google Arts & Culture artist-in-residence Mario Klingemann did. He used a neural network called Pix2Pix that tries to anticipate the next frame in a video, and trained it using pairs of consecutive frames from his fireworks videos. Because the algorithm only knows what occurred in the previous frame, it often doesn’t work very well. For instance, Klingemann has experimented with this technique before using a video of human motion. Instead of creating a visually interesting movie, it “just became a beige soup,” he tells Co.Design in an email. “So every [type of] footage leads to different results–a few work, many don’t,” he continues. “But I had a gut feeling that fireworks might work since there is constant alternation between creation and destruction.”

His fake fireworks, which have the same sparkling, explosive characteristics as the real thing, also have an ethereal charm to them–unlike most neural net art, which tends to be downright creepy. “The instability of the model is of course also its beauty,” Klingemann adds. “Because if unstable enough it becomes unpredictable what we might see in five, 10, or 20 seconds.”
In one video, which he dubs The Adventures of Little Spark, a neural net-generated spot of light takes on an almost anthropomorphic quality, flickering and fluttering as great cavalcades of light twist and shimmer around it.

Klingemann compares the swirls of his generated fireworks to Turing patterns, which occur frequently in nature–for instance, the spots in the fur of a leopard or the complex rippling patterns found on fish or in seashells. “[Turing patterns] are temporary islands of stability in a sea of chaos shaped by opposing forces,” Klingemann says. “Ah, you see I can get quite philosophical when I look at these systems because I believe that in some way these patterns happen in any system, not only physical ones. I am pretty sure human societies and communities show these patterns, too–it is only hard to see them.”

Klingemann plans to take his fireworks experiments and turn them into higher-resolution artworks based on the same algorithmic principles. (One such work will be on display in an exhibition that opens next week at the Boston Cyberarts Gallery.) But he also hopes to investigate how the same algorithmically predictive mechanisms could create what he calls an “inspiration engine” that could function within another neural network. That engine, he says, “could act as a constantly mutating seed that other models can build upon.”


About the author

Katharine Schwab is an associate editor at Co.Design based in New York who covers technology, design, and culture.