advertisement
advertisement

The Internet Is In Love With Google’s Fine Art Selfie-Matching App

Does Google’s Arts & Culture app think you look like a Vermeer duchess? Or Goya’s terrifying Saturn?

The Internet Is In Love With Google’s Fine Art Selfie-Matching App
[Source Images: Erik Lucatero/Unsplash, Pvt. Francis Calder Emerson, Francis Vandeveer Kughler, 1944/Google]

Last month, the Google Arts & Culture app released an experimental new feature that uses machine learning to match your selfies to famous paintings. After percolating for a few weeks, the feature took the internet by storm over the weekend, with many thousands of people sharing their results online.

advertisement

The tool is built on top of the vast archive of digitized art that Google Arts & Culture has created over the last seven years by partnering with art institutions and museums around the world. It uses facial recognition to create a biometric fingerprint of your facial features, then compares it to the more than 70,000 works in the app’s database. The tool is AI at work–designed to show off both the Arts & Culture project’s mission of making art more accessible and Google’s machine learning technology. The company includes a disclaimer that it won’t use the data from your selfie, nor will it store the photo beyond the time it takes to find a matching painting.

Some people were thrilled with the results of the app, which asks you to take a photo of yourself and the presents a series of comparable painted people, along with the percentage match.

But for others, the app presented some rather horrifying results, making it more amusing than accurate. What’s particularly entertaining when looking at other people’s results is seeing bits of the algorithm’s logic–even when the two images truly don’t look alike. In this photo, a woman with curly hair and arched eyebrows matches with a wig-wearing man with similarly arched eyebrows.

advertisement
advertisement

I too spent quite a bit of time retaking selfies to see if I could get the algorithm to match me with a painting that looks a little bit more like me. But try as I might, I’m a 53% match with Mrs. Robinson from a painting by Sir Joshua Reynolds and a 48% match with Marchesa Elena Grimaldi Cattaneo by Sir Anthony van Dyck. Not terrible, but not great either.

[Screenshot: courtesy of the author]
The app is both a testament to the progress Google has made with machine learning, and its failings. Some users have turned the tool’s outputs into memes, choosing their own “match” images to highlight how silly the tool can be.

The app’s deficiencies, particularly for people of color, also point out how white-washed the history of art is–and how neither Google nor its partner museums have chosen to include much art that represents all types of people.

The tool is available via the Google Arts & Culture app, but so far it hasn’t shown up in every user’s app. Many have expressed frustration online that they downloaded the app hoping to try the feature but were disappointed when it wasn’t there. That’s because Google has limited the number of people who can try it by region. I’ve reached out to the company to inquire when the tool will be released more broadly and will update this story when I hear back.

advertisement

You can try your luck in the roulette of machine learning selfie matching by downloading the app here.

advertisement

About the author

Katharine Schwab is an associate editor at Co.Design based in New York who covers technology, design, and culture.

More