I live mere blocks from a world-class park in Brooklyn, but when I take my soon-to-be-born kid there someday, what am I going to say when he/she says, “What’s that tree, Daddy? And that one? And that one?” Luckily there’s an app for that: Leafsnap, a collaboration between Columbia University, the University of Maryland, the Smithsonian Institution, and FindingSpecies.org that repurposes facial-recognition algorithms to identify local flora instead.
Leafsnap identifies tree species by the shapes of their leaves.
Leafsnap’s scientific pedigree is impeccable, but the interaction design is truly ingenious: Computer scientists Peter Belhumeur and David Jacobs realized that the same algorithms that tag idiots in Facebook photos could also be used to visually identify trees by the shapes of their leaves. You just pluck a leaf off a branch (or pick one off the ground, if you’re one of those “but trees feel pain, too!” types) snap a pic of it, and Leafsnap tells you whether it’s it gingko or dogwood — and also displays high-resolution photos of the tree’s flowers and other goodies. (For some back story about how the app was developed, click on our previous story here.)
The way it works is actually pretty simple. Leafsnap identifies tree species by the shapes of their leaves, so the app’s algorithms extract the edge contours of the leaf in question (that’s why the app asks you to snap your pics on a solid background) and match it against an online database assembled by the Smithsonian Institution. “We use a different palette of visual characteristics for plant species identification than we do for face recognition, but the later stage machine learning methods are much the same,” Belhumeur tells Co.Design.
The Smithsonian images that Leafsnap uses for pattern-matching don’t get displayed in the app, though — that’s where FindingSpecies.org‘s photos come in. They’re high-res, factually accurate, and just plain gorgeous. And every time you use Leafsnap to ID a tree, you make that database and matching process even more robust. So in three to five years when “TK Pavlus” starts peppering me with botanical questions I am in no way qualified to answer, Leafsnap should be a no-brainer.