How do you identify a neighborhood’s affluence? Turns out, you can do it from space.
Th satellite mapping company DigitalGlobe tapped San Francisco design studio Stamen Design to build a machine learning-powered mapping tool that takes income data and satellite imagery to predict average income of city blocks. Called Penny, the program analyzes the shapes, colors, and lines that make up a satellite image. Using corresponding Census data, it looks for patterns between different urban features and income levels. With that information, the algorithm can then guess what the income level is of any given area you point it at–provided it’s in the same city the algorithm was trained on.
Take, for example, the New York version: Hover over the World Trade Center, and Penny is 86% confident that it’s a high-income area; hover over Harlem, and Penny is 99% confident that it’s a low-income area. The program also shows you how Penny’s prediction matches up with Census data for that part of the city.
Penny also lets you drag and drop urban features–trees, baseball diamonds, parking lots, helipads, brownstones, pools–into any area of the city and see what happens to the program’s predictions. In the New York version, you can drop a freeway in front of Trump Tower, and it drops from high-income to low-income. Put a baseball diamond in front of the Metropolitan Museum of Art, and it’s no longer so ritzy.
“What’s interesting is you can go to East Harlem and you can take one of those lower to mid-income highrises, but you can surround the entire building with grass and trees and that will significantly raise the income prediction of that area,” says Jordan Winkler, a geographer and data specialist at DigitalGlobe who worked with Stamen on the project. “It’s showing you that it’s not any one single thing [that determines income]. It’s not the building, it’s the building and the context into which that building is placed.”
Penny’s accuracy–an average of 86%–means that it’s not perfect, but it does point to correlations between urban design features, from green space to parking lots, and how a neighborhood is perceived. And when adding urban markers of higher income, like trees or tennis courts, doesn’t have an impact on Penny’s income prediction for low-income neighborhoods, the program points to how there are often more structural issues at play in cities that can’t be fixed so easily.
“This is not a tool to say go plant a tree, and this neighborhood will become healthier,” Winkler says. “It is a tool to help us question our assumptions about if human activity and human behavior can be understood through the lens of satellite imagery.”
But he also has something more ambitious in the works for Penny. “We’re working on asking Penny, okay, here’s a project in Queens or in the Bronx,” Winkler says. “How can we make this [wealthier]? And then have Penny suggest and place the objects on the map for us.”
This could be used by cities or developers to examine what might happen to a neighborhood if you add certain types of urban features to it. But for now, Winkler says, the technology is not robust enough and the model is entirely city-specific; because it learns based on localized data, it can’t be generalized effectively. Penny also lacks important nuance: It doesn’t take the distribution of income in a particular area into account. For instance, Penny displays Midtown Manhattan as a high-income area but doesn’t offer any insight on how to help the neighborhood’s homeless population.
“This says nothing of income equality,” Winkler says. “The patterns that we display on the surface of the Earth that we see from space are not always going to be perfectly correlated with the human condition in that area.”
But it can be used to identify intriguing tipping points, Winkler says–where planting two trees makes little difference, but planting three makes a big one. “What kind of patterns in the ways in which we organize ourselves as humans in cities are visible to machines in a way that can predict the quality of our life on the ground?” he says.
From playing around with the program it’s clear that it’s far from perfect, so using it to inform any kind of real-life decisions would be unwise without further research. Ultimately it shows that while machine learning-enabled programs like Penny might provide insight into patterns in cities that would be difficult to model without such large amounts of data, it still can’t capture the complexity and nuance of cities and the humans that live within them.