Google Is Using Machine Learning To Study The Eclipse

It’s been a century since the U.S. has seen an eclipse like this–and Google has spent years preparing to learn from it.

Google Is Using Machine Learning To Study The Eclipse
[Image: Google]

On Monday, as the entire country points their phones at the sky in hopes of capturing the highly anticipated solar eclipse, a team at Google has its own plans.


The company’s Making and Science Lab, a department dedicated to supporting science and engineering education, has recruited 1,300 volunteers to capture the phenomenon, each armed with a high-resolution camera and a telephoto lens. The group–mostly astronomy buffs and amateur photographers–are stationed along the path stretching from Salem, Oregon, to Charleston, South Carolina, where they’ll be able to see the total eclipse as well as the corona, or the sun’s tenuous atmosphere. This fleet of photographers will send their images to Google, which will use an algorithm it created to stitch the images together into what it calls an “Eclipse Megamovie” showing a time-expanded video of the total solar eclipse as it crosses North America.

The idea is to gather a rich data set around the first total solar eclipse to cross a large portion of the United States in almost 100 years. Technology has changed exponentially in the last century; this rare cosmic event is the first time many will experience a total eclipse, and it’s also an opportunity to experience it with new technology. And in Google’s case, that means using their machine learning to study this eclipse and develop new ways to study cosmic events in the future.

Google’s impending Eclipse Megamovie has its roots in the science community. The initiative is in collaboration with a group of scientists led by University of California, Berkeley’s Space Sciences Laboratory, who came up with the idea of crowdsourcing an image archive of next week’s total solar eclipse back in 2011.

One of the most significant things about this eclipse compared to other total eclipses–the last one seen from the U.S. was in 1979, according to NASA--is that it will cross over land for an incredibly long time. The last time it crossed land for a comparable amount of time was 1860, when astronomers had to mail each other hand-drawn sketches of the images from their telescopes in order to study the corona. This eclipse, by contrast, is visiting in the age of the iPhone and selfies. The Berkeley scientists knew there would be no shortage of images to study if people were willing to send theirs in. They decided to crowdsource those images and stitch them together into an Megamovie to further study the sun’s corona.

“The huge advantage of the archive will be in its high degree of oversampling,” UC Berkeley professors Alexei V. Filippenko and Hugh Hudson told Co.Design over email. “From a good single site, such as Salem, Oregon, we might have 10 asynchronous samples from a hundred cameras with 10 megapixels each, thus providing some three terabytes of data. We would thus like to think that nothing the corona does (and it varies ceaselessly!) can escape scrutiny from such a fine-toothed comb.”


[Photo: Google]
In 2014, the group approached Google to see how else to take advantage of new technology to study the 2017 total eclipse. In turn, Google consulted with the design studio Ideum, who built an app that anyone can use to take photos of the eclipse (though they’re most interested in photographs taken within the line of “totality”) and upload them for use in the project. These images will be used by UC Berkeley scientists for their study, and released in an open data set by Google later this year. For the Megamovie itself, Google decided to recruit a team of “citizen scientists” with DSLR cameras and telephoto lenses to capture high-resolution photos that will be used in the movie.

Google recruited these volunteers by reaching out to amateur astronomy groups, photography schools, and colleges located in the path of the total eclipse. On Monday, they will set up in fields and on rooftops to capture the eclipse as it passes over them, then run to a spot with Wi-Fi to send them to Google. The team at Google will use an algorithm it developed to align the photos by GPS data and time stamps, so that eventually the movie will show the eclipse and its movement from views across the country. The group plans to release it as it goes, starting at around 5 p.m. Eastern Time. “At first it will be heavily skewed toward the West Coast,” says Calvin Johnson, program manager for Google’s Making and Science team. “Then, as we get new images, we’ll start putting out new versions of the video. Eventually, the gaps that we have will get filled.”

For the Megamovie on Monday, Johnson and his team have been training the algorithm on data sets of other eclipses. But just as the scientists have never had a chance to study an eclipse of this length, the Googlers don’t have a test data set comparable to what they’ll be using. The only precedent for this eclipse was 100 years ago. Once they do have the imagery from across the country, however, they’ll use it not only to make the Megamovie, but also to improve their algorithm.

When the project was being planned in 2014, machine learning was still in its infancy. Just two years ago, Google wasn’t sure exactly how they could leverage it to create the movie. Within that time, machine learning has developed in leaps and bounds; now engineers train algorithms to do everything from create their own video footage to driving our cars. As a company, Google is continuing to develop its own machine learning capacities and implement them across its various products. In the long term, with regards to the eclipse project, the company intends to incorporate new tech as the field advances, making its algorithm even faster at stitching together more images more accurately.

By 2024, when the next total eclipse will be visible from the U.S., Johnson says they hope to start exploring entirely new ways to use machine learning to do this type of image stitching. Until them, they encourage others to go through the data set when they release it–for discoveries both astronomical and technological.

Clarification: An earlier version of the article stated that Google built the eclipse app. In fact, Ideum built the app and Google provided guidance. The piece has been updated to reflect that. 

About the author

Meg Miller is an associate editor at Co.Design covering art, technology, and design.