Augmenting Satellite Image Search with Machine Learning

Gathering relevant examples is an initial and important step for a scientific study related to environmental change or natural disasters. Locating these examples can require researchers to painstakingly inspect daily satellite imagery covering Earth’s 197 million square miles over several years. While such an effort can produce a valuable trove of data, manual searching is laborious, expensive, and impractical, which can ground many scientific studies before they ever take off. Enter machine learning, where researchers can train algorithms or models to learn from data. Machine learning models can rapidly search archives, saving money and hours of work. As part of this effort, team members from IMPACT and Development Seed developed the Similarity Search tool, which helps users search rapidly across image archives. This tool builds upon initial work done by Anirudh Koul from SpaceML. We interviewed Dr. Manil Maskey, Iksha Gurung and Sowmya Subramanian from IMPACT for more information.

The availability, or lack thereof, of large-scale training datasets is a problem for supervised machine learning. Similarity Search uses a self supervised learning (SSL) algorithm that does not require training datasets. However, Manil says Similarity Search can be used to build such datasets, speeding up science by increasing the diversity and representativeness of open training datasets. He believes the tool would be particularly helpful to data science communities because researchers can use it to show trends, label images, and build downstream applications. Asked about Similarity Search’s overall relevance to the scientific community and beyond, Iksha says:

"There are really interesting phenomena happening around the world. Anyone who is interested in any of these visibly identifiable phenomena can use the tool to learn more about where else these phenomena occur, their seasonality, and so on."

SSL generates features from the images, which are vector information called embeddings. Currently, Similarity Search creates SSL-based embeddings using Moderate Resolution Imaging Spectroradiometer (MODIS) data from NASA’s Global Image Browse Services (GIBS). The technical architecture is built around an Amazon Web Service ECS (elastic container service) cluster.

The team intends to build on the current beta release of Similarity Search. They look to enhance user experience by allowing a subselection of images to further refine the results, add multiple zoom levels, and filter results based on spatial and temporal criteria such as area of interest or specific time period. Sowmya says someday the platform could be used to search image archives across different planets and exoplanets, amplifying research in other disciplines such as astronomy.

Additionally, we are seeking feedback from the community on this tool.

Check out the Similarity Search tool and Similarity Search Jupyter Notebooks.

View more information about the Global Imagery Browse Services (GIBS). Learn more about the TERRA satellite. View more information about self-supervised learning.

Discover more about SpaceML SSL Github. See a previous blog post highlighting SpaceML and Anirudh Koul’s work. Check out SpaceML.

More information about IMPACT can be found on the NASA Earthdata and IMPACT project website.