Machine Learning the Impact of COVID-19

Recently NASA released a BETA version of a COVID-19 dashboard that tracks various impacts of the global response to the COVID-19 epidemic as reflected in the Earth observation data collected daily by satellites.

IMPACT’s machine learning team, led by Muthukumaran Ramasubramanian, assisted in this effort by building machine learning models trained to identify shipping, vehicle, and airplane traffic in satellite imagery. Here is how team member Zhuangfang NaNa Yi described this opportunity:

"Applying statistical analysis skills to provide insights for good and impactful decisions has been my greatest motivation. Deep learning and cloud computing can drive massive analytical applications in Earth science, and its application in this dashboard could potentially shape better decision outcomes for COVID-19 responses."

The IMPACT team utilized high-temporal resolution small satellite imagery to derive multi-class object detections over many locations and dates. The models developed by the team were trained on open source benchmark datasets. Then the results were synthesized into a reflection of the change in tracked indicators over time.

The models utilized the SSD Resnet 101 FPN architecture and were written in TensorFlow. The training examples were sourced from a combination of the DOTA and xView datasets. Each of those datasets featured more classes than needed for the scope of the problem, so the team filtered and aggregated classes to include airplanes, small vehicles, large vehicles (including trucks) and ships.

For the most ubiquitous Planet imagery product — PlanetScope — the team had to downsample training images to match PlanteScope’s 3-meter spatial resolution specification. For the high resolution planet imagery product — training images were downsampled to match a 0.8-meter spatial resolution specification. Additionally, colorspace correction was applied to training images in order to mimic the colorspace and contrast of the respective Planet imagery products.

Applying machine learning to satellite imagery posed many challenges. Some complexities that the team encountered included cloud cover, shadows, haze, pixel color, and object shape shifts due to the differences in time when the satellite images were captured.

The IMPACT machine learning team explored potential solutions to several questions such as:

• which data manipulation methods and tactics are effective when the training data is significantly different from the validation dataset in color, feature, and spatial resolution; and
• a method in which humans work more efficiently together with machines by validating model outputs which are then looped back to ‘retrain’ the model.

The approach taken by the IMPACT team showed how COVID-19 curve-flattening measures resulted in potential impacts on economies and society as represented by airplanes, cars, trucks and ships. The analysis identified a statistical set of trends that may be used to infer correlations with future shutdowns and other policies implemented internationally. Additionally, the analyses show how detections of changes in various economic and societal activities might correlate with detected changes in atmospheric or environmental indicators.

Team member Lilly Thomas speaks to this potential:

"My greatest passion has always been to address Earth science problems with machine learning to support innovative solutions. This project provided an interesting opportunity to leverage machine learning as a way of understanding the impacts of a global pandemic from an Earth observation perspective."

Currently the dashboard only displays ship detections. Airplane and vehicle detections are planned for dashboard integration in upcoming releases.

Check out LinkedIn profiles for Zhuangfang, Lilly, and Kumar.