Skip to main content

Esri Contributes Compression Algorithm to Tiled Storage Format

We're happy to report that Esri has contributed a data compression algorithm to the Meta Raster Format repository called Limited Error Raster Compression.

When we originally open sourced GIBS' tiled imagery storage format and server two years ago, we hoped that it would encourage usage and contributions from those outside of our project. Originally developed at NASA's Jet Propulsion Laboratory, the Meta Raster Format (MRF) is a key piece of what allows GIBS to efficiently serve over 200 trillion pixels (and counting!) to thousands of users per day without breaking a sweat. Today we're happy to report that Esri has contributed a data compression algorithm to the MRF repository called Limited Error Raster Compression (LERC).

LERC is designed to efficiently serve mapping data (as opposed to imagery) such as elevation or temperature values which can then be rapidly retrieved online. A primary feature of LERC is the ability to tune the size of the data based on the application's needs and the characteristics of the data itself. It accomplishes this by allowing the data provider to precisely control the amount of error allowed between the original data values and the compressed data values. This could range somewhere from lossless (i.e., no difference between original and compressed data) and lossy (e.g., 0.1 degree Celsius tolerance for a temperature dataset).

This compression and decompression is also designed to be fast enough to perform in the browser. A JavaScript decoder is currently available in the LERC repository.

Esri has contributed this patented code to the MRF repository under an Apache v2.0 open source license for "GIS, terrestrial and extra-terrestrial mapping, and other related earth sciences applications."

While GIBS does not currently serve data using LERC, NASA's Earth Science Data and Information System (ESDIS) plans to investigate how GIBS-like access to data could better support its mission. Besides accessing through standard tile-based retrievals, we imagine it might also gain traction by supporting the use of scripts to rapidly scrape only the data of interest.

We hope that you'll also find some good uses! Please let us know at support@earthdata.nasa.gov if you do or have questions.

Details

Last Updated

Published