The OlmoEarth Platform: Geospatial inference at planetary scale

Ai2 previously released the OlmoEarth series of Earth observation foundation models, pretrained on approximately 10 TB of multimodal satellite data, applicable to deforestation monitoring, food security, and wildfire risk. However, Ai2 found that most organizations in the environmental domain lack engineering teams to manage the full model lifecycle, including data annotation, fine-tuning, and large-scale inference. Drawing on over a decade of experience operating platforms like Skylight and EarthRanger, Ai2 built the OlmoEarth platform to provide infrastructure from fine-tuning evaluation to large-scale inference.

The scale of Earth observation inference differs from typical ML models: a single fine-tuning task may move terabytes of data and run for hours. Input data may come from multiple providers, use different projections and resolutions, and may be missing or cloud-obscured. The output itself is a map, where each prediction must precisely align with surrounding areas. Data acquisition and preprocessing often take more time than the model inference itself, making efficient data pipelines critical.

The platform divides inference tasks into three stages: data acquisition and preprocessing (CPU, high I/O), inference (GPU), and postprocessing (CPU). Each stage is matched to different hardware configurations, avoiding the use of expensive GPUs for CPU-suited tasks. The platform uses a multi-process data loader to continuously feed data to the GPU and streams output directly to object storage.

The OlmoEarth Run execution layer divides the geographic area into partitions, each assigned to a compute instance, and further subdivided into smaller windows for model processing. Partitions slightly overlap, and seams are eliminated during final stitching. Since partitions are independent, the same stage can run in parallel on thousands of instances.

For example, generating a wildfire risk map covering all of North America used approximately 19,600 CPUs and 994 GPUs at peak, with network throughput exceeding 168 GB/s. Parallelization reduced an estimated 4,737 hours of serial computation to about 30.5 hours, a 155x speedup. However, the degree of parallelism is limited by cloud quotas, which is one of the tunable parameters. Other tunable parameters include output resolution, model size, and caching strategy.

Given a geographic area and time range, the platform must determine which satellite images should be input to the model. This involves querying across catalogs, formats, and publication delays from different providers. For optical imagery (e.g., Sentinel-2), scenes with the least cloud cover are typically selected; for synthetic aperture radar, polarization channels may be more important.

The platform relies on public STAC catalogs and open standards, but large-scale inference can generate thousands of concurrent metadata queries, far exceeding the capacity of external STAC APIs such as ESA or Microsoft Planetary Computer. To address this, the platform maintains its own metadata index, updated via AWS Open Data SNS notifications or periodic polling of upstream indexes. The index stores scene metadata and pixel location pointers, and at runtime selects the best source and performs windowed reads on cloud-optimized formats (e.g., COG or Zarr), avoiding downloading entire scenes.