Accessing EO data

The EO data access landscape has expanded significantly in recent years, but it is also fast-moving and not cleanly delineated: providers and platforms regularly restructure their offerings, pricing, and access policies, and the boundaries between “canonical provider”, “cloud re-distributor”, and “compute platform” are increasingly blurred. A unifying constraint across this landscape is data gravity: modern EO archives have grown to a scale where even freely available data is often impractical to move, and the position of compute relative to data has become a primary design factor. This is the reason some large-scale EO workflows today rely on platforms that co-locate data and compute instead of directly downloading from canonical providers.

Canonical EO data providers maintain authoritative archives of specific satellite missions and datasets. The Copernicus Data Space Ecosystem (CDSE) serves as the primary distribution point for Sentinel missions and related European products, while also providing access to Copernicus Contributing Missions (CCM) to complement core open collections with commercial and national high-resolution imagery. The United States Geological Survey (USGS) provides access to Landsat observations. NASA Earthdata hosts MODIS, GEDI, ICESat-2, and other NASA Earth science products. The European Organisation for the Exploitation of Meteorological Satellites (EUMETSAT) distributes data from MSG, MTG, Sentinel-3, Sentinel-6, and related systems. The Alaska Satellite Facility (ASF) offers NISAR, the ERS, JERS-1 and RADARSAT-1 legacy missions, and terrain-corrected Sentinel-1 products. Additional canonical providers include the JAXA Earth Observation Center for ALOS PALSAR, DLR Geoservice for TanDEM-X, and various other agencies maintaining specialized datasets on dedicated platforms.

Re-distribution platforms have emerged to simplify this access, though their roles vary considerably. Google Earth Engine (GEE) [1] is best understood as a compute ecosystem with co-located data rather than a pure data provider. Access is primarily through server-side operations and the platform is highly effective for tasks that exploit this paradigm, including extracting moderate-scale ML training datasets, a common use in the community. However, large-scale data extraction is not what GEE is built for, and users quickly run into export quotas and rate limits on the free and non-commercial tiers. While commercial tiers relax these limits, bulk exports require routing through Google Cloud Storage and incur significant egress fees. This presents a key structural barrier for academic and open-science teams, who produce the vast majority of public EO benchmark datasets, as few research budgets can accommodate high-throughput cloud infrastructure and egress costs. Microsoft Planetary Computer (MPC), by contrast, is essentially a data access layer: a STAC catalog of cloud-hosted Cloud-Optimized GeoTIFFs (COGs) in Azure Blob Storage, with no built-in server-side processing since the retirement of its free compute environment in mid-2024¹. Amazon Web Services (AWS) Simple Storage Service (S3) hosts EO data on both free buckets (e.g., Cloud-Optimized GeoTIFF collections for Sentinel-2 L2A and Landsat 8/9 via the AWS Open Data Program) and requester-pays buckets (e.g., the primary USGS Landsat Collection 2 archive or complete Sentinel-2 SAFE zips, imposing transfer costs directly on the user). Additional platforms include Sentinel Hub, Google Cloud Storage public datasets, and CloudFerro². Relying on re-distribution platforms introduces several trade-offs. Since these platforms ingest data from canonical EO providers, there may be latency between acquisition and availability, and their processing pipelines may differ from those of the original providers. As an example, we provide an overview of the processing differences between the native Sentinel-1/2 data products and their GEE equivalents in Appendix: Sentinel-1 and Sentinel-2 processing across access pathways. Data gaps can also occur, as regularly reported by users on the MPC GitHub repository.

Table 3.1: Overview of major Earth observation data providers. Host: cloud infrastructure where the data resides, with geographic region when documented. Online Ops: whether server-side processing such as multi-scene compositing or cloud-masking is exposed through an API. Bulk Egress: feasibility of bulk archive download, ✓ = supported on the standard tier modulo cloud egress costs; ~ = requires commercial or institutional arrangements beyond standard egress; × = the access API is not architecturally suited to bulk download regardless of budget. Cloud Compute: feasibility of running very large workloads such as deep learning inference at planetary resolution on infrastructure adjacent to the data, ✓ = the data resides on a hyperscaler (AWS, Azure, or Google Cloud Platform) offering large GPU/TPU pools in the same region; ~ = compute is available on the underlying infrastructure but does not reach hyperscaler scale, or the path between data and compute is interrupted by mandatory exports. ^*GEE data is not directly user-accessible at the storage layer; access requires the GEE platform’s own constrained compute environment.

Provider	Host	Online Ops	Bulk Egress	Cloud Compute
Canonical providers
Copernicus Data Space Ecosystem	CloudFerro (PL, DE), T Cloud (DE, NL, CH)	✓	✓	~
NASA Earthdata	Amazon S3 (`us-west-2`)	×	✓	✓
USGS	Amazon S3 (`us-west-2`)	×	✓	✓
EUMETSAT	Own data centers, Amazon S3	×	✓	~
Redistribution platforms
Microsoft Planetary Computer	Azure Blob Storage (NL)	×	✓	✓
GCS public datasets	Google Cloud Storage (various)	×	✓	✓
CloudFerro	Own data centers	~	✓	✓
Source Cooperative	Amazon S3 (`us-west-2`)	×	✓	✓
AWS Open Data	Amazon S3 (mostly `us-west-2`)	×	✓	✓
Google Earth Engine	Google Cloud Storage^*	✓	×	~
Sentinel Hub	Amazon S3 (`eu-central-1`)	✓	×	~

Accessing EO data therefore involves practical choices across multiple dimensions. Each platform offers distinct trade-offs in where the data is hosted, support for server-side operations, suitability for large-scale download, and feasibility of running large-scale co-located compute; we summarize them in Table 3.1. A practical pattern that follows from data gravity is to treat EO data access as a two-stage process. Model development typically involves modest data volumes that can be downloaded comfortably from almost any provider; here, convenience and catalog coverage dominate the choice. Inference at scale, on the other hand, is dominated by data movement: pulling petabyte-scale imagery from one provider into a different compute environment is rarely tractable. As a result, practitioners typically either (i) co-locate inference compute with the data, or (ii) accept the time and storage cost of moving the EO data into their compute environment. Co-location only helps if the compute you need is available where the data is: large GPU pools are a hyperscaler feature, so GPU-bound inference is restricted to data hosted on AWS, Azure or GCP; CPU-bound pipelines have more options. The platform that minimizes friction for training is therefore often not the one that minimizes cost for inference, and moving between them introduces a risk of distribution shift that should not be taken lightly. No universally optimal solution exists: the most appropriate choice depends on the scale of analysis, computational requirements, institutional constraints, the practical cost of download and storage, and the acceptable trade-off between immediate convenience and the risk of lock-in to proprietary platforms.

Try it · Same granule, different providers

The compare_eo_providers notebook fetches the same Sentinel-2 L2A granule from Planetary Computer and Earth Search, compares it with the canonical CDSE product, and shows how inconsistent offset metadata and platform-specific processing change reflectance and NDVI.

Decision · Provider

Alternatives. A canonical EO data provider vs. a re-distribution platform.

Trade-off. Canonical providers give provenance certainty; re-distributors give convenience, but introduce silent processing divergence. Co-located archives and compute remove the transfer bottleneck but restrict access to what they host.

Constrains. The processing levels available for each provider; egress cost and lock-in. Because switching provider between training and inference introduces a distribution shift, this decision belongs at the start of the project rather than at the inference stage.

Running example · Provider and compute

WorldCereal has been produced twice. For the first phase (the 2021 global products), Sentinel-1, Sentinel-2 and thermal Landsat 8 time series, the Copernicus Digital elevation model (DEM) and Ecoregions2017 all came from AWS, and the pipeline ran there entirely, on CPU. Keeping compute beside the archive made iteration fast and inference cheap; most of the compute budget (≈100k EUR) went into preprocessing the hundreds of terabytes of imagery. The second phase moved to CDSE and openEO.

CHM used Sentinel-1, Sentinel-2, ALOS PALSAR-2, and the TanDEM-X DEM and Forest/Non-Forest maps. The ≈50 TB of data was downloaded from AWS and Google Earth Engine, to a university server (with 2 PiB of local storage available). Training required GPUs, from the university HPC cluster. Inference then ran for five months. There were no extra costs.

Both drew mainly on the Sentinel-1 and Sentinel-2 archives; they differed in where the computation ran, and in what it cost.

[1]

N. Gorelick, M. Hancher, M. Dixon, S. Ilyushchenko, D. Thau, and R. Moore, “Google Earth Engine: Planetary-scale geospatial analysis for everyone,” Remote Sensing of Environment, 2017, doi: 10.1016/j.rse.2017.06.031.

https://github.com/microsoft/PlanetaryComputer/discussions/347 ↩︎
CloudFerro currently acts as the canonical Copernicus data provider by providing infrastructure supporting the CDSE platform.↩︎