Skip to main content

Data preprocessing

Processing steps for Earth observation data vary considerably across the major data sources, but share common goals of converting sensor measurements into analysis-ready data formats. While specific steps differ between optical and SAR, and between missions within each, one constraint governs them all: choices made here shape the distribution of the training data. If any preprocessing choice differs when the model is run at scale, the resulting distribution shift might cause silent failure, with a plausible-looking map. Model evaluation cannot anticipate this; only validating the map against independent reference data can (see Validation). Another related constraint is availability: all processing steps are not available on all platforms. The practical question is therefore not only which processing is theoretically preferable, but which is appropriate enough for the application and available consistently at the scale required.

SAR preprocessing

Synthetic aperture radar (SAR) data requires a distinct and often more involved preprocessing chain than optical imagery. SAR sensors measure the coherent backscatter of microwave pulses, and the resulting images are affected by multiplicative speckle noise, radiometric variations driven by local terrain geometry, and geometric distortions inherent to the side-looking acquisition (as described in SAR data selection). Unlike optical sensors, typically delivered as atmospherically corrected products, SAR products such as Sentinel-1 Ground Range Detected (GRD) products are distributed at a relatively early processing stage and require several user-side corrections before they can be meaningfully compared across space and time.

NoteRunning example · Two routes through the speckle decision

WorldCereal filters spatially: a Gamma-MAP filter with a kernel of 7 and three equivalent looks, applied per acquisition and chosen to preserve the original spatial resolution. The series is then composited to 12-day means, with missing values linearly interpolated.

CHM filters temporally: no per-acquisition filter, with quarterly compositing doing the work instead.

What separates both approaches is what each was willing to give up: a little spatial resolution in the first case, the individual acquisitions in the second.

Processing steps

The corrections needed to convert GRD products into analysis-ready backscatter can be grouped into four stages. First, thermal noise removal suppresses additive noise artifacts that are particularly visible near swath edges and in cross-polarization channels. Second, radiometric calibration converts digital numbers to a physically meaningful backscatter coefficient by normalizing by a reference area; the conventions in use (\(\sigma^0\), \(\beta^0\), \(\gamma^0\)) differ in which area they assume, and in whether that area comes from an ellipsoid or from a digital elevation model (see Appendix: Radiometric calibration and terrain correction of SAR data). Third, speckle reduction addresses the multiplicative speckle noise produced by the nature of SAR imaging. Spatial filtering [1], [2], [3] gives up spatial resolution for radiometric smoothness; multi-temporal filtering [4] denoises an image using a stack of acquisitions at higher computational cost, preserving resolution; and temporal averaging, the strategy common in the SatML community [5], [6], preserves spatial detail but discards individual observations. Fourth, geometric and radiometric terrain correction resolves the distortions topography introduces. GRD products are projected onto an Earth ellipsoid and do not account for local relief, so Range-Doppler correction against a DEM is required to recover accurate geolocation, indispensable whenever SAR data is co-located with optical imagery over anything but flat ground. Radiometric terrain correction separately normalizes against the true illuminated area per pixel, without which intensity remains partly a function of slope rather than of ground properties (see SAR data selection). Established toolboxes including pyroSAR [7] and the ESA SNAP toolbox [8] provide implementations of these workflows.

Platform-specific processing

The absence of a standardized preprocessing pipeline for Sentinel-1 GRD data means that the choice of data provider is, in practice, inseparable from the preprocessing approach. Different platforms deliver Sentinel-1 data at different stages of processing, using different DEMs, backscatter conventions, and speckle treatments — differences that can propagate silently into downstream models. Details are provided in Appendix: Sentinel-1 processing across access pathways. Within the Copernicus Data Space Ecosystem (CDSE) [9], data are available from GRD products to the more recent Sentinel-1 RTC product. Google Earth Engine applies its own processing to native GRD data; the reference implementation [10] and its associated package [11] provide guidance on proper analysis-ready data preparation on GEE. AWS S3 hosts the GRD archive from CDSE in cloud-optimized GeoTIFF format [12] as well as a Sentinel-1 RTC collection over the contiguous United States [13]. Microsoft Planetary Computer [14] provides both a GRD collection from ESA and an RTC collection that they process themselves; for the former, they encourage users to apply geometric or radiometric terrain correction on the fly using any DEM available on the platform. For practitioners that would rather not maintain a SAR processing chain, two products deliver already processed SAR data: the Sentinel-1 Level-3 ARD monthly mosaics (Copernicus Data Space Ecosystem), global at 20m, and the PALSAR/PALSAR-2 yearly mosaics (JAXA), global at 25m. Both are free and both are delivered as terrain-corrected \(\gamma^0\), requiring little user-side preprocessing. We compare them stage by stage in Appendix: Analysis-ready SAR products.

NoteDecision · How much of the SAR chain to perform

Alternatives. Start from GRD and run the four corrections in-house; take a provider’s partially processed product, where thermal noise removal, calibration and terrain correction are applied but speckle is not; or take a finished Level-3 mosaic, where compositing has handled speckle too.

Trade-off. Owning the chain buys control over every convention and keeps individual acquisitions addressable, at the cost of building and maintaining it at scale. Inheriting it buys global consistency produced by people who know the sensor, at the cost of accepting their DEM, their normalization and their compositing rule.

Constrains. Taking a composited product fixes the cadence, so the temporal decision (see Temporal distribution) is made in advance, and orbit direction and incidence angle are no longer selectable, foreclosing the choices discussed in SAR data selection.

Optical preprocessing

Optical data is typically distributed at two processing levels: top-of-atmosphere (TOA) radiance or reflectance, and bottom-of-atmosphere (BOA) reflectance after atmospheric correction. Most ML workflows operate on the latter (Sentinel-2 Level-2A, Landsat Collection 2 Level-2) but several steps remain before the data is ready for model ingestion.

Cloud masking

Cloud and shadow contamination affects all passive optical sensors. Because Sentinel-2 dominates the recent ML literature, most dedicated detection methods have been developed and benchmarked on it. Many such methods transfer to other optical sensors but may require retraining for different bands and resolutions. The available methods range from the Scene Classification Layer shipped with every Sentinel-2 L2A product to dedicated deep learning models, compared in Appendix: Cloud masking methods for optical data. The choice is largely governed by compute: SCL and, for GEE users, Cloud Score+ are already computed, while other methods must be run per product, and the deep learning methods that outperform traditional ones can be substantially more expensive. Furthermore, whether to apply cloud and cloud shadow masking at all depends on the intended use of the data. For workflows that temporally aggregate observations into composites, median aggregation already suppresses most cloud contamination, though masking cloudy and shadowed pixels beforehand generally yields cleaner composites and, for that purpose, the SCL layer is typically sufficient. For workflows that operate on individual scenes, the decision involves a trade-off between input quality and model robustness: aggressive masking produces cleaner inputs but discards data, while retaining cloudy pixels forces the model to learn to identify and handle these artifacts itself, consuming part of its representational capacity.

NoteTry it · Compare cloud masks

The cloud_masks notebook compares tile-level cloud metadata with cloud over an area of interest, contrasts the SCL with Sen2Cor’s cloud probability, and measures what each does to a median composite.

Radiometric conventions

Reflectance is distributed as scaled integers, and the scaling is a convention rather than a constant. Sentinel-2 is a cautionary example: since Processing Baseline 04.00, a radiometric offset is applied to all bands, and because the entire archive was reprocessed in the Collection-1 campaign, the offset must be determined from the PROCESSING_BASELINE metadata field rather than the acquisition date. Some platforms apply the offset upstream, in which case it must not be applied again. The conversion equations and the divisors used in practice are given in Appendix: Radiometric conventions for optical data. The general point is that a scaling convention can change mid-archive, can differ between providers of the same nominal product, and produces no error when applied incorrectly, only a shifted input distribution.

Illumination corrections

Level-2 products still contain systematic reflectance variation unrelated to surface properties. The largest source is viewing and illumination geometry: pixels at swath edge are observed at a different angle than those at nadir, and sun position changes across orbits and dates, so the same ground pixel yields different values purely because of geometry. Bidirectional Reflectance Distribution Function (BRDF) normalization adjusts reflectance to a standard viewing and illumination configuration, producing smoother time series and reducing visible seam lines at tile boundaries, at the cost of surface structure assumptions and an additional preprocessing step. Implementations are available through Sentinel Hub [15], the sen2nbar package [16] and the FORCE framework [17]. Landsat, and its harmonization with Sentinel-2, is covered in Appendix: Combining Landsat and Sentinel-2.

NoteRunning example · Who performed the preprocessing

WorldCereal ran its own chain. In the first phase, it started from Sentinel-2 L1C on AWS and applied atmospheric correction and cloud masking in-house, because the L2A archive was not yet consistent across the globe, and computed backscatter from Sentinel-1 GRD in the same pass. In the second phase, openEO on CDSE computes backscatter on the fly, but the chain remains theirs to specify.

CHM inherited most of it. Sentinel-1 was taken from Google Earth Engine, which fixes the convention at \(\sigma^0\) and the DEM at SRTM; ALOS PALSAR-2 was taken already terrain-corrected and composited. Only Sentinel-2 was processed from the archive directly.

Neither choice was free. WorldCereal owned the chain because no available product met its consistency requirement; the canopy height group inherited it because running that chain over seven years of global data was not feasible for a university group. Circumstance settled it in both cases, not preference.

[1]
J.-S. Lee, “Digital Image Enhancement and Noise Filtering by Use of Local Statistics,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. PAMI–2, no. 2, pp. 165–168, 1980, doi: 10.1109/TPAMI.1980.4766994.
[2]
V. S. Frost, J. A. Stiles, K. S. Shanmugan, and J. C. Holtzman, “A Model for Radar Images and Its Application to Adaptive Digital Filtering of Multiplicative Noise,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. PAMI–4, no. 2, pp. 157–166, 1982, doi: 10.1109/TPAMI.1982.4767223.
[3]
J.-S. Lee, M. R. Grunes, and G. de Grandi, “Polarimetric SAR speckle filtering and its implication for classification,” IEEE Transactions on Geoscience and Remote Sensing, vol. 37, no. 5, pp. 2363–2373, 1999, doi: 10.1109/36.789635.
[4]
S. Quegan and J. J. Yu, “Filtering of multichannel SAR images,” IEEE Transactions on Geoscience and Remote Sensing, vol. 39, no. 11, pp. 2373–2379, 2001, doi: 10.1109/36.964973.
[5]
J. Pauls et al., “Capturing Temporal Dynamics in Large-Scale Canopy Tree Height Estimation,” arXiv preprint arXiv:2501.19328, 2025, Available: https://arxiv.org/abs/2501.19328
[6]
M. Schwartz et al., “Retrieving yearly forest growth from satellite data: A deep learning based approach,” Remote Sensing of Environment, vol. 330, p. 114959, Dec. 2025, doi: 10.1016/j.rse.2025.114959.
[7]
pyroSAR, “A Python Framework for Large-Scale SAR Satellite Data Processing, GitHub repository.” https://github.com/johntruckenbrodt/pyroSAR, n.d.
[8]
European Space Agency, “SNAP — Sentinel Application Platform.” http://step.esa.int, n.d.
[9]
Copernicus Dataspace Ecosystem, “Sentinel-1 Documentation — Derived products & Processing options.” https://perma.cc/5YKB-NXSS, n.d.
[10]
A. Mullissa et al., “Sentinel-1 SAR Backscatter Analysis Ready Data Preparation in Google Earth Engine,” Remote Sensing, vol. 13, no. 10, p. 1954, May 2021, doi: 10.3390/rs13101954.
[11]
A. Mullissa, “Sentinel-1 SAR Backscatter Analysis Ready Data Preparation in Google Earth Engine, GitHub repository.” https://github.com/adugnag/gee_s1_ard, 2021.
[12]
Registry of Open Data on AWS, “Sentinel-1.” https://registry.opendata.aws/sentinel-1/, n.d.
[13]
Registry of Open Data on AWS, “Analysis Ready Sentinel-1 Backscatter Imagery.” https://registry.opendata.aws/sentinel-1-rtc-indigo/, n.d.
[14]
Microsoft Planetary Computer, “Sentinel-1 Synthetic Aperture Radar (SAR) Datasets.” https://planetarycomputer.microsoft.com/dataset/group/sentinel-1, n.d.
[15]
Sentinel Hub, “BRDF Normalization Script — Sentinel-2 Custom Scripts.” https://custom-scripts.sentinel-hub.com/custom-scripts/sentinel-2/brdf/, n.d.
[16]
D. Montero, M. D. Mahecha, C. Aybar, C. Mosig, and S. Wieneke, “Facilitating advanced Sentinel-2 analysis through a simplified computation of Nadir BRDF Adjusted Reflectance,” The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, vol. XLVIII–4/W12–2024, pp. 105–112, Jun. 2024, doi: 10.5194/isprs-archives-xlviii-4-w12-2024-105-2024.
[17]
FORCE, “FORCE Documentation.” https://force-eo.readthedocs.io/en/latest/about.html, n.d.