Skip to main content

Sources of uncertainty

The total uncertainty in a map prediction arises from multiple, largely independent sources that are usefully organized into the classical distinction between aleatoric uncertainty (irreducible noise inherent in the data) and epistemic uncertainty (reducible uncertainty arising from limited knowledge) [1]. In the context of EO-derived maps, however, this abstract distinction maps onto pipeline stages.

Upstream (input) uncertainty

Every observation that enters the model carries errors introduced before any machine learning is applied. Sensor noise, imperfect atmospheric correction, cloud mask failures, co-registration errors between sensors, compositing artifacts, and BRDF effects all contribute. These sources are predominantly aleatoric at the pixel level though the magnitude of the noise varies spatially (e.g., atmospheric correction is harder over bright surfaces) and can therefore appear epistemic at the map level. In principle, one would propagate input uncertainty through the model via Monte Carlo simulation over perturbed inputs [2]. In practice, this is rarely done since the computational cost is substantial, and the input uncertainty distributions are themselves poorly characterized. Nevertheless, it is important to acknowledge that model-derived uncertainty is only one component of the total error.

Label (reference data) uncertainty

The training targets are themselves uncertain. For example, GEDI footprint-level AGBD estimates carry uncertainty from allometric models and instrument noise [3], national forest inventory plots carry measurement error and representativeness uncertainty, and crowdsourced labels carry annotator disagreement. Label noise is a form of aleatoric uncertainty that sets a floor on achievable prediction accuracy. Methods that assume noise-free labels, which are most standard loss functions, will conflate label noise with model error in their uncertainty estimates.

Model (epistemic) uncertainty

This encompasses everything the model does not know such as gaps in the training data distribution, misspecification of the functional form, and sensitivity to hyperparameter choices. Model uncertainty is highest where the inference domain differs from the training domain, which is exactly the regime that matters for global maps, which inevitably extrapolate beyond the spatial and temporal extent of their training data.

Many UQ methods address only model uncertainty. The gap between what is quantified and what contributes to the total error budget should be made explicit.

[1]
A. Der Kiureghian and O. Ditlevsen, “Aleatory or epistemic? Does it matter?” Structural Safety, vol. 31, no. 2, pp. 105–112, 2009, doi: 10.1016/j.strusafe.2008.06.020.
[2]
G. B. M. Heuvelink, Error Propagation in Environmental Modelling with GIS, 1st ed. London: CRC Press, 1998. doi: 10.4324/9780203016114.
[3]
L. Duncanson et al., “Aboveground biomass density models for NASA’s Global Ecosystem Dynamics Investigation (GEDI) lidar mission,” Remote Sensing of Environment, vol. 270, p. 112845, Mar. 2022, doi: 10.1016/j.rse.2021.112845.