The research group investigates approaches to use deep learning for process understanding via explainable machine learning (XAI) and hybrid modeling, the combination of physically-based modeling and machine learning. Scientific insights can either be achieved via built-in mechanisms (e.g., hybrid modeling) or via post-hoc explanations. Both approaches are motivated by the ever-growing amounts of Earth observation data, the limited capability of traditional, physically-based models to reproduce observed patterns, and the capability of modern deep learning approaches to approximate the behavior of complex Earth system processes.
Explainable machine learning
The dynamics of ecosystems and Earth system processes in general are often difficult to model and predict, and even today many machine learning approaches fail to encode temporal dependencies that typically characterize dynamic natural systems. However, deep learning approaches in the time domain (e.g. recurrent neural networks) are conceptually capable of solving such problems. Such models are difficult or impossible to interpret physically, which limits the scientific knowledge gained from them.
However, appraoches from XAI can support the process of model development and debugging, justify model behavior, and even enable the destillation of knowlege from machine learning models. We focus on temporal dependencies between forcing variables like precipitation and the ecosystem response (for example represented by vegetation state).
Memory effect on vegetation state found by a block-permutation approach using recurrent neural networks.
Memory effect on vegetation state found by a block-permutation approach using recurrent neural networks.
In a first attempt to apply deep learning to better understand ecosystem dynamics (Kraft et al., 2019), we used a permutation-based approach to identify memory effects on vegetation state. This research will be continued with more recent appraoches from explainable machine learning.
Explainability via buildt-in mechanisms: Hybrid modeling
Hybrid models combine physically-based modeling and machine learning.
Hybrid models combine physically-based modeling and machine learning.
The combination of machine learning and physically-based modeling, hybrid modeling , is intended to research and further develop possible ways of gaining scientific knowledge about the observed phenomena despite the black box character of the models used and thus the potential of temporal deep learning approaches for to be able to use the modeling of ecosystem processes.
First application of hybrid modeling to Earth system modeling
Hydrological hybrid model that combines physically-based (e.g. mass conservation) and deep learning approaches to represent the major states and fluxes of the global water cycle, from Kraft et al. (2022).
Hydrological hybrid model that combines physically-based (e.g. mass conservation) and deep learning approaches to represent the major states and fluxes of the global water cycle, from Kraft et al. (2022).
In a recent publication (Kraft et al., 2022), we presented a physics-aware machine learning model of the global hydrological cycle. As the model uses neural networks under the hood, the simulations of the water cycle are learned from data, and yet they are informed and constrained by physical knowledge. The simulated patterns lie within the range of existing hydrological models and are plausible. The hybrid modeling approach has the potential to tackle key environmental questions from a novel perspective.
A key result was the decomposition of the terrestrial water storage signal into the components of snow, soil moisture, and groundwater in a data-driven yet physically consistent way.
Contribution af single water storages—snow (SWE), soil moisture (SM), and groundwater (GW)—to the terrestrial water storage found by the hybrid model (H2M) and a selection of physically based hydrological models.
Contribution af single water storages—snow (SWE), soil moisture (SM), and groundwater (GW)—to the terrestrial water storage found by the hybrid model (H2M) and a selection of physically based hydrological models.
The DUKE project
Funded by the Federal Ministry for Economic Affairs and Climate Action based on a resolution of the German Bundestag. Duration: 01/2022 – 12/2023
Funded by the Federal Ministry for Economic Affairs and Climate Action based on a resolution of the German Bundestag. Duration: 01/2022 – 12/2023
The DUKE project (Deep-learning-based hybrid uncertainty-aware modelling of the coupled water and carbon cycle with Earth observation data) aims to further develop and combine the research paths of hybrid-modelling and uncertainty assessment. The project is a collaboration between the Max Planck Institute for Biogeochemistry adn the Chair of Remote Sensing Technology at the Technical University of Munich.
DUKE focuses on the real-world problem of modelling global matter transport cycles, specifically the currently still open question of coupled water and carbon cycles. Here, a data-driven perspective offers enormous potential, as the mostly rigid boundary conditions and parameterisations in process models lead to systematic deviations in the simulations. Furthermore, the quantification of uncertainties is essential for model development as well as for testing alternative hypotheses.
The appraoches from the uncertainty quantification (TUM) will be integrated into the coupled hybrid model developed by the MPI group to achieve an uncertainty-aware hybrid model of the coupled carbon-water cycle.
Key publications
Basil Kraft, Martin Jung, Marco Körner, Sujan Koirala, and Markus Reichstein, "Towards hybrid modeling of the global hydrological cycle," Hydrology and Earth System Sciences 26 (6), 1579-1614 (2022).
Basil Kraft, Martin Jung, Marco Körner, Christian Requena Mesa, José Cortés, and Markus Reichstein, "Identifying dynamic memory effects on vegetation state using recurrent neural networks," Frontiers in Big Data 2, 31 (2019).
Hoon Taek Lee, Martin Jung, Nuno Carvalhais, Tina Trautmann, Basil Kraft, Markus Reichstein, Matthias Forkel, and Sujan Koirala, "Diagnosing modeling errors in global terrestrial water storage interannual variability," Hydrology and Earth System Sciences 27 (7), 1531-1563 (2023).
Reda ElGhawi, Basil Kraft, Christian Reimers, Markus Reichstein, Marco Körner, Pierre Gentine, and Alexander J. Winkler, "Hybrid modeling of evapotranspiration: inferring stomatal and aerodynamic resistances using combined physics-based and machine learning," Environmental Research 18, 034039 (2023).
Markus Reichstein, Bernhard Ahrens, Basil Kraft, Gustau Camps-Valls, Nuno Carvalhais, Fabian Gans, Pierre Gentine, and Alexander Winkler, "Combining system modeling and machine learning into hybrid ecosystem modeling", in Knowledge-Guided Machine Learning, edited by Ramakrishnan Kannan and Vipin Kumar (Chapman & Hall, London, 2022), pp. 327-352.
Basil Kraft, Martin Jung, Marco Körner, Sujan Koirala, and Markus Reichstein, "Towards hybrid modeling of the global hydrological cycle," Hydrology and Earth System Sciences 26 (6), 1579-1614 (2022).
Basil Kraft, Simon Besnard, and Sujan Koirala, "Emulating ecological memory with recurrent neural networks", in Deep Learning for the Earth Sciences: A Comprehensive Approach to Remote Sensing, Climate Science, and Geosciences, edited by Gustau Camps-Valls, Devis Tuia, Xiao Xiang Zhu, and Markus Reichstein (John Wiley & Sons Ltd, Hoboken, New Jersey, 2021), pp. 269-281.
Basil Kraft, Martin Jung, M. Körner, and Markus Reichstein, "Hybrid modeling: Fusion of a deep approach and physics-based model for global hydrological modeling," The International Archives of Photogrammetry, Remote Sensing and Spatial Information Sciences XLIII-B2-2020, 1537-1544 (2020).
Basil Kraft, Martin Jung, Marco Körner, Christian Requena Mesa, José Cortés, and Markus Reichstein, "Identifying dynamic memory effects on vegetation state using recurrent neural networks," Frontiers in Big Data 2, 31 (2019).
Christian Requena-Mesa, Markus Reichstein, Miguel D. Mahecha, Basil Kraft, and Joachim Denzler, "Predicting landscapes from environmental conditions using generative networks", in Pattern Recognition, DAGM GCPR 2019, edited by G. A. FInk, S. Frintrop, and X. Jiang (Springer, Cham, 2019), pp. 203-217.
Markus Reichstein, Simon Besnard, Nuno Carvalhais, Fabian Gans, Martin Jung, Basil Kraft, and Miguel D. Mahecha, "Modelling landsurface time-series with recurrent neural nets", in 2018 IEEE International geoscience and remote sensing symposium (IGARSS), (Valencia, 2018, 2018), pp. 7640-7643.
