Bayesian, causal, universal differential equation learner

PN 5-6 (II)

Project description

We are interested in machine-learning a PDE and/or a set of governing equations, with all relevant constitutive or closure relations, by combining partial knowledge and data. The initial condition for learning includes all relevant forms and items of existing domain knowledge, be it explicit or implicit, and all equipped with statements of (un)certainty. In areas where knowledge is incomplete, we can discover previously unknown relations of interest while providing statements of (un)certainty. The chosen learning paradigm should encode causal relations and useful problem structures in its own internal structure, ensuring that the learned relations are causal and interpretable for scientific modelling. To approach this vision, we develop B-CUDE, a Bayesian, causal extension for the recent concept of universal differential equations (UDE). UDEs are known to retain the causal structure of a system as far as known, even in aspects delegated to learning, and this aspect helps to ensure interpretability. Building upon our previously developed Finite Volume Neural Network (FINN), our emphasis lies in systematically representing relevant domain knowledge using a Bayesian framework, ensuring algorithmic efficiency for the resulting Bayesian learning task, and translating the findings into symbolic expressions with quantified uncertainty. Additionally, we address specific modelling challenges in demonstrator cases. As our motivating problem, we focus on multi-phase flow (MPF), a complex system where scientific modelling faces challenges such as non-equilibrium laws for hysteretic and dynamic effects. Data for MPF microfluidic experiments are provided by partners from Project Network 1 and CRC 1313.

This motivating problem is representative of many others: PDE-based models exist but still fail, as they rest on simplified/uncertain assumptions. However, experimental data are too scarce to rely solely on data-driven models while ignoring uncertainties. Therefore, it is advisable to incorporate physics-informed, causal learning. Conducting (Bayesian) uncertainty analysis becomes inevitable. The benefits of innovative approaches like B-CUDE are essential in advancing the scientific field.

Project information

Project title Bayesian multiscale spatio-temporal modelling of extreme events
Project leaders Wolfgang Nowak (Sergey Oladyshkin, Ingo Steinwart)
Project staff Stefania Scheurer, doctoral researcher
Project partners Holger Steeb and Nikolaos Karadimitriou (PN1 & CRC 1313 Project B05, experimental data from the Porous Media Lab PML)
Project duration January 2023 – December 2025
Project number PN 5-6 (II)

Publications PN 5-6 and PN 5-6 (II)

  1. 2024

    1. J. Bartsch, P. Knopf, S. Scheurer, and J. Weber, “Controlling a Vlasov-Poisson Plasma by a Particle-in-Cell Method based on a Monte Carlo Framework,” SIAM Journal on Control and Optimization, vol. 62, no. 4, Art. no. 4, Jul. 2024, doi: 10.1137/23M1563852.
  2. 2023

    1. S. Oladyshkin, T. Praditia, I. Kroeker, F. Mohammadi, W. Nowak, and S. Otte, “The Deep Arbitrary Polynomial Chaos Neural Network or how Deep Artificial Neural Networks could benefit from Data-Driven Homogeneous Chaos Theory,” Neural Networks, vol. 166, pp. 85–104, Sep. 2023, doi: 10.1016/j.neunet.2023.06.036.
    2. C. C. Horuz et al., “Physical Domain Reconstruction with Finite Volume Neural Networks,” Applied Artificial Intelligence, vol. 37, no. 1, Art. no. 1, 2023, doi: https://doi.org/10.1080/08839514.2023.2204261.
  3. 2022

    1. M. Takamoto et al., “PDEBench: An Extensive Benchmark for Scientific Machine Learning,” in 36th Conference on Neural Information Processing Systems (NeurIPS 2022) Track on Datasets and Benchmarks, in 36th Conference on Neural Information Processing Systems (NeurIPS 2022) Track on Datasets and Benchmarks. 2022.
    2. M. Karlbauer, T. Praditia, S. Otte, S. Oladyshkin, W. Nowak, and M. V. Butz, “Composing Partial Differential Equations with Physics-Aware Neural Networks,” in Proceedings of the 39th International Conference on Machine Learning, in Proceedings of the 39th International Conference on Machine Learning. Baltimore, USA, 2022.
    3. C. C. Horuz et al., “Inferring Boundary Conditions in Finite Volume Neural Networks,” in International Conference on Artificial Neural Networks and Machine Learning -- ICANN 2022, E. Pimenidis, P. Angelov, C. Jayne, A. Papaleonidas, and M. Aydin, Eds., in International Conference on Artificial Neural Networks and Machine Learning -- ICANN 2022. Cham: Springer International Publishing, 2022, pp. 538–549. doi: https://doi.org/10.1007/978-3-031-15919-0_45.
    4. T. Praditia, M. Karlbauer, S. Otte, S. Oladyshkin, M. V. Butz, and W. Nowak, “Learning Groundwater Contaminant Diffusion-Sorption Processes with a Finite Volume Neural Network,” Water Resources Research, vol. 58, no. 12, Art. no. 12, 2022, doi: 10.1029/2022WR033149.
  4. 2021

    1. T. Praditia, M. Karlbauer, S. Otte, S. Oladyshkin, M. Butz, and W. Nowak, “Finite Volume Neural Network: Modeling Subsurface Contaminant Transport,” in Deep Learning for Simulation ICLR Workshop 2021, in Deep Learning for Simulation ICLR Workshop 2021. 2021. [Online]. Available: https://simdl.github.io/files/33.pdf
    2. S. Xiao, T. Praditia, S. Oladyshkin, and W. Nowak, “Global sensitivity analysis of a CaO/Ca(OH)2 thermochemical energy storage model for parametric effect analysis,” Applied Energy, vol. 285, p. 116456, 2021.
    3. S. Scheurer et al., “Surrogate-based Bayesian Comparison of Computationally Expensive Models: Application to Microbially Induced Calcite Precipitation,” Computational Geosciences, vol. 25, pp. 1899–1917, 2021.
To the top of the page