How can machine learning help to replicate computational demanding models and catch the measurements data? Active learning for the Gaussian process emulator based on information theory provides a pathway for inverse modelling at the low computational costs. This is also the topic of a new SimTech publication on “Bayesian3 Active Learning for the Gaussian Process Emulator Using Information Theory" by Sergey Oladyshkin, Farid Mohammadi, Ilja Kroeker and Wolfgang Nowak. The paper just has been published in Entropy as part of the Special Issue “Theory and Applications of Information Theoretic Machine Learning”.
Gaussian process emulators (GPE) are a machine learning approach that replicates computational demanding models using training runs of that model. Constructing such a surrogate is very challenging and, in the context of Bayesian inference, the training runs should be well invested. The current paper offers a fully Bayesian view on GPEs for Bayesian inference accompanied by Bayesian active learning (BAL). We introduce three BAL strategies that adaptively identify training sets for the GPE using information-theoretic arguments. The first strategy relies on Bayesian model evidence that indicates the GPE’s quality of matching the measurement data, the second strategy is based on relative entropy that indicates the relative information gain for the GPE, and the third is founded on information entropy that indicates the missing information in the GPE. We illustrate the performance of our three strategies using analytical- and carbon-dioxide benchmarks. The paper shows evidence of convergence against a reference solution and demonstrates quantification of post-calibration uncertainty by comparing the introduced three strategies. We conclude that Bayesian model evidence-based and relative entropy-based strategies outperform the entropy-based strategy because the latter can be misleading during the BAL. The relative entropy-based strategy demonstrates superior performance to the Bayesian model evidence-based strategy.
For further information as well as the paper, please check https://www.mdpi.com/1099-4300/22/8/890.