Resource-efficient domain adaptation and simulation-enhanced learning in spatiotemporal data

Spatiotemporal information is the cornerstone of many complex systems, such as physiological- and climate systems, scene- or traffic dynamics. This project targets novel resource-efficient simulation-enhanced deep learning methods to supplement missing information, improve domain invariance, and mitigate dataset bias when learning specifically from spatiotemporal data. First, methods for few-shot domain adaptation are to be developed to minimize the discrepancy between synthetic and real data while requiring only a very small number of training examples. At the same time, challenges of the dataset bias and domain invariance should be addressed by using learned simulations and data augmentation techniques while taking spatiotemporal consistency and temporal patterns into account. We target one important application of these techniques - the estimation of metabolic rates from wearable sensors placed on the human body (e.g., 3D accelerometer and muscle data).  This field holds substantial implications for health monitoring, assistive systems, and rehabilitation robotics, but has been hampered by the scarcity of large-scale datasets covering a wide array of situations and users, making it an ideal scenario for simulation-enhanced learning.