Data-integrated simulation of human perception and cognition

PN 6-5

Project description

The overall goal of this project is to investigate the simulation of human cognitive processes by bridging between cognitive architectures and data-driven methods. As such, it aims to address a core challenge towards realizing the vision of ”Digital Human Model” and complements models of biomechanics and biological systems explored in other EXC SimTech projects.

The main research questions addressed in this project will circle around the three following aspects:

  1. How to build adaptive cognitive systems simulating human behaviours using machine learning esp. deep learning and human signals such as gaze information?
  2. How to develop a machine learning esp. deep learning system that reflects cognitive processes of human being as proposed in the EPIC architecture – a simulation model and how to use the knowledge in EPIC to enhance machine learning systems?
  3. How to make cognitive architectures interactive and how to extend them to support human-machine interaction in real-world applications?
Project title Data-integrated simulation of human perception and cognition
Project leaders Ngoc Thang Vu (Andreas Bulling)
Project duration June 2019 - November 2022
Project number PN 6-5

Publications PN 6-5 and PN 6-5 (II)

  1. 2024

    1. P. Tilli and N. T. Vu, “Intrinsic Subgraph Generation for Interpretable Graph Based Visual Question Answering,” in Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), N. Calzolari, M.-Y. Kan, V. Hoste, A. Lenci, S. Sakti, and N. Xue, Eds., Torino, Italia: ELRA and ICCL, May 2024, pp. 9204–9223. [Online]. Available: https://aclanthology.org/2024.lrec-main.806

  2. 2023

    1. N. Schäfer et al., “Visual Analysis of Scene-Graph-Based Visual Question Answering,” in Proceedings of the 16th International Symposium on Visual Information Communication and Interaction, in VINCI ’23. Guangzhou, China: Association for Computing Machinery, 2023. doi: 10.1145/3615522.3615547.
    2. E. Dönmez, P. Tilli, H.-Y. Yang, N. T. Vu, and C. Silberer, “HNC: Leveraging Hard Negative Captions towards Models with Fine-Grained Visual-Linguistic Comprehension Capabilities,” in Proceedings of the 27th Conference on Computational Natural Language Learning (CoNLL), J. Jiang, D. Reitter, and S. Deng, Eds., Singapore: Association for Computational Linguistics, Dec. 2023, pp. 364–388. doi: 10.18653/v1/2023.conll-1.24.

  3. 2021

    1. D. Väth, P. Tilli, and N. T. Vu, “Beyond Accuracy: A Consolidated Tool for Visual Question Answering Benchmarking,” in Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, 2021, pp. 114–123.
To the top of the page