MAGiC: A multimodal framework for analysing gaze in dyadic communication

  • Ülkü Arslan Aydin Middle East Technical University Cognitive Science Department
  • Sinan Kalkan Middle East Technical University Computer Engineering Department
  • Cengiz Acarturk Middle East Technical University Cognitive Science Department http://orcid.org/0000-0002-5443-6868
DOI: https://doi.org/10.16910/jemr.11.6.2
Keywords: Gaze analysis, speech analysis, automatic face detection, automatic speech segmentation

Abstract

The analysis of dynamic scenes has been a challenging domain in eye tracking research. This study presents a framework, named MAGiC, for analyzing gaze contact and gaze aversion in face-to-face communication. MAGiC provides an environment that is able to detect and track the conversation partner’s face automatically, overlay gaze data on top of the face video, and incorporate speech by means of speech-act annotation. Specifically, MAGiC integrates eye tracking data for gaze, audio data for speech segmentation, and video data for face tracking. MAGiC is an open source framework and its usage is demonstrated via publicly available video content and wiki pages. We explored the capabilities of MAGiC through a pilot study and showed that it facilitates the analysis of dynamic gaze data by reducing the annotation effort and the time spent for manual analysis of video data.

Author Biographies

Ülkü Arslan Aydin, Middle East Technical University Cognitive Science Department

Ülkü Arslan Aydin is a PhD Student at the Middle East Technical University Cognitive Science Department

Sinan Kalkan, Middle East Technical University Computer Engineering Department
Sinan Kalkan is a faculty member, an associate professor at the Computer Engineering Department of the Middle East Technical University, Turkey. His current research interests cover developmental robotics (Life-long Learning, Context, Concepts and Representation in Robots, Human-Robot Interaction), Deep Learning, Computer Vision, Pattern Recognition (Computer Vision, Cognitive Vision, Image Processing, Pattern Recognition)
Cengiz Acarturk, Middle East Technical University Cognitive Science Department

Cengiz Acartürk is a faculty member, an associate professor at the Cognitive Science Program at the Informatics Institute of the Middle East Technical University, Turkey. His current research interests cover human computer interaction, eye tracking applications, group eye tracking, reading, and cybercognition. He teaches graduate courses on visual cognition, eye tracking and cybersystems.

The study reported here has been partially supported by TUBITAK Turkish Technological and Scientific Research Foundation Project No. 117E021

 

Published
2018-11-12
How to Cite
Arslan Aydin, Ülkü, Kalkan, S., & Acarturk, C. (2018). MAGiC: A multimodal framework for analysing gaze in dyadic communication. Journal of Eye Movement Research, 11(6). https://doi.org/10.16910/jemr.11.6.2
Issue
Vol. 11 No. 6 (2018)
Section
Articles

License

Copyright (c) 2018 Ülkü Arslan Aydin, Sinan Kalkan, Cengiz Acarturk

Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

Most read articles by the same author(s)