Audio-visual integration during overt visual attention

Authors

  • Cliodhna Quigley Neurobiopsychology Department, Institute of Cognitive Science, University of Osnabrueck
  • Selim Onat Neurobiopsychology Department, Institute of Cognitive Science, University of Osnabrueck
  • Sue Harding Speech and Hearing Group, Department of Computer Science, University of Sheffield
  • Martin Cooke Speech and Hearing Group, Department of Computer Science, University of Sheffield
  • Peter König Neurobiopsychology Department, Institute of Cognitive Science, University of Osnabrueck

DOI:

https://doi.org/10.16910/jemr.1.2.4

Keywords:

eye movements, attention, crossmodal integration

Abstract

How do different sources of information arising from different modalities interact to control where we look? To answer this question with respect to real-world operational conditions we presented natural images and spatially localized sounds in (V)isual, Audio-visual (AV) and (A)uditory conditions and measured subjects' eye-movements. Our results demonstrate that eye-movements in AV conditions are spatially biased towards the part of the image corresponding to the sound source. Interestingly, this spatial bias is dependent on the probability of a given image region to be fixated (saliency) in the V condition. This indicates that fixation behaviour during the AV conditions is the result of an integration process. Regression analysis shows that this integration is best accounted for by a linear combination of unimodal saliencies.

Downloads

Additional Files

Published

2008-09-17

Issue

Section

Articles

How to Cite

Audio-visual integration during overt visual attention. (2008). Journal of Eye Movement Research, 1(2). https://doi.org/10.16910/jemr.1.2.4