Does descriptive text change how people look at art? A novel analysis of eye-movements using data-driven Units of Interest

Keywords: Art, paintings, eye tracking, eye movement, painting narration, art perception, areas of interest, regions of interest, Markov chain


Does reading a description of an artwork affect how a person subsequently views it? In a controlled study, we show that in most cases, textual description does not influence how people subsequently view paintings, contrary to participants’ self-report that they believed it did. To examine whether the description affected transition behaviour, we devised a novel analysis method that systematically determines Units of Interest (UOIs), and calculates transitions between these, to quantify the effect of an external factor (a descriptive text) on the viewing pattern of a naturalistic stimulus (a painting). UOIs are defined using a grid-based system, where the cell-size is determined by a clustering algorithm (DBSCAN). The Hellinger distance is computed for the distance between two Markov chains using a permutation test, constructed from the transition matrices (visual shifts between UOIs) of the two groups for each painting. Results show that the description does not affect the way in which people transition between UOIs for all but one of the paintings -- an abstract work -- suggesting that description may play more of a role in determining transition behaviour when a lack of semantic cues means it is unclear how the painting should be interpreted. The contribution is twofold: to the domain of art/curation, we provide evidence that descriptive texts do not effect how people view paintings, with the possible exception of some abstract paintings; to the domain of eye-movement research, we provide a method with the potential to answer questions across multiple research areas, where the goal is to determine whether a particular factor or condition consistently affects viewing behaviour of naturalistic stimuli.

Author Biographies

Alan Davies, University of Manchester

School of computer science

PhD student, Research assistant, research software engineer (RSE)

Manuele Reani, University of Manchester

School of computer science

PhD student

Markel Vigo, University of Manchester

School of computer science

Lecturer in Health Informatics


Simon Harper, University of Manchester

School of Computer Science

Reader human computer interaction (HCI) 

Martin Grimes, Manchester City Galleries

Web Manager

Manchester City Galleries
Growth and Neighbourhoods Directorate

Manchester City Council

Clare Gannaway, Manchester Art Gallery

Curator: Contemporary Art

Manchester Art Gallery

Caroline Jay, University of Manchester

Senior Lecturer in Computer Science (software engineering)

How to Cite
Davies, A., Reani, M., Vigo, M., Harper, S., Grimes, M., Gannaway, C., & Jay, C. (2017). Does descriptive text change how people look at art? A novel analysis of eye-movements using data-driven Units of Interest. Journal of Eye Movement Research, 10(4).