Science and art are often considered to be parallel disciplines with little interaction between the two ( 1 ); here we provide a scientific perspective on the perception of art, which emerged from a collaborative project between the University of Manchester and Manchester Art Gallery. Manchester Gallery were specifically interested in understanding the behaviour of their web visitors for curatorial purposes. We explored whether reading a description of an artwork affects the way a person subsequently views it in a controlled study, leading to a richer understanding of how people view art, and a generalizable method that can be used by researchers in eyemovement research. The method presented can help to answer similar questions about differences in viewing behaviour between groups, when using stimuli where Areas of Interest (AOI) segmentation is challenging.

Art is a unique and subjective perceptual experience( 1 ). Although arguably the best context for some forms of art, museums can be difficult for some people to visit, including older people, those who have disabilities, and those who are unable to travel to them. It is also known that the time people spend viewing artworks decreases as people move through an exhibition, a phenomenon termed “museum fatigue” ( 2 ). As it is now possible to view many paintings online, more people can potentially access artworks than previously, and can access those works faster. It is known that the context in which art is viewed has an effect on how people evaluate it ( 3 ). With more and more art consumed online, new questions are emerging as to how best to digitally present it. Eyetracking can play a valuable role in understanding how people perceive art, and has the potential to provide information that can be used to support its curation. Quantitatively analysing gaze data over artwork can be challenging, due to the fact that images are often naturalistic (representing the various colours and forms as they appear in nature), and do not generally contain explicit semantic regions that can be labelled as Areas/Regions of Interest (AOIs/ROIs). In this paper we introduce a method of stimulus segmentation for subsequent data analysis that reduces researcher bias and aids in the segmentation of stimuli with difficult to identify or subjective semantic details. The method is used to quantitatively examine whether presenting a descriptive text to people before they view a painting subsequently affects their viewing pattern. The text consists of a short description written by a curator or other expert, providing information about the painting; in the current study, we used texts taken from the Art UK website ( http://www.artuk.org ), which accompany each painting displayed online. The following example is taken from one of the paintings used in the study, entitled, ‘Self Portrait’, by Louise Jopling: Figure 1

“A frontal bust portrait of the artist as a young woman with her hair tied up, wearing a pale coat with white collar and matching hat, set at an angle. At her neck she wears a decorative pink neck scarf. Her skin and features are smoothly and evenly painted, in comparison to her more textured clothes. She is set against a dark plain background.”

This study is the first to consider the impact of a descriptive text on subsequent gaze patterns over a painting. As visual scanning is the genesis of aesthetic experience ( 4 ), we apply a quantitative method to determine if the presence or absence of a description has any impact on the visual behaviour of participants by using eye-tracking, an established measure of visual attention ( 5 ) that has previously been identified as a meaningful method for quantifying how people view artworks ( 1 ).

The texts examined in the current study are primarily used for describing the stimulus such that it can be searched for within an online collection. We demonstrate that reading such descriptions does not generally appear to affect people's viewing behaviour in terms of the nature of fixation frequency or duration, and that whilst transition behaviour between UOIs is generally similar across groups, it appears to vary more when a work is abstract.

Areas of interest (AOIs) are used to identify semantic regions in a stimulus that are of importance to an experiment ( 6 ). It can be challenging to apply these to artwork, due to the non-uniform and naturalistic nature of the stimuli, which make it harder to determine how and where to draw the boundaries for AOIs. Our work addresses this issue by segmenting the image into regions using a data-driven clustering algorithm, before going on to compare differences in gaze transitions between these areas across two groups. We begin with a review of other work that has used eye-tracking to explore how people view art, and highlight the effect of the environment on how a work is perceived. The second section looks at different methods of segregating images to produce areas of interest, to provide context for our approach.

A between-subjects experiment was conducted. The main factor was “stimulus presentation”. This had two levels: 1) “no-textual description”, henceforth referred to as the “no-description” condition, where 8 paintings were shown sequentially without any description, and 2) “description” condition, where the same 8 paintings were presented sequentially, preceded by a descriptive narrative written by experts presented before each painting. The order of the presentation of the paintings was fixed, and the same for both conditions. Participants were randomly allocated to one of the two conditions. Neither of the groups were told the titles of the paintings or given any information concerning the artists. No specific task(s) were given to the participants, allowing them to view the paintings naturally.

All the analysis reported here was carried out using the R project for statistical computing, version 3.3.2. ( 19 ). Note that where effect size (partial eta squared) is reported, it was calculated on untrimmed data. The full code and data is available from ( 20 ). The Densitybased spatial clustering of applications with noise (DBSCAN) algorithm ( 21 ) was used to cluster visual fixations for each of the paintings, using the DBSCAN package ( 22 ) available for R. The algorithm was selected as it is widely used for cluster discovery, without a requirement to state the number of clusters in advance. This was important, as we did not know a priori where the fixations would be clustered, or how many clusters there would be. Density is determined by counting the points in a specific radius (termed Eps). Where the number of these points exceeds the threshold defined by a value called MinPts, it is considered a “core point” by the algorithm. Noise points are those that are neither core points, nor contain a core point within the Eps radius ( 21 ). Formally the Eps-neighbourhood (Eps) for a point is defined as N_Eps(P) = {q ∈ D | dist(p,q) ≤ Eps}. Points can also be directly density-reachable, defined as p ∈ N_Eps(q) and | N_Eps (q)| ≤ MinPts ( 21 ). The optimal Eps value was selected for each painting by computing the k-nearest neighbour distances and plotting them in ascending order to visualise the “knee of the curve” (point of curve with significant change) that corresponds to the optimal Eps value. The MinPts value, referring to the minimum number of points that are required to form “core points”, was set to 4. This value was used as the original authors state that k distance graphs did not alter significantly with values > 4, but did, however, require greater computational effort ( 21 ).

A grid of squares was then applied to each painting, with the cell dimensions (height and width) set as double the average of the optimal DBSCAN Eps value (radius). Each of the cells represented a Unit of Interest (UOI), for which fixation data was calculated.

To ensure the analysis considered only fixations on the painting itself, we added an offset to the grid using a bespoke algorithm to calculate the position of the painting inside the black container area Figure 3. Additionally as the division of individual cell dimensions into the image space may leave a remainder, we accounted for this by locating the grid in the centre of the image so that any additional space between the area of the grid and that of the painting will be around the edges of the painting. This was done instead of locating the grid in the top left of the image on the assumption that the salient features for a given painting are located centrally rather than peripherally.

As there is an inherent error in gaze accuracy for each eye-tracker, we consider the appropriateness of the size of the cells used in the grid to determine if the cell size is small enough to be impacted significantly by gaze accuracy error. The units spanned by the visual field (x) can be calculated by first determining the visual angle (Φ) given the object size (s) and the object distance (d) converted from radians into degrees (Equation 1).

The smallest of the cell sizes used in this study (1.55°) is greater than the 1 to 1.5° that is suggested as the minimum practical AOI size ( 6 ). A transition matrix was then constructed for each condition (description and no-description), representing the number of transitions from and to each cell in the grid. This is then converted into a Markov chain representing the probability of transitioning from a given UOI to the same UOI, or to a different one. This technique has been used to compare differences between clinicians making correct and incorrect interpretations of medical scans with the Jensen-Shannon distance ( 23 ), and modified by Reani, ( 24 ) to use the Hellinger distance, which is more appropriate for comparing transition behaviour, as it permits values of 0 in the transition matrix. The Hellinger distance (Equation 2) is used to determine the difference between the Markov chains representing each condition and can be used as a proxy for dissimilarity.

This value can then be compared against that obtained for two groups of equivalent size, but containing participants chosen at random. By performing this operation 10,000 times using a permutation test ( 25 ), we obtain a distribution of the difference between groups compared chosen at random, which against the difference can then between be the description and no-description groups. This allows a threshold to be set, where “p-values” to the right of the critical value allow the rejection of the null hypothesis. This allows us to see if there is something “special” about the two conditions that is unlikely to be explained by random chance, given that permutation tests are known to be robust against Type I error ( 26 ).

The individual participants' scanpaths were also compared against one another for both conditions (description and no-description). The order in which the participants transition their gaze around the UOIs was represented as a “string” of text. This essentially represents each participant's scanpath around the stimulus in terms of the sequences of UOIs visited. Although this does not account for the temporal dimension of the scanpath sequence, it does allow comparison between the areas visited. The Levenshtein distance applies a cost for each operation (insertion, deletion and substitution) used to transform one string of text into another ( 27 ). Visualising the resulting distance in a matrix allows us to rapidly visually compare all participants against each other and detect any outliers, or participants that appear to use similar visualisation strategies. The darker the shade of red the more similar the scanpaths are; the darker the shade of blue the less similar they are. Figure 2 shows an example of the Levenshtein distance for the 2 groups (description and no-description) for the “Rhyl Sands” painting, where the participants' scanpaths can be compared with one another in that condition and between conditions. This allows for rapid initial analysis of the spatial and sequential aspects of the participants’ eye-movements as they viewed the paintings. In this representative example we can see that most of the cells are red in both groups, implying that the sequence of transitions is fairly similar for most of the participants.

The visualisations were generated for each stimulus for the two groups (description and no-description). The visualisations allow for rapid high level comparison of the two conditions per painting. As mentioned previously this also makes it easier to identify outliers among the participants for further examination, or possible exclusion from subsequent data analysis. A summary of the Levenshtein distance results for each painting can be seen in Table 2.

The results indicate that the majority of fixations made for both groups tend to occur in the 100-300 ms duration range, suggesting that relatively short fixations are predominant in both conditions. There were, on average, 893 (SD = 72) fixations in the no-description group and 815 (SD = 97) fixations in the description group. The mean fixation duration for the no-description group was 239 ms (SD = 191) and 227 ms for the description group (SD = 128). Figure 5 shows the fixations for both conditions along with the fixation durations. A two-way mixed ANOVA with trimmed means (ɣ = 0.2) which was used due to data violating parametric assumptions ( 28, 29 ), showed that there was no significant difference between fixation counts for the description and nodescription groups. There was a significant but small main effect of painting (Q = 4.15, p = .006, ɳ² = .04), which post-hoc pairwise t-tests (Bonferroni correction) showed was significant for “Flask-walk, Hampstead” and “Sir Gregory Page-Turner” paintings (p = .013).

Figure 6 summarises the average fixation durations and number of fixations for each paining for both groups.

The results of the permutation test, with the exception of the painting “Release” did not show any significant differences in transitions between the groups, resulting in p values greater than .10. The painting “Release” did indicate a difference between groups (Hd = 0.859, p = .08), see Figure 7. Although this is not significant for α = .05.

This form of analysis is known to be very robust, and there is thus a very low risk of a type I error.

We examined the recording quality between the two groups to see if this could have impacted on any differences between the groups. Recording quality pertains to a percentage value that is derived from the number of gaze samples identified by the eye-tracking software that is divided by the number of attempts. Problems arising from a failure to detect the eyes and participants looking away from the screen can all contribute to reducing the recording quality value. To this end a t-test was carried out comparing the recording quality percentage values for both groups. The difference was not significant t(37) = 0.61, p > .05 suggesting that any differences detected were due to behavioural factors rather than as a result of uneven recording quality between the groups.

The majority of the paintings (n=7) did not demonstrate any difference in terms of fixation and transition behaviour. The painting that was associated with a difference between groups was “Release”. As the permutation test is robust against type I error ( 26 ), this result suggests that there was a real difference between the groups for this painting.

It is notable that this painting lacks distinctive features differentiating one area of the painting from any other. The text, which provided an explanation of what the features of the painting represent (images of chromosomes viewed through an electron microscope) could explain why transitional behaviour differs between the groups in this case. Here, the description appeared to provide information that could not be gleaned from the painting itself, and it thus caused people to examine the features of the painting differently. With no salient features and a fairly uniform pattern, there may not otherwise be cues to drive viewing behaviour.

68% of the participants who read the description thought that it did make a difference to how they subsequently viewed the painting. A difference in gaze behaviour was not observed between groups in this experiment, and it would thus be interesting to further explore the potential nature of this difference, if it exists.

Recently there has been a move towards providing descriptive information in a form that departs from formal language, using instead descriptions that focus on placing the work in context and describing the artist's intentions ( 30 ). The work reported here focuses on detecting whether reading a descriptive text leads to a difference in visual behaviour; future work could systematically address how different forms and formats of curatorial narrative affect gaze patterns as well as different types of image.

The method presented here goes some way toward removing biases that can be introduced by researchers when manually defining areas/regions of interest. Although in the current study it was applied to detecting differences in visual behaviour with paintings, the method could also be applied to other domains. Applying a grid to the stimulus allows for comparison between these uniform spatial Units of Interests (UOIs), defined using a data-driven approach. This could be applied to any type of stimulus that lacks obvious or predefined semantic areas or regions. AOIs are typically added to segment a stimulus in response to a hypothesis. Changing the AOI changes the hypothesis, and adding AOIs after recording data thus equates to formulating a post-hoc hypothesis ( 6 ). Data-driven UOIs allow the segmentation of the stimuli space into equally sized regions based on the clustering of the fixation data. This allows for the easy comparison of such UOIs to determine areas of the stimulus that the participants focus most attention on, and how they move between these areas. This data-driven method allows an unbiased, exploratory approach to interpreting gaze data across a stimulus. To mitigate the arbitrary selection of grid size when using a gridded AOI system, Holmqvist et al. ( 6 ) recommends several different cell sizes be used. As changing the AOI size has an effect on the results, there may be a temptation for researches to do this until they find a statistically significant result, a practice that can be problematic from a scientific perspective ( 18 ). The method presented in this work addresses this issue by using a combination of clusters within the gaze data and pragmatic considerations to determine the size of the grid.