The interplay between task difficulty and microsaccade rate: Evidence for the critical role of visual load

In previous research, microsaccades have been suggested as psychophysiological indicators of task load. So far, it is still under debate how different types of task demands are influencing microsaccade rate. This piece of research examines the relation between visual load, mental load and microsaccade rate. Fourteen participants carried out a continuous performance task (n-back), in which visual (letters vs. abstract figures) and mental task load (1-back to 4-back) were manipulated as within-subjects variables. Eye tracking data, performance data as well as subjective workload were recorded. Data analysis revealed an increased level of microsaccade rate for stimuli of high visual demand (i.e. abstract figures), while mental demand (n-back-level) did not modulate microsaccade rate. In conclusion, the present results suggest that microsaccade rate reflects visual load of a task rather than its mental load.


Introduction
Research on microsaccades has been very popular in the past decade (Engbert, 2006a;Martinez-Conde, Macknik, Troncoso, & Hubel, 2009), with a considerable deal of effort being put into investigating causes for the modulation of microsaccade rate (Betta & Turatto, 2006;Otero-Millan et al., 2008). In this context, the influence of task load on microsaccade rate attracted The interplay between task difficulty and microsaccade rate: Evidence for the critical role of visual load considerable interest in research (e.g. Hicheur, Zozor, Campagne, & Chauvin, 2013;Siegenthaler et al. 2013;Gao, Yan, & Sun, 2015;Dalmaso, Castelli, Scatturin, & Galfano, 2017;Krueger et al., 2019). However, the result pattern of studies addressing the relation of microsaccade rate and task load is rather diverse and complex. While it can be assumed that task difficulty has a decisive influence on microsaccade rate (Pastukhov & Braun, 2010;Siegenthaler et al. 2013;Gao et al., 2015;Dalmaso et al., 2017;Benedetto, Pedrotti, & Bridgeman, 2011;Hicheur et al., 2013;Xue, Huang, Ju, Chai, Li, & Chen, 2017), the direction of this effect is less apparent. Some studies indicate an increase while others report a decrease in microsaccade rate as consequence of an increase in task difficulty. In this context, visual load, i.e. the amount of information involved in the perceptual processing of the task stimuli (see Lavie, 2010 for review), and/or mental load, i.e. demands of working memory (Lavie, Hirst, de Fockert, & Viding, 2004) have been suggested to play a major role (Xue et al., 2017). Although results of some initial studies comparing different types of cognitive load (e.g. Krueger et al., 2019;Xue et al., 2017) indicate that the visual component of the load is highly important for microsaccade occurrence, there is still some lack of clarity regarding the influence of non-visual task load.
Recent studies showed differing patterns of microsaccadic activity in relation to applied tasks, depending on task modality (visual task, e.g. visual search task, or mental task, e.g. arithmetic task) and the variation of difficulty within those tasks (Benedetto et al., 2011;Siegenthaler et al. 2013;Gao et al., 2015;Dalmaso et al., 2017). Some research revealed a positive correlation between microsaccade rate and task demand. For instance, Benedetto and colleagues (2011) used a simulated driving task to compare a low load task (control task) and a high load task (dual task including visual search task) and reported more microsaccades being produced in the high load condition. However, other studies reported lower microsaccade rate associated with an increase of mental load. Those studies applied non-visual tasks (Siegenthaler et al. 2013;Gao et al., 2015;Dalmaso et al., 2017). For example, Siegenthaler and colleagues (2013) used a mental arithmetic task without any visual component (mental counting) and reported that microsaccade rate decreased in the high load condition. A replication of Siegenthaler et al. (2013) showed the same inverse relationship between microsaccade rate and task difficulty (Gao et al., 2015). Dalmaso and colleagues (2017) applied twodigit (low load) and five-digit (high load) memorizing tasks to investigate the association between microsaccade rate and working memory load. Their results indicated also a reduced microsaccade rate for the task with high mental load.
Overall, studies have shown that increasing difficulty in tasks with a strong but not exclusive visual component increases microsaccade rate (Benedetto et al., 2011) and that increased task difficulty in nonvisual tasks is linked with a decreased microsaccade rate (Siegenthaler et al., 2013;Gao et al., 2015;Dalmaso et al., 2017). These contradicting results indicate that specific interactions between the effects of visual load and mental load might occur.
To learn more about those interactions, Krueger and colleagues (2019) manipulated visual and mental load systematically in a dual-task paradigm (visual search task and mental arithmetic task). They showed that if processing resources are allocated to a visual task (i.e. a difficult visual task combined with an easy mental task), microsaccade rate increased with difficulty level. On the other hand, microsaccade rate decreased with increasing difficulty level of the mental task (i.e. a difficult mental task combined with an easy visual task). The authors hence concluded that microsaccade rate indicates how much processing resources are allocated to a visual task.
In summary, previous research has shown that task load modulates microsaccade rate; visual load is associated with an increase in microsaccade rate while mental load is associated with a decrease in microsaccade rate (c.f. Krueger, et al., 2019). But it is not clear yet whether inducing visual or mental load is decisive for changing microsaccade rates. Therefore, the intriguing remaining open question is how the microsaccade rate is modulated when mental load is induced by a task that requires visual information processing. Or put differently, the question is whether the microsaccade rate responds to an increase in difficulty of mental load in a task where visual information processing is required to solve the task.

The present study
In contrast to all previous research, the task used in this study coupled visual and mental demand. This means that some degree of visual processing must be maintained during the entire task. Processing resources have thus to be split between the visual and the mental processing. For this purpose, two versions of a n-back task were created, to manipulate visual difficulty of the task. The figure version used stimuli that are novel, visually complex and difficult to process in comparison to the letter version using well-known stimuli that are easy to process. Mental load was manipulated by increasing difficulty level (increasing n).
Based on previous studies we expected 1) higher microsaccade rate in the condition inducing higher visual load (n-back task with figures) compared to the task inducing a minimal amount of visual load (nback task with letters), and 2) microsaccade rate to decrease with increasing mental load.

Participants
Fourteen participants (55% females) with an average age of 21.45 years (SD ± 1.29) took part in the study. All participants were students of the University of Fribourg. Three participants were excluded from the eye movement analyses due to technical problems.

Task and Stimuli
Participants completed two versions of the n-back task. The n-back tasks are continuous recognition measures that present sequences of stimuli, such as letters or pictures (originally introduced by Kirchner, 1958). For each item in a sequence, participants judge whether it matches the one presented n items ago. n is a variable that can be adjusted to respectively increase or decrease mental load (Au et al., 2015). In the current study, we used two different sets of stimuli, letters and Attnaeve figures (Attneave, 1957;Attneave & Arnoult, 1956). The presentation of unknown figures is associated with an increase in visual load (as compared to well-known stimuli like letters). In addition, participants cannot use verbal strategies when Attnaeve figures are presented.
In addition, a probe detection task (PDT) was used as a control task. In this task, participants were required to identify a predefined stimulus (a star) among distractors (a diamond).
Each version of the n-back task contained eight trials. Starting with 1-back, it increased up to 4-back and reversed. Furthermore, each version started and ended with a probe detection task, for a total of 10 trials (control, 1-back, 2-back, 3-back, 4-back, 4back, 3-back, 2-back, 1-back and control). Eighteen stimuli appeared in each trial. Each stimulus was presented for 500ms, followed by a black fixation cross for 2500ms (see Figure 1). Six target stimuli appeared in each trial at random positions.
Participants were asked to press the spacebar as fast as possible when they detected a target and do nothing when a distractor was displayed. To avoid any predictability in the n-back tasks, the stimuli sets were divided into three groups which served an equal amount of times as targets and distractors across observers. This means that each stimulus was a target in about a third of the trials, a distractor in another third and was not present in the last third of the trials. The use of each stimulus (target, distractor or not used) was counterbalanced across trials and participants. The stimuli set in the letter condition consisted of 18 capital consonants (B, C, D, F, G, H, J, K, M, N, P, Q, R, S, T, V, X, Z, e.g. Ravizza et al., 2004). Meaningless Attneave figures composed the figure stimuli set to ensure that observers could not remember the figures due to their meaning. They had between four and eight sides and were selected among about 200 figures randomly generated with the Matlab® code developed by Collin and McMullen (2002, see Figure 2). They were selected based on the work of Vanderplas and Garvin (1959). Letter and figure stimuli were about the same size (height of 2.33° by width of 2.37° of visual angle, with a standard deviation of 0° and 0.77°, respectively). The presentation order of the n-back versions were counterbalanced across participants.

Experimental design
In this study, a 2 x 10 within-subject design was used, with visual load and difficulty order as factors. Visual load was manipulated by using different types of visual stimuli for the n-back task on two levels, letters (i.e. easily processed stimuli) and Attneave figures (i.e. hard-to-process stimuli, Attnaeve, 1957, Attneave & Arnoult, 1956). Difficulty order corresponded to the sequence in which trials were presented according to their difficulty.

Apparatus
The experiment was conducted in a quiet room, which was equally illuminated for each session. Participants were seated at a desk approximately 70cm away from an LCD monitor (1650 by 1050 px, 60 Hz refresh rate), facing a desktop-mounted Eye-Link 1000 eye tracker. Head position was maintained constant by an EyeLink 1000 head/chin support. Stimuli presentation and data collection was controlled by a Matlab script and Psychophysics and EyeLink Toolbox extensions (Brainard, 1997;Pelli, 1997;Kleiner et al., 2007; see http://psychtoolbox.org/).

Measures
Microsaccades per second and response accuracy were recorded. Response accuracy (i.e. percentage of correct responses) was used as performance measures.
In addition, participants completed a modified NASA-Task Load Index (NASA-TLX; Hart & Staveland, 1988;Hart, 2006) for mental demand (How mentally demanding was the task?) and effort (How hard did you have to work to accomplish your level of performance?), as a measure of perceived workload. A visual analog scale ranging from 0 to 100 with a title and a bipolar descriptor (very low/very high) at each end was presented for both NASA-TLX dimensions.

Eye movement recordings and analyses
A non-invasive fast video-based eye tracker (Eyelink 1000, SR research, Ontario, Canada) sampling the eye position binocularly at a frequency of 500Hz was used to assess eye movement data. Blinks and semiblinks were removed for data analysis. Blinks correspond to a full occlusion of the pupil and were identified by the missing information about the pupil. Semiblinks are periods with very fast changes of the pupil area during which the pupil is never completely occluded. A decrease or increase larger than 50 pupil size units per sample was considered as a semiblink . Additionally, to avoid any partial occlusion, data samples 200ms before and after blinks and semiblinks periods were removed . Microsaccades were automatically identified with an objective algorithm (see Engbert & Kliegl, 2003 for details) using a λ of 6 to determine the velocity threshold and a selection criterion of an amplitude smaller than one degrees of visual angle (c.f. also Beer, Ehckel, & Greenlee, 2008;Bettea & Turatto, 2006;Hafed, Goffart, & Krauzlis, 2009; based on the entire trial. The minimum duration for a microsaccade was set at 10ms. To reduce potential noise (Engbert, 2006a(Engbert, , 2006b, only binocular microssacades were retained for the analyses (e.g. Engbert, 2006a;Engbert & Mergenthaler, 2006;. To avoid categorizing overshoots as microsaccades, an intersaccadic interval of 20ms was applied (Møller, Laursen, Tygesen, & Sjølie, 2002).

Procedure
Participants were welcomed in the eye tracking laboratory and guided to their place. The experimenter explained the procedure as well as the tasks and asked for the following information about the participant: gender, age, and handedness. Then, participants were seated at a desk in front of the LCD monitor. Participants performed both versions of the n-back task which differed in type of stimuli presented (letters or Attnaeve figures). For each version, a nine-point calibration of the eye position was performed before completing the ten experimental trials. Furthermore, to maintain the accuracy of the eye position signal, a drift correction was performed at the beginning of each trial. Finally, they had to respond to the two questions about their subjective state at the end of each trial.

Data analysis
Data were analysed with a two factorial repeated measures ANOVA with stimuli type (two levels: letters and Attneave figures) and task difficulty (5 levels, control task to 4-back) as within-subjects factors with performance and mental workload as dependent variables. A second two factorial repeated measures ANOVA with stimuli type (two levels: letters and Attneave figures), task difficulty (five levels, control task to 4-back) as within-subjects factors was calculated to analyze the effect of stimuli type, task difficulty and object processing on microsaccade rate. Furthermore, a 2 (letters, Attneave figures) x 8 (task difficulty from 1-back to 4-back and from 4back to 1-back) x 2 (displayed stimulus: fixation cross and stimulus) repeated measures ANOVA was also used to investigate how the microsaccade rate behavior was affected by the stimuli type, variations of task difficulty across trials and the relevance of the displayed object for the task. Partial eta-squared (ηp 2 ) was calculated as effect size measure.
Finally, we investigated whether microsaccade rate was influenced by expected response to the stimulus (target vs distractor). We used a similar approach as described in Engbert & Kliegl (2003) where microsaccade rate was averaged overall all trials of all participants for each level of task difficulty. A moving time-window of 100ms was used to smooth the data.

Performance and subjective workload
Analysis of performance data (see Figure 3) indicates reduced performance when the Attnaeve figures were used compared to the letter stimuli, F(1, 10) = 56.21, p < .001, ηp 2 = 0.85. In addition, performance decreased with an increase in task difficulty, F(2.56, 25.58) = 63.66.0, p < .001, ηp 2 = 0.86, with within-subjects contrasts revealing a significant linear trend, F(1, 10) = 150.78.0, p < .001, np 2 = 0.85. Also the interaction of stimulus type and n-back level reached significance level, F(2.50,25.02) = 6.00, p < .001, ηp 2 = 0.71. Accuracy was similar in the control and 1-back conditions for both visual load conditions but decreased faster in for figures than letters from the 2-back to the 4-back condition.
The reverse pattern was observed for perceived workload (see Figure 4). Subjective workload was higher with Attneave figures than with letters, F(1, 10) = 51.89, p < .001, ηp 2 = 0.84. response accuracy) as a function of task difficulty (mental load) and stimulus type (visual load). Data of the respective two levels of mental load (e.g. the two 1-back trials with letter stimuli) were pooled. Furthermore, it increased according to task difficulty, F(1.16, 11.64) = 57.77, p < .001, np 2 = 0.85, in a linear way, F(1, 10) = 64.08, p < .001, np 2 = 0.84. Finally, this increase was steeper for the figures compared to the letters stimuli, as revealed by the significant interaction effect F(2.22, 22.19) = 9.30, p < .001, np 2 = 0.48. Altogether, these results indicate that the experimental manipulation was successful.
The interaction between stimulus type and mental demand level was significant, F(4, 40) = 11.18, p = .007, ηp 2 = 0.53. As displayed in Table 1, further analyses revealed that the microsaccade rate was significantly higher for figure stimuli compared to the letter stimuli in each mental load condition except for the control condition.
The visual inspection of the temporal evolution of microsaccade rate showed a single peak around 270ms after the onset of the to-be-processed stimulus (see Figure 6). Overall, we observed a similar pattern on microsaccade rate as revealed by the 2x5 ANOVA. Regarding the visual load, a higher peak was observed in the figure condition than in the letter condition. The peak amplitude was also affected by the task type, where higher microsaccade rates were showed in the n-back task than in the control task. Furthermore, there was no noticeable difference across n-back levels for both letter and figure conditions. Finally, the onset of target stimuli generated a similar microsaccade rate as distractors. This suggests that microsaccadic response was modulated by the displayed stimuli but not by differences in required stimulus response (distractor vs target).  Table 1. Descriptive data (mean and standard deviation) and t-test results (t-value, one-tailed significance level and effect size) of microsaccade rate per second as a function of stimulus type (visual load). Figure 6. Time evolution of microsaccade rate as a function of stimuli type (visual load) and n-back level (mental load). The blue dashed line represents the microsaccade rate for the distractors and the solid black line the microsaccade rate for the target stimuli (which required an answer from the participants).

Discussion
The aim of this study was to investigate the influence of visual and mental task demand on microsaccade rate in visual tasks. Results showed that microsaccade rate is linked with visual load but not with mental load per se. Microsaccade rate increased from the control condition (probe detection task) to the onset of the n-back figure task (rather simple mental task but visually complex).
Also, microsaccade rate was higher in the figure condition, which induced more visual load, compared to the letter condition and the control condition. The visual analysis of the temporal evolution of microsaccade rate confirmed that a higher peak amplitude was observed in the figure condition compared to the letter condition. Therefore, we can confirm our first hypothesis, stating that we expect higher microsaccade rate in the n-back figure condition compared to the letter condition. These findings are in line with previous studies (Benedetto et al., 2011;Krueger et al., 2019) which found that visual load in a nonexclusive visual task leads to an increase in microsaccade rate. Interestingly, the increase in mental demand did not lead to a significant change in microsaccade rate. Since the manipulation check was successful (i.e. subjective evaluation of task difficulty increased with increasing n-back level), this implies that we have to reject our second hypothesis, stating that microsaccade rate decreases with increasing mental load. These findings contradict previous studies which argue that tasks with high mental demand reduce microsaccade rate (Siegenthaler et al., 2013;Gao et al., 2015;Dalmaso et al., 2017). Therefore, our study challenges the assumption that microsaccade rate could be a measure for mental workload in tasks which require a certain amount of visual information processing. In our study, we tied the increase in mental demand to a visual task. In all five difficulty levels, participants were asked to identify a letter or a figure and therefore, had to process the visual stimuli to complete the mental task. Therefore, we argue that the amount of required visual processing resources stays the same over the five difficulty levels of the n-back task, which would explain why microsaccade did not decrease despite the increase in mental load.
It could be assumed that in studies showing a decrease in microsaccade rate in tasks of high mental task demand, participants attributed less resources to their visual system and concentrated on solving the demanding mental tasks. In the previous studies showing a decrease of microsaccade rate, mental load was induced through tasks with no visual component whatsoever (e.g. a mental arithmetic task). This resource reduction on visual processes might have led to the decrease in microsaccade rate. In the present study, a minimal amount of visual processing is required in all levels of mental load and hence this could not result in a resource deallocation, suggesting that microsaccade rate is solely linked with visual load and not with mental load.
In comparison to Krueger et al. (2019), our mental load condition required some degree of visual information processing and visual load was induced through a task demanding some mental information processing. In Krueger et al. (2019) the mental load was induced trough a mental arithmetic task requiring no visual information processing and the visual load was induced through a visual task requiring some mental information processing. Krueger et al. (2019) found that the increase in visual load lead to an increase in microsaccade rate. However, in combination with a difficult mental arithmetic task, this increase did not happen. Hence, the conclusion was that the mental arithmetic task pulled away resources from the visual task. However, it would also be possible to argue, that the visual task pulls away resources from the arithmetic task because if presented alone, the arithmetic task lead to a reduction in microsaccade rate.
The results of this study show that mental demand, if bound to a visual task, does not lead to a decrease in microsaccade rate in relation to difficulty increase. Therefore, we conclude that microsaccade rate indeed primarily is influenced by visual load and that a change in microsaccade rate through mental load displays a resource redistribution away from the visual stimuli.

Task manipulation
In order to exclude potential other influencing factors, we analyzed whether microsaccade rate was modulated by the required action after stimulus presentation (i.e. pushing a button on the keyboard or not). Data analysis revealed no considerable influence of the required action on micorsaccade rate which did not differ between target (requiring action) and distractor stimuli (requiring no action). On the other hand, miccrosaccade rate was increased during stimulus presentation (i.e. target and distractor) compared to the presentation of a fixation cross for both stimuli types (letter and figure). This allows us to conclude that the increase in microsaccade rate observed for figure stimuli (as compared to the letter stimuli) was due to the increase in visual load induced by the figure stimulus type.

Limitations and future research
In the control task of this experiment, participants were required to identify a predefined stimulus (a star) among distractors (a diamond). However, it would have been more accurate to use letters and Attnaeve figures as control stimuli in this probe detection task. In addition, it could be speculated that differences between stimulus types might be due to the task domain (verbal versus visual-spatial) instead of visual complexity. In a fMRI study however, Ragland and colleagues (2002) showed that the same cortical areas are activated during a n-back task using either overlearned (letters) or meaningless stimuli (fractal figures). This suggests that both types of stimuli are processed by the same working memory circuitry. Therefore, the difference in microsaccade rate observed in this study may be primarily attributed to the difference in visual complexity of the stimuli and not to their processing by different working memory systems. While the focus of this piece of research lies in the modulation of microsaccade rate through different task modalities, further research should investigate the link between sustained level of microsaccade rate and event-related modulation (e.g. oculomotor inhibition). In this context, more microsaccades would be expected in focal attention mode of visual inspection compared to ambient attention in a free viewing task (e.g. Krejtz et al. 2016).

Implications
The findings presented have several implications for research and practice. Our results support the assumption that in tasks which hold a strong visual component, microsaccade production is increased compared to tasks with no visual component. This implies that a high microsaccade rate (compared to a baseline measurement for the respective person) can indicate if a subject is processing visual information in a task. Microsaccade rate increases in a task if visual load increases. In addition, findings of this study indicate that the decrease of microsaccade rate reported in other studies might not be due to an increase in mental demand as suggested in previous research but be linked with a shift of resources from visual towards mental processes. It remains to be evaluated whether tasks induced by other sensory modalities (e.g. auditory or tactile system) would lead to the same result pattern.
Within an applied context, these findings suggest that visual information processing might be monitored in real time through continuous microsaccade detection. As mentioned by Krueger and colleagues (2019), being able to continuously assess the visual attention of a person, for example whether their focus is drifting into inattentional blindness, could be of great value. This could be useful in applied settings where human factors are critical, such as airport baggage screening, air traffic control, radar operating or driving.