The binocular eye tracker (see Figure 1a) consists of two infrared sensitive monochrome USB3.0 cameras (The Imaging Source, www.theimagingsource.com, camera model DMK33UX174). Both cameras are run at a video frame size of 640x480 pixel and 8-bit grey levels (software selectable monochrome video format: Y800) with a frame rate of 395 Hz (specified maximal frame rate of the cameras and checked by counting the number of frames that were processed in 60 secs). Both cameras are equipped with a lens with 50 mm focal length and a f/# of 1.4 (ROCOH TV Lens 50 mm 1:1.4). The camera sensor specifications are as follows: ¹/_1.2-inch Sony CMOS Pregius sensor (IMX174LLJ); pixel size is H: 5.86 µm, V: 5.86 µm. The number of effective pixels is 1936 (H) x 1216 (V), with the maximum resolution being 1920 (H) x 1200 (V). The lenses are covered by a daylight cut-off filter (The Imaging Source, https://www.theimagingsource.de/produkte/optik/filter/, #092, 46 x 0.75). Three 5 mm extension rings are necessary to focus the cameras on the eyes at a distance of 250 mm which results in a video magnification of 39.7 pixel/mm. Both eyes are illuminated by a single circular arrangement with a diameter of 40 mm of 15 high power IR LEDs emitting at 875 nm (https://www.conrad.de/de/p/hp-hdsl-4230-hp-ir-emitter-875-nm-17-5-mm-radial-bedrahtet-185809.html). The LED field was placed 85 mm below the cameras and adjusted to illuminate both eyes from below and generate two bright and large Purkinje images in the two eyes. We used a gaming computer (Memory PC Intel i7-7700K 4X 4.2 GHz, 4 GB DDR4, 500 GB Sata3) and a computer screen with a refresh rate of 240 Hz (Acer Predator XB252Q, 24.5”, resolution of 1920 x 1080 pixels) (see Figure 1b), although neither is mandatory to do binocular eye tracking at the full speed of 395 Hz.

Software was developed under Visual C++ 8.0 to merge both camera inputs into one video buffer and to track both pupil centers and first Purkinje images (see Figure 2). Bright and large first Purkinje images were generated by the circular field of 15 infrared LEDs below the cameras. It can be simply estimated how precisely the center of the pupil and the first Purkinje image position must be determined to achieve an angular resolution of 1 arcmin. It is known (17, 30) that, on average, the first Purkinje image moves one millimeter relative to the pupil center when the eye rotates about 12 degrees (Hirschberg ratio). Accordingly, for one degree, the displacement would be 83 µm; for 1 arcmin of eye rotation, it would only be 1.39 µm – close to one thousandth of a millimeter. This estimation illustrates how precisely the pupil center and first Purkinje image center need to be detected to reliably measure fixational eye movements, for example. Pixel magnification in the current set-up was 39.7 pixel/mm or 25.2 µm/pixel. Accordingly, a one-pixel change in position in pupil center of the first Purkinje image was equivalent to 18.1 arcmin, not yet the range of fixational eye movements. However, because a 4 mm pupil already generates about 20,000 dark pixels and a bright first Purkinje image about 400 pixels, their centers of mass could be determined with subpixel resolution for their positions. In our setup, the positions were determined with a resolution of 0.2 pixels, equivalent to about 3.6 arcmin. The pupil was located by a simple thresholding procedure – all pixels that were darker than an adjustable threshold (default: 0.6 darker than the average image brightness) were stored, the center of mass determined, and the pupil area measured as the number of dark pixels. Pupil radius was determined as

The pupil border was graphically denoted by a circle and could be optimized by manually adjusting the threshold. The same procedure with an inverted threshold was applied to determine the center and diameter of the first Purkinje image, which was also marked with a green circle. The pixels in the Purkinje image are typically close to saturation, and the pixel threshold for their detection was set to 250, independently from the average brightness of the video image. That is, pixels higher than 250 in intensity were considered part of the Purkinje image. We have also included a simple focus detection algorithm, counting the number of pixels in the Purkinje image. The size of the Purkinje image is determined by the size and distance of the IR LED field that generates it, and also of defocus.

We used the PC-CR (pupil center – corneal reflection) vector technique to measure the angular position of the eyes (31). The detected eye positions are initially written down to the file in pixel coordinates (the coordinate system of the image), but we used the calibration procedures described below to also obtain degrees of visual angle.

To determine how stable the detection of the pupil center and the center of the first Purkinje image was, a running standard deviation was determined, continuously taking the data of the latest 25 samples. Sampling at 395 Hz, 25 data points are equivalent to 63 ms, which is too short to be severely affected by ocular drifts. It therefore reflects how repeatably the positions are detected in each frame. Standard deviations ranged from 0.2 to 0.5 pixels. These data are continuously displayed on the screen for both eyes to be able to judge the reliability of eye tracking. In addition, a more conservative measure of measurement noise was performed – determining the average absolute difference between two subsequent measurements in the horizontal direction, again determined over the latest 25 measurements. These data were also displayed.

Since the standard deviations of pupil sizes over the latest 25 frames were also available, they could be used as a sensitive way to detect blink artifacts. During blinks, the pupil is rapidly covered by eye lids and the number of black pixels declines. A standard deviation of pupil sizes exceeding 0.2 mm was found to denote blinks (since pupil size cannot change fast, and pupil responses are slow in the absence of blinks). In this case, data were continuously written, but the data file contained zeros in all data columns.

Because the same LED field served as a light source for both eyes, the first Purkinje images were not at the same position in the pupils of both eyes, and a calibration procedure was done simultaneously for both eyes, but independently. Four red fixation points (diameter 4 arcmin) appeared on the screen, one after the other. They were arranged in a rectangle, which could be adjusted in size from the keyboard before calibration. When the subject fixated, the running standard deviation of eye positions dropped to a value below 0.5 degrees. This triggered the fixation point to turn green, and the averages of 100 samples and 100 first Purkinje images were stored. The next fixation point appeared, and the procedure was repeated. After the calibration procedure was completed (i.e. after approximately 2-3 seconds), any eye position within the rectangular field could be inferred by linear extrapolation. At this point, it is necessary to consider how linearly the distance between pupil center and Purkinje image center are related to the true eye position. Linearity of this procedure was tested for the central +20 degrees (display size of approximately +20 cm from the screen center) of the visual field in the experiments described below. Outside this range, irregularities of corneal curvature as well as changes in the position of the center of rotation of the eyeball cause non-linear conversions into eye positions, which were not analyzed for the present paper. More sophisticated calibration procedures can account for such non-linearities, depending on the intended application of the eye tracker (17).

Since a stationary pupil center position cannot be assumed when pupil size changes (15), we implemented an automatic procedure to correct for potential pupil center drifts when measuring binocular fixational eye positions. These binocular fixation measurements are the measurements for which pupil center drifts caused by pupil size changes are the most problematic, given the similar amplitudes of the movements and the pupil size artifacts. After calibration, a single fixation point was presented on a black background in the center of the screen for 4 seconds. Due to the black screen, the pupils dilated. While fixation was maintained, the screen suddenly turned brighter to a pixel grey level of 150 (on an 8-bit gray scale) for a duration of 30 frames (about 75 ms), which elicited a prominent pupil constriction. While such a manipulation can also alter fixational eye position (32), the effect on eye position is minute in comparison to pupil center drifts and also occurs before pupil dilation. Eye positions were continuously recorded during the pupil responses. After another 600 ms, the angle of eye convergence was automatically plotted against pupil size. If the pupil center position was not stationary but rather moved when pupil size changed, then this became evident as a significant convergence of eye position. Specifically, the software plotted pupil sizes versus convergence measures and performed a linear regression. If the slope of the regression was significantly different from zero, a correction was necessary, and it was implemented in the subsequently recorded data. This was done by simply taking the regression equation in the plot of vergence versus pupil size, and re-calculating vergence for each of the known pupil sizes.

To make sure that our device can measure eye movements and fixations correctly, we compared it to the well-known eye tracker, EyeLink 1000 Plus (SR Research, Ottawa, Ontario, Canada). This is one of the most popular and established commercial devices used for binocular measurements. We built a set-up that included the two eye tracking systems simultaneously: the first one was our computer (Memory PC Intel i7-7700K 4X 4.2 GHz, 4 GB DDR4, 500 GB Sata3), the monitor (Acer Predator XB252Q, 24.5”, resolution of 1920 x 1080 pixels), and the custom-built device, and the second one was the EyeLink 1000 Plus system with its own computer. Stimuli and calibration points were presented on our monitor. We used the chin rest to fix participants’ head in order to avoid any unnecessary movements. The calibration procedure included four points appearing on the screen in sequence. The EyeLink 1000 Plus (desktop mode, binocular, 25 mm lens, stabilized head) was recording data without calibration; we calibrated it later offline using the same calibration points as those used for the custom-built eye tracker. To compare the temporal and spatial performance of our eye tracker with an established device (EyeLink 1000 Plus), a TTL (transistor-transistor logic) signal was generated by the custom-built eye tracker each time a new fixation target appeared. This signal was fed into the EyeLink system as an external event (similar to button press devices connected to the EyeLink system). This served as a time stamp for simultaneous data recording with both devices (see Figure 3). We used the infrared illuminator (LED) from the custom-built eye tracker for both devices. This was acceptable because the spectral content of our illuminator was very similar to that of the EyeLink system (as we confirmed experimentally by measuring them). Before the experiment started, we made sure that the eye was perfectly illuminated in the EyeLink eye tracker. This allowed us to make simultaneous recordings.

After the recording session, the following data could be written to a file for each single frame: computer time, frame number, pupil sizes for left and right eyes (mm), x positions for left and right eyes in screen coordinates of the fixation points (pixel), vergence in arcmin with and without pupil centration correction, x and y positions for fixation targets, “1” when a TTL signal was emitted and “0” if there was none.

We measured ten participants (three male, age range 21–26 years). They had no known ocular pathologies or binocular irregularities, other than moderate refractive errors that were corrected by their habitual spectacles or contact lenses. The experiment was conducted in agreement with the Code of Ethics of the World Medical Association (Declaration of Helsinki) and approved by the Ethics Commission of the University of Tuebingen. Informed consent was obtained from all participants.

The only way to completely eliminate any eye movements and other biological factors from the eye tracker signal is to use artificial eyes (33). For better and more optimal comparison of the precision between our eye tracker and the commercial system, we first used the artificial eyes shown in Figure 4 (MagiDeal, 16 mm, https://www.amazon.de/St%C3%BCck-H%C3%A4lfte-Acryl-Dollfie-Eyeballs/dp/B008S3S9H2). These artificial eyes were very similar to real ones since they also had an iris, cornea, and even a corneal reflection.

The eyes were made of acrylic, and they had a diameter of 16 mm. Pupil diameter was 4 mm. The eyes were mounted on the chin rest at the same distance and height as the participants’ eyes, and we proceeded to perform simultaneous measurements with both eye trackers. We avoided desk vibrations as much as possible, to avoid measuring artifactual displacements.

In order to demonstrate that our eye tracker was well prepared for doing binocular measurements, we performed an additional experiment exercising vergence eye movements. We asked a participant to look at three different targets located at different distances from the computer screen while measuring eye movements with our eye tracker. We used the same calibration procedure as described above before asking the participant to look at the different distances.

For each trial we used two targets between which the participant fixated. One target was located on the computer screen, and the other one was located on a special holder similar to the one that we used to hold the cameras of our eye tracker. The holder was mounted on a metal horizontal panel. This panel allowed us to move the target back and forth depending on the distance of the target that we wanted to apply. Both targets were 1x1 mm yellow squares. They were created using yellow tape.

The monitor was located at 54 cm from the participant’s eyes. We first put one target at a distance of 49 cm from participant’s eyes. Next, the target was located at 44 cm, and then the last one was at a distance of 29 cm. During the first trial, the participant was asked to look first at the target located on the screen (this corresponded to 6.360 degrees of vergence angle), and then to the target located at 49 cm (7.008 degrees) from the eyes. The next task was to look at the first target (6.360 degrees) and then to the target located at a distance of 44 cm (7.800 degrees). During the last trial, the participant was looking at the target with the distance of 54 cm (6.360 degrees) and then at 29 cm (11.812 degrees).

For the offline calibration of the EyeLink 1000 Plus system, we first chose fixation periods (free of saccades and microsaccades) of 100 ms for each calibration point (similar to our calibration approach of our custom-built eye tracker). After that, the average eye position of this piece of data was found. For each of the five calibration points (including the center point), we obtained a best-fit second-order polynomial for the measurements (34, 35).

Saccades and microsaccades were detected using U’n’Eye – a deep neural network for the detection of saccades and other eye movements (36). First, we trained the network on our data. For this, we took 60 seconds of data that included fixations, saccades, and microsaccades. For the training set, saccades and microsaccades were manually labeled with 1 and fixations with 0. The output of the training process was the trained network weights that were later used for the saccade detection. In the end, we had a file with saccade labels for each trial.

An eye tracker performance is usually described using two metrics: precision and accuracy. Precision is the ability of the eye tracker to reliably reproduce the same gaze point measurement (36). Precision values of currently available pupil-based eye trackers range from 0.01 to 1 degree (36). Accuracy is the average difference between the real stimulus position and the measured gaze position. Typical accuracy values for pupil-based eye trackers fall in a range between 0.5 and 1 degrees (36). Accuracy and precision are usually measured separately for horizontal and vertical positions, and for the right and the left eye, or as an average of both eyes.

We estimated the precision of our binocular eye tracker using two methods: 1) by calculating the horizontal and vertical root mean square (RMS) noise (that is: the RMS of the inter-sample angular distances) over all samples, and 2) by calculating the horizontal and vertical standard deviation of the samples. The RMS noise was calculated using the following equation

where θ means the angular distance between successive fixation data samples (x _i, y _i ) to (x _{i + 1}, y _{i + 1}) (sample-to-sample distances). The resulting values were averaged across trials.

For the best comparison of precision between devices, we used artificial eyes. The measurements took place under the same light conditions for both eye trackers. We used the same methods of calculating precision RMS and precision standard deviation as described above. One trial of recording the data took 15 seconds, and we later divided the measurements into one-second epochs. Precision was calculated across 15 epochs and then averaged across them. The results are summarized in Table 1 for both our eye tracker as well as the EyeLink 1000 Plus system. As can be seen, our eye tracker outperformed the much more expensive system for horizontal eye movements, and it exhibited similar performance for vertical eye movements. This is despite the fact that our eye tracker had a lower sampling rate. However, this is not a major issue given the bandwidth of eye movements in general and given that precision standard deviation measures are less dependent on the sampling rate of the eye tracker (36).

Having established the robustness of our eye tracker with artificial eyes, we next aimed to validate its performance with real data obtained from human participants. We recruited a total of ten participants who performed simple fixation and saccade tasks. Figure 5 shows raw data plots obtained from one sample participant. The curves in blue show the measurements of eye position with our custom-built binocular eye tracker.

The curves in orange show the measurements with the EyeLink 1000 Plus system. The participant was asked to track the position of a fixation spot as it jumped on the display (fixation spot locations are shown in the figure with dashed black lines; note that there is a delay between fixation spot jump and reactive saccade due to physiological reaction times). For simplicity, we show only the positions of the left eye, but the measurements were naturally binocular. As can be seen, simultaneous measurements of eye position between the two systems largely overlapped. In particular, saccade times were coincident. However, there were also subtle differences in eye position reports in some cases. Our summary analyses below explain the possible reasons for such discrepancies.

Across participants, we obtained an accuracy estimate by picking a fixation interval in a given trial and averaging horizontal and vertical eye position during this interval. The intervals included periods of time when participants were fixating the certain target excluding saccades, microsaccades, and blinks. During these intervals, participants were given an instruction to fixate the target for 1.5 seconds. Figure 6 shows example measurements from one participant for all five fixation points. As can be seen, both eye trackers performed well, but the error between target and eye positions in the EyeLink 1000 Plus system was bigger. To quantify this, we calculated a horizontal or vertical average offset within a trial from the true target location. We did this for each participant after excluding missing data, blinks, and microsaccades. All precision and accuracy calculations were done using the data obtained from the left eye of each participant. The resulting values were averaged across all participants. For precision, we used similar procedures to those described above with artificial eyes.

For the participant in Figure 6 (the same person as that shown in Figure 5), the average eye position error with our eye tracker was 0.4304 degrees, whereas it was 0.7848 degrees with the EyeLink 1000 Plus system. Thus, our eye tracker outperformed the EyeLink 1000 Plus system.

Across participants, Table 2 provides quantitative numbers.

The figure and table also provide our precision estimates. We found that accuracy was better with our eye tracker when compared to the EyeLink 1000 Plus system, but precision RMS was worse. This is explained in part by the higher sampling rate of the EyeLink 1000 Plus system. On the other hand, the superior accuracy performance of our eye tracker is likely due to a much more optimal placement of the cameras – almost level with the participants’ eyes (see Discussion).

We next measured saccade metrics. We detected saccades in the recorded traces independently for our eye tracker and for the EyeLink 1000 Plus system. For this purpose, we used a machine learning approach (37), and we trained a neural network on each eye tracker’s data individually. We then ran the network on the rest of the data.

We measured saccade latency, saccade duration, saccade amplitude, and saccade peak velocity. Saccade latency (ms) was defined as the difference between time of the fixation point appearance and the time when saccade happened. Saccade duration (ms) is the time passed from the first point of a saccade to the last one. Saccade amplitude (degrees) was defined as the Euclidean distance between the start point of the saccade and the last point. Peak velocity (degrees/second) was defined as the maximum value in the radial velocity trace. Correlations between the metrics given by the custom-built eye tracker and the metrics given by EyeLink 1000 Plus system were obtained. The results are shown in Figure 7. As can be seen, saccadic metrics were highly correlated in two eye trackers. Although, sometimes small differences in saccade latency and duration existed.

Since fixational eye movements are also of interest in a wide array of psychophysical applications ( 38, 39, 40, 41, 42, 35, 43), we also evaluated how well our eye tracker can detect microsaccades. Both eye trackers could detect microsaccades well. Agreement between the eye trackers was assessed using analyses like those shown in Figure 8.

It is clear from the results that our eye tracker was able to measure microsaccadic metrics. However, some discrepancies existed between the two eye trackers for some saccadometry measures. For example, it can be seen that the correlation between microsaccade amplitude was not as perfect as it was for larger saccades. However, all other parameters, such as duration and velocity, showed a very high statistically significant correlation between the two eye trackers.

We also checked whether our eye tracker missed some microsaccades that were detected by the EyeLink 1000 Plus system, or vice versa. To do this, we took all microsaccades detected by one eye tracker, and we asked what fraction of them was also detected by the other. For all microsaccades detected by our eye tracker, 100% were also detected by the EyeLink 1000 Plus system. However, for all microsaccades detected by the EyeLink 1000 Plus system, 92% of them were detected by ours. This is likely attributed to the lower precision performance of our eye tracker with the real eyes, perhaps due to the lower sampling rate.

Finally, in order to demonstrate that our eye tracker was well prepared for doing binocular measurements, we performed an additional experiment exercising vergence eye movements. We asked a single participant from our lab to look at three different targets located at different distances from the computer screen while we measuring eye movements with our eye tracker.

The participant was asked to look at two targets in sequence: the first one was always the target located on the screen, and then the other one was located on the holder that was nearer to the participant’s eyes. This induced vergence eye movements that are shown in Figure 9. The subject then alternated back and forth between the target depths. As can be seen, our eye tracker was capable of tracking both small and large convergence and divergence eye movements. This means that our eye tracker is suitable for a wide range of experiments involving binocular vision.

The author(s) declare(s) that the contents of the article are in agreement with the ethics described in http://biblio.unibe.ch/portale/elibrary/BOP/jemr/ethics.html and that there is no conflict of interest regarding the publication of this paper.

This study was supported by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation), project number 276693517, SFB 1233, project TP11.

We are grateful to Dr. Torsten Strasser for his help to merge both camera inputs into one video buffer in Visual C++, and to our participants for their patience and efforts.