Nowadays, technological advances and vendor competition are steadily lowering the price of the eye-tracking technology. Research institutions can buy more eye-tracking equipment, i.e., more individual eye-tracking stations (12). So much so, the number of eye-trackers can easily surpass the number of rooms available to house them, or the number of personnel available (and able) to use them in a “traditional” study setup. By a traditional setup, we mean studies, in which the participants work one-at-a-time on a single eye-tracking station. In such setup, the study moderators manage the participants one-to-one. As the prices go lower, an institution may want to furnish couple more “traditional setup” eye-tracking labs. This way, however, the low prices of eye-tracking technology cannot really be exploited, because the personnel and overhead costs would need to scale as well.

It may, however, make sense to arrange multiple eye-trackers in a different setup. Multiple eye-tracking stations (e.g., 20 PCs) can be placed together into a group eye-tracking room. In such physical setup, studies will no longer have the one-to-one, but one-to-many design. In such design, the moderator can (at the same time) manage multiple participants working in parallel.

A group eye-tracking setup requires a special infrastructure. The system must provide means to design the experiments, effectively distribute the scenarios to workstations, orchestrate the work and collect the recorded data.

To this day, laboratories with the group eye-tracking setups have existed for some years in several (but not many) institutions worldwide (14, 2, 8, 6, 19). Software solutions with features allowing the control of multiple eye-trackers also started to emerge (12). However, as a discipline, the group eye-tracking is not yet well described, discussed and methodologically established. Only recently, fora dedicated to this field started to emerge. And, seasoned infrastructural solutions are not yet available.

In comparison with the traditional setup, group eye-tracking has several advantages, but also limitations and challenges. Depending on the study requirements, the trade-off between the pros and cons can, in many cases, play in favor of the group setup.

The advantages and benefits of the group eye-tracking include:

The limitations and challenges of the group eye-tracking include:

Today, the group eye-tracking requires a custom software and hardware infrastructure. The available eye-tracking software tools (e.g., Tobii Studio, SMI Experiment Center, OGama) are suited for single-user experiments and are generally inadequate for the group studies. To be fair, we are aware of the initiatives of some traditional eye-tracker vendors and other companies to develop a solution for the group eye-tracking use. There are also some open source initiatives 12). Still, as in our case, laboratories tend to develop and maintain their own solutions for practical reasons.

The main source of inadequacy of existing tools is the absence of proper study management features. Especially missing are the features for the centralized remote control and monitoring of the study process. A group study also requires an effective distribution and collection of the required data (stimuli, tasks, logs) to and from the workstations. Finally, a high-level programmatic control of the eye-trackers is seldom available outside of the vendor’s canned (closed) tools. Although the eye-trackers do have low-level SDKs, these require a lot of programming effort to be set up for studies. This hampers the integration of external applications, often required in the studies.

We have overcome some of these challenges by building a custom infrastructure for the group eye-tracking laboratory at our institute.

Our system, the Group Studies, was developed and is currently deployed at the User Experience and Interaction Research Center (UXI) at the Slovak University of Technology in Bratislava.

The principal high-level requirements of the Group Studies system are:

Following these requirements, we designed the Group Studies system as a thick client-server application. The system consists of two principal components:

Our system works primarily with the Tobii technology but allows the integration with devices from other vendors. It is implemented in C#, utilizes .NET and Windows ecosystem and relies on a fast intranet connection between its elements (10 Gbps in our case). However, since the bulk of the recorded data (screen recordings, eye-tracker logs, etc.) is sent from the individual workstations to the server after the session end, it could be in theory used also in a setup, in which the server and clients do not reside on the same local network.

The system was designed iteratively and incrementally. We base it on our experience with the study organization systems and experimental education platforms, which support group classroom experiments (25, 26). We were also inspired by crowdsourcing systems such as Mechanical Turk and systems for interactive experiment support (21).

Our system distinguishes between the two types of users: (1) the study owners, who interact with both UXR and UXC and (2) the study participants who interact with the UXC. The study owner role covers the study designers, moderators, and analysts.

Following is a typical workflow of an experiment in the Group Studies (see also Figure 1):

The client application (UXC) is autonomous to a large extent. The UXC can be run on its own, without the connection to the UXR (the server-side application of our system). This has several advantages. First, when experiments are run in the group session laboratory, the system is less prone to server and network failures. Second, it allows the experiments to be designed and tested anywhere, using only a single machine, even without the eye-tracker (which can be substituted by a mock input).

The experiment is defined using a data structure called the Session definition. This structure is stored as a JSON file. It contains various setup parameters and most importantly, the timeline. The timeline is a sequence of stimuli, questionnaires, calibrations, calibration validations and other events, that the participants encounter during the session.

Defining the timeline through JSON files differs from other eye-tracking tools, which usually use graphical interfaces. We chose this approach, because of its:

A downside of using such "programmatic" approach is, of course, the lower accessibility of our system for study owners with a non-technical background. Yet, using the learn-by-example approach, even the non-technical persons can quickly grasp the principles of the JSON session definition, especially when they have the access to a battery of example scenarios.

The following section lists the functionality provided by the Group Studies system. We present it component-wise (first UXR, then UXC). In addition, we describe the options available for Session definition JSON file, which is defined outside of the both components.

The Group Studies system has two principal components (1) UXR – the web application for experiment management and (2) UXC – the desktop client application for operating the eye-trackers and stimuli. The system also allows the use of (3) external applications, which are often required to serve as stimuli. Figure 7 (appendix) shows interconnection of these system components. Based on the use case, an external application may use any of the interfaces provided by the client application, i.e., push events, read gaze data or even control the experiment timeline.

The role of the UXR (which runs on a web server) is to support the use cases for the study setup and control, as well as retrieving data after the experiment. The UXR also serves for distributing UXC updates. Figure 8 (appendix) shows the main internal components of the management application, built on top of the Microsoft ASP.NET MVC framework and Microsoft SQL Server database.

The client application (UXC) is autonomous during the session execution. The architectural style of the system is a thick-client. The client application can receive information (e.g., a session definition) and commands (e.g., a synchronized recording start) from the server (the management application), but apart from that, the client application manages the session autonomously. The client application implements the eye-tracker calibration, records the session data (e.g., eye-tracking data, screen recording, user camera, keyboard, and mouse events) and sends them back to the server after the session finishes.

The autonomous character of the client application is important, because it increases the robustness of the system, which is thus less prone to server and network failures caused by the bottlenecks.

Figure 9 (appendix) shows the main internal components of the UXC with the Sessions Control module for controlling the session recording. The data sources (devices) are controlled automatically by the Sessions Control through the Adapters Control module. The data source components are adapters, which implement routines required for collecting the specific data types, but which share the same internal interface for the Adapters Control module.

The Eye-tracker component uses Tobii Pro SDK2 library to communicate with a Tobii Pro Eye-tracker device. FFmpeg3 is used for recording multimedia: the participant’s screen with the UScreenCapture4 software and a webcam available on the workstation. The Mouse & Keyboard component records participant’s keystrokes, mouse clicks and movements using the WinAPI provided by the Microsoft Windows operating system. A special data source type is External Events which allows external applications to add events recorded during the experiment. During the whole session recording, the experiment timeline is played and gaze data may be accessed by an external application.

When the external applications are going to be used, the study owner, or a developer of the application must implement communication with the UXC local web services, either using REST API or web sockets. The external applications can be either desktop applications or browser-based applications with their own web servers. The external applications communicate with the UXC through the localhost domain. This helps to preserve the overall workstation autonomy. A problem arises if the external application is secured (i.e., uses HTTPS). This can be solved with advanced configuration of the workstation, which we provide details about in the project documentation.

Physically, the Group Studies system is, with exception of the server, entirely deployed in the room where the group experiments take place. The room can be seen in the Figure 4 during an experiment. 20 workstations are positioned to form a classroom. In our setup, each workstation is equipped with a 60Hz eye-tracker (Tobii Pro X2-60) and a web camera (Creative Senz3D). One additional workstation is dedicated for the study owner and is equipped with a projector. The study owner can use the workstation for controlling the recording of an experiment session. The server side of the system (the management application) runs on a dedicated server, which also hosts the data storage, allowing direct and single-point access to the recorded data.

So far, we have used our infrastructure for several studies. These included studies on cleaning pupillary dilation data from the real-world stimuli lightning effects (10), student attention during an interactive lecture (26) (see also Figure 4), visual search on real websites (7), detecting deception in the questionnaires (20), or the eye-tracking aided crowdsourcing (24).

To demonstrate the use of our Group Studies system in the real lab settings, we present a setup from a study for our ongoing research on the program comprehension (27, 11) where participants’ task is to read the source code fragments, understand them and answer the comprehension questions.

We made the software components of our infrastructure publicly available as source code and documentation. We publish the software in several GitHub repositories:

The documentation for the source code is placed within the wiki sections of these repositories.

The Group Studies system is currently best suited for scaling up the eye-tracking experiments, which otherwise have a “single-participant nature”. This means experiments, which do not involve any interaction between the participants (workstations). Many studies are like this and the group eye-tracking simply helps them to be finished faster.

There is, however, an entire line of research dealing with the between-participant interactive eye-tracking scenarios (1, 13, 18, 5). Such scenarios are currently not supported by our system, as there is no direct native support for the data exchange between the workstations.

Despite that, our system does not prevent collaborative scenarios and provides suitable basis for implementing them. By a suitable basis, we mean the local API capabilities of the UXC application. Using this API, an external application can request gaze data from the UXC (UXC can provide the actual scalar or buffered historical data). Therefore, if a collaborative scenario were required to be run on our infrastructure, the data exchange would have to be implemented in the external application.

The prices of the eye-tracking technologies are steadily dropping and allow larger (hardware) purchases. This allows institutions to furnish more conventional eye-tracking labs, but it also opens the possibility to furnish labs with the group eye-tracking setups. However, running studies in such setups requires special infrastructure to support them.

This paper presented the Group Studies system, a software part of a group eye-tracking infrastructure deployed at the User Experience and Interaction Research Center (UXI). The system is based on a thick client-server architecture. It allows flexible preparation of the experiment scenarios and integration of the 3^rd party software and is extendable in the future.

We have described the functionality of the system, which supports all required phases of a group eye-tracking experiment. We have also looked at the system from the architectural perspective and shown a high-level overview of its components. This overview serves as a good introduction into the entire implementation of our system, which we made publicly available on GitHub.

The system was primarily designed to support our specific needs in conducting the group eye-tracking studies (at UXI Research Center). It was designed through multiple iterations and evolved over time. Despite that we believe that it can inspire new labs as well. Moreover, researchers can use our code and modify, tailor and deploy this system at their own lab sites.

The system does not support all possible scenarios for the group eye-tracking or the user study designs right now. But also, it does not prevent them. For example, we did not focus it on the collaborative scenarios. Therefore, researchers pursuing this path would be required to put additional effort to use it for these studies. Nevertheless, the ability of our system to integrate external software into the infrastructure would enable such scenarios.

We see several possible directions for the future work. First, we understand that the stimuli timeline structure in its current state may be limiting for certain studies because of its linearity. It is possible to define alternative session timelines when scheduling the session in the UXR or control the stimuli timeline and insert new steps during the recording using the UXC local API from a 3^rd party application. However, it is currently not possible to randomize or counterbalance the order of the pre-defined timeline steps, nor is it possible to conditionally select the next step during the recording, possibly based on the results from the previous steps. Another possible feature (and direction for future work), which we identified the need for during our studies, is to validate the eye-tracking data on completion of the Eye-tracker validation step during the recording and request the participant to re-calibrate the eye-tracker. Thanks to the design and architecture of the presented system, it will be possible to implement these features in the system in the future.

The authors declare that the contents of the article are in agreement with the ethics described in http://biblio.unibe.ch/portale/elibrary/BOP/jemr/ethics.html and that there is no conflict of interest regarding the publication of this paper.

This article was created with the support of the Slovak Research and Development Agency under the contracts No. APVV-15-0508 and APVV-17-0267, the Scientific Grant Agency of the Slovak Republic, grant No. VG 1/0646/15, the Ministry of Education, Science, Research and Sport of the Slovak Republic within the Research and Development Operational Programme for the project “University Science Park of STU Bratislava”, ITMS 26240220084, co-funded by the ERDF and the project “Development of research infrastructure STU”, project no. 003STU-2-3/2016 by the Ministry of Education, Science, Research and Sport of the Slovak Republic.

We wish to thank Tobii Pro for fruitful collaboration with setting up and maintaining our lab infrastructure.