Bleisch ( 1 ) defines 3D geovisualization as a generic term used for a range of 3D visualizations representing the real world, parts of the real world, or other data with a spatial reference. With the advent of virtual globes such as Google Earth, or perhaps even earlier with the notion of a digital earth ( 2 ), they have become increasingly popular, and many people already know about 3D geovisualizations even though they may not call them as such. Most 3D geovisualizations are digital elevation models draped with ortho or satellite imagery and relatively detailed 3D city models ( 1 ). These perspective views are often referred to as 3D maps. The overview of the usability and usefulness of 3D geovisualizations was presented by Çöltekin, Lokka ( 3 ). Authors categorized the results from existing empirical studies according to visualization type, task type, and user type.

3D geovisualization is not limited to the depiction of terrain where the Z axis represents elevation. The development of a phenomenon in time is often displayed, for example, with the aid of a so-called Space-Time-Cube (STC). Hägerstraand ( 4 ) proposed a framework for time geography to study social interaction and the movement of individuals in space and time. The STC is a visual representation of this framework where the cube’s horizontal plane represents space, and the 3D vertical axis represents time ( 5 ). With a Space-Time-Cube, any spatio-temporal data can be displayed. That data can be, for example, information recorded by GPS devices, statistics with location and time components, or data acquired with eye-tracking technology ( 6 ).

3D maps and visualizations can generally be divided into two categories: static and interactive. Static visualizations are essentially perspective views (images) of any 3D scene. In interactive 3D visualizations, the user can control and manipulate the scene. The disadvantages of static 3D maps are mainly overlapping objects in the 3D scene and the distortion of distant objects. Inexperienced users could have problems with scene manipulation using a mouse ( 7 ).

Most of the cases referred to as 3D geovisualization are not true 3D, but a pseudo 3D (or 2.5D – each X and Y coordinate corresponds to exactly one Z value). According to Kraak ( 8 ), true 3D can be used in those cases where special equipment achieves realistic 3D projection (i.e. 3D LCD displays, holograms, stereoscopic images, anaglyphs or physical models).

Haeberling ( 9 ) notes that there is almost no cartographic theory or principles for creating 3D maps. In his dissertation, Góralski ( 10 ) also argues that solid knowledge of 3D cartography is still missing. A similar view can be found in other studies ( 7, 11-13 ). The authors report that there is still very little known about how and in which cases 3D visualization can be effectively used. Performing an appropriate assessment of the usability of 3D maps is necessary.

Due to the massive increase in map production in recent years, it is important to focus on map usability research. Maps can be modified and optimized to better serve users based on the results of this research.

One of the first works dealing with map usability research was published by Petchenik ( 14 ). In her work "Cognition in Cartography", she states that for the successful transfer of information between the map creator and map reader, it is necessary for the reader to understand the map in the same way as the map creator. The challenge of cognitive cartography is understanding how users read various map elements and how the meanings of those elements between different users vary.

The primary direction of cognitive cartography research leads to studies in how maps are perceived, to increase their efficiency, and adapt their design to the needs of a specific group of users. The International Cartographic Association (ICA) has two commissions devoted to map users, the appraisal of map effectiveness, and map optimization – the Commission on Use and User Issues ( http://use.icaci.org/ ) and the Commission on Cognitive Visualization ( http://cogvis.icaci.org/ ). User aspects are examined in respect of the different purposes of maps (for example Staněk, Friedmannová ( 15 ) or Kubíček, Šašinka ( 16 ).

Haeberling ( 17 ) evaluated the design variables employed in 3D maps (camera angle and distance, the direction of light, sky settings and the amount of haze). Petrovič and Mašera ( 18 ) used a questionnaire to determine user preferences between 2D and 3D maps. Participants of their study had to decide which type of map they would use to solve four tasks: measuring distances, comparing elevation, determining the direction of north, and evaluating the direction of tilt. Results of the study of Petrovič and Mašera ( 18 ) showed that 3D maps are better for estimating elevation and orientation than their 2D equivalents, but 3D maps may cause potential problems for distance measuring.

Savage, Wiebe ( 19 ) tried to answer the question whether using 3D perspective views has an advantage over using traditional 2D topographic maps. Participants were randomly divided into two groups and asked to solve spatial tasks with a 2D or a 3D map. The results of the study showed no advantage in using 3D maps for tasks that involved estimating elevation. Additionally, in tasks where it was not necessary to determine an object’s elevation (e.g. measuring distances), the 3D variant was not as good.

User testing of 3D interactive virtual environments is relatively scarce. One of the few articles describing such an environment is presented by Wilkening and Fabrikant ( 20 ). Using the Google Earth application, they monitored the proportion of applied movement types – zoom, pan, tilt, and rotation. Bleisch, Burkhard ( 21 ) assessed the 3D visualization of abstract numeric data. Although speed and accuracy were measured, no information about navigation in 3D space was recorded in this study. Lokka and Çöltekin ( 22 ) investigated memory capacity in the context of navigating a path in a virtual 3D environment. They observed the differences between age groups.

Previous studies ( 20, 23, 24 ) indicate that there are considerable differences between individuals in how they read maps, especially in the strategies and procedures used to determine an answer to a question. To understand map reading strategy, the use of eye-tracking facilitates the study.

Although eye-tracking to study maps was first used in the late 1950s, it has seen increased use over the last ten to fifteen years. Probably the first eye-tracking study for evaluating cartographic products was the study of Enoch ( 25 ), who used as stimuli simple maps drawn on a background of aerial images. Steinke ( 26 ) presented one of the first published summaries about the application of eyetracking in cartography. He compiled the results of former research and highlighted the importance of distinguishing between the perceptions of user groups of different age or education.

Today, several departments in Europe and the USA conduct eye-tracking research in cartography ( 27 ). In Olomouc, Czech Republic, eye-tracking has been used to study the output of landscape visibility analyses ( 28 ) and to investigate cartographic principles ( 29 ). In Zurich, Switzerland, Fabrikant, Rebich-Hespanha ( 30 ) evaluated a series of maps expressing the evolution of phenomenon over time and weather maps ( 31 ). Çöltekin from the same university analyzed users’ visual analytics strategies ( 32 ). In Ghent, Belgium paper and digital topographic maps were compared ( 33 ) and differences in attentive behavior between novice and expert map users were analyzed ( 34 ). Ooms, Coltekin ( 35 ) proposed the methodology for combining eye-tracking with user logging to reference eyemovement data to geographic objects. This approach is similar to ours, but instead of 3D model a dynamic map is used.

The issue of 3D visualization on maps has so far only been addressed marginally. At the State University of Texas, Fuhrmann, Komogortsev ( 36 ) evaluated the differences in how a traditional topographic map and its 3D holographic equivalent were perceived. Participants were asked to suggest an optimal route. Analysis of the eyetracking metrics showed the better option to be the holographic map.

One of the first and more complex studies dealing with eye-tracking and the evaluation of 3D maps is the study by Putto, Kettunen ( 37 ). In this study, the impact of three types of terrain visualization was evaluated while being required to solve four tasks (visual search, area selection, and route planning). The shortest average length of fixation was observed for the shaded relief, indicating that this method is the easiest for users.

Eye-tracking for evaluating 3D visualization in cartography is widely used at Palacký University in Olomouc, Czech Republic, with studies examining the differences in how 3D relief maps are perceived ( 38 ), 3D maps of cities ( 39 ), a 3D model of an extinct village ( 40 ), and tourist maps with hill-shading ( 41 ) being produced there. These studies showed that it is not possible to generalize the results and state that 3D is more effective than 2D or vice versa. The effectivity of visualization depends on the exact type of stimuli and also on the task.

In all these studies static images were used as stimuli. Nevertheless, the main advantage of 3D models is being able to manipulate them (pan, zoom, rotate). An analysis of eye-tracking data measured on interactive stimuli is costly, as eye-trackers produce video material with overlaid gaze-cursors and any classification of fixations requires extensive manual effort ( 42 ). Eye tracking studies dealing with interactive 3D stimuli typically comprise a time-consuming frame-by-frame analysis of captured screen recordings with superimposed scanpaths. One of the few available gaze visualization techniques for 3D contexts is the representation of fixations and saccades as 3D scanpaths ( 43 ). A challenge with 3D stimuli is mapping fixations onto the correct geometrical model of the stimulus ( 44 ).

Several attempts to analyze eye-tracking data recorded during the work with interactive 3D stimuli exist. Probably the most extensive work has been done by Stellmach, who developed tool called SWEETER – a gaze analysis tool adapted to the Tobii eye-tracker system and XNA Framework. SWEETER offers a coherent framework for loading 3D scenes and corresponding gaze data logs, as well as deploying adapted gaze visualizations techniques ( 45 ).

Another method for visualizing the gaze data of dynamic stimuli was developed by Ramloll, Trepagnier ( 46 ). It is especially useful for 3D objects on retail sites allowing shoppers to examine products as interactive, non-stereoscopic 3D objects on 2D displays. In this approach, each gaze position and fixation point is mapped to a 3D object’s relevant polygon. A 3D object is then flattened and overlaid with the appropriate gaze visualizations. The advantage of this flattening is that the output can be reproduced on a 2D static medium (i.e. paper).

Both approaches used a remote eye-tracker to record data. Pfeiffer ( 42 ) used a head-mounted eye-tracking system by Arrington Research. This study extended recent approaches of combining eye-tracking with motion capture, including holistic estimations of the 3D point of regard. In addition, he presented a refined version of 3D attention volumes for representing and visualizing attention in 3D space.

Duchowski, Medlin ( 47 ) developed an algorithm for binocular eye-tracking in virtual reality, which is capable of calculating the three-dimensional virtual coordinates of the viewer’s gaze.

A head-mounted eye-tracker from the SMI was used in the study of Baldauf, Fröhlich ( 48 ), who developed the application KIBITZER – a wearable gaze-sensitive system to explore urban surroundings. The eye-tracker is connected via a smartphone and the user’s eye-gaze is analyzed to scan the visible surroundings for georeferenced digital information. The user is informed about points of interest in his or her current gaze direction.

SMI glasses were also involved in the work of Paletta, Santner ( 49 ), who used them in combination with Microsoft Kinect. A 3D model of the environment was acquired with Microsoft Kinect and gaze positions captured by the SMI glasses were mapped onto the 3D model.

Unfortunately, all the presented approaches work with specific types of device and are not generally available for the public. For this reason, we decided to develop our own application called 3DgazeR (3D Gaze Recorder). 3DgazeR can place recorded raw data and fixations into the 3D model’s coordinate system. The application works primarily with geographical 3D models (DEM – Digital Elevation Models in our pilot study). Majority of visualization of results of the case study is performed in open source Geographic Information System QGIS. The application works with data from an SMI RED 250 device and a low-cost, EyeTribe eye-tracker. This eye-tracker is connected with open source application OGAMA. Many different eye-trackers could be connected with OGAMA and then our tool will work with their data.

The data acquisition module is used to collect primary data. Its main component is a window containing the 3D model used as a stimulus. This 3D scene can be navigated or otherwise manipulated. The rendering of virtual content inside a graphics pipeline is the orthographic or perspective projection of 3D geometry onto a 2D plane. The parameters for this projection are usually defined by some form of virtual camera. Only main parameters, position, and orientation of the virtual camera are recorded in the proposed solution. The position and orientation of the virtual camera are recorded every 50 milliseconds (frequency of records 20 Hz). The recording is performed using functions from X3DOM runtime API and JavaScript in general. The recorded position and orientation of the virtual camera is sent every two seconds to a server and stored using a PHP script to a CSV (Comma Separated Value) file. Storage of the 3D scene loading time is necessary for subsequent combination with eye-tracking data. Similarly, termination of the 3D scene is also stored. The interface is designed as a full-screen 3D scene while input for answers is provided on the following screen (after the 3D scene).

The connecting module combines two partial CSV files based on timestamps. The first step is joining trimmed data (from the eye-tracker and from the movement of the virtual environment) by markers of the beginning and end of the depiction of a 3D scene. The beginning in both records is designated as time 0. Each record from the eye-tracker is then assigned to the nearest previous recorded position of the virtual camera (by timestamp), which is the simplest method of joining temporal data and it was not difficult to implement. The maximum time deviation (inaccuracy) is 16.67 ms.

Four variants of the connecting module were created – for SMI RED 250 and EyeTribe, and for raw data and fixations. The entire connecting module is implemented in JavaScript.

The calculating module comprises a similar window and 3D model to those used in the test module. The same screen resolution must be used as during the acquisition of data. For every record, the intersection of the viewing ray with the displayed 3D model is calculated. A 3D scene is depicted with a virtual camera’s input position and orientation. The X3DOM runtime API function getViewingRay and screen coordinates as input data are used for this calculation. Setting and calculating the virtual camera’s parameters is automated using the FOR cycle. The result is a table containing timestamps, 3D scene coordinates (X, Y, Z), the DEF element the ray intersects with, and optionally, a normal vector to this intersection. If the user is not looking at any particular 3D object, this fact is also recorded, including whether the user is looking beyond the dimensions of the monitor. This function is based on ray casting method (see Figure 3) and can be divided into three steps:

• calculation of the direction of the viewing ray from the virtual camera position, orientation and screen coordinates (function calcViewRay);

• ray casting to the scene;

• finding the intersection with the closest object (function hitPnt).

For more information about ray casting see Hughes, Van Dam ( 54 ).

For additional processing, analysis, and visualization of calculated data, GIS software is used. It was primarily open source program QGIS, but ArcGIS with 3D Analyst and ArcScene (3D viewing application) can also be used. We worked with QGIS version 2.12 with several additional plug-ins. Most important was Qgis2threejs plug-in. Qgis2threejs creates 3D models and exports terrain data, map canvas images, and overlaid vector data to a web browser supporting WebGL).

For the testing, we chose a low-cost EyeTribe device. Currently, the EyeTribe tracker is the most inexpensive commercial eye-tracker in the world at a price of $99 ( https://theeyetribe.com) Popelka, Stachon ( 55 ) compared the precision of the EyeTribe and the professional device SMI RED 250. The results of the comparison show that the EyeTribe tracker can be a valuable tool for cartographic research. The eye-tracker was connected to OGAMA software ( 56 ). The device operated at a frequency of 30Hz. EyeTribe also works with a frequency of 60Hz, however saving information about camera orientation caused problems with a frequencies higher than 20Hz. Some computer setups were not able to store camera data correctly when the frequency was higher than 20Hz. The length of the file was shorter than real recording, because some rows were omitted.

Two versions of the test were created – variant A and variant B. Each variant included eight tasks over almost the same 3D models (differing only in the used texture). The 3D models in variant A had no transparency, and the terrain was covered with a hypsometric color scale (from green to brown). The same hypsometric scale covered four 3D models in variant B, but transparency was set at 30%. The second half of the models in variant B had no transparency, but the terrain was covered with satellite images from Landsat 8. The order of the tasks was different in both variants. A comparison of variant A and variant B for the same task is shown in Figure 4. Four tasks were required:

• Which object has the highest elevation? (Variant A – tasks 1, 5; Variant B – tasks 1, 2)

• Find the highest peak. (Variant A – tasks 2, 6; Variant B – tasks 3, 4)

• Which elements are visible from the given position? (Variant A – tasks 3, 7; Variant B – tasks 5, 6)

• From which positions a given object is visible? (Variant A – tasks 4, 8; Variant B – tasks 7, 8)

The first two tasks had only one correct answer, while the other two had one or more correct answers.

Before the pilot test we decided that 20 participants would be tested on both variants, with an interval of at least three days between both testing sessions. Participants were recorded on both variants, but not influenced by a learning effect when performing the second variant of the test because of the interval.

Half of the participants were students of the Department of Geoinformatics with cartographic knowledge, half of them were cartographic novices. Half of the participants were men, half women. The age range was 18-32 years.

Screen resolution was 1600 x 900 and the sampling frequency was set to 30 Hz. Each participant was seated at an appropriate distance from the monitor with an eye-tracking device calibrated with 16 points. Calibration results of either Perfect or Good (on the scale used in OGAMA) were accepted. An external keyboard was connected to the laptop to start and end the tasks (F2 key for start and F3 for the end). A researcher controlled the keyboard. The participant performed the test using only a common PC mouse.

The experiment began with calibrating the device in the OGAMA environment. After that, participants filled in their ID and other personal information such as age, sex, etc. The experiment was captured as a screen recording.

Prepared with individual HTML pages, the experiment included questions, tasks, 3D models, and input screens for answers. The names of the CSV files where recording of the virtual camera movement would be stored coincided with the task in the subsequently created eye-tracking experiment in OGAMA. This would allow correct combination in the connecting module.

As recording began, a page with initial information about the experiment appeared. The experiment ran in Google Chrome in full-screen mode and is available (in Czech) at http://eyetracking.upol.cz/3d/. Each task was limited to 60 seconds duration, and the whole experiment lasts approximately 10 to 15 minutes. Longer total time of the experiment may affect the user performance. From the evidence from previous experiments, we found out that when the recording is longer than e.g. 20 minutes, the participants started to be tired and they lose concentration.

Care was taken with the correct starting time for tasks. A screen with a target symbol appeared after the 3D model had loaded. The participant switched to the task by pressing the F2 key. This keypress was recorded by OGAMA and used by 3DGazeR to divide recording according to the task. After that, a participant could manipulate the 3D model to try to discover the correct answer. The participant then pressed F3, and a screen with a selection of answers appeared.

It is necessary to store the data for each task separately, alternatively control or manually modify (e.g. delete the unnecessary lines at the end of recording). The data is then processed in the connecting module where data from the eyetracking device is combined with virtual camera movement. The output is then sent to the calculation module which must be switched to full-screen mode. The calculation must take place on the same screen resolution as the testing. The output should be modified for import into GIS software and visualized. For example, the data format of the time column had to be modified into a form required to subsequently create animation.

These adjusted data can be imported into QGIS. CSV data are loaded and displayed here using the Create a Layer from a Delimited Text File dialog. The retrieved data can be stored in GML (Geography Markup Language) or Shapefile as point layers. After the export and re-render of this new layer above the 3D model, it is possible that some data may have the wrong elevation (see Figure 5). This distortion occurs when the 3D model is rotated while eyes are simultaneously focused on a specific place, or when the model is rotated, and eyes track with a smooth pursuit. To remove these distortions and correctly fit eye-tracking data exactly on the model, the Point Sampling Tool plug-in in Qgis2threejs was used.

For the evaluation of the validity of 3DGazeR output, we have created the short animation of 3D model with one red sphere in the middle. The diameter of the sphere was approximately 1/12 of the 3D model width. In the beginning, the sphere was located in the middle of the screen. After five seconds, the camera changed its position (it took two seconds), and the sphere moved to the upper left side of the screen. The camera stayed there for six seconds and then moved again, so the sphere was displayed in the next corner of the screen. This process was repeated for all four corners of the screen. The validation study is available at http://eyetracking.upol.cz/3d/.

The task of the participant was to look at the sphere all time. The validation was performed on five participants. Recorded data were processed in connection and calculating modules of 3DGazeR. For the evaluation of the data validity, we decided to analyze how many data samples were assigned to the sphere. Average values for all five participants are displayed in Figure 6. Each bar in the graph represents one camera position (or movement). The blue color corresponds to the data samples where the gaze coordinates were assigned to the sphere; the red color is used when the gaze was recorded out of the sphere. It is evident that inaccuracies were observed for the first position of the sphere because it took some time to participants to find the sphere. A similar problem was found when the first movement appeared. Later, the percent of samples recorded out of the sphere is minimal. In total, average amount of samples recorded out of the sphere is 3.79 %. These results showed that the tool works correctly and the inaccuracies are caused by the inability of the respondents to keep eyes focused on the sphere that was verified by watching the video recording in OGAMA.

First, we tried to visualize raw data as simple points placed on a 3D surface. This method is very simple, but its main disadvantage is the poor arrangement of depicting data in this way, mainly in areas with a high density of points. The size, color, and transparency of symbols can be set in used GIS software. With this type of visualization, data from different groups of participants can be compared, as shown in Figure 7. Raw data displayed as points were used as input for creating other types of visualizations. Figure 7 shows the 3D visualization of raw data created in QGIS. Visualization of a large number of points in the 3D scene in a web browser through Three.js is hardware demanding. Thus, visualization of raw data is more effective in ArcScene.

The usual approach for depicting eye-tracking data is scanpath visualization superimposed on a stimulus representation. Scanpaths show the eye-movement trajectory by drawing connected lines (saccades) between subsequent fixation positions. A traditional spherical representation of fixations was chosen, but ( 45 ) also demonstrate different types of representation. Cones can be used to represent fixations or viewpoints and view directions for camera paths.

The size of each sphere was determined from the length of the fixation. Fixations were detected in OGAMA environment with the use of I-DT algorithm. The settings for thresholds were set to maximum distance of 30px and minimum number of three samples per fixation. Fixation length was used as the attribute for the size of each sphere. Transparency (30 %) was set because of overlaps. In the next step we created 3D saccades linking fixations. The PointConnector plug-in in QGIS was used for this purpose.

This visualization method is quite clear. It provides an overview of the duration of individual fixations, their position, and relation to each other. It tells where the participant’s gaze lingered and where it stayed only briefly. Lines indicate if a participant skipped between remote locations and back or if the observation of the stimulus was smooth. The scanpath from one participant solving variant A, task 4 is shown in Figure 8. From the length of fixations, it is evident that the participant observed locations near spherical bodies defining the target points crucial for task solving. His gaze shifted progressively from target to target, whereby the red target attracted the most attention.

Visual gaze analysis in three-dimensional virtual environments still lacks the methods and techniques for aggregating attentional representations. Stellmach, Nacke ( 45 ) introduced three types of attention maps suitable for 3D stimuli – projected, object-based, and surface-based attention maps. In Digital Elevation Models, the use of projected attention maps is the most appropriate. Object-based attention maps, which are relatively similar to the concept of Areas of Interest, can also be used for eye-tracking analysis of interactive 3D models with 3DgazeR. In this case, stimuli must contain predetermined components (objects) with unique identifiers (attribute DEF in X3DOM library).

Projected attention maps can be created in the ArcScene environment using the Heatmap plug-in in QGIS function. Heatmap calculates the density of features (in our case fixations) in a neighborhood around those features. Conceptually, a smoothly curved surface is fitted over each point. The important factors for creating Heatmap are grid cell size and search radius. We used a cell size of 25 m (it is about one thousandth of the terrain model size) and search radius as an implicit value (see Figure 9).

The advantage of projected attention maps is their clarity for visualization of a large amount of data. In a Geographic Information System, the exact color scheme of the attention map can be defined (with minimum and maximum values).

An interesting result was obtained from task 6, variant B. Figure 9 compares the resultant attention maps from participants with cartographic knowledge with those from the general public. For cartographers, the most important part of the terrain was around the blue cube. Participants without cartographic knowledge focused on other objects in the terrain. An interpretation of this behavior could be that the cartographers were consistent with the task and looked at the blue cube from different areas. By contrast, novices used the opposite approach and investigated which objects were visible from the blue cube’s position.

A suitable tool for evaluating user strategies is animation. Creating an animation with a 3D model is not possible in QGIS software, so we used ArcScene (with the function Create Time Animation) for this purpose. The model can also be rotated during the animation, providing interactivity from data acquisition through to final analysis. Animations can be used to study fixations of individuals or to compare several users. Animations can be exported from ArcScene software as video files (e.g. AVI), but it loses its interactivity. AVI files exported from ArcScene are available at http://eyetracking.upol.cz/3d/. A similar method to animation is taking screenshots, which can also be alternatively used in the qualitative (manual) analysis of critical task solving moments, such as at the end or when entering an answer.

When analyzing 3D eye-tracking data, it would be appropriate to concentrate on analyzing the Z coordinate (height). From the data recorded with 3DgazeR, the Z coordinate’s changes over time can be displayed, so the elevations the participants looked at in the model during the test can be investigated. Data from the program ArcScene were exported into a DBF table and then analyzed in OpenOffice Calc. A scatter plot with data points connected by lines should be used here. A graph for one participant (see Figure 10) or multiple participants can be created. A graph of Z coordinate raw data values was created in this case.

It is apparent from this graph when participants looked at higher ground or lowlands. In Figure 10, we can see how the participants initially fluctuated between elevations in observing locations and focused on the highest point around the 27^th second during the task. In general, we con-clude that this participant studied the entire terrain quite carefully and looked at a variety of low to very high eleva-tions.