The first, SoleWay, offers indoor wayfinding support through textual instructions (see Figure 1, on the left). The app and related website are based on a crowd-based outsourcing platform. As such, a community of (potential) wayfinders is created. On the one hand, people who are familiar with a certain route can add written route descriptions to the SoleWay database. Added route descriptions are purely textual. Each description is geographically located by pinpointing the approximate location of the starting point (e.g. main entrance) or the building in which the described route is situated on a map (i.e. Google Maps). On the other hand, wayfinders can find these descriptions by searching for the destination through a search box. The SoleWay platform will then provide the user with all descriptions of routes that lead to that destination and are in the vicinity of the user or the building of interest. Consequently, there is no need to spatially model the indoor environment or to use indoor positioning techniques which would require the installation of sensors (e.g. RFID tags, Bluetooth beacons, WLAN), which in turn may require costly infrastructure or augmentation of the building. A member of the community only needs a device with network capabilities (e.g. a smartphone). As a result, the cost and complexity of the application is minimised. SoleWay was developed by co-author Prof. Nico Van de Weghe (Department of Geography, UGent - https://soleway.ugent.be).

These textual instructions offer the advantage that (more) abstract concepts can concisely be conveyed to the user. Navigators have a good understanding of locations like the main entrance, a reception (desk) and a meeting room, regardless of their appearance or the extent to which they can change over time in terms of design. Consequently, these locations can easily be incorporated in an instruction without lengthy or detailed descriptions [9]. Analogously, it can be assumed that these textual route instructions (only) contain the most relevant information for the wayfinding task at hand and are not cluttered with unnecessary elements. In contrast, photographs may contain a high level of information and details as these depict everything present at a specific location.

Unfortunately, this is a double-edged sword as it can be difficult to determine what the most relevant, essential or suitable (type of) wayfinding information is. There are no unequivocal criteria for selecting salient physical features that can be used to describe the location where an action should take place as part of previously mentioned view-action pairs. Several studies have shown that landmarks are the most commonly used cues to enable wayfinding decisions and that route instructions containing landmarks as descriptive features are rated as highly effective [10, 11]. The main reason for this is that they allow fast reasoning and efficient communication for directing a person from A to B. Firstly, because they act as points of correspondence between different forms of spatial knowledge (e.g. reality, wayfinding tools [such as maps] and the cognitive model of the environment) [12]. Secondly, landmarks define a place with reduced representational complexity. While a place may exhibit a high level of information and details, a landmark is an anchor point that is abstracted to a node without internal structure [7]. As Streeter, Vitello and Wonsiewicz (1985) put forward, however, the landmark selection process is highly individual [13]. It depends on the perception and individual preferences of the observer, which are influenced by gender, age, social and cultural background, experience, familiarity with the environment and intentions [14]. For example, women prefer three-dimensional objects over two-dimensional elements [15]. In addition to the selection of the correct wayfinding information, the assessment of the adequate amount of information may be problematic as well. An instruction that is too brief may lead to uncertainty, while too much information can result in confusion and both will lead to higher cognitive load levels [16].

The other smartphone application, Eyedog, provides wayfinding support by means of ‘street-view’ like photographic imagery (see Figure 1, on the right). The route is presented as a sequence of photographs wherein the user can (manually) swipe back and forth. The photographs are augmented with textual or schematic (e.g. arrows) directions to clarify the intended wayfinding instruction. Although Eyedog can operate in combination with indoor positioning systems, similar to SoleWay it can function without the use of external hardware. In contrast to SoleWay, the indoor environment is spatially modelled with the help of a network of nodes and edges attributed with weights and photographs. Based on this network, shortest paths are generated automatically. Eyedog was developed by co-author Ralph Michels (PhD researcher and CTO of Eyedog Indoor Navigation - http://www.eyedog.mobi).

Photographs, as used by Eyedog, represent the indoor environment at a certain location. This way, the landmark and wayfinding information selection process is to a large extent in the hands of the user. Photographs support this process as the visual sense contributes greatly to the recognition of landmarks and the estimation of distance and orientation during navigation [8]. Indoors this is of great value. Indoor routes are often characterised by frequent shifts in direction and, therefore, require a higher density of landmarks to be clearly described. Moreover, the number of object categories from which landmarks can be selected is usually limited indoors [17]. Accordingly, several studies have shown that, in comparison to paper and mobile maps, participants prefer images to visualise the environment while executing wayfinding tasks indoors as the use of photographs leads to improved wayfinding performance in terms of task duration and success rate [ 17, 18].

The comparison between different modalities to convey route descriptions to a navigator is generally based on the time needed to complete a wayfinding task and whether or not a person is able to reach the destination with the help of a specific modality. In most of these indoor wayfinding studies, however, it is very rare that a person does not reach the destination point. Furthermore, the observed task duration differences may be statistically significant, but in practice these are very small. Other performance measures that may be more adequate to reflect the usability of a description are location and orientation accuracy [3], numbers of errors, feeling lost episodes and/or dwell points [19], smartphone interaction recordings to identify wayfinding strategies [20] and user ratings with respect to quality and usefulness [ 16].

Another research tool, which is relatively new in the domain of indoor wayfinding, is eye tracking. The analysis of gaze characteristics can provide useful insights regarding a navigator’s use of environmental and wayfinding information, and the interplay of both [21]. Consequently, in recent years eye tracking has frequently been used in a wide range of settings within the field of pedestrian navigation (e.g. spatial decision making, map interaction, wayfinding aids) [22]. However, the number of studies, specifically within the context of communication modalities of indoor pedestrian wayfinding systems, is limited.

Ohm, Müller and Ludwig (2017) used eye tracking measures as fixation time, number of fixations and revisits to the mobile phone screen to demonstrate that participants preferred a reduced interface displaying landmarks and simplified route segments instead of an interface using floor plans [23]. In contrast to the original experimental design, the smart phone screen was seen as a single area of interest. The small screen and the limited accuracy of the eye tracking device did not allow fixations to be attributed to different interface elements. Following, Schnitzler et al. (2016) investigated what the effect of a wayfinding aid (i.e. no map, paper map, digital map) was on fixation frequencies at decision points [21]. The number of fixations was determined for three areas of interest: signage, correct route option and incorrect option. Next, Li (2017) [18] used the number of fixations and their duration to create heat maps and gaze plots on photographs and maps to investigate the role of maps in combination with other aids during indoor wayfinding. In these three studies, eye tracking data was collected by a group of test persons that individually completed an indoor route to a destination with the help of a wayfinding aid. During this task, participants were equipped with a mobile eye tracker.

Most studies on communication modalities of (indoor) pedestrian wayfinding applications focus on the usability of such a modality compared to or in combination with (mobile) maps. Moreover, no studies take into account the aspect of mental effort. Mental effort refers to the proportion of working memory capacity that is allocated to the (instructional) demands of the task and can be used as an index to assess the cognitive load that the execution of a task imposes on a person [24, 25]. Photographs and textual descriptions are both external representations of (wayfinding) information that are employed to support memory and thinking [9]. A person must translate such a representation (e.g. of a location), which is briefly stored in the short-term memory, to reality (e.g. the corresponding actual location in his/her surroundings). This translation requires the interaction of the short-term memory with previously acquired knowledge and skills (e.g. orientation skills) stored in the long-term memory. In turn, this interaction (i.e. working memory) and, as such, this translation demand mental effort [26]. Central in the cognitive load theory is that the working memory capacity is limited [27]. Visitors of large-scale spaces, especially first-time visitors, may already experience high stress levels and a significant working memory load caused by factors other than the wayfinding task. The wayfinding aid and the method used to present wayfinding information (e.g. words or images) can reduce the complexity of the decision-making process and therefore the cognitive load [28].

In general, there are four ways to assess mental effort: (1) indirect (performance) measures, (2) subjective measures, (3) secondary task measures, and (4) physiological measures [27]. In this study, subjective measures (i.e. rating scales) and physiological measures (i.e. eye tracking) will be used to assess which communication modality (i.e. written or photographic-based route instructions) requires less mental effort to understand and to act in accordance with. A large number of studies have used a nine-grade rating scale as a subjective measure to examine the experienced mental effort [24, 29, 30]. For an extensive overview of these studies, we refer to [31] and [24]. This frequent use has proven that the numerical values of a (nine-grade) rating scale enable test persons to veraciously express the required mental effort. Furthermore, multiple measurements during an experiment are possible. This way, a more detailed analysis of mental effort and task complexity variations can be conducted [27, 31].

Following, eye tracking can also be used as a measure of the processing demands of a task [32]. Especially in problem solving tasks, longer fixations and shorter saccadic amplitudes are linked to more effortful cognitive processing and indicate that a person has (more) difficulties in extracting information or relating this information to internalised representations. In scene perception, features that are considered more important, interesting or semantically informative generate longer fixations and more revisits compared to those elements that are perceived less important. Additionally, several studies assume that the number of fixations and saccadic rate overall is negatively correlated with the search efficiency and could be an indication of the difficulties a person experiences while collecting relevant information [ 33, 34].

Ooms (2016) emphasizes the importance of using mixed methods in usability research [35]. Using multiple eye tracking measures and mental effort ratings makes it possible to verify the results across datasets. This improves the reliability and validity of the study. However, some authors have argued that fixation measures and mental effort ratings measure different aspects of cognitive load [27, 32]. Fixations represent parts of the task or an individual problem, while an effort rating represents the mental effort of the overall task or process (e.g. the total number of problems). As such, these may result in non-equivalent assessments of the invested mental effort. By asking mental effort ratings at multiple intermediate points, the authors hope to minimise such distortion.

In total 14 male and 15 female subjects participated in the experiment. The questionnaire, wherein participants were asked to rate a series of statements, revealed the following. Participants were, on average, relatively familiar with (parts of) the test environment (see materials section). As such, they are acquainted with the building’s structure and design. In contrast, the destinations along the route were not known to them. Furthermore, test persons had used smartphone applications as wayfinding aid before outdoors, but rarely in an indoor setting. Their ages ranged between twenty and sixty years old ( M = 34, SD = 9). During the test, they were not distracted by the researcher following them or by the mobile eye tracking device. Five participants were excluded from the eye tracking results, because the tracking ratio was too low. The required tracking ratio was set to 95%.

During the completion of both routes, participants wore a SMI ETG 2.1 mobile eye tracking device (60 Hz / 30 FPS). Fixations were calculated with the help of the SMI Event Detection (dispersion-based) algorithm and were transferred manually to four reference images (i.e. one for each route and application). Each reference image displayed two categories (i.e. (screen of) smartphone, signage (along the route)), which were attributed with areas of interest by using the semantic gaze mapping tool of BeGaze 3.6.

Both wayfinding apps (i.e. Eyedog Indoor Navigation 1.0.0 and SoleWay v15) were installed and presented on the same smartphone, namely an LG Nexus 4. As mentioned in section 2, Eyedog automatically provided the shortest routes between the selected destinations based on a network model of the building. The SoleWay routes were formulated and entered manually. A SoleWay route is a text depicted on a single screen. Each line in the text consisted of one view-action pair specifying the location and the required wayfinding action as simple as possible (see Figure 1). For each route, the number of lines was comparable to the number of photographs.

The star shaped building (see Figure 2) was considered to be fairly complex by most participants. It was built in 1976 and has a traditional interior (see Figure 1 and 4). Within this building two routes were selected. Both routes had had a total length of approximately 360 meters and consisted of three connected route segments leading to a destination. All participants completed the same routes (see Figure 2). The first route went from the starting point on the second floor level to the men’s toilet (destination A) on the ground floor. From there, participants were asked to go to the secretariat of the Marine Biology Research Group (B). Finally, they were asked to find the Geophysics Processing Lab (C) on the first floor level. The second route started at the same starting point. The first intermediate destination was the small garage (D) in the curved outer corridor on the ground floor level. The following destination was the office of prof. V. Cnudde on the first floor level (E). The final destination of the second route was lecture room 3.065 on the third floor (F). Route 1 includes 22 decision points and route 2 counts 24 decision points.

Finally, the following statements (subdivided in four categories) are rated on a seven-grade scale in the concluding questionnaire:

Evaluation of the route

(1) “The route was complex.”

Evaluation of the wayfinding instructions

(2) “I often had doubts about the further course of the route.” (3) “The wayfinding instructions were clear.” (4) “The wayfinding instructions were easy to follow.” (5) “The wayfinding instructions were detailed enough.” (6) “The wayfinding instructions were adequate to convey the route.”

Memory of the route

(7) “After the experiment, I am able to complete the same route without help of a wayfinding aid.” (8) “After the experiment, I am able to verbalise route instructions to a person who is not familiar with the route.” (9) “After the experiment, I am able to draw the route on a floor plan.”

Application recommendation

(10) “I would recommend the application for other buildings.”

At the beginning of the experiment, the participants were instructed as follows. “After the calibration of the eye tracking device, you will be asked to complete two routes while wearing the eye tracking device. This device will register your eye movements. I will follow while completing these routes. One route will be explained with the help of SoleWay. During the other route, you will be guided by Eyedog. Each route starts in this office and at the end of each route we will check if the eye tracking device is still correctly calibrated. Each route consists of a number of destinations or intermediate stops. At each stop, I will ask you to rate the mental effort that was needed to reach this destination with the help of the wayfinding application. You can rate this effort on a nine-grade scale: zero being very, very low and nine being very, very high. Then I will give you the next destination. After the completion of both routes, you will be asked to fill in a small questionnaire. During the experiment, you may always ask for help or clarification. I will intervene if you would get lost.”

The experiment proceeded as described. Five point targets placed at approximately 1.5 meters were used to calibrate the eye tracking device. For all five points, the gaze error is corrected, making the calibration more accurate for every additional point. Participants always completed the routes in the same order. However, the wayfinding app used to guide a participant along a route was randomised in order to assess the potential influence of familiarity with the experimental setup. As such, half of the participants completed the first route with SoleWay and the second with Eyedog, while the other half first used Eyedog and then SoleWay. The guidelines as expressed by Holmqvist et al. (2011) were taken into account during calibration, instruction giving and route completion [33]. After completing the two routes, participants filled in the final questionnaire.

The significance of potential differences in eye tracking measures between both groups (i.e. SoleWay users and Eyedog users) was determined by a parametric test (see Table 2). As there is disagreement about whether (ordinal) rating scale data should be analysed with parametric statistics or nonparametric statistics [37], the normality of the mental effort and questionnaire data was tested with the Shapiro-Wilk Test. Because not all data samples are normally distributed, both the parametric t-test and the non-parametric Mann-Whitney test are conducted to determine the significance of potential differences between both SoleWay users and Eyedog users regarding mental effort. Both tests showed the same result and are listed in Table 1. The significance of potential differences between Soleway and Eyedog in the general questionnaire is also analysed with both a parametric and a non-parametric test, for the same reason. In this case, a Wilcoxon signed-rank test and a dependent t-test is executed, because all participants used both Soleway and Eyedog and are therefore in both groups. The results of both tests are indicated in Table 3.