JemrJournal of Eye Movement Research1995-8692Bern Open PublishingBern, Switzerland10.16910/jemr.11.1.5Research ArticleWhen Words and Graphs Move the Eyes: The Processing of Multimodal Causal RelationsParodiGiovanniJulioCristóbalRecioInésPontificia Universidad Católica de Valparaíso, ChileHeidelberg Universität, Germany9032018201811110.16910/jemr.11.1.52018Parodi, G., Julio, C., & Recio, I.This work is licensed under a Creative Commons Attribution 4.0 International License,
(
https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use and redistribution provided that the original author and source are credited.
Research on causal relations in multisemiotic texts constituted by words and graphs has been scarce with only a few exceptions. In the current study, eye movement behavior was studied in seventy-six Chilean high school students, who read a set of twelve causally-related economics texts in Spanish in four experimental conditions. The objective is twofold. We aimed, on the one hand, to observe the main effects of the causal discourse marker (DM) por tanto and the statistical causal graph (G), as well as the interaction effect of both variables on different eye tracking measures. On the other, we seek to observe the effects of the DM on the same eye tracking measures for the graph system (GS) area of interest (AOI). The findings showed that the conjoint presence of the DM and the G did not positively influence the processing of selected AOIs. Analyses also reveal no significant effects on the GS AOI. Thus, the results indicate that the DM tend to decrease processing times, while the G increases them. Additional analyses conducted on the integrative transitions between the verbal system and the graph system reveal that more transitions were identified between the consequence segment and the graph system, thereby confirming that the consequence segment is crucial for the integration of both semiotic systems.
Multisemiotic written texts, composed of multiple
semiotic systems (e.g., words, graphs, images, diagrams),
are widely employed in academic and professional contexts and
especially in cross-disciplinary genres (
59, 60
).
Graph/word texts, for example, have been identified as fundamental
written discourse tools in the construction
and transmission of specialized knowledge (
36, 83, 11, 62, 63
).
Similarly, economics reports are written using a wide range of
diverse semiotic systems, such as verbal and
diagrammatic (
61
).
Both systems may express causation, which has been
defined as the most significant relation in the world (
41
), not
only through descriptive textual analyses (
20, 43, 15, 3, 40
), but also
through psycholinguistic cognitive processing
research (
77, 78, 96, 23, 54, 1, 58, 4
).
Nevertheless, research on causal relations in
multisemiotic texts constituted by words and graphs is scarce
with only a few exceptions(
24, 25
). Even more limited is the
use of eye tracking measures in the study of reading
processing with texts written in Spanish and a varying
degree of specialization (
62, 33
). As is well known, eye tracking
technology allows for the determination of the exact
location of the point of gaze of a subject's eye, which in
turn allows researchers to register the intermodal
connections between words and graphs in the construction of a
mental representation while reading on line in a
momentto-moment processing of written discourse. The current
state of knowledge, therefore, leaves important questions
unanswered.
In light of these shortcomings, the objective of this
study is twofold. Firstly, we aim to observe the main
effects of the causal discourse marker (DM) por tanto
(therefore) (i.e., presence/absence), and the causal
statistical graph (G) (i.e., presence/absence), as well as the
interaction effect of both variables on different eye
tracking measures (e.g., Total Fixation Time, First and Second
Pass Reading Time), when students read a set of
causallyrelated texts in Spanish. We expect to observe that the
presence of both the DM and the G diminishes reading
times, separately. In addition, we expect that the
interaction (combined effect) of both variables further decreases
reading times. These hypotheses were tested for the
verbal system (VS), the causal discourse segment (S1), and
the consequence discourse segment (S2).
The second goal of this study is to observe the effects
of the DM (i.e., presence/absence) on the same eye
tracking measures for the graph system (GS) area of interest
(AOI). We expect to observe that the presence of the DM
decreases the reading times on the GS AOI.
The article is structured as follows. First, in the
theoretical background, we review some relevant distinctions
of causal semantic discourse relations when considering
multisemiotic texts, discourse markers and graphs. The
second section describes the methodology, focusing on
the the participants,experimental design, the material
andprocedures. Section three presents the results. In the
final section, we summarize the results and discuss the
findings, highlighting some of the problems encountered,
as well as possible solutions and avenues for further
research.
Causal Discourse RelationsThe Verbal System
As a phenomenon of dual component – a cause and a
consequence – discursive causality must be necessarily
treated as an instantiation of connection (or
‘connectedness’ (
74
)). This is the case whether only one discourse
segment has been expressed and (causally) linked to an
assumption available in the common ground of the
interlocutors and, therefore, only contextually accessible or
whether the causally related segments have both been
uttered. The latter type constitutes the focus of this study
(1).
(1) Los incendios forestales en el sur
aumentaron. CAUSEPor tanto, la producción
maderera descendió.REASONED CONSEQUENCE
[Forest fires in the South increased.
Therefore, wood production decreased.]
Two discourse segments holding a causal relation can
be linked by means of an explicit semantic mark –
prototypically a connective (por tanto, in Spanish; ‘therefore’
in English) or a causal phrase (a causa de, in Spanish;
‘due to’ in English) (
92
).
In (1), the Spanish argumentative connective por tanto
provides readers with an explicit instruction on how to
process the text. As a connective, thesemantics of
por tanto is mainly of procedural nature, as opposed to
conceptual meaning. Procedural meanings act as constraints on
inferential processes and can be defined as “encoded
instructions that specify computational operations to be
performed during interpretation and, more precisely to
access a particular context for interpretation.” (
17
).
Specifically, por tanto constrains the access to the context for
readers and instructs them to process the host segment as
a reasoned consequence of the mental representation
arisen from the propositional content of the previous
discourse segment, that is, the cause of the discourse
relation (
3
). As a subgroup of discourse markers (DMs),
connectives guide inferential processes to interpret
utterances linking “semantically and pragmatically a discourse
member with a previous one” (
43
). As such, readers would
not construct such a complex mental representation, if
they process only a single segment.
However, despite the inference-constraining role of
DMs in general, two discourse segments can also be
processed as causally-related when these segments are
merely juxtaposed (2), resulting in what is commonly
referred to as ‘implicit causality’:
(2) Los incendios forestales en el sur
aumentaron.CAUSELaproducción maderera
descendió.CONSEQUENCE
[Forest fires in the South increased. Wood
production decreased.]
Such relations not explicitly signaled by a DM (i.e.
implicit relations) have been a concern of traditional
grammar for a long time (
21, 39, 68
). Particularly, implicit causal
relations are an interesting phenomenon for discourse
research due to their special cognitive status and the fact
that causality seems to be processed by default:
Because readers aim at building the most informative
representation, they start out assuming
the relation between two consecutive sentences
is a causal relation (given certain characteristics
of two discourse segments). (
79
)
While processing two juxtaposed discourse segments,
readers are guided by two kinds of expectations: one of
maximal informativity, or optimal relevance (
86, 87
), and one
of causality. Both are intimately entrenched: causality is
deemed to be more relevant than other potentially
implicit discourse relations, like temporal or additive relations,
because it entails them and, therefore, it also leads to
greater contextual effects.
Contrary to explicit causality, where causal
processing is supported by the conventional implicature
semantically fixed by the instantiation of the connective
(por tanto, in this study), implicit causality is processed by
means of a pragmatic enrichment (
43
). Driven by the
causality-by-default principle, readers try to recover the
assumption communicated by the writer resorting to
previous assumptions available in their minds in which they
integrate the mental representation derived from the
propositional content of the juxtaposed segments. In the
absence of prior sufficient assumptions to do so, i.e. when
for readers the propositional content of the segments does
not sufficiently allow for a causal interpretation,
establishing a causal link between the two adjacent discourse
segments becomes more difficult. This does not mean
that causality may not be recovered; it rather implicates
that failure to easily activate “a rule from world
knowledge or context” (
12
) that leads to a causal
interpretation may trigger an attempt to at least momentarily seek
for a non-default interpretation – very commonly, for
instance, temporal or additive relations (
27
). In other words,
in absence of a procedural guide that makes the discourse
relation explicit, readers have less constraints to access
the proper context to build a situation model and,
therefore, the array of possible interpretations is wider, which
may result in higher cognitive efforts. By contrast, when
causality is marked, the main procedural meaning of the
connective, as opposed to conceptual words (
18, 52
), compels
readers to build a causal relation. Hence, in the case of
our study, por tanto confers on the two discourse
segments a specific semantic role (cause-consequence) and
triggers a process in readers’ minds to integrate the
segments and build a causal representation.
Causal discourse relations have been commonly
categorized in terms of their degree of subjectivity (
89, 77, 78, 56, 76
);
that is, to the presence or absence of the so-called Subject
of Consciousness who assumes the responsibility for the
discourse relation (
57
). Within a continuum of
subjectivity (
74, 37
), according to the content of the texts employed in
this study, the critical stimuli reflect relations close to the
objectivity pole: one event causes another in the real
world; they belong to the non-volitional content domain
(as opposed to epistemic or speech-act domain), and they
describe physical facts (as opposed to mental facts,
judgements, or speech acts) (see 1).
The Graph System
Statistical graphs are complex multisemiotic
systems (
59, 62, 63
). They represent visual data through the
combined use of points, lines, numbers, symbols, shading,
and color, together with a coordinate system (
2, 95
). There
exists a great diversity of graphs, such as bar charts,
Cartesian graphs, curve-difference charts, juxtaposed
Cartesian graphs, and pie charts (
2, 8, 95
). Graphs are widely used
across disciplines, and they play an important role in the
communication of scientific and technological
knowledge, as well as in business, education and mass
media in general (
8, 13
).
For Zacks and Tversky (
100
), statistical graphs are
cognitive artefacts used in scientific discourse both to reason
and to communicate data, as they help interpret the
results obtained in scientific research and allow
communication and dissemination of results and conclusions.
Statistical graphs are of paramount importance in the
development and communication of scientific research, which
has led to conceptualization of them as powerful
objects (
36, 7, 95, 11
). Furthermore, the visual representation of data
facilitates the interpretation and comprehension of
numerical or statistical values associated with variables
under study. The subsequent wide dissemination and
frequent use of graphs have motivated researchers to
identify and describe the different cognitive processes
involved in the reading of graphs (
1, 26, 83, 88
), and to develop
theoretical-empirical models (
8, 65, 66, 99, 85, 46, 47
).
Instead of simply showing associated values,
statistical graphs link two or more variables (
95
). This is possible
because graphs display quantitative information that
reveals patterns through the visual distribution of the
data (
80
). This variable-matching function of graphs allows:
... encouraging and even imploring the viewer
to assess the possible causal relationship
between the plotted variables. It confronts causal
theories that x causes y and with empirical
evidence as to the current relationship between x
and y. (
95
)
However, the simple juxtaposition of variables does
not necessarily imply causality between them (
22
). In the
analysis of statistical data, certain conditions must be
fulfilled in order to establish causality between
variables (
64
). Nevertheless, graphs can present causal relations
between variables whenever the analysis of the statistical
information verifies so or when such causal relation is
confirmed. Graphs, in this way, fulfill the purpose of
explaining causal relations, even serving as evidence of
the relation between two or more variables.
Two types of relations between variables can be
established: direct relations, also called positive relations,
and inverse relations, known as negative relations. In
positive relations, the increase or decrease in one variable
causes an identical action in the other, while in negative
relations, the increase or decrease of one variable leads to
the opposite action in the other (
64
). The direction of the
relation is important to establish the influence of one
variable on the other; at the same time, it determines the
way in which these variables must be represented in the
graph.
In this research, the statistical graphs presented to the
participants are composed of two variables, a cause and a
consequence that are related in a negative way, where the
increase of one variable causes the decrease of the other.
MethodsParticipants
Seventy-six Chilean students attending a public high
school (39 female, 37 male, mean age = 16.6 years,
S.D. = 2.2) took part in the study. All were native Spanish
speakers. At the time of the experiment, all participants
were in 11th grade. Their parents gave their written
informed consent to the experimental procedure, as
required by the National Commission for Scientific and
Technological Research (CONICYT) in Chile. The
participants did not present vision disorders that could
interfere with the eye tracking methodology.
The a priori sample size estimation considered the
following parameters: a) significance level α = .05, b) (1-β)
= 0.9, and c) effect size = .2 (small (
9
)). As a result, the
minimum required sample size was seventy-two
participants. All analyses were conducted using GPower 3.0 (
19
).
Design, Areas of Interest, and Dependent
VariablesDesign
In order to carry out the first objective of the study, a
two-factor within-subject design was implemented, which
encompassed four experimental conditions (A, B, C, and
D), as follows (see Figure 1):
(A) presence of the DM and presence of the G (+DM+G);
(B) presence of the DM and absence of the G (+DM-G);
(C) absence of the DM and presence of the G (-DM+G); and
(D) absence of both the DM and the G (-DM-G).
Illustration of the two factors DM and G and their possible combinations.
In order to accomplish our second objective, a one
factor within-subject design was implemented. The
within-subject factor was the DM (i.e., presence or absence).
The dependent variables were the same used to test the
first hypothesis, which are described in detail below.
As is usual in within-subject designs, all participants
received all four experimental conditions (
16, 55, 84
). To
minimize the carry-over and learning effects (
84
), the order of
the four experimental conditions was counterbalanced
(e.g., ABCD, BCDA, etc.) and randomly assigned to each
participant.
Areas of Interest (AOIs)
The AOIs were segmented manually with the Tobii
Studio software (Tobii Technology AB) and
corresponded to the verbal system (VS: S1+S2), the S1 being the
cause segment, and the S2, the consequence segment.
Figure 2 shows an example of a verbal/graph text and the
four AOIs.
The four AOIs (VS, S1, S2 and GS).
Dependent Variables
Three dependent variables were included in the
experiment: a) Fixation Time, b) First Pass Reading Time, c)
Second Pass Reading Time (
30, 70, 28
). These measures were
analyzed for the verbal system (VS), for the cause and the
consequence discourse segments (S1 and S2,
respectively), and for the graph system (GS). These dependent
variables correspond to what Holmqvist et al. (
28
) have
classified as numerosity measures (event-counting
measures).
The Fixation Time (also called Total Reading Time)
amounts to the total time spent on an AOI, including
rereading and reinspections of the target region (
30, 72
). First
Pass Reading Time is obtained by summing up the
duration of the fixations within the target AOI before exiting
it, whether moving forward or looking back in the
text (
71, 69, 30
). Second Pass Reading Time (
30, 70, 32
) is the
summed duration of all the fixations that occur after the
first pass reading, when the eyes reenter the target AOI.
First and second reading measures also include the
duration of all reinspections within the target region before
exiting it.
The values of the eye tracking measures reported here
correspond to the aggregated time needed for each
participant to read all three versions of the texts integrating
each experimental condition (see Materials section).
Equipment
Reading data was recorded using a Tobii Eye Tracker
TX-300 set on a desk in front of the subject. The TX-300
Eye Tracker is a screen-based eye tracking system that
capture gaze data at 300 Hz. The accuracy of the system
is less than 0.5 degrees in optimal conditions. The screen
resolution for the experiment was 1920 x 1080. The
software Tobii Studio Pro was used for the design of the
experiment, to collect the data, and to draw the AOIs. A
special script programmed in Java was employed to
compute the eye tracking measures, since Tobii Studio Pro
does not provide First and Second Pass Reading times as
part of the manufacturer software.
MaterialsTexts
The twelve target texts focused on economic
variableson a range of topics, such as dollar value, copper
production, oil exportation, and mining resources. The
twelve texts were designed in order to display the four
experimental conditions as described in Figure 1. All
participants read all critical items in all condition, and
each condition was read – in different versions – three
times by each participant. Filler items were added to the
critical stimuli in a 2:1 ratio. The original texts from
which the stimuli were constructed were part of curricular
available information the participants have encountered
in previous school activities.
The graphs designed for the experiments
independently display a negative or inverse cause and consequence
relation (graph system). The graph in Figure 1 shows two
variables that clearly represent the semantic relation
under study: the increasing of one variable generates the
reduction of the other (e.g., the increase of forest fires in
the South caused the decrease of wood production). The
same explicit/implicit cause/consequence relation is
observed in the verbal system: two juxtaposed sentences
that may or may not be connected by the discourse
marker por tanto and thatexpress a cause-consequence
semantic relation (see Figure 1). Therefore, the information
presented in the VS and the information displayed in the
GS are the same in terms of semantic content, making it
possible to state that there is a synonymic relation (
94
). Both
semiotic systems express the equivalent causes and
consequences, only with words in one case, and by means of
lines, and a layout of numbers and words in the other.
Procedure
The presentation of the twelve texts was created using
the Tobii Studio software (Tobii Technology AB).
Participants were seated in a chair facing a computer
monitor in a quiet room, at a distance of approximately
70 cm from the monitor. A chin rest was used to
minimize head movements. Next, the eye tracker was adjusted
for optimal recording. An initial calibration pattern was
displayed to participants before running the eye tracking
session.
After calibration, participants were told that they
would be shown a series of texts on the computer monitor
while their eyes’ position was recorded. Participants were
asked to read normally, for comprehension, at their own
pace. They were also told that after each text they would
need to answer multiple-choice comprehension
questions.After reading the instructions, participants moved to
the next screen by pressing a key on the keyboard. Each
trial began with the presentation in the upper left corner of
a black cross on a white background for one second,
followed by the presentation of the text. Each experimental
stimulus was followed by a comprehension task. When an
answer was selected, a new trial started. The entire
session took approximately fifteen minutes. There were no
time limitations in the experiment.
Data Analysis
To achieve the first objective of this study, main
effects and interaction analyses were performed for DM
and G factors. For these analyses, a two-factor repeated
measures ANOVA was conducted. Both normality and
sphericity assumptions were tested. While ANOVA tests
are usually robust against normality transgressions,
Greenhouse-Geisser (GG) corrections were obtained in
case of violations of the sphericity assumption.
In order to accomplish the second objective, we
conducted a Wilcoxon Signed-Rank test for related samples,
since the normality assumption for these measures was
not met.
All analyses were conducted using IBM SPSS
Statistics for Macintosh (
31
), version 24.0.
Results
Comprehension rates were high (M = 89%, SD = 8.9%).
This data confirmed that the group of readers is rather
homogeneous in terms of their level of comprehension of
the texts and the causal relations included as part of them.
At the same time, these results show that the students
constructed coherent mental representations, and that the
situation models were satisfactorily built up (
97, 34, 103
).
Results on the VS AOIVerbal system AOI: Fixation Time
Table 1 shows mean (M) and standard deviation (SD)
for all experimental conditions for Fixation Time on the VS.
Mean and Standard Deviation for Fixation Time on the verbal system (SV).
Factor
Discourse Marker (DM)
Presence
Absence
M
SD
M
SD
n
Graph (G)
Presence
22946a
10189a
23330
9969
76
Absence
20532
7767
22108
8307
76
aFixation Time is expressed in milliseconds (ms).
Results showed a significant main effect of the graph (G)
for the Fixation Time(F(1,75)=5.234,p=.025,η²=0.029),
but no significant effects for both discourse marker (DM)
(F(1,75)=2.347,p=.130,ns) and interaction between G
and DM (F(1,75)=.850,p=.359,ns). The effect size for the
G main effect on Fixation Time may be considered
between small and moderate, according to Cohen (
10
) criteria
(i.e., η²=0.06).
Cause and consequence AOIs: Fixation Time
Table 2 shows M and SD for all experimental conditions
for Fixation Time on the cause segment (S1).
Mean and Standard Deviation for Fixation Time on the cause segment (S1).
Factor
Discourse Marker (DM)
Presence
Absence
M
SD
M
SD
n
Graph (G)
Presence
9323a
8339a
8316
6551
76
Absence
8109
5672
7627
4819
76
aFixation Time is expressed in milliseconds (ms).
The results show no significant main or interaction effects
for the Fixation Time on cause segment (S1). By contrast, significant
main effects of the DM were observed on the consequence segment (S2)
(F(1,75)=15.553,p<.001,η²=0.060). The effect size for
DM main effect for Fixation Time may be considered
moderate. Nevertheless, no significant effects were
observed for either the G factor (F(1,75)=3.651, p=.060,ns)
orinteraction between the G and the DM
(F(1,75)=.539,p=.465,ns). Table 3 shows M and SD for
all experimental conditions for Fixation Time on the
consequence segment (S2).
Mean and Standard Deviation for Fixation Time on the consequence segment (S2).
Factor
Discourse Marker (DM)
Presence
Absence
M
SD
M
SD
n
Graph (G)
Presence
13623a
5844a
15013
6423
76
Absence
12422
5224
14481
6430
76
aFixation Time is expressed in milliseconds (ms).
Cause and consequence AOIs: First
Pass Reading Time and Second Pass Reading Time
This subsection presents more fine-grained measures by
including First Pass Reading Time and Second Pass Reading Time
on S1 and S2. Table 4 shows M and SD for all experimental conditions
for both parameters on the cause segment (S1).
Mean and Standard Deviation for First Pass and Second Pass on the cause segment (S1).
Factor
Discourse Marker (DM)
Presence
Absence
M
SD
M
SD
n
First Pass
Graph (G)
Presence
2928a
2574a
2756
2295
76
Absence
2530
2072
2922
2152
76
Second Pass
Graph (G)
Presence
6395
6865
5560
5156
76
Absence
5580
4456
4705
3766
76
aFirst and Second Pass times are expressed in milliseconds (ms).
The results showed no significant interaction effects
for the First Pass on S1. However, for the Second Pass a main effect
of the DM factor was observed (F(1,75)=5.970,p=.017,η²=0.019).
Nevertheless, the effect size for DM main effect may be consider rather
small.
Table 5 shows M and SD for all experimental
conditions for both parameters on the consequence segment
(S2).
Mean and Standard Deviation for First Pass and Second Pass on the consequence segment (S2).
Factor
Discourse Marker (DM)
Presence
Absence
M
SD
M
SD
n
First Pass
Graph (G)
Presence
1874a
1350a
2465
2142
76
Absence
2354
1552
2751
1918
76
Second Pass
Graph (G)
Presence
11749
5840
12547
6246
76
Absence
10067
5541
11729
6510
76
aFirst and Second Pass times are expressed in milliseconds (ms).
Regarding First Pass, main effects for both DM factor
(F(1,75)=9.304, p=.003, η²=0.033) and G factor
(F(1,75)=4.954, p=.029, η²=0.020) were observed.
Similarly, for the Second Pass, main effects of both DM factor
(F(>1,75)=6.908, p=.010, η²=0.029) and G factor
(F(1,75)=6.860, p=.011, η²=0.029) were observed. All
effect sizes may be considered between small and
moderate (i.e., η²=0.01=small, η²=0.06=moderate). No
interaction effects were observed.
Results on the GS AOI
Table 6 shows M and SD for two experimental
conditions (A and C) for Fixation Time, First Pass and Second Pass on
the GS AOI.
Mean and Standard Deviation for Fixation Time, First Pass, and Second Pass on the graph system (GS).
Discourse Marker (DM)
Presence
Absence
M
SD
M
SD
n
Fixation Time
10147
7857
9752
8333
76
First Pass
3807
4201
3543
5317
76
Second Pass
6339
6533
6208
6437
76
All analyses on the GS AOI showed no statistical
significant differences between conditions A and C for
Fixation Time (Z= - .730, p=.465; ns), First Pass Reading
Time (Z= - .764, p=.445; ns), or Second Pass Reading
Time (Z= - .096, p=.923; ns).
Discussion
We had expected to observe that the presence of the
DM and the G would diminish reading times, separately.
Moreover, we had expected that the interaction of both of
them would also decrease reading times. These
hypotheses were tested for the VS, the S1, and the S2. An
additional expectation was that the presence of the DM would
decrease reading times on the GS.In general terms, results
regarding the first objective showed main effects of DM
and of G. However, no interaction effects (i.e., DM x G)
were observed. The analyses for the second objective
revealed that no significant effects for DM on the GS
were observed.
According to the results obtained, no main effects of
the DM were observed on the complete verbal system
(S1+S2); however, the presence of the connective showed
varying influences on the two independent discourse
segments constituting the cause-consequence semantic
relation. Considering the S1,results show that the
presence of the DM slows-down Second Pass Reading Time
on the causal segment. More interestingly, the DM
showed important effects on the consequence discourse
segment (S2) for all the three measures involved in this
study. The results revealed that the DM reduces Fixation
Time, and First and Second Pass Reading Time on the S2
segment.By contrast, no main effects of the DM were
observed on the GS. These findings partially support one
of the hypotheses of this study: the DM reduces cognitive
efforts while reading causally-related texts by speeding
up the time needed to process the consequence segment.
As mentioned, for the causal discourse segment (S1)
an effect of the DM in Second Pass was found in the
inverse direction than for S2. In fact, the S1 is processed
more slowly in presence of the connective por tanto. Its
procedural instruction leads readers to return to the S1
more often than in the implicit condition in order to build
a situation model Explaining this result deserves
reference to how both S1 and S2 are processed during First
Pass reading. No significant effects of the DM were
found on S1 in First Pass, which may find a
straightforward explanation based on the ‘principle of continuity’ (
51
):
reading starts at S1, and it is not until readers finish
reading it that they start to have expectations about the type of
discourse relation that holds between both segments.
Also, the significantly lower First Pass Reading Time of
S2 in the explicit condition shows that the presence of
por tanto allows readers to acknowledge, from the
beginning of the processing, that the S2 following the DM is a
consequence that must be reasoned out from the content
of S1. Such facilitating effect of the DM seems to be
suppressed during the Second Pass of S1, which is longer
in the explicit condition. However, the facilitating effect
of the DM endures if global processing is considered, as
reflects in the fact that no significant differences were
observed in the total reading time (Fixation Time) for S1.
In other words, the participants of the study are
compelled to link S1 and S2 causally, and this produces a
slow-down effect of the Second Pass on S1 (cancelled out
in Fixation Time), probably due to the readers’ need to
re-check the semantic status of S1 as the cause of the
discourse relation in order to build the situation model.
For the consequence discourse segment, the global
facilitating effect of the DM throughout reading (in First
and Second Pass as well as in Fixation Time) may be due
to the nature of connection in general and to the structure
of forward consecutive relations in particular. Instances
of connection need at least two discourse segments for
the relation to arise. In the case of forward consecutive
relations (a cause is followed by a consequence), the first
segment argumentatively points to the second, either
based on its lexical content and/or by means of the
presence of a forward consecutive connective. In other words,
the event expressed in S1 moves readers towards S2,
where they will carry out further inferential processes
until a complete mental representation is constructed. A
procedural instruction like the one coded in por tanto, as a
generator of readers’ expectations about ‘what comes
next’, facilitates the integration of the argument contained
in S2, which, as the findings revealed, is read faster when
the DM is provided.
In general terms, in this study por tantocontributes
toclarify the underlying causal relation between the
discourse segments by means of its procedural instruction:
“process the second discourse segment as a consequence
deduced from the events depicted in the first segment” (
43
).
Even if por tanto is not itself the source of the discourse
relation – given it does not create it – the DM appears to
force the connecting of the segments, “to add some
meaning to the reading of the overall fragments” (
14
). Thus, it
makes the relation explicit and conveys a more
constrained semantic status to each of the discourse segments
that constitute the text, particularly to the S2.
In short, in spite of the fact that in general readers
expect two juxtaposed verbal segments to be arranged
linearly and to be causally related, in this study, the implicit
condition demands, in general, longer processing times
compared to the explicit condition. This finding supports
the importance of explicit linguistic markers in order to
establish discourse relations; particularly, in relation to
the facilitating role of causal connectives to construct
causal relations, especially when the access to a stored
assumption in which to integrate the processed text does
not seem to be sufficiently constrained by the
propositional content of the discourse segments (
42, 79, 92, 93, 98, 102, 101
)(see
also (
52, 53
) on Spanish por tanto for different results when
participants are confronted with causal relations
reflecting assumptions clearly stored in their long-term memory
as everyday world knowledge).
In relation to the G, in global terms, that is,
considering Fixation Time, the G showed a slow-down effect on
the VS. For S1 AOI, no main effects of the G
wereobserved, neither in Fixation Time, nor in First or Second
Pass Reading. Regarding the S2 AOI, even if the G
reduced First Pass reading, it increased Second Pass, which
cancelled out the earlier effect, as can be seen in the
significantly longer Fixation Time of S2. The fact that a G
effect in the same direction of the VS (slow-down) was
found for S2 during Second Pass, allows us to
maintainthat all the detrimental impacts of the G concentrate
particularly on S2.
Contrary to our hypothesis, the results revealed that
reading the VS (S1+S2) of the texts including the G required
more cognitive efforts and led to a more demanding task.
Thus, the presence of a graph does not facilitate the
processing of causally-related information; on the contrary, it
seems to delay moment-to-moment reading processes,
increasing one reading measure (Fixation Time). While
the G showed an effect on the VS, the DM did not reveal
any main effects.
The discussion so far suggests that constructing a
mental coherent representation of multisemiotic texts
(+DM+G) is not favored when the text presentation
includes, at the same time, synonymous information from
two different sources (‘Redundancy Effect’ (
5, 6, 90, 91
)). The
finding that the presence of the DM haspositive main
effects supports, therefore, only partially the general
hypothesis of this study: only the procedural meaning of
the DM – and not the pictorial visualization of the causal
relation by means of a statistical graph– seems to speed
up reading times of causally-related texts.
Processing multisemiotic causal texts constituted by a
verbal system and a graph system may produce a
cognitive overload in working memory and, consequently, may
delay processing in semantic memory. This may occur
preferably when the information is presented in spatial
and temporal contiguity (
46
). In this line,Schnotz's (
82
) Integrated Model
of Text and Picture Comprehension predicts that the
combination of text and pictures could also have
detrimental effects. This negative effect (‘Redundancy
effect’)may occur when information is presented in
multiple additional forms or is unnecessarily elaborated. For
texts written in Spanish, Parodi and Julio (
63
) observed better
results among university students in writing summaries
based on comprehension of economics texts when readers
were given a single semiotic system version (only verbal
or only graphic) of the Monetary Policy Report
genre.When texts present this kind of information, it is
possible that readers be affected by the so called
‘Split-Attention Principle’ (
49, 81, 47
); therefore, their overloaded
attentional resources focus on only one system at a time,
and then on the other, not being able to connect them
immediately.Using eye tracking techniques, Parodi and
Julio (
62
) also reported findings that support the idea that
when presented with verbal+graphic information in
economics texts, university students preferred reading in first
and for longer period of time the verbal system rather
than the graphic. This could imply that integrating
information from verbal and graph systems, in some contexts
and for some readers, demands more cognitive effort, so
readers tend to select and to concentrate preferably on the
verbal system due to an overload on the visual working
memory (‘Split-Attention Principle’).
However, on the other hand, the dual-coding
theory (
73
) assumes that adding pictures to texts always leads to
better learning (two codes in memory are better than
one). Similarly, Schnotz (
82
) also points out that numerous
studies have shown that -depending on specific
conditions- students usually learn better from words and
pictures than from words alone (
45, 48
) ('Multimedia Effect'). In
the same line, Holsanova, Holmqvist and Holmberg (
29
) in a
naturalistic newspaper eye tracking study, found that an
integrated format with spatial contiguity between text and
graphics facilitates integration and prolongs reading.
Complementarily, we may advance another possible
explanation for the findings. Readers of our experiment
have been educated to pay more attention to the verbal
system than to any other semiotic system (‘Logocentric
Principle’ (
62
)). In most cases, they certainly read and
comprehend multisemiotic texts (
63
), but they have been formally
cultured to believe that the most important component of
a written text isthe words. Moreover, Parodi and
Julio (
62
) asked university students in economics and in
language studies three questions about perception and
preference of relevance of the verbal or graphic information
in written texts. More than 70% of the students in each
discipline agreed that germane information was found in
words rather than in graphs. No statistically differences
were observed between disciplinary origin of the
university students. For Radford (
67
), the eyes have been
domesticated, that is, culturally educated to read a semiotic
system with a specific emphasis. This seems a promising
hypothesis that may deserve further research.
Nevertheless, considering the findings of the present
study,it is not only the case that readers did not pay
attention to the graph at all; in fact, they did read it (see Table
6). The specific findings regarding the G showed that the
VS and, particularly, the consequence segment (S2) were
read for a longer time in the presence of the G. This
indicates that when reading a text in which different semiotic
systems are present, the reader should read them both and
then integrate the propositional contents of each system
into one coherent mental representation (
82, 46, 47, 44
). This
additional process should increase second reading times
and the integrative transitions between semiotic systems.
In order to find out whether the increase on the reading
times on the VS and especiallyon the S2 was due to an
effect of the transitions between S1 or S2 and GS, we
conducted further preliminary analyses. To this end,
transitions (from and back) were obtained, and a
Wilcoxon Signed-Rank test for related samples was conducted to
test the hypothesis. Our interim results showed that
transitions between the S2 and the GS were higher in number
than the transitions between the S1 and the GS (Z =
7.366; p = .000) (see Appendix 1).
These results may suggest that the integration
processes of the consequence segment (S2) may occur
differently from those of the cause segment (S1), which
would offer a preliminary explanation for the longer
reading times on the consequence. Furthermore, the
provisional data opens the possibility that the integration of
both semiotic systems, the graphic and the verbal, could
take place with focus on the consequence segment. More
analyses would be needed to specify the precise region in
the graph from which the information is contrasted with
the verbal information. We are certain that these analyses
were beyond the scope of the current study, but this
preliminary data mayinspirehypotheses for future research.
Conclusions
After decades of intensive research primarily focused on
the verbal system of written texts, processing
multisemiotic discourse is now receiving increasing interest, aimed
in particular at the comprehension of texts that include
graphs. In this article, we have argued that words and
graphs working together may help readers construct a
coherent mental representation of the text. Thus, when
constructing a complex mental model, readers would
benefit from a more informationally dense text containing
a DM and a G, expressing an inverse causal objective,
content-related semantic relation (
97, 35, 73, 47, 83
). In light of
these arguments, our initial hypothesis was that a higher
informative text would lead to faster processing of the
information expressed verbally and graphically, and that
subsequent reading times would thenslow down. As
discussed above, however, the joint presence of the DM and
the G as an integral unity that facilitates discourse
processing was supported only partially. Although we found
no evidence for interaction effects between the
presence/absence of the cause-consequence discourse marker
por tanto and the statistical causal-consequence graph, we
observed main effects on the DM and the G, separately.
The effects of the DM are particularly clear and reveal
that its instructional value endures until the construction
of a coherent mental representation has been completed.
The presence of the G, however, slows down the
processing of the verbal system, particularlyin the
consequence segment. This is due to the fact that the G requires
to be integrated and this is reflected on the VS.
Complementarily, additional preliminary analyses on
integrative transitions between the S1, the S2 and the GS
revealedthat the cause and the consequence discourse
segments were proceeded differently in their interplay
with the GS. More interactions were observed between
the consequence discourse segment and the graph.
Considering methodological implications, we are
aware that the argument over which measure is best to
use as an index of cognitive processing partly depends on
what is the determined focus of the examination at hand.
The currently available measures do not always help the
researcher capture or reflect the whole reality involved in
cognitive processing, particularly at discourse level. In
this vein, the preliminary evidence advanced here,
analyzing discourse segments with particular distinctive
functional and semantic properties by resorting to
different eye tracking measures, helped reveal that the cause
segment was processed differently. This suggests that the
analysis of global processing indicators is well
complemented by precise parameters, which in this study helped
disclose effects of combining different semiotic systems
in texts, an effect particularly visible on S2.
As well known, research with eye tracking
technology for a long time focused mainly on the processing of
syllables, words, and isolated short sentences; thus,
exploring and defining new related fine-grained measures
are challenges for researchers, particularly when
studyingprocessing, for example, atmultisemiotic discourse
level or on global or macro dimension (
38, 30, 50, 28
).
As for the limitations of this paper, in future research,
we should move, on the one hand, beyond synonymic
relations between words and graph. A more demanding
and probably diverse data could be explored studying
multiple relations between systems, such as antonym or
more specifically what we could call ‘complementarity’:
a multisemiotic relation in which a graph would add
crucial exclusive information to the words forcing the
reader to critically integrate both representational systems
in order to construct a unified coherent mental
representation of the whole text. On the other hand, this study
focuses in cause-consequence content relations marked by
the Spanish connective por tanto in which the related
segments present an inverse negative relation. The effects
of other types of causal relations and of other connectives
on multimodal processing should be investigated in the
future, as well as the effects of other types of causal
relations and of other connectives, also those that require a
different order of the cause and consequence segments.
Despite these limitations, the current research has
scientific significance as it suggests the potential of a novel
approach to combine words and graphs from a
discourseoriented perspective; also, it can be taken as a starting
point to further examine, for instance, the theoretical
implications of the various formats of multisemiotic text
representations and their integration processes. As of our
knowledge, even though there exist some advances, there
is no up-to-date systematic available description of the
words-graphs intermodal connections across disciplinary
discourse genres and their respective levels in
comprehension, as well as in learning. A more in-depth analysis
should take into account the present findings.
After a decade of research on multimodal
words/graph text processing and comprehension, we still
adhere to Acartürk et al.’s (
1
) words “…contra to the
models for eye movement control in reading, there is no
model for eye movement control in text-graphics documents,
despite their potential for formal descriptions”.
Finally, although our study only conducted
preliminary analyses on integrative transitions, it reveals a
promising future for this line of research. Investigating on the
reading routes ofmultisemiotic texts and the processes
involved in comprehension and discourse integration in
more natural settings is a challenging niche. In particular,
some questions for future multimodal research are:
• Which are the reading routes that lead to integration
of specialized disciplinary words/graph texts?
• At which exact moments do readers look at the graph?
• From which verbal system region do readers move to
the graph system?
• How often do readers pay attention to the graph
compared to the verbal system?
• How often do readers switch between the verbal
system and the graph?
Ethics and Conflict of Interest
The authors declare that the contents of the article are
in agreement with the ethics described in
http://biblio.unibe.ch/portale/elibrary/BOP/jemr/ethics.htmland
that there is no conflict of interest regarding the
publication of this paper.
Acknowledgements
This research was supported partially by grant CONICYT
20150058 and FONDEQUP 150119, from the National
Commission for Scientific and Technological Research (CONICYT).
We wish to thank PhD. Alonso Ortega, University of
Valparaíso, Chile, for providing expert assistance in
experimental methodology and statistics in this article.
Appendix
Mean and standard deviation of the Number of Transitions between S1 ↔ GS AOI, and S2 ↔ GS AOI.
Transitions
M
SD
S1 ↔ GS
3.68a
3.51a
S2 ↔ GS
15.92∗
8.49
a Number of transitions; ∗ Z= - .7366, p=.000
Acartürk, C., Habel, C., Cagitay, K., & Alacam, O. (2007). Multimodal comprehension of language and graphics: Graphs with and without annotations.Journal of Eye Movement Research, 1(3), 1–15. 10.16910/jemr.1.3.21995-8692Bertin, J. (1983). Semiology of graphics. Madison, WI: University of Wisconsin Press.Briz, A., Pons, S., & Portolés, J. (2017). Diccionario de Partículas Discursivas del Español. Retrived from http://www.dpde.esCanestrelli, A., Mak, W., & Sanders, T. (2013). Causal connectives in discourse processing: How differences in subjectivity are reflected in eye movements.Language and Cognitive Processes, 28(9), 1394–1413. 10.1080/01690965.2012.6858850169-0965Chandler, P., & Sweller, J. (1991). Cognitive Load Theory and the format of instruction.Cognition and Instruction, 8(4), 293–332. 10.1207/s1532690xci0804_20737-0008Chandler, P., & Sweller, J. (1996). Cognitive load while learning to use a computer program.Applied Cognitive Psychology, 10(2), 151–170. 10.1002/(SICI)1099-0720(199604)10:2<151::AID-ACP380>3.0.CO;2-U0888-4080Cheng, P., & Simon, H. A. (1995). Scientific discovery and creative reasoning with diagrams. In S.Smith, T.Ward, & R.Finke (Eds.),The Creative Cognition Approach (pp. 205–228). Cambridge, MA: MIT Press.Cleveland, W., & McGill, R. (1984). Graphical perception: Theory, experimentation, and application to the development of graphical methods.Journal of the American Statistical Association, 79(387), 531–554. 10.1080/01621459.1984.104780800162-1459Cohen, J. (1992). A power primer.Psychological Bulletin, 112(1), 155–159. 10.1037/0033-2909.112.1.155195656830033-2909Cohen, J. (1988). Statistical Power Analysis for the Behavioral Sciences. Hillsdale: Erlbaum.Danielson, R., & Sinatra, G. (2016). A Relational reasoning approach to text-graphic processing.Educational Psychology Review, •••, 1–18. 10.1007/s10648-016-9374-21040-726Xde Saussure, L. (2007). Procedural pragmatics and the study of discourse.Pragmatics & Cognition, 15(1), 139–159. 10.1075/pc.15.1.10sau0929-0907Der, G., & Everitt, B. (2015). A Handbook of Statistical Graphics Using SAS ODS. CRC Press.Degand, L. (1998). On classifying connectives and coherence relations. In Coling/ACL Workshop on Discourse Relations and Discourse Markers (pp. 29–35). Montreal, Canada, Domínguez, M. (2007). Conectores en Textos Argumentativos Breves. Madrid: Arco Libros.Domínguez, M. (2007). Conectores en Textos Argumentativos Breves. Madrid: Arco Libros.Duchowski, D. (2007). Eye Tracking Methodology: Theory and Practice. London: SpringerVerlag.Escandell, V., & Leonetti, M. (2011). On the rigidity of procedural meaning. In M. V.Escandell, M.Leonetti, & A.Ahern (Eds.),Procedural Meaning: Problems and Perspectives. Bingley: Emerald.Escandell, V., Leonetti, M., & Ahern, A. (Eds.). (2011). Procedural Meaning: Problems and Perspectives. Bingley: Emerald.Faul, F., Erdfelder, E., Lang, A. G., & Buchner, A. (2007). G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences.Behavior Research Methods, 39(2), 175–191. 10.3758/BF03193146176953431554-351XGarcía Izquierdo, I. (1998). El análisis textual como paso previo a la traducción. La tipología textual y su interpretación.TRANS: Revista de Traductología, 3, 133–140.Gili Gaya, S. (1943). Curso Superior de Sintaxis Española. Mexico: Minerva.Glymour, M. (2006). Using causal diagrams to understand common problems in social epidemiology. In J.Oakes & J.Kaufman (Eds.),Methods in Social Epidemiology (pp. 387–422). San Francisco, CA: Wiley.Graesser, A., & Bertus, E. (1998). The construction of causal inferences while reading expository texts on science and technology.Scientific Studies of Reading, 2(3), 247–269. 10.1207/s1532799xssr0203_41088-8438Green, N. (2005). Design of information graphics for causal arguments. In Proceedings of the International Joint Conference on Artificial Intelligence, Workshop on Computer Models of Natural Argument. Edinburgh, UK.Habel, C., & Acartürk, C. (2011). Causal inference in graph-text constellations: Designing verbally annotated graphs.Tsinghua Science and Technology, 16(1), 7–12. 10.1016/S1007-0214(11)70002-51007-0214Hegarty, M., Canham, M. S., & Fabrikant, S. I. (2010). Thinking about the weather: How display salience and knowledge affect performance in a graphic inference task.Journal of Experimental Psychology. Learning, Memory, and Cognition, 36(1), 37–53. 10.1037/a0017683200530430278-7393Hoek, J., & Zufferey, S. (2015). Factors Influencing the Implicitation of Discourse Relations across Languages.Proceedings of the 11th Joint ACL-ISO Workshop on Interoperable Semantic Annotation (ISA-11), 39-45.Holmqvist, K., Nystrom, M., Andersson, R., Dewhurst, R., Jarodzka, H., & van de Weijer, J. (2011). Eye Tracking – A Comprehensive Guide to Methods and Measures. Oxford: Oxford University Press.Holsanova, J., Holmqvist, K., & Holmberg, N. (2009). Reading information graphics: The role of spatial proximity and dual attentional guidance.Applied Cognitive Psychology, 23(9), 1215–1226. 10.1002/acp.15250888-4080Hyönä, J., Lorch, R., & Rinck, M. (2003). Eye movement measures to study global text processing. In J.Hyönä, R.Radach, & H.Deubel (Eds.),The Mind’s Eye: Cognitive and Applied Aspects of Eye Movement Research (pp. 313–334). Amsterdam: Elsevier. 10.1016/B978-044451020-4/50018-9IBMCorp. (2016). IBM SPSS Statistics for Macintosh, Version 24.0. Armonk, NY: IBM Corp.Juhasz, B., & Pollatsek, A. (2011). Lexical influences on eye movements in reading. In S.Liversedge, I.Gilchrist, & S.Everling (Eds.),The Oxford Handbook of Eye Movements (pp. 873–893). Oxford: Oxford University Press.Julio, C., Parodi, G., & Loureda, Ó. (2018). (in press). Congruencia entre sistemas semióticos: Estudio de palabras y gráficos con el uso de eye tracker.Revista Estudios Filológicos, 61.Kintsch, W. (1998). Comprehension: A Paradigm for Cognition. New York: Cambridge University Press.Kintsch, W. (2013). Revisiting the Construction- Integration Model of text comprehension and its implications for instruction. In D. E.Alvermann, N. J.Unrau, & R. B.Ruddell (Eds.),Theoretical Models and Processes of Reading (pp. 807–839). Newark, DE: International Reading Association. 10.1598/0710.32Kosslyn, S. (1994). Elements of Graph Design. New York: W. H. Freeman.Li, F., Sanders, T., & Evers-Vermeul, J. (2016). On the subjectivity of Mandarin reason connectives: Robust profiles or genre-sensitivity? In N.Stukker, W.Spooren, & G.Steen (Eds.),Genre in Language, Discourse and Cognition (pp. 15–50). Berlin: Mouton De Gruyter. 10.1515/9783110469639-003Liversedge, S., Patterson, K., & Pickering, M. (1998). Eye movements and measures of reading time. In G.Underwood (Ed.),Eye guidance in Reading and Scene Perception (pp. 55–75). Oxford: Elsevier Science. 10.1016/B978-008043361-5/50004-3López García, L. (1999). Relaciones paratácticas e hipotácticas. In I. Bosque & V. Demonte (Coords.), Gramática Descriptiva de la Lengua Española (pp. 3507–3549). Madrid: Espasa Calpe.Loureda, Ó., & Acín, E. (2010). Los Estudios sobre Marcadores del Discurso en Español, hoy. Madrid: Arco Libros.Mackie, J. (1974). The Cement of the Universe. London: Oxford University Press.McNamara, D., Kintsch, E., Songer, N., & Kintsch, W. (1996). Are good texts always better? Text coherence, background knowledge, and levels of understanding in learning from text.Cognition and Instruction, 14(1), 1–43. 10.1207/s1532690xci1401_10737-0008Martín Zorraquino, M. A., & Portolés Lázaro, J. (1999). Los marcadores del discurso. In I. Bosque & V. Demonte (Coords.), Gramática Descriptiva de la Lengua Española (pp. 4051–4213). Madrid: Espasa Calpe.Mason, L., Pluchino, P., & Tornatora, M. C. (2015). Eyemovement modeling of integrative reading of an illustrated text: Effects on processing and learning.Contemporary Educational Psychology, 41, 172–187. 10.1016/j.cedpsych.2015.01.0040361-476XMayer, R. (1997). Multimedia learning: Are we asking the right questions?Educational Psychologist, 32(1), 1–19. 10.1207/s15326985ep3201_10046-1520Mayer, R. (2005). Introduction to multimedia learning. In R.Mayer (Ed.),The Cambridge Handbook of Multimedia Learning (pp. 1–18). New York: Cambridge University Press.Mayer, R. (2009). Multimedia Learning. New York: Cambridge University Press; 10.1017/CBO9780511811678Mayer, R. (2017). Instruction based on visualizations. In R.Mayer & P.Alexander (Eds.),Handbook of Research on Learning and Instruction (pp. 483–501). New York, London: Routledge.Mayer, R., & Moreno, R. (1998). A split-attention effect in multimedia learning: Evidence for dual processing systems in working memory.Journal of Educational Psychology, 90(2), 312–320. 10.1037/0022-0663.90.2.3120022-0663Mikkilä-Erdmann, M., Penttinen, M., Anto, E., & Olkinuora, E. (2007). Constructing Mental Models during Learning from Science Text. In D.Ifenthaler, P.Pirnay-Dummer, & J.Spector (Eds.),Understanding Models for Learning and Instruction: Essays in Honor of Norbert M. Seel (pp. 63–79). New York: Routledge; 10.1007/978-0-387-76898-4_4Murray, J. D. (1997). Connectives and narrative text: The role of continuity.Memory & Cognition, 25(2), 227–236. 10.3758/BF0320111490990730090-502XNadal, L., Cruz, A., Recio, I., & Loureda, Ó. (2016). El significado procedimental y las partículas discursivas del español: Una aproximación experimental. Revista Signos.Estudios de Lingüística, 49, 52–77. 10.4067/S0718-09342016000400004Nadal, L., & Recio, I. (in press). Processing Implicit and Explicit Causality in Spanish. In Ó.Loureda, I.Recio, L.Nadal, & A.Cruz (Eds.),Empirical Approaches to the Construction of Discourse: Information Structures, Argumentative Relations and Discourse Units. Amsterdam: Benjamins.Otero, J., León, J., & Graesser, A. (Eds.). (2002). The Psychology of Science Text Comprehension. Mahwah, NJ: Erlbaum.Pagano, R. (2011). Estadística para las Ciencias del Comportamiento. México: Cengage Learning.Pander Maat, H., & Degand, L. (2001). Scaling causal relations and connectives in terms of speaker involvement.Cognitive Linguistics, 12, 211–245.0936-5907Pander Maat, H., & Sanders, T. (2001). Subjectivity in causal connectives: An empirical study of language in use.Cognitive Linguistics, 12, 247–273.0936-5907Parodi, G. (2005). Discurso especializado y lingüística de corpus: Hacia el desarrollo de una competencia psicolingüística.Revista Boletín de Lingüística, 17(23), 61–68.Parodi, G. (2010). Multisemiosis y lingüística de corpus: Artefactos (multi)semióticos en los textos de seis disciplinas en el corpus PUCV-2010.Revista de Lingüística Teórica y Aplicada, 48(2), 33–70. 10.4067/S0718-48832010000200003Parodi, G. (2015). Variation across university genres in seven disciplines: A corpus based study on academic written Spanish.International Journal of Corpus Linguistics, 20(4), 469–499. 10.1075/ijcl.20.4.03par1384-6655Parodi, G., Julio, C., & Vásquez-Rocca, L. (2015). Los géneros del Corpus PUCVUCSC-2013 del discurso académico de la economía: El caso del Informe de Política Monetaria.Revista ALED, 15(3), 179–200.Parodi, G., & Julio, C. (2016). ¿Dónde se posan los ojos al leer textos multisemióticos disciplinares? Procesamiento de palabras y gráficos en un estudio experimental con eye tracker. Revista Signos.Estudios de Lingüística, 49(1), 149–183. 10.4067/S0718-09342016000400008Parodi, G., & Julio, C. (2017). More than words: Contending semiotic systems and the role of disciplinary knowledge in specialized text comprehension.Ibérica (New York, N.Y.), 33, 11–35.1056-5000Pearl, J., Glymour, M., & Jewell, N. (2016). Causal Inference in Statistics. John Wiley & Sons.Pinker, S. (1982). A Theory of Graph Comprehension. Occasional Paper No. 15, Cambridge, MA: MIT Center for Cognitive Sciences.Pinker, S. (1990). A theory of graph comprehension. In R.Freedle (Ed.),Artificial Intelligence and the Future of Testing (pp. 73–126). Hillsdale, NJ: Erlbaum.Radford, L. (2010). The eye as a theoretician: Seeing structures in generalizing activities.For the Learning of Mathematics, 30(2), 2–7.0228-0671RAE (2009). Nueva Gramática de la Lengua Española. Madrid: Espasa Calpe.Rayner, K. (1998). Eye movements in reading and information processing: 20 years of research.Psychological Bulletin, 124(3), 372–422. 10.1037/0033-2909.124.3.37298491120033-2909Rayner, K. (2009). Eye movements and attention in reading, scene perception, and visual search.Quarterly Journal of Experimental Psychology, 62(8), 1457–1506. 10.1080/17470210902816461194492611747-0218Rayner, K., & Sereno, S. (1994). Eye movements in reading: Psycholinguistic studies. In M.Gernsbacher (Ed.),Handbook of Psycholinguistics (pp. 57–82). New York: Academic Press.Rayner, K., Chace, K., Slattery, T., & Ashby, J. (2006). Eye movements as reflections of comprehension processes in reading.Scientific Studies of Reading, 10(3), 241–255. 10.1207/s1532799xssr1003_31088-8438Sadoski, M., & Paivio, A. (2001). Imagery and Text. A Dual Coding Theory of Reading and Writing. Mahwah, NJ: Erlbaum.Sanders, T., & Spooren, W. (2001). Text representation as an interface between language and its users. In T.Sanders, J.Schilperoord, & W.Spooren (Eds.),Text Representation: Linguistic and Psycholinguistic Aspects (pp. 1–26). Amsterdam: Benjamins. 10.1075/hcp.8.02sanSanders, T., & Spooren, W. (2015). Causality and subjectivity in discourse: The meaning and use of causal connectives in spontaneous conversation, chat interactions and written text.Linguistics, 53(1), 53–92. 10.1515/ling-2014-00340024-3949Sanders, J., Sanders, T., & Sweetser, E. (2012). Responsible subjects and discourse causality. How mental spaces and perspective help identifying subjectivity in Dutch backward causal connectives.Journal of Pragmatics, 44(2), 191–213. 10.1016/j.pragma.2011.09.0130378-2166Sanders, T., Spooren, W., & Noordman, L. (1992). Toward a taxonomy of coherence relations.Discourse Processes, 15(1), 1–35. 10.1080/016385392095448000163-853XSanders, T., Spooren, W., & Noordman, L. (1993). Coherence Relations in a Cognitive Theory of Discourse Representation.Cognitive Linguistics, 4(2), 93–133. 10.1515/cogl.1993.4.2.930936-5907Sanders, T. (2005). Coherence, Causality and Cognitive Complexity in Discourse. In M. Aurnague, M. Bras, A. Le Draoulec & L. Vieu (Eds.), Proceedings/Actes SEM-05, First International Symposium on the exploration and modelling of meaning (pp. 105-114). Toulouse: Université de Toulouse le Mirail.Schmid, C. (1954). Handbook of Graphic Presentation. New York: Ronald Press.Schnotz, W. (2005). An integrated model of text and picture comprehension. In R.Mayer (Ed.),The Cambridge Handbook of Multimedia Learning (pp. 49–69). Cambridge: Cambridge University Press; 10.1017/CBO9780511816819.005Schnotz, W. (2014). Integrated Model of Text and Picture Comprehension. In R.Mayer (Ed.),Cambridge Handbook of Multimedia Learning (pp. 72–103). Cambridge: Cambridge University Press. 10.1017/CBO9781139547369.006Schnotz, W., & Baadte, C. (2015). Surface and deep structures in graphics comprehension.Memory & Cognition, 43(4), 605–618. 10.3758/s13421-014-0490-2254658980090-502XSeltman, H. (2015). Experimental Design and Analysis. Pittsburgh: Carnegie Mellon University.Shah, P. (1997). A model of the cognitive and perceptual processes in graphical display comprehension. In M.Anderson (Ed.),Reasoning with Diagrammatic Representations II (pp. 94–101). Menlo Park, CA: AAAI Press.Sperber, D., & Wilson, D. (1986). Relevance: Communication and Cognition. Oxford: Blackwell.Sperber, D., & Wilson, D. (1997). Remarks on Relevance Theory and the Social Sciences.Multilingua, 16(2-3), 145–152. 10.1515/mult.1997.16.2-3.1450167-8507Strobel, B., Sass, S., Lindner, M., & Köller, O. (2016). Do graph readers prefer the graph type Most Suited to a Given Task? Insights from Eye Tracking.Journal of Eye Movement Research, 9(4), 1–15. 10.16910/jemr.9.4.41995-8692Sweetser, E. (1990). From Etymology to Pragmatics: Metaphorical and Cultural Aspects of Semantic Structure. Cambridge: Cambridge University Press. 10.1017/CBO9780511620904Sweller, J. (2005). Implications of cognitive load theory for multimedia learning. In R.Mayer (Ed.),The Cambridge Handbook of Multimedia Learning (pp. 19–30). New York: Cambridge University Press. 10.1017/CBO9780511816819.003Sweller, J., Ayres, P., & Kalyuga, S. (2011). Cognitive Load Theory. New York: Springer. 10.1007/978-1-4419-8126-4Taboada, M. (2006). Discourse markers as signals (or not) of rhetorical relations.Journal of Pragmatics, 38(4), 567–592. 10.1016/j.pragma.2005.09.0100378-2166Taboada, M. (2009). Implicit and explicit coherence relations. In J.Renkema (Ed.),Discourse, of Course: An Overview of Research in Discourse Studies (pp. 127–140). Amsterdam: Benjamins. 10.1075/z.148.13tabTaboada, M., & Habel, C. (2013). Rhetorical relations in multimodal documents.Discourse Studies, 1(1), 65–89. 10.1177/14614456124664681461-4456Tufte, E. (2001). The Visual Display of Quantitative Information. Cheshire, Connecticut: Graphics Press.van den Broek, P. (1990). Causal inferences in the comprehension of narrative texts. In A.Graesser & G.Bower (Eds.),Psychology of Learning and Motivation: Inferences and Text Comprehension (pp. 175–196). New York: Academic Press.van Dijk, T., & Kintsch, W. (1983). Strategies of Discourse Comprehension. New York: Academic Press.van Silfhout, G., Evers-Vermeul, J., & Sanders, T. (2014). Connectives as processing signals: How students benefit in processing narrative and expository texts.Discourse Processes, 52(1), 47–76. 10.1080/0163853X.2014.9052370163-853XWinn, W. (1994). Contributions of perceptual and cognitive processes to the comprehension of graphics. In W.Schnotz & R.Kulhay (Eds.),Comprehension of Graphics (pp. 3–28). North-Holland: Elsevier. 10.1016/S0166-4115(09)60105-9Zacks, J., & Tversky, B. (1999). Bars and lines: A study of graphic communication.Memory & Cognition, 27(6), 1073–1079. 10.3758/BF03201236105865820090-502XZunino, G. (2017). Procesamiento de causalidad y contracausalidad: Interacciones entre estructura sintáctica y conocimiento del mundo en la comprensión de relaciones semánticas. Revista Signos.Estudios de Lingüística, 50(95), 472–491. 10.4067/S0718-09342017000300472Zunino, G., Abusamra, V., & Raiter, A. (2012). Causalidad: Relación entre conocimiento de mundo y conocimiento lingüístico.Pragmalingüística, 20(20), 200–219. 10.25267/Pragmalinguistica.2012.i20.09Zwaan, R. A., & Radvansky, G. A. (1998). Situation models in language comprehension and memory.Psychological Bulletin, 123(2), 162–185. 10.1037/0033-2909.123.2.16295226830033-2909