THE IMPACT OF EXTERNAL GRAPHICAL REPRESENTATIONS IN DIFFERENT KNOWLEDGE DOMAINS : IS THERE A DOMAIN EFFECT ?

The impact of external graphical representations in different knowledge domains
THE IMPACT OF EXTERNAL GRAPHICAL
REPRESENTATIONS IN DIFFERENT KNOWLEDGE DOMAINS:
IS THERE A DOMAIN EFFECT?

KATRIEN DE WESTELINCK & MARTIN VALCKE
Department of Education, Ghent University

Correspondence concerning this paper should be addressed to K. De Westelinck,
Ghent University, Department of Education, H. Dunantlaan 2, B9000 Ghent,
Belgium, katrien.dewestelinck@UGent.be

This study was partially funded under the Flemish BOF program, nr. 011D00103.

Abstract

In this paper three studies question the generalizability of Mayer’s Cognitive Theory of
Multimedia Learning (CTML) to the knowledge domain of the social sciences. The first
research builds on the assumption that this knowledge domain differs in the way instructional
designers develop depictive external graphical representations and the way students
understand these representations. The key variable in this process is being (un)acquainted
with the iconic sign system that is used to develop the representations in this knowledge
domain. The research results reveal that studying learning materials enriched with external
graphical representations in the social sciences domain does not result in higher test
performance and does not result in lower levels of cognitive load. The second research builds
on these former results and questions under which condition the performance on retention and
transfer tests can be increased. Therefore a new design principle activation is put forward.
Results show that the performance on the retention and transfer tests is higher when the
learner actively builds his/her personal external graphical representations. The active
construction of personal external graphical representations positively influences the
performance on retention and transfer tests. The results of the second study are the starting
point of the third research that focuses on the differential impact of the types of personal
external graphical representations learners add to textual learning materials.

Multimedia Learning in Social Sciences: Impact of the Knowledge Domain

The Cognitive Theory of Multimedia Learning (CTML), put forward by Mayer
(2001a), presents a clear framework to direct instructional design of both printed and
interactive multimedia materials. Despite the theoretical and practical appeal of the
theory for instructional designers, daily teaching experience of the authors of the
present paper, responsible for freshman courses in the knowledge domain of
educational sciences, is not always in line with the CTML. It appears that students
cope with external graphical representations such as schemas, tables and graphs
added to learning materials. And, as will be discussed in the next sections, recent
research is not always able to replicate the positive findings that have been reported
in earlier CTML-studies in other knowledge domains. The authors suggest that the
nature of the knowledge domain and the nature of the external graphical
representations interact with the validity of the CTML-design principles. By testing
the original CTML-design principles in another knowledge domain questions about
extending and/or generalizing the cognitive theory of multimedia learning are raised.





1

---

The impact of external graphical representations in different knowledge domains
Basic Assumptions and Design Guidelines of CTML

Cognitive processing of information is the concept from which Mayer (2001a, 2003)
starts to formulate three central assumptions of the Cognitive Theory of Multimedia
Learning (CTML): the dual channel assumption (Baddeley, 1992; Chandler &
Sweller, 1991; Moore, Burton & Myers, 1996; Paivio, 1978; Paivio, 1991; Nelson,
1979; Broadbent, 1956; Shannon & Weaver, 1949; Mayer, 2001a; Mayer, 2003), the
active processing assumption (Wittrock, 1989; Mayer, 2001a; Mayer, 2003) and the
limited capacity assumption (Sweller, 1988; Sweller, 1989; Sweller, 1994; Mayer,
2001a; Mayer, 2003).

This leads to the following design principles (Mayer, 2001a; Mayer 2001b;
Mayer, 2003): (a) the multimedia principle: learners benefit more from printed text
enriched with pictures than from printed text alone, (b) the temporal contiguity
principle: learners perform better when corresponding printed text and pictures are
presented simultaneously instead of successively, (c) the spatial contiguity principle:
learning is fostered when printed text and pictures are presented close to one another
on a page or on screen, (d) the coherence principle: learning performance is higher
when extraneous sounds, words, pictures are excluded, (e) the modality principle:
learners learn more from animation enriched with audio (narration) than from
animation enriched with printed text, (f) the redundancy principle: learners perform
better when presented with animation and narration instead of animation and
narration combined with printed text matching the narration, and (g) the individual
differences principle: all design principles have a stronger impact with low prior
knowledge learners and learners with higher spatial abilities (see Mayer, 2001a,
2001b and 2003 for an overview). The generic nature of these design principles for
all knowledge domains and types of external graphical representations is stressed by
Mayer (2001a), but is questioned in this paper.

External Graphical Representations

First generation research focused on generic principles to understand consistencies
in the processing of verbal and visual information (Anglin, Towers, & Levie, 1996;
Goldman, 2003). Second generation research focuses rather on the affordances of
and acquaintance of the students with iconic sign system as reflected in external
graphical representations and this in relation with the nature of knowledge domain.

Verbal and pictorial representations behold a clear difference subscribed by
Mayer. Verbal representations such as text (descriptive), require more mental effort
to be processed by the learner and pictorial representations (depictive) are
considered as being more original modes of knowledge representation and are more
intuitive, closer to visual experience (Mayer; 2001a, p. 68).
Being acquainted with descriptive and depictive representations can
influence the processing of the representations by the learners. As to the depictive
representations, the question is raised whether the learners have sufficient and
adequate prior knowledge that directs their understanding of these representations.
Prior knowledge related to the mastery of the iconic sign system that is at the base of
the external graphical representations is needed. When learners are confronted with
new or unknown iconic sign systems, this is expected to result in learner difficulties
(e.g. weaker processing and/or more processing time needed). It points at a
mismatch between the iconic sign system of a learner and the one used in the
external graphical representations (De Westelinck, Valcke, De Craene and
Krischner, in press). The same idea has been put forward by other authors in the
literature, but has yet not been related to CTML (Stenning, 1999; Dobson, 1995;
Goodman, 1976; Lewalter, 2003; Goldman, 2003).
2

---

The impact of external graphical representations in different knowledge domains
Considering the issue of familiarity with the iconic sign system, the
activation principle is introduced in this paper. According to this principle, learners
are invited to develop their own external graphical representation; thereby building
on an iconic sign system they are familiar with. By developing a personal external
graphical representation, learners are forced to process the new knowledge elements
and to construct a visual mental model, next to the model based on text. This process
also reinforces the familiarity with the personal iconic sign system. Several authors
promote this activation principle (Marzono, Pickering & Pollock, 2001; Wileman,
1993).

The first study: CTML in the social sciences

Hypotheses

The central hypothesis put forward in this first research is that learners in the social
sciences knowledge domain will experience difficulties with external graphical
representations; thus resulting in non-confirmation of the original CTML
hypotheses. The unfamiliarity with the iconic sign system used to develop the
external graphical representations can have an influence on the selection, processing
and organisation processes of mental models. Students are e.g., expected to
experience cognitive load due to being not familiar with the iconic sign system.
Confirmation of this overall hypothesis implies that the CTML-design guidelines are
to be extended by a supplementary design principle that considers the mastery of the
iconic sign system by the learner and the nature of the knowledge domain. The
theoretical position about the nature of external graphical representations is
applicable to both static (e.g., graphs) and dynamic (e.g., animations) external
depictive representations.
In the first study, hypotheses related to five CTML-design principles were
tested: the multimedia, spatial contiguity, coherence, modality and redundancy
principle. In addition a sub-hypothesis was tested, in relation to each of the former
hypotheses, whether application of the principles did lead to higher or lower levels
of cognitive load as reported by the subjects.

Research Method

Participants
In total 190 freshmen, the entire first year student population studying educational
sciences, participated in this study (academic year 2002-2003). Participation was a
formal part of the course “Instructional Sciences”. Informed consent was obtained
from all students prior to experimentation.

Procedure
Three to six experiments were set up during two consecutive weeks in relation to
each of the five CTML design principles. Students were randomly assigned to the
experimental conditions. Each experimental package consisted of (a) a prior
knowledge test consisting of a retention and transfer test, (b) a specific elaboration
of the learning materials to be studied, and (c) a post-test to test the mastery of the
complex knowledge elaborated in the learning materials (retention and transfer
questions). Twice, the students were asked to estimate their perceived mental effort.
Literature and research confirm the validity and realibility of this approach to
determine cognitive load (Van Gerven et al., 2002; Paas et al. 1994; Paas, 1992;
Gopher & Braune, 1984). No time limit was set to study the materials and/or to fill
out the tests. The answers to the retention and transfer questions were scored by
3

---

The impact of external graphical representations in different knowledge domains
three independent researchers based on a scoring checklist. Scores for the tests were
standardized.

Materials
The content of the learning materials was both complex and new to the students: an
introduction to the learning styles literature (the learning content). Nine themes were
outlined to be presented to the students. Mayer’s recommendations were taken into
account when developing the external graphical representations. Figure 1 depicts a
sample page of printed learning materials with integrated external graphical
representations about Kolb’s learning style approach. It is clear from the example
that the external graphical representations do not build on a formal and/or existing
iconic sign system. Moreover, the approach is similar to the typical external
graphical representations found in most psychology and educational sciences
textbooks.


Figure 1. Example of learning materials related to the coherence principle building
on the learning style model of Kolb.





4

---

The impact of external graphical representations in different knowledge domains
Results and Discussion

Table 1 Summary of significant ANOVA results in study 1
Multimedia
Spatial
Coherence
Modality Redundancy
principle
contiguity
principle
principle
principle
Week 1
F (1, 69)
F (1, 67)
naa
F (1, 49)
na
Exp
1:
Transfer

8.74**

Exp 1 Total test

8.74**



Exp
2
Transfer

21.56*

Exp
2
Total
test
15.49*

Week 2
F (1, 48)
na
F (1,85)
na
F (1,49)
Cognitive
load

7.99*

aNot applicable. No experiments were set up to test this specific hypothesis during
the session of this week. * p < .01 ** p< .05.

Multimedia principle
Students studying learning materials with no external graphical representations
always attain higher mean post-test scores. Analysis of variance reveals (see Table
1) that these differences are significant for the second experiment with very large to
large effect sizes d = 1.12 for the transfer test and d = .95 for the total post-test
score. This suggests that the original CTML-hypothesis can not be confirmed for
any of the six experiments related to this principle. This might confirm that learners
have problems when studying from external graphical representations because of
inadequate experience with or knowledge of the iconic sign system used. These
results are consistent with Cox (1999), Lowe (2003) and Schnotz and Bannert
(2003), and Dobson (1999).

Spatial contiguity
In 2 of the three experiments that helped to test this hypothesis, students obtain
higher post-test scores when illustrations are not integrated into the text. These
differences are significant in the first experiment, reflecting a medium effect size of
d = 0.72. These results are not surprising in the light of the discussion of the first
principle.

Coherence principle
In 2 of the three experiments to test the hypothesis about the impact of the coherence
principle, students studying learning materials that consist of summaries with
external graphical representations perform better on post-test questions. But none of
these differences are significant. This questions the generic nature of the coherence
design principle when external graphical representations are used that are depictive
that are not based on a previously acquired iconic sign system.

Modality principle
In the three experiments, inconsistent results were observed in student performance.
The condition where animations were enriched with audio did not consistently lead
to higher performance as compared to conditions where the animations were
enriched with text. Moreover, none of the differences in performance were
significant. Also Tabbers, Martens and van Merriënboer (in pressb , 2001) could not
replicate the CTML-findings in their studies. Of particular importance is the fact that
in their study they also focused on less formal symbolic icon systems, used in the
knowledge domain “research methodologies in social sciences”.


5

---

The impact of external graphical representations in different knowledge domains
Redundancy principle
The post-test scores of students studying non-redundant learning materials; i.e.,
animation with narration and without additional text, were mostly higher. But the
differences are not significant.


Cognitive load
The impact on cognitive load was tested in relation to each of the former five
hypotheses. At a descriptive level, minor differences in cognitive load could be
observed, but most differences are not significant. Only in the experiments related to
the coherence principle, a significantly higher cognitive load was reported, reflecting
a medium effect size of d = 0.72. But this significant effect is not in line with the
CTML. Materials building on summaries with external graphical representations are
– contrary to the CTML assumptions – invoking higher cognitive load than
expanded versions with external graphical representations. This is in line with the
research of Tabbers and others (in pressb). They also report inconsistent results as to
the impact of external graphical representations on cognitive load.

The discussion in relation to the five experiments leads to rather consistent
conclusions. In relation to each experiment, the analysis results do not confirm or
reject the CTML-hypothesis. The practical implications of this finding are that
instructional designers are expected to pay additional attention to the nature of the
knowledge domain when developing external graphical representations. The results
suggest that this is especially the case when students are not acquainted with the
iconic sign system used to develop the representations.

The second research: the activation principle

Background and Hypotheses

The results of the first study revealed that the positive findings of Mayer could not
be replicated in the social sciences. It was hypothesized that in the social science
domain there is not an established conic symbol system shared by instructional
designers, experts and novices to develop external graphical representations. From
this hypothesis follows that students will not benefit from external graphical
representations and that the CTML design principles would not hold. In a second
study, this hypothesis was tested in an alternative way by adding a new design
principle to Mayer’s list: the activation principle. The central hypothesis for this
second study is stated as follows: learners that are actively engaged in designing and
developing their personal external graphical representations will perform better on
post tests than learners studying learning materials with external graphical
representations developed by others.
The activation principle can be readily linked to the CTML when focusing on the
active processing assumption that founds this theory (2003). The selection,
organisation and integration processes that foster the development of mental models
presuppose that the learners are active. Developing personal external graphical
representations is in the context of the present study put forward as a catalyst to
promote these internal cognitive processes. Also other authors have suggested ways
to foster the active processing (Burton, Moore, Magliaro, 1996; Seels, Berry,
Fullerton, Horn, 1996; Driscoll, 2004; Van Hout-Wolters, Simons, and Volet, 2000).
The activation principle can also be linked to the approach proposed by Novak
(1998) and the theoretical base he puts forward to ground the use of concept maps.
But it might also be hypothesized that asking learners to develop external graphical
representations might invoke cognitive load (Sweller,1988). Next to processing new
6

---

The impact of external graphical representations in different knowledge domains
complex knowledge, students are asked to develop external representations. This
extra process might hinder the construction of schemas in working memory and
confront learners even faster with the limited capacity of their working memory;
thus invoking a higher level of intrinsic cognitive load (Sweller, van Merriënboer &
Paas, 1998). In the literature, there is plenty of reference to the relationship between
cognitive load and prior knowledge (Kirschner, 2002). Several authors have made
suggestions to cope with cognitive load and to reduce extraneous cognitive load.
They put forward a variety of instructional design ideas and there is empirical
evidence to support the positive impact of such measures (Paas, 1993; Stark et al.
2002; Sweller, 1999; Sweller et al (1998); van Merrienboer et al, 2002):
• Worked examples: learners work with exemplary partially solved problems.
• Goal-free problems: this helps learners to redirect their attention from a means-
end strategy to a strategy in which they are invited to work forward from the
given information.
• Hierarchical approach: this implies that based on a task analysis, learners tackle
first the sub-components of the knowledge base before working on the more
complex knowledge elements.
• Emphasis-manipulation approach: In the context of a problem, students are
invited to tackle a specific sub-part of the problem.
• Completion strategy: students complete incomplete solutions. The next time
they are expected to work on a larger incomplete problem.
• Expert-like problem analysis: students follow a specific set of questions that
replicate the type of approach adopted by an expert.
In the present study, we take into account the potential negative impact on cognitive
load and add an experimental condition in which student are presented with pre-
worked external graphical representations. If the traditional CTML principles hold,
the presentation of pre-worked representations is expected to require lower levels of
mental effort from the learners. If the alternative hypothesis holds – the assumption
that representations has to build on familiar iconic symbol systems – presenting
learners with these pre-worked representations will still require more mental effort;
thus reflecting more cognitive load.

Research Method

Participants
The entire population of freshmen, studying educational sciences (N=217)
participated in this study (2003-2004). Participation was a formal part of the course
“Instructional Sciences”. Informed consent was obtained prior to experimentation.

Procedure
Students were randomly assigned to one of the four experimental conditions: (1)
studying printed learning materials with no external graphical representations, (2)
learning materials with ready-made external graphical representations, (3) learning
materials to which the students were invited to add their personal external graphical
representations, and (4) learning materials with pre-worked external graphical
representations.
In each condition, the students studied the learning materials that focussed at content
level on complex and new knowledge about theories and models of Instructional
Design (ID). Three ID-sub themes were presented to the students in one single two
hour session.
The three sub themes were presented in three subsequent experimental packages
consisting of (a) a prior knowledge test consisting of a retention and transfer test, (b)
a specific presentation of the learning materials to be studied, and (c) a post-test to
7

---

The impact of external graphical representations in different knowledge domains
test the mastery of the complex knowledge elaborated in the learning materials
(retention and transfer questions). Comparable to the first study, the students were
asked twice to estimate their perceived mental effort. Again no time limit was set to
study the materials and/or to fill out the tests. The answers to the retention and
transfer questions were scored by three independent researchers based on a scoring
checklist.

Results and Discussion

Table 2 presents a summary of the descriptive results. Analysis of variances, with
the pretest results as the co-variable were carried out to compare post test results and
measures of cognitive load. In case of statistically significant differences, Cohen’s d
was calculated to determine the effect size (Talheimer & Cook, 2002).

Table 3 gives an overview of the significant F-values when comparing the
performance and cognitive load measures in the different conditions. The third
experiment did not result in significant differences. This might be due to fatigue,
given the duration of the study. But the results in relation to the first two
experiments are clear. Students who are actively engaged in developing personal
external graphical representations score higher on the retention and transfer tests
(effect size d = .66 for the first experiment and d = 0.54 for the second experiment).
These results suggest that the activation principle leads to better performance, but
the results also confirm the CTML multimedia principle.

Significant differences are found in experiment two when comparing the
group receiving learning materials with external graphical representations and the
group developing personal external graphical representations, in favour of the latter
(medium effect size d = 0.63). These results suggest that the activation principle is
confirmed in this knowledge domain and when no formal iconic symbol system is
available. The result in the other experiments show a comparable trend, but the
differences are not significant.

Completing pre-worked external graphical representations versus
developing personal external graphical representations results in significant
differences with a large effect size (d = .86) in the second experiment. Students who
develop personal representations show superior performance. The same trend is seen
in the other experiments, but it is yet not significant.

These results are in line with the outcomes of other studies where activation
was built in the learning materials. But the way activation is introduced is
sometimes different. Lowe (2003), e.g., reports the positive impact of adding cues to
focus the learners’ attention to particular aspect of the representation, especially for
learners with limited prior knowledge. A more direct activation was studied by
Stern, Aprea and Webner (2003) who also asked their students to construct external
graphical representations. concluded that active construction of graphs by students
improves significantly knowledge transfer. Their results also underscore that the
passive study of available graphs leads to lower test performance.

The results in relation to cognitive load reveal significant differences.
Consistently, students that have to develop their personal external representations
report significantly lower levels of mental load. But there is no significant difference
between conditions that ask to develop completely new representations or to further
develop pre-worked representations. These results obviously underscore the
importance of the activation principle.
8

---

The impact of external graphical representations in different knowledge domains
Table 2. Mean scores and standard deviation for performance measures and cognitive load in the second study

Text without
Text with representations
Completing pre-worked
Developing
representations
representations
representations

Ma
SD Mb
SD Mc
SD Md
SD
Experiment 1
Retention test
3.61
0.95
3.69
1.16
4.16
1.23
4.04
1.14

Transfer te

st
7.91
3.41 9.22
3.45 8.44
3.36 9.96
3.48

Total test score
11.52
3.60
12.91
3.70
12.59
4.01
14.00
3.87
Experiment 2
Retention test
1.57
0.89
1.65
0.83
1.71
0.87
1.95
0.84

Transfer te

st
2.81
1.10 2.67
0.93 2.23
1.06 3.21
1.04

Total test score
4.38
1.59
4.32
1.40
3.94
1.55
5.17
1.29
Experiment 3
Retention test
1.12
0.51
1.15
0.58
1.10
0.59
1.07
0.69

Transfer te

st
1.91
1.92 2.18
1.79 1.89
1.69 1.56
1.57

Total test score
3.03
1.93
3.33
2.10
2.99
1.98
2.62
1.99












Ma
SD Mb
SD Mc
SD Md
SD

Cognitive load 1
5.41
1.47
4.81
1.60
4.67
1.60
4.17
1.42

Cognitive load 2
7.00
1.30
6.24
1.43
6.17
1.71
5.83
1.69

Total cognitive load
12.43
2.34
11.07
2.69
10.35
2.99
9.94
2.59
aN = 54. bN = 55. cN = 54. dN = 54.


9

---

The impact of external graphical representations in different knowledge domains
Table 3. Overview of significant F-values ANOVA results

Text without


Text without
Text without
Text with
Text with
Completing pre-
representations
representations
representations
representations
representations
worked
vs
vs
vs
vs
vs
representations
Text with
Completing pre-
Developing
Completing pre-worked
Developing
vs
representations
worked
representations
representations
representations
Developing

representations
representations

F (1, 108)
F (1, 107)
F (1, 107)
F (1, 108)
F (1, 108)
F (1, 107)
Experiment 1 Retention test

6.69**
4.45**
4.16**


Transfer test



9.61*





Total test score


11.89*



Experiment 2 Retention test


5.23**



Transfer
te
st
7.57*
5.33*
8.15* 23.43**

Total test score


7.97**

10.87**
19.90**


F (1, 108)
F (1, 107)
F (1, 107)
F (1, 108)
F (1, 108)
F (1, 107)

Cognitive load 1

6.26*
19.80*

4.94*


Cognitive load 2

7.93*
15.94*




Total cognitive load

9.23**
26.73*

4.87**

* p < .01.
** p< .05.

10

---

Document Outline

• ??
• ??
• ??
• ??
• ??
• ??
• ??
• ??
• ??
• ??
• ??
• ??
• ??
• ??
• ??
• ??
• ??
• ??
• ??
• ??
• ??
• ??
• ??
• ??

---