Information Design Journal 16(2), 107–124
© 2008 John Benjamins Publishing Company
José Marconi Bezerra de Souza and Mary Dyson
Are animated demonstrations the clearest and most comfortable
way to communicate on-screen instructions?
Keywords: static representation, dynamic representation,
instruction, animation, multimedia learning
This study uses preference measurements to compare
participants’ perceptions of nine versions of on-screen
instructions and aims to discover the ones users
consider to be the clearest and most comfortable to use.
Instructions are divided into 3 categories: composite,
synoptic and animated demonstrations. A group of
twenty-four adult participants ranked instructions, first
within the same category and then between categories.
Even though no data was gathered to suggest which
type of instruction will lead to better learning, the results
show that animations are perceived as being the clearest
and most comfortable to use. Users are likely to welcome
animations, particularly for learning highly dynamic and
non-intuitive software drawing tasks (i.e., using Bezier
tools).
Introduction
Ideally, on-screen instructions enable people to learn
how to use software on their own at the most convenient
time. But, what do users consider to be the clearest and
most comfortable way to demonstrate instructions? This
study explores this question by asking 24 participants
d o i : 10.1075/idj.16.2.03bez
with varied levels of expertise to compare 9 versions of
on-screen instructions. Six instructions use static and 3
use animated representations: they represent the basic
range of solutions available in the instructional market.
The emphasis here is on individual perceptions and not
on learning efficiency, so only preference measurements
were used. This study is part of PhD research (in prog-
ress) which aims to contribute to knowledge of how to
optimize on-screen instructions that are purely visual
(no sound effects or voice narrations). The instructions
tested are targeted at a general audience and show how
to use highly dynamic devices (i.e. Bezier tools) that are
available in a broad range of amateur and professional
software packages.
In addition to analysing learners’ views on instruc-
tions’ clarity and comfort of use, this study has meth-
odological and practical implications: (1) a ranking
technique was used to compare a range of equivalent
on-screen instructions which have not previously been
systematically compared; and (2) the list of verbalized
reasons for preferring certain instructions will help
practitioners to predict learners’ reactions to simi-
lar designs. The literature review focuses on learners’
preferences, but is complemented by a brief review of
research on performance.
107
José Marconi Bezerra de Souza and Mary Dyson  •  On-screen instructions
Motivation
Existing research has examined users’ views and attitudes
towards instructions in general terms (Despotakis et al.,
2007; Kim et al., 2007). Judging by the greater number of
studies, researchers have prioritized investigation of how
instructions affect learners’ performance efficiency rather
than preferences (Lowe, 2001). Furthermore, experi-
ments from within psychology and education appear
to be limited to comparing extreme forms of animated
instruction (self-running with limited or standard inter-
action devices) against traditional forms of static graph-
ics (sequences of images with arrows printed on paper).
Therefore, there is little evidence of how users compare
a variety of both animated and static on-screen instruc-
tions specifically in relation to clarity and comfort.
This lack of empirical data contrasts with the view
that it is designers’ responsibility to develop products
that are the best balance between performance and pref-
erence requirements (Lidwell et al., 2003). According to
Wright (2003), it is a “waste of time” to create informative
messages that could be understood, but that, for some
personal preference, users postpone reading them. So,
this study focuses on learners’ preferences because these
are likely to influence their selection of and intention to
use equivalent instructional formats. This is particularly
the case when learners exert choice and are not a captive
audience restrained by a teacher or trainer.
Background
According to a survey on how users search for infor-
mation as they are dealing with computing problems
(Schriver, 1997), novices tend to solve their problems by
asking someone for help. Novices think they got “better,
more direct, and more understandable answers” from
friends than from formal written instructions (Schriver,
1997, p. 383). A national survey (Barker & Gardiner, 2006)
108
idj 16(2), 2008, 107-124
found that 45% of United Kingdom’s adults acquired
computer skills by receiving informal assistance from
colleagues, friends and family. Indeed, experienced
friends can demonstrate instructions rather than just
describe them. Demonstration is often followed by imita-
tive practice and when, for some reason, the practice
fails, the same friend will help to diagnose which step has
been missed.
Unfortunately, an experienced and dedicated friend
is not always available. Nor is it reasonable to expect
them to provide visual memory aids, even though it
can be difficult to remember all of the instructional
details provided (Twidale, 1999). In such circumstances
structured visual demonstrations, such as on-screen
instructions, could work as (rather limited) alternatives
to real life demonstrations (Plaisant & Shneiderman,
2005). Research suggests that even cross-cultural differ-
ences might have an effect on the acceptability of such
on-screen demonstrations (Chavan, 2005). For example,
first-time users, non-literate people from Indian rural
areas, were shown in a recent study (Medhi, 2007)
to prefer to learn how to use computers by observ-
ing Bollywood-style instructional videos. In these
videos, the story line dramatized the relevance of using
computers and presented popular actors learning how
to use software tools. The audience felt able to overcome
their “fear and mistrust” of the new technology and
improve their performance (for example, the rate of
success in a job search task increased from 30 to 100%)
(Medhi, 2007, p. 14).
Furthermore, it is currently affordable and apparently
easy to record demonstrations of instructors teach-
ing how to use software. The traditional, low resolution
video-camera recording technique has been replaced
by screen recorder software, which creates interactive
animated versions of instructors’ real life demonstrations
direct from the computer screen (e.g., Camtasia Studio®,
Viewlet Builder®, Captivate®, Winkdebug®, etc.).
José Marconi Bezerra de Souza and Mary Dyson  •  On-screen instructions
Screen recorder software companies claim that
animated demonstrations are very effective for teaching
software skills. However, the gap between making such
demonstrations easier to create and making them effec-
tive learning tools has not been systematically explored.
This study does so, using instructions that were created
with a computer screen-recording system.
Research on learning performance
Some researchers are confident about the instructional
benefit of teaching software using animated demonstra-
tions. Their argument is that animation is a more direct
and engaging way to show novices how to use software
features instead of indirectly telling them what to do
(Baecker, 2002; Plaisant & Shneiderman, 2005). Accord-
ing to Höffler and Leutner (2007) the animation instruc-
tional promise has been fulfilled. They, for the first time
in this field, carried out a meta-analysis of 76 pair-wise
comparisons (embedded in 26 studies from 1973 to 2003)
between non-interactive animations and static picto-
rial instructions. They found a medium-sized overall
advantage of animations over static pictures. Even when
non-interactive, animation seems to be especially effec-
tive (large effect size) for acquiring procedural-motor
knowledge (Höffler & Leutner, 2007).
In spite of empirical evidence in favour of anima-
tions, the choice to use them to depict dynamic computer
instructions is being contested. For example, Tversky
and Morrison (2002) ascertain that animation (when
compared with static graphics) is not the most effective
way to learn about dynamic content. According to this
view, animations may be ineffective because they can be
difficult to perceive and learn from when the animation
is too fast or has too many moving parts. Another argu-
ment is that animations do not usually segment actions
into meaningful steps (Lozano & Tversky, 2003), since
learning is inhibited when continuous events depicted
idj 16(2), 2008, 107-124
in the animation could be better comprehended through
discrete segments. It is possible, as Tversky and Morrison
(2002) conclude, that interaction flexibility may facilitate
learning from animation by helping the user to overcome
these difficulties of perception and comprehension.
It has been suggested that when users are learning
how simple mechanical systems work (for example, a
pulley system) from static instructions, they will create
a “mental animation” (Hegarty, 1992). In order to do
this, learners have to infer the system’s dynamism from
the small changes that occur from picture to picture
(or in sections of a diagram). According to Mayer et al.
(2005), the cognitive effort required to create this “mental
animation” affects learning positively, making it, in some
circumstances, superior to learning from actual anima-
tion. The “static media hypothesis” (Mayer et al., 2005)
indicates that this superiority is partially due to the fact
that static graphics with text seem to encourage learners
to be more active and to self-explain the changes from
one segment to the next, and this “active processing”
leads to deeper learning. In contrast, animation with
spoken text (when compared to text-oriented instruc-
tions) may encourage “superficial processing” and, as a
consequence, poor retention of the instructional message
(Palmiter & Elkerton, 1993).
In addition, “clever schematizations”, such as arrows,
may be as effective as animation in communicating direc-
tion of motion and temporal sequences in mechanical
systems diagrams (Tversky & Morrison, 2002, p. 258). This
may be, as suggested by Krull & Sharp (2006) because
arrows are “visual verbs” that depict “physical forces on
physical objects” (p. 192). Similarly, other graphic devices
that suggest movement, such as motion lines and over-
lapping multiples, are commonly used in instructions
(Mijksenaar & Westendorp, 1999; Tufte, 2002) and in
comics (Kunzle, 1990; McCloud, 1993). A recent review of
motion representation in instructions has demonstrated
that, in order to depict distinct aspects of motion (i.e.,
109
José Marconi Bezerra de Souza and Mary Dyson  •  On-screen instructions
direction, trajectory and displacement), graphics devices
should be used in combination (Souza & Dyson, in press);
this review concludes that arrows are frequently used to
indicate direction, motion lines to indicate trajectory, and
overlapping multiples to indicate displacement of motion.
However, with few exceptions (Friedman & Stevenson,
1975; Lowe & Pramono, 2006) there is little data on their
comparative communicative effectiveness.
The nature of the task to be learned, research
suggests, is also playing an important role in the effec-
tiveness of instructional formats. For example, animated
demonstrations of software instructions may be effective
when the task:
− involves continuous changes (e.g., location, size,
shape, orientation, speed, etc.) which are not easily
inferred by the user (Harrison, 1995);
− requires using “direct manipulation interface” devices
(Shneiderman & Plaisant, 2004), such as resizing a
graphical shape by dragging its corners with a mouse.
In this circumstance, the cause-and-effect relation-
ship between user action and software response can
be easily inferred by means of demonstration (Water-
son & O’Malley, 1992);
− is clearly segmented into procedural steps (Harrison,
1995) and this segmentation is somehow visually
labeled (Ertelt et al., 2005); and
− is to be performed immediately after a training
session (Palmiter & Elkerton, 1991, 1993).
Research on learners’ preferences
Research also suggests that young students (i.e., fourth
graders) will feel motivated to learn from animation
(Rieber, 1991). This was recently confirmed by Kim et al.’s
(2007) research, in which fourth grade students found
continuous animations more attractive and motivat-
ing than static versions. This effect disappeared with
age, since experienced and mature students (i.e., sixth
110
idj 16(2), 2008, 107-124
grade) did not rate these non-interactive animations in
the same way. Mature students preferred the control-
lability of “self paced” static graphics. As the researchers
proposed, it may be that young students prefer anima-
tions because they do not want to be engaged in the
cognitive effort necessary to make sense of static graph-
ics. Young students, they suggest, might even be unable to
make inferences from static graphics in the ways mature
ones can. Thus “continuous animation” is unlikely to be
welcomed by mature students because they want to be in
control of presentation pace. Students’ feelings concern-
ing self paced animation were not investigated, but this
research does suggest that users’ prior knowledge, need
for controlling the pace of presentation, and willingness
to make inferences might all affect their preferences.
First-time users of narrated animated demonstrations
(Despotakis et al., 2007) find, among others, the follow-
ing advantages to their use:
− they require low cognitive effort to understand the
instructional message;
− they can be played on familiar interactive devices
(such as the one used by most media players), which
facilitates replaying sections that were not fully
comprehended;
− they make the learning process more pleasant and
entertaining.
However, first-time users also indicated (to Despotakis et
al., 2007) at least the following disadvantages of animated
demonstrations:
− the animations sometimes exceed the user’s ability
to assimilate the contents because they are presented
too fast;
− interactive devices do not provide sufficient informa-
tion for locating specific points within and between
animations;
− the animations might give the impression that learn-
ers had acquired the knowledge, when actually they
had not.
José Marconi Bezerra de Souza and Mary Dyson  •  On-screen instructions
The perception of effectiveness of instructions might also
depend on how comfortable the user feels in interacting
with them, some research has shown. This concept is in
harmony with the “interactivity principle” (Betrancourt,
2005) that establishes that people learn better when they
have control over the pace of the presentation. Indeed, as
some data suggests, some level of interaction flexibility,
such as that offered by the controls of traditional media
players, improves user’s performance while learning
procedural tasks of increasing difficulty (i.e., 4 types of
nautical knots in Schwan & Riempp, 2004). However,
providing a high level of interaction flexibility is not a
guarantee that users will focus on relevant aspects of the
animation’s instructional message, particularly when the
task to be learned is related to abstract and conceptual
knowledge (Lowe, 2004). People’s perception of comfort
in using animation or static instruction’s interaction
controls has also not been fully investigated.
Exploring the concept of animated demonstrations
Traditionally, animation is defined as a successive
presentation of multiple images that gives the illusion
of motion. The motion is represented realistically and
animation is self-running at a minimum rate of 12 images
(frames) per second, thus producing a smooth transition
between frames (this is attributed to the persistence of
vision phenomenon). This type of animation (Figure 7)
relies on the fact that the illusion of real-time motion can
only be sustained if each frame is displayed for no longer
than a fraction of second (Lowe, 2006).
Besides the traditional self-running format, this study
presents two alternatives forms of animation (Figure 8
and 9), which result from the user manually controlling
the succession of frames. This feature allows the user to
be in full control of the speed and direction of frames’
succession. This can create a rather jagged illusion of
motion (similar to flicker books). Hopefully, this loss in
idj 16(2), 2008, 107-124
realism does not compromise the instructional role of the
animation. This type of interaction control is consistent
with the proposal that realism per se does not make the
animations more effective (Tversky & Morrison, 2002).
On the contrary, in order to be perceived accurately,
some animations should be less realistic and be “slow
and clear enough for observers to perceive movements,
changes, and their timing, and to understand the changes
in relations between the parts and the sequence of events”
(Tversky & Morrison, 2002, p. 258). This study explores
the idea that the users’ comprehension of the instruction-
al message is intertwined with the way they interact with
the mechanisms that trigger the illusion of motion.
Exploring the concept of static instructions
Traditionally, static instructions are presented as a sequence
of pictorial segments that show “frozen” moments of an
action as it is being performed. Like designers of other
forms of visual narratives such as comics (McCloud, 1993),
instructional designers use their discretion to select the
“frozen” moments of a certain step that are important for
learning and discard those that are not.
These pictorial segments are organized as long
horizontal or vertical sequences of images, which are
sometimes called a “paragraph of pictographs” (Marcel
& Barnard, 1977) or a “composite image” (Twyman,
1985). The presentation of composite instructions on a
computer screen will be invariably limited to the size
of the computer screen window. Due to this size limita-
tion, the user will not be able to see the whole composite
instruction at once, only a few adjacent segments. The
inspection of sequential segments will therefore require
scrolling devices (such as “scroll bars”), although it has
also been suggested that when people are reading from
screens, excessive scrolling disorients them, and that the
invisibility of scrolled off texts makes them difficult to
remember (Lynch & Horton, 2002).
111
José Marconi Bezerra de Souza and Mary Dyson  •  On-screen instructions
Besides composite instructions, this study also
examines the “synoptic image” (Twyman, 1985) category.
Synoptic instructions represent a sequence of steps,
which were originally separate in time and space, inside
of a unified single image (Figures 4, 5 and 6). In this
case numbers are used as reading guides that specify the
order in which the instructions should be put into prac-
tice. Synoptic instructions are very good at showing the
sequence of procedures as a whole and because of their
compact nature do not require scrolling devices. Howev-
er, when compared with composite instructions, they
might be perceived as visually cluttered and too dense. It
seems that achieving the right balance between informa-
tion complexity and visual clarity is a difficult challenge
for designers of synoptic instructions, particularly when
such instructions are embedded with illustrated captions
and diverse representations of motion, as is the case in
this study. Indeed, this design issue should be further
investigated in the future.
Method
Materials
The instructions for this experiment teach how to use
graphic vector-based software Bezier tools (CorelDraw’s
“pen tool”) to construct a round arc. This task was
chosen because it requires the demonstration of direct
manipulation interface controls that are visually highly
dynamic (e.g. “drag and release” mouse manipulation);
and also because the accurate use of these controls seems
Table 1.  Characteristics that differentiate the instruction groups
Group Visual representations Interaction controls
1 Static composite instruction Horizontal scroll-bar
2 Static synoptic instruction None
3 Animated demonstration Buttons and interactive timeline
112
idj 16(2), 2008, 107-124
to defy people’s prior knowledge about the way in which
arcs are traditionally drawn (e.g., free-hand or using a
compass). Therefore, it is expected that these dynamic
and non-intuitive actions should act as encouragement
for the user to read, observe and reflect on the instruc-
tion details as carefully as possible. In addition, the
selected task is sufficiently complex, so that its explana-
tion may be segmented into four sequential steps. Finally,
it is important to research new ways to teach the use of
direct manipulation devices, such as Bezier tools, because
they are available in the majority of professional and
non-professional graphic software (including popular
word-processors, such as Microsoft Word).
Instruction groups
As outlined previously, three groups of instructions were
developed: 1 – Static composite instruction, 2 – Static
synoptic instruction, and 3 – Animated demonstration.
All instructions share the following characteristics:
– they are segmented into four steps (e.g., 1: click, 2:
drag, 3: release, and 4: double-click);
– each step is presented through captions (similar to
comic “speech bubbles”), which contain textual and
illustrated directions; and
– all captions are placed near the area where the action
will occur.
Besides their representational nature, the instructions
differ in terms of how the user interacts with them
(Table 1):
José Marconi Bezerra de Souza and Mary Dyson  •  On-screen instructions idj 16(2), 2008, 107-124

Table 2.  Differences between instructions in Group 1 – Static composite instruction

Static composite instruction Number of segments Representation of motion involved in the action
A 14 Three images show frozen segments of the moving objects.
B 8 Image with arrow represents each motion.
C 8 Image with overlapping multiples represents each motion.

– Due to its length, the composite instruction requires
a scroll-bar to move the sequence of images from left
to right and vice-versa.
– Due to its compact nature, the synoptic instruction
does not require any interaction control.
– Due to its dynamism, the animated demonstration
requires more sophisticated controls, such as buttons
and an interactive timeline, so the user can trigger the
mechanism for creating illusion of motion (this will
be described below).
Group 1 – Static composite instruction. As can be seen
in Table 2, three variations within this group were devel-
oped, labeled (A), (B) and (C). Basically, static composite
instructions vary in terms of how the motions involved
in each action are represented by static means:
– A depicts each single motion as a sequence of three
small frozen moments (Figure 1). Frozen moments are
pictures that show the segments between the start and
end point of a certain motion. A contains 14 segments;
– B is derived from A but uses arrows as graphic
devices to replace the three frozen moments of each
motion (Figure 2). B contains 8 segments; and
– C is similar to B, but uses overlapping multiples as a
graphic device (e.g., three segments of the motion are
represented as if they are overlapping, transparent
layers encapsulated into a single picture) (Figure 3).
C also contains 8 segments.

Figure 1.  Static composite instruction A in which motion Figure 2.  Static composite instruction B in which motion is
is represented by a sequence of frozen fragments of represented by arrows
movement

113
José Marconi Bezerra de Souza and Mary Dyson • On-screen instructions idj 16(2), 2008, 107-124

Figure 4. Static synoptic instruction D in which arrows are

used to represent motion

Figure 3. Static composite instruction C in which motion is

represented by overlapping multiple segments

Group 2 – Static synoptic instruction. As can be seen in

Table 3, three variations within this group were devel-
oped, and labeled as D, E and F. The variation within
this group is based on how graphic devices are used to
represent small segments of motion. Instruction D uses Figure 5. Static synoptic instruction E in which overlapping
arrows (Figure 4), E overlapping multiples (Figure 5), and multiples are used to represent motion
F motion lines (Figure 6).

Table 3. Differences between instructions in Group 2 – Static

synoptic instruction
Static synoptic Representation of motion
instruction involved in the action
D Arrows
E Overlapping multiples images
F Motion lines

Figure 6. Static synoptic instruction F in which motion lines

are used to represent motion

114
José Marconi Bezerra de Souza and Mary Dyson  •  On-screen instructions idj 16(2), 2008, 107-124

Table 4.  Differences between the instructions in Group 3 – animated demonstrations

Animated demonstration Representation of motion involved Number of frames Interaction controls
in the action
G Self-running smooth animation 136 “Play/pause” buttons.
(12 images/second).
H Each image is superimposed on   14 “Next” and “previous” buttons
the previous creating a jagged that control image
illusion of continuous motion. substitution.
I Each image is superimposed on   14 “Interactive timeline” with
the previous creating a jagged numbers from 1 to 14 that
illusion of continuous motion. control image substitution.

Group 3 – Animated demonstration. As can be seen in Smooth animations, such as (G), demand a large number
Table 4, three variations within this group were devel- of frames to depict motion realistically (136 frames, 12 per
oped, labeled as G, H and I. second). In contrast, user manipulated animation, such as
H and I, use only 14 images (including transitional frames
– G presents the traditional self-running animation
between the start and end point of the motion) to create
format, where the user has only to press the “play/
a similar, but less realistic, effect. H and I are considered
pause” button and observe the animation’s progres-
manually controlled animations because the user is in full
sion (Figure 7). If necessary, the user can monitor
control of the speed and direction of the frame’s succession.
the progression of the animation by looking at the
moving “play-head” in the timeline display;
– H creates the illusion of motion by the user control-
ling the progressive substitution of images (Figure 8).
The substitution mechanism is triggered by the user
pressing the “next” and “previous” buttons (e.g., analo-
gous to slide presentation); and
– I is similar to H, but the trigger mechanism is more
sophisticated. The user manipulates the progressive
substitution of images by dragging the mouse cursor
(pointer) over the interactive timeline region (Figure
9). The interactive timeline contains 14 “hot spot”
numbers corresponding to 14 frames. The speed of
the succession of frames is controlled by the manner
in which the timeline is manipulated (a distant
analogue to flicker books).
Figure 7.  Animated demonstration G in which the self-
running animation is controlled by basic “play/pause” buttons

115
José Marconi Bezerra de Souza and Mary Dyson  •  On-screen instructions
Figure 8.  Animated demonstration H in which “next” and
“previous” buttons control the substitution of each frame of
the animation
Figure 9.  Animated demonstration I in which the
“interactive timeline” (hot spot numbers from 1 to 14) controls
the substitution of each of the animations
116
idj 16(2), 2008, 107-124
Test design
The method adopted for collecting participants’ opinions
was ranking. Due to the large number of instruction
types to be ranked (9), this process was divided into two
parts: first, each participant ranked 3 types of instruc-
tions within the same group (such as: “Which composite
instruction do you consider the clearest: A, B or C?”).
Participants repeated this for each group, thus totaling
three judgements.
Twenty-four participants were divided into two equal
groups. One group of 12 participants ranked in terms
of clarity and the other comfort. This separation was
intended to help participants focus on one criterion;
otherwise, they might mix up the concepts. Participants
were allowed to form their own conception of “clarity”
and “comfort” as no definitions were provided.
Second, after each participant had ranked the 3 best
types of instructions within their 3 groups (in terms of
clarity or comfort), they were asked to compare these
highest-ranked instructions to one another. For example,
the participants who had ranked instructions in terms
of clarity were asked the following: “now that you have
ranked the best instruction of each group, which highest-
ranked instruction do you consider the clearest, interme-
diary and least clear?”
To minimize any bias that might be caused by all
participants comparing instruction groups in the same
order, the sequence order was varied (Table 5) so that
each participant compared individual instructions
between groups in a slightly different sequential order.
To gather qualitative data as well as quantitative,
two questions were asked immediately after partici-
pants finished all judgments. The group that had ranked
instructions in terms of clarity was asked the following
questions: (1) “Why do you think Instruction ‘X’ is clearer
than the others?”; (2) “Is there any group of instructions
that you think contains the best instructions in terms of
José Marconi Bezerra de Souza and Mary Dyson  •  On-screen instructions idj 16(2), 2008, 107-124

Table 5.  Order that group of instructions was presented to participants

Participants
1 2 3 4 5 6 7 8 9 10 11 12
Order of groups of instructions 123 132 213 231 312 321 123 132 213 231 312 321

Table 6.  Ranking results of synoptic instructions using clarity as the criterion. The
numbers in each cell represent the number of participants (out of total of 12) who
assigned the particular rank (column heading) to the instruction type (row heading).
Instruction type 1st 2nd 3rd
The clearest Intermediary The least clear
D 7 5 0
E 3 3 6
F 2 4 6

clarity? If so, why?”. The group that had ranked instruc-
tions in terms of comfort was asked: (1) “Why do you
think Instruction ‘X’ is more comfortable to use than the
others?”; (2) “Is there any group of instructions that you
think contains the best instructions in terms of comfort?
If so, why?”
Participants and basic procedure
The participants took part voluntarily, on a one-to-one
basis. Their levels of education varied: 3 undergradu-
ates, 13 graduates, 2 masters and 6 PhDs. All were adults
(ranging from 18 to 64 years old). The 2 groups of 12
participants rated their previous graphic software experi-
ence as follows: 7 had little or no experience, 9 were
intermediate and 8 were expert users. This mix of experi-
ence was considered helpful in ensuring that the results
were representative, and in keeping with the general aim
of optimizing on-screen instructions used by a diverse
audience. The test was administered by the researcher
and conducted in a quiet environment where partici-
pants would not be disturbed while making their judge-
ments. One laptop, with a 14 inch display (resolution of
1024 x 768 pixels) was used to carry out the test.
Results
The results were analysed using Kendall’s Coefficient of
Concordance, which establishes whether or not there is
significant agreement on the ranks assigned by different
participants. For practical reasons, this section will only
present ranking results with significant agreement. The
general results can be examined in Appendices 1 (Clarity)
and 2 (Comfort of use).
Clarity
In the group of synoptic instructions, participants were
in agreement as to which instruction was the clearest
(W=0.25, χ2=6.1, p<0.05). Participants ranked instruc-
tion D (Median Q2=1, quartile deviation QD=0.5), which
uses arrows to depict motion, as the clearest. Instruc-
tions E (Q2=2.5, QD=0.87) and F (Q2=2.5, QD=0.5) may
be considered equivalent to each other, since they were
similarly ranked (Table 6).
Participants were in agreement (W=0.36, χ2=8.66,
p<0.02) that the group of animated instructions (Q2=1,
QD=0.37) contains the clearest instructions. The groups
of synoptic (Q2=2, QD=0.87) and composite instructions

117
José Marconi Bezerra de Souza and Mary Dyson  •  On-screen instructions idj 16(2), 2008, 107-124

Table 7.  Overall ranking results using clarity as the criterion. Group of animated demonstrations
was ranked 9 times (out of total of 12) as the clearest of all instructions
Group of instructions 1st 2nd 3rd
The clearest Intermediary The least clear
Group 1 – Static composite instructions 0 6 6
Group 2 – Static synoptic instructions 3 4 5
Group 3 – Animated demonstrations 9 2 1

Table 8.  Overall ranking results using comfort of use as the criterion. Again, group of animated demonstrations was ranked 9
times (out of total of 12) as the most comfortable to use
Group of instructions 1st 2nd 3rd
The most comfortable to use Intermediary The least comfortable to use
Group 1 – Composite instruction 1 5 6
Group 2 – Synoptic instruction 2 5 5
Group 3 – Animated demonstrations 9 2 1

(Q2=2.5, QD=0.5) were ranked roughly equally, with the
synoptic instructions ranked slightly higher (3 partici-
pants considered them the clearest).
Comfort of use
In terms of comfort of use, there is no significant agree-
ment on judgements within each group of instructions.
In contrast, significant agreement was found when
the highest ranked instructions of each group were
compared to one another. Participants were in agree-
ment (W=0.34, χ2=8.16, p<0.02) that, again, the group of
animated demonstrations (Q2=1, QD=0.37) contained
the most comfortable to use instructions. The groups
of composite (Q2=2.5, QD=0.5) and synoptic (Q2=2,
QD=0.5) instructions can be considered as equivalent
(Table 8) since they were ranked similarly.
Participants’ observations
Immediately after the judgements, participants were
asked what they thought makes one specific instruction
or group of instructions clearer or more comfortable
to use than the others. These answers are summarized
below and classified according to instruction group.
Group 1 - Static composite instruction. The scroll-bar
was considered uncomfortable to use because it:
– is time consuming
– diverts attention from the instructions
– causes optical discomfort and a sense of disori-
entation when images are moved backwards and
forwards.
In terms of clarity it was pointed out that it:
– is difficult to make sense of long sequences of images
when the visibility is limited by one small window.

118
José Marconi Bezerra de Souza and Mary Dyson  •  On-screen instructions
Group 2 – Static synoptic instruction. Some of the
observations about clarity were conflicting. While some
participants found instructions very difficult to under-
stand, others expressed the opposite.
For example, synoptic instructions were considered:
– too dense in information
– too cluttered visually.
On the other hand, others considered synoptic instruc-
tions:
– straightforward and economic
– potentially easy to follow and to put into practice
– a good complement to animated instructions
because, contrary to animation, they are printable.
Group 3 – Animated demonstration. The observations
were mostly related to the way in which the interactive
controls either assisted or interfered with the process of
making sense of instructions:
Instruction G (self running animation) was considered:
– too slow
– lacking animation speed control
– lacking controls that stop the animation at specific
steps
Instruction H (next and previous buttons) was consid-
ered:
– very easy and comfortable to use.
Instruction I (interactive timeline) raised a series of
conflicting observations. For some the interactive control
was:
– flexible and intuitive
– made it easy to return to specific points of the anima-
tion in which there was a need for re-inspection
On the other hand, other participants observed that the:
– efficient use of the interactive timeline might require
some practice
idj 16(2), 2008, 107-124
– the interactive timeline requires fine manipulation of
the mouse cursor; otherwise it would “jump” unin-
tentionally to the next step.
– demand for careful control of the mouse diverted
their attention from the animation.
Discussion
This study demonstrates that animated instructions
with different levels of interaction flexibility are likely
to be welcomed by the majority of users. Animations
were perceived as clearer and more comfortable to use
than static versions, with a few reservations. In terms
of methodology, the ranking method was successful in
detecting differences between groups of animated and
static instructions, but less sensitive to minor differences
within groups (except the group of synoptic instructions
in which differences in terms of clarity were found). This
is unsurprising, since assessments that rely on subjective
opinion can reveal high levels of disagreement (Wright,
2003). Participants’ observations on why they think one
specific instruction is better than another have practi-
cal implications: (1) they might help designers to predict
learners’ reactions to similar designs; (2) they can be
converted into a check list of aspects that might influ-
ence learners to choose one instruction over another. To
determine the generality of these results, replication with
bigger samples should be carried out by others. Finally,
the variety of designs investigated can be used as a start-
ing point for further examination and/or inspiration for
better and more innovative design solutions.
The results encourage further investigation of how
different levels of interactive flexibility affect users’
performance while using animated demonstrations to
learn and apply on-screen instructions. This is important
because learners in general will welcome interaction
controls (as confirmed here); however, novices’ ability
to easily convert this flexibility in their favor is being
119
José Marconi Bezerra de Souza and Mary Dyson  •  On-screen instructions
debated. As Lowe (2006) suggests, when learners are
novices in an abstract and complex domain, “unfash-
ionable” non-interactive animation might be superior
to “free user control” versions. This contrasts with the
design guideline that interaction controls should always
be made available when the animation “is longer than 15
seconds” (Plaisant & Shneiderman, 2005), and also with
research (Schwan & Riempp, 2004) that has shown that
standard media player controls facilitate learning practi-
cal procedures (i.e., nautical knots). Indeed, this issue
deserves further investigation.
Earlier research has not fully explored the concept
that the illusion of motion contained in animations can
be created in different manners (Lowe, 2006; Tversky
& Morrison, 2002), and that these other forms could
be considered by users as beneficial for their learning
(Despotakis et al., 2007; Kim et al., 2007). Traditional
animations do not require the user to be aware or in
control of the slight changes in position and shape of
moving objects. In such cases, users could be consid-
ered to be passive observers with no deep involve-
ment in the process of creating the illusion of motion.
However, in this study manually controlled animations
allowed participants to feel in control of the succession
of frames. Potentially, this user interaction could make
the process of learning from animation more active and
better, in accordance with the “interactivity principle”
(Betrancourt, 2005). In this study, participants’ observa-
tions confirm that features allowing easy re-inspection
of instructional segments (contained in manually
controlled animations) make interaction with animation
more comfortable and clearer, even though, for some
participants, the effective use of these features might
require further practice. In this respect, it is possible to
speculate that with the increasing popularity of online
video-sharing services (e.g., Youtube, Dailymotion, and
Metacafe) more people will become familiar with basic
video interaction controls. Again, the extent to which
120
idj 16(2), 2008, 107-124
interaction flexibility might affect learning from dynamic
instruction (including amateurish “how-to” video shared
online) seems to be a valuable issue for further research.
Previous research on learner’s preference (Kim et
al., 2007) and performance (Mayer et al., 2005; Tversky
& Morrison, 2002) has demonstrated that the intrinsic
permanence of static instructions helps users to feel in
control of pace and order of presentation of instructions.
Indeed, Hegarty (1992) has used learning performance
measures and big samples to demonstrate that this
control will enable the user to create a “mental anima-
tion” of the task (Hegarty, 1992). However, preference
measures gathered here suggest that static instructions
(whether composite or synoptic) are considered the least
clear and comfortable to use of all instructions. One
could speculate that this is another interesting case in
which preference and performance might not be posi-
tively correlated (Nielsen & Levy, 1994). The possible
explanation for the observed disparity can be partially
explained by participants’ complaints that their inspec-
tion of static segments in small windows requires too
much scrolling (i.e., moving the whole sequence from
left to right and vice-versa). Furthermore, some partici-
pants observed that scrolling was time consuming and
caused optical discomfort and this made them confused.
This also coincides with the suggestion that excessive
scrolling might disorient users (Lynch & Horton, 2002).
This suggests that further research could explore novel
interactive devices, such as zooming user interface and
transition effects inspired by the “infinite canvas” concept
(McCloud, 2001), available in software such as “infinite-
canvas” and “graphywall”, which could make the inspec-
tion of long sequences of instructions in small windows
clearer and more comfortable.
Surprisingly, static synoptic instructions are considered
by participants as slightly clearer than composite images.
However, while some find the compact nature of synoptic
instruction to be straightforward and economic, others
José Marconi Bezerra de Souza and Mary Dyson  •  On-screen instructions
feel that this type of instruction is too cluttered and, there-
fore, potentially confusing. The effectiveness of synoptic
instructions might also depend on participants’ familiar-
ity with the task to be learnt, which would allow them to
recognize individual actions and interpret the sequence of
instructions correctly. This assumption might also explain
why participants have suggested that synoptic instructions
should be used as a complement to animated demonstra-
tion (as a printable job aid or visual summary). Indeed,
the integration of synoptic instructions with animation
deserves more attention from researchers.
Confirming results from a previous study (Lowe
& Pramono, 2006) arrows are considered the clearest
graphic device used to convey motion, particularly for
synoptic instructions. However, further research should
be carried out to test if arrows continue to be considered
as superior to motion lines and overlapping multiples
when, besides direction, other dimensions of motion
(such as displacement and trajectory) have to be clearly
communicated (Souza & Dyson, in press).
The non-intuitive and highly dynamic nature of the
task chosen for this study might have influenced partici-
pants’ opinion towards choosing animations as the best
instruction of all. Indeed, this result concurs with the
suggestion made by Waterson & O’Malley (1992) that
learning how to use direct manipulation devices (such as
Bezier tools) is not comfortably inferred by the observa-
tion of static instructions.
Literature suggests that the definition of the level
of dynamism embedded into the task seems to be the
decisive factor influencing the success of either static or
animated on-screen instructions. In other words, if the
dynamic content is easy to grasp through static graph-
ics, then animation is likely to be redundant (Mayer et
al., 2005). Conversely, if dynamism is difficult to imagine
and-or the learner is not familiarized with it, then anima-
tion is likely to be clarifying, if not essential (Waterson
& O’Malley, 1992). One might speculate that research
idj 16(2), 2008, 107-124
that demonstrates that static graphics are superior to
animation (Mayer et al., 2005) has actually confirmed
that some topics are easier to learn when they are broken
down into manageable discrete segments. This condition
for learning is in harmony with the permanent nature
of static graphics (in this case, printed) because they are
easily controllable and segmented into discrete moments.
In addition, Tversky and Morrison (2002) propose
that animations will not improve clarity when they are
uncontrollable and present gratuitous (although, realis-
tic) dynamism.
In summary, even though this study has shown that
animation is likely to be welcomed by learners, it does
not suggest that this type of representation is always
the best option for learning instructions. The nature of
the task (its level of dynamism, participant’s familiarity
with the task, its complexity, declarative or procedural
information, etc.) should be carefully examined before
making the decision to adopt animation or static graph-
ics as the instructional tool.
Conclusion
Although there are limitations of this study (such as its
small sample and absence of performance-based data),
the results suggests that highly dynamic and non-intu-
itive on-screen instructions are likely to be perceived
by people as clearer and more comfortable to use when
using animated demonstrations rather than static
versions. They also show that much work remains to be
done to optimize the effectiveness of animated demon-
strations. Further research could continue to explore
ways in which instructional formats could be used in
integration (e.g., synoptic with animation, composite
instruction with small animated segments, animations
with arrows, etc.). Indeed, research that has been explor-
ing the opposition between instructional formats has
already revealed that no single type of instruction will be
121
José Marconi Bezerra de Souza and Mary Dyson  •  On-screen instructions
the best for all learning scenarios. The systematic explo-
ration of the boundaries between static and dynamic
forms of instructions seems to be a promising and chal-
lenging field of research.
References
Baecker, R. (2002). Showing instead of telling. Paper presented at
the Proceedings of the 20th annual International Conference
on Computer Documentation, Toronto, Ontario, Canada.
Barker, R., & Gardiner, J. (Eds) (2006). Focus on the digital age: E-
education and e-skills. Dept. of Education and Skills & Dept
of Trade and Industry. (pp. 15–22): The Office for National
Statistics.
Betrancourt, M. (2005). The animation and interactivity principles
in multimedia learning. In R. E. Mayer (Ed.), The Cambridge
handbook of multimedia learning (pp. 287–296). Cambridge,
USA: Cambridge University Press.
Chavan, A. L. (2005). Another culture, another method. Paper pre-
sented at the Proceedings of the 11th International Confer-
ence on Human-Computer Interaction, 22–27 July, Las Vegas,
Nevada, USA.
Despotakis, T. C., Palaigeorgiou, G. E., & Tsoukalas, I. A. (2007).
Students’ attitudes towards animated demonstrations as
computer learning tools. Journal of Educational Technology &
Society, 10(1), 196–205.
Ertelt, A., Renkl, A., & Spada, H. (2005). Learning a new computer
application using on-screen videos. In A. Méndez-Vilas, B.
González-Pereira, J. M. González & J. A. M. González (Eds.),
Recent Research Developments in Learning Technologies (2005)
(Vol. II) pp. 779–784. Badajoz, Spain: FORMATEX.
Friedman, S. L., & Stevenson, M. B. (1975). Developmental
changes in the understanding of implied motion in two-
dimensional picture. Child Development, 46(3), 773-778.
Harrison, S. M. (1995). A comparison of still, animated, or non-
illustrated on-line help with written or spoken instructions in a
graphical user interface. Paper presented at the Conference
on Human Factors in Computing Systems, 7–11 May, Denver,
Colorado, USA.
Hegarty, M. (1992). Mental animation: Inferring motion from static
displays of mechanical systems. Journal of Experimental Psy-
chology: Learning, Memory, and Cognition, 18(5), 1084– 1102.
122
idj 16(2), 2008, 107-124
Höffler, T. N., & Leutner, D. (2007). Instructional animation versus
static pictures: A meta-analysis. Learning and Instruction,
17(6), 722-738.
Kim, S., Yoon, M., Whang, S. M., Tversky, B., & Morrison, J. B. (2007).
The effect of animation on comprehension and interest.
Journal of Computer Assisted Learning, 23(3), 260-270.
Krull, R., & Sharp, M. (2006). Visual verbs: Using arrows to depict
the direction of actions in procedural illustrations. Informa-
tion Design Journal, 14(3), 189-198.
Kunzle, D. (1990). The history of the comic strip (Vol. 1). Los Ange-
les, California, USA: University of California Press.
Lidwell, W., Holden, K., & Butler, J. (2003). Universal principles of
design (1st Edn). Gloucester, Massachusetts, USA: Rockport
Publishers, Inc.
Lowe, R. (2001). Beyond “eye-candy”: Improving learning with
animations. Paper presented at the Apple University Consor-
tium Conference, James Cook University.
Lowe, R. (2004). Interrogation of a dynamic visualization during
learning. Learning and Instruction, 3(14), 257–274.
Lowe, R. (2006). Educational animation: Who should call the
shots. Paper presented at the 23rd Annual Conference of the
Australasian Society for Computers in Learning in Tertiary
Education, 3–6 December, Sydney, Australia.
Lowe, R., & Pramono, H. (2006). Using graphics to support
comprehension of dynamic information in texts. Information
Design Journal, 14(1), 22–34.
Lozano, S. C., & Tversky, B. (2003). Demonstrations: Clues to
effective animated explanations, Workshop on Interactive
Graphics. London, UK.
Lynch, P. J., & Horton, S. (2002). Web style guide: basic design
principles for creating web sites (2nd Edn). New Haven and
London: Yale University Press.
Marcel, T., & Barnard, P. (1977). Paragraphs of pictographs: The use
of non-verbal instructions for equipment. Paper presented at
the Processing of Visible Language, Eindhoven, The Nether-
lands.
Mayer, R. E., Hegarty, M., Mayer, S., & Campbell, J. (2005). When
static media promote active learning: Annotated illustra-
tions versus narrated animations in multimedia instruction.
Journal of Experimental Psychology: Applied, 11(4), 256-265.
McCloud, S. (1993). Understanding comics: The invisible art. New
York, NY: Harper Collins.
McCloud, S. (2001). Reinventing comics. New York: Perennial.
José Marconi Bezerra de Souza and Mary Dyson  •  On-screen instructions idj 16(2), 2008, 107-124

Medhi, I. (2007). User-centered design for development. Interac- Waterson, P., & O’Malley, C. (1992). Using animated demonstra-
tions, 14(4), 12-14. tions to teach graphics skills. Paper presented at the HCI
Mijksenaar, P., & Westendorp, P. (1999). Open here: the art of ‘92 Conference on People and Computers VII, York, United
instructional design. London: Thames & Hudson. Kingdom.
Nielsen, J., & Levy, J. (1994). Measuring usability − preference vs. Wright, P. (2003). Criteria and ingredients for successful patient
performance. Communications of the ACM, 37(4), 66-75. information. Journal of Audivisual Media in Medicine, 26(1),
Palmiter, S., & Elkerton, J. (1991). An evaluation of animated 6–10.
demonstrations of learning computer-based tasks. SIGCHI
Conference on Human Factors in Computing Systems: Reach-
ing through technology, 257–263. About the authors
Palmiter, S., & Elkerton, J. (1993). Animated demonstrations for
José Marconi Bezerra de Souza is a PhD student in the Depart-
learning procedural computer-based tasks. Human-Comput-
ment of Typography & Graphic Communication at the Univer-
er Interaction, 8, 193-216.
sity of Reading, UK. He is supported by a doctoral grant from
Plaisant, C., & Shneiderman, B. (2005). Show Me! Guidelines for
CNPq–Brazil (National Council for Scientific and Technological
producing recorded demonstrations. Paper presented at the
Development).
2005 IEEE Symposium on Visual Languages and Human-
Centric Computing, 21–24 September. Mary Dyson is José’s PhD supervisor, teaches and researches in
Rieber, L. P. (1991). Animation, incidental learning, and continuing the Department of Typography & Graphic Communication into
motivation. Journal of Educational Psychology, 83(3), 318–328. reading from screen, and focuses on empirical work.
Schriver, K. A. (1997). Dynamics in document design (1st Edn). New
York, USA: Wiley.
Schwan, S., & Riempp, R. (2004). The cognitive benefits of inter- Contact
active videos: Learning to tie nautical knots. Learning and
José Marconi Bezerra de Souza
Instruction, 14(3), 293–305.
j.m.b.desouza@reading.ac.uk
Shneiderman, B., & Plaisant, C. (2004). Designing the user inter-
marconi2006@googlemail.com
face. London: Addison-Wesley Computing.
Souza, J. M. B., & Dyson, M. (in press). An illustrated review of how
Mary Dyson
motion is represented in static instructional graphics. Paper
m.c.dyson@reading.ac.uk
presented at the Visual literacies: Exploring critical issues, 3–5
July 2007, Oxford, UK.
Tufte, E. R. (2002). Visual explanations (5th edn). Cheshire, Con-
necticut, USA: Graphics Press.
Tversky, B., & Morrison, J. B. (2002). Animation: Can it facilitate?
International Journal of Human-Computer Studies, 57(4),
247-262.
Twidale, M. B. (1999). Over the shoulder learning: Supporting brief
informal learning embedded in the work context. Illinois, USA:
University of Illinois at Urbana-Champaign.
Twyman, M. (1985). Using pictorial language: A discussion of the
dimensions of the problem. In R. W. Thomas M. Duffy (Ed.),
Designing Usable Texts (pp. 245–312). Orlando, Florida, USA:
Academic Press, Inc.

123
José Marconi Bezerra de Souza and Mary Dyson  •  On-screen instructions idj 16(2), 2008, 107-124

Appendix 1.  Overall results of ranking related to clarity.

The ranking process was divided into two parts: first, each to one another. Median (Q2) and quartile deviation (QD)
participant ranked 3 types of instructions within the same indicate level of variability of ordinal data. Kendall’s coefficient
group (which composite instruction do you consider the of concordance establishes whether or not there is significant
clearest: A, B or C?). Participants repeated this for each group, agreement on the ranks assigned by different participants. Chi
thus totalling three judgements. Second, each participant Square (χ2 ) and corresponding probability (p value) indicate
compared the 3 highest-ranked of each group of instructions the strength of evidence for rejecting the null hypothesis.
Clarity ranks
Category Instructions 1st 2nd 3rd Median Quartile Kendall χ2 p value
(Q2) Deviation Coefficient of
(QD) Concordance (W)
Static composite  A 3 6 3 2 0.75 0.02 0.6 p>0.05
B 6 2 4 1.5 1
C 3 4 5 2 0.87      
Static synoptic D 7 5 0 1 0.5 0.25 6.1 p<0.05
E 3 3 6 2.5 0.87
F 2 4 6 2.5 0.5      
Animated G 7 4 1 1 0.5 0.21 5.1 p>0.05
demonstration H 0 7 5 2 0.5
I 5 1 6 2 1      
Best of each   Static composite 0 6 6 2.5 0.5 0.36 8.6 p<0.02
group Static synoptic 3 4 5 2 0.87
Animated 9 2 1 1 0.37    

Appendix 2. Overall results of ranking related to comfort of use.

Comfort of use ranks
Category Instructions 1st 2nd 3rd Median Quartile Kendall χ2 p value
(Q2) Deviation Coefficient of
(QD) Concordance (W)
Static composite  A 5 5 2 2 0.5 0.18 4.5 p>0.10
B 5 5 2 2 0.5
C 2 2 8 3 0.5      
Static synoptic D 5 6 1 2 0.5 0.09 2.1 p>0.10
E 3 3 6 2.5 0.5
F 4 3 5 2 1      
Animated G 5 5 2 2 0.5 0.13 3.1 p>0.10
demonstration H 5 4 3 2 0.87
I 2 3 7 3 0.5      
Best of each   Static composite 1 5 6 2.5 0.5 0.34 8.1 p<0.02
group Static synoptic 2 5 5 2 0.5
Animated 9 2 1 1 0.37    

124