Out PDF

INFORMATION TO U SERS
This manuscript has been reproduced from the microfilm master. UMI films
the text directly from the original or copy submitted. Thus, some thesis and
dissertation copies are in typewriter face, while others may be from any type of
computer printer.
The quality of th is reproduction is dependent upon th e quality of the

copy subm itted. Broken or indistinct print, colored or poor quality illustrations
and photographs, print bleedthrough, substandard margins, and improper
alignment can adversely affect reproduction.
In the unlikely event that the author did not send UMI a complete manuscript
and there are missing pages, these will be noted. Also, if unauthorized
copyright material had to be removed, a note will indicate the deletion.
Oversize materials (e.g., maps, drawings, charts) are reproduced by

sectioning the original, beginning at the upper left-hand comer and continuing
from left to right in equal sections with small overlaps.
Photographs included in the original manuscript have been reproduced

xerographically in this copy. Higher quality 6” x 9” black and white
photographic prints are available for any photographs or illustrations appearing
in this copy for an additional charge. Contact UMI directly to order.
Bell & Howell Information and Learning

300 North Zeeb Road, Ann Arbor, Ml 48106-1346 USA
800-521-0600
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without perm ission.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
INSTRUCTION IN MICROBURSTS: THE STUDY
OF MINIMALIST PRINCIPLES APPLIED
TO ONLINE HELP
by
Jean A. Pratt
A dissertation submitted in partial fulfillment

o f the requirements for the degree
of
DOCTOR OF PHILOSOPHY
in
Instructional Technology
M ajor Professor
A.J.,
)r J. Sleven4oulier D r.'tw via HaMey
Committee Member Committee Member
Dr. TimotKy A. Slocum r. Noelle E. Cockett

Committee Member Interim Dean of Graduate Studies
UTAH STATE UNIVERSITY

Logan, Utah
2000
R e p ro d u c e d with perm ission of the copyright owner. Further reproduction prohibited without permission.
UMI Number: 9999108
___ __©
UMI
UMI Microform 9999108
Copyright 2001 by Bell & Howell Information and Learning Company.
All rights reserved. This microform edition is protected against
unauthorized copying under Title 17, United States Code.
Bell & Howell Information and Learning Company

300 North Zeeb Road
P.O. Box 1346
Ann Arbor, Ml 48106-1346
Copyright © Jean A. Pratt 2000
AH Rights Reserved
ABSTRACT
Instruction in Microburst: The Study
o f Minimalist Principles Applied
to Online Help
by
Jean A. Pratt. Doctor o f Philosophy
Utah State University. 2000
Major Professor: Dr. M. David Merrill

Department: Instructional Technology
The purpose o f this research was to define ways in which minimalism, an instructional
methodology designed for print-based training materials, could be effectively modified and applied to
online help systems, which are computer-based and typically informational and referential. Five different
online help systems were designed, each implementing a slightly different implementation o f minimalism.
Together they were used to test the following hypotheses:
1. Users require (and prefer) less instructional support to complete and learn software
application procedural skills.
2. Users who are prompted to interact with the application spend longer amounts o f time learning
tasks, but. once learned, complete tasks faster without instructional support.
3. Users will be more effective and efficient at identifying and correcting errors if they have
direct access to error information.
4. Users using an online help system that is matched to their learning style will complete
software application procedural skills more quickly and with fewer errors than users using a non-matched
type o f online help.
First, there was a lack o f overall statistically significant differences among the treatment groups.
However, statistically significant differences were obtained for all groups between the pretest and posttests.
Gains in performance were retained after a one-week delay in assessment. All the implementations o f
minimalism were equivalently effective in teaching subjects how to complete software application
procedural tasks.
Second, time to complete tasks was not an indication o f successful performance. Instead, today’s
target audience possesses enough computer expertise to learn new software application skills at
approximately the same speed, regardless o f variances in the instruction.
Third, placement o f error information (either next to the error-prone step(s) or in a distant location)
was not beneficial in reducing the number o f errors subjects committed, increasing the percentage o f errors
they corrected, or reducing the amount o f time they spent recovering from errors.
Fourth, a subject’s preference for a verbal- or visual-learning style is not a predictor of student
performance given a specific implementation o f minimalist online instruction. There was no apparent
pattern o f correlations among time, completeness, and accuracy measures among the different
implementations o f minimalist online instruction.
(208 pages)
V
DEDICATION
This doctoral dissertation is dedicated first to my Lord and God,
whose mercy and grace endures forever,
and whose unending love supports me every single day!
I also dedicate this work to Brendan. Adelle. and Cassidy Pratt,
who oftentimes felt as though the dissertation was more important than they were,
but who were ALWAYS more important.
I love you all more than more can ever say.
vi
ACKNOWLEDGMENTS
Funding for this dissertation research was provided in large part by the Society for Technical
Communication. Near-matching funds were provided by Brendan J. Pratt. Additional funding was provided
by the Women and Gender Research Institute at Utah State University. Major in-kind support was provided
by the Department o f Business Information Systems and Education and the Department o f Instructional
Technology. This research would not have been possible without the above support.
I am indebted to my mentor and dissertation chairperson. Dr. M. David Merrill, who hired me at
the beginning o f my master’s program and then mentored me through the doctoral program. Dr. Merrill,
thank you for your belief in me and support o f me.
Special thanks are owed to Dr. Hans van der Meij. the principal co-theorist behind the principles
o f minimalism (which were implemented in this research). Hans, thanks so much for your patient
explanations, support, guidance, and virtual pats on the back.
Special thanks are also owed to Dr. Lloyd Bartholome. who made magic happen when it needed
to. Lloyd, you’re a great friend. I d o n ’t know how you did what you did— thanks from the heart.
Special thanks are not enough to express my gratitude to Kasey Child, who watched hundreds o f
hours o f digitized video and coded nearly one half the data for me. Kasey, you're the greatest!!! I literally
could not have done this without your tireless help. How can I ever repay you?
Special thanks are definitely due to my friend. Dr. Thomas Hardy, whose generous creation o f the
AVI Player saved me literally hundreds o f data-coding hours. Thanks, Thom!
Thanks are definitely due to Tim Slocum and Dan Robertson who were always available to work
through the statistical challenges with me. You guys are the best!! 1 had a blast in SPSS!
1 extend thanks also to my buddies, Rick Cline and Bob Mills, who traversed the path before me,
warned me o f the barriers to avoid, supported me when I encountered barriers o f my own. and were
extremely good sounding boards for new ideas. Thanks, guys, for being such great buds.
Jean A. Pratt
vii
CONTENTS
Page
ABSTRACT ................................................................................................................................................................ iii
DEDICATION ............................................................................................................................................................... v
ACKNOW LEDGMENTS ...........................................................................................................................................vi
LIST OF TABLES ..................................................................................................................................................... xi
LIST OF FIGURES ................................................................................................................................................ xiii
CHAPTER
I. INTRODUCTION ......................................................................................................................................... 1
Statement o f Purpose...................................................................................................................................I
R atio n ale.................................................................................................................................................... 1
II. REVIEW OF THE LITER A TU RE...............................................................................................................4
Focusing the Review of L iteratu re .......................................................................................................... 4
Locating Relevant Research A rtic les.............................................................................................. 4

Inclusion/Exclusion Criteria .............................................................................................................5
Categorizing Included Research A rticles....................................................................................... 5
Trends in Software Documentation ........................................................................................................ 6

M inim alism ................................................................................................................................................. 7
Problems with M inim alism ....................................................................................................................... 8
Review o f the Related Research S tu d ie s...............................................................................................11
Black et al.: Alternative Manuals ................................................................................................11

Lazonder: Minimal and Self-Study M an u a ls............................................................................... 12
Reznich: Minimalism and Computer A nxiety.............................................................................. 14
W arner: Test o f Two Minimalist Principles ................................................................................ 15
Vanderlinden et al.: M inimalist Tutorials......................................................................................16
Frese et al.: Guided E x p lo ratio n .................................................................................................... 17
Other. Mixed-Results S tu d ie s.........................................................................................................18
Minimalism Applied to Online H e lp ..................................................................................................... 18

Subject Preference for a Verbal o r Visual Learning S ty le ................................................................20
III. APPLICATION OF THE PRINCIPLES OF MINIMALISM TO ONLINE HELP .......................... 23
Common Characteristics o f AH Online Help S y stem s.......................................................................24
Tasks in Microsoft W o r d ................................................................................................................24

Style G uidelines............................................................................................................................... 24
W indow F o rm at................................................................................................................................24
viii
Table o f Contents .............................................................................................................................26

Title B a r s ........................................................................................................................................... 27
Action T e x t........................................................................................................................................ 29
Caution N o te s ....................................................................................................................................30
Unique Characteristics o f Each Online Help System ..........................................................................31
Skeletal Online Help System ........................................................................................................ 3 1

Elaborative Online Help S ystem .................................................................................................... 32
Inferential Online Help System ..................................................................................................... 36
Inferential-Distant Online Help S y stem ........................................................................................ 38
Inferential-Proximal Online Help S ystem .....................................................................................39
IV. STATEMENT OF THE METHODS AND PROCEDURES ............................................................... 43
Presession M aterials................................................................................................................................. 43
Letter o f Informed C onsent............................................................................................................. 43

Background Information Q uestionnaire.................................................. 43
Verbalizer-Visualizer Questionnaire ............................................................................................ 44
Kelly Services Skills T e stin g ..........................................................................................................44
Experimental M aterials............................................................................................................................44
Four Assessment Newsletters .........................................................................................................45

S o ftw are............................................................................................................................................. 45
MS Word 97 Tasks ...........................................................................................................................51
Instructions for Using Online H e lp ................................................................................................ 52
Online Help S y ste m ..........................................................................................................................52
Help Design Evaluation Questionnaire ........................................................................................ 52
Treatment D esig n ......................................................................................................................................53

Pilot S tu d ie s............................................................................................................................................... 54
External Review o f T reatm ents...............................................................................................................54
Population Sample Selection and A ssignm ent.....................................................................................54
Selection ............................................................................................................................................ 55
Stratified-Random Assignment to G ro u p s ....................................................................................55
Subject Attrition ............................................................................................................................... 56
Use o f Lab C onsultants............................................................................................................................56

Experimental P ro ced u res........................................................................................................................ 57
Presession A ctiv ities........................................................................................................................ 57

Session 1 Activities ..........................................................................................................................59
Session 2 Activities ........................................................................................................................ 61
Procedure for Coding D a ta ..................................................................................................................... 62
AVI P la y e r......................................................................................................................................... 62
Dissertation Data Sheet ...................................................................................................................63
Activity Tasks C h eck list..................................................................................................................63
Help Tasks C h eck list....................................................................................................................... 63
ix
Research A ssistan t...........................................................................................................................64
Procedure for Statistical Analyses ........................................................................................................65
V. RESU LTS........................................................................................................................................................67
Description o f Sample P o p u latio n ........................................................................................................67

Description o f Help Systems ............................................................................................................... 7 1
Description o f D a ta ..................................................................................................................................71
Hypothesis 1 Data Analysis ................................................................................................................... 75
Hypotheses 2 and 3 Data A n aly ses.......................................................................................................82
Hypothesis 4 Data Analysis ...................................................................................................................90
Hypothesis 5 Data Analysis ................................................................................................................... 97
Verbal x Online Help x Time ........................................................................................................97

Verbal x Online Help x A ccuracy............................................................................................. 100
Exploration o f Pretest Group Demographic D a ta ................................................................... 100
Hypothesis 6 Data Analysis ............................................................................................................... 101
VI SUMMARY. CONCLUSIONS. AND RECOM M ENDATIONS...................................................... 105
Summary o f the Study.......................................................................................................................... 105
Statement o f the Problem ............................................................................................................. 105

Statement o f the Purpose ............................................................................................................ 106
Research Procedures.................................................................................................................... 106
Data A nalysis................................................................................................................................ 107
Summary o f the Results ...................................................................................................................... 108
Finding 1 ........................................................................................................................................ 108

Finding 2 ........................................................................................................................................ 108
Finding 3 ........................................................................................................................................ 109
Finding 4 ........................................................................................................................................ 110
Conclusions .......................................................................................................................................... 110

D iscussion ............................................................................................................................................. 112
Overall Experimental D esign....................................................................................................... 112

Subjects’ Prerequisite Experience and Knowledge ................................................................113
Sampling E r r o r ...............................................................................................................................113
Time F a c to r.................................................................................................................................. 114
Experimental M ortality................................................................................................................. 115
Lack o f Emotional Support for S ubjects................................................................................... 115
Recommendations for Further R esearch............................................................................................ 116
R EFEREN C ES......................................................................................................................................................... 117
A PPEN D IC ES.......................................................................................................................................................... 123
Appendix A: Review o f Research on Minimalism ........................................................... 124
X
Appendix B: Kelly Temporary Services Training O u tlin e .............................................. 131

Appendix C: Letter oflnform ed C o n sen t........................................................................... 133
Appendix D: Background Information Q uestionnaire...................................................... 135
Appendix E: Verbalizer-Visualizer Q uestionnaire........................................................... 137
Appendix F: Session 1 Experimental M aterials................................................................ 139
Appendix G: Session 2 Experimental M aterials................................................................ 166
Appendix H: Online Help Instructions............................................................................... 176
Appendix I: Help Design Evaluation Q uestionnaire....................................................... 180
Appendix J: Expert Reviewer B iography.......................................................................... 189
VITA ......................................................................................................................................................................... 191
LIST OF TABLES
Fable Page
1. Reasons for Subject Attrition .................................................................................................................... 56
2. Gender and Age Description .................................................................................................................... 68
3. Experience with Computers, Word, and W ordPerfect........................................................................... 68
4. Platform and Ownership o f C om puter......................................................................................................69
5. Demographic Data by Treatment Group ................................................................................................. 69
6. Comparative Evaluation o f Online Help Systems ( a ) ............................................................................ 72
7. Comparative Evaluation o f Online Help Systems ( b ) ............................................................................ 72
8. Difference in Subject Perception o f Online Help System ......................................................................73
9. Descriptive Summary o f Time, Completeness, and Accuracy: ExpI .................................................73
10. Descriptive Summary o f Error Commission, Correction, and Recovery: Exp 1 ...............................74
11. Descriptive Summary o f Time. Completeness, and Accuracy: Exp2 ................................................ 75
12. Descriptive Summary o f Error Commission. Correction, and Recovery: Exp 2 ...............................76
13. Preferred Learning Style ............................................................................................................................ 76
14. Preferred Visual Learning Style by Treatment G ro u p ........................................................................... 77
15. Preferred Verbal Learning Style by Treatment Group .......................................................................... 77
16. Hypothesis 1 Interaction Assessment ....................................................................................................... 78
17. Effect o f Instructional Effectiveness per A ssessm ent.............................................................................79
18. Contribution to Instructional Effectiveness: Hypo I ..............................................................................79
19. Differences in Task Completeness and Accuracy per G roup................................................................ 80
20. Significance o f Learning over Testing T im e s..........................................................................................82
21. Differences in Performance over Testing Times: Hypo I ......................................................................83
22. Trend Analysis o f Performance M easures............................................................................................... 85
23. Hypotheses 2 and 3 Interaction A ssessm ent............................................................................................ 88
24. Effect o f Time on P erform ance................................................................................................................. 89
R e p ro d u c e d with perm ission of th e copyright owner. Further reproduction prohibited without perm ission.
xii
25. Significance o f Time to Complete Tasks over Testing Tim es.............................................................. 89
26. Differences in Time to Complete Tasks over Testing Times ...............................................................90
27. Trend Analysis o f Time to Complete T asks............................................................................................ 90
28. Hypothesis 4 Interaction Assessment .......................................................................................................92
29. Effect o f Error Information on Performance .......................................................................................... 93
30. Significance o f Error Information over Testing T im e s..........................................................................93
3 1. Differences in Use o f Error Information over Testing T im e s.............................................................. 94
32. Trend Analysis o f Error Inform ation........................................................................................................95
33. Interaction Effects: Verbal Learning by Experiment 1 ..........................................................................98
34. Demographic Data for Pretest Groups ............................................................................................. 101
35. Significant Differences in Pretest S u b jec ts.......................................................................................... 104
36. Interaction Effects: Visual Learning by Experiment 1 ...................................................................... 104
R e p ro d u c e d with perm ission of the copyright owner. Fu rther reproduction prohibited without permission.
xiii
LIST OF FIGURES
Figure Page
1. Example o f main window........................................................................................................................... 25
2. Example o f procedural w in d o w ................................................................................................................ 26
3. Table o f contents at Level 2 ......................................................................................................................27
4. Skeletal and elaborative table o f contents at Level 4 ..............................................................................28
5. Inferential table o f contents at Level 4 .................................................................................................... 28
6. Skeletal procedural to p ic ........................................................................................................................... 29
7. Format Text Box dialog box .....................................................................................................................30
8. Default caution note ..................................................................................................................................31
'•). Caution note in Inferential help system ...................................................................................................32
10. Definition 1 ..................................................................................................................................................33
11. Definition 2 ..................................................................................................................................................33
12. D efinitions ..................................................................................................................................................33
13. Elaborative help with links to graphical feedback.................................................................................. 34
14. "Click here ..." graphical feedback..........................................................................................................35
15. “What you should see" graphical feedback.............................................................................................35
16. Elaborative help with hint ......................................................................................................................... 36
17. Inferential procedural topic with h in ts ..................................................................................................... 37
18. Inferential procedural topic with "On your own" prom pt......................................................................38
19. Inferential “On your own" p o p u p .............................................................................................................38
20. Elaborative procedural topic .....................................................................................................................39
21. Inferential-distant online help system table of contents......................................................................... 40
22. "Troubleshooting” button in inferential-distant online help system .................................................... 40
23. General access to ‘Troubleshooting” topics via "Troubleshooting b u tto n .........................................41
24. Troubleshooting topic ................................................................................................................................41
xiv
25. Proximal error information in inferential-proximal online help ........................................................... 42
26. Pop-up error information in inferential-proximal online help system ................................................ 42
27. Pretest new sletter...........................................................................................................................................46
28. Treatment n ew sletter....................................................................................................................................47
29. Posttest n ew sletter........................................................................................................................................ 48
30. Delayed-posttest new sletter....................................................................................................................... 49
31. Pretest Task I ................................................................................................................................................ 50
32. Computer-group assignment s h e e t...........................................................................................................58
33. Partial introduction page ............................................................................................................................60
34. Components o f data on C D -R O M ..............................................................................................................61
35. Top part o f dissertation data s h e e t..............................................................................................................63
36. Bottom part o f dissertation data sheet ............................................................................................. 64
37. Partial Activity Tasks Checklist ................................................................................................................. 64
38. Partial Help Tasks Checklist .......................................................................................................................65
39. Partial coding sheet ...................................................................................................................................... 66
40. Percentage o f tasks completed: S k eletal...................................................................................................81
41. Percentage o f tasks completed: E laborative.......................................................................................... 81
42. Trend: Percentage o f tasks co m p leted ......................................................................................................84
43. Trend: Percentage o f accuracy in task completion .................................................................................86
44. Trend: Number o f errors co m m itted.......................................................................................................... 86
45. Trend: Percentage o f errors c o rre c te d .......................................................................................................87
46. Trend: Error recovery tim e .......................................................................................................................... 87
47. Trend: Time to complete tasks ..................................................................................................................91
48. Trend: Percentage o f errors corrected over ti m e .....................................................................................95
49. Trend: Number o f errors committed over tim e ........................................................................................ 96
50. Trend: Error recovery time over tim e ........................................................................................................96
51. Interaction o f online help, verbal learning, and time during p re te s t.................................................... 99
XV
52. Interaction o f online help, verbal learning,and accuracy during treatm ent........................................ 99
53. Years o f computer experience by g ro u p .............................................................................................. 102
54. Level o f experience with Microsoft W o rd ........................................................................................... 102
55. Level o f experience with Corel W ordP erfect...................................................................................... 103
CHAPTER I
INTRODUCTION
Statement o f Purpose
online help systems, which are computer-based and typically informational and referential. Two separate
experiments (using the same sample population) were conducted with the following general hypotheses:
2. Users who are prompted to interact with the application spend longer amounts o f time learning
tasks, but. once learned, complete tasks faster without instructional support.
software application procedural skills more quickly and with fewer errors than subjects using a nonmatched
Rationale
Most computer software is accompanied by documentation, much o f which is placed online. The
trend in software application development to place as much o f the documentation as possible online
(Boggan. Farkas, & Welinske, 1996) is causing a shift in the way we view the role o f technical
communicators (Hailey & Hailey. 1997) and online help (Pratt, 1998). Technical communicators are now
asked to develop training materials (Techwriters, 1997), and the quality o f software applications is judged
by the effectiveness o f the online help system to teach the user how to operate the software (Greer. 1984;
Rosenbaum. 1998). Effective, well-written, helpful documentation strongly influences the perceived
effectiveness o f the software by users (Greer, 1984; Rosenbaum, 1998) to the point that documentation can
"make or break a software application” (Greer, 1984, p. 7). Inadequate or poorly written online help results
in user frustration with and possible rejection o f the software (Kearsley, 1985b; Lieberman, 1987) and
possible loss o f business (Rosenbaum, 1998). Specific guidelines are needed to help technical
communicators implement principles o f instructional design in the development o f online help.
Technical communicators are starting to apply the principles o f minimalism, an instructional
design approach used to develop training manuals, to the development o f online help systems. Minimalism
was developed by John Carroll and his colleagues at the IBM W atson Research Center as a “ less is more”
approach to addressing problems users encountered when learning a new software application via self-
instruction. Carroll suggested that this “new” approach was in sharp contrast to the traditional instructional
systems design (ISD) approach proposed by Gagne’ (1985) and used by most instructional designers in the
development o f training materials. The main focus o f minimalism is to conduct iterative testing and analysis
(similar to the iterative nature o f formative evaluation in ISD) in order to provide users with “ju st enough"
information to guide their exploration and learning o f the software. The end product is similar to a well-
developed job aid (Horn. 1992).
The implementation o f minimalist principles to online help instruction could result in increased
learning o f the subject matter content and acceptance o f the software application; however, the application
of minimalism to training materials has met with mixed reviews, and its application to online help has not
been evaluated thoroughly. There are a few problems associated with adopting the minimalism principles
for use in the development o f online help.
First. Carroll intended these principles to be used only as general guidelines, not a prescriptive
"recipe" for “[cranking] out a training manual” (Carroll. 1990). Technical communicators differ in their
interpretation o f how to implement the minimalist principles.
The second problem is that minimalism is a model o f instruction applied to training manuals. It
was not designed for use in online help systems, but its “less is more” approach is directly applicable to the
limited availability o f screen space in online help. Microsoft and Lotus are shifting their help systems to
minimalism (Boggan et al., 1996). Some o f the leading authors in online help development (Boggan et al.,
1996; Brockman, 1990b; Farkas, 1993; Hackos & Stevens, 1997) include minimalism in their online help
guidelines. Other technical communicators propose a variety o f ways to apply the minimalism principles to
online help (Techwriters. 1997). Logical heuristics seem to be the main guidelines for the transition o f
minimalism from print to an online media.
This brings us to the third, related problem. The limited empirical research conducted on
minimalism used training manuals, not online help. These studies resulted in contradictory findings. Carroll
(1984. 1990) reported case studies indicating support for minimalism. Carroll (1990) and Van der Meij and
Carroll (1995) cited a number o f studies they and others have conducted which have "repeatedly proven the
effectiveness o f minimalist instruction” (p. 243). However, Jansen (1994) discovered that minimalism is not
an etl'ective approach for all types o f learners. The findings from the Nowaczyk and James (1993) study
supported one minimalist principle but refuted another. No empirical support for the application of
minimalist principles to online help was located.
Research on the application o f the principles o f minimalism to online help was needed to assist
technical communicators in the design and development o f effective online help instruction. The results o f
this research can be used by technical communicators to develop online help systems: it can also be used by
instructional designers to develop online instruction. This research supports the minimalist approach to
developing online help for subjects who have an intermediate to advanced skill level and who are learning
and performing intermediate- to advanced-level skills. This research does not support the implementation o f
one variation o f minimalist instruction over another. That is, given a solid base o f minimalist instruction,
subjects can leam equivalently despite slight variations in the instruction. These findings can be extended to
the development o f any type o f online instruction: provide users with less information and require them to
interact with the content more in order to increase learning.
R e p ro d u c e d with permission of the copyright owner. Further reproduction prohibited without perm ission.
4
CHAPTER II
REVIEW OF THE LITERATURE
The purpose o f this literature review was to provide a foundation for applying the principles o f
minimalism to online help and design a study to test that application. In a deductive manner, the review of
literature begins with the procedures used to locate, include/exclude, and categorize the research for ease o f
use in this study. It then examines a trend in technical communication that is leading practitioners toward a
researched use o f minimalism, then follows with an overview o f and problems with minimalism. It
concludes with an analysis o f the related research on minimalism and suggestions for how this research may
answer questions and/or fill in the gaps identified through previous research.
Focusing the Review o f Literature
Since this is a new area o f research, several related fields o f research had to be investigated for
their applicability to the current research study.
Locating Relevant Research Articles
The two primary sources of data for this literature review were the listserves and computer-assisted
searches o f relevant research databases. Utah State University’s online public access catalog (OPAC) was
also used to locate books.
The archives for two listserves, Techwriters (1997) and WinHelp (1997), were searched for
discussions o f online help and minimalism. Once a discussion thread was located, it was followed until
dropped. Some o f the same people participated in the discussions on both listserves. Although not empirical
data, discussions on the listserves provide timely input on what practitioners (the target audience) are doing
and the challenges they face.
The empirical data for determining what to research and how to conduct the research study were
obtained by searching relevant databases back to 1980. (Carroll’s seminal paper on minimalism was
published in 1 9 8 4 :1 searched back to 1980 to guarantee that no other studies had been conducted prior to
that.) Databases searched included ERIC, USU Libraries— Electronic Journals, Dissertation Abstracts,
5
Education Abstract, Social Science, and Applied Science and Technology Abstracts. The following
descriptors svere used alone and in combination to expand and narrow the searches: minimalist, minimalism,
online help, online help, ICAl, expert system, intelligent tutoring, embedded training, coaching, feedback,
tutoring, multimedia, computer-based instruction, computer-based training, adaptive help, adult learning,
and adult education.
Inclusion/Exclusion Criteria
A wide search was used in order to identify studies in related areas that could be used as a guide to
develop the current study since no studies with direct application were located. Studies were included if
they described the use o f instructional features (e.g., examples and feedback via graphics or text) that could
be used with minimalist instruction. Studies were excluded if they (a) referenced library or corporate
information databases without applications to help or training systems, (b) provided general descriptive
information without any specific reference to instructional design features, or (c) required advanced
programming for the implementation o f the instructional design features (e.g.. "intelligent" agents).
Exclusion o f articles based on the above criteria did not bias the study: rather, it focused the study for more
direct application to the target audience: online help developers and instructional designers.
A separate, smaller search was conducted on the Verbalizer-Visualizer Questionnaire (a
measurement tool used in this research). All articles were included (although only three were even in related
fields o f study) because it was the test— not the subject matter content— that was o f importance. (A brief
summary o f these studies is included at the end o f the Review o f Literature.)
Categorizing Included Research Articles
Carroll’s (1990), Van der Meij and Carroll’s (1995), and Black, Carroll, and McGuigan’s (1987)
works were the principal sources for setting up the study and creating the five different online help systems
used for the treatments. However, since their works are based on manuals and simulated software, other
included articles were used to aid in the application o f the principles o f minimalism to online help. Such
articles were comprised mostly o f computer-based/assisted instruction/training.
The remaining theoretical and empirical articles were used to identify the current trend in software
6
documentation, previous research conducted on the principles o f minimalism and the related principles, and
the need for this research.
Trends in Software Documentation
One trend in software application development is to place as much o f the documentation as
possible online (Boggan et al.. 1996). Documentation could include tutorials, reference and training
manuals, links to information databases, web pages, and online help systems. Brockman (1986) suggested
that "manual-less" software would be in common use by the year 2000 either because documentation would
be put "online" or software would become increasingly intuitive. However. Kearsley (1985a. 1985b) points
out that what is intuitive to one may not be intuitive to another and that there are very few software
applications that are so simple and intuitive to use that they do not require some type o f training or
embedded documentation.
Two ramifications o f this trend seem to be emerging. First, technical communicators are taking on
new roles such as trainers and webmasters (Techwriters, 1997). Hailey and Hailey (1997) suggest that
industry demands force technical communicators to “ lead instructional technology into the workplace" (p.
3 1). a role typically assumed by instructional designers. Technical communicators are also becoming
information suppliers by providing access to information and the ability to cooperatively create more
information. Second, there is a blurring in the purposes and uses o f the different forms o f online
documentation. Pratt (1998) suggested that online help could be used to teach new users how to use the
software, an instructional role typically reserved for tutorials and training manuals.
Several leaders in the field o f technical communication have assisted technical communicators in
their transition from producers o f print documents to producers of online documents. Their publications
provide general and specific guidelines for producing online documents as well as specific emphases on the
technical aspects (Boggan et al., 1996), standards (Hackos & Stevens, 1997). evaluation (Duffy. Palmer. &
Mehlenbacher, 1992). and conversion from paper media to digital (Brockman, 1986. 1990b; Horton, 1990,
1994). An implicit assumption in these publications (with the exception o f Duffy et al., 1992) is that the
same guidelines can be applied across the various forms o f online documentation (both informational and
7
instructional). This assumption is indicated by the types o f online documentation the authors list as the
target o f their guidelines (Brockman, 1990b; Hackos & Stevens, 1997; Horton, 1994) and by the use o f a
training methodology (minimalism) to guide the development o f referential online documents (Boggan et al.
1996: Brockman, 1990b; Hackos & Stevens, 1997). But what is minimalism?
Minimalism
Minimalism was first introduced in the early 1980s (Carroll, 1984), but there has been a resurgence
in the discussion o f minimalism and its application to documentation. Past discussion on the listserves
(Techwriters. 1997; WinHelp, 1997) leads one to believe that minimalism is new to many, very familiar to a
few. and not well defined, understood or applied by the majority. This is despite the fact that Microsoft has
adopted it as a standard, Mary Deaton used it in a jo b description. JoAnn Hackos presented a one-day
Society o f Technical Communication seminar on it and Stephanie Rosenbaum conducts semiannual, one-
day seminars on it at University' o f California, Santa Cruz. What is minimalism? Why is it important to
technical communication research? How can we apply its principles to online help systems? According to
Carroll (1990):
The key idea in the minimalist approach is to present the smallest possible obstacle to
learners" efforts, to accommodate, even to exploit, the learning strategies that cause
problems for learners using systematic instructional materials. The goal is to let the
learner get more out o f the training experience by providing less overt training structure.
(pp. 77-78)
Minimalism was developed by Carroll and his colleagues at the IBM Watson Research Center as a
"less is more" approach to addressing problems users encountered when learning a new software
application via self-instruction. This "new"’ approach was in contrast to the traditional instructional systems
design (ISD) approach proposed by Gagne’ (1985) and used by m ost instructional designers in the
development o f training materials. Carroll’s impression o f instruction designed using the ISD process is that
it "is designed with little consideration o f the learners and no consideration for the context within which
learning will occur” (Carroll, 1990, p. 74). However, a comparison o f minimalism and the ISD process
8
rev eals they both utilize an iterative design based on analysis o f the user needs, learning environment, and
content/tasks to be learned (Horn, 1992). Carroll’s main complaint about systematic design is the tendency
for instructional designers using this approach to break down tasks into minute, detailed components
preceded and followed by lengthy descriptions and conceptual explanations (Carlson, 1992; Hallgren.
1992). Instead, Carroll advocates providing users with just enough information to learn the application as
they explore its interface to solve their own problem tasks.
The number and focus o f the specific principles o f minimalism have changed over the years. The
most recent listing by Van der Meij and Carroll (1995, 1998) condenses minimalism into four principles
and 11 "heuristics";
Choose an action-oriented approach.
Provide an immediate opportunity to act.
Encourage and support exploration and innovation.
Respect the integrity o f the user's activity.
Anchor the tool in the task domain.
Select or design instructional activities that are real tasks.
The components o f the instruction should reflect the task structure.
Support error recognition and recovery.
Prevent mistakes whenever possible.
Provide error information when actions are error prone or when correction is difficult.
Provide error information that supports detection, diagnosis and recovery.
Provide on-the-spot error information.
Support reading to do, study, and locate.
Be brief; don’t spell out everything (formerly “slash the verbiage”).
Provide closure for chapters.
Problems with Minimalism
The main problem with minimalism is that there are no specific application guidelines for how to
9
implement the principles and heuristics. Hallgren (1992) criticized Carroll’s book on minimalism. The
Xurnherg Funnel, for "providing no direct help for me to perform my work” (p. 16). Indeed, Carroll was
adamantly against prescribing a "recipe” for creating instruction:
There is no deductive theory o f minimalist instruction; that is, given a set o f minimalist
principles, we cannot just crank out a training manual. Design never works in this way.
What is important is the key idea o f minimizing the extent to which instructional materials
obstruct learning and o f refocusing the designing o f training materials on the goal of
supporting learner-directed activity and accomplishment. (Carroll, 1990, p. 91)
Without specific "how to” guidelines, practitioners are left on their own to implement the
minimalist principles. Draper (1996) taught students how to design a "graded series” o f training manuals
using the minimalist principles. Farkas (1993) suggested that balloon help, especially with a "layering” or
"filtering" option for providing different levels o f balloon help for different needs, is an implementation o f
minimalism. Hackos and Stevens (1997) highlighted five "tips" based on the 11 heuristics. Brockman
( 1990b) included adopting the minimalist philosophy (using five tips and four case studies) as part o f the
tirst step in the documentation process. Boggan et al. (1996) integrated what they believe are the applicable
minimalist principals throughout their guidelines. Practitioners on the Techwriters (1997) ListServe suggest
a variety of implementation guidelines, some of which are contrary to minimalism.
Another problem with minimalism is that it is targeted toward one type o f learner and may not
prov ide the flexibility necessary to reach a spectrum o f learners with varying degrees o f experience.
Minimalism is good for the holistic, independent explorer type o f learner using a simple, noncritical
application, but is not applicable for all types o f learners in all types o f situations (Carlson. 1992; Horn,
1992; Jansen. 1994; Reddish, 1988). Jansen (1994) found that “manual-oriented subjects were often
confused by the deliberate incompleteness o f the instructions and by the absence o f introductory sections.
They missed the step-by-step approach and sometimes felt abandoned” (p. 236). In fact. Carroll reported
that some of his experimental subjects complained that they wanted the structure o f a self-instruction
manual (Carroll, 1990; Reddish, 1988). “Three participants became frustrated enough to ask to be excused
from the experiment and were replaced” (Carroll, 1990, p. 167). In another study. Carroll had to provide his
10
experimental subjects with a typical ISD-type o f reference manual to complete the experim ent (Horn,
1992). These findings are supported by other studies o f guided exploration hypermedia that indicate that
users lacking sufficient prerequisite knowledge were frustrated because they could not decide what
information was needed or where to find it (Cline. 1991; Lawless & Brown, 1997).
Brockman (1990a. 1990b) identified other problems with the minimalist approach (which I will
simply summarize here in paragraph format). Minimalism is based on guided exploration, which is a self-
discovery type o f learning. Self-discovery learning is less predictable and may result in shallow learning.
Minimalism assumes a motivated audience that can choose effective and attainable goals. Instructional
design research indicates that this is an erroneous assumption. Users are not motivated to explore the
interface. They need specific directions to complete a task and get back to work. Furthermore, they lack the
prerequisite knowledge about the system and their own learning to be able to set effective learning goals.
There is also a potential for documentation to become cryptic as technical communicators "slash the
verbiage" instead o f rewriting the content. Moreover, technical communicators are uncomfortable slashing
the verbiage.
Rosenbaum's (1998) survey o f technical communicators who had taken a minimalist course
revealed that practitioners were uncomfortable with the minimalist principle to specify information
incompletely. This stemmed in large part from the fact that most o f them had neither the time nor the budget
to implement iterative testing so had to rely on their own heuristics for what information users needed. They
also had difficulty deciding what kind and size o f tasks to modularize, especially for com plex applications.
Some o f them "implemented” minimalism into one medium (e.g., the training manual) by shifting all the
wordy documentation to another medium (e.g., online help).
In another study. Draper (1996) identified conflicting purposes o f minimalism: task completion
(job aid) and learning (tutorial). Slashing the verbiage to create a job aid necessarily means eliminating
explanations and other details that learners need in a tutorial. The lack o f detail could prove disastrous to
users who lack the prerequisite knowledge. Another principle that conflicts with “slash the verbiage” is
"provide on-the-spot error information” which, by definition, means providing redundant verbiage because
no one can anticipate where and when the user may make an error. Users spend between 25% and 50% o f
II
the time correcting errors (Van der Meij & Carroll, 1995) and need to have the error recovery information
immediately accessible. However, providing access to additional details through hypertext links in online
help would resolve these conflicts.
Review o f the Related Research Studies
This review o f the research is comprised o f those studies that were published. A detailed table o f
the studies and their findings is located in Appendix A. Other research has been conducted in the
commercial sector but has either not been reported or has been reported anecdotally.
Black et al.: Alternative Manuals
Black et al. ( 1987) designed four different manuals with varying amounts o f verbiage in each and
then compared the time it took subjects to complete the specified tasks. The manuals they created were
skeletal (brief, task-based commands), elaborative (commands with explanations, feedback and
summaries), inferential (partial commands with "Try this on your own" prompts for self exploration), and
rehearsal (commands with built-in practice).
The current study was a modified replication o f the Black et al. study. It differed from the original
study in the following ways: (a) the principles o f minimalism were applied to online help rather than to
training manuals, (b) the rehearsal implementation was dropped, and (c) the content was more advanced.
Online help was the more appropriate medium since so much documentation is being converted that way.
The rehearsal manual was not an appropriate implementation o f online help. The use o f intermediate and
advanced tasks was more appropriate given today's higher level o f experience with word processing
applications.
The tasks the subjects were required to perform in the Black et al. (1987) study were easy to
perform. The simple learning test was to insert a letter or word; the command sequence task was to center or
move text: the realistic task was to create or revise a letter or memo. The authors admitted that they did not
know how well the inferential manual would teach advanced tasks. The tasks the subjects were required to
perform in the current study were rated as intermediate- to advanced-level tasks by the Kelly Temporary
12
Sen/ices training outline (Appendix B). They included creating and manipulating text boxes and tables,
inserting and formatting text and graphics, and creating a mail merge document and database.
Black et al. (1987) concluded that the inferential manual, emphasizing guided exploration, was
superior to the other types o f manuals. The results o f this study indicate that subjects using any o f the three
implementations o f minimalism to online help were able to achieve equivalent completion and accuracy
scores for the given tasks. Black et al. also suggested that the study o f alternative manuals for teaching
advanced topics was an area for future research. The results o f this study indicate that minimalism is an
effective approach to teaching intermediate and advanced tasks.
Lazonder: Minimal and Self-Studv Manuals
Lazonder (1994) conducted a series o f four tests on the principles o f minimalism. His study is
especially relevant to the current study because he conducted his studies as part o f his dissertation work
under the guidance o f Van der Meij who. with Carroll, refined and published a more definitive set o f
minimalist principles (Van der Meij & Carroll. 1995. 1998).
Lazonder's first experiment was a replication o f Carroll. Smith-Kerker. Ford, and Mazur-Rimetz's
(1987) research comparing minimal and traditional training manuals. Overall, his findings supported
minimalism as an effective instructional design strategy for software applications, but only during the
training phase o f the experiment. In fact, the mean task-completion score o f subjects using traditional
manuals was higher than the mean task-completion scores o f subjects using the minimal manual during the
retention and transfer phases o f the experiment, but not significantly so. Lazonder also found that novice
users required more time and were less successful, less efficient, less capable o f recovering from errors, and
less efficient at transferring knowledge than more experienced users.
Lazonder's second experiment was designed to assess the effectiveness o f error information for
detecting, diagnosing, and correcting errors. He found no statistically significant differences, so he followed
up this study with a qualitative study designed to see if new users made enough errors in learning the
software to warrant further study o f error information. His findings provided support for continued study o f
the integration o f error information in instructional materials: 25% o f users' actions were in error.
13
The fourth experiment Lazonder conducted was a modified replication o f his second (error
information). He removed error-prone tasks, provided more distinctive headings, faded error information
out over the course o f the manual and placed error information next to steps that had some sort o f visual cue
on the screen. Thirty-five percent o f the minimal manual was error information. He also eliminated the
delayed testing situation and relied only on the training, practice, and immediate performance scores.
Although there were some statistically significant differences in speed and quantity o f detected and
corrected errors during the training and practice phases, there were no statistically significant differences
between manual types during the test. Lazonder’s concluded that overall "subjects from both conditions
were equally skilled at detecting and correcting their own errors” (p. 111).
Lazonder chose to focus his study on novice users performing very basic tasks. Given the prolific
use o f computers and word processing applications, the current study was comprised o f users who already
had a working knowledge o f a word processing application and were interested in learning additional skills
to become more productive in their work. They represent a more typical target audience. The tasks chosen
to meet the needs of this target audience were more complex (e.g.. creating a data source and performing a
mail merge) than those used by Carroll (1990) and Lazonder (1994). The ability o f minimalist online help
systems to teach subjects complex tasks to subjects with intermediate skills addresses the criticism by
Lazonder (1994) and Horn (1992) that minimalism is only an effective strategy to use to teach basic tasks to
novice users.
Testing was one problem with Lazonder’s first experiment: he let the subjects use their training
manuals during the assessment phase. He inadvertently tested the ability to complete the basic tasks using a
minimal manual rather than the ability to learn tasks with it. The current study used an approach sim ilar to
the one used by Palmiter, Elkerton, and Baggett (1 9 9 1): training with instruction followed by immediate
and delayed testing without instruction. Subjects in each o f the different treatm ent groups o f the current
study maintained their level o f task completion and accuracy while decreasing the number o f errors they
committed during the posttest and delayed posttest.
Another problem with Lazonder’s study was his inconsistent time o f assessments. There was a 1-
week delay between assessments in Experiment I, a 3-day delay for Experiment 2 and no delayed
R e p ro d u c e d with perm ission of th e copyright owner. F urther reproduction prohibited without permission.
14
assessments for Experiment 4 (Experiment 3 was a qualitative, exploratory test). Immediate performance
tests assess the ability o f users to recall and apply procedural information from short term memory, but do
not assess the more practical issue o f retention beyond the initial training phase (which is more important in
a workplace environment). The current study used a I-week delayed performance assessment. A I-week
delay introduces a possible history effect— subjects could potentially practice the newly learned tasks
during the course o f their everyday work efforts. However, this type o f practice would be typical o f this
target audience and its effects distributed equally across the sample. Moreover, there were no statistically
significant differences in scores between the immediate and delayed posttest to suggest any possible threat
o f history .
Lazonder removed those tasks that were error prone (Carroll blocked them in his use o f the
Training Wheels simulated software) and provided extra error information for the remaining tasks (35% o f
the manual). The current study neither blocked nor removed from the instruction error-prone tasks. Rather,
sub jects were allowed to flounder in frustration (just as they would in the real world) if they made an error.
Subjects were, however, provided with error prevention/recovery information in Experiment 2. Again,
subjects were able to maintain their level o f task completion and accuracy while decreasing the number o f
errors they committed during the posttest and delayed posttest. However, as in Lazonder's (1994) study,
findings from the current research indicated that subjects exposed to minimalist instruction with variations
o f error information will be "equally skilled at detecting and correcting their own errors" (p. III).
Reznich: Minimalism and Com puter Anxietv
Reznich (1993) tested the efFect o f minimalist design principles in reducing computer anxiety for
users learning word processing tasks. He implemented six minimalist principles in a “getting started" type
o f document to teach basic w ord processing skills in W ordPerfect. He found no statistically significant
differences between the experimental and control groups for performance achievement (time and success to
complete tasks) or decrease in com puter anxiety. He concluded that well-designed instruction (regardless o f
the selected instructional design model) could teach new users word processing tasks and that computer
anxiety is reduced over time with continued exposure to a computer application.
R e p ro d u c e d with perm ission of th e copyright owner. Further reproduction prohibited without permission.
15
There were a few problems associated with this study. Reznich (1993) had a 30% differential
mortality rate, ceiling effects o f performance tests, and testing problems— all of which could have impacted
the outcome. All the students who achieved an initial decrease in computer anxiety stayed with the
experiment: the experimental group was administered the very same questionnaire four different times:
almost all the subjects in both groups completed the word processing tasks within the given time. Reznich
followed Carroll’s example and used basic tasks.
In an attempt to avoid those problems in the current study, the following controls were
implemented: (a) to prevent differential mortality, temporary employee subjects were not paid unless they
attended both sessions, (b) to prevent any ceiling effects, subjects were told to complete tasks that had been
identified by expens as moderately difficult, and (c) to prevent subjects from learning the test, the pretest
and posttests were administered only once, with different content for each. Even with these controls, there
was differential mortality in this experiment (12 o f the 22 subjects who left were from the inferential group).
1louever. the inclusion o f their scores would not have changed the results o f the experiment.
Warner: Test o f Two Minimalist Principles
Warner (1989) selected two minimalist principles to study: slash the verbiage and force the
coordination o f the system and the training (implemented via Teaming activities). He created four different
training manuals to study high versus low verbiage and activities at end o f lesson versus activities
throughout the lesson. His dependent variables were time to complete tasks, written and performance
achievement o f tasks, attitude towards learning, and level o f anxiety. As he expected, he found that subjects
using the training manuals with low verbiage scored higher on both the written and performance
achievement tests, completed their tasks faster, and had less anxiety. Subjects using the training manuals
with learning activities distributed throughout the lesson scored higher on the performance achievement test
and had less anxiety.
Contrary to his expectations, he found that subjects using the training manuals with high verbiage
and with activities at the end o f the lesson had a more positive attitude toward the learning experience than
did their counterparts using a low-verbiage version with activities distributed throughout the lesson. Those
16
using the manuals with the activities at the end o f the lesson also finished almost 12 minutes fa ste r than
those engaged in learning activities throughout the lesson.
Vanderlinden et al.: Minimalist Tutorials
Vanderlinden. Cocklin, and McKita (1988) wanted to design more effective tutorials for Hewlett-
Packard's CAD system. They were intrigued by the guided exploration approach Carroll. Mack, Lewis.
Grischowskv. and Robertson (1985) had used to teach word-processing applications. However. HP’s CAD
s\stem was far more complex than a word processing application, so they conducted a case study o f 16
users (62% o f which had previous CAD experience) to determine if the guided exploration would work for
their situation. They were also interested in whether prior computer experience would have an influence on
users' preference for tutorial type (guided exploration versus self study).
To answer these questions, they designed two types o f tutorials: a guided exploration (plus
procedures, when necessary) and a typical self-study tutorial. They found that subjects using the guided
exploration tutorial took 10% longer to work through the tutorial, but completed the post-tutorial test in one
half the time and with far fewer errors than those subjects using the self-study tutorial. In fact, those using
the self-study tutorial had to refer back to the tutorial four times more often than those using the guided
exploration tutorial. Subjects using the guided exploration tutorials reported that the guided exploration
tutorial u'as more vague than what they were accustomed to. They also found that prior computer
experience made no difference.
Although this was an exploratory case study, their findings lend support to the use o f the guided
exploration approach in the development o f software application documentation. Further, the fact that prior
computer experience made no difference in learning indicated that the current study could eliminate prior
computer experience as a critical variable and focus instead on learning styles (e.g., visual vs. verbal).
However, the results o f the current study were in direct contrast to the Vanderlinden et al. results. Prior
computer experience was statistically significantly correlated with task completion and accuracy, and
preferred learning style was not directly correlated.
17
Frese et al.: Guided Exploration
Frese et al. (1988) were also interested in the guided exploration and active learning approaches
espoused by Carroll et al. (1985). They conducted a study comparing the effectiveness o f three different
types of instruction to teach word processing skills: sequential steps to complete unrelated tasks (e.g., copy,
delete, print): sequential steps for tasks that were integrated using a diagram depicting their relationships to
one another and to the entire word processing system plus explanatory elaborations: no written assistance,
but the guidance o f Socratic dialogue by the researcher. In other words, they were comparing steps, steps
with diagrams, and full (but guided) exploration. Overall, they concluded that guided exploration was a
more effective means o f teaching word processing tasks than the use o f unrelated task instruction that is
t\ pical of traditional training manuals and online help systems.
All but one o f the statistically significant differences they found were between the steps and the
guided exploration. They found that subjects who explored the software interface as the researcher guided
them (via questions) to develop hypotheses and a mental model of the system could use the commands
better in the performance test than those subjects who learned the software by completing instruction on
specific steps. Those who learned the system via exploration could also complete transfer tasks better and in
less time and use fewer keystrokes to complete a task than those who learned via standard instruction. All
the subjects made errors, but those in the guided exploration group and the steps-with-diagram groups made
fewer errors and corrected them faster than those who learned via standard instruction. Subjects in the
guided exploration group also recalled more commands than those in the steps-with-diagrams group in a
free-reeall session on the second day. However, there were no statistically significant differences in recall
between the three groups on the third day.
There were a few problems with the Frese et al. study that could have accounted for some o f the
outcome. The most significant problem was that the researchers were actively involved in all three
conditions. When the subjects made errors, the researchers would either tell them how to correct the error or
use questions to guide them in the error correction. Frese et al. noticed that subjects o f a different gender
than the researchers performed better. Also, the variation in researcher personality and teaching expertise is
a variable they did not take into account. In the current study, only lab consultants were able to interact with
18
the subjects. The lab consultants were, in turn, monitored by the researcher to make sure they provided only
emergency and administrative help to the subjects.
Other. Mixed-Results Studies
Other empirical support for minimalism comes from studies conducted by Carroll and his group
(Carroll. 1984, 1990; see also Van der Meij & Carroll. 1995. 1998). Carroll found that experimental
subjects using his Minimal Manual completed tasks 40% faster and scored higher on achievement tests (by
a factor o f 10) than subjects using traditional manuals. Although they did not directly reference minimalism.
Rogers and Brown (1993) reported that the use o f minimalist-type written instructions resulted in a greater
quantity o f responses and more timely responses. Wenger and Spyridakis (1993) combined the minimalist
approach with findings from reading research for an increased performance in user reading, comprehension,
and recall. However. Jansen (1994) indicated that minimalism is not an effective approach for all types o f
users. Nowaczyk and James (1993) reported that their findings supported the “slash the verbiage" principle
but that screen captures (recommended by the minimalist approach) interfered with tasks. These studies
addressed the application o f minimalism to training manuals, not online help systems. Minimalism is a
training and instructional methodology. The application o f minimalist principles to online help is a new area
o f research.
Minimalism Applied to Online Help
The hypertext characteristics o f online help may make it a better medium for the application o f
minimalist principles than a print-based medium. Carroll (1987) stated that he saw “no useful distinction
between help and training” (p. 264). He suggested that the hypertextual nature o f online help made it an
effective media by which to provide different levels o f detail for different users. He focused his research on
intelligent online help but conceded that “only very fragile and limited examples o f intelligent help now
exist and none has been demonstrated to be successful” (p. 267). Microsoft’s w izard is good representation
o f the progress made in intelligent help since Carroll’s assessment, but there is still not yet the technology
and enough knowledge o f different learning behaviors to develop a robust, effective intelligent help system.
19
Farkas (1993) suggested that balloon help with "layering” was an application o f minimalism. This
suggestion met with mixed reviews (Price. 1993; Talburt, 1993) but merits further investigation. Layering is
a means o f providing users with a minimum amount (and different types) o f information and access to
vary ing levels o f more detailed information. Providing different layers o f information is a possible solution
to the problem o f using minimalism for users with differing levels o f prerequisite knowledge and different
task needs.
Farkas and his colleagues (Boggan et al.. 1996) proposed the use o f layering as one solution to the
problem o f different learning sty les. Carlson (1992) also sees layering as a modification o f minimalism to
address different types o f learners, referring to it as "intelligent hypertext.” She suggested placing the full
text online but adjusting the delivery according to the circumstances and the user. The interrelationships
among the intelligent nodes could be "sorted and filtered for specific combinations” o f user tasks and needs
(p. 25). Nichols’s (1994) research group developed three types o f online help panels (general, procedural
and conceptual) and two layers o f help (novice and advanced) to accommodate the different levels o f
background knowledge and user expertise.
Layering o f information is based on the hypertext structure and navigation research which is
directly applicable to the "guided exploration” philosophy behind minimalism. There are mixed findings in
regard to the most effective hypertext structure for learning and information retrieval. McKnight. Dillon,
and Richardson (1996) conducted a review o f the literature and found that overall subjects performed better
in a nonlinear environment and were able to adapt the material to their own learning style. The hypermedia
environment encouraged exploration and enabled users to see subtasks as part o f a whole task. Jonassen and
W ang's research (1993) supported these findings for field independent learners. However. Hailey and
Hailey (1997) found that a more linear structure o f the information was more effective. The very nature o f
hypermedia introduced a frustrating level o f confusion to some users. Nichols (1994) found that people do
not yet have a good schema for processing information so it was better to provide all users with structural
guidance. In their opinion paper. Grice and Ridgway (1995) suggested that "minimal information,
minimally connected” is better (p. 39).
In summary, minimalism is a buzzword that is used among technical communicators to refer to a
20
set o f principles for writing documentation. These principles were derived for application to training
manuals as "self instruction” for users learning new software applications. However, many o f the minimalist
principles are directly applicable to online help. Because the application o f minimalist principles to online
help has not been thoroughly investigated, the research on print-based media and hypertext/hypermedia was
evaluated for research guidelines. The resulting research was based, in part, on the above studies.
Subject Preference fo ra Verbal or Visual Learning Style
Because several different researchers (Carlson. 1992; Horn. 1992; Jansen, 1994; Reddish. 1988)
suggested that minimalism was not applicable for all types o f learners in all types o f situations, it seemed
prudent to incorporate at least a simple measure o f learning style into this research. The tool selected for use
in this research was the Verbalizer-Visualizer Questionnaire (VVQ). This 20-question assessment tool was
designed by Richardson (1977 as cited in Kirby. Moore, & Schoefield. 1988) and later modified by Kirby et
al. ( 1988). Their tests o f the modified VVQ showed that the two learning constructs (visual and verbal)
possessed good construct validity in the principal component analysis and adequate reliability (the alpha
coefficients). They also found that verbal and visual learning styles could be measured and that there was a
moderate correlation between visual and verbal learning styles and their associated abilities, but that the
learning styles explained even more o f the variance in scores than did ability.
Several dissertation studies which used the VVQ were located. They had mixed results. Blair
(1980). Kini (1993). Kuchler (1983). and Spiegel (1985) all obtained nonsignificant differences associated
with scores from the VVQ. Blair’s (1980) study was an attempt at increasing subjects’ ability to create
visual images. Subjects’ scores on the VVQ did not change after the visual imagery training. Kini’s (1993)
study compared subjects’ cognitive style, field independent (FI) and field dependent (FD) scores, and scores
from the VVQ with performance outcome on a concept-learning task. The FI-FD and VVQ dimensions
were independent, but subjects who received instruction that matched their preferred learning style did not
improve their learning performance. Kuchler (1983) administered a whole battery o f tests— including the
VVQ— in an attempt to determine the relationship between cerebral “hemispheric style” and the traditional
"cognitive style.” She obtained very low correlations (.00 to -.54) and concluded that there was
21
considerable diversity in cognitive style and that there is a need for further test development to measure
such constructs. Spiegel (1985) also found no significant differences in scores from two models o f math
instruction as associated with the VVQ.
Although two other researchers also obtained no significant differences, they did obtain some
interesting findings. Fishel (1984) obtained no significant difference in her study o f creativity and preferred
learning style. However, she did find that the mean scores for creative ability by females with a preferred
verbal learning style and males with a preferred visual learning style were slightly higher than the same
scores by their counterparts. In another study, Parzivand (1984) obtained similar results in his study o f
creativity and "m odes o f thinking,” as he called the scores from the VVQ. Although not significant, he
found that subjects with higher verbal-leaming scores tended to have higher creative production scores than
did their visualizer and mixed counterparts.
However, some researchers did obtain statistical significance in association with VVQ scores.
Ackerman-Efron (1985) found a significant positive correlation between the proportion o f upward eye
movements and stares (in response to modally ambiguous statements) and higher visual scores on the VVQ.
Dern (1982) found a significant positive correlation between higher visual scores on the VVQ and higher
scores on anxiety and depression measures. In experiments more closely related to this research. Garro
(1982) and Pantin (1982) both obtained statistical significance in the association o f scores from the VVQ
and performance outcomes. Garro (1982) found that subjects with a visual preferred learning style scored
better in both recall and recognition o f visual stimuli. This is important as it relates to the display o f content
in either graphic or textual format. Pantin (1982) found that the use o f visual imagery in text is also
important. Subjects rated higher those excerpts o f text that matched their own visual/verbal preference (i.e.,
subjects who had a higher visual score on the VVQ preferred more visual imagery in the written word).
The scores from the Verbalizer-Visualizer assessment tool have not been normed. Kirby (personal
communication. June 5. 1998) has suggested that although the constructs are independent, the range o f
scores for each construct would fall along a continuum, requiring a multiple regression analysis. Kirby
suggested that the comparison o f an individual’s score to the scores o f the related group (in this case,
secretarial employees) would provide the m ost useful information. The VVQ scores indicate the learner’s
22
preference for style o f learning (either verbal or visual) and were classified as an attribute variable (Borg &
Gall, 1989) and used to test any interaction between preference for learning style and type o f online help.
Subjects' scores from the VVQ were compared to their ability to learn and complete tasks using different
types o f online help systems which varied in the amount o f verbal and visual information provided.
23
CHAPTER III
APPLICATION OF THE PRINCIPLES OF MINIMALISM
TO ONLINE HELP
The first goal o f this research was to test the application o f the general principles o f minimalism to
an online help system. The first general hypothesis was that users required (and preferred) less instructional
support to complete tasks, but more instructional support to learn tasks. The second general hypothesis was
that users who were prompted to interact with the application would spend longer amounts o f time learning
tasks, but, once learned, complete tasks faster without instructional support.
The second goal o f this research was to test the application o f Principle 3 o f minimalism
(supporting error recognition and recovery) to an online help system. The general hypothesis was that users
will be more effective and efficient at identifying and correcting errors if they have direct access to error
information.
The third goal o f this research was to determine the most effective combination o f information
types (e.g.. procedures, explanations, descriptions) for different preferences for learning style. The general
hypothesis was that users using an online help system that was matched to their learning style would
complete software application procedural skills more quickly and with fewer errors than subjects using a
nonmatched type o f online help.
Five different online help systems were developed in an attempt to implement selected principles
o f minimalism and test the above hypotheses. The first three online help systems followed those described
by Carroll (1990) and Black et al. (1987) in their experiments with the Minimal Manual. Those three help
systems are referred to as skeletal, inferential, and elaborative. Two additional help systems were developed
(using the Inferential help system as a base) to evaluate the addition and location o f error identification and
recovery information. These two help systems are referred to as inferential-proximal and inferential-distant.
All five help systems contained some identical characteristics so that variance could be associated
with their purposed differences. This section identifies those characters that were both common and unique
among the help systems.
24
Common Characteristics o f All Online Help Systems
Care was taken to ensure that all the help systems were designed and developed in a similar
manner so that any differences among them would be more easily attributable to their purposed differences
in design. The attributes that each had in common included the same tasks in the same software application,
style guidelines, window format, table o f contents, title bars, action text, and use o f caution notes.
Tasks in Microsoft Word
All the online help systems represented a tiny slice o f the MS Word software application. They
each contained instructions on how to complete the following general tasks: inserting and formatting text,
graphics, and tables: creating, formatting, and merging a mail merge data source; accessing, customizing,
and navigating the online help system. However, please note that although the help systems contained the
procedures for all the tasks listed above, the subjects used only those procedures associated with text for
Experiment I and then used only those procedures associated with graphics, tables, and mail merge for
Experiment 2. The procedures for text, graphics, tables, and mail merge were customized according to the
principles being tested in each help system.
Stvle Guidelines
Each help system also followed the same style guideline, as specified by Boggan et al. (1996): use
o f gerund headings for Level 1 and Level 2 conceptual topics (e.g.. Inserting and Formatting Text), use o f
root headings for Level 3 headings that could be further subdivided into procedural topics (e.g.. Create a
Text Box), and use o f infinitive headings for Levels 3 and 4 procedural topics (e.g.. To Create a Text Box).
Window Format
The other constants across all five online help systems were type, color, size, and placement o f the
online help windows. Two windows were used to denote the difference between conceptual and/or
introductory topics and procedural topics.
Figure 1 shows a main window containing an introductory topic for creating text boxes. Main
windows were formatted for landscape display and had a soft blue title bar over a soft yellow background.
25
Uvimj MS Woril
File £dit Bookmark Options
Contents Index a^ck
Use text boxes to flow tex t automatically from one page to another or to create w ate rm arks
R elated T asks:
To create a text box
To create an A u to S h a o e text box
~o insert text into a text box
To link to another tex t box
To link to an .AutoShaoe text box
"o change the a p p e a ra n c e of a text box
Fiaure I. Example o f main window.
These colors were chosen because they were softer and more aesthetically pleasing to the eye. The main
windows were larger in size (predominantly because some o f them contained much conceptual text), and
were positioned in the lower right com er o f the screen. Main windows had a default o f "Not On Top"
display (that is, they would disappear from view when the subject clicked anywhere outside the main help
window, thus allow ing the user to view more o f the task-application screen). They were set to an absolute
size with automatic scrollbars. Main windows generally did not contain any procedural steps (there was one
exception), but hypertext links to related procedural steps.
In contrast. Figure 2 shows a procedural window containing the procedures necessary for creating
text boxes. Procedural windows were formatted for portrait display and had a bright blue title bar over a
white background. T hese colors were brighter, drawing attention and providing a stark contrast o f the
emphasis formatting (e.g., red and green colors, bolded/italicized text). Procedural windows were smaller in
size— even the title b ar was changed to a narrow height— to allow the user to see more o f the screen while
simultaneously viewing the steps in the help system. Procedural windows were positioned in the upper right
comer o f the screen with a default o f “On Top” display (that is, they would remain visible on top of the
task-application screen until the subject closed them). They were set to relative size (that is. they expanded
vertically as necessary to accommodate the text) with automatic scrollbars when the amount o f text
26
Helplopics Back Erint Options <<

>>
1 Click on insert | Text Box.

2 Click, hold and drag the crossbars to the desired text
box size.
What Can I Do With a Text Box?

C r e a t e a n A u t o S h a o e t e x t bo x
In sert t e x t into a t e x t bo x
Link to a n o t h e r t e x t b o x
Link to a n A u t o S h a o e t e x t bo x
C h a n g e its a o o e a r a n c e
U s e it to c r e a t e a w a t e r m a r k
C h a n g e t h e d ir e c tio n of t h e te x t in s id e a te x t b o x
Figure 2. Example o f procedural window.
surpassed the height o f the computer window. Procedural windows contained the procedural steps with
hypertext links to related procedural steps.
Forward/backward chevrons ( » « ) were placed on the procedure windows, but not on the main
windows. Thus, subjects were able to view conceptual topics in the smaller, stay-on-top procedural window,
but not procedural topics in the larger, not-on-top window. Both windows contained a “ Back” button which
took subjects to the previously viewed screens. Subjects could access the Index directly from the Main
window, but had to first click on a “ Help Topics” button to access Index or Find from the procedural
window. This decision was made in an attempt to keep the procedural window simple with a focus on the
procedures.
Table o f Contents
The second principle o f minimalism is to anchor the tool in the task domain (Van der Meij &
Carroll. 1995). The two heuristics to support this principle are (a) select or design instructional activities
that are real tasks and (b) make the components o f the instruction reflect the task structure. Based on that
27
principle and those heuristics, all the online help systems were designed to assist subjects in completing
specific tasks rather than describe the functionality o f each dialog box and button in MS Word (the selected
software application). Figures 3 and 4 show an example o f the skeletal table o f contents at Level 2 and
Level 4. respectively. The tasks chosen represent typical end-user tasks in this software application.
With the exception o f the inferential-distant help system, all the help systems were identical down
to Level 2 o f the table o f contents (the former included a "Troubleshooting” section). The skeletal and
elaborative online help systems each contained the same listing o f procedural topics in Levels 3 and 4. The
inferential online help system displayed less procedural tasks (see Figure 5) in order to force the subject to
infer related tasks from the tasks given (compare the elaborative and inferential online help systems).
Title Bars
The title bars for the table o f contents and the main windows in all help systems displayed the
"Help Topics: Using MS Word” text. The title bars for the procedural windows in all help systems
displayed the "How to ...” text.
Help Topics: Usinq MS W ord
Content* | index | Find |
Click a book, and then click Open. Ot dick another tab. such a t Index.
^ Inserting and Formatting Text

Insertng and Formatting Graphics
^ Inserting and Formatting Tables
( j j Mergng Busn e s s CommwicaMon Documents
Creating a Man Document
% Creating a Data Source
^ Editing/Formatting the Layout
^ Fitemg/Sorting Lists via Query
Mergng a List w th a Main Oocunert
(U Help on Help
% Accessing help Topics
^ Navigating n Help
^ Customizng Help
Prnt.. Cancel
Figure 3. Table o f contents at Level 2.
28
Help Topics: Using MS Word 21*1

Contents | Index | Find |
Click a topic, and then click Display. Or cSck another tab. such a t Index.
Creating Business Communications Documents

( | j inserting and Formatting Text
?] Inserting and Formatting Text
^ Insert Text
?] Create a Text Box

jf] To create a text box
?] To create an AutoShape text box
?] To insert text into a text box
_?] To ink to another text box
?] To ink to an AutoShape text box
?] To change the appearance or a text box
.2] To use a text box to create a watermark
_?] To change the direction of tha text nside a text box
^ Create Text and Headng Styles y|
Close Print.. Cancel
Figure 4. Skeletal and elaborative table o f contents at Level 4.
Help Topics: Using MS W ord 1 -U2S!

Contents | index | Fnd |
Click a tope, and then click Display. Or cSck another tab. such as Index.
I j j Creating Business Communications Documents

I j j Insertng and Formatting Text
J ) inserting and Formatting Text
^ Insert Text
iy Q Q m Q U l
7] Create a Text Box
?] To create a text box
7] To link to another text box
7] To change the appearance o t a text box
^ Create Text and Heating Styles
% Format Text In Columns
^ Create Headers £ Footers
Inserting and Formatting Graphics
^ Inserting and Formatting Tables
Mergng Busmess Communication Documents
Zl
Cjose | Print.. Cancel
Figure 5. Inferential table o f contents at Level 4.
29
Action Text
Principle 1 o f minimalism is that designers should "‘choose an action-oriented approach” ; its
supporting heuristic is that we should provide an immediate opportunity to act (Van der Meij & Carroll.
1995). This principle and heuristic is closely related to Principle 4 (support reading to do, study, and locate)
and its supporting heuristic (be brief, don’t spell out everything). The implementation o f the first principle
and heuristic meant placing everything in active voice and designing topics around the associated tasks that
a given user may be attempting to complete (see, for example. Figure 6).
The implementation o f Principle 4 and its supporting heuristic meant eliminating some
information. Van der Meij and Carroll (1995 refer to two types o f screen information: the content and th
location. They suggest that “ incomplete screen information can be created by omitting one o f the two and
by making the information less explicit” (1995. pp. 256-257). Being brief, while simultaneously supporting
the user in his or her activity, meant eliminating the obvious (e.g.. "Click OK to close the dialog box."), but
it also meant going one step further and eliminating the steps leading to the location o f the information. For
.r jn jx j
Help Xopici fiack Print Options <<
>>
1 Right-click on the text.b.ox border

2 Click on Format Text Box.
3 Select a solid Fill Color or a special Fill.J=ffect.
4 Select solid or patterned Ijng Color.
5 Click on the Wrapping tab and select a text wrap
option.
6 Click OK to exit.
R elated Tasks:
To insert text into a text box
To link to another text box
To link to an AutoShape text box
To use a text box to create a watermark
To change the direction of the text inside a text box
Figure 6. Skeletal procedural topic.
30
example, in the illustration in Figure 6, Step 3 is to “Select a solid Fill C olor or a special Fill Effect,” but
the user is not told where to find the fill color effect in the resulting dialog box (see Figure 7). The user
must infer that the "Fill Color” option would be located under the "Colors and Lines” tab. They must then
infer that they must then click on the down arrow to select one o f the displayed colors in the color palette, or
click on "Fill Effect” and select a predefined fill effect or create a unique fill effect. The net effect o f being
brief while supporting the user’s activity is that the user is then forced to interact m ore with the interface
(and hopefully, therefore, learn and remember how to complete the given tasks in that application).
Caution Notes
Boggan et al. (1996) differentiated among the different types o f warning notes: danger (injury to
persons), warning (damage to software or hardware), and caution (potential loss o f data). The only type o f
warning note used in all the help systems is the caution note (Figure 8). The frequency o f the caution note
differs, however, among the help systems. All the help systems have at least the caution note illustrated in
Figure 8. The inferential help systems (including its revisions used in Experiment 2) include additional
caution notes as error prevention measures (see example in Figure 9).
Form at T ext Box
Colors and Lines | Stee | Layout | | Text Box | Web |
Fill ------
£olor: ] ■ej I- Sem(ransparenc
Line — - NoFS
Color: ■■■■■■■■
Cashed:
■■■■■■■■
Arrows —
□ c c □ □ □ □ (□
More Colors...
fi* Effects...
Fiuure 7. Format Text Box dialog box.
31
mmm. - |D |X |
Help Topict
>>
Sack Prr* l
Qpfora ln
1 Create 3 data so urce.
2 Click on Tools I Mail Merge I S etu p ....
3 S e t the desired print and label options or envelope
options and d ic k OK.
J Click on the Insert Merge Field button and select
the layout of the fields you want to merge.
© CAUTION: Insert spaces between fields (for

example, between first and lest name) If you do not
insert spaces, you may have to delete the resulting
mai.ling.Jist. document and stan over again.
5 Click OK.
You are now ready to filter and/or sort and merge
R ela ted T asks:

To create a data source
To add new data tields
To arid data to a data source
To -reate a main document
Filtennq/Sortmq Lists via Query
Merging a List with a Main Document
Fieure 8. Default caution note.
Unique Characteristics o f Each Online Help System
The first three help systems (used in Experiment 1) were based on the skeletal help system, but
then had content and instructional features either added or removed. The three help systems were skeletal,
inferential, and elaborative.
Skeletal Online Help System
The skeletal online help system included all the functions using terse, but explicit definitions and
procedural steps. Subjects had to click on hypertext to get definitions. There were no conceptual or
explanatory sections included in the procedural topics. Subjects learned by doing. Figure 6 is an illustration
□
X
i
Help Topics £ack £nnt Options <<
>>
On the page
1 Click and hold the mouse over the graphic.
2 Move the graphic to the desired position; release
mouse.
© Hint: You can resize a graphic for easier positioning.

Just ciick and drag one of the four com er selection
boxes.
© CAUTION: If you click and drag the top or side

selection boxes, then you will distort, rather than
resize, the graphic.
Within layers
1 Select the graphic
2 Click on Draw | Order.
3 Select one of the layering options.
O nYourO wn...
Related Tasks:
To in s e rt a oraohic
To wrap text around a graphic
Figure 9. Caution note in inferential help system.
o f a procedural topic in the skeletal online help system. Figures 10 - 12 illustrate examples o f what the
subject would see if he or she clicked on some o f the definition hyperlinks in this window.
Elaborative Online Help System
The elaborative online help system is based on the skeletal version o f help, but includes the i
instructional support available in any o f the online help systems.
Explanatory information. Elaborative help topics generally included a two-to-three sentence
explanation o f what the task does and why subjects might want to perform it. This explanation was
displayed below the heading and immediately above the procedural steps.
33
T ext Box
A co ntain er for text. Word h a s several different s h a p e s for
text b ox es. You can resize and position text box es
anywhere on the document. You c a n also link text box es
tog eth er s o th at text flows automatically from one text box
to another.
Figure 10. Definition 1.
Fill E ffect
Special effects for text boxes. There are four different ty p es
of fill effects: Gradient, .Texture, Pattern and .Picture,
G radient
A special fill-effect for drawing objects. There are three
Gradient options: (1) one solid color, (2) two colors in a
variety of shading options, (3) p re s e t gradient color th e m e s
{for example. Chrome, Ocean).
Notes. Elaborative help topics included supplemental information in the form o f notes. Although
not critical to complete the task, notes provided users with helpful information. When the information was
relevant to the entire task, then the note was placed below the steps; however, when the note was relevant
only to a step within the task (see Figure 13), then the note was placed immediately after that step (Boggan
et al.. 1996).
Graphic feedback. Elaborative help topics also included two types o f instructional, graphic
feedback: (a) "Click here for a graphic o f . . . ” text (see Figure 13), and (b) "W hat you should see” hypertext
located either at the end o f the sequence o f the tasks or (in the case o f a complex task) at appropriate
divisions within the task (see Figure 13). Figure 14 illustrates what a subject would see if they clicked on
the “Click h e re ...” text pictured in Figure 13; Figure 15 illustrates what they would see if they clicked on
the "W hat you should see” text. The graphical feedback confirmed to the users that they were or were not in
34
L?jCHa IBgHi'f
Help lopics
>>
Back Print Options <<
r
Select the text, cell, or box for which you want to
change the default Vi pt. black line border.
2 Click on £ormat | Borders and Shading.
(C.lick here for a graphic of the Borders and
Shading dialog box.)
3 Click on a border Setting to select it. Choose from

• Box
' Al)
• Shadow
• 3-D
■ find
« C u sto m
4 Scroll through the selection of line Styles; click on

one to select it.
5 Click on the down arrow and scroll through the
selection of line Colors; dick on one to select it.
6 Click on the down arrow to display the different line
Widths; click on one to select it.
7 Preview your border selection in the Preview panel.
Note: If you selected the Custom setting, then you

need to click on the borders in the Preview panel to
apply the selected border as desired.
(W hat y.Q.u shp.uld s s s if you selected a custom

setting.)
8 Click the down arrow next to the Apply to: field

and select the border area (text, cell, paragraph,
table, page). Zl
Figure 13. Elaborative help with links to graphical feedback.
the right place to complete the step o r that they had completed the step correctly. The main difference
between the two types o f graphical feedback is that the ‘‘What you should see” graphics included a b rief
explanation (in blue type) at the bottom o f the graphic.
35
B o ld e r: an d S h ad in g
gprders | Page Border | S v rin g |

Setting: S t*: Preview
r fitpne
Bos
Cleft on diagram bebw or use
buttons to apply borders
ID
Al
Color:
6
Gri£
&dth:
'A p t App(y to:
asm
C^tofli 3 | Table
C p tijr * ...
Showloobar OK Cancel
Figure 14. "Click h ere..." graphical feedback.
B o ld e r: an d S h ad in g
gprders Page Border Siadhg j

Setting: Staler Preview
r Ckkon diagram below or usk-

buttons to apply borders^
[□ Bos
[s' ®
[Âuto
Grig
i
w Mfldtht1 -
'Apt Apptttor
Qfftom
, p s m
Opffcnt.
> i • oit'dr OK j. Cancel.
Notice in this grap h ic th a t th e Custom box is selected,

different borders a re a p p lie d a ro u n d th e selected
c ells in the P review p a n e l, a n d Cell is selected for
th e a rea of application.
Figure 15. “What you should see” graphical feedback.
36
Hints. The hints in the elaborative online help system differ from those found in the inferential
online help systems. Elaborative hints provided additional information to help the user complete the given
task (see Figure 16). while the inferential hints encouraged the user to reason out how to com plete a similar
task from the given task (see Figure 17).
Inferential Online Help Svstem
The inferential online help system was used in both experiments. In Experiment 1 it was used in
direct contrast to the skeletal and elaborative online help systems. In Experiment 2 it provided the base for
the error-recovery information added to the inferential-distant and inferential-proximal online help systems.
It was also used in its original state (i.e., without any error information included) for comparison purposes,
resulting in three online help systems for Experiment 2: inferential without, inferential-distant, and
inferential-proximal.
■ ■ ■ I
Hdpjopics Back Print Options <<
>>
1 Right-click on th e table.
2 S e lec t Bordeis and S hading.
3 Click on th e G rid icon
© 2 B & the Grid option inserts 1/2 pt. lines to borders

inside the table while retaining your other border
selection for the outside border. The A il option
applies your border selection to both internal and
external table borders.
R elated Tasks:
To insert a table
To change the alignm ent o f the table text
To chanoe the direction o f the table text
To change table borders
To shade or pattern c e lls
To meroe c e lls togeth er
To divide c ells
Figure 16. Elaborative help with hint.
37
Help Topics
1 Click on Insert I T e st Box.

2 Click, hold and drag the crossbars to the
desired text box size.
© H int There are several things you can do

wth text boxes For example you can:
□ Create AutoShape text boxes: U se the

A u toS h ap es menu on the Drawing.
Toolijar. After drawing the AutoShape,
right-click on it to add text.
□ Insert existing text into a text box: Use

the Insert I Fife menu commands to
insert text rather than typing it in.
□ Use a text box as a wajgrmark that is

displayed on each page: Just open the
H eader/Footer view before you insert
the text box
□ Change the direction of the text inside

a text box: Use the C h ange T ext
Direction icon on the T.abJjes.flp.d.
Borders t.op.I.tjar
OnYoiirQyrn...
What Can I Do With a Text Box?

i.;niv 10 another text box
'har.'ie its appearance
Fitiure 17. inferential procedural topic with hints.
The inferential online help system used the skeletal version o f help as a base, but with less
information and more encouragement to "reason and improvise” the application’s functions (Carroll. 1990).
For example, where the subject had the option to select special fill effects for a text box. the procedural
steps were included for only one effect (see Figure 18) and the subject was encouraged (via “On Your
Own" text) to explore the rest (see Figure 19). The inferential online help system required the learner to
attend to the system during the course o f training (Carroll, 1984).
Another, more significant way that the inferential online help system forced users to reason and
improvise with the software application was to provide the directions for completing related tasks within
"Hints" located at the end o f the procedural steps for a base task. For example. Figure 17 shows the steps
38
Options
1 Right-click on the Is.x.t.b.Qx border.

2 Click on Format Text Box.
3 Select a solid Fill Color or a special /rilLEffscx
4 Select solid or patterned U.n.e Color.
5 Click on the W rapping tab and select a text wrap
option
6 Click OK to exit
On Your Own...
Related Tasks:
To create a te xt box
To linn to a n o t h e n e x t box
Figure 18. Inferential procedural topic with "On your own” prompt.
D id you find all th e P r e s e t g r a d i e n t fills? How ab ou t th e I

T exture th a t lo o ks like fish lying in t h e s a n d ? I
Figure 19. Inferential "On your own" pop-up.
for the base procedure o f creating a text box. However, notice the inclusion o f four related tasks (creating
an Autoshape text box. inserting text into a text box. using a text box as a watermark, and changing the
direction o f text inside a text box) located under “Hints” in the same window. Notice also that there are only
two related tasks as hypertext at the bottom o f the window. Compare this with the same procedural task
from the elaborative online help system (see Figure 20). By providing related tasks as hints in the same
window as the base procedure, users can apply the same knowledge to different, but similar, tasks and see
how those tasks are related to one another within an overall goal.
Inferential-Distant Online Help Svstem
Based largely on the inferential online help system, this online help (used only in Experiment 2),
was designed to test the effectiveness o f error-recovery information placed in a ‘Troubleshooting” section
39
Help l o p e s | Enr* Option* (<
1 Click on [nsart I Text Box.

2 Click, hold and drag the crossbars to the desired text
b o x s iz e
Note: You need to be m the Page Layout vtew to see

te x t b o x e s
Note: Text boxes dont have to be boxes, per se You can

select an Auto Shape from the Drawing toolbar and Add
Text to it
W hat C a n I Do With a T e x t B ox?

: : a a t e an A u to S h a o e te x t bo x
i i s e rt t e x t .n to a te x t b c <
j - n k tn - m o t h e r S e t t h o *
I 'in k tn o n A .u tT S h a o e h fix
i ■ tn e a p p e a ra n c e
I
I J -se it to c r e a te a w a te r m a rk
| > . n - e tn e e r e c t io n o f th e te x t in s id e a t e x t b o x
Fiuure 20. Elaborative procedural topic.
at the end o f the help system. Users had two ways o f accessing the error-recovery information: (a) through a
"Troubleshooting" button located on each procedural window and some main windows, and (b) through the
table of contents. Figure 21 shows the table o f contents for the inferential-distant online help system. It was
the only help system that difFered in the table o f contents at Level 2.
Most subjects, however, used the 'Troubleshooting” button (Figure 22) to access error-recovery
information. Clicking on that button resulted in the display o f the general “Troubleshooting” main window
(Figure 23). Subjects could then select the area in which they were having troubles and then access the
appropriate error-recovery information (Figure 24).
Inferential-Proximal Online Help System
Again based largely on the inferential online help system, the inferential-proximal online help
(used only in Experiment 2), was designed to test the effectiveness o f error-recovery information placed
40
Help Topics: Using MS W ord M -: jJ i< l

Contents | index | find |
Click a book, and then dick Opea Or dick another tab. such as Index.
^ Creating Business Communications Documents

^ UyWHBWIiHMMIIiim M HIlm i
^JJ Troubleshooting
^ Problems w th Borders and Shadng
^ Problems w th Cokmns
% Problems w th Flea
^ Problems w th Graphics
^ Problems w th Headers and Footers
^ Problems w th Mai Merge
Problems w th Styles
Problems w th Tables
Problems w th Text
^ Problems w th Text Boxes
Problems w th Watermarks
Help on Help
Open E m it.. Cancel
Figure 21. Inferential-distant online help system table o f contents.
S s liiP ih ’:'"
r
_ -!□ ! x |
Helplopics fiack Print Options L<
>>
Troubleshooting
1. Click on Io o ls | Mail Mecge | Query Options;...
2. In Field, select the field you want lo use as a filter.
3 In Comparison, select a filter.
4 Enter the value of the filter in the Compare to field.
5 Click OK.
© Hint: You can aiso sort the records. Just click on

the Sort Records tab and define the sorting criteria.
Related Tasks:
Editing/Formatting the Layout
Merging a List with a Main Document
Figure 22. “Troubleshooting” button in inferential-distant online help system.
41
& U sing MS W ord

File £<At Bookrrvark Options Help
Contents
P ro b le m s with B orders and Sh adin g

P ro b le m s with C olum ns
P ro b le m s with Files
P ro b lem s with G raphics
P ro b lem s with H ea d e rs an d Footers
P ro b lem s with Mail Merge
P ro b lem s with S tv le s
P ro b lem s with Tables
Figure 23. General access to "Troubleshooting" topics via "Troubleshooting" button.
119 I
Help Topics Back Print Options <<
>>
Solution: Go back into the Mai.l.Merge.He!P.er and click

on either
(a) Get Data and then open your completed .d?lt?..§.QUCCS or
(b) Edit and add records (for example, names and
addresses) to your data source.
Explanation: There is an option in the middle of the mail
merge process to set up the document or create the data
source. If you set up the majn.cjgc.gment before creating
the data source, then there will be no records to filter or
sort.
Figure 24. Troubleshooting topic.
next to where users need it most (Lazonder, 1994: Van der Meij & Carroll, 1995). In line with the iterative-
testing approach to minimalism (Carroll, 1990), two pilot studies were conducted for this research to
determine the error-prone steps. Basic procedural steps were revised for clarity o f understanding. Error-
recovery information (see Figure 25) was then placed in the procedural topics at a frequency o f about once
42
Help Topic*
1 Click on l o o t s I M ail M e ig e I Q u e ry O p tio n s....

2 In F ield, sele c t th e field you w an t to u se as a
filter.
3 In C o m p a riso n , se le c t a filter.
4 Enter the value of the filter in th e C o m p a re to
field
5 Click OK
Error Infqrmati.Qn
© H in t You can also sort the records Just click

on the §.ort R ecords tab and define the sorting
criteria.
Figure 25. Proximal error information in inferential-proximal online help.
for every three to five actions (Van der Meij & Carroll [1995] suggested inserting error information at a rate
o f about once every three actions).
Subjects could view the error information related to the task or a substep o f the given task ju st by
clicking on the "error information" hypertext link. Unfamiliar words in the error information were
highlighted as hypertext pop-up definitions for further understanding. Notice that the error information for
both the inferential-proximal (Figure 26) and the inferential-distant (Figure 24) online help systems
remained the same; it is just access to that information that changed.
You g e t an error m e ssa g e s a y in g that Word

found no data records to filter or sort? Go back
into the .Mai.l.Marije H e lp e r ancj cjjck on either
(a) G et Data and then open your completed data.
puree or
(b) Edit and add records (for exam ple, nam es and
addresses) to your data source.
Explanation: There is an option in the middle of the

mail merge process to set up the document or create
the data source. If you set up the main.dpcument
before creating the data source, then there will be no
records to filter or sort.
Figure 26. Pop-up error information in inferential-proximal online help system.
43
CHAPTER IV
STATEM ENT OF THE METHODS AND PROCEDURES
All subjects for this empirical research had to qualify for entrance based on their scores on a Kelly
Services Skills Testing o f MS Word 97 tasks. Once qualified, they completed a background-information
survey and a Verbalizer-Visualizer survey. Kelly Services personnel then scheduled them for two sessions,
one week apart, at the Business Information Systems and Education computer lab.
During the first session subjects used MS Word 97 to create three similar-looking newsletters as a
form o f a pretest, treatment, and posttest. During the second session, a week later, subjects created a final
newsletter. Subjects evaluated one o f the two online help systems they had used the week before.
This chapter identifies and describes the experimental materials that the subjects received. It also
describes the treatment design and specific hypotheses tested, the pilot studies, the external review o f
treatments, the population sample selection and assignment, the use o f lab consultants, the actual
experimental procedures, the procedure for coding data, and the procedures for statistical analyses.
Presession Materials
Subjects signed an informed consent form, completed questionnaires, and took a qualification test
prior to actually being admitted to the research study.
Letter o f Informed Consent
Subjects were required to sign a letter o f informed consent (Appendix C) prior to being admitted to
the study. The letter o f informed consent specified that the subjects would be paid only upon completion o f
both experimental sessions, one week apart. The letter also specified the prerequisite qualifications for
participating in the study (basic word-processing expertise) and suggested that subjects could become more
marketable after participating.
Background Information Questionnaire
An eight-question assessment tool (Appendix D) was designed specifically for this study to obtain
44
simple demographic information. The results o f this tool provided data on whether gender, age, level o f
general com puter experience, and experience with previous computer applications had any impact on how
effectively subjects used different types o f online help systems.
Verbalizer-Visualizer Questionnaire
The Verbalizer-Visualizer Questionnaire is a 20-question assessment tool (Appendix E) designed
by Richardson (1977 as cited in Kirby et al., 1988) and later modified by Kirby et al. (1988) to assess a
person's preference for a verbal or visual learning style. All subjects completed this questionnaire before
being admitted to the study. The visual scores o f the experimental subjects were then used to stratify their
random assignment to the different experimental groups.
Kellv Services Skills Testing
Subjects used a Kelly Services' proprietary, computer-based assessment tool to complete a variety
o f tasks in a simulated version o f MS Word 97. The skills-testing simulation recorded the time it took for
subjects to complete tasks using their choice o f method (e.g.. menu or keyboard). Subjects for this study
completed the basic version o f this test, which assessed mastery o f skills such as opening, closing, and
saving tiles: inserting, copying, moving, and deleting text (see Appendix B). Subjects had to receive a "Q"
(qualify) in order to participate in the study. Subjects could take the Kelly’s training in order to pass the
test. Originally subjects were required to complete correctly 80% o f the tasks in this test in 18 minutes in
order to receive a "Q .” However, some subjects were still struggling so much over basic Windows
operations (e.g.. open, close, save) that were not part o f the research study, that the minimum qualification
rate was changed to 100%.
Experimental Materials
Subjects went through the same procedures and performed the same tasks throughout the course o f
this experiment. The only thing that differed among the subjects was the type o f online help instruction they
received. Everything that was the same for each subject is described in this section: the four newsletters the
subjects created, the software they used to create the newsletters, and the tasks that comprised the
45
newsletters. Also described in this section are the instructions on how to use the online help systems, the
names o f the five different online help systems, and the evaluation instrument for the online help systems.
Four Assessment Newsletters
All subjects created (or at least attempted to create) four newsletters (Figures 27, 28, 29. and 30)
as a pretest, treatment, posttest, and delayed posttest assessment o f their expertise with intermediate and
advanced MS Ward tasks. Each assessment newsletter was comprised o f five main steps:
1. Create a banner heading (using text boxes)
2. Insert and format text into columns with a banner heading
3. Insert and format a graphic (i.e., resize, position, apply word wrap)
4. Create and format a table (i.e., apply borders and shading; rotate text)
5. Create a data source and mailing labels for a mail merge.
Each o f the five tasks was timed for the pretest and posttests. For the pretest, subjects were allotted
the amount o f time it would take for an expert to complete the same task. For the immediate and delayed
posttests, subjects were given a total o f 45 minutes (divided proportionately among the five tasks) to
complete all five tasks. The resulting time for each task was approximately one and one half to two times
the amount o f time it would take an expen to complete the same task. Subjects were not timed during the
completion o f the treatment tasks, but were encouraged to move ahead without completing the tasks after a
specified time (they had approximately 2 hours to complete the five tasks— a lab m onitor encouraged them
to move to the next task after each task’s proportion o f the 2 hours had elapsed).
Each subject also received directions (Figure 31) on what was required to com plete the newsletters
and where to find the text and graphic resources. The complete packet o f instructions received by each
experimental subject for Session I (pretest, treatment, and posttest) is located in Appendix F. The complete
packet o f instructions for Session 2 (delayed posttest) is located in Appendix G.
Software
Carroll (1984) advocated the use o f commercial applications for research, suggesting that the use
o f noncommercial applications is analogous to creating “toy” systems that have no real application to the
45
newsletters. Also described in this section are the instructions on how to use the online help systems, the
names o f the five different online help systems, and the evaluation instrument for the online help systems.
Four Assessment Newsletters
All subjects created (or at least attempted to create) four newsletters (Figures 27, 28, 29, and 30)
as a pretest, treatment, posttest, and delayed posttest assessment o f their expertise with intermediate and
advanced MS Word tasks. Each assessment newsletter was comprised o f five main steps:
2. Insert and format text into columns with a banner heading
3. Insert and format a graphic (i.e., resize, position, apply word wrap)
4. Create and format a table (i.e., apply borders and shading; rotate text)
Each o f the five tasks was timed for the pretest and posttests. For the pretest, subjects were allotted
the amount o f time it would take for an expert to complete the same task. For the immediate and delayed
posttests, subjects were given a total o f 45 minutes (divided proportionately among the five tasks) to
complete all five tasks. The resulting time for each task was approximately one and one half to two times
the amount o f time it would take an expert to complete the same task. Subjects were not timed during the
completion o f the treatment tasks, but were encouraged to move ahead without completing the tasks after a
specified time (they had approximately 2 hours to complete the five tasks— a lab monitor encouraged them
to move to the next task after each task’s proportion o f the 2 hours had elapsed).
Each subject also received directions (Figure 3 1) on what was required to complete the newsletters
and where to find the text and graphic resources. The complete packet o f instructions received by each
experimental subject for Session 1 (pretest, treatment, and posttest) is located in Appendix F. The complete
packet o f instructions for Session 2 (delayed posttest) is located in Appendix G.
Software
Carroll (1984) advocated the use o f commercial applications for research, suggesting that the use
o f noncommercial applications is analogous to creating '‘toy” systems that have no real application to the
46
;n»
! B«<*
01
0» ____
A 1i
ML
i StstTadcl
1 BstTadcl: >
! StatTn*k2 •
I BtiTukZ:
| StatTukJ
1
o Sales Report
i BbtTadcJ: u
Jg
j StartTuka.
i BtdTtdc* 9
| StetTukf
! BstT«*lc5
r
£
t u By-Oik r Coapay * ott coB onkaam t* kaoatd a n a* a n itaaa a id out By-orkk

o rat itaatg to* am* dn eid o i onandmk coKtiaat* aid acogtet* tU(* ooictni aid
t i root it s k tt« aorta (*%*. O nrUt par m ay*as. ByOttik a t a n red b (otora*
so<
m kkiitM M iaia kar k«odioidaaniiforiQiirrioct**ni pmxcaoi oaoooaaotaa
Uy-Orkt i n i acrtatUly paxMc* inotK oeanik aid two earn*. »* ctatgt it aandk.
u td a td « h m « i Dmd Tk km lit rtctkid *>• i t r a t o r itit
m t paxuctii t t Mtn. pndics kar t M i n o i n ora 1a total Foodfttak*
t u p tipat ora* upon* t> i BD f w i t By-onucoapBtY
paBdi OyOrkk rtKtWUtB t u rtootd aaorraut
<xn at iionraidkg or*« So*Dn*Makt£ g titatx to riit BY-Ornk
ptodtctdtatMpauttli ■* Coapaty* a t prodicsoi or
ooopaty Skct Byorkk TU dy-omi Coap»Yiociit( tkt aaotty n « tCtr ByOrak
otgai B optraoow k 1902. it orst itgtratitopporBiBc* ot t u a at (taairti atd dkmot at
iat tikxnctdr to ta l dk*»ct roaortik a a m tn c a u t o* * (08 UgiVUataproaictr ti*
prodtc* aid naming o n ta a m t o n s a d« ‘ CakTBI lu orprodiem kar Ottt
caapakju tkat ka*t kdtoic ■tdiM atitaida tgt rau ptodaok «x try-omt c into
tiaiciaifaoavaid aanut orrtttn By- Orttk kar otatgtitCt sou a t itcaarto
i aiong Scat or tit aes ito tn td iia t oyS patctitn tsst. Saw
p M t i M s t i a k k t i i or dkMiepacitor OktOnik. o rm n e t* a* kcrta*kg
Uy-Orkk itaooBcia*: Otcaafcattu DktOtnk. dn B tu tg t e o tfia tr
a tttiM .a * rtportcoitatu a atdCktny Orkk. k (kaatdtertiraiie*. it a t
otgat or piar i993.Bt dtaatd «ti a . Byo rk k i q u c tt»
or sorrdrkB (ici ar c ^ ia e t ot u c tig t a a t t
Predict Dtstto f c a . a «m kcrtand D a ita dtaatd atotgk itcaartd
ktyaoeot itmr 2Tp*IC*lt pndtcttalt*.
Uy-Orkt piodtcttki co u * * or ornttortorkk a a m t ly
aioitonik aanataid a t>od 1996. a* dkt — • drkk
n a ik t Tit n oo a a o tt* stgat it* tottcaino tiutddQ ptict itor
k »a« d pm anv « n tl * Qt (Ottdrlik lidary. fortcar* n e t a*
prodtctDi ornituict*. Tit a « t m kctttt c o tc tru c o u ta ts last
Tab la o f Qoaxtexijr Sala*
0aartadi (alt* ly I n t e l m l Rada*

C S aT Gttt I Hornet Totals
p ScdCndu 6 7 6 | 19
*
doddle
Food
3
2
5
4
4
4
B8 U
U
H B n iM f “ I T 16 14 r r ™ ....
Fiuure 27. Pretest newsletter.
47
ID# 123
Date: 60899
S tsrtlfck I: 1*5:36:54
Bui T « k l: 16*139
Start T*k2: 1*5:4155
Bui T * k2 :165041
Start Tfck3: 1*5:3>48
Adventure
Bui T«k3 465348
Start Ifckd: 16:56:43 T ravel
B ulT «k4:170959
Start B dc5:17:10:14
Bui Tadc5:17:1045
R«iew:
Freeport—Island o f Action and Relaxation

Omd&tWA irtla ham* otf hveUdterhstaiief mdidattebasly
fte tpojfiyfjgj. fluensmt sacend
cay.w th r&oerpece w ieaare
camopaUtsi uo flp b n flwttoe old- WBt enemeraldocemtat anprfuptbe
w a ll scphsticaicri cfNasrai. ciledfee mostu r idly ajlarflin dl of experience3
the*l*ris.»d4l*dscspe a ra ta h
fteepaitduceyaiaiU M . cenbe fbwd w «vtoictroa The DalffcinE<p«i«ce afirs inumber
whole of Gtand Bdume. Du* _ 9 - oodanrtsons to deserted of irurulixepiDgwni wahits pod of
tslm diselfhajbem ' s*eds.mdun$d*mce4» of Attatjcbottlrose doymrs.The CLOSE
settled tb rce rtu iis. ehiccavets.QraidBibjen ENCOUNTERt i t esyouby ferryboat
piotridrg elastic _ hoUsthepotatieUbx s a y sea <to*nft*«*ih*hort afthe islsxlto
o t nnmenr-- tfyotrwwt ”af Ceribbonwecttim. Serunmy B yjinm e afthe dnfchins.
a re-to y o arstiy .Q w st Then ■501 cm dbtm* deesewadedul
HenntungweyalLentespaxid TJm Sfcw&Beedi Resort.®! animals cbseqp .Ttkepiants. Hex an
the sm dy-fbaedbes an tis M aiuis locoedandue souftwectcoart Btensttogtakbydre aunal ceresta£
tsl»d.The easternmost edge of of GfcwdffetotH Masd and x comafthe Adtipesttns.Thx is a t e swim-wih-
theK lstdhK beexhexdtidivt dose* ofelliclnl detonators. thed&tteitpropeB .batpoanaytlan
tn n rey deeded. Citanri Beheaa Q&gW ftbmss todtenpottxes theds& patjo'rfoLinftttntttrand
holiith e potertial&r or a y soot i«eeiboBtheaid-70noftubw90^. elbwttndcfchersto canetp mdtaadr
of Carfcbeanvwaim. The ^M feBaadrRmtis a yaa.QrsH> aao ufoDavwadiae
The towns aflhe WestEhd, p la te n end itgaa.wlhthe d o )n is h
whididtmQHnhbtmwere wesfemdgto-fggBL. thew*er.PlMy afr s ts tsdr^
rhjdeatt fan»m-ititn«4iar« m wferaMinri.You MDSTbrhgyjur
oU'Worlddum . bitMpoki afthe At iEsw ttof'fl *«*« ta n 49 specious camera. Price is aar«Jy$36 perpastxi.
tsl»d.(ru cmfindnsmwds ofHie on-btdnm staw « i B7 sia(UD«s hdbodec b o ttn lt. tboti S h ras.
islxids tsrlest. &3K& axflm iax. ellraw^rwcrcated; ellafertogbeicMes
wdthe mare recert rtmnmby a rt drawnraim s.Seen es below.
p rates. CSsud Bjfciflwdoe* htect
Xanadu Resort
R oom D escrip tio n R ate

- StqMdor $125
Deluxe $155
“ VQk Two-Bedroom. $325
Figure 28. Treatment newsletter.
48
ID# 133
Dd*: 4/1&09W
Hope
Stilt T u k l: 10:40*10
Bi4T*ikl: 10:4435
StntT «ik2:10:44:40
ai4T *jk2:10:40 35
StartTaikl: 10:4*00
BnlT«ik3:10:5225
StartT uk4:10:5230
Oil T «k 4:10:3805
StartTaikS:10:58:10
Bid T*ik5:11:1425
Bacardi Neva ... Tm Ca Um !

U « ie 0 4 r)%*(l^Hlt«lpiMi «b »*)te
t v » iM ta r f io la i# k w rcp eU A
C I Mllutytatlmtnîul
p*t»at «a 6*s&t»xi» teUARUudDti* j i6utt»«bx«Me»« Skywcateu.
avis •tamf<fa*l*xpmal4arti«kaftM SwiulHiul laynutlk k
itpz&uttV antaaar 0i»rtahaKaptyadMii. 1 kav
4fix ^4a £1 tea «aaktafcaaar ak*4»campas. W
s»tlirfc&t» a v i s . i4»w ( a rittftaajaiMautliiyr
lit tl* > atuat tet I1&w» jteartmctaaaart (iuiKayi
iMibit <mfV\iw fed £ tMMaal kCiaSvMyteaWlt ia
4a4tf»1^pW)BC £• M m i a t Aantflaulbi.'
a 4a afriatet
"(AXAfiAfSMa. im-vimf M t» H n i
ktaiax Bk«i p»um» nidiip teraa Ck»ia4»c»t<
tU iM l »fficu*a vttlmyi u tm . tyrte** ate
I&»SAfojialifujy; • a —ja tte r kia <«ik4«te|atbm&iaaa teaa4ac«te.
♦2Yc*f p4tAStf*04*11BAT a k a te Caaaakzaaasaac a£a#:t<a*k atdmr
adAttWai 4ft iflBOtBW ft *vn»&it ayataftctte ftaitft tta u u # VW
a*<£x*a»&«&4a\&ty ah. (aaaa^ a a y k*te]anks 4*str^crfiaa te
*Bm*£k**2 <«BUBAfBfi2ft91 Pto. tern * a « y artMm<V>Crtaclay aaraljaar* Kara*
4* a6aBt»»A&4ftAH»mSMa44l]k «ai <4% &xtecM * k taK tm r aazjaaas, n l a y y
b t f i hit i f k t e « jb m w M«z«tte laciaxt
frna. a » B ia ^ > « ta a S iH t e k l t in |t lt X 4 ( |M ) k ate <a
UtA 4 k&4t ftVTWmt atffaAHih m •Ouaa4ie*teaat2te yatirriUl W |H lti< |a n a u O n t e W U
fr a a a iy illa a a . AatexM aiBUmta naa*ife*tin*«a4te«V a a c a c ik fa tk fcccfr «£Bea te » t « M t e
<iVmiT|»>V Skaa. u*i a t iiu f e i
Uxlxitou B cRImA
AM^rii <f VtRtahrDMi0yjtem*G«Ufc«g)
VDitfi Mild nrlm n A t tevularlM f?

DIET ANALYSIS IMPROVABLE?
BarohrHSls Dirt. MfceatyWt nfaimiM No
1 Dr.Atkj»N«rDirt Mitfiuc leu cfn tn kx No
at H L o rfttlta t Il&rtOTinfciE Qstbrowiiici
o T-ftct* M ssiic totprt. *k»e.E Prooidrt WWBDA
Figure 29. Posttest newsletter.
49
ED# 123
D dc:6A & t9»
S tart T ta k l: 1040:00
B id T aak l: 10.4432
S tart I * k 2 : 10:44:40
B u lT ad c2 :104335
S tart 1 fc lc 3 .10.40:00
B td T « k 3 :105235
S ta rtT U c 4 :105230
f i l l T « k 4 :105305
S tart T W c5:105010
B ui T * k £ : 11:1425
V U n t a t iM n n M ^ r h y r t m ilH iiu i
RafchBndgtsh* agm stdaftw

csgsrrstwniiyccr
Wt an rttngtw hcfoar c o iro ra ^ v tm
g o tff afOTimtmbtrrtoilTOrt mitmbtrstD lifitest
aabmcBdtocamatayov port* coma iso tbtir
hamttartitfety
wptt±m.lhtst k f ^c oSn pt *k «Sd d ^ ^ ^ ta r td w w f f lb » p n « f l* d b 7
w htgahtt® *' Gtapi.
ptscmtluh) ttcanl Alhoogtftritcctbaccnt lifelong
Kart m lajarwil psojt<l. carofiHrg>«l*to caatrxltwt.
irten * x i MID haJfafththomK iitadicA c* ana how
bomtaftcy. Amfly
Qnaafthtao* am basaf goqpc.but tftta tn n zK ocans m
impoitat.5 C«ps oar ©caps. mowt* fit* p o s t . tachpocp a m H a ty
thiywillbet8g«iig M alta n p t a l? « I5bcm*s h th a n a i
wHlbt ftmiEtrtwlhyxng dnUrat. M trfcttt tw*y«arp«no<L/ifffirt.p o n d artr.4u
Acadtrtal dnfiifiryoong dtitirat wpaarttfwtwfittoli plaxktontum tD a smallerprop ot
:itrr\gdKtcs&rthK*
irthe grwtflMlwtfoqrtodlto. suresudnpatiqpang npaaxBtMBStoba andhblt laiyum*.
Ra^H ysfiutnthoflutnfflibei anitxprdhgajTTUiLbtr Otr stttslctl depsnmetwilbt trading
h*a*oiytot*Hoihcw«fity dvokzutB . Oaswfll tht «4tct afth* projta. Hi* Safety
irtpt<tiat»hart OT<dlr«s. Hum grtu* atojpailmiytD h duatat Qi±* cf/amcaplantD vstthz
ptoplt anm*fctro*sb«<ax«ttwy mchouttodl an vb igt aia l* *4t*nxl*tiyw*its.So
«rapwr«tiigflfe*.w*»llfflfir*o crmmnntMt.no M a r fir (vapau hitbM i aAaxasbc mmols
grtcîoiLAlhaaûtxbafin howfeabftftMyaaybt. &—tffladtlVtiic* anaoBo fistUt 7 «r
^ fta ^ rtu k ttta rtU D O R Tteawfflbtvaao, is “Bpfegtadxcthrto safety."
nspairart. «nfl*ltfiw toB y
SchaMtaf fesptdfan
City Sections
North W art East South
1 fen Fab Bfir /p H
% 2 M*y tan Mr Aug
p
o Stp. 0<t Mar Dtc
3
Figure 30. Delayed-posttest newsletter.
50
Task 1: Use regular and shaped TEXT BOXES to cresite heading
Start:
Position your cursor after"Start Task 1” and. insert the time from Invert | Date and Time.
Use the very last time format (14:30:25).
I. Create a text box spanning from, the harder o f the dateAime text box to the right margin and
as long as the date/time text box (i.e., align the battomborders o f the two text boxes).
2. Type and center " S a le s R q r o r t " inthe text box.
3. Make " S a le s E e p a r t " Anal, bold, 28 point.
4. Apply either a texture or pattern fill to the text box.
5. Apply a special border to the text box.
6. Apply to the text box atight text wrap with a 0.1 external margin.
7. Create a *star* AutoShape text box
8. Apply a different texture fill to the star.
9. Place the star over the “Sales Report” text box as shown.
10. Make the 'W e're #1" text white, Arial, 14 point font, bold, centered.
I I . Make a text bac with rotated text as shown inthe sample.
12. Apply the same texture and border as the “Sales Report” text box.
13. Make the text Arial, 12 point font, bold, centered.
14. Positiontext box over “Sales Report” text box as shown.
Stop:
Position your cursor after“End T ask 1” and insert the time from Insert | Date and Time.
Use the very fort time format (14:30:25).
* SAVE your document
* Wait far the lab consultant to give you Task 2.
Figure 31. Pretest Task 1.
51
real world and real problems. M S Word 97 was selected as the software application for this experiment
because it is a commercially distributed and widely used word-processing application. Its prolific use will
increase the generalizability o f any findings to a large target audience. The use o f M S Word also increases
the available sample size (i.e.. many people are interested in extending their expertise in a word-processing
application).
MS Word 9" Tasks
Carroll (1990) criticized the use o f artificial tasks in research as being nonrelevant to the subject.
The tasks chosen for this research were identified as either intermediate or advanced skills in the Kelly
Testing & Training Management System (see Appendix B for a list o f basic, intermediate, and advanced
skills). They represent functions o f the software application that are not normally used by the typical end
user, but are requested by employers and would increase the end user’s productivity. The subjects in this
research were advised that learning these tasks would make them more employable and effective in their
work settings.
Subjects completed two tasks for Experiment 1:
2. Insert and format text into columns with a banner heading.
Subjects then completed the following three tasks for Experiment 2:
1. Insert and format a graphic (i.e.. resize, position, apply word wrap)
2. Create and format a table (i.e.. apply borders and shading; rotate text)
It is important to note that all five tasks were completed for each newsletter activity; the subjects
were unaware that they were transitioning from one experiment to another within the context o f the same
newsletter.
It should also be noted here that Mirel (1998) suggested that some tasks that technical
communicators define as complex (she specifically refers to creating and applying print merges) are not
complex tasks because they are not open ended, situational, and contingent She defined complex tasks as
52
those requiring problem solving or the creation o f new perspectives for novel purposes. For the purposes o f
this study. I chose to define complex tasks as complicated procedures that require an intermediate or
advanced level o f expertise.
Instructions for Using Online Help
The lab consultants provided instruction and demonstration on how to navigate through and use the
online help system. Subjects were provided a copy o f the instructions (Appendix H) to which they could
refer at any time during the treatment.
Online Help System
Subjects had access to one o f three different online help systems to complete Tasks 1 and 2 o f the
treatment newsletter (Experiment 1). The three different online help systems were the skeletal, elaborative.
and inferential.
Subjects had access to another o f three different online help systems to complete Tasks 3 .4 . and 5
of the treatment newsletter (Experiment 2). The three different online help systems were inferential-without.
inferential-distant, and inferential-proximal. (The inferential-without system for Experiment 2 was identical
to the inferential system from Experiment I .) The online help instructions were the only instructions
available for completing the tasks.
Help Design Evaluation Questionnaire
After completing the second session, subjects used the Help Design Evaluation Questionnaire
(HDEQ: Duffey et al„ 1992) to evaluate one o f the two online help systems they had used the previous
week. This eight-category assessment tool (Appendix I) was designed, developed, and tested by Duffy et al.
(1992) to measure the usability o f online help systems. It measures an online help system's support for
problem representation, access to help, topic selection, ease o f scanning help, appropriateness o f the type o f
help information presented, comprehensibility o f help content, ease o f navigation, and ease o f transferring
information from help to the application. The HDEQ allows the omission o f specific items that are not
relevant to the given online help system. The “Access” section was omitted since the research had no access
to Microsoft’s proprietary code for Word and was therefore unable to create context-sensitive help for the
application. Scores from the HDEQ were used to measure subjects’ evaluation o f the various online help
systems. They were also used to compare subjects’ evaluation o f the online help systems with ability to
perform tasks in those systems. The purpose o f the HDEQ was to assess which style o f help users preferred
(as compared to measures indicating which style o f help was more effective).
Treatment Design
The experimental design selected for this research was the pretest-posttest, 2 x 4 factorial design.
Six specific hypotheses were tested:
1. Inferential help (I) will be more effective in teaching software application procedural skills
than skeletal (S). which will be more effective in teaching procedural skills than elaborative (E) online help
(i.e.. I -■S > E).
2. While learning procedural tasks, subjects who use the skeletal help (S) will complete tasks
more quickly than subjects using elaborative help (E). who will complete tasks more quickly than subjects
using inferential help (1) (i.e.. S < E < 1).
3. Subjects who used the inferential help (1) to learn tasks will complete those tasks more quickly
than subjects using skeletal help (S), who will complete tasks more quickly than subjects using elaborative
help (E) (i.e.. I < S < E).
4. Inferential-proximal (P) online help will be more effective in helping learners identify and
correct errors than inferential-distant (T) online help, which will be more effective than inferential-without
( W) online help (i.e., P > T > W).
5. Subjects who prefer a verbal style o f learning will complete tasks more accurately and
efficiently using skeletal online help.
6. Subjects who prefer a visual style o f learning will complete tasks more accurately and
efficiently using elaborated online help.
54
Pilot Studies
One usability and two pilot studies were conducted to test the treatments and procedures. The
usability study was conducted over a year prior to the actual experiment. The researcher and two assistants
each monitored a different subject, drawn from the sampling pool, as they completed the five tasks selected
for the experiment. Entrance qualification criteria and treatment modifications were based on the findings
from the usability study.
The first pilot study was conducted at the same time the experimental treatments were sent out for
an external review by Laurie Kantner from Tec-Ed (see Appendix J). The pilot study was administered to
11 subjects under the actual test conditions for this study. Modifications to the lab-assistant procedures,
subject experimental materials, and treatments were based on the findings from the first pilot study.
The second pilot study was conducted immediately prior to the experiment. The pilot study was
administered to 6 subjects under the actual test conditions for this study. There were no problems. Data
from four o f the subjects were used as experimental data.
External Review o f Treatments
Laurie Kantner, o f Tec-Ed, completed iterative expert reviews o f the treatments and the subject
experimental materials. Kantner and colleague Stephanie Rosenbaum teach workshops on the
implementation o f minimalism.
Population Sample Selection and Assignment
Carroll (1984) criticized the use o f undergraduates as experimental subjects, suggesting they are
compliant and lack real-world experience. The 1 11 subjects for this research were obtained from the clerical
pool of a local temporary employment agency (Kelly Temporary Services). These subjects were very
ty pical o f the target audience. Clerical employees are predominant users o f word-processing applications.
They may have an intrinsic motivation to gain expertise in M S W ord in order to function m ore effectively in
their work.
55
Selection
Kelly Temporary Services (KTS) routinely screens all applicants according to their expertise with
word-processing applications. Subjects from the clerical pool received monetary compensation from the
employment agency (who in turn billed the researcher). Subjects were paid only at the completion o f both
experimental sessions.
Subjects from the pool who did not qualify at the basic level o f word-processing expertise and
subjects who did not speak English were eliminated from the sample. Subjects had to be familiar with a
word-processing software (either M S Word or WordPerfect), but not be an expert programmer or hacker.
This was based on research conducted by Roper Starch Worldwide (Snapshot. 1998) that states that the
majority o f computer users (58%) think that computer technology is important to know and somewhat
confusing. This is the target benefactor o f these research findings. They are neither the upper 15% that are
programmers nor are they the lower 15% that are technophobes.
Stratitied-Random Assignment to Groups
All subjects were assigned a Kelly Service ID number upon selection for participation. They used
that number on all testing materials throughout the study in order to protect their anonymity. Subjects for
each Kelly-scheduled session were randomly assigned (by the researcher) to both the first and second
experimental conditions based on their visual score from the VVQ. Subjects were ranked from highest to
lowest based on their visual score. Three different coins were then used to randomly assign the top three
subjects to each o f the first three experimental conditions (skeletal, inferential, elaborative) and then again
to each o f the second three experimental conditions (inferential-without, inferential-distant, inferential-
proximal).
The subjects’ KTS ID numbers were placed in a box and shaken. A single coin flip and a draw
from the box were then used to select which online help system each subject would evaluate using the
HDEQ (Appendix I). For example, if three subjects were using the combined skeletal/inferential-proximal
online help systems, then their ID numbers were tossed in the box. The coin flip determined if the first ID
drawn would evaluate the skeletal or the inferential-proximal online help system. The next ID drawn would
R e p ro d u c e d with perm ission of the copyright owner. F urther reproduction prohibited without permission.
56
evaluate the other online help system. The remaining ID was paired up with another user o f either the
skeletal or the inferential-proximal online help system, and the procedure was repeated.
Subject Attrition
Subjects who did not complete a full session or who failed to show up for the second session were
eliminated from the study and were not paid for any time they had already invested. Fifty-one subjects went
through the preliminary, 2-hour qualification, registration, and scheduling process (unpaid) at Kelly
Temporary Services, but never showed up for the experiment. Twenty-two subjects completed all or part o f
the first session but either left before completing the first session or did not show up for the second session.
Four subjects were so frustrated that they were excused from the study. The data from 33 subjects were
accidentally erased by the computer lab manager. The data from the remaining 111 subjects was used for
this dissertation study.
Table 1
Reasons for Subject Attrition
No Show No Return Incomplete Excused Technical
^ •%
51 13 9 4 JJ
Use o f Lab Consultants
The same four, paid lab consultants were used for the entire data-collection phase o f the
experiment (which lasted 6 months). Three o f the lab consultants were selected specifically for their
troubleshooting expertise in the microcomputer labs: the fourth was selected for her speaking ability (she
welcomed the subjects and read all the directions) and troubleshooting expertise.
AH four lab consultants were trained. They each completed the same activities required o f the
subjects, with the exception that the researcher was there to guide them if they ran into problems. The lab
consultants were blind to the hypotheses o f the experiments and the differences among the help systems.
57
For the most pan the lab consultants, not the researcher, interacted with the subjects (the few
exceptions were when there were subjects who had to be excused from the experiment because o f
belligerence or total frustration and inability to continue). The lab consultants monitored the subjects’
activities and intervened when the subjects had technical difficulties or general Windows questions; the
researcher monitored the lab consultants to make sure they were not providing instructional support to the
subjects (the researcher’s windowed office was in the back o f the lab).
Experimental Procedures
Subjects, in groups o f 9 to 15, were schedule for two experimental sessions, one week apart. This
section describes the presession. Session I. and Session 2 activities.
Presession Activities
All sessions for this research were conducted in the same College o f Business microcomputer lab
at Utah State University. The lab consisted o f 27 Dell. Pentium II computers connected to a LAN.
Prior to the beginning o f the data collection phase o f this research, all computers in the designated
lab were "ghosted" with an image that included the online help systems (the treatments) and access to those
help systems from the S ta rt | P rogram s pop-up menu. “Ghosting” refers to the process o f configuring one
computer and then copying an image o f that computer (including all hard drive contents and pointers to
server applications) via the server to all the computers in that lab.
MS Camcorder©, a Microsoft computerized screen capture software, was installed on each
computer. Camcorder was used to record subjects’ mouse moves and keyboard strokes as they completed
each of the experimental activities. The resulting data were saved as an .avi file.
Subjects completed the qualification testing, the background survey, and the W Q at the Kelly
Temporary Services (KTS) facility. A KTS person would either bring that data to the researcher, or she
would pick it up. The researcher ranked the subjects according to their visual score and then randomly
(using pennies) assigned them to one o f the three experimental conditions for each experiment. The
researcher assigned each subject to a com puter (in order to space the subjects out) and then created an
58
assignment sheet (Figure 32) displaying each subject’s assignment to a computer, lab assistant, and
experimental conditions.
Just prior to the arrival o f the subjects for each session, the researcher removed the access to
IYard's online help (FI and the Help menu) and disabled the Help wizard. The text and graphic resources
for the session were downloaded from the server onto each computer: Camcorder was launched and placed
in the upper right com er o f the screen: and Word 97 was launched, set to Page Layout, and set to 80% view
Uo allow easier visibility o f the Camcorder button).
9/11 and 9/18 Assignments

Please seat subjects at assigned computers both times. Thanks!
EL____________ ________ __________ __________ ___
Lab Consultant Croup # Computer # Subjects Help Systems
Troy 1 27 231 A/F
25 227 A/D
24 228 A/D
Tommy 2 15 219 B/D

16 223 B/E
19 224 B/E
20 181 B/F
G tstehaa 3 11 203 C/D

12 220 C/F
13 202 C/F
James 4 2 225 A/E

3 181 B/F
4 243 BD
6 230 C/D
□
Figure 32. Computer-group assignment sheet
59
The researcher arranged the materials for each subject onto tables for fast access by the lab consultants. The
materials included a general description o f the session activities, samples o f the newsletters, and instructions
for completing each o f the five tasks.
The researcher then wrote the following general instructions on the front white board:
■ The lab consultants are not allowed to help you complete your tasks. Please d on’t ask them.
■ All text and graphic resources are located in the C:\Newsletter folder.
= Undo
■ A = Font color
Session 1 Activities
The researcher greeted the subjects in the hallway outside the computer lab. checked them in
according to their KTS ID number, had them sign the KTS time sheet, and organized them into groups
according to their assigned lab consultant. The researcher then introduced the subjects (by ID number) to
their assigned lab consultant and turned them over to the lab consultant for seating.
One lab consultant greeted the subjects, read the "Welcome/Overview” materials (Figure 33), and
guided them through the initial open/save/time procedures (see the complete Packets in Appendices F and
G). Another subject demonstrated the open/save/time procedures on the demonstration computer, while the
other two lab consultants assisted subjects.
When all subjects were ready, the lab consultants distributed Task 1, set the timer, and told the
subjects to begin. At the completion o f Task 1. the lab consultants picked up the Task 1 instructions and
replaced them with instructions for completing Task 2. This procedure was repeated until all five pretest
tasks had been completed. This portion o f the experiment took approximately 45 to 50 minutes.
At the completion o f the five tasks, the subjects were excused from the lab while the lab
consultants launched the assigned online help system for each subject and distributed the directions for
navigating the help systems. The subjects, meanwhile, were treated to some sort o f refreshment (e.g., sweet
breads or cookies).
60
Overview
Welcome>Thank you fo r participating m this study.
This orientation packet is designed to guide you through the activities for this session. We will
preview the session activities as a group before you start You may refer back at any time to the
instructions included in this packet
Session Activities
During today’s session you will complete three very similar activities using different content, but
comprised of the same five tasks:
1 Create and format regular and AutoShape text boxes
2 Insert text; format it into columns
3 Insert graphics; wrap the text around them
4 Create a table, format it with borders, shading, etc.
5 Use mail merge to create mailing labels
All three activities you complete today resemble partially-completed newsletters. A description of
each activity follows:
» Create a pardon of a sales report (see enclosed sample): m e mail men*e to create mailing
labels. (35 minutes)
The intent of this first activity is to introduce you to intermediate- and advanced-level skills m
MS Word that will make you more employable You will attempt to complete five tasks in M S
Word without any help. You will have 35 minutes to complete this newsletter/sales report
Note: Although each task is timed, feel free to work at your own pace Note: It is not expected
that you know how to complete each task—these are intermediate and advanced-level tasks m
M S Word It's OK if you do not know how to complete them now—you will be able to use the
online help system to complete these same tasks in the second portion of this session. Just do the
best you can.
* Create a newsletter; use mail merge to create mailing labels. (approximately 2 hours)
________This second activity is to enable you to learn the same MS Word skills using an online help_____
Fiuure 33. Partial introduction page.
After the subjects were again seated, one lab consultant read the online help navigational
instructions (Appendix H), another demonstrated the procedures, and the other two assisted subjects.
When all subjects were ready, the lab consultants distributed Tasks I and 2 for the treatment
newsletter and told the subjects to begin. The subjects were told they had approximately 2 hours to
complete the five tasks. The time was divided proportionately among the five tasks. One lab consultant was
in charge o f the time and, at the end o f the identified time for each task, would encourage the subjects to
move to the next task even if they had not yet completed the task on which they had been working.
61
When a subject completed Task 2, he or she was instructed to raise his or her hand and notify a lab
consultant. The lab consultant then stopped Camcorder and saved the resulting .avi file, exchanged Tasks I
and 2 for Tasks 3, 4. and 5. closed the first online help system, and launched the second o f the two assigned
online help systems. At the completion o f Task 5, the subject was given the opportunity to review any o f the
procedures again using the online help system. When subjects felt comfortable enough to complete the
procedures without the assistance o f an online help system, they again notified a lab consultant. The lab
consultant stopped Camcorder, saved the resulting .avi file, and picked up Tasks 3, 4, and 5 from the
treatment newsletter.
The lab consultant then, on a subject-by-subject basis, distributed Task 1 o f the posttest newsletter
to each subject, recorded the start time for that subject, and told the subject to begin. When the subject
completed Task 5 o f the posttest newsletter, he or she was thanked for participating, reminded that payment
was contingent on completion o f the next week's activities, and reminded to sign out on the KTS time sheet.
At the end o f Session 1, the researcher moved all the .doc and .avi files generated by the subjects
to the server, then moved them once again to the demo computer, where they were then burned onto CD-
ROMs. It generally took one CD-ROM per subject (see Figure 34).
Session 2 Activities
The researcher completed all the same presession procedures for Session 2 as she did for Session
I : picked up the qualification testing scores and surveys, ranked and assigned subjects, prepared computers
000listl.doc 32 KB Microsoft Word Doc... 7/16/1999 11:57 AM

Sj34list2.doc 19 KB Microsoft Word Doc... 7/16/1999 1:34 PM
Sj34list3.doc 31 KB Microsoft Word Ooc... 7/16/1999 2:29 PM
34mail2.doc 19 KB Microsoft Word Doe... 7/16/1999 1:41 PM
S o m a lia .d o c 19 KB Microsoft Word Doc... 7/16/1999 2:27 PM
Sj34main3.doc 21KB Microsoft Word Doc... 7/16/1999 2:29 PM
0j34News.doc 638 KB Microsoft Word Doc... 7/16/1999 1:47 PM
Sj34sales.doc 36 KB Microsoft Word Ooc... 7/16/1999 12:01 PM
ssjjj34tl.avi 102,979 KB video dip 7/16/1999 11:59 AM
■sajj34t2a.avi 80,716 KB Video O p 7/16/1999 12:38 PM
■sjjj34t2b.avi 232,851KB Video Ctp 7/16/1999 1:49 PM
•=w]34t3.avi 124,482 KB Video O p 7/16/1999 2:33 PM
■JTE5TDIR.TMP 0KB TMPFiie 7/17/1999 9:04 AM
Figure 34. Components o f data on CD-ROM.
62
for the session, arranged subject materials, wrote reminders on the white board, and checked in subjects
according to their KTS ID number.
The lab consultants seated the subjects at the same computers they had used the previous week,
demonstrated the open/save/time procedures once again, distributed Task 1, set the timer, and told the
subjects to begin. At the completion o f Task 1, the lab consultants picked up the Task I instructions and
replaced them with instructions for completing Task 2. This procedure was repeated until all five delayed
posttest tasks had been completed.
At the completion o f Task 5, the lab consultants stopped Camcorder and saved the .avi file,
launched the online help system the subject was assigned to evaluate, and distributed the Help Design
Evaluation Questionnaire (Appendix I). After completing the HDEQ, each subject was thanked for his or
her participation and reminded to sign the KTS time sheet.
At the end o f Session 2. the researcher again moved all the .doc and .avi files generated by the
subjects to the server, then moved them once again to the demo computer, where they were then burned
onto CD-ROMs.
Procedure for Coding Data
The A VI Player was used to watch the .avi video files on the computer. Five different checklists
were used to code each subject’s data. A research assistant was trained to assist the researcher in the data
coding.
AVI Plaver
Dr. Thomas Hardy, Utah State University, created the A VI Player, the tool used to play the
resulting .avi subject data files. Camcorder produces unique .avi files that even the Windows Media Player
would not play. Camcorder has only Stop and Play buttons, requiring the user to start over from the very
beginning if it were necessary to stop for a phone call or meeting. With approximately five hours o f
recorded data per subject, that option was unacceptable. The A VI Player had regular VCR-like controls plus
a frame-by-frame forward/backward option, and an invaluable option for starting at any given frame.
63
Dissertation Data Sheet
The starting and stopping times (Figure 35) for each task were recorded onto the Dissertation Data
Sheet. The online help information (e.g., features accessed and available) and the subjects Verbalizer-
Visualizer scores (Figure 36) were also recorded here.
Activity Tasks Checklist
Task completeness was measured using the Activity Tasks Checklist. The same five tasks and their
associated steps were listed for each activity (Figure 37). There were 28 steps total to complete each task.
Subject received a check next to the step if it was completed, even if the subject did not use the correct or
the most efficient means o f completing the task.
Help Tasks Checklist
Accuracy was measured by comparing the basic steps for completing each o f the tasks (as
identified in the online help systems) to the steps the subject took to complete the same tasks. The
procedures from the online help systems were compiled into a separate Help Tasks Checklist document.
Where a different way o f completing the task was actually more efficient but was not consistent with the
fc m trfn w m tl D tre ti 1 / m t 1 1 D IS SE R T A T IO N DATA S H U T S ubject*___
L ab C aiw u lttrtt/C n n m * tfc ls S ts tm u fA -F u C n t a D
PRETEST
A c tk itg h n m i f a k i A j k l i A ccuaK r
T m ik F tm t S ta r t Stop T o to i 96 C onpL Pom Co XT % Extoes / * C o st R oc. T i m
C reate a b a n n e r h e a d in g ( te x t b o x es)
A d d a n d fo rm at text
A d d a n d fo rm at p ictu re
A d d an d fo rm at table
C reate a n d p n n t m ailin g lab els
To tale t
T R EA TM EN T S
A cthdty 2 1N e w le tr e d la W b Ac c u b k t
T ask Fnm S to rt Stop T o ta l % CuiopL Pom C o rr % E rx o c * / % C a r R oc. T i m
C reate a logo (te x tb o x e s )
A d d a n d fo rm at tex t
A d d a n d fo rm at p ictu re
A d d a n d fo rm a! tab le
C reate a n d p n n t m ailin g lab els
f o ta k t
R rrin rt
M A ST ER Y PO STT EST
ill liiipi 1\ N n w h ltir f f olw li i | A c c u to tr
Tuk ham S tu t Stop T o to l I 91 CmopL Pom C o rr % E x x o rs / % C o e r R oc. T n m
. - £ » t o a o .h i n u o r hoobim ftffoH hnT or) L _— ...
Figure 35. Top part o f dissertation data sheet.
64
HELP FEATURES/ERROR INFORMATION

Tonic* Aco. Fniuaf Uoii % hato ilW
C I R • c D E e H L N 0 R T W AM * %
Vinal U..U1 lunfccQuSiniitt BUpSnfemTn*/ Scot*
Fttfiw:
daCmiiai H-Om
I>Dkfiraiian. 7*Troahkal»<rt(»cBtttim
E*Emr In&eynetim N-Note W-Poo-u#cipMt(vA at milumld—..*)
cnpiacCcEdthsia*»** ) OOm YourOwi
Accm:
C iC o ttn * W iukx R>RikfedT«)« ♦»
Fiuure 36. Bottom part o f dissertation data sheet.
! Activity 1: NewsletterA.aDeis I
i i Task 1: Create a banner heading Task 2: Add and format ted Task 3: Add and format picture |
I I 1 Create a regular textbox l Insert text file l Insert a p tc ttse 1
i 2 Create an au to sh g re textbox i Format first h ae into heading; form at type 2 A pp h rftg h f text w rap
ZZJ 3
4
E nter an d form at text
Rotate text m text box
3
4
Form at text m to three colum ns; 2 .2
Form at lest Ime into heading; form at type.
3. Edit w rap around d a n k
4 P o sitio n picture
5 A pply eith er a texture o r p a r e m M
1 16 A pply a special b order
| 7 A pply a fight text text uoqa
I 3 P ositio n text b o ats
__ ! Task 4: Add and format table Task 5: Create the mailing labels
l Create table I Create moil m erge docum ent
2 Form at h o rd e s 2. Create d a ta source |
3. F oim at shading 3. S etup d o cu m eg !
4 M erge cells. 4 S ort the labels a asctndm g order.
5 R otate text i 3 M erer th e labels to a r a r Document.
6 Align text vertically
7 Align text h crtro n taily
Fiizure 37. Partial Activity Tasks Checklist.
main approach used throughout the help system, then both methods were listed (see Figure 38) and the
accuracy score was based on whichever method the subject used.
Error-commitment and error-recovery information was recorded on the lines below each task. The
number o f errors the subject successfully resolved were divided into the total number o f errors committed.
The amount o f time subjects spent on error recovery was recorded to the nearest 30-second interval.
Research Assistant
The data from three different subjects (selected because they increased in level o f data coding
judgment) were used to train a research assistant. The researcher first coded the data and then had the
research assistant code it. The researcher and research assistant discussed the differences and agreed on a
65
To cbaaga tbaappaarwiret of a tart box

1 RiQrt-eiickonthetodboxbontet(ot did*onFornet}
2 CldconForm at Text B
est.
3. Selectasobdfil ColoreraspeoalBiEttoct
4 SeleasoMerpatternedUneColor.
5. CldcontheUfrappngtabandselectatextw *apopfton.
O
R
V Selectteetedbox
2. DisplayOrbingtoolbar
3. Usethefill Coloricontochangethetextboxfl.
4. UsetheLineColoricontochangethetextboxInecolor.
Resolved? Tim
e
Erron Start 9tco
Figure 38. Partial Help Tasks Checklist.
coding rubric. This procedure was repeated for each o f the next two subjects. Thereafter, the research
assistant and researcher discussed any new coding situations and added them to the master coding sheet
(Figure 39). The mean interrater reliability rate was 96%.
Procedure for Statistical Analyses
The two statistical analyses used in this research were a multiple multivariate analyses o f
covariance followed by analyses o f variance, simple effects and pairwise comparisons for statistically
significant findings. The multivariate analysis o f covariance was selected in order to provide a benchmark
comparison point when conducting a more complete and detailed description o f differences among the
groups, especially with correlated variables (Stevens. 1996). The analysis o f variance is the appropriate
methodology for “describing the relationship between a continuous independent variable and one or more
dependent variables” (Kleinbaum, Kupper. & Muller, 1988. p. 12). The Bonferroni approach (Green,
Salkind. &. Akey. 1997) was used to prevent Type I error. Wilks lambda A was the statistic used to
determine the multiple eta squared ( q - ) effect size.
A repeated measures analysis was conducted on an exploratory basis to provide a different
perspective o f the data. The repeated measures methodology prescribed by Pedhazur and Schmelkin (1991)
was used as the guide for this analysis. The exploratory analysis was conducted to determine if any learning
occurred over time.
66
hzxcenmentai Dales: / / ana / / H elp T a sk * C h e c k lis t
1 2 3 4
To create a textbox
t Click on Insect I Text Box or.
Click on the test box icon
2 Click, hold ana drag the crossbars la me desired test bos size.
Rl
Errors
a Count as 3 steps irS creates a rectangular AutoSnape ana m en changes it tc a lest bos
j
; 3 can form at ane test box ana then cocv ana cnange :t w ith out error (prevents having to ‘armat twice)
t ; - n o r if S clicks anc men arous test aos reserved after S resizes it
|
i
I To c r e a t e an AutoShape textbox
1 Click on me Draw icon on me Standard toolbar to display the Drawing toolbar tor insert;
I 2 Click on me A utoShapes menu on me Drawing toolbar car Picture » AutoShaoesj
'! I 3 Select an AutoShape

i
4 Click, noia and drag me crossbats to me desired AutoShaoe text bos size.
i I 5 Right-Click on [he AutoShaoe and select Add Text tor Test button on Craw toolbar)
Fimire 39. Partial coding sheet.
67
CHAPTER V
RESULTS
The general purpose o f this research was to define ways in which minimalism, an instructional
online help systems. Two separate experiments (using different sets o f online help systems) were designed
to evaluate different principles o f minimalism. The first experiment focused on comparing variations o f
minimalist instruction for increased learning. The second experiment focused on identifying the effect that
presence and location oferror-recovery information had on subject errors. General error-
eommissiorv'recovery data were also obtained for Experiment 1; instructional-effectiveness data were also
obtained for experiment 2. Post hoc analyses were conducted for only significant findings from the
multivariate analyses.
A related purpose o f this research was to determine if a subject's performance in learning a
software application would increase if his or her preference for learning style would be matched to an
instructional approach. The two hypotheses (with associated data analysis) for this research question follow
the primary analyses.
The data analysis follows a deductive approach: description o f the sample population, help
systems, and data set are first described followed by a multivariate analysis o f specific variables by
research hypotheses. The general support (or no support) for the experimental hypotheses are presented
first followed by the data analysis to authenticate any support.
Description o f Sample Population
The 111 experimental subjects used in this study were randomly assigned to the different
treatment groups for Experiment I and Experiment 2.
The majority o f the sample population was comprised o f females aged 18-25 with 1 to 10 years o f
computer experience (Tables 2 and 3). Sixty-seven percent o f the sample population indicated that they
possessed an intermediate level o f experience in Microsoft Word or Corel WordPerfect (Table 3). Nearly
68
Table 2
Gender and Age Description
Gender Number Percentage Age Number Percentage
Male 23 20.7 18-25 92 82.9
Female 88 79.3 26-30 6 5.4
Total 1 11 100.0 31-35 6 5.4
36-40 1 0.9
41-45 3 2.7
46-50 1 0.9
51-55 i 1.8
Total III 100.0
Table 3
Experience with ComDuters. Word, and WordPerfect
Exp Number Percentage Word Number Percentage WP Number Percentage
0.0 1 0.9 None 4 3.6 None 12 10.8
0-1 6 5.4 Beg 38 34.2 Beg 23 20.7
1-2 19 17.1 Med 62 55.9 Med 59 53.2
2-5 40 36.0 Adv 7 6.3 Adv 17 15.3
5-10 31 27.9 Total III 100.0 Total 111 100.0
over 10 11 9.9
Total 108 97.3
Missing 3 2.7
111 100.0
69
90% o f the sample population was familiar with the WSkV Windows platform, and nearly 65% owned their
own computer (Table 4). The sample population used for this research was much more advanced in their
use o f computer technology than the sample population used by Carroll in his experiments in the early
1980s: however, the sample population more accurately represents the current target population. A detailed
breakdown o f all the demographic data by group is displayed in Table 5.
Table 4
Platform and Ownership o f Computer
Computer Number Percentage Own Number Percentage
Mac 11 9.9 Yes 72 64.9
IBM 99 89.2 No 39 35.1
3.0 0.9 Total 111 100.0
Total 111 100.0
Table 5
Deinocraphic Data bv Treatment Group
Experiment I Experiment 2
Variable Skeletal Inferential Elaborative Without Distant Proximal
N 38 36 38 36
(percentage) (percentage) (percentage) (percentage) (percentage) (percentage)
Gender Male 24.3 13.9 23.7 16.7 21.6
Female 76.3 75.7 86.1 76.3 83.3 78.4
Age 18-25 78.9 86.5 83.3 81.6 77.8 89.2
26-30 7.9 5.4 2.8 7.9 5.6 2.7
(table continues)
70
Experiment I Experiment 2
Age 31-35 7.9 2.7 5.6 0.0 13.9 2.7
36-40 0.0 2.7 0.0 2.6 0.0 0.0
41-45 2.6 2.7 2.8 2.6 0.0 5.4
46-50 0.0 0.0 2.8 0.0 2.8 0.0
51-55 2.6 0.0 2.8 5.3 0.0 0.0
Comp
Exp 0-1 2.6 5.4 8.3 5.3 2.8 8.1
1-2 13.2 21.6 16.7 18.4 16.7 16.2
2-5 36.8 37.8 33.3 36.8 33.3 37.8
5-10 28.9 29.7 25.0 18.4 38.9 27.0
over 10 13.2 2.7 13.9 13.2 5.6 10.8
Word
Exp None 2.6 2.7 5.6 5.3 2.8 2.7
Beg 42.1 35.1 25.0 31.6 30.6 40.5
Med 44.7 59.5 63.9 63.2 50.0 54.1
Adv 10.5 2.7 5.6 0.0 16.7 2.7
WP Exp None 2.6 10.8 19.4 10.5 13.9 8.1
Beg 28.9 18.9 13.9 13.2 27.8 21.6
Med 52.6 56.8 50.0 60.5 38.9 59.5
Adv 15.8 13.5 16.7 15.8 19.4 10.8
(table continues!
71
Experiment 1 Experiment 2
Platform Mac 7.9 8.1 13.9 10.5 11.1 8.1
IBM 92.1 91.9 83.3 89.5 86.1 91.9
Own Yes 63.2 64.9 66.7 65.8 69.4 59.5
No 36.8 35.1 33.3 34.2 30.6 40.5
Note. The same subjects were randomly assigned to both Experiment 1 and Experiment 2.
Description o f Help Systems
All subjects in the experimental groups completed a Help Design Evaluation Questionnaire
(Duffey et al„ 1992) at the completion o f the delayed posttest. They evaluated one o f the two online help
systems used during the treatment, evaluating that system along seven measures. The means and standard
deviations for these measures across all help systems are displayed in Tables 6 and 7. T he greatest mean
differences were associated with Content. These differences approached significance (p = .07) at the .05
level o f confidence. However, a multivariate analysis o f the help systems on these measures revealed no
statistically significant differences (Table 8 ). The subject rating o f each online help system was equivalent
to the ratings o f the other systems.
Description o f Data
Prior to conducting an in-depth analysis o f the data to search for statistically significant
differences among the online help systems, simple descriptive tests were conducted. The greatest mean
differences and variability in data among the treatment groups for Experiment 1 were associated with the
completeness variable during the treatment session (Table 9). The mean completion scores o f subjects in
the elaborative group were 8% higher than the scores o f their counterparts in the skeletal group. This
finding is contrary to what was anticipated. Other interesting mean differences from T able 9 are associated
72
Table 6
Comparative Evaluation o f Online Help Systems (at
Representation Selection Format Content
Help system M SD M SD M SD M SD
Skeletal 0.77 0.21 0.76 0.13 0.81 0.11 0.58 0.22
Inferential 0.76 0.17 0.73 0.17 0.76 0.05 0.58 0.27
Elaborative 0.73 0.31 0.79 0.12 0.83 0.07 0.73 0.17
Without 0.68 0.17 0.74 0.19 0.81 0.14 0.74 0.24
Distant 0.79 0.19 0.80 0.11 0.86 0.09 0.74 0.21
Proximal 0.71 0.16 0.74 0.18 0.80 0.10 0.71 0.19
Total 0.73 0.21 0.76 0.15 0.81 0.10 0.69 0.22
Table 7
Comparative Evaluation o f Online Help Svstems (bl
Comprehension Navigation Link to app HDEQ total score
Help system M SD M SD M SD M SD
Skeletal 0.71 0.12 0.80 0.13 0.74 0.24 0.74 0.10
Inferential 0.75 0.14 0.72 0.22 0.74 0.26 0.72 0.14
Elaborative 0.76 0.12 0.79 0.13 0.78 0.17 0.77 0.10
Without 0.74 0.14 0.79 0.19 0.79 0.21 0.76 0.13
Distant 0.78 0.16 0.81 0.16 0.78 0.27 0.79 0.12
Proximal 0.74 0.12 0.81 0.16 0.78 0.19 0.75 0.10
Total 0.74 0.13 0.79 0.16 0.77 0.22 0.76 0.11
with the accuracy variable. Accuracy scores o f subjects in the inferential group were 7% higher than those
o f subjects in the skeletal group during the treatment and 5% higher than those o f subjects in both the
73
Table 8
Difference in Subject Perception o f Online Help System
Effect F df Sig. Partial
Help Systems 0.91 35 0.61 0.06
Table 9
Descriptive Summary o f Time. Completeness, and Accuracy: Exp 1
Pretest Treatment Posttest Delayed posttest
Measure Exp. 1 M SD M SD M SD M SD
Time (secs) Skeletal 803.95 70.53 2577.97 1466.65 840.53 162.37 861.21 98.20
Inferential 813.43 62.84 2483.86 930.66 818.84 125.33 874.00 76.94
Elaborative 802.75 65.22 2547.00 835.58 803.39 132.64 868.14 84.24
Complete Skeletal 0.41 0.18 0.79 0.18 0.77 0.21 0.76 0.24
Inferential 0.44 0.14 0.86 0.14 0.75 0.22 0.77 0.18
Elaborative 0.44 0.14 0.87 0.14 0.78 0.19 0.76 0.20
Accuracy Skeletal 0.64 0.20 0.81 0.13 0.80 0.15 0.81 0.13
Inferential 0.63 0.20 0.87 0.11 0.85 0.15 0.84 0.14
Elaborative 0.60 0.20 0.86 0.11 0.80 0.16 0.80 0.16
skeletal and elaborative groups during the immediate posttest. The only other initially interesting finding
was that the subjects in the elaborative group corrected 5 to 7% more errors than the subjects in the skeletal
group during the pretest, treatment, and immediate posttest (Table 10).
The greatest mean differences and variability in data among the treatment groups for Experiment 2
were associated with the completeness variable (Table 11). Subjects in the inferential-without group
74
Table 10
Descriptive Summary o f Error Commission. Correction, and Recovery: Exp 1
Errors Skeletal 6.63 4.45 8.29 6.00 4.42 3.13 3.89 2.70
Inferential 5.41 2.53 9.70 7.62 4.73 3.25 4.68 3.56
Elaborative 6.19 3.40 9.89 6.23 3.92 2.58 3.78 2.44
"o Correct Skeletal 0.69 0.21 0.86 0.18 0.63 0.32 0.79 0.27
Inferential 0.71 0.24 0.87 0.17 0.68 0.27 0.74 0.26
Elaborative 0.74 0.24 0.91 0.12 0.70 0.31 0.78 0.26
Rec time Skeletal 194.21 142.73 515.53 426.94 131.84 138.19 139.74 117.19
Inferential 174.32 124.06 455.68 370.26 142.70 118.41 130.54 128.49
Elaborative 186.67 120.02 613.33 469.02 127.50 119.22 125.83 113.80
completed 10% more tasks than did their counterparts in the inferential-proximal group during the delayed
posttest: subjects in the inferential-distant group completed 9% more tasks than did their counterparts in the
inferential-proximal group during the immediate posttest. These findings are contrary to what was expected.
There were also some unexpected findings associated with the percentage o f errors corrected for
Experiment 2 (Table 12). Subjects in the inferential-distant group corrected 16% more errors than did their
counterparts in the inferential-without group during the pretest. They also corrected 12% and 8% more than
those in the inferential-proximal group in the posttest and delayed posttest, respectively.
In addition, m ore subjects preferred a visual (versus verbal) learning style (Table 13). The
complete breakdown o f visual and verbal preferences for learning are displayed in Tables 14 and 15. The
distribution o f visual and verbal learners is equivalent across all treatment groups.
75
Table 11
Descriptive Summary o f Time. Completeness, and Accuracy: Exp 2
Time (secs) Without 1207.58 106.63 4447.16 1223.60 1527.79 222.15 1600.87 234.94
Distant 1190.47 89.55 4359.28 1553.58 1513.61 284.98 1595.75 243.37
Proximal 1212.24 151.79 4084.68 1066.02 1523.41 260.03 1627.86 257.26
Complete Without 0.22 0.14 0.77 0.24 0.73 0.26 0.79 0.24
Distant 0.27 0.17 0.75 0.15 0.75 0.18 0.76 0.18
Proximal 0.26 0.18 0.71 0.19 0.66 0.23 0.69 0.25
Accuracy Without 0.72 0.17 0.90 0.13 0.92 0.11 0.92 0.09
Distant 0.73 0.17 0.90 0.08 0.90 0.10 0.93 0.08
Proximal 0.68 0.20 0.88 0.11 0.89 0.10 0.91 0.10
Hypothesis 1 Data Analysis
Hypothesis I stated that “ inferential help (I) will be more effective in teaching software
application procedural skills than skeletal (S), which will be more effective in teaching procedural skills
than elaborative (E) online help. “Effective” refers to the combination o f the following measures:
1. Completeness: the ability to complete the given task
2. Accuracy: the exactness by which the subject’s method for completing a given task matches
the method outlined in the online help systems
3. Errors committed: the number o f errors the subject made while completing a task
4. Percentage o f errors corrected: the number o f corrected errors divided by the total number o f
errors committed
76
5. Error recovery time: the total time spent correcting errors per experiment and activity.
Analysis o f covariance was the statistical test selected in order to best determine the magnitude o f
differences among the groups after exposure the testing and treatment materials. Therefore, the interaction
between the group and the performance measures was assessed to ensure that the group regression lines
Table 12
Descriptive Summary o f Error Commission. Correction, and Recovery: Exp 2
Error Without 9.13 3.29 15.84 9.02 6.39 6.39 7.76 6.33
Distant 10.69 4.15 17.56 8.87 7.25 5.52 7.67 4.71
Proximal 8.43 3.58 15.38 9.51 7.27 5.92 7.49 5.06
11o Correct Without 0.57 0.25 0.79 0.15 0.75 0.29 0.75 0.25
Distant 0.73 0.17 0.81 0.16 0.77 0.27 0.80 0.22
Proximal 0.68 0.19 0.76 0.18 0.65 0.31 0.72 0.23
Rec time Without 292.18 197.37 959.21 681.06 211.66 217.15 294.47 212.34
Distant 301.67 165.56 857.50 554.03 222.58 189.84 304.17 253.36
Proximal 252.97 146.21 923.51 714.67 243.24 277.29 265.95 266.10
Table 13
Preferred Learning Stvle
Preferred learning style N Range Minimum Maximum M SD
Visual 111.0 21.0 -1.0 20.0 10.0 4.5
Verbal 111.0 25.0 -8.0 17.0 6.8 5.3
77
Table 14
Preferred Visual Learning Stvle bv Treatment Group
Group N Range Minimum Maximum M SD
Skeletal 38.00 21.00 -1.00 20.00 9.76 4.80
Inferential 37.00 19.00 0.00 19.00 10.16 4.19
Elaborative 36.00 18.00 1.00 19.00 10.00 4.69
Without 38.00 20.00 -1.00 19.00 10.61 4.35
Distant 36.00 20.00 0.00 20.00 9.72 5.28
Proximal 37.00 14.00 2.00 16.00 9.57 3.94
Table 15
Preferred Verbal Leamina Stvle bv Treatment GrouD
Group N Range Minimum Maximum M SD
Skeletal 38.00 22.00 -8.00 14.00 6.03 6.14
Inferential 37.00 16.00 0.00 16.00 6.70 4.21
Elaborative 36.00 19.00 -2.00 17.00 7.67 5.27
Without 38.00 18.00 -2.00 16.00 6.87 4.88
Distant 36.00 24.00 -8.00 16.00 6.94 5.04
Proximal 37.00 25.00 -8.00 17.00 6.54 5.96
were parallel to the overall regression lines (Table 16). The analysis o f covariance procedure adjusts
dependent variable means for each group along the slope o f the overall regression line. Accordingly,
straightforward interpretation o f analysis o f covariance results requires the assumption that regression lines
for each group be parallel. The absence o f an interaction between the independent variable and the
covariates means that regression lines are parallel, that the adjustment employed by the analysis o f
covariance procedure is appropriate, and that analysis o f covariance results may be interpreted
78
straightforwardly. Since none o f the interaction terms was statistically significant, this assumption is met to
a reasonable degree, and use o f analysis o f covariance is appropriate for the data.
Three different multivariate analyses o f covariance were conducted to evaluate the relationship
between types o f online minimalist instruction and specific performance measures. The pretest scores on
the above five measures were used as the covariate to measure any differences in subjects' performance
after exposure to the testing materials.
Table 17 represents the combined output o f the multivariate analyses o f the five performance
measures listed above applied to Experiment 1 (skeletal, inferential, and elaborative) for the treatment,
posttest, and delayed posttest using the pretest measures as covariates. Only treatment was significant at the
multivariate level (F=1.92, p = .04). An intermediate effect size ( q - = .09) with a strong power (1- |3 = .86)
indicated that the magnitude o f the variance obtained was related to the treatment effects and the
probability o f obtaining such a difference by accident would be less than 15%. (Stevens [1996] lists q - =
.01 as a small, q - = .06 as a medium, and q - = . 14 as a large effect size for social science research.)
Separate univariate analyses o f covariances were conducted (Table 18) for each o f the five
measures to determine which measures contributed to the overall level o f significance for the multivariate
analysis o f covariance during the treatment (Table 18). Both the completeness and accuracy variables were
significant at the .05 level o f confidence (F = 3.37. p = 0.04; F = 3.36. p = 0.04. respectively). A medium
effect size ( q - = .06) and level o f power (1- P = .62) were also obtained for both measures.
An additional pairwise comparison (using the Bonferroni approach) o f the completeness measure
Table 16
Hypothesis 1 Interaction Assessment
Activity Effect F df Sig. Partial q - Power
Treatment Exp. 1 0.63 15 0.84 0.03 0.38
Posttest Exp. 1 0.79 15 0.69 0.04 0.48
Delayed posttest Exp. 1 0.85 15 0.62 0.05 0.51
79
Table 17
Effect ot' Instructional Effectiveness per Assessment
Test Effect F df Sig- Partial p - Power
T reatment Exp. I 1.92 10.00 0.04 0.09 0.86
Posttest Exp. 1 0.78 10.00 0.64 0.04 0.41
Delayed posttest Exp. I 1.30 5.00 0.27 0.07 0.44
Table 18
Contribution to Instructional Effectiveness: Hvdo 1
Measure Effect F df Sig. Partial p - Power
Completeness Exp. 1 3.37 2.00 0.04 0.06 0.62
Accuracy Exp. 1 3.36 2.00 0.04 0.06 0.62
Errors Exp. 1 0.74 2.00 0.48 0.01 0.17
% Corrected Exp. 1 0.47 2.00 0.63 0.01 0.13
Rec time Exp. 1 1.26 2.00 0.29 0.02 0.27
revealed a statistically significant difference (e = .05) between subjects in the elaborative and skeletal
groups (Table 19). The mean completion scores for subjects using elaborative online help (M = .87. SD =
.14) was statistically significantly greater at the .05 level of confidence than the completion scores for
subjects using skeletal online help (M = -79, SD = .18). A standardized m ean difference effect size
(SM DES) o f .49 indicated that subjects in the elaborative group completed about 7% more tasks (.49 x .14)
than did subjects in the skeletal group. Figures 40 and 41 illustrate the m agnitude o f difference between the
two groups.
Although not statistically significant, the standardized mean difference effect size (0.49) for the
accuracy measures between the subjects in the inferential and skeletal groups (Table 19) tended toward
significance (p = .0.07), which contributed to the overall significance o f the accuracy measure during
R e p ro d u c e d with perm ission of the copyright owner. Furth er reproduction prohibited without permission.
80
Table 19
Differences in Task Completeness and Accuracy per Group
SMDES
Measure (I) Exp. I (J) Exp. 1 SE Sig. (I—J/SD)
Completeness Inferential Skeletal 0.03 0.15 0.40
Elaborative Skeletal 0.03 0.05 0.49
Elaborative Inferential 0.03 1.00 0.09
Accuracy Inferential Skeletal 0.03 0.07 0.49
Inferential Elaborative 0.03 1.00 0.03
treatment (Table 17). This effect size indicates that subjects in the inferential group were 5% more accurate
(0.40 x 0 .1 1) in completing their tasks during the treatment than were their counterparts in the skeletal
group.
The hypotheses for this research study compared different implementations o f minimalism during
different vertical "time slices'”: pretest, treatment, posttest, delayed posttest. O f additional significance for
practitioners and educators is the horizontal perspective afforded by analyzing each online help system
across all four testing times. For Hypothesis 1, such an analysis would address the question o f whether or
not any learning occurred given any o f the different implementations o f minimalism.
A multivariate repeated measure analysis was conducted to evaluate the differences in subjects'
performance outcome scores (completeness, accuracy, errors committed, percentage o f errors corrected,
error recovery time) over four different points in time (pretest, treatment, posttest, delayed posttest).
Statistically significant differences (F = 46.84, g = 0.000) were obtained for a combination o f all
performance measures across time (Table 20).
An additional pairwise comparison (using the Bonferroni approach) o f the performance measures
across time revealed several statistically significant differences. As shown in Table 21, the greatest effect
sizes were associated with the differences in performance measures between the pretest and the treatment;
81
20
Std. Dev = .18

Mean = .79
N = 38.00
.25 .38 .50 .63 .75 .88 1.00
Percent of Tasks Completed

Figure 40. Percentage o f tasks completed: Skeletal.
20
.50 .63 .75 .88 1.00
Percent Tasks Completed

Figure 41. Percentage o f tasks completed: Elaborative
82
Table 20
Significance o f Learning over Testing Times
Grouping if F Sig. Partial q - Power
Performance measures 15 46.84 0.000 0.45 1.00
the next greatest effect sizes were associated with the differences in performance measures between the
pretest and both posttests.
A trend analysis (Table 22) was conducted to focus the analysis and mathematically model the
form o f the subjects' performance scores across the different time periods. There were strong statistically
significant differences (Table 22) along each o f the trend lines. Figures 42 through 46 illustrate the
statistically significant differences within the performance scores over time. As suggested by the data in
Table 2 1 and illustrated in Figures 42. 4 3,44, and 46. the sharp differences in scores between the pretest
and treatment and pretest and posttests and then the nearly negligible differences in scores between the
posttest and delayed posttest suggest that the data are best represented by linear and quadratic trend models.
The cubic trend model better represents Figure 45, percentage o f errors corrected.
Hypotheses 2 and 3 Data Analyses
Hypothesis 2 stated that "While learning procedural tasks, subjects who use skeletal help (S) will
complete tasks more quickly than subjects using elaborative help (E), who will complete tasks more
quickly than subjects using inferential help (I).” Hypothesis not supported at the 0.05 level o f confidence.
Hypothesis 3 stated that "Subjects who used inferential help (I) to learn tasks will complete those
tasks more quickly than subjects using skeletal help (S), who will complete tasks more quickly than
subjects using elaborative help (E).” Hypothesis not supported at the 0.05 level o f confidence.
"M ore quickly’ (for both hypotheses) was m easured by tracking the start and stop time (as entered
by the subjects) for each task in the pretest, treatment, posttest, and delayed posttest.
First, the interaction between the group and the time measure was assessed to ensure that the
83
Table 21
Differences in Performance over Testing Times: H vdo 1
Measure (I) TEST (J) TEST 11 S i^ SMDES
Completeness Treatment Pretest 0.02 0.00 2.05
Completeness Treatment Posttest 0.02 0.00 0.35
Completeness Treatment Delayed post 0.02 0.00 0.39
Completeness Posttest Pretest 0.02 0.00 1.70
Completeness Posttest Delayed post 0.02 1.00 0.04
Completeness Delayed Pretest 0.02 0.00 1.66
Accuracy Treatment Pretest 0.02 0.00 1.11
Accuracy Treatment Posttest 0.01 0.42 0.14
Accuracy Treatment Delayed post 0 .0 1 0.31 0.14
Accuracy Posttest Pretest 0.02 0.00 0.98
Accuracy Posttest Delayed post 0.01 1.00 0.00
Accuracy Delayed Pretest 0.02 0.00 0.98
Accuracy Delayed Posttest 0.01 1.00 0.00
Errors Pretest Treatment 0.75 0.00 -0.94
Errors Posttest Pretest 0.44 0.00 -0.44

00
Errors Posttest Treatment 0.62 0.00
Errors Delayed Pretest 0.39 0.00 -0.53
Errors Delayed Treatment 0.63 0.00 -1.47
Errors Delayed Posttest 0.30 1.00 -0.09
Corrected Pretest Posttest 0.04 1.00 0.15
Corrected Treatment Pretest 0.02 0.00 0.76
Corrected Treatment Posttest 0.03 0.00 0.91
Corrected Treatment Delayed post 0.03 0.00 0.46
(table continues)
84
Measure tl) TEST (J) TEST SE Sig- SMDES
Corrected Delayed Pretest 0.03 0.17 0.30
Corrected Delayed Posttest 0.03 0.02 0.45
Rec time Pretest Posttest 19.40 0.17 0.33
Rec time Pretest Delayed post 17.24 0.04 0.37
Rec time Treatment Pretest 45.20 0.00 2.71
Rec time Treatment Posttest 43.46 0.00 3.04
Rec time Treatment Delayed post 42.21 0.00 3.08
Rec time Posttest Delayed post 15.18 1.00 0.04
.90
.8 0 -
.7 0 -
-O
<U
0) .60-
Q.
f—
C
o
O
CO .5 0 - Help 1
-SC
O)
CO
h- Skeletal
° .40-
c Inferential
0)
au .
0
CL 30 Elaborative
Pretest Treatment Posttest Delayed Posttest
TEST
Figure 42. Trend: Percentage o f tasks completed.
85
Table 22
Trend Analysis o f Performance Measures
Measure Test SS MS df F Sig. Partial r|- Power
Completeness Linear 4.20 4.20 1 214.04 0.000 0.69 1.00
Quadratic 4.29 4.29 1 278.90 0.000 0.75 1.00
Cubic 1.43 1.43 1 115.18 0.000 0.55 1.00
Accuracy Linear 1.52 1.52 1 97.47 0.000 0.51 1.00
Quadratic 1.21 1.21 1 95.44 0.000 0.50 1.00
Cubic 0.37 0.37 1 37.54 0.000 0.28 1.00
Errors Linear 558.16 558.16 1 73.24 0.000 0.44 1.00
Quadratic 336.71 336.71 I 19.42 0.000 0.17 0.99
Cubic 830.45 830.45 1 45.09 0.000 0.32 1.00
Corrected Linear 0.00 0.00 1 0.00 0.997 0.00 0.05
Quadratic 0.12 0.12 I 2.23 0.139 0.02 0.32
Cubic 2.40 2.40 1 71.25 0.000 0.43 1.00
Rec time Linear 1428949.04 1428949.04 1 69.31 0.000 0.42 1.00
Quadratic 3135772.23 3135772.23 1 57.15 0.000 0.38 1.00
Cubic 6365360.61 6365360.61 1 73.84 0.000 0.44 1.00
group regression lines were parallel to the overall regression lines (Table 23). There was a statistically
significant interaction (F = 4.50. g = 0.01) during the treatment, which suggested that time measure during
the pretest differed enough from the time measure during the treatment to cause biased results. However, a
statistically significant difference in time between the treatment and the pre- and posttests is expected since
the pre- and posttests were time controlled, and the treatment was not. Therefore, the use o f analysis o f
covariance is still appropriate for the data.
Three univariate analyses o f variance were then conducted to evaluate the relationship between
types o f online minimalist instruction and time. The pretest score on the time measure was used as the
86
90
80
.70
>.
o
CD
Help 1
3
O
U
< Skeletal
4—
o
c Inferential
(D
O
q! 50 Elaborative
P retest Treatment Posttest D elayed Posttest
TEST
Figure 43. Trend: Percentage o f accuracy in task completion.
12
10
8
■o
0)
E
E
o 6
O
C/5
u. Help 1
o
Skeletal
V *—
o 4
<D Inferential
n
E
3
2 ____ Elaborative
Pretest Treatment Posttest D elayed Posttest
TEST
Fiuure 44. Trend: Num ber o f errors committed.
87
1.00
Help 1
ow
w .70
LU Skeletal
o
c Inferential
d)
u\_
(U
0. Elaborative
P re test T reatm ent P o sttest D elayed P o stte st
TEST
Figure 45. Trend: Percentage o f errors corrected.
700
600
500
(A
400
</)
(D
E
I - 300 Help 1
&
<D
> Skeletal
0
flj 200
cr1 Inferential
O
u.
UJ 100 E laborative
P re test T reatm en t P o sttest D elayed P o stte st
TEST
Figure 46. Trend: Error recovery time.
88
Table 23
Hypotheses 2 and 3 Interaction Assessment
Test SS (secs) MS (secs) df F Sig. Partial q - Power
T reatment 10204592.33 5102296.167 2 4.50 0.01 0.08 0.76
Posttest 49030.03 24515.017 2 1.24 0.29 0.02 0.27
Delayed posttest 23583.88 11791.938 2 1.63 0.20 0.03 0.34
eovariate to measure any differences in subjects’ performance after exposure to the testing materials. Table
24 represents the combined output o f the univariate analyses o f time as applied to Experiment 1 (skeletal,
inferential, and elaborative) for the treatment, posttest, and delayed posttest. There were no statistically
significant differences among the groups at the 0.05 level o f confidence in time to complete the tasks.
The variability o f the time measure was constrained during the time-controlled pretest and
posttests. Subjects who completed the tasks prior to the allotted time continued to tweak their work until
stopped by the lab consultant: subjects who did not complete the task were also stopped at the designated
time, resulting in similar completion times for all subjects. The data coders considered using the observed
ending time (rather than the ending time as entered by the subject) for the time measure, but decided
against it in order to prevent any instrumentation effects.
different vertical "time slices”: pretest, treatment, posttest, delayed posttest. O f additional significance for
across all four testing times. For Hypotheses 2 and 3, such an analysis would address the question o f
whether or not subjects could decrease the time taken to complete tasks given any o f the different
implementations o f minimalism.
performance outcome scores (time) over four different points in time (pretest, treatment, posttest, delayed
posttest). A statistically significant differences (F = 112.66, p = 0.000) was obtained (Table 25), which was
expected since the treatment was untimed.
89
Table 24
Effect of Time on Performance
Test SS (secs) MS(secs) if F Sig- Partial t^2 Power
Treatment 335398.892 167699.446 2 0.14 0.87 0.003 0.07
Posttest 26068.319 13034.159 2 0.66 0.52 0.012 0.16
Delayed posttest 2082.786 1041.393 2 0.14 0.87 0.003 0.07
Table 25
Significance o f Time to Complete Tasks over Testing Times
Grouping Jf F Sig. Partial r|- Power
Performance measures J 112.66 0.000 0.76 1.00
An additional pairwise comparison (using the Bonferroni approach) o f the time tc complete tasks
revealed several statistically significant differences (Table 26). The findings that are relevant to this
research are the differences (or lack thereof) among the timed tests: pretest, posttest, and delayed posttest.
There was no statistically significant difference between the posttest and pretest even though subjects had
nearly 14 more minutes in the posttest to complete tasks. Instead, they were able to complete tasks during
the posttest within about 24 seconds o f the time (0.22 x 65.9 / 60) they took to complete the same tasks in
the pretest. Thus, a nonsignificant finding here actually is important. The other finding o f interest is the
statistically significant difference (g = 0.03; MDES =0.71) in time to complete tasks between the posttest
and delayed posttest. After a I-week delay, subjects could complete the tasks within just over a minute o f
the time it took to complete the same tasks the previous week. This finding, coupled with the fact that
subjects completed about the same amount o f tasks with nearly the same amount o f accuracy, is relevant.
form o f the time it took for subjects to complete tasks across the different time periods. There were strong
statistically significant differences (Table 27) along each o f the trend lines; however, the data from the
timed tests best fit a linear trend model (Figure 47).
90
Table 26
Differences in Time to Complete Tasks over Testing Times
(1) TEST (J) TEST SE Sig. MDES
Treatment Pretest 105.09 0.000 26.25
T reatment Posttest 100.05 0.000 26.03
Treatment Delayed post 103.35 0.000 25.32
Posttest Pretest 14.04 1.000 0.22
Delayed post Pretest 9.31 0.000 0.93
Delayed post Posttest 13.15 0.003 0.71
Table 27
Trend Analysis o f Time to Complete Tasks
Measure Test SS MS df F Sig. Partial n~ Power
Time Linear 13022055.78 13022055.78 1 240.10 0.000 0.69 1.00
Time Quadratic 78535662.81 7S535662.81 1 235.97 0.000 0.69 1.00
Time Cubic 150412184.96 150412184.96 1 297.35 0.000 0.73 1.00
Hypothesis 4 stated that "Inferential-proximal (P) online help will be more effective in helping learners
identity and correct errors than inferential-distant (T) online help, which will be more effective than
inferential-without (W) online help (i.e., P > T > W).” Hypothesis not supported at the .05 level o f
confidence. "Effective” referred to a combination o f the following measures:
1. Errors committed: the number o f errors the subject made while com pleting a task
2. Percentage o f errors corrected: the number o f corrected errors divided by the total number o f
errors committed
3. Error recovery time: the total time spent correcting errors per experim ent and activity.
91
3000
20 0 0 -
tn
o
(D
w
<n
-S£
(/)
ro
-SJ 1000- Help 1

Jl)
CL [
E S keletal
o
O
o
Inferential
<D
E
I- E laborative
P retest T re a tm e n t P o s tte s t D elay ed P o stte st
TEST
Fitmre 47. Trend: Time to complete tasks.
Analysis o f covariance was the statistical test selected in order to best determine the magnitude o f
differences among the groups after exposure the testing and treatment materials. Therefore, the interaction
between the group and the performance measures was assessed to ensure that the group regression lines
were parallel to the overall regression lines (Table 28). The absence o f an interaction between the
independent variable and the covariates means that regression lines are parallel, that the adjustment
employed by the analysis o f covariance procedure is appropriate, and that analysis o f covariance results
may be interpreted straightforwardly. Since none o f the interaction terms was statistically significant, this
assumption was met to a reasonable degree, and use o f analysis o f covariance was appropriate for the data.
Three different multivariate analyses o f variance were then conducted to evaluate the relationship
between types o f online minimalist instruction and specific, error-related performance measures. The
pretest scores on the above three measures were used as the covariate to measure any differences in
subjects’ performance after exposure to the testing materials.
92
Table 28
Hypothesis 4 Interaction Assessment
Test Effect If F Sig. Partial q - Power
T reatment Exp. 2 9 0.86 0.56 0.03 0.34
Posttest Exp. 2 9 0.64 0.76 0.02 0.25
Delayed post Exp. 2 9 0.97 0.47 0.03 0.39
Table 29 represents the combined output o f the multivariate analyses o f the three measures listed
above applied to Experiment 2 (inferential-without, inferential-distant, and inferential-proximal) for the
treatment, posttest, and delayed posttest. No statistically significant differences were obtained (at the 0.05
level o f confidence) regarding the presence or location o f error information and the effect o f that error
information on decreasing the number o f errors subjects committed, increasing the percentage o f errors
they corrected, or decreasing the amount o f time they spent correcting errors.
different vertical "tim e slices” : pretest, treatment, posttest, delayed posttest. O f additional significance for
across all four testing times. For Hypothesis 4. such an analysis would address the question o f whether or
not subjects could decrease the number o f errors they committed, increase the percentage o f errors they
corrected, or decrease the amount o f time they spent correcting errors given any o f the different
implementations o f minimalism.
performance outcome scores (errors committed, percentage corrected, error recovery time) over four
different points in time (pretest, treatment, posttest, delayed posttest). A statistically significant difference
(F = 23.29. p = 0.000) was obtained (Table 30).
An additional pairwise comparison (using the Bonferroni approach) o f the error performance
measures revealed several statistically significant differences (Table 31). The findings that are relevant to
this research are the differences and effect sizes between the pretest and each o f the posttests. Subjects
93
Table 29
Effect o f Error Information on Performance
Test Effect df F Sig- Partial rj2 Power
Treatment Exp. 2 6 0.89 0.50 0.03 0.35
Posttest Exp. 2 6 1.00 0.42 0.03 0.39
Delayed posttest Exp. 2 6 0.30 0.94 0.01 0.13
Table 30
Significance o f Error Information over Testing Times
Grouping df F Sig. Partial r|- Power
Performance measures 9 23.29 0.000 0.69 1.00
committed approximately 4 fewer errors (0.62 x 5.95) during the posttest (p = 0.001. SMDES = -0.62) and
approximately 2.25 fewer errors (0.42 x 5.33) during the delayed posttest (p = 0.031. SMDES = -0.42) than
the> did during the pretest. There were no statistically significant differences in the amount o f time taken to
recover from errors during the timed tests.
Although subjects corrected approximately 12% more errors (0.50 x 0.23) during the delayed
posttest (p = 0.031, SMDES = -0.42) than they did during the pretest, the statistically significant difference
among the three groups during the pretest (see Figure 48) suggests sampling error and therefore makes any
comparison differences suspect. However, the scores from the subjects in the inferential-without group (the
most divergent during the pretest) were comparable to the scores from the other groups during the
treatment and posttests.
form o f the time it took for subjects to complete tasks across the different time periods. There were strong
statistically significant differences (Table 32) along each o f the trend lines; however, the data from the
timed tests best fits a cubic trend model (Figures 4 8 ,4 9 , and 50). O f real interest here also is the fact that
the subjects in the proximal group corrected less errors overall than did the subjects in the other group
94
Table 31
Differences in Use o f Error Information over Testing Times
Measure (1) TEST (J) TEST SE Sig. MDES
Errors Pretest Treatment 0.89 0.000 -1.83
Errors Posttest Pretest 0.59 0.001 -0.62
Errors Posttest Treatment 0.98 0.000 -2.44
Errors Posttest Delayed post 0.54 1.000 -0.20
Errors Delayed post Pretest 0.55 0.031 -0.42
Errors Delayed post Treatment 0.84 0.000 -2.24
Corrected Treatment Pretest 0.02 0.000 0.59
Corrected Treatment Posttest 0.03 0.272 0.27
Corrected T reatment Delayed post 0.02 1.000 0.09
Corrected Posttest Pretest 0.04 0.342 0.32
Corrected Delayed post Pretest 0.03 0.002 0.50
Corrected Delayed post Posttest 0.03 1.000 0.18
Rec time Pretest Treatment 62.26 0.000 -3.64
Rec time Pretest Delayed post 28.32 1.000 -0.09
Rec time Posttest Pretest 24.13 0.187 -0.31
Rec time Posttest Treatment 61.92 0.000 -3.94
Rec time Posttest Delayed post 25.93 0.060 -0.39
Rec time Delayed post Treatment 67.91 0.000 -3.55
(Figure 48) although their correction rate nearly parallels that o f subjects in the inferential-without group.
In other words, the subjects who had no error information at all corrected more errors than did the subjects
w ho had error information located right next to the error-prone steps. In fact, they corrected even fewer
errors during the posttest than did their counterparts in the inferential-without group.
95
.90
.80-
L-
L_ Ir
o
O
(/)
L_
Help 2
o
i5 60 ■ Without
o
c ^ Distant
<u
o
<u
0. Proximal
P retest Treatment Posttest Delayed Posttest
TEST
Fitzure 48. Trend: Percentage o f errors corrected over time.
Table 32
Trend Analysis o f Error Information
Measure Test SS MS df F Sig. Partial r|- Power
Errors Linear 1051.15 1051.15 I 54.48 0.000 0.34 1.00
Quadratic 1021.87 1021.87 1 43.84 0.000 0.30 1.00
Cubic 3683.01 3683.01 1 76.75 0.000 0.42 1.00
Corrected Linear 0.35 0.35 1 7.59 0.007 0.07 0.78
Quadratic 0.20 0.20 1 4.16 0.044 0.04 0.52
Cubic 0.41 0.41 1 10.38 0.002 0.09 0.89
Rec time Linear 2147523.84 2147523.84 1 30.91 0.000 0.23 1.00
Quadratic 8358484.45 8358484.45 1 61220 0.000 0.37 1.00
Cubic 22584698.38 22584698.38 1 126.53 0.000 0.55 1.00
96
20
18
16
-o 14
aj
1
£ 12
o
O
(/)
W. Help 2
o1
I—
— 10
UJ
.«— Without
o
k-
(U 8
-Q ^ Distant
E
2
6 ___ Proximal
Pretest Treatment Posttest Delayed Posttest
TEST
Figure 49. Trend: Number o f errors committed over time.
1000
800-
600-
(0
o
CD
U)
^ 400-
i- Help 2
^ f
<u [
Without
g 200-
<u
DC Distant
Proximal
Pretest Treatment P osttest Delayed P osttest
TEST
Figure 50. Trend: Error recovery time over time.
97
Hypothesis 5 stated that "Subjects who prefer a verba! style o f learning will complete tasks more
accurately and efficiently using skeletal online help.” An attempt was made to associate an instructional
style with verbal aptitude. "A ccurately and efficiently” refer to the combination o f the following measures:
1. Time: the start and stop time for each task as entered by the subject
2. Completeness: the ability to complete the given task (see Figure 37 for a sample o f the data
coding sheet listing the five tasks and the steps to complete them)
the method outlined in the online help systems (see Figure 38 for a sample o f the related coding sheet)
First, univariate analyses o f variance were conducted for each outcome measure (time,
completeness, accuracy) for the pretest, treatment, and posttests to evaluate any aptitude treatment
interaction between verbal learning scores and type o f online instruction. The fixed factors were the
different Experiment I treatment groups: skeletal, inferential, and elaborative. The covariate was the
subjects' verbal learning scores. The main interaction effects for the outcome measures between verbal-
learning score and online help system for each o f the testing phases are identified in Table 33.
Simple scatterplots (Figures 51 and 52) were created to illustrate the magnitude o f the interactions
relative to each treatment group. Linear regressions were then conducted for each group to generate the
intercept and the slope coefficients for the group regression lines. The slope and intercept data for each
regression line are included in the scatterplot legends.
Verbal x Online Help x Time
A statistically significant interaction between time, preference for verbal learning, and type o f
online help was obtained (F = 4.28: p = .02) during the pretest (Table 33). A medium effect size ( q - = .08)
and intermediate level o f power (1 - P = 0.74) associated with the interaction suggest that the occurrence o f
this interaction was probably due to sampling error. That is, the groups differed from the target population
along some characteristics (even though they were randomly assigned to the various groups). However, the
obtained significant interaction with tim e was not maintained across the treatment or posttests.
98
Table 33
Interaction Effects: Verbal Learning by Experiment 1
Measure Test SS M2 df F Sig. n2 Power
Time (mins) Pretest 597.06 298.53 2.00 4.28 0.02 0.08 0.74
Treatment 42962.17 21481.08 2.00 1.09 0.34 0.02 0.24
Posttest 434.84 217.42 2.00 0.65 0.52 0 .0 1 0.16
Delayed post 194.87 97.44 2.00 0.81 0.45 0.02 0.18
Complete Pretest 0.06 0.03 2.00 0.64 0.53 0 .0 1 0.16
Treatment 0.12 0.06 2.00 2.58 0.08 0.05 0.51
Posttest 0.20 0.10 2.00 2.36 0.10 0.04 0.47
Delayed post 0.05 0.03 2.00 0.61 0.55 0.01 0.15
Accuracy Pretest 0.06 0.03 2.00 0.82 0.44 0.02 0.19
Treatment 0.17 0.09 2.00 6.68 0.002 0.11 0.91
Posttest 0.08 0.04 2.00 1.71 0.19 0.03 0.35
Delayed post 0.02 0.01 2.00 0.54 0.58 0.01 0.14
Figure 51 illustrates the relative magnitude and direction o f interaction between subjects' verbal-
learning scores and time to complete tasks that contributed to the main interaction effects that occurred
during the pretest. There was an inverse relationship between time and verbal learning preference
associated with the skeletal group and a positive correlation between time and verbal learning preference
associated with the elaborative and inferential groups. That is, subjects in the skeletal group who scored
higher on the verbal learning measure took less time to complete tasks during the pretest, whereas subjects
in the elaborative and inferential groups who scored higher on the verbal learning measure took more time
to complete tasks during the pretest.
99
1100
^ 1000 -
c/3
o
0
C/3
0 900-
E
K
re
£ 800- ' ^>4 Help 1
XX
C/3
0 * Elaborative
0
700- Y = 7 77.27 + 3.32X
CL
Inferential
c
| 600- Y = 7 89.57 + 3.56X
o3 * Skeletal
Q.
QJ 500. Y = 825.08 - 3.51X
-10 0 10 20
Verbal Learning Score

Figure 5 1. Interaction o f online help, verbal learning, and time during pretest.
c
0
CJ
<D
Q.
<U
O
O
CO
>N ■ Help 1
o
03
w
3 * Elaborative
O
u
< Y = 0.91 -0 .0 1 X
* Inferential
c
<D Y = 0.90 - 0.01X
E
3 ■ ■ ■ i
0 * Skeletal
Q.
X
0 Y = 0.76 + 0.01X
-10 0 10 20
Verbal Learning Score

Figure 52. Interaction o f online help, verbal learning, and accuracy during treatment.
100
Verbal x Online Help x Accuracy
As shown in Table 33, a statistically significant interaction between accuracy, verbal learning
preference, and type o f online help was obtained during treatment (F = 6.68, g = 0.002). A large effect size
(H~ = 0 .11) and level o f power ( I - (3 = 0 .9 1) indicate that over 10% o f the subject’s ability to complete a
task accurately was associated with his or her preference for a verbal learning style. In addition, there was
less than a 10% chance that the interaction obtained from this sample would have occurred by chance.
Figure 52 illustrates the relative magnitude and direction o f interaction between subjects’ verbal-
learning scores and accuracy scores that contributed to the main interaction effects that occurred during the
treatment. There was a positive relationship between accuracy scores and verbal learning preference
associated with the skeletal group and an inverse relationship between accuracy scores and verbal learning
preference associated with the elaborative and inferential groups. That is, subjects in the skeletal group who
scored higher on the verbal learning measure were more accurate in task completion during the treatment,
whereas subjects in the elaborative and inferential groups who scored higher on the verbal learning measure
w ere less accurate in task completion during the treatment.
Exploration o f Pretest Group Demographic Data
Because there was a statistically significant interaction between the groups during the pretest, a
further exploration o f the demographic makeup o f the groups was warranted. The focus o f the investigation
was on the skeletal group and the ways in which those subjects differed from the subjects in the inferential
and elaborative groups. As shown in Table 34, subjects in the skeletal group tended to differ from the other
subjects along the characteristics o f years o f computer experience and level o f specific experience in Word
and WordPerfect. However, there was no real trend in the differences. For example. Figures 53, 54. and 55
illustrate that although some o f the subjects in the skeletal group had less general and specific computer
knowledge than did the subjects in the other two groups, others in the skeletal group either matched or
exceeded the top level o f general and specific computer experience possessed by subjects in the other two
groups. There did not appear to be any statistically significant differences among the groups in regard to
their entrance characteristics. However, a multivariate analysis o f variance was conducted (Table 35) to
confirm that indeed there were no statistically significant differences obtained.
101
Table 34
Demographic Data for Pretest Groups
Experiment I groups
Skeletal Inferential Elaborative
Variable Years (percentage) (percentage) (percentage)
Comp exp 0-1 2.6 5.4 8.3
1-2 13.2 21.6 16.7
2-5 36.8 37.8
5-10 28.9 29.7 25.0
over 10 13.2 2.7 13.9
Word exp None 2.6 2.7 5.6
Beg 42.1 35.1 25.0
Med 44.7 59.5 63.9
Adv 10.5 2.7 5.6
WP exp None 2.6 10.8 19.4
Beg 28.9 18.9 13.9
Med 52.6 56.8 50.0
Adv 15.8 13.5 16.7
Hypothesis 6 stated that “Subjects who prefer a visual style o f learning will complete tasks more
accurately and efficiently using Elaborated online help.” In other words, an attempt was made to associate
an instructional style with visual aptitude. “Accurately and efficiently” refer to the combination o f the
following measures:
102
14-
12-
10-
Help 1
S keletal
Inferential
E laborative
Missing o 0-1 1-2 2-5 5-10 o v e r 10
Y ears of Com puter E xperience

Figure 53. Years ot'com puter experience by group.
20-
Help 1
Skeletal
o
<D
•Q Inferential
E
3
2 Elaborative
None Beg Med Adv
Level of Experience with Word

Figure 54. Level o f experience with Microsoft Ward.
103
30
Skeletal
Inferential
Elaborative
None Beg Med Adv
Level of Experience with Word Perfect

Fiiiure 55. Level o f experience with Core! WordPerfect.
1. Time: the start and stop time for each task as entered by the subject
2. Completeness: the ability to complete the given task (see Figure 37 for a sample o f the data
coding sheet listing the five tasks and the steps to complete them)
the method outlined in the online help systems (see Figure 38 for a sample o f the related coding sheet)
Univariate analyses o f variance were conducted for each outcome measure (time, completeness,
accuracy) for the pretest, treatment, and posttests to evaluate any aptitude treatment interaction between
visual learning scores and type o f online instruction. The fixed factors were the different Experiment 1
treatment groups: skeletal, inferential, and elaborative. The covariate was the subjects' visual learning
scores. The main interaction effects for the outcome measures between visual-learning score and online
help system for each o f the testing phases are identified in Table 36. There were no statistically significant
interactions at the .05 level o f confidence between preferred visual learning style, online help system, or
performance measures.
104
Table 35
Significant Differences in Pretest Subjects
-)
Test df F Sig- n- Power
Wilks' lambda 6 0.52 0.79 0.02 0.21
Table 36
Interaction Effects: Visual Learning bv Experiment I
Measure Test SS df M2 F Sig. h~ Power
Time (secs) Pretest 1111.97 2 555.99 0.12 0.88 0.002 0.07
Treatment 4906.23 2 2453.11 0.002 1.00 0.000 0.05
Posttest 8804.32 2 4402.16 0.22 0.80 0.004 0.08
Delayed post 16526.63 2 8263.32 1.11 0.33 0.02 0.24
Completeness Pretest 0.02 2 0.01 0.21 0.81 0.004 0.08
T reatment 0.03 2 0.02 0.67 0.52 0.01 0.16
Posttest 0.05 2 0.03 0.61 0.55 0.01 0.15
Delayed post 0.08 2 0.04 0.90 0.41 0.02 0.20
Accuracy Pretest 0.02 2 0.01 0.28 0.76 0 .0 1 0.09
Treatment 0.02 2 0.01 0.87 0.42 0.02 0.20
Posttest 0.02 2 0.01 0.50 0.61 0.01 0.13
Delayed post 0.04 2 0.02 0.90 0.41 0.02 0.20
105
CHAPTER VI
SUMMARY, CONCLUSIONS, AND RECOMMENDATIONS
The general purpose o f this research was to define ways in which minimalism, an instructional
online help systems. Two separate experiments (using different sets o f online help systems) were designed
to evaluate different principles o f minimalism. The first experiment was focused on comparing variations
o f minimalist instruction for increased learning. The second experiment was focused on identifying the
effect that presence and location o f error-recovery information had on subject error commission and
recovery’. A related purpose o f this research was to determine if a subject’s performance in learning a
software application would increase if his or her preference for learning style would be matched to an
instructional approach. The summary o f this study, conclusions drawn from findings, and further
recommendations based on findings from this research are presented in this chapter.
Summary o f the Study
This summary section includes the statement o f the problem, statement o f purpose, research
procedures, data analysis, and findings for each hypothesis.
Statement o f the Problem
Most computer software is accompanied by documentation, much o f which is placed online.
Effective, well-written, helpful documentation strongly influences the perceived effectiveness o f the
software by users (Greer, 1984; Rosenbaum, 1998) to the point that documentation can “make or break a
software application" (Greer, 1984, p. 7). Inadequate or poorly written online help results in user frustration
with and possible rejection o f the software (Kearsley, 1985b; Lieberman, 1987) and possible loss o f
business (Rosenbaum, 1998). Technical communicators are starting to apply the principles o f minimalism,
an instructional design approach used to develop training manuals, to the developm ent o f online help
systems. The implementation o f minimalist principles to online help instruction could result in increased
learning o f the subject matter content and acceptance o f the software application; however, the application
106
o f minimalism to training materials has met with mixed reviews, and its application to online help has not
been evaluated thoroughly. Research on the application o f the principles o f minimalism to online help is
needed to assist technical communicators in the design and development o f effective online help
instruction: the results o f this research could also be extended to the development o f any online instruction.
Statement o f the Purpose
online help systems, which are computer-based and typically informational and referential. Two separate
experiments (using the same sample population) were conducted with the following general hypotheses:
2. Users who are prompted to interact with the application spend longer amounts o f time
learning tasks, but, once learned, complete tasks faster without instructional support.
software application procedural skills more quickly and with fewer errors than subjects using a nonmatched
Research Procedures
A sample o f 170 subjects for this research was obtained from the clerical pool o f Kelly Temporary
Services (KTS). where they completed prequalification testing in M S Word 97 tasks, a background survey,
and a learning preference survey. Subjects were randomly assigned to two different experimental
conditions. The data from 111 subjects were used for this research.
All sessions for this research were conducted in the same College o f Business microcomputer lab
at Utah State University. Loaded onto each computer were two sets o f online help systems for M S Word 97
107
( for a total o f five different online help systems), a screen-capture application, and text and graphic
resources necessary to complete the experiment.
Subjects created four different newsletters (pretest, treatment, posttest, and delayed posttest) which
were comprised o f five tasks identified as intermediate or advanced in difficulty by the KTS training guide.
Experiment 1 consisted o f the first two tasks (create a banner heading; insert and format text into columns);
Experiment 2 consisted o f the last three tasks (insert and format a graphic, create and format a table, create
a data source and mailing labels for a mail merge). Online help instruction was available only during the
treatment newsletter. Subjects used one online help system to complete the first two tasks and a different
online help system to create the remaining three tasks: lab consultants switched the online help systems for
the subjects. After completing the fourth newsletter (the second week), subjects evaluated one o f the two
online help systems they had used the previous week.
Data Analysis
MS Cam corder£>, a Microsoft computerized screen capture software, was used to record subjects’
mouse moves and keyboard strokes as they completed each o f the experimental activities. The resulting
data were saved as an .avi file and burned onto CD-ROMs. Coding checklists were used to record all
pertinent data for analysis.
SPSS 10.0 was used to analyze the data. Multivariate analyses o f covariance (followed by
univariate analyses o f variance and pairwise comparisons for statistically significant findings) were used to
determine if one online help system was more effective than another in assisting subjects in both learning
and remembering how to complete software application procedural tasks. Bivariate correlations were used
to identity any relationships among subjects’ preferred learning style, type o f online help system used, and
effectiveness in completing software application procedural tasks.
Univariate analyses o f variance with subjects’ preferred visual or verbal learning style were
conducted to assess the effect on performance outcomes associated with an interaction between preferred
learning style and type o f online instruction. Linear regressions were used to plot the slope and intercept for
the interaction plots.
108
Summary o f Results
There were four general findings associated with the four general research hypotheses. Each
finding and general hypothesis is listed below followed by the summary o f results for each hypotheses.
Finding 1
It was generally hypothesized that users would require (and prefer) less instructional support to
complete and learn software application procedural skills. Specifically, it was predicted that subjects who
used the inferential online help system (least instructional support) would have better completion and
accuracy scores than subjects using either the elaborative or skeletal online help systems. The findings were
mixed. Although not quite statistically significant at the 0.05 level o f confidence, the mean accuracy scores
for subjects using inferential online help were greater during the teaching'treatment session than the
accuracy scores for subjects using skeletal online help. Instead, the mean completion scores for subjects
using elaborative online help (most instructional support) were statistically significantly greater during the
teaching treatment session than the completion scores for subjects using skeletal online help. This finding
was opposite what was expected.
However, additional repeated measure analysis revealed statistically significant differences
between the pretest and posttest on the following performance outcome measures: completeness, accuracy,
number o f errors committed, and amount o f time taken to recover from errors. In other words, subjects
completed more tasks, completed them more accurately, committed fewer errors, and corrected the errors in
less time after exposure to any o f the three minimalist online help systems.
F inding 2
It was generally hypothesized that users who are prompted to interact with the application would
spend longer amounts o f time learning tasks but. once learned, would complete tasks faster without
instructional support. Specifically, it was predicted (Hypothesis 2) that while learning procedural tasks (i.e.,
during the treatment), subjects who used skeletal help would complete tasks m ore quickly than subjects
using elaborative help, who would, in turn, complete tasks more quickly than subjects using inferential
help. It was also predicted (Hypothesis 3) that subjects who used inferential help during the treatment to
learn tasks would complete those tasks more quickly during the posttests than subjects who used skeletal
help, who would, in turn, complete tasks more quickly than subjects who used elaborative help. Instead,
this was not the case. There were no statistically significant differences among the treatm ent groups.
However, additional repeated measure analysis revealed no statistically significant differences
between the posttest and pretest, which means that even though subjects had nearly 14 more minutes in the
posttest to com plete tasks, they were able to complete tasks during the posttest in the same amount o f time
they took to complete the same tasks in the pretest. Thus, a nonsignificant finding here actually is
important. The other finding o f interest is that after a I-week delay, subjects could complete the tasks
within ju st over a minute o f the time it took to complete the same tasks the previous week. This finding,
coupled with the fact that subjects completed about the same amount o f tasks with nearly the same amount
o f accuracy, is relevant.
FitnJinu 3
It was generally hypothesized that users would be more effective and efficient at identifying and
correcting errors if they had direct access to error information. Three online help systems were developed:
one without any error information (without), one with error information located in a '"Troubleshooting"
section (distant), and one with error information located right next to the error-prone steps (proximal). It
was predicted (Hypothesis 4) that subjects in the proximal group would outperform subjects in the other
two groups. Instead, the presence and location o f error information did not seem to effect the number o f
errors subjects committed, the percentage they corrected, or the time they spent recovering from errors.
However, additional repeated measure analysis revealed statistically significant differences
between the pretest and posttests on the number o f errors committed. In other words, subjects committed
fewer errors after exposure to one o f the three minimalist online help systems. O f special interest was the
fact that the subjects in the proximal group corrected less errors overall than did the subjects in the other
groups even though they had error information located right next to the error-prone steps. Their rate o f
errors corrected paralleled that o f subjects in the inferential-without group (without any error information)
and even dropped below during the posttest.
110
Finding 4
It was generally hypothesized that users who used an online help system that was matched to their
learning style would complete software application procedural skills more quickly and with fewer errors
than subjects who used a nonmatched type o f online help. Specifically, it was predicted that subjects who
preferred a verbal style o f learning would complete tasks more accurately and efficiently using skeletal
online help (Hypothesis 5), and subjects who preferred a visual style o f learning would complete tasks
more accurately and efficiently using elaborated online help (Hypothesis 6).
An interaction between verbal learning score, time to complete tasks, and online help system was
obtained during the pretest (that is, before the subjects were actually exposed to any online help system),
which indicated that groups differed along this attribute prior to exposure to the different treatments.
Subjects in the skeletal group were different from the subjects in the other two groups: there was inverse
correlation between learning score and time to complete tasks for subjects in the skeletal group. This
difference among the groups was also evident in the interaction between accuracy and verbal-learning
measure as assessed during the treatment: there was a positive correlation between learning score and
accuracy o f task completion for subjects in the skeletal group. However, because the skeletal group differed
in the pretest, the differences in their scores on the accuracy measure during the treatment are suspect.
There were no interactions between visual learning score, the performance measures, and online
help systems (Hypothesis 6).
Conclusions
Based on the findings from the research hypotheses, the following conclusions were made:
1. As evidenced by the equivalent posttest scores related to the completion and accuracy o f tasks
among the three groups, it is possible to say that, based on the way the subjects used the different help
systems, the different applications o f minimalism to online instruction were fairly equivalent in their overall
effect on learning. That is, although the online help systems were different in the amount and type o f
instructional support provided, the subjects’ use o f the features which were consistent across the systems
Ill
and nonuse o f the features which varied across the systems may have resulted in the systems appearing
equivalent in their effectiveness.
In addition, based on the increase in scores from the pretest to the posttests related to the
completion and accuracy o f tasks and decrease in scores related to the number o f errors committed and time
taken to recover from errors, it is possible to say that learning did occur. That is, given a strong base o f
minimalism, the slight design differences preferred by different instructional designers (e.g., providing
more or less explanations, feedback, graphic support) will make no real difference to the end users: they
will still learn how to complete the tasks.
2. As evidenced by the lack o f any statistically significant findings associated with the time it
took for subjects in the treatment groups (who were intermediate-level users) to learn or complete tasks
( Finding 2) during the untimed testing period, it is possible to say that time is not a differentiating factor
among more experienced users using different implementations o f minimalist online help.
3. As evidenced by equivalent pretest and posttests times to complete tasks with increased
completion and accuracy rates, it is possible to say that using minimalist instruction increases the speed
with which subjects can complete tasks accurately.
4. As evidenced by the lack o f any statistically significant findings associated with the presence
and location o f error information in online instruction (Finding 3). it is possible to say that the placement o f
error information (either next to the error-prone step(s) or in a distant location) may not be beneficial in
reducing the number o f errors users commit or helping them recover from errors they do commit.
However, lack o f use o f error information may actually be an issue o f access rather than o f
minimalist content. Getting subjects to access error information by clicking on a button or hypertext link
appeared to be a problem. Therefore, one may conclude that providing separate error information via button
or hypertext link may be ineffective in assisting subjects in preventing or recovering from errors.
5. As evidenced by a decrease in the number o f errors subjects committed during Experiment 1
and Experiment 2 (Finding 1 and Finding 3), it is possible to say that subjects who are exposed to a solid
base o f minimalist instruction will leam to complete more tasks with fewer errors.
112
6. As evidenced by lack o f interaction between time, completeness, and accuracy measures and
both verbal- and visual-learning preference scores for subjects in the two treatment groups that did not
differ in the pretest assessment (Finding 4), it is possible to say that a subject’s preference for a verbal- or
visual-learning style is not a predictor o f student performance given a specific implementation o f
minimalist online instruction.
Discussion
Minimalism is an effective approach to developing online instruction. Different limitations in this
research may have accounted for the lack o f statistically significant findings in support o f one
implementation o f minimalism over another.
Overall Experimental Design
Taking a more minimalist approach to this study may have been beneficial. This research effort
was an attempt to complete on one grand scale in a condensed time period what Carroll (1990) and Black et
al. (1987) completed iteratively on a smaller scale over different time periods. Carroll (1990) used 19
subjects in three 8-hour work days for Test 1. used 32 subjects in a 5-hour time period for Test 2. joined
with Black et al. (1987) and used 20 subjects in about a 2-hour time period for Test 3. and used six subjects
in three 8-hour workdays for Test 4. Carroll revised and refined his experimental conditions between each
experimental session to test a different principle o f minimalism or to refine the testing o f the same
principle. The current research used the data from 111 subjects (which, with a 2.5% unemployment rate in
the valley, took 6 months to obtain), gathered in one 4- to 5-hour time period at one session and another 1-
liour session a week later.
In addition, where Carroll focused on one set o f principles for each test, this research was an
attempt to split the experimental session into two different experiments. The first two tasks the subjects
completed were associated with the application o f Carroll’s (1990) and Black and others’ (1987) skeletal,
elaborative, and inferential manuals to online instruction. The last three tasks o f the same activity were
associated with the Principle 3, support error recognition and recovery, as described by Van der Meij and
113
Carroll (1995) and tested by Lazonder (1994). It may have been more beneficial to run a series o f smaller
experiments focused first on one principle o f minimalism and then on another.
Subjects' Prerequisite Experience and Knowledge
The sample population for this research is typical o f today’s intended target audience, but is quite
different from the sample population used by Carroll (1990), Black et al. (1987), and Lazonder ( 1994).
Whereas the early experiments testing the different implementations o f the principles o f minimalism used
subjects with little or no computer experience (i.e.. nearly novice users), this research indicated that the
typical users o f online instruction to learn intermediate- to advanced-level tasks in an office setting would
be 18 to 25 years o f age, have up to 10 years o f computer experience, possess at least an intermediate level
o f experience in the given application, and own their own Windows platform computer. This great
difference in sample populations may have accounted for much o f the lack o f statistical differences.
Slight differences in the design o f good instruction make little difference to a target audience with
intermediate to advanced computer skills: their ability to leam and complete computer-based tasks
outweighs slight variances in the instruction. The fact that subjects felt confident enough in their own
ability' to explore and leam from the interface without (as Carroll would say) letting the instruction get in
the way o f learning, lends support to the theory o f minimalism. Subjects used the base o f minimalist
instruction in the different online help systems and tended to ignore the different instructional components
that varied among the systems. Instead, they interacted directly with the system to leam how to complete a
given task.
Samplinu Error
Although great care was taken to randomly assign subjects to the different experimental groups,
there was a statistically significant difference among the groups associated with the percentage o f errors
corrected (inferential-without group was different). This difference in groups was not maintained across the
treatment or posttests. The difference during the pretest could have been attributed to either sampling error
or coding error (although no coding errors were located after a cursory investigation). The statistically
114
significant differences here caused the statistically significant differences on this measure between the
pretest and posttests to be suspect.
Another sampling-error limitation was associated with the statistically significant interactions
related to subjects’ scores on the verbal-learning measure. Although there were no statistically significant
differences among the groups on visual- or verbal-leaming scores, there were statistically significant
differences among the groups during the pretest. Subjects in the skeletal group differed from the other two
groups on the interaction between verbal-leaming score and completion rates: subjects in the inferential
group differed from the other two groups on the interaction between verbal-leaming score and accuracy
rates. It is possible that the groups differed along some other characteristic that was not identified by any of
the measurement instruments.
Time Factor
Related to the first limitation, overall experimental design, the time factor may have been another
limitation o f this study. For the pretest, subjects were given the amount o f time it took an expert (about 32
minutes total) to complete all five tasks. Subjects then took a short refreshment break before going back
into the lab for Activity 2, which lasted between 2 and 3 hours. Subjects generally took a quick break about
one to one and one-half hours into Activity 2 while the lab consulted saved the first video and loaded the
second help system. Subjects then completed the last three tasks, which took approximately one to one and
one half hours. By the time subjects began Activity 3 (the immediate posttest), they had been in a learning
situation for approximately two and one half to three hours with little nutritional support (the sessions
started at 6 p.m.: many subjects came directly from work). Subjects made mistakes during Activity 3 on
tasks that they had executed correctly in the previous activity, which suggests that perhaps the internal
validity' threat o f testing effect (i.e., subjects were getting tired) had an effect on performance outcomes.
Another time-related limitation was the amount o f time allocated for the posttests (which was
determined in large part on the budget: an additional 15 minutes per subject multiplied by all the subjects
was very' expensive). The posttests were each 45 minutes long, divided proportionally among the five tasks.
The resulting time per task was one and one half the time it would take an expert to complete the same
task. This may have created an artificial ceiling for some. Others completed the tasks well within the
115
allotted time, but then spent the remainder o f the time tweaking their work. The data coders discussed the
option o f ending the time upon observed completion o f task (which may have resulted in statistically
significant differences for time: Hypothesis 2), but decided against that option because it would introduce
too much o f an instrumentation threat to the internal validity o f the study (Campbell & Stanley, 1963).
Instead the stopping time for each timed task was the time the subject entered as the stopping time.
Experimental Mortality
Campbell and Stanley (1963) defined experimental mortality as the "differential loss o f
respondents from the comparison groups” (p. 5). A quick analysis o f the 22 subjects who forfeited their pay
for the first session by either not showing up for the second session or leaving the first session before
completing the tasks revealed that a disproportionate amount o f these subjects (12) were in the inferential
group for Experiment 1. However, it is highly unlikely that the inclusion o f their scores would have
changed the direction o f the outcome. That is, the overall scores o f the inferential group would not have
increased with the inclusion o f the score o f those who left the experiment.
Lack o f Emotional Support for Subjects
In light o f research ethics, the researcher did not provide as much emotional support as perhaps
was possible. Carroll (1990) mentioned that three subjects became too frustrated with the study and were
excused. A similar situation occurred with at least three subjects in this research (presumably more if one
takes into account those that did not return the following week). O f the three known subjects, two were
struggling to the point o f tears (one just sat staring at her computer, unable to even move) and the
researcher quietly excused them from the study. The third subject broke into hysterical tears after the
pretest and would not even reenter the computer lab to retrieve her purse. All attempts to console her
magnified the wailing and sobbing.
If subjects were frustrated enough to leave or not return (even when it meant forfeiting their
payment), then it could be assumed that they did not leam the software application tasks. Ethically, the
researcher should have provided an alternative method o f instruction to ensure that subjects learned how to
complete the tasks before leaving the study.
116
Recommendations for Further Research
The following recommendations were based on the findings and conclusions o f this study in
relation to the discussion o f possible limitations:
1. Since there was a lack o f statistically significant differences (supporting the hypotheses)
among the treatment groups, then future research is recommended to test the different implementations o f
minimalism on a larger scale. Subjects were only required to complete two tasks in the current study
t Experiment I ). Future research may require that subjects complete all five tasks using either the skeletal,
inferential, or elaborative online instruction.
2. Again, since there was a lack o f statistically significant differences (supporting the
hypotheses) among the treatment groups, then future research is recommended with an emphasis on the
quantity and type o f learning that occurs. That is, given any type o f minimalist online instruction, will users
be prompted to explore and leam their software application better? Will they leam how to perform known
tasks in a more efficient manner?
3. Since the presence and location o f error information made no difference to performance
outcome (Finding 3. Conclusion 3), then future research is recommended to identify better ways to get
subjects to access the error information. The conflict arises between placing the error information directly
on the screen (where the subject could see it immediately) and keeping the contents o f the screen brief (by
having the subject access the error information via a hypertext link).
117
REFERENCES
Ackerman-Efron, J. (1985). States and reflective eye movements in the vertical dimension as an index o f
cognitive function in a verbal or visual modality. [On-line], Dissertation Abstracts Item. 46-06B. 2053.
Abstract from: DAI Accession No.: A A G 8516383
Black. J. B., Carroll. J. M.. & McGuigan, S. M. (1987). What kind o f minimal instruction manual is the
most effective? In H.J. Bullinger, B. Shackel & K. Komwachs (Eds.), Proceedings o f the second IF1P
conference on human-computer interaction ( dp . 159-162). Amsterdam: Elsevier.
Blair. J. D. (1980). Effects o f a self-administered audio-taped program on image awareness, use, vividness
and control as reported by college students. [On-line]. Dissertation Abstracts Item. 41-03 A. 0940.
Abstract from: DAI Accession No.: AAG80I9165
Boggan. S., Farkas, D.. & Welinske, J. (1996). Developing online help for Windows 95. London:
International Thomson Computer Press.
Borg. W.R., & Ball. M.D. (1989). Educational research (5th ed.). New York: Longman.
Brockman. J. R. (1986). Writing better computer-user documentation. New York: Wiley.
Brockman. J. R. (1990a). The why, where and how o f minimalism. SIGDOC Asterisk. J_4(4), 11 l- l 19.
Brockman, J. R. (1990b). Writing better computer user documentation: From paper to hypertext. New
York: John Wiley.
Campbell, D. T„ & Stanley, J. C. (1963). Experimental and auasi-experimental designs for research.
Boston: Houghton Mifflin.
Carlson. P. A. (1992). From document to knowledge base: Intelligent hypertext as minimalist instruction.
The Journal o f Computer Documentation. 16(1). 17-31.
Carroll. J. M. (1984). Minimalist training. Datamation. 30(18L 125-136.
Carroll, J. M. (1987). Five gambits for the advisory interface dilemma. In M. Frese, E. Ulich, & W. Dzida
(Eds.), Psychological issues o f human computer interaction in the work place (pp. 257-274). North
Holland: Elsevier Science Publishers B. V.
118
Carroll, J. M. (1990). The Numberg funnel: Designing minimalist instruction for practical computer skill.
Cambridge. MA: MIT Press.
Carroll. J. M„ Mack. R. L., Lewis. C. H.. Grischkowsky, N. L., & Robertson, S. R. (1985). Exploring a
word processor. Human Computer Interaction, i , 283-307.
Carroll. J. M.. Smith-Kerker. P. L., Ford. J. R„ & Mazur-Rimetz, S. A. (1987). The minimal manual.
Human-Computer Interaction. 3. 125-153.
Cline. J. A. (1991). Cognitive stvle in system design (Report No. IR 053 868). Unpublished master’s thesis.
Kent State University, Kent, OH. (ERIC Document Reproduction Service No. ED 343 586)
Dem. R. W. (1982). Learning instructions and cognitive coding in depressed and nondepressed college
students. [On-line]. Dissertation Abstracts Item. 43-05B. 1610. Abstract from: DAI Accession No.:
AAG8223271
Draper, S. W. (1996). Practical problems and proposed solutions in designing action-centered
documentation. [On-line]. Available: http://www.psy.gla.ac.uk/~steve/MinMan2.html
DutYy. T. M.. Palmer. J. E.. & Mehlenbacher, B. (1992). Online help design and evaluation. Norwood, NJ:
Ablex.
Farkas. D. K. (1993). The role o f balloon help. The Journal o f Computer Documentation. 17(2), 3-19.
Fishel. A. K. (1984). Clinical effects and personality correlates o f being a creative person and participating
in creative activity. rOn-linel. Dissertation Abstracts Item. 45-08B. 2685. Abstract from: DAI
Accession No.: A A G 8415804
Frese. M.. Albrecht, K.. Altmann, J. L., Papstein, P. V., Peyerl, R„ Prumper, J.. Schulte-Gocking, H.,
Wankmuller, I.. & Wendel, R. (1988). The effects o f an active development o f the mental model in the
training process: Experimental results in a word processing system. Behaviour and Information
Technologv. 7(3), 295-304.
Gagne'. R. M. (1985). The conditions o f learning and theory o f instruction. Chicago: Holt, Rinehart &
Winston.
119
Garro, L. C. (1982). Imagery’s role in cognition: Integrating imagery theory with an individual differences
approach. [On-line]. Dissertation Abstracts Item. 43-07B. 2371. Abstract from: DAI Accession No.:
AAG8227832
Green. S. B.. Salkind, N. J.. & Akey, T. M. (1997). Using SPSS for Windows: Analyzing and
understanding data. Upper Saddle River, NJ: Prentice Hall.
Greer. M. (1984). A trainer’s guide to software documentation. Performance and Instruction Journal. 23(3),
6-7.
Grice. R. A., & Ridgway. L. S. (1995). Presenting technical information in hypermedia format: Benefits and
pitfalls. Technical Communication Quarterly. 4( 1), 35-46.
Hackos. J. T„ & Stevens. D. M. (1997). Standards for online communication. New York: Wiley.
Hailey. D. E. Jr.. & Hailey, C. E. (1997). Hypermedia, multimedia, and reader cognition: An empirical
study. 45(3). 330-342.
Mallgren. C. (1992). Numberg funnel: A minimal collection. The Journal o f Computer Documentation.
16(1). 11-16.
Horn. R. E. (1992). Commentary on the Numberg funnel. The Journal o f Computer Documentation. 16(1).
3-11.
Horton. W. K. ( I990). Designing and writing online documentation: Help files to hypertext. New York:
Wiley.
Horton. W. K. (1994). Designing and writing online documentation: Hypermedia for self-supporting
products. New York: Wiley.
Jansen. C. (1994). Research in technical communication in the Netherlands. Technical Communication.
44(2). 234-239.
Jonassen, D. H.. & Wang, S. (1993). Acquiring structural knowledge from semantically structured
hypertext. Journal o f Computer-Based Instruction. 20(1). 1-8.
Kearslev. G. (1985a, May). Helps, simulations, and intelligent tutors: An overview o f embedded training.
Data Training. 32-38.
120
Kearsley. G. (1985b). Embedded training: The new look o f computer-based instruction. Machine-Mediated
Learning. J_(3), 279-296.
Kini. A. S. (1993). Effects o f cognitive learning style and verbal and visual presentation modes on concept
learning in computer-based instruction (verbal presentation, presentation modes). [On-line].
Dissertation Abstracts Item. 54-08A. 2954. Abstract from: DAI Accession No.: AAG9403528
Kirby. J. R., Moore, P.J.. & Schoefield, N.J. (1988). Verbal and visual learning styles. Contemporary
Educational Psychology. 13. 169-184.
Kleinbaum. D. G.. Kupper, L. L.. & Muller. K. E. (1988). Applied regression analysis and other
multivariable methods (2nd ed.). Belmont, CA: Duxbury.
Kuchler. C. J. (1983). Hemispheric laterality and cognitive style. [On-line], Dissertation Abstracts Item. 44-
Q5B. 1597. Abstract from: DAI Accession No.: AAG8320841
Lawless. K. A.. & Brown. S. W. (1997). Multimedia learning environments: Issues o f learner control and
navigation. Instructional Science. 25. 117-131.
Lazonder. A. W. (1994). Minimalist computer documentation: A study o f constructive and corrective skills
development. Enschede, The Netherlands: Universiteits Bibliotheek, Universiteit Twente.
Liebenman. J. (1987). Implication o f adult learning characteristics and learning styles for the design o f
software documentation. The Technical Writing Teacher. 14(2). 219-231.
McKnight. C.. Dillon, A.. & Richardson. J. (1996). User-centered design o f hypertext/hypermedia for
education. In D. Jonassen (Ed.). Handbook o f research for educational communications and technology
(pp. 622-633). New York: Simon & Schuster Macmillan.
Mirel. B. (1998). Minimalism for complex tasks. In J. M. Carroll (Ed.), Minimalism bevond the N um bers
funnel (pp. 179-218). Cambridge, MA: MIT Press.
Nichols. M. C. (1994). Using style guidelines to create consistent online information. Technical
Communication. 41(3). 432-438.
Nowaczyk. R. H.. & James, E. C. (1993). Applying minimal manual principles for documentation o f
graphical user interfaces. Journal o f Technical Writing and Communication. 23(4). 379-388.
121
Palmiter, S. Elkerton, J., & Baggett, P. (1991). Animated demonstrations vs. written instructions for
learning procedural tasks: A preliminary investigation. International Journal o f Man-Machine Studies.
34*687-701.
Pantin. H. M. (1982). The relationship between subjects’ predominant sensory predicate use, their preferred
representational system and self-reported attitudes towards similar versus different therapist-patient
dyads. [On-line], Dissertation Abstracts Item. 43-07B. 2350. A bstract from: DAI Accession No.:
AAG8229208
Parzivand, A. (1984). Effects o f habitual modes o f thinking and stress on production o f creative responses.
[On-line], Dissertation Abstracts Item. 46-02B. 0678. Abstract from: DAI Accession No.:
AAG8507881
Pedhazur. E. J.. & Schmelkin, L. P. (1991). Measurement, design, and analysis. Hillsdale. NJ: Erlbaum.
Pratt, J. A. (1998). Where is the instruction in online help systems? Technical Communication. 45(1). 33-
37.
Price. J. (1993). Comments on balloon help. The Journal o f Computer Documentation. 17(2). 21-22.
Reddish. J. C. (1988). Reading to leam to do. The Technical Writing Teacher. 15(3), 223-233.
Reznich. C. B. (1993). Minimalist design principles applied to the problem o f computer anxiety (Doctoral
dissertation. Michigan State University, 1993). Dissertation Abstracts International. 54(10 A), A AC
9406544.
Rogers. H. G.. & Brown, F. W. (1993). The impact o f writing style on compliance with instructions.
Journal o f Technical Writing and Communication. 23(1), 53-71.
Rosenbaum, S. (1998). Follow-up on training in minimalism: How are technical communicators using
minimalism? In J. M. Carroll (Ed.), Minimalism beyond the N um berg funnel (pp. 119-147).
Cambridge, MA: MIT Press.
Snapshot. (1998. Spring). Stages: The Fidelity Investments Magazine o f Personal Finance, p. 3.
Spiegel, C. A. (1985). A comparison o f graphic and conventional teaching o f polynomial operations by
means o f CAI (algebra, mathtiles, manipulatives, visualizer). [On-line]. Dissertation Abstracts Item. 46-
10A. 2954. Abstract from: DAI Accession No.: AAG8528003
122
Stevens. J. (1996). Applied multivariate statistics for the social sciences (3"* ed.). Mahwah, NJ: Erlbaum.
Techwriters. (1997). Minimalism/minimalist discussion. Cambridge Publications [On-line], Available:
http:'w \vw .documentation.com/techwrit
Talburt, J. R. (1993). Comments on "The role o f balloon help” by David Farkas. The Journal o f Computer
Documentation. 17(2). 20-21.
Vanderlinden. G., Cocklin. T.G., McKita, M. (1988.). Testing and developing minimalist tutorials: A case
history. Paper presented at the 35th International Technical Communications Conference. Washington.
DC.
Van der Meij, H.. & Carroll. J. M. (1995). Principles and heuristics for designing minimalist instruction.
Technical Communication. 42(2), 243-261.
Van der Meij, H.. &. Carroll. J. M. (1998). Principles and heuristics for designing minimalist instruction. In
J. M. Carroll (Ed.), Minimalism bevond the Numberg tunnel (pp. 19-53). Cambridge. MA: M IT Press.
Warner, J. W. (1989). The effectiveness on adult learners of minimalism design theory on learning, attitude,
anxiety, and time-on-task for self-study learning environments (Doctoral dissertation. Michigan State
University. 1988). Dissertation Abstracts International. 49(091. AAC 8814915.
Wenger, M. J.. & Spyridakis. J. H. (1993). Reduced text structure at two text levels: impacts on the
performance o f technical readers. Journal o f Technical Writing and Communication. 23(4). 333-352.
WinHelp. (1997). Minimalism/minimalist discussion. Cambridge Publications [On-line]. Available:
http://www.documentation.com/winheIp
123
APPENDICES
124
Appendix A
Review o f Research on Minimalism
125
Table X
Review o f Research on Minimalism
Author(s) Focus o f research Statistically significant findings ES
Black. J. B„ Compared four Inferential (I) and Skeletal groups N/A, but the
Carroll. J. M.. different types o f completed the overall learning task 24 findings are in
& McGuigan, manuals: inferential, and 18 minutes, respectively, faster than minutes for ease
S. M. (1987). skeletal, elaborative, the Elaborative group. o f comparison
rehearsal Inferential (I) and Skeletal groups
completed the simple learning test 1.7
and .8 seconds, respectively, faster than
the Elaborative group.
Inferential (I) and Skeletal groups
completed the command sequence test
10.5 and 7.1 seconds, respectively, faster
than the Elaborative group.
Inferential (I) and Skeletal groups
completed the realistic task tests 7.6 and
6.3 minutes, respectively, faster than the
Elaborative group for the memo task and
7.5 and 6.2 minutes, respectively for the
letter task.
(table continues)
126
Frese et al. steps alone vs. steps Performance: Guided exploration (GE)
(1988) with mental model could use the commands taught in the
diagrams vs. guided course better than Steps (S) (A = 1.5) 1.5
exploration to teach Transfer: GE could complete the task
basic word processing better and in less time than S (A = 1.2) 1.2
tasks Efficiency: GE slightly more efficient
(i.e., used less keystrokes to complete a
task) than S (A = .91). .91
Corrected error time: GE and E made
less errors and corrected them faster than
S (A = 1.43 and 1.12, respectively). 1.43 ' 1.12
Recall 2nd day: GE recalled more
commands than steps with mental model
(A = 1.36). Note: these differences 1.36
washed out by the 3rd day.
Lazonder la. minimal manual MM took less time to train than SSM -1.10
(1994) MM started tasks sooner than SSM 1.44
1b. experience MM printed (i.e., completed tasks)
sooner than SSM .74
MM completed basic and retention tasks
more quickly 67/.4S
(table continues)
127
MM completed retention tasks more
quickly .5
more experienced faster than novice -1.11.62
MM had less errors than SSM
more experienced corrected more errors .3
MM completed more tasks .93
more experienced completed more tasks .93
MM more efficient (tasks/time) .96
more experienced more efficient 1.23
MM more efficient with error recovery .51
MM more effective with error recovery
error information error information provided no
statistically significant differences on
procedural skills
MM faster at correcting errors in
training/practice .51/.8
MM faster at detecting errors in practice .5
SMM made more syntactic errors in
testing .59
MM detected more syntactic errors in
testing 67
MM corrected more semantic &
syntactic errors during testing ,79/.65
(table continues)
128
Nahl- Elaborated vs. no statistically significant differences fo r
Jakobovits unelaborated text to type o f instructional text (i.e.. elaborated
(1994). teach CD-ROM vs. unelaborated) on success,
searching software satisfaction o f searching, level o f
frustration, perceived self-efficacy or N/A
knowledge scores.
Users o f the elaborated text reported
they found the instructions more helpful
and comprehensible (F = 5.4 and 3.9.
respectively; p>.05; no A).
Reznich minimalism to reduce no statistically significant differences
(1993) computer anxiety for:
* ability to complete word processing
tasks
* time to complete word processing N/A
tasks
* reduced computer anxiety
the only statistically significant
difference favored the control group:
* higher motivation to leam more
computer skills
(table continues)
129
A u th o rs) Focus o f research Statistically significant findings ES
Vanderlinden, Guided exploration Guided exploration plus procedures N/A
Cocklin. & (plus procedures, when (GE+P) spent 10% more time working
M cK ita necessary) vs. self- through the tutorial. N/A
(1988) study tutorials to leam GE+P had less problems and required
a CAD system; slightly less interventions, N/A
emphasis on quantity o f GE+P completed the post-tutorial test 2
errors and ability to times faster (GE+P mean = 11 minutes
complete complex tasks compared with self-study mean o f 22
minutes) and more accurately N/A
Self-study subjects referred back to the
tutorial 4 times as much as the GE-^P to
complete tasks. N/A
W arner effect o f verbiage Written Achievement:
( 1987) quantity and placement a. no diff. between Activities at end
o f activities on (AE) and Activities throughout (AT).
achievement, time, A=0 0
anxietv and attitude b. low verbiage (LV) scored higher
than high verbiage (HV). A = .52 .52
Performance Achievement:
a. AT scored higher than AE. A = .38 -38
b. LV scored higher than HV. A = .54 .54
(table continues)
130
Attitude:
a. AE had more positive attitude than
AT. A = .83 .83
b. HV had more positive attitude than
LV. A = .69 .69
Anxiety:
a. AE had higher anxiety than AT. A =
.36 36
b. HV had higher anxiety than LV. A =
.69 69
Time on Task:
a. AE faster than AT. A = 1.94 1.94
1.45
b. VL faster than VH. A = 1.45
131
Appendix B
Kelly Temporary Services
Training Outline
132
K e lly T e s tin g & T r a in in g M a n a g e m e n t S y s t e m
PinPoint
The Power o f Learning at Work
Word 97
Training Outline
B a s i c S k ills I n t e r m e d i a t e S k ills A d v a n c e d S k ills

• Create a New Document Copy between Files Record a Macro
Using a Template Create a Text Box Run a Macro
• Save a File Insert a Text File Format Text Columns
: . Close a File Create a Header Insert a Picture
i • Open a File Create a Different First Move a Picture
j • Save As Page Header Add a Bookmark
• Display the Formatting Format Text as a Go to a Bookmark
Toolbar Numbered List
Insert a Footnote
; • Replace Text Use AutoFormat to Create
Insert a Table of Contents
{ • Insert Text a Bulleted List
Mark an Index Entry
! • Delete Text Insert a Symbol
Insert an Index
! • Insert Text Apply a Style
Add a Callout
I • Move Text (Cut and Paste) Modify a style
Add a Border to Text
• Copy Text Modify a Template
Insert a Section Break
• Undo Last Action Create a Table
Add a Border to an Object
. Enhance Text (Bold, Move a Column
Wrap Text around an
Underline) Copy a Row
Object
• Use Format Painter Insert a Row
Apply Outline Numbering
• Align Text (Center) Delete a Row
Insert a Comment
• Find and Replace Sort Table Data
Create a Chart
• Display the Ruler Add Borders to a Table
Change the Chart Type
• Clear All Tabs Create an AutoText Entry
Insert a Date
• Set a Tab Insert an AutoText Entry
Create a Data Source
• Move a Tab Create an AutoCorrect
Insert Merge Fields
; • Insert a Page Break Entry
Merge
i • Spell Check Insert an AutoCorrect Entry
Create Mailing Labels
• Change to Normal View Indent a Paragraph
• Use Office Assistant Change Page Margins
• Help (Contents, Index) Insert Page Numbers
R 1 2 /9 7 Word 97 Training
© 1997 Kdly S e r v i c e s . Inc
US P a t 5.602.982. Pinpoint is a registered Trademark of Kelly Services. Inc
Appendix C
Lener o f Informed Consent
p 't o r - 'i £ d
Instruction in Microbunts:
OEPAitTstENrc?iNsrsLcncH^L TcCM,sct.c<:Tê Study o f Minimalist Principles
College o» Educmon Applied to Online Help
logon. uunM J” .:jJU
ty«pl'Qnei3i>l I
fixiaon.~ir-:si]
6/5/91
Dear Participant:
Utah S ate University a iavalved la a research study to improve die eSedyesess of anBee help systems. You were
selected lor this study because of your registration in die derieal pool o f the Kedy Services temporary employment agency.
Your participation in this study is based on the assumptions chat you are (a) familiar with a word processing application
(e.g., MS Ward or WordPerfect) and (b) not an expert in all the advanced funeioas of.US Ward You may be ineligible to
participate in this study if you are either x.ocperienced with computers and ’word processors or demcnsoare a high level o f
expertise.
Your involvement in this study is comprised of completing some iatbnaationai questionnaires, learning and completing
some new word processing sksils, and evaluating an online help system. You will csmplem these acdvities over two
sessions, scheduled one week apart, at a Utah State University computer lab. The first session is scheduled to last
between four and five hours (although you may be able to complete it -in three hours): the second session is scheduled to
last approximately one and one-half hours. You m ust complete each session in order :o paid:
You may withdraw from this study at any time without ronsequer.ee I f you are a Utah State University student, then
withdrawal will not arTrc: your grades in any way. However, please keep in mind Sat payment for Sis temporary
employment Srough Kelly Services is contingent upon your completion o f hath sessions, a week apart.
Your concdendaiicy in this study 'will be maintained in the following manner
♦ You will use a special number instead o f your name on ail dccumemanon. This number will be assigned to you
by Kelly Services and will be similar to OCO-OQ-OlOl Tae research personnel wi3 idem!?/ you by that number
only and your responses to anything in the study will be associated with that number only The only association
between your name and your assigned number will be in the Kelly Services: secured database tor payment
purposes. .Vote: you must use your assigned number when checking in at bed: sessions o f the srody in order to
be paid by Kelly Services.
♦ Ad scores, responses and data collected from your participation in tae study wiE be kept :n a locked ale cabinet
at the researcher's home omce. Only the researcher will have access to the ale cabinet All informittcn collected
at Keily Services wiil be secured In their employee ale cabinets until retrieved by the researcher
♦ Payment to you for mis temporary employment will be made directly through Keily Services.
B enefit to you. We appreciate your paructpiticn In this study 1: is our hope h at you will beaesc mom It by learning
some word-processing skills that will make you more marketable. There is no risk to you associated wtth this study
Flense sign both eopies and keep one for your records.
Agreem ent fo r participation. 1 foteae er-.it same hcrei ... have read the above
research descnpr.cn and consent to participate Ia both sessions o f this srody of oniine heip systems. I further agree to
allow all data generated by me and collected by computer or surveys to be used in the research sctdy
Signature of Participant
ir. M. David Mhrril lean A. Pratt

Major Professor Ph.D. Candidate Researcher
Department at Instructional Technology Department of Instmcnonal Technology
Utah State University Utah State University
Questions about this study cstt be directed to Jean A. Pratt at prattb/Spcuntt.
135
Appendix D
Background Information Questionnaire
136
Background Information Questionnaire ID Number
Please circle the answer that applies to you fo r each question.
1. G ender: M F
2. Age: 18-25 26-30 31-35 36-40 41-45 46-50 51-55 56-60 61-65 o v e r6 5
3. C o m p u ter Experience: 6-12 months 1-2 years 2-5 years 5-10 years over 10 years
4. W ord processing applications used (please check all that apply and circle level o f expertise):
MS Word Beginner Intermediate Advanced
Corel/W ord Perfect Beginner Intermediate Advanced
Other Beginner Intermediate Advanced
G rap h ics/d raw in g applications (please check all that apply and circle level o f expertist
Paitu Beginner Intermediate Advanced
Paintbrush Beginner Intermediate Advanced
Paint Shop Pro Beginner Intermediate Advanced
Adobe Photoshop Beginner Intermediate Advanced
Other Beginner Intermediate Advanced
6. O th e r types o f com puter applications (please name and circle level o f expertise):
________________________ Beginner Intermediate Advanced
_________________________ Beginner Intermediate Advanced
________________________ Beginner Intermediate Advanced
7. C o m p u ter platfo rm with w hich I am most fam iliar: Macintosh PC/IBM clone
8. I own a p erso n al com puter. Yes No
137
Appendix E
Verbaiizer-Visuaiizer Questionnaire
138
Learning Preferences Questionnaire CD Number_
Please circle the number that applies to you.
Agree — Disagree
1. I enjoy doing work that requires the use o f words. 4 5
2. 1 don't believe that anyone can think in terms o f mental
pictures.
3. I enjoy learning new words. 4 3
4. I find illustrations or diagrams help me when I'm reading. 4 5
1 have a hard time making mental pictures o f a place that 4 5
I've only been a few times.
6. 1 can easily think o f synonyms for words. 4 5
7. I read rather slowly. 4 5
8. I seldom use diagrams to explain things. 4 5
9. I like newspaper articles that have graphs. 4 5
10. 1 prefer to read instructions about how to do something 4 5
rather than have someone show me.
I don't like maps or diagrams in books. j 4 5
12. I have a better than average fluency in using words. 2 3 4 5
13. 1 spend little time attempting to increase my vocabulary. 2 j 4 5
14. When 1 read books with maps in them, 2 3 4 5
I refer to the maps a lot.
15. I dislike word games like crossword puzzles. 2 3 4 5
16. The old saying "A picture is worth a thousand words" is 2 3 4 5
certainly true for me.
17. 1 dislike looking words up in the dictionary. 2 3 4 5
18. 1 have always disliked jigsaw puzzles. 2 3 4 5
19. I find maps helpful in finding my way around a new city. 2 3 4 5
20 . I have a hard time remembering the words to songs. 7 j 4 5
Note: I will put your learning preference score in your file for you in case you are curious.
Thanks for your participation.
Appendix F
Session I Experimental Materials
140
O verview
WelcomeI Thank you fo r participating in this study.
This orientation packet is designed to guide you through the activities for this session. We will preview the
session activities as a group before you start. You may refer back at any time to the instructions included in
this packet.
S e s s io n A ctivities
During today's session you will complete three very similar activities using different content, but comprised
o f the same five tasks:
1. Create and format regular and AutoShape text boxes

2. Insert text; format it into columns
3. Insert graphics; wrap the text around them
4. Create a table; format it with borders, shading, etc.

5. Use mail merge to create mailing labels
All three activities you complete today resemble partially-completed newsletters. A description o f each
activity follows:
• Create a portion o f a sales report (see enclosed sample): use mail merge to create mailing labels. (35
minutes)
The intent o f this first activity is to introduce you to intermediate- and advanced-level skills in MS
Word that will make you more employable. You will attempt to complete five tasks in MS Word
without any help. You will have 35 minutes to complete this newsletter/sales report.
Note: Although each task is timed, feel free to work at your own pace. Note: it is not expected that you
know how to complete each task— these are intermediate and advanced-level tasks in MS Word. It's
OK if you do not know how to complete them now— you will be able to use the online help system to
complete these same tasks in the second portion o f this session. Just do the best you can.
* Create a newsletter; use mail merge to create mailing labels, (approximately 2 hours)
This second activity is to enable you to leam the same MS Word skills using an online help system.
Note: this is the only time you will have any instruction to leam these new skills. Everything you need
to know is included in the online help system. The online help was developed specifically for this study
so is limited to just the help you need. Using the online help system will prevent you from experiencing
a lot o f frustration as you attempt to leam these skills.
A c c e s s th e o n lin e help by clicking on Help | C o n t e n ts and Index.
You have two hours to complete all five tasks in this second activity. Please pace yourself so that you can
complete all five tasks and have time to review the skills prior to the third activity. You will not have any
online help available to you during the third activity, so you will want to spend some time reviewing the
skills prior to starting the third activity. Activity 3 takes about 50 minutes to complete. Leave yourself
enough time to complete it before the time you are scheduled to leave.
141
■ Create a portion o f a newsletter; use m ail merge to create m ailing labels. (SO m inutes)
This third activity provides you with an opportunity to demonstrate your new knowledge by
completing the same MS Word skills without the aid o f an online help system. N o online help will
be available for this activity.
A dditional In stru c tio n s
1. Work at your own pace to complete each task.

2. A complete set o f instructions for preparing the newsletter and completing a mail merge are included in
an abbreviated form o f the MS Word online help system. This help system will be available to you onlv
during the second activity.
3. You will receive a completed sample o f each newsletter as you begin each activity.
Use the sample as a guide.
N O TE: The sample is onlv a guide—
you do not have to m ake your copy look exactly like the sam ple.
Instead, focus y o u r attention on learning the principles o f the skills
(i.e., could you complete these sam e skills w ithout any assistance fo r a fu tu re em ployer?)
4. Most o f the text and all the graphic files for the newsletter are located in the Newsletter folder on the
hard drive. The file names are printed next to the corresponding text and graphics in the sample
newsletter. T he Newsletter folder is on the C:\ drive!
5. Please follow the instructions for each o f the activities. Make sure you have completed all the tasks
before you notify a lab consultant that you are through.
6. You will be directed to insert the time periodically throughout this ID# 123
session. D ate: 6/16/1999
S ta rt T ask 1: 13:59:03
a. Place your cursor next to the corresponding task number in the text End T ask 1: 14:04:25
box in the upper left comer.
S ta rt T ask 2: 14:04:45
b. Click on Insert | Date and Time. End T a sk 2: 14:08:03
S ta rt T ask 3: 14:08:25
c. Make sure there is no ch eckm ark in the Update A utom atically
End T a sk 3: 14:13:41
box!
S ta rt T ask 4: 14:13:54
d. Click on the very last time option (for example, 13:00:05) and click End T a sk 4: 14:26:57
O K . You will end up with something like this box in the upper left S ta rt T ask 5: 14:27:13
com er o f your screen. End T a sk 5: 14:35:35
7. There is a Stop button in the upper right comer o f your screen. You may click on the title bar and move
the Stop button, but do not click on th a t button. If for some reason the computer closes the recording
application, then notify a lab consultant immediately.
8. You will be prompted to save your w ork as you go.
It is very important that you include y o u r ID# (e.g. 15) in the filename o f all documents— use a /, 2, or 3 on
different documents to prevent overwriting earlier documents by the same name (e.g., I5listl.doc,
I5list2.doc. lSlist3.doc).
142
S a v e a ll d o c u m e n t s in t h e C : \N e w s le tte r fo ld e r .
a. In Word, click on File | Save As ...

b. Click on the broken up-arrow folder until y o d s^ e the B115 (C :) drive.
c. Scroll down (or over) until you find the Newsletter folder.
d. Type "00 O nam e. d o c " (where 000 IS YOUR ID# and name is the name o f your document) in
the File Name field.
e. Click on the Save button.
h. The lab consultants can only help you with computer problems. The online help system has the answers
to everything you need to do: it was developed specifically for this study— p le a se u se th e
online help system. In the first and third activities (those without online help), ju st do the best
you can to figure out or remember how to complete the given task.
10. We will be taking two five-minute breaks today: one between each o f the activities. Your lab consultant
will direct you to the restrooms and water fountains.
143
A ctivity Item #1: S a le s R eport & Mailing L abels
Creating a portion o f a newsletter and some mailing labels to mail it is comprised o f five main tasks. There
is a certain amount o f time allotted to complete each task. Just work at your own pace to complete each
task. If you do not know how to complete a particular task, think about how you might complete it and try
that. However, don't spend too long trying to figure out how to do any one part o f a task. If you can't figure
out how to do something, then ju st complete the task as best as you can.
I sample copy o f the partial sales report is enclosed in this packet. You may use it as a guide.
G etting S ta rte d :
l. Open the start.doc document (located in the Newsletter folder).
a. From Word, click on the O pen icon.

b. Click on the broken up-arrow folder l ^ l |until you see the B 115 (C :) drive.
d. Double-click on the Newsletter folder to open it.
e. Find and double-click on the start.doc file to open it.
2. Save the document as 000sales.doc in the C:\Newsletter folder (where 000 IS YOUR ID#).
b. Click on the broken up-arrow folder untiiy cu see the B115 (C :) drive.
d. Type " o o O s a l e s . d o c " (where 000 IS YOUR ID#) in the File Name field.
3. Type your ID# (e.g., #54, no* your social security number) in the text box at the top o f the page.
4. Position your icon next to “D ate:” in the text box and insert the date from Insert | D ate and Time.
Use the very first date format (6/18/1999).
5. W ait for the lab consultant to direct you to begin Task I.
144
Task 1: Use regular and shaped TEXT BOXES to create heading

Start:
Position your cursor after ’‘S ta rt T ask I” and insert the time from Insert | D ate and Tim e. Use the
very last time format (14:30:25).
I. Create a text box spanning from the border o f the date/time text box to the right margin and as
long as the date/time text box (i.e., align the bottom borders o f the two text boxes).
2. Type and center "Sales Report" in the text box.
3. Make "Sales Report" Arial, bold. 28 point.
4. Apply either a texture or pattern fill to the text box.
5. Apply a special border to the text box.
6. Apply to the text box a tight text wrap with a 0.1 external margin.
7. Create a “star" A utoSltape text box
8. Apply a different texture fill to the star.
9. Place the star over the "Sales Report” text box as shown.
10. Make the "W e're # 1 ” text white, Arial, 14 point font, bold, centered.
I I. Make a text box with rotated text as shown in the sample.
12. Apply the same texture and border as the “Sales Report” text box.
13. Make the text Arial, 12 point font, bold, centered.
14. Position text box over “Sales Report” text box as shown.
Stop:
Position your cursor after “ E nd T ask I” and insert the time from Insert | Date and Time.
Use the very last time format (14:30:25).
* SAVE your docum ent.
* Wait for the lab consultant to give you Task 2.
145
Task 2: Add and format text

Start:
Position your cursor after ’‘S tart Task 2” and insert the time from Insert | Date and Time. Use the
1. Bring in some existing text (use the M yD rink.doc file in the Newsletter folder); place it below the
text boxes.
2. Make the first line o f text ("Introduction") into a one-column heading.
3. Make the "Introduction" heading centered, bold. 12 point.
4. Insert a blank line after the "Introduction" heading
5. Place the remainder o f the text into three columns using the following dimensions:
a. Colum ns 1 and 3: 2 inches wide with a .25-inch space between columns
b. Colum n 2: 2.5 inches wide with a .25-inch space between columns
6. Make the last line o f text ("Table o f Quarterly Sales") and the remaining blank lines into a
one-column heading.
7. Make this heading centered, bold, 12 point.
S. Place the cursor at the beginning o f the "Table o f Quarterly Sales" heading and insert a blank
line.
Stop:
Position your cursor after “ End Task 2” and insert the time from Insert | Date and Time. Use the very
last time format (14:30:25).
w SAVE y o u r docum ent.
146
Task 3: Add and format picture

Start:
Position your cursor after “S ta rt T a sk 3” and insert the time from Insert | D ate an d Time. Use the
1. Position your cursor anywhere in the "Soft Drink Market" section o f the text.
2. Bring in a picture. (Use the drink.bm p file in the Newsletter folder.)

Note: the computers in this computer lab automatically go to the
“C lip a rt’ folder on the server. Just click on the broken up-folder
you see the B115 (C :) drive where you can find the Newsletter folder.
3. Resize the graphic so that it fits better in the text (use the sample as a guide).
4. Apply a tight text wrap to the graphic; edit the wrap points so the text contours the graphic.
5. Position the "drink" graphic so that the text wraps uniformly around it.
Stop:
Position your cursor after “ E nd T ask 3” and insert the time from Insert | D ate and Time. Use the
* SAVE y o u r d o c u m e n t
147
Task 4: Add and format table

Start:
Position your cursor after “S tart Task 4” and insert the time from Insert | Date and Time. Use the
1. Insert a few blank lines below the "Table o f Quarterly Sales" heading text.
2. Create the following table below the "Quarterly Sales" heading— pay attention to
10 point. Times New Roman, regular font
use o f bold for headings
I Vi point external b o rd er,'/: point internal border. 'A point double-lined internal border
cell merges
10% shading
text centered vertically and horizontally in cells
table centered horizontally on page
Use the sample as a guide!
3. Make sure all items fit on one page.
Stop:
Position your cursor after “ End Task 4” and insert the time from Insert | Date and Time. Use the very
* SAVE your document.
148
T a sk 5: C rea te th e m ailing lab els
Start:
Position your cursor after "S ta rt T ask 5” and insert the time from Insert | Date and Time. Use the
1. Use the Mail Merge Helper to create a list o f addresses for mailing labels in a new main
docum ent.
DO NOT C R EA TE TH E M AILING LABELS O VER YOUR “Sales Report”,
Use the following names and addresses:
<Your ID#> PersonA Jane Adams

123 E. 204 N. 4153 W. 544 S.
Logan. UT 84321 Providence, UT 84332
Marissa Reams Samuel Hodge

3897 E. 783 S. 1123 N. Main Street
Springdale, UT 84767 Laketown. UT 84038
2. Save the mailing list as "000listl.doc" (WHERE 000 IS YOUR ID#) in the C:\Newsletter folder.

b. Click on the broken up-arrow folder ^1 until you see the B 115 (C :) drive.
d. Type " 0 0 0 1 i s t l . d o c " (where 000 IS YOUR ID#) in the File Name field.
3. Format the mailing list as you would like it to be printed on 2160 Mini-address Avery Standard
labels.
4. Sort the labels in ascending order by Postai-code.
5. Merge the labels to a new docum ent: save as "000m aill.doc" (where 000 IS YOUR ID#).

b. Click on the broken up-arrow folder f^ lj until you see the B I15 (C :) drive.
149
c. Scroll down (or over) until you find the N ewsletter folder.
d. Type " 0 0 0 m a i l l . d o c " (where 000 IS YOUR ID#) in the File N am e field.
6. Close 000m aill.doc and 000listl.doc (where 000 IS YOUR ID#).
7. Save the new main document (Document/V) as 000m aitil.doc (where 000 IS YOUR ID#);
close it.

b. Click on the broken up-arrow folder S i until you see the B 115 (C :) drive.
d. Type " 0 0 0 m a i n l .d o c " (where 000 IS YOUR ID#) in the File N am e field.
8. Return to the Sales Report.
Stop:
Position your cursor after ” End T ask 5” and insert the time from Insert | D ate an d Xime. Use the very
* SAVE y o u r docum ent.
* Close the Sales Report.
******PLEA SE r a is e y o u r h a n d t o s u m m o n a l a b c o n s u l t a n t *******
150
Activity Item #2: C reate a N ew sle tte r
N ew sletter S c e n a rio
You have been hired temporarily (through Kelly Services) to work for a small, locally owned and operated
travel agency. The duties you normally perform in situations like this typically include invoicing customers,
paying suppliers, and preparing company correspondence. In this situation, however, the ow ner o f the
company wants you to publish a promotional newsletter for a January vacation get-away. Publishing a
newsletter involves many o f the skills that you could also use to complete other tasks (e.g., using mail
merge to prepare the form letters and mailing labels for customer invoicing).
You have about two hours to complete this small newsletter. Work at your own pace.
You have a dual goal:
■ complete the newsletter
* use th e o n lin e help system to leam new skills to be more efficient in performing other tasks.
You will be more employable i f you know these new skills.
Remember...
■ the text and graphics for this newsletter are contained in the Newsletter folder on the hard drive
* use the enclosed checklist to make sure you complete all tasks
* use the enclosed sample newsletter as a guide
* use y o u r online help system!
151
Getting Started
l. Open the newsletter.doc file in the Newsletter folder on the hard drive.

b. Click on the broken up-arrow folder until you see the B115 (C :) drive.
|-=j B115 (O) 3
d. Double-click on the N ewsletter folder to open it.
e. Find and double-click on the newsletter.doc file to open it.
2. Save the document as 000news.doc in the C:\Newsletter folder (where 000 IS YOUR ID#).

d. Type " 0 0 0 n e w s .d o c " (where 000 IS YOUR ID#) in the File Name field.
3. Type your ID# in the text box at the top o f the page.
4. Position your icon after “ D ate:” and insert the date from Insert | Date and Time.
Start:
Position your cursor after “S ta rt T ask 1” and insert the time from Insert | D ate and Tim e. Use
the very last time format (14:30:25).
152
l. Use the following formatting for the logo text
a. "Adventure" text box = Arial, bold, 28 point, centered white on black
b. "Travel" text box = Arial, bold, 28 point, centered, black on white, 2.25 pt. border
c. "January 1999" text box = Arial, bold, 10 point, all caps, rotated text; 3 pt. double
lined border
d. "Special Issue" AutoShape text box = Arial, bold, 12 point, centered, AutoShape
f. Highlight text boxes with borders and colors as shown
g. Position text boxes as shown in sample
h. Apply a tight text wrap to text boxes
Stop:
Position your cursor after “ E nd T ask 1” and insert the time from Insert | Date an d Tim e. Use the
* SAVE y o u r docum ent— rem em ber to save often while you work!
* Proceed to Task 2.
153
Task 2: Add and format text
Start:
Position your cursor after “S ta rt T ask 2” and insert the time from Insert | D ate an d X>me. Use
1. Position your cursor after the logo
2. Bring in first the fre e p o rtd o c and then the dolphins.doc text files
a. Create a one-column heading from the first line o f text “Freeport - Island o f Action
and Relaxation”
b. Change the heading font type and size to Times New Roman, bold, italic,18 point
c. Place the rest o f the text in three columns (2.17, 2.4, and 2.37) with .27 and .29
spacing
d. Delete the blank line before the first paragraph o f the "Freeport" article
e. Make the last line o f text ("Xanadu Resort") into a one-column heading.
f. Center the "Xanadu Resort" heading.
Stop:
Position your cursor after “ End T ask 2” and insert the time from Insert | D ate a n d Time. Use the
* SAVE y o u r docum ent— rem em ber to save often w hile you work!
*******PLEA SE RAISE Y O U R HAND T O SU M M O N A LAB C O N SU LTA N T *******
154
Task 3: Add and format graphics
Start:
Position your cursor after “S ta rt T ask 3” and insert the time from In sert | D ate and Time. Use
1. Place your cursor anywhere in the text.
2. Bring in pictures/graphics: dolphins.bm p, logo.bmp, palnubm p

"C lipart' folder on the server. Just click on the broken up-folder l*T)| until
you see the B115 (C:) drive where you can find the Newsletter folder.
a. Resize and position graphics in the text as illustrated in the sample
b. Position "dolphins.bm p” behind the "Travel” box: align the waterline and bottom box
border
c. Apply a tight text wrap to each o f the graphics
d. Edit the wrap on palm .bm p so that it goes in between each palm leaf
Stop:
Position your cursor after “ End T a sk 3” and insert the time from In sert | D ate and Jim e. Use the
155
Task 4: Add and format table (replicate the table in the sample)
Start:
Position your cursor after “S ta rt T ask 4” and insert the time from Insert | Date and Tim e. Use
1. Create the "Xanadu Resort" table illustrated in the sample
a. Format the text as shown (i.e., 12 point. Times New Roman and Arial, bold headings)
b. Left-align text in the left column; center text in the right column; center the titles
c. Vertically align all the text in the center o f each cell
d. Insert a 10% shading in the title cells
e. Make the external border a 1 '/: point solid line
f. Make the internal ceil borders '/: pt. single and double-lines
Insert double-lined internal cell borders as shown in the sample
h. Center the table on the page bottom
Stop:
Position your cursor after “ End T ask 4” and insert the time from In sert | D ate and Tim e. Use the
* SAVE your d o c u m e n t
156
Task 5: Create the mailing labels (i.e., performing a mail merge)
Start:
Position your cursor after " S ta rt T ask 5” and insert the time from In se rt | Date and Tim e. Use
1. Use the Mail Merge Helper to create a list o f addresses for mailing labels in a new m ain
docum ent.
DO N O T C REA TE T H E M A ILIN G LABELS OVER Y O U R TRAVEL N EW SLETTER.
<Your ID#> PersonB

130 E 200 S
Logan, UT 84321
Gerald Trunnell
4352 W 5200 N
Richmond, UT 84333
Serena Hoffman
1814 E 1200N
Providence, UT 84332
Ailene Gregory
434 N. Main St.
Cedar Valley, UT 84013
2. When prompted, save the mailing list as 000list2.doc (where 000 IS YOUR ID#) in the
C:\Newsletter folder.

b. Click on the broken up-arrow folder Hill until you see the B115 (C :) drive.
d. Type " 0 0 0 1 i s t 2 . d o c ” (where 000 IS YOUR ID#) in the File Nam e field.
157
3. Format the mailing list as you would like it to be printed on 2160 Mini-address Avery Standard
labels.
4. Sort the mailing list in descending order by postal code
5. Merge the labels to a new document; save as "000mail2.doc" (where 000 IS YOUR ID#).

b. Click on the broken up-arrow folder S | until you see the B 1 15 (C:) drive.
d. Type " 0 0 0 m a il2 .d o c " (where 000 IS YOUR ID#) in the File Name field.
6. Close 000mail2.doc (the mailing labels) and return to the main document (looks like this:)
«FirstName» «LastName»
«Address I»
«City». «State» «PostalCode»
«Next Record»«FirstName» «LastName»

«A ddressl»
«City», «State» «PostalCode»
7. Save the new main document (Document#) as 000main2.doc (where 000 IS YOUR ID#); close
it: click on "Y es" to save the associated "I231ist.doc."

b. Click on the broken up-arrow folder Gl 1| until you see the B 1 15 (C :) drive.
d. Type " 0 0 0 m a in 2 .d o c " (where 000 IS YOUR ID#) in the File Name field.
Stop:
Position your cursor after "E n d T a sk 5” and insert the time from In sert | D ate an d Tim e. Use the very
158
COULD YOU PERFORM THESE SAME TASKS
WITHOUT USING THE ONLINE HELP SYSTEM?
If NO. then please review the procedures in online help until you feel confident o f your ability.
(M ake sure you KNOW HOW to do all the above procedures
W ITHOUT assistance before proceeding.)
If Y ES, then please raise your hand to sum m on a lab consultant.
Let th e lab c o n s u lta n t know if you have ju s t fin ish ed th e n e w s le tte r of ju s t

fin ish ed review ing.
159
Activity Item #3: N ew sletter & Mailing L a b e ls
You now have the opportunity to demonstrate the skills you learned by creating a small portion o f a
newsletter and some mailing labels without the use o f an online help system. W ork at your own pace to
complete each task. If you do not know how to complete a particular task, think about how you might
complete it and try that. However, don't spend too long trying to figure out how to do any one part o f a
task. If you can't figure out how to do something, then just complete the task as best as you can.
.1 sam ple copy o f th e partial newsletter (taken fr o m Utah State University's H ope H ealth Letter) is
enclosed in this packet. You m ay use it as a guide.
Getting Started:
I. Open the end.doc document (located in the Newsletter folder),
=D B115 (C:)
e. Find and double-click on the end.doc file to open it.
2. Save the document as 000hope.doc (where 000 IS YOUR ID#) in the C :\N ew sletter folder.

b. Click on the broken up-arrow folder until you see the B 115 (C :) drive.
d. Type " o o o h o p e . d o c " (where 000 IS YOUR ID#) in the File N am e field.
4. Position your icon after “ D ate:” and insert the date from Insert | Date a n d T im e.
5. Raise your hand to let the lab consultant know you are ready to start Task I .
160
Start:
Position your cursor after “S ta rt T ask 1” and insert the time from In sert | D ate and Time. Use
1. Create a text box spanning from the border o f the date/time text box to the right margin.
2. Type and center the word "Hope" in the text box.
3. Make the word "Hope" Arial. bold, 125 point, white
4. Fill the text box with solid green.
5. Apply a special, black border to the text box.
6. Create a smaller text box with the same solid green fill, but with no border.
7. Type in HEALTH LETTER (Arial, centered, white, 12 point, all caps)
8. Rotate and position the "Health Letter” text box as shown
9. Apply a tight text wrap with a 0.1 external margin to the "Hope" text box
10. Create the “News for You!” AutoShape text box: Arial, 12 point, centered.
11. Fill the AutoShape with a solid color.
Stop:
Position your cursor after “ End T ask 1” and insert the time from In sert | D ate and lim e . Use the
* SA V E y o u r d o c u m e n t
161
Task 2: Add and format the text
Start:
Position your cursor after “S tart Task 2” and insert the time from Insert | Date and Time. Use
1. Position your cursor a few lines below the "Hope" banner.
2. Bring in some existing text. (Use the ltealth.doc file in the Newsletter folder.)
3. Use the first line o f text I" Research News... You Can Use!") as a one-column heading.
4. Center and bold the heading.
5. Format the remainder o f the text into three columns using the following dimensions:
a. Columns 1 and 3: 2.5 inches wide with a 0.25-inch space between columns
b. Column 2: 1.5 inches wide with a 0.25-inch space between columns.
6. Make the last line o f text (“Analysis o f Popular Diets...") it into a one-column heading.
7. Center the "Analysis..." heading.
Stop:
Position your cursor after “End Task 2” and insert the time from Insert | Date and Time. Use the
162
Task 3: Add and format a graphic
Start:
Position your cursor after "S tart Task 3” and insert the time from Insert | Date and Time. Use
1. Position your cursor somewhere in the text.
2. Bring in a picture. (Use the m edicine.bmp file in the Newsletter folder.)

"Clipart' folder on the server. Just click on the broken up-folder 1^11 until
you see the B115 (C :) drive where you can find the Newsletter folder.
3. Resize the graphic so that it fits better in the text.
4. Position the picture in the text so that it overlaps any two columns.
5. Apply a tight text wrap; edit the wrap so that the text wraps in a circle around the picture.
Stop:
* SAVE your do c u m e n t
163
Task 4: Add and format a table
Start:
Position your cursor after “Start Task 4” and insert the time from Insert | Date and Time. Use
1. Create the following table— meet the following criteria:
2 V* point external border with V- point internal cell borders
'/i point, double-lined internal cell borders under sub-headings
Merged cells
Text direction
Shading: 15% for title. 10% for "Bad" category, 5% for "OK” category
Text formatting: 9 point, centered and bold as shown, caps as shown
Make rows an even height
Vertically align all text in center o f each cell
Horizontally align the table in the center o f the page
Stop:
* SAVE your document
164
Task 5: Create the mailing labels
Start:
Position your cursor after '‘S ta r t T a sk 5” and insert the time from In sert | D ate and Time. Use the
I. Use the Mail Merge Helper to create a list of addresses for mailing labels in a new main
docum ent.
n o NOT CREATE THE MAILING LABELS OVER YOUR H E A L T H NEWSLETTER.
<Your ID#> PersonC Glen Hansen

277 N. 300 W. 400 N. 585 E.
Randolph, UT 84064 Petersboro, UT 84651
Thelma Smedley George Nourse

597 Eaale Dr. 583 N. Riverdale Rd.
Layton. UT 84745 Huntsville. UT 8 4 3 17
2. Save the mailing list as "000list3.doc" (where 000 is your ID#).
a. In Word, click on File | Save A s ...

b. Click on the broken up-arrow folder S | until you see the B I 15 (C :) drive.
d. Type " 0 0 0 1 i s t 3 .d o c " (where 000 IS YOUR ID#) in the File N am e field.
3. Format the mailing list as you would like it to be printed on 2 160 M ini-address Avery Standard
labels.
4. Sort the labels in descending order by Postal-code.
5. Merge the labels to a new docum ent: save the mailing labels as "000mail3.doc" (where 000 IS
YOUR ID#) in the C:\Newsletter folder.
a. In Word, click on File | Save A s ...

b. Click on the broken up-arrow folder © J until you see the B l 15 (C :) drive.
d. Type " 0 0 0 m a il3 . d o c " (where 000 IS YOUR ID#) in the File N am e field.
165

6. Close 000mail3.doc and 000list3.doc (where 000 IS YOUR ID#).
7. Save the new main document (DocumentAO as 000main3.doc (where 000 IS YOUR ID#); close it.

b. Click on the broken up-arrow folder untikvcu see the B 115 (C :) drive.
d. Type "0 0 0 m a in 3 .d o c " (where 000 IS YOUR ID#) in the File Name field.
8. Return to the Hope health letter.
Stop:
Position your cursor after "E n d T ask 5” and insert the time from Insert | Date and Tim e. Use the
* Close the Hope health letter.
Thank you very much for participating in the first half o f this study.
The second half o f the study will be conducted in one week.
Please check with your Kelly Services representative for your scheduled time.
Note: you will be paid only after participation in both th is an d next w eek's
se s sio n .
*******PLEA SE RAISE Y O U R HAND T O SUM M ON A LAB C O N SU LTA N T *******
166
Appendix G
Session 2 Experimental Materials
167
O verview
Welcome back to the second session o f this study.
This orientation packet is designed to guide you through the activities for this session. We will preview the
session activities as a group before you start. You may refer back at any time to the instructions included in
this packet.
S e s s io n A ctivities
During today's session you will have an opportunity to demonstrate your employable knowledge o f
intermediate and advanced word-processing skills in Microsoft Word. You will create a portion o f a
newsletter, completing the sam e five tasks you completed last week while learning advanced skills in Word.
After you finish the partial newsletter, you will have an opportunity to evaluate the online help system that
you used last week. You w ill have access to that online help system to assist you in this evaluation. A more
detailed description o f today's activities follows:
/. Create a portion o f a newsletter. (60 minutes)
a. Although each task is timed, feel free to work at your own pace. Try to remember what you learned
from the previous session o f MS Word using an online help system.
b. You will be com pleting MS Word tasks without the use o f an online help system.
c. Wait to start the first task until a lab consultant directs you to begin.
2. Complete an evaluation o f the online help system (30 minutes)

a. You will have access to one of the online help systems you used previously.
b. Go through the help system and evaluate it according to the given criteria.
c. Identify what was most/least useful to you and what changes you would recommend.
168
A dditional In stru ctio n s
I. You will be directed to insert the time periodically throughout this session
ID# 123
a. Place your cursor next to the corresponding task number in the text D ate: 6/16/1999
box in the upper left comer. S ta rt T ask 1: 13:59:03
End T ask I: 14:04:25
b. Click on Insert | D ate and Time. S ta rt T ask 2: 14:04:45
End T ask 2: 14:08:03
c. Make sure there is no checkm ark in the U pdate Automatically S ta rt T ask 3: 14:08:25
box! End T a sk 3 : 14:13:41
S ta rt T ask 4: 14:13:54
d. Click on the very last time option (for example, 13:00:05) and E nd T ask 4: 14:26:57
click O K . You will end up with something like this box in the S ta rt T ask 5: 14:27:13
upper left com er o f your screen. End T a sk S : 14:35:35
2. There is a Stop bunon in the upper right comer o f your screen. You may click on the title bar and m ove
the Stop button, but do not click on th at button. If for some reason the com puter closes the recording
application, then notify a lab consultant immediately.
3. You will be prompted to save your work as you go.
It is verv important that you include your ID# (e.g. 15) in the filename o f all documents.
S a v e a ll d o c u m e n t s in t h e C : \N e w s le tte r fo ld e r .

b. Click on the broken up-arrow folder ^ 1 until you see the B I 15 (C :) drive.
d. Type "00 O nam e. d o c ” (where 000 IS YOUR ID# and name is the name o f your document)
in the File Nam e field.

4. The lab assistants are here to help you with com puter problems. They will not help you perform the
tasks. Please do not ask them for specific task-related assistance. Just try to remember how to complete
each o f the tasks from what you learned last week.
169
A ctivity Item #1: N ew sletter & M ailing L abels
You have another opportunity to demonstrate the skills you learned by creating a small portion o f a
newsletter and some mailing labels without the use o f an online help system. Work at your own pace to
complete each task. If you do not know how to complete a particular task, think about how you might
complete it and try that. However, don't spend too long trying to figure out how to do any one part o f a
task. If you can't figure out how to do something, then just complete the task as best as you can.
.1 sam ple copy o f the p a rtial newsletter (taken fr o m an actual newsletter) is enclosed in this p a c k e t You
m ay use it as a guide.
Getting Started:
1. Open the fin a l.d o c document (located in the Newsletter folder),

b. C until you see the B 115 (C :) drive.
e. Find and double-click on the fin a L d o c file to open it.
2. Save the document as 000safe.doc (where 000 IS YOUR ID#) in the C:\Newsletter folder.
b. Click on the broken up-arrow folder , ® | until you see the BUS (C:) drive.
d. Type " o O O s a f e . d o c " (where 000 IS YOUR ID#) in the File Name field.
4. Position your icon after “ Date:” and insert the date from In sert | Date and Time.
5. Raise your hand to let the lab consultant know you are ready to start Task 1.
170
Start:
Position your cursor after “Start Task 1" and insert the time from Insert | Date and Time. Use
the very last tim e format (14:30:25).
1. Create a text box spanning from the border o f the date/time text box to the right margin.
2. Type and center the words "Safety News" in the text box.
3. Make the word "Safety" Arial Narrow, 72 pt. font
4. Make the w ord ”News” Arial Black, 72 pt. font, bold, italics
5. Fill the text box with a texture: apply a special, black border to the text box.
6. Apply a tight text box with a 0.1 external margin.
7. Create another text box with the same texture fill and a special border.
8. Type in the “Safety C lubs...” text (Arial, centered, 12 point, bold).
9. Rotate and position the “Safety C lubs...” text box as shown.
10. Apply a tight text wrap with a 0.25 external margin.
11. Create the “#1 in Safety!” AutoShape text box: Arial, 12 point, bold, white.centered.
12. Fill the AutoShape with color or texture; apply a special border.
13. Apply a tight text wrap with a 0.1 external margin.
14. Position the AutoShape text box as shown.
Stop:
Position your cursor after “End Task I” and insert the time from Insert | Date and Time. Use the
* SAVE your d o cu m en t
• Wait for the lab consultant to give you Task 2.
171
Task 2: Add and format the text
Start:
Position your cursor after “S tart T ask 2” and insert the time from Insert | Date and Time. Use
1. Position your cursor a few lines below the "Safety News" banner.
2. Bring in some existing text. (Use the safety.doc file in the Newsletter folder.)
3. Use the first line o f text (" Volunteers to Perform ...") as a one-column heading.
4. Center the heading.
5. Format the remainder o f the text into three columns using the following dimensions:
a. Columns I and 3: 2.5 inches wide with a 0.25-inch space between columns
b. Column 2: 1.5 inches wide with a 0.25-inch space between columns.
6. Make the last line o f text (“Schedule o f Inspections ") and remaining lines into a one-column
heading.
7. Center the "Inspections ” heading.
Stop:
Position your cursor after “End T ask 2” and insert the time from Insert | D ate and Time. Use the
172
Task 3: Add and format a graphic
Start:
Position your cursor after “S ta rt T ask 3” and insert the time from In se rt | D ate an d Time. Use
1. Position your cursor somewhere in the text.
2. Bring in a picture. (Use the safety.bm p file in the Newsletter folder.)
N ote: the computers in this computer lab automatically go to the

“C liparf' folder on the server. Just click on the broken
folder until you see the B115 (C :) drive where you can find the
Newsletter folder.
3. Resize the graphic so that it fits better in the text.
4. Position the graphic in the text so that it overlaps any two columns.
5. Apply a tight text wrap; edit the wrap so that the text wraps in the shape o f the graphic.
Stop:
Position your cursor after "E nd T ask 3” and insert the time from In se rt | D ate and Time. Use the
173
Task 4: Add and format a table
Start:
Position your cursor after "Start Task 4” and insert the time from Insert | Date and Time. Use
1. Create the following table— meet the following criteria:
2 'A point external border with '/: point internal cell borders
'/: point, double-lined internal cell borders under sub-headings
Merged cells
Text direction
Shading: 12.5% for all titles
Text formatting: 10 point, centered and bold as shown
Make rows an even height and columns an even width
Vertically align all text in center o f each cell
Horizontally align the table in the center o f the page
Stop:
Position your cursor after "End Task 4” and insert the time from Insert | Date and Time. Use the
174
Task 5: Create the mailing labels
Start:
Position your cursor after “S ta rt T ask 5” and insert the time from In sert | Date a n d Tim e. Use the
I. Use the Mail Merge Helper to create a list o f addresses for mailing labels in a new main
docum ent.
n o NOT CREATE THE MAILING LABELS OVER YOUR H E A L T H NEWSLETTER.
‘-Your ID#> PersonD Joe Schrnoe

155 Locust Grove 58 Sycamore Lane
Fairway. UT 82121 Leeway. UT 85054
Betty Boo Mike Tucker

100 Brilliant Avenue 4897 Giant Rd.
Airway. UT 84000 Bigwav, UT 89997
'
2. Save the mailing list as "000list4.doc" (where 000 is your ID#).

b. Click on the broken up-arrow folder ^ 1 until you see the B I 15 (C:) drive.
d. Type " 0 0 0 1 i s t 4 .d o c " (where 000 IS YOUR ID#) in the File Name field.
3. Sort the labels in descending order by Postal-code.
4. Merge the labels to a new docum ent: save the mailing labels as "000mail4.doc" (where 000 IS
YOUR ID#) in the C:\Newsletter folder.

b. Click on the broken up-arrow folder d ll| until you see the B l 15 (C:) drive.
d. Type "0 0 O m ai.14. d o c " (where 000 IS YOUR ID#) in the File Name field.

5. Close 000mail4.doc and 000list4.doc (where 000 IS YOUR ID#).
175
6. Save the new main document (Document/V) as Q00main4.doc (where 000 IS YOUR ID#);
close it.
a. In Word, click on File | S ave As ...

b. Click on the broken up-arrow folder until you see the B 1I5 (C :) drive.
d. Type " 0 0 0 m a in 4 .d o c " (where 000 IS YOUR ID#) inthe File Nam e field.
7. Return to the "Safety N ew s" newsletter.
Stop:
* SAVE y o u r d o cu m en t
* Close the "Safety News " newsletter.
*******PLEASE RAISE YOUR HAND TO SUMMON A LAB CONSULTANT *******
176
Appendix H
Online Help Instructions
177
O p e n in g Help
Your lab consultant will open the correct online help system for you. If you accidentally close your
online help system, please raise your hand and ask the lab consultant to reopen it.
Please do not reopen help on your own.
A c c e s s in g Help T o p ic s
This help system is designed to provide you with the greatest amount o f flexibility to get the
information you need in a hurry. Choose from the following methods o f accessing help to complete
any given task.
To u se the Table o f Contents

The T able o f C ontents is represented as a series o f purple "books" which can be "opened" to
display specific "sheets" o f information.
1. Open online help.
Contents I index 1 Find 1

2. Click on the C o n te n ts tab 1
3. Double-click on any book ^ you think contains the needed information. (The book
"opens" ^ to display its "sheets" o f help information.)
4. Double-click on any help sheet to access the corresponding help information.
© Hint: not sure what to do next? Look to the Table o f C ontents for the correct order o f tasks.
178
To use the Find Function

Find searches every word in the help system to locate the key word(s) you enter. It's slower, but
more thorough.
1. Open the online help system.
2. Click on the Find ta b . jJ
Find Setup Wizaid
Find enables;you to watch fortpedfc wordsand

... phiaaash heip topct; tw eed of.searching For
•*" fafiarnwtianbst category.
■ Before you can u«Find.Windowr must fsst aeate a fat
■ (or datiase). which contain* every wardftcrn your help
Beftl.
. To create this fat now dfcfc Next
15inim^databa»iael(Bconwiendedl
^t~-M^piK8»earchcapab®iBt
?C ‘&itoiii»»eatdicapaMMes-1
Hext > ' | Cancel
Figure 1. Find Setup W izard: What you see after clicking on the Find tab
3. Click on Next.
Help Topics. Leominij MS Woid 9 /
Corter**| Index; Fe*k- J,
11upe the;wordf*) youwank to find

|mai merge
2 Select lomematehfag,words to nanow your tearch Options™

1 »3
F n d Sm B af
... merged
.. MERGEF1ELD
fnti Now
■^| Rebuild..
I.] I | , — L i I ■■■ H i ' I . - I I : . ! i U n -n l

To tier lists
m
Tosortfats
Understanding Mai Merge
zl
l4TopcsFctmd‘ Atwords.'BegstAuto:Pause
D-gptag,-.^ c [ Cancel-
Figure 2. Find W izard : W hat you see after clicking on the Next button.
179
4. Type in the word or phrase that describes the task you want to complete.
5. Click on a word from the alphabetized list to narrow your search.
6. Double-click on the topic that most closely matches your task.
N avigating in Help
There are different ways to get around in help to find related topics. You can use any o f the most
frequently used methods:
<< >>
Browse b u tto n s =1 L — take you to the topics before and after the topic that is
currently displayed. Use this method to browse back and forth through the help system
after you access a closely related help topic.
g ack
Back button returns you in succession to previously displayed help topics.
H ypertext jum ps are green with a solid underline. They are typically preceded with a
Related Tasks: heading. Click on hypertext jum ps to go directly to related topics— use
the Back button to return.
Hypertext pop-ups are green with a dotted underline. Click on them to get additional
information.
D isplaying Help
Once your online help system is open, you can move or minimize it to provide more working space
on your screen.
To m ove the online help window

Click and drag the help window by its title bar to a different section o f the screen.
^ U s i n g MS W o i d B E E
To minimize th e online help window

Click on the horizontal button in the upper right com er o f the window.
BEQE
To restore th e online help window

Click on the appropriate help button on the bottom o f your screen.
UsingM I Click on this button for overview information.
F?j H ow to,: ! Click on this button for procedural information.
180
Appendix I
Help Design Evaluation Questionnaire
181
Help Design E valuation Q uestionnaire ID N um ber____________
Category 1: Problem Representation
I. Does the help system support different naming fo r tasks and com mands?
A. Are alternative menu systems available to reflect differing options when seeking help (for example,
alphabetical menus to support searching for a particular command, task-based menus that
correspond to real-world tasks, task-based menus to reflect computing tasks, or expert and novice
organizations)?
1. Yes, alternative representations are available, and the alternative organizations make a lot o f
sense.
2. Yes, alternative representations are available, but one or more o f the alternative organizations
is not very helpful.
3. There is only one menu system, but there are both tasks and commands on it.
4. There is only a command-based menu.
B. Does the help system enable the novice user to use familiar terminology to get to the relevant
topics?
1. An index or keyword search system (for example, a Find function) is available and it includes
an extensive list o f synonyms for both the functions inthe application program and the real-
world tasks comprised by these functions (for example, the "cut" and "paste" commands point
to the task "moving a paragraph").
2. Either the tasks or synonyms (but not both) are part o f a keyword search system or index, o r
both tasks and synonym-based access is available but very limited.
3. There is a glossary that defines terms in the application program in a manner that the novice
user can understand.
4. The help system does not contain a glossary, index, or keyword search system.
Category 2: Selection (Menus!
II. How easy is it to select a topic to see using the menus?
A. How many items are there on the main menu? Consider the main menu to be the first menu that has
a list o f actual help topics that the user may see. not an introductory menu offering navigational or
"branching" choices (for example, the "books” are navigational; the "pages" are main menu items).
1. 15 to 50
2. 5 1 to 70, or 8 to 14
3. More than 70, o r fewer than 8
4. There are no menus, ju st a keyword search system.
B. What is the average number o f items that are on the submenus (including embedded menus, that is,
words in the help text that act as menu choices)?
1. Between 7 and 15.
2. Fewer than 7 or more than 15.
C. Are all the items on menus visible at once?

I. Yes.
182
2. No. In some menus, users must scroll or page to see all menu items - but never for more than
two screens.
3. No. In most menus, users must scroll or page to see all menu items - but never for more than
two screens.
4. No. In most menus there are more than two screens o f menu items.
D. How are higher-level menus and submenus linked?

1. Submenus can be seen without leaving the higher-level menu (for example, through the use o f
pop-up lists, pop-up windows, or split screens).
2. Submenus are included in the information or text on a help topic (for example, through
hypertext or as a list at the end o f the help information).
3. Submenus replace higher-level menus (that is, users must traverse a path o f menus to see help
text).
E. How are the items on a menu organized?

1. Menu items are grouped according to shared relevance (for example, items are used in the
same task or are part o f the same general topic) and there are headings for the groups.
2. Menu items are grouped as in 1, but there are no headings for the groupings.
3. Menu items are presented alphabetically.
4. Menu items have neither an alphabetical or common use organization (the organization
appears to be random).
F. How do you select menu items?

1. By clicking on the item with a mouse or, on systems that do not support a mouse, by typing a
letter or number associated with the item.
2. By moving the cursor to the item with full cursor control (up. down, left, and right).
3. By typing the first letter or letters o f the item.
4. By moving the cursor without full cursor control (for example, using only the Tab key or only
the Up and Down arrow keys) and the items are in more than one column.
Category 3: Format
III. Does the fo rm a l o f the help text facilitate searching fo r and understanding the needed
information?
A. If the help text frequently takes two or three screens per topic, but not more, how do you move
through the text?
1. Page through.
2. Scroll.
B. If the individual help texts tend to be more than three screens in length, as in an online document,
how do you move through the text?
1. Page through, and there's a visible menu available to jum p to particular subtopics.
2. Page through, but there's no menu available to jum p to particular subtopics.
3. Paging is not available; the user must scroll through the text. There is. however, a menu
available to support jum ping to particular subtopics.
4. No paging or menu o f subtopics is available; the user must scroll through the text.
C. How are lists formatted?

1. Usually presented as a list, with bullets, dashes, numbers and so on.
2. Usually presented as a list, but with no bullets, dashes, numbers and so on.
3. Sometimes presented as a list
183
4. Almost never presented as a list; presented as sentences o r phrases in a paragraph.
D. How much o f the screen appears to be given to margins and spacing between chunks o f
information? (Do not include spacing between lines in your estimate.)
1. Looks like at least 50%.
2. Looks like between 30% and 50%.
3. Looks like less than 30%.
E. Are different types o f information (command syntax, descriptions, examples, and so on) clearly
separated by spacing?
1. Frequently.
2. Sometimes.
3. Never.
F. Are headings or some form o f highlighting (such as underlining, boldface, indenting, or extra line
spacing) used to identify the different types of information in help text?
1. Almost always.
2. Frequently.
3. Occasionally.
4. Very seldom.
G. Are basic organization and formatting principles (layout, highlighting) applied consistently across
the different help texts?
1. Yes, the appearance is highly structured and consistent from topic to topic.
2. Yes. formatting principles are applied consistently, but not often (more structuring would be
helpful).
3. Formatting principles are applied, but not consistently.
4. No formatting principles were applied.
H. Is the text easy to read (is the typeface legible)?

1. Yes, very consistently.
2. Most text is easy to read, but some information is presented insmall or difficult-to-read
typefaces, or poor highlighting strategies make it difficult to read some o f the text.
3. Most o f the text is very difficult to read.
I. Are both uppercase and lowercase letters used?

1. Yes.
2. No.
184
Category 4: Content
/I What information is presented in the help text?

A. Is the content task oriented? (Consider a task to be the job or action typical users want to complete
when they consult the help system.)
1. The help focuses on tasks and how to use commands to complete those tasks.
2. The help describes commands (including their syntax, function, and related information such
as bugs, if appropriate) but some attention is given to tasks or examples o f how to apply the
commands.
3. The help only describes commands (including their syntax, function, and related information
such as bugs, if appropriate). It may provide examples o f the command syntax, but it does not
provide example applications o f the command (why or how to use it), or these examples are
trivial.
4. The help only describes the command syntax or function and it does not show the way in
which the syntax would be entered.
B. Is the help interactive?

1. Yes, and it is well done.
2. Yes, but it is not done well or it is done in an inconsistent manner.
3. No.
C. Are there levels o f explanation (for example, quick reference and elaborated, pop-up explanations
or definitions, simplified and technical, and so on)?
1. Yes. and it's very complete and distinguishable.
2. Yes. but it's not very complete or different.
3. No.
For each o f the following questions, enter the appropriate score if the feature is present in the help system.
Score:
• Present as necessary in all help, score 1.
• Present for some help but should be in all, score 1.5.
• Absent but should be present, score 2.
D. Is syntax information provided?
E. Is function information provided?
F. Is a list o f related commands given?
G. Are possible applications suggested?
H. Is a concrete example o f how a command is used presented with enough detail that you could
imitate it?
I. Are bugs, warnings and troubleshooting advice given?
J. Are tutorials available from the help?
185
Category 5: Comprehensibility
V. How clearly is the help text written?\

A. How easy is it to understand (not apply) the help information while browsing through it?
1. Almost always easy to understand.
2. Often very easy to understand.
3. Often very difficult to understand.
4. Almost always difficult to understand.
B. Are sentences overly complicated?

1. Almost always uses simple sentences.
2. Usually uses simple sentences.
3. Usually uses complex sentences.
4. Always uses complex sentences.
C. Are sentences in the passive voice?

1. Alm ost never.
2. Sometimes.
3. Frequently.
4. Alm ost always.
D. Do lists use parallel structure?

1. Almost always.
2. Frequently.
3. Sometimes.
4. Alm ost never.
E. How often does the text contain noun strings?

1. Alm ost never.
2. Sometimes.
3. Frequently.
4. Alm ost always.
F. Does the text refer to users with person pronouns (for example, "you") and use the imperative
("Press the Return key")?
1. Frequently.
2. Sometimes.
3 Infrequently.
G. Is the vocabulary, beyond the use o f command names and task names, unnecessarily difficult or
technical?
1. A lm ost always easy to understand.
2. Usually easy to understand.
3. Usually difficult to understand.
4. Alm ost always difficult to understand.
H. Does the help use functional graphics (functional graphics are used for informational purposes, not
just for motivational or aesthetic reasons)?
1. Yes, graphics are used consistently.
2. Yes, but only occasionally.
3. No.
186
I. Are the functional graphics easy to understand?

1. Very easy to understand.
2. Reasonably easy to understand.
3. Very difficult to understand.
Category 6: Navioation
1 /. How easy is it to navigate the help system?

A. How helpful is the information on how to use the help system?
1. Help on help gives an overview and clear, substantive information on using help, and it is
readily available (that is, there is a prompt for accessing help on help and the prompt is visible
when users enter the help system).
2. Help on help is provided, but does not describe the organization o f the help or give strategies
to use.
3. Help on help is clear and complete, but the user must leave the help system to read it (for
example, it's in a separate file).
4. Help on help is not available within the help system and it does not describe the organization
o f the help or give strategies for use.
5. Help on help is not provided.
B. How helpful is the overview on the application program?

1. Provides a conceptual model to help users think about the program and gives hints on
strategies and potential pitfalls.
2. Provides some useful, detailed information but does not give a general view that helps users
think globally about the program.
3. Provides neither useful, detailed information nor a general view o f the program.
4. Does not provide an overview o f the program.
C. How do you move around in the help system?

1. There is a network o f submenus that leads directly to topics and, from a topic itself, that goes
to related topics or to a menu o f related topics.
2. There is a single menu, but you can go from one topic to another using a keyword or "see
also" listing on the topic itself without returning to the menu.
3. There is a single menu and you must use it to access all help topics.
4. There are multiple menus, but you must follow a fixed path through the menus or help
information (for example, you must backtrack through one or more menus in order to follow a
branch on a different menu).
D. How long does it take to move from one help topic to another (or within the same general topic
area if the help is broken into topic areas)?
1. Less than 2 seconds.
2. Between 2 and 5 seconds.
3. More than 5 seconds.
187
E. How well does the help system support navigation?

Support for navigation includes:
• Navigation commands that are always visible (or the method for accessing the commands is
always visible
• Bookmarking (users can mark places in the help system and return directly to them).
• Easy recovery from "getting lost” (for example, an ever-present option to return to a main
menu).
A map o f the help system showing the user's current location.
• The ability to preview topics before going to them or to see the context surrounding the
discussion o f a topic.
1. The help provides 4 or 5 o f the navigational supports.

2. The help provides 2 or 3 o f the navigational supports.
3. The help provides one o f the navigational supports.
4. The help does not provide any o f the navigational supports.
Category 7: Link to Application
1II. Do the help system and the application program facilitate applying the information to the users’
tasks or problem s?
A. Can you transfer help information to the application program (for example, using cut and paste)?
1. Yes.
2. No.
B. Can you view the relevant parts o f the application program while in the help system? .......
1. Yes, windowing permits viewing both the help and the application program simultaneously.
2. Sometimes (there is an overlapping window with fixedplacement).
3. No, but you can switch between the help and the application screen easily.
4. No, you must leave help to see the application program.
C. Can you work in the application program while help is on the screen?
1. Yes. for all help content.
2. Yes, but only for a small portion o f the help content.
3. No.
D. Does the help use the same functions as the application program where appropriate (for example,
does it use the same movement commands or the same menu system)?
1. Yes, fully o r almost fully mimics the application program.
2. Somewhat, not consistent in all respects.
3. No.
188
Category 8: Usefulness
1. Do you generally use the online help that comes with computer applications? Yes No
Why or why n o t ? ____________________________________________________________________
2. Did you use the online help to learn the Word tasks in this study? Yes No
3. Was the online help system used in this study helpful to you? Yes No
4. What was the most useful feature o f this online help system?
5. What was the least useful feature o f this online help feature?
Thank you for participating in this study.

Your input will have an input on the design and development o f future online help systems.
*********** PLEASE RAISE YOUR HAND TO SUMMON A LAB ASSISTANT
189
Appendix J
Expert Reviewer Biography
190
E x p ert R e v ie w e r fo r C o rrect Im plem entation o f M in im alist P rinciples
Laurie Kantner, U sa b ility Specialist, Inform ation A rchitect, an d P ro je ct M an a g er,
T ec-E d , A nn A rb o r, MI
L au rie K a n tn e r has been a u sability sp ecia list, inform ation a rc h ite c t, an d p ro jec t m a n a g e r
for o v e r 20 y ears, w ith experience based on w ell o v e r 200 projects. L au rie sp ec ialize s in h e lp in g
c lie n ts d e fin e a n d c a rry out usability stu d ies and docu m en tatio n pro jects to a n sw e r c ritical
q u e stio n s a b o u t p ro d u c t o r system ease o f use. L aurie also teaches a o n e-d ay c la ss on m in im a list
d esig n for d o c u m e n ta tio n at the U niversity o f C alifo rn ia, S anta C ru z E x te n sio n . L aurie is o n th e
B oard o f D ire c to rs o f th e U sability P rofessio n als A sso ciatio n (U P A ) and is C o n te n t C h a ir o f th e
U PA 2000 C o n fe re n c e .
191
V ITA
Jean A. Pratt
Department o f Business Information Systems jpratt@ b202.usu.edu
and Education http://www.bus.usu.edu/bise/faculty/jp/
3515 Old Main Hill Work: 435-797-0331
Utah State University Home: 435-752-4595
Louan. UT 84322-3515 Fax: 435-797-2351
EDUCATION
Ph.D. Instructional Technology, em phasis in online help, Utah State University, December 2000
M.S. Instructional Technology, emphasis in Instructional Design and Development, Utah State
University, January 1995
B.S. Organizational Communication, Summa Cum Laude, University o f Idaho. May 1991
EX PERIEN CE
Utah State University: Assistant Professor (2000-current)
Utah State University: Lecturer (1997-2000)
■ INST 581(0): Instructional Design
• BISE 5450/6450: Advanced Web Design
■ BISE 5450/6450: Computerized Business Presentations
■ ENG 5410: Interactive Media: The Development o f Online Help Systems
» BISE 2550: Business Communications
■ BISE 2400: Desktop and W eb Design
■ BISE 1400: Microcomputer Applications in Business
W ashington State University (1991-1992)
• Public Speaking
■ Small Group Communication
■ Team leader for a university-funded educational hypermedia project
ID2 Research Group, Utah State University: Project M anager, Technical Com municator
(1992-1997)
■ Managed the corporate and educational public education projects (over $500,000 in
funding)
■ Wrote a $ 133,888 grant which was awarded to develop an interactive CD-ROM for higher
education
■ Supervised and contributed to the design and development o f instructional products
■ Created context-sensitive help systems for different versions o f five instructional
development software applications
■ Created hard-copy training and reference documentation for three instructional development
software applications
Moore Business Communication Services: Training Developer (Summers of 1992 and 1993)
■ Developed and upgraded the 1099 training program (both the instructor and participant
materials)
■ Developed the Standard Operating Procedures for all facets o f processing work orders
192
Northgate Emporium, Boise, ID: Sales Trainer/Assistant M anager, Department M anager,

Sales (1984— 1989)
■ Developed and conducted sales training for sales personnel
■ Managed two departments (successively) with up to 15 employees per department
GRANT AND CONTRACT AWARDS

Utah Higher Education Technology initiative, 1994: Funded for 5133,888, ID2 Research Group
Society for Technical Communications, 1998: Funded 510,000 for dissertation research
INVITED CONFERENCE PRESENTATIONS

Pratt, J. A. (September 1998). Now Cut That Out! Minimalist Design, Instructional Technology
Institute, Logan, UT.
Pratt, J. A. (April 1998). Designing Minimalist Documentation: Practical Guidelines. Society for
Technical Communication, Salt Lake City, UT
REFEREED CONFERENCE PRESENTATIONS

Pratt. J. A. (April 2000). Analysis o f Instructional Components in an Online Setting, National
Association for Business Teacher Education Research Conference. Anneheim, CA.
LaBonty, D. & Pratt, J. A. (November 1999). Integrating Computer Applications to Communicate
Research Results, Delta Pi Epsilon National Conference, St. Louis, MO.
Pratt. J. A. (May 1997). Where is the Instruction in Online Help Systems'! Society for Technical
Communication, International Conference, Toronto, Canada.
Pratt. J. A. (May 1996). Examining the Three Personalities o f Help Files, Computers & Writing
Conference, Logan, UT.
Pratt. J. A. (February 1996). Creating Help Systems Which Really Are Helpful! Association for
Educational Communications and Technology National Conference
Pratt. J. A. (February 1995). ID Expert fo r Individualized Authoring and Learning, Association for
Pratt, J. A. (May 1994). How to Create Effective, Multimedia using ID Expert. Association for
Educational Communications and Technology Regional Conference, Park City, UT.
Pratt, J. A. (February 1994). Self-Description: Using ID Expert to Teach ID Expert, Association for
Pratt, J. A. (February 1994). The Use o f Instructional Transaction Shells to Develop Instructional
Courseware, Association for the Development o f Computer-Based Instructional Systems
National Conference
Pratt. J. A. (May 1993). The Use o f Instructional Transaction Shells to Develop Educational
Courseware, Association for Educational Communications and Technology Regional
Conference, Park City, UT.
PROFESSIONAL PUBLICATIONS
Pratt. J. A. (1998). Where is the Instruction in Online Help Systems? Technical Communication.
45(1), 33-37.
Merrill, M.D., Drake, L., Lacy, M., & Pratt, J.A. (1996). Reclaiming Instructional Design.
Educational Technology. 36(5), 5-7.
Pratt, J. A. (1994). Instructional Design Theory [Review o f Instructional Design T heory].
ETR&D. 42(41. 101-104.
Pratt, J. A. (1994). AECT chapters: Filling a niche. Tech Trends. 39(1). 61.
193
PROFESSIONAL MEMBERSHIPS
Association for Educational Communications and Technology
Delta Pi Epsilon
National Business Education Association
Society for Technical Communication
Utah Business and Computer Education Association
PROFESSIONAL ORGANIZATION COMMITTEES/POSITIONS

President/President-elect, AECT Utah State University Chapter, 1992 - 1994
AECT National Membership Committee, 1994 - 1996
AECT Leadership Development Committee, 1994 - 1996
HONORS AND AWARDS

Outstanding Graduate Student o f the Year (1995). Department o f Instructional Technology, Utah
State University
AECT Memorial Scholarship (1994)
ECT Leadership Development Grant (1994 )
Phi Kappa Phi Honor Society
Represented Utah State University at the "W omen as Leaders” Conference (1993). Washington,
D.C.
Dean's List, Honor Roll: Utah State University, Washington State University. University o f Idaho,
Boise State University

Out PDF

Hochgeladen von

Dokumentinformationen

Originaltitel

Copyright

Verfügbare Formate

Dieses Dokument teilen

Dokument teilen oder einbetten

Freigabeoptionen

Stufen Sie dieses Dokument als nützlich ein?

Sind diese Inhalte unangemessen?

Copyright:

Verfügbare Formate

Out PDF

Hochgeladen von

Copyright:

Verfügbare Formate

INFORMATION TO U SERS

The quality of th is reproduction is dependent upon th e quality of the

Oversize materials (e.g., maps, drawings, charts) are reproduced by

Photographs included in the original manuscript have been reproduced

Bell & Howell Information and Learning

OF MINIMALIST PRINCIPLES APPLIED

A dissertation submitted in partial fulfillment

Dr. TimotKy A. Slocum r. Noelle E. Cockett

UTAH STATE UNIVERSITY

Bell & Howell Information and Learning Company

Instruction in Microburst: The Study

o f Minimalist Principles Applied

Jean A. Pratt. Doctor o f Philosophy

Utah State University. 2000

Major Professor: Dr. M. David Merrill

Together they were used to test the following hypotheses:

application procedural skills.

direct access to error information.

type o f online help.

approximately the same speed, regardless o f variances in the instruction.

implementations o f minimalist online instruction.

This doctoral dissertation is dedicated first to my Lord and God,

whose mercy and grace endures forever,

and whose unending love supports me every single day!

I also dedicate this work to Brendan. Adelle. and Cassidy Pratt,

but who were ALWAYS more important.

I love you all more than more can ever say.

thank you for your belief in me and support o f me.

explanations, support, guidance, and virtual pats on the back.

AVI Player saved me literally hundreds o f data-coding hours. Thanks, Thom!

ABSTRACT ................................................................................................................................................................ iii

ACKNOW LEDGMENTS ...........................................................................................................................................vi

LIST OF TABLES ..................................................................................................................................................... xi

LIST OF FIGURES ................................................................................................................................................ xiii

II. REVIEW OF THE LITER A TU RE...............................................................................................................4

Focusing the Review of L iteratu re .......................................................................................................... 4

Locating Relevant Research A rtic les.............................................................................................. 4

Trends in Software Documentation ........................................................................................................ 6

Black et al.: Alternative Manuals ................................................................................................11

Minimalism Applied to Online H e lp ..................................................................................................... 18

III. APPLICATION OF THE PRINCIPLES OF MINIMALISM TO ONLINE HELP .......................... 23

Common Characteristics o f AH Online Help S y stem s.......................................................................24

Tasks in Microsoft W o r d ................................................................................................................24

Table o f Contents .............................................................................................................................26

Unique Characteristics o f Each Online Help System ..........................................................................31

Skeletal Online Help System ........................................................................................................ 3 1

IV. STATEMENT OF THE METHODS AND PROCEDURES ............................................................... 43

Letter o f Informed C onsent............................................................................................................. 43

Four Assessment Newsletters .........................................................................................................45

Treatment D esig n ......................................................................................................................................53

Use o f Lab C onsultants............................................................................................................................56

Presession A ctiv ities........................................................................................................................ 57

Procedure for Coding D a ta ..................................................................................................................... 62

Research A ssistan t...........................................................................................................................64

Procedure for Statistical Analyses ........................................................................................................65

Description o f Sample P o p u latio n ........................................................................................................67

Verbal x Online Help x Time ........................................................................................................97

Hypothesis 6 Data Analysis ............................................................................................................... 101

VI SUMMARY. CONCLUSIONS. AND RECOM M ENDATIONS...................................................... 105

Summary o f the Study.......................................................................................................................... 105

Statement o f the Problem ............................................................................................................. 105

Summary o f the Results ...................................................................................................................... 108

Finding 1 ........................................................................................................................................ 108