Trust, Learning, and Usability
Laurian C. Vega, Yeong-Tay Sun, D. Scott McCrickard
Center for Human-Computer Interaction, Virginia Tech
2202 Kraft Drive
Blacksburg, Virginia 24060
{Laurian, YTSun, McCrickS}@vt.edu
STATEMENT OF TOPIC
In this poster presentation we will introduce the findings
from a study that evaluated trust and its effects on usability
and learning within online information repositories.
SUMMARY
Designing software and online tools so that students find
them not only usable, but also credible to learn from is a
growing issue. In this poster presentation we will introduce
the findings from a study that evaluated trust and its effects
on usability and learning within online information
repositories. We investigated what design element was
being trusted by the e-learner and what aspects of usability
(e.g., errors) influenced trust. Using a novel method of
evaluation, we will present preliminary findings that lead to
a model of trust and implications for designing e-Learning
tools.
SIGNIFICANCE AND RELEVANCE
Problem & Motivation
As learning continues to moves from the classroom to the
Internet, concerns over how students use and evaluate
online materials becomes increasingly important [1].
Indeed, the assessment of credible online resources
becomes a priority for evaluating and designing web-
materials.
Prior work has suggested that the usability of a system
affects how people will learn [2]. Also supported is the idea
that students’ trust of online materials will affect their use
[3, 4]. However, what has been minimally researched is the
effect of usability on trust in e-Learning. We question, what
is a user trusting in e-Learning situations? Is the learner
trusting the creator of the information, as suggested by [5]?
Or, is the learner trusting the online materials, as suggested
by [6]? Would the trust in one affect the other? What effect
does usability have when trust is examined under this lens?
Background & Related Work
In a recent search for related work on trust and online
materials, there were over two thousand results from peer-
reviewed publications. While most of these publications
concentrate on the topics of trust and e-Commerce and how
to increase trust in the buyer, the fact that trust has been
researched and evaluated for the past fifty years is apparent.
Therefore, we have limited out view of trust in this research
1
by defining the domain we are examining: formal
instruction.
Education is a largely social endeavor guided by instructors
and tools; lectures are given by individuals, books are
written by individuals. Learning is an in situ activity that is
guided by participation with the group. Therefore, trust is a
social value that must be encapsulated in the design of
educational tools like software. It is for this reason that
Jones et al. [8] have cited trust as a key factor that must be
accounted for in the continuing development of e-Learning.
Approach & Uniqueness
To begin probing these questions we ran an exploratory
study of twenty participants using two different e-Learning
websites: the Usability Case Study Library (UCS), and
Wikipedia. The UCS is an online repository of software
design materials. Wikipedia is an online encyclopedic
repository of user provided content. The participants spent
twenty minutes within each website to complete four tasks.
To measure usability, our usability goals were efficiency,
learnability, utility, and effectiveness. We measured the
number of errors encountered, subjective beliefs, and the
time users spent on each task.
To measure trust, we employed a unique method. Every
time users clicked on the website, and if the click was at
least thirty seconds apart from the previous click, a pop-up
appeared asking how much they trusted the websites as a
whole, the individual webpages, and the information they
were using.
Results & Contributions
These results are still preliminary; however, they are
encouraging. In examining subjective measures of usability
and using a Fit Least Squares Effect Test, there were a
number of findings. The relationship between the UCS
website as a whole and usability show that website
instructions and feedback had a positive effect on trust.
Frustration and the overall feel of the website had a
negative effect on trust for the UCS. Also having a negative
effect on the UCS was the belief that operations completed
on the website related to the task at hand. Focusing on these
aspects of usability can help when designing a trustworthy
e-Learning website.

Figure 1. One participant’s responses to trusting the entire
website, a single webpage, or the information to guide them
across one twenty minute interval.
Wikipedia is statistically trusted more then the UCS (UCS
Trust = 495, Wikipedia Trust = 617, p= 1.1E-35). This
finding holds across each individual question. Next, if we
break this average down into its separate questions, we can
see a trend: users trust the entire site more then they trust an
individual page within that site. Second, users trust that the
page they are on will lead them to their task answer less
then both of the previous questions. (Correlation= .959).
For a software designer, focusing on the interaction of the
whole website can be a better place to start when designing
for trust.
In examining error rates and their effects on trust, using a
Pearson’s ChiSquare test we found that usability errors had
a detrimental effect on all measured aspects of trust
(p<.0001, df=1). After encountering an error a users trust
would decrease.
One interesting interaction effect to be examined is the
hypothesis that when trust levels are on average lower, they
are more likely to be affected by interaction effects like
usability. When trust levels are higher, usability -
specifically bad usability - may not cause such a large
effect.
As illustrated by looking at Figure 1, the data is very rich
and tells an interesting story.
REFERENCES
1. Rogers, D. and K. Swan, Self-regulated learning and
internet searching. Teachers College Record, 2004.
106(9): p. 1804-1824.
2. Rosson, M.B. and J.M. Carroll, Usability Engineering:
Scenario-Based Development of Human-Computer
Interaction. 2002, San Francisco: Morgan Kaufmann
Publishers.
3. Fogg, B.J., et al. What makes web sites credible? A
report on a large quantitative study. in Conference on
Human Factors in Computing Systems CHI 2001
Anyone. Anywhere, Mar 31-Apr 5 2001. 2001.
4. Tschannen-Moran, M. and W.K. Hoy, A
Multidisciplinary Analysis of the Nature, Meaning, and
Measurement of Trust. Review of Educational Research,
2000. 70(4): p. 547-93.
5. Friedman, B., P.H. Kahn, Jr., and D.C. Howe, Trust
Online, in Communications of the ACM. 2000. p. 34-40.
6. Nass, C.I., et al., Computers are Social Actors: A
Review of Current Research, in Human Values and the
Design of Computer Technology, B. Friedman, Editor.
1997, Cambridge University Press: Cambridge. p. 137-
162.
7. Jones, C., L. Dirckinck-Holmfeld, and B. Lindstrom, A
relational, indirect, meso-level approach to CSCL
design in the next decade. International Journal of
Computer-Supported Collaborative Learning, 2006.
1(1): p. 35-56.
8. Rogers, Y., H. Sharp, and J. Preece, Interaction Design.
2007, Chichester, West Sussex: John Wiley & Sons Ltd.