Forms Talk

Symposium Catalog

•
Calendar of Events
• 01 CFSP Exam Application Deadline
04/15/2010
• 02 Canadian Forms Institute East
04/19/2010 - 04/20/2010
• 03 Symposium San Jose
05/03/2010 - 05/06/2010
• 04 CFSP Exam Period
06/05/2010 - 06/19/2010
• 05 Master Series: Best Practices in Forms Management 6/10/2010 - 7/15/2010
• 06 CFSP Study Group 8/2010
• 07 Forms and Technology Conference
9/13/2010 - 9/15/2010
Categories
• Association News
• Books
• Certification
• Chapter Events
• Expertise
• Industry News
• Marketing Council
• Membership Council
• Message from the President
 • Professional Opportunities
• Program Council
• Spotlight & Awards
• Symposium
• TalkBack
Gold Level Sponsors
• IBM
• Microsoft
Silver Level Sponsors
• Adobe, Inc.
• Exstream Software
• Kaye-Smith
• Snap Forms
Bronze Level Sponsors
• AM+A
• Amgraf
• Cerenade
• DocuStep
• Easel Solutions
• Essociates Group
• FormSoft
• Integriform
• Mi-Co
• Resource International
• Texcel Systems, Inc.
« BFMA Membership and Professional Growth | Main | CANCGSB-6.2-2009 - What It Is and
Why You Want to Know »
09/09/2009
Usability Series: Overview and Exercise
By Sandra Thwaites
Cognitive Systems Engineering
This is the first of three articles on usability. The current article includes a basic overview of usability and
an exercise in usability testing. The author believes that going through this exercise will give readers a
better appreciation for the steps involved in usability and the value that it provides. You can reach Sandra
at thwaites@magma.ca.
WHAT IS USABILITY?
Usability can be defined as the ease with which the intended group of users can interact with or use a
product to achieve something the product is designed to do. More broadly speaking, usability contributes
to the ‘good’ or ‘bad’ user experience of a product.
Usability is most typically done when development follows a user-oriented, iterative design approach. The
product being developed is anything that has a human interaction component. Instances of such products
may include services offered on a website, an insurance company application for approving clients for
insurance, an electronic thermometer for infants, a vote ballot, or take-out cup and lid for hot liquids.
The term “user-oriented” means there is a belief in the value of including the users’ needs in the product
design, along with business needs and technology. The value of usability is easiest to see when there is a
strong correlation between business needs and user needs. For example, the business may need to
increase sales by 10% (e.g., of cell phones) and users may be more willing to purchase that company’s
product if it enables users to ‘easily’ accomplish some core task (e.g., send text messages).
Iterative development essentially follows these steps:
1. design
2. usability test
3. refine design if indicated
4. repeat until the usability goal(s) is met.
Usability goals should be specified for a product as soon as possible and a priori, that is, before any
testing is done and any results are known. Stating usability goals, or destinations, before testing helps to
avoid the tendency for people to argue about whether the results of testing indicate the design is 'good
enough' to release the product to market. By specifying goals a priori, before testing begins, then the
decision to reiterate or evolve product design will be based on fact (testing results) rather than opinion.
Usability goals deal with either performance or subjective opinion. Performance ultimately comes down to
the speed and errors that result in order to accomplish a task. It may be easiest to set a usability goal in
cases where user performance levels with a product will result in life or death for the users or other
individuals. In the case of a radiation machine for example, a usability goal may be that an acceptable or
good enough design will be one that results in zero patient deaths after a user administers radiation with
the machine. (Set Phasers on Stun, by Steven Casey, presents true stories of interface designs with
deadly consequences, including one story about a radiation machine:
http://www.aegeanpublishing.com/phaser1.html)
Most products will not result in life or death but rather will enable users to complete a task ‘quickly’, ‘more
slowly’, ‘with a few errors’ or ‘with many errors’. These high level statements of usability goals need to be
“operationalized” or stated in a way that makes them measurable. Examples of measurable statements
for usability goals include: users will perform all core tasks in 5 minutes or less; users will complete all
core tasks at a level of satisfaction that is 2 points higher than the satisfaction rating for the way they
currently complete the core tasks. Another example is a goal which states that core task x will be
completed in less than 2 minutes while the other core tasks will be completed in less than 5.5 minutes.
Subjective usability goals may be of interest instead of or in addition to performance usability goals.
Subjective goals relate to ratings on say a 5 point scale Subjects give in response to opinion statements.
For example, a subjective usability goal may be that Subjects rate the product 4 or higher on a scale of
perceived ease-of-use (e.g., I find the product easy to use).
WHAT KIND OF PEOPLE ARE USABILITY PROFESSIONALS?
Usability specialists come from a variety of backgrounds but all hold the common desire to contribute to
the development of products that are usable; products that enable users to accomplish a specific task(s).
This author has a background in Cognitive Psychology as well as several years in a doctoral Human
Oriented Technology program within a Psychology Department. Skills the author believes to be invaluable
include an understanding of research methods, statistics, limitations and strengths of the human brain,
and the ability to be an objective observer.
Relevant courses are available at many universities as well as online.
WHEN DOES USABILITY TESTING OCCUR DURING DEVELOPMENT?
Usability testing is ideally done early and often. Testing can begin as soon as one has an idea for a
product. At the idea or conceptual phase, testing would likely take the form of a focus group where
opinions and visions for the product idea are voiced by participants.
Beyond the conceptual phase, a user interface designer will be able to provide some rough sketches for
the proposed product. Early sketches fall into the category of “low fidelity” and can be tested with potential
users to provide invaluable feedback on whether these high level designs make sense to the user. Still
later in the process, designers and/or developers will be able to deliver a partially-functioning or high-
fidelity prototype. Typically, such a prototype will only be functioning for a specific set of core tasks that
the product is to support.
Testing early means that one can essentially get the design ‘good enough’ before any expensive and
costly-to-change programming has begun.
WHO ARE THE TEST SUBJECTS?
Usability testing can be done on 5 to 7 Subjects who are representative of the target user population.
More often than not, Subjects are tested one at a time on a set of tasks that can be completed in
approximately one hour.
It is important that Subjects understand that it is the product that is being tested rather than them.
Perhaps the most difficult part of testing is reminding the Subject to think aloud. The object of the testing
is usually to collect qualitative data. The tester needs to know and note what the Subject is doing or trying
to do and understand their reasons for anticipating any interaction behaviours. Thinking aloud is a foreign
concept to most people and so the tester needs to gently and frequently prod the Subject to continue to
think aloud.
WHAT IS THE TEST PROTOCOL?
The usability specialist needs to devise a protocol which lays out the tasks for the Subject to try and
complete with the prototype materials. These tasks should represent the core tasks that the product is
designed to do.
It is common for a Usability specialist to run one or two ‘pilot’ or practice Subjects to try and gain an
appreciation for where common interface issues might be for Subjects. In this way, the specialist can be
prepared with a series of hints to try and help the Subject complete a task.
Writing everything down in a protocol better ensures that every Subject will be run through the test in the
same way.
ANALYZING TEST RESULTS
For each task, the problems that Subjects encounter during a usability test may be categorized as a
Critical, Major, or Minor problem according to the following definitions:
Critical: User cannot complete the task because it is not possible with the current interface or because
they do not know how.
Major: User experiences great difficulty completing the task. They may commit many errors and require
several hints to be able to complete the task.
Minor: User completes the task but is annoyed along the way with specific elements of the interface as
indicated by comments to that effect or facial expressions of annoyance.
One wants to focus on common problems found during usability testing. That is, problems which are
repeated for almost every Subject. It is necessary to test representative Subjects because it is impossible,
even for a seasoned usability professional, to predict 100% accurately where exactly users will have
problems with an interface. When testing is done by a usability practitioner alone without any Subjects, it
is called a Heuristic Evaluation. While this Heuristic Evaluation is better than doing no Usability testing, it
is generally inferior to the results that will be found through testing a representative sample of Subjects.
From the analysis, the usability practitioner must pinpoint problems with the current interface design and
make recommendations for the way in which that design needs to be changed to better meet the users
mental model or understanding of how the application works.
THE EXERCISE
This exercise is designed to be simplistic and will hopefully provide the reader with an opportunity to
perform a usability test on 5 to 7 people.
The material to be usability tested is presented in Figure 1 which is a first pass sketch of a website for a
company called GroceriesOnline.com. Assume that this is a new yet to be launched website intended for
local users. The owners of the company believe that this design is sufficient to allow their potential
customers to order such things as
1. Bananas
2. Bar-b-q tongs
3. Large watermelon
Your tasks are to:
1. Set one or more usability goals.
2. Develop a test protocol whereby each Subject must tell you how they would order large bananas,
bar-b-que tongs and large watermelon. The test protocol needs to be further specified.
3. Run yourself as a pilot Subject to get a feel for what might happen during actual testing.
4. Run 5 to 7 Subjects through the test.
5. Record & analyze your data.
6. Make at least two key recommendations for creating a better design for the website.

Figure 1: Material for Usability testing (kindly provided by Lise Whitewolf and Hilary Little, User Interface Designers, CBSA,
as part of a lunch-and-learn exercise Summer, 2009)
Usability is an art and a science and, like anything, improves with practice. You are bound to learn
something by going through this exercise and may even find yourself eager to usability test some of your
own designs.
The collective results of many usability studies have resulted in the development of standards of best
practice for design. Nevertheless we must not forget that the best design for today’s technology will be
replaced with a design more suited to tomorrow’s technology. This point is underscored by a recent study
in The BBC Magazine. They marked the 30th anniversary of the Sony Walkman by inviting a 13 year-old
boy to swap his iPod for a Walkman for a week. Some of the boy’s comments included, “It took me three
days to figure out that there was another side to the tape!”
The entire article can be found at http://news.bbc.co.uk/2/hi/uk_news/magazine/8117619.stm
The next article in this series will discuss some of the many issues with the initial GrocerersOnline.com
website design. Additionally, the second article will discuss some publicly available usability work, noting
some key elements to further expand your understanding of usability.
SOME CLASSIC BOOKS & A WEBSITE ON USABILITY
Bias, Randolph G. & Mayhew, Deorah (Eds.), Cost Justifying Usability, Elsevier, 2005.
Krug, Steve & Black, Roger, Don’t Make Me Think: A Common Sense Approach to Web Usability, New
Riders Publishing, 2000.
Nielsen, Jakob, www.UseIt.com
Nielsen, Jakob, www.useit.com/papers/heuristic/heuristic_list.html (10 principles of good interface design)
Nielsen, Jakob, Usability Engineering, Morgan Kaufmann, 2001.
Norman, Donald, The Design of Everyday Things, The MIT Press, 1998.
Rubin, Jeffrey, Handbook of Usability Testing: How to Plan, Design and Conduct Effective Tests, Wiley,
2008.
Posted by BFMA on 09/09/2009 in Expertise | Permalink
Digg This | Save to del.icio.us
TrackBack
TrackBack URL for this entry:
http://www.typepad.com/services/trackback/6a00e54ef14a7988330120a53ee117970b
Listed below are links to weblogs that reference Usability Series: Overview and Exercise:
Comments
Verify your Comment
Previewing your Comment
Posted by: |
This is only a preview. Your comment has not yet been posted.
Top of Form

Post Edit

Bottom of Form
Your comment could not be posted. Error type:
Your comment has been posted. Post another comment
The letters and numbers you entered did not match the image. Please try again.
As a final step before posting your comment, enter the letters and numbers you see in the image
below. This prevents automated programs from posting comments.
Having trouble reading this image? View an alternate.

Top of Form
 Continue

Bottom of Form
Top of Form
6a00e54ef14a79 1273453873-88ff

Post a comment
Comment below or sign in with TypePad Facebook Twitter and more...
You are currently signed in as (nobody). Sign Out

(URLs automatically linked.)

Your Information
(Name is required. Email address will not be displayed with the comment.)
Name

Name is required to post a comment

Email Address

Please enter a valid email address

Web Site URL

Invalid URL
Post Preview

Bottom of Form

BFMA
• www.bfma.org
BFMA International: The Organization for Forms and Business Process Professionals
• Meet your Board Members
• Current Newsletter
• BFMA Newsletter Archive
Enrollment Info
• Become a Member of BFMA
 • Subscribe to the BFMA Newsletter
• Receive Information on Educational Events
• Stay in the Know! Bookmark This Site!
Photo Albums

•
1 Fall Forms Institute 2009

•
2 Symposium 2009

•
3 Canadian Forms Institute West 2009

•
4 Fall 2008 Forms Institute & PDF Conference

•
5 Symposium 2008

•
6 Canadian Forms Institute 2008
 •
7 Fall 2007 Forms Institute

•
8 Fall 2007 Board Meeting

•
9 Symposium 2007
Recent Comments
• Corin Salnave on New CFSP Recognized
• Web developer on Have You Submitted Your Accomplishments for 2008?
• Caroline Jarrett on Usability Pitfalls and a Vision for the Future
• Caroline Jarrett on BFMA Members Respond to the Loss of Rob Barnett
• Jessica Enders on San Jose is Looming Large
• Justiss Boyer on Demonstrating Contribution
• Carol Romero on Do You Know the Way to San Jose?
• Joyce Endres on Long Time Member Says Goodbye
• Carl W. Brannon on Do You Know the Way to San Jose?
• Oliver Wischmeyer on Do You Know the Way to San Jose?
Archives
• April 2010
• March 2010
• February 2010
• January 2010
• December 2009
• November 2009
• October 2009
 • September 2009
• August 2009
• July 2009
More...
Address
• BFMA
3800 Old Cheney Road,
Suite 101-285
Lincoln, Nebraska 68516
Ph: 888-367-3078
Fax: 877-204-5979
E-mail: bfma@bfma.org
www.bfma.org