Beruflich Dokumente
Kultur Dokumente
Executive Summary
For our ENGL 4181 Writing User Documents class, we were tasked with analyzing the usability of the WorldCat
Discovery beta interface. Because the library plans to implement WorldCat Discovery in place of WorldCat Local,
these usability tests were developed both in hopes of challenging problems from the old interface and the sight that
they would reveal additional problems with the Discovery interface. We chose 5 users in an attempt to represent a
diverse range of undergraduate students, with users being both male/female, native and international students,
freshman to senior levels of study, and including varying
majors among the mix.
The usability tests were performed using our groups laptops,
a printed task list, and users were observed by a group
member either recording or taking notes. The users had very
few problems performing the tasks required of them on the
interface, and all of the participants finished the tasks in
under 10 minutes. Interestingly enough, since our users
ranged from novice to advanced experience with the previous
interface WorldCat Local, the tasks on the Discovery
interface seemed to be an equal difficulty level for all
participants, and both the problems encountered and comments received were similar. The most notable difficulties
were able to be broken down into two main groups of problems:
1.
2.
Table of Contents
Title Page
Executive Summary
ii
Table of Contents
iii
Introduction
iv
Methodology
iv
Test Results
1. Layout and Design Problems
2. Content, Language, and the User Problems
Supplemental Qualitative Results
Visualizing Quantitative Results
vi
vi
vii
viii
viii
ix
References
xii
List of Figures
Figure 1.1: User Demographics
Figure 1.2: Usability Test Task List
Figure 1.3: User Post-Test Questions
Figure 2.1: WorldCat Local versus WorldCat Discovery
Figure 2.2: Distracting sidebar
Figure 2.3: Example A. and B. Search Functions and Text Size
Figure 2.4: Example C. Refining the Search
Figure 3.1: User Quotes by Task
iv
iv
v
vi
vi
vii
viii
viii
x
x
ix
Methodology
In an attempt to replicate the wide variety of target users the J. Murrey Atkins Library serves, our group recruited
participants from different majors, levels of study, gender, and expertise with the library interface. Our group
recruited participants by word of mouth. Each member of the group asked people from their social circles. This
included friends, roommates, etc.
Before the start of the test, the participants were asked a series of pre-questions to create a demographic of the
library interface users, in Figure 1.1 below:
The group provided participants with a laptop, cleared of history and cookies for each test. One of the five usability
tests were conducted using laptops installed with the OBS Studio software, which captured both audio, and video of
the webcam and screen captured the browser. The other four usability tests were supplemented with a note taker to
record observations on body language, signs of frustration, and where and how the user had problems with the tasks.
Facilitators and notetakers encouraged the users to think out loud through their execution of the tasks, and reminded
users that the test was on the library interface, and not of the user, himself. The group did not provide assistance to
the users when they had questions about the site, and only spoke if a user took an exceptionally long amount of time
or expressed great frustration with a task to move on to the next task, keeping the user test between the average 1015 minutes.
Following the test, the users were asked a series of post-test questions. These questions helped give a better picture
of what the users thought of the interface. The users were also given the opportunity to provide any other additional
feedback the test and questions may not have covered, in Figure 1.3, below:
Our user participants all completed the usability tasks and questions. Our findings of these usability tests were
constructed using charts, tables, screenshots, and other visual aids from the data gathered by note-taking and
reviewing the two user test videos.
Test Results
While four out of five users were able to complete all of the tasks, the users did encounter difficulties as they
attempted to complete the tasks. It was noted that the problems that the users encountered were mostly the same
among all the users. The group was able to divide the main problems into two categories, which further delve into
their respective solutions.
1.) Layout and Design Problems
There could be a potential bias of the usability test results regarding layout and design, considering that because two
out of the five users had never used the library interface, the remaining three had experience with the WorldCat
Local layout, and had preferences for the initial layout. During the post-test questions, when asked what changes
could be made to the interface, one user suggested the the WorldCat Discovery search mimic the old WorldCat Local
layout, in terms of having the tabs present to sort the search. The old and new search engines are below, in Figure
2.1:
It was brought to the groups attention by one user who admitted she never used the library site in apprehension of
having no idea how to use it, that the layout of WorldCat to a novice user may not appear to be user friendly. The
WorldCat Discovery search offers no visible help or tutorial function on either the initial search page or the search
results page.
One complaint our group received from the a user was that after clicking on a search result and directed onto the
next page, that the sidebar with other relevant searches to the left side of the screen was overwhelming to the page
[outlined in red in Figure 2.1 above] , or in the users words, This is so distracting. Is it for what I clicked on, or
what? The user then realized that to find the information required of the tasks, she would use the drop down bars on
the right instead.
The interface lacks the ability to collapse that side search bar. Perhaps, like the right side of the page, where the
information is not revealed without being clicked on, the relevant searches on the left side of the page could be a
drop-down option. Otherwise, the width of the left sidebar with relevant searches should be able to be collapsed or
minimized to a degree which would not distract from the source that the user clicked on.
2.) Content, Language, and the User Problems
When users were asked in Task 1 to add the first search result to their feature My List, we had five out of five
users complain that the My List function was difficult to find due to the size of the text. In turn, users did not
know the function of either of the two buttons in Example B., or that the second button was the Add to My List
function. Most users seemed hesitant to click on the buttons without knowing their function.
The tasks regarding changing the formats of the search, like sorting for an audiobook or having only peer-reviewed
articles come up revealed much about the librarys users and the user-friendliness of the library interface. When
users were asked to sort articles by peer-reviewed results only, one user expressed her frustration with formats
available to her, as, I dont even know what the difference between a peer-reviewed article and normal article is.
With all five users complaining about the size of the text when searching for the function, and two of the five
thinking out loud that they did not know the function of the My List button, it is clear to our group that the lack of
explanation on of the sites features poses a problem for users navigation and understanding of the site and the
features available to them.
Supplemental Qualitative Results:
Our users were encouraged to think out loud while navigating the beta interface to that we could get a better idea of
what they were thinking. Some of our users became distraught, while more of them were unsure if they found the
right feature we were looking for. Here are some of the direct quotes with what the users said and how they went
through some of the tests:
10
11
Because of our own usability tests, which revealed that some users did not understand the language of searching
academic sites, the final quick-fix to the site before its actual release would be to provide definitions of the unclear
terms and functions on the site. This could be accomplished by highlighting the terms, and as users hover their
mouse over the term or function, a short definition or explanation could appear. In our example below, if the user
were to hover over a peer-reviewed article to refine their results, the definition of the term would appear, or if they
did not know what the My-List function was for, the user could hover their mouse of the button and receive a
result without having to click away to another page.
As for long term changes, the group would like to see the design of the interface be adaptable to user preference, so
that if a user finds the left sidebar present in searches distracting, they should be able to collapse or minimize the
width of the column. With the idea of user preference in mind, the user should also be to adjust the size of the text
on screen to suit their individual needs, just as users are able to choose a language in which to navigate the site.
The group would also like to see a tutorial service provided for the users. Both novice and advanced users of the site
may not know or be aware of the functions which make their library search easier, and by providing a tutorial,
perhaps either by video or PDF, the users would at least have the option to receive help navigating and
understanding the site. This function would be particularly useful to first-year students, English Language Learners,
non-traditional students, and international students, all of which may be new to searching on WorldCat and using
academic language to refine their results. Overall, the group believes that with the preferences being adaptable by
the user and providing explanations both throughout the site and as a function throughout it, the idea of user
friendliness towards the interface will become more positive.
12
References
Usability Test Participants:
Greg Wickliff, Rachael Winterling, Questions from ENGL 4181/5181 Students: Responses from Rachael
Winterling, 2/01/2016
13