CIS 385 Contemporary Multimedia Tools

Web Conferencing Evaluation Assignment

THE USABILITY TEST
Spring 2017

Make it or Break it: Conduct a Usability Test

Phase 1. Bring out the Crystal Ball: Make a Prediction (3)
Phase 2. Honey Do: Write a List of Features (6)
Phase 3. Drum Roll Please... Complete the Usability Rubric (12)
Phase 4. Ta-Da! Usability Test Summary (12)
<Briar Bahnmiller>
Instructions

1. Insert your name above and include it in the name of this file. I am not particular about
how you name your file, but I want your name included.
2. Please use Calibri 11-point font.
3. Fill in the blanks where I have indicated. Use complete sentences and paragraphs.
4. I have included examples for each task I ask of you (indicated by italics). Some examples
are of my own making, some are from previous students. I suggest you read them.
5. Blue highlight and enlarged Arial font are used to indicate the areas in which you are to
enter your work.
6. Read each rubric. I am very clear about what it means to do exemplary work. If something
is not clear to you, ask me before I grade your work (not after).
7. Use APA Style for formatting in-text citations and references. Our Moodle course site
contains a chart entitled How to Cite Something Your Found on a Website in APA Style:
What to Do When Information is Missing. Please use it!
8. As you work through each phase of the ADDIE process, think about how you could apply
this to your daily life.

1
Make it or Break it: Conduct a Usability Test
The Usability Test is an extremely important aspect of the mTool Evaluation. It will be completed
in four phases; 1) Predict your results; 2) Identify desired features, write a features list and add
any needed items to the Usability Test (below); 3) Complete the Usability Test and calculate the
score; 4) Write the Summary.

Phase 1. Bring out your Crystal Ball: Make a Prediction

Before you get too far into this process, take a few moments to bring out your crystal ball. Dont
have one? No problem! Just describe what you predict will be the results of this Usability Test
and explain why you predict them. Think this through you will get a Crystal Ball Score later!

Note: Be sure predictions can be tested within the scope of the evaluation.

Heres a non-mTool example: I predict my iPhone will need to update iTunes once in a while
because I have used iTunes on other devices and I know it updates periodically. I also
predict that I will have to buy a high quality case because I know recent iPhone models
have issues with breaking screens.

Example of a Crystal Ball Prediction by a former student:

I predict that there will be some learning required and that the FAQ, if there is one, will be very
limited in information. This is because after looking at the main page and a few other pages
around their website, I was unable to find a FAQ. There may be an FAQ within the tool itself
but it should be easy to find on their website as well.

Navigation will be simplistic. This is because with all of the features this tool provides such as
file storage, time management, notes, project risks, billing, people, links, and milestones
(Problem Solved, 2016), there needs to be a simplistic way to navigate to all of these features
or it will discourage individuals from wanting to use the application.

Updates will be thoroughly tested prior to being released to the public. According to National
Instruments description of software upgrade best practices Create a test that clearly defines
all functionality of the code including every state the code runs. Ensure the application
handles errors as expected. Define a test for any data logging, confirming files are saved
correctly. Determine the necessary execution timing for each state of the code. While
execution timing often affects functionality of certain states, pay close attention to time
sensitive tasks that may complete successfully but outside of the allotted timing. Upgrading
code can affect both memory and processing usage. Benchmark this usage for reference after
the upgrade. Once the tests and benchmarks are clearly defined and documented proceed
with the upgrade. (Software, 2014).

Teamwork will also have integration with other applications as well. Reports will be able to be
exported in formats that are available in the Microsoft Office Suite. With my years of
experience working with project management and tools that involve generating reports, there
was always an option to pull a report at minimum in Excel format.

References
Problem Solved. (n.d.). Retrieved April 28, 2016, from
https://problemsolved1.teamwork.com/projects/232478-test-project/tasks

2
Software Upgrade Best Practices. (2014, May 02). Retrieved April 29, 2016, from
http://www.ni.com/tutorial/52136/en/

3
Write your Crystal Ball Prediction.
Prediction:

Skype for Business will be super easy to navigate through. There are really only 3 options in
the main screen for you and about 4 options when you click on a contact to do. The 3 options
on the main menu are contacts, schedule a meeting, and conversations (instant messages)
between your contacts.

It will be easy to get help with because there is a list of help instruction in the settings button
of the application. If you are having problems with anything else or understanding the Help,
you can call them at any time and they will likely answer. I tried calling them several times
during the day at different times of the day and they answered every time between 7 a.m. and
9 p.m.

I predict that this app will always stay up-to-date with updates, be better for teamwork than
any other app, and that people using the app will figure out how to use it very easily when they
first begin.

References
Bahnmiller, B. (2017, January 25). Briar Bahnmiller's Prediction on Business for Skype. Mayville,
North Dakota, United States.

4
 Your Crystal Ball Prediction will be graded according to these criteria:
Criteria Unacceptable Developing Acceptable Exemplary
0 points 1 points 2 points 3 points
4 or more 4 or more
3 or fewer
predictions were predictions were
predictions were
made made
Crystal Ball No predictions were made
and and
Prediction made or
Some predictions All predictions were
Some predictions
were superficially thoroughly
were not explained.
explained. explained.
Rating &
Explanation
Total /3

5
Phase 2. Honey Do: Write a List of Features
Write a list of features that should be explored during the Usability Test. This list should come
from the Problem Statement and the Needs Assessment.
1. Write a list of features that you will explore as part of the Usability Test.
2. Explain how this list is derived from the Problem Statement and Needs Assessment
sections.
3. Features that are not already listed in the Usability Rubric must be added to the Usability
Test in rows 21 through 25. Add more rows as needed.

Example of a features list from a previous student

1. Is the application available via a distance, or is it only available locally? This is part of the
barriers portion of the analysis. Some developers will need access to currently project
tasks while deployed to different parts of the United States.
2. Does this application provide a means for notes to be added to each task? This is taken
from the Findings and Implications section of the analysis. Due to each developer having a
certain style for resolving a specific problem, notes must be added to define the function
used to resolve the problem.
3. The application must be able to generate a report that includes the time taken to
complete each individual task and billing for each task. This was taken from the Needs
Assessment of the analysis. This is very important as it will help Problem Solved develop
more efficient ways to complete tasks that are taking a large amount of time to complete.

Enter your list of features.

Features:
1. The app can be used anywhere and at any time with internet access. You must have
internet access in order to use this application. The convenience in key in with this
application.
2. This is effective in problem solving because of its convenience and usability. The only
problem that one may have with skype for business is not having internet.
3. The application must allow for people to talk face to face (via video) and exchange data
in the meantime. Its very important that you are able to file share just in case someone
wants an example of something that is orally stated during the meeting.

6
Your Usability Test Features List will be graded according to these criteria:
Criteria Unacceptable Developing Acceptable Exemplary
0 points 1 points 2 points 3 points
Two relevant Three or more
One relevant
List of features were relevant features
feature was added
Features No list of features added to the were added to the
to the Feature List
or Features List Features List
Relevance and
No relevant features and and
and added features
were added to the added features added features
Inclusion were incorporated
list. were incorporated were incorporated
into the Usability
into the Usability into the Usability
Test
Test Test
Rating &
Explanation
Note: If you do not add features to the Usability Test, you will get
0 points.
Most of the items All of the items on
List is not derived
on the list are the list are clearly
List of from the needs
clearly tied to the tied to the needs
Features - No list of features assessment and/or
needs assessment assessment and/or
Alignment the problem
and/or the problem the problem
statement.
statement. statement.
Rating &
Explanation
Total /6

7
Step 3. Drum Roll Please... Complete the Usability Test

Use the table below to rate usability of the mTool product you have chosen. Explain each
of your ratings in detail.
If a criterion listed below is not applicable to the mTool, you may enter N/A as necessary.
Please explain why you think the criterion is not applicable.
Remember to remove anything marked N/A from the score calculations.
Compare the items from your Features List to your Usability Test. Add features to the
Usability Test as needed. Do not delete items from this chart.

Hint: Take screen shots (or record your screen) as you work through the Usability Test. You
will need them later!

Example of a Usability Test from a former student

In this example, I left an error made by this student (#7 highlighted in yellow) to demonstrate a
common error and how to avoid it. Notice in #21-24 (highlighted in blue), the student added the
features discussed above, then rated them.

Poor Medio Excell

Criteria 1 cre ent
2 3
1. Purpose. The purpose of the mTool is clearly stated; users X
understand the purpose of the mTool.
Explain your rating: The application clearly states the purpose and uses of the mTool on the main page
and tour.
2. Features. The mTool has a comprehensive set of features to
X
meet the needs of its users.
Explain your rating: The mTool does have a comprehensive list of features that has met and succeeded
all requirements laid out previously.
3. Ease of use. The mTool is easy to access and self-
X
explanatory to operate.
Explain your rating: The mTool is easy to access but I would not say its easy to operate without
instruction.
4. Installation. The mTool was easy to download and install (if
X
applicable).
Explain your rating: There is a link only available in one place in order to download and install apps that
run the tool if needed. You would have to exit the tool while using it and go to the main page of the
website and navigate to the Projects application page in order to download the tool for which ever
platform your using.
5. Help. Help options are prominently displayed and easy to
X
use and understand.
Explain your rating: The website states what you can do with the tool but does not really explain how to
get places and how to do specific tasks. There was also no FAQ to be found on the products webpage or
in the application itself.
6. Learnability: Basic tasks are easy to accomplish the first
X
time they are attempted.
Explain your rating: The tool requires some learning in order to find and use specific features. I do not
think any task is easily accomplished the first time its attempted.
7. Instructions. Instructions for use are easy to find and
X
clearly stated.
Explain your rating: Student Explanation of Rating: Did not find any instructions for the tool other
than a tour of the product itself. The tour only explained the details of its features. Instructor
Response: They call it Help docs. There is a large set of questions, answers and videos. I did a Google
search for Teamwork Projects tutorials and also found it at the bottom of the Home page under Support
> pick any link > Learn More. http://support.teamwork.com/desk

8
8. Transparent pricing. Prices are clearly displayed. There
are no hidden costs or surprises in the terms (yes, you must X
read them).
Explain your rating: Pricing is very transparent and no credit card is required in order to use the
application for evaluation purposes. The application is also free to use but you are limited to 100mb of
storage and 2 open projects at a time (Problem Solved, 2016).
9. Privacy. IP-based geolocation is not enabled without user's
X
permission. (If you do not know what this means, look it up.)
Explain your rating: IP-based geolocation is not used by this application. However, your IP address is
tracked for usage statistics.

9
10. Privacy. There is a privacy policy. X
Explain your rating: There is a privacy policy that is viewable by clicking the Privacy Policy link located in
the footer of each page.
11. Navigation. Navigation of the mTool is logical and
X
consistent.
Explain your rating: Navigation to specific features of Teamwork is logical and consistent. When you
move to a specific page it highlights the page you are currently viewing so that you know exactly where
you are. When you select a link over relevant information to the link is displayed.
12. Links. Links are descriptive. There are no click here
X
links.
Explain your rating: Links adequately describe where they are taking you.
13. Visuals. Images and videos on the mTool website and the
X
mTool itself are relevant and meaningful.
Explain your rating: There is an abundance of videos available that show the features Teamwork
provides.
14. Layout. Important content is prominently presented. X
Explain your rating: When you login youre presented with an overview of all of your projects.
15. Look and Feel. Styles and colors are complementary and
X
consistent.
Explain your rating: Colors are not overwhelming and are consistent on each page of the tool.
16. Efficiency: Once users have learned the design, tasks are
X
performed quickly.
Explain your rating: With knowledge of the design scheme, some tasks can be performed quicker, but
others require quite a bit of information
17. Updates. Updates, their frequency and method are
X
explained to the user.
Explain your rating: Updates are tested prior to release and they are explained to the user via a blog. The
blog can be accessed from the Teamwork website.
18. Integration. The mTool integrates with other tools and
X
programs as needed.
Explain your rating: Teamwork does provide a means to upload content to cloud services such as Google
Drive, Dropbox, and also has the ability to download files in Excel format.
19. Storage. Storage capacities and limits are clearly
X
explained.
Explain your rating: Storage capacities are explained in the plans available when the user signs up for a
plan.
20. Free account. Limits and privileges of a free account are
X
clearly stated.
Explain your rating: The features of a free account are not clearly stated and it was very difficult to find.
During the trial period users have the SMALL OFFICE plan but once their trial has been completed they
only have 100mb of storage available to them as well as only 2 projects that can be opened at a time.
21. Can the application generate reports? The mTool provides the X
ability to create reports.
Explain your rating: Yes, reports can be generated in the application itself or exported in a PDF or Excel
format.
22. Are you able to add notes to created tasks? Notes can be X
added to tasks upon creation.
Explain your rating: Not only are you able to add notes to a task during its creation you can also go back
and make changes to the notes.
23. Is the application available over a distance? The application X
can be accessed anywhere.
Explain your rating: This application is cloud based so you are able to access information from anywhere
as long as you have an active internet connection.
24.
Explain your rating:
25.
Explain your rating:

10
Totals 5 2 16

11
Usability Scoring: 90% to 100% = Outstanding; 80% to 89% = Exemplary; 70% to 79% =
Acceptable; 0% to 69% = Unacceptable

OVERALL USABILITY SCORE

(5 questions x 1 point) + (2 questions x 2 points) + (16 questions x 3
points) = 57
57 earned points / 69 total possible points = 82.6%
Exemplary

Conduct your Usability Test.

Poor Medio Excell
Criteria 1 cre ent
2 3
1. Purpose. The purpose of the mTool is clearly stated; users X
understand the purpose of the mTool.
Explain your rating: Skype is considered one of the largest video chat applications in the world. The title
of this application even has business in the name. Meaning that it is a video tool made for people who
want to hold business meetings over the internet.
2. Features. The mTool has a comprehensive set of features to
X
meet the needs of its users.
Explain your rating: It is excellent because of its simplicity. You are able to add contacts, view them
through outlook, message people, schedule meetings, and video chat with anyone at any time pending
internet connection.
3. Ease of use. The mTool is easy to access and self-
X
explanatory to operate.
Explain your rating: Refer to purpose and features explanation
4. Installation. The mTool was easy to download and install (if
X
applicable).
Explain your rating: This tool comes with almost all products that come with Microsoft. Skype for business
was already downloaded on my Laptop and is on most peoples laptop that I had the pleasure of asking.
5. Help. Help options are prominently displayed and easy to
X
use and understand.
Explain your rating: Skype has a help function that is rather helpful. But they defer to their call center for
most of the problems.
6. Learnability: Basic tasks are easy to accomplish the first
X
time they are attempted.
Explain your rating:
7. Instructions. Instructions for use are easy to find and
X
clearly stated.
Explain your rating: Skype comes with the Microsoft account that you sign up for when you receive an
Microsoft product. It also automatically links to your email.
8. Transparent pricing. Prices are clearly displayed. There are
no hidden costs or surprises in the terms (yes, you must read X
them).
Explain your rating: Skype for business is free
9. Privacy. IP-based geolocation is not enabled without user's
X
permission. (If you do not know what this means, look it up.)
Explain your rating: Itll ask for your location on setup. It goes by Microsofts standards.
10. Privacy. There is a privacy policy. X
Explain your rating: You can block people, adjust what goes on your profile page, adjust who is your
friends, adjust if you are online or not and much more.
11. Navigation. Navigation of the mTool is logical and X
12
consistent.
Explain your rating: Super simple. There are only 3 buttons. Conversations, meetings, and contacts.
12. Links. Links are descriptive. There are no click here links. X
Explain your rating: There are no click here links
13. Visuals. Images and videos on the mTool website and the
X
mTool itself are relevant and meaningful.
Explain your rating: Plane blue and white coloring. Super simple and not confusing to the user
14. Layout. Important content is prominently presented. X
Explain your rating: Refer to navigation
15. Look and Feel. Styles and colors are complementary and
X
consistent.
Explain your rating: Kind of blah. Just 2 colors but the 2 colors make the app simpler to navigate and use
16. Efficiency: Once users have learned the design, tasks are
X
performed quickly.
Explain your rating: Refer to explanations above on simplicity
17. Updates. Updates, their frequency and method are
X
explained to the user.
Explain your rating: Had trouble finding. But I would fair to guess that Microsoft updates it regularly
18. Integration. The mTool integrates with other tools and
X
programs as needed.
Explain your rating: Helps bring people together and helps business meetings occur with simplicity.
19. Storage. Storage capacities and limits are clearly
X
explained.
Explain your rating: internet storage. The program doesnt take up computer storage
20. Free account. Limits and privileges of a free account are
X
clearly stated.
Explain your rating: It comes with a Microsoft account.
21.
Explain your rating:
22.
Explain your rating:
23.
Explain your rating:
24.
Explain your rating:
25.
Explain your rating:
Totals 1 2 17

Note: To add another row, hover over a beginning or end of a horizontal line and look for the icon.
Click it once a row will be inserted below.

Usability Scoring: 90% to 100% = Outstanding; 80% to 89% = Exemplary; 70% to 79% =
Acceptable; 0% to 69% = Unacceptable

OVERALL USABILITY SCORE

93%

13
14
 Your Usability Test will be graded according to these criteria:
Criteria Unacceptable Developing Acceptable Exemplary
0 points 4 points 5 points 6 points
Usability Usability Test was
84% or fewer
85% to 90% of More than 90% of
Test Usability Test items
not completed Usability Test items Usability Test items
were scored.
or were scored. were scored
or
Items were deleted and and
Overall score was
from the Usability Overall score was Overall score was
NOT calculated
Rubric calculated correctly. calculated correctly.
correctly.
Rating &
Explanation
Some ratings are Some ratings are All ratings are
explained explained explained
Ratings are not or and and
Ratings explained Explanations are Explanations are All explanations are
not related to the related to the related to the
questions asked. questions asked questions asked
Rating &
Explanation
Total /12

15
 Phase 4. Ta-Da! Usability Test Summary
Write a summary of the Usability Test and its results.
Answer these questions in the space provided below
In the finished product, delete the list of bullets and questions.
Use complete sentences and paragraphs.
Include screen shots and their citations as needed to support your comments.
Using APA Style, insert citations in the text and references in the References section.

The results and your Crystal Ball

The Testing Process
1. Were you able to complete the tasks you set
out to accomplish?
2. Did you need help?
3. Were there any stumbling blocks or
problems? If so, describe them.
4. How much time did it take you to conduct the
test and why?
Score
1. Did the results turn out the way you
predicted?
2. What predictions did you get right and
which did you get wrong?
3. Generally speaking, do you consider this
mTool usable? Explain why or why not by
citing specific results of the Usability Test
to support your comments. You may also
use other sources.
4. How does what you learned during the
Usability Test influence your use of this
mTool?

Example of a Usability Test Summary by Previous Student

I was only able to test the functionality of the Teamwork application that was available in the trial
version. I did not have access to export files into Dropbox and Google Drive, so I was unable to
test those features. With the exception of those features, I was able to test the rest of the
application successfully.
I did require assistance from my brother in order to test the ability to assign multiple people to a
single project and have him change the notes within a task. With his help we were able to ensure
that he was able to make changes to the note I originally added under the Test Task.

(Problem Solved, 2016)

The biggest issue I ran into was trying to find some sort of FAQ or tutorial on how to use the
application. There seems to be no information anywhere on the website or in the tool itself to
instruct you on how to use the application. At a minimum it would have been nice if the
application had written instructions and images. This was the biggest issue because it caused
me to have to learn how to use the application myself. It took me two hours to go through the
entire application and test it thoroughly with the assistance of my brother. The testing would
have been easier if we had written or visual instruction available to us on the website. [Note:
Help files do exist but they were not found by this student]
I was correct with my prediction that there would be a learning curve required and that the FAQ
would be nonexistent or very limited. I was however, incorrect in my prediction that the tool
would be easy to navigate. There was a lot of information (links) available on each page that
16
made it very difficult to navigate. If the links could be consolidated it would make it a lot more
simplistic to navigate.

I personally after conducting the usability test do not find Teamwork to be a suitable real world
project management application. There are a lot of links being presented on almost every page
that makes it seem a little over whelming at times.

(Problem Solved, 2016)

What I learned during the testing I conducted for this application is that a tool with these many
features absolutely needs a tutorial that shows how to navigate and utilize all of its features.
Without a FAQ, written, or visual instruction it is very difficult to learn how to operate this kind of
tool. As stated before I would not recommend this tool for real world use, because it would be
difficult to instruct a team without some form of instructions.
[Note: The student would not recommend this mTool due to lack of Help files. Since the Help
files really do exist, if they had been found earlier, the recommendation would likely change. An
article in PC Magazine does recommend this mTool.]

Write the Usability Test Summary.

Usability Test Summary.
This app is perfect as an mTool for communication at any point in time. The key to this mTool is
convenience and ease of use. This product basically comes on any device and you
automatically sign up for it when you sign up for a Microsoft account. The only thing you need
after you begin using it is a somewhat stable internet connection. Besides that, the app is
super convenient. I didnt need any help in figuring out how to use the application because of
how simple the setup of the app is. It was super easy to navigate through the entirety of the
app with very simple instruction. (Bahnmiller, 2017)
The whole test didnt take me long at all to do because Skype was such a good mTool and I
gave it a great score. There wasnt much bad to say about the product. (Bahnmiller, 2017)

References
Bahnmiller, B. (2017, January 25). Briar Bahnmiller's Prediction on Business for Skype. Mayville,
North Dakota, United States.

17
 Your Usability Test Summary will be graded according to these criteria:
Unacceptable Developing Acceptable Exemplary
Criteria
0 points 1 points 2 points 3 points
Summary is stated
Summary is
clearly and
Summary written with described and
described
superficial thought clarified so that
comprehensively,
and preparation. understanding is
delivering all
Or not seriously
relevant
Phase 3: Summary does not impeded by any
No Usability Test information
Usability Test specifically address omissions that may
Summary necessary for a full
Summary Usability. have occurred.
understanding.
or and
and
Statements are not Some statements
All statements are
supported with facts are supported with
supported with
and/or reflections. facts and/or
facts and/or
reflections.
reflections.
Rating &
Explanation
Fully addresses and
None of the Addresses up to 50% Addresses more
develops all aspects
Questions questions listed of the questions than 50% of the
of all questions
are addressed asked. questions asked
asked
Rating &
Explanation
All
Some All
questions/numbers/
questions/numbers/ questions/numbers/
No bullets are deleted.
bullets are deleted. bullets are deleted.
questions/numb and
Formatting and and
ers/ bullets are Complete
Some sentences and Most sentences and
deleted. sentences and
paragraphs are paragraphs are
paragraphs are
complete. complete.
used.
Rating &
Explanation
Some in-text All in-text citations
Some in-text citations
citations exist exist
exist
and and
No in-text and
Style Some in-text All in-text citations
citations. No References section
citations are are supported in
supports the in-text
supported in the the References
citations.
References section. section.
Rating &
Explanation
Total /12

18