B5_Team 20: Cognitive Walkthrough and Heuristic

Evaluation

Cognitive Walkthrough Questions

1. The user does not understand that an action is needed to reach the goal,
2. The user doesn’t see the correct control (e.g., button or link),
3. The user does not realize that the correct choice actually is correct (e.g., does not
understand the label) or might choose another alternative instead,
4. System feedback does not make it clear to the user that he/she made the right
choice.

Task 1 - Adding products and comparing them.

Sara wants to buy an automatic car for city drive. She is looking for a hatchback that fits well
in her budget. She wants to search and compare the different models available in an
automatic variant. Sare also wants her friends and family to help her decide.

Step 1a. Search Products. On the main menu, Sara clicks on “Search Products”
Step 1b: From the options, Sara can decide to “Add a product” or “Search for products” or
“Compare Products” or view “Saved products” or “Share for Voting”. Sara clicks on “Add a
product”.
Step 2: Select a way to add a new product. Sara has 3 choices to add a product:
● Search the web or device apps: “Search the web” will open the search section of the app,
which will allow Sara to search and save products (as discussed in step 4). Similarly,
searching through device apps, will allow her to choose an app on her device and search for
products, while a plug-in will allow addition of that product to the saved list.
● Paste a url link: Sara can also paste the url link of the product and the app will sync the
details of that product from the web to the saved list.
● Add a product manually: Sara can manually enter the details of the product and upload an
image.
Sara selects “search web”.

Step 3: Search options of products. The search screen comes up (which can also be reached directly
from the main menu). Sara enters “Automatic” in the search bar, selects “vehicle” as her category and
sets the budget between $9000 to $12000. She selects Amazon and Ebay as the websites and
selects Search.
Step 4: Select products. The search shows the desired results to Sara. Sara reviews and selects
“Save” for the relevant products.

Step 5a: Compare products. Sara selects “Compare” from the main menu, and checks off products 1,
3 and 5 to compare. She selects the “compare” button.
Step 5b: Review comparison. Sara reviews the products basis the list of metrics of comparison. She
can expand and collapse the metrics, or search for a metric in a long list.
Step 6a: View Saved Products. Sara can see the saved products, she chooses to delete product 5.
Step 6b: A warning pops up informing how to view deleted products and that they will be stored for 30
days.
Step 6c: View Deleted Products. Sara clicks on restore button next to Product 5 to restore it back to
the saved list.
Step7: Nominate and share products for voting. Sara selects the “Share for Voting” tab from the main
menu and stars products 1, 3 and 5 to nominate them. She then selects the “update” button to make
them available to the other collaborators to allow them to vote for their favourite product.

Scenario Walkthrough Walkthrough Walkthrough Walkthrough

Steps: Question 1 Question 2 Question 3 Question 4

Scenario It is not understood

Step 1b that ‘Search’ button
[Main is for searching
menu] new products

Scenario Too many options User will not System does not
Step 2 and detail present understand that give feedback when
[Add a on screen to add a web search means the search buttons
product] product search through the are clicked or if url
app, neither how link is working. No
will the system feedback on
search through manual image
device apps upload.

Scenario User won’t

Step 3 understand what to
[Search] ‘search’, does the
user need to enter
the product name
or keywords?

Scenario It is easy to miss On click, the

Step 4 the filter button, no system should let
[Search label present either. the user know that
Results] the product has
 been saved or
added to compare
list.

Scenario User will not

Step 5 understand how to
[Compare expand the metric.
Products] Mention “tap to
expand”

Scenario “Saved Products” User will not see System needs to

Step 6 will not indicate that where the deleted give feedback that
[Saved deleted products products list unless the product has
Products] list is also available the instruction been deleted or
in that section. comes, only on restored.
deleting a product.

Scenario The user will not

Step 7 understand what
[Shared does “shared for
Products] voting” would entail
and who all will be
able to see the
products.

Task 2 - Collaborating with Other members.

2. Adding team members, collaborating, and managing them.
Scenario 1 Inviting new members:
1. In the main menu, click collaboration to enter collaboration section.
2. In collaboration section, click menu icon on the top right.
3. Click the invite icon (first icon).
4. Input contact information in the top fields. Click add icon.
5. Write a personal note to your invitees in the bottom field. Check email or text to select message
delivery method. Click invite to send invitation.

Scenario 2, Managing members:

1. Click the top right menu icon.
2. Click the remove member icon to remove members from your group.
3. Click remove to remove members.
4. Click the settings icon to select member access.
5. Uncheck boxes for information you want to be invisible.

Scenario 3, Collaborating and messaging other teammates:

1. Input messages to your team, upload files and images, share links and videos.
2. View comparison pages. click the like icon to upvote. click the comment button to leave a
comment on a product.
3. click menu in upper right corner. Click the social icon to view your friend's purchase plans.
Scenario Walkthrough Walkthrough Walkthrough Walkthrough
Steps: Question 1 Question 2 Question 3 Question 4

Scenario 1, All actions are not

Step 2 visible.

Scenario 1, Needs feedback

Step 5. that the invitation
has been sent.

Scenario 2 All actions are not

Step 1 visible.

Scenario 2 Unclear this icon is

Step 4 where member’s
settings are.

Scenario 3 Add more detail

Step 2 and feedback for
how that interaction
would look.
Task 3 - Setting a financial goal and creating a savings plan.

John wants to save money for a new house. He wants to track his progress in the app and
he also hope to find some advice.

Step 1. Go to the Finance section. Open the app and click the “Finance” button.
Step 2. Press the “+” button to create new goal.

Step 3. Fill in the goal’s details and press DONE:

Goal name: House

Required amount: $100 000
Category: Property
Deadline: 10/08/2022
Minimal monthly savings: $1500
Send related materials via email: YES
Send progress notifications: YES
Step 4: Click on the created goal and press GET ADVICE and check your email for property
saving/buying related advices:

Step 5: Press “+” to transfer money to your goal’s account. Enter the payee account and the
amount you’d like to send. Press “send”
Step 6: Observe your progress and repeat step 5 until the goal is reached.

Scenario Walkthrough Walkthrough Walkthrough Walkthrough

Steps: Question 1 Question 2 Question 3 Question 4

Scenario The “Finance” label

Step 1 can be confusing.
 Probably it’s too
vague.

Scenario Plus button should

Step 2 have distinctive
color.

Scenario
Step 3

Scenario It’s not clear how Plus button should User should see
Step 4 he will get an have distinctive some sort of
advice. Should be color. feedback, maybe
more clear. an alert that says
“we’ve sent you
some advices via
email”

Scenario
Step 5

Scenario
Step 6

Task 4 - Setting alerts and notifications

Anna would like to set the alerts and announcements for her budget this month. She would
like also like to receive and help and assistance if she has any queries.

Step 1a:​ Search the help button. On the main menu, Anna will click on the ‘Help’ button.
Step 1b:​ From the options listed, Anna can choose from ‘alerts and reminders’, ‘shared
queries’, ‘get advice’ or the ‘trash’ option. Anna clicks on the ‘Alerts and reminders’ button.
Step 2a: ​Anna swipes the buttons to activate the various options. Anna chooses to ‘set reminders’ ,
‘set alerts’ and ‘set notifications’.
Step 2b: ​Anna also decided to set the payment dates in which she chooses to make her monthly
contributions towards her budget plan. Thus, she clicks on ‘set payment dates’ whereby she will select
the date from the calendar that pops up.
Step 2c:​ When the alerts and reminders have been activated, alerts and notifications will pop up
when certain milestones have been reached, etc.
Step 3a: ​Anna selects the ‘shared queries’ from the main menu. Various queries from
different people will appear.
Step 3b:​ Anna can select a query that she wishes to respond to or assist with by click on the
‘+’ icon next to the person’s query.
Step 3c:​ When the ‘+’ icon is clicked, a text block will pop up where Anna can write her
response in. When the response is written, Anna will click ‘send’ and the response will
appear on the person’s query post.

Step 4a: ​Anna selects the ‘get advice’ button from the main menu.
Step 4b:​ She uploads an image of herself by clicking on the upload button (this is optional),
selects a problem category, gives the problem name and writes her description of the
problem. Anna then clicks ‘save’.
Scenario Walkthrough Walkthrough Walkthrough Walkthrough Question
Steps: Question 1 Question 2 Question 3 4

Scenario The user doesn’t

Step 1 (main see the correct
menu) control

Scenario The user does System feedback

Step 2 (set not understand does not make it
alerts and that an action is clear to the user that
reminders)
needed to reach he/she made the
the goal right choice.

Scenario The user doesn’t

Step 3 see the correct
(Shared control - there is
queries)
no button that lets
the person know
to send/share
their response

Scenario The user does not

Step 4 realize that the
(Get advice) correct choice
actually is correct.
Heuristic Evaluation

Combined Report

Screen 1: Main Menu

Sever # Rule Description

ity evalua
tors

2 2 8. Flexibility and User should immediately be taken to a working part of the

efficiency of use app. The main menu need to be simple and easy to use for
first time users

Screen 2: Compare and Add Products Menu

Sever # Rule Description
ity evalua
tors

1 1 7. Recognition rather Users will need to depend on recall to know that "deleted
than recall products" come under "Saved Products"

2 1 8. Flexibility and Needs some acceloration to get to usueful parts, rather than
efficiency of use having suers click through menus.

Screen 3: Add Product

 Sever # Rule Description
ity evalua
tors

3 1 9. Aesthetic and Too much clutter in terms of information regarding adding a

minimalist design product

2 1 10. Help and Not clear how the system will help the user add products
Documentation from device apps or web

3 1 1. Visibility of system No system feedback on link's validity

status

4 2 3. User control and There is no back button

freedom

Screen 4: Add Product screen 2

Screen 5: Search screen 1

Sever # Rule Description

ity evalua
tors

2 1 8. Flexibility and Should be integrated with saved products.

efficiency of use

2 1 2. Match between simplify the design and structure

system and the real
 world

Screen 6: Search screen 2

Sever # Rule Description

ity evalua
tors

3 2 1​. Visibility of system System doesnt give any feedback when the product is
status added to saved list or compare

Screen 7: Compare products screen 1

Sever # Rule Description
ity evalua
tors

2 1 8. Flexibility and Should be integrated with saved products rather than

efficiency of use having its own page.

Screen 8: Compare products screen 2

 Sever # Rule Description
ity evalua
tors

1 1 7. Recognition rather Users will not know that they need to tap to expand the
than recall other comparison metrics

2 1 6. Consistency and Different from other product list interfaces. May be hard for
standards users to learn.

Screen 9: Saved products screen 1

Sever # Rule Description

ity evalua
tors

3 2 3. User control and No button to lead users to the deleted products or back
freedom button to return to saved products.

Screen 10: Saved products screen 2

Screen 11: Deleted Products

Sever # Rule Description

ity evalua
tors

3 3 1. Visibility of System doesnt give any feedback when the product has
system status been restored

Screen 12: Shared Products

 Sever # Rule Description
ity evalua
tors

2 3 8. Flexibility and Seems to be an extra page that isn't necessary.

efficiency of use

Screen 13: Shared products screen 2

Sever # Rule Description
ity evalua
tors

3 3 4. Error prevention Delete button may cause users to delete the product,
instead of removing from nominated list

2 3 1. Visibility of system could be more clear on the different interactions.

status

Screen 15: Collaboration Sub Menu

 Sever # Rule Description
ity evalua
tors

2 3 7. Recognition rather Has to remember what options are in menu when opening.
than recall Also what icons mean.

Screen 16: Invite

Sever # Rule Description
ity evalua
tors

3 3 3. User control and Needs a back button

freedom

Screen 17: Remove

 Sever # Rule Description
ity evalua
tors

1 3 9. Aesthetic and May look cluttered in case a longer list of collaborators

minimalist design

2 3 8. Flexibility and Separate menu for each individual action. Not very efficient.
efficiency of use

Screen 18: Collaboration Settings

Sever # Rule Description
ity evalua
tors

2 3 9. Aesthetic and Possiblity of having too much information on screen in case

minimalist design of another category or more colloborators.

2 3 8. Flexibility and Separate menu for each individual action. Not very efficient.
efficiency of use

Screen 19: Goals

Sever # Rule Description
ity evalua
tors

2 3 10. Help and There is no tutorial and no help button

Documentation

1 3 8. Flexibility and There are no search or filtering options for goals

efficiency of use

3 3 6. Consistency and Different interface from products. Two sections could be

standards more consistent with adding new items.

Screen 20: Add Goal

Sever # Rule Description

ity evalua
tors

1 3 3. User control and It’s not possible to quickly un-add the created goal
freedom

2 3 4. Error prevention No validation and no validation errors

1 3 1. Visibility of system There is no toast or dialog that will notify about successful
status goal creation.

1 3 10. Help and There is no way of knowing the meaning of every field.
Documentation

2 3 6. Consistency and Done button on top. Normally on bottom.

standards

Screen 21: View Goal

 Sever # Rule Description
ity evalua
tors

2 3 7. Recognition rather Missing the “latest transactions” header. It’s not clear what
than recall those rows mean.

2 3 3. User control and No way for user to add a loan amount, how much is due,
freedom how much has been repaid etc.

Screen 22: View Goal + Overflow

 Sever # Rule Description
ity evalua
tors

3 3 7. Recognition rather People would have to remember where the chart option was
than recall located.

Screen 24: Transfer Money

Sever # Rule Description

ity evalua
tors

2 3 8. Flexibility and No accelerators available to add a transaction.

efficiency of use

1 3 1. Visibility of A notification should be made if the transaction is

system status successful (or not) after user pressed “send”

4 3 4. Error prevention System does not warn the user about the transfer of money.

4 3 4. Error prevention Needs more warning in case a transfer would cause

overdrafts or other financial issues.

Screen 25: Friend’s Goals

Sever # Rule Description
ity evalua
tors

2 3 8. Flexibility and User should be able to pin favorite goals to the top (or
efficiency of use mark/find them in some other way)

1 3 8. Flexibility and We don’t have search but we should

efficiency of use

Screen 26: Help Homepage

 Sever # Rule Description
ity evalua
tors

2 3 8. Flexibility and It is not clear to a user that various settings are under help
efficiency of use

Screen 27: Alerts and reminders

Sever # Rule Description

ity evalua
tors

3 3 3. User control and It is not clear that the user can customise the alerts,
freedom reminders and notifications.

2 3 2. Match between It's not clear that alerts and reminders are situated under
system and the real help
world

Screen 28: Alert 1 - missed payment

 Sever # Rule Description
ity evalua
tors

2 3 1. Visibility of system System doesn't give any feedback once noticed that
status payment has been missed

Screen 29: Alert 2 - budget goal reached

Sever # Rule Description

ity evalua
tors

1 3 1. Visibility of System does not give feedback when goal has been
 system status reached and viewer clicks 'view now'

Screen 30: Alert 3 - congratulate friend

Sever # Rule Description

ity evalua
tors

1 3 1. Visibility of It is not clear as to how to congratulate a friend even though

system status user has accepted pop-up. Would this be via email,
Facebook, etc?

Screen 32: Shared queries 1

 Sever # Rule Description
ity evalua
tors

2 3 8. Flexibility and It is not clear to user that advice should be given to others -
efficiency of use the '+' button does not communicate this effectively.

Screen 34: Get Advice

Sever # Rule Description
ity evalua
tors

2 3 8. Flexibility and Could be more integrated, into one page rather than 3
efficiency of use seperate pages.

2 3 8. Flexibility and It's not clear that query has been saved or sent for others to
efficiency of use view.
5 Most Severe Problems And Their Potential Solutions

Screen Severity Rule # Of Problem Potential Solution

Evaluators

24 4 4. Error 3 System does not System should

prevention warn the user double check the
about the transfer payment details with
of money. the user.

24 4 4. Error 3 Needs more Add overdraft

Prevention warning in case a warnings or security
transfer would measures like
cause overdrafts passcode for
or other financial transfers.
issues.

3 4 3. User control 3 There is no back Add back button

and freedom button

9 3 9. User Control 3 No button to lead Add those buttons

and freedom users to the
deleted products
or back button to
return to saved
products.

3 3 1. Visibility of 3 No system Allow system to visit

system status feedback on link's link once the
validity complete link is
entered

Individual Evaluations

Evaluator 1

Screen Rule Severity Description Potential Solution

2 4 1 Users will need to depend Rename button as "Listed

on recall to know that Products" or another suitable title
"deleted products" come etc.
under "Saved Products"

3 9 3 Too much clutter in terms Simplify design

of information regarding
adding a product

3 10 2 Not clear how the system Mention how it will work if the
will help the user add user selects these options
products from device apps
 or web

3 1 4 No system feedback on Allow system to visit link once the

link's validity complete link is entered

6 1 3 System does not give any Alert should pop up to inform

feedback when the product user that the product has been
is added to saved list or saved or added to compare list
compare

8 7 1 Users will not know that Mention "tap to expand" next to

they need to tap to expand the metric.
the other comparison
metrics

11 1 3 System does not give any Alert should pop up to inform

feedback when the product user that the product has been
has been restored restored

13 4 3 Delete button may cause Remove delete button

users to delete the product,
instead of removing from
nominated list

17 9 1 May look cluttered in case Can be simplified

a longer list of collaborators

18 9 2 Possibility of having too Consider making settings full

much information on screen, instead of a panel
screen in case of another
category or more
collaborators.

19 10 2 User will not know how Provide some help or

to go about adding a instructions
goal and payment
options

21 3 2 No way for user to add a Consider giving user the control

loan amount, how much is to add different financial options
due, how much has been
repaid etc.

24 4 4 System does not warn the System should double check the
user about the transfer of payment details with the user.
money.

27 3 3 It is not clear that the user Allow user to customize alert,

can customise the alerts, reminders and notification
reminders and notifications. settings.

Evaluator 2

Screen Rule Severity Description Potential Solution

1 User should immediately be
Default page could be compare
8 2 taken to a working part of the
page
app.

2 Needs some acceleration to

get to useful parts, rather Make default the compare
8 2
than having users click products page.
through menus.

3 3 4 Needs a back button. add a back button to every page

5 Should be integrated with Create a search bar for the save

8 2
saved products. section that allows for search.

7 Should be integrated with

Add a compare checkboxes in
8 2 saved products rather than
saved products page.
having its own page.

8 Different from other product

Organize the UI similarly to Saved
6 2 list interfaces. May be hard
products and other product lists.
for users to learn.

9 No button to lead users to

the deleted products or back
3 3 Add those buttons
button to return to saved
products.

12 Seems to be an extra page Integrate with Saved Products

8 2
that isn't necessary. page.

13 could be more clear on the

1 2 Better UI and labeling.
different interactions.

15 Has to remember what

Could place icons in top menu and
options are in menu when
7 2 label. Add list of active users in
opening. Also what icons
main page.
mean.

16 3 3 Needs a back button Provide a back button

17 Separate menu for each

Integrate all member actions into
8 2 individual action. Not very
one menu.
efficient.

18 Separate menu for each

Integrate all member actions into
8 2 individual action. Not very
one menu.
efficient.

19 Different interface from

products. Two sections could Make all lists use the same
6 3
be more consistent with interface.
adding new items.

20 Done button on top.

6 2 Shift position
Normally on bottom.

22 People would have to Better to integrate it into the

7 3 remember where the chart current interface, rather than
option was located. having a side menu.
24 Needs more warning in case
a transfer would cause Add overdraft warnings or security
overdrafts or other financial measures like passcode for
4 4 issues. transfers.

34 Could be more integrated, into one

8 2 Too segregated. page rather than 3 separate
pages.

Evaluator 3

Screen Rule Severity Description Potential Solution

The main menu need to be
Simplify the headings, add
8 2 simple and easy to use for
short-cuts
1 first time users
Insert a back button for user to go
3 3 There is no back button
3 back
simplify the design and incorporate system that will
2 2
5 structure simplify the process of searching
A pop-up message should appear
1 2 No feedback to let user know that product has
6 been saved
There is no back button for
3 2 Add a back button
9 user to return
It is not clear to a user that A different heading name would be
8 2 various settings are under needed for help such as budget
26 help controls, etc
It's not clear that alerts and
Potentially give a different name
2 2 reminders are situated under
instead of using the word 'help'
27 help
System doesn't give any Feature to take user to show
1 2 feedback once noticed that missed payment and give the
28 payment has been missed option to 'make payment now'
System does not give
System feature that takes user to
feedback when goal has
1 1 view that budget goal has been
been reached and viewer
reached and the different statistics
29 clicks 'view now'
It is not clear as to how to
congratulate a friend even
Give user different methods once
1 1 though user has accepted
'yes' has been selected
pop-up. Would this be via
30 email, Facebook, etc?
It is not clear to user that Have a better way of
advice should be given to communicating that user can give
others - the '+' button does advice with the use of a more
32 8 2 not communicate this striking button that says 'give
 effectively. advice'
It's not clear that query has
been saved or sent for A pop-up should appear to state
34 8 2 others to view. that query has been submitted