Human Computer Interaction

Dimensions of Interface Variability

PART 1

Direct Manipulation &

Virtual Environments

Introduction
Positive feelings associated with good user
interfaces:
Mastery of the interface
Competence in performing tasks
Ease in learning the system originally and in assimilating
advanced features
Confidence in the capacity to retain mastery over time
Enjoyment in using the system
Eagerness to show the system off to novices
Desire to explore more powerful aspects of the system

Examples of
Direct-Manipulation Systems
Command line vs. display editors and word processors
Training times with display editors are much less than line editors
Line editors are generally more flexible and powerful
The advances of WYSIWYG word processors:
Display a full page of text
Display of the document in the form that it will appear when the final
printing is done
Show cursor action
Control cursor motion through physically obvious and intuitively
natural means
Use of labeled icon for actions
Display of the results of an action immediately
Provide rapid response and display
Offer easily reversible actions

Examples of
Direct-Manipulation Systems: WYSIWYG
word processing

Examples of
Direct-Manipulation Systems (cont.)
Technologies that derive from the word processor:
Integration
Desktop publication software
Slide-presentation software
Hypermedia environments
Improved macro facilities
Spell checker and thesaurus
Grammar checkers

Examples of
Direct-Manipulation Systems (cont.)
The VisiCalc spreadsheet and its descendants
VisiCalc users delighted in watching the program propagate
changes across the screen.
In some cases, spatial representations provide a better model
of reality
Successful spatial data-management systems depend on
choosing appropriate:
Icons
Graphical representations
Natural and comprehensible data layouts

Examples of
Direct-Manipulation Systems (cont.):
spreadsheet

Examples of
Direct-Manipulation Systems (cont.)
spatial data management

Examples of
Direct-Manipulation Systems (cont.)
Video games
Nintendo Wii, Sony PlayStation, and Microsoft Xbox
Field of action is visual and compelling
Commands are physical actions whose results are immediately shown on
the screen
No syntax to remember
Most games continuously display a score
Direct manipulation in SimSity
Second Life virtual world
Spore
Myst well received
DOOM and Quake controversial

Examples of
Direct-Manipulation Systems (cont.)
Guitar Hero video game

Examples of
Direct-Manipulation Systems (cont.)
Computer-aided design

Computer-aided design (CAD) use direct manipulation

Manipulate the object of interest
Generate alternatives easily
Explain the impact
Problem solving by analogy to the real-world

Office automation

Xerox Star was a pioneer with sophisticated formatting

Apple Lisa System
Rapid and continuous graphical interaction
Microsoft Windows is a descendant

Continuing Evolution of
Direct-Manipulation Systems
Direct-Manipulation interfaces are being used in a wide range of applications,
e.g. management dashboard for a retail store

Continuing evolution of
Direct-Manipulation Systems (cont.)

Discussion of
Direct Manipulation
Problems with direct manipulation
Spatial or visual representations can be too spread out
High-level flowcharts and database-schema can become
confusing
Designs may force valuable information off of the screen
Users must learn the graphical representations
The visual representation may be misleading
Typing commands with the keyboard may be faster

Principles of
Direct Manipulation
1. Continuous representations of the objects and
actions of interest with meaningful visual
metaphors.
2. Physical actions or presses of labeled buttons,
instead of complex syntax.
3. Rapid, incremental, reversible actions whose effects
on the objects of interest are visible immediately.

Interface-Building Tools
Visual Thinking and Icons

The visual nature of computers can challenge the first

generation of hackers
An icon is an image, picture, or symbol representing a
concept
Icon-specific guidelines

Represent the object or action in a familiar manner

Limit the number of different icons
Make icons stand out from the background
Consider three-dimensional icons
Ensure a selected icon is visible from unselected icons
Design the movement animation
Add detailed information
Explore combinations of icons to create new objects or actions

3D Interfaces
Pure 3D interfaces have strong utility in some contexts, e.g.,
medical, product design. In other situations, more
constrained interaction may actually be preferable to simplify
interactions.

Enhanced interfaces, better than reality, can help reduce

the limitations of the real-world, e.g., providing simultaneous
views.
Avatars in multiplayer 3-D worlds
First person games

3D Interfaces (cont.)

3D Interfaces (cont.)
Features for effective 3D

Use occlusion, shadows, perspective, and other 3D techniques

carefully.
Minimize the number of navigation steps for users to accomplish
their tasks.
Keep text readable.
Avoid unnecessary visual clutter, distraction, contrast shifts, and
reflections.
Simplify user movement.
Prevent errors.
Simplify object movement
Organize groups of items in aligned structures to allow rapid visual
search.
Enable users to construct visual groups to support spatial recall.

3D Interfaces (cont.)
Guidelines for inclusion of enhanced 3D features:
Provide overviews so users can see the big picture
Allow teleoperation
Offer X-ray vision so users can see into or beyond objects.
Provide history keeping
Permit rich user actions on objects
Enable remote collaboration
Give users control over explanatory text and let users
select for details on demand.
Offer tools to select, mark, and measure.

3D Interfaces (cont.)
Guidelines for inclusion of enhanced 3D features
(cont.):
Implement dynamic queries to rapidly filter out unneeded
items.
Support semantic zooming and movement
Enable landmarks to show themselves even at a distance
Allow multiple coordinated views
Develop novel 3D icons to represent concepts that are
more recognizable and memorable.

Teleoperation
Two parents: direct manipulation in personal computers
and process control in complex environments
Physical operation is remote
Complicating factors in the architecture of remote
environments:
Time delays
transmission delays
operation delays
Incomplete feedback
Feedback from multiple sources
Unanticipated interferences

Virtual and
Augmented Reality
Virtual reality breaks the physical limitations of space and
allow users to act as though they were somewhere else
Augmented reality shows the real world with an overlay of
additional overlay
Situational awareness shows information about the real world
that surrounds you by tracking your movements in a computer
model
Augmented reality is an important variant
Enables users to see the real world with an overlay of
additional interaction.

Virtual and
Augmented Reality (cont.)
Successful virtual environments depend on the
smooth integration of:

Visual Display
Head position sensing
Hand-position sensing
Force feedback
Sound input and output
Other sensations
Cooperative and competitive virtual reality

Impact of this
technology in our everyday lives

PART 2

Menu, Form & Dialogue Box

Task-Related
Organization
"The primary goal for menu, form fill-in, and
dialog-box designers is to create a sensible,
comprehensible, memorable, and convenient
organization relevant to the user's task."

Single Menus
Binary Menus
Mnemonic letters
Radio Buttons
Button Choice
3. What is your marital status?
o Single
o Married
o Widowed/divorced/separated

Single Menus (cont.)

Multiple-item Menus
Multiple-selection menus or check boxes

Single Menus (cont.)

Pull-down, pop-up, and toolbar menus
Pull-down menus
Always available to the user by making selections on a
top menu bar
Pull-down menus (cont.)
Key board shortcuts
E.g., Ctrl-C important to support expert user
efficiency
Toolbars, iconic menus, and palletes
Offers actions on a displayed object
Pop-up menus
Appear on a display in response to a check or tap
with a pointing device.

Single Menus (cont.)

Single Menus (cont.)

To see updates from friends, photos and feeds, the Zumobi Ziibii interface
(http://www.zumobi.com) allows users to choose between two styles of presentation.
On the left is a static list of text/image items with a gestural swipe used to control
paging, and on the right is a dynamic scrolling ticker (called River) which
horizontally scrolls titles and images across the screen.

Single Menus (cont.)

Single Menus (cont.)

Menus for long lists
Scrolling menus, combo boxes, and fisheye menus
Scrolling menus display the first portion of the menu
and an additional menu item, typically an arrow that
leads to the next set of items in the menu sequence.
Combo boxes combine a scrolling menu with a textentry filed.
Fisheye menus display all of the menu items on the
screen at once, but show only items near the cursor at
full size.

Single Menus (cont.)

Menus for long lists (cont.)

Sliders and alphasliders

When items consist of ranges or numerical
values, a slider is a natural choice to allow the
selection of a value.
The alphaslider uses multiple levels of
granularity in moving the slider thumb and
therefore can support tens or hundreds of
thousand of items.

Single Menus (cont.)

Menus for long lists (cont.)
Two-dimensional menus
Fast and vast two-dimensional menus give users a
good overview of the choices, reduce the number of
required actions, and allow rapid selection.

Single Menus (cont.)

Single Menus (cont.)

Embedded menus and hotlinks
Embedded menus are an alternative to explicit menus
It is natural to allow users reading about people, events,
and places to retrieve detailed information by selecting
menus in context.

Single Menus (cont.)

Combination of
Multiple Menus
Linear menu sequences and simultaneous menus
Linear
Guide the user through complex decision-making
process.
E.g. cue cards or "Wizards"
Effective for novice users performing simple tasks
Simultaneous
Present multiple active menus at the same time and
allows users to enter choices in any order

Combination of
Multiple Menus (cont.)
Tree-structured menus
Designers can form categories of similar items to create a
tree structure
E.g., fonts, size style, spacing
Fast retrieved if natural and comprehensive
Use terminology from the task domain
Expanding menus maintain the full context of each choice
E.g., Windows Explorer

Combination of
Multiple Menus (cont.)
Menu Maps
Menu maps can help users stay oriented in a large menu
tree
Effective for providing overviews to minimize user
disorientation.
Acyclic and Cyclic Networks
Useful for
social relationships
transportation routing
scientific-journal citations
Can cause confusion and disorientation.

Content Organization
Task-related grouping in tree organization
Create groups of logically similar items
Form groups that cover all possibilities
Make sure that items are nonoverlapping
Use familiar terminology, but ensure that items are distinct
from one another

Content Organization (cont.)

Item Presentation Sequence
The order of items in the menu is important, and should
take natural sequence into account when possible:
Time
Numeric ordering
Physical properties
When cases have no task-related orderings, the designer
must choose from such possibilities as:
Alphabetic sequence of terms
Grouping of related items
Most frequently used items first
Most important items first.

Content Organization (cont.)

Adaptive menus in Microsoft Office.
A font-selection menu lists the
recently used fonts near the top of
the menu (as well as in the full list),
making it easier to quickly select
the popular fonts.

Content Organization (cont.)

Menu layout

Content Organization (cont.)

Menu layout (cont.)
Titles
For single menus, use a simple descriptive title.
For tree-structured menus, use the exact same words in
the higher-level menu items as in the titles for the next
lower-level menu.
E.g. if a menu item is called Business and Financial
Services, the next screen should have that phrase as
its title.

Content Organization (cont.)

Menu layout
Titles (cont.)
Phrasing of menu items
Use familiar and consistent terminology
Ensure that items are distinct from one another
Use consistent and concise phrasing
Bring the keyword to the left

Content Organization (cont.)

Menu layout (cont.)
Graphic layout and design
Constraints
screen width and length
display rate
character set
highlighting techniques

Content Organization (cont.)

Menu layout (cont.)
Establish guidelines for consistency of at least these
menu components:
Titles
Item placement
Instructions
Error messages
Status reports

Content Organization (cont.)

Menu layout (cont.)
Techniques
Indentation
Upper/lower case characters
Symbols such as * or - to create separators or outlines
Position markers
Cascading or walking menus
Magic lens

Fast Movement Through Menus

Keyboard shortcuts
Supports expert use
Can make translation to a foreign language more difficult
Bookmarks in browsers
User configured toolbars

Data Entry with Menus: Form

Fill-in, Dialog Boxes, and Alternatives
Form Fill-in
Appropriate when many fields of data must be entered:
Full complement of information is visible to user.
Display resembles familiar paper forms.
Few instructions are required for many types of entries.
Users must be familiar with:
Keyboards
Use of TAB key or mouse to move the cursor
Error correction methods
Field-label meanings
Permissible field contents
Use of the ENTER and/or RETURN key.

Data Entry with Menus: Form

Fill-in, Dialog Boxes, and Alternatives

Data Entry with Menus: Form

Fill-in, Dialog Boxes, and Alternatives

Data Entry with Menus: Form

Fill-in, Dialog Boxes, and Alternatives

Format-specific field
Coded fields
Telephone numbers
Social-security numbers
Times
Dates
Dollar amounts (or other currency)

Data Entry with Menus: Form

Fill-in, Dialog Boxes, and Alternatives
Dialog Boxes
Combination of menu and form fill-in techniques.
Internal layout guidelines:

Meaningful title, consistent style

Top-left to bottom-right sequencing
Clustering and emphasis
Consistent layouts (margins, grid, white space, lines, boxes)
Consistent terminology, fonts, capitalization, justification
Standard buttons (OK, Cancel)
Error prevention by direct manipulation

Data Entry with Menus: Form

Fill-in, Dialog Boxes, and Alternatives

Dialog Boxes (cont.)

External Relationship
Smooth appearance and disappearance
Distinguishable but small boundary
Size small enough to reduce overlap problems
Display close to appropriate items
No overlap of required items
Easy to make disappear
Clear how to complete/cancel

Data Entry with Menus: Form

Fill-in, Dialog Boxes, and Alternatives

Novel design combining menus and direct manipulation

Pie menus (example here)
Control menus
Marking menus
Flow menus
Toolglass

Audio Menus and

Menus for Small Displays
Menu systems in small displays and situations where hands
and eyes are busy are a challenge.
Audio menus
Verbal prompts and option descriptions
Input is normally verbal or keypad
Not persistent, like a visual display, so memorization is
required.
Request users can avoid listening to options

Audio Menus and

Menus for Small Displays (cont.)
Menu for small displays
E.g., entertainment, communication services
Learnability is a key issue
Hardware buttons
Navigation, select
Expect interactions
Tap interface
GPS and radio frequency identification provides same
automatic input

Audio Menus and

Menus for Small Displays (cont.)

Telephone menus use soft keys to present context-dependent menu items. The
convention used here is to consistently place selections on the left side and back
or exit options on the right side. Hard buttons control the connect and disconnect
functions. Dedicated buttons facilitate scrolling through lists. The current position in
the list is indicated on the right side of the screen.

Audio Menus and

Menus for Small Displays (cont.)

The Zumobi interface (http://www.zumobi.com) on a mobile phone starts with

four tiles using a two-level zoom interaction to see the tile details (left side).
The user can specify which tiles are in their zoomspace. Then, when they
become more familiar with the interface, they can add up to a total of 16 tiles
using a three-level zoom interaction to smoothly go between overview, zone
view, and detail view (right side). The application accommodates thumb use on
touchscreens, numeric key pads for zone-based zooming, 4-way D-Pads, and even
thumb-roller controllers.

PART 3

Command & Natural Languages

Basic Goals
of Language Design

Precision
Compactness
Ease in writing and reading
Speed in learning
Simplicity to reduce errors
Ease of retention over time

Higher-Level
Goals of Language Design
Close correspondence between reality and the
notation
Convenience in carrying out manipulations relevant
to user's tasks
Compatibility with existing notations
Flexibility to accommodate novice and expert users
Expressiveness to encourage creativity
Visual appeal

Functionality to
Support Users Tasks
Users do wide range of work:
text editing
electronic mail
financial management
airline or hotel reservations
inventory
manufacturing process control
gaming

Functionality to
Support Users Tasks (cont.)
Designers should
determine functionality of the system by studying users' task domain
create a list of task actions and objects
abstract this list into a set of interface actions and objects
represent low-level interface syntax
create a table of user communities and tasks, with expected use frequency
determine hierarchy of importance of user communities (i.e. prime users)
evaluate destructive actions (e.g. deleting objects) to ensure reversibility
identify error conditions and prepare error messages
allow shortcuts for expert users, such as macros and customizing system
parameters

Command-Organization
Strategies
A unifying interface concept or metaphor aids
learning
problem solving
retention

Designers often err by choosing a metaphor closer to machine

domain than to the user's task domain.
Simple command set
Each command is chosen to carry out a single task. The number of
commands match the number of tasks.
For small number of tasks, this can produce a system easy to learn and
use.
E.g. the vi editor of Unix.

Command plus
arguments/options

Command plus arguments

Follow each command by one or more arguments that indicate objects to be
manipulated, e.g.
COPY FILEA, FILEB
DELETE FILEA
PRINT FILEA, FILEB, FILEC

Keyword labels for arguments are helpful for some users, e.g. COPY FROM=FILEA
TO=FILEB.

Commands may also have options to indicate special cases, e.g.:

PRINT/3,HQ FILEA
PRINT (3, HQ) FILEA
PRINT FILEA -3, HQ

to produce 3 copies of FILEA on the printer in the headquarters building.

Error rates and the need for extensive training increase with the number of
possible options.

Benefits of Structure
Human learning, problem solving, and memory are greatly facilitated by
meaningful structure.
Beneficial for
task concepts
computer concepts
syntactic details of command languages

Consistent Argument Ordering

Inconsistent order of arguments

Consistent order of arguments

SEARCHfile no, message id

SEARCH
TRIM
message id, segment sizeTRIM
REPLACE
message id, code no
INVERT group size, message id
INVERT

REPLACE

message id, file no

message id, segment size
message id, code no
message id, group size

Hierarchical
command structure
The full set of
commands is
organized into a
tree structure
5x3x4 = 60 tasks
with 5 command
names and 1 rule of
formation

Action

Object

Destination

CREATE

File

File

DISPLAY

Process

Local printer

REMOVE

Directory

Screen

COPY
MOVE

Remote
printer

Symbols vs. Keywords

Command structure affects performance
Symbol Editor

Keyword Editor

FIND:/TOOTH/;-1

BACKWARD TO "TOOTH"

LIST;10

LIST 10 LINES

RS:/KO/,/OK/;*

CHANGE ALL "KO" TO "OK"

Naming & Abbreviations

There is often a lack of consistency or obvious strategy for construction
of command abbreviations.
Specificity Versus Generality
Infrequent, discriminating words

insert

delete

Frequent, discriminating words

add

remove

Infrequent, nondiscriminating words

amble

perceive

Frequent, nondiscriminating words

walk

view

General words (frequent, nondiscriminating)

alter

correct

Nondiscriminating nonwords (nonsense)

GAC

MIK

Discriminating nonwords (icons)

abc-adbc

abc-ab

Six Potential
Abbreviation Strategies
1.
2.

3.
4.
5.
6.

Simple truncation: The first, second, third, etc. letters of each

command.
Vowel drop with simple truncation: Eliminate vowels and use some of
what remains.
First and last letter: Since the first and last letters are highly visible,
use them.
First letter of each word in a phrase: Use with a hierarchical design
plan.
Standard abbreviations from other contexts: Use familiar
abbreviations.
Phonics: Focus attention on the sound.

Guidelines for
using Abbreviations
Ehrenreich and Porcu (1982) offer this set of guidelines:

A simple primary rule should be used to generate abbreviations for most items; a simple
secondary rule should be used for those items where there is a conflict.
Abbreviations generated by the secondary rule should have a marker (for example, an
asterisk) incorporated in them.
The number of words abbreviated by the secondary rule should be kept to a minimum.
Users should be familiar with the rules used to generate abbreviations.
Truncation should be used because it is an easy rule for users to comprehend and
remember. However, when it produces a large number of identical abbreviations for
different words, adjustments must be found.
Fixed-length abbreviations should be used in preference to variable-length ones.
Abbreviations should not be designed to incorporate endings (ING, ED, S).
Unless there is a critical space problem, abbreviations should not be used in messages
generated by the computer and read by the user.

Command-language
Guidelines

Natural Language
in Computing

Natural-language interaction
Natural-language queries and question answering
Text-database searching
Natural-language text generation
Adventure games and instructional systems

Natural Language
in Education

CognitiveTutor traces student progress in mastering skills and

concepts, then assigns individually tuned problems
Communicating with students via Natural Language

PART 4

Interaction Devices

Keyboard Layouts
QWERTY layout
1870 Christopher Latham Sholes
good mechanical design and a clever placement of the letters that slowed
down the users enough that key jamming was infrequent
put frequently used letter pairs far apart, thereby increasing finger travel
distances

Dvorak layout

1920
reduces finger travel distances by at least one order of magnitude
Acceptance has been slow despite the dedicated efforts of some devotees
it takes about 1 week of regular typing to make the switch, but most users
have been unwilling to invest the effort

Keyboard Layouts

Keyboard Layouts (cont.)

ABCDE style
26 letters of the alphabet laid out in alphabetical order
nontypists will find it easier to locate the keys

Additional keyboard issues

IBM PC keyboard was widely criticized because of the
placement of a few keys
backslash key where most typists expect SHIFT key

placement of several special characters near the ENTER key

Number pad layout

wrist and hand placement

Keyboard Layouts (cont.)

Keys

1/2 inch square keys

1/4 inch spacing between keys
slight concave surface
matte finish to reduce glare finger slippage
40- to 125-gram force to activate
3 to 5 millimeters displacement
tactile and audible feedback important
certain keys should be larger (e.g. ENTER, SHIFT, CTRL)
some keys require state indicator, such as lowered position or light
indicator (e.g. CAPS LOCK)
key labels should be large, meaningful, permanent
some "home" keys may have additional features, such as deeper cavity
or small raised dot, to help user locate their fingers properly (caution no standard for this)

Keyboard Layouts (cont.)

Function keys
users must either remember each key's function, identify them from the screen's
display, or use a template over the keys in order to identify them properly
can reduce number of keystrokes and errors
meaning of each key can change with each application
placement on keyboard can affect efficient use
special-purpose displays often embed function keys in monitor bezel
lights next to keys used to indicate availability of the function, or on/off status
typically simply labeled F1, F2, etc, though some may also have meaningful labels, such
as CUT, COPY, etc.
frequent movement between keyboard home position and mouse or function keys can
be disruptive to use
alternative is to use closer keys (e.g. ALT or CTRL) and one letter to indicate special
function

Keyboard Layouts (cont.)

Cursor movement keys

up, down, left, right

some keyboards also provide diagonals
best layout is natural positions
inverted-T positioning allows users to place their middle
three fingers in a way that reduces hand and finger
movement
cross arrangement better for novices than linear or box
typically include typamatic (auto-repeat) feature
important for form-fillin and direct manipulation
other movements may be performed with other keys, such
as TAB, ENTER, HOME, etc.

Keyboard Layouts (cont.)

Keyboard and keypads for small devices
Wireless or foldable keyboards
Virtual keyboards
Cloth keyboards
Soft keys

Pens and touchscreens

Keyboard Layouts (cont.)

The popular RIM Blackberry (http://www.blackberry.com) shown here on the left

demonstrated that many people could use a reduced-size keyboard on a regular basis;
users typically type with one finger or with both thumbs. The Nokia device in the middle
shows that non-English-speaking countries may use different keyboard layouts (here, a
French AZERTY keyboard). On the right, a larger keyboard uses the longer dimension of
the device and can be slid back into the device when not needed (http://www.nokia.com).

Keyboard Layouts (cont.)

Dasher predicts probable characters and words as users make their selections
in a continuous two-dimensional stream of choices

Other text entry methods

The virtual keyboard of

the Apple iPhone gains
precision by allowing finger
repositioning and then
activates on lift-off

Other text
entry methods (cont.)

Another method is to handwrite on a touch sensitive

surface, typically with a stylus using Graffiti on the Palm devices

Pointing Devices
Pointing devices are applicable in six types of interaction tasks:

1. Select:

2. Position:

user rapidly performs a series of position and orient operations.

may be realized as a curving line in a drawing program, the instructions for a cloth cutting machine, or the route on
a map.

5. Quantify:

user chooses a direction in a two-, three-, or higher-dimensional space.

direction may simply rotate a symbol on the screen, indicate a direction of motion for a space ship, or control the
operation of a robot arm.

4. Path:

user chooses a point in a one-, two-, three-, or higher-dimensional space

used to create a drawing, to place a new window, or to drag a block of text in a figure.

3. Orient:

user chooses from a set of items.

used for traditional menu selection, identification of a file in a directory, or marking of a part in an automobile
design.

user specifies a numeric value.

usually a one-dimensional selection of integer or real values to set parameters, such as the page number in a
document, the velocity of a ship, or the amplitude of a sound.

6. Text:

user enters, moves, and edits text in a two-dimensional space. The

pointing device indicates the location of an insertion, deletion, or change.
more elaborate tasks, such as centering; margin setting; font sizes; highlighting, such as boldface or underscore;
and page layout.

Pointing Devices

Direct-Control
Pointing Devices
Lightpen
enabled users to point to a spot on a screen and to
perform a select, position, or other task
it allows direct control by pointing to a spot on the
display
incorporates a button for the user to press when the
cursor is resting on the desired spot on the screen
lightpen has three disadvantages: users' hands
obscured part of the screen, users had to remove their
hands from the keyboard, and users had to pick up the
lightpen

Direct-Control
Pointing Devices (cont.)
Touchscreen
allows direct control touches on the screen using a finger
early designs were rightly criticized for causing fatigue, handobscuring-the-screen, hand-off-keyboard, imprecise pointing, and the
eventual smudging of the display
lift-off strategy enables users to point at a single pixel
the users touch the surface
then see a cursor that they can drag around on the display
when the users are satisfied with the position, they lift their fingers off
the display to activate
can produce varied displays to suit the task
are fabricated integrally with display surfaces

Direct-control
pointing devices (cont.)
Tablet PCs and Mobile Devices:
Natural to point on the LCD surface
Stylus
Keep context in view
Pick up & put down stylus
Gestures and handwriting recognition

Indirect
Pointing Devices

mouse

the hand rests in a comfortable position, buttons

on the mouse are easily pressed, even long
motions can be rapid, and positioning can be
precise

trackball

usually implemented as a rotating ball 1 to 6

inches in diameter that moves a cursor

joystick

are appealing for tracking purposes

graphics tablet

a touch-sensitive surface separate from the

screen

touchpad

built-in near the keyboard offers the convenience

and precision of a touchscreen while keeping the
user's hand off the display surface

Comparison of
Pointing Devices
Human-factors variables

speed of motion for short and long distances

accuracy of positioning
error rates
learning time
user satisfaction

Other variables

cost
durability
space requirements
weight
left- versus right-hand use
likelihood to cause repetitive-strain injury
compatibility with other systems

Comparison of
Pointing Devices (cont.)
Some results
direct pointing devices faster, but less accurate
graphics tablets are appealing when user can remain with device for long periods
without switching to keyboard
mouse is faster than isometric joystick
for tasks that mix typing and pointing, cursor keys a faster and are preferred by users to
a mouse
muscular strain is low for cursor keys

Fitts' Law

Index of difficulty = log2 (2D / W)

Time to point = C1 + C2 (index of difficulty)
C1 and C2 and constants that depend on the device
Index of difficulty is log2 (2*8/1) = log2(16) = 4 bits
A three-component equation was thus more suited for the high-precision pointing task:
Time for precision pointing = C1 + C2 (index of difficulty) + C3 log2 (C4 / W)

Novel Devices
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.

Foot controls
Eye-tracking
Multiple-degrees-of-freedom devices
DataGlove
Haptic feedback
Bimanual input
Ubiquitous computing and tangible user interfaces
Handheld devices
Smart pens
Table top touch screens
Game controllers

Novel Devices (cont.)

Speech and
Auditory Interfaces

Speech recognition still does not match the

fantasy of science fiction:
demands of user's working memory
background noise problematic
variations in user speech performance impacts
effectiveness
most useful in specific applications, such as to
benefit handicapped users

Speech and
Auditory Interfaces (cont.)

Speech and
Auditory Interfaces (cont.)

Discrete word recognition

recognize individual words spoken by a specific person; can work with 90- to 98percent reliability for 20 to 200 word vocabularies
Speaker-dependent training, in which the user repeats the full vocabulary once or
twice
Speaker-independent systems are beginning to be reliable enough for certain
commercial applications
been successful in enabling bedridden, paralyzed, or otherwise disabled people
also useful in applications with at least one of the following conditions:

voice-controlled editor versus keyboard editor

speaker's hands are occupied

mobility is required
speaker's eyes are occupied
harsh or cramped conditions preclude use of keyboard

lower task-completion rate

lower error rate

use can disrupt problem solving

Speech and
Auditory Interfaces (cont.)

Continuous-speech recognition

Not generally available:

Speech store and forward

Voice mail users can

difficulty in recognizing boundaries between spoken words

normal speech patterns blur boundaries
many potentially useful applications if perfected

receive messages
replay messages
reply to caller
forward messages to other users, delete messages
archive messages

Systems are low cost and reliable.

Speech and
Auditory Interfaces (cont.)

Voice information systems

Stored speech commonly used to provide information about tourist

sites, government services, after-hours messages for organizations
Low cost
Voice prompts
Deep and complex menus frustrating
Slow pace of voice output, ephemeral nature of speech, scanning
and searching problems
Voice mail
Handheld voice recorders
Audio books
Instructional systems

Speech and
Auditory Interfaces (cont.)

Speech generation

Michaelis and Wiggins (1982) suggest that speech generation is

"frequently preferable" under these circumstances:

The message is simple.

The message is short.
The message will not be referred to later.
The message deals with events in time.
The message requires an immediate response.
The visual channels of communication are overloaded.
The environment is too brightly lit, too poorly lit, subject to severe
vibration, or otherwise unsuitable for transmission of visual
information.
The user must be free to move around.
The user is subjected to high G forces or anoxia

Speech and
Auditory Interfaces (cont.)

Audio tones, audiolization, and music

Sound feedback can be important:

to confirm actions
offer warning
for visually-impaired users
music used to provide mood context, e.g. in games
can provide unique opportunities for user, e.g. with
simulating various musical instruments

Displays Small and Large

The display has become the primary source of

feedback to the user from the computer

The display has many important features, including:

Physical dimensions (usually the diagonal dimension and

depth)
Resolution (the number of pixels available)
Number of available colors, color correctness
Luminance, contrast, and glare
Power consumption
Refresh rates (sufficient to allow animation and video)
Cost
Reliability

Displays Small and Large (cont.)

Usage characteristics distinguish displays:
Portability
Privacy
Saliency
Ubiquity
Simultaneity

Display technology

Monochrome displays

are adequate, and are attractive because of their lower cost

RGB shadow-mask displays

small dots of red, green, and blue phosphors packed closely

Raster-scan cathode-ray tube (CRT)

electron beam sweeping out lines of dots to form letters

refresh rates 30 to 70 per second

Liquid-crystal displays (LCDs)

voltage changes influence the polarization of tiny capsules of liquid crystals

flicker-free
size of the capsules limits the resolution

Plasma panel

rows of horizontal wires are slightly separated from vertical wires by small glass-enclosed
capsules of neon-based gases

Light-emitting diodes (LEDs)

certain diodes emit light when a voltage is applied

arrays of these small diodes can be assembled to display characters

Display technology (cont.)

Electronic ink
Paper like resolution
Tiny capsules with negatively and positively
charged particles

Braille displays
Pins provide output for the blind

Displays Large and Small (cont.)

Large displays
Informational wall
displays
Interactive wall
displays
Multiple desktop
displays

Displays Large and Small (cont.)

Heads-up and helmet mounted displays

A heads-up display can, for instance, project
information on a partially silvered widescreen of
an airplane or car
A helmet/head mounted display (HMD) moves
the image with the user
3D images

Mobile Device Displays

Currently mobile devices used for

brief tasks, except for game playing
Optimize for repetitive tasks
Custom designs to take advantage
of every pixel
DataLens allows compact overviews
Web browsing difficult
Okay for linear reading, but making
comparisons can be difficult

Animation, image, and video

Accelerated graphics hardware

More information shared and downloaded on the
web
Scanning of images and OCR
Digital video
CD-ROMs and DVDs
Compression and decompression through MPEG
Computer-based video conferencing

Reference
Shneiderman, B. and
Plaisant, C. (2004).
Designing the User
Interface. 5th Edition,
Pearson.