CSE 291 - Introduction to

Virtual Environments
Interaction concepts, 3D GUIs

Jürgen P. Schulze

California Institute for Telecommunications and

Information Technology (Calit2)

jschulze@ucsd.edu October 26, 2006

 Overview
 User interface metaphors
 Desktop metaphor

 Ultimate interface

 Key interactions:
 manipulation

 navigation

 Formal user studies

 User Interface Metaphors:
Desktop Metaphor
 Familiar entities:
 files

 folders

 trash can

 Pitfall:
user may feel constrained to activities allowed on
real desktop
 User Interface Metaphors:
Ultimate Interface
 Real-life interactions in virtual world
 Example: reach out, pick up, move
 Pitfall: laws of physical world sometimes don't
apply in virtual world:
 selection/manipulation from distance

 instant movement between places

 Manipulating a Virtual World
 WIMP = windows, icons, menus, pointing devices
 VR evolves from WIMP
 Most manipulations have 2 phases:
 selection

 action

 Manipulation methods:
 Direct user control

 Physical control

 Virtual control

 Agent control
 Direct User Control
 User interacts with objects in VR just as in real
world
 Gesture or gaze make a selection
 Example: user grabs table and moves it with hand
as long as button remains pressed
 Physical Control
 Based on real-world apparatus
 Haptic feedback from pressing buttons, switches,
sliders, dials
 Examples: steering wheel, dashboard, brush
 Virtual Control
 Control manifested entirely in virtual world
 Often computer generated representations of
physical counterparts: button, trackball, steering
wheel
 Reasons for virtual controls:
 reduce number of physical devices

 less expensive

 can show up when needed, hidden when not

 Common places for virtual controls:

 world, hand, front of view, on display, on panel
 Agent Control
 Commands through intermediary – intelligent
agent
 Agent is person or computer-controlled entity
 Communication is voice or gestures
 If agent is person: can simulate voice recognition
system (Wizard of Oz)
 Properties of Manipulation
 Feedback: more important for virtual devices
 Ratcheting: repeat input to create greater effect
 Constraints: limited DOF, positions on grid
 Control visibility: controls show up when needed (button click, wand
orientation)
 Distance: manipulate objects beyond physical reach
 Pointer beam scope: laserbeam, spotlight
 Hysteresis: compensates for shaky hands or unstable tracking
 Control location: see above (world, hand, front of view, on display, on
panel)
 Frame of reference: wrist rotation for dials, wand motion, joystick on
wand
 Bimanual interface: hold menu in one hand, select with the other
 Selection
 Direction selection:
 pointer, gaze, crosshair, torso,

device, coordinate, landmark

 Item selection:
 Contact: avatar makes contact

 Point to object

 3D cursor: point selector

 Aperture: space between fingers

creates aperture
 Menu: list of items

 World in miniature

 Speech recognition
 Selection
 Alphanumeric value selection:
 Physical input: keyboard, tablet/pen

 Virtual controls: virtual keyboard, sliders

 Agent controls: speech recognition

 Manipulation Operations
 Positioning and sizing objects
 Exerting force on virtual object: relocate object
 Modifying object attributes
 Modifying global attributes
 Altering state of virtual controls
 Navigation
 Navigation
 Navigation = wayfinding + travel
 Wayfinding: direction and path

 Travel: actual motion

NPSNET: large scale military training

 Wayfinding
 Aids to improve wayfinding:
 path following

 maps

 landmarks

 memorable placenames

 leaving a trail (bread crumbs)

 compass

 instrument guidance

 exocentric view (eg, bird's eye view)

 coordinate display

 constrained travel
 Travel
 Manipulation method:
 physical controls: used in vehicle simulators

 virtual controls: emulate physical devices

 agent controls: spoken or gesture commands

 Constraints:
 user stays on the ground, follows terrain

 ground stays horizontal

 Movement formula:
 how far to move upon user input?

 multiple of real-world movement, physical

value (meters, etc)

 Travel
 Travel methods:
 physical locomotion

 ride along

 towrope (river)

 fly/walk through

 pilot through: control virtual vehicle

 move the world

 scale the world: scale down, move, scale up

 put me there: go directly to target

 orbital viewing: object swivels about user on

head motion
 Formal User Studies
 Important means to evaluate interaction methods
 Subjects with specified background use VR
application
 Subject performance is recorded
 Heavy use of statistical methods to find
significance in results
 This Week's Class Paper

 Kelso, Arsenault, Satterfield, Kriz: DIVERSE: A

Framework for Building Extensible and
Reconfigurable Device Independent Virtual
Environments, In Proceedings of IEEE Virtual
Reality 2002

 Questions? Comments?
 Research Paper Presentation
 Presenter: Daniel Rohrlick
 Paper: Schmalstieg, Raitmayr: The World as a
User Interface: Augmented Reality for Ubiquitous
Computing, In Proceedings of the Central
European Multimedia and Virtual Reality
Conference (CEMVRC 2005)
 Presentation: max. 20 minutes
 Discussion: max. 10 minutes
 Announcements
 Next week's topics: VR applications, software packages
 Optional read: textbook chapters 7 and 8
 Paper to read and summarize:
Pertaub, Slater, Barker: An Experiment on Fear of Public
Speaking in Virtual Reality, Stud Health Technol Inform,
81:372-8, 2001
 Programming project #2 proposal due today