Research Issues in Ubiquitous Computing

Syed Imran Jami PhD Student FAST-NU

Outline
What is Ubiquitous Computing Scenarios Computer Science Issues in Ubiquitous Computing Labs working on UbiComp Web Sites / Conferences / Magazines and Journals Some good books

What is Ubiquitous Computing

Vision
Also known as Pervasive Computing Some Visions are
Computing Everywhere for Everyone Embed Computing devices in the environment Keep the computers in the background presence

Definitions
It is the trend towards increasingly ubiquitous, connected computing devices in the environment.
a trend being brought about by a convergence of advanced electronic technologies and the Internet.
Wireless enabled

Ubiquitous computing devices are not personal computers, but very tiny - even invisible devices.
Can be either mobile or embedded in almost any type of object imaginable. This may include cars, tools, appliances, clothing and various consumer goods - all communicating through increasingly interconnected networks.

According to Dan Russell, director of the User Sciences and Experience Group at IBM's Almaden Research Center,
By 2010 computing will have become so naturalized within the environment that people will not even realize that they are using computers In future smart devices all around us will maintain current information about their locations, the contexts in which they are being used, and relevant data about the users

Definition: by Mark Weiser [4]

Mark Weiser is the founder of this idea. According to him: The goal of Ubiquitous computing is to enhance computer use by making many computers available throughout the physical environment, but making them effectively invisible to the user. Ubiquitous computing envisions a world of fully connected devices, with cheap wireless networks everywhere. You need not carry anything with you, since information will be accessible everywhere. Ubiquitous computing envisions computation primarily in the background where it may not even be noticed. Ubiquitous computer gives the feeling as though you did it yourself.

Ubiquitous Systems require Embedded processors In everyday objects Small cheap and light weight Wireless Communications Sensors

Characteristics
Ubiquitous Systems
Can remember pertinent events
they have a memory

Show context-sensitive behavior

they may have sensors location / situation awareness

Are responsive
communicate with their environment networked with other smart objects

Main Theme
With Ubiquity of Computers
Information processing moves to the background
human centered: concentrate on the task, not the tool the notion computer as a tool does no longer hold

New picture of computing as an invisible, ubiquitous background assistance

specialized, invisible computers will become an integral part of the natural human environment

Computing without Computers

Some reasons for this revolution

As per Moores law
Processing speed and storage capacity double every 18 months
cheaper, smaller, faster

This leads to the fact that most important technology parameters double every 1 3 years:
computation cycles memory, magnetic disks bandwidth

Some reasons for this revolution

New devices
Smart papers E Ink Live boards

Philips prototype for smart paper and E-Ink

Some reasons for this revolution

Progress in Communication Technologies Fiber optics: from Gbit/s to Tbit/s Powerline technique coffee maker automatically connected to the Internet Wireless mobile phone: GSM, UMTS wireless LAN (> 10 Mbit/s) Body area networks A set of mobile and compact units (on body) enable transfer of parameters between the bodies. The data flow passes a chain of BAN modules from each sensor to a main body station, which consolidates the data streams of all sensor modules attached. Transmits the data to a home base station, from where they can be forwarded via telephone line or internet. Wide application in remote health care

Some reasons for this revolution

Better Sensors
Very small cameras and microphones
pattern recognition, assisted by heuristics
user is in a meeting

speaker recognition, speech controlled devices

Fingerprint sensor on mobile objects Many other types of sensors (e.g. location) Autonomous perception of the users environment
establishing contextual relations
recognition of objects

Scenarios

WatchPad1.5 by IBM

Use as a Universal Remote Controller

WatchPad 1.5
Use as a Personal Identificator
Automated check-in at hotel and air counter Cashless payment at restaurant and station Medical history and prescription retrieval Specify your preference

Use as a Display for Location Based Services

Provide personalized advertisement and offering information Timetable and flight schedule at station and airport Navigate you at stations

Radio Sensors
Applications
Mobile devices Wireless light switch Fire detectors Temperature surveillance Remote control

Computer Science Issues in Ubiquitous Computing

10

Ubiquitous Computing is an emerging discipline bringing together different areas of Computer Science. This includes:
Mobile Computing Sensors & Ad hoc Networks Computer Architecture HCI Data Management Privacy & Security

Computer Architecture/Hardware
the computer artifact is of many sizes and shapes, including tiny inexpensive ones that could bring computing to everyone. three new kinds of hardware devices are required: [4]
very low power computing, low-power high-bits communication, and pen devices.

11

Mobile and Ubiquitous Systems

Several famous projects are going on in this area Famous Research labs are working on this area Software Competence Center Hagenberg (SCCH) Austria,
www.scch.at

Computer Science and Artificial Intelligence Laboratory, MIT

Mobile and Ubiquitous Systems: Oxygen[1]

Enables pervasive, human-centered computing through a combination of specific user and system technologies Speech and vision technologies enable to communicate with Oxygen as if we're interacting with another person, Working on collaboration technologies Oxygen's device, network, and software technologies dramatically extend range by delivering user technologies at home or at work. Computational devices embedded in homes, offices, and cars sense and affect immediate environment. Handheld devices empower us to communicate and compute anywhere Dynamic, self-configuring networks help machines locate each other as well as the people, services, and resources Software that adapts to changes in the environment or in user requirements help us do what we want when we want to do it.

12

Mobile and Ubiquitous Systems: Hydrogen [2]

Vision: Bringing together People, Information and Things Investigation and evaluation of middleware for:
Supporting social interaction among people Mobile ad hoc cooperation among team members Taking into account the very dynamic execution context of applications running on mobile devices

Development of experimental prototypes

Mobile meeting agenda Location aware reminder

13

Mobile and Ubiquitous Systems: Location awareness

Where I am?

Mobile and Ubiquitous Systems: Location awareness

Lot of labs working on it Lot of opportunities for collaborations

14

Mobile and Ubiquitous Systems

The central theme is to try and hide the impact of mobility Borrowed on notions of distribution transparency popular at the time (access, location, migration, replication, failure etc.)

Privacy
What is Privacy? An ability of an individual or group to stop information about themselves from becoming known to people other than those they choose to give the information to. [Wikipedia] Can be seen as an aspect of Security Different Facets of privacy [3] Bodily Privacy
Strip Searches, Drug Testing,

Territorial Privacy
Privacy Of Your Home, Office,

Privacy Of Communications
Phone Calls, (E-)mail,

Informational Privacy
Personal Data (Name, Address, Hobbies, )

15

Privacy: Issues

Different Ubiquitous systems are coming that will compromise privacy

Privacy: Bodymedia [3]

Communication Platform for wireless Transmission of Body Sensor Readings Bodymedia Data Center translates Raw Data into Lifestyle Data which is accessible via Web Interface on Company-Site

16

Privacy: Virtual Dad

Road Safety International Sells Black Box for Car Detailed Recording of Position, Acceleration, etc. Audio Warnings When Speeding, Cutting Corners Continuous Reckless Driving is Reported Home Sold as Piece of Mind for Parents Imagine if you could sit next to your teenager every second of their driving. Imagine the control you would have. Would they speed? Street race? Hard corner? Hard brake? Play loud music? Probably not. But how do they drive when you are not in the car? Source: http://www.roadsafety.com/Teen_Driver.htm

Privacy
Car Monitoring
ACME Rent-A-Car, New Jersey
Automatically Fines Drivers US$450.-at Speeds Over 79mph GPS Records Exact Position of Speed Violation

AutographSystem
Pilot Program 1998/99, Houston, TX Insurance based on individual driving habits (When, Where, How) GPS Tracking, Mobile Communication, Data Center

Future: Tracking Your Personal Mobile Phone Source: Insurance& Technology Online, Jan 2nd 2002 (http://www.insurancetech.com/story/update/IST20020108S0004)

Other Examples
Electronic Toll Gates Consumer Loyalty Cards Electronic Patient Data Computer Assisted Passenger Screening (CAPS)
Improved Systems in the Works (post 9/11)

Plans: Link Travel Data, Credit Card Records, Address Information,

17

Privacy: Some partial solutions

No hidden data collection!
Legal requirement in many countries

Establish privacy policies How to publish policies in Ubicomp?

Periodic broadcasts
Too many devices? Countless announcements an annoyance

Privacy: Some partial solutions

Use Anonymous data for analysis and experiment comes cheap no consent, security, access needed Pseudonyms allow for customization user can discard at any time Sometimes one cannot hide No anonymizing cameras & microphones Real-world data hard to anonymized Even pseudonyms can reveal true identity Real-world has complex situation-dependant security requirements Free access to medical data in emergency situations

18

Privacy: Some partial solutions

Identifiable data must be accessible
Users can review, change, sometimes delete

Collectors must be accountable

Try some Privacy-aware storage technology?

Ubicomp applications need lots of data

Increased need for accounting and access

Privacy: Some partial solutions

No spying, please (Proximity)
Devices only record if owner is present

Rumors should not spread (Locality)

Local information stays local Walls and Flower-Pots can talk (but wont do so over the phone)

19

Sensors in Ubiquitous Computing

Issues [3]
Why sensing is important for Ubiquitous computing Examples of how sensing features used in ubicomp projects Some new trends in Sensing Issues in distributed sensing

Sensors in Ubiquitous Computing

Why we need Sensing Ubiquitous Computing need systems that adapt to people, as opposed to people adapting to systems Reactive to what people do Proactive, anticipating what people want to do Situated, sharing context with human user Necessary for interaction between people and systems All this requires ability for observation of human activity if a computer merely knows what room it is in, it can adapt its behaviour without even a hint of AI

20

Sensors in Ubiquitous Computing

Enabling Technology As per Moores Law
sensors in overdrive dramatic drop in price Drop in size

Require energy

Sensors: Examples
Examples of Sensing in Ubiquitous Computing Location Sensing [4]
Active Badge System
ORL, Cambridge/UK, 1989-92 Locating people (and devices) Room-level accuracy Badges worn by people emit beacons Sensors with known location

21

Sensors: Examples
The Bat Ultrasonic Location System [3] Highly accurate indoor positioning 95% of readings within 3cm Bat device emits short pulse of ultrasound Ceiling mounted sensor array Sentient Computing [3] Use sensors to construct model of the environment Shared view of the world between system and user

Sensors: Examples
Weight Lab [3]
An environment in which all surfaces are load-sensitive Floor, tables, chairs,shelves, trays Activity tracking with infrastructure

22

Sensors: Smart device

Orientation-aware Newton MessagePad [5] Sensors as UI element

Sensors: Smart device

The Aware Home [3] Research initiative at GaTech A Living Lab for Ubicomp Research Large-scale deployment of sensors for perception of everyday activities Smart Home

23

Sensors: Smart device

Smart Palm PC
Microsoft Research Hinckley et al Sensors to improve user interaction Detecting simple percepts
holding & duration tilt, orientation etc

Detecting simple motion

dictaphone gesture scrolling

Sensors: Smart device

TEA Mobile Phone [3] Integration of diverse simple sensors (light, audio, acceleration, temperature, touch) Sensor fusion for perception of device context (car, meeting, home, ...) Shared context among phone users context call context phonebook

24

Sensors: Smart device

Wearable Sensing StartleCam [3]
MITMediaLab Example for sensing the user Sensing generally important in wearables

Sensors: Smart device

The Mediacup [3] TecO Karlsruhe, 1999-2000 Wireless sensor device embedded in ordinary coffee cup Movement, weight, temperature sensing On-board computation of user-level context: filled up, gone cold, etc. >95% reliable context prediction in everyday use

25

Sensors in Ubiquitous Computing

Device-based sensing (Portable, Wearable) Sense the user, the location, the immediate environment
Enable proactive/reactive behaviours, novel UI techniques

Environment-based sensing
Homogeneous sensing infrastructure to supply devices Smart environment control, responsive rooms etc

Wireless sensor devices and networks

Heterogeneous sensors, ad hoc organized Large-scale observation of the physical world Deep embedding in physical objects

Sensors: Distributed
Issues In current trends we require distributed sensing To facilitate combination of distributed observations Factoring out sensing from devices into infrastructure Separation of sensing and application into distributed entities Some implications Location and time need to be considered Data delivery from sensor to application Where to sense: device vs. infrastructure

26

Sensors: Location and Time

There are real-time and real-place issues Real-time issues Value of observation time-dependent
e.g. can become irrelevant after some time

Latency can contribute to inaccuracy

e.g. location reading of moving objects

Synchronization of distributed observations Real-place issues Arising with mobile/flexible sensor nodes Value of observation location-dependent
e.g. less relevant the greater the distance between sensor node and observed entity

Location also relevant for combination of sensors Localization hot issues for wireless sensor networks!

Sensors: Sensor Data Delivery

Application-level Delivery Models
Continuous: sensors communicate their data at prespecified rate Event-driven: report data only if event of interest occurs Request-reply: report only response to an application request

Network-level Routing Models

Flooding: broadcasting observations to neighbors, who rebroadcast until application is reached Directed Diffusion: data-centric protocol
Data is named by attribute-value pairs Applications submit queries, diffused through the network Nodes satisfying the query start transmitting data

27

Sensors
Where to Sense Smart Device vs Smart Environment e.g. location sensing
GPS model: infrastructure sends its coordinates, device computes its position Active Badge model: device/client sends beacon, infrastructure computes position

Privacy issues: whos in control over location information Distributed systems issues System-wide location management Client reliance on infrastructure Protocols to talk about location

Conclusion: What will happen

28

Labs & Projects

The Labscape Project - Ubiquitous Computing in the Cell Biology Laboratory

Larry Arnstein, Research Assistant Professor, Department of Computer Science & Engineering, University of Washington

Stephen S. Intille's work on ubiquitous sensing

Massachusetts Institute of Technology

Ubiquitous Computing research at Yano Lab, Tokushima university, Japan Ubiquitous Chip Research Laboratory
Internet Systems Research Laboratories, NEC Corp. and Osaka University collaboration lab

The Handheld Devices for Ubiquitous Learning Project (HDUL) Project Aura - Distraction-free Ubiquitous Computing
CMU

The Ubiquitous Communications (UbiCom) program at Delft University. Ubiquitous Networking Laboratory.
Harvard

Ubiquitous Computing research - The Intelligence Engineering Lab (IEL) of Institute of Software, Chinese Academy of Science (ISCAS) Ubiquitous Computing Lab
Department of Information Systems and Multimedia Design School of Engineering, Tokyo Denki University

UC Lab, Keio University M-Lab - The Mobile And Ubiquitous Computing Lab Ubiquitous Networked Media Computing Projects.
Nara Institute of Science and Technology

Ubiquitous Computing & Communication Laboratory, Keio University, Japan Multi-Agent Distributed Mobile and Ubiquitous Computing Lab at the Computer Science Department University of Saskatchewan

29

Books

Pervasive Computing: Technology and Architecture of Mobile Internet Applications by Horst Henn, Stefan Hepper, Klaus Rindtorff, Thomas Schack (Editor), Provides both a theoretical overview and a practical guide The first part of the book walks through the essential protocols, standards and architectures involved The second part uses a continuous real-world example to present an end-toend architecture for implementing pervasive computing applications. first book to offer an understanding of the scope and the implications of pervasive computing Good for IT managers, professionals, architects, consultants, developers, and students concerned with internet and web technologies. Contents of Pervasive Computing include:
Introduction to pervasive devices and their applications Overview of the key technologies and protocols Web application concepts WAP and beyond Voice Technology Server-side programming in Java Pervasive web application architecture Device-independent example application Accessing the example application via PC, PDA, WAP and voice

30

Books
Mobile and Ubiquitous Information Access by Fabio Crestani, Mark Dunlop, Stefano Mizzaro Contains thoroughly refereed postproceedings of the International Workshop on Mobile and Ubiquitous Information Access held in Udine, Italy in September 2003 during Mobile HCI 2003. The 21 articles in the book are organized in topical sections on foundations: concepts, models, and paradigms; interactions; applications and experimental evaluations; context and location.

Security for Ubiquitous Computing) by Frank Stajano Focuses on two very hot topics: ad-hoc wireless networking and security Covers security issues for wellestablished and widely used technologies such as GSM, Bluetooth, and IP Includes basics on security and cryptography Describes the security issues in peerto-peer networking

31

UbiComp 2005: Ubiquitous Computing: 7th International Conference, UbiComp 2005, Tokyo, Japan, September 11-14, 2005, Proceedings (Lecture Notes in Computer Science) (Paperback) by Michael Beigl (Editor), Stephen Intille (Editor), Jun Rekimoto (Editor), Hideyuki Tokuda (Editor)

Fundamentals of Mobile and Pervasive Computing by Frank Adelstein, Sandeep KS Gupta, Golden Richard III, Loren Schwiebert Provide engineering principles underlying mobile computing Good as a text book for graduates and senior undergraduates. Contains extensive Exercises, projects, and solutions. Gives the preview of the latest research that will help realize the full potential of pervasive computing

32

References
[1] MIT Project Oxygen, Computer Science and Artificial Intelligence Laboratory, MIT http://oxygen.lcs.mit.edu/index.html [2] The Hydrogen Project at Software Competence Center Hagenberg, Austria http://www.scch.at/index.jsp [3] Lecture notes and presentations, Summer School on Ubiquitous and Pervasive Computing August 7-14, 2002 Schloss, Dagstuhl, Germany http://www.vs.inf.ethz.ch/events/dag2002/ [4] Mark Wieser, Computer Science Laboratory at Xerox PARC, http://www.ubiq.com/weiser/ [5] Newton MessagePad, Newton Team, Apple. http://www.msu.edu/~luckie/gallery/mp100.htm

33