CHAPTER 11

What is multimedia?
M lti Multimedia can h di have a many d fi iti definitions th these

include: Multimedia means that computer i f l i di h information can b i be represented through audio, video, and animation in addition t t diti dditi to traditional media (i l di (i.e., t t text, graphics/drawings, images).

General definition
A good general working d fi iti d l ki definition f thi module i for this d l is: Multimedia is the field concerned with the

computer controlled integration of text, ll d i i f graphics, drawings, still and moving images (video), i ti ( id ) animation, audio, and any other media di d th di where every type of information can be represented, stored, transmitted and processed digitall st d t s itt d d ss d digitally.

Multimedia Application Definition pp

A M lti Multimedia A li ti i an application which di Application is li ti hi h

uses a collection of multiple media sources e.g. text, graphics, images sound/audio graphics images, sound/audio, animation and/or video.

What is HyperText and HyperMedia? yp yp

H Hypertext is a text which contains li k t other t t i t t hi h t i links to th

texts. The term was i h invented b Ted Nelson around 1965. d by d l d 6

HyperText Navigation yp g
Traversal through pages of hypertext is therefore

usually non-linear (as indicated below).

This has implications in layout and organisation of

material and d t i l d depends a l t on th application at d lot the li ti t hand.

Hypermedia yp
H HyperMedia i not constrained t b t t b d It M di is t t i d to be text-based.

can include other media, e.g., graphics, images, and especially the continuous media sound and video video.

Example Hypermedia Applications? p yp pp

Th World Wide Web (WWW) i the best example of The W ld Wid W b is th b t l f

ahypermedia application. Powerpoint i Adobe Acrobat Many Others?

Multimedia Applications pp
E Examples of M lti di A li ti l f Multimedia Applications i l d include: World Wide Web M lti Multimedia Authoring, e.g. Ad b /M di A th i Adobe/Macromedia Di t di Director Hypermedia courseware Video on demand Video-on-demand Interactive TV Computer Games Virtual reality Digital video editing and p g g production systems y Multimedia Database systems

Multimedia Systems y
A M lti Multimedia S t di System i a system capable of is t bl f

processing multimedia data and applications. A Multimedia S l i di System i characterised b the is h i d by h processing, storage, generation, manipulation and rendition of M lti di i f diti f Multimedia information. ti

Characteristics of a Multimedia System y

A M lti Multimedia system h f di t has four b i characteristics: basic h t i ti Multimedia systems must be computer controlled. M lti Multimedia systems are i t di t integrated. t d The information they handle must be represented digitally. The interface to the final presentation of media is usually interactive.

Challenges for Multimedia Systems g y

Di t ib t d N t Distributed Networks k Temporal relationship between data Render different data at same time continuously. Sequencing within the media

playing frames in correct order/time frame i video l i f i d /i f in id

Synchronisation inter-media scheduling

E.g. E g Video and Audio Lip synchronisation is clearly important for humans to watch playback of video and audio and even animation and audio. Ever tried watching an out of (l ) sync f l d h f (lip) film f a l for long time?

Key Issues for Multimedia Systems y y

Th k i The key issues multimedia systems need to d l with lti di t d t deal ith

here are:

How to represent and store t H t t d t temporal i f l information. ti How to strictly maintain the temporal relationships on play back/retrieval What process are involved in the above. Data has to represented digitally AnalogDigital p g y g g Conversion, Sampling etc. Large Data Requirements bandwidth, storage,

Data compression is usually mandatory

Desirable Features for a Multimedia System y

Given the above challenges the following feature a

desirable (if not a prerequisite) for a Multimedia System:

Very High Processing Power needed to deal with large data processing and real time delivery of media.
Special hardware commonplace.

Multimedia Capable File System needed to deliver real time real-time media e.g. Video/Audio Streaming. Special Hardware/Software needed e.g. RAID technology. Data Representations Fil F D R i File Formats that support h multimedia should be easy to handle yet allow for compression/decompression in real-time.

Desirable Features for a Multimedia System y

Efficient and High I/O input and output to the file input

subsystem needs to be efficient and fast. Needs to allow for realtime recording as well as playback of data. e.g. Direct to Disk recording systems. g y Special Operating System to allow access to file system and process data efficiently and quickly. Needs to support direct transfers to disk, real-time scheduling, fast interrupt processing, I/O streaming etc. / i Storage and Memory large storage units (of the order of hundreds of Tb if not more) and large memory (several Gb or more). L ) Large C h also required and hi h speed b Caches l i d d high d buses f efficient for ffi i management. Network Support Client-server systems common as distributed di t ib t d systems common. t Software Tools user friendly tools needed to handle media, design and develop applications, deliver media.

Components of a Multimedia System p y

N Now let us consider th C l t id the Components (H d t (Hardware and d

Software) required for a multimedia system:

Capture devices Video Camera, Video Recorder, Audio Camera Recorder Microphone, Keyboards, mice, graphics tablets, 3D input devices, tactile sensors, VR devices. Digitising Hardware Storage Devices Hard di k CD-ROMs, DVD-ROM, etc i d disks, Communication Networks Local Networks, Intranets, Internet, Multimedia or other special high speed networks. Computer Systems Multimedia Desktop machines, Workstations, MPEG/VIDEO/DSP Hardware Display Devices CD-quality speakers, HDTV,SVGA, Hii l i li k i Res monitors, Colour printers etc.

Applications pp
E Examples of M lti di A li ti l f Multimedia Applications i l d include: World Wide Web H Hypermedia courseware di Video conferencing Video on demand Video-on-demand Interactive TV Groupware Home shopping Games Virtual reality Digital video editing and p g g production systems y

A Brief Look at Multimedia Data:Input and o at Format

T t and St ti D t Text d Static Data

Source: keyboard, speech input, optical character recognition, data stored on disk. Stored and input character by character: Storage of text is 1 byte per character (text or format character). h t ) For other forms of data (e.g. Spreadsheet files). May store format as text (with formatting) others may use binary encoding. Format: Raw text or formatted text e.g HTML, Rich Text Format (RTF), Word or a program language source (C, Pascal, etc.. Not temporal BUT may have natural implied sequence e g e.g. HTML format sequence, Sequence of C program statements. Size Not significant w.r.t. other Multimedia data.

A Brief Look at Multimedia Data:Input and o at Format

Graphics Format: constructed by the composition of primitive objects such as lines, polygons, circles, curves and arcs. , p yg , , Input: Graphics are usually generated by a graphics editor program (e.g. Illustrator) or automatically by a program (e.g. Postscript). Postscript) Graphics are usually editable or revisable (unlike Images). Graphics input devices: keyboard ( p p y (for text and cursor control), ), mouse, trackball or graphics tablet. graphics standards : OpenGL, PHIGS, GKS Graphics files usually store the primitive assembly Do not take up a very high storage overhead.

A Brief Look at Multimedia Data:Input and o at Format

Images

Still pictures which (uncompressed) are represented as a bitmap (a grid of pixels). Input: digitally scanned photographs/pictures or direct from a digital camera. Input: May also be g p y generated by p g y programs similar to g p graphics or animation programs. Stored at 1 bit per pixel (Black and White), 8 Bits per pixel (Grey Scale, Colour Map) or 24 Bits per pixel (True Colour) Size: a 512x512 Grey scale image takes up 1/4 Mb, a 512x512 24 bit image takes 3/4 Mb with no compression. This overhead soon increases with image size modern high digital camera 10+ Megapixels 29Mb uncompressed! Compression is commonly applied.

A Brief Look at Multimedia Data:Input and o at Format

A di Audio Audio signals are continuous analog signals. I Input: microphones and th di iti d and stored t i h d then digitised d t d CD Quality Audio requires 16-bit sampling at 44.1 KHz

Even higher audiophile rates (e g 24-bit, 96 KHz) (e.g. 24-bit

1 Minute of Mono CD quality (uncompressed) audio requires 5 Mb. 1 Minute of Stereo CD quality (uncompressed) audio requires 10 Mb. Usually compressed (E.g. MP3, AAC, Flac, Ogg Vorbis).

A Brief Look at Multimedia Data:Input and o at Format

Video

Input: Analog Video is usually captured by a video camera and then digitised. There are a variety of video (analog and digital) formats Raw video can be regarded as being a series of single images. There yp y 5, 30 50 per are typically 25, 3 or 5 frames p second. E.g. A 512 512 size monochrome video images take 25 x 0.25 = 6.25Mb for a minute to store uncompressed. Typical PAL digital video (720 576 pixels per colour frame) ~ 1 2 x 1.2 25 = 30Mb for a minute to store uncompressed. High Definition DVD (14401080 = 1.5 Megapixels per frame) ~ 4.5 x 25 = 112 5Mb for a minute to store uncompressed. (There are higher 112.5Mb uncompressed possible frame rates!) Digital video clearly needs to be compressed for most times.