Multimedia Building Blocks

Text Sound MIDI Digital Audio Audio File Formats MIDI under Windows environment Audio & Video Capture
Multimedia Systems
UNIT II . 1

Text
There are three types of text that are used to produce pages of documents. Unformatted Text: enables pages to be created which comprise of strings of fixed-sized characters from a limited character set, also known as plaintext. Formatted Text: enables pages and complete documents to be created which comprise of strings of characters of different styles, size and shape with tables, graphics and images inserted at appropriate points, also known as richtext. Hypertext: enables an integrated set of documents to be created which have defined linkages between them.
Multimedia Systems
UNIT II . 2

Sound
The most sensuous element of multimedia. How the power of sound is used, can make the difference between an ordinary multimedia presentation & a professionally one. Misuse of sound can wreck the project. When something vibrates in the air by moving back & forth, it creates waves of pressure. These waves spread like ripples from pebble tossed into a still pool, and when they reach your eardrums, you experience the change of pressure, or vibrations, as sound.
Multimedia Systems
UNIT II . 3

Sound (contd.)
Acoustics is the branch of physics that studies sound. Sound pressure levels (loudness or volume) are measured in decibels (dB), which is actually the ratio between a chosen reference point on a logarithmic scale and the level that is actually experienced.

Multimedia Systems

UNIT II . 4

Use of sound
Use of sound in multimedia projects will not require highly specialized knowledge of harmonics, intervals, sine waves, notation, octaves, or the physics of acoustics and vibration. Need to know the following:
How to make sounds How to record and edit sounds on your computer How to incorporate sounds into multimedia work
Multimedia Systems
UNIT II . 5

Multimedia System Sounds

In Windows, system sounds are WAV files, and they reside in the Windows\Media subdirectory. Available system event sounds include start.wav, chimes.wav, chord.wav, ding.wav, logoff.wav, notify.wav, recycle.wav, tada.wav, and the Microsoft sound.wav which plays when Windows starts up. After installing Microsoft Office, more choices are available: applause.wav, camera.wav, carbrake.wav, cashreg.wav, chimes.wav, clap.wav, driveby.wav, dumbroll.wav, explode.wav, glass.wav, gunshot.wav, laser.wav, projector.wav, ricochet.wav, type.wav, and whoosh.wav.
Multimedia Systems
UNIT II . 6

Contd.
Newly created sound files can be added and installed in a manner that they play when system events occur: place the WAV sound files in ~\Windows\Media directory and use the Sound Control Panel to select them. Most sound used in a multimedia production is either digitally recorded audio or MIDI (Musical Instrument Digital Interface) music.
Multimedia Systems
UNIT II . 7

Digital Audio
It is created when a sound wave is represented using numbers a process referred to as digitizing. Sound can be digitized from a microphone, a synthesizer, existing tape recordings, live radio and television broadcasts, and popular CDs, i.e., any source, natural or prerecorded. Digitized sound is sampled sound. Every nth fraction of a second, a sample of sound is taken and stored as digital information in bits and bytes.
Multimedia Systems
UNIT II . 8

Contd
The quality of this digital recording depends upon how often the samples are taken (sampling rate or frequency, measured in kHz or thousands of samples per second) and how many numbers are used to represent the value of each sample (bitdepth, sample size, resolution, or dynamic range). The more often a sample is taken and the more data is stored about that sample, the finer the resolution and quality of the captured sound is, when it is played back.
Multimedia Systems
UNIT II . 9

Contd
Digital audio is said to be device independent, because the quality of audio is based on the quality of recording and not the device on which end user will play the audio. The three sampling frequencies most often used in multimedia are CD-quality 44.1 kHz, 22.05 kHz, and 11.025 kHz. Sample sizes are either 8 bits or 16 bits. The larger the sample size, the better the data describes the recorded sound.
Multimedia Systems
UNIT II . 10

Quantization & Clipping

The process of rounding off the value of each sample to the nearest integer is called quantization. If the amplitude is greater than the intervals available, clipping of the top and bottom of the wave occurs. Quantization can produce an unwanted background hissing noise, and clipping may severely distort the sound.
Multimedia Systems
UNIT II . 11

Preparation of Digital Audio Files

Having analog source materials music or sound effects that have been recorded on analog media, i.e., cassette tapes the first step is to digitize the analog material by recording it onto computer-readable digital media. Generally, this is done by playing sound from one device (such as a tape recorder) into the computer, using appropriate audio digitizing software.
Multimedia Systems
UNIT II . 12

Contd
One should focus on two crucial aspects of preparing digital audio files:
Balancing the need for sound quality against file size. Higher quality means larger files, requiring longer download times on the Internet and more storage space on a CD or DVD. Setting proper recording levels to get a good, clean recording.

Multimedia Systems

UNIT II . 13

File Size versus Quality

Sampling rate determines the frequency at which samples will be drawn for the recording. Sampling at higher rates more accurately captures the high-frequency content of the sound. Audio resolution (such as 8- or 16-bit) determines the accuracy with which a sound can be digitized. Using more bits for the sample size yields a recording that sounds more like its original.
Multimedia Systems
UNIT II . 14

Contd
Stereo recording are more lifelike and realistic because human beings have two ears. Mono recordings are fine, but tend to sound a bit flat and uninteresting in comparison to stereo recordings. Logically, stereo sound files require twice as much storage space as mono files for the same length of play time.
Multimedia Systems
UNIT II . 15

Contd
Formulas for determining the size (in bytes) of a digital recording. For monophonic recording: sampling rate duration of recording in sec. (bit resolution / 8) 1 For stereo recording: sampling rate duration of recording in sec. (bit resolution / 8) 2
(Note: sampling rate is measured in kHz, or thousand samples per sec, so to convert from kHz to a whole number, multiply by 1000. Resolution is measured in bits per sample. Since there are 8 bits in a byte, therefore divide the bit resolution by 8.)
Multimedia Systems
UNIT II . 16

Setting Proper Recording Levels

Any good piece of digital audio recording and editing s/w displays digital meter to let you know how loud your sound is. Watch the meters closely during recording, and you will never have a problem. Unlike analog meters, digital meters peak out. To avoid distortion, do not cross over the limit. If it happens, lower your volume and try again. Try to keep peak levels b/w -3 & -10. If you go over the peak, whether you can hear it or not, you introduce distortion into the recording.
Multimedia Systems
UNIT II . 17

Editing Digital Recordings

Audacity is a free open source sound editing application for Windows, Macintosh, and Linux OS. With this tool you can create sound tracks and digital mixes. Basic sound editing operations:
Multiple Tracks Trimming Splicing & Assembly Volume Adjustments Format Conversion Resampling or Downsampling Fade-ins and Fade-outs Equalization Time Stretching Digital Signal Processing (DSP) Reversing Sounds
Multimedia Systems
UNIT II . 18

Making MIDI Audio

MIDI (Musical Instrument Digital Interface) is the quickest, easiest, and the most flexible tool for composing your own original score one of the most creative and rewarding aspects of making a multimedia project; though creating an original MIDI score is hard work requires something about music, playing the piano, and lot of good ideas; beyond that it takes time and musical skill to work with MIDI.
Multimedia Systems
UNIT II . 19

Contd
The process of creating MIDI is quite different from digitizing existing audio. Digitized audio is thought as analogous to a bitmapped graphic image (both use sampling of the original analog medium to create a digital copy), while MIDI is thought analogous to structured or vector graphics (both involve instructions provided to s/w to be able to re-create the original one on demand)
Multimedia Systems
UNIT II . 20

Contd
For digitized audio, the audio is played through a computer or device which can digitally record the sound, while to make MIDI scores, sequencer software and a sound synthesizer (built into the sound board on PCs) will be needed. A MIDI keyboard is also useful for simplifying the creation of musical scores.

Multimedia Systems

UNIT II . 21

Contd
Using sequencer s/w, you can record and edit MIDI data. Rather than recording a note, the s/w creates data about each note as it is played on a MIDI keyboard (or another MIDI device), e.g., which note it is, how much pressure was used on the keyboard to play the note, how long it was sustained, and how long it takes for the note to decay or fade away. This information, when played back through a MIDI device, allows the note to be reproduced exactly.
Multimedia Systems
UNIT II . 22

Contd
Because the quality of the play back depends upon the end user s MIDI device rather than the recording, MIDI is device dependent. The sequencer software quantizes the score to adjust for time inconsistencies, and it may also print a neatly penned copy of the score to paper.

Multimedia Systems

UNIT II . 23

Contd
Advantage of structured data such as MIDI is the ease with which the data can be edited. E.g., you have a piece of music being played on a honky-tonk piano, but your client wants the sound of saxophone. In this case if you had the music in digitized audio, you would have to rerecord and redigitize the music. But when it is in MIDI data, there is a value that designates the instrument to be used for playing back the music. So to change the instruments, just change that value.
Multimedia Systems
UNIT II . 24

General MIDI
Instruments that can be synthesized are identified by a General MIDI numbering system that ranges from 0 to 127 (see table in Book by Tay Vaughan). Some MIDI devices offset the numbers by one, using 1 to 128. Most of the softwares have a switch to accommodate these devices.

Multimedia Systems

UNIT II . 25

Contd
It is so easy to edit MIDI data, therefore, many fine adjustments can be made to the music. Since MIDI is device dependent and the quality of consumer MIDI playback h/w varies greatly, MIDI may be used as a production tool rather than a delivery medium. Till now, MIDI is the best way to create original music for multimedia projects, so use MIDI to get the flexibility and required creative control.
Multimedia Systems
UNIT II . 26

Components of a MIDI System

Synthesizer: A sound generator. A good synthesizer often has a microprocessor, keyboard, control panels, memory, etc. Sequencer: A stand-alone unit or a s/w pgm for a PC. It is used to be a storage server for MIDI data. Now a days it is a software music editor on the computer. It has one or more MIDI INs and MIDI OUTs. Track: It is used in sequencer to organize the recordings. Tracks can be turned on or off on recording or playing back.
Multimedia Systems
UNIT II . 27

Contd
Channel: These are used to separate information in a MIDI system. There are 16 MIDI channels in one cable. Channel numbers are coded into each MIDI message. Timbre: The quality of sound, e.g., flute sound, cello sound, etc. Multitimbral capable of playing many different sounds at the same time (e.g., piano, brass, drums, etc.) Pitch: Musical note that the instrument plays. Voice: The portion of the synthesizer that produces sound. Synthesizers can have may (12, 20, 24, 36, etc.) voices. Each voice works independently and simultaneously to produce sounds of different timbre and pitch. Patch: The control settings that define a particular timbre.
Multimedia Systems
UNIT II . 28

MIDI Connectors
These are 5-pin ports found on the back of every unit. MIDI IN: The connector via which the device receives all MIDI data. MIDI OUT: The connector through which the device transmits all the MIDI data it generates itself. MIDI THROUGH: The connector by which the device echoes the data receives from MIDI IN.
Multimedia Systems
UNIT II . 29