IMD20503 – Digital Audio and Video

Basic of Sound
How sound works
Analog and digital sound technology
Sound design
How sound works
Importance of sound

• Sound is essential in communicating knowledge.

• Sound greatly enhances images and, especially, video or
animation.
• Sound can manipulate the mood of spectators with
background music and sound effects.
• Sound can provide continuity to jumpy frame cuts, but
audiences react gingerly to out-of synch dialog and
misplaced sound effects.
• Enhanced MM in the form of voice-overs, narrations and
instructions.
How sound works
What is sound?

• Sound is basically a vibration. (analog)

• Sound is a force:emotional, perceptual, physical.
• It can excite feeling and convey meaning
• Sound is omnidirectional Æ everywhere.
• When something “makes a sound”, it vibrates and emits
pressure waves that are received in the human ear. The
brain interprets these as sound.
How sound works
What is sound?

• Simple waveform can be explained as follows;

– Imagine a rectangular glass tank full of water.
– Looking down from above imagine a vibrating object (an
electric toothbrush) being lowered at the side of the tank.
– Ripples will spread out let say from left to right.
– Picture yourself looking through the side of the tank, with
your eyes at water level as the ripples pass before you.
– What you see is the repeating pattern of the waves created
by the object vibrating in the water.
– If we speed up the vibration, it will increase the frequency.
How sound works
Waveform / Sound wave
• A waveform or sound wave gives us a measurement of the
speed of the air particles and the distance, which they
travel for a given sound.
• Period is the distance between two crests, and measured
in seconds.
• A number of crests (peaks) may occur within a second, so
the number of peaks that occur in one second is called the
frequency.
How sound works
Noise

• Unwanted sound in a recording.

• Always avoid noise because it makes it harder to hear the
message that you want to deliver.
• Examples:
– Vehicles (planes, trains and automobiles)
– Nature (wind, rain, dogs, birds and cats)
– Machines (computer fans, phones)
– Electrical (fluorescent light, refrigerator, electrical
circuits)
– Recording errors (cheap microphones, unbalanced
audio signals, analog to digital conversion inside the
computer)
How sound works
Components of waveform

• Acronym : WAVE PFH

– Wavelength
– Amplitude
– Velocity
– Envelope
– Phasing
– Frequency
– Harmonics
How sound works
Wavelength
• Literally, the length of the wave.
• The physical distance through a medium such as atmosphere,
that a soundwave travels to complete cycle.
• The length of a wave is measured from one point on the
waveform to the exact point of the next repeating cycle.
• It is usually applied to continuous, predictable waves whose
cycles are uniform and easily measurable.
Wavelength (distance)
How sound works
Amplitude
• The amplitude measures the relative loudness of the
sound, which is the distance between a valley and a crest.
• The amplitude determines the sound’s volume.
• Volume is measured by the amount of energy it takes to
create the sound.
• The volume of sound or amplitude is measured in decibels
(dB).
• A decibel is the smallest change in loudness that a human
ear can detect.
• Human hearing ranges from 1 decibel, called the threshold
of hearing, to over 120 decibels, the threshold of pain.
How sound works
Amplitude
How sound works
Velocity

• Speed through the atmosphere.

• The speed or velocity in air and other gases depends on
temperature.
• Speed of sound increase in warm, moist air and decrease
in cold, dry air.
• Sounds at high amplitude might go father but travel slower
which is the opposite for lower amplitude.
How sound works
Envelope
• Refers to an imaginary curve that wraps the entire waveform,
generated by musical instruments
• All sound posses a characteristic change in amplitude or
loudness as time increases.
• The features included in an envelope are attack, decay, sustain
and release(ADSR).
i. Attack
• The sudden increase of amplitude to full sound intensity.
ii. Decay
• occurs just after the attack, is a slight decrease in amplitude.
iii. Sustain
• the main part of a note, when it plays for its longest duration or the
sound is held.
iv. Release
• which the amplitude fades out (when lifting a finger from a key).
How sound works
Envelope

Attack Decay Sustain Release

A
m
p
li
t
u
d
e

Duration
How sound works
Phasing
• Think of phase as the information our ears use to tell from
where a sound originated, because sound arrives at our
ears at slightly different times.
• measured in degrees (0-360)
• Types of phase :
– In phase
– Out of phase
How sound works
Phase

When two waveforms match

each other and are playing
simultaneously they are in
phase. Sound waves which are
exactly in phase add together
to produce a stronger wave.
How sound works
Phase
If one of the two
waveforms is playing
back at slightly
different speed, they
are out of phase. It will
caused them to weaken
each other.
If one waveform is
exactly 180 degrees out
of phase, the waveform
will cancel each other
out and produce silence.
How sound works
Phase

Sound waves which

have varying phase
relationships produce
differing sound
effects.
How sound works
Frequency and Pitch
• Frequency is the number of cycles that a vibration completes in
one second.
• Pitch is measured by frequency.
• Pitch is the relative tonal highness or lowness of a sound.
• As an object oscillates more rapidly, it creates a high pitch
sound, which is registered as a high frequency sound.
• A low-frequency sound is produced by a slowly vibrating object,
such as the thicker strings on a guitar.
• Frequency is measured in Hertz.
• The human ear can perceive a range of frequencies from 20 Hz
(Hertz) to 20,000 Hz (Also expressed as 20 KHz).
How sound works
Frequency and Pitch

• Comparison of frequency range

Name Low frequency High frequency

Human 20 Hz 20,000 Hz
Cat 45 Hz 85,000 Hz
Dog 50 Hz 45, 000 Hz
Elephant 5 Hz 10,000 Hz
Bat 10,000 Hz 120,000 Hz
Dolphin 1,000 Hz 120,000 Hz
How sound works
Others

ƒ Few objects produce sounds at a single frequency. Most

musical instruments generate multiple frequencies for every
note, so that the difference between two same-pitch
instruments can be noted, e.g. a violin and a flute or a piano
and a bass.
ƒ The combinations of frequencies in an instrument is known as
the timbre.
ƒ A sound wave has a main frequency and minor frequencies,
referred as overtones or harmonics.
ƒ Duration measures frequency, amplitude and timbre over time.
ƒ Fourier Theorem – “A sound wave is a combination of sinus
waves created during vibration”.
How sound works
Complex sound wave

A complex sound wave showing a main

frequency and minor overtones.
How sound works
Octave
• a musical term for doubling of frequency.
• in other words, the interval between a note that plays at a
given frequency and another note that plays at double this
frequency.
• measurement unit for pitch
• Example a pitch at 20Hz is one octave from a pitch at
40Hz.
An Octave shown as two sound
waves with the same amplitude,
but with different pitches. The
black wave has a frequency double
the red one.
How sound works
Shape
• Indicates its tonal quality.
• 3 different shapes:

i) Sine Wave
• indicates a gentle and smooth, sweet sound

ii) Square Wave

• reflects a rich and full, harsh sound

iii) Sawtooth Wave

• indicates a piercing and colourful sound

• Generally, the smoother the sound wave, the cleaner and

sweeter it sounds.
Analog and Digital Sound
History

• Evolution from analog to digital technology.

• The analog century begins with a nursery rhyme….
Analog and Digital Sound
Analog sound

• Analog means “the same as” or “similar to”.

• Root words “analogous” and “analogy”, both of which
mean “a similar copy”.
• Analog signals are exact replicas of the original sound
source.
• Noise exist in the recording ( not the sound source but part
of the analog recording).
• Analog media :
– Audio cassette
– LP record
– Reel-to-reel tape
– Broadcast television
Analog and Digital Sound
Analog sound – analog media

Audio Cassette

LP record / Vinyl

Reel-to-reel tape
Analog and Digital Sound
How analog recording works?

• A process of transduction.
• Transducers are the keys to sound recording and listening.
• A microphone, a loudspeaker and our ears are all types of
transducer.
• A transducer is a device that helps to change one type of
energy into another.
• In terms of audio, it is a device that converts sound waves
into electrical signals and vice versa.
Analog and Digital Sound
How analog recording works?
Analog and Digital Sound
Digital sound

• “Digital” means numbered.

• Using number is often a more precise way of expressing
information.
• The conversion of analog signals into a form that a
computer can breakdown.
• Digital media :
– Compact Disc
– DAT
– MiniDisc
Analog and Digital Sound
Digital sound – digital media

Compact Disc

Digital Audio Tape (DAT)

MiniDisc
Analog and Digital Sound
How digital recording works?

• Audio becomes ‘digital audio’ when it passes from an

analog source, such as microphone or audio out of a
keyboard to you PC.
Analog and Digital Sound
How digital recording works?

• The recording process of a digital system takes samples of

the analog signals as it enters the system.
• In order to get good digital recording, it has to have
enough :
– Samples per second
– Bit depth to accurately reproduce the source material
• Audio CD uses 44,100hz, stereo and 16 bit samples per
second.
Analog and Digital Sound
Analog vs digital
• Which is better?
• There are differences between analog and digital but
neither is better or worse than the other.
• They are simply different.
• But now our focus is on digital audio because :
– We will use digital audio most of the time through
software and hardware to create, record and process
the sound work.
– The start up expenses are relatively low compared with
purchasing analog gear.
– Analog systems (recording and editing) are becoming
obsolete.
Analog and Digital Sound
Analog vs digital

• Today, every sound designer works in the digital arena.

• Many professional music recordings in the studio are
recorded on tape.
• Usually this recording is digitized, processes and edited in
a Digital Audio Workstation (DAW) and then sent back to
tape or storage medium like a hard drive.
• Both the analog and digital processes are at work but this
is changing.
Sound Design
What is sound design?
• What is sound design?
• How important it is to a visual and nonvisual experience?
• Where does it all begins?
• Sound design is a technical/conceptually creative field.
• Sound Design can also be defined as: "The manipulation
of audio elements to achieve a desired effect."
• It covers all non-compositional elements of a film, a play, a
music performance or recording, computer game software
or any other multimedia project.
• A person who practices the art of sound design is known
as a Sound Designer.
Sound Design
What is sound design?
• Area that uses sound designer :
1. Theatre
• Use large mechanical devices to produce sound
effects.
• Now, there are 2 roles of sound design in theatre :
– Technical Sound Design
ƒ Design sound system
– Conceptual Sound Design
ƒ Design what sound to be used to create mood
and setting of the play.
2. Radio
• Use large mechanical devices then better technology
allows sound effects to be recorded and library of
sound was created.
Sound Design
What is sound design?
3. Film
• Originally the sound effects was recorded live as
the film was being projected onto a large screen in
a recording studio.
• Now, sound effects track is complied of many
tracks layered together and mixed to form a
dynamic soundtrack.
• Foley sound effects – create sound effects in the
studio by Foley artists.
4. Music
• Involved in the creation of the overall sound of a
recording and sometimes live performance.
Sound Design
What is sound design?
Sound Design
Sound effects design in “Star Wars”
Name Effects

R2D2 Half of the sound are electronic and the other half are
vocalization, water pipes and whistles

Imperial walkers A machinist’s punch press along with the sounds of bicycle
chains dropping on cement.

Light saber An old tv set along with the hum of a 35mm projecter mixed
together.

Laser blasts A hammer hitting the wires of a radio tower.

Sound Design
Sound effects design in “Star Wars”
Name Effects
Tiefighter An elephant’s bellow altered in various way.
Speeder bike A P-5 Mustang airplane and a P-38 Lockheed Interceptor
combined and mixed.
Luke Skywalker’s The sound of Los Angeles Harbor Freeway through a vacuum
landspeeder cleaner pipe.

Chewbacca The sounds of walruses and other animals combined together.

Ewokese The combination and layering Tibetan, Mongolian and Nepali

language languages.
Sound Design
Stages in sound design
1. Pre production
– Story, design document, proposal
– “Spot” for sound effects
– Defining sound effects and sources
– Sound blueprint and approval process
– Technical playback considerations
2. Production
– Sound asset collection
– Inventing original sound
3. Post production
– Sound processing
Sound Design
Pre production
• Story, design document, proposal
– The sound design process starts with a working
concepts.
– Can be anything from a design document to the mood
of the director or producer has in mind.
– Need to get a feel for what the message is going to be
in the project.
– If it is a script, the text need to be reviewed for the
sound conceptualization.
Sound Design
Pre production
• “Spot” for sound effects
– “Spotting” for sound is simply gaining some ownership
to the design document or screenplay.
– It is a process of literally writing down aural ideas as the
reading occurs.
– The processes include:
• the first impression of the target
• sketch out rough sound map with the outcome
expected from the project.
• Write down as many ideas as can fit on the page
because it will save many hours of planning.
Sound Design
Pre production
• Defining sound effects and sources
– Details on what the sound are and where they are to be
found such as at sound libraries and live recording.
– Sound designer need to be familiar with sound libraries
and recording techniques.
– There are also many online sites that have free
downloadable sounds and fee-based sound for usage
and collection Æ not of high quality sound.
– A good tips is to start collecting sound your own sound
library even if you do not need them in projects because
we can use them in future works.
Sound Design
Pre production
www.soundrangers.com
Sound Design
Pre production
www.partnersinrhyme.com
Sound Design
Pre production
• Sound blueprint and approval process
– Need to get approval for the work.
– Categorize the sound based on frequency ranges,
types of sound used and any other characteristic that
can be defined.
Sound Design
Pre production
• Technical playback considerations
– When creating the initial plan for the sound design,
consider :
• What the playback device will be?
• What platforms?
• If it is for the Internet, the bandwidth and the average
playback system need to be decided.
• Compression formats
• In film, need to consider theater or cinema standards
ie Dolby, THX, DTS
Sound Design
Production
• Sound asset collection
– Start collecting the sound asset based on the pre-
production stage.
– The sounds can be from sound libraries or source
recordings.
– Sound object falls into two types:
• Single sound
• Mixed sounds
Sound Design
Production
• Inventing original sound
– Originally recorded sounds are better, more fitting for the
scenario and more original.
– 2 types of sound effects:
• Small subject sounds
• Glasses clinking, paper being crumpled, door
squeaks, footsteps, cloth movement and human
noises (sneezing, coughing).
• Recorded in a studio or by Foley artists.
• Big subject sounds
• Sounds that need to be recorded outside such as
explosion, gunfire, crowds, cars, planes, trains and
machinery.
Sound Design
Post Production
• Sound processing
– Altering and enhancing sounds to fit the scenario.
– The final stage is mixing and mastering the sound objects.
– In film, the re-recording process integrates all of the
elements together and mixes them into a composite
soundtrack.
– Soundtrack consists of the dialogue track, music track and
sound effects track.
– The three entities need to be properly mixed and placed in
order to be effective.
– The soundtrack should support the visual content and
storyline.
Summary

• How sound works

– Definition
– Terms : WAVE PFH
• Analog and digital sound technology
– Analog vs digital
• Sound design
– Definition
– Stages in sound design; pre production, production and
post production.