Part V

Sound Field Synthesis

History of Sound Reproduction

1881 Clément Ader Théatrophone – 2-channel telephone-based

spatial audio transmission
1930-40 Harvey Fletcher Wall of Sound
1931 Alan Dower Blumlein Patent on stereophonic recording and reproduc-
tion
12.3.1932 Stokowski First stereo recording on (vinyl) disc
1940 Harvey Fletcher Demonstration of 3-channel system in Carnegie
Hall
1940 Walt Disney Fantasound
1962 BBC First radio program in stereo (FM)
1973 M.A. Gerzon Publication on Ambisonics
1988/93 A.J.Berkhout Publication on Wave Field Synthesis

Spors | VL-STiASP WS 2017/18 | Sound Field Synthesis | Spatial Sound Reproduction 5-1
Terminology

scene
representation

virtual scene

recording/
playback
sound event
auditory event
DSP
sound event auditory event sound event

sound scene audio signals

recording room auditory scene

listening room

Spors | VL-STiASP WS 2017/18 | Sound Field Synthesis | Spatial Sound Reproduction 5-2
Physical versus Listener Centric Viewpoint

Spors | VL-STiASP WS 2017/18 | Sound Field Synthesis | Spatial Sound Reproduction 5-3
Objectives

Authenticity
time- and/or space-shifted reproduction of acoustic scene without noticeable difference
to original
Plausibility
plausible reproduction of acoustic scene
Immersion
involvement of listener in acoustic scene (’suspension of disbelief’)
...

Two substantially different (technical) options

synthesis of sound pressure at the ears using headphones or loudspeakers
synthesis of sound field using loudspeakers

Spors | VL-STiASP WS 2017/18 | Sound Field Synthesis | Spatial Sound Reproduction 5-4
Approaches

Binaural
synthesis of sound pressure at the ears using headphones or loudspeakers
replication of acoustic properties of torso/outer ear
Stereophonic
several loudspeakers, perceptually motivated,
level- or time-differences between pairs of loudspeakers, creation of a phantom source
Wave Field Synthesis (WFS)
many loudspeakers (at small distance), physically motivated,
synthesis of the sound field of a virtual source
Higher-Order Ambisonics (HOA):
several up to many loudspeakers, physically motivated, circular/spherical setups,
synthesis of the sound field of a virtual source, different variants
Numerical approaches
several up to many loudspeakers, synthesis of sound field at selected control points,
synthesis of the sound field of a virtual source, different variants
Spors | VL-STiASP WS 2017/18 | Sound Field Synthesis | Spatial Sound Reproduction 5-5
Loudspeaker Array
Institute of Communications Engineering

Spors | VL-STiASP WS 2017/18 | Sound Field Synthesis | Sound Field Synthesis 5-6
Sound Field Synthesis

Synthesized sound field

N
X
P (x, ω) = D(xn , ω) G(x|xn , ω)
n=1

G(x|xn , ω) sound field emitted by individual loudspeakers (secondary sources)

D(xn , ω) driving signal of individual loudspeakers
Goal: Synthesize sound field of virtual source S(x, ω) within listening area V
with an ensemble of individually driven secondary sources surrounding V
Spors | VL-STiASP WS 2017/18 | Sound Field Synthesis | Sound Field Synthesis 5-7
The Single Layer Potential

Synthesized sound field

I
P (x, ω) = D(x0 , ω) G(x − x0 , ω) dA(x0 )
∂V

solve for D(x0 , ω) under the condition P (x, ω) = S(x, ω) for x ∈ V

requires solution of integral equation (Fresnel integral of first kind)

Spors | VL-STiASP WS 2017/18 | Sound Field Synthesis | Sound Field Synthesis 5-8
Approaches

all secondary sources emit the same field G(x|x0 , ω) = G(x − x0 , ω)

free-field propagation
⇒ model G(x − x0 , ω) as point source at position x0 (Green’s function)
I
P (x, ω) = D(x0 , ω) G(x − x0 , ω) dA(x0 )
∂V

1. Explicit solution
series expansion of Kernel into (geometry dependent) basis functions
solution by comparison of coefficients
2. Equivalent scattering approach
interpretation of boundary ∂V as sound-soft virtual scatterer
solution of scattering problem
3. High-frequency approximation of Kirchhoff integral equation

Spors | VL-STiASP WS 2017/18 | Sound Field Synthesis | Sound Field Synthesis 5-9
The Kirchhoff Integral Equation


I   x ∈V P (x, ω)

∂G(x − x0 , ω) ∂P (x0 , ω)
P (x0 , ω) −G(x−x0 , ω) dA(x0 ) = 12 P (x, ω) x ∈ ∂V
∂V ∂n(x0 ) ∂n(x0 ) 
0 x∈/V


Spors | VL-STiASP WS 2017/18 | Sound Field Synthesis | Wave Field Synthesis → Foundations 5-10
Synthesis with Secondary Monopoles and Dipoles

I  S(x, ω)
 x ∈V

∂G(x − x0 , ω) ∂S(x0 , ω) 1
S(x0 , ω) −G(x−x0 , ω) dA(x0 ) = 2 S(x, ω) x ∈ ∂V
∂V ∂n(x0 ) ∂n(x0 ) 
0 x∈/V


secondary monopole sources G(x − x0 , ω)

∂G(x−x ,ω)
secondary dipole sources ∂n(x 0)
0

Spors | VL-STiASP WS 2017/18 | Sound Field Synthesis | Wave Field Synthesis → Foundations 5-11
High-Frequency Approximation


I  S(x, ω)
 x ∈V

∂G(x − x0 , ω) ∂S(x0 , ω) 1
S(x0 , ω) −G(x−x0 , ω) dA(x0 ) = 2 S(x, ω) x ∈ ∂V
∂V ∂n(x0 ) ∂n(x0 ) 
0 x∈/V


Introduce point source as Green’s function (for x ∈ V )

1 + j ωc r
I  
∂S(x0 , ω)
S(x0 , ω) cos φ − G(x − x0 , ω) dA(x0 ) = S(x, ω)
∂V r ∂n(x0 )

Far-field/high-frequency approximation with cos φ = 1, ω

cr 1
I  
ω ∂S(x0 , ω)
j S(x0 , ω) − G(x − x0 , ω) dA(x0 ) ≈ S(x, ω)
∂V c ∂n(x0 )
| {z }
D(x0 ,ω)

sound field outside V does not vanish

Spors | VL-STiASP WS 2017/18 | Sound Field Synthesis | Wave Field Synthesis → Foundations 5-12
Driving Function for the Synthesis of a Plane Wave

I h ω
−j ω nT x i
j · 1 + hnpw , ni · e c pw 0 G(x − x0 ) dA(x0 ) ≈ S(x, ω)
∂V | c {z }
D(x0 ,ω)

Further approximation for WFS (1 + hnpw , ni ≈ 2hnpw , ni)

ω T
(
2j ωc hnpw , nie −j c npw x0 for hnpw , ni > 0
D(x0 , ω) =
0 otherwise
⇒ selection of active secondary sources
Spors | VL-STiASP WS 2017/18 | Sound Field Synthesis | Wave Field Synthesis → Foundations 5-13
Example – 2D Synthesis of a Plane Wave
Circular Secondary Source Distribution, Frequency f = 1000 Hz

2D, R = 1.50 m, αpw = 270o

Spors | VL-STiASP WS 2017/18 | Sound Field Synthesis | Wave Field Synthesis → Foundations 5-14
Example – 2D Synthesis of a Plane Wave
Rectangular Secondary Source Distribution, Frequency f = 1000 Hz

2D, αpw = 315o

Spors | VL-STiASP WS 2017/18 | Sound Field Synthesis | Wave Field Synthesis → Foundations 5-15
Driving Function for the Synthesis of a Point Source

1 + j ωc rs
I  
ω
j + cos(φs ) G(x0 − xs ) G(x − x0 ) dA(x0 ) ≈ S(x, ω)
∂V c rs
| {z }
D(x0 ,ω)

Further approximation for WFS (cos(φs ) ≈ 1, ω

c rs  1)
(
2j ωc cos(φs )G(x0 − xs ) for cos(φs ) > 0
D(x0 , ω) ≈
0 otherwise
⇒ selection of active secondary sources
Spors | VL-STiASP WS 2017/18 | Sound Field Synthesis | Wave Field Synthesis → Foundations 5-16
Example – 2D Synthesis of a Line Source
Frequency f = 1000 Hz

2.D, R = 1.50 m, xls = (0, 2.2) m

Spors | VL-STiASP WS 2017/18 | Sound Field Synthesis | Wave Field Synthesis → Foundations 5-17
2.5-dimensional Synthesis

Secondary point sources are typically used for synthesis in a plane

mismatch of secondary source type (point vs. line source)
21/2-dimensional reproduction

Methods to account for secondary source type mismatch

stationary phase approximation [Berkhout et al., 1993]
modified 2D driving function [Spors et al., 2010]
primary source correction [Völk et al., 2012]

Artifacts
amplitude deviations within the listening area
deviating amplitude decay for virtual point sources
slight spectral changes

Spors | VL-STiASP WS 2017/18 | Sound Field Synthesis | Wave Field Synthesis → Foundations 5-18
Example – 2.5D Synthesis of a Plane Wave
Rectangular Secondary Source Distribution, Frequency f = 1000 Hz

2.5D, R = 1.50 m, αpw = 315o

Spors | VL-STiASP WS 2017/18 | Sound Field Synthesis | Wave Field Synthesis → Foundations 5-19
Spatial Sampling of Secondary Source Distribution
Signal Processing Model of Spatial Sampling

Continuous distribution of secondary sources

Spatially discrete distribution of secondary sources

may result in spatial aliasing and reconstruction errors

analytic treatment only for regular geometries

Spors | VL-STiASP WS 2017/18 | Sound Field Synthesis | Wave Field Synthesis → Sampling 5-20
Spatial Sampling of Secondary Source Distribution
Continuous Case

Driving Function D̊(ν, R) Secondary Source G̊(ν, r )

[dB] [dB]
3 3 0
0
−10
−10
f −> (kHz)

f −> (kHz)
2 2
−20
−20

D
−30 −30
1 1
−40 −40

0 −50 −50
−100 −50 0 50 100 −100 −50 0 50 100
ν −> ν −>
(2.5D, R = 1.50 m, r = 1.50 m, N = 56, αpw = 270o )

broadband plane wave synthesized by WFS

circular secondary source distribution

Spors | VL-STiASP WS 2017/18 | Sound Field Synthesis | Wave Field Synthesis → Sampling 5-21
Spatial Sampling of Secondary Source Distribution
Discrete Case

Driving Function D̊S (ν, R) Secondary Source G̊(ν, r )

[dB] [dB]
3 3 0
0
−10
−10
f −> (kHz)

f −> (kHz)
2 2
−20
−20

D
−30 −30
1 1
−40 −40

0 −50 −50
−100 −50 0 50 100 −100 −50 0 50 100
ν −> ν −>
(2.5D, R = 1.50 m, r = 1.50 m, N = 56, αpw = 270o )

spatial aliasing and reconstruction error

Spors | VL-STiASP WS 2017/18 | Sound Field Synthesis | Wave Field Synthesis → Sampling 5-22
Example – Synthesis of a Plane Wave
Monofrequent – Frequency 1000 Hz
2

1.5

0.5
y/m

−0.5

−1

−1.5

−2
−2 −1 0 1 2
x/m

(2.5D, R = 1.50 m, N = 56, αpw = 270o )

Spors | VL-STiASP WS 2017/18 | Sound Field Synthesis | Wave Field Synthesis → Sampling 5-23
Example – Synthesis of a Plane Wave
Monofrequent – Frequency 2000 Hz
2

1.5

0.5
y/m

−0.5

−1

−1.5

−2
−2 −1 0 1 2
x/m

(2.5D, R = 1.50 m, N = 56, αpw = 270o )

Spors | VL-STiASP WS 2017/18 | Sound Field Synthesis | Wave Field Synthesis → Sampling 5-23
Example – Synthesis of a Plane Wave
Monofrequent – Frequency 4000 Hz
2

1.5

0.5
y/m

−0.5

−1

−1.5

−2
−2 −1 0 1 2
x/m

(2.5D, R = 1.50 m, N = 56, αpw = 270o )

Spors | VL-STiASP WS 2017/18 | Sound Field Synthesis | Wave Field Synthesis → Sampling 5-23
Simulation – Synthesis of a Plane Wave
Broadband
2

1.5

0.5
y/m

−0.5

−1

−1.5

−2
−2 −1 0 1 2
x/m

(2.5D, R = 1.50 m, N = 56, αpw = 270o )

Spors | VL-STiASP WS 2017/18 | Sound Field Synthesis | Wave Field Synthesis → Sampling 5-24
Simulation – Synthesis of a Plane Wave
Broadband
2

1.5

0.5
y/m

−0.5

−1

−1.5

−2
−2 −1 0 1 2
x/m

(2.5D, R = 1.50 m, N = 56, αpw = 270o )

Spors | VL-STiASP WS 2017/18 | Sound Field Synthesis | Wave Field Synthesis → Sampling 5-24
Simulation – Synthesis of a Plane Wave
Broadband
2

1.5

0.5
y/m

−0.5

−1

−1.5

−2
−2 −1 0 1 2
x/m

(2.5D, R = 1.50 m, N = 56, αpw = 270o )

Spors | VL-STiASP WS 2017/18 | Sound Field Synthesis | Wave Field Synthesis → Sampling 5-24
Perception of Synthetic Sound Fields

The perceptual properties of current techniques depend

on the human perception of synthetic sound fields

Relevant psychoacoustic mechanisms

precedence effect, law of the first wave front
summing localization
perception of timbre

Spors | VL-STiASP WS 2017/18 | Sound Field Synthesis | Wave Field Synthesis → Perception 5-25
Experimental Evaluation of Localization and Coloration

Coloration [Wierstorf, Raake, Spors, 2013]

pointing into perceived direction by head orientation (Laserpointer)
periodic white noise pulses 700 ms length, 300 ms pause
12 subjects, 16 different listener positions
binaural resynthesis of circular loudspeaker array (with tracking)

Coloration [Wierstorf 2014]

MUltiple Stimuli with Hidden Reference and Anchor (MUSHRA)
periodic pink noise pulses 700 ms length, 500 ms pause
16 subjects, central listener position
binaural resynthesis of circular loudspeaker array

Spors | VL-STiASP WS 2017/18 | Sound Field Synthesis | Wave Field Synthesis → Perception 5-26
Results – Localization
Synthesis of a virtual plane wave by 56 loudspeakers

mean deviation: loudspeaker distance:

1.2◦ 2 0.17 m
confidence interval:
1.4◦
1
2.3◦
y/ m

40◦
0 0.6◦ 30◦
20◦
0.7◦ 10◦
−1
0◦

−2 −1 0 1 2
x/ m
Spors | VL-STiASP WS 2017/18 | Sound Field Synthesis | Wave Field Synthesis → Perception 5-27
Results – Localization
Synthesis of a virtual plane wave by 28 loudspeakers

mean deviation: loudspeaker distance:

2.5◦ 2 0.34 m
confidence interval:
1.5◦
1
4.4◦
y/ m

40◦
0 1.9◦ 30◦
20◦
1.3◦ 10◦
−1
0◦

−2 −1 0 1 2
x/ m
Spors | VL-STiASP WS 2017/18 | Sound Field Synthesis | Wave Field Synthesis → Perception 5-27
Results – Localization
Synthesis of a virtual plane wave by 14 loudspeakers

mean deviation: loudspeaker distance:

4.3◦ 2 0.67 m
confidence interval:
2.3◦
1
5.8◦
y/ m

40◦
0 4.9◦ 30◦
20◦
2.1◦ 10◦
−1
0◦

−2 −1 0 1 2
x/ m
Spors | VL-STiASP WS 2017/18 | Sound Field Synthesis | Wave Field Synthesis → Perception 5-27
Results – Coloration
Point source – central listener position

1
coloration (mean, CI)

0.5

−0.5

−1
anchor stereo 14 28 56 3584 ref(hidden)

no (+1) / strong (−1) perceived difference with respect to reference

Spors | VL-STiASP WS 2017/18 | Sound Field Synthesis | Wave Field Synthesis → Perception 5-28
Psychoacoustic Properties of WFS

WFS performs wavefront synthesis

underlying perceptual mechanism not clear
precedence effect, law of the first wave front?

Psychoacoustic Properties
reconstruction of first wavefront
⇒ very stable localization throughout the listening area
spatial sampling artifacts after first wavefront
⇒ coloration of the virtual source
truncation of secondary source distribution
⇒ coloration of the virtual source due to diffraction effects
secondary source type mismatch
⇒ incorrect amplitude decay with respect to listener distance

Spors | VL-STiASP WS 2017/18 | Sound Field Synthesis | Wave Field Synthesis → Perception 5-29
 Realization of a Typical Research System
sounds

Sound Field Synthesis

Digital Audio Interface

D/A Converters
DSP, GUI, ...

Amplifiers
N
/
parameters
source

PC

pre-computation and playback of loudspeaker signals

realtime computation of loudspeaker signals

Spors | VL-STiASP WS 2017/18 | Sound Field Synthesis | Wave Field Synthesis → Realization 5-30
Digital Signal Processing

source–loudspeaker angle of
distance incidence

source loud-
signal Filter Delay Weight + speaker

other
other
loudspeakers
sources

Basic Blocks
FIR/IIR filters (SISO, MIMO)
(fractional) delay lines
weighting, summation, ...

Spors | VL-STiASP WS 2017/18 | Sound Field Synthesis | Wave Field Synthesis → Realization 5-31
The SoundScape Renderer (SSR)

software tool for scene-based realtime spatial audio reproduction

several different reproduction methods
Binaural Renderer
Binaural Room Synthesis (BRS)
Wave Field Synthesis (WFS)
NFC-HOA Renderer
Vector Base Amplitude Panning (VBAP)
Ambisonic Amplitude Panning (AAP)
Generic Renderer

Spors | VL-STiASP WS 2017/18 | Sound Field Synthesis | Wave Field Synthesis → Realization 5-32
The SoundScape Renderer (SSR)

Spors | VL-STiASP WS 2017/18 | Sound Field Synthesis | Wave Field Synthesis → Realization 5-32
The SoundScape Renderer (SSR)

Spors | VL-STiASP WS 2017/18 | Sound Field Synthesis | Wave Field Synthesis → Realization 5-32
The SoundScape Renderer (SSR)

runs on Linux and Mac OS X

uses the Jack Audio Connection Kit (JACK)
graphical user interface (Qt) and network interface (TCP/IP)
Free and Open Source Software (GPLv3)
http://spatialaudio.net/ssr

Spors | VL-STiASP WS 2017/18 | Sound Field Synthesis | Wave Field Synthesis → Realization 5-32