Sie sind auf Seite 1von 97

# STFT (revisited)

## • Time/Frequency localization depends on window size.

• Once you choose a particular window size, it will be the same
for all frequencies.
• Many signals require a more flexible approach - vary the
window size to determine more accurately either time or
frequency.
The Wavelet Transform

## • Overcomes the preset resolution problem of the STFT by

using a variable length window:

## – Narrower windows are more appropriate at high frequencies

(better time localization)

## – Wider windows are more appropriate at low frequencies

(better frequency localization)
The Wavelet Transform (cont’d)

## Wide windows do not provide good localization

at high frequencies.
The Wavelet Transform (cont’d)

## A narrower window is more appropriate at high frequencies.

The Wavelet Transform (cont’d)

## Narrow windows do not provide good localization

at low frequencies.
The Wavelet Transform (cont’d)

## A wider window is more appropriate at low frequencies.

What is a wavelet?
• It is a function that “waves” above and below the x-axis;
it has (1) varying frequency, (2) limited duration, and (3)
an average value of zero.

## • This is in contrast to sinusoids, used by FT, which have

infinite energy.
Sinusoid Wavelet
Wavelets
• Like sine and cosine functions in FT, wavelets can define
basis functions ψk(t):

f (t )   ak k (t )
k

## • Span of ψk(t): vector space S containing all functions f(t)

that can be represented by ψk(t).
Wavelets (cont’d)

## Haar Morlet Daubechies

Basis Construction - Mother Wavelet

  jk (t )

Basis Construction - Mother Wavelet
(cont’d)

 jk (t )  2 j / 2  2 j t  k 

scale =1/2j j
(1/frequency)

k
Continuous Wavelet Transform (CWT)

## translation parameter, scale parameter

measure of time (measure of frequency)

1  t  
Forward C ( , s)   f  t 

 dt
CWT: s t  s 

Mother wavelet
normalization (window)
constant
scale =1/2j=1/frequency
CWT: Main Steps

## 1. Take a wavelet and compare it to a section at the start

of the original signal.

## 2. Calculate a number, C, that represents how closely

correlated the wavelet is with this section of the
signal. The higher C is, the more the similarity.
CWT: Main Steps (cont’d)

3. Shift the wavelet to the right and repeat steps 1 and 2 until
you've covered the whole signal.
CWT: Main Steps (cont’d)

## 5. Repeat steps 1 through 4 for all scales.

Coefficients of CTW Transform
• Wavelet analysis produces a time-scale view of the input
signal or image.

1  t  
C ( , s )   f  t    dt
s t  s 
Continuous Wavelet Transform (cont’d)

• Inverse CWT:

1 t 
f (t ) 
s
 s C ( , s) ( s )d ds

## note the double integral!

FT vs WT
weighted by F(u)

weighted by C(τ,s)
Properties of Wavelets

## • Simultaneous localization in time and scale

- The location of the wavelet allows to explicitly represent
the location of events in time.
- The shape of the wavelet allows to represent different
detail or resolution.
Properties of Wavelets (cont’d)

## • Sparsity: for functions typically found in practice, many of the

coefficients in a wavelet representation are either zero or very
small.

1 t 
f (t ) 
s  s C ( , s) ( s )d ds
• Adaptability: Can represent functions with discontinuities or
corners more efficiently.
• Linear-time complexity: many wavelet transformations can be
accomplished in O(N) time.
Discrete Wavelet Transform (DWT)

t

## f (t )   a jk jk (t ) (inverse DWT)

k j

where  jk (t )  2 j/2
  t k
2 j
DFT vs DWT

## • FT expansion: one parameter basis

or f (t )   a l l (t )
l

## • WT expansion two parameter basis

f (t )   a jk jk (t )
k j
Multiresolution Representation using  jk (t )

f (t )

fine
details
wider, large translations

f (t )   a  jk (t )
jk
k j j

coarse
details
Multiresolution Representation using  jk (t )

f (t )

fine
details

f (t )   a  jk (t )
jk
k j j

coarse
details
Multiresolution Representation using  jk (t )

f (t )

fine
details
narrower, small translations

f (t )   a  jk (t )
jk
k j j

coarse
details
Multiresolution Representation using  jk (t )

f (t ) high resolution
(more details)

fˆ1 (t )
j
fˆ2 (t )

low resolution
fˆs (t ) (less details)

f (t )   a  jk (t )
jk
k j
Prediction Residual Pyramid - Revisited
• In the absence of quantization errors, the approximation
pyramid can be reconstructed from the prediction residual
pyramid.

## • Prediction residual pyramid can be represented more

efficiently.

(with sub-sampling)
Efficient Representation Using “Details”

details D3

details D2

details D1
L0
(without sub-sampling)
Efficient Representation Using Details (cont’d)

representation: L 0 D 1 D2 D3
in general: L0 D1 D2 D3…DJ
A wavelet representation of a function consists of
(1)a coarse overall approximation
(2)detail coefficients that influence the function at various scales.
Reconstruction (synthesis)
H3=H2+D3

H2=H1+D2
details D3

details D2 H1=L0+D1

details D1
L0
(without sub-sampling)
Example - Haar Wavelets

of 4 pixels:
[9 7 3 5]

## • The Haar wavelet transform is the following:

L 0 D 1 D2 D3
(with sub-sampling)
Example - Haar Wavelets (cont’d)
• Start by averaging the pixels together (pairwise) to get
a new lower resolution image:
[9 7 3 5]

## • To recover the original four pixels from the two

averaged pixels, store some detail coefficients.

1
Example - Haar Wavelets (cont’d)
• Repeating this process on the averages (i.e., low
resolution image) gives the full decomposition:

Harr decomposition:
Example - Haar Wavelets (cont’d)

## • We can reconstruct the original image by adding or

subtracting the detail coefficients from the lower-
resolution versions.
2 1 -1
Example - Haar Wavelets (cont’d)

Detail coefficients
become smaller and
smaller as j increases.

Dj

Dj-1 How to
compute Di ?

D1
L0
Multiresolution Conditions
• If a set of functions can be represented by a weighted
sum of ψ(2jt - k), then a larger set,
set including the
original, can be represented by a weighted sum of
ψ(2j+1t - k):
high
resolution

low
resolution
Multiresolution Conditions (cont’d)
• If a set of functions can be represented by a weighted
sum of ψ(2jt - k), then a larger set, including the
original, can be represented by a weighted sum of
ψ(2j+1t - k):

k

## Vj+1: span of ψ(2j+1t - k): f j 1 (t )   bk ( j 1) k (t )

k

V j  V j 1
Nested Spaces Vj
Basis functions: Vj : space spanned by ψ(2jt - k)
ψ(t - k) V0 f (t )   a  jk (t )
f(t) ϵ Vj k j
jk

ψ(2t - k) V1

ψ(2jt - k) Vj

## Multiresolution conditions  nested spanned spaces: V j  V j 1

i.e., if f(t) ϵ V j then f(t) ϵ V j+1
How to compute Di ? (cont’d)

f(t) ϵ Vj f (t )   a  jk (t )
jk
k j

## IDEA: define a set of basis

functions that span the
difference between Vj+1 and
Vj
Orthogonal Complement Wj
• Let Wj be the orthogonal complement of Vj in Vj+1

Vj+1 = Vj + Wj
How to compute Di ? (cont’d)
If f(t) ϵ Vj+1, then f(t) can be represented using basis
functions φ(t) from Vj+1:

Vj+1 f (t )   ck (2 j 1 t  k )
k

## Alternatively, f(t) can be represented using two sets of basis functions,

φ(t) from Vj and ψ(t) from Wj:
Vj+1 = Vj + Wj

f (t )   ck (2 j t  k )   d jk (2 j t  k )
k k
How to compute Di ? (cont’d)

## Think of Wj as a means to represent the parts of a function

in Vj+1 that cannot be represented in Vj

f (t )   ck (2 j 1 t  k )
k

## f (t )   ck (2 j t  k )   d jk (2 j t  k ) differences

k k between
Vj and Vj+1
Vj Wj
How to compute Di ? (cont’d)
• Vj+1 = Vj + Wj  using recursion on Vj:

Vj+1 = Vj-1+Wj-1+Wj = …= V0 + W0 + W1 + W2 + … + Wj

## if f(t) ϵ Vj+1 , then:

f (t )   ck (t  k )   d jk (2 j t  k )
k k j

## V0 W0, W1, W2, …

basis functions basis functions
Summary: wavelet expansion (Section 7.2)
• Wavelet decompositions involve a pair of waveforms
(mother wavelets):
encode low encode details or
resolution info φ(t) ψ(t) high resolution info

f (t )   ck (t  k )   d jk (2 j t  k )
k k j

1D Haar Wavelets

φ(t) ψ(t)

## computes average computes details

1D Haar Wavelets (cont’d)

## • Think of a one-pixel image as a function that is constant

over [0,1)

width: 1
Example:
0 1
1D Haar Wavelets (cont’d)

## • V1 represents the space of all two-pixel images

• Think of a two-pixel image as a function having 21
equal-sized constant pieces over the interval [0, 1).

## Examples: 0 ½ 1 width: 1/2

• Note that V0  V1

e.g., = +
1D Haar Wavelets (cont’d)
• V j represents all the 2j-pixel images
• Functions having 2j equal-sized constant pieces over
interval [0,1).

## Examples: width: 1/2j

• Note that V j 1  V j

e.g., ϵ Vj ϵ Vj
1D Haar Wavelets (cont’d)

## V0, V1, ..., V j are nested

i.e., V j  V j 1

VJ fine details

V1
V0 coarse details
Define a basis for Vj

1
0 1

## note alternative notation:  i ( x )   ji ( x )

j
Define a basis for Vj (cont’d)
Define a basis for Wj (cont’d)
• Mother wavelet function:

1
-1

0 1/2 1

## note alternative notation:  i ( x )   ji ( x )

j
Define a basis for Wj

Note
that the dot
product
between basis
functions in Vj
and Wj is zero!
Define a basis for Wj (cont’d)

Basis functions ψ ji of W j
form a basis in V j+1
Basis functions φ ji of V j
Define a basis for Wj (cont’d)

V3 = V2 + W2
Define a basis for Wj (cont’d)

V2 = V1 + W1
Define a basis for Wj (cont’d)

V1 = V0 + W0
Example - Revisited

f(x)=

V2
Example (cont’d)

f(x)=
φ2,0(x)

V2 φ2,1(x)

φ2,2(x)

φ2,3(x)
Example (cont’d)

## (divide by 2 for normalization)

V1and W1 φ1,0(x)
V2=V1+W1
φ1,1(x)

ψ1,0(x)

ψ1,1(x)
Example (cont’d)
Example (cont’d)
(divide by 2 for normalization)
V0 ,W0 and W1
φ0,0(x)
V2=V1+W1=V0+W0+W1
ψ0,0(x)

ψ1,0 (x)

ψ1,1(x)
Example
Example (cont’d)
Filter banks
• The lower resolution coefficients can be calculated
from the higher resolution coefficients by a tree-
structured algorithm (i.e., filter bank).

Subband
encoding
(analysis)

## h0(-n) is a lowpass filter and h1(-n) is a highpass filter

Example (revisited)

[9 7 3 5]
low-pass, high-pass,
down-sampling down-sampling

## (9+7)/2 (3+5)/2 (9-7)/2 (3-5)/2 V1 basis functions

Filter banks (cont’d)
Next level:
Example (revisited)

[9 7 3 5]

low-pass, high-pass,
down-sampling down-sampling

V1 basis functions

(8+4)/2 (8-4)/2
Convention for illustrating
1D Haar wavelet decomposition
x x x x x x … x x
average
detail

re-arrange:

V1 basis functions
re-arrange:
Examples of lowpass/highpass
(analysis) filters

h0
Haar
h1

h0

Daubechies
h1
Filter banks (cont’d)
• The higher resolution coefficients can be calculated from
the lower resolution coefficients using a similar structure.

Subband
encoding
(synthesis)
Filter banks (cont’d)
Next level:
Examples of lowpass/highpass
(synthesis) filters

g0
Haar (same as
for analysis):
g1

g0
Daubechies
+ g1
2D Haar Wavelet Transform

## • The 2D Haar wavelet decomposition can be computed

using 1D Haar wavelet decompositions.
– i.e., 2D Haar wavelet basis is separable

## • Two decompositions (i.e., correspond to different

basis functions):
– Standard decomposition
– Non-standard decomposition
Standard Haar wavelet decomposition

• Steps

## (1) Compute 1D Haar wavelet decomposition of each row of

the original pixel values.

## (2) Compute 1D Haar wavelet decomposition of each

column of the row-transformed pixels.
Standard Haar wavelet decomposition
(cont’d)
average
(1) row-wise Haar decomposition: detail

re-arrange terms
xxx … x … …
xxx … x …
… … . … … .
… … .
xxx ... x …
Standard Haar wavelet decomposition
(cont’d)
average
(1) row-wise Haar decomposition: detail

row-transformed result

… …

… … . …
… … .

Standard Haar wavelet decomposition
(cont’d)
average
detail
(2) column-wise Haar decomposition:

## row-transformed result column-transformed result

… …
… …

… … . … … .
… …
Example

row-transformed result

re-arrange terms …

… … .

… … .
Example (cont’d)

column-transformed result

… … .

2D Haar basis for standard decomposition
To construct the standard 2D Haar wavelet basis, consider
all possible outer products of 1D basis functions.

φ0,0(x)

Example:
ψ0,0(x)
V2=V0+W0+W1

ψ1,0(x)

ψ1,1(x)
2D Haar basis for standard decomposition
To construct the standard 2D Haar wavelet basis, consider
all possible outer products of 1D basis functions.

## φ00(x), φ00(x) ψ00(x), φ00(x) ψ10(x), φ00(x)

i j ( x)   ji ( x)  i j ( x)   ji ( x)
2D Haar basis of standard decomposition

i j ( x)   ji ( x)
V2

 i j ( x)   ji ( x)
Non-standard Haar wavelet decomposition

## (1) Perform one level decomposition in each row (i.e., one

step of horizontal pairwise averaging and differencing).

## (2) Perform one level decomposition in each column from

step 1 (i.e., one step of vertical pairwise averaging and
differencing).

## (3) Repeat the process on the quadrant containing averages

only (i.e., in both directions).
Non-standard Haar wavelet decomposition
(cont’d)

## one level, horizontal one level, vertical

Haar decomposition: Haar decomposition:
xxx … x … …
xxx … x … …
… … . … … .
… … .
xxx ... x … …
Non-standard Haar wavelet decomposition
(cont’d)
re-arrange terms

… one level, horizontal one level, vertical
Haar decomposition Haar decomposition
… … .

… … .

Example

re-arrange terms

… … .

Example (cont’d)

… … .

2D Haar basis for non-standard decomposition

## • Defined through 2D scaling and wavelet functions:

 ( x, y )   ( x) ( y ) 000 ( x, y )   ( x, y )

 ( x, y )   ( x) ( y )  kfj ( x, y )  2 j  (2 j x  k , 2 j y  f )
 ( x, y )   ( x) ( y ) kfj ( x, y )  2 j (2 j x  k , 2 j y  f )
 ( x, y )   ( x) ( y )  kfj ( x, y )  2 j (2 j x  k , 2 j y  f )
2D Haar basis for non-standard decomposition
(cont’d)

i j ( x)   ji ( x)
V2
 i j ( x)   ji ( x)
2D Haar basis for non-standard decomposition
(cont’d)
• Three sets of detail coefficients (i.e., subband encoding)

## 000 ( x, y )   ( x, y ) LL: average LL

Detail coefficients

## LH: intensity variations along

 ( x, y )  2  (2 x  k , 2 y  f )
j
kf
j j j
columns (horizontal edges)

## kfj ( x, y )  2 j (2 j x  k , 2 j y  f ) HL: intensity variations along

rows (vertical edges)
 kfj ( x, y )  2 j (2 j x  k , 2 j y  f ) HH: intensity variations along
diagonals
2D DWT using filter banks (analysis)
LL LH

HL HH
2D DWT using filter banks (analysis)

LL
H LH
V HL
LL LH LL
D HH
HL HH

## The wavelet transform is applied again on the LL sub-image

2D DWT using filter banks (analysis)
2D Inverse DWT using filter banks
(synthesis)

H LH
V HL
D HH
Wavelets Applications
• Noise filtering
• Image compression
– Fingerprint compression
• Image fusion
• Recognition
G. Bebis, A. Gyaourova, S. Singh, and I. Pavlidis, "Face Recognition by
Fusing Thermal Infrared and Visible Imagery", Image and Vision
Computing, vol. 24, no. 7, pp. 727-742, 2006.
• Image matching and retrieval
Charles E. Jacobs Adam Finkelstein David H. Salesin, "Fast
Multiresolution Image Querying", SIGRAPH, 1995.
Fast Multiresolution Image Querying
painted low resolution target

queries

## Charles E. Jacobs Adam Finkelstein David H. Salesin, "Fast

Multiresolution Image Querying", SIGRAPH, 1995.