Lect2 PDF

1
ImProc. Digital Image Processing

Lecture 1 (draft)
Basic Tools in Image Processing
7 Oct. 2019
https://my.eurecom.fr/jcms/p0_2027226/en/improc
jean-luc.dugelay@eurecom.fr
Typical Sequence
• Pre-processing
– Noise reduction: low-pass Filtering
• Processing
– Gradient maps + Threshold
– Gradient based descriptor
• Post-processing
– Edge thinning, closing
– Outlier removal
• Representation & Description
– Hough Transform
jld@eurecom.fr 1
3
Spatial Masks
 w1 w2 w3 
 
( x −1, y −1) ( x −1, y) ( x −1, y +1) T [ f (x, y)] = w1 f ( x −1, y −1) + w2 f ( x −1, y)
 
 w4 w5 w6  + w3 f ( x −1, y +1) + w4 f ( x, y −1)
 
 ( x, y −1) ( x, y) ( x, y + 1)  + w5 f ( x, y) + w6 f ( x, y + 1) + w7 f ( x + 1, y −1)
 
 w7 w8 w9  + w8 f ( x + 1, y) + w9 f ( x + 1, y +1)
 
 
( x + 1, y −1) ( x + 1, y) ( x + 1, y + 1)
w1,...,w9: mask coefficents • size

of the template
8-neighbors of (x,y)
• values of coefficients
Direct Low pass filtering

by averaging
1 1
1 1 1
4 4 1
1 1 1 1 1
  9 
averaging 2x2 4 4 1 1 1 averaging 3x3
Image smoothing k − 1 k − 1
+ noise reduction ∑ ∑
p = 0 q = 0
h (p , q ) = 1
- edge smoothing
jld@eurecom.fr 2
5
Low pass filtering by averaging

3x3
5x5 7x7
Median Filtering
Algorithm
1. Classify S = { f ( xj , yj ), ( xj , yj ) ∈W }
2. f ' ( xi, yi ) = med ( S )
Non linear,
med (α .I1 + β .I 2 ) ≠ α .med ( I1 ) + β .med ( I 2 )
Useful in preserving edges while reducing noise (image smoothing)
jld@eurecom.fr 3
7
Edges : Gradient & Laplacian
f ( x, y )
 ∂f 
 
∇f ( x, y ) =  ∂∂x 
 f
 ∂y 
 
2
 ∂f   ∂f 
2
∇f ( x, y ) =   +  
 ∂x   ∂y 
∂ 2 f ( x, y ) ∂ 2 f ( x, y )
∇ 2 f ( x, y ) = +
∂x 2 ∂y 2
Image Sharpening
Gradient Pattern
• Gradient 0 0 0  0 1 0
Gx  1 0 − 1 G y 0 0 0
   
0 0 0 0 − 1 0
1 0 − 1  1 0 − 1 if ∇f ( x, y ) > Threshold at ( x0 , y0 )
1 0 − 1  2 0 − 2 then ( x0 , y 0 ) is an edge point
    else ( x0 , y0 ) is not an edge point
1 0 − 1  1 0 − 1
Prewitt Sobel
jld@eurecom.fr 4
9
Gradient Pattern
 10 10 10 10 
10
 10 10 10 10 
10

0 0 0 0
0
   10 10 10 10 10 
−10 −10 −10 −10
−10
 10
−10 10 10 6 2 
−10 −10 −10 −10 
8 4 0 −4 −8 
  10 10 10 10 0
0°  −2 −6 −10 −10 −10 10 10 10 0 −10
−10 −10 −10 −10 −10 
10 10 0 −10 −10
 
22.5°
10 0 −10 −10 −10
 0 −10 −10 −10 −10
45°
10
Questions
1 1 1 1 2 1
1
1
1 1 1  2 4 2
9  16
1 1 1 1 2 1
If we apply it two times? 2D filtering or 2 times 1D?
jld@eurecom.fr 5
11
Gradient based descriptor: SURF, FAST, DAISY, SIFT, etc.

Local binary Pattern (LBP)
p −1
LBP = ∑ s ( g p − g c ) 2 p
p =0
 1 if x ≥ 0
s ( x) = 
0, otherwise
χ 2 (S , M ) = ∑
n
(Si − M i )2
Widely used to compare two face images i Si + M i
12
Laplacian Pattern
• Laplacian
 0 +1 0 
+ 1 - 4 + 1
 
 0 + 1 0 
Edges: zero-crossing points
∇ 2 f [(x,y )] ≈ f(x + 1,y) + f(x − 1,y) + f(x,y + 1) + f(x,y − 1) − 4f(x,y)
∂f f ( x + a, y ) − f ( x − b, y )
( x, y ) ≅
∂x a+b
jld@eurecom.fr 6
13
Basic question on Filtering
• What are the differences between averaging

and median filters (e.g. of size 3x3) in terms
of implementation and impact?
14
Hough Transform
Originally designed for Line detection
but can also be used for any analytical curve (circle, ellipse, etc.)
IMAGE J HOUGH H
Image Space defined by {Mi (xi,yi)}

A set of pixels defines a curve described by a parametric equation,
f(a1,a2,..., an, x, y) = 0
Ex.
Line {y = a.x + b or ρ = x.cos(θ) + y. sin(θ)}: 2 parameters (a,b) or (ρ, θ)
Circle {(x - a)2 + (y - b)2 = c2}: 3 parameters (center (a,b) and radius c, or (a, w, R)}
jld@eurecom.fr 7
15
Hough Transform (Cont.)
A point in H corresponds to a straight line in I

A point in I corresponds to a sinusoid in H
Points on the same line in I give curves passing through a common point in H
Points on the same curve in H give lines passing through a common point in I
16

Several kind of transformations:
• From m to 1
Any “m-uplets” from the image space is associated to a parametric curve {ai} in the Hough space;
• From 1 to n
Any pixel (xi,yi) from the image space is associated to m parametric curves {ai} in the Hough space;
• From m to m’
Any “m-uplets” from the image space is associated to m’ parametric curves {ai} in the Hough space;
jld@eurecom.fr 8
17

Line detection : from m to 1
• m=2
Mi(xi,yi) and Mj(xj,yj)
x i y j − x j yi x j − xi
ρk = θl = − Arctg
( y j − yi )2 + (x j − x i )2 y j − yi
18

Line detection : from 1 to m
Algo.
A- Partition H into cells ACC(ρk, θl) initialized to 0;

B- For each pixel Mi(xi,yi) in I,
do ACC(ρk, θl) ← ACC(ρk, θl) +1 if f(ρk,θl,xi,yi) ≈ 0 is verified
C- Hence, for a given cell, that is to say ACC(ρm, θn),
its value is incremented as many times as a pixel is on the straight
line (ρm, θn) ;
D- If ACC(ρm, θn) > Threshold in H, then a straight line (ρm, θn) in I is
detected.
jld@eurecom.fr 9
19
Quad tree
Bounding box
20
Quad tree
root
A0,1,1
A0,0 A0,1 A0,2 A0,3
A0,0
A0,1,2,3
A0,1,1 A0,1,2
A0,3 A0,2
A0,1,2,3
jld@eurecom.fr 10
21
Split & Merge segmentation
• Split:
Split the picture into smaller and smaller areas until reaching
a given uniformity criterion;
• Merge:
Merge neighboring areas according to a similarity criterion
(mean, variance, etc.);
• Split & Merge:
Combination of both previous approaches
22
1 1 1 1 1 1 1 1 Split
1 1 1 1 1 1 1 2
1 1 1 1 1 2 2 2
1 1 1 1 2 2 2 2
1 1 1 1 1 2 2 2
1 1 1 1 1 2 2 2
2 2 2 2 2 2 2 2
2 2 2 2 2 2 2 2
∑ (g ( x ) − m ( R l ) ) < Threshold
1 2
TEST : E ( R l ) = i _1
card [ R l ] xi∈ Rl
jld@eurecom.fr 11
23
1.0 1.0 1.3 Split

1.8 2.0
1.0 1.0 1.9
2.0 2.0
∑ (g ( x ) − m ( R l ) ) < Threshold
1 2
TEST : E ( R l ) = i _1
24
0 0 0 0 0 0 .3 .3 Split
0 0 0 0 0 0 .3 .7
0 0 0 0 .8 .2 0 0
0 0 0 0 .2 .2 0 0
0 0 0 0 .9 .1 .1 .1
0 0 0 0 .9 .1 .1 .1
0 0 0 0 .1 .1 .1 .1
0 0 0 0 .1 .1 .1 .1
∑ (g ( x ) − m ( R l ) ) < Threshold
1 2
TEST : E ( R l ) = i _1
jld@eurecom.fr 12
25
Merge
Region adjacency graph
1.0 1.0 1.3 0.9
0.9
1.43
2.6 1.9
1.8 2.0 0.9
0
0.9 2.6
1.0 1.0 1.9 0 1.8
4.2
2 2
2.0 2.0
1.8
0
∑ (g(x ) − µ(R , R ))
2
E(R l , R k ) = i l k
x i ∈R l ∪ R k
1
µ(R l , R k ) = ∑ g(x i )
card{R l ∪ R k } x i ∈R l ∪R k
26
Merge
µ=1.0 1.3
0
2.6 1.9
1.8 2.0 0.9
0
0 2.6
1.0 1.0 1.9 0 1.8
4.2
2 2
2.0 2.0
1.8
0
∑ (g(x ) − µ(R , R ))
2
E(R l , R k ) = i l k
x i ∈R l ∪ R k
1
µ(R l , R k ) = ∑ g(x i )
jld@eurecom.fr 13
27
L
SEQ(L) = ∑ ∑ (g(x ) − µ(R ))
2
> Threshold _ 2
l =1 x i ∈R l
i l
Merge
µ=1..03
15.9
µ=1.91
∑ (g(x ) − µ(R , R ))
2
E(R l , R k ) = i l k
x i ∈R l ∪ R k
1
µ(R l , R k ) = ∑ g(x i )
28
Math. Morpho.: Intro.
union
intersection
complementary
inclusion
difference
jld@eurecom.fr 14
29
Math. Morpho.: Basic operations
• Object A
• Structuring element Bp
• EROSION { }
er (A, Bp ) ≡ p B p ⊂ A
• DILATATION {
dil(A, Bp ) ≡ p B p ∩ A ≠ ∅ }
• OPENING dil(er(A,Bp),Bp)
• CLOSING er(dil(A,Bp),Bp)
30
Math. Morpho.: Some properties

(Cont.)
jld@eurecom.fr 15
31
Math. Morpho.: Intro.(Cont.)

isthmus island cape
lake
gulf
32
Math. Morpho.: Some properties

• Translation invariant
• Erosion & dilatation are not inverses of each other
Duality: dil(A, B p ) = [er (A c , B p )]
c
• Increasing: X ⊂ X’ => T(X) ⊂ T(X’)

• ...
Opening: cuts narrow isthmus;

removes small islands and narrow capes.
Closing: fills narrow canals;
removes small lakes and narrow gulfs.
It is more severe to erode prior to dilate than the contrary.
jld@eurecom.fr 16
Erosion
Original image Eroded image
Erosion
Eroded once Eroded twice
jld@eurecom.fr 17
Opening and Closing
OPENING: The
original image
eroded twice and
dilated twice
(opened). Most
noise is removed
CLOSING: The
original image
dilated and then
eroded. Most
holes are filled.
36
Optical Flow, or apparent motion field

relative motion between an observer (an eye or a camera) and the scene
Motion vector pattern

resulting from camera panning and zooming
Camera panning direction
zoom out zoom in Camera panning and tilting
jld@eurecom.fr 18
37
http://extra.cmis.csiro.au/IA/changs/motion/
38
Apparent motion
Optical flow
between two consecutive image frames taken at t and t'(t +δt)
STEP 1. Local Estimation:
displacement: di (dx ,dy ) for each pixel: pi (x i , yi )

i i
d x i ≡ x 'i − x i

d yi ≡ y'i − y i
jld@eurecom.fr 19
39
Apparent motion (Cont.)
{z _ f , pan x , pan y }:
STEP 2. Global Interpretation  x'   x   pan x 
∀i, i  = z _ f . i  +  

y
 i'  y i   pan y 
parameter name interpretation

zf zoom factor IF z_f is closed to 0
THEN no zoom
ELSE
zf>> 1 => backward zoom
zf<< 1 => forward zoom
tx vertical pan mean vertical displacement
in pixels of the whole image
ty horizontal pan mean horizontal displacement
in pixels of the whole image
40
Estimation of the Optical Flow

Hypothesis: the luminance of a pixel is constant over time.
I ( x, y, t ) = I ( x + δx, y + δy, t + δt )
Assuming that δx and δy are smalls
I(x,y,t) with Taylor series can be developed to get:
∂I ∂I ∂I
I ( x + δx, y + δy, t + δt ) = I ( x, y, t ) + δx + δy + δt + H .O.T .
∂x ∂y ∂t
∂I ∂I ∂I ∂I ∂I ∂I
δx + δy + δt = 0 Vx + Vy = −
∂x ∂y ∂t ∂x ∂y ∂t
• Technique based on a relation between spatial and temporal gradients

(i.e. derivatives of the image at (x,y,t)
• 1 equation – 2 unknowns
• Compute Vx and Vy via an iterative process:
V (i ) = V ( i −1) ( p , t ) − ε .DFD ( p , t , V (i −1) )∇ I ( x − V x( i −1) , y − V y( i −1) , t − 1)

jld@eurecom.fr 20
d (i) = d (i−1) (p, t) − ε .sign {DFD (p, t, d (i−1) )}.sign {∇ I ( x − dx(i−1) , y − dy(i−1) , t − 1)}
41
Optical flow: basic illustration

(iteration #1)
Pixel #6 at time t (from which position at t-1 does this pixel come from ?)
Assumption: d(0) = 0
Gradients
Temporal I(6,t) - I(6,t-1) x x x x x x x x x x
t-1
Spatial I(7,t-1) - I(5,t-1) t

x x x x x x x x x x
position 1 2 3 4 5 6 7 8 9 10
Gain:ε = 1
Temporal > 0
Spatial > 0
d (i) = d (i−1) (p, t) − ε .sign {DFD (p, t, d (i−1) )}.sign {∇ I ( x − dx(i−1) , y − dy(i−1) , t − 1)}
42
Optical flow: basic illustration (Cont.)

(iteration #2)
Pixel #6
d(1) = d(0) - 1 = -1
Gradients
Temporal I(6,t) - I(7,t-1) x x x x x x x x x x
t-1
Spatial I(8,t-1) - I(6,t-1) t

x x x x x x x x x x
position 1 2 3 4 5 6 7 8 9 10
jld@eurecom.fr 21

Lect2 PDF

Hochgeladen von

Dokumentinformationen

Originaltitel

Copyright

Verfügbare Formate

Dieses Dokument teilen

Dokument teilen oder einbetten

Freigabeoptionen

Stufen Sie dieses Dokument als nützlich ein?

Sind diese Inhalte unangemessen?

Copyright:

Verfügbare Formate

Lect2 PDF

Hochgeladen von

Copyright:

Verfügbare Formate

1

ImProc. Digital Image Processing

w1,...,w9: mask coefficents • size

Direct Low pass filtering

Low pass filtering by averaging

Useful in preserving edges while reducing noise (image smoothing)

Edges : Gradient & Laplacian

If we apply it two times? 2D filtering or 2 times 1D?

Gradient based descriptor: SURF, FAST, DAISY, SIFT, etc.

∇ 2 f [(x,y )] ≈ f(x + 1,y) + f(x − 1,y) + f(x,y + 1) + f(x,y − 1) − 4f(x,y)

Basic question on Filtering

• What are the differences between averaging

Image Space defined by {Mi (xi,yi)}

Hough Transform (Cont.)

A point in H corresponds to a straight line in I

Hough Transform (Cont.)

Hough Transform (Cont.)

Hough Transform (Cont.)

A- Partition H into cells ACC(ρk, θl) initialized to 0;

Split & Merge segmentation

1.0 1.0 1.3 Split

1.0 1.0 1.9

Math. Morpho.: Intro.

Math. Morpho.: Basic operations

Math. Morpho.: Some properties

Math. Morpho.: Intro.(Cont.)

Math. Morpho.: Some properties

• Increasing: X ⊂ X’ => T(X) ⊂ T(X’)

Opening: cuts narrow isthmus;

It is more severe to erode prior to dilate than the contrary.

Original image Eroded image

Eroded once Eroded twice

Optical Flow, or apparent motion field

Motion vector pattern

Camera panning direction

zoom out zoom in Camera panning and tilting

STEP 1. Local Estimation:

displacement: di (dx ,dy ) for each pixel: pi (x i , yi )

Apparent motion (Cont.)

parameter name interpretation

Estimation of the Optical Flow

• Technique based on a relation between spatial and temporal gradients

V (i ) = V ( i −1) ( p , t ) − ε .DFD ( p , t , V (i −1) )∇ I ( x − V x( i −1) , y − V y( i −1) , t − 1)

Optical flow: basic illustration

Spatial I(7,t-1) - I(5,t-1) t

Optical flow: basic illustration (Cont.)

Spatial I(8,t-1) - I(6,t-1) t

Das könnte Ihnen auch gefallen