Beruflich Dokumente
Kultur Dokumente
ij
f
2
(i , j )/
ij
2
(i , j ) ,
Fig. 0. "olor images 4ith Gaussian noise a!!e! 4ith a
mean of 8 an! a
stan!ar! !eviation of 28 (right. (he signal to noise ratios
of the noisy images are 01.1 2 3a4son.
0alt an# -epper Eoise
2mpulse noise is corruption &ith in#ivi#ual noisy pi)els &hose bri"htness #iffers si"nificantly
from that of the nei"hborhoo# 0alt an# pepper noise is a type of impulse noise &here saturate# impulse
noise affects the ima"e 8i.e. it is corrupte# &ith pure &hite an# blac* pi)els9.
Eoise Mo#els
A##itive Eoise
2n the case of #ata in#epen#ent noise 8i.e. noise &here the amount of noise is not relate# to the
ima"e #ata itself9$ an a##itive noise mo#el is appropriate:
f 8i$ 9 O "8i$ 9 P v8i$ 9
&here "8i$ 9 is the i#eal ima"e$ v8i$ 9 is the noise an# f8i$ 9 is the actual ima"e.
Multiplicative Eoise
2n the case of #ata #epen#ent noise 8i.e. noise &here the amount of noise is relate# to the ima"e
#ata itself9$ a multiplicative noise mo#el is appropriate:
f 8i$ 9 O "8i$ 9 P "8i$ 9 Uv8i$ 9
&here "8i$ 9 is the i#eal ima"e$ v8i$ 9 is the noise an# f8i$ 9 is the actual ima"e.
Fig. ,. "olor images 4ith 189 Salt an! pepper noise
(right. (he signal to
noise ratio of the noisy images are :., 2 3a4son.
/isto"rams
An ima"e histo"ram is an abstraction of an ima"e &here the fre>uency of each ima"e
8bri"htness=intensity9 value is #etermine#.
1, /isto"rams
2n the case of a "rayscale ima"e in &hich there are 25; "rayscale intensities 80$ 2559$ 25;
counts are compute# in#icatin" ho& many pi)els each of the "rayscales in the ima"e have. !he
histo"ram contains "lobal information about the ima"e an# that information is completely in#epen#ent
of the position an# orientation of obects in the scene. 2n some cases$ the histo"ram or information
#erive# from it 8such as the avera"e intensity an# its stan#ar# #eviation9 can be use# to perform
classification. /o&ever$ care must be ta*en as ima"e histo"rams are not uni>ue an# hence many
very #ifferent ima"es may have similar 8or even the same9 histo"ram.
'olor /isto"rams
Another issue that arises is &hat to #o &ith color ima"es. Often histo"rams are #etermine# for
each channel in#epen#ently. !he choice of color mo#el can have a hu"e effect on the usefulness of the
color histo"ram. 1?($ 'MF$ FQV.
/isto"ram=2ma"e 6>ualiDation
Often ima"es may be #ifficult for a human observer to #ecipher #ue to the picture bein"$ for
e)ample$ too #ar*. 2t has been #etermine# that humans can #istin"uish bet&een 300 an# 700 sha#es of
"rey un#er optimal vie&in" con#itions$ althou"h in very #ar* or bri"ht sections of a ima"e the ust
noticeable #ifference re#uces si"nificantly. /o&ever$ it is also clear that it is easier for humans to
#istin"uish lar"er #ifferences$ so if the #istribution of "rayscales in an ima"e is improve#. One
techni>ue for improvin" the #istribution of "rayscales in an ima"e is histo"ram e>ualiDation !his
Fig. -. (he histogram !erive! from a grayscale image 2
3a4son.
techni>ue attempts to #istribute the "rayscales in an ima"e evenly$ so that the resultin" histo"ram is
flat 8i.e. all "rayscales have e)actly the same number of points9.
Eote that &hen e>ualiDin" a color ima"e 8such as that in +i"ure 39 &e "enerally only e>ualiDe
the luminance channel as other&ise the colors can become #istorte#.
(inary 2ma"es.!hreshol#in"
?rayscale ima"es "enerally have 4 bits per pi)el. :hile processin" these ima"es is easier in
some &ays than processin" color ima"es$ there is a simpler form of ima"e$ the binary ima"e$ in &hich
processin" is even more strai"htfor&ar#. 2n fact$ a si"nificant portion of practical applications of
computer vision have been #evelope# usin" binary vision.
A binary ima"e is one in &hich there is only a sin"le bit per pi)el 8e.". blac* or &hite9. !hese
ima"es are create# by threshol#in" &here the threshol#s use# are #etermine# in a variety of &ays.
!hreshol#in"
A binary ima"e is create# from a "rayscale ima"e by threshol#in". !he binary threshol#in"
al"orithm is simply:
+or all pi)els 8i$ 9
f8i$ 9 O 1 &here f 8i$ 9 GO !
O 0 &here f 8i$ 9 J !
Fig. :. ; color image an! its luminance histogram
(left, together 4ith a histogram7e5uali6e! image
an! luminance histogram (right 2 3a4son.
Often "ray level 255 is use# instea# of binary 1 8so that the resultin" ima"e can be represente#
usin" a 4Cbit format an# #isplaye# in the same manner as the ori"inal "rayscale ima"e9.
!he threshol#in" operation is "enerally use# in or#er to separate some obects of interest from
the bac*"roun#. Most typically the obect8s9 of interest are represente# by 1 8or 2559$ but sometimes
the binary ima"e &oul# have to be inverte# for this to be the case.
!hreshol#in" -roblems
!he most important thin" to note about binary ima"in" is that the fore"roun# an# the
bac*"roun# that are bein" separate# nee# to be #istinct in the ima"e bein" threshol#e#. 2f they are not
#istinct then it &ill be #ifficult 8or even impossible9 to accurately se"ment them usin" threshol#in".
/o&ever$ there are a number of techni>ues 8e.". a#aptive threshol#in"9 &hich are use# to try to #eal
&ith situations &here the #istinction bet&een fore"roun# an# bac*"roun# is not clear.
A#aptive !hreshol#in"
2n all of the e)amples sho&n so far$ "lobal threshol#in" has been use# 8i.e. a sin"le threshol#
has been applie# to all points in the ima"e9. 2n some situations it is possible to si"nificantly improve
threshol#in" results by usin" multiple threshol#s. 'onsi#er the threshol#in" in +i" 7$ &here an optimal
"lobal threshol# hi#es most of the &ritten #etails of the #ia"ram. Qsin" a#aptive threshol#in" 8&ith
;4 threshol#s=ima"e bloc*s9 most of the #etails appear correctly.
Fig. /. "orrect threshol!ing (top, too lo4 ($ottom left
an! too high (right 2 3a4son.
!he a#aptive threshol#in" al"orithm is:
1. ,ivi#e the ima"e into subCima"es 8e.". ;4 subCima"es &ere use# an 4V4 "ri#9.
2. +or each of the subCima"es compute a threshol#.
<. +or each point in the ima"e #etermine a threshol# by interpolatin" a threshol# value from the four
nearest threshol#s usin" bilinear interpolation.
Key points
A si"nal is a function &hich #epen#s on some physical variable. !he ima"e as a si"nal can be
2C#imensional or <#. +unctions may be cate"oriDe# as continuous$ #iscrete or #i"ital. A continuous
function has a continuous #omain an# ran"e. ,iscrete C if the #omain is #iscrete$ if the ran"e is also
#iscrete then &e have a #i"ital function. (ri"htness values boun#e# by these limits are "rayClevels. A
continuous ima"e is #i"itiDe# at samplin" points.
!hese samplin" points are or#ere# in the plane an# their "eometric relation is calle# the "ri#. !he
raster is the "ri# on &hich a nei"hborhoo# relation bet&een points is #efine#.
One infinitely small samplin" point in the "ri# correspon#s to one picture element calle# a pi)el. 2n <#
vo)el 8volume element9.
An ima"e histo"ram is an abstraction of an ima"e &here the fre>uency of each ima"e
8bri"htness=intensity9 value is #etermine#.
A binary ima"e is one in &hich there is only a sin"le bit per pi)el.
2ma"e +ormation
1. 'amera Mo#el. Lens !ransformation
2. 'alibration
+i". 1. !he pinhole camera mo#el
!he ima"e is forme# by li"ht rays issue# from the scene facin" the hole. 2f the pinhole &ere
re#uce# to a point$ e)actly one li"ht ray &oul# pass throu"h each point in the plane of the plate 8or
ima"e plane9$ the pinhole$ an# some scene point. 2n reality$ the pinhole &ill have a finite 8albeit small9
siDe$ an# each point in the ima"e plane &ill collect li"ht from a cone of rays. 0o this i#ealiDe# an#
e)tremely simple mo#el of the ima"in" "eometry &ill not strictly apply. 8(runelleschi$ be". @V cent.9
Most cameras use either '', 8char"eCcouple# #evice9 or 'MO0 8complementary metal o)i#e
semicon#uctor9 photosensitive elements both usin" photovoltaic principles. !hey capture bri"htness of
a selecte# color channel or a monochromatic ima"e.
2n a '', sensor$ every pi)elMs char"e is transferre# throu"h ust one output no#e to be
converte# to volta"e$ buffere#$ an# sent offCchip as an analo" si"nal. !he basic sensor element inclu#es
a 0chott*y photo#io#e.
'MO0 technolo"y starte# bein" technolo"ically mastere# from about the 1770s. 2n a 'MO0
sensor$ each pi)el has its o&n char"eCtoCvolta"e conversion$ an# the sensor often inclu#es amplifiers$
noiseCcorrection$ an# #i"itiDation circuits$ so that the chip outputs bits.
!he a#vanta"e of 'MO0 cameras compare# to '', is a hi"her ran"e of sense# intensities 84
times9$ hi"h spee# of rea#Cout 8about 100 ns9 an# ran#om access to in#ivi#ual pi)els. !he #isa#vanta"e
is a hi"her level of noise.
+i". 2. !ra#itional color cameras
Lenses
Most real cameras are e>uippe# &ith lenses. !here are t&o reasons for this. !he first reason is
to "ather li"ht. !he secon# reason is *eepin" the picture in sharp focus &hile "atherin" li"ht from a
lar"e area. 2"norin" #iffraction$ interferences$ an# other physical optics phenomena$ the behavior of
lenses is #ictate# by the la&s of "eometric optics$ see +i".<:
819 li"ht travels in strai"ht lines 8li"ht rays9 in homo"eneous me#ia%
829 &hen a ray is reflecte# from a surface$ this ray$ its reflection$ an# the surface normal are coplanar$
an# the an"les bet&een the normal an# the t&o rays are complementary% an# 8<9 &hen a ray passes
from one me#ium to another$ it is refracte#$ i.e. its #irection chan"es.
Accor#in" to 0nellMs la& the an"les S 1 an# S2 bet&een the normal an# the t&o rays are relate#
by
n
1
sin W
1
O n
2
sin W
2
.
!racin" li"ht rays as they travel throu"h a lens is simpler &hen the an"les bet&een these rays
an# the refractin" surfaces of the lens are assume# to be small$ &hich is the #omain of para)ial 8or
firstCor#er9 "eometric optics$ an# 0nellMs la& becomes n1 S1 S n2 S2. Let us also assume that the lens is
rotationally symmetric about a strai"ht line$ calle# its optical a)is$ an# that all refractive surfaces are
spherical.
Fig. 1. Reflection an! refraction at the interface
$et4een t4o homogeneous me!ia 4ith
in!ices of refraction n1 an! n2 .
Fig. 0. ; thin lens. Rays passing through # are not
refracte!. Rays parallel to the optical a)is are focuse!
on the focal point F<.
'onsi#er a point - locate# at 8ne"ative9 #epth N off the optical a)is$ an# #enote by 8- O9 the
ray passin" throu"h this point an# the center O of the lens 8+i". 49. 2t easily follo&s from the para)ial
form of 0nellMs la& that 8- O9 is not refracte#$ an# that all the other rays passin" throu"h - are focuse#
by the thin lens on the point p &ith #epth D alon" 8- O9 such that$ so calle# the thin lens e>uation$
1=D C 1=N O 1=f$
f O 1 = H28nC19I is the focal len"th of the lens.
!he t&o points + an# + locate# at #istance f from the lens center on the optical a)is are calle#
the focal points of the lens. 2n practice$ obects &ithin some ran"e of #istances 8calle# #epth of fiel# or
#epth of focus9 &ill be in acceptable focus. As sho&n in the problems at the en# of the lecture$ the
#epth of fiel# increases &ith the fCnumber of the lens$ e.". the ratio bet&een the focal len"th of the lens
an# its #iameter.
+i". 5. !he fiel# of vie& of a camera.
Eote that the fiel# of vie& of a camera$ e.". the portion of scene space that actually proects
onto the retina of the camera$ is not #efine# by the focal len"th alone but also #epen#s on the effective
area of the retina 8e.". the area of the sensor in a #i"ital camera9.
A more realistic mo#el of simple optical systems is the thic* lens. !he e>uations #escribin" its
behavior are easily #erive# from the para)ial refraction e>uation$ an# they are the same as thin lens
proection e>uations$ e)cept for an offset.
+i". ;. A simple thic* lens &ith t&o spherical surfaces.
0imple lenses suffer from a number of aberrations. !o un#erstan# &hy$ let us remember that
the para)ial refraction mo#el is only an appro)imation$ vali# &hen the an"le S bet&een each ray alon"
the optical path an# the optical a)is of the len"th is small an# sin .
+i". 3. 0pherical aberration.
!he "rey re"ion is the para)ial Done &here the rays issue# from - intersect at its para)ial ima"e
p. 2f an ima"e plane S &ere erecte# in p$ the ima"e of p in that plane &oul# form a circle of confusion
of #iameter e. !he focus plane yiel#in" the circle of least confusion is in#icate# by a #ashe# line. !he
siDe of that circle &ill chan"e if &e move S alon" the optical a)is. !he circle &ith minimum #iameter is
calle# the circle of least confusion$ an# its center #oes not coinci#e 8in "eneral9 &ith p.
'amera 'alibration
(esi#es spherical aberration$ there is a chromatic aberration: !he in#e) of refraction of a
transparent me#ium #epen#s on the &avelen"th 8or color9 of the inci#ent li"ht rays.
!here are other types C coma$ asti"matism$ fiel# curvature$ an# #istortion.
,istortion chan"es the shape of the ima"e as a &hole. !his effect is #ue to the fact that #ifferent areas
of a lens have sli"htly #ifferent focal len"ths.
Let us estimate the intrinsic an# e)trinsic parameters of a camera from the *no&n ima"e
positions of scene in some fi)e# coor#inate system. 2n this conte)t$ camera calibration can be mo#ele#
as an optimiDation process$ &here the #iscrepancy bet&een the observe# ima"e features an# their
theoretical positions is minimiDe# &ith respect to the cameraMs intrinsic an# e)trinsic parameters.
2n this e)ample$ the calibration ri" is forme# by three "ri#s #ra&n in ortho"onal planes. Other
patterns coul# be use# as &ell an# they may involve lines or other "eometric fi"ures. A precise
#efinition of the camera calibration proce#ure is beyon# the scope of our course.
2n a linear approach to calibration &e #ecompose the calibration process into 819 the
computation of the perspective proection matri) M associate# &ith the camera$ follo&e# by 829 the
estimation of the intrinsic an# e)trinsic parameters of the camera.
!here is no an e)act solution an# an appro)imate one has to be foun# as the solution of a leastC
s>uares minimiDation problem.
A Eonlinear Approach
A nonlinear approach to camera calibration ta*es into account all the relevant constraints. :e
assume once a"ain that &e observe n fi#ucial points -i 8i O 1$ . . . $ n9 &hose positions in some fi)e#
coor#inate system are *no&n$ an# minimiDe the meanCs>uare# #istance bet&een the measure#
positions of their ima"es an# those pre#icte# by the perspective proection e>uation &ith respect to a
vector of camera parameters S$ &hich mi"ht inclu#e various #istortion coefficients in a##ition to the
usual intrinsic an# e)trinsic parameters.