Beruflich Dokumente
Kultur Dokumente
FollowThatFace!
d e s c r i b e di n l a s t m o n t h ' sa r t i c l e T
. o do rithmin moredetail
Lastmonth'sarticlein this t h a t , h o w e v e ry, o u ' d n e e dt o d e c i d ei f C a m s h i f tw a s o r i g i n a l l yd e v e l o p e d
f o r h a n d s - f r egea m i n g .l t ' s d e s i g n e dt o
seriesexplained how to tt hh ee fsaacme yeofua dc ee .t el fc tt eh dei ndeeat ec hcftroarmf i nedi ss be veryfast and
"lightweight"
so the
implement and configurem o r e t h a n o n e f a c e i n a f r a m e ,y o u ' d computer can do other tasks while
n e e d t o d e c i d ew h i c h d e t e c t i o ni s t h e t r a c k i n g .S i n c ei t w a s d e v e l o p e da s a
Thismonth, o n e y o u ' r e t r a c k i n g . F i n a l l y ,i f a
facedetection. g a m i n gi n t e r f a c eC t l s oh a sa n
, a m s h i fa
( l i m i t e d )a b i l i t y t o d e t e c t c h a n g e si n
l'll showyou howto use p e r s o n ' s h e a d t i l t s t o wp raorfdi lse o n e h e a d p o s i t i o n s, u c ha s t i l t i n gt h e h e a d
s h o u l d e ro, r t u r n st o w a r d s view,
OpenCV to trackataceonce t h e f r o n t a lf a c ed e t e c t o rw i l l n o l o n g e r t o o n e s i d e .C o u l dy o u u s e t h a t a b i l i t y
you'vedetectedit. r i o t c r t i t c n v n r r ' dn e p dt o h a n d l et h a t
Jv t o c o m m u n i c a t ew i t h y o u r r o b o t ?
tvu
"Come
s i t u a t i o na, s w e l l . M a y b et w o f a s t h e a dt i l t s m e a n
Fortunately,OpenCV includes h e r e ,r o b o t ! "
s p e c i a l i z e dc o d e f o r t r a c k i n g a f a c e F i g u r e 1 s h o w s O p e n C V ' sf a c e
FaceTrackingin e f f i c i e n t l y ,u s i n g c o n t i n u i t y b e t w e e n t r a c k e ri n a c t i o n- f o l l o w i n ga f a c e a s
OpenCV f r a m e st o h e l p f i n d t h e b e s t m a t c hf o r i t t i l t s t o o n e s i d ea n d d u r i n g a t u r n t o
t h e f a c ei t ' sf o l l o w i n g . p r o fi l e .
T r a c k i n ga f a c e i s m o r e d r f f i c u l t T h e a l g o r i t h mt h a t O p e n C Vu s e s
t h a n t r a c k i n ga s t r o n g l y - c o l o r eodb j e c t . for face tracking is called Camshift TheCamshiftDemo
S k i nr e f l e c t st h e a m b i e n tl i g h ti n s u b t l e , C a m s h i f tu s e s c o l o r i n f o r m a t i o n ,b u t
c h a n g i n g w a y s a s a p e r s o n ' sh e a d T h e O p e n C V s a m p l e sd i r e c t o r Y
r a t h e rt h a n r e l y i n go n a s i n g l ec o l o r ,i t
t u r n so r t i l t s . c o n t a i n sa p r o g r a m c a l l e d c a m s h r f t -
t r a c k sa c o m b i n a t i o no f c o l o r s .S i n c ei t
I n p r i n c i p l ey, o u c o u l dt r a c ka f a c e tracks by color, it can follow a face d e m o .Y o u c a n g e t s o m eg o o d h a n d s -
b y l o c a t i n gi t o v e r a n d o v e r I n e v e r y on experience
t h r o u g h o r i e n t a t i o nc h a n g e st h a t t h e a n d a n i n t u i t i v ef e e l f o r
frame, using the Haar detector H a a rd e t e c t o rc a n ' th a n d l eT the Camshifa
. h es i d e b a r , t l g o r i t h mw i t h t h i s d e m o
"How O p e n C V ' s p r o g r a m .H e r ea r e t h e s t e p sf o r d o i n g
FaceTracker
Works," t h a t :
e x p l a i n st h i s a l g o -
1 ) P l u gi n a w e b c a m .
F I G U R E1 . O p e n C V ' s
face tracker in action, 2 ) L a u n c ht h e p r o g r a mc a l l e dc a m s h i f t -
It's able to follow a d e m o i n t h e s a m p l e sd i r e c t o r y .
face as it tilts to one
s i d e a n d d u r i n ga t u r n ;
to profile. 3 ) U s ey o u r m o u s et o s e l e c a t rectangle
c e n t e r e dt i g h t l yo n y o u r f a c e .
4 ) C l i c k i n t h e v i d e o - d i s p l aw
yindow
and type the letter b. (The display
s h o u l dc h a n g et o l o o k s o m e t h i n gl i k e
t h e v i e w i n F i g u r e2 . )
36 srnvoo3.eoo7
t.t
- : e n C V ' sf a c et r a c k e ru s e sa n a l g o r i t h mc a l l e dC a m s h i f t , FIGUREA. Two examplesof
' - ' : c o n s i s tos f f o u r s t e P s : th€ color histogram that
Camshiftusesto represenla
.z-:e a color histogramto representthe face. face,
in OPenCV.
, :- ate a "faceprobability"for each pixelin the incoming backprojection"
'-':'::
Ihere's a built-in method
t h a t i m p l e m e n t si t , c a l l e d
^-.^^f ^n^^r,- ^l^^
- 'i the location .:_-_!!L^'
of the face rectanglein eachvid e o f r a m e . c v ( d . c d o c K P r o l e c .
..
Figure C shows the
- : : , l a t e t h e s i z ea n d a n g l e , f a c e - p r o b a b i l i t yi m a g e i n
o n e v i d e o f r a m ea s C a m s h i f t
'l : 'row each stePworks: tracks my face. Blackpixels
have the lowest probabilitY
white,the high-
.'eate a histogram.Camshift representsthe face it's v a l u g a n d
^ S ( a l s oc a l l e da b a r c h a r to) f c o l o rv a l u e s . est. Gray pixels lie somewherein the middle.
-. a sa h i s t o g r a m
'a A shows two example histogramsproduced by the
- =^ rft demo programthat shipswith OpenCV.The heightof 3) Shift to a new location. With each new video frame,
"shifts"its estimateof the face location,keepingit
-- colored bar indicateshow many pixels in an image Camshift
: . ldve that
"hue." Hue is one of three valuesdescribing centeredover the area with the highestconcentrationof
, e s c o l o r i n t h e H S V( H u e ,S a t u r a t i o nV,a l u e c) o l o r m o d e l . bright pixels in the face-probabilityimage.lt finds this new
' -ore on color and color models, see "Ihe World of f,ra"utlon OVttarting at the previow to.u-tionand computing
:' SERVO Magazine,November'05,) the cenler of gravityof the face-probabilityvalueswithin a
- the imageregionrepresentedby the top histogram,a rectangle.lt then shifts the rectangleso it's right over the
^ h u e i s m o s tc o m m o n ,a n d a s l i g h t l ym o r e l a v e n d e h r u e center of gravity. lt does this a few times to center the
:-e next most common, The bottom histogram shows a rectangle well. The OpenCV f unction cvcars rif - r )
location.
:a in which the most common hue is the rightmostbin. implementsthe stepsfor shiftingto the new
This process of shifting the rectangle to correspond with
. rue is almos! but not quite, red, "Mean
the center of gravityis based on an algorithmcalled
-atculate face probability - simpler than it sounds! fhe S h i f t , " b y D o r i n C o m a n i c i u ,I n f a c t , C a m s h i f ts t a n d s f o r
" C o n t i n u o u sAl yd a p t i v eM e a nS h i f t . "
::gram is created only once, at the start of tracking.
"face-probability" value to
:e'wards, it's used to assigna
-- rrrl6g€ pixel in the video frames that follow' 4) Calcutate size and angte. The OpenCV method is
"ContinuouslyAdaptive" and not just "Mean Shift"
Faceprobability"soundsterribly complicatedand heav- called
* athematical, but it's neither!Here'show it works.FigureB because it also adjusts the size and angle of the face
- ^'s the barsfrom a histogramstackedone atop the other. rectangleeach time it shifts it, lt does this by selectingthe
':e' stacking t h e m ,i t ' sc l e a rt h a t t h € r i g h t m o sbt a r a c c o u n t s s c a l e a n d o r i e n t a t i o n F I G U R E C. The normal and face-
- :oout 457oof the pixels in the region.That meansthe that are the best fit to probabilityviewsas Camshift tracks
:caoility that a pixel seleded randomlyfrom this region t h e f a c e - p r o b a b i l i t y m y f a c e . I n t h e f a c e - p r o b a b i l i t y
"faceprob- pixels inside the n€w view, black pixelshave the lowest
: - d fall into the rightmostbin is 45%.That'sthe
'- i,' for a pixelwith this hue.The samereasoningindicates r;."ii."
iu.ii"si" il:#2i1,"f"'f,lf,fil;f.',?
;?];?
:: tf'e face probabilitYfor
'z rext histogram b i nt o t h e
:-: ls about 2O7o,since it
-:rutts for about 207oof
stack's total
- :: s all there isto it..
As new video frames
' .e, the hue value for
':. pixel is determined.
'': r that,the facehistogram
, ..;sed to assign a face
-h€ight.
%
: c a b i l i t yt o t h e p i x e l .T h i s
"histogram
:cess is called
-:JRE B. To see what
..e probability" means/
:grre stacking t h e b a r si n
: histogram on€ atoP
'-e other. The probability
. . ; o c i a t e dw i t h e a c hc o l o ri s
' z p e r c e n lt h a t c o l o r b a r
: : rtributes to the total
-:ght of this stack.
3V
SERVOO3.2OO1
FIGURE3. The main not onlymy face,but halfthe roomas
program listing for well!Thereasonfor the oversized face
1- //// Constants detectinga face in a detectionis clearlyvisiblein the face-
2 const char * DISPLAY_WINDOUI= "Di-splayWindow"; live video stream,then
3 #def ine OPniCV_R@I "C: /Proqrarn Files/@enCv/1.0" tracking it using the probabilityview. Backgroundpixels
A Camshift wrapperAPl. with a nearlyneutralshadecontributed
5 //// cLoi:p.l variables too muchnoisewhen vrnindfld smin
5 Ipllmage * pvideoFramecopy = 0;
gray,or black).Color wereat theirdefaultvalues.
1
B void main( inc argc, char** argTv ) canbe computed for The middle and right views in
g { pixels that are Figure2 showthe effectof increasing
10 CvRect. * pFaceRect = 0; almost neutral,but first snj-n,then rzmin.In the right-hand
11 if( linitAllO ) exitprogram(-1); theircolorvaluesare view, noisy pixelshave been largely
L2
13 // Capture and display video frames until a face
unstable,and these eliminated,but the face region still
L4 // is detected pixels contribute producesa strongsignal.Trackingis
15 while( I ) noisethat interferes now quitegood,and the ellipseis well
1 6 { with tracking. oositioned.
L] / / Ir,ok for a face in the next video frame
fB capEureVideoFrameO;
Camshift uses
L9 pFaceRect = detectFace(pvideoFramecopy); two parameters Ihe SimpleCamshift
smin ood vmin - to
20
2I // Showthe display image screenout thisnoise.
Wrapper
22 cvSlor./Tfiage( DISPI,AY-WINffi,i. pVideoFrarreCoEl ) ;
23 if( (char)27==cvlvaitKey(1)) exitprogram(0);
These parameters OoenCVincludessourcecodefor
24 definethresholds for camshiftdemo, but it's not easy to
25 // *it loop when a face is detected ignoringpixelsthat adapt, since it combinesuser-input
26 if(pFaceRect) break; are too close to handlersand view togglingwith the
21 )
28
neutral.rrminsetsthe stepsfor facetracking.
"almost
29 // inLtLalize tracking threshold for lf you're programmingin C++,
30 startTracking(pVideoFrameCogl, pFaceRect); black,"and smi-nfor ratherthan in C, you could use the
31 "almostgray."
These CvCamShiftTracker class,defined in
32 / / Track the detected face using Camshift
33 while( I )
two thresholdlevels cvaux.hpp. Again,however,this class
3 4 { will need to be is fairly complex, with many I
35 CvBolD faceBox; adjusted for your interfaces, and is onlyavailableto C++ J
38 sEnvoo3.9oo7
FIGURE 4. ThehelperfunctionsinirAll ( )
and exitProgramO handle program
initializationand cleanup. 1 t-nt initAll o
2 t
3 if{ linicCapture{} ) return 0;
Combinins Face
-and 4
5
if ( !initFaceDet (OPn{CV_R@T
" /daLa/haarcascades/haarcascade_fronLaI f ace-def aulc . xn] " ) )
Detection Tracking 6 return 0;
7
In camshiftdemo, you neededto 8 // Sfarr'rn mpqq^da fplls rqpr how ro l-eoin and how tO exit
manuallyinitializetrackingwith the 9 pfintf ( \\ni*******************************************\n"
!
"To
mouse.For a roboticsapplication,it 10 exit, click inside the video display,\n"
1t- "then press
wouldbe muchnicerto initializetrack- the ESC key\n\n"
\ L2 "Press <mJIER>to begin"
'l ing automatically,usinga face detec- 13 \\\n********************************************\n/ ),
tion that the Haardetectorreturned. I4 fdof./cl-din\.
(Seelastmonth'sarticlefor detailson 15
implementing facedetection.) l6 // CrcarP fhe disnlaw windOw
L] cvNamedwindow( DISPLAY_WINmW, I );
Thissectionshowshow to do that t-8
usingthe Camshiftwrapperdescribed L9 / / IntLialtze t.racker
above.The programdescribedhere 20 n:nfr rrol/i donE r^ma / I '
9 closeFaceDet ( ) ;
1) Detecta face.
10 -^t ^-^^m-^^t,^-
lElgoJg1!aLAYr
/ \ -
\ / i
)t e s 2) Startthe tracker. 11
3) Trackthe face. L2 exit (code) ;
L3 ]
B if( lpVideoFrameCopy )
Q n\/i danFr^maf-^n/ - .\raro:f FTm:ao /errCci-Qi zc ln\/i dcotrr:mel R ? \ .
tTGURE 5. The helper function J Pr Luuvl r\r,'LLwJ uY! \! vvvev+g
SERVO
03.e00739
detectFace ( )
. OpenCVon Sourceforge
http :// sourcef orge.na/ Proleds/
opencvllbrary
a,= .,
. Official OpenCVusergrouP
hllp: / / lech.groups.yahoo.com/
group/Open€V
ComingUp
Showthe punk with the volcono S o f a r ,t h e f a c e sw e ' v eb e e nf i n d i n g
a n d f o l l o w i n gh a v e b e e n a n o n y m o u s
40 sEnvoo3.eoo7