REAL-TIME OBJECT DETECTION AND TRACKING

By: Vanya V. Valindria Hammad Naeem Rui Hua

Outline
Introduction Hardware in RT Object Detetion & Tracking Methods
Traditional Methods: Absolute Differences Census Method Feature Based Method Modern Methods:
KLT Meanshift

Result and Conclusion

Introduction
Definition: Object detection detect a particular object in an image
Object tracking to track an object (or multiple objects) over a sequence of images

Application: Traffic Information

http://www.youtube.com/watch?v=vA35sXbn7zs

Application: Surveillance

http://www.youtube.com/watch?v=o25fClk9cdg

Application: Mobile Robot

http://www.youtube.com/watch?v=Q4zycRGJFFs

Problems??
Temporal variation/dynamic environment Abrupt object or camera motion Multi-camera? Multi-objects? Computational expensive

Hardware in Real-time Tracking

MEMORY
Important Tracking system encountering limited memory problems.

FRAME RATE
~30 FPS

PROCESSORS - DSP
Allow saturated arithmetic operation Powerful operation ability Can do several memory accesses in a single instruction

Object Detection and Tracking

In a video sequence an object is said to be in motion, if it is changing its location with respect to its background The motion tracking is actually the process of keeping tracks of that moving object in video sequence i.e. position of moving object at certain time etc.

Flow Chart
Idle Image acquisition Object Detection Image acquisition Object tracking

Object Lost?

No

Y es

Method 1: Absolute Differences

= Image subtraction
D(t)=I(ti) I(tj) Gives an image frame with changed and unchanged regions

Ideal Case for no motion: I(ti) = I(tj), D(t)=0

Moving objects are detected

Results:

Frame1

Frame10

Difference of Two Frames

Absolute Difference
Methods for Motion Detection Frame Differencing Background Subtraction Draw Backs: involves a lot of computations Not feasible for DSP implementation

Method 2: Census Transforms

124 124 74 64 32 18
If (Center pixel < Neighbor pixel) Neighbor pixel = 1

1 1 1

1 x 1

0 0 1

157

116

84

Signature Vector Generation

1 1

1 x 1

0 0 1

11011101

Signature Vector

Image
128 26 96 76 125 243 87 43 236 125 Signatur vector generation for all pixels

128 129 235 229 209 228 251 229 221 234 227 221 35 58 98

Signature Vectors 10110101 00101011 . . . 10111010

List Generation

10110101 00101011 . . . 10111010

List population

Generated List

Signature vector matching

Census Transform:
Advantages: Compare only two values 0 or 1. Similar Illumination Variation for pixel and neighbouring pixels Draw Backs: As we only deal with only 0`s and 1`s, this method is sensitive to noise. Calculate, store and match process computationally Expensive

Method 3: Morphology Based Object Tracking

Background estimation

Frame differencing

Object Registration

Morphology Based Object Tracking

Image Differencing Background Thresholding
Estimation

Object Registration

Contours are registered Width, height and histogram are recorded for each contour

Feature Vector

Each object represented by a feature vector (the length, width, area and histogram of the object)

Segmented object

Tracked object

Morphology Based Techniques

Advantages: Can Track Multiple objects Objects are registered based on their anatomy Helpful for Object Merging Draw Backs: Object registration complex and slow process For multiple object registration per frame more complex

Method 4: Lucas-Kanade Technique

Visual motion pattern of objects and surface in a scene by Optical Flow

Frame 1

Frame 2

Method 5: Mean shift

An algorithm that iteratively shifts a data point to the average of data points in its neighborhood
Choose a search window size in the initial location Compute the MEAN location in the search window

Repeat until convergence

Center the search window at the mean

Intuitive Description

Region of interest Center of mass

Mean Shift vector

Objective : Find the densest region Distribution of identical balls

Intuitive Description

Region of interest Center of mass

Mean Shift vector

Objective : Find the densest region Distribution of identical balls

Intuitive Description
Region of interest Center of mass

Mean Shift vector

Objective : Find the densest region Distribution of identical balls

Intuitive Description
Region of interest Center of mass

Mean Shift vector

Objective : Find the densest region Distribution of identical balls

Intuitive Description
Region of interest Center of mass

Mean Shift vector

Objective : Find the densest region Distribution of identical balls

Intuitive Description
Region of interest Center of mass

Mean Shift vector

Objective : Find the densest region Distribution of identical balls

Intuitive Description
Region of interest Center of mass

Objective : Find the densest region Distribution of identical balls

Process

CAMSHIFT --Continously Adaptive Meanshift

Modified to adapt dynamically to the colour probability distributions More real time
For each frame-> MEAN-SHIFT is applied with several iteration

Store the location of the mean and calculate new window size for next frame

New development
Combine with different features. SIFT features, colour feature & texture information Camshift algorithm combined with the Kalman filter.

Result
Algorithm Absolute Differencing Census Transform Morphological Tracking Kanade Lucas

Arithmetic and Logic operations

4230100 2416000 352210 500825

Time taken by Algorithm

16 5. 4 14.2 0.486

Comparison
Easy to implement
Absolute Differences Allows continuous tracking Computationally expensive Slow and low accuracy Computationally expensive Complex if Multiple objects per frame Slow Large Memory consumption

Census Transform

Immune to noise and Illumination changes

Feature Based

Can track multiple objects well

Comparison
High accuracy
KLT Less execution time Robust to noise and dynamic scene Large memory

MeanShift & CAMShift

Computationally less expensive

Ineffective if there is heavy occlusion

Conclusion
KLT algorithm has the best performance with higher accuracy and less computation time It requires combination of methods to achieve the appropriate object detection and tracking according to the proposed scenario

References
S. Shah, T. Khattak, M. Farooq, Y. Khawaja, A. Bais, A. Anees, and M. Khan, Real Time Object Tracking in a Video Sequence Using a Fixed Point DSP, Advances in Visual Computing, pp. 879 888. K. Huang, L. Wang, T. Tan, and S. Maybank, A real-time object detecting and tracking system for outdoor night surveillance, Pattern Recognition, vol. 41, no. 1, pp. 432444, 2008. J. Li, F. Li, and M. Zhang, A Real-time Detecting and Tracking Method for Moving Objects Based on Color Video, in 2009 Sixth International Conference on Computer Graphics, Imaging and Visualization. IEEE, 2009, pp. 317322. W. Junqiu and Y. Yagi, Integrating color and shapetexture features for adaptive real-time object tracking, IEEE Trans on Image Processing, vol. 17, no. 2, pp. 235240, 2008. Q. Wang and Z. Gao, Study on a Real-Time Image Object Tracking System, in Computer Science and Computational Technology, 2008. ISCSCT08. International Symposium on, vol. 2, 2008. Y. Meng, Agent-based reconfigurable architecture for real-time object tracking, Journal of Real-Time Image Processing, vol. 4, no. 4, pp. 339351, 2009. [Y. Yao, C. Chen, A. Koschan, and M. Abidi, Adaptive online camera coordination for multicamera multi-target surveillance, Computer Vision and Image Understanding, 2010.