Computer Vision and Video Processing Using MATLAB

, Senior Application Engineer

Agenda

Introduction Video and Image Processing Blockset Demo: optical flow Demo: stereo vision Questions

Example Computer Vision and Video Processing Tasks

Detect and track objects Count objects in a scene Stabilize camera motion Deinterlace video frames Create mosaicks and panoramas Generate depth maps from stereo image pairs

Common Challenges

Accessing and analyzing video data Exploring algorithms and what-if scenarios Understanding the system-wide context Rebuilding standard algorithms Visualizing intermediate results Testing and validating under real-world conditions

44

Video and Image Processing Blockset

Design and simulate video and image processing systems
Multimedia file I/O Video display Text and graphic overlays Pre- and post-processing Motion-based processing Object detection and tracking Feature recognition Computer vision

Demo: Video Processing in MATLAB

Use optical flow on a video sequence

66

System Objects

Definition

MATLAB objects that represent time-based and data-driven algorithms, sources, and sinks
hReader = video.MultimediaFileReader(viptraffic.avi) step(hReader);

Instantiate and configure

Execute within a loop

Other methods available

reset isDone close

For related information, see the following webinar:

Object-Oriented Programming in MATLAB
77

Batch Processing
Load the entire video file and process it all at once

MATLAB

Memory
Batch Video Algorithm

Source

88

Stream Processing
Load a video frame and process it before moving on to the next frame

MATLAB

Memory

Stream Source

Stream Processing

99

Stream Processing in MATLAB is Hard

myVid = mmreader(myvideofile.avi); numFrames = myVid.NumberOfFrames; currentFrame = read(myVid,1); numIter = 10; opticalFlowOutput = zeros([size(currentFrame) numFrames]); i = 1;

while i <= numFrames prevFrame = currentFrame; currentFrame = read(myVid,i);

Explicit state management

flow = opticalFlow(currentFrame,prevFrame,horn-schunck,... numIter,magitude-squared); opticalFlowOutput(:,:,2:end) = opticalFlowOutput(:,:,1:end-1); opticalFlowOutput(:,:,1) = flow; i = i+1; end implay(opticalFlowOutput)

Explicit indexing Need to maintain buffer

10

System Objects Make it Easier

reader = videoMultimediaFileReader reader.Filename = myvideofile.avi; viewer = video.VideoPlayer optical = video.OpticalFlow optical.Method = horn-schunck; optical.OutputValue = Magitude-squared; optical.ReferenceFrameDelay = 3; optical.MaximumIterationCount = 10;

Initialize objects

while ~isDone(reader) currentFrame = step(reader); OF = step(optical, currentFrame); step(viewer, OF); end

In-the-loop code is much simpler Implicit states, buffering, and indexing Video player runs in-the-loop

11

Demo: Video Processing in MATLAB

Use optical flow to detect and counting moving vehicles on a road

12 12

Video and Image Processing System Objects

AlphaBlender Autocorrelator2D Autothresholder BinaryFileReader BinaryFileWriter BlobAnalysis BlockMatcher BoundaryTracer ChromaResampler ColorSpaceConverter Deinterlacer DemosaicInterpolator DeployableVideoPlayer EdgeDetector FFT2D GammaCorrector GeometricRotator GeometricScaler GeometricTransformer IFFT2D ImageComplementer MorphologicalErode MorphologicalOpen ImageDataTypeConverter MorphologicalTopHat ImageFilter ImagePadder LocalMaximaFinder MarkerInserter Maximum Mean MultimediaFileReader MultimediaFileWriter OpticalFlow PSNR Pyramid ShapeInserter StandardDeviation TemplateMatcher TextInserter Variance VideoPlayer

GeometricTransformEstimator Median MedianFilter2D MedianFilter2D Minimum MorphologicalBottomHat MorphologicalClose MorphologicalDilate

ConnectedComponentLabeler GeometricTranslator ContrastAdjuster Convolver2D CornerDetector Crosscorrelator2D DCT2D Histogram2D HistogramEqualizer HoughLines HoughTransform IDCT2D

13

Computer Vision Example 1: Mosaicking and Stabilization

Register neighboring frames to create a large view of the scene or stabilize camera motion Applications: Security License plate recognition Aerial surveying Medical imaging

14 14

Computer Vision Example 2: Object Detection and Tracking

Detect, classify, and track objects in a scene Applications: Traffic monitoring Cell counting Lane departure warning system

15 15

Computer Vision Example 3: Stereo Vision

Extract 3D information from a pair of stereo images

Applications: Obstacle avoidance Automotive safety Face recognition Scene reconstruction Part pickers

16 16

Stereo Vision Basics

Left

Right

17 17

Stereo Vision Algorithm

Stereo rectification Align images horizontally Block matching Find disparity map Backprojection Calculate 3D points

18 18

Video and Image Processing Blockset Advanced Capabilities

Graphical design in Simulink Fixed point modeling Code generation Targeting and verification
DSPs FPGAs

For more information, see the following webinar:

Image and Video Processing with DSPs and FPGAs

19

Image Acquisition Toolbox

Live video and image acquisition directly into MATLAB and Simulink Device property configuration Live video previewing GUI or functional interface Support for multiple hardware vendors

20

Why Use MATLAB for Computer Vision and Video Processing?

System objects simplify coding and reduce errors Algorithms for computer vision and video processing Flexible environment enables algorithm exploration Support for reading and writing many video file formats Text and graphics annotations on video data Support for cameras and frame grabbers Embedded hardware design considerations

21

For More Information

mathworks.com/products/viprocessing/demos.html

Other examples:
People tracking

Video mosaicking Video stabilization Lane departure warning system Abandoned object detection

22

For More Information

Experiment with product by downloading a trial http://www.mathworks.com/products/viprocessing

Related demos and webinars
More detailed product information Links to product documentation

User stories

Contact us

Sales representative

23

Questions?

24