3D MODELLING OF OBJECTS FOR BETTER ROBOTIC VISION

Project Guide : Asst. Prof. Shabin P. Submitted by,

Jansher K.M Kiran Surendran A Midhul Manoj Kumar Mithun Binny Mohd. Jasheer T.P

Overview

One of the method of path planning of automated robots is image processing 2D images make path prediction almost impossible 2D to 3D conversion helps path prediction Matlab programming is used for 2D to 3D conversion

INTRODUCTION

Visual perception is the ability to interpret information and surroundings from the effects of visible light reaching the eye Robots uses cameras for vision which gives 2D image Visual perception of robots cannot be done using 2D images; need of computer vision

Contd

2D positions converted to 3D depth through stereo matching Stereo matching is the process of taking two or more images and estimating a 3D model of the scene by finding matching pixels in the images and converting their 2D positions into 3D depths The depth information is obtained as a 3D model of the scene

OBJECTIVE

Produce a 3D model of a scene

Using

Stereo images from a single camera Pixel wise disparity estimation Applying it to corresponding equation to get the depth information

DESIGN
Image 1 Image 2 Disparity estimation

Camera parameters

Z co-ordinate generation

3D reconstruction

3D image

DIFFERENT BLOCKS 1. Obtaining stereo image pair 2. Disparity estimation 3. Camera parameters 3.1 Intrinsic parameters 3.2 Extrinsic parameters 4. z- coordinate generation 5. 3D - reconstruction 6. 3D - image

1. Obtaining stereo image pair

closely resemble images Images taken at a time with same camera parameter or Single camera kept at 2 position in same horizontal plane but differing by small angle variation

To MATLAB via IMAQ tool

2.

Disparity estimation

Convert Image to Grey Scale

Pixel wise Comparison

Relative disparity plotting

Disparity Map

Disparity estimation is the process to locate corresponding points in the images Gray scale digital image is an image in which the value of each pixel is a single sample, that is, it carries only intensity information. Gray scale consists of only 1 channel, which helps in better analysis

3.

Camera parameters

3.1

Intrinsic parameters

Intrinsic matrix

Five intrinsic parameters

Focal length Image format Principle point Scale factor Skew coefficient

3.2

Extrinsic parameters

R,Tare theextrinsic parameters

3D world coordinates camera coordinates

3D

Defines the position of camera center & cameras heading in world coordinates

4.

z- coordinate generation

Camera intrinsic matrix

fo c a l_ le n g th _ x s k e w _ x K = 0 0 0

ca m e ra _ c e n te r_ x; 1

fo ca l_ le n g th _ y ca m e ra _ c e n te r_ y;

 Disparity measurement of the stereo depth as: Zworld = f (1+B/d) B f d baseline distance between the two cameras focal length of the camera. disparity : difference in image location of an object seen by left and right images

5.

3D - reconstruction

Parallel stereo camera geometry

Every physical point M yields a pair of 2D projections m1 and m2 on two images

Depth of a point M can be simply calculated by

baseline distance between the two cameras focal length of the camera.

disparity : difference in image location of an object seen by left and right images
d

 Depth is inversely proportional to disparity As the distance between cameras and an object increases, depth errors, caused by disparity errors, are increasingly amplified since the depth is inversely proportional to the disparity. As a result, the accuracy of depth reconstruction depends on the distance of the object as well as the accuracy of the disparity fields. However, this can be overcome by adjusting camera parameters

3D image

3D image Contains

the final product

X,Y,Z coordinate information

3D images can be improved by Disparity regulation Dynamic programming

Flow chart

SOFTWARE USED

Matlab

High-level language for technical computing

Tools for building custom graphical user interfaces Provides interactive environment for algorithm development, data visualization, data analysis, and numeric computation

Toolbox
Image processing toolbox Image acquisition toolbox Camera calibration toolbox

LEFT IMAGE

RIGHT IMAGE

Pixel wise disparity estimation is relatively less time consuming

Left image is taken as reference Each pixel is compared with pixels in 2nd image of same row

Minimum disparity found Relative disparity :Near points have more disparity and far points have less disparity

Merged Image

Disparity map

3D model

FUTURE WORKS

Block matching algorithm for better disparity. Dynamic programming and image pyramiding for smooth disparity map.

APPLICATION

Path planning of automated robots Space missions Explorations

ADVANTAGES

Easier Less time consuming method Less hardware

THANK YOU

 R t

3X3 rotation matrix 3X1 translation vector. Extrinsic matrix

Reconstruct the 3D plotting(X,Y,Z)

 Apply

in

for each

pixel. Obtain camera parameters using camera calibration toolbox P K , (x0, y0)

camera matrix camera intrinsic matrix scale factors=focal length coordinate of the principal point

REAL TIME CONSIDERATIONS

Have to perform image rectification

Rectified images