VISION ASSISTED PICK AND PLACE

ROBOTIC ARM GUIDED BY IMAGE

PROCESSING CONCEPTS FOR OBJECT
SORTING SYSTEM

BY,
ANJANA ANILKUMAR
S7 CSE
COLLEGE OF ENGINEERING MUNNAR
 OVERVIEW
 Introduction
 Concepts
 Machine vision
 Working demonstration
 Advantages and
Disadvantages
 Applications
 Conclusion and
Discussions
 References
 INTRODUCTION

• George Charles Devol -Inventor of the first industrial robot Unimate and this was a
turning point
• Pick and place robotic arms thus came into existence followed by a change in
industrial field by the application of robotics
• A robotic arm is a robotic manipulator, usually programmable, with similar functions to
a human arm.
 CONCEPTS

• Block diagram
• Image Acquisition
• Image Processing
• Data bus tools
• Object identification using signature
method
• The process behind the operation
 Getting in to the concepts
• Main function of a robotic manipulator is to pick some object from a fixed position and
to place it in some other fixed area
• Basic model of a robot arm consists of
• The servomotor is controlled by a pulse width modulated signal (control
signal) from the controller unit through its control wire
• We can either use a SG-5010 servo motor or Tower pro SG-90 servo motor
according to the specifications needed
• Block diagram
MACHINE VISION
 The process behind
• Usually a high quality webcam is used for image acquisition
• The image captured will then converted into grey scale image
• Crop the image
• Adjust Contrast
• Gray scale image is converted to binary image
• Sobel Edge Detection Algorithm
• Image Dilation
• Fill closed spaces (create blobs)
• Find bounding box of each object
 IMAGE PROCESSING

Acquired image Grey scale image Crop the image and

contrast adjustment
 IMAGE PROCESSING

Convert to Binary Image Edge detection Image dilation

IMAGE PROCESSING
Fill Holes
IMAGE PROCESSING
Find Bounding Box of each Object
 SOBEL EDGE DETECTION

• Performs a 2-D spatial gradient

measurement on an image.
• Uses a pair of 3x3 convolution masks, one
estimating the gradient in the x-direction
(columns) and the other estimating the
gradient in the y-direction (rows).
• A convolution mask is usually much
smaller than the actual image and is slid
over the image, manipulating a square of
pixels at a time.
SOBEL MASKS
SOBEL PROCEDURE
THE PROCESS
 ALGORITHM FOR THE OPERATION

• This method is used to convert colour image to greyscale image using Median filter to remove
noise from the image and then convert greyscale image to binary image.
• The binary image is then processed to find object.
• .As we need to find centroid of object, binary image is then, processed with image negative
algorithm.
• After finding centroid of object, signature of object is generated. Signature is a 1-dimensional
plot of distance from centre to boundary of object at regular increment in angle from 0 to 360.
 OBJREC ALGORITHM FOR OPERATIONAL USE

• Objrec algorithm is developed in MATLAB to recognize

the colour and send command to the controller using
serial communication
• Various steps involved are
Objrec algorithm
 TRANSFER OF CONTROL SIGNALS

• Object coordinates computed in the previous step are used to develop a

program to generate and send digital signals to robot controller via the
parallel port
• Interfacing circuit contains isolation circuit with optocoupler.
• Control signal send from PC to Optocoupler via parallel port.
• Optocoupler is necessary to prevent system from damage
WORKING MODEL
 ADVANTAGES

• Consistency of performance.
• 24/7 continuous working.
• Reduced amount of operator errors.
• It can move from one location to another
location.
• Robotic workers never get tired.
• Do not need to be paid.
• Can be made to perform even the most
dangerous tasks without concern.
 DISADVANTAGES

• High standard of maintenance required

• Precise programming needed (time, training,
specialist knowledge)
• when computer systems failure will cause
breakdown
• New products require complete reprogramming
• Certain processes still need a skilled operator
• Complex and expensive equipment to buy and
install
 APPLICATIONS
 Industries for assembly,process
automation,welding
 Medical applications
 Hazardous enviornments
 PCB manufacturing units.
 Space exploration
 Furnace manufacturing units.
 Oil refineries.
 Notable robotic arms and low cost robotic arms
 Pick and place robotic arms can be used as a
good teaching aid
 Enhancements that can be applied

• Range Estimation for Robot arm Applications using Image

Segmentation and Curve Fitting Tool
• Control Design and Implementation of Intelligent Vehicle with Robot
Arm and Computer Vision
• Automated Library System Using Robotic Arm Robot arm Applications
using Image Segmentation and Curve Fitting Tool
• Wireless Control of Pick and Place Robotic Arm Using an Android
Application
• Control Design and Implementation of Intelligent Vehicle with Robot
Arm and Computer Vision
• Object Locator and Collector Robotic Arm Using Artificial Neural
Network
 References

• INTERNATIONAL JOURNAL OF SCIENTIFIC & TECHNOLOGY RESEARCH VOLUME 5, ISSUE 06, JUNE 2016 Position
Control Method For Pick And Place Robot Arm For Object Sorting System -Khin Moe Myint, Zaw Min Min Htun, Hla Myo
Tun
• International Journal of Advanced Computational Engineering and Networking, -, Sept-2013 Object Detection Using Image
Processing For An Industrial Robot -OBJECT DETECTION USING IMAGE PROCESSING FOR AN
• INDUSTRIAL ROBOT P.B.VIJAYALAXMI, 2ROHAN PUTTA, 3GAYATRI SHINDE, 4PUNIT LOHANI,Assistant Professor, Department of
Computer Engineering, Fr. C. Rodrigues Institute of Technology, Vashi, India,Department of Computer Engineering, Fr. C.
Rodrigues Institute of Technology, Vashi, India
• Paul RP, Shimano B, Mayer GE. Kinematic control equation for simple manipulators. IEEE Trans. Syst. Man.Cybern. SMC-
11. 198; 449-455.
• International Journal of Robotics and Automation (IJRA)-Vol. 2, No. 3, September 2013-End-Effector Position Analysis
Using Forward Kinematics for 5 DOF Pravak Robot Arm-Jolly Shah1, S.S.Rattan1, B.C.Nakra21ME Department, NIT,
Kurukshetra, 136118, India
• International Journal of Mechanical & Mechatronics Engineering IJMME-IJENS , October 2013 - Development of a
Prototype Robot Manipulator for Industrial Pick-and-Place Operations
• 1.Ayokunle A. Awelewa;
• 2.Kenechukwu C. Mbanisi;
• 3.Samuel O.Majekodunmi;
• 4.Ishioma A. Odigwe;
• 5.Ayoade F. Agbetuyi;
• 6.Isaac A. Samuel
• -Department of Electrical and Information Engineering, College of Science & Technology, Covenant University, Ota, Ogun
State, Nigeria.
• Determining the Benefit of Human Input in Human-in-the-Loop Robotic
• Systems,Christine Bringes, Yun Lin, Yu Sun, Redwan Alqasemi-2013 IEEE RO-MAN: The 22nd IEEE International
Symposium on Robot and Human Interactive Communication Gyeongju, Korea
• Project on― Pick n Place Robot‖, made by Bharat Jain and Dinesh Rajput, under guidance of Prof. KaviArya, IIT
Mumbai,2010.
 Conclusion and discussion

• The analysed robots for pick and place application using image
processing are able to detect the fully desired object and to place it at
the desired location.
• The colour detection capability is enhanced by using high speed
processers and low cost cameras that use little power.
• MATLAB is used for GUI and for processing the image acquired. A new
software
• RoboCIM has been used which has inbuilt program which controls the
robot.
• The co-ordinates of the object are acquired through the camera and
edited in the program.
• Then this software can send signals to he robot and the robot then
moves toward the object,picks it and places it in the desired location
THANK YOU