Summary and future work

Robotics and computer vision are relatively young disciplines but they are on the
rise. In this thesis, we study to integrate the capability of object detection, from computer
vision, to control a robot manipulator.
The proposed objectives for this thesis were to study a robot manipulator with 5
degrees-of-freedom and to be able to control it using computer vision. The first objective
was achieved when I was able to come up with the kinematics of the manipulator:
forward kinematics using Denavit-Hartenberg convention, inverse kinematics with the
geometrical method. After discovering the kinemtics of the manipulator, I proceed with
implementing path planning for the end-effector. In this objective, MATLAB with the
Peter Corke’s robotic toolbox played a major role in the formulation and simulation.
After getting the desired algorithm, I applied it to manipulator using Python, which is a
high level language and bears similarities with MATLAB.
Parallel with the first objective, the second objective was to come up with an
object detection algorithm that would enable the manipulator perform a pick-and-place
task on an object with a designated color and shape. This was done with the utilization of
the open source OpenCV library, which is the most adopted open source computer vision
library. With the use of OpenCV and Python, I was able to formulate a method to
recognize an object with basic shape and color. The method was to convert the captured
RGB image to the more natural HSV color space and then create a mask with upper and
lower bounds of the desired color in the HSV color space and applied it to the HSV
image. The result of this method is a binarized copy of the wanted object. This copy will
then be used to identify the shape of the object using contour approximation, which
estimate the number of vertices in the given contour.
Finally, combining all of the formulated algorithms of kinematics and object
detection, the manipulator was able to perform the pick-and-place task with a good
chance of success. It is estimated that under good lighting condition, the computer was
able to identify the center of the designated object with 100% chance of getting on point
with low deviation between sample and for the grabbing process, due to errors in
mechanical parts and uncalculated factors, the success chance was lower at 70%.
After all, due to the constraints in time and knowledge, the thesis still has some
drawbacks:
 The system is open-loop which means no feedback from the manipulator’s
joints. This makes it hard to control the real joint to match the calculated angle.
 The design of the manipulator permitted a lot of vibration from the servos and
does not leave room for improvement, e.g. installing another joint to increase
the degree-of-freedom.
  The computer vision method was still a classic 2D vision which is now
obsolete and surpassed by many modern methods. This method, which is
sufficient for the pick-and-place task, cannot measure depth without the use of
reference points.

Future work
As remedies for all the stated problems, in the future work, a new robot
manipulator will be built based on the experience of this one. The new robot will have a
bigger support base and an increased size for the lower joints which will form a counter
balance for all vibration created by the top joints. Also, the new robot will abandon the
use of RC servos and move on to more suitable motors like Stepper or AC servos.
A new type of camera will be used, preferably stereo camera. This type of camera
enable he detection of depth which will be used to estimate the height of objects.
Therefore, the manipulator will be able to move the grabbed object to higher destination
or a tilted one.