Beruflich Dokumente
Kultur Dokumente
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
presents
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
The word robot originally was supposed to mean a slave It is a machine which performs a variety of tasks, either using manual external control or intelligent automation A manually controlled car or a ASIMOV trying to kick a football are all robots
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
ROBOTS
CONTROL
APPLICATIONS
INTERFACE
AUTONOMOUS
MANUAL
INDUSTRIAL MEDICAL
HARDWARE
SOFTWARE
INTERLINKED
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Locomotion System Actuators Power Supply System Transmission System Switches Sensory Devices For Feedback Sensor Data Processing Unit
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Each type of locomotion requires different number of degrees of freedom More degrees of freedom means more the number of actuators you will have to use
Although one actuator can be used to control more than one degree of freedom
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
The kind of locomotion most frequently used in robotics at the undergrad level This involves conversion of electrical energy into mechanical energy (mostly using motors)
The issue is to control these motors to give the required speed and torque
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
We have a simple equation for the constant power delivered to the motor:
P=X
Note that the torque and angular velocity are inversely proportionally to each other So to increase the speed we have to reduce the torque
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
The dc motors available have very high speed of rotation which is generally not needed
At high speeds, they lack torque For reduction in speed and increase in pulling capacity we use pulley or gear systems
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Differential Drive Dual Differential Drive Car-type Drive Skid-steer Drive Synchronous Drive
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
In-place (zero turning radius) rotation is done by turning the drive wheels at the same rate in the opposite direction Arbitrary motion paths can be implemented by dynamically modifying the angular velocity and/or direction of the drive wheels Total of two motors are required, both of them are responsible for translation and rotational motion
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Simplicity and ease of use makes it the most preferred system by beginners
Independent drives makes it difficult for straight line motion. The differences in motors and frictional profile of the two wheels cause them to move with slight turning effect
The above drawback must be countered with appropriate feedback system. Suitable for human controlled remote robots
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
The two can be directly mechanically coupled as they always move in the same direction with same speed
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
The direction of motion is given by black arrow. The alignment of the machine is shown by red arrow
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
The use of separate motors for translation and wheel turning guarantees straight line motion without the need for dynamic feedback control This system is somewhat complex in designing but further use is much simpler
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Actuators, also known as drives, are mechanisms for getting robots to move.
Most actuators are powered by pneumatics (air pressure), hydraulics (fluid pressure), or motors (electric current).
They are devices which transform an input signal (mainly an electrical signal)) into motion
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Widely used because of their small size and high energy output. Operating voltage: usually 6,12,24V. Speed: 1-20,000 rpm.. Power: P = X
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
The stator is the stationary outside part of a motor. The rotor is the inner part which rotates. Red represents a magnet or winding with a north polarization. Green represents a magnet or winding with a south polarization. Opposite, red and green, polarities attract. Commutator contacts are brown and the brushes are dark grey.
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Stator is composed of two or more permanent magnet pole pieces. Rotor composed of windings which are connected to a mechanical commutator. The opposite polarities of the energized winding and the stator magnet attract and the rotor will rotate until it is aligned with the stator. Just as the rotor reaches alignment, the brushes move across the commutator contacts and energize the next winding.
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
It is an electric motor that can divide a full rotation into a large number of steps. The motor's position can be controlled precisely, without any feedback mechanism. There are three types:
Permanent Magnet Variable (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha Resistance
Stepper motors work in a similar way to dc motors, but where dc motors have 1 electromagnetic coil to produce movement, stepper motors contain many.
Stepper motors are controlled by turning each coil on and off in a sequence.
Every time a new coil is energized, the motor rotates a few degrees, called the step angle.
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Full Step Stepper motors have 200 rotor teeth, or 200 full steps per revolution of the motor shaft. Dividing the 200 steps into the 360's rotation equals a 1.8 full step angle. Achieved by energizing both windings while reversing the current
Servos operate on the principle of negative feedback, where the control input is compared to the actual position of the mechanical system as measured.
Any difference between the actual and wanted values (an "error signal") is amplified and used to drive the system in the direction necessary to reduce or eliminate the error Their precision movement makes them ideal for powering legs, controlling rack and pinion steering, (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa Ajusal Sugathan & Utkarsh Sinha to move a sensor around |etc.
Suitable power source is needed to run the robots Mobile robots are most suitably powered by batteries The weight and energy capacity of the batteries may become the determinative factor of its performance
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
For a manually controlled robot, you can use batteries or voltage eliminators (convert the normal 220V supply to the required DC voltage 12V , 24V etc.)
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Gear Belt Pulley Chain Sprocket Rack and Pinion Pick Place Mechanisms
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Gears are the most common means of transmitting power in mechanical engineering Gears form vital elements of mechanisms in many machines such as vehicles, metal tooling machine tools, rolling mills, hoisting etc. In robotics its vital to control actuator speeds and in exercising different degrees of freedom
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
To achieve torque magnification and speed reduction They are analogous to transformers in electrical systems It follows the basic equation:
1 x r1 = 2 x r2
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
An arrangement of gears to convert rotational torque to linear motion Same mechanism used to steer wheels using a steering In robotics used extensively in clamping systems
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
It allows for mechanical power, torque, and speed to be transmitted across axes If the pulleys are of differing diameters, it gives a mechanical advantage In robotics it can be used in lifting loads or speed reduction Also it can be used in a differential drive to interconnect wheels
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Sprocket is a profiled wheel with teeth that meshes with a chain It is similar to the system found in bicycles It can transfer rotary motion between shafts in cases where gears are unsuitable Can be used over a larger distance Compared to pulleys has lesser slippage due to firm meshing between the chain and sprocket
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Image Processing is a tool for analyzing image data in all areas of natural science It is concerned with extracting data from real-world images Differences from computer graphics is that computer graphics makes extensive use of primitives like lines, triangles & points. However no such primitives exist in a real world images.
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Increasing need to replicate human sensory organs Eye (Vision) : The most useful and complex sensory organ
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Automated visual inspection system Checking of objects for defects visually Remote Sensing Satellite Image Processing Classification (OCR), identification (Handwriting, finger prints) etc. Detection and Recognition systems (Facial recognition..etc) Biomedical applications
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Camera, Scanner or any other image acquisition device PC or Workstation or Digital Signal Processor for processing Software to run on the hardware platform (Matlab, Open CV etc.) Image representation to process the image (usually matrix) and provide spatial relationship A particular color space is used to represent the image
Captures data from a single light receptor at a time CCD Charge Coupled Devices CMOS Complementary MOSFET Sensor based
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Digital Cameras
CCD Cameras High quality, low noise images Genarates analog signal converted using ADC Consumes high power CMOS Cameras Lesser sensitivity Poor image quality Lesser power
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Matlab
Open CV
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Two types: Vector and Raster Vector images store curve information Example: Indias flag Three rectangles, one circle and the spokes
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
If you think about it, your laptops display is a raster display Also, vector images are high level abstractions Vector representations are more complex and used for specific purposes
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Raster
Matrix
Vector
Quadtrees Chains Pyramid
Of the four, matrix is the most general. The other three are used for special purposes. All these representations must provide for spatial relationships
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Computers cannot handle continuous images but only arrays of digital numbers So images are represented as 2-D arrays of points (2-D matrix)(Raster Represenatation) A point on this 2-D grid (corresponding to the image matrix element) is called PIXEL (picture element) It represents the average irradiance over the area of the pixel
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Each pixel requires some memory Color depth : Amount of memory each pixel requires Examples
1-bit 8-bit
32-bit
64-bit
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Pixels are tiny little dots of color you see on your screen, and the smallest possible size any image can get When an image is stored, the image file contains information on every single pixel in that image i.e
The number of pixels used to represent the image digitally is called Resolution More the number of pixels used, higher the resolution Higher resolution requires more processing power
Pixel Location Intensity
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
MATLAB stands for MATrix LABoratory, a software developed by Mathworks Inc (www.mathworks.com). MATLAB provides extensive library support for various domains of scientific and engineering computations and simulations
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
When you click the MATLAB icon (from your desktop or Start>All Programs), you typically see three windows: Command Window, Workspace and Command History. Snapshots of these windows are shown below
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
This window shows the variables defined by you in current session on MATLAB
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Command History stores the list of recently used commands for quick reference
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
In MATLAB, variables are stored as matrices (singular: matrix), which could be either an integer, real numbers or even complex numbers
These matrices bear some resemblance to array data structures (used in computer programming)
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Let us start with writing simple instructions on MATLAB command window To define an integer, Type a=4 and hit enter >>a=4 To avoid seeing the variable, add a semicolon after the instruction >>a=4;
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Similarly to define a 2x2 matrix, the instruction in MATLAB is written as >> b=[ 1 2; 3 4]; If you are familiar with operations on matrix, you can find the determinant or the inverse of the matrix. >> determin= det(b) >> d=inv(b)
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Images as we have already seen are stored as matrices So now we try to see this for real on MATLAB We shall also look into the basic commands provided by MATLABs Image Processing Toolbox
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Once you have started MATLAB, type the following in the Command Window
>> im=imread(sample.jpg');
This command stores the file image file sample.jpg in a variable called im It takes this file from the CurrentDirectory specified Else, entire path of file should be mentioned
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
You can display the image in another window by using imshow command >>figure,imshow(im);
This pops up another window (called as figure window), and displays the image im
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
The imview command can also be used in order toview the image imview(im);
Difference is that in this case you can see specific pixel values just by moving the cursor over the image
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
To know the breadth and height of the image, use the size function, >>s=size(im);
The size function basically gives the size of any array in MATLAB Here we get the size of the IMAGE ARRAY
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Now that we have our image stored in a variable we can observe and understand the following: How pixels are stored? What does the values given by each pixel indicate? What is Image Resolution?
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
>>im(1:10,1:10);
Or Say view the pixel range 50:150 on both axis >> figure,imshow(im(50:150,50:150));
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
1-bit = BLACK or WHITE 8-bit = 28 different shades 24-bit = 224 different shades 64-bit images High end displays Used in HDRI, storing extra information per pixel, etc
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
This is another name for 1-bit images Each pixel is either White or Black Technically, this is a black & white image
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Another name for 8-bit images Each pixel can be one of 256 different shades of gray These images are popularly called Black & White. Though, this is technically wrong.
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Again, each pixel gets 8 bits But each of the 256 values maps to a color in a predefined palette If required, you can have different bit depths
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
8-bits is too less for all the different shades of colors we see So 24-bits is generally used for color images Thus each pixel can have one of 224 unique colors
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Now, a new problem arises: How do you manage so many different shades? Programmers would go nuts Then came along the idea of color spaces
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
A color space can be thought of as a way to manage millions of colors Eliminates memorization, and increases predictability Common color spaces:
RGB
HSV
YCrCb or YUV YIQ
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Each pixel stores 3 bytes of data The 24-bits are divided into three 8-bit values The three are: Red, Green and Blue i.e the primary colours Mixing of primary colours in right proportions gives any particular colour Each pixel has these 3 values
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
1 byte = 8 bits can store a value between 0-255 We get pixel data in the form RGB values with each varying from 0-255 That is how displays work So there are 3 grayscale channels
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Advantages:
Intuitive Very widely used
Disadvantages:
Image processing is relatively tough
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
HSV makes image processing easier Again, 24 bits = three 8-bit values or 3 channels The 3 channels are:
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
RGB HUE
SATURATION VALUE
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Advantages:
The color at a pixel depends on a single
Disadvantages:
Something
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Intuitively RGB might seem to be the simpler and better colour space to deal with Though HSV has its own advantages especially in colour thresholding As the colour at each pixel depends on a single hue value it is very useful in separating out blobs of specific colours even when there are huge light variations Thus it is very useful in processing real images taken from camera as there is a large amount of intensity variation in this case Hence, ideal for robotics applications
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Hence Cr and Cb components represent the colour called Color Difference Components
Y Component: Gives luminance or intensity Cr Component: It is the RED component minus a reference value Cb Component: It is the BLUE component minus a reference value
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Advantages:
Used in video processing Gives you a 2-D colour space hence helps
Disadvantages:
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
The camera returns images in a certain color space You might want to convert to different color spaces to process it Colour space conversions can take place between RGB to any other colour space and vice versa
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Since cameras usually input images in rgb We would like to convert these images into HSV or YCrCb Conversions:
RGB->HSV
HSV->RGB
RGB->YCrCb YCrCb->RGB
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
>>h = rgb2hsv(im)
This converts the RGB image to HSV The new colour space components can be seen using >> imview(h) >> imview(h(:,:,1)) HUE >> imview(h(:,:,2)) Saturation >> imview(h(:,:,3)) Value
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
>>R = hsv2rgb(im)
This converts the HSV image to RGB The new colour space components can be seen using >> imview(R) >> imview(R(:,:,1)) Red >> imview(R(:,:,2)) Green >> imview(R(:,:,3)) Blue
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
>> Y = rgb2ycbcr(im);
This converts the RGB image to YCbCr The new colour space components can be seen using >> imview(Y) >> imview(Y(:,:,1)) Luminance >> imview(Y(:,:,2)) Differenced Blue >> imview(Y(:,:,3)) Differenced Red
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
>> R = ycbcr2rgb(im);
This converts the YCbCr image to RGB The new colour space components can be seen using >> imview(R) >> imview(R(:,:,1)) Red >> imview(R(:,:,2)) Green >> imview(R(:,:,3)) Blue
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Formulae for conversion are very complex But the best thing is, you dont need to remember these formulae Matlab and OpenCV have built-in functions for these transformations :-)
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
OpenCV is a collection of many functions that help in image processing You can use OpenCV in C/C++, .net languages, Java, Python, etc as well We will only discuss OpenCV in C/C++
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
It is blazingly fast
Quite simple to use and learn Has functions for machine learning, image processing, and GUI creation
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Install the package, and note where you installed it (like C:\Program Files\OpenCV\)
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Now, we need to tell Microsoft Visual Studio that weve installed OpenCV So, we tell it where to find the OpenCV header files Start Microsoft Visual Studio 2008
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
2 1
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Right now, Visual Studio knows where to find the OpenCV include files and library files
Now we create a new project
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Accept all default settings in the project Youll end up with an empty project with a single file (like Mybot.cpp) Open this file, well write some code now
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
int main() {
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Windows bitmaps - BMP, DIB JPEG files - JPEG, JPG, JPE Portable Network Graphics - PNG Portable image format - PBM, PGM, PPM Sun rasters - SR, RAS TIFF files - TIFF, TIF OpenEXR HDR images - EXR JPEG 2000 images - jp2
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Shows img in the window with caption title If no such window exists, nothing happens
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Waits for time milliseconds, and returns whatever key is pressed If time=0, waits till eternity
Here, weve used it to keep the windows from vanishing immediately
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Right now, Visual Studio knows where OpenCV is But it does not know, whether to use OpenCV or not We need to tell this explicitly
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Got errors?
Check if the syntax is correct Copy all DLL files in *\OpenCV\bin\ into
C:\Windows\System32
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
src is the original image dst is the destination code is one of the follow:
CV_BGR2HSV
CV_RGB2HSV CV_RGB2YCrCb
CV_HSV2RGB
CV_<src_space>2<dst_space>
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
src should be a valid image. Or an error will pop up dst should be a valid image, i.e. you need a blank image of the same size code should be valid (check the OpenCV documentation for that)
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Allocates memory for an image of size size, with bits bits/pixel and chan number of channels
Used for creating a blank image
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Example:
IplImage* blankImg =
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Wired
Motor Driving module Interface with PC (Parallel/Serial)
Wireless
The Motor-driving module The Wireless Receiver Circuit
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
MCT2E Opto-Coupler
Parallel Port Male-Connector
RF-RX Connector
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Its a three terminal linear 5 volt regulator used to supply the board and other peripherals Prescribed input voltage to this component is about 7-9 Volts
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Voltage fluctuations can be controlled by using low pass filter capacitors across output and input Higher input voltage can be applied if heatsink is provided
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Used to control Dc and Stepper Motors Uses a H-Bridge which is an electronic switching circuit that can reverse direction of current Its a Dual-H bridge Basically used to convert a low voltage input into a high voltage output to drive the motor or any other component Eg: Microcontroller Motor Driver Motor (5 Volts) (12 Volts)
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
L293D 600mA Current Rating Dual H-bridge (Dc and Stepper Motors) L298N 1 Amp Current Rating Dual H-bridge (Dc and Stepper Motors) L297-L298 (Coupled) For stepper motor overdriving Dual H-bridge (Dc and Stepper Motors) 2 Ics in parallel ULN2003/ULN2803 500mA Current Rating For unipolar stepper motors
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Output Current:
600 mA
Output Voltage
Wide Range 4.5 V 36 V
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
There are many situations where signals and data need to be transferred from one subsystem to another within a piece of electronics Relays are too bulky as they are electromechanical in nature and at the same time give lesser efficiency In these cases an electronic component called Optocoupler is used
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
They are generally used when the 2 subsystems are at largely different voltages These use a beam of light to transmit the signals or data across an electrical barrier, and achieve excellent isolation
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
In our circuit, Opto-isolator (MCT2E) is used to ensure electrical isolation between motors and the PC parallel port during wired connection The Viz-Board has four such chips to isolate the four data lines (pin 2, pin 3, pin 4, pin 5) coming out of the parallel port
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Transmitt er
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Receive r
Radio frequency modules are used for data transmission wirelessly at a certain frequency It sends and receives radio waves of a particular frequency and a decoder and encoder IC is provided to encode and decode this information Wireless transmission takes place at a particular frequency Eg. 315Mhz Theses modules might be single or dual frequency
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Antenna is recommended on both of them - just connect any piece of 23 cm long to the Antenna pin The kit has a dual frequency RF module with frequencies 315/434 Mhz
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
The encoder IC encodes the parallel port data and sends it to the RF transmitter module for wireless transmission
They are capable of encoding information which consists of N address bits and (12-N) data bits
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
The HT12E Encoder IC has 8 address bits and 4 data bits A DIP-Switch can be used to set or unset the address bits A0-A7
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
A0-A7Address Bits
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
AD8-AD11Data Bits
A0-A7Address Bits
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
AD8-AD11Data Bits
The decoder IC decodes the RF transmitter data and sends it to the parallel port for wireless transmission
They are capable of encoding information which consists of N address bits and (12-N) data bits
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
The HT12D Decoder IC has 8 address bits and 4 data bits A DIP-Switch can be used to set or unset the address bits A0-A7
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
A0-A7Address Bits
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
D8-D11Data Bits
A0-A7Address Bits
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
D8-D11Data Bits
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Serial Port
Parallel Port USB
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Data is transferred serially i.e packets are sent one after the other through a single port
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Data is transferred in parallel through different data pins at the same time Communication is pretty fast
Found in old printer ports
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Parallel port is faster than serial A mass of data can be transmitted at the same time through parallel ports Though parallel and serial ports are not found these days in laptops Desktops and old laptops have these ports
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Sampling: Digitizing the coordinate values (spatial resolution) Quantization: Digitizing the amplitude values (intensity levels)
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
1 bit /pixel
2 possible values 2 gray levels -> 0 or 1 (binary image)
B bits/pixel
2B gray levels 1 byte = 8 bits > 256 levels
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
All this sampling and quantization puts in extra noise on the image! Noise can be reduced by
Using hardware Using software: filters
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Large amounts of external disturbances in real images Due to different factors like changing lighting and other real-time effects To improve quality of a captured image to make it easier to process the image
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Mean Filter
Median Filter Gaussian Smoothing
Histogram Equalization
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
The Averaging filter is used to sharpen the images by taking average over a number of images
It eliminates noise by assuming that different snaps of the same image have different noise patterns
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
This is extremely useful for satellites that take intergalactic photographs The images are extremely faint, and there is more noise than the image itself
Millions of pictures are taken, and averaged to get a clear picture
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
The center pixel would be changed from 77 to 92 as that is the mean value of all surrounding pixels This filter is often used to smooth images prior to processing It can be used to reduce pixel flicker due to overhead fluorescent lights
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
This replaces each pixel value by the median of its neighbors, i.e. the value such that 50% of the values in the neighborhood are above, and 50% are below This can be difficult and costly to implement due to the need for sorting of the values However, this method is generally very good at preserving edges
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
If the neighborhood under consideration contains an even number of pixels, the average of the two middle pixel values is used
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Used to `blur' images and remove detail and noise The effect of Gaussian smoothing is to blur an image
The Gaussian outputs a `weighted average' of each pixel's neighborhood, with the average weighted more towards the value of the central pixels
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
A Gaussian provides gentler smoothing and preserves edges better than a similarly sized mean filter
Before Blurring
After Blurring
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Options in imfilter
Convolution is same as correlation except that the h matrix is inverted before applying the filter
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
The process of adjusting intensity values can be done automatically by the histeq function
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Original Image
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Things arent as simple as they were in Matlab C/C++ needs a bit of syntax and formalities
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Bilateral filter
Simple blur Averaging filter
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
#include <cv.h> #include <highgui.h> int main() { IplImage* img = cvLoadImage(C:\\noisy.jpg); IplImage* imgBlur = cvCreateImage(cvGetSize(img), 8, 3); IplImage* imgGaussian = cvCreateImage(cvGetSize (img), 8, 3); IplImage* imgMedian = cvCreateImage(cvGetSize (img), 8, 3); IplImage* imgBilateral = cvCreateImage(cvGetSize (img), 8, 3);
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
imgBlur, CV_BLUR, 3, 3); imgGaussian, CV_GAUSSIAN, 3, 3); imgMedian, CV_MEDIAN, 3, 3); imgBilateral, CV_BILATERAL, 3, 3);
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
cvNamedWindow(original); cvNamedWindow(blur); cvNamedWindow(gaussian); cvNamedWindow(median); cvNamedWindow(bilateral); cvShowImage(original, img); cvShowImage(blur, imgBlur); cvShowImage(gaussian, imgGaussian); cvShowImage(median, imgMedian); cvShowImage(bilateral, imgBilateral); cvWaitKey(0); return 0; }
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
IplImage* imgBlue[25]; for(int i=0;i<25;i++) { IplImage* img; char filename[150]; sprintf(filename, "%d.jpg", (i+1)); img = cvLoadImage(filename);
Generate the strings 1.jpg, 2.jpg, etc and store them into filename Load the image filename
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
imgBlue[i] = cvCreateImage(cvGetSize(img), 8,
1);
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
cvReleaseImage(&img);
}
Split img into constituent channels Note the order: B G R Release img
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
We created 75 grayscale images: 25 for red, 25 for green and 25 for blues Loaded 25 color images in the loop
Split each image, and stored in an appropriate grayscale image
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
CvSize imgSize = cvGetSize(imgRed[0]); IplImage* imgResultRed = cvCreateImage(imgSize, 8, 1); IplImage* imgResultGreen = cvCreateImage(imgSize, 8, 1); IplImage* imgResultBlue = cvCreateImage(imgSize, 8, 1); IplImage* imgResult = cvCreateImage(imgSize, 8, 3);
This will hold the final, filtered image It will be a combination of the grayscale channels imgResultRed, imgResultGreen and imgResultBlue
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
for(int x=0;x<imgSize.width;x++) { int theSumRed=0; int theSumGreen=0; int theSumBlue=0; for(int i=0;i<25;i++) {
To figure out the average, we need to find the numerator (the sum) over all 25 images
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
x);
theSumGreen+=cvGetReal2D(imgGreen[i], y, x); theSumBlue+=cvGetReal2D(imgBlue[i], y, x); }
To figure out the average, we need to find the numerator (the sum) over all 25 images
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
theSumRed = (float)theSumRed/25.0f; theSumGreen = (float)theSumGreen/25.0f; theSumBlue = (float)theSumBlue/25.0f; cvSetReal2D(imgResultRed, y, x, theSumRed); cvSetReal2D(imgResultGreen, y, x, theSumGreen); cvSetReal2D(imgResultBlue, y, x, theSumBlue); } } Once we have the sum, we divide by 25 and set the appropriate pixels
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
cvMerge(imgResultBlue, imgResultGreen, imgResultRed, NULL, imgResult); cvNamedWindow("averaged"); cvShowImage("averaged", imgResult); cvWaitKey(0); return 0; }
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
cvGetReal2D to get value at a pixel in grayscale images cvSetReal2D to set the value at a pixel
CvSize to store an images size
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
The process of extracting image components that are useful in representation of image for some particular purpose Basic morphological operations are:
Dilation Erosion
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
The operation that grows or thickens objects in a binary image The specific manner of thickening is controlled by a shape referred to as structuring element
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Binary Image
Structuring Element
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Dilated Image
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Erosion shrink or thins objects in a binary image The manner of shrinkage is controlled by the structuring element
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Binary Image
Structuring Element
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Eroded Image
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
In practical image processing dilation and erosion are performed in various combinations
An image can undergo a series for diltions and erosion using the same or different structuring element
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
In practical image processing dilation and erosion are performed in various combinations An image can undergo a series for diltions and erosion using the same or different structuring element Two Common Kinds:
It is basically one erosion followed by one dilation by the same structuring element
They are used to smooth object contours, break thin connections and remove thin protrusions
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
It is basically one dilation followed by one erosion by the same structuring element They are used to smooth object contours like opening But unlike opening they generally join narrow breaks, fill long thin gulfs and fills holes smaller than the structuring element
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
cvErode(src, dst)
cvDilate(src, dst) Opening & closing: use the appropriate sequence
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
By default, OpenCV uses the zero structuring element (all are zeros) You can explicitly specify your structuring element as well Check the OpenCV Documentation for more information
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Computers can manipulate images very efficiently But, comprehending an image with millions of colors is tough Solution: Figure out interesting regions, and process them
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Demo thresholdRGB
Demo thresholdHSV
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
MATLAB provides a facility to execute multiple command statements with a single command. This is done by writing a .m file
Goto File > New > M-file For example, the graythresh function can be manually written as a m-file as:
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Observe that, comments (in green) can be written after the symbol %. A commented statement is not considered for execution
M-files become a very handy utility for writing lengthy programs and can be saved and edited, as and when required We shall now see, how to define your own functions in MATLAB.
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Functions help in writing organized code with minimum repetition of logic Instead of rewriting the instruction set every time, you can define a function Syntax: Create an m-file and the top most statement of the file should be the function header
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
The inbuilt graythresh function in matlab is used for thresholding of grayscale images
It uses the Otsus Method Of thresholding
A sample thresholding opreation has been shown in the next slide
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Thresholding of a grayscale image can be done in MATLAB using the following commands:
>> level=graythresh(imGRAY); >> imBW = im2bw(imGRAY,level); >> imview(imBW);
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
The graythresh command basically gives an idea as to what exactly the threshold value should be
Graythresh returns a value that lies in the range 0-1
This gives the level of threshold which is obtained by a complex method called the Otsus Method of Thresholding
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
This level can be converted into pixel value by multiplying by 255 Lets say, level=.4
Then threshold value for the grayscale image is: 0.4 x 255 =102
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
What this indicates is that for the given image the values below 102 have to be converted to 0 and values from 103-255 to the value 1
Conversion from grayscale to binary image is done using the function: >>imBW = im2bw(imGRAY,level);
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Here level is the threshold level obtained from graythresh function This function converts pixel intensities between 0 to level to zero intensity (black) and between level+1 to 255 to maximum (white)
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
In order to threshold an RGB colour image using the graythresh function, the following have to be done:
Conversion of the RGB image into its 3
grayscale components Subtracting each of these components from the other 2 to get the pure colour intensities Finding level for each of the grayscale using graythresh Thresholding the image using imbw and the level
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Commands: Im=Imread(rgb.jpg);
R = im(:,:,1); --Red
G = im(:,:,2); --Green B = im(:,:,3); --Blue
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Levelr=graythresh(Ronly);
Levelg=graythresh(Gonly); Levelb=graythresh(Bonly);
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Rthresh=im2bw (Ronly,levelR);
Gthresh=im2bw(Gonly,levelG); Bthresh=im2bw(Bonly,levelB);
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
for j=1:s(1,2)
Splitting of HSV image into components Using the Hue channel and thresholding it for different values Since the hue value of a single colour is constant it is relatively simple to threshold and gives better accuracy
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Splitting of HSV image into components Using the Hue channel and thresholding it for different values Since the hue value of a single colour is constant it is relatively simple to threshold and gives better accuracy
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
function [temp] = ht(im,level1,level2) s=size(im); temp=im; for i=1:s(1,1) for j=1:s(1,2) if (temp(i,j)<level2 & temp(i,j)>level1) temp(i,j)=1; else temp(i,j)=0; end end end imview(temp);
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
To this function we give the input arguments as the upper and lower bounds of the threshold levels
These levels can be obtained by having a look at the range of hue values for the particular colour
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
cvInRangeS(src, scalarLower, scalarUpper, dst); scalarLower = cvScalar(chan1, chan2, chan3, chan4); scalarUpper = cvScalar(chan1, chan2, chan3, chan4);
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Ultra basics: motors, drives, etc Digital image representation Color spaces Inter-conversion of color spaces Electronics Filtering Thresholding Morphology
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
After thresholding, we get a binary image We want useable information like centers, outlines, etc There geometrical properties can be found using many methods. Well talk about moments and contours only.
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Consider xorder=0 and yorder=0 for a binary image So youre just summing up pixel values
This means, youre calculating the area of the white pixels
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Now consider xorder=1 and yorder=0 for a binary image You sum only those x which are white
So youre calculating the numerator of an average
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
The number of points where the pixel is white is the area of the image So, dividing this particular moment (xorder=1, yorder=0) by the earlier example (xorder=0, yorder=0) gives the average x
This is the x coordinate of the centroid of the blob
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
The order of a moment = xorder+yorder So, the area is a zero order moment
The centroid coordinate = a first order moment / the zero order moment
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
There are entire books written on this topic You can find complex geometrical properties, like the eccentricity of an ellipse, radius of curvature of objects, etc
Also check for Hu invariants if youre interested
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
These are pixels of an image that are conencted to each other forming separate blobs in an image They can be seperated out and labelled
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
>>L = bwlabel(BW,n)
Returns a matrix L, of the same size as BW, containing labels for the connected objects in BW n can have a value of either 4 or 8, where 4 specifies 4-connected objects and 8 specifies 8-connected objects; if the argument is omitted, it defaults to 8
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
>>L = bwlabel(BW,n)
Returns a matrix L, of the same size as BW, containing labels for the connected objects in BW n can have a value of either 4 or 8, where 4 specifies 4-connected objects and 8 specifies 8-connected objects; if the argument is omitted, it defaults to 8
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
STATS = regionprops(L,properties)
Measures a set of properties for each labeled region in the label matrix L The set of elements of L equal to 1 corresponds to region 1; the set of elements of L equal to 2 corresponds to region 2; and so on
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
'Area' The actual number of pixels in the region 'Centroid'-- The center of mass of the region. Note that the first element of Centroid is the horizontal coordinate (or x-coordinate) of the center of mass, and the second element is the vertical coordinate (or y-coordinate) 'Orientation' -- Scalar; the angle (in degrees) between the x-axis and the major axis of the ellipse that has the same second-moments as the region. This property is supported only for 2-D input label matrices
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Binary area open remove small objects BW2 = bwareaopen(BW,P) Removes from a binary image all connected components (objects) that have fewer than P pixels, producing another binary image, BW2.
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Example
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
For robotics purposes, moments are fine till have one single object If we have multiple objects in the same binary image
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
The output you get for the previous binary image is:
Four chains of points
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Contour plotting
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
im = imread(img.jpg'); imcontour(im,level)
Level=number of equally spaced contour levels if level is not given it will choose automatically
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
OpenCV linked lists to store the chains Well see some code to find out the squares in the thresholded image you saw
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
The chains are stored in contours result is a temporary variable storage is for temporary memory allocation
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
img is a grayscale thresholded image storage is for temporary storage All chains found would be stored in the contours sequence The rest of the parameters are usually kept at these values Check the OpenCV documentation for details information about the last four variables
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
The previous command makes contours point to the first chain Were approximating the contour right now
After this command, result stores the approximate contour as a polygon (many points)
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
if(result->total==4) { CvPoint *pt[4]; for(int i=0;i<4;i++) pt[i] = (CvPoint*)cvGetSeqElem(result, i); } Were looking for quadrilaterals, so we check if the number of points in this particular polygon is 4
Then, get extract each point using the command cvGetSeqElem Once you have the points, you can actually check the shape of the object as well (by checking angles, lengths, etc)
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
MATLAB has an image acquisition toolbox which helps capture images Now-a-days most of the cameras are available with USB interface Once you install the driver for the camera, the computer detects the device whenever you connect it
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
In MATLAB, you can check if the support is available for your camera MATLAB has built-in adaptors for accessing these devices An adaptor is a software that MATLAB uses to communicate with an image acquisition device
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
COMMANDS:
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
To get more information about the device, type >>dev_info = imaqhwinfo('winvideo',1) Instead of winvideo, if imaqhwinfo shows another adaptor, then type that adaptor name instead of winvideo.
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
You can preview the video captured by the image by defining an object and associate it with the device
>>vid=videoinput(winvideo,1,RGB24 _320x240)
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Now to see the video insert the following command: >> preview(vid)
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
You should see a window pop-up, that displays what your camera is capturing
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
The camera may support multiple video formats. To see for yourself all the supported formats, type
Check out for yourself the display of other formats, by replacing `RGB24_320x240` with other formats, in the definition of the object vid
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Now to capture an image from the video, define the object vid as described before and use getdata to capture a frame from the video
>>start(vid); % This command initiates capturing of frames and stores the frames in memory >>im=getdata(vid,1); >>imview(im);
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
You can store the captured image as a .jpg or .gif file using imwrite function >>imwrite(im,'testimage.gif'); The image will be stored in MATLAB71\work folder
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Static Processing
Step Processing Real-Time Processing
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Take a single picture of the arena and process it Find out critical regions and points
Apply some geometry and mathematical calculations Then blindly follow a specified path
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Advantages
Simplest to implement Can be fairly accurate with stepper motors
Disadvantages
Bot goes blind because only one pic
determines the bot motion Accuracy is very low especially with DC motors
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Advantages
Quite simple to implement Can be very accurate with stepper motors
Disadvantages
Bot goes blind for a particular period of
time Accuracy is compromised with DC motors Dynamic environmental changes cannot be accounted for
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Advantages
Very accurate for both DC and Stepper motors Gives dynamic feedback and accounts for
Disadvantages
Requires more processing power
Requires more memory for taking so many
images
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Real-time image processing is the best approach for any real application Dynamic feedback systems give excellent accuracy and precision and hence is the best approach
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Every time you want to capture an instantaneous image, you have to stop the video, start it again and use the getdata function
To avoid this repetitive actions, the Image Acquisition toolbox provides an option for triggering the video object when required and capture an instantaneous frame
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Every time you want to capture an instantaneous image, you have to stop the video, start it again and use the getdata function
To avoid this repetitive actions, the Image Acquisition toolbox provides an option for triggering the video object when required and capture an instantaneous frame
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
for i=1:5 trigger(vid); im= getdata(vid,1); figure,imshow(im); end stop(vid); delete(vid); clear vid;
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
In the above code, object im gets overwritten while execution of each of the interations of the for loop
To be able to see all the five images, replace im with im(:,:,:,i)
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
In the above code, object im gets overwritten while execution of each of the interations of the for loop
To be able to see all the five images, replace im with im(:,:,:,i)
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
triggerconfig sets the object to manual triggering, since its default triggering is of type immediate In immediate triggering, the video is captured as soon as you start the object vid The captured frames are stored in memory. Getdata function can be used to access these frames But in manual triggering, you get the image only when you trigger the video
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
FramesPerTrigger decides the number of frames you want to capture each time trigger is executed TriggerRepeat has to be either equal to the number of frames you want to process in your program or it can be set to Inf If set to any positive integer, you will have to start the video capture again after trigger is used for those many number of times
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Once you are done with acquiring of frames and have stored the images, you can stop the video capture and clear the stored frames from the memory buffer, using following commands:
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Getsnapshot function returns one image frame and is independent of FramesPerTrigger property
So if you want to process your images in real-time, this is all you need:
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
vid=videoinput(winvideo,1) triggerconfig(vid,'manual'); set(vid,'FramesPerTrigger',1); set(vid,'TriggerRepeat', Inf); start(vid); while(1) { trigger(vid); im= getdata(vid,1); % write your image processing algorithm here % % you may break this infinite while loop if a certain condition is met }
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Using MATLAB
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
MATLAB provides support to access serial port (also called as COM port) and parallel port (also called as printer port or LPT port) of a PC
MATLAB has an adaptor to access the parallel port (similar to adaptor for image acquisition)
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
To access the parallel port in MATLAB, define an object >> parport= digitalio('parallel','LPT1');
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
You have to define the pins 2-9 as output pins, by using addline function
>> addline(parport, 0:7, 'out') Now put the data which you want to output to the parallel port into a matrix; e.g. >> dataout = logical([1 0 1 0 1 0 1 1]);
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
You can connect the pins of the parallel port to the driver IC for the left and right motors of your robot, and control the left, right, forward and backward motion of the vehicle
You will need a H-bridge for driving the motor in both clockwise and anticlockwise directions
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Using MATLAB
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Just follow the instructions that come along, and youll be sending data to your robot!
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
port itself (parallel, serial, etc) Keep this file open And keep writing to this file So, data is automatically send to the desired port
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
HANDLE hPort;
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Step 2: Create function to create the virtual file, and store its handle in hPort
bool SerialOpen(LPCWSTR strPort) { // Open the serial port. hPort = (HANDLE)CreateFile (strPort, GENERIC_READ | GENERIC_WRITE, 0, NULL, OPEN_EXISTING, 0, (long)NULL); // // // // // // // // Pointer to the name of the port Access (read-write) mode Share mode Pointer to the security attribute How to open the serial port Port attributes Handle to port with attribute to copy
DCB PortDCB; DWORD dwError; // Initialize the DCBlength member. PortDCB.DCBlength = sizeof (DCB); // Get the default port setting information. GetCommState (hPort, &PortDCB);
(contd)
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
// Change the DCB structure settings. PortDCB.BaudRate = 9600; // Current baud PortDCB.fBinary = TRUE; // Binary mode; no EOF check PortDCB.fParity = TRUE; // Enable parity checking PortDCB.fOutxCtsFlow = FALSE; // No CTS output flow control PortDCB.fOutxDsrFlow = FALSE; // No DSR output flow control PortDCB.fDtrControl = DTR_CONTROL_ENABLE; // DTR flow control type PortDCB.fDsrSensitivity = FALSE; // DSR sensitivity PortDCB.fTXContinueOnXoff = TRUE; // XOFF continues Tx PortDCB.fOutX = FALSE; // No XON/XOFF out flow control PortDCB.fInX = FALSE; // No XON/XOFF in flow control PortDCB.fErrorChar = FALSE; // Disable error replacement PortDCB.fNull = FALSE; // Disable null stripping PortDCB.fRtsControl = RTS_CONTROL_ENABLE; // RTS flow control PortDCB.fAbortOnError = FALSE; // Do not abort reads/writes on error PortDCB.ByteSize = 8; // Number of bits/byte, 4-8 PortDCB.Parity = NOPARITY; // 0-4=no,odd,even,mark,space PortDCB.StopBits = ONESTOPBIT; // 0,1,2 = 1, 1.5, 2
(contd)
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
// Configure the port according to the specifications of the DCB // structure. if (!SetCommState (hPort, &PortDCB)) { // Could not configure the serial port. dwError = GetLastError(); printf("Serial port creation error: %d", dwError); MessageBox(NULL, L"Unable to configure the serial port", L"Error", MB_OK); return false; } return true; }
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
This function works equally well for both serial ports and parallel ports
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Step 2: Ive assumed youre creating a function This function returns a true when the port is created successfully If youre not, replace the return statements with printfs
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
This is actually how you can access ports in DOS as well using COM8: and LPT1: instead of C:, D:, etc The L before the quotes is just syntax for C/C++
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Step 3: Next, well write functions to write to the virtual file weve created
bool SerialWrite(byte theByte) { // The port wasn't opened if(!hPort) return false;
Port handle Pointer to the data to write Number of bytes to write Pointer to the number of bytes written Must be NULL
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
You pass the byte you want to write as a parameter, and it gets written to the port
Example: SerialWrite(12)
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Example: SerialClose()
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Again, Ill emphasize that these functions will work equally well for both parallel ports and serial ports
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
There are different kinds of edge detectors based on the criteria of derivatives An edge detector can be sensitive to horizontal or vertical lines or both
In detection we try to find out regions where the derivative or gradient is greater than a specified threshold
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
g=edge(im,method,t); t-threshold
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
g=edge(im,method,t); t-threshold
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Image Acquisition Image Cropping Image Filtering Image Segmentation Image Thresholding Finding critical points (Centroids) Finding bot centroidand orientation Robot Feedback control Robot Control
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Static processing
Step processing Real-time processing
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Manual Cropping:
I2 = imcrop(I);
Coordinate Cropping:
I2 = imcrop(I,[x1 y1 x2 y2]);
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Mean filter
Median filter Histogram equalization
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Canny
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Thresholding
In RGB In HSV
Separating colors
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Using orientation
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
The idea of an orientation tag is to precisely indicate the orientation of the robot
It should happen with the least number of operations
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
But if you have a team of bots, this wont be the best choice You need to have multiple colors for each bot
So, you have more operations Slowing down the program
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
OpenCV lets you access cameras through the CvCapture structure So we create a CvCapture structure
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Then we check if we were able to get exclusive control of the camera. If not, quit.
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Now, well display a live stream in the window we had created Because the getSnapshot() function returns a single frame, we need to take snaps regularly
So it goes into the dowhile loop
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
If you try compiling the program right now, youll get an error
getSnapshot() comes after the main function So the compiler doesnt know if it exists Hence we need the so called prototype
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
The cvCaptureFromCam works only for some supported cameras DirectX is a much bigger library and supports almost all cameras that exist
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
First, download the Microsoft Platform SDK for Windows Server 2003 R2
http://www.microsoft.com/downloads/deta
ils.aspx?familyid=E15438AC-60BE-41BDAA14-7F1E0F19CA0D&displaylang=en
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
ils.aspx?FamilyID=77960733-06e9-47ba914a-844575031b81&DisplayLang=en
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
ils.aspx?FamilyID=8af0afa9-1383-44b4bc8b-7d6315212323&DisplayLang=en
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
nput/
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
nput/
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
These are external libraries. So we need to tell Visual Studio where to find the required external files
So we follow steps similar to the OpenCV ones.
The VideoInput package comes with lots of sample code, so you shouldnt have much problem
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
tShow
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
For real time, you need to process captured frames as quickly as possible So we use a loop of some kind (usually a dowhile) Within the loop, you do the following tasks:
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
1: 2: 3: 4: 5: 6:
Capture an image Preprocessing it Process the image Take a decision Move the bot (with feedback) Go to Task 1
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Quite obvious
You need an image Only then can you process something
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Once you have the image, you need to enhance it with pre-processing Increase contrast, reduce noise, smoothen it out, etc
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
With the pre-processed image, you figure out the location of each object in the arena
Use moments, contours, or anything else Thresholding, morphology, etc are helpful here
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
With the location of each object, you can decide what to do next Bot: Should I go to the red ball because its the closest? Or should I go to the green ball because it has the maximum number of points
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Once decided where to go, you make the robot move You must include feedback in this step itself (maybe another dowhile loop)
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(have all the balls been potted?) If required, only then go to Task 1 Blindly go to Task 1 (the decision module will tell the bot when to stop)
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
For the simplest feedback mechanism, you use pixels Everything is measure in terms of pixels: distances, coordinates, etc Angles are usually represented in radians (cos, sin, etc work well with radians)
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
First, you need to decide your coordinate system And you need to stick with it throughout your code Heres a coordinate system I used several times
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
This is a top-down view of the arena, just like the camera sees it
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
So all your calculations MUST use this particular coordinate system In particular, you must make sure all your angle calculations are consistent And that they cycle through 359-1 degrees perfectly
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
TurnBotToAngle(angle)
Turns the bot to angle degrees of the
coordinate system This would be the very basic feedback function Within this function, you have a loop This loop keeps running as long as the bot isnt oriented at angle degrees Youll take snapshots within this function as well
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
MoveBotToPosition(x,y)
Moves the bot until it reaches the
coordinates (x,y) in the image If required, you can also put a call to TurnBotToAngle (to orient the bot to move) Again, theres a loop which keeps running until the bot reaches the desired position And youll need to take multiple snapshots to check where the bot actually is
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Reminder: All x, y and angle weve talked about are in the image, in terms of pixels
We have no idea how they relate to physical distances and angles
But were sure that they are proportional to physical distances and angles
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Though, you CAN calibrate your camera and actually figure out physical distances
For example, 5pixels = 3cm
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Use classes and structures as much as possible. And you dont need to know OOPs to use them. They really simplify your work, and even make the code more readable
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
What weve described requires that the bot stops and then checks if the angle is correct or not, etc Try working on something which checks the bots angle without stopping the bot
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Working in the YUV space (this is what cameras use so thresholding in YUV itself will eliminate processing time consumed by YUV to RGB conversion)
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Kalman filters: If a bot is static, still its position might be calculated as different for different frames.
If you use Kalman filters, you can smooth out the position data and get precise positioning and angle data
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
You now know enough to participate in image processing based competitions All this knowledge can even serve as a start point for further studies in image processing
Enjoy!
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
Utkarsh Sinha
Ajusal Sugathan
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha
(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha