Kendall Vision Symposium

2015
In-Sight Basic Vision Tools Lab
  

Table of Contents

Lab 1: Histogram

Lab 2: Edge Detection

Lab 3: Blob Extraction

Lab 4: Pattern

 
 Histogram Tool Lab
Introduction:

A histogram is the most basic of machine vision tools as it uses only the intensity value of pixels to
perform whatever inspection is needed. Other tools employ a histogram as part of their overall
function. The ExtractHistogram tool is the basis for all other histogram tools as these tools reference
the ExtractHistogram to perform their function. Using the ExtractHistogram tool in specific ways can
produce functionality like a gray scale pixel counter. This lab will explore the functionality of the
ExtractHistogram and related tools.

Start:

On the desktop click on the icon labeled Launch Histogram Lab. This will open In-Sight Explorer,
set the emulator to the desired system and setup the image playback that will be needed for this lab;
this will take about thirty seconds. In the filmstrip control at the bottom of the window you will see
only one image for this lab.

Place your cursor in an open cell and click the left mouse button to select the cell. Next hover over
the Palette label on the right side of the window to open the tool palette. On the Functions tab,
under Vision Tools click the triangle in front of the Histogram heading to expand the menu; find the
tool ExtractHistogram (Fig. 1) and either double click on it to insert it into the cell you selected earlier
or click on the tool and drag and drop it into the desired cell in the spread sheet. The tool property
sheet will open.

Fig. 1 - ExtractHistogram Tool

1
The first step will be to define the region of the tool. In the property sheet click on the Region label
and then on the Edit Region hot key in the tool bar of the property sheet (see red oval Fig. 2).

Fig. 2 - ExtractHistogram Region Property

When the hot key is clicked the property sheet and spread sheet grid will disappear and you should
see something similar to the image below (Fig. 3). When you place your cursor inside the rectangle
it will appear similar to that in the image; if you click and hold the mouse button you can drag the
region to the desired location.

Fig. 3 - Moving the Region

2
When the region has been placed over the black rectangle (similar to what is shown below in Fig. 4),
you can adjust the size as desired. Placing your cursor on one of the corners will cause the cursor
to appear as is shown below (Fig. 4). By clicking and holding the left mouse button and dragging
you can change the height or width of the region. A similar change can be made by placing the
cursor over one of the sides, except that you will only be able to change the size in that direction.
On Figure 3, you will notice a curved arrow at the bottom of the region; this can be used to rotate the
entire region. The curved rectangle at the right side can be used to introduce curvature to the region
to the point of turning it into an annular region. You won’t be using either of these functions for this
lab. When you have the region in the location and sized similarly to what is shown in Fig. 5 press
the Enter key on you keyboard to accept the changes and return the property sheet.

Fig. 4 - Resizing the Region

Fig. 5 - Set Region

3
 Fig. 6 - Extract Histogram Property Sheet
The properties for the ExtractHistogram tool will be described below.

Image: This property specifies the image upon which the tool will function; the default is A0
which is the acquired image but it could also reference an image filtering function, a calibration or
other select tools.

Fixture: This property allows reference to Row, Col and Angle values from a tool used for
software fixturing so that the histogram tool could be localized to a specific region of the image.

Region: Allows setting the inspection region for the tool (which you have already done).

External Region: Allows referencing to a cell with an EditRegion or Region structure to be

used as the region for the tool. When used, the Region property is disabled.

Show: This property allows selection of what will be shown in the image. Options are hide
all, input graphics only and show all: input and chart. As this tool only extracts information
from the image there are no results markings.

4
When the property sheet is open and the tool region is positioned in the desired place a graph will
be visible at the bottom of the window similar to that shown below (Fig. 7).

Fig. 7 - Histogram Chart

This chart shows the distribution of pixels at all intensities from 0 to 255 (8 bit gray scale). The
vertical axis automatically scales to the maximum number of pixels at any one intensity value. The
horizontal scale shows the intensities from 0 to 255. Looking closely you will see a vertical line on
the chart with the intensity marked below it, in this case 104 (see red arrow in Fig. 7). This is the
automatically calculated threshold based on the number and distribution of pixels at all intensities
within the region. This threshold is the dividing line between what is considered dark and what is
considered light. As the image within the region is made up of very distinct bands of intensity we
see very narrow, discrete peaks that are distributed evenly across the spectrum of intensities. As
the region does not go all the way through the bands on the ends we see slightly smaller peaks at
the extremes that would signify a lower pixel count at those intensities.

Click OK at the bottom of the property sheet to close it. Information is automatically inserted into the
spread sheet as shown below (Fig. 8).

Fig. 8 ExtractHistogram Inserted Information

In the leftmost cell is the Hist structure and the inserted information is described below.

Thresh: This is the automatically calculated threshold described above and uses the function
HistThresh().

Contrast: This is a calculation based on the difference between the mean gray scale value
above and below the calculated threshold. The function HistContrast() is used to calculate this
value.

DarkCount: This is the number of pixels present in the region whose intensities are between
zero and the calculated threshold minus one. This uses the function HistCount().

BrightCount: This is the number of pixels present in the region whose intensities are
between the threshold and 255. This also uses the function HistCount.().

5
Average: This is a calculation of the average intensity of all pixels in the region. The function
HistMean() is used to calculate this value.

While this may be interesting, in a manner, what does this really do for us? The ExtractHistogram
tool primarily analyzes intensity information in a region and it is only by manipulating the available
data that this tool becomes useful. As was mentioned earlier this tool could be used to create a gray
scale pixel counting tool; the remainder of this lab will be concerned with building this tool.

The HistCount tool that is used in the ExtractHistogram output to count the dark and bright pixels will
be the main function used in our pixel counting tool. The structure of the tool is as follows.

HistCount(Structure,FirstBin,LastBin,Color)

In the function above Structure refers to the Hist structure inserted into the spread sheet when an
ExtractHistogram tool is created. In the function you would make a reference to the cell that
holds that structure.

FirstBin refers to the intensity value at the low end of the intensity range you want to
consider.

LastBin refers to the intensity value at the high end of the intensity range you want to
consider.

Color is an optional argument that would only be used on color systems when referencing an
ExtractColorHistogram tool. You will not need to add anything for this argument.

To build our pixel counting tool we then need an ExtractHistogram tool, the HistCount function and
some values for the intensity range. As this might be something that you would use on a regular
basis so it would be helpful to make it easy to use.

The first step will be to create an EditRegion function that will be used for several purposes. Click
on an open cell in the spread sheet and type “EditRegion(“; a property sheet will open. In the Name
property, type the name PixCt and click OK at the bottom of the property sheet to close it. An
EditRegion tool will be inserted into the spread sheet as will other information similar to the
image below (Fig. 9). Place your cursor in the cell with the Row label, click and hold the left mouse
button and drag to select all the cells through to the value underneath the Curve label and press the
Delete key on your keyboard. This will help to keep your tool compact.

Fig. 9 - EditRegion Tool

6
In the cell next to the EditRegion function insert an ExtractHistogram function. When the property
sheet opens click on the ExternalRegion label and then the AbsoluteReference hot key in the
property sheet tool bar (see red oval in Fig. 10). The property sheet will disappear; place your
cursor in the cell with the EditRegion function and double click the mouse button. When the property
sheet reappears there will be a cell reference in the value field of the External Region property.
Click OK at the bottom of the property sheet. Delete the inserted information from this function as
well so that you are only left with the Hist structure.

Fig. 10 - ExtractHistogram External Region

In the cell next to the Hist structure type “EditFloat(“; a property sheet will open with a Min and a Max
property. Leave Min at its default of zero and set Max to 255; click OK at the bottom of the property
sheet to close it. Right click in this cell and select Copy; place your cursor in the cell to the right of
this one and right click and select Paste so that you have two identical functions. In the cell above
the first of these type ”’Low Limit” making sure to include the single quote mark before the letters;
this signifies that it is text and not a function. Above the second one type “’High Limit”.

In the cell next to the second EditFloat function type “HistCount(“; this time a property sheet does not
open but the grid disappears from the cell you are in as well as the cells in that row to the right of it.
The cursor will be blinking next to the open parenthesis awaiting you to enter the rest of the
arguments of the function. You have two options at this point, you could either type in cell
references to the appropriate cells or you can simply click on the desired cells with the desired
information. Using the latter method, as you move the pointer cursor outside the cell with the
function in it you will see that it has a small spread sheet grid icon to the lower right of it and as you
point to various cells a tool tip comment will appear saying “Click to insert reference” followed by the
row and column coordinate of the cell to which the cursor is pointing. Place the cursor in the cell
containing the Hist structure and click the mouse button; you will see the row and column coordinate
of the cell appear in the function. Type a comma after this cell reference. As you move the pointer
cursor you will again see that it is suggesting to make a reference to another cell; click in the first of
the two EditFloat functions you entered; again the cell reference will appear in the function. Type
another comma and make the final cell reference to the second EditFloat function but this time
double click in the cell. This will complete the function (adds in the closing parenthesis) and you
should see an integer value in the cell. In the cell above this value, type “’Pixel Count”.

7
You now have a gray scale pixel counting tool. By clicking on the EditRegion function, the region of
the histogram tool will open and you can position and size it according to your needs. By entering
values into the EditFloat controls you select the gray scale range for the intensity of pixels you want
to count. But how do you know which pixels are being counted. Time for some visual feed back.

In the cell next to the HistCount function type “PointFilter(“; a property sheet will open (Fig. 11). This
is an image filter function of the “Point” type. This means that the intensity value of each pixel in the
image is changed based on its intensity alone and the operation of the filter selected. For the
Operation property click on the drop list arrow to expose the list of choices, scroll to the
bottom of the list and select Threshold Range. This operation makes all pixels with intensities within
the set intensity range black and those outside the set range, white.

Make a relative reference for the Minimum property by clicking the label and then selecting the
RelativeReference hot key in the property sheet tool bar (see red oval Fig. 11). As before,
when this button is clicked the property sheet will disappear and you will be able to select a cell in
the spread sheet; make a reference to the first of the EditFloat functions by double clicking in that
cell. When the property sheet reappears do the same with the Maximum property and make a
reference to the second of the EditFloat functions. For the Show property make an absolute
reference to the cell directly above the cell holding this function; this will be handled soon.

We still need to define the Region property for this tool and we would really like it to be the same as
the region used for the ExtractHistogram function which is the EditRegion function. However this
tool does not have an ExternalRegion property. This means you will need to handle this in a manner
used, before tools had ExternalRegion properties.

Fig. 11 - PointFilter Property Sheet

8
Start by clicking on the plus sign in front of the Region property to expand the values that make up
the region. You will see labels and values for Row, Col, High, Wide and Angle. References can
be made for each of these to cells in the spread sheet; however you deleted (as instructed) these
values that were inserted when the EditRegion was created. These values still exist in the
EditRegion structure, you just have to extract them.

First click the label Row and then the AbsoluteReference hot key in the tool bar and double click in
the cell with the EditRegion function. When the property sheet reappears the Region label will be in
red text, the Row label will have a red background and there will be a red circle with an exclamation
mark in it next to the OK button at the bottom of the property sheet. Do not panic. Click in the value
field of the Row and press the Home key on the keyboard to move the cursor to the beginning of the
field. Type “GetRow(“ and then move to the end of the field (after the cell reference) and type a
close parenthesis; press tab on the keyboard; all of the red will disappear. You will repeat this
process for all of the values associated with the Region parameter. Follow the table below for each
of the other values where CellRef is a reference to the cell with the EditRegion function.

Col GetCol(CellRef)
High GetHigh(CellRef)
Wide GetWide(CellRef)
Angle GetAngle(CellRef)

9
When done you should have something like the image below (Fig. 12)

Fig. 12 - PointFilter Complete

Click OK at the bottom of the property sheet to close it when all settings have been made. Click in
the cell above the filter just created and type “CheckBox(“; a property sheet will open with only one
property being Name. Enter the name Show for this property and click OK at the bottom of the
property sheet. You have already made the reference to this control earlier so nothing more needs
to be done with it. When the check box in checked the result graphics for the filter will appear.
Pixels whose intensity is within the set region will appear black. What if the background is black?
This will not be an effective visual feedback. The answer is another filter.

10
Place your cursor in the cell with the check box, click and hold the mouse button and drag to select it
and the filter just created; release the mouse button. Place the cursor within the selected region and
right click and select Copy. Now put the cursor in the cell to the right of the check box, right click and
select Paste; another check box and filter will be inserted into the spread sheet though the cell where
the image filter should be will likely have #ERR in it. Double click in this cell to open the property
sheet for the PointFilter function. Notice that you have created a replica of the previous filter with
one exception, the cell references to the two EditFloat functions point to one of the EditFloat
functions and the other points to the cell with the HistCount function. This is the reason for the
#ERR in the cell. Click in each value and type the number zero to clear the reference. From
the Operation property drop list select Invert and finally for the Image property make an absolute
reference to the first PointFilter. The Invert operation does exactly that, so that what was black
would now be white and what was white would now be black. If the background is dark you can use
the second filter to make the counted pixels appear white; if the background is light you can use the
first filter to make the counted pixels appear black. By checking one of the two check boxes you can
mark the image appropriately. Now your pixel counting tool is finished. Try it out by setting the
region to fit across two adjacent gray scale bands in the image and adjusting the lower and upper
limits to find the pixels in one band but not in the other. Now reverse it. Play with your image
marking to see which works best for the background in question.

The point of creating a user friendly tool was so that it could be reused, but how do you make this
tool available to use in other programs? Place your cursor in the cell above the EditRegion function,
click and hold the mouse button and drag to select all of the cells in your pixel counting tool. Place
the cursor in the selected region and right mouse click . From the menu that appears hover over
Snippet to expose the sub-menu and then select Export (Fig. 13). A Windows navigation dialog will
open and the default location is the Snippet folder for the currently opened version of In-Sight
Explorer. You can create a folder for your own snippets within this folder, put it in one of the
existing folders or navigate to some other folder on the computer. The advantage to putting it into
the Snippet folder is that it would then be available from the Snippet tab of the tool palette. Please
shut down In-Sight Explorer when done.

Fig. 13 - Export Snippet

Conclusion:

The ExtractHistogram tool seems to be of little use until you find ways to apply the analysis it
provides. Many of the other tools use the functionality of the ExtractHistogram tool in the
background to be able to determine thresholds for the functionality of that tool. The functionality of
this tool is essential as image processing relies on knowing the intensity of each pixel in the image.

11
  

 
 Edge Tool Lab
Introduction:

Edge detection is a commonly used, basic machine vision tool. It can be used for fixturing other
tools to a part, determining the presence or absence of an object, or as part of a metrology
application. In-Sight Explorer has a variety of edge detection tools that find straight edges, circular
edges, curved edges, multiple edges, edge pairs or segments but they all function basically the same
as the FindLine tool which is the focus of this lab. There is also a Caliper tool under the heading of
edge detection tools that functions quite differently than FindLine, but it is the subject of another lab
in the advanced vision tools section.

Start:

On the desktop click on the icon labeled Launch Edges Lab. This will open In-Sight Explorer, set the
emulator to the desired system and setup the image playback that will be needed for this lab; this will
take about thirty seconds. Using the filmstrip control at the bottom of the window, scroll through the
images to see what you will be working with. Leave the first image showing.

Place your cursor in an open cell and click the left mouse button to select the cell. Next hover over
the Palette label on the right side of the window to open the tool palette. On the Functions tab, under
Vision Tools click the triangle in front of the Edges heading to expand the menu; find the tool
FindLIne (Fig. 1) and either double click on it to insert it into the cell you selected earlier or
click on the tool and drag and drop it into the desired cell in the spread sheet. The tool property
sheet will open.

Fig. 1 - FindLine Tool

1
The first step will be to define the region of the tool. In the property sheet click on the Region label
and then on the Edit Region hot key in the tool bar of the property sheet (see red oval Fig. 2).

Fig. 2 - FindLine Region Property

When the hot key is clicked the property sheet and spread sheet grid will disappear and you should
see something similar to the image below (Fig. 3). When you place your cursor inside the rectangle
it will appear similar to that in the image; if you click and hold the mouse button you can drag the
region to the desired location. The arrow along the top edge of the region shows the direction of
scan for the tool; this will be important when choosing the type of edge transition expected.

Fig. 3 - Moving the Region

2
When the region has been placed over the black rectangle (similar to what is shown below in Fig. 4),
you can adjust the size as desired. Placing your cursor on one of the corners will cause the cursor
to appear as is shown below (Fig. 4). By clicking and holding the left mouse button and dragging
you can change the height or width of the region. A similar change can be made by placing the
cursor over one of the sides, except that you will only be able to change the size in that direction.
On Figure 3, you will notice a curved arrow at the bottom of the region; this can be used to rotate the
entire region. The curved rectangle at the right side can be used to introduce curvature to the region
to the point of turning it into an annular region. You won’t be using either of these functions for this
lab. When you have the region in the desired location and the appropriate size, press the Enter key
on you keyboard to accept the changes and return the property sheet.

Fig. 4 - Resizing the Region

3
 Fig. 5 - FindLine Property Sheet

The properties for the FindLine tool are described below.

Image: This property specifies the image upon which the tool will function; the default is A0
which is the acquired image but it could also reference an image filtering function, a calibration or
other select tools.

Fixture: This property allows reference to Row, Col and Angle values from a tool used for
fixturing.

Region: Allows setting the inspection region for the tool (which you have already done).

Polarity: This drop list has choices of black to white, white to black and either; the default is
either. This sets the desired transition to be detected. Leave this at the default setting.

Find By: This drop list has choices of best score, first edge, and last edge; it specifies, along
with Polarity, which edge to find. Leave this at the default setting of best score.
4
Accept Threshold: This specifies the minimum contrast for an edge to be found. The
contrast is normalized to the histogram in the region to a range of zero to 100. Leave this set at 25.

Normalize Score: When checked, the edge score is normalized to the grayscale histogram in
the region. When low contrast is found within the region this will accentuate edges to make them
more easily detectable. When contrast is high within the region this can be unchecked. Uncheck
this feature to begin.

Angle Range: This sets the allowable variation between the orientation of the region and the
orientation of the edge before the edge score is significantly affected. Ideally the tool region would
be perpendicular to the edge to be found; of course this will not always be the case. The maximum
setting for this is ten degrees. While the tool may find edges that are rotated by more than ten
degrees to the scan direction, edge scores will be reduced; greater rotation will result in increasingly
reduced scores until no edge is found. This can be left at default.

Edge Width: This property sets how many pixels across the edge are used to determine the
location of the edge. This should be set according to the width of the edge in the image. For edges
that transition from light to dark (or vice versa) over one or two pixels, this should be set to a low
number. If the transition from light to dark (or vice versa) occurs over four or five pixels this should
be set to a higher number. This helps to insure that the edge is detected in a reasonable location.
Leave this set at default for now.

Show: This sets the graphics that will be displayed on the image. The choices are hide all,
result graphics only, input and result graphics, and show all: input, result and chart. Result
graphics are the found edge, input and result graphics are the found edge and the tool region and
show all adds the chart.

When the property sheet is open and the tool region positioned across an edge, a graph like the one
below will be displayed at the bottom of the window. The vertical axis of the graph is the normalized
score from -100 to +100. The Accept Threshold is shown by the two horizontal lines on either side
of zero as a negative to positive range. The horizontal axis is representative of the tool region with
the scale along that axis showing the length of the region in pixels. The found edges are shown as
either valleys or peaks placed to show the offset from the origin of the tool. A narrow peak or valley
represents a sharp edge and a more gradual peak or valley represents an edge that transitions over
more pixels. As the Edge Width property is increased these will become wider and may decrease in
height depending on the contrast across the edge.

Fig. 6 - FindLine Graph

5
Click OK at the bottom of the property sheet, the property sheet will close and information will be
inserted into the spread sheet (Fig. 7).

Fig. 7 - FindLine Inserted Information

In the leftmost cell is the Edges structure that holds the properties of the tool. In the following cells
are the row and column coordinates of the two endpoints of the edge; these are designated as
Row0, Col0, Row1, Col1. The last cell shows the score of the edge. This value is negative for
white to black transitions and positive for black to white transitions. The value is a reflection of how
well the edge matches the set properties and its strength. For the image used the edge is highly
contrasted and very sharp so the score is high. Which edge was found in your inspection?

While the information shown above is automatically inserted into the spread sheet it is not the only
information available from the tool. For every tool there is additional information that can be
extracted from the tool by using what Cognex terms “getters”. The name comes from the fact that
each function’s syntax begins with “Get(“. These functions can be found under Vision Data Access
in the tool palette at the right of the window and are divided into headings for the vision tools to
which they relate; be careful when applying them as not all functions can be used with all tools under
the specified heading. The help files will specify with which tools they will work.

Common additional information for edges would be its angle or perhaps the midpoint of the found
line. If you want, in the cell next to the value for the score you can type “GetAngle(“. After typing
this, the function will be opened for editing and when you move the cursor you will see a small
spread sheet icon by it; this indicates that the function is looking for a reference to another cell and
in this case the Edges structure. If you place your cursor in the cell with the Edges structure and
double click you will make a relative reference to it and complete the function. If you hold the shift
key on the keyboard as you double click you will make an absolute reference.

Open the property sheet for the FindLine tool by double clicking on the Edges structure. You will be
dragging and dropping property controls into the spread sheet so that you can change them without
having the property sheet open. Starting with the Show property place your cursor on the Show
label, click and hold the left mouse button and drag the property toward the spread sheet. As soon
as the cursor leaves the bounds of the property sheet, the cursor will appear as in the image below
(Fig. 8). The plus sign means that the control could be placed in that cell; if the cell were occupied
there would be a circle with a line through it. As what is inserted into the spread sheet will typically
take more than just one cell, if an adjoining cell, into which information would be inserted, was
already occupied you would also see the circle with a line through it. Place the cursor in the cell four
cells above the Edges structure and release the mouse button. A label will inserted into the spread
sheet with a drop list control below it.

Fig. 8 - Drag and Drop Cursor

6
Do the same thing with the properties for Polarity, Find By, Accept Thresh, Angle Range and
Edge Width. When done you might have something that looks like the image below (Fig. 9).

Fig. 9 - FindLine Property Controls

With the first image showing, change the polarity to white to black so that the left edge of the
rectangle is detected. Scroll to the next image and change a property so that the left edge of the
rectangle on the right is detected. You may need to adjust the size of the region so that the desired
edge is included in the region. Which property did you need to change?

Scroll to the next image with the thin line between the two rectangles. What do you need to do to
detect the leading edge of the thin line? Understand that this may take more than changing a single
property.

When you have found the leading edge of the thin line, adjust the Edge Width property to different
values. You can turn off the spread sheet grid by clicking on Overlay under the View menu. When
the grid is turned off, you can press and hold the Ctrl button on the keyboard and left click the mouse
to zoom in on the image. Zoom in enough so that the rectangles nearly fill the screen; turn the
overlay back on. As you adjust the Edge Width property observe where the edge is found. What
happens when the Edge Width is set to about five or more? Why does this happen? Use the graph
to help understand.

Scroll to the fourth image, the first image of a dark part on a dark background. Adjust the properties
to detect the edge at the left side of the part (either white to black or black to white transition). Scroll
to the fifth image where the contrast is diminished; is the edge still found? What do you need to
change to make the edge detectable. Scroll to the sixth image. Can you make changes to detect
the edge in this image. If you have not done so already, open the FindLine property sheet and
check the box for Normalize Score and pay attention to the graph at the bottom of the window when
you do this. This checkbox can allow low contrast edges to be detected but by unchecking it you
can remove noise that may be in the image that you do not want to detect.

Looking next at angle, set the Angle property to zero if you have changed it at some point and scroll
to the seventh image. Notice that the angle value you added in has not changed even though the
part is clearly rotated in reference to the tool region. Also notice that the line is still vertical rather
than slanted. With the Angle property set to zero you are not allowing any tolerance of rotation.
Start changing the Angle property and observe the reported value as you increase the tolerance
value. Does the value change linearly? Scroll through the next three images and observe how the
reported value changes; also observe how the Score value changes. When you reach the last
image what happens? Why does it happen? What can you change so that the edge is found?

You are formally finished with the lab; you can continue to investigate edge tools if you so desire.
When you are finished please close In-Sight Explorer.

7
Conclusion:

Edge detection tools can be very useful but are also limited in their capability. As you have seen
there is a limitation to the angle of the edge that can be detected but there is also sufficient strength
in the algorithm and its adjustable properties to be able to deal with variations in angle, contrast,
edge sharpness and transition type.

8
 Blob Tool Lab
Introduction:

Particle analysis is one of the more basic concepts in machine vision. In In-Sight Explorer that
concept is realized in the ExtractBlobs tool. A blob is a collection of connected pixels of similar
intensity; it does not need to be a specific size nor do the pixels need to be in a specific spatial order
(a pattern). The ExtractBlobs tool can be used to detect the presence of one or more objects, count
objects or identify objects by specific physical attributes.

This lab will show the use and setup of the ExtractBlobs tool, the information provided about found
blobs and how that information relates to the found blob. There are two additional blob tools,
FindBlobs and SortBlobs, both of which reference an ExtractBlobs tool. The FindBlobs tool
uses information about the found blobs to score each according to how well they match the user set
properties. The SortBlobs tool creates an order of the found blobs based on a choice of preset
possible methods; the default order from the ExtractBlobs tool is by area in descending order.
Neither of these last two tools will be covered in this lab, but feel free to explore on your own.

Start:

On the desktop click on the icon labeled Launch Blob Lab. This will open In-Sight Explorer, set the
emulator to the desired system and setup the image playback that will be needed for this lab; this will
take about thirty seconds. Using the filmstrip control at the bottom of the window, scroll through the
images to see what you will be working with.

Place your cursor in an open cell and click the left mouse button to select the cell. Next hover over
the Palette label on the right side of the window to open the tool palette. On the Functions tab, under
Vision Tools click the triangle beside Blobs to expand the heading (Fig. 1). Either double
click on the ExtractBlobs tool or drag and drop it to the desired cell. The tool property sheet will open
(Fig. 2 on next page).

Fig. 1 - ExtractBlobs Tool

1
 Fig. 2 - ExtractBlobs Property Sheet
The next step is to set the region for the tool; in this case we will use the whole image. Click the
Region property to highlight it and then click the tool bar hot key to expand the region to the
full image (see the red oval in Fig. 2). You should see a red border around the image; this is the tool
region.

The various properties are described below.

Image: This property points to an Image structure on which the tool will operate. The default
is A0 which is the acquired image. This could also reference an image filter tool that would process
the acquired image prior to inspection by the ExtractBlobs tool.

Fixture: This property allows reference to Row, Col and Angle values from a tool used for
fixturing. As you have set the region to the entire image we do not need any fixture reference.

Region: Sets the region size and location of the tool, in which blobs would be detected.

2
External Region: Allows referencing to a cell with an EditRegion or Region structure to be
used as the region for the tool. When used the Region property is disabled.

Number to Sort: The number of blobs you would like to find within the region; by default this
is set to 1. Entering a number up to ten will automatically insert information for the corresponding
number of blobs in the spread sheet; if a number greater than ten is entered, only ten lines of
information will be automatically inserted into the spread sheet; additional lines can be added by
providing an index number and copying and pasting the information. Entering a value of zero in this
property will allow the tool to find as many blobs as possible that fit the other properties and the only
information inserted into the spread sheet will be the total number of blobs found. For now leave this
at the default setting.

Threshold: This property sets the binary intensity threshold between what is considered
black and what is considered white. This can have a value between 0 and 255 for the eight bit gray
scale intensity range used in the system. A value of -1 dynamically sets the threshold based on the
histogram found within the tool region; this changes as the intensity of pixels within the region
change. For now leave this at the default value.

Fill Holes: When checked any holes within the blob that are the same color as the chosen
background will be included in the information provided about the blob. The holes will not be
graphically filled in, in the image.

Boundary Blobs: When checked, blobs that touch the boundary of the region will be
included. When unchecked, blobs that touch the boundary are excluded.

Color: Blob: This specifies the expected color of the blob and the choices are black, white,
and either. Choosing either would allow either black or white blobs to be detected. For this lab set
this to black.

Color: Background: This specifies the expected color of the background and the choices are
black and white. In addition to being a specific choice this property also affects connectivity.
Since pixels are distinct structures, each pixel is surrounded by eight other pixels, four on connecting
sides and four on the corners. When the background color is set to the opposite color as the blob
color, the intensities of all eight pixels around any given pixel are considered for determination of
being connected to the blob. If the background and blob color are set to the same color only the
intensities of the pixels on the sides of a given pixel are considered; the pixels at the corner are
ignored. This could potentially affect the size of blobs where there is significant variation around the
edges of the blob. For this lab set this to white.

Area Limit: Min: This sets the minimum size allowed for consideration as a blob.

Area Limit: Max: This sets the maximum size allowed for consideration as a blob.

Show: This defines what markings will be shown on the image. Set this to result graphics
only.
When a blob is found it will be bordered in a green line that follows the contour of the blob. With the
settings suggested do you find a blob (an area bordered in green)? If not why do you think nothing
has been found.? Try setting the Area Limit: Max to 200000.000. Now do you find a blob? Click
OK at the bottom of the property sheet.
3
When the property sheet closes information about the blob will be automatically inserted into the
spread sheet (Fig. 3).

Fig. 3 - ExtractBlobs Automatically Inserted Information

The information inserted is described as follows.

Index: This specifies the index of each found blob; as only one blob was selected to be
found, the information is only reported about one blob. If more blobs were specified to be found
there would be multiple lines (up to ten automatically) and each line would have its own index
number; like members of an array.

Row: This is the row value of the centroid of the found blob. A centroid is the location of the
geometric center of the blob.

Col: This is the column value of the centroid of the found blob.

Angle: This is the angle of the major axis of the found blob relative to the image coordinate
frame. The major axis points in the direction of the greatest distribution of pixels that are furthest
from the centroid. Blobs that are more symmetrical will have a more ambiguous major axis. A circle
would have the most ambiguous major axis as the distribution of pixels from the centroid would be
highly consistent in all directions.

Color: This specifies whether the blob is black (0) or white (1).

Score: This rates how well the blob matches the criteria for being a blob. With the
ExtractBlobs tool the only criteria would be color and size; if a blob meets both of those
criteria the score will be 100 so all found blobs will have a score of 100. This becomes more
important when using the FindBlobs tool which references an ExtractBlobs tool and uses various
criteria set by the user to determine how closely a blob fits a desired set of criteria.

Area: The number of pixels which are included in the blob.

Elongation: This specifies the eccentricity of the blob. Eccentricity is a measure of how
evenly the pixels are distributed from the centroid. A circle would have a very low eccentricity, an
oval a slightly higher eccentricity and a long narrow rectangle would have a much higher
eccentricity.

Holes: The number of areas within the bounds of the found blob that match the background
color, as determined by the threshold setting (see Fill Holes property).

Perimeter: The distance in pixels around the outer border of the blob.

Spread: This value is similar to Elongation but is calculated differently. This relates more to
the density of a blob. A circle would have a low spread but a washer would have a higher spread;
both would have a low value for Elongation.
4
In the cell just to the right of the value for spread, type “PlotCross(“; a property sheet will open (Fig.
4).

Fig. 4 - PlotCross Property Sheet

Highlight the label Cross and click the tool bar hotkey for inserting a relative reference (red oval in
Fig. 4). The property sheet will disappear and the spread sheet will be visible with no image behind
it. Place the cursor in the cell below the Row label for the found blob, click and hold the left mouse
button and drag so that a red box appears around the values for Row, Col and Angle (Fig. 5).
Press the Enter button on your keyboard; the property sheet will reappear.

Fig. 5 - PlotCross Location

Finish setting the properties for the tool exactly as shown below (Fig. 6). Click OK at the bottom of
the property sheet when done. This will create a cross at the centroid with the long axis pointing in
the direction of the major axis.

Fig. 6 - PlotCross Completed

5
Scroll through the first eight images and note the angles of the blobs, especially on the images of the
round parts. How close are the angles on these two images; compare your values to the people
next to you. On the third and fourth images the angle of the cross seems to point in the logical
direction. On the fifth and sixth images the triangle is more symmetrical; does the cross point where
you would expect? In the sixth image the triangle was rotated clockwise so that the vertex that was
at the top in the fifth image is now at the lower right. Pay attention to the other information supplied
and verify that it makes sense.

When you reach the ninth image there are now two blobs of less regular shape and different
intensities. You will likely see that the darker blob is found with the tool that you have already setup;
what if we want to find both blobs? Double click on the Blobs structure in the spread sheet to open
the property sheet. Set the Number to Sort to two (Fig. 7). Do you see a second blob found? If you
don’t see a second blob, why not? What do you need to do to find both blobs with one tool?

Fig. 7 - Set Number to Sort to 2

When you figure out what will allow both blobs to be found click OK at the bottom of the property
sheet. Notice that changing the Number to Sort to two did not add a second line of reported
information. Information is only automatically inserted into the spread sheet when the tool is
originally setup; if you make a change to this number after the property sheet has been closed you
will need to add the information manually, but this is fairly easy. In the cell below the zero index,
type the number one. Then place the cursor in the cell below the Row label for the blob at index
zero, click and hold the left mouse button and drag so that all the cells through the PlotCross
function are highlighted (Fig. 8). Either right click in the selected area and select Copy or simply type
Ctrl-C on the keyboard. Place the cursor in the cell next to the index one and either right
click and select Paste or click in the cell and type Ctrl-V. The information and PlotCross tool will
be pasted into the appropriate cells and will show the information for the second blob.

Fig. 8 - Select Information

6
Scroll through the next two images and verify that both blobs are found.

If you have time and want to try something more; see if you can find only the lighter colored blob.
This may take more than just adjusting settings in the blob tool.

At times knowing the extents of a blob can be useful and for this there is a function called
BoundingRectangle, which creates a rectangle that encompasses the perimeter of the blob.
Click in the cell next to the PlotCross tool; from the Palette at the right click on the triangle next to
the Geometry menu (just below the Vision Tools menu) to expand the heading. Click on the triangle
next to the Fit menu to expand it and double click on the BoundingRectangle tool (Fig. 9) to insert it
into the selected cell; its property sheet will open.

Fig. 9 - BoundingRectangle Tool

7
First click on the Blobs label to select it and then click on the tool bar hot key to insert an absolute
reference (red oval in Fig. 10) to the Blobs structure of the ExtractBlobs tool. Next click on the Index
label to select it and then the tool bar hot key to insert a relative reference to the index number of the
first blob (Index 0). Set the Show property to result graphics only. The Alignment property has three
choices, Blob Coordinate Frame, Image Coordinate Frame, and User Coordinate Frame. The
first option means that the bounding rectangle will be oriented according to the blob’s orientation
(angle of the rectangle will align to the blob’s major axis). The second choice will orient the
bounding rectangle to the image frame of reference; the “X” direction on the rectangle will point
straight down in keeping with row values starting at zero at the top of the image and increasing
toward the bottom of the image and the “Y” direction starting at zero at the left side of the image and
increasing toward the right side of the image. The last selection will orient the bounding rectangle to
the orientation of a calibration (assumes the ExtractBlobs tool Image property references a
calibration). This might be used if information from a blob tool were used to guide a robot to pick up
an object. You can copy and paste the tool and associated information (as you had done with the
information for the second blob) to create a second BoundingRectangle tool for the second found
blob. Scroll through the images and see each blobs associated bounding rectangle. Switch the
Alignment property to see how the orientation of the rectangle changes. When you are done
with the lab close In-Sight Explorer.

Fig. 10 - BoundingRectangle Property Sheet

8
Conclusion: Particle analysis is a cornerstone of machine vision and has a variety of
applications from determination of the presence of an object, to counting objects, to segmenting
characters in a string for Optical Character Recognition. Blob extraction is a fairly basic
implementation of particle analysis but can be very useful in a variety of applications.

9
  

 
 Pattern Tool Lab
Introduction:

Pattern matching is a commonly used tool in machine vision. It can be used to determine the
presence of an object, counting objects or for locating a part in the field of view. It is most effective
when the object to be detected is unique, as this will provide the best information about location,
scale and orientation. Using this to detect patterns that are not unique will provide questionable
results. This lab will demonstrate the setup and use of the FindPatterns pattern matching tool.

Start:

On the desktop click on the icon labeled Launch FindPatterns Lab. This will open In-Sight Explorer,
set the emulator to the desired system and setup the image playback that will be needed for this lab;
this will take about thirty seconds. Using the filmstrip control at the bottom of the window, scroll
through the images to see what you will be working with.

Place your cursor in an open cell and click the left mouse button to select the cell. Next hover over
the Palette label on the right side of the window to open the tool palette. On the Functions tab under
the Vision Tools group click on the triangle in front of the Pattern Match group to expand the group.
Locate the FindPatterns tool (Fig. 1) and either double click on it to insert it into the selected cell or
drag and drop it into the desired cell in the spread sheet. It will be inserted into the spread sheet and
its property sheet will open (Fig. 2 on next page)

Fig. 1- FindPatterns

1
 Fig. 2 - FindPatterns Property Sheet
The first step is to set the model region. Click on the Model Region label (Fig. 3) and click the
EditRegion hot key (highlighted hot key in Fig. 3) in the property sheet tool bar. When you do
this the property sheet and spread sheet grid will disappear, leaving only the image with a red
rectangle in it (Fig. 4 on next page).

Fig. 3 - Edit Model Region

2
 Fig. 4 - Editing Model Region Step 1
When you place your cursor inside the region it will resemble what is shown in the image above (Fig.
4). Clicking the mouse button and holding it will allow you to drag the region to a desired location.
Placing your cursor on one of the corner handles will change the cursor shape (Fig. 5) and allow you
to change the width and height of the region. Placing your cursor on one of the sides will also allow
you to change the size but only in that direction.

Fig. 5 - Editing Model Region Step 2

3
Adjust the region to resemble the image below (Fig. 6) and press Enter on your keyboard to accept
the region and return to the property sheet. When the property sheet reappears you should see the
part outlined in green to show the pattern edges. Repeat the above process with the Find Region; it
should appear similar to the image below (Fig. 7).

Fig. 6 - Editing Model Region - Finished

Fig. 7 - Find Region

4
Once the Model Region and the Find Region are set, the Model Settings (Fig. 8) can be adjusted.
These settings determine how the model is created and found; the properties are described below.

Fig. 8 - Model Settings

Model Type: This drop list offers two choices, Edge model and Area model. The Area Model
is based on the intensities of the pixels in the Model Region and the Edge Model is based on the
gradients of the pixels in the Model Region.

Coarseness: This specifies the size of the smallest features to be included in the trained
model. This drop list offers choices of Fine, Medium and Coarse. If your model has small
features that need to be included in the model to make it unique then use the Fine setting. For
many patterns with larger features the default setting of Medium is usually sufficient.

Accuracy: This specifies the compromise between speed and the accuracy with which the
pattern is found. This drop list offers choices of Accurate, Medium and Fast.

Offset Row: Specifies in pixels by how many rows the location of the centroid is moved from
its nominally found location. This might be used in a robot guidance application to change the pick
point of an object from its nominally reported location.

Offset Column: Specifies in pixels by how many columns the location of the centroid is
moved from its nominally found location. This might be used in a robot guidance application to
change the pick point of an object from its nominally reported location.

Force Train: Checking this box forces retraining of the model. Leaving it checked will retrain
the model with each image acquisition. Leave this unchecked.

Patterns: This property allows reference to another FindPatterns tool to use its model
information for search purposes. When set to zero the model trained in the local FindPatterns tool is
used.

You can leave all of these properties at the default values.

5
 Fig. 9 - FindPatterns Property Sheet
The rest of the properties are described below.

Image: This is the image upon which the tool will work. The default is A0, the acquired
image, but it could also reference any other image structure.

Fixture: This allows reference to the Row, Col, and Angle values of another tool to fixture it to
a specific relative location in the image.

Number to Find: Specifies the number of instances of the model to be found in the image.
Specifying a number up to ten will automatically insert the same number of lines of information into
the spread sheet; specifying more than ten will only insert ten lines. If more are to be found then
additional lines of information can be added by copying and pasting.

Angle Range: Specifies the rotational tolerance before the match score begins to be affected.
Instances of the object may be rotated by more than this angle and still be found but the reported
angle may not be accurate. This value represents the positive and negative rotation allowed based
on the rotational angle of the model.

Scale Tolerance: This checkbox allows an amount of variation in the scale of the part before
the match score is affected. When unchecked no scale variation is tolerated though an instance
may still be found.

Thresh: Accept: This specifies the minimum accepted match score to report a found instance
and is one of the factors in finding matches and assigning a final match score. By lowering this
value instances may be found with a match score above the original setting, as the instance was
ignored in the initial search for potential matches.

6
Thresh: Confuse: This specifies the maximum score for a pattern in the image that is not a
true instance of the model. This aids in knowing which possible instances to examine and which to
safely ignore.

Timeout: This sets a time limit for searching for a match before #ERR is returned. Setting
this value to zero removes any time limit.

Show: Specifies what markings are displayed in the image. The choices in this drop list
control are hide all, result graphics only, and input and result graphics. The result graphic is a
cross at the centroid (geometric center) of the pattern pointing in the direction of orientation based
on the orientation of the trained model. The input graphic is the search region.

Leave all of the properties at their defaults for now.

It is possible to drag and drop properties from the first tab of a property sheet into the spread sheet
so that the controls can be adjusted without needing the property sheet open. You will do this with
several of the properties so that you can view the data and make changes with the property sheet
closed. Click on the Show property and hold the mouse button down and begin to drag it into the
spread sheet; when the cursor moves beyond the bounds of the property sheet the cursor will have
a plus sign in the lower right corner (Fig. 10) if the control can be placed in that cell. If the
information to be inserted into the spread sheet would overwrite existing information in a cell you will
see a black circle with a line through it. Drag the Show parameter into the spread sheet and place it
four cells above the Patterns structure. Drag and drop the Angle Range and Scale Tolerance
controls into the spread sheet as well, leaving room for each label and control to be easily read.
Click OK at the bottom of the property sheet. When you are done you should have something
similar to the image below (Fig. 11).

Fig. 10 - Drag and Drop Cursor

Fig. 11 - Inserted Information

7
The information automatically inserted into the spread sheet is described below.

In the left most cell is the Pattern structure that holds the properties of this tool. In the cell next to it
is the Index which specifies the instance of a found match. As only one instance was specified in
Number to Find, only one line has be inserted with an Index of zero. If more instances were
specified to be found, more lines (up to ten) would be automatically inserted with an index number
for each (this happens only when the property sheet is initially opened; after it has been closed for
the first time, no information will be automatically inserted). The Row, Col and Angle specify the
location of the centroid and orientation of the found instance in the image. Scale specifies the size
of the found instance in relation to the trained model. The Match Score is the calculated value of
how closely the found instance matches the trained model. The algorithm for calculating this value
is not specified and relies on multiple properties.

Set the Show drop list to result graphics only and scroll through the images to see in which images
instances are found. Pay close attention to the reported angle and match score as well as the
orientation of the result graphic cross (direction the arrows point in reference to the notch at the top)
especially in relation to the first image (Fig 12). In image PART004.bmp (the name will appear in
the bottom left corner of the window) the part has been flipped over.

Fig. 12 - First Image

8
Now change the Angle Range parameter to 60 and scroll through the images again. What
differences do you observe? Pay close attention to the result graphic orientation on images
PART003.bmp and PART007.bmp. What happens if you change the Angle Range to 90? On
image PART008.bmp which of the two parts is found?

Now check the Scale Tolerance check box and scroll through the images again; observe the
orientation angle, match score and orientation of the result graphic. When you are done
investigating please close In-Sight Explorer.

These same images are used with the PatMax Lab in the advanced vision tools and it is capable of
finding an instance of the model in all of these images.

Conclusion:

The FindPatterns tool can be useful for detecting the presence of an object, counting objects or for
use in software fixturing. As you have seen, proper setting of the properties is critical in producing
accurate position and orientation information. A pattern that is unique will produce more reliable
results than one that is highly symmetrical or is very generic.

If using it for software fixturing where position and orientation of the tools to be fixtured is critical,
then using only one FindPatterns tool may not be sufficient. Using two FindPatterns tools and using
the PointToPoint function to calculate the distance between the centroids will result in the definition
of a line; using the midpoint and angle of this line will provide much more accurate fixturing
information.