TheFundamentalsof

MachineVision

DavidDechow
StaffEngineer,Intelligent
Robotics/MachineVision
FANUCAmericaCorporation
TheFundamentalsofMachineVision

INTRODUCTIONANDOVERVIEW
WhatisMachineVision
TheMachineVisionMarket
IndustrialUsesofMachineVision
IntroductionandOverview
WhatisMachineVision
Machinevisionisthesubstitutionofthehumanvisualsenseand
judgmentcapabilitieswithavideocameraandcomputerto
performaninspectiontask.Itistheautomaticacquisitionand
analysisofimagestoobtaindesireddataforcontrollingor
evaluatingaspecificpartoractivity.

KeyPoints:
Automated/NonContact
Acquisition
Analysis
Data
IntroductionandOverview
WhatisMachineVision

Image
Acquisition
Image
Sensors Acquisition
Optics
Lighting

Integration
and
Integrationand Applications
Applications
Results
Communications
Automation

Analysis

Components
Software Analysis
Algorithms
IntroductionandOverview
TheMachineVisionMarket
Choices
Wellover400manufacturersandsuppliers
Diverseproductofferings
Confusion
Product/componentdifferentiationsometimesisunclear
Endusers(thebuyers)oftendontunderstandwhattheyare
getting
Whatsimportant
Componentsandtechniquesneedtobebetterunderstoodat
theenduserlevel
Advancedtechnologyskillsarenecessaryforcompetent
specificationandintegration
IntroductionandOverview
TheMachineVisionMarket
GeneralPurposeMachineVisionSystems

Camera
Lens Computer
Electronics
Imager
Power/Control FrameGrabber
Signal orothersignal Digital
conversion Image

PCbasedsystem
Singleormultiplecamerasinterfacedtoacomputer,standard(Windows,Linux)
operatingsystem
Diverseimagingdevicesavailable
analog(RS170),anddigital(GigEVision,FireWire,CameraLink,USB)interfaces
IntroductionandOverview

TheMachineVisionMarket
GeneralPurposeMachineVisionSystems,continued
Camera
Lens Computer
Electronics
Imager
Power/Control FrameGrabber
Signal orothersignal Digital
conversion Image Optionalext.
computerfor
operatorinterface

VisionSystemPackage
AllinOneSystem
SmartCamera
VisionSensor

Camerasensorandproprietarycomputerinonepackage,proprietary
operatingsystem,ethernet communications
Applicationconfigurationexternaltothedevice
IntroductionandOverview

TheMachineVisionMarket
GeneralPurposeMachineVisionSystems,continued
Camera
Lens Computer
Electronics
Imager
Power/Control FrameGrabber
Signal orothersignal Digital
conversion Image

VisionSystemPackage
HybridSmartCamera

Camerasensorormultipletetheredcameraswithfullcomputer(keyboard,
mouse,monitor,USB,Ethernet),standard(Windows,Linux)orproprietary
operatingsystem,
IntroductionandOverview
TheMachineVisionMarket
Systemfeatureoverview
PCbasedsystems
Mostflexibleandpowerfulsystemdesign
Degreeofdifficultyvariesbyimplementation
Pricingvariesdependinguponarchitecture
SmartCamera/SmartSensorvisionsystem
Includestheeasiesttousesystems
Somearemoredifficulttouse
Greaterdangerofoverspecifyingcapability
Pricingvarieswidely canbequiteinexpensive
HybridSmartCameravisionsystem
Includessomeofthefeaturesofbothdependinguponproduct
Somearchitecturesmayposeintegrationchallenges
IntroductionandOverview
TheMachineVisionMarket
Camera/processorhardwareisjustanimagedeliverysystem!!
Differentiationofproductsatthehardwarelevelislimitedto:
Physicalstructureandsystemarchitecture
Singleormultipleviews?
Smartcamera distributedsystem
PCbased centralizedsystem
Customorfixedinterfaceoptions
Availablecameraresolutions
Processingspeeds
Input/outputoptions
Otherhardwareintegrationissues
IntroductionandOverview
TheMachineVisionMarket
Peripheralcomponents
Lighting
Optics
I/Odevices
FrameGrabbers
IntroductionandOverview

TheMachineVisionMarket
ApplicationSpecificMachine
VisionSolutions(ASMV)
Standalonedevices
designedfortargeted
inspectiontasks
Imaging,lighting,
optics,automation
Benefitisagenerally
uncomplicatedandeasy
touseinspectiondevice
forafocusedapplication
area
IntroductionandOverview
TheMachineVisionMarket
Targetedapplicationcomponents
Bar and2D codereaders
Othersmartsensors

ProductimagescopyrightCognex,
Siemens
IntroductionandOverview
IndustrialUsesofMachineVision

Electronics/El
0Wood ectrical

Automotive

Semiconductor

Pharma/
Medical
Device

UsersofMachineVisionby
Industry
IntroductionandOverview
Industrialusersofmachinevision
Agriculture,Automotive,Biometrics/Security,Container,
Cosmetic,Electronics/Electrical,Entertainment,Fabricated
Metal,Fastener,Food/Beverage,Glass,LabAutomation,
Lumber/Wood,MedicalDevices,MedicalImaging,
Military/Aerospace,MiscellaneousMfg.,Nanotechnology,Paper,
Pharmaceutical,Plastics,PrimaryMetal,Printing,Rubber,
ScientificImaging,Semiconductor,Telecommunications,
Textile/Apparel,Tobacco,Transportation
IntroductionandOverview
IndustrialUsesofMachineVision
Machinevisionapplicationcategories
Defectdetection
Gauging
Guidanceandparttracking
Identification
OCR/OCV
Packaginginspection
PatternRecognition
ProductInspection
SurfaceInspection
WebInspection
 Acquisition

Integration

TheFundamentalsofMachineVision
Analysis

IMAGEACQUISITION
Sensors&Imaging
Optics
Lighting
ImageAcquisition
Nothinghappensinamachinevisionapplicationwithoutthe
successfulcaptureofaveryhighqualityimage
Imagequality:correctresolutionforthetargetapplicationwith
bestpossiblefeaturecontrast
Resolution determinedbysensorsizeandqualityofoptics
Featurecontrast determinedbycorrectlightingtechnique
andqualityofoptics

Imagingissaidtocontributemorethan85%tothesuccessof
anymachinevisionapplication
Thegoalofmachinevisionimageacquisitionistocreatean
imagethatisusablebythetechnology notnecessarilyone
thatspleasingtothehumaneye
ImageAcquisition

SensorsandImaging
Allmachinevisioncameras
createanimagebyexposing
arraysofphotosensitive
materialtolightenergy
Thinkofphotonbuckets
Exposuredurationistime
limitedandtypically
adjustable
Theenergyinabucket
capturedduringan
exposureperiodbecomesa
microvoltageforthat
bucket
 ImageAcquisition

SensorsandImaging
Eachelementinacamera
sensorarrayiscalleda
pixel(pictureelement)
Theenergyvalueforeach
individualpixelisoutput
asamicrovoltageupon
acquisitionofeachimage
thevoltageultimately
determinesthecolorlevel
forthatpixel
Thepixelanddata
transferarchitecture
variesbysensortype
mostwidelyusedare
CCDandCMOS
ImageAcquisition
SensorsandImaging
Theimagingsensorarraycomesindifferentphysicallayouts
Area
Line
Sizeofthechipvarieswidelyasdoesthenumberofindividual
pictureelements(pixels)
Typicalareachipformachinevision:from.3to5+Mpix
640to2048+pixels(horizontal)
Physicalsizesfromdiag.upto1+
Typicallinescanarray:from1Kto12K+pixels
Physicalsizesfromabout15mmto90mm+
ImageAcquisition
SensorsandImaging
Imagerepresentationinthecomputer

255 255 255 105 51 41 43 49 101 255 255 255

255 255 255 116 62 44 42 57 120 255 255 255

255 255 255 112 68 41 46 58 117 255 255 255

105 110 111 109 60 42 48 61 115 112 114 108

60 68 62 57 42 41 46 41 43 49 42 41

44 42 41 46 46 42 48 44 42 42 46 42

41 46 42 48 44 42 41 41 46 43 49 42

59 54 60 59 41 46 42 46 46 42 48 46

100 120 120 115 51 41 43 49 110 116 118 105

255 255 255 118 62 44 42 57 115 255 255 255

255 255 255 121 68 41 46 58 120 255 255 255

255 255 255 100 60 42 48 61 105 255 255 255

ImageAcquisition
SensorsandImaging
ImageAcquisition
SensorsandImaging
Whataboutcolor?
Bayerfilter
Threechip
ImageAcquisition
SensorsandImaging
Imageresolution
Keyelementinvisioncomponentselection
Thesmallestfeatureresolvedbytheimager
Determinants:Whatisthesizeofthefieldofview(FOV),
andwhatistherequiredaccuracyoftheimaging.

Howmanypixels? Inspection Pixelsrequired(usually)

(forexampleonly Defectdetection Min.2x2

actualrequirement Featurelocation Min. 3x3

variesbyapplication) Featuredifferentiation Min.5x5

Gauging Subpixel resolutionmustbe

1/10th thedesiredtolerance
ImageAcquisition
SensorsandImaging
ImageResolution
Examples
Weneedtodetecta0.1(diam.)defect(highcontrast)onasurfacethatis3square.
Givengoodlightingandhighqualityoptics,whatcameraresolutiondoweneed?

Thedefectdiametershouldspanabout2pixelssoapixelmustcover0.05.Over36
therefore,theremustbe720pixels(36/0.05).Wemustselectacamerawithatleast
thatresolutionintheminoraxis(vertical) probablyonewitha1024x780sensor.

Wemustdifferentiateanemblemthatisapproximately1highrelativetoavery
similarfeatureinalowcontrastimage.Ifweuseastandardresolutioncamera(640x
480),howlargeshouldthefieldofviewbe?

Atminimum,adifferentiableobjectmustcover5pixels.Duetothelowcontrast,we
decidetodoublethatcoverageto10pixels.Thetargetpixelsizewillbe0.1(1/10),
andthefieldofviewmustbenolargerthan48(480x.01).
ImageAcquisition
Optics
Applicationofopticalcomponents
Machinevisionrequiresfundamentalunderstandingofthe
physicsoflensdesignandperformance
Goal:specifythecorrectlens
Createadesiredfieldofview(FOV)
Achieveaspecificoracceptableworkingdistance(WD)
Projecttheimageonaselectedsensorbasedonsensor
size primarymagnification(PMAG)
Createthehighestlevelofcontrastbetweenfeaturesof
interestandthesurroundingbackground;withthe
greatestpossibleimagingaccuracy
 ImageAcquisition
Optics
Considerationsforlensselection
Magnification,focallength,depthoffocus(DOF),fnumber,resolution,diffractionlimits,
aberrations(rolloff,chromatic,spherical,fieldcurvature,distortion),parallax,imagesize,etc.
Somegeometricaberrationmaybecorrectedincalibration
Thephysicsofopticaldesigniswellknownandcanbemathematicallymodeledand/or
empiricallytested
Specificationorcontrolofmostofthelenscriteriaisoutofourhands

S5 S6
Images:EdmundOptics;www.edmundoptics.com
ImageAcquisition

Optics
Considerationsforlensselection
Practicalspecificationsformachinevision:
PMAG(asdictatedbyfocallength)and
WDtoachieveadesiredFOV
Useasimplelenscalculatorand/or
manufacturerlensspecifications
Simple statetherequiredFOV,the
sensorsizebasedonphysical
selectionofcameraandresolution,
andadesiredworkingdistance
calculatethelensfocallength
Note specifiedworking
distancemaynotbeavailable
foragivenlens review
specifications Images:PPTVision;
Testyourresults pptvision.com

Alwaysuseahighresolutionmachine
visionlensNOTasecuritylens
 ImageAcquisition
Optics
Whyusemachinevisionlensesonly
Lightgatheringcapability
andresolution

Images:EdmundOptics;www.edmundoptics.com
 ImageAcquisition
Optics
SpecialtyLenses
Telecentric
Microscopestages
Macro,longWD
Zoom(cautionrecommended)

Images:EdmundOptics;www.edmundoptics.com,
Navitar;www.navitar.com
ImageAcquisition
Lighting
Scienceorart???
Correctlightingmust
Highlightfeaturestobedetectedrelativetobackground
Createrepeatableimagesregardlessofpartvariation
Incorrectlightingwillputthe
success/reliability/repeatability/easeofuseofthevision
applicationatrisk
MachinevisioncamerasandsoftwarealgorithmsCANNOT
makeupforinadequateilluminationtechniques
ImageAcquisition

Lighting
Illuminationformachinevisionmust
bedesignedforimaging,nothuman
viewing
Selectionmustbemaderelative
tolightstructure,position, Diffuse
color,diffusion Reflection
LightSource Specular
Weneedtoknowhowlight Reflection
workssoourlightselectionsare
nothitandmissguesswork
Lightisbothabsorbedand
reflectedtosomedegreefrom
allsurfaces Refraction,Absorption

Whenanobjectisclearor
translucent,lightisalso
transmitted
TransmittedLight(if
Angleofincidence=angle objectisnot
ofreflection completelyopaque)
ImageAcquisition
Lighting
Dedicatedlightingmustbeusedformachinevisionwithfew
exceptions.
Wherefeasible,LEDilluminationisthebestsource
Longlifewithminimaldegradationofintensity
Abletobestructuredintoavarietyofshapes
Maybedirectionalordiffuse
Maybestrobedatveryhighdutycyclesandoverdriventomany
timesnominalcurrentspecifications
Availableinmanyvisibleandnonvisiblecolors
Othersources fluorescent,fiberoptics
Fluorescent brightandhighlydiffusebutcanbeinconsistent
Fiberoptic glass/plasticfibersdeliveringlightfromhalogen,
tungstenhalogenorxenonsource,bright,shapeable,focused
ImageAcquisition
Lighting
LightingTechniques
Thegoaloflightingformachinevisionapplicationsusuallyis
tomaximizethecontrast(grayscaledifference)between
featuresofinterestandsurroundingbackground
Techniquesarecategorizedgenerallybythedirectionofthe
illuminationsource
Mostmaybeachievedwithdifferentsources
 ImageAcquisition

LightingTechniques
Directbrightfield
illumination
Sources:highangle
ringlights(shown),
spotlights,barlights
(shown);LEDsor
Fiberopticguides
Uses:general
illuminationof
relativelyhigh
contrastobjects;light
reflectiontocamerais
mostlyspecular

Images:CCSAmerica;www.ccsamerica.com
 ImageAcquisition

LightingTechniques
Diffusebrightfield
illumination
Sources:highangle
diffuseringlights(shown),
diffusebarlights;LEDsor
fluorescent
Uses:generalillumination
ofrelativelyhighcontrast
objects;lightreflectionto
cameraismostlydiffuse

Images:CCSAmerica;www.ccsamerica.com
 ImageAcquisition

LightingTechniques
Directdarkfieldillumination
Sources:lowanglering
lights(shown),spotlights,
barlights;LEDsorFiber
opticguides
Uses:illuminationof
geometricsurfacefeatures;
lightreflectiontocamerais
mostlyspecular
Darkfieldismisleading
thefieldorbackground
maybelightrelativeto
surfaceobjects

Images:CCSAmerica;www.ccsamerica.com
 ImageAcquisition

LightingTechniques
Diffusedarkfieldillumination
Sources:diffuse,lowangle
ringlights(shown),spot
lights,barlights;LEDsor
fluorescent
Uses:nonspecular
illuminationofsurfaces,
reducingglare;mayhide
unwantedsurfacefeatures

Images:CCSAmerica;www.ccsamerica.com
 ImageAcquisition

LightingTechniques
Diffusebacklight
Sources:highly
diffusedLEDor
fluorescentarea
lighting
Uses:providean
accuratesilhouetteof
apart

Images:CCSAmerica;www.ccsamerica.com
 ImageAcquisition

LightingTechniques
Structuredlight
Sources:FocusedLED
lineararray,focusedor
patternedlasers
Uses:highlight
geometricshapes,create
contrastbasedupon
shape,provide3D
informationin2Dimages

Images:CCSAmerica;www.ccsamerica.com,
Stocker&Yale;www.stockeryale.com
 ImageAcquisition

LightingTechniques
Onaxis(coaxial)illumination
Sources:directed,diffused
LEDorfiberopticarea
Uses:producemoreeven
illuminationonspecular
surfaces,mayreducelow
contrastsurfacefeatures,
mayhighlighthighcontrast
geometricsurfacefeatures
dependingonreflectiveangle

Images:CCSAmerica;www.ccsamerica.com
 ImageAcquisition
NondirectionalCollimated

LightingTechniques
Collimatedillumination
Sources:specialtyilluminator
(LED,Fiber)utilizingopticsto
guidethelight
Uses:highlyaccurate
backlighting,reducingstraylight,
highlightingsurfacefeaturesasa
frontlight

Images:EdmundOptics;www.edmundoptics.com,
CCSAmerica;www.ccsamerica.com
 ImageAcquisition

LightingTechniques
ConstantDiffuseIllumination
(CDI cloudyday
illumination)
Sources:specialty
integratedlighting
Uses:providescompletely
nonspecular,non
reflectingcontinuous
lightingfromallreflective
angles;goodforreflective
orspecular surfaces

Images:Siemens;www.nerlite.com
ImageAcquisition
LightingTechniques
Otherlightingconsiderations
Color
Monochromaticlight
oncoloredfeatures
Cameraresponsetodifferentcolors
Whitelightandcolorimaging
Nonvisiblecolors
Lightdegradationovertime;
componentlife,heatdissipation
Lightintensityanduniformity
Strobing
Eliminationofambientand
otherstraylight
 Acquisition

Integration

TheFundamentalsofMachineVision
Analysis

IMAGEANALYSIS
MachineVisionSoftware
GeneralMachineVisionAlgorithms
ImageAnalysis
MachineVisionSoftware
Machinevisionsoftwaredrivescomponentcapability,reliability,
andusability
Mainmachinevisioncomponentdifferentiationisinthe
softwareimplementation
Availableimageprocessingandanalysistools
Abilitytomanipulateimagingandsystemhardware
Methodforinspectiontaskconfiguration/programming
Interfacetohardware,communicationsandI/O
Operatorinterfaceanddisplaycapability
Often,systemsoftwarecomplexityincreaseswithsystem
capability
ANDgreatereaseofuseusuallyisattheexpenseofsome
algorithmicand/orconfigurationcapabilities
ImageAnalysis
MachineVisionSoftware
Adizzyingvarietyofsoftwarepackagesandlibraries
ImageAnalysis
MachineVisionSoftware
WhatsImportant
Sufficientalgorithmdepthandcapabilitytoperformthe
requiredinspectiontasks
Consider:
Speedofprocessing
Leveloftoolparameterization
Easewithwhichtoolscanbecombined
Adequateflexibilityintheprocessconfigurationtoservice
theautomationrequirements
EnoughI/Oandcommunicationscapabilitytointerfacewith
existingautomationasnecessary
Appropriatesoftware/computerinterfacingtoimplementan
operatorinterfaceasneededfortheapplication
 ImageAnalysis
GeneralMachineVisionAlgorithms
Imagetransformation/geometricmanipulation
ContentStatistics
Imageenhancement/preprocessing
Connectivity
EdgeDetection
Correlation
GeometricSearch
OCR/OCV
Colorprocessing

Machinevisionalgorithmsfrequentlyexecuteoverasubsetofthe
imageratherthantheentireimage.Theareaisoftencalledaregion
ofinterestorROI.
ImageAnalysis
GeneralMachineVisionAlgorithms
Geometricmanipulationoftheimage
Shifting
Rotating
Mirroring
Inverting
Unwrapping
Sampling
Binarization
Warping/Shifting
 ImageAnalysis
GeneralMachineVisionAlgorithms
Image/ROIcontentstatistics
Animagehistogramshowsthecountofpixelsateachgray
scalewithineithertheimageoraspecifiedregion.
Thelistcanyieldavarietyofstatisticalinformationaboutthe
imageorROI.
PixelCount

Intensity(grayscalecolor)
ImagingAnalysis
GeneralPurposeMachineVisionAlgorithms
Applicationsofhistogramstatistics
Thresholding
Imageequalization
Featurepresence/absence
Surfaceanalysis
Color/grayscaleanalysis
Lighting/camerastatus
ImageAnalysis
GeneralPurposeMachineVisionAlgorithms
ImageEnhancement/PreProcessing
Algorithmsthatchangetheimagebyphysicallyreplacing
pixelvalues
Morphology
SpatialFiltering
Applications
Reducenoise
Createbettercontrast
Extractedgefeatures
ImageAnalysis
GeneralMachineVisionAlgorithms
Extractionandanalysisof2dimensionalconnectedshapes
(blobs)
Connectivitycanbeaveryusefulandpowerfultool
Successoftendependsupontheimageandthelevelofpre
processing
Suitedforimageswithhighcontrastandconsistentcolor
levels
ImageAnalysis
GeneralMachineVisionAlgorithms
BlobStatistics
Usedtofilterandcategorizetargetshapes
Area
Perimeter
CenterofGravity,median
BoundingBox,Length,width,angle
Circularity,elongation,aspectratio
UsesforConnectivity
Objectlocation,identification
Cursorygauging
Presence/absence
ImageAnalysis
(231.651, 172.295)

GeneralMachineVision
Algorithms
Edgedetection:isolationof
localchangesincontrast
withintheimage
Edgetoolslocatewholeor
subpixeledgepoints
Usesforedgetools
Gauging
Featurepresence,
verification
Otheredgetools
Line,curveorobject
approximation
Regressions
Hough
transformation
ImageAnalysis
GeneralMachineVisionAlgorithms
NormalizedCorrelation
Findspretrainedfeatures
Othernames
Templatematching
Search
Patternmatching
ImageAnalysis
GeneralMachineVisionAlgorithms
NormalizedCorrelation
Amodelortemplateistrainedfromanexistingimage
Modelsmaybe,butrarelyare,synthesized
Themodelisstoredasacompletegrayscaleimage
ThetargetimageorROIissearchedbymathematicallycomparingthemodeltemplateatall
points
Typicaluses
Featurepresence/absence
Positionverification
Guidance

Search Process

Target Image

Trained and Stored

Model Template
ImageAnalysis

GeneralMachineVisionAlgorithms
GeometricSearch
Locatesfeatureswithinanimagebased
upongeometricstructure,feature
relationships
Alsocalledpatternmatching
algorithmhassomebranded
Thesearchpatternistrainedfroman
imageormaybesynthesized.
Geometricsearchpatternisa
mathematicalrepresentationofthe
targetobject,notanactualimage
Trainingandsearchprocessbothuse
contour(e.g.,edge)image
Processallowsforfast,reliablesearch
withfulltransformationandrotation.
ImageAnalysis

GeneralMachineVisionAlgorithms
GeometricSearch
Patternmaybeeditable,or
otherwisemanipulatedto
optimizeperformance
Typicaluses
Guidance
Featurelocation
Partverification
ImageAnalysis
GeneralMachineVisionAlgorithms
Commonchallengesinsearchalgorithms
Modelselection
Confusingscenes
ImageAnalysis
GeneralMachineVisionAlgorithms
OCR/OCV
OpticalCharacterRecognition/Verification readingor
verifyingprintedcharacters
Canbefooledbyprintvariations
Verificationisdifficultdependingupontheapplication
ImagingIssues
Consistentpresentationofthecharacterstring
Mayrequireextensivepreprocessing
 Acquisition

Integration

TheFundamentalsofMachineVision
Analysis

INTEGRATIONANDAPPLICATIONS
Integration
ResultsandCommunications
BasicApplicationConcepts
IntegrationandApplictions

Hardware
execution
Initiate Camera and (if
Acquire applicable)
Inspection
Image strobe trigger
external event
-Part
Tracking
-Multiple
results
Software -Other data
Analysis execution of
inspection
program

-Recipe
changeovers Determine
Results part status and
-Multiple communicate Process
images/lights results Result
external
-Part tracking event
IntegrationandApplications
Integration
Utilizeappropriatehandshakingwhereapplicable,particularlywith
interfacetoaPLCorexternalcontrolsystem(nottimers)
Whenpartisinmotion,interfacefromvisiondevicetotheautomation
mustbediscrete,digitalI/Otoavoidvariablelatencies

Trigger>

Acquisition<

Processing<

Result<

Othercomponents encoders/sensors/rejectmechanisms
IntegrationandApplications
Integration
MechanicalIssues
Camera/lightingfixturing
Incorporateadjustmentasneededtoaccommodatethe
applicationrequirements
Goal:placecameraonthespecificplanerequiredfor
theinspectionatapointwherethepartcanbe
triggeredforthespecifiedfieldofview,withlighting
atthecorrectpositionrelativetothecamera
Ofteneasiertoadjustpartsensorratherthancamera
Fixturesmustberobustandlockable
Somedonts.
IntegrationandAutomation
Integration
MechanicalIssues
Partpresentation
Afterlightingissues,partpresentationisthemostcritical
anddifficultpartofmostindustrialapplications
Directimpactonthelightingandimaging
Partmustbepresentedinagenerallyrepeatable
planarrepresentationrelativetothecamera.
Ifnewautomation,incorporateappropriateparthandling
toenhance/benefittheinspection
Wherepossiblemodifyexistingautomationtoconstrain
partposition
IntegrationandAutomation
Integration
Softwareandsystemconfiguration
Wherepossible,prepareapreliminaryimagingsetupusingthe
finallightingandopticsconfigurationandcapturerealproduction
images
Useproductionimagesforinspectionprogramdevelopment
Configuringabasicmachinevisioninspection
Imageacquisition/processing
Configurecameraparameters
Shutter,gain,offset,partialscanning,triggering
Strobelightcontrol
Executeaninspectionprocessuponimageacquisition
Tunetheimageifneeded
Filtering
Morphology
 IntegrationandAutomation
Integration
Configuringabasic
machinevisioninspection
Processingsteps
Locatethepartandextract
anominalorigin
Adjustregionsrelativeto
theorigin
Extractappropriatedata
withinregionsofinterest
Makedecisionsbased
uponapplication
parametersforthatfeature
IntegrationandApplications
ResultsandCommunications
Installation/testing/startup
Alwaysimplementcontrolhandshakingfirst
Imageacquisition,dataexchange,andrejecttiming/co
ordinationmakeupthelargestpartofonsitesystems
integrationwithanautomationdevice
Determinefinalcamera/lightingmountingbasedupon
liveproductionimaging
Lock/pinallmountsonceimageiscorrectforallparts
Testinspectionalgorithmsonallsampleswithrepresentative
failuremodes
IntegrationandApplications
ResultsandCommunications
DiscretedigitalI/Omayberequiredduetotiming/signallatency
Othercommunicationprotocolscanbeconsideredwhere
appropriate
Incorporateproperhandshakingwhenimplementingsignalswith
externalcontroldevices
Noncriticaldatacommunications(notrequiringdeterministic
timing),suchaspartrecipes,canbeimplementedwithserial
(RS232/422),TCP/IP,Ethernet/IP,orspecialty(Modbus,
DataHighway)interfaces.
Thesystemdesignmusttakeintoaccountinherentlatenciesin
theseprotocols
Aboutnetworkcommunications
IntegrationandApplications
BasicApplicationConcepts
Configurethedesiredinspectiontaskutilizingappropriatetools
providedbytheselectedcomponents
Typicalgeneralpurposefactoryfloorinspections
Flawdetection
AssemblyVerification/Recognition
Gauging/Metrology
Location/Guidance
OCR/OCV
Notethatvirtuallyallapplicationswillrequirethe
implementationofmultipletoolstosuccessfullyextractthe
imagedata
IntegrationandApplications
BasicApplicationConcepts
Defect/FlawDetection
Aflawisanobjectthatisdifferentfromthenormalimmediate
background
ImagingIssues
Musthavesufficientcontrastandgeometricfeaturestobe
differentiablefromthebackgroundandothergoodobjects
Typicallymustbeaminimumof3x3pixelsinsizeandpossibly
upto50x50pixelsifcontrastislowanddefectclassificationis
required
Reliableobjectclassificationmaynotbepossibledepending
upongeometricshapeoftheflaws
Machinevisiontools
Binarypixelcounting,morphology,edgeextractionand
counting,imagesubtraction/goldentemplate,blobanalysis
IntegrationandApplications
BasicApplicationConcepts
AssemblyVerification/ObjectRecognition
Featurepresence/absence,identification,differentiationofsimilar
features
ImagingIssues
Mustcreateadequatecontrastbetweenfeatureand
background
Accommodatepartandprocessvariations locateandcorrect
forpartpositionalchanges
Mayrequireflexiblelighting/imagingforvaryingfeatures
Forfeaturepresence/absence,featureshouldcoverapprox.
1%ofthefieldofview(med.resolutioncamera),morefor
identificationordifferentiation
Machinevisiontools
Edgedetectionandmeasurementtools,blobanalysis,
normalizedcorrelationandpatternmatching
IntegrationandApplications
BasicApplicationConcepts
Gauging/Metrology
Note:Therearephysicaldifferencesbetweengauging
featuresinanimageproducedbyacamera,andtheuseofa
gagethatcontactsapart.Thesedifferencesusuallycannot
bereconciled
Gaugingconcepts
Resolution,repeatability,accuracy
Subpixelmeasurement
Measurementtolerances
Resolutionmustbeapproximately1/10ofrequired
accuracyinordertoachievegauge
reliability/repeatability
IntegrationandApplications
BasicApplicationConcepts
Gauging/Metrology
ImagingIssues
Lightingtogetarepeatableedge
Backlighting,collimatedlight
Telecentriclenses
Calibration
Correctionforimageperspective/plane
Calibrationerrorstackup
Machinevisiontools
Edgedetectionandmeasurement,blobanalysis
IntegrationandApplications
BasicApplicationConcepts
Location/Guidance
Identificationandlocationofanobjectin2Dor3Dspace
Maybeinaconfusingfieldofview
ImagingIssues
Measurementtolerancesandaccuraciesasdescribedfor
gauging/metrologyapplications
Subpixelresolutionsmaybebetterthandiscretegauging
results
Forguidanceapplications,thestackuperrorinrobot
motionmaybesignificant
Machinevisiontools
Blobanalysis,normalizedcorrelation,patternmatching
IntegrationandApplications
BasicApplicationConcepts
OCR/OCV
OpticalCharacterRecognition/Verification readingor
verifyingprintedcharacters
Canbefooledbyprintvariations
Verificationisdifficultdependingupontheapplication
ImagingIssues
Consistentpresentationofthecharacterstring
Mayrequireextensivepreprocessing
Machinevisiontools
OCR/OCV,goldentemplatematch
 IntegrationandApplications
BasicApplicationConcepts
CameraCalibration
Mappingrealworldcoordinatestothecamera(observed)
pixelcoordinates
Correctionofplanarandopticaldistortion

Images:EdmundOptics;www.edmundoptics.com,
Cognex;www.cognex.com
Conclusion

Wheredowegofromhere?
DavidDechow
StaffEngineer,Intelligent
Robotics/MachineVision
FANUCAmericaCorporation
3900W.HamlinRoad
RochesterHills,Michigan48309
USA

Phone:+12482764058
Cell:+15174495173
Email:david.dechow@fanucamerica.com

www.fanucamerica.com