Image Acquisition and Preprocessing for Machine Vision Systems

661 Pages·2012·20.38 MB·English

by  Sinha

Image Acquisition
and Preprocessing for
Machine Vision Systems
P. K. Sinha
Bellingham, Washington USA
Library of Congress Cataloging-in-Publication Data
Sinha, P. K. (Pradip K.), 1947-
Image acquisition and preprocessing for machine vision systems / Pradip K. Sinha.
p. cm. -- (Press monograph ; 197)
Includes bibliographical references and index.
ISBN 978-0-8194-8202-0
1. Computer vision. 2. Identification. 3. Electronic data processing--Data preparation.
I. Title.
TA1634.S56 2012
006.3'7--dc22
2010038733
Published by
SPIE
P.O. Box 10
Bellingham, Washington 98227-0010 USA
Phone: [phone]
Fax: [phone]
Email: [email protected]
Web: http://spie.org
Copyright © 2012 Society of Photo-Optical Instrumentation Engineers
All rights reserved. No part of this publication may be reproduced or distributed in any
form or by any means without written permission of the publisher.
The content of this book reflects the work and thought of the author(s). Every effort has
been made to publish reliable and accurate information herein, but the publisher is not
responsible for the validity of the information or for any outcomes resulting from reliance
thereon. For the latest updates about this title, please visit the book’s page on our website.
Cover image courtesy of Eastman Kodak, Rochester, NY
Printed in the United States of America.
First printing
Preface
Fromanapplicationspointofview,machinevisionreferstotherecoveryofquanti-
tativedatafromdigitalimages.Thesetupforsuchrecoverytasksrequireshardware
forimagesensingandstorage,andpreprocessingsoftwaretoconvertcapturedim-
ages into image data. From an end-user’s perspective, a machine vision system
consistsof threefunctionally cognate subsystems:acquisition, preprocessing,and
application-specificanalysisandmeasurementsoftware.Thisbookcoversthefirst
twosubsystemsbypresentingsomeofthefundamentalprinciplesandcharacteris-
ticsoffront-endhardwareandderivationsofacoresetofpreprocessingfunctions.
Examples are included primarily to illustrate the use of some preprocessing func-
tions rather than to provide an account of specific applications. I have taken this
approach because algorithms and software for the third subsystem are application
specific,andthedetailsofmanyofthoseapplicationsarereadilyavailable.Incon-
trast,aunifiedaccountofimageacquisitionhardwareandpreprocessingfunctions
isnotavailableinanycomparableliterature.
In selecting the contents for this book, I excluded several areas associated
withimageprocessing,suchasmathematicalmorphology,featuredetection,shape
recognition, and texture analysis, and I give only an outline description of corre-
lation,moments,andtheHoughtransform.Allofthesetopicsarewellcoveredin
severalothertextbooks.Instead,Ichosetoprovidein-depthcoverageofthetopics
tied to image capture and spatial- and frequency-domain preprocessing functions
forreaderswhoaremigratingtothemachinevisionfieldfromotherareas,aswell
asforpracticingengineerswhoareseekingaconceptualaccountoffront-endelec-
tronicsandtherootsofpreprocessingalgorithms.
The increasing degree of “smartness” of plug-and-play cameras and frame-
grabbers allows many preprocessing operations to be performed using default
settings. However, the integration of an image-based measurement system, with
emphasis on smaller memory space and shorter execution time, requires a higher
level of awareness of design principles and associated performance parameters.
Withthiscontext,thebookcoversprinciplesrelatedtotheintrinsiccharacteristics
ofcapturedimages,thehardwareaspectsofimagesignalgeneration,andthemath-
ematical concepts of image signal processing from an algorithmic perspective of
developingpreprocessingsoftware.Inadditiontobridgingthehardware–software
gap, this book provides a basis to identify some of the key design parameters and
potential interface or processing limitations at an early stage of application devel-
opment. In this respect, topics covered in the book are suitable for students and
researchers as well as for a wide spectrum of end users, application development
engineers, and system integrators from both the image processing and machine
visioncommunities.
In building an algorithmic framework for preprocessing tasks, I adopted
an approach akin to mathematical modeling and signal analysis to provide a
xv
xvi Preface
conceptual understanding of the basic principles and their relationship to image
acquisitionparameters.Mostofthehardwaremodulesandpreprocessingfunctions
covered in this book are underpinned by an extensive collection of models and
derivations. Other than providing insight to the design features of the front-end
components (optics, sensors, and interface), this mathematical framework helps
to (1) highlight some of the underlying assumptions in the operation of imaging
hardware and (2) identify sources of various preprocessing parameters generally
assigned by default in commercial application software. With an increasing
trend toward customization and embedded design, this approach also offers a
frameworktoselectandevaluateimaginghardwareandfunctions,andtosequence
individualprocessingfunctionsinthecontextofspecificapplicationrequirements.
Furthermore, since such requirements may be subsumed in the early stages of
hardware design, selection, integration, and algorithm development, this book
offers the theoretical foundations necessary to adapt many generic results. I
hope that these design details and mathematical concepts will enable readers to
effectivelyintegratethefront-endhardwareandpreprocessingfunctionsintotheir
applicationplatform.
Although I have included a significant number of original derivations, I have
drawn much of the material from the literature. I have attempted to cite original
sources as far as possible; however, due to the continued growth of the related
subjectareasandthelargenumberofpublicationsthathostimagingliterature,my
referencelistsareincomplete.WhileIhavetakencaretoensurethatallderivations
andsupportingalgorithmicdescriptionsarecorrect,someerrorsandomissionsare
likely to be present due to the involved nature of the analytical work. I will take
responsibilityforsucherrorsandwouldappreciateitifreadersbroughtthemtomy
attention.
P.K.Sinha
December2011
Contents
Preface................................................................................................................ xv
Acknowledgments...............................................................................................xvii
AcronymsandAbbreviations............................................................................. xix
Chapter1 Introduction........................................................................................ 1
1.1 Acquisition......................................................................................................... 1
1.2 Preprocessing..................................................................................................... 3
1.3 AnalysisandMeasurement........................................................................... 5
1.4 OverviewofText.............................................................................................. 8
References....................................................................................................................... 11
Chapter2 HumanVision..................................................................................... 13
2.1 SourcesofLight............................................................................................... 13
2.2 TheHumanEye................................................................................................ 15
2.3 StimulusMeasurement................................................................................... 18
2.4 BrightnessThresholds.................................................................................... 24
2.4.1 Absolutethreshold........................................................................... 24
2.4.2 Differentialthreshold...................................................................... 25
2.4.3 Adaptation.......................................................................................... 26
2.5 Contrast............................................................................................................... 27
2.6 VisualAcuity..................................................................................................... 30
2.7 Flicker.................................................................................................................. 31
2.8 Spatio-TemporalEffects................................................................................ 32
References....................................................................................................................... 35
Chapter3 Image-FormingOptics......................................................................... 39
3.1 OpticalGlass...................................................................................................... 40
3.2 GeometricalOptics.......................................................................................... 41
3.3 LensEquations.................................................................................................. 44
3.3.1 Simplethinlens................................................................................ 44
3.3.2 Compoundthinlens........................................................................ 46
3.3.3 Thicklens........................................................................................... 48
3.3.3.1 Balllens............................................................................. 48
vii
viii Contents
3.3.3.2 Cylindricallens................................................................ 48
3.3.3.3 Condenserlens................................................................. 49
3.3.3.4 Fresnellens....................................................................... 50
3.3.3.5 Microlenses...................................................................... 50
3.3.3.6 Extensiontube.................................................................. 50
3.4 ApertureStop, f-Number,andSpeed....................................................... 52
3.5 FocusingandDepthofField........................................................................ 59
3.6 ResolvingPower............................................................................................... 60
3.7 Aberration........................................................................................................... 65
3.7.1 Monochromaticaberrations......................................................... 65
3.7.1.1 Sphericalaberrations..................................................... 65
3.7.1.2 Astigmatism...................................................................... 66
3.7.1.3 Coma................................................................................... 66
3.7.1.4 Fieldcurvature................................................................. 66
3.7.1.5 Distortion........................................................................... 66
3.7.2 Chromaticaberrations.................................................................... 68
3.8 OpticalCoatings............................................................................................... 69
3.9 OpticalFilters.................................................................................................... 70
3.9.1 Absorptionfilters............................................................................. 71
3.9.2 Interference(bandpass)filters..................................................... 71
3.10 Plasticlens.......................................................................................................... 72
References....................................................................................................................... 73
Chapter4 SceneIllumination............................................................................... 75
4.1 RadiantSources................................................................................................ 75
4.2 TypesofIlluminators...................................................................................... 77
4.3 OpticalPropertiesofTargets........................................................................ 78
4.3.1 Reflectivematerials......................................................................... 78
4.3.2 Transmissivematerials................................................................... 79
4.3.3 Absorptivematerials....................................................................... 80
4.4 LightingMethods............................................................................................. 80
4.4.1 Frontlighting..................................................................................... 80
4.4.2 Backlighting...................................................................................... 81
4.4.3 Specularillumination..................................................................... 82
4.4.4 Beamsplitterandsplitmirror....................................................... 83
4.4.5 Retroreflector.................................................................................... 83
4.4.6 Structuredlighting........................................................................... 85
4.5 PolarizationofLight....................................................................................... 85
4.6 FiberOpticLighting........................................................................................ 92
4.6.1 Lightgathering................................................................................. 95
4.6.2 Transmissioncharacteristics........................................................ 97
References....................................................................................................................... 101
Contents ix
Chapter5 ImageSensors....................................................................................103
5.1 Photogeneration................................................................................................ 104
5.1.1 Criticalwavelength......................................................................... 104
5.1.2 Absorptioncoefficient.................................................................... 105
5.2 Photoconductor................................................................................................. 107
5.3 Photodiode.......................................................................................................... 109
5.4 CMOSImageSensor...................................................................................... 115
5.5 Metal-OxideGate............................................................................................. 118
5.6 Charge-CoupledDevices............................................................................... 121
5.6.1 Charge-transferefficiency............................................................. 125
5.6.2 Blooming............................................................................................ 127
5.6.3 Illuminationgeometry.................................................................... 127
5.7 Line-ScanImaging.......................................................................................... 127
5.8 Area-ScanImaging.......................................................................................... 130
5.9 RelatedTopics................................................................................................... 136
5.9.1 Pixelsize............................................................................................. 136
5.9.2 Color-filterarray.............................................................................. 137
5.9.3 Noise.................................................................................................... 139
5.9.4 CMOSversusCCD......................................................................... 140
5.9.5 Scanpattern....................................................................................... 142
5.9.6 Pixelordering.................................................................................... 145
5.9.7 Incidentphotons............................................................................... 146
Appendix5A:SemiconductorProperties............................................................. 150
5A.1 OpticalMaterials.............................................................................. 151
5A.2 DopingofSemiconductors........................................................... 151
5A.3 CarrierGeneration........................................................................... 152
5A.4 OpticalSensors................................................................................. 154
5A.5 SemiconductorTerminology........................................................ 154
References....................................................................................................................... 156
Chapter6 ImagingHardware...............................................................................161
6.1 ImageDisplay................................................................................................... 161
6.2 LiquidCrystalDisplay................................................................................... 169
6.3 Framegrabber..................................................................................................... 173
6.3.1 Analogfrontend.............................................................................. 179
6.3.2 Timingpulses.................................................................................... 179
6.3.3 Pixelclock.......................................................................................... 179
6.3.4 Gray-leveldigitization................................................................... 180
6.3.5 Look-uptable.................................................................................... 180
6.3.6 Imagestore......................................................................................... 181
6.3.7 Dedicatedprocessor........................................................................ 182
6.3.8 Videosamplingfrequency............................................................ 182
x Contents
6.4 LatencyParameters.......................................................................................... 184
6.4.1 Capturelatency................................................................................. 185
6.4.2 Transferlatency................................................................................ 190
6.4.3 Effectsoflatency............................................................................. 191
6.5 Resolution........................................................................................................... 194
6.5.1 Gray-levelresolution...................................................................... 194
6.5.2 Pixelresolution................................................................................. 194
6.5.3 Spatialresolution............................................................................. 194
6.5.4 Assessmentofresolution.............................................................. 198
References....................................................................................................................... 201
Chapter7 ImageFormation.................................................................................203
7.1 FieldofView..................................................................................................... 208
7.2 DepthofField.................................................................................................... 211
7.3 ImageIntensity.................................................................................................. 215
7.4 ImageFunctions .............................................................................................. 221
7.4.1 Point-spreadfunction..................................................................... 222
7.4.2 Line-spreadfunction....................................................................... 227
7.4.3 Edge-spreadfunction...................................................................... 227
7.4.4 Opticaltransferfunction................................................................ 228
7.4.5 MTFandcontrast............................................................................. 230
7.5 ImageModeling................................................................................................ 234
7.5.1 Wavefrontmodel............................................................................. 235
7.5.2 Diffraction ......................................................................................... 237
7.6 LensMTF........................................................................................................... 241
7.6.1 Resolution.......................................................................................... 251
7.6.2 Imagequality..................................................................................... 255
7.7 SensorMTF....................................................................................................... 261
7.7.1 SpatialMTF....................................................................................... 261
7.7.2 DiffusionMTF.................................................................................. 266
7.7.3 Charge-transferMTF...................................................................... 271
References....................................................................................................................... 279
Chapter8 CameraCalibration.............................................................................283
8.1 Projection............................................................................................................ 284
8.2 IdealIntrinsicModel....................................................................................... 289
8.3 ExtrinsicModel................................................................................................ 292
8.3.1 Translation.......................................................................................... 294
8.3.2 Rotation............................................................................................... 294
8.4 GeneralCameraModel.................................................................................. 295
8.5 TsaiCalibration................................................................................................ 298
8.5.1 Cameramodel................................................................................... 298
8.5.2 Scalingandorigintransfer............................................................ 300
Contents xi
8.5.3 Stage 1 calibration: Parameters embedded in image
abscissa................................................................................................ 301
8.5.4 Stage2calibration:Parametersrelatedtoimageordinate. 304
8.5.5 Resolutionanddistortion.............................................................. 305
8.6 StereoImaging.................................................................................................. 311
8.6.1 Epipolargeometry........................................................................... 313
8.6.2 Matchingwithepipolarconstraints........................................... 321
8.7 FeatureMatching............................................................................................. 323
8.7.1 Intensitymatching........................................................................... 323
8.7.2 Cross-correlation.............................................................................. 324
8.7.3 Edgefeature....................................................................................... 326
8.8 InclinedCamera................................................................................................ 331
8.8.1 Viewingdirection............................................................................. 333
8.8.2 Scalingfactors.................................................................................. 333
References....................................................................................................................... 336
Chapter9 Gray-LevelTransformation...................................................................341
9.1 Pixel-to-PixelMapping.................................................................................. 342
9.2 GammaCorrection.......................................................................................... 343
9.3 ImageHistogram.............................................................................................. 345
9.4 HistogramEqualization................................................................................. 348
9.5 HistogramHyperbolization.......................................................................... 350
9.6 HistogramSpecification................................................................................. 354
9.7 LocalHistogram............................................................................................... 357
9.8 StatisticalDifferencing................................................................................... 357
9.9 Thresholding...................................................................................................... 361
9.9.1 Triangularminimummethod....................................................... 363
9.9.2 Iterativemeanmethod.................................................................... 363
9.10 Co-occurrenceMatrix..................................................................................... 365
Appendix9A:HistogramProperties...................................................................... 371
9A.1 Definitions.......................................................................................... 371
9A.2 VariateTransformation.................................................................. 372
References....................................................................................................................... 375
Chapter10 SpatialTransformation........................................................................379
10.1 Interpolation....................................................................................................... 380
10.2 GeometricOperations..................................................................................... 384
10.2.1 Forwardtransformation................................................................. 387
10.2.2 Backwardtransformation.............................................................. 387
10.2.3 RescalingCartesiancoordinates................................................. 390
10.3 BilinearInterpolation...................................................................................... 391
10.4 CubicInterpolation.......................................................................................... 394
10.5 Zero-OrderConvolution................................................................................ 404
xii Contents
10.6 AffineTransformation.................................................................................... 408
10.7 PerspectiveTransformation.......................................................................... 411
Appendix10A:BasisFunctionsandSplines....................................................... 419
10A.1 HermiteCurves................................................................................. 422
10A.2 CardinalSplines............................................................................... 425
10A.3 BézierCurves.................................................................................... 427
10A.4 CubicSplines.................................................................................... 435
References....................................................................................................................... 441
Chapter11 SpatialFiltering...................................................................................445
11.1 NoiseModels..................................................................................................... 447
11.2 AveragingFilters.............................................................................................. 453
11.2.1 Gaussianfilter................................................................................... 455
11.2.2 Rotatingaverage............................................................................... 456
11.2.3 Sigmafilter......................................................................................... 457
11.2.4 Outlierfilter....................................................................................... 458
11.2.5 Unsharpmask.................................................................................... 458
11.3 Rank-OrderFilters........................................................................................... 459
11.4 AdaptiveFilters................................................................................................. 461
11.4.1 Additivenoise................................................................................... 462
11.4.2 Impulsenoise.................................................................................... 463
11.4.3 Multiplicativenoise......................................................................... 465
11.5 First-OrderGradients...................................................................................... 467
11.5.1 Robertsoperator............................................................................... 469
11.5.2 Prewittoperator................................................................................ 469
11.5.3 Sobeloperator................................................................................... 471
11.6 Second-OrderGradients................................................................................ 471
11.7 AnisotropicFilters........................................................................................... 477
11.7.1 Bilateralfilters.................................................................................. 477
11.7.2 Diffusionfilters................................................................................. 478
Appendix11A:ConvolutionKernels..................................................................... 482
11A.1 DiscreteConvolution...................................................................... 483
11A.2 Two-DimensionalConvolution................................................... 486
References....................................................................................................................... 490
Chapter12 DiscreteFourierTransform..................................................................495
12.1 DiscreteFourierSeries................................................................................... 495
12.2 DiscreteFourierTransform........................................................................... 501
12.3 Decimation-in-TimeFFT.............................................................................. 506
12.4 ImageFrequencySpectrum.......................................................................... 515
12.5 BasisFunction................................................................................................... 520
12.6 MatrixForm....................................................................................................... 528
12.6.1 DITinmatrixform.......................................................................... 530
12.6.2 DIFinmatrixform.......................................................................... 533