Machine Vision : Automated Visual Inspection: Theory, Practice and Applications PDF
Preview Machine Vision : Automated Visual Inspection: Theory, Practice and Applications
Machine Vision Jürgen Beyerer • Fernando Puente León Christian Frese Machine Vision Automated Visual Inspection: Theory, Practice and Applications 123 Jürgen Beyerer Christian Frese Fraunhofer-Institut für Optronik, Fraunhofer-Institut für Optronik, Systemtechnik und Bildauswertung Systemtechnik und Bildauswertung and Karlsruhe Institute of Technology Karlsruhe, Germany Karlsruhe, Germany Fernando Puente León Karlsruhe Institute of Technology Karlsruhe, Germany Translator: Johannes Meyer ISBN 978-3-662-47793-9 ISBN 978-3-662-47794-6 (eBook) DOI 10.1007/978-3-662-47794-6 Library of Congress Control Number: 2015947141 Springer © Springer-Verlag Berlin Heidelberg 2016 This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of the material is concerned, specifi cally the rights of translation, reprinting, reuse of illustrations, recita- tion, broadcasting, reproduction on microfi lms or in any other physical way, and transmission or in- formation storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publica- tion does not imply, even in the absence of a specifi c statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. The publisher, the authors and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors give a warranty, express or implied, with respect to the material contained herein or for any errors or omissions that may have been made. Printed on acid-free paper Springer is a brand of Springer Fachmedien Wiesbaden Springer Berlin Heidelberg is part of Springer Science+Business Media (www.springer.com) DedicatedtoProfessorDr.-Ing.FranzMesch VII Preface Machine vision and automated visual inspection are domains of automation technology withasteadilyincreasingeconomicalrelevance.Althoughtherelatedindustryisnotablyex- pandingsincethepasttwodecades,onlyapartoftoday’svisualinspectiontaskshavebeen automated.Thisiswhythereisagreatpotentialforeconomizationinhighincomecountries whichmayleadtobothreducedcostsandincreasedqualityoftheproducedgoods. Asvisualperceptionisthemainhumansensation,theautomationofvisualinspectionis somehowfascinating—atleastfortheauthorsofthisbook.Whentalkingabouttheautoma- tionofvisualinspectiononemighteasilythinkthatitcannotbethathardtoteachatechnical visualinspectionsystemtoperceivewhatahumancaneasilyseewithonlyaglimpse. Ac- tually,usersoftenstate:‘Ashumanscanseethatinstantly,itcan’tbethathardtoachievethe sameusingamachine’. Theanswerisnoteasyatallbutitdependsontheindividualcase: forhumans,somethingsrepresenteasytaskswhichhoweveraredifficulttoautomate—on thecontrary,manyotherthingscanbedonemorepreciselyandparticularlymorereliably bymachines,ifanautomationispossible. Automatedvisualinspectionisacomplexandmulti-disciplinarytopicinvolvingoptics, mechanicalandelectricalengineering,mathematicsandcomputerscience. Systemsforau- tomatedvisualinspectionareusuallymoreorlesscomplexmechatronicsystems,whichcan onlyachievetherequestedperformanceinaneconomicalway,ifallthenecessarydisciplines collaborate. Everythingstarts witha visualinspection taskthat isto becarried outusing anauto- matedapproach.Inthiscontext,imageacquisitionplaysanimportantrole:lossofinforma- tionduringthatstepcanhardlybecompensatedduringlaterimageprocessingsteps. The successofavisualinspectionsolutiondependsheavilyonthequalityofthisfirststep. Fortunately,whendesigninganautomatedvisualinspectionsystem,oneusuallyhasthe benefitofseveraldegreesoffreedom,inordertoobtainimagedatawithsufficientquality andsignificance. Thisiswhythesuitabilityoftheimageacquisitionforagivenproblemat leastpartlydependsontheengineer.Inordertoexploitthosedegreesoffreedomatthebest, thisbookdrawsparticularattentiontoimageacquisitionandtheacquisitionconstellation, consistingofthetestobject,theilluminationandtheacquisitionsystem. Thisbookhastheambitiontothoroughlyintroducethereaderintothetermsofauto- matedvisualinspection. Forthispurpose,theChapters2to6ofthebook’sfirstpartdeal withthephysicsofimageformationandtherequiredopticalprinciplesandtechniquesin anadequatelyextensiveway. Basedonthatfoundations,imageacquisitionforautomated visualinspectionwillbetreatedinChapter7. Inthiskeychapter,amultitudeofdifferent techniquesforimageacquisitionwillbeexplainedinasystematicway,aswellasimportant hintsandtrickswillbeshownwhichareindispensableforagoodvisualinspectionsystem. Inordertoenableautomatedanalysisofimagesinacomputer,theanalogimagesignals havetobetransformedintodigitalsignals. Theunderlyingtheoryofsignalprocessingand the effects of local sampling and quantization will be extensively discussed, especially in termsofsystemtheory.Amongothers,Chapter8isdevotedtothebasicsofdigitalprocess- ingofanalogsignalsandpreparesthereaderforthesecondpartofthebook,whichfocuses onimageanalysis.Chapters9to15covermethods,whichformtheindividualstepsleading toafinalinspectionresultbasedontheacquiredimagedata. VIII Preface Thedepthoftheexplanationsofallcoveredsubjectsischosentoprovidethereaderwith insightintotherespectivemotivationandbackgrounds. Nofactsaresupposed‘toappear fromnowhere’;theunderlyingconceptsshouldbethoroughlyunderstood. Sometheorems howeverwillnotbeproveninastrictmathematicalway.Infact,therewillbesketchesofthe proofs,whichwillpresenttheiressentialideaandhelptounderstandimportantconcepts. Foranapplication-orientedreaderwhoisneverthelessinterestedinwhathappensbehind thescenes,consciouslyomittingtechnicallyflawlesslyledproofsincreasesthebook’sread- abilityandleadstoahandyamountofpages. ThebookonhandispartiallybasedonlecturesheldbytheauthorJ.BeyererattheKarls- ruheInstituteofTechnology(KIT,formerlyUniversityofKarlsruhe)since1994andbythe authorF.PuenteLeón,initiallyatTechnischeUniversitätMünchen(TUM)since2003andat KITsince2008. Itaddressesitselftostudentsstudyinginthefieldsofengineeringscience, computerscience,physicsandmathematics.Asallneededconceptsandmethodsareintro- ducedinasufficientlyexhaustiveway,itshouldbepossibleforadvancedbachelorstudents toclearlyunderstandthepresentedcontent. Furthermore,scientists,PhDstudentsandes- peciallymasterstudentsdealingwithautomatedvisualinspectioncanprofitfromreading thebookasitstopicsareappropriatelyelaborated. Besidestheory,practiceisnotmissedout.Theauthors’industrialexperience,whichisin- corporatedintomanytopicsofthebook,bringsbenefitseventopracticallyorientedreaders whoseekforrobustandeconomicsolutionsforconcretevisualinspectiontasks. Nonethe- less,thebookdoesnotlooseitselfintosuperficialrecipesbutyieldsenoughsubstancefora deepunderstandingofthepresentedcontent. Theauthorswanttoparticularlythanktheirfollowingcolleaguesforsupportingthemin thecreationofthisbook: Dr.UlrichBreitmeier(BreitmeierMesstechnikGmbH,Ettlingen,Germany)foranexam- pleimageofacylindricscanner Dr.MichaelFried(UniversityofErlangen, Germany)foranexampleoftheMumford- Shah-Method Dr.JanHorn(DepartmentofMeasurementandControl,KarlsruheInstituteofTechnol- ogy)foranimageofcamerabasedvelocitymeasurement Dr.UdoNetzelmannandDr.GünterWalle(FraunhoferIZFP,Saarbrücken,Germany)for exampleimagesoftheimpulsethermography ArneNowak(FraunhoferIIS,Erlangen,Germany)forexampleimagesacquiredwiththe POLKAcamera DirkNüßler(FraunhoferFHR,Wachtberg,Germany)forexampleimagesofinspection usingterahertzradiation Prof.Dr.WolfgangOsten(UniversityofStuttgart,Germany)forimagesofinterferometric methods Prof.Dr.JerryL.Prince(JohnsHopkinsUniversity,Baltimore,USA)forexamplesofac- tivecontours Dr. Andreas Purde (Institute for Measurement Systems and Sensor Technology, TUM, Germany)foranexampleforthepyramidlinkingmethod Dr.AnnaRemelliandDr.ClaudioSedazzari(OptoEngineering,Mantova,Italy)forex- amplesofthehypercentricperspective Dr.NormanUhlmann(FraunhoferEZRT,Fürth,Germany)forexampleimagesofX-ray inspection Preface IX Bernhard Schmitt M.A. (ONUK, Karlsruhe, Germany) for an aerial photograph of the KarlsruhePalace DirkvomSteinandThomasWinkel(InspectomationGmbH,Mannheim,Germany)for exampleimagesshowingtheinspectionofcastingcoresandbrakediscsaswellastele- centricimages Dr.MarcoKruse(InstituteofIndustrialInformationTechnologyIIIT,KarlsruheInstitute ofTechnologyKIT)fortheimageofthecheckerboard-shadowillusionaswellasforex- ampleimagesconcerningtherestorationofuniformmotionblur MarioLietz(IIIT,KIT)forexampleimagesshowingthedivisionbyareferenceimage Dr.IoanaGhe¸ta(VisionandFusionLaboratoryIES,InstituteforAnthropomatics,Karls- ruheInstituteofTechnology)forexamplesconcerningtheanalysisofspectralimagese- ries Dr.RobinGruna(IES,KIT)forimagesrecordedwithinverseillumination Dr. Matthias Michelsburg (IIIT, KIT) and Dr. Robin Gruna for hyperspectral images of food ThomasStephan(IES,KIT)forimagesrecordedwithalight-fieldcameraandforexam- plesshowingtherestorationofparticipatingmedia Dr. Matthias Hartrumpf (Fraunhofer Institute for Optronics, System Technologies and ImageExploitationIOSB,Karlsruhe,Germany)forexampleimagesshowingphotoelas- ticityandtheinspectionofglasspreforms Prof.Dr.MichaelHeizmann(FraunhoferIOSB)forexamplesoftextureanalysisaswell asshapefromshading ChristianNegara(FraunhoferIOSB)forexamplesofthegraphcutmethod JohannesPallauf(IIIT,KIT)fortheexampleofimagerestorationforuniformmotionblur GünterSaurandWolfgangRoller(FraunhoferIOSB)forsupplyingTerraSAR-Ximages Günter Struck and Dr. Kai-Uwe Vieth (Fraunhofer IOSB) for examples of fluorescence spectroscopy Chen-Ko Sung (Fraunhofer IOSB) for example images of inspection based on flatbed scanners Dr.StefanWerling(FraunhoferIOSB)fordeflectometricimages Dr.AlexanderSchwarzandMartinaRichter(FraunhoferIOSB)forimagesshowingan exampleBRDFmeasurement Dr.MiroTaphanel(IES,KIT)forimagesconcerningtheCCTsensor JohannesMeyer(IES,KIT)forexampleimagesacquiredwithaSchlierensetup Dr.YaokunZhang(IPR,KIT)andJohannesMeyer(IES,KIT)forexampleOCTimages Special thanks go to Johannes Meyer (IES, KIT) for the incorporation of numerous exten- sionsintotheGermanversionofthisbook(‘AutomatischeSichtprüfung’)andfortranslat- ingitintoEnglish.Abigcontributiontothecontentspresentedinthisbookhasbeenmade especially by Dr. Stefan Werling, Dr. Christoph Lindner, Dr. Ana Pérez Grassi, Dr. Robin Gruna, SebastianHöferandDr.MichaelTeutschbyassistingthementionedlectures. The authorswouldliketoalsothankallthestudentswhowereinvolvedincreatingimagesand diagrams. Inaddition,numerousstudentshavesentinvaluablesuggestionsforimproving thebook’sdidacticcomponent. Special thanks are directed to Andrey Belkin, Yvonne Fischer, Peter Frühberger, Dr. Robin Gruna, Jan Hendrik Hammer, Pilar Hernández Mesa, Christian Herrmann, Sebas- tian Höfer, Chettapong Janya-Anurak, Mahsa Mohammadi Kaji, Dr. Marco Kruse, Achim X Preface Kuwertz,DingLuo,ThomasNürnberg,Dr.AlexeyPak,JohannesPallauf,JuliusPfrommer, Chengchao Qu, Matthias Richter, Masoud Roschani, Lars Sommer, Thomas Stephan, Dr. MiroTaphanel,Dr.MichaelTeutsch,SebastianVater,MarkusVogelbacher,Dr.StefanWer- ling,PhilippWoockandMathiasZiebarthforvaluablecommentsandproofreadingofthe manuscript.FinallywewouldliketothankourpublisherSpringerfortheexcellentcooper- ation. Karlsruhe,Summer2015 JürgenBeyerer FernandoPuenteLeón ChristianFrese XI Contents 1 Introduction 1 1.1 Visualinspection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 1.2 Opticalcapturingoftestobjects . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 1.3 Formationanddefinitionofanimage . . . . . . . . . . . . . . . . . . . . . . . . 9 1.4 Machinevision . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 1.5 Practicalapproachforperformingmachinevisionprojects . . . . . . . . . . . . 15 1.6 Bibliography. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 I ImageAcquisition 19 2 Light 21 2.1 Thephenomenonoflight . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 2.1.1 Theelectromagneticspectrum . . . . . . . . . . . . . . . . . . . . . . . 23 2.2 Lightasanelectromagneticwave . . . . . . . . . . . . . . . . . . . . . . . . . . 24 2.2.1 Maxwell’sequations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 2.2.1.1 Waveequationsoftheelectricfield . . . . . . . . . . . . . . . 26 2.2.2 Polarization. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 2.2.2.1 Thepolarizationellipse . . . . . . . . . . . . . . . . . . . . . . 31 2.2.2.2 Stokesparameters . . . . . . . . . . . . . . . . . . . . . . . . 33 2.2.2.3 ThePoincare´ sphere . . . . . . . . . . . . . . . . . . . . . . . 35 2.2.2.4 Muellermatrices . . . . . . . . . . . . . . . . . . . . . . . . . 36 2.2.2.5 Measuringthepolarizationstate . . . . . . . . . . . . . . . . . 41 2.2.2.6 TheJonescalculus . . . . . . . . . . . . . . . . . . . . . . . . 42 2.2.3 Huygens’principle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44 2.2.4 Coherence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44 2.2.5 Interference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46 2.2.6 Diffraction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48 2.2.6.1 Resolutionofimagingsystems. . . . . . . . . . . . . . . . . . 58 2.2.7 Speckle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60 2.3 Lightasaquantumphenomenon . . . . . . . . . . . . . . . . . . . . . . . . . . 61 2.4 Theraymodelofgeometricaloptics . . . . . . . . . . . . . . . . . . . . . . . . 65 2.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66 2.6 Interactionoflightandmatter . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66 2.6.1 Absorption . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67 2.6.2 Thelawofreflection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67 2.6.3 Thelawofrefraction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68 2.6.4 Scattering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70 2.6.5 TheFresnelcoefficientsforreflectionandtransmission. . . . . . . . . . 71 2.6.6 Electromagneticwavesinconductivemedia . . . . . . . . . . . . . . . . 76 2.6.6.1 Complexindexofrefraction . . . . . . . . . . . . . . . . . . . 77
The list of books you might like
