Texts in Computer Science Editors DavidGries FredB.Schneider Forfurthervolumes: www.springer.com/series/3191 Richard Szeliski Computer Vision Algorithms and Applications 123 Dr. Richard Szeliski Microsoft Research One Microsoft Way 98052-6399 Redmond Washington USA [email protected] SeriesEditors DavidGries FredB.Schneider DepartmentofComputerScience DepartmentofComputerScience UpsonHall UpsonHall CornellUniversity CornellUniversity Ithaca,NY14853-7501,USA Ithaca,NY14853-7501,USA ISSN 1868-0941 e-ISSN 1868-095X ISBN 978-1-84882-934-3 e-ISBN 978-1-84882-935-0 DOI 10.1007/978-1-84882-935-0 Springer London Dordrecht Heidelberg New York BritishLibraryCataloguinginPublicationData AcataloguerecordforthisbookisavailablefromtheBritishLibrary LibraryofCongressControlNumber:2010936817 © Springer-VerlagLondonLimited2011 Apart from any fair dealing for the purposes of research or private study, or criticism or review, as permittedundertheCopyright,DesignsandPatentsAct1988,thispublicationmayonlybereproduced, stored or transmitted, in any form or by any means, with the prior permission in writing of the publishers,orinthecaseofreprographicreproductioninaccordancewiththetermsoflicensesissuedby theCopyrightLicensingAgency.Enquiriesconcerningreproductionoutsidethosetermsshouldbesent tothepublishers. Theuseofregisterednames,trademarks,etc.,inthispublicationdoesnotimply,evenintheabsenceofa specificstatement,thatsuchnamesareexemptfromtherelevantlawsandregulationsandthereforefree forgeneraluse. Thepublishermakesnorepresentation,expressorimplied,withregardtotheaccuracyoftheinformation containedinthisbookandcannotacceptanylegalresponsibilityorliabilityforanyerrorsoromissions thatmaybemade. Printedonacid-freepaper SpringerispartofSpringerScience+BusinessMedia(www.springer.com) Thisbookisdedicatedtomyparents, ZdzisławandJadwiga, andmyfamily, Lyn,Anne,andStephen. 1 Introduction 1 Whatiscomputervision? Abriefhistory • • Bookoverview Samplesyllabus Notation • • n^ 2 Imageformation 27 Geometricprimitivesandtransformations • Photometricimageformation • Thedigitalcamera 3 Imageprocessing 87 Pointoperators Linearfiltering • • Moreneighborhoodoperators Fouriertransforms • • Pyramidsandwavelets Geometrictransformations • • Globaloptimization 4 Featuredetectionandmatching 181 Pointsandpatches • Edges Lines • 5 Segmentation 235 Activecontours Splitandmerge • • Meanshiftandmodefinding Normalizedcuts • • Graphcutsandenergy-basedmethods 6 Feature-basedalignment 273 2Dand3Dfeature-basedalignment • Poseestimation Geometricintrinsiccali•bration 7 Structurefrommotion 303 Triangulation Two-framestructurefrommotion • • Factorization Bundleadjustment • • Constrainedstructureandmotion 8 Densemotionestimation 335 Translationalalignment Parametricmotion • • Spline-basedmotion Opticalflow • • Layeredmotion 9 Imagestitching 375 Motionmodels Globalalignment • • Compositing 10 Computationalphotography 409 Photometriccalibration Highdynamicrangeimaging • • Super-resolutionandblurremoval • Imagemattingandcompositing • Textureanalysisandsynthesis 11 Stereocorrespondence 467 Epipolargeometry Sparsecorrespondence • • Densecorrespondence Localmethods • • Globaloptimization Multi-viewstereo • 12 3Dreconstruction 505 ShapefromX Activerangefinding • • Surfacerepresentations Point-basedrepresentations • • Volumetricrepresentations Model-basedreconstruction • • Recoveringtexturemapsandalbedos 13 Image-basedrendering 543 Viewinterpolation Layereddepthimages • • LightfieldsandLumigraphs Environmentmattes • • Video-basedrendering 14 Recognition 575 Objectdetection Facerecognition • • Instancerecognition Categoryrecognition • • Contextandsceneunderstanding • Recognitiondatabasesandtestsets Preface Theseedsforthisbookwerefirstplantedin2001whenSteveSeitzattheUniversityofWash- ingtoninvitedmetoco-teachacoursecalled“ComputerVisionforComputerGraphics”. At thattime,computervisiontechniqueswereincreasinglybeingusedincomputergraphicsto createimage-basedmodelsofreal-worldobjects,tocreatevisualeffects,andtomergereal- world imagery using computational photography techniques. Our decision to focus on the applicationsofcomputervisiontofunproblemssuchasimagestitchingandphoto-based3D modelingfrompersonalphotosseemedtoresonatewellwithourstudents. Sincethattime,asimilarsyllabusandproject-orientedcoursestructurehasbeenusedto teachgeneralcomputervisioncoursesbothattheUniversityofWashingtonandatStanford. (ThelatterwasacourseIco-taughtwithDavidFleetin2003.) Similarcurriculahavebeen adoptedatanumberofotheruniversitiesandalsoincorporatedintomorespecializedcourses oncomputationalphotography.(Forideasonhowtousethisbookinyourowncourse,please seeTable1.1inSection1.4.) Thisbookalsoreflectsmy20years’experiencedoingcomputervisionresearchincorpo- rateresearchlabs,mostlyatDigitalEquipmentCorporation’sCambridgeResearchLaband atMicrosoftResearch. Inpursuingmywork, Ihavemostlyfocusedonproblemsandsolu- tiontechniques(algorithms)thathavepracticalreal-worldapplicationsandthatworkwellin practice. Thus,thisbookhasmoreemphasisonbasictechniquesthatworkunderreal-world conditionsandlessonmoreesotericmathematicsthathasintrinsicelegancebutlesspractical applicability. Thisbookissuitableforteachingasenior-levelundergraduatecourseincomputervision to students in both computer science and electrical engineering. I prefer students to have either an image processing or a computer graphics course as a prerequisite so that they can spendlesstimelearninggeneralbackgroundmathematicsandmoretimestudyingcomputer visiontechniques. Thebookisalsosuitableforteachinggraduate-levelcoursesincomputer vision(bydelvingintothemoredemandingapplicationandalgorithmicareas)andasagen- eralreferencetofundamentaltechniquesandtherecentresearchliterature.Tothisend,Ihave attemptedwhereverpossibletoatleastcitethenewestresearchineachsub-field,evenifthe technicaldetailsaretoocomplextocoverinthebookitself. Inteachingourcourses,wehavefounditusefulforthestudentstoattemptanumberof smallimplementationprojects,whichoftenbuildononeanother,inordertogetthemusedto workingwithreal-worldimagesandthechallengesthatthesepresent. Thestudentsarethen askedtochooseanindividualtopicforeachoftheirsmall-group,finalprojects. (Sometimes these projects even turn into conference papers!) The exercises at the end of each chapter contain numerous suggestions for smaller mid-term projects, as well as more open-ended
Description: