ebook img

Smart Camera Design: Algorithms, Architectures, and Art PDF

234 Pages·2018·7.6 MB·English
Save to my drive
Quick download
Download
Most books are stored in the elastic cloud where traffic is expensive. For this reason, we have a limit on daily download.

Preview Smart Camera Design: Algorithms, Architectures, and Art

Marilyn Wolf Smart Camera Design Algorithms, Architectures, and Art Smart Camera Design Marilyn Wolf Smart Camera Design Algorithms, Architectures, and Art MarilynWolf SchoolofECE GeorgiaInstituteofTechnology Atlanta,GA,USA ISBN978-3-319-69522-8 ISBN978-3-319-69523-5 (eBook) https://doi.org/10.1007/978-3-319-69523-5 LibraryofCongressControlNumber:2017957575 ©SpringerInternationalPublishingAG2018 Thisworkissubjecttocopyright.AllrightsarereservedbythePublisher,whetherthewholeorpartof the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilarmethodologynowknownorhereafterdeveloped. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publicationdoesnotimply,evenintheabsenceofaspecificstatement,thatsuchnamesareexempt fromtherelevantprotectivelawsandregulationsandthereforefreeforgeneraluse. Thepublisher,theauthorsandtheeditorsaresafetoassumethattheadviceandinformationinthis book are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors give a warranty, express or implied, with respect to the material contained hereinor for anyerrors oromissionsthat may havebeenmade. Thepublisher remainsneutralwith regardtojurisdictionalclaimsinpublishedmapsandinstitutionalaffiliations. Printedonacid-freepaper ThisSpringerimprintispublishedbySpringerNature TheregisteredcompanyisSpringerInternationalPublishingAG Theregisteredcompanyaddressis:Gewerbestrasse11,6330Cham,Switzerland For Dad, Who set me on this path. Preface This book draws from three areas of computing: image processing, computer vision,andcomputergraphics.Imageprocessingandcomputervisioninparticular havelongbeenseparatefieldswithoverlappinginterests.Thisispartlyasociolog- ical phenomenon—image processing comes from electrical engineering, while computer vision comes from computer science. These two fields blend smoothly indigitalcameradesign.Themodernimagingchainstartsattraditionalfilteringand endswithfeatureanalysis. Parts of this book draw upon my research work with my students at Princeton and Georgia Tech. Cheng-Yao Chen, Santanu Dutta, Jason Fritts, Se Hun Kim, Changhong Lin, Chung-Ching Lin, Tiehan Lv, Jason Schlessman, Senem Velipasalar,JiangXu,HeatherYu,andShengqiYanghaveallworkedonaspects ofmultimediacomputingandembeddedcomputervision.Iamgratefultothemfor the opportunity to work with them and learn from them. Burak Ozer was not my officialstudent,buthehasbeenmyfriendandcollaboratoronsmartcamerasforthe past15years. Theinspirationforthisbookcomesfrommyfather,aninventorwhocreatedtwo differentpanoramiccameras.Thecameraonthededicationpage,whichwasknown commercially as the CycloPan 360, uses a cylindrical mapping. Here is another pictureofDadwithhisdonutcamera. vii viii Preface He also created a motion picture version of this camera for use in a flight simulator. That camera used 500 aerial film. The projector used the same optical pathasthecamerabutrotatedcontinuouslytosweeptheimage;itwasanimposing machine.IranDad’scolordarkroomduringthoseyears—Iworkedcheap. Dad,thisbookisentirelyyourfault.Ihavespentnightafternighttypingaway becauseyoutaughtmehowtothink.Thisbookismytributetoyou. Atlanta,GA,USA MarilynWolf August2016 Contents 1 DigitalPhotography. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1.1 Introduction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1.2 PrevisualizationandAutoprevisualization. . . . . . . . . . . . . . . . . 1 1.3 EnhancedImages. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 1.4 BeyondImagestoAnalysis. . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 1.5 StillandMovingImages. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 1.6 TakingaPicture. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 1.7 HowtoReadthisBook. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 2 Light,Optics,andImaging. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 2.1 Introduction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 2.2 ImageFormation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 2.2.1 LightandImages. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 2.2.2 ThePhysicsofLight. . . . . . . . . . . . . . . . . . . . . . . . . . 11 2.3 TheHumanVisualSystem. . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 2.4 ColorScience. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 2.4.1 TheoriesofColorVision. . . . . . . . . . . . . . . . . . . . . . . 19 2.4.2 ColorModels. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 2.5 Lenses. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 2.5.1 LensesandImageFormation. . . . . . . . . . . . . . . . . . . . 23 2.5.2 RayOptics. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 2.5.3 LensDesign. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 2.5.4 Panoramas. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 2.5.5 AssessingLenses. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38 2.6 GeometryandtheCameraModel. . . . . . . . . . . . . . . . . . . . . . . 41 2.6.1 ProjectiveGeometry. . . . . . . . . . . . . . . . . . . . . . . . . . 41 2.6.2 TheCameraModel. . . . . . . . . . . . . . . . . . . . . . . . . . . 44 2.6.3 CameraCalibration. . . . . . . . . . . . . . . . . . . . . . . . . . . 46 2.7 ImageDisplay. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48 2.8 PracticalImageCapture. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50 ix x Contents 2.8.1 ExposureSettings. . . . . . . . . . . . . . . . . . . . . . . . . . . . 51 2.8.2 WhichExposureSetting?. . . . . . . . . . . . . . . . . . . . . . . 52 2.8.3 ColorTemperature. . . . . . . . . . . . . . . . . . . . . . . . . . . . 56 2.8.4 ImageComposition. . . . . . . . . . . . . . . . . . . . . . . . . . . 58 2.8.5 ImageQualityAssessment. . . . . . . . . . . . . . . . . . . . . . 63 2.9 Summary. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66 3 ImageCaptureSystemsandAlgorithms. . . . . . . . . . . . . . . . . . . . . . 67 3.1 Introduction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67 3.2 TheGenericCameraArchitecture. . . . . . . . . . . . . . . . . . . . . . . 68 3.3 TheCameraDesignSpace. . . . . . . . . . . . . . . . . . . . . . . . . . . . 69 3.3.1 Trade-Offs.. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. 69 3.3.2 UseCasesforCameras. . . . . . . . . . . . . . . . . . . . . . . . 70 3.3.3 FourExamplesofCameraDesigns. . . . . . . . . . . . . . . . 70 3.4 ImageSensors. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73 3.4.1 ImageSensorArchitectures. . . . . . . . . . . . . . . . . . . . . 73 3.4.2 Photosensors. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75 3.4.3 Charge-CoupledDevices. . . . . . . . . . . . . . . . . . . . . . . 77 3.4.4 APSCMOSImageSensors. . . . . . . . . . . . . . . . . . . . . 78 3.4.5 AdvancedImageSensors. . . . . . . . . . . . . . . . . . . . . . . 82 3.4.6 ImageSensorCharacteristics. . . . . . . . . . . . . . . . . . . . 83 3.4.7 ShuttersandIrises. . . . . . . . . . . . . . . . . . . . . . . . . . . . 89 3.5 PreexposureOperations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91 3.5.1 Autofocus. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92 3.5.2 Exposure. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96 3.5.3 ImageStabilization. . . . . . . . . . . . . . . . . . . . . . . . . . . 98 3.5.4 FaceDetectionandTracking. . . . . . . . . . . . . . . . . . . . 99 3.6 PostexposureOperations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102 3.6.1 ColorFilterArrayInterpolation. . . . . . . . . . . . . . . . . . 102 3.6.2 WhiteBalance. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105 3.6.3 Sharpening. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107 3.7 ImageandVideoCompression. . . . . . . . . . . . . . . . . . . . . . . . . 108 3.7.1 LossyCompression. . . . . . . . . . . . . . . . . . . . . . . . . . . 108 3.7.2 ImageCodingandJPEG. . . . . . . . . . . . . . . . . . . . . . . 108 3.7.3 VideoCoding,H.264/AVC,andHEVC/H.265. . . . . . . 113 3.7.4 QualityAssessmentofCompressedImages. . . . . . . . . . 120 3.8 ComputingPlatforms. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121 3.8.1 CamerasasHeterogeneousMultiprocessors. . . . . . . . . 121 3.8.2 Buffering. . . .. . . .. . . .. . . . .. . . .. . . .. . . .. . . . .. 122 3.8.3 InputandOutput. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124 3.8.4 FileFormats. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124 3.8.5 OperatingSystemsandFileSystems. . . . . . . . . . . . . . . 126 3.8.6 Accelerators. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126 Contents xi 3.9 ImageCharacteristicsandImageCapture. . . . .. . . . . .. . . . . .. 130 3.10 StereoandMulticameraSystems. . . . . . . . . . . . . . . . . . . . . . . . 131 3.11 Trade-OffsRevisited. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132 4 ImageandVideoEnhancement. . . . . . . . . . . . . . . . . . . . . . . . . . . . 135 4.1 Introduction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135 4.2 UsefulAlgorithms. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136 4.3 TonalMappingandColorGrading. . . . . . . . . . . . . . . . . . . . . . 138 4.4 High-DynamicRangeImages. . . . . . . . . . . . . . . . . . . . . . . . . . 143 4.5 SharpeningandSuperresolution. . . . . . . . . . . . . . . . . . . . . . . . 146 4.6 BokehIntroduction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149 4.7 LensCorrections. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149 4.8 FocusStacking. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 150 4.9 KeystoneCorrection. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152 4.10 MosaicComposition. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153 4.11 VideoStabilization. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156 4.11.1 OpticalFlow. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156 4.11.2 StabilizationAlgorithms. . . . . .. . . . . . . .. . . . . . . .. . 157 4.12 SoftwareDesignforImageEnhancement. . . . . . . . . . . . . . . . . . 158 4.13 PracticalImageEnhancement. . . . . . . . . . . . . . . . . . . . . . . . . . 160 5 ImageandVideoAnalysis. .. . . . . . . . . . . . . .. . . . . . . . . . . . . .. . . 163 5.1 Introduction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163 5.2 ImageAnalysisAlgorithms. . . . . . . . . . . . . . . . . . . . . . . . . . . . 164 5.3 ImageandVideoCharacteristics. . . . . . . . . . . . . . . . . . . . . . . . 167 5.3.1 ImageStatistics. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 167 5.3.2 Saliency. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 168 5.3.3 KeyFrameSelection. . . . . . . . . . . . . . . . . . . . . . . . . . 170 5.4 SceneAnalysis. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171 5.4.1 VisualSearch. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171 5.4.2 FaceDetectionandRecognition. . . . . . . . . . . . . . . . . . 172 5.5 Tracking. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173 5.5.1 BackgroundElimination. . . .. . . . . . . . .. . . . . . . . .. . 173 5.5.2 TrackingfromaFixedCamera. . . . . . . . . . . . . . . . . . . 175 5.5.3 AppearanceModels. . . . . . . . . . . . . . . . . . . . . . . . . . . 178 5.5.4 ActivityAnalysis. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 181 5.5.5 TrackingfromaMovingCamera. . . . . . . . . . . . . . . . . 183 5.6 MulticameraSystems. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 186 5.6.1 MulticameraSystemsasDistributedComputing Systems. . . . .. . . . . .. . . . . .. . . . . .. . . . .. . . . . .. . 187 5.6.2 MulticameraCalibration. .. . . . .. . . . .. . . . . .. . . . .. 189 5.6.3 MulticameraTracking. . . . . . . . . . . . . . . . . . . . . . . . . 191 5.7 UseCasesandWorkflows. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197

Description:
This book describes the algorithms and computer architectures used to create and analyze photographs in modern digital cameras. It also puts the capabilities of digital cameras into context for applications in art, entertainment, and video analysis. The author discusses the entire range of topics re
See more

The list of books you might like

Most books are stored in the elastic cloud where traffic is expensive. For this reason, we have a limit on daily download.