ebook img

Real-Time Reconstruction of Static and Dynamic Scenes PDF

196 Pages·2015·31.25 MB·English
Save to my drive
Quick download
Download
Most books are stored in the elastic cloud where traffic is expensive. For this reason, we have a limit on daily download.

Preview Real-Time Reconstruction of Static and Dynamic Scenes

Real-Time Reconstruction of Static and Dynamic Scenes Der Technischen Fakultät der Friedrich-Alexander-Universität Erlangen-Nürnberg zur Erlangung des Grades Dr.-Ing. vorgelegtvon Michael Zollhöfer ausHerzogenaurach AlsDissertationgenehmigt vonderTechnischenFakultät derFriedrich-Alexander-UniversitätErlangen-Nürnberg TagdermündlichenPrüfung: 22.12.2014 VorsitzendedesPromotionsorgans: Prof.Dr.-Ing.habil.MarionMerklein Gutachter: Prof.Dr.GüntherGreiner Prof.Dr.Christianeobalt Revision1.00 ©2014,CopyrightMichaelZollhöfer [email protected] AllRightsReserved AlleRechtevorbehalten Abstract WiththereleaseoftheMicrosoXbox360Kinect,anaffordablereal-time RGB-Dsensorisnowavailableonthemassmarket. ismakesnewtech- niques and algorithms, which have previously been only available to re- searchersandenthusiasts, accessibleforaneverydayusebyabroadaudi- ence. Applicationsrangefromtheacquisitionofdetailedhigh-qualityre- constructionsofeverydayobjectstotrackingthecomplexmotionsofpeo- ple.Inaddition,thecaptureddatacanbedirectlyexploitedtobuildvirtual reality applications, i.e. virtual mirrors, and can be used for gesture con- trolofdevicesandmotionanalysis.Tomaketheseapplicationseasy-to-use inoureverydaylife, theyshouldbeintuitivetocontrolandprovidefeed- backatreal-timerates.Inthisdissertation,wepresentnewtechniquesand algorithmsforbuildingthree-dimensionalrepresentationsofarbitraryob- jectsusingonlyasinglecommodityRGB-Dsensor, manuallyeditingthe acquiredreconstructionsandtrackingthenon-rigidmotionofphysically deforming objects at real-time rates. We start by proposing the use of a statisticalpriortoobtainhigh-qualityreconstructionsofthehumanhead usingonlyasinglelow-qualitydepthframeofacommoditysensor.Weex- tendthisapproachandobtainevenhigherqualityreconstructionsatreal- timeratesbyexploitingallinformationofacontiguousRGB-Dstreamand jointly optimizing for shape, albedo and illumination parameters. ere- aer,weshowthatamovingsensorcanbeusedtoobtainsuper-resolution reconstructionsofarbitraryobjectsatsensorratebyfusingalldepthobser- vations. Wepresentstrategiesthatallowustohandleavirtuallyunlimited reconstructionvolumebyexploitinganewsparsescenerepresentationin combinationwithanefficientstreamingapproach. Inaddition,wepresent ahandlebaseddeformationparadigmthatallowstheusertoeditthecap- turedgeometry,whichmightconsistofmillionsofpolygons,usinganinter- activeandintuitivemodelingmetaphor. Finally,wedemonstratethatthe motionofarbitrarynon-rigidlydeformingphysicalobjectscanbetracked atreal-timeratesusingacustomhigh-qualityRGB-Dsensor. i Acknowledgements is dissertation captures the compressed technical contributions and al- gorithmicachievementsoffoursolidyearsofwork. Istartedmyworkin January2011withastronginterestingeometryprocessing, optimization problems, rendering and GPGPU programming, but quite clueless about a suitable field of research. Little did I know that all my rather different interestswouldmagicallylineup... afewyearslater. isallwouldnothavebeenpossiblewithouttheconstantsupport,friendly encouragements and selfless contributions of my colleagues and cowork- ers.MysupervisorGüntherGreineralwayssupportedmeandgavemethe freedom to fully pursue my own research interests. Jochen Süßmuth in- troducedmetotheworldofnon-rigidregistrationandsupervisedmedur- ingmystudiesandinthefirstyearofmyPhD.MatthiasNießnerhelped me to rediscover my will to conduct research and fueled my curiosity in online 3D reconstruction methods. Marc Stamminger helped during all deadlineswithhisadviceandcraedalotofamazingillustrations. Frank Bauer, our Blender guru, shared his tricks and helped with the produc- tion of videos. I am thankful for all this support and feel in great debt. Whatimpressedmemostisthatallofmycolleaguesmanagedtoendure myconstantrantsaboutthealgorithmsthatbreakunderrealworldcondi- tions:ChristianSiegl,KaiSelgrad,MagdalenaPrus,MichaelMartinek,Jan Kretschmer, Quirin Meyer, Roberto Grosso, Matteo Colaianni, Matthias Innmann, Henry Schäfer, Benjamin Keinert, Franziska Bertelshofer and ChristophWeber. I feelprivileged for theopportunity towork in such a greatandcreativeenvironmentwithpeoplethathavebecomemorethan justcolleaguestomeoverthelastfouryears. Iwouldalsoliketothankallofmystudentsthatworkedwithmeduring theirBachelorandMasterprojects. Togetherwelearnedalotandgotex- posedtonewinterestingtopicsandideas. EzgiSerthelpeddevelopingthe presentedlatticebaseddeformationapproach. Justusieshelpedmesub- stantially with my first DFG project and the presented interactive model iii basedreconstructionmethod.Iamhappythathewillcontinuemyresearch atthechair. ankstoShahramIzadiforgivingmetheopportunitytoconductresearch in his group at Microso Research Cambridge. I met a lot of cool peo- pleduringmystay:AndrewFitzgibbon,ChristophRhemann,Christopher Zach,DavidKim,CemKeskinandSeanFanello.Itwasathrillingtimefull ofhardwork,excellentfoodandgreatparties. ankstoMatthiasNießner,MatthewFisher,ChengleiWuandChristian eobalt for helping with my real-time non-rigid reconstruction project andAngelaDaiforthenicevoiceover. IamalsogratefultotheGermanResearchFoundation(DFG)forfunding myworkoverthelastfouryearsundergrantSTA-662/3--1andGRK-1773. Lastbutnotleast,thankstomyparentsKerstinandFranzforsupporting meonallofmyendeavors<3. October 2014 MZ iv Contents 1 MotivationandFundamentals 1 1.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1.2 Real-TimeRGB-DSensors. . . . . . . . . . . . . . . . . . 2 1.3 GeneralPurposeGPUProgramming . . . . . . . . . . . . 7 2 OptimizationTheory 11 2.1 ModelFitting . . . . . . . . . . . . . . . . . . . . . . . . . 11 2.2 LeastSquaresOptimization . . . . . . . . . . . . . . . . . 13 2.3 LinearLeastSquaresOptimization . . . . . . . . . . . . . 15 2.4 Non-linearLeastSquaresOptimization . . . . . . . . . . . 16 3 ContributionandOutline 19 I ReconstructionofPersonalizedAvatars 23 4 Introduction 25 5 RelatedWork 27 6 Method 29 6.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 6.2 DataAcquisition . . . . . . . . . . . . . . . . . . . . . . . 29 6.2.1 DataPreparation . . . . . . . . . . . . . . . . . . 30 6.2.2 FaceandFeatureDetection . . . . . . . . . . . . . 31 6.2.3 FaceSegmentation. . . . . . . . . . . . . . . . . . 33 6.3 FittingaGenericFaceModel . . . . . . . . . . . . . . . . 33 6.3.1 FittingEnergyTerm . . . . . . . . . . . . . . . . . 35 6.3.2 RegularizationEnergyTerm . . . . . . . . . . . . 36 6.3.3 Optimization . . . . . . . . . . . . . . . . . . . . 36 6.3.4 Fittingthemorphablefacemodel . . . . . . . . . . 37 v Contents 7 Results 39 8 Conclusion 41 II ModelbasedReconstructionoftheHumanHead 43 9 Introduction 45 10 RelatedWork 47 10.1 Model-free3D-Reconstruction . . . . . . . . . . . . . . . 47 10.2 Model-based3D-Reconstruction . . . . . . . . . . . . . . 48 11 Method 51 11.1 PipelineOverview . . . . . . . . . . . . . . . . . . . . . . 51 11.2 HeadPoseEstimation . . . . . . . . . . . . . . . . . . . . 52 11.3 DataFusion . . . . . . . . . . . . . . . . . . . . . . . . . . 53 11.4 EstimatingModelParameters . . . . . . . . . . . . . . . . 54 11.4.1 StatisticalShapeModel . . . . . . . . . . . . . . . 54 11.4.2 ObjectiveFunction . . . . . . . . . . . . . . . . . 55 11.4.3 ParameterInitialization . . . . . . . . . . . . . . . 56 11.4.4 JointNon-LinearGPUOptimizer . . . . . . . . . 57 12 Results 59 12.1 RuntimeEvaluation . . . . . . . . . . . . . . . . . . . . . 59 12.2 ReconstructionQuality . . . . . . . . . . . . . . . . . . . 60 12.3 Applications . . . . . . . . . . . . . . . . . . . . . . . . . 60 13 Conclusion 65 III Online3DReconstructionatScale 67 14 Introduction 69 15 Relatedwork 71 vi

Description:
Together these techniques have the potential to redefine the way we think . the 3D world to the 2D image plane (blue) of the sensor, see Figure 1.4 (left). such as World of Warcraft, Aion or Second Life have gained tremendous.
See more

The list of books you might like

Most books are stored in the elastic cloud where traffic is expensive. For this reason, we have a limit on daily download.