Multimodal Scene Understanding: Algorithms, Applications and Deep Learning PDF
Preview Multimodal Scene Understanding: Algorithms, Applications and Deep Learning
Multimodal Scene Understanding Multimodal Scene Understanding Algorithms, Applications and Deep Learning Edited by Michael Ying Yang Bodo Rosenhahn Vittorio Murino AcademicPressisanimprintofElsevier 125LondonWall,LondonEC2Y5AS,UnitedKingdom 525BStreet,Suite1650,SanDiego,CA92101,UnitedStates 50HampshireStreet,5thFloor,Cambridge,MA02139,UnitedStates TheBoulevard,LangfordLane,Kidlington,OxfordOX51GB,UnitedKingdom Copyright©2019ElsevierInc.Allrightsreserved. Nopartofthispublicationmaybereproducedortransmittedinanyformorbyanymeans,electronicormechanical,including photocopying,recording,oranyinformationstorageandretrievalsystem,withoutpermissioninwritingfromthepublisher.Detailson howtoseekpermission,furtherinformationaboutthePublisher’spermissionspoliciesandourarrangementswithorganizationssuchas theCopyrightClearanceCenterandtheCopyrightLicensingAgency,canbefoundatourwebsite:www.elsevier.com/permissions. ThisbookandtheindividualcontributionscontainedinitareprotectedundercopyrightbythePublisher(otherthanasmaybenoted herein). Notices Knowledgeandbestpracticeinthisfieldareconstantlychanging.Asnewresearchandexperiencebroadenourunderstanding,changesin researchmethods,professionalpractices,ormedicaltreatmentmaybecomenecessary. Practitionersandresearchersmustalwaysrelyontheirownexperienceandknowledgeinevaluatingandusinganyinformation,methods, compounds,orexperimentsdescribedherein.Inusingsuchinformationormethodstheyshouldbemindfuloftheirownsafetyandthe safetyofothers,includingpartiesforwhomtheyhaveaprofessionalresponsibility. Tothefullestextentofthelaw,neitherthePublishernortheauthors,contributors,oreditors,assumeanyliabilityforanyinjuryand/or damagetopersonsorpropertyasamatterofproductsliability,negligenceorotherwise,orfromanyuseoroperationofanymethods, products,instructions,orideascontainedinthematerialherein. LibraryofCongressCataloging-in-PublicationData AcatalogrecordforthisbookisavailablefromtheLibraryofCongress BritishLibraryCataloguing-in-PublicationData AcataloguerecordforthisbookisavailablefromtheBritishLibrary ISBN:978-0-12-817358-9 ForinformationonallAcademicPresspublications visitourwebsiteathttps://www.elsevier.com/books-and-journals Publisher:MaraConner AcquisitionEditor:TimPitts EditorialProjectManager:JoshuaMearns ProductionProjectManager:KameshRamajogi Designer:VictoriaPearson TypesetbyVTeX Contents ListofContributors............................................................... vii Chapter1: IntroductiontoMultimodal SceneUnderstanding ..................... 1 Michael YingYang,BodoRosenhahn,VittorioMurino Chapter2: DeepLearning forMultimodal DataFusion........................... 9 AsakoKanezaki,RyoheiKuga,YusukeSugano,YasuyukiMatsushita Chapter3: MultimodalSemantic Segmentation: FusionofRGBand DepthDatain Convolutional Neural Networks....................................... 41 ZoltanKoppanyi,DorotaIwaszczuk, BingZha,CanJozef Saul,Charles K.Toth,Alper Yilmaz Chapter4: Learning Convolutional NeuralNetworksforObjectDetectionwith Very LittleTraining Data............................................. 65 ChristophReinders,HannoAckermann,Michael YingYang, Bodo Rosenhahn Chapter5: MultimodalFusionArchitecturesforPedestrianDetection ............ 101 DayanGuan,JiangxinYang,YanlongCao,MichaelYingYang, YanpengCao Chapter6: Multispectral PersonRe-IdentificationUsing GANforColor-to-Thermal ImageTranslation .................................................... 135 VladimirV.Kniaz,VladimirA.Knyaz Chapter7: AReviewand Quantitative Evaluation ofDirectVisual–Inertial Odometry ............................................................ 159 Lukas vonStumberg,VladyslavUsenko,DanielCremers v Contents Chapter8: MultimodalLocalizationforEmbeddedSystems:A Survey ............ 199 ImaneSalhi,MartynaPoreba,Erwan Piriou,ValerieGouet-Brunet, Maroun Ojail Chapter9: Self-SupervisedLearning fromWebDataforMultimodalRetrieval ... 279 RaulGomez, LluisGomez,JaumeGibert,DimosthenisKaratzas Chapter10:3DUrbanSceneReconstructionand InterpretationfromMultisensor Imagery .............................................................. 307 HaiHuang,Andreas Kuhn,MarioMichelini,MatthiasSchmitz, HelmutMayer Chapter11:DecisionFusionofRemote-SensingDataforLandCover Classification ......................................................... 341 Arnaud LeBris,NesrineChehata,WalidOuerghemmi, CyrilWendl, TristanPostadjian,AnnePuissant,Clément Mallet Chapter12:Cross-modalLearning by Hallucinating Missing ModalitiesinRGB-D Vision ................................................................ 383 Nuno C.Garcia,PietroMorerio,VittorioMurino Index ............................................................................. 403 vi List of Contributors HannoAckermann InstituteforInformationProcessing,LeibnizUniversityHanover,Hanover, Germany YanlongCao StateKeyLaboratoryofFluidPowerandMechatronicSystems,SchoolofMechanical Engineering,ZhejiangUniversity,Hangzhou,China KeyLaboratoryofAdvancedManufacturingTechnologyofZhejiangProvince,SchoolofMechani- calEngineering,ZhejiangUniversity,Hangzhou,China YanpengCao StateKeyLaboratoryofFluidPowerandMechatronicSystems,SchoolofMechani- calEngineering,ZhejiangUniversity,Hangzhou,China KeyLaboratoryofAdvancedManufacturingTechnologyofZhejiangProvince,SchoolofMechani- calEngineering,ZhejiangUniversity,Hangzhou,China NesrineChehata Univ.Paris-Est,LASTIGMATIS,IGN,ENSG,Saint-Mande,France EAG&EBordeauxINP,UniversitéBordeauxMontaigne,Pessac,France DanielCremers DepartmentofInformatics,TechnicalUniversityofMunich,Munich,Germany ArtisenseGmbH,Garching,Germany NunoC.Garcia PatternAnalysis&ComputerVision(PAVIS),IstitutoItalianodiTecnologia(IIT), Genova,Italy Universita’degliStudidiGenova,Genova,Italy JaumeGibert Eurecat,CentreTecnològicdeCatalunya,UnitatdeTecnologiesAudiovisuals, Barcelona,Spain LluisGomez ComputerVisionCenter,UniversitatAutònomadeBarcelona,Barcelona,Spain RaulGomez Eurecat,CentreTecnològicdeCatalunya,UnitatdeTecnologiesAudiovisuals, Barcelona,Spain ComputerVisionCenter,UniversitatAutònomadeBarcelona,Barcelona,Spain ValerieGouet-Brunet Univ.Paris-Est,LASTIGMATIS,IGN,ENSG,Saint-Mande,France DayanGuan StateKeyLaboratoryofFluidPowerandMechatronicSystems,SchoolofMechanical Engineering,ZhejiangUniversity,Hangzhou,China KeyLaboratoryofAdvancedManufacturingTechnologyofZhejiangProvince,SchoolofMechani- calEngineering,ZhejiangUniversity,Hangzhou,China HaiHuang InstituteforAppliedComputerScience,BundeswehrUniversityMunich,Neubiberg, Germany DorotaIwaszczuk Photogrammetry&RemoteSensing,TechnicalUniversityofMunich,Munich, Germany AsakoKanezaki NationalInstituteofAdvancedIndustrialScienceandTechnology,Tokyo,Japan vii ListofContributors DimosthenisKaratzas ComputerVisionCenter,UniversitatAutònomadeBarcelona,Barcelona, Spain VladimirV.Kniaz StateRes.InstituteofAviationSystems(GosNIIAS),Moscow,Russia MoscowInstituteofPhysicsandTechnology(MIPT),Moscow,Russia VladimirA.Knyaz StateRes.InstituteofAviationSystems(GosNIIAS),Moscow,Russia MoscowInstituteofPhysicsandTechnology(MIPT),Moscow,Russia ZoltanKoppanyi DepartmentofCivil,Envir.&GeodEng.,TheOhioStateUniversity,Columbus, OH,UnitedStates RyoheiKuga GraduateSchoolofInformationScienceandTechnology,OsakaUniversity,Osaka, Japan AndreasKuhn InstituteforAppliedComputerScience,BundeswehrUniversityMunich,Neubiberg, Germany ArnaudLeBris Univ.Paris-Est,LASTIGMATIS,IGN,ENSG,Saint-Mande,France ClémentMallet Univ.Paris-Est,LASTIGMATIS,IGN,ENSG,Saint-Mande,France YasuyukiMatsushita GraduateSchoolofInformationScienceandTechnology,OsakaUniversity, Osaka,Japan HelmutMayer InstituteforAppliedComputerScience,BundeswehrUniversityMunich,Neu- biberg,Germany MarioMichelini InstituteforAppliedComputerScience,BundeswehrUniversityMunich,Neu- biberg,Germany PietroMorerio PatternAnalysis&ComputerVision(PAVIS),IstitutoItalianodiTecnologia(IIT), Genova,Italy VittorioMurino PatternAnalysis&ComputerVision(PAVIS),IstitutoItalianodiTecnologia(IIT), Genova,Italy Universita’degliStudidiVerona,Verona,Italy MarounOjail CEA,LIST,Gif-sur-YvetteCedex,France WalidOuerghemmi Univ.Paris-Est,LASTIGMATIS,IGN,ENSG,Saint-Mande,France Aix-MarseilleUniversité,CNRSESPACEUMR7300,Aix-en-Provence,France ErwanPiriou CEA,LIST,Gif-sur-YvetteCedex,France MartynaPoreba Univ.Paris-Est,LASTIGMATIS,IGN,ENSG,Saint-Mande,France TristanPostadjian Univ.Paris-Est,LASTIGMATIS,IGN,ENSG,Saint-Mande,France AnnePuissant CNRSUMR7362LIVE-UniversitédeStrasbourg,Strasbourg,France ChristophReinders InstituteforInformationProcessing,LeibnizUniversityHanover,Hanover, Germany BodoRosenhahn InstituteforInformationProcessing,LeibnizUniversityHanover,Hanover,Ger- many ImaneSalhi CEA,LIST,Gif-sur-YvetteCedex,France Univ.Paris-Est,LASTIGMATIS,IGN,ENSG,Saint-Mande,France CanJozefSaul RobertCollege,Istanbul,Turkey MatthiasSchmitz InstituteforAppliedComputerScience,BundeswehrUniversityMunich,Neu- biberg,Germany viii ListofContributors YusukeSugano GraduateSchoolofInformationScienceandTechnology,OsakaUniversity,Osaka, Japan CharlesK.Toth DepartmentofCivil,Envir.&GeodEng.,TheOhioStateUniversity,Columbus, OH,UnitedStates VladyslavUsenko DepartmentofInformatics,TechnicalUniversityofMunich,Munich,Germany LukasvonStumberg DepartmentofInformatics,TechnicalUniversityofMunich,Munich,Ger- many ArtisenseGmbH,Garching,Germany CyrilWendl Univ.Paris-Est,LASTIGMATIS,IGN,ENSG,Saint-Mande,France StudentatEcolePolytechniqueFédéraledeLausanne(EPFL),Lausanne,Switzerland JiangxinYang StateKeyLaboratoryofFluidPowerandMechatronicSystems,SchoolofMechani- calEngineering,ZhejiangUniversity,Hangzhou,China KeyLaboratoryofAdvancedManufacturingTechnologyofZhejiangProvince,SchoolofMechani- calEngineering,ZhejiangUniversity,Hangzhou,China MichaelYingYang SceneUnderstandingGroup,UniversityofTwente,Enschede,TheNetherlands AlperYilmaz DepartmentofCivil,Envir.&GeodEng.,TheOhioStateUniversity,Columbus,OH, UnitedStates BingZha DepartmentofCivil,Envir.&GeodEng.,TheOhioStateUniversity,Columbus,OH, UnitedStates ix
The list of books you might like
