Deep Learning Applications, Volume 3 (Advances in Intelligent Systems and Computing) PDF
Preview Deep Learning Applications, Volume 3 (Advances in Intelligent Systems and Computing)
Advances in Intelligent Systems and Computing 1395 M. Arif Wani Bhiksha Raj Feng Luo Dejing Dou Editors Deep Learning Applications, Volume 3 Advances in Intelligent Systems and Computing Volume 1395 SeriesEditor JanuszKacprzyk,SystemsResearchInstitute,PolishAcademyofSciences, Warsaw,Poland AdvisoryEditors NikhilR.Pal,IndianStatisticalInstitute,Kolkata,India RafaelBelloPerez,FacultyofMathematics,PhysicsandComputing, UniversidadCentraldeLasVillas,SantaClara,Cuba EmilioS.Corchado,UniversityofSalamanca,Salamanca,Spain HaniHagras,SchoolofComputerScienceandElectronicEngineering, UniversityofEssex,Colchester,UK LászlóT.Kóczy,DepartmentofAutomation,SzéchenyiIstvánUniversity, Gyor,Hungary VladikKreinovich,DepartmentofComputerScience,UniversityofTexas atElPaso,ElPaso,TX,USA Chin-TengLin,DepartmentofElectricalEngineering,NationalChiao TungUniversity,Hsinchu,Taiwan JieLu,FacultyofEngineeringandInformationTechnology, UniversityofTechnologySydney,Sydney,NSW,Australia PatriciaMelin,GraduateProgramofComputerScience,TijuanaInstitute ofTechnology,Tijuana,Mexico NadiaNedjah,DepartmentofElectronicsEngineering,UniversityofRiodeJaneiro, RiodeJaneiro,Brazil NgocThanhNguyen ,FacultyofComputerScienceandManagement, WrocławUniversityofTechnology,Wrocław,Poland JunWang,DepartmentofMechanicalandAutomationEngineering, TheChineseUniversityofHongKong,Shatin,HongKong Theseries“AdvancesinIntelligentSystemsandComputing”containspublications on theory, applications, and design methods of Intelligent Systems and Intelligent Computing.Virtuallyalldisciplinessuchasengineering,naturalsciences,computer and information science, ICT, economics, business, e-commerce, environment, healthcare,lifesciencearecovered.Thelistoftopicsspansalltheareasofmodern intelligentsystemsandcomputingsuchas:computationalintelligence,softcomput- ingincludingneuralnetworks,fuzzysystems,evolutionarycomputingandthefusion oftheseparadigms,socialintelligence,ambientintelligence,computationalneuro- science, artificial life, virtual worlds and society, cognitive science and systems, Perception and Vision, DNA and immune based systems, self-organizing and adaptive systems, e-Learning and teaching, human-centered and human-centric computing, recommender systems, intelligent control, robotics and mechatronics including human-machine teaming, knowledge-based paradigms, learning para- digms,machineethics,intelligentdataanalysis,knowledgemanagement,intelligent agents, intelligent decision making and support, intelligent network security, trust management,interactiveentertainment,Webintelligenceandmultimedia. The publications within “Advances in Intelligent Systems and Computing” are primarily proceedings of important conferences, symposia and congresses. They cover significant recent developments in the field, both of a foundational and applicable character. An important characteristic feature of the series is the short publication time and world-wide distribution. This permits a rapid and broad disseminationofresearchresults. Indexed by DBLP, INSPEC, WTI Frankfurt eG, zbMATH, Japanese Science and TechnologyAgency(JST). AllbookspublishedintheseriesaresubmittedforconsiderationinWebofScience. Moreinformationaboutthisseriesathttp://www.springer.com/series/11156 · · · M. Arif Wani Bhiksha Raj Feng Luo Dejing Dou Editors Deep Learning Applications, Volume 3 Editors M.ArifWani BhikshaRaj DepartmentofComputerScience LanguageTechnologiesInstitute UniversityofKashmir CarnegieMellonUniversity Srinagar,India Pittsburgh,PA,USA FengLuo DejingDou ClemsonUniversity UniversityofOregon Clemson,SC,USA Eugene,OR,USA ISSN2194-5357 ISSN2194-5365 (electronic) AdvancesinIntelligentSystemsandComputing ISBN978-981-16-3356-0 ISBN978-981-16-3357-7 (eBook) https://doi.org/10.1007/978-981-16-3357-7 ©TheEditor(s)(ifapplicable)andTheAuthor(s),underexclusivelicensetoSpringerNature SingaporePteLtd.2022 Thisworkissubjecttocopyright.AllrightsaresolelyandexclusivelylicensedbythePublisher,whether thewholeorpartofthematerialisconcerned,specificallytherightsoftranslation,reprinting,reuse ofillustrations,recitation,broadcasting,reproductiononmicrofilmsorinanyotherphysicalway,and transmissionorinformationstorageandretrieval,electronicadaptation,computersoftware,orbysimilar ordissimilarmethodologynowknownorhereafterdeveloped. Theuseofgeneraldescriptivenames,registerednames,trademarks,servicemarks,etc.inthispublication doesnotimply,evenintheabsenceofaspecificstatement,thatsuchnamesareexemptfromtherelevant protectivelawsandregulationsandthereforefreeforgeneraluse. Thepublisher,theauthorsandtheeditorsaresafetoassumethattheadviceandinformationinthisbook arebelievedtobetrueandaccurateatthedateofpublication.Neitherthepublishernortheauthorsor theeditorsgiveawarranty,expressedorimplied,withrespecttothematerialcontainedhereinorforany errorsoromissionsthatmayhavebeenmade.Thepublisherremainsneutralwithregardtojurisdictional claimsinpublishedmapsandinstitutionalaffiliations. ThisSpringerimprintispublishedbytheregisteredcompanySpringerNatureSingaporePteLtd. The registered company address is: 152 Beach Road, #21-01/04 Gateway East, Singapore 189721, Singapore Preface Machinelearningalgorithmshaveinfluencedmanyaspectsofourday-to-dayliving and transformed major industries around the world. At the frontier of machine learning innovation are deep learning systems, a class of multi-layered networks capable of automatically learning meaningful hierarchical representations from a variety of structured and unstructured data. Breakthroughs in deep learning allow us to generate new representations, extract knowledge, and draw inferences from raw images, video streams, text and speech, time series, and other complex data types.Thesepowerfuldeeplearningmethodsarebeingappliedtonewandexciting real-worldproblemsinmedicaldiagnostics,factoryautomation,publicsafety,envi- ronmental sciences, autonomous transportation, military applications, and much more. Thisbookexploressomeofthelatestapplicationsindeeplearningandincludes avarietyofarchitecturesandnoveldeeplearningtechniques,whicharedescribedin twelvechapters.EachchapterisaccompaniedbyPythonSoftware/Codealongwith testdatasets.Asummaryofthesechaptersisgivenbelow. Chapter1discussesDeepRapidClassAugmentation(DeepRCA)fordeepneural networks.TheapproachattemptstoeliminatetheissuesassociatedwithCatastrophic Forgetting. Deep RCA’s ability to train only on the new task data while jointly optimizingacrossalltasksisshowntoreduceaugmentationtrainingtimesanddata storagerequirements.ThismakesDeepRCAwellsuitedforapplicationsthatrequire rapidsequentiallearningandwheredatastoragecapacityislimited. Chapter2presentsananalysisofusingtheBERTmodelforfivelanguageswith richmorphology(Finnish,Czech,Hungarian,Turkish,Japanese).Theworkevaluates cross-lingual, multilingual, and monolingual BERT models and three non-BERT- basedmodelsonfivemorphologicallyrichlanguagesandcomparestheperformance ofthemodelsontheEnglishlanguage. Chapter3discussesfusionofRGBandDepthimagesforimprovingtheperfor- manceofobjectdetectionincomplexscenes.Earlyfusionarchitectureispresented thatemploysanunsupervisedlearningdepthestimationtechniquetoautomatically infer a dense depth image from a single RGB input image. The depth image is v vi Preface concatenated to the RGB image to perform object detection using deep learning models. Chapter4presentsdimensionestimationusingAutoencoders(AEs).Dimension estimationattemptstoestimatetheintrinsicdimensionalityornumberoflatentvari- ables in a set of measurements of a random vector. The work discusses AE archi- tectural choices and regularization techniques that allow it to transform AE latent layerrepresentationsintoestimatesofintrinsicdimension.Theeffectivenessofthe techniques is demonstrated on benchmark image processing problems, analysis of financialmarkets,andnetworksecurity. Chapter 5 discusses the problem of high demand for computing and storing resources associated with the deep learning approach. Transforming the original large pre-trained networks into new smaller models, by utilizing Model Compres- sionandAcceleration(MCA)techniquesisdescribed.WithintheMCAframework, a clustering-based approach is discussed that is able to increase the number of employedcentroids/representativesforanaccelerationgain.Thetheoreticalacceler- ationgainsarepresentedandthekeysystemhyper-parametersthataffectthegainare identified.Extensiveevaluationstudiesarecarriedoutusingvariousstate-of-the-art DNN models trained on image classification and object detection tasks to validate theeffectivenessoftheclusteringmethodinMCA. Chapter 6 presents Deep Learning-based time series forecasting. The work describesRNN,CNN,andTransformerbasedstructuresfortimeseriesforecasting. Experimentsareperformedtocomparethesetechniquesonfourdifferentpublicly availabledatasets. Chapter7discussesDeepEvidentialActiveLearning(DEAL)forImageClassifi- cation.ActiveLearning(AL)ispresentedtomitigatetheproblemoflimitedlabeled data.AnovelALalgorithmthatefficientlylearnsfromunlabeleddatabycapturing highpredictionuncertaintyisdiscussed.ByreplacingtheSoftMaxstandardoutput ofaCNNwiththeparametersofaDirichletdensity,themodellearnstoidentifydata instancesthatcontributeinimprovingmodelperformanceduringtraining.Results of several experiments performed on publicly available data are presented. On the real-worldmedicaldatasetsforpneumoniadetectioninchestX-rayimagesarealso presented. Chapter8discussesConvolutionalNeuralNetworkwithLatentBinarizationfor large-scalemulti-classclassification.Error-correctingoutputcodesareincorporated intoconvolutionalneuralnetworksbyinsertingalatentbinarizationlayerintheCNN classification layer. This approach encapsulates both encoding and decoding steps of error-correcting output codes into a single CNN architecture that is capable of discoveringanoptimalcodingmatrixduringtraining.Theresultsofapplyingthese modelstoseveralimagerecognitiontasksarepresented. Chapter 9 discusses a framework for deployment of pre-trained deep learning models on robots. An open-source framework, that is easy to deploy and is inde- pendent of the underlying deep learning models, is employed. The framework is testedandevaluatedonsevenpre-traineddeeplearningmodelsforhazarddetection in supermarket floors in a simulated environment. Experimental results of testing Preface vii the framework in deploying the deep learning models in Robot Operating System environmentarepresented. Chapter10discussesextendedFeatureAugmentedandTransformedGAN(FAT- GAN)modelforconditionalsimulationofElectron–ProtonScatteringEventswith Variate Beam Energies. The model produces inclusive event feature distributions andcorrelationsforacontinuousrangeofreactionenergiesbyautomaticallyinter- polatingandextrapolatingfromasetoftrainedenergies.Acontinuousenergyfeature representationisemployedtoenablethenetworkstolearnthedistributionrelation- shipsbetweendifferentreactionenergies,layingthegroundworkforaccessingevents atuntrainedenergies. Chapter 11 discusses the practical challenges of performing multi-agent data processing. The issues of coordinating connected Distributed Energy Resources (DERs) in the Internet of Things (IoT) are discussed. The results on case studies regardingtheissuesofloadforecastingandcoordinationchallengesarepresented. Chapter12discussestheuseofDeepLearningapproachfordonorjourneytasks. ExperimentalresultsofusingseveralvariantsofRecurrentNeuralNetworks(RNNs) andConvolutionalNeuralNetworks(CNNs),aswellaswithvarioussizedwindows, ontheabovetaskarepresented. PythonSoftware/CodeAssociatedwitheachchapter Srinagar,India M.ArifWani Pittsburgh,USA BhikshaRaj Clemson,USA FengLuo Eugene,USA DejingDou Contents DeepRapidClassAugmentation;ANewProgressiveLearning ApproachthatEliminatestheIssueofCatastrophicForgetting ........ 1 HannaWitzgall AComprehensiveAnalysisofSubwordContextualEmbeddings forLanguageswithRichMorphology ............................... 31 ArdaAkdemir,TetsuoShibuya,andTungaGüngör RGBandDepthImageFusionforObjectDetectionUsingDeep Learning ......................................................... 73 FahimehFarahnakianandJukkaHeikkonen DimensionEstimationUsingAutoencodersandApplication ........... 95 NitishBahadur,BrianLewandowski,andRandyPaffenroth ANewClustering-BasedTechniquefortheAccelerationofDeep ConvolutionalNetworks ............................................ 123 ErionVasilisPikoulis, ChristosMavrokefalidis, StavrosNousias, andArisS.Lalos DeepLearning-BasedTimeSeriesForecasting ....................... 151 KushagraAgarwal,LalasaDheekollu,GauravDhama,AnkurArora, SiddharthaAsthana,andTanmoyBhowmik DEAL:DeepEvidentialActiveLearningforImageClassification ...... 171 PatrickHemmer,NiklasKühl,andJakobSchöffer LB-CNN:ConvolutionalNeuralNetworkwithLatentBinarization forLargeScaleMulti-classClassification ............................ 193 TimothyReeseandYuMichaelZhu EfficientDeploymentofDeepLearningModelsonAutonomous RobotsintheROSEnvironment .................................... 215 M.G.SarwarMurshed,JamesJ.Carroll,NazarKhan,andFarazHussain ix x Contents cFAT-GAN:ConditionalSimulationofElectron–ProtonScattering Events with Variate Beam Energies by a Feature Augmented andTransformedGenerativeAdversarialNetwork ................... 245 LuisaVelasco, EvanMcClellan, N.Sato, PawelAmbrozewicz, TianboLiu, W.Melnitchouk, MichelleKuchera, YasirAlanazi, andYaohangLi Building Power Grid 2.0: Deep Learning and Federated ComputationsforEnergyDecarbonizationandEdgeResilience ........ 263 CarolynGoodman,JesseThornburg,ShankarKoduvayurRamaswami, andJavadMohammadi Deep Learning the Donor Journey with Convolutional andRecurrentNeuralNetworks .................................... 295 GregLee,AjithKumarRaghavan,andMarkHobbs AuthorIndex ...................................................... 321
The list of books you might like
