Machine Learning for Adaptive Many-Core Machines - A Practical Approach PDF
Preview Machine Learning for Adaptive Many-Core Machines - A Practical Approach
Studies in Big Data 7 Noel Lopes Bernardete Ribeiro Machine Learning for Adaptive Many- Core Machines – A Practical Approach Studies in Big Data Volume 7 Serieseditor JanuszKacprzyk,PolishAcademyofSciences,Warsaw,Poland e-mail:[email protected] Forfurthervolumes: http://www.springer.com/series/11970 AboutthisSeries Theseries“StudiesinBigData”(SBD)publishesnewdevelopmentsandadvances in the variousareas of Big Data- quickly and with a high quality. The intent is to coverthetheory,research,development,andapplicationsofBigData,asembedded inthefieldsofengineering,computerscience,physics,economicsandlifesciences. The booksof the seriesrefer to the analysisand understandingof large,complex, and/ordistributeddatasetsgeneratedfromrecentdigitalsourcescomingfromsen- sors or other physical instruments as well as simulations, crowd sourcing, social networksor other internet transactions, such as emails or video click streams and other.Theseriescontainsmonographs,lecturenotesandeditedvolumesinBigData spanningtheareasofcomputationalintelligenceincl.neuralnetworks,evolutionary computation,softcomputing,fuzzysystems, as wellas artificialintelligence,data mining, modern statistics and Operationsresearch, as well as self-organizingsys- tems. Of particular value to both the contributorsand the readership are the short publicationtimeframeandtheworld-widedistribution,whichenablebothwideand rapiddisseminationofresearchoutput. · Noel Lopes Bernardete Ribeiro Machine Learning for Adaptive Many-Core Machines – A Practical Approach ABC NoelLopes BernardeteRibeiro PolytechnicInstituteofGuarda DepartmentofInformaticsEngineering Guarda FacultyofSciencesandTechnology Portugal UniversityofCoimbra,PoloII Coimbra Portugal ISSN2197-6503 ISSN2197-6511 (electronic) ISBN978-3-319-06937-1 ISBN978-3-319-06938-8 (eBook) DOI10.1007/978-3-319-06938-8 SpringerChamHeidelbergNewYorkDordrechtLondon LibraryofCongressControlNumber:2014939947 (cid:2)c SpringerInternationalPublishingSwitzerland2015 Thisworkissubjecttocopyright.AllrightsarereservedbythePublisher,whetherthewholeorpartof thematerialisconcerned,specificallytherightsoftranslation,reprinting,reuseofillustrations,recitation, broadcasting,reproductiononmicrofilmsorinanyotherphysicalway,andtransmissionorinformation storageandretrieval,electronicadaptation,computersoftware,orbysimilarordissimilarmethodology nowknownorhereafterdeveloped.Exemptedfromthislegalreservationarebriefexcerptsinconnection with reviews or scholarly analysis or material supplied specifically for the purpose of being entered and executed on a computer system, for exclusive use by the purchaser of the work. Duplication of this publication or parts thereof is permitted only under the provisions of the Copyright Law of the Publisher’slocation,initscurrentversion,andpermissionforusemustalwaysbeobtainedfromSpringer. PermissionsforusemaybeobtainedthroughRightsLinkattheCopyrightClearanceCenter.Violations areliabletoprosecutionundertherespectiveCopyrightLaw. Theuseofgeneraldescriptivenames,registerednames,trademarks,servicemarks,etc.inthispublication doesnotimply,evenintheabsenceofaspecificstatement,thatsuchnamesareexemptfromtherelevant protectivelawsandregulationsandthereforefreeforgeneraluse. Whiletheadviceandinformationinthisbookarebelievedtobetrueandaccurateatthedateofpub- lication,neithertheauthorsnortheeditorsnorthepublishercanacceptanylegalresponsibilityforany errorsoromissionsthatmaybemade.Thepublishermakesnowarranty,expressorimplied,withrespect tothematerialcontainedherein. Printedonacid-freepaper SpringerispartofSpringerScience+BusinessMedia(www.springer.com) To SaraandPedro To my family Noel Lopes To Miguel andAlexander To my family BernardeteRibeiro Preface Motivationand Scope Todaytheincreasingcomplexity,performancerequirementsandcostofcurrent(and future) applications in society is transversal to a wide range of activities, from science to business and industry. In particular, this is a fundamental issue in the Machine Learning (ML) area, which is becoming increasingly relevant in a wide diversity of domains. The scale of the data from Web growth and advances in sensor data collectiontechnologyhave beenrapidlyincreasingthe magnitudeand complexityoftasksthatMLalgorithmshavetosolve. Much of the data that we are generating and capturing will be available “indefinitely” since it is considered a strategic asset from which useful and valuable information can be extracted. In this context, Machine Learning (ML) algorithms play a vital role in providing new insights from the abundant streams and increasingly large repositories of data. However, it is well-known that the computational complexity of ML methodologies, often directly related with the amount of data, is a limiting factor that can render the application of many algorithms to real-world problems impractical. Thus, the challenge consists of processing such large quantities of data in a realistic (useful) time frame, which drivestheneedtoextendtheapplicabilityofexistingMLalgorithmsandtodevise parallelalgorithmsthatscale well with the volumeof data or, in other words, can handle“BigData”. This volume takes a practical approach for addressing this problematic, by presentingwaystoextendtheapplicabilityofwell-knownMLalgorithmswiththe help of high-scalable Graphics Processing Unit (GPU) parallel implementations. Modern GPUs are highly parallel devices that can perform general-purpose computations, yielding significant speedups for many problems in a wide range of areas. Consequently, the GPU, with its many cores, represents a novel and compellingsolutiontotackletheaforementionedproblem,byprovidingthemeans toanalyzeandstudylargerdatasets. VIII Preface Rationally, we can not view the GPU implementations of ML algorithms as a universal solution for the “Big Data” challenges, but rather as part of the answer, which may requirethe use of differentstrategiescoupledtogether. In this perspective, this volume addresses other strategies, such as using instance-based selection methods to choose a representative subset of the original training data, whichcaninturnbeusedtobuildmodelsinafractionofthetimeneededtoderivea modelfromthecompletedataset.Nevertheless,largescaledatasetsanddatastreams may require learning algorithms that scale roughly linearly with the total amount of data. Hence, traditional batch algorithms may not be up to the challenge and thereforethebookalsoaddressesincrementallearningalgorithmsthatcontinuously adjusttheirmodelswithupcomingnewdata.Theseembodythepotentialtohandle the gradual concept drifts inherent to data streams and non-stationary dynamic databases. Finally,inpracticalscenarios,theawarenessofhandlinglargequantitiesofdata is often exacerbated by the presence of incomplete data, which is an unavoidable problemformostreal-worlddatabases.Therefore,thisvolumealsopresentsanovel strategy fordealing with this ubiquitousproblemthatdoesnotaffectsignificantly eitherthealgorithmsperformanceorthepreprocessingburden. The book is not intended to be a comprehensive survey of the state-of-the-art of the broad field of Machine Learning. Its purpose is less ambitious and more practical: to explain and illustrate some of the more important methods brought to a practical view of GPU-based implementation in part to respond to the new challengesoftheBigData. PlanandOrganization Thebookcomprehendsninechaptersandoneappendix.Thechaptersareorganized intofourparts:thefirstpartrelatingtofundamentaltopicsinMachineLearningand GraphicsProcessingUnitsenclosesthefirsttwochapters;thesecondpartincludes fourchaptersandgivesthemainsupervisedlearningalgorithms,includingmethods to handle missing data and approaches for instance-based learning; the third part withtwochaptersconcernsunsupervisedandsemi-supervisedlearningapproaches; in the fourthpartwe concludethe bookwith a summaryofmany-corealgorithms approaches and techniques developed across this volume and give new trends to scale up algorithmsto many-coreprocessors. The self-containedchaptersprovide anenlightenedviewoftheinterplaybetweenMLandGPUapproaches. Chapter 1 details the Machine Learning challenges on Big Data, gives an overviewof the topicsincludedin the book,and containsbackgroundmaterialon MLformulatingtheproblemsettingandthemainlearningparadigms. Chapter2presentsanewopen-sourceGPUMLlibrary(GPUMachineLearning Library–GPUMLib)thataimsatprovidingthebuildingblocksforthedevelopment ofefficientGPUMLsoftware.Inthiscontext,weanalyzethepotentialoftheGPU in the ML area, coveringits evolution.Moreover,an overviewof the existingML Preface IX GPU parallel implementations is presented and we argue for the need of a GPU ML library. We then present the CUDA (Compute Unified Device Architecture) programming model and architecture, which was used to develop GPU Machine LearningLibrary(GPUMLib)andwedetailitsarchitecture. Chapter 3 reviews the fundamentals of Neural Networks, in particular, the multi-layered approaches and investigates techniques for reducing the amount of time necessary to build NN models. Specifically, it focuses on details of a GPU parallel implementation of the Back-Propagation (BP) and Multiple Back- Propagation (MBP) algorithms. An Autonomous Training System (ATS) that reducessignificantlytheeffortnecessaryforbuildingNNmodelsisalsodiscussed. A practicalapproachtosupporttheeffectivenessof theproposedsystemsonboth benchmarkandreal-worldproblemsispresented. Chapter 4 analysesthe treatmentof missing data and alternativesto dealwith this ubiquitous problem generated by numerous causes. It reviews missing data mechanisms as well as methods for handling Missing Values (MVs) in Machine Learning.Unlikepre-processingtechniques,suchas imputation,a novelapproach Neural Selective Input Model (NSIM) is introduced. Its application on several datasets with both different distributions and proportion of MVs shows that the NSIM approach is very robust and yields good to excellent results. With the scalabilityinmindaGPUparalellimplementationofNeuralSelectiveInputModel (NSIM)tocopewithBigDataisdescribed. Chapter 5 considers a class of learning mechanisms known as the Support Vector Machines (SVMs). It provides a general view of the machine learning framework and describes formally the SVMs as large margin classifiers. It explorestheSequentialMinimalOptimization(SMO)algorithmasanoptimization methodologyto solve an SVM. The rest of the chapteris dedicatedto the aspects related to its implementation in multi-thread CPU and GPU platforms. We also present a comprehensive comparison of the evaluation methods on benchmark datasets and on real-world case studies. We intend to give a clear understanding ofspecificaspectsrelatedtotheimplementationofbasicSVMmachinesinamany- core perspective. Further deploymentof other SVM variants are essential for Big Dataanalyticsapplications. Chapter 6 addresses incremental learning algorithms where the models incorporate new information on a sample-by-sample basis. It introduces a novel algorithm the Incremental Hypersphere Classifier Incremental Hypersphere Classifier (IHC) which presents good properties in terms of multi-class support, complexity, scalability and interpretability. The IHC is tested in well-known benchmarks yielding good classification performance results. Additionally, it can beusedasaninstanceselectionmethodsinceitpreservesclassboundarysamples. Details of its application to a real case study in the field of bioinformatics are provided. Chapter 7 deals with unsupervised and semi-supervised learning algorithms. It presents the Non-Negative Matrix Factorization (NMF) algorithm as well as a new semi-supervised method,designated by Semi-Supervised NMF (SSNMF). In addition,thisChapteralsocoversahybridNMF-basedfacerecognitionapproach. X Preface Chapter8motivatesforthedeeplearningarchitectures.Itstartsbyintroducing theRestrictedBoltzmannMachines(RBMs)andtheDeepBeliefNetworks(DBNs) models.Beingunsupervisedlearningapproachestheirimportanceisshowninmul- tiple facetsspecifically bythefeaturegenerationthroughmanylayers,contrasting with shallowarchitectures.We addresstheir GPU parallelimplementationsgiving adetailedexplanationofthekernelsinvolved.Itincludesanextensiveexperiment, involvingtheMNISTdatabaseofhand-writtendigitsandtheHHrecomulti-stroke symboldatabaseinordertogainabetterunderstandingoftheDBNs. InthefinalChapter9wegiveanextendedsummaryofthecontributionsofthe book.Inadditionwepresentresearchtrendswithspecialfocusonthebigdataand streamcomputing.Finally,tomeetfuturechallengesonreal-timebigdataanalysis fromthousandsof sourcesnew platformsshouldbe exploitedto accelerate many- coresoftwareresearch. Audience The book is designed for practitioners and researchers in the areas of Machine Learning (ML) and GPU computing (CUDA) and is suitable for postgraduate studentsincomputerscience,engineering,informationtechnologyandotherrelated disciplines. Previous backgroundin the areas of ML or GPU computing (CUDA) willbebeneficial,althoughweattempttocoverthebasicsofthesetopics. Acknowledgments Wewouldliketoacknowledgeandthankallthosewhohavecontributedtobringing thisbooktopublicationfortheirhelp,supportandinput. We thank many stimulating user’s requirementsto include new perspectivesin the GPUMLib due to many downloads of the software. It turn out possible to improveandextendmanyaspectsofthelibrary. WealsowishtothankthesupportofthePolytechnicInstituteofGuardaandof theCentreofInformaticsandSystemsoftheInformaticsEngineeringDepartment, FacultyofScienceandTechnologies,UniversityofCoimbra,forthemeansprovided duringtheresearch. Our thanks to Samuel Walter Best who reviewed the syntactic aspects of the book. Ourspecialthanksandappreciationtooureditor,ProfessorJanuszKacprzyk,of StudiesinBigData,Springer,forhisessentialencouragement. Lastly,toourfamiliesandfriendsfortheirloveandsupport. Coimbra,Portugal NoelLopes February2014 BernardeteRibeiro
Description:The list of books you might like
