Improving Classifier Generalization. Real-Time Machine Learning based Applications PDF
Preview Improving Classifier Generalization. Real-Time Machine Learning based Applications
Studies in Computational Intelligence 989 Rahul Kumar Sevakula Nishchal K. Verma Improving Classifier Generalization Real-Time Machine Learning based Applications Studies in Computational Intelligence Volume 989 SeriesEditor JanuszKacprzyk,PolishAcademyofSciences,Warsaw,Poland The series “Studies in Computational Intelligence” (SCI) publishes new developments and advances in the various areas of computational intelligence—quickly and with a high quality. The intent is to cover the theory, applications, and design methods of computational intelligence, as embedded in the fields of engineering, computer science, physics and life sciences, as well as themethodologiesbehindthem.Theseriescontainsmonographs,lecturenotesand editedvolumesincomputationalintelligencespanningtheareasofneuralnetworks, connectionist systems, genetic algorithms, evolutionary computation, artificial intelligence, cellular automata, self-organizing systems, soft computing, fuzzy systems,andhybridintelligentsystems.Ofparticularvaluetoboththecontributors and the readership are the short publication timeframe and the world-wide distribution,whichenablebothwideandrapiddisseminationofresearchoutput. IndexedbySCOPUS,DBLP,WTIFrankfurteG,zbMATH,SCImago. AllbookspublishedintheseriesaresubmittedforconsiderationinWebofScience. · Rahul Kumar Sevakula Nishchal K. Verma Improving Classifier Generalization Real-Time Machine Learning based Applications RahulKumarSevakula NishchalK.Verma SignalProcessing DepartmentofElectricalEngineering WhoopInc IndianInstituteofTechnologyKanpur Boston,MA,USA Kanpur,India ISSN 1860-949X ISSN 1860-9503 (electronic) StudiesinComputationalIntelligence ISBN 978-981-19-5072-8 ISBN 978-981-19-5073-5 (eBook) https://doi.org/10.1007/978-981-19-5073-5 ©SpringerNatureSingaporePteLtd.2023 Thisworkissubjecttocopyright.AllrightsarereservedbythePublisher,whetherthewholeorpartof thematerialisconcerned,specificallytherightsoftranslation,reprinting,reuseofillustrations,recitation, broadcasting,reproductiononmicrofilmsorinanyotherphysicalway,andtransmissionorinformation storageandretrieval,electronicadaptation,computersoftware,orbysimilarordissimilarmethodology nowknownorhereafterdeveloped. Theuseofgeneraldescriptivenames,registerednames,trademarks,servicemarks,etc.inthispublication doesnotimply,evenintheabsenceofaspecificstatement,thatsuchnamesareexemptfromtherelevant protectivelawsandregulationsandthereforefreeforgeneraluse. Thepublisher,theauthors,andtheeditorsaresafetoassumethattheadviceandinformationinthisbook arebelievedtobetrueandaccurateatthedateofpublication.Neitherthepublishernortheauthorsor theeditorsgiveawarranty,expressedorimplied,withrespecttothematerialcontainedhereinorforany errorsoromissionsthatmayhavebeenmade.Thepublisherremainsneutralwithregardtojurisdictional claimsinpublishedmapsandinstitutionalaffiliations. ThisSpringerimprintispublishedbytheregisteredcompanySpringerNatureSingaporePteLtd. The registered company address is: 152 Beach Road, #21-01/04 Gateway East, Singapore 189721, Singapore Preface Classificationalgorithmsformthebasisofdecision-makinginmostpatternrecog- nitionproblems,e.g.imagerecognition,speechandspeakerrecognition,irisrecog- nition, and spam mail detection. With the horizon of their applications expanding at a fast pace, the need for further research has only increased. This fact becomes particularly true because (a) each application poses its own set of challenges and (b)onewouldalwaysfindaclassifierwithaparticularimprovisationthatbestsuits thesituation.Nomatterwhichclassificationapproachisused,generalizationisan importantaspect.Generalizationessentiallyindicateshowwellthetrainedclassifier worksinrealtime,i.e.onunseentestdata. This monograph begins with the fundamentals of classifiers, bias-variance tradeoff, statistical learning theory (SLT), probably approximate correct (PAC) framework,maximummarginclassifiers,andpopularmethodswhichimprovegener- alization like regularization, boosting, transfer learning, dropout in deep learning, etc.Furthermore,themonographsolvesfourindependentproblemsthathavegreat relevanceforcertainreal-timeapplications. Thefirstpartofthemonographaimsatfindingclassifierswhichexhibitextremely lowvariance.Classificationalgorithmsaretraditionallydesignedtosimultaneously reduce errors caused by bias as well as variance. In many situations, low variance becomesextremelycrucialforgettingtangibleclassificationsolutionsandevenslight overfittingcanhaveseriousconsequencesonthetestresults.Classifierswithlowvari- ancehavetwomainadvantages:(1)theclassifierstatisticallymanagestokeepthe testerrorsclosetothetrainingerror,and(2)theclassifierlearnseffectivelyevenwith asmallnumberofsamples.Themonographintroducesaclassofclassifierscalled MajorityVotePointClassifier(MVPC),whichonaccountofthelowerVapnikCher- vonenkis(VC)dimensioncanexhibitlowervariancethanevenlinearclassifiers.The monographcontributesbyestimatingatrendfortheMVPclassifier’sVCdimension, andvalidatesitslowvarianceontworeal-timeproblems. The monograph then focuses on the real-time application of condition-based monitoring of machines using acoustic and vibration measurements. Signal data acquired from machines are often found to change with time, wear and tear, and subsequent repair of the machine. Classifiers are typically trained to perform the v vi Preface decision-makingprocedureduringfaultdiagnosis/detection.Sincedatamaychange withtime,lowgeneralizationerrorisessentialtoavoidoverfittingduringclassifica- tion.Therefore,MVPCisseentobebestsuitedforthissituation.However,MVPC hasalimitationthatitmaynotbeabletofitthedatasufficiently,andmayhavehigh trainingerrors.Themonographpresentsanovelframeworkforpatternrecognition, where novel procedures for optimal data source (sensitive position) identification, dataacquisition,andfeatureselectionaretailoredtogivethebestpossibletraining performancewiththeMVPclassifier.TheunderstandinghereisthatifMVPgiveslow trainingerror,real-timefaultdiagnosisofmachinesbecomesfeasiblewithconsistent accuracy.Theintroducedframeworkwasexperimentallyimplementedandtestedfor anaircompressorconditionmonitoringapplication;associatedreal-timeexperiments showedasignificantimprovementinthereliabilityoffaultdetection. ThethirdpartofthemonographfocusesondealingwithclassnoiseintheFuzzy SupportVectorMachine(FSVM)classifier.FSVMisconsideredtobeasignificant additionoversoftmarginSVMlikeC-SVM,fortheformercanguardagainstoutlier sensitivity and the latter cannot. The ability of FSVM to absorb outliers strongly depends on how well the training samples are assigned fuzzy membership values (MVs).Traditionally,themembershipfunctions(MFs)usedforFSVMwerecustom- madeforapplications,andMFsusedforonecouldingeneralnotbeusedforothers. Toovercomethelimitation,GeneralPurposeMembershipFunctions(GPMFs)are definedinthismonographasthoseMFswhichcanuniversallybeusedformultiple applications, and which allow FSVM to statistically perform better than C-SVM. ThemonographcontributestotheGPMFliteratureintwostages.Firstlywithhelp of convex hulls, it presents a few limitations that FSVM faces while treating all samplesofaclasswithasingleMF.Further,itrecommendsdifferentialtreatmentof databycategorizingthemintotwofuzzysets:onecontainingpossiblenon-outliers andtheothercontainingpossibleoutliers.Whilepossibleoutliersaremodeledwith anormalMF,possiblenon-outliersarerecommendedtohaveaconstantMVof‘1’. The chapter then introduces novel GPMFs which use clustering-based techniques to detect possible outliers, and use Hausdorff Distance and pt-set to characterize those possible outliers. To establish conclusions, the introduced GPMFs are thor- oughlyevaluatedandstatisticallycomparedwithearlierGPMFsonnumerousreal- worldbenchmarkdatasets.TheresultsshowthatproposedGPMFsnotonlyperform significantly better in treating class noise, but also execute with efficient run time complexity. Finally, a novel scheme to introduce deep learning in Fuzzy Rule-based classi- fiers(FRCs)ispresented.FRCshavegainedprominencefortheiruniqueabilityof givinggoodclassificationperformance,andallowingexistingexpertknowledgetobe usedconjointlywithtrainingdata.RecentinnovationsinDeepNeuralNetworksare allowingresearcherstotacklesomeverycomplexproblemswithimprovedtheoret- icalandempiricaljustifications,e.g.imageclassificationandaudioclassification.The monographpresentsaschemetoincorporatestackeddenoisingsparseautoencoders withintheFRCframework.Whilestackingofdenoisingsparseautoencodershelps learnthecomplexnon-linearrelationshipsamongdataandrepresenttheinputdata inareduced compact feature space, theframework builttoward FRC allows users Preface vii to input expert knowledge to the system. To make denoising sparse autoencoders learn more effectively, data pre-processing strategies have been proposed. Further, to improve the classification performance and rule reduction performance of the FRC,threefine-tuningstrategieshavealsobeenproposed.Theproposedframework istestedacrossreal-worldbenchmarkdatasets,andanelaboratecomparisonacross literature shows that proposed methods are capable of building FRCs that provide state-of-the-artaccuraciesand/orafewrules,aspertheuser’sdemand. The monograph ends with an epilogue, on the use of autoencoders in transfer learning, tumor classification, and condition monitoring problems. Furthermore, pertainingtotheresearchcontributionsmadeherein,thedirectionsforpossiblefuture workhavealsobeendiscussed. Boston,USA RahulKumarSevakula Kanpur,India NishchalK.Verma Contents 1 Introduction ................................................... 1 1.1 Basics ................................................... 1 1.2 StatisticalLearningTheoryandVCDimension ................ 3 1.2.1 VCDimension ..................................... 3 1.2.2 Probably Approximate Correct (PAC) Learning Framework ......................................... 4 1.2.3 GrowthFunction .................................... 6 1.3 Bias-VarianceTrade-Off .................................... 6 1.3.1 Occam’sRazor ..................................... 7 1.4 Outline .................................................. 8 References ..................................................... 9 2 MethodsUsedtoImproveGeneralizationPerformance ............ 11 2.1 MaximumMarginClassifier ................................ 12 2.1.1 UpperBoundonVCDimensionofLinearClassifiers .... 12 2.1.2 SupportVectorMachine(SVM) ....................... 13 2.2 BaggingandBoosting ...................................... 15 2.3 Semi-SupervisedLearning,ActiveLearning,andTransfer Learning ................................................. 16 2.4 Regularization ............................................ 17 2.5 Stochastic Gradient Descent, Momentum, Adaptive LearningRates,Dropout,SkipConnections,Etc. ............... 18 2.6 Conclusions .............................................. 20 References ..................................................... 20 3 MVPC—AClassifierwithVeryLowVCDimension ............... 23 3.1 MajorityVoteClassifier .................................... 23 3.1.1 TheMajorityVotePoint(MVP)Classifier .............. 24 3.1.2 ImplementationofMVPClassifier .................... 25 ix x Contents 3.2 EvaluatingandComparingVCDimension .................... 25 3.2.1 UpperBoundonVCDimensionofMVPClassifier ...... 25 3.2.2 EmpiricalEstimationofVCDimension ................ 29 3.2.3 ComparingwithVCDimensionofLinearClassifiers ..... 32 3.3 CaseStudies .............................................. 32 3.3.1 RotatingMachineFaultDiagnosis ..................... 32 3.3.2 BiologicalData ..................................... 35 3.4 Summary ................................................ 38 References ..................................................... 39 4 FrameworkforReliableFaultDetectionwithSensorData ......... 41 4.1 LiteratureSurveyonFaultDiagnosisinRotatingMachines ...... 41 4.2 ProblemFormulationandSummary .......................... 44 4.3 Data Acquisition Framework to Simulate Real-Time Environment .............................................. 44 4.4 DataPre-processing ....................................... 46 4.4.1 NormalizationRobusttoOutliers ...................... 47 4.5 FeatureExtraction ......................................... 51 4.6 FeatureSelectionAlgorithmUsingGraphicalIndices ........... 54 4.6.1 FeatureRanking .................................... 55 4.6.2 DatasetRejection ................................... 60 4.6.3 DatasetRetrieval .................................... 62 4.6.4 FeatureSelectionArchitecture ........................ 62 4.7 Classification ............................................. 63 4.8 SensitivePositionAnalysis(SPA) ............................ 64 4.8.1 ParameterRangeIdentification(PRI) .................. 65 4.9 CaseStudyonAirCompressorFaultDetection ................ 66 4.9.1 LeakageInletValve(LIV)FaultDetection .............. 66 4.9.2 LeakageOutletValve(LOV)FaultDetection ........... 69 4.9.3 OfflineTesting ..................................... 70 4.9.4 Real-TimeTesting .................................. 72 4.10 Summary ................................................ 73 References ..................................................... 74 5 MembershipFunctionsforFuzzySupportVectorMachine inaNoisyEnvironment ......................................... 77 5.1 DealingwithClassNoise ................................... 78 5.2 FuzzySupportVectorMachine(FSVM) ...................... 79 5.2.1 AvailableMembershipFunctionsforFSVM ............ 80 5.2.2 LimitationsofEarlierMembershipFunctions ........... 81 5.2.3 ConvexHullAnalysisonFSVM ...................... 82 5.3 Set Measures—Distance Measure Between Points andNon-emptySets ....................................... 85 5.3.1 DistanceBetweenaPointandaNon-emptySet ......... 86 5.3.2 HausdorffDistance(HD) ............................. 87
The list of books you might like
