An Exploration of the Adaptive Neuro-Fuzzy Inference System (ANFIS)

212 Pages·2013·5.02 MB·English

An Exploration of the Adaptive Neuro-Fuzzy
Inference System (ANFIS) in Modelling
Survival
By
Hazlina Hamdan, BSc, MSc
Thesis submitted to The University of Nottingham
for the Degree of Doctor of Philosophy
School of Computer Science
The University of Nottingham
Nottingham, United Kingdom
March 2013
An Exploration of the Adaptive Neuro-Fuzzy Inference
System (ANFIS) in Modelling Survival
Hazlina Hamdan
Submitted for the degree of Doctor of Philosophy
March 2013
Abstract
Medical prognosis is the prediction of the future course and outcome of a disease and
an indication of the likelihood of recovery from that disease. Prognosis is important be-
causeitisusedtoguidethetypeandintensityofthemedicationadministeredtopatients.
Patients are usually concerned with how long they will survive after diagnosis. Survival
analysis describes the analysis of data that corresponds to the time from when an indi-
vidual enters a study until the occurrence of some particular event or end-point. It is
concerned with the comparison of survival curves for different combinations of risk fac-
tors. Analyticalmethodsthataretransparentfortheclinician’sunderstandingandexplain
individualinferencesneedtobeconsideredwhendealingwithmedicaldata.
This thesis describes a methodology for modelling survival by utilising the applica-
tionoftheAdaptiveNeuro-FuzzyInferenceSystem(ANFIS).Ahybridintelligentsystem
whichcombinesthefuzzylogicqualitativeapproachandadaptiveneuralnetworkcapabil-
itiestowardsbetterperformance. TheANFISapproachwasappliedinmodellingsurvival
of breast cancer based on patient groups derived from the Nottingham Prognostic Index
(NPI). A comparison of the proposed method with the existing methods in the capability
to predict the survival rate is presented. The use of a fuzzy inference system (FIS) in
modellingsurvivalisexpectedtoofferthecapabilitytodelivertheprocessofturningdata
intoknowledgethatcanbeunderstoodbypeople.
The design of rules can be performed either by human experts or using appropriate
iii
approaches to build high quality FIS to represent the knowledge. In this thesis, represent
an automatic generation of membership functions and rules from the data. Further, cor-
responding subsequent adjustments have been made to the model to give towards more
satisfactoryperformance. Thefinalpremiseandconsequentparametersobtainedarethen
usedtopredictthesurvivalforeachtimeinterval.
A framework for modelling survival with the application of fuzzy inference system
andback-propagationneuralnetworkwasdevelopedandisdescribedinthisthesis. Inthis
framework, a different way of partitioning the input space can be selected to define the
membershipfunctionsforexamplesusingexpertknowledge,equaliserpartitioning,fuzzy
c-means or subtractive clustering techniques. Further, the rule base can be established by
enumerating all possible combinations of membership functions of all inputs. After the
initialisationofthefuzzyinferencestructure,thereplicationdata(untiltimetoevent)will
be subject to training using the gradient descent and nonnegative least square algorithm
toestimatetheconditionaleventprobability. Thisframeworkisvalidatedoverasynthetic
datasetandanoveldatasetofpatientsfollowingoperativesurgery ofovariancancer. The
proposed framework can be applied to estimate the hazard and survival curve between
differentprognosticfactorsandsurvivaltimewiththeexplanationcapabilities.
Declaration
The work in this thesis is based on research carried out at the Intelligent Modelling and
AnalysisResearchGroup,theSchoolofComputerScience,theUniversityofNottingham,
England. No part of this thesis has been submitted elsewhere for any other degree or
qualificationanditallmyownworkunlessreferencedtothecontraryinthetext.
Copyright© 2013byHazlinaHamdan.
“The copyright of this thesis rests with the author. No quotations from it should be pub-
lished without the author’s prior written consent and information derived from it should
beacknowledged”.
iv
Acknowledgments
Firstly, it gives me great pleasure to take this opportunity to thank Prof. Jonathan M.
Garibaldi for invaluable guidance and supervision in making this research possible for
me. ThanksalsototheheadofIntelligentModellingandAnalysis(IMA)researchgroup,
Prof. Uwe Aickelin for giving me the opportunity to study my PhD in the IMA research
group.
Beside that, I would like to express my appreciation to Dr. Andy Green and all the
peopleinvolvedinthebreastcancerresearchwithwhichIhavecollaborated,forproviding
the data and also guidance and sharing of vast experience in the field of breast cancer.
Many thanks also to Prof. Lindy Durrant from Department of Oncology, University of
Nottingham,forprovidingmewiththedataforvalidationofthelastpieceofthiswork.
I would also like to thank the Ministry of Higher Education (MOHE) of Malaysia
and the Universiti Putra Malaysia (UPM) for the doctoral scholarship and other financial
supportthroughoutthecourseofmyPhDstudy.
Special thanks to my lovely husband, Kamal Taib with his encouragement and indi-
rectlyhelpinginmakingthisresearchareality,mydaughterNurAtiqah,mysonMuham-
mad Asyraf and Muhammad Afiq for their love, patience and understanding throughout
the study. Another sincere thank you message to my family in Malaysia for all their
supportwithdailyprayersallthetimeformyfamilyhereformanyyears.
To all IMA Group members, thank you for the friendly working environment in the
group,especiallyfriendsinroomB38.
v
Contents
Abstract ii
Acknowledgments v
ListofFigures xv
ListofTables xvii
ListofSymbols xviii
1 Introduction 1
1.1 BackgroundandMotivation . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 Objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.3 ExpectedOutcome . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.4 OverviewofThesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
2 LiteratureReview 9
2.1 SurvivalAnalysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
2.2 ArtificialIntelligenceTechniquesinCancerPrediction . . . . . . . . . . 15
2.3 ArtificialNeuralNetworks . . . . . . . . . . . . . . . . . . . . . . . . . 16
2.3.1 ANNapproachinBreastCancersurvivalprediction . . . . . . . . 20
2.3.2 PartialLogisticArtificialNeuralNetworkinmodellingsurvival . 22
2.3.3 IssuesofANNapplicationincancerprediction . . . . . . . . . . 25
2.4 FuzzySystems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
vi
Contents vii
2.4.1 Fuzzysetsandmembershipfunctions . . . . . . . . . . . . . . . 28
2.4.2 Linguisticvariablesandfuzzyrules . . . . . . . . . . . . . . . . 30
2.4.3 Fuzzyinferencesystem . . . . . . . . . . . . . . . . . . . . . . . 33
2.4.4 Automaticdeterminationoffuzzysetsandrules . . . . . . . . . . 36
2.4.5 Fuzzyapplicationsincancerprediction . . . . . . . . . . . . . . 43
2.5 AdaptiveNeuro-FuzzyInferenceSystem . . . . . . . . . . . . . . . . . . 46
2.5.1 ForwardPass . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
2.5.2 BackwardPass . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
2.6 Datamanipulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
2.6.1 Datapre-processing . . . . . . . . . . . . . . . . . . . . . . . . 53
2.6.2 Datareplicationuntiltimetoevent . . . . . . . . . . . . . . . . . 53
2.7 PerformanceEvaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
2.8 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
3 OntheuseofFuzzyInferenceinModellingBreastCancerSurvival 57
3.1 BreastCancerdataset . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
3.2 ComparisonofPLANNandANFISinmodellingsurvival . . . . . . . . . 59
3.2.1 PLANNconfigurations . . . . . . . . . . . . . . . . . . . . . . . 59
3.2.2 ANFISconfigurations . . . . . . . . . . . . . . . . . . . . . . . 62
3.2.3 Datareplication . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
3.2.4 ResultsandDiscussion . . . . . . . . . . . . . . . . . . . . . . . 66
3.3 ANFISmethodinhandlingcontinuousvalues . . . . . . . . . . . . . . . 69
3.3.1 ResultsandDiscussion . . . . . . . . . . . . . . . . . . . . . . . 70
3.4 Problemonthenegativevalueofconditionaleventprobability . . . . . . 77
3.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
4 RefinementofANFISforModellingSurvival 79
4.1 ANFISlearningandparameterupdate . . . . . . . . . . . . . . . . . . . 79
4.2 ProposedalterationtoANFISupdatemechanism . . . . . . . . . . . . . 81
Contents viii
4.3 Experimentssettings . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
4.4 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84
4.5 ValidationusingcategoricalNPI . . . . . . . . . . . . . . . . . . . . . . 91
4.6 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94
4.7 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
5 AutomaticDeterminationofMembershipFunctionsandRules 100
5.1 InitialisationofFuzzyModel . . . . . . . . . . . . . . . . . . . . . . . . 101
5.2 Methodsconsidered . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102
5.2.1 TheEqualizedUniversemethod . . . . . . . . . . . . . . . . . . 103
5.2.2 Fuzzyc-meansclustering . . . . . . . . . . . . . . . . . . . . . . 105
5.2.3 Subtractiveclustering . . . . . . . . . . . . . . . . . . . . . . . . 117
5.3 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124
5.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128
6 AFrameworkforAutomaticModellingofSurvivalusingFuzzyInference 130
6.1 Frameworkprocessandparametersetting . . . . . . . . . . . . . . . . . 133
6.2 Frameworkconfigurations . . . . . . . . . . . . . . . . . . . . . . . . . 137
6.3 Validationover‘synthetic’dataset . . . . . . . . . . . . . . . . . . . . . 137
6.3.1 Experimentalsetting . . . . . . . . . . . . . . . . . . . . . . . . 138
6.3.2 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141
6.3.3 ApplyingsyntheticdatasetintothePLANNmodel . . . . . . . . 143
6.4 ValidationoverOvarianCancerdataset . . . . . . . . . . . . . . . . . . . 146
6.4.1 Experimentalsetting . . . . . . . . . . . . . . . . . . . . . . . . 147
6.4.2 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149
6.4.3 Prognosticindexofepithelialovariancancer(PIEPOC) . . . . . . 150
6.5 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164
6.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165
Contents ix
7 Conclusions 167
7.1 Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 168
7.2 ThesisLimitation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172
7.3 Futurework . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172
7.4 Dissemination . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 174
7.4.1 Papers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 174
7.4.2 Presentations . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175
A AJCCBreastCancerStaging 177
References 180
List of Figures
2.1 Survivaltimeofeightpatients(Collet,1994). . . . . . . . . . . . . . . . 10
2.2 Kaplan-MeierSurvivalCurveof10breastcancerpatients. . . . . . . . . 14
2.3 Thebiologicalneuron(Simon,2012). . . . . . . . . . . . . . . . . . . . 16
2.4 Activationfunctions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
2.5 Back-propagationANN. . . . . . . . . . . . . . . . . . . . . . . . . . . 19
2.6 PLANNarchitecture(Biganzolietal.,1998). . . . . . . . . . . . . . . . 23
2.7 Crispsetsoftallmen. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
2.8 Fuzzysetsoftallmen. . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
2.9 Exampleofmembershipfunctionsforfuzzysets. . . . . . . . . . . . . . 30
2.10 Fuzzysetsofshort,averageandtallmen(Negnevitsky,2005). . . . . . . 31
2.11 Fuzzysetoperations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
2.12 FuzzyInferenceSystem. . . . . . . . . . . . . . . . . . . . . . . . . . . 34
2.13 First-orderSugenofuzzymodel(Jang&Sun,1995). . . . . . . . . . . . 35
2.14 MembershipfunctionofNPIusingequalizeduniversemethod. . . . . . . 37
2.15 Equalizeduniversemethodintoninefuzzyregions(Jang,1993). . . . . . 38
2.16 Clusteringbasedpartitioning. . . . . . . . . . . . . . . . . . . . . . . . . 43
2.17 AdaptiveNeuro-FuzzyInferenceSystem(ANFIS)(Jang,1993). . . . . . 46
2.18 Gaussianmembershipfunctionwithcentre(c)andwidth(σ) . . . . . . 48
i i
3.1 ConditionalEventProbabilityofPLANNmodel(withhiddenlayernodes=12
andweightdecay=0.075). . . . . . . . . . . . . . . . . . . . . . . . . . . 61
x