Variational Algorithms for Approximate Bayesian Inference

281 Pages·2003·3.88 MB·English

by  Matthew James Beal

VARIATIONAL ALGORITHMS FOR
APPROXIMATE BAYESIAN INFERENCE
by
Matthew J. Beal
M.A., M.Sci., Physics, University of Cambridge, UK (1998)
TheGatsbyComputationalNeuroscienceUnit
UniversityCollegeLondon
17 QueenSquare
LondonWC1N3AR
AThesissubmittedforthedegreeof
DoctorofPhilosophyoftheUniversityofLondon
May2003
Abstract
TheBayesianframeworkformachinelearningallowsfortheincorporationofpriorknowledge
in a coherent way, avoids overfitting problems, and provides a principled basis for selecting
between alternative models. Unfortunately the computations required are usually intractable.
This thesis presents a unified variational Bayesian (VB) framework which approximates these
computationsinmodelswithlatentvariablesusingalowerboundonthemarginallikelihood.
Chapter1presentsbackgroundmaterialonBayesianinference,graphicalmodels,andpropaga-
tionalgorithms. Chapter2formsthetheoreticalcoreofthethesis,generalisingtheexpectation-
maximisation (EM) algorithm for learning maximum likelihood parameters to the VB EM al-
gorithmwhichintegratesovermodelparameters. Thealgorithmisthenspecialisedtothelarge
family of conjugate-exponential (CE) graphical models, and several theorems are presented to
pave the road for automated VB derivation procedures in both directed and undirected graphs
(BayesianandMarkovnetworks,respectively).
Chapters 3-5 derive and apply the VB EM algorithm to three commonly-used and important
models: mixtures of factor analysers, linear dynamical systems, and hidden Markov models.
It is shown how model selection tasks such as determining the dimensionality, cardinality, or
number of variables are possible using VB approximations. Also explored are methods for
combining sampling procedures with variational approximations, to estimate the tightness of
VB bounds and to obtain more effective sampling algorithms. Chapter 6 applies VB learning
to a long-standing problem of scoring discrete-variable directed acyclic graphs, and compares
the performance to annealed importance sampling amongst other methods. Throughout, the
VB approximation is compared to other methods including sampling, Cheeseman-Stutz, and
asymptotic approximations such as BIC. The thesis concludes with a discussion of evolving
directions for model selection including infinite models and alternative approximations to the
marginallikelihood.
2
Acknowledgements
I am very grateful to my advisor Zoubin Ghahramani for his guidance in this work, bringing
energyandthoughtfulinsightintoeveryoneofourdiscussions. Iwouldalsoliketothankother
senior Gatsby Unit members including Hagai Attias, Phil Dawid, Peter Dayan, Geoff Hinton,
CarlRasmussenandSamRoweis,fornumerousdiscussionsandinspirationalcomments.
My research has been punctuated by two internships at Microsoft Research in Cambridge and
inRedmond. Whilstthisthesisdoesnotcontainresearchcarriedoutintheselabs,Iwouldlike
to thank colleagues there for interesting and often seductive discussion, including Christopher
Bishop,AndrewBlake,DavidHeckerman,NebojsaJojicandNeilLawrence.
AmongstmanyothersIwouldliketothankespeciallythefollowingpeoplefortheirsupportand
usefulcomments: AndrewBrown,NandodeFreitas,OliverDowns,AlexGray,YoelHaitovsky,
Sham Kakade, Alex Korenberg, David MacKay, James Miskin, Quaid Morris, Iain Murray,
Radford Neal, Simon Osindero, Lawrence Saul, Matthias Seeger, Amos Storkey, Yee-Whye
Teh,EricTuttle,NaonoriUeda,JohnWinn,ChrisWilliams,andAngelaYu.
I should thank my friends, in particular Paola Atkinson, Tania Lillywhite, Amanda Parmar,
James Tinworth and Mark West for providing me with various combinations of shelter, com-
panionship and retreat during my time in London. Last, but by no means least I would like
to thank my family for their love and nurture in all my years, and especially my dear fiance´e
CassandreCreswellforherlove,encouragementandendlesspatiencewithme.
TheworkinthisthesiswascarriedoutattheGatsbyComputationalNeuroscienceUnitwhichis
fundedbytheGatsbyCharitableFoundation. IamgratefultotheInstituteofPhysics,theNIPS
foundation,theUCLgraduateschoolandMicrosoftResearchforgeneroustravelgrants.
3
Contents
Abstract 2
Acknowledgements 3
Contents 4
Listoffigures 8
Listoftables 11
Listofalgorithms 12
1 Introduction 13
1.1 Probabilisticinference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
1.1.1 Probabilisticgraphicalmodels: directedandundirectednetworks . . . 17
1.1.2 Propagationalgorithms . . . . . . . . . . . . . . . . . . . . . . . . . . 19
1.2 Bayesianmodelselection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
1.2.1 MarginallikelihoodandOccam’srazor . . . . . . . . . . . . . . . . . 25
1.2.2 Choiceofpriors. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
1.3 PracticalBayesianapproaches . . . . . . . . . . . . . . . . . . . . . . . . . . 32
1.3.1 Maximumaposteriori(MAP)parameterestimates . . . . . . . . . . . 33
1.3.2 Laplace’smethod . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
1.3.3 Identifiability: aliasinganddegeneracy . . . . . . . . . . . . . . . . . 35
1.3.4 BICandMDL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
1.3.5 Cheeseman&Stutz’smethod . . . . . . . . . . . . . . . . . . . . . . 37
1.3.6 MonteCarlomethods . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
1.4 Summaryoftheremainingchapters . . . . . . . . . . . . . . . . . . . . . . . 42
2 VariationalBayesianTheory 44
2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
2.2 VariationalmethodsforML/MAPlearning . . . . . . . . . . . . . . . . . . . 46
2.2.1 Thescenarioforparameterlearning . . . . . . . . . . . . . . . . . . . 46
2.2.2 EMforunconstrained(exact)optimisation . . . . . . . . . . . . . . . 48
4
Contents Contents
2.2.3 EMwithconstrained(approximate)optimisation . . . . . . . . . . . . 49
2.3 VariationalmethodsforBayesianlearning . . . . . . . . . . . . . . . . . . . . 53
2.3.1 Derivingthelearningrules . . . . . . . . . . . . . . . . . . . . . . . . 53
2.3.2 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
2.4 Conjugate-Exponentialmodels . . . . . . . . . . . . . . . . . . . . . . . . . . 64
2.4.1 Definition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
2.4.2 VariationalBayesianEMforCEmodels . . . . . . . . . . . . . . . . . 66
2.4.3 Implications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
2.5 Directedandundirectedgraphs . . . . . . . . . . . . . . . . . . . . . . . . . . 73
2.5.1 Implicationsfordirectednetworks . . . . . . . . . . . . . . . . . . . . 73
2.5.2 Implicationsforundirectednetworks . . . . . . . . . . . . . . . . . . 74
2.6 ComparisonsofVBtoothercriteria . . . . . . . . . . . . . . . . . . . . . . . 75
2.6.1 BICisrecoveredfromVBinthelimitoflargedata . . . . . . . . . . . 75
2.6.2 ComparisontoCheeseman-Stutz(CS)approximation . . . . . . . . . . 76
2.7 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80
3 VariationalBayesianHiddenMarkovModels 82
3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82
3.2 InferenceandlearningformaximumlikelihoodHMMs . . . . . . . . . . . . . 83
3.3 BayesianHMMs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88
3.4 VariationalBayesianformulation . . . . . . . . . . . . . . . . . . . . . . . . . 91
3.4.1 DerivationoftheVBEMoptimisationprocedure . . . . . . . . . . . . 92
3.4.2 PredictiveprobabilityoftheVBmodel . . . . . . . . . . . . . . . . . 97
3.5 Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98
3.5.1 Synthetic: discoveringmodelstructure . . . . . . . . . . . . . . . . . 98
3.5.2 Forwards-backwardsEnglishdiscrimination . . . . . . . . . . . . . . . 99
3.6 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104
4 VariationalBayesianMixturesofFactorAnalysers 106
4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106
4.1.1 Dimensionalityreductionusingfactoranalysis . . . . . . . . . . . . . 107
4.1.2 Mixturemodelsformanifoldlearning . . . . . . . . . . . . . . . . . . 109
4.2 BayesianMixtureofFactorAnalysers . . . . . . . . . . . . . . . . . . . . . . 110
4.2.1 ParameterpriorsforMFA . . . . . . . . . . . . . . . . . . . . . . . . 111
4.2.2 InferringdimensionalityusingARD . . . . . . . . . . . . . . . . . . . 114
4.2.3 VariationalBayesianderivation . . . . . . . . . . . . . . . . . . . . . 115
4.2.4 Optimisingthelowerbound . . . . . . . . . . . . . . . . . . . . . . . 119
4.2.5 Optimisingthehyperparameters . . . . . . . . . . . . . . . . . . . . . 122
4.3 Modelexploration: birthanddeath . . . . . . . . . . . . . . . . . . . . . . . . 124
4.3.1 Heuristicsforcomponentdeath . . . . . . . . . . . . . . . . . . . . . 126
4.3.2 Heuristicsforcomponentbirth . . . . . . . . . . . . . . . . . . . . . . 127
5
Contents Contents
4.3.3 Heuristicsfortheoptimisationendgame . . . . . . . . . . . . . . . . . 130
4.4 Handlingthepredictivedensity . . . . . . . . . . . . . . . . . . . . . . . . . . 130
4.5 Syntheticexperiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132
4.5.1 Determiningthenumberofcomponents . . . . . . . . . . . . . . . . . 133
4.5.2 EmbeddedGaussianclusters . . . . . . . . . . . . . . . . . . . . . . . 133
4.5.3 Spiraldataset . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135
4.6 Digitexperiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138
4.6.1 Fully-unsupervisedlearning . . . . . . . . . . . . . . . . . . . . . . . 138
4.6.2 ClassificationperformanceofBICandVBmodels . . . . . . . . . . . 141
4.7 CombiningVBapproximationswithMonteCarlo . . . . . . . . . . . . . . . . 144
4.7.1 Importancesamplingwiththevariationalapproximation . . . . . . . . 144
4.7.2 Example: TightnessofthelowerboundforMFAs . . . . . . . . . . . . 148
4.7.3 Extendingsimpleimportancesampling . . . . . . . . . . . . . . . . . 151
4.8 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157
5 VariationalBayesianLinearDynamicalSystems 159
5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159
5.2 TheLinearDynamicalSystemmodel . . . . . . . . . . . . . . . . . . . . . . 160
5.2.1 Variablesandtopology . . . . . . . . . . . . . . . . . . . . . . . . . . 160
5.2.2 Specificationofparameterandhiddenstatepriors . . . . . . . . . . . . 163
5.3 Thevariationaltreatment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 168
5.3.1 VBMstep: Parameterdistributions. . . . . . . . . . . . . . . . . . . . 170
5.3.2 VBEstep: TheVariationalKalmanSmoother . . . . . . . . . . . . . . 173
5.3.3 Filter(forwardrecursion) . . . . . . . . . . . . . . . . . . . . . . . . . 174
5.3.4 Backwardrecursion: sequentialandparallel . . . . . . . . . . . . . . . 177
5.3.5 Computingthesingleandjointmarginals . . . . . . . . . . . . . . . . 181
5.3.6 Hyperparameterlearning . . . . . . . . . . . . . . . . . . . . . . . . . 184
5.3.7 CalculationofF . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185
5.3.8 Modificationswhenlearningfrommultiplesequences . . . . . . . . . 186
5.3.9 Modificationsforafullyhierarchicalmodel . . . . . . . . . . . . . . . 189
5.4 SyntheticExperiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189
5.4.1 Hiddenstatespacedimensionalitydetermination(noinputs) . . . . . . 189
5.4.2 Hiddenstatespacedimensionalitydetermination(input-driven) . . . . 191
5.5 Elucidatinggeneexpressionmechanisms . . . . . . . . . . . . . . . . . . . . 195
5.5.1 Generalisationerrors . . . . . . . . . . . . . . . . . . . . . . . . . . . 198
5.5.2 Recoveringgene-geneinteractions . . . . . . . . . . . . . . . . . . . . 200
5.6 Possibleextensionsandfutureresearch . . . . . . . . . . . . . . . . . . . . . 201
5.7 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 204
6 Learning the structure of discrete-variable graphical models with hidden vari-
ables 206
6
Contents Contents
6.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 206
6.2 CalculatingmarginallikelihoodsofDAGs . . . . . . . . . . . . . . . . . . . . 207
6.3 Estimatingthemarginallikelihood . . . . . . . . . . . . . . . . . . . . . . . . 210
6.3.1 MLandMAPparameterestimation . . . . . . . . . . . . . . . . . . . 210
6.3.2 BIC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 212
6.3.3 Cheeseman-Stutz . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213
6.3.4 TheVBlowerbound . . . . . . . . . . . . . . . . . . . . . . . . . . . 215
6.3.5 AnnealedImportanceSampling(AIS) . . . . . . . . . . . . . . . . . . 218
6.3.6 Upperboundsonthemarginallikelihood . . . . . . . . . . . . . . . . 222
6.4 Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 223
6.4.1 ComparisonofscorestoAIS . . . . . . . . . . . . . . . . . . . . . . . 226
6.4.2 Performanceaveragedovertheparameterprior . . . . . . . . . . . . . 232
6.5 Openquestionsanddirections . . . . . . . . . . . . . . . . . . . . . . . . . . 236
6.5.1 AISanalysis,limitations,andextensions . . . . . . . . . . . . . . . . 236
6.5.2 Estimatingdimensionalitiesoftheincompleteandcomplete-datamodels 245
6.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 247
7 Conclusion 250
7.1 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 250
7.2 Summaryofcontributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 254
AppendixA ConjugateExponentialfamilyexamples 259
AppendixB Usefulresultsfrommatrixtheory 262
B.1 Schurcomplementsandinvertingpartitionedmatrices . . . . . . . . . . . . . . 262
B.2 Thematrixinversionlemma . . . . . . . . . . . . . . . . . . . . . . . . . . . 263
AppendixC Miscellaneousresults 265
C.1 Computingthedigammafunction . . . . . . . . . . . . . . . . . . . . . . . . 265
C.2 Multivariategammahyperparameteroptimisation . . . . . . . . . . . . . . . . 266
C.3 MarginalKLdivergenceofgamma-Gaussianvariables . . . . . . . . . . . . . 267
Bibliography 270
7
List of figures
1.1 TheeliminationalgorithmonasimpleMarkovnetwork . . . . . . . . . . . . . 20
1.2 FormingthejunctiontreeforasimpleMarkovnetwork . . . . . . . . . . . . . 22
1.3 ThemarginallikelihoodembodiesOccam’srazor . . . . . . . . . . . . . . . . 27
2.1 VariationalinterpretationofEMforMLlearning . . . . . . . . . . . . . . . . 50
2.2 VariationalinterpretationofconstrainedEMforMLlearning . . . . . . . . . . 51
2.3 VariationalBayesianEM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
2.4 Hidden-variable/parameterfactorisationsteps . . . . . . . . . . . . . . . . . 59
2.5 HyperparameterlearningforVBEM . . . . . . . . . . . . . . . . . . . . . . . 62
3.1 GraphicalmodelrepresentationofahiddenMarkovmodel . . . . . . . . . . . 83
3.2 EvolutionofthelikelihoodforMLhiddenMarkovmodels,andthesubsequent
VBlowerbound. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100
3.3 ResultsofMLandVBHMMmodelstrainedonsyntheticsequences.. . . . . . 101
3.4 Test data log predictive probabilities and discrimination rates for ML, MAP,
andVBHMMs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103
4.1 MLMixturesofFactorAnalysers . . . . . . . . . . . . . . . . . . . . . . . . 110
4.2 BayesianMixturesofFactorAnalysers . . . . . . . . . . . . . . . . . . . . . . 114
4.3 Determinationofnumberofcomponentsinsyntheticdata . . . . . . . . . . . . 134
4.4 Factorloadingmatricesfordimensionalitydetermination . . . . . . . . . . . . 135
4.5 TheSpiraldatasetofUedaet.al . . . . . . . . . . . . . . . . . . . . . . . . . 136
4.6 BirthanddeathprocesseswithVBMFAontheSpiraldataset . . . . . . . . . . 137
4.7 EvolutionofthelowerboundF fortheSpiraldataset . . . . . . . . . . . . . . 137
4.8 TrainingexamplesofdigitsfromtheCEDARdatabase . . . . . . . . . . . . . . 138
4.9 AtypicalmodelofthedigitslearntbyVBMFA . . . . . . . . . . . . . . . . . 139
4.10 Confusiontablesforthetrainingandtestdigitclassifications . . . . . . . . . . 140
4.11 DistributionofcomponentstodigitsinBICandVBmodels . . . . . . . . . . . 143
4.12 LogarithmofthemarginallikelihoodestimateandtheVBlowerboundduring
learningofthedigits{0,1,2}. . . . . . . . . . . . . . . . . . . . . . . . . . . 150
4.13 Discrepancies between marginal likelihood and lower bounds during VBMFA
modelsearch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152
8
Listoffigures Listoffigures
4.14 Importance sampling estimates of marginal likelihoods for learnt models of
dataofdifferentlyspacedclusters . . . . . . . . . . . . . . . . . . . . . . . . 156
5.1 Graphicalmodelrepresentationofastate-spacemodel . . . . . . . . . . . . . 161
5.2 Graphicalmodelforastate-spacemodelwithinputs . . . . . . . . . . . . . . . 162
5.3 GraphicalmodelrepresentationofaBayesianstate-spacemodel . . . . . . . . 164
5.4 RecoveredLDSmodelsforincreasingdatasize . . . . . . . . . . . . . . . . . 190
5.5 Hyperparametertrajectoriesshowingextinctionofstate-spacedimensions . . . 191
5.6 Datafortheinput-drivenLDSsyntheticexperiment . . . . . . . . . . . . . . . 193
5.7 Evolutionofthelowerboundanditsgradient . . . . . . . . . . . . . . . . . . 194
5.8 Evolutionofprecisionhyperparameters,recoveringtruemodelstructure . . . . 196
5.9 Geneexpressiondataforinput-drivenexperimentsonrealdata . . . . . . . . . 197
5.10 GraphicalmodelofanLDSwithfeedbackofobservationsintoinputs . . . . . 199
5.11 Reconstruction errors of LDS models trained using MAP and VB algorithms
asafunctionofstate-spacedimensionality . . . . . . . . . . . . . . . . . . . . 200
5.12 Gene-gene interaction matrices learnt by MAP and VB algorithms, showing
significantentries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202
5.13 Illustration of the gene-gene interactions learnt by the feedback model on ex-
pressiondata . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 203
6.1 Thechosenstructureforgeneratingdatafortheexperiments . . . . . . . . . . 225
6.2 IllustrationofthetrendsinmarginallikelihoodestimatesasreportedbyMAP,
BIC,BICp,CS,VBandAISmethods,asafunctionofdatasetsizeandnumber
ofparameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 228
6.3 Graph of rankings given to the true structure by BIC, BICp, CS, VB and AIS
methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 230
6.4 Differences in marginal likelihood estimate of the top-ranked and true struc-
tures,byBIC,BICp,CS,VBandAIS . . . . . . . . . . . . . . . . . . . . . . 232
6.5 The median ranking given to the true structure over repeated settings of its
parametersdrawnfromtheprior,byBIC,BICp,CSandVBmethods . . . . . 233
6.6 Medianscoredifferencebetweenthetrueandtop-rankedstructures,underBIC,
BICp,CSandVBmethods. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 234
6.7 ThebestrankinggiventothetruestructurebyBIC,BICp,CSandVBmethods 235
6.8 The smallest score difference between true and top-ranked structures, by BIC,
BICp,CSandVBmethods . . . . . . . . . . . . . . . . . . . . . . . . . . . . 236
6.9 OverallsuccessratesofBIC,BICp,CSandVBscores,intermsofrankingthe
truestructuretop . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 237
6.10 ExampleofthevarianceoftheAISsamplerestimateswithannealingschedule
granularity, using various random initialisations, shown against the BIC and
VBestimatesforcomparison . . . . . . . . . . . . . . . . . . . . . . . . . . . 238
9
Listoffigures Listoffigures
6.11 AcceptanceratesoftheMetropolis-Hastingsproposalsasafunctionofsizeof
dataset . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 240
6.12 Acceptance rates of the Metropolis-Hastings sampler in each of four quarters
oftheannealingschedule . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 242
6.13 Non-linearAISannealingschedules . . . . . . . . . . . . . . . . . . . . . . . 244
10