Parameter Estimation and Inverse Problems

316 Pages·2005·1.607 MB·English

by  Richard C. Aster, Brian Borchers and Clifford H. Thurber (Eds.)

Parameter Estimation
and Inverse Problems
ThisisVolume90inthe
INTERNATIONALGEOPHYSICSSERIES
Aseriesofmonographsandtextbooks
EditedbyRENATADMOWSKA,JAMESR.HOLTON,andH.THOMASROSSBY
Acompletelistofbooksinthisseriesappearsattheendofthisvolume.
Parameter Estimation
and Inverse Problems
Richard C.Aster, Brian Borchers, and Clifford H. Thurber
Amsterdam • Boston (cid:127) Heidelberg (cid:127) London (cid:127) NewYork (cid:127) Oxford
Paris (cid:127) SanDiego (cid:127) SanFrancisco (cid:127) Singapore (cid:127) Sydney (cid:127) Tokyo
AcquisitionEditor FrankCynar
ProjectManager KyleSarofeen
EditorialCoordinator JenniferHelé
MarketingManager LindaBeattie
CoverDesign SuzanneRogers
Composition CephaImagingPrivateLtd.
CoverPrinter PhoenixColorCorporation
InteriorPrinter Maple-VailBookManufacturingGroup
ElsevierAcademicPress
30CorporateDrive,Suite400,Burlington,MA01803,USA
525BStreet,Suite1900,SanDiego,CA92101-4495,USA
84Theobald’sRoad,LondonWC1X8RR,UK
Thisbookisprintedonacid-freepaper.
Copyright©2005,ElsevierInc.Allrightsreserved.
Nopartofthispublicationmaybereproducedortransmittedinanyformorbyanymeans,
electronicormechanical,includingphotocopy,recording,oranyinformationstorageand
retrievalsystem,withoutpermissioninwritingfromthepublisher.
PermissionsmaybesoughtdirectlyfromElsevier’sScience&TechnologyRightsDepartment
inOxford,UK:phone:[phone],fax:[phone],e-mail:
[e-mail]questonlineviatheElsevier
homepage(http://elsevier.com),byselecting“CustomerSupport”andthen“ObtainingPermissions.”
LibraryofCongressCataloging-in-PublicationData
Applicationsubmitted
BritishLibraryCataloguinginPublicationData
AcataloguerecordforthisbookisavailablefromtheBritishLibrary
ISBN: 0-12-065604-3
ForinformationonallElsevierAcademicPressPublications
visitourWebsiteatwww.books.elsevier.com
PrintedintheUnitedStatesofAmerica
04 05 06 07 08 09 9 8 7 6 5 4 3 2 1
Contents
Preface xi
1 INTRODUCTION 1
1.1 ClassificationofInverseProblems 1
1.2 ExamplesofParameterEstimationProblems 4
1.3 ExamplesofInverseProblems 7
1.4 WhyInverseProblemsAreHard 11
1.5 Exercises 14
1.6 NotesandFurtherReading 14
2 LINEAR REGRESSION 15
2.1 IntroductiontoLinearRegression 15
2.2 StatisticalAspectsofLeastSquares 17
2.3 UnknownMeasurementStandardDeviations 26
2.4 L Regression 30
1
2.5 MonteCarloErrorPropagation 35
2.6 Exercises 36
2.7 NotesandFurtherReading 40
3 DISCRETIZING CONTINUOUS INVERSE
PROBLEMS 41
3.1 IntegralEquations 41
3.2 QuadratureMethods 41
v
vi Contents
3.3 ExpansioninTermsofRepresenters 46
3.4 ExpansioninTermsofOrthonormalBasisFunctions 47
3.5 TheMethodofBackusandGilbert 48
3.6 Exercises 52
3.7 NotesandFurtherReading 54
4 RANK DEFICIENCYAND ILL-CONDITIONING 55
4.1 TheSVDandtheGeneralizedInverse 55
4.2 CovarianceandResolutionoftheGeneralizedInverseSolution 62
4.3 InstabilityoftheGeneralizedInverseSolution 64
4.4 AnExampleofaRank-DeficientProblem 67
4.5 DiscreteIll-PosedProblems 73
4.6 Exercises 85
4.7 NotesandFurtherReading 87
5 TIKHONOV REGULARIZATION 89
5.1 SelectingaGoodSolution 89
5.2 SVDImplementationofTikhonovRegularization 91
5.3 Resolution,Bias,andUncertaintyintheTikhonovSolution 95
5.4 Higher-OrderTikhonovRegularization 98
5.5 ResolutioninHigher-OrderTikhonovRegularization 103
5.6 TheTGSVDMethod 105
5.7 GeneralizedCrossValidation 106
5.8 ErrorBounds 109
5.9 Exercises 114
5.10 NotesandFurtherReading 117
6 ITERATIVE METHODS 119
6.1 Introduction 119
6.2 IterativeMethodsforTomographyProblems 120
6.3 TheConjugateGradientMethod 126
6.4 TheCGLSMethod 131
Contents vii
6.5 Exercises 135
6.6 NotesandFurtherReading 136
7 ADDITIONALREGULARIZATIONTECHNIQUES 139
7.1 UsingBoundsasConstraints 139
7.2 MaximumEntropyRegularization 143
7.3 TotalVariation 146
7.4 Exercises 151
7.5 NotesandFurtherReading 152
8 FOURIER TECHNIQUES 153
8.1 LinearSystemsintheTimeandFrequencyDomains 153
8.2 DeconvolutionfromaFourierPerspective 158
8.3 LinearSystemsinDiscreteTime 161
8.4 WaterLevelRegularization 164
8.5 Exercises 168
8.6 NotesandFurtherReading 170
9 NONLINEAR REGRESSION 171
9.1 Newton’sMethod 171
9.2 TheGauss–NewtonandLevenberg–MarquardtMethods 174
9.3 StatisticalAspects 177
9.4 ImplementationIssues 181
9.5 Exercises 186
9.6 NotesandFurtherReading 189
10 NONLINEAR INVERSE PROBLEMS 191
10.1 RegularizingNonlinearLeastSquaresProblems 191
10.2 Occam’sInversion 195
10.3 Exercises 199
10.4 NotesandFurtherReading 199
viii Contents
11 BAYESIAN METHODS 201
11.1 ReviewoftheClassicalApproach 201
11.2 TheBayesianApproach 202
11.3 TheMultivariateNormalCase 207
11.4 MaximumEntropyMethods 212
11.5 Epilogue 214
11.6 Exercises 216
11.7 NotesandFurtherReading 217
A REVIEW OF LINEARALGEBRA 219
A.1 SystemsofLinearEquations 219
A.2 MatrixandVectorAlgebra 222
A.3 LinearIndependence 228
A.4 SubspacesofRn 229
A.5 OrthogonalityandtheDotProduct 233
A.6 EigenvaluesandEigenvectors 237
A.7 VectorandMatrixNorms 240
A.8 TheConditionNumberofaLinearSystem 242
A.9 TheQRFactorization 244
A.10 LinearAlgebrainSpacesofFunctions 245
A.11 Exercises 247
A.12 NotesandFurtherReading 249
B REVIEW OF PROBABILITYAND STATISTICS 251
B.1 ProbabilityandRandomVariables 251
B.2 ExpectedValueandVariance 257
B.3 JointDistributions 258
B.4 ConditionalProbability 262
B.5 TheMultivariateNormalDistribution 264
B.6 TheCentralLimitTheorem 265
B.7 TestingforNormality 265
Contents ix
B.8 EstimatingMeansandConfidenceIntervals 267
B.9 HypothesisTests 269
B.10 Exercises 271
B.11 NotesandFurtherReading 272
C REVIEW OF VECTOR CALCULUS 273
C.1 TheGradient,Hessian,andJacobian 273
C.2 Taylor’sTheorem 275
C.3 LagrangeMultipliers 276
C.4 Exercises 278
C.5 NotesandFurtherReading 280
D GLOSSARYOF NOTATION 281
283
Bibliography
291
Index