Numerical Analysis

665 Pages·2011·9.29 MB·English

| i
Numerical Analysis
This page intentionally left blank
| iii
Numerical Analysis
S E C O N D E D I T I O N
Timothy Sauer
GeorgeMasonUniversity
Boston Columbus Indianapolis NewYork SanFrancisco UpperSaddleRiver Amsterdam CapeTown
Dubai London Madrid Milan Munich Paris Montréal Toronto Delhi MexicoCity SãoPaulo
Sydney HongKong Seoul Singapore Taipei Tokyo
EditorinChief:DeirdreLynch
SeniorAcquisitionsEditor:WilliamHoffman
SponsoringEditor:CarolineCelano
EditorialAssistant:BrandonRawnsley
SeniorManagingEditor:KarenWernholm
SeniorProductionProjectManager:BethHouston
ExecutiveMarketingManager:JeffWeidenaar
MarketingAssistant:CaitlinCrane
SeniorAuthorSupport/TechnologySpecialist:JoeVetere
RightsandPermissionsAdvisor:MichaelJoyce
ManufacturingBuyer:DebbieRossi
DesignManager:AndreaNix
SeniorDesigner:BarbaraAtkinson
ProductionCoordinationandComposition:IntegraSoftwareServicesPvt.Ltd
CoverDesigner:KarenSalzbach
CoverImage:TimTadder/Corbis
Photocredits:Page1ImageSource;page24NationalAdvancedDrivingSimulator(NADS-1Simulator)located
attheUniversityofIowaandownedbytheNationalHighwaySafetyAdministration(NHTSA);page39Yale
BabylonianCollection;page71Travellinglight/iStockphoto;page138RosenfeldImagesLtd./PhotoResearchers,
Inc;page188Pincasso/Shutterstock;page243Orhan81/Fotolia;page281UPPA/Photoshot;page348Paul
Springett04/Alamy;page374BillNoll/iStockphoto;page431DonEmmert/AFP/GettyImages/Newscom;
page467PictureAlliance/Photoshot;page495ChrisRout/Alamy;page505ToniAngermayer/Photo
Researchers,Inc;page531JinxPhotographyBrands/Alamy;page565PhilDegginger/Alamy.
Manyofthedesignationsusedbymanufacturersandsellerstodistinguishtheirproductsareclaimedas
trademarks.Wherethosedesignationsappearinthisbook,andPearsonEducationwasawareofatrademark
claim,thedesignationshavebeenprintedininitialcapsorallcaps.
LibraryofCongressCataloging-in-PublicationData
Sauer,Tim.
Numericalanalysis/TimothySauer.–2nded.
p.cm.
Includesbibliographicalreferencesandindex.
ISBN-13:978-0-321-78367-7
ISBN-10:0-321-78367-0
1. Numericalanalysis. I. Title.
QA297.S3482012
518–dc23
2011014232
Copyright©2012,2006PearsonEducation,Inc.
Allrightsreserved.Nopartofthispublicationmaybereproduced,storedinaretrievalsystem,ortransmitted,in
anyformorbyanymeans,electronic,mechanical,photocopying,recording,orotherwise,withouttheprior
writtenpermissionofthepublisher.PrintedintheUnitedStatesofAmerica.Forinformationonobtaining
permissionforuseofmaterialinthiswork,pleasesubmitawrittenrequesttoPearsonEducation,Inc.,Rights
andContractsDepartment,501BoylstonStreet,Suite900,Boston,MA02116,faxyourrequestto
617-671-3447,ore-mailathttp://www.pearsoned.com/legal/permissions.htm.
1 2 3 4 5 6 7 8 9 10—EB—15 14 13 12 11
ISBN10: 0-321-78367-0
ISBN13:978-0-321-78367-7
Contents
PREFACE xiii
CHAPTER0 Fundamentals 1
0.1 EvaluatingaPolynomial 1
0.2 BinaryNumbers 5
0.2.1 Decimaltobinary 6
0.2.2 Binarytodecimal 7
0.3 FloatingPointRepresentationofRealNumbers 8
0.3.1 Floatingpointformats 8
0.3.2 Machinerepresentation 11
0.3.3 Additionoffloatingpointnumbers 13
0.4 LossofSignificance 16
0.5 ReviewofCalculus 19
SoftwareandFurtherReading 23
CHAPTER1 SolvingEquations 24
1.1 TheBisectionMethod 25
1.1.1 Bracketingaroot 25
1.1.2 Howaccurateandhowfast? 28
1.2 Fixed-PointIteration 30
1.2.1 Fixedpointsofafunction 31
1.2.2 GeometryofFixed-PointIteration 33
1.2.3 LinearconvergenceofFixed-PointIteration 34
1.2.4 Stoppingcriteria 40
1.3 LimitsofAccuracy 43
1.3.1 Forwardandbackwarderror 44
1.3.2 TheWilkinsonpolynomial 47
1.3.3 Sensitivityofroot-finding 48
1.4 Newton’sMethod 51
1.4.1 QuadraticconvergenceofNewton’sMethod 53
1.4.2 LinearconvergenceofNewton’sMethod 55
1.5 Root-FindingwithoutDerivatives 61
1.5.1 SecantMethodandvariants 61
1.5.2 Brent’sMethod 64
RealityCheck1:KinematicsoftheStewartplatform 67
SoftwareandFurtherReading 69
CHAPTER2 SystemsofEquations 71
2.1 GaussianElimination 71
2.1.1 NaiveGaussianelimination 72
2.1.2 Operationcounts 74
vi | Contents
2.2 TheLUFactorization 79
2.2.1 MatrixformofGaussianelimination 79
2.2.2 BacksubstitutionwiththeLUfactorization 81
2.2.3 ComplexityoftheLUfactorization 83
2.3 SourcesofError 85
2.3.1 Errormagnificationandconditionnumber 86
2.3.2 Swamping 91
2.4 ThePA=LUFactorization 95
2.4.1 Partialpivoting 95
2.4.2 Permutationmatrices 97
2.4.3 PA=LUfactorization 98
RealityCheck2:TheEuler–BernoulliBeam 102
2.5 IterativeMethods 106
2.5.1 JacobiMethod 106
2.5.2 Gauss–SeidelMethodandSOR 108
2.5.3 Convergenceofiterativemethods 111
2.5.4 Sparsematrixcomputations 113
2.6 Methodsforsymmetricpositive-definitematrices 117
2.6.1 Symmetricpositive-definitematrices 117
2.6.2 Choleskyfactorization 119
2.6.3 ConjugateGradientMethod 121
2.6.4 Preconditioning 126
2.7 NonlinearSystemsofEquations 130
2.7.1 MultivariateNewton’sMethod 131
2.7.2 Broyden’sMethod 133
SoftwareandFurtherReading 137
CHAPTER3 Interpolation 138
3.1 DataandInterpolatingFunctions 139
3.1.1 Lagrangeinterpolation 140
3.1.2 Newton’sdivideddifferences 141
3.1.3 Howmanydegreedpolynomialspassthroughn
points? 144
3.1.4 Codeforinterpolation 145
3.1.5 Representingfunctionsbyapproximatingpolynomials 147
3.2 InterpolationError 151
3.2.1 Interpolationerrorformula 151
3.2.2 ProofofNewtonformanderrorformula 153
3.2.3 Rungephenomenon 155
3.3 ChebyshevInterpolation 158
3.3.1 Chebyshev’stheorem 158
3.3.2 Chebyshevpolynomials 160
3.3.3 Changeofinterval 162
3.4 CubicSplines 166
3.4.1 Propertiesofsplines 167
3.4.2 Endpointconditions 173
3.5 BézierCurves 179
RealityCheck3:FontsfromBéziercurves 183
SoftwareandFurtherReading 187
Contents | vii
CHAPTER4 LeastSquares 188
4.1 LeastSquaresandtheNormalEquations 188
4.1.1 Inconsistentsystemsofequations 189
4.1.2 Fittingmodelstodata 193
4.1.3 Conditioningofleastsquares 197
4.2 ASurveyofModels 201
4.2.1 Periodicdata 201
4.2.2 Datalinearization 203
4.3 QRFactorization 212
4.3.1 Gram–Schmidtorthogonalizationandleastsquares 212
4.3.2 ModifiedGram–Schmidtorthogonalization 218
4.3.3 Householderreflectors 220
4.4 GeneralizedMinimumResidual(GMRES)Method 225
4.4.1 Krylovmethods 226
4.4.2 PreconditionedGMRES 228
4.5 NonlinearLeastSquares 230
4.5.1 Gauss–NewtonMethod 230
4.5.2 Modelswithnonlinearparameters 233
4.5.3 TheLevenberg–MarquardtMethod. 235
RealityCheck4:GPS,Conditioning,andNonlinearLeastSquares 238
SoftwareandFurtherReading 242
CHAPTER5 NumericalDifferentiationand
Integration 243
5.1 NumericalDifferentiation 244
5.1.1 Finitedifferenceformulas 244
5.1.2 Roundingerror 247
5.1.3 Extrapolation 249
5.1.4 Symbolicdifferentiationandintegration 250
5.2 Newton–CotesFormulasforNumericalIntegration 254
5.2.1 TrapezoidRule 255
5.2.2 Simpson’sRule 257
5.2.3 CompositeNewton–Cotesformulas 259
5.2.4 OpenNewton–CotesMethods 262
5.3 RombergIntegration 265
5.4 AdaptiveQuadrature 269
5.5 GaussianQuadrature 273
RealityCheck5:MotionControlinComputer-AidedModeling 278
SoftwareandFurtherReading 280
CHAPTER6 OrdinaryDifferentialEquations 281
6.1 InitialValueProblems 282
6.1.1 Euler’sMethod 283
6.1.2 Existence,uniqueness,andcontinuityforsolutions 287
6.1.3 First-orderlinearequations 290
6.2 AnalysisofIVPSolvers 293
6.2.1 Localandglobaltruncationerror 293
viii | Contents
6.2.2 TheexplicitTrapezoidMethod 297
6.2.3 TaylorMethods 300
6.3 SystemsofOrdinaryDifferentialEquations 303
6.3.1 Higherorderequations 304
6.3.2 Computersimulation:thependulum 305
6.3.3 Computersimulation:orbitalmechanics 309
6.4 Runge–KuttaMethodsandApplications 314
6.4.1 TheRunge–Kuttafamily 314
6.4.2 Computersimulation:theHodgkin–Huxleyneuron 317
6.4.3 Computersimulation:theLorenzequations 319
RealityCheck6:TheTacomaNarrowsBridge 322
6.5 VariableStep-SizeMethods 325
6.5.1 EmbeddedRunge–Kuttapairs 325
6.5.2 Order4/5methods 328
6.6 ImplicitMethodsandStiffEquations 332
6.7 MultistepMethods 336
6.7.1 Generatingmultistepmethods 336
6.7.2 Explicitmultistepmethods 339
6.7.3 Implicitmultistepmethods 342
SoftwareandFurtherReading 347
CHAPTER7 BoundaryValueProblems 348
7.1 ShootingMethod 349
7.1.1 Solutionsofboundaryvalueproblems 349
7.1.2 ShootingMethodimplementation 352
RealityCheck7:BucklingofaCircularRing 355
7.2 FiniteDifferenceMethods 357
7.2.1 Linearboundaryvalueproblems 357
7.2.2 Nonlinearboundaryvalueproblems 359
7.3 CollocationandtheFiniteElementMethod 365
7.3.1 Collocation 365
7.3.2 FiniteelementsandtheGalerkinMethod 367
SoftwareandFurtherReading 373
CHAPTER8 PartialDifferentialEquations 374
8.1 ParabolicEquations 375
8.1.1 ForwardDifferenceMethod 375
8.1.2 StabilityanalysisofForwardDifferenceMethod 379
8.1.3 BackwardDifferenceMethod 380
8.1.4 Crank–NicolsonMethod 385
8.2 HyperbolicEquations 393
8.2.1 Thewaveequation 393
8.2.2 TheCFLcondition 395
8.3 EllipticEquations 398
8.3.1 FiniteDifferenceMethodforellipticequations 399
RealityCheck8:Heatdistributiononacoolingfin 403
8.3.2 FiniteElementMethodforellipticequations 406
Contents | ix
8.4 Nonlinearpartialdifferentialequations 417
8.4.1 ImplicitNewtonsolver 417
8.4.2 Nonlinearequationsintwospacedimensions 423
SoftwareandFurtherReading 430
CHAPTER9 RandomNumbersandApplications 431
9.1 RandomNumbers 432
9.1.1 Pseudo-randomnumbers 432
9.1.2 Exponentialandnormalrandomnumbers 437
9.2 MonteCarloSimulation 440
9.2.1 PowerlawsforMonteCarloestimation 440
9.2.2 Quasi-randomnumbers 442
9.3 DiscreteandContinuousBrownianMotion 446
9.3.1 Randomwalks 447
9.3.2 ContinuousBrownianmotion 449
9.4 StochasticDifferentialEquations 452
9.4.1 Addingnoisetodifferentialequations 452
9.4.2 NumericalmethodsforSDEs 456
RealityCheck9:TheBlack–ScholesFormula 464
SoftwareandFurtherReading 465
CHAPTER10 TrigonometricInterpolationand
theFFT 467
10.1 TheFourierTransform 468
10.1.1 Complexarithmetic 468
10.1.2 DiscreteFourierTransform 470
10.1.3 TheFastFourierTransform 473
10.2 TrigonometricInterpolation 476
10.2.1 TheDFTInterpolationTheorem 476
10.2.2 Efficientevaluationoftrigonometricfunctions 479
10.3 TheFFTandSignalProcessing 483
10.3.1 Orthogonalityandinterpolation 483
10.3.2 Leastsquaresfittingwithtrigonometricfunctions 485
10.3.3 Sound,noise,andfiltering 489
RealityCheck10:TheWienerFilter 492
SoftwareandFurtherReading 494
CHAPTER11 Compression 495
11.1 TheDiscreteCosineTransform 496
11.1.1 One-dimensionalDCT 496
11.1.2 TheDCTandleastsquaresapproximation 498
11.2 Two-DimensionalDCTandImageCompression 501
11.2.1 Two-dimensionalDCT 501
11.2.2 Imagecompression 505
11.2.3 Quantization 508
11.3 HuffmanCoding 514
11.3.1 Informationtheoryandcoding 514
11.3.2 HuffmancodingfortheJPEGformat 517