Numerical recipes: the art of scientific computing

1262 Pages·2007·7.02 MB·English

by  William H. Press, Saul A. Teukolsky, William T. Vetterling, Brian P. Flannery

(cid:2) (cid:2)
“nr3” — 2007/5/1 — 20:53 — page ii — #2
(cid:2) (cid:2)
This page intentionally left blank
(cid:2) (cid:2)
(cid:2) (cid:2)
“nr3” — 2007/5/1 — 20:53 — page i — #1
(cid:2) (cid:2)
NUMERICAL
RECIPES
The Art of Scientific Computing
Third Edition
(cid:2) (cid:2)
(cid:2) (cid:2)
“nr3” — 2007/5/1 — 20:53 — page ii — #2
(cid:2) (cid:2)
(cid:2) (cid:2)
(cid:2) (cid:2)
“nr3” — 2007/5/1 — 20:53 — page iii — #3
(cid:2) (cid:2)
NUMERICAL
RECIPES
The Art of Scientific Computing
Third Edition
WilliamH.Press
RaymerChairinComputerSciencesandIntegrativeBiology
TheUniversityofTexasatAustin
SaulA.Teukolsky
HansA.BetheProfessorofPhysicsandAstrophysics
CornellUniversity
WilliamT.Vetterling
ResearchFellowandDirectorofImageScience
ZINKImaging,LLC
BrianP.Flannery
Science,StrategyandProgramsManager
ExxonMobilCorporation
(cid:2) (cid:2)
CAMBRIDGEUNIVERSITY PRESS
Cambridge, New York, Melbourne, Madrid, Cape Town, Singapore, São Paulo
Cambridge University Press
The Edinburgh Building, Cambridge CB2 8RU, UK
Published in the United States of America by Cambridge University Press, New York
www.cambridge.org
Information on this title: www.cambridge.org/9780521880688
© Cambridge University Press 1988, 1992, 2002, 2007 except for 13.10, which is placed into
the public domain, and except for all other computer programs and procedures, which are
This publication is in copyright. Subject to statutory exception and to the provision of
relevant collective licensing agreements, no reproduction of any part may take place
without the written permission of Cambridge University Press.
First published in print format 2007
ISBN-13 978-0-511-33555-6 eBook (NetLibrary)
ISBN-10 0-511-33555-5 eBook (NetLibrary)
ISBN-13 978-0-521-88068-8 hardback
ISBN-10 0-521-88068-8 hardback
Cambridge University Press has no responsibility for the persistence or accuracy of urls
for external or third-party internet websites referred to in this publication, and does not
guarantee that any content on such websites is, or will remain, accurate or appropriate.
Without an additional license to use the contained software, this book is intended as a text
and reference book, for reading and study purposes only. However, a restricted, limited
free license for use of the software by the individual owner of a copy of this book who
personally keyboards one or more routines into a single computer is granted under terms
described on p. xix. See the section “License and Legal Information” (pp. xix–xxi) for
information on obtaining more general licenses. Machine-readable media containing the
software in this book, with included license for use by a single individual, are available
from Cambridge University Press. The software may also be downloaded, with immediate
purchase of a license also possible, from the Numerical Recipes Software Web site (http:
//www.nr.com). Unlicensed transfer of Numerical Recipes programs to any other format,
or to any computer except one that is specifically licensed, is strictly prohibited. Technical
questions, corrections, and requests for information should be addressed to Numerical
Recipes Software, P.O. Box 380243, Cambridge, MA 02238-0243 (USA), email info@nr.
com, or fax [phone].
(cid:2) (cid:2)
“nr3” — 2007/5/1 — 20:53 — page v — #5
(cid:2) (cid:2)
Contents
PrefacetotheThirdEdition(2007) xi
PrefacetotheSecondEdition(1992) xiv
PrefacetotheFirstEdition(1985) xvii
LicenseandLegalInformation xix
1 Preliminaries 1
1.0 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.1 Error,Accuracy,andStability . . . . . . . . . . . . . . . . . . . . 8
1.2 CFamilySyntax . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
1.3 Objects,Classes,andInheritance . . . . . . . . . . . . . . . . . . 17
1.4 VectorandMatrixObjects . . . . . . . . . . . . . . . . . . . . . . 24
1.5 SomeFurtherConventionsandCapabilities . . . . . . . . . . . . . 30
2 SolutionofLinearAlgebraicEquations 37
2.0 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
2.1 Gauss-JordanElimination . . . . . . . . . . . . . . . . . . . . . . 41
2.2 GaussianEliminationwithBacksubstitution . . . . . . . . . . . . 46
2.3 LU DecompositionandItsApplications . . . . . . . . . . . . . . 48
2.4 TridiagonalandBand-DiagonalSystemsofEquations . . . . . . . 56
2.5 IterativeImprovementofaSolutiontoLinearEquations . . . . . . 61
2.6 SingularValueDecomposition. . . . . . . . . . . . . . . . . . . . 65
2.7 SparseLinearSystems . . . . . . . . . . . . . . . . . . . . . . . . 75
2.8 VandermondeMatricesandToeplitzMatrices . . . . . . . . . . . . 93
2.9 CholeskyDecomposition . . . . . . . . . . . . . . . . . . . . . . 100
2.10 QRDecomposition . . . . . . . . . . . . . . . . . . . . . . . . . 102
2.11 IsMatrixInversionanN3 Process? . . . . . . . . . . . . . . . . . 106
3 InterpolationandExtrapolation 110
3.0 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110
3.1 Preliminaries: SearchinganOrderedTable . . . . . . . . . . . . . 114
3.2 PolynomialInterpolationandExtrapolation . . . . . . . . . . . . . 118
3.3 CubicSplineInterpolation . . . . . . . . . . . . . . . . . . . . . . 120
3.4 RationalFunctionInterpolationandExtrapolation . . . . . . . . . 124
v
(cid:2) (cid:2)
(cid:2) (cid:2)
“nr3” — 2007/5/1 — 20:53 — page vi — #6
(cid:2) (cid:2)
vi Contents
3.5 CoefficientsoftheInterpolatingPolynomial . . . . . . . . . . . . 129
3.6 InterpolationonaGridinMultidimensions . . . . . . . . . . . . . 132
3.7 InterpolationonScatteredDatainMultidimensions . . . . . . . . 139
3.8 LaplaceInterpolation . . . . . . . . . . . . . . . . . . . . . . . . 150
4 IntegrationofFunctions 155
4.0 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155
4.1 ClassicalFormulasforEquallySpacedAbscissas . . . . . . . . . . 156
4.2 ElementaryAlgorithms . . . . . . . . . . . . . . . . . . . . . . . 162
4.3 RombergIntegration . . . . . . . . . . . . . . . . . . . . . . . . . 166
4.4 ImproperIntegrals . . . . . . . . . . . . . . . . . . . . . . . . . . 167
4.5 QuadraturebyVariableTransformation . . . . . . . . . . . . . . . 172
4.6 GaussianQuadraturesandOrthogonalPolynomials . . . . . . . . 179
4.7 AdaptiveQuadrature . . . . . . . . . . . . . . . . . . . . . . . . . 194
4.8 MultidimensionalIntegrals . . . . . . . . . . . . . . . . . . . . . 196
5 EvaluationofFunctions 201
5.0 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201
5.1 PolynomialsandRationalFunctions. . . . . . . . . . . . . . . . . 201
5.2 EvaluationofContinuedFractions. . . . . . . . . . . . . . . . . . 206
5.3 SeriesandTheirConvergence . . . . . . . . . . . . . . . . . . . . 209
5.4 RecurrenceRelationsandClenshaw’sRecurrenceFormula. . . . . 219
5.5 ComplexArithmetic . . . . . . . . . . . . . . . . . . . . . . . . . 225
5.6 QuadraticandCubicEquations . . . . . . . . . . . . . . . . . . . 227
5.7 NumericalDerivatives . . . . . . . . . . . . . . . . . . . . . . . . 229
5.8 ChebyshevApproximation. . . . . . . . . . . . . . . . . . . . . . 233
5.9 DerivativesorIntegralsofaChebyshev-ApproximatedFunction . . 240
5.10 PolynomialApproximationfromChebyshevCoefficients . . . . . 241
5.11 EconomizationofPowerSeries . . . . . . . . . . . . . . . . . . . 243
5.12 Pade´ Approximants . . . . . . . . . . . . . . . . . . . . . . . . . 245
5.13 RationalChebyshevApproximation . . . . . . . . . . . . . . . . . 247
5.14 EvaluationofFunctionsbyPathIntegration . . . . . . . . . . . . . 251
6 SpecialFunctions 255
6.0 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 255
6.1 GammaFunction,BetaFunction,Factorials,BinomialCoefficients 256
6.2 IncompleteGammaFunctionandErrorFunction . . . . . . . . . . 259
6.3 ExponentialIntegrals . . . . . . . . . . . . . . . . . . . . . . . . 266
6.4 IncompleteBetaFunction . . . . . . . . . . . . . . . . . . . . . . 270
6.5 BesselFunctionsofIntegerOrder . . . . . . . . . . . . . . . . . . 274
6.6 Bessel Functions of Fractional Order, Airy Functions, Spherical
BesselFunctions . . . . . . . . . . . . . . . . . . . . . . . . . . . 283
6.7 SphericalHarmonics . . . . . . . . . . . . . . . . . . . . . . . . . 292
6.8 FresnelIntegrals,CosineandSineIntegrals . . . . . . . . . . . . . 297
6.9 Dawson’sIntegral . . . . . . . . . . . . . . . . . . . . . . . . . . 302
6.10 GeneralizedFermi-DiracIntegrals. . . . . . . . . . . . . . . . . . 304
6.11 InverseoftheFunctionxlog.x/ . . . . . . . . . . . . . . . . . . . 307
6.12 EllipticIntegralsandJacobianEllipticFunctions . . . . . . . . . . 309
(cid:2) (cid:2)
(cid:2) (cid:2)
“nr3” — 2007/5/1 — 20:53 — page vii — #7
(cid:2) (cid:2)
Contents vii
6.13 HypergeometricFunctions . . . . . . . . . . . . . . . . . . . . . . 318
6.14 StatisticalFunctions . . . . . . . . . . . . . . . . . . . . . . . . . 320
7 RandomNumbers 340
7.0 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 340
7.1 UniformDeviates . . . . . . . . . . . . . . . . . . . . . . . . . . 341
7.2 CompletelyHashingaLargeArray . . . . . . . . . . . . . . . . . 358
7.3 DeviatesfromOtherDistributions . . . . . . . . . . . . . . . . . . 361
7.4 MultivariateNormalDeviates . . . . . . . . . . . . . . . . . . . . 378
7.5 LinearFeedbackShiftRegisters . . . . . . . . . . . . . . . . . . . 380
7.6 HashTablesandHashMemories . . . . . . . . . . . . . . . . . . 386
7.7 SimpleMonteCarloIntegration . . . . . . . . . . . . . . . . . . . 397
7.8 Quasi-(thatis,Sub-)RandomSequences . . . . . . . . . . . . . . 403
7.9 AdaptiveandRecursiveMonteCarloMethods . . . . . . . . . . . 410
8 SortingandSelection 419
8.0 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 419
8.1 StraightInsertionandShell’sMethod . . . . . . . . . . . . . . . . 420
8.2 Quicksort . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 423
8.3 Heapsort . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 426
8.4 IndexingandRanking . . . . . . . . . . . . . . . . . . . . . . . . 428
8.5 SelectingtheMthLargest . . . . . . . . . . . . . . . . . . . . . . 431
8.6 DeterminationofEquivalenceClasses . . . . . . . . . . . . . . . . 439
9 RootFindingandNonlinearSetsofEquations 442
9.0 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 442
9.1 BracketingandBisection . . . . . . . . . . . . . . . . . . . . . . 445
9.2 SecantMethod,FalsePositionMethod,andRidders’Method . . . 449
9.3 VanWijngaarden-Dekker-BrentMethod . . . . . . . . . . . . . . 454
9.4 Newton-RaphsonMethodUsingDerivative . . . . . . . . . . . . . 456
9.5 RootsofPolynomials . . . . . . . . . . . . . . . . . . . . . . . . 463
9.6 Newton-RaphsonMethodforNonlinearSystemsofEquations . . . 473
9.7 GloballyConvergentMethodsforNonlinearSystemsofEquations 477
10 MinimizationorMaximizationofFunctions 487
10.0 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 487
10.1 InitiallyBracketingaMinimum . . . . . . . . . . . . . . . . . . . 490
10.2 GoldenSectionSearchinOneDimension . . . . . . . . . . . . . . 492
10.3 ParabolicInterpolationandBrent’sMethodinOneDimension . . . 496
10.4 One-DimensionalSearchwithFirstDerivatives . . . . . . . . . . . 499
10.5 DownhillSimplexMethodinMultidimensions . . . . . . . . . . . 502
10.6 LineMethodsinMultidimensions . . . . . . . . . . . . . . . . . . 507
10.7 DirectionSet(Powell’s)MethodsinMultidimensions . . . . . . . 509
10.8 ConjugateGradientMethodsinMultidimensions . . . . . . . . . . 515
10.9 Quasi-NewtonorVariableMetricMethodsinMultidimensions . . 521
10.10 LinearProgramming: TheSimplexMethod . . . . . . . . . . . . . 526
10.11 LinearProgramming: Interior-PointMethods . . . . . . . . . . . . 537
10.12 SimulatedAnnealingMethods . . . . . . . . . . . . . . . . . . . . 549
10.13 DynamicProgramming . . . . . . . . . . . . . . . . . . . . . . . 555
(cid:2) (cid:2)
(cid:2) (cid:2)
“nr3” — 2007/5/1 — 20:53 — page viii — #8
(cid:2) (cid:2)
viii Contents
11 Eigensystems 563
11.0 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 563
11.1 JacobiTransformationsofaSymmetricMatrix . . . . . . . . . . . 570
11.2 RealSymmetricMatrices . . . . . . . . . . . . . . . . . . . . . . 576
11.3 Reduction of a Symmetric Matrix to Tridiagonal Form: Givens
andHouseholderReductions . . . . . . . . . . . . . . . . . . . . 578
11.4 EigenvaluesandEigenvectorsofaTridiagonalMatrix . . . . . . . 583
11.5 HermitianMatrices . . . . . . . . . . . . . . . . . . . . . . . . . 590
11.6 RealNonsymmetricMatrices . . . . . . . . . . . . . . . . . . . . 590
11.7 TheQRAlgorithmforRealHessenbergMatrices . . . . . . . . . 596
11.8 ImprovingEigenvaluesand/orFindingEigenvectorsbyInverse
Iteration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 597
12 FastFourierTransform 600
12.0 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 600
12.1 FourierTransformofDiscretelySampledData . . . . . . . . . . . 605
12.2 FastFourierTransform(FFT) . . . . . . . . . . . . . . . . . . . . 608
12.3 FFTofRealFunctions . . . . . . . . . . . . . . . . . . . . . . . . 617
12.4 FastSineandCosineTransforms . . . . . . . . . . . . . . . . . . 620
12.5 FFTinTwoorMoreDimensions . . . . . . . . . . . . . . . . . . 627
12.6 FourierTransformsofRealDatainTwoandThreeDimensions . . 631
12.7 ExternalStorageorMemory-LocalFFTs . . . . . . . . . . . . . . 637
13 FourierandSpectralApplications 640
13.0 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 640
13.1 ConvolutionandDeconvolutionUsingtheFFT . . . . . . . . . . . 641
13.2 CorrelationandAutocorrelationUsingtheFFT . . . . . . . . . . . 648
13.3 Optimal(Wiener)FilteringwiththeFFT . . . . . . . . . . . . . . 649
13.4 PowerSpectrumEstimationUsingtheFFT . . . . . . . . . . . . . 652
13.5 DigitalFilteringintheTimeDomain . . . . . . . . . . . . . . . . 667
13.6 LinearPredictionandLinearPredictiveCoding . . . . . . . . . . . 673
13.7 PowerSpectrumEstimationbytheMaximumEntropy(All-Poles)
Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 681
13.8 SpectralAnalysisofUnevenlySampledData . . . . . . . . . . . . 685
13.9 ComputingFourierIntegralsUsingtheFFT . . . . . . . . . . . . . 692
13.10 WaveletTransforms . . . . . . . . . . . . . . . . . . . . . . . . . 699
13.11 NumericalUseoftheSamplingTheorem . . . . . . . . . . . . . . 717
14 StatisticalDescriptionofData 720
14.0 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 720
14.1 MomentsofaDistribution:Mean,Variance,Skewness,andSoForth 721
14.2 DoTwoDistributionsHavetheSameMeansorVariances?. . . . . 726
14.3 AreTwoDistributionsDifferent? . . . . . . . . . . . . . . . . . . 730
14.4 ContingencyTableAnalysisofTwoDistributions . . . . . . . . . 741
14.5 LinearCorrelation . . . . . . . . . . . . . . . . . . . . . . . . . . 745
14.6 NonparametricorRankCorrelation . . . . . . . . . . . . . . . . . 748
14.7 Information-TheoreticPropertiesofDistributions. . . . . . . . . . 754
14.8 DoTwo-DimensionalDistributionsDiffer? . . . . . . . . . . . . . 762
(cid:2) (cid:2)