Numerical Methods with Worked Examples: Woodford, Phillips

259 Pages·2012·1.977 MB·English

by  Woodford, Chris;Phillips, Chris

Numerical Methods with Worked Examples:
Matlab Edition
C. Woodford (cid:2) C. Phillips
Numerical Methods
with Worked
Examples:
Matlab Edition
Second Edition
C.Woodford Prof.C.Phillips
DepartmentofComputingService SchoolofComputingScience
NewcastleUniversity NewcastleUniversity
NewcastleuponTyne,NE17RU NewcastleuponTyne,NE17RU
UK UK
[email protected]
Additionalmaterialtothisbookcanbedownloadedfromhttp://extras.springer.com.
ISBN978-94-007-1365-9 e-ISBN978-94-007-1366-6
DOI10.1007/978-94-007-1366-6
SpringerDordrechtHeidelbergLondonNewYork
LibraryofCongressControlNumber:2011937948
1stedition:©Chapman&Hall(aspartofSpringerSBM)1997
©SpringerScience+BusinessMediaB.V.2012
Nopartofthisworkmaybereproduced,storedinaretrievalsystem,ortransmittedinanyformorby
anymeans,electronic,mechanical,photocopying,microfilming,recordingorotherwise,withoutwritten
permissionfromthePublisher,withtheexceptionofanymaterialsuppliedspecificallyforthepurpose
ofbeingenteredandexecutedonacomputersystem,forexclusiveusebythepurchaserofthework.
Coverdesign:deblik
Printedonacid-freepaper
SpringerispartofSpringerScience+BusinessMedia(www.springer.com)
Preface
Thisbookisasurveyofthenumericalmethodsthatarecommontoundergraduate
coursesinScience,Computing,EngineeringandTechnology.Theaimistopresent
sufficientmethodstofacilitatethenumericalanalysisofmathematicalmodelslikely
tobeencounteredinpractice.Examplesofsuchmodelsincludethelinearequations
describing the stress on girders, bridges and other civil engineering structures, the
differential equations of chemical and thermal reactions, and the inferences to be
drawnfromobserveddata.
Thebookiswrittenprimarilyforthestudent,experimentalscientistanddesign
engineer for whom it should provide a range of basic tools. The presentation is
novelinthatmathematicaljustificationfollowsratherthanprecedesthedescription
ofanymethod.Weencouragethereaderfirsttogainafamiliaritywithaparticular
methodthroughexperiment.Thisistheapproachweusewhenteachingthismate-
rial in university courses. We feel it is a necessary precursor to understanding the
underlyingmathematics.Theaimatalltimesistousetheexperienceofnumerical
experiment and a feel for the mathematics to apply numerical methods efficiently
andeffectively.
Methodsarepresentedinaproblem–solution–discussionorder.Thesolutionmay
notbethemostelegantbutitrepresentstheonemostlikelytosuggestitselfonthe
basisofprecedingmaterial.Theensuingdiscussionmaywellpointthewaytobetter
things. Dwelling on practical issues we have avoided traditional problems having
neat, analytical solutions in favour of those drawn from more realistic modelling
situationswhichgenerallyhavenoanalyticsolution.
It is accepted that the best way to learn is to teach. But even more so, the best
way to understand a mathematical procedure is to implement the method on a to-
tally unforgiving computer. Matlab enables mathematics as it is written on paper
tobetransferredtoacomputerwithunrivalledeaseandsoofferseveryencourage-
ment.Thebookwillshowhowprogramsforawiderangeofproblemsfromsolving
equationstofindingoptimumsolutionsmaybedeveloped.Howeverwearenotrec-
ommendingre-inventingthewheel.Matlabprovidesanenormousrangeofreadyto
useprograms.Ouraimistogiveinsightintowhichprogramstouse,whatmaybe
expectedandhowresultsaretobeinterpreted.Tothisendwewillincludedetailsof
theMatlabversionsoftheprogramswedevelopandhowtheyaretobeemployed.
v
vi Preface
Wehopethatreaderswillenjoyourbook.Ithasbeenarefreshingexperienceto
reverse the usual form of presentation. We have tried to simplify the mathematics
asfaraspossible,andtouseinferenceandexperienceratherthanformalproofasa
firststeptowardsadeeperunderstanding.Numericalanalysisisasmuchanartasa
scienceandlikeitsbestpractitionersweshouldbepreparedtopickandchoosefrom
themethodsatourdisposaltosolvetheproblemathand.Experience,areadinessto
experimentandnotleastahealthyscepticismwhenexaminingcomputeroutputare
qualitiestobeencouraged.
NewcastleUniversity,NewcastleuponTyne,UK ChrisWoodford
ChrisPhillips
Contents
1 BasicMatlab . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.1 Matlab—TheHistoryandtheProduct . . . . . . . . . . . . . . . . 1
1.2 CreatingVariablesandUsingBasicArithmetic . . . . . . . . . . . 2
1.3 StandardFunctions . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.4 VectorsandMatrices . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.5 M-Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.6 ThecolonNotationandtheforLoop . . . . . . . . . . . . . . . . 6
1.7 Theif Construct . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
1.8 ThewhileLoop . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
1.9 SimpleScreenOutput . . . . . . . . . . . . . . . . . . . . . . . . 9
1.10 KeyboardInput . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
1.11 UserDefinedFunctions . . . . . . . . . . . . . . . . . . . . . . . 10
1.12 BasicStatistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
1.13 Plotting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
1.14 FormattedScreenOutput . . . . . . . . . . . . . . . . . . . . . . 12
1.15 FileInputandOutput . . . . . . . . . . . . . . . . . . . . . . . . 14
1.15.1 FormattedOutputtoaFile. . . . . . . . . . . . . . . . . . 14
1.15.2 FormattedInputfromaFile . . . . . . . . . . . . . . . . . 14
1.15.3 UnformattedInputandOutput(SavingandRetrievingData) 15
2 LinearEquations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
2.2 LinearSystems . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
2.3 GaussianElimination . . . . . . . . . . . . . . . . . . . . . . . . 22
2.3.1 RowInterchanges . . . . . . . . . . . . . . . . . . . . . . 24
2.3.2 PartialPivoting. . . . . . . . . . . . . . . . . . . . . . . . 26
2.3.3 MultipleRight-HandSides . . . . . . . . . . . . . . . . . 30
2.4 SingularSystems. . . . . . . . . . . . . . . . . . . . . . . . . . . 32
2.5 SymmetricPositiveDefiniteSystems . . . . . . . . . . . . . . . . 33
2.6 IterativeRefinement . . . . . . . . . . . . . . . . . . . . . . . . . 35
2.7 Ill-ConditionedSystems . . . . . . . . . . . . . . . . . . . . . . . 37
2.8 Gauss–SeidelIteration . . . . . . . . . . . . . . . . . . . . . . . . 37
vii
viii Contents
3 NonlinearEquations . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
3.2 BisectionMethod . . . . . . . . . . . . . . . . . . . . . . . . . . 49
3.2.1 FindinganIntervalContainingaRoot . . . . . . . . . . . 50
3.3 RuleofFalsePosition . . . . . . . . . . . . . . . . . . . . . . . . 51
3.4 TheSecantMethod . . . . . . . . . . . . . . . . . . . . . . . . . 52
3.5 Newton–RaphsonMethod . . . . . . . . . . . . . . . . . . . . . . 55
3.6 ComparisonofMethodsforaSingleEquation . . . . . . . . . . . 58
3.7 Newton’sMethodforSystemsofNonlinearEquations . . . . . . . 59
3.7.1 HigherOrderSystems . . . . . . . . . . . . . . . . . . . . 63
4 CurveFitting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
4.2 LinearInterpolation . . . . . . . . . . . . . . . . . . . . . . . . . 72
4.2.1 Differences . . . . . . . . . . . . . . . . . . . . . . . . . . 75
4.3 PolynomialInterpolation . . . . . . . . . . . . . . . . . . . . . . 77
4.3.1 NewtonInterpolation . . . . . . . . . . . . . . . . . . . . 77
4.3.2 NevilleInterpolation . . . . . . . . . . . . . . . . . . . . . 80
4.3.3 AComparisonofNewtonandNevilleInterpolation . . . . 81
4.3.4 SplineInterpolation . . . . . . . . . . . . . . . . . . . . . 83
4.4 LeastSquaresApproximation . . . . . . . . . . . . . . . . . . . . 86
4.4.1 LeastSquaresStraightLineApproximation. . . . . . . . . 86
4.4.2 LeastSquaresPolynomialApproximation . . . . . . . . . 89
5 NumericalIntegration . . . . . . . . . . . . . . . . . . . . . . . . . . 97
5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98
5.2 IntegrationofTabulatedFunctions . . . . . . . . . . . . . . . . . 98
5.2.1 TheTrapeziumRule . . . . . . . . . . . . . . . . . . . . . 99
5.2.2 QuadratureRules . . . . . . . . . . . . . . . . . . . . . . 101
5.2.3 Simpson’sRule . . . . . . . . . . . . . . . . . . . . . . . 101
5.2.4 IntegrationfromIrregularly-SpacedData . . . . . . . . . . 102
5.3 IntegrationofFunctions . . . . . . . . . . . . . . . . . . . . . . . 104
5.3.1 Analyticvs.NumericalIntegration . . . . . . . . . . . . . 104
5.3.2 TheTrapeziumRule(Again) . . . . . . . . . . . . . . . . 104
5.3.3 Simpson’sRule(Again) . . . . . . . . . . . . . . . . . . . 106
5.4 HigherOrderRules . . . . . . . . . . . . . . . . . . . . . . . . . 109
5.5 GaussianQuadrature . . . . . . . . . . . . . . . . . . . . . . . . . 110
5.6 AdaptiveQuadrature . . . . . . . . . . . . . . . . . . . . . . . . . 112
6 NumericalDifferentiation . . . . . . . . . . . . . . . . . . . . . . . . 119
6.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120
6.2 Two-PointFormula . . . . . . . . . . . . . . . . . . . . . . . . . 120
6.3 Three-andFive-PointFormulae . . . . . . . . . . . . . . . . . . . 122
6.4 HigherOrderDerivatives . . . . . . . . . . . . . . . . . . . . . . 125
6.4.1 ErrorAnalysis . . . . . . . . . . . . . . . . . . . . . . . . 126
6.5 Cauchy’sTheorem . . . . . . . . . . . . . . . . . . . . . . . . . . 128
Contents ix
7 LinearProgramming . . . . . . . . . . . . . . . . . . . . . . . . . . . 135
7.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136
7.2 FormingaLinearProgrammingProblem . . . . . . . . . . . . . . 136
7.3 StandardForm . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140
7.4 CanonicalForm . . . . . . . . . . . . . . . . . . . . . . . . . . . 141
7.5 TheSimplexMethod. . . . . . . . . . . . . . . . . . . . . . . . . 142
7.5.1 StartingtheSimplexMethod . . . . . . . . . . . . . . . . 146
7.6 IntegerProgramming . . . . . . . . . . . . . . . . . . . . . . . . 149
7.6.1 TheBranchandBoundMethod . . . . . . . . . . . . . . . 151
7.7 DecisionProblems . . . . . . . . . . . . . . . . . . . . . . . . . . 153
7.8 TheTravellingSalesmanProblem . . . . . . . . . . . . . . . . . . 155
7.9 TheMachineSchedulingProblem . . . . . . . . . . . . . . . . . . 156
8 Optimisation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169
8.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 170
8.2 GridSearchingMethods . . . . . . . . . . . . . . . . . . . . . . . 171
8.2.1 SimpleGridSearch . . . . . . . . . . . . . . . . . . . . . 171
8.2.2 GoldenSectionSearch. . . . . . . . . . . . . . . . . . . . 173
8.3 UnconstrainedOptimisation . . . . . . . . . . . . . . . . . . . . . 175
8.3.1 TheMethodofSteepestDescent . . . . . . . . . . . . . . 176
8.3.2 ARank-OneMethod. . . . . . . . . . . . . . . . . . . . . 178
8.3.3 GeneralisedRank-OneMethod . . . . . . . . . . . . . . . 181
8.4 ConstrainedOptimisation . . . . . . . . . . . . . . . . . . . . . . 184
8.4.1 MinimisationbyUseofaSimplePenaltyFunction . . . . . 185
8.4.2 MinimisationUsingtheLagrangian . . . . . . . . . . . . . 187
8.4.3 TheMultiplierFunctionMethod . . . . . . . . . . . . . . 188
9 OrdinaryDifferentialEquations . . . . . . . . . . . . . . . . . . . . 197
9.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 198
9.2 First-OrderEquations . . . . . . . . . . . . . . . . . . . . . . . . 200
9.2.1 Euler’sMethod. . . . . . . . . . . . . . . . . . . . . . . . 200
9.2.2 Runge–KuttaMethods . . . . . . . . . . . . . . . . . . . . 202
9.2.3 Fourth-OrderRunge–Kutta . . . . . . . . . . . . . . . . . 204
9.2.4 SystemsofFirst-OrderEquations . . . . . . . . . . . . . . 206
9.2.5 HigherOrderEquations . . . . . . . . . . . . . . . . . . . 207
9.3 BoundaryValueProblems . . . . . . . . . . . . . . . . . . . . . . 208
9.3.1 ShootingMethod . . . . . . . . . . . . . . . . . . . . . . 208
9.3.2 DifferenceEquations . . . . . . . . . . . . . . . . . . . . 209
10 EigenvaluesandEigenvectors . . . . . . . . . . . . . . . . . . . . . . 215
10.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 215
10.2 TheCharacteristicPolynomial . . . . . . . . . . . . . . . . . . . 217
10.3 ThePowerMethod . . . . . . . . . . . . . . . . . . . . . . . . . . 218
10.3.1 PowerMethod,Theory . . . . . . . . . . . . . . . . . . . 219
10.4 EigenvaluesofSpecialMatrices . . . . . . . . . . . . . . . . . . . 222
10.4.1 Eigenvalues,DiagonalMatrix . . . . . . . . . . . . . . . . 222
10.4.2 Eigenvalues,UpperTriangularMatrix . . . . . . . . . . . 223
x Contents
10.5 ASimpleQRMethod . . . . . . . . . . . . . . . . . . . . . . . . 223
11 Statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 231
11.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 232
11.2 StatisticalTerms . . . . . . . . . . . . . . . . . . . . . . . . . . . 232
11.2.1 RandomVariable . . . . . . . . . . . . . . . . . . . . . . 232
11.2.2 FrequencyDistribution . . . . . . . . . . . . . . . . . . . 232
11.2.3 ExpectedValue,AverageandMean . . . . . . . . . . . . . 234
11.2.4 VarianceandStandardDeviation . . . . . . . . . . . . . . 234
11.2.5 CovarianceandCorrelation . . . . . . . . . . . . . . . . . 236
11.3 LeastSquaresAnalysis . . . . . . . . . . . . . . . . . . . . . . . 239
11.4 RandomNumbers . . . . . . . . . . . . . . . . . . . . . . . . . . 241
11.4.1 GeneratingRandomNumbers . . . . . . . . . . . . . . . . 242
11.5 RandomNumberGenerators . . . . . . . . . . . . . . . . . . . . 243
11.5.1 CustomisingRandomNumbers . . . . . . . . . . . . . . . 243
11.6 MonteCarloIntegration . . . . . . . . . . . . . . . . . . . . . . . 244
MatlabIndex . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 249
Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 253