Nonlinear Parameter Optimization Using R Tools

305 Pages·2014·3.422 MB·English

by  John C. Nash

Nonlinear Parameter
Optimization Using R Tools

Nonlinear Parameter
Optimization Using R Tools
John C. Nash
TelferSchoolofManagement
UniversityofOttawa
Thiseditionfirstpublished2014
©2014JohnWileyandSonsLtd
Registeredoffice
JohnWiley&SonsLtd,TheAtrium,SouthernGate,Chichester,WestSussex,PO198SQ,UnitedKingdom
Fordetailsofourglobaleditorialoffices,forcustomerservicesandforinformationabouthowtoapplyforper-
missiontoreusethecopyrightmaterialinthisbookpleaseseeourwebsiteatwww.wiley.com.
TherightoftheauthortobeidentifiedastheauthorofthisworkhasbeenassertedinaccordancewiththeCopy-
right,DesignsandPatentsAct1988.
Allrightsreserved.Nopartofthispublicationmaybereproduced,storedinaretrievalsystem,ortransmitted,in
anyformorbyanymeans,electronic,mechanical,photocopying,recordingorotherwise,exceptaspermittedby
theUKCopyright,DesignsandPatentsAct1988,withoutthepriorpermissionofthepublisher.
Wileyalsopublishesitsbooksinavarietyofelectronicformats.Somecontentthatappearsinprintmaynotbe
availableinelectronicbooks.
LimitofLiability/DisclaimerofWarranty:Whilethepublisherandauthorhaveusedtheirbesteffortsinpreparing
thisbook,theymakenorepresentationsorwarrantieswithrespecttotheaccuracyorcompletenessofthecontents
ofthisbookandspecificallydisclaimanyimpliedwarrantiesofmerchantabilityorfitnessforaparticularpurpose.
Itissoldontheunderstandingthatthepublisherisnotengagedinrenderingprofessionalservicesandneither
thepublishernortheauthorshallbeliablefordamagesarisingherefrom.Ifprofessionaladviceorotherexpert
assistanceisrequired,theservicesofacompetentprofessionalshouldbesought.
MATLAB®isatrademarkofTheMathWorks,Inc.andisusedwithpermission.TheMathWorksdoesnotwarrant
theaccuracyofthetextorexercisesinthisbook.Thisbook’suseordiscussionofMATLAB®softwareorrelated
productsdoesnotconstituteendorsementorsponsorshipbyTheMathWorksofaparticularpedagogicalapproach
orparticularuseoftheMATLAB®software.
LibraryofCongressCataloging-in-PublicationData
Nash,JohnC.,1947-
NonlinearparameteroptimizationusingRtools/JohnC.Nash.
pagescm
Includesbibliographicalreferencesandindex.
ISBN978-1-118-56928-3(cloth)
1.Mathematicaloptimization.2.Nonlineartheories.3.R(Computerprogramlanguage)I.Title.
QA402.5.N342014
519.60285′5133–dc23
2013051141
AcataloguerecordforthisbookisavailablefromtheBritishLibrary.
ISBN:9781118569283
Typesetin10/12ptTimesLTStdbyLaserwordsPrivateLimited,Chennai,India
PrintedandboundinSingaporebyMarkonoPrintMediaPteLtd.
1 2014
Thisworkisapartofandisdedicatedtothateffortbythemany
community-minded peoplewhocreate,support,promote,andusefree
andopensourcesoftware,andwhogenerouslysharetheseideas,
withoutwhichRinparticularwouldnotexist.

Contents
Preface xv
1 Optimizationproblemtasksandhowtheyarise 1
1.1 Thegeneraloptimizationproblem 1
1.2 Whythegeneralproblemisgenerallyuninteresting 2
1.3 (Non-)Linearity 4
1.4 Objectivefunctionproperties 4
1.4.1 Sumsofsquares 4
1.4.2 Minimaxapproximation 5
1.4.3 Problemswithmultipleminima 5
1.4.4 Objectivesthatcanonlybeimpreciselycomputed 5
1.5 Constrainttypes 5
1.6 Solvingsetsofequations 6
1.7 Conditionsforoptimality 7
1.8 Otherclassifications 7
References 8
2 Optimizationalgorithms–anoverview 9
2.1 Methodsthatusethegradient 9
2.2 Newton-likemethods 12
2.3 ThepromiseofNewton’smethod 13
2.4 Caution:convergenceversustermination 14
2.5 DifficultieswithNewton’smethod 14
2.6 Leastsquares:Gauss–Newtonmethods 15
2.7 Quasi-Newtonorvariablemetricmethod 17
2.8 Conjugategradientandrelatedmethods 18
2.9 Othergradientmethods 19
2.10 Derivative-freemethods 19
2.10.1 Numericalapproximationofgradients 19
2.10.2 Approximateanddescend 19
2.10.3 Heuristicsearch 20
2.11 Stochasticmethods 20
2.12 Constraint-basedmethods–mathematicalprogramming 21
References 22
viii CONTENTS
3 Softwarestructureandinterfaces 25
3.1 Perspective 25
3.2 Issuesofchoice 26
3.3 Softwareissues 27
3.4 Specifyingtheobjectiveandconstraintstotheoptimizer 28
3.5 Communicatingexogenousdatatoproblem
definitionfunctions 28
3.5.1 Useof“global”dataandvariables 31
3.6 Masked(temporarilyfixed)optimizationparameters 32
3.7 Dealingwithinadmissibleresults 33
3.8 Providingderivativesforfunctions 34
3.9 Derivativeapproximationswhenthereareconstraints 36
3.10 Scalingofparametersandfunction 36
3.11 Normalendingofcomputations 36
3.12 Terminationtests–abnormalending 37
3.13 Outputtomonitorprogressofcalculations 37
3.14 Outputoftheoptimizationresults 38
3.15 Controlsfortheoptimizer 38
3.16 Defaultcontrolsettings 39
3.17 Measuringperformance 39
3.18 Theoptimizationinterface 39
References 40
4 One-parameterroot-findingproblems 41
4.1 Roots 41
4.2 Equationsinonevariable 42
4.3 Someexamples 42
4.3.1 Exponentiallyspeaking 42
4.3.2 Anormalconcern 44
4.3.3 LittlePollyNomial 46
4.3.4 Ahypothequialquestion 49
4.4 Approachestosolving1Droot-findingproblems 51
4.5 Whatcangowrong? 52
4.6 Beingasmartuserofroot-findingprograms 54
4.7 Conclusionsandextensions 54
References 55
5 One-parameterminimizationproblems 56
5.1 Theoptimize()function 56
5.2 Usingaroot-finder 57
5.3 Butwhereistheminimum? 58
5.4 Ideasfor1Dminimizers 59
5.5 Theline-searchsubproblem 61
References 62