Table Of Content

Nonlinear Parameter Optimization Using R Tools Nonlinear Parameter Optimization Using R Tools John C. Nash TelferSchoolofManagement UniversityofOttawa Thiseditionfirstpublished2014 ©2014JohnWileyandSonsLtd Registeredoffice JohnWiley&SonsLtd,TheAtrium,SouthernGate,Chichester,WestSussex,PO198SQ,UnitedKingdom Fordetailsofourglobaleditorialoffices,forcustomerservicesandforinformationabouthowtoapplyforper- missiontoreusethecopyrightmaterialinthisbookpleaseseeourwebsiteatwww.wiley.com. TherightoftheauthortobeidentifiedastheauthorofthisworkhasbeenassertedinaccordancewiththeCopy- right,DesignsandPatentsAct1988. Allrightsreserved.Nopartofthispublicationmaybereproduced,storedinaretrievalsystem,ortransmitted,in anyformorbyanymeans,electronic,mechanical,photocopying,recordingorotherwise,exceptaspermittedby theUKCopyright,DesignsandPatentsAct1988,withoutthepriorpermissionofthepublisher. Wileyalsopublishesitsbooksinavarietyofelectronicformats.Somecontentthatappearsinprintmaynotbe availableinelectronicbooks. LimitofLiability/DisclaimerofWarranty:Whilethepublisherandauthorhaveusedtheirbesteffortsinpreparing thisbook,theymakenorepresentationsorwarrantieswithrespecttotheaccuracyorcompletenessofthecontents ofthisbookandspecificallydisclaimanyimpliedwarrantiesofmerchantabilityorfitnessforaparticularpurpose. Itissoldontheunderstandingthatthepublisherisnotengagedinrenderingprofessionalservicesandneither thepublishernortheauthorshallbeliablefordamagesarisingherefrom.Ifprofessionaladviceorotherexpert assistanceisrequired,theservicesofacompetentprofessionalshouldbesought. MATLAB®isatrademarkofTheMathWorks,Inc.andisusedwithpermission.TheMathWorksdoesnotwarrant theaccuracyofthetextorexercisesinthisbook.Thisbook’suseordiscussionofMATLAB®softwareorrelated productsdoesnotconstituteendorsementorsponsorshipbyTheMathWorksofaparticularpedagogicalapproach orparticularuseoftheMATLAB®software. LibraryofCongressCataloging-in-PublicationData Nash,JohnC.,1947- NonlinearparameteroptimizationusingRtools/JohnC.Nash. pagescm Includesbibliographicalreferencesandindex. ISBN978-1-118-56928-3(cloth) 1.Mathematicaloptimization.2.Nonlineartheories.3.R(Computerprogramlanguage)I.Title. QA402.5.N342014 519.60285′5133–dc23 2013051141 AcataloguerecordforthisbookisavailablefromtheBritishLibrary. ISBN:9781118569283 Typesetin10/12ptTimesLTStdbyLaserwordsPrivateLimited,Chennai,India PrintedandboundinSingaporebyMarkonoPrintMediaPteLtd. 1 2014 Thisworkisapartofandisdedicatedtothateffortbythemany community-minded peoplewhocreate,support,promote,andusefree andopensourcesoftware,andwhogenerouslysharetheseideas, withoutwhichRinparticularwouldnotexist. Contents Preface xv 1 Optimizationproblemtasksandhowtheyarise 1 1.1 Thegeneraloptimizationproblem 1 1.2 Whythegeneralproblemisgenerallyuninteresting 2 1.3 (Non-)Linearity 4 1.4 Objectivefunctionproperties 4 1.4.1 Sumsofsquares 4 1.4.2 Minimaxapproximation 5 1.4.3 Problemswithmultipleminima 5 1.4.4 Objectivesthatcanonlybeimpreciselycomputed 5 1.5 Constrainttypes 5 1.6 Solvingsetsofequations 6 1.7 Conditionsforoptimality 7 1.8 Otherclassifications 7 References 8 2 Optimizationalgorithms–anoverview 9 2.1 Methodsthatusethegradient 9 2.2 Newton-likemethods 12 2.3 ThepromiseofNewton’smethod 13 2.4 Caution:convergenceversustermination 14 2.5 DifficultieswithNewton’smethod 14 2.6 Leastsquares:Gauss–Newtonmethods 15 2.7 Quasi-Newtonorvariablemetricmethod 17 2.8 Conjugategradientandrelatedmethods 18 2.9 Othergradientmethods 19 2.10 Derivative-freemethods 19 2.10.1 Numericalapproximationofgradients 19 2.10.2 Approximateanddescend 19 2.10.3 Heuristicsearch 20 2.11 Stochasticmethods 20 2.12 Constraint-basedmethods–mathematicalprogramming 21 References 22 viii CONTENTS 3 Softwarestructureandinterfaces 25 3.1 Perspective 25 3.2 Issuesofchoice 26 3.3 Softwareissues 27 3.4 Specifyingtheobjectiveandconstraintstotheoptimizer 28 3.5 Communicatingexogenousdatatoproblem definitionfunctions 28 3.5.1 Useof“global”dataandvariables 31 3.6 Masked(temporarilyfixed)optimizationparameters 32 3.7 Dealingwithinadmissibleresults 33 3.8 Providingderivativesforfunctions 34 3.9 Derivativeapproximationswhenthereareconstraints 36 3.10 Scalingofparametersandfunction 36 3.11 Normalendingofcomputations 36 3.12 Terminationtests–abnormalending 37 3.13 Outputtomonitorprogressofcalculations 37 3.14 Outputoftheoptimizationresults 38 3.15 Controlsfortheoptimizer 38 3.16 Defaultcontrolsettings 39 3.17 Measuringperformance 39 3.18 Theoptimizationinterface 39 References 40 4 One-parameterroot-findingproblems 41 4.1 Roots 41 4.2 Equationsinonevariable 42 4.3 Someexamples 42 4.3.1 Exponentiallyspeaking 42 4.3.2 Anormalconcern 44 4.3.3 LittlePollyNomial 46 4.3.4 Ahypothequialquestion 49 4.4 Approachestosolving1Droot-findingproblems 51 4.5 Whatcangowrong? 52 4.6 Beingasmartuserofroot-findingprograms 54 4.7 Conclusionsandextensions 54 References 55 5 One-parameterminimizationproblems 56 5.1 Theoptimize()function 56 5.2 Usingaroot-finder 57 5.3 Butwhereistheminimum? 58 5.4 Ideasfor1Dminimizers 59 5.5 Theline-searchsubproblem 61 References 62