Fundamentals of optimization. Methods, minimum principles and applications

252 Pages·2018·7.112 MB·English

by  French M.;Luby

Mark French
Fundamentals
of Optimization
Methods, Minimum Principles and
Applications for making Things better
MarkFrench
SchoolofEngineeringTechnology
PurdueUniversity
WestLafayette,IN,USA
ISBN978-3-319-76191-6 ISBN978-3-319-76192-3 (eBook)
https://doi.org/10.1007/978-3-319-76192-3
LibraryofCongressControlNumber:2018935277
©SpringerInternationalPublishingAG,partofSpringerNature2018
Foreword
I have had the great pleasure in getting to know Mark French’s dedication and
innovativewaysofteachingengineeringprinciplesoverthelastfewyears.Markis
trulyanexceptionalteacherandthisisreflectedinhislatestwork:Fundamentalsof
Optimization: Methods, Minimum Principles, and Applications for Making Things
Better. The best textbook authors for education tend to be those that have a strong
desiretosimplifyandpresentdifficulttopics insuchawayastocarefullyleadthe
studentalongalearningjourneythatbuildsselfconfidenceinthesubject.Beingan
author of numerous textbooks myself, I know that dedicated teachers make great
textbookauthorsandMark’sdedicationisreflectedinhisbook.
Thisbookisanintroductorytreatiseonthetopicofoptimization inengineering
design.Thepointofthisbookistoprovideareferenceforstudentsandprofessionals
who need a clear introduction to optimization, with actionable information and
examples that show how the calculations work. This is basically the book Mark
Frenchneededwhenhelearnedtouseoptimizationasacivilianaerospaceengineer
withtheUSAirForce.
There are a number of mathematically rigorous optimization books in print and
someareverygood,buttheyaregenerallygearedtowardsresearchersandadvanced
graduate students. They tend toward abstract and theoretical presentations, often
goingsofarastointroducesubjectsusingamathematicallyformalstatement-proof
structure.Whilethisisquiteappropriatefortheresearchcommunity,itisthewrong
approachforstudentswhoneedtolearnhowtoimproveprocessesandproducts.
The main features of the book that make it useful for teaching and learning
includethefollowing:
(cid:129) Afocusonmeaningful,lessabstractproblems
(cid:129) Samplecalculationsincludingintermediateresults
(cid:129) Preferenceforaccessible,robustmethods
(cid:129) ExamplesshowingimplementationinMATLAB
(cid:129) Graphicalresultswheneverpossible
(cid:129) Historical and industrial examples to give context to the development of
optimization
Each chapter steps the reader through example problems, which include text
explainingtheprocess,thecalculations,andthegraphicrepresentationoftheresults.
This step-by-step learning process is an ideal teaching methodology that leads the
learner through the concept being presented in a format that is easy to follow and
understand. The use of real-world examples is especially compelling as it has the
potentialtodrawthelearnerintotheproblem.Asweknowaself-motivatedlearner
tendstobemoreengagedandinterestedinthetopic.
Overall,MarkFrenchhaswrittenabookthatisverycompellinginitsapproach,
and his writing style and use of examples are some of the best I have seen in
mathematics textbooks. Mark is an outstanding classroom teacher and that is
reflectedinthisbook.
PurdueUniversity,WestLafayette, GaryBertoline
IN,USA
Preface
Thisisanoptimizationbookforpeoplewhoneedtosolveproblems.Itiswrittenfor
those who know very little – maybe nothing – about optimization, but need to
understandbasicmethodswellenoughtoapplythemtopracticalproblems.Asurvey
ofavailablebooksonoptimizationfindsanumberofverygoodones,butoftenones
written using high-level mathematics and sometimes even written in a statement-
proofformat.
Clearly, these books are valuable for people wishing to work at a high level of
technicalsophistication,buttheyaresometimesopaquetopeopleneeding,rather,an
accessibleintroduction.Ithinkthereneedstobeabookthatbringsareaderfroma
pointofknowingnexttonothingtoapointofknowingthebigideas,thenecessary
vocabulary, a group of simple and robust methods, and, above all, a sense of how
optimizationcanbeused.
This book has resulted from years of teaching optimization to both engineering
and engineering technology students and from years spent applying optimization
methods to problems of practical interest. Its content has been tested on many
classrooms full of inquisitive students, and only the lessons they think were suc-
cessfulappearhere.
The number of methods presented here is small; I have selected a few simple,
robust methods to describe in detail so that readers can develop a useful mental
framework for understanding how information is processed by optimization soft-
ware.Whilemoresophisticatedmethodsarecertainlymoreefficient,theydon’tlook
thatdifferenttotheuser.Thus,thisbookstickstothesimple,robust,andaccessible
methods.
Ihaveusedtwocalculationtools,MathcadandMATLAB.MATLABis,atthis
writing, probably the standard tool for general technical calculation, at least in the
UnitedStatesandmuchofEurope.Themoresophisticatedplotsandmoreextensive
calculationsinthisbookweregenerallydoneusingMATLAB.However,MATLAB
is a programming language that looks familiar to anyone who ever learned FOR-
TRAN.Whilequitepowerful,MATLABcode,especiallyefficient,vectorizedcode,
canbeverydifficulttoread–arealproblemforabooklikethis.
Mathcadisaverydifferentanimal,designedtomakesmallercalculationssimply
andwithlittleneedforformalprogramming.Itisessentiallyamathscratchpadthat
iseasytouseandeasytoread.Ichosetoillustratethedetailsofspecificcalculations
usingMathcadbecauseevenreaderswhoarenewtoMathcadcanprobablyreadthe
samplefiles,eveniftheycouldnotwritethem.TheMathcadfilesinthisbookwere
made using Mathcad V15. The name Mathcad is not, in my opinion, a very good
one,thoughIverymuchlikethesoftware.IthasnothingtodowithCADorgraphics
software.
Finally,I hope this book will serve the needs of analysts for whom theresultis
moreimportantthanthealgorithm.Thosewhowisheventuallytostudythebehavior
of algorithms and to perhaps even develop better methods may, I hope, find this
volumeausefulstartbeforemovingontomoreadvancedtexts.
WestLafayette,IN,USA MarkFrench
Contents
1 Optimization:TheBigIdea. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.1 DesignSpace. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.2 WhatIsOptimum?. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
References. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
2 GettingStartedinOptimization:Problems
ofaSingleVariable. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
2.1 TheLifeguardProblem. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
2.2 MaximumRangeofaProjectilewithAerodynamics. . . . . . . . . . 18
2.3 MaximumElectricalPowerDeliveredtoaLoad. . . . . . . . . . . . . 21
2.4 ShortestTimePathforaToyCar. . . . . . . . . . . . . . . . . . . . . . . . 24
2.5 WeightonaSnap-ThroughSpring. . . . . . . . . . . . . . . . . . . . . . . 27
2.6 SolutionMethods. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
2.6.1 AnalyticalSolution. . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
2.6.2 MonteCarloMethod. . . . . . . . . . . . . . . . . . . . . . . . . . . 32
2.6.3 BinarySearch. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
2.6.4 QuadraticApproximationMethod. . . . . . . . . . . . . . . . . . 36
2.6.5 HybridMethods. .. . . . . . . . .. . . . . . . . .. . . . . . . . .. . 39
2.7 FindingRootsUsingMinimization. . . . . . . . . . . . . . . . . . . . . . 41
2.8 OptimizingaProcess:OceanShippingRoutes. . . . . . . . . . . . . . 43
2.9 MATLABExamples. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
References. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
3 MinimumPrinciples:OptimizationintheFabricoftheUniverse. . 55
3.1 Evolution. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
3.2 MinimumEnergyStructures. . . . . . . . . . . . . . . . . . . . . . . . . . . 58
3.3 Optics:Fermat’sPrincipleandSnell’sLaws. . . . . . . . . . . . . . . . 66
3.4 GeneralRelativity. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
References. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
4 ProblemswithMorethanOneVariable. . . . . . . . . . . . . . . . . . . . . 71
4.1 Two-VariableLifeguardProblem. . . . . . . . . . . . . . . . . . . . . . . . 71
4.2 LeastSquaresCurveFitting. . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
4.3 Two-VariableSoapFilmProblem. . . . . . . . . . . . . . . . . . . . . . . 77
4.4 SolutionMethods. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
4.4.1 MonteCarloMethod. . . . . . . . . . . . . . . . . . . . . . . . . . . 80
4.4.2 MarchingGridAlgorithm. . . . . . . . . . . . . . . . . . . . . . . . 82
4.4.3 SteepestDescentMethod. . . . . . . . . . . . . . . . . . . . . . . . 85
4.4.4 ConjugateGradientMethod. . . . . . . . . . . . . . . . . . . . . . 94
4.4.5 Newton’sMethod. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
4.4.6 Quasi-Newton’sMethods. . . . . . . . . . . . . . . . . . . . . . . . 103
4.5 ApproximateSolutiontoaDifferentialEquation. . . . . . . . . . . . . 104
4.6 Evolution-InspiredSemi-RandomSearch:
FollowingNature’sLead. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108
4.7 ConvexProblems. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114
4.7.1 SequentialLinearProgramming. . . . . . . . . . . . . . . . . . . 117
4.8 OptimizationattheLimits:UnlimitedClassAirRacers. . . . . . . . 119
References. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125
5 Constraints:PlacingLimitsontheSolution. . . .. . . . . . .. . . . . . .. 127
5.1 MaximumVolumeofaBox. . . . . . . . . . . . . . . . . . . . . . . . . . . 127
5.2 ConstrainedLifeguardProblem:ExteriorPenaltyFunction. . . . . 130
5.3 MinimumSurfaceAreaofaCan. . . . . . . . . . . . . . . . . . . . . . . . 133
5.4 EqualityConstraints. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137
5.5 ApproximateSolutions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138
5.5.1 HangingChainProblem. . . . . . . . . . . . . . . . . . . . . . . . . 138
References. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142
6 GeneralConditionsforSolvingOptimizationProblems:
Karush-Kuhn-TuckerConditions. . . . . . . . . . . . . . . . . . . . . . . . . . 143
6.1 GeneralFormofKKTConditions. . . . . . . . . . . . . . . . . . . . . . . 143
6.2 ApplicationtoanUnconstrainedProblemofOneVariable. . . . . 144
6.3 ApplicationtoanUnconstrainedProblemofTwoVariables. . . . 145
6.4 ApplicationtoaSingleVariableProblemwithanEquality
Constraint. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147
6.5 ApplicationtoaMultivariableProblemwithInequality
Constraints. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 150
6.6 MultipleConstraints. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155
References. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157
7 DiscreteVariables. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159
7.1 TheTravelingSalesmanProblem. . . . . . . . . . . . . . . . . . . . . . . . 159
7.2 NearestNeighborAlgorithm. . . . . . . . . . . . . . . . . . . . . . . . . . . 160
7.3 ApplicationsoftheTravelingSalesmanProblem. . . . . .. . . . . .. 164
7.3.1 TheVehicleRoutingProblem. . . . . . . . . . . . . . . . . . . . . 164
7.3.2 ComputerCircuitDesign. . . . . . . . . . . . . . . . . . . . . . . . 165
7.4 DiscreteVariableProblems. . . . . . . . . . . . . . . . . . . . . . . . . . . . 168
7.5 ExamplesofDiscreteVariableDesignProblems. . . . . . . . . . . . . 170
7.5.1 MarineDieselEngine. . . . . . . . . . . . . . . . . . . . . . . . . . . 171
7.5.2 WindTurbines. . .. . . . . . . . . . . . . . . . . . . . . . . . .. . . . 173
References. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175
8 AerospaceApplications. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177
8.1 MonoplaneorBiplane. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179
8.2 WingPlanforms. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 181
8.3 VehiclePerformance. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 186
8.4 AircraftDesignOptimization. . . . . . . . . . . . . . . . . . . . . . . . . . . 189
References. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189
9 StructuralOptimization. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 191
9.1 TrussStructures. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192
9.2 UpdatingFiniteElementModels. . . . . . . . . . . . . . . . . . . . . . . . 194
9.3 AeroelasticallyScaledWindTunnelModelDesign. . . . . . . . . . . 197
References. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 199
10 MultiobjectiveOptimization. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201
10.1 TheNeedforMultiobjectiveOptimization. . . . . . . . . . . . . . . . 201
10.2 ASimpleMultiobjectiveProblem. . . . . . . . . . . . . . . . . . . . . . 202
10.3 WeightedObjectivesMethod. . . . . . . . . . . . . . . . . . . . . . . . . . 203
10.4 HierarchicalOptimizationMethod. . . . . . . . . . . . . . . . . . . . . . 205
10.5 GlobalCriterionMethod. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 207
10.6 ParetoOptimality. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 208
10.7 AMultiobjectiveInverseSpectralProblem. . . . . . . . . . . . . . . . 208
10.8 AComplexMultiobjectiveDesignProblem. . . . . . . . . . . . . . . 212
References. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213
AppendixA:Derivatives. . . . .. . . . . . . . . .. . . . . . . . . .. . . . . . . . . .. . 215
DerivativesofaSingleVariable. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 215
DerivativesofMorethanOneVariable. .. . . . . . .. . . . . . .. . . . . . .. 217
AppendixB:TestFunctions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 219
SingleVariableFunctions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 219
TwoVariableFunctions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 223
ThreeVariableFunction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 228
AppendixC:SolvingOptimizationProblemsUsingMATLAB. . . . . . . . 229
FindingRootswithfzero. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 230
SolvingSystemsofEquationsUsingFsolve. . . . . . . . . . . . . . . . . . . . 232
MinimizationUsingfminsearch. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 238
UnconstrainedMinimizationUsingFminunc. . . . . . . . . . . . . . . . . . . 240
ConstrainedMinimizationUsingFmincon. . . . . . . . . . . . . . . . . . . . . 242
Index. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 245