Table Of ContentSpringer Optimization and Its Applications 123
Panos M. Pardalos
Antanas Žilinskas
Julius Žilinskas
Non-Convex
Multi-
Objective
Optimization
Springer Optimization and Its Applications
Volume 123
ManagingEditor
PanosM.Pardalos(UniversityofFlorida)
Editor-CombinatorialOptimization
Ding-ZhuDu(UniversityofTexasatDallas)
AdvisoryBoard
J.Birge(UniversityofChicago)
C.A.Floudas(TexasA&MUniversity)
F.Giannessi(UniversityofPisa)
H.D.Sherali(VirginiaPolytechnicandStateUniversity)
T.Terlaky(LehighUniversity)
Y.Ye(StanfordUniversity)
AimsandScope
Optimizationhasbeenexpandinginalldirectionsatanastonishingrateduringthe
lastfewdecades.Newalgorithmicandtheoreticaltechniqueshavebeendeveloped,
thediffusionintootherdisciplineshasproceededatarapidpace,andourknowledge
ofallaspectsofthefieldhasgrownevenmoreprofound.Atthesametime,oneof
themoststrikingtrendsinoptimizationistheconstantlyincreasingemphasisonthe
interdisciplinarynatureofthefield.Optimizationhasbeenabasictoolinallareas
ofappliedmathematics,engineering,medicine,economics,andothersciences.
The series Springer Optimization and Its Applications publishes undergraduate
and graduate textbooks, monographs and state-of-the-art expository work that
focusonalgorithmsforsolvingoptimizationproblemsandalsostudyapplications
involvingsuchproblems.Someofthetopicscoveredincludenonlinearoptimization
(convex and nonconvex), network flow problems, stochastic optimization, optimal
control, discrete optimization, multi-objective programming, description of soft-
warepackages,approximationtechniquesandheuristicapproaches.
Moreinformationaboutthisseriesathttp://www.springer.com/series/7393
Panos M. Pardalos • Antanas Žilinskas
Julius Žilinskas
Non-Convex Multi-Objective
Optimization
123
PanosM.Pardalos AntanasŽilinskas
DepartmentofIndustrial InstituteofMathematics&Informatics
andSystemsEngineering VilniusUniversity
UniversityofFlorida Vilnius,Lithuania
Gainesville,FL,USA
ResearchUniversityHigherSchool
ofEconomics,Russia
JuliusŽilinskas
InstituteofMathematics&Informatics
VilniusUniversity
Vilnius,Lithuania
ISSN1931-6828 ISSN1931-6836 (electronic)
SpringerOptimizationandItsApplications
ISBN978-3-319-61005-4 ISBN978-3-319-61007-8 (eBook)
DOI10.1007/978-3-319-61007-8
LibraryofCongressControlNumber:2017946557
©SpringerInternationalPublishingAG2017
Thisworkissubjecttocopyright.AllrightsarereservedbythePublisher,whetherthewholeorpartof
thematerialisconcerned,specificallytherightsoftranslation,reprinting,reuseofillustrations,recitation,
broadcasting,reproductiononmicrofilmsorinanyotherphysicalway,andtransmissionorinformation
storageandretrieval,electronicadaptation,computersoftware,orbysimilarordissimilarmethodology
nowknownorhereafterdeveloped.
Theuseofgeneraldescriptivenames,registerednames,trademarks,servicemarks,etc.inthispublication
doesnotimply,evenintheabsenceofaspecificstatement,thatsuchnamesareexemptfromtherelevant
protectivelawsandregulationsandthereforefreeforgeneraluse.
Thepublisher,theauthorsandtheeditorsaresafetoassumethattheadviceandinformationinthisbook
arebelievedtobetrueandaccurateatthedateofpublication.Neitherthepublishernortheauthorsor
theeditorsgiveawarranty,expressorimplied,withrespecttothematerialcontainedhereinorforany
errorsoromissionsthatmayhavebeenmade.Thepublisherremainsneutralwithregardtojurisdictional
claimsinpublishedmapsandinstitutionalaffiliations.
Printedonacid-freepaper
ThisSpringerimprintispublishedbySpringerNature
TheregisteredcompanyisSpringerInternationalPublishingAG
Theregisteredcompanyaddressis:Gewerbestrasse11,6330Cham,Switzerland
Preface
Optimization is a very broad field of research with a wide spectrum of important
applications. Until the 1950s, optimization was understood as a single-objective
optimization, i.e., as the specification and computation of minimum/maximum
of a function of interest taking into account some constraints for the solution.
Suchoptimizationproblemswerethefocusofmathematiciansfromancienttimes.
The earliest methods of calculus were applied for the analysis and solution of
optimization problems immediately following their development. Moreover, these
applications gave way to important results in the basics of natural sciences. The
importance of optimization for the understanding of nature is well formulated by
LeonardEuler:
SincethefabricoftheuniverseismostperfectandtheworkofamostwiseCreator,nothing
atalltakesplaceintheuniverseinwhichsomeruleofmaximumorminimumdoesnot
appear.1
Further developments of optimization theory and computational methods were
successfullyappliednotonlyinnaturalsciencesbutalsoinplanninganddesign.Let
usmentionthatseveralNobelMemorialPrizesinEconomicswereawardedforthe
application of optimization methods to the problems of economics. Nevertheless,
in the middle of the last century, it was understood that the model of single-
objective optimization is not universal. In many problems of planning and design,
a decisionmaker aims to minimize/maximize not a single but several objective
functions.Asanexample,inindustry,whenproducingmetalsheets,theobjectives
are to minimize energy consumption, maximize process speed, and maximize
the strength of the product at the same time. The purpose of multi-objective
1Cum enim Mundi universi fabrica sit perfectissima atque a Creatore sapientissimo absoluta,
nihilomninoinmundocontingit,inquononmaximiminimiveratioquaepiameluceat;quamo-
brem dubium prorsus est nullum, quin omnes Mundi effectus ex causis finalibus ope Methodi
maximorumetminimorumaequefeliciterdeterminariqueant,atqueexipsiscausisefficientibus
(L.Euler,Methodusinveniendilineascurvasmaximiminimiveproprietategaudentes,sivesolutio
problematisisoperimetricilattissimosensuaccepti,LausanneandGeneva,1744).
v
vi Preface
optimizationistogiveanunderstandingofhowtheseobjectivesareconflictingand
to provide the user the possibility to choose an appropriate trade-off between the
objectives.
Asanotherexample,considermulti-objectivepathfindingproblemswhichhave
received a lot of attention in recent years. Routing problems are part of everyday
activity. We move material through transportation networks, and we move huge
amountsofdatathroughtelecommunicationnetworksortheInternet.Manytimes,
we are looking for the shortest path, but in real life, other objectives can be
considered. Suppose we consider the problem to route hazardous materials in a
transportation network. In addition to the minimum distance, we need to have
objectivesonminimizingenvironmentalrisksandriskstohumanpopulations.
When multiple objectives are present, the concept of an optimal solution as
in the single-objective problems does not apply. Naturally, first of all, the well-
known classical single-objective optimization methods were generalized for the
multi-objective case, e.g., methods of multi-objective linear programming and of
multi-objective convex optimization were developed. The single-objective non-
convex optimization problems are known as the most difficult. The difficulties
certainlyincreaseincaseofseveralobjectives.Thegeneralizationofmathematical
methods of single-objective global optimization to multi-objective case and the
development of new methods present a real challenge for researchers. Heuristic
methodsaremorepronetovariousmodifications,andthegeneralizationofheuristic
methods of global optimization for the multi-objective case has been booming.
Meanwhile, many multi-objective optimization problems of engineering can be
solved by software implementing the heuristic methods. Nevertheless, the math-
ematical analysis of non-convex multi-objective optimization problems is urgent
from the point of applications as well as of general global optimization theory.
A subclass of those problems waiting for a more active attention of researchers
is multi-objective optimization of non-convex expensive black box problems; this
book is focused on the theoretically substantiated methods for problems of such a
type.
Acknowledgements WewouldliketothankSpringerfortheirhelpandtheopportunitytopublish
this book. Work of the authors was supported by RSF grant 14-41-00039 (National Research
University Higher School of Economics) and a grant (No. MIP-063/2012) from the Research
CouncilofLithuania.
Gainesville,FL,USA PanosM.Pardalos
Vilnius,Lithuania AntanasŽilinskas
2017 JuliusŽilinskas
Contents
PartI BasicConcepts
1 DefinitionsandExamples.................................................. 3
1.1 Definitions............................................................ 3
1.2 OptimalityConditions ............................................... 5
1.3 IllustrativeExamples................................................. 7
2 Scalarization ................................................................ 13
2.1 GeneralIdea.......................................................... 13
2.2 TchebycheffMethod................................................. 14
2.3 AchievementScalarizationFunction................................ 16
2.4 kth-ObjectiveWeighted-ConstraintProblem........................ 17
2.5 PascolettiandSerafiniScalarization ................................ 18
3 ApproximationandComplexity .......................................... 19
3.1 SomeResultsontheComplexityofMulti-Objective
Optimization.......................................................... 19
3.2 ApproximateRepresentationofParetoSets ........................ 25
3.2.1 The"-ConstraintMethod................................... 25
3.2.2 TheNormalizedNormalConstraintMethod.............. 27
3.2.3 TheMethodofNormalBoundaryIntersection............ 28
3.2.4 TheMethodofkth-ObjectiveWeighted-Constraint....... 29
3.2.5 AnExampleofAdaptiveMethod.......................... 30
4 ABriefReviewofNon-convexSingle-ObjectiveOptimization........ 33
4.1 Introduction........................................................... 33
4.2 LipschitzOptimization............................................... 34
4.3 StatisticalModels-BasedGlobalOptimization ..................... 36
4.3.1 GeneralAssumptions....................................... 36
4.3.2 StatisticalModelsforGlobalOptimization ............... 37
4.3.3 Algorithms.................................................. 40
4.4 BranchandProbabilityBoundMethods ............................ 41
vii
viii Contents
PartII TheoryandAlgorithms
5 Multi-ObjectiveBranchandBound...................................... 45
5.1 BranchandBoundforContinuousOptimizationProblems........ 47
5.2 BranchandBoundforCombinatorialOptimizationProblems..... 50
6 Worst-CaseOptimalAlgorithms.......................................... 57
6.1 OptimalAlgorithmsforLipschitzFunctions ....................... 57
6.1.1 Introduction................................................. 57
6.1.2 MathematicalModel........................................ 58
6.1.3 OptimalPassiveAlgorithm ................................ 61
6.1.4 OptimalSequentialAlgorithm............................. 61
6.1.5 Discussion .................................................. 62
6.2 One-StepOptimalityforBi-objectiveProblems.................... 63
6.2.1 Introduction................................................. 63
6.2.2 BoundsfortheParetoFrontier............................. 64
6.2.3 PropertiesofLipschitzBounds ............................ 65
6.2.4 TheImplementationofOne-StepOptimality ............. 68
6.2.5 NumericalExperiments .................................... 74
6.2.6 Remarks..................................................... 79
6.3 MultidimensionalBi-objectiveLipschitzOptimization............ 80
6.3.1 Introduction................................................. 80
6.3.2 LipschitzBoundfortheParetoFrontier................... 80
6.3.3 PropertiesofLocalLowerLipschitzBound .............. 84
6.3.4 Worst-CaseOptimalBisection............................. 87
6.3.5 TrisectionofaHyper-Rectangle........................... 89
6.3.6 ImplementationoftheAlgorithms......................... 89
6.3.7 NumericalExamples ....................................... 91
6.3.8 Remarks..................................................... 95
7 StatisticalModelsBasedAlgorithms..................................... 97
7.1 Introduction........................................................... 97
7.2 StatisticalModel ..................................................... 98
7.3 Multi-ObjectiveP-Algorithm........................................ 99
7.4 Multi-Objective(cid:2)-Algorithm........................................ 102
7.4.1 ANewApproachtoSingle-ObjectiveOptimization...... 102
7.4.2 TheGeneralizationtotheMulti-ObjectiveCase.......... 104
7.5 ExperimentalAssessment............................................ 107
7.5.1 MethodologicalProblems.................................. 107
7.5.2 TestFunctions .............................................. 108
7.5.3 ExperimentswiththeP-Algorithm ........................ 109
7.5.4 Experimentswiththe(cid:2)-Algorithm........................ 116
7.6 DiscussionandRemarks............................................. 120
8 ProbabilisticBoundsinMulti-ObjectiveOptimization ................ 121
8.1 Introduction........................................................... 121
8.2 StatisticalInferenceAbouttheMinimumofaFunction ........... 122
Contents ix
8.3 ConditionsontheIntersectionofParetoFronts .................... 124
8.4 UpperandLowerEstimatesfortheParetoFront................... 126
8.5 BranchandProbabilityBoundMethods ............................ 127
8.6 Visualization.......................................................... 130
8.7 DiscussionandRemarks............................................. 135
PartIII Applications
9 VisualizationofaSetofParetoOptimalDecisions ..................... 139
9.1 Introduction........................................................... 139
9.2 ADesignProblem.................................................... 139
9.3 VisualizationoftheOptimizationResults........................... 140
9.4 TheAnalysisofExploratoryGuess ................................. 144
9.5 Remarks............................................................... 145
10 Multi-ObjectiveOptimizationAidedVisualizationofBusiness
ProcessDiagrams........................................................... 147
10.1 Introduction........................................................... 147
10.2 VisualizationofSequenceFlow..................................... 148
10.2.1 ABriefOverviewofSingle-ObjectiveAlgorithms
AimedatVisualizationofSequenceFlow................. 149
10.2.2 DescriptionoftheProblem................................. 150
10.2.3 Binary-LinearModel....................................... 151
10.2.4 OptimizationProblemsofInterest......................... 154
10.2.5 OptimizationbyHeuristicMethods ....................... 155
10.2.6 NumericalExperiments .................................... 158
10.2.7 DiscussionandRemarks ................................... 161
10.3 Multi-ObjectiveAllocationofShapes............................... 161
10.3.1 AProblemoftheAllocationofShapes.................... 161
10.3.2 AllocationofShapesbyMulti-ObjectiveOptimization .. 162
10.3.3 BranchandBoundAlgorithmforShapeAllocation...... 164
10.3.4 ComputationalExperiments ............................... 169
References......................................................................... 179
Index............................................................................... 191
Notation and Symbols
Rd d-DimensionalEuclideanspace
A2Rd Feasibleregion
x2Rd Avectorind-dimensionalspace
f.(cid:2)/ Objectivefunction
i
m Numberofobjectivefunctions
f.x/D.f1.x/;:::;fm.x//T Vector-valuedobjectivefunction
P.f/ ThesetofParetooptimalsolutions
O
P.f/ ThesetofParetooptimaldecisions
D
L Lipschitzconstantoff.(cid:2)/
i i
(cid:3) Randomvariableusedtomodelanunknownvalue
x
ofanobjectivefunctionatpointx
(cid:3).x/ Randomfield
(cid:4).x/ Vectorrandomfield
xi
