Numerical Optimization with Computational Errors

308 Pages·2016·2.269 MB·English

by  Alexander J. Zaslavski (auth.)

Springer Optimization and Its Applications 108
Alexander J. Zaslavski
Numerical
Optimization with
Computational
Errors
Springer Optimization and Its Applications
VOLUME108
ManagingEditor
PanosM.Pardalos(UniversityofFlorida)
Editor–CombinatorialOptimization
Ding-ZhuDu(UniversityofTexasatDallas)
AdvisoryBoard
J.Birge(UniversityofChicago)
C.A.Floudas(PrincetonUniversity)
F.Giannessi(UniversityofPisa)
H.D.Sherali(VirginiaPolytechnicandStateUniversity)
T.Terlaky(McMasterUniversity)
Y.Ye(StanfordUniversity)
AimsandScope
Optimization has been expanding in all directions at an astonishing rate
during the last few decades. New algorithmic and theoretical techniques
havebeendeveloped,thediffusionintootherdisciplineshasproceededata
rapidpace,andourknowledgeofallaspectsofthefieldhasgrownevenmore
profound.Atthesametime,oneofthemoststrikingtrendsinoptimization
is the constantly increasing emphasis on the interdisciplinary nature of the
field.Optimizationhasbeenabasictoolinallareasofappliedmathematics,
engineering,medicine,economics,andothersciences.
The series Springer Optimization and Its Applications publishes under-
graduate and graduate textbooks, monographs and state-of-the-art exposi-
tory work that focus on algorithms for solving optimization problems and
alsostudyapplicationsinvolvingsuchproblems.Someofthetopicscovered
include nonlinear optimization (convex and nonconvex), network flow
problems, stochastic optimization, optimal control, discrete optimization,
multi-objectiveprogramming,descriptionofsoftwarepackages,approxima-
tiontechniquesandheuristicapproaches.
Moreinformationaboutthisseriesathttp://www.springer.com/series/7393

Alexander J. Zaslavski
Numerical Optimization
with Computational Errors
123
AlexanderJ.Zaslavski
DepartmentofMathematics
TheTechnion–IsraelInstitute
ofTechnology
Haifa,Israel
ISSN1931-6828 ISSN1931-6836 (electronic)
SpringerOptimizationandItsApplications
ISBN978-3-319-30920-0 ISBN978-3-319-30921-7 (eBook)
DOI10.1007/978-3-319-30921-7
LibraryofCongressControlNumber:2016934410
MathematicsSubjectClassification(2010):47H09,49M30,65K10
©SpringerInternationalPublishingSwitzerland2016
Thisworkissubjecttocopyright.AllrightsarereservedbythePublisher,whetherthewholeorpartof
thematerialisconcerned,specificallytherightsoftranslation,reprinting,reuseofillustrations,recitation,
broadcasting,reproductiononmicrofilmsorinanyotherphysicalway,andtransmissionorinformation
storageandretrieval,electronicadaptation,computersoftware,orbysimilarordissimilarmethodology
nowknownorhereafterdeveloped.
Theuseofgeneraldescriptivenames,registerednames,trademarks,servicemarks,etc.inthispublication
doesnotimply,evenintheabsenceofaspecificstatement,thatsuchnamesareexemptfromtherelevant
protectivelawsandregulationsandthereforefreeforgeneraluse.
Thepublisher,theauthorsandtheeditorsaresafetoassumethattheadviceandinformationinthisbook
arebelievedtobetrueandaccurateatthedateofpublication.Neitherthepublishernortheauthorsor
theeditorsgiveawarranty,expressorimplied,withrespecttothematerialcontainedhereinorforany
errorsoromissionsthatmayhavebeenmade.
Printedonacid-freepaper
ThisSpringerimprintispublishedbySpringerNature
TheregisteredcompanyisSpringerInternationalPublishingAGSwitzerland
Preface
Thebookisdevotedtothestudyofapproximatesolutionsofoptimizationproblems
inthepresenceofcomputationalerrors.Wepresentanumberofresultsonthecon-
vergencebehaviorofalgorithmsinaHilbertspace,whichareknownasimportant
tools for solving optimization problems and variational inequalities. According to
the results known in the literature,these algorithms should converge toa solution.
In this book, we study these algorithms taking into account computational errors
whicharealwayspresentinpractice.Inthiscasetheconvergencetoasolutiondoes
nottakeplace.Weshowthatouralgorithmsgenerateagoodapproximatesolution,
if computational errors are bounded from above by a small positive constant. In
practiceitissufficienttofindagood approximate solutioninsteadofconstructing
a minimizing sequence. On the other hand, in practice computations can induce
numericalerrorsandifoneusesoptimizationmethodstosolveminimizationprob-
lems these methods usually provide only approximate solutions of the problems.
Ourmaingoalis,foraknowncomputationalerror,tofindoutwhatanapproximate
solutioncanbeobtainedandhowmanyiteratesoneneedsforthis.
This monograph contains 16 chapters. Chapter 1 is an introduction. InChap.2,
we study the subgradient projection algorithm for minimization of convex and
nonsmooth functions. The mirror descent algorithm is considered in Chap.3. The
gradient projection algorithm for minimization of convex and smooth functions
is analyzed in Chap.4. In Chap.5, we consider its extension which is used for
solvinglinearinverseproblemsarisinginsignal/imageprocessing.Theconvergence
of Weiszfeld’s method in the presence of computational errors is discussed in
Chap.6.InChap.7,wesolveconstrainedconvexminimizationproblemsusingthe
extragradient method. Chapter 8 is devoted to a generalized projected subgradient
method for minimization of a convex function over a set which is not necessarily
convex. In Chap.9, we study the convergence of a proximal point method in a
Hilbert space under the presence of computational errors. Chapter 10 is devoted
to the local convergence of a proximal point method in a metric space under
the presence of computational errors. In Chap.11, we study the convergence of
a proximal point method to a solution of the inclusion induced by a maximal
monotone operator, under the presence of computational errors. In Chap.12, the
v
vi Preface
convergenceofthesubgradientmethodforsolvingvariationalinequalitiesisproved
under the presence of computational errors. The convergence of the subgradient
methodtoacommonsolutionofafinitefamilyofvariationalinequalitiesandofa
finitefamilyoffixedpointproblems,underthepresenceofcomputationalerrors,is
showninChap.13.InChap.14,westudycontinuoussubgradientmethod.Penalty
methods are studied in Chap.15. Chapter 16 is devoted to Newton’s method. The
resultsofChaps.2–6,14,and16arenew.Theresultsofotherchapterswereobtained
andpublishedduringthelast5years.
Theauthorbelievesthatthisbookwillbeusefulforresearchersinterestedinthe
optimizationtheoryanditsapplications.
RishonLeZion,Israel AlexanderJ.Zaslavski
October19,2015
Contents
1 Introduction................................................................. 1
1.1 SubgradientProjectionMethod .................................... 1
1.2 TheMirrorDescentMethod........................................ 4
1.3 ProximalPointMethod............................................. 6
1.4 VariationalInequalities ............................................. 8
2 SubgradientProjectionAlgorithm ....................................... 11
2.1 Preliminaries ........................................................ 11
2.2 AConvexMinimizationProblem.................................. 13
2.3 TheMainLemma................................................... 17
2.4 ProofofTheorem2.4............................................... 19
2.5 SubgradientAlgorithmonUnboundedSets....................... 20
2.6 ProofofTheorem2.8............................................... 22
2.7 Zero-SumGameswithTwo-Players............................... 25
2.8 ProofofProposition2.9 ............................................ 29
2.9 SubgradientAlgorithmforZero-SumGames..................... 35
2.10 ProofofTheorem2.11.............................................. 39
3 TheMirrorDescentAlgorithm ........................................... 41
3.1 OptimizationonBoundedSets..................................... 41
3.2 TheMainLemma................................................... 44
3.3 ProofofTheorem3.1............................................... 47
3.4 OptimizationonUnboundedSets.................................. 48
3.5 ProofofTheorem3.3............................................... 49
3.6 Zero-SumGames ................................................... 53
4 GradientAlgorithmwithaSmoothObjectiveFunction............... 59
4.1 OptimizationonBoundedSets..................................... 59
4.2 AnAuxiliaryResultandtheProofofProposition4.1 ............ 61
4.3 TheMainLemma................................................... 62
4.4 ProofofTheorem4.2............................................... 66
4.5 OptimizationonUnboundedSets.................................. 68
vii
viii Contents
5 AnExtensionoftheGradientAlgorithm ................................ 73
5.1 PreliminariesandtheMainResult................................. 73
5.2 AuxiliaryResults ................................................... 75
5.3 TheMainLemma................................................... 78
5.4 ProofofTheorem5.1............................................... 82
6 Weiszfeld’sMethod......................................................... 85
6.1 TheDescriptionoftheProblem.................................... 85
6.2 Preliminaries ........................................................ 87
6.3 TheBasicLemma................................................... 91
6.4 TheMainResult .................................................... 94
6.5 ProofofTheorem6.10.............................................. 98
7 TheExtragradientMethodforConvexOptimization.................. 105
7.1 PreliminariesandtheMainResults................................ 105
7.2 AuxiliaryResults ................................................... 109
7.3 ProofofTheorem7.1............................................... 113
7.4 ProofofTheorem7.3............................................... 115
8 AProjectedSubgradientMethodforNonsmoothProblems .......... 119
8.1 PreliminariesandMainResults.................................... 119
8.2 AuxiliaryResults ................................................... 122
8.3 ProofofTheorem8.1............................................... 126
8.4 ProofofTheorem8.2............................................... 131
9 ProximalPointMethodinHilbertSpaces ............................... 137
9.1 PreliminariesandtheMainResults................................ 137
9.2 AuxiliaryResults ................................................... 140
9.3 ProofofTheorem9.1............................................... 144
9.4 ProofofTheorem9.2............................................... 146
10 ProximalPointMethodsinMetricSpaces............................... 149
10.1 PreliminariesandtheMainResults................................ 149
10.2 AuxiliaryResults ................................................... 156
10.3 TheMainLemma................................................... 158
10.4 ProofofTheorem10.1.............................................. 160
10.5 AnAuxiliaryResultforTheorem10.2 ............................ 161
10.6 ProofofTheorem10.2.............................................. 163
10.7 Well-PosedMinimizationProblems ............................... 164
10.8 AnExample......................................................... 167
11 MaximalMonotoneOperatorsandtheProximalPointAlgorithm... 169
11.1 PreliminariesandtheMainResults................................ 169
11.2 AuxiliaryResults ................................................... 173
11.3 ProofofTheorem11.1.............................................. 175
11.4 ProofsofTheorems11.3,11.5,11.6,and11.7.................... 176
Contents ix
12 TheExtragradientMethodforSolvingVariationalInequalities...... 183
12.1 PreliminariesandtheMainResults................................ 183
12.2 AuxiliaryResults ................................................... 189
12.3 ProofofTheorem12.2.............................................. 193
12.4 TheFinite-DimensionalCase ...................................... 196
12.5 AConvergenceResult.............................................. 198
13 ACommonSolutionofaFamilyofVariationalInequalities .......... 205
13.1 PreliminariesandtheMainResult................................. 205
13.2 AuxiliaryResults ................................................... 210
13.3 ProofofTheorem13.1.............................................. 212
13.4 Examples............................................................ 221
14 ContinuousSubgradientMethod......................................... 225
14.1 BochnerIntegrableFunctions...................................... 225
14.2 ConvergenceAnalysisforContinuousSubgradientMethod ..... 226
14.3 AnAuxiliaryResult................................................. 228
14.4 ProofofTheorem14.1.............................................. 229
14.5 ContinuousSubgradientProjectionMethod....................... 231
15 PenaltyMethods............................................................ 239
15.1 AnEstimationofExactPenaltyinConstrainedOptimization.... 239
15.2 ProofofTheorem15.4.............................................. 243
15.3 Infinite-Dimensional Inequality-Constrained
MinimizationProblems............................................. 246
15.4 ProofsofAuxiliaryResults......................................... 253
15.5 ProofofTheorem15.12 ............................................ 255
15.6 ProofofTheorem15.15 ............................................ 258
15.7 AnApplication...................................................... 260
16 Newton’sMethod........................................................... 265
16.1 Pre-differentiableMappings........................................ 265
16.2 ConvergenceofNewton’sMethod................................. 269
16.3 AuxiliaryResults ................................................... 272
16.4 ProofofTheorem16.3.............................................. 274
16.5 Set-ValuedMappings............................................... 279
16.6 AnAuxiliaryResult................................................. 281
16.7 ProofofTheorem16.8.............................................. 285
16.8 Pre-differentiableSet-ValuedMappings........................... 287
16.9 Newton’sMethodforSolvingInclusions.......................... 292
16.10 AuxiliaryResultsforTheorem16.12.............................. 294
16.11 ProofofTheorem16.12 ............................................ 296
References......................................................................... 297
Index............................................................................... 303