Introduction to Global Optimization Exploiting Space-Filling Curves

135 Pages·2013·2.277 MB·English

by  Yaroslav D. Sergeyev, Roman G. Strongin, Daniela Lera (auth.)

SPRINGER BRIEFS IN OPTIMIZATION
Yaroslav D. Sergeyev
Roman G. Strongin
Daniela Lera
Introduction
to Global
Optimization
Exploiting Space-
Filling Curves
123
SpringerBriefs in Optimization
SeriesEditors
PanosM.Pardalos
Ja´nosD.Pinte´r
StephenRobinson
Tama´sTerlaky
MyT.Thai
SpringerBriefs in Optimization showcases algorithmic and theoretical tech-
niques,casestudies,andapplicationswithinthebroad-basedfieldofoptimization.
Manuscripts related to the ever-growing applications of optimization in applied
mathematics, engineering, medicine, economics, and other applied sciences are
encouraged.
Forfurthervolumes:
http://www.springer.com/series/8918

Yaroslav D. Sergeyev (cid:129) Roman G. Strongin
Daniela Lera
Introduction to Global
Optimization Exploiting
Space-Filling Curves
123
YaroslavD.Sergeyev RomanG.Strongin
Universita`dellaCalabria N.I.LobachevskyUniversity
DepartmentofComputerEngineering, ofNizhniNovgorod
Modeling,ElectronicsandSystems SoftwareDepartment
Rende,Italy NizhniNovgorod,Russia
DanielaLera
UniversityofCagliari
DepartmentofMathematics
andComputerScience
Cagliari,Italy
ISSN2190-8354 ISSN2191-575X(electronic)
ISBN978-1-4614-8041-9 ISBN978-1-4614-8042-6(eBook)
DOI10.1007/978-1-4614-8042-6
SpringerNewYorkHeidelbergDordrechtLondon
LibraryofCongressControlNumber:2013943827
Mathematics Subject Classification (2010): 90C26, 14H50, 68W01, 65K05, 90C56, 90C30, 68U99,
65Y99
©YaroslavD.Sergeyev,RomanG.Strongin,DanielaLera2013
Thisworkissubjecttocopyright.AllrightsarereservedbythePublisher,whetherthewholeorpartof
thematerialisconcerned,specificallytherightsoftranslation,reprinting,reuseofillustrations,recitation,
broadcasting,reproductiononmicrofilmsorinanyotherphysicalway,andtransmissionorinformation
storageandretrieval,electronicadaptation,computersoftware,orbysimilarordissimilarmethodology
nowknownorhereafterdeveloped.Exemptedfromthislegalreservationarebriefexcerptsinconnection
with reviews or scholarly analysis or material supplied specifically for the purpose of being entered
and executed on a computer system, for exclusive use by the purchaser of the work. Duplication of
this publication or parts thereof is permitted only under the provisions of the Copyright Law of the
Publisher’slocation,initscurrentversion,andpermissionforusemustalwaysbeobtainedfromSpringer.
PermissionsforusemaybeobtainedthroughRightsLinkattheCopyrightClearanceCenter.Violations
areliabletoprosecutionundertherespectiveCopyrightLaw.
Theuseofgeneraldescriptivenames,registerednames,trademarks,servicemarks,etc.inthispublication
doesnotimply,evenintheabsenceofaspecificstatement,thatsuchnamesareexemptfromtherelevant
protectivelawsandregulationsandthereforefreeforgeneraluse.
While the advice and information in this book are believed to be true and accurate at the date of
publication,neithertheauthorsnortheeditorsnorthepublishercanacceptanylegalresponsibilityfor
anyerrorsoromissionsthatmaybemade.Thepublishermakesnowarranty,expressorimplied,with
respecttothematerialcontainedherein.
Printedonacid-freepaper
SpringerispartofSpringerScience+BusinessMedia(www.springer.com)
Preface
BeforeIspeak,Ihavesomethingimportanttosay.
GrouchoMarx
Intheliteraturethereexistalotoftraditionallocalsearchtechniquesthathavebeen
designedforproblemswheretheobjectivefunctionF(y),y∈D⊂RN,hasonlyone
optimumandastronga prioriinformationisknownaboutF(y)(forinstance,itis
supposedthat F(y) is convexand differentiable).In such cases it is used to speak
aboutlocaloptimizationproblems.However,inpracticetheobjectsandsystemsto
beoptimizedarefrequentlysuchthattherespectiveobjectivefunctionF(y)doesnot
satisfy these strongsuppositions.In particular,F(y) can be multiextremalwith an
unknownnumberoflocalextrema,non-differentiable,eachfunctionevaluationcan
beaverytime-consumingoperation(fromminutestohoursforjustoneevaluation
ofF(y)onthefastestexistingcomputers),andnothingisknownabouttheinternal
structure of F(y) butits continuity.Very often when it is requiredto find the best
among all the existing locally optimal solutions, in the literature problemsof this
kind are called black-box global optimization problems and exactly this kind of
problemsandmethodsfortheirsolvingareconsideredinthisbook.
TheabsenceofastronginformationaboutF(y)(i.e.,convexity,differentiability,
etc.) doesnotallow one to use traditionallocalsearch techniquesthatrequirethis
kind of information and the necessity to develop algorithmsof a new type arises.
In addition, an obvious extra difficulty in using local search algorithms consists
of the presence of several local solutions. When one needs to approximate the
globalsolution(i.e.,thebestamongthelocalones),somethingmoreisrequiredin
comparisonwithlocaloptimizationproceduresthatleadtoalocaloptimumwithout
discussingthemainissueofglobaloptimization:whetherthefoundsolutionisthe
globaloneweareinterestedinornot.
Thus, numerical algorithms for solving multidimensional global optimization
problemsare the main topic of this book and an importantpartof the lives of the
authorswhohavededicatedseveraldecadesoftheircareerstoglobaloptimization.
Results of their research in this direction have been presented as plenary lectures
v
vi Preface
atdozensofinternationalcongresses.Togetherwiththeircollaboratorstheauthors
havepublishedmorethanahundredofresearchpapersandseveralmonographsin
EnglishandRussiansince1970softhetwentiethcentury.Amongthesepublications
thefollowingthreevolumes[117,132,139]canbespeciallymentioned:
1. Strongin, R.G.: Numerical Methods in Multi-ExtremalProblems: Information-
StatisticalAlgorithms.Nauka,Moscow(1978),inRussian
2. Strongin, R.G., Sergeyev, Ya.D.: Global Optimization and Non-Convex Con-
straints: Sequentialand ParallelAlgorithms.Kluwer AcademicPublishers, DD
(2000)
3. Sergeyev, Ya.D., Kvasov, D.E.: Diagonal Global Optimization Methods. Fiz-
MatLit,Moscow(2008),inRussian
Eachofthese volumeswasin somesensespecialatthe timeofitsappearance:
themonographof1978wasoneofthefirstbooksintheworldentirelydedicatedto
globaloptimization;thesecondmonographforthefirsttimehaspresentedresultsof
theauthorsinEnglishinacomprehensiveformgivingaspecialemphasistoparallel
computations—apeculiaritythatwasextremelyinnovativeatthattime;finally,the
monographof2008wasoneofthefirstbooksdedicatedtoglobaloptimizationand
publishedinRussian sinceeventsoccurredintheSovietUnionin1990sfilling so
inthegapinpublicationsinthisdirectionthatwas15–20yearslong.
ThedecisiontowritethepresentBriefhasbeenmadeduetothefollowingtwo
reasons. First, as it becomes clear from the format of this publication (Springer
Brief) and the title of the book—Introduction to Global Optimization Exploiting
Space-FillingCurves—theauthorswishedtogiveabriefintroductiontothesubject.
In fact, the monograph [139] of 2000 has been also dedicated to space-filling
curves and global optimization. However, it considers a variety of topics and is
verydetailed(itconsistsof728pages).Second,itismorethan10yearssince the
monograph[139]hasbeenpublishedin2000andtheauthorswishedtopresentnew
resultsanddevelopmentsmadeinthisdirectioninthefield.
The present book introduces quite an unusual combination of such a practical
field as global optimization with one of the examples per eccellenza of pure
mathematics—space-filling curves. The reason for such a combination is the
following.The curveshave been first introducedby Giuseppe Peanoin 1890who
hasprovedthattheyfillinahypercube[a,b]⊂RN,i.e.,theypassthrougheverypoint
of[a,b],andthisgaverisetothetermspace-fillingcurves.Then,inthesecondhalf
of the twentiethcenturyithas beenindependentlyshownin the SovietUnionand
theUSA(see[9,132,139])that,byusingspace-fillingcurves,themultidimensional
global minimization problem over the hypercube [a,b] can be turned into a one-
dimensionalproblemgivingsoanumberofnewexcitingpossibilitiestoattackhard
multidimensionalproblemsusingsuchareduction.
The book proposes a number of algorithms using space-filling curves for
solvingthecoreglobaloptimizationproblem—minimizationofamultidimensional,
multiextremal, non-differentiableLipschitz (with an unknown Lipschitz constant)
function F(y) over a hypercube [a,b]⊂RN. A special attention is dedicated both
to techniques allowing one to adaptively estimate the Lipschitz constant during
Preface vii
the optimization process and to strategies leading to a substantial acceleration
of the global search. It should be mentioned that there already exist a lot of
generalizations of the ideas presented here in several directions: algorithms that
use new efficient partition techniques and work with discontinuousfunctions and
functions having Lipschitz first derivatives; algorithms for solving multicriteria
problems and problems with multiextremal non-differentiable partially defined
constrains; algorithms for finding the minimal root of equations (and sets of
equations)having a multiextremal(and possibly non-differentiable)left-handpart
overaninterval;parallelnon-redundantalgorithmsforLipschitzglobaloptimization
problemsandproblemswithLipschitzfirstderivatives,etc.Duetotheformatofthis
volume(SpringerBrief)thesegeneralizationsarenotconsideredhere.However,in
ordertoguidethereaderinpossiblefutureinvestigations,referencestoanumberof
themwerecollectedandsystematized(seep.117).
In conclusion to this preface the authors would like to thank the institutions
they work at: University of Calabria, Italy; N.I. Lobachevsky State University of
Nizhni Novgorod, Russia; University of Cagliari, Italy; and the Institute of High
PerformanceComputingandNetworkingoftheNationalResearchCouncilofItaly.
During the recent years the research of the authors has been supported by Italian
and Russian Ministries of University, Education and Science and by the Italian
NationalInstitute ofHigh Mathematics“F. Severi.”Actuallyresearchactivitiesof
the authors are partially supported by the Ministry of Education and Science of
Russian Federation,project 14.B37.21.0878as well as by the grant11-01-00682-
a of the Russian Foundation for Fundamental Research and by the international
program“Italian-RussianUniversity.”
The authors are very grateful to the following friends and colleagues for their
inestimable help and useful discussions: K. Barkalov, M. Gaviano, V. Gergel,
S. Gorodetskiy, V. Grishagin, and D. Kvasov. The authors thank Prof. Panos
Pardalos for his continuous support that they do appreciate. The authors express
theirgratitudetoSpringer’spublishingeditorRaziaAmsadforguidingthemduring
thepublicationprocess.
Finally, the authors cordially thank their families for their love and continuous
supportduringthepreparationofthisbook.
Rende,Italy YaroslavD.Sergeyev
NizhniNovgorod,Russia RomanG.Strongin
Cagliari,Italy DanielaLera

Contents
1 Introduction .................................................................. 1
1.1 ExamplesofSpace-FillingCurves..................................... 1
1.2 StatementoftheGlobalOptimizationProblem ....................... 6
2 ApproximationstoPeanoCurves:AlgorithmsandSoftware........... 9
2.1 Space-FillingCurvesandReductionofDimensionality.............. 9
2.2 ApproximationstoPeanoCurves ...................................... 13
2.2.1 PartitionsandNumerations..................................... 13
2.2.2 TypesofApproximationsandTheirAnalysis................. 28
2.3 StandardRoutines for ComputingApproximations
toPeanoCurves......................................................... 37
3 GlobalOptimizationAlgorithmsUsingCurvestoReduce
DimensionalityoftheProblem ............................................. 47
3.1 Introduction ............................................................. 47
3.2 One-DimensionalInformationandGeometricMethods
inEuclideanMetrics.................................................... 50
3.2.1 ConvergenceConditionsandNumericalExamples........... 54
3.2.2 Relationship Between the Information
andtheGeometricApproaches ................................ 59
3.3 One-DimensionalGeometricMethodsinHo¨lderianMetrics......... 60
3.3.1 Algorithms...................................................... 61
3.3.2 ConvergencePropertiesandNumericalExperiments......... 65
3.4 AMultidimensionalInformationMethod.............................. 73
3.5 AMultidimensionalGeometricMethod............................... 81
4 IdeasforAcceleration ....................................................... 91
4.1 Introduction ............................................................. 91
4.2 LocalTuningandLocalImprovementinOneDimension............ 93
ix