Concentration Inequalities for Sums and Martingales

131 Pages·2015·1.763 MB·English

by  Bernard Bercu, Bernard Delyon, Emmanuel Rio (auth.)

SPRINGER BRIEFS IN MATHEMATICS
Bernard Bercu
Bernard Delyon
Emmanuel Rio
Concentration
Inequalities
for Sums and
Martingales
123
SpringerBriefs in Mathematics
SeriesEditors
NicolaBellomo
MicheleBenzi
PalleE.T.Jorgensen
TatsienLi
RoderickMelnik
OtmarScherzer
BenjaminSteinberg
LotharReichel
YuriTschinkel
G.GeorgeYin
PingZhang
SpringerBriefs in Mathematics showcases expositions in all areas of
mathematics and applied mathematics. Manuscripts presenting new results
or a single new result in a classical field, new field, or an emerging topic,
applications, or bridges between new results and already published works,
are encouraged. The series is intended for mathematicians and applied
mathematicians.
Moreinformationaboutthisseriesathttp://www.springer.com/series/10030

Bernard Bercu • Bernard Delyon • Emmanuel Rio
Concentration Inequalities
for Sums and Martingales
123
BernardBercu BernardDelyon
InstitutdeMathe´matiques InstitutdeRechercheMathe´matique
deBordeaux deRennes
Universite´deBordeaux Universite´deRennes
Talence,France Rennes,France
EmmanuelRio
LaboratoiredeMathe´matiques
deVersailles
Universite´deVersailles
St.QuentinenYvelines
Versailles,France
ISSN2191-8198 ISSN2191-8201 (electronic)
SpringerBriefsinMathematics
ISBN978-3-319-22098-7 ISBN978-3-319-22099-4 (eBook)
DOI10.1007/978-3-319-22099-4
LibraryofCongressControlNumber:2015945946
MathematicsSubjectClassification(2010):60-01,60E15,60F10,60G42,60G50
SpringerChamHeidelbergNewYorkDordrechtLondon
©TheAuthors2015
Thisworkissubjecttocopyright.AllrightsarereservedbythePublisher,whetherthewholeorpartof
thematerialisconcerned,specificallytherightsoftranslation,reprinting,reuseofillustrations,recitation,
broadcasting,reproductiononmicrofilmsorinanyotherphysicalway,andtransmissionorinformation
storageandretrieval,electronicadaptation,computersoftware,orbysimilarordissimilarmethodology
nowknownorhereafterdeveloped.
Theuseofgeneraldescriptivenames,registerednames,trademarks,servicemarks,etc.inthispublication
doesnotimply,evenintheabsenceofaspecificstatement,thatsuchnamesareexemptfromtherelevant
protectivelawsandregulationsandthereforefreeforgeneraluse.
Thepublisher,theauthorsandtheeditorsaresafetoassumethattheadviceandinformationinthisbook
arebelievedtobetrueandaccurateatthedateofpublication.Neitherthepublishernortheauthorsor
theeditorsgiveawarranty,expressorimplied,withrespecttothematerialcontainedhereinorforany
errorsoromissionsthatmayhavebeenmade.
Printedonacid-freepaper
SpringerInternationalPublishingAGSwitzerlandispartofSpringerScience+BusinessMedia(www.
springer.com)
InmemoryofourfriendAbderrahmenTouati

Preface
Overthelasttwodecades,therehasbeenarenewedinterestintheareaofconcentra-
tioninequalities.Thestartingpointofthisshortbookwasaprojectonexponential
inequalitiesformartingaleswithaviewtowardapplicationsinprobabilityandstatis-
tics.Duringthepreparationofthisbook,werealizedthattheclassicalexponential
inequalities for sums of independent random variables were not well reported in
the literature. This motivated us to write a chapter entirely devoted to sums of in-
dependent random variables, which includes the classical deviation inequalities of
Bernstein, Bennett, and Hoeffding as well as less-recognized inequalities and new
results. Some of these inequalities are extended to martingales in the third chap-
ter,whichdealswithconcentrationinequalitiesformartingalesandself-normalized
martingales.Weendthisbookwithabriefchapterdevotedtoafewapplicationsin
probability and statistics, which shows the striking efficiency of martingales tech-
niques on some examples. We wish to emphasize that this short book does not
provide a complete overview of martingale exponential inequalities and their ap-
plications. More sophisticated results can be found in the literature. We hope that
researchers interested in concentration inequalities for sums and martingales will
findinthisbookusefultoolsfortheirfutureresearch.
Talence,France BernardBercu
Rennes,France BernardDelyon
Versailles,France EmmanuelRio
June2015
vii

Contents
1 Classicalresults................................................ 1
1.1 Sumsofindependentrandomvariables ........................ 1
1.1.1 Stronglawoflargenumbersforsums ................... 1
1.1.2 Centrallimittheoremforsums ......................... 2
1.1.3 Largedeviations ..................................... 2
1.2 Martingales ............................................... 6
1.2.1 Stronglawoflargenumbersformartingales.............. 8
1.2.2 Centrallimittheoremformartingales ................... 9
References..................................................... 10
2 Concentrationinequalitiesforsums .............................. 11
2.1 Bernstein’sinequalities...................................... 11
2.1.1 One-sidedinequalities ................................ 11
2.1.2 Two-sidedinequalities................................ 17
2.1.3 AboutthesecondterminBernstein’sinequality........... 18
2.2 Hoeffding’sinequality ...................................... 21
2.3 Binomialratefunctions...................................... 26
2.4 Bennett’sinequality ........................................ 31
2.5 SubGaussianinequalities .................................... 36
2.5.1 Randomvariablesboundedfromabove.................. 37
2.5.2 Nonnegativerandomvariables ......................... 40
2.5.3 Symmetricconditionsforboundedrandomvariables ...... 42
2.5.4 Asymmetricconditionsforboundedrandomvariables ..... 43
2.6 Alwaysalittlefurtheronweightedsums ....................... 45
2.7 SumsofGammarandomvariables ............................ 48
2.8 McDiarmid’sinequality ..................................... 52
2.9 ComplementsandExercises.................................. 57
References..................................................... 59
ix