Simulated annealing in convex bodies and an

26 Pages·2006·0.32 MB·English

JournalofComputerandSystemSciences72(2006)392–417
www.elsevier.com/locate/jcss
Simulated annealing in convex bodies and an
O∗(n4) volume algorithm
László Lovásza, SantoshVempalab,∗,1
aMicrosoftResearch,OneMicrosoftWay,Redmond,WA98052,USA
bDepartmentofMathematics,MIT,Cambridge,MA02139,USA
Received21April2004;receivedinrevisedform27July2005
Availableonline9November2005
Abstract
We present a new algorithm for computing the volume of a convex body in Rn. The main ingredients of the
algorithm are (i) a “morphing” technique that can be viewed as a variant of simulated annealing and (ii) a new
roundingalgorithmtoputaconvexbodyinnear-isotropicposition.ThecomplexityisO∗(n4),improvingonthe
previousbestalgorithmbyafactorofn.
©2005ElsevierInc.Allrightsreserved.
Keywords:Convexbodies;Volumecomputation;Randomwalks;Simulatedannealing;Logconcavefunctions
1. Introduction
Efficientvolumecomputationinhighdimensionisanimportantquestionboththeoreticallyandprac-
tically.ThefirstpolynomialtimerandomizedalgorithmtocomputethevolumeofaconvexbodyinRn
wasgivenbyDyeretal.intheirpathbreakingpaper[6].Theconvexbodyisspecifiedeitherbyasepa-
rationoracleorbyamembershiporacleandapointinthebody[8].Thisresultisquitesurprising,given
that no deterministic polynomial-time algorithm can approximate the volume to within a factor that is
exponentialinn[7,2].Averyhighpowerofthedimensionn(about23)occurredinthecomplexitybound
of the algorithm, but subsequent improvements [14–17,1,5,11] brought the exponent down to 5. In this
∗
Correspondingauthor.
E-mailaddresses:[email protected](L.Lovász),[email protected](S.Vempala).
1SupportedbyNSFCareerawardCCR-9875024andaSloanfoundationfellowship.
0022-0000/$-seefrontmatter©2005ElsevierInc.Allrightsreserved.
doi:10.1016/j.jcss.2005.08.004
L.Lovász,S.Vempala/JournalofComputerandSystemSciences72(2006)392–417 393
paper,wefurtherimprovetherunningtimetoO∗(n4)(wheretheasteriskindicatesthatthedependence
onerrorparametersandonlogarithmicfactorsinnisnotshown).
The main ingredient of our algorithm is a method that can be viewed as a variation of simulated
annealing. Introduced by Kirkpatrick et al. [12], simulated annealing is a general-purpose randomized
search method for optimization. It does a random walk in the space of possible solutions, gradually
adjustingaparametercalled“temperature”.Athightemperature,therandomwalkconvergesfasttothe
uniformdistributionoverthewholespace;asthetemperaturedrops,thestationarydistributionbecomes
moreandmorebiasedtowardstheoptimalsolutions.Simulatedannealingoftenworkswellinpractice,
butitisnotoriouslydifficulttoobtainanytheoreticalguaranteesforitsperformance.
To explain the connection between volume computation and simulated annealing, let us review the
common structure of previous volume algorithms.All these algorithms reduce volume computation to
samplingfromaconvexbody,usingthe“Multi-phaseMonte-Carlo”technique.Oneconstructsasequence
ofconvexbodiesK ⊆ K ⊆ ··· ⊆ K = K,whereK isabodywhosevolumeiseasilycomputed,and
0 1 m 0
oneestimatestheratiosvol(Ki−1)/vol(Ki)(i = 1,... ,m)bygeneratingsufficientlymanyindependent
uniformly distributed random points in Ki and counting the fraction of them that fall in Ki−1. The
generation of random points in K is done by some version of the Markov chain method (lattice walk,
i
ballwalk,hit-and-run),whosedetailscanbeignoredforthemoment.
Ofcourse,onewouldliketochoosethenumberofphases,m,tobesmall.Anysavinginthenumberof
phasesentersasitssquareintherunningtime:notonlythroughthereducednumberofiterationsbutalso
throughthefactthatwecanallowlargererrorsineachphase,whichmeansasmallernumberofsample
pointsareneeded.
However, reducing the number of phases is constrained by the fact that in order to get a sufficiently
goodestimatefortheratiovol(Ki−1)/vol(Ki),oneneedsaboutmvol(Ki)/vol(Ki−1)randompoints.It
followsthattheratiosvol(Ki)/vol(Ki−1)mustnotbetoolarge;sincethevolumeratiobetweenvol(K)
andvol(K )isn(cid:1)(n) intheworstcaseforanyconceivablechoiceofK ,itfollowsthatmhastobe(cid:1)(n)
0 0
just to keep the ratios vol(Ki)/vol(Ki−1) polynomial size. It turns out that the best choice is to keep
theseratiosbounded;thiscanbeachievede.g.ifK0 = B istheunitballandKi = K ∩(2i/nB)f√ori =
1,2,... ,m = (cid:2)(nlogn). (After appropriate preprocessing, one can assume that B ⊆ K ⊆ O( n)B;
we will discuss such “roundings” in more detail.) Reducing m any further (i.e., o(n)) appears to be a
fundamentalhurdle.
Ontheotherhand,volumecomputationisaspecialcaseofintegration.SincethepaperofApplegate
and Kannan [1], the flexibility obtained by extending the problem to the integration of special kinds
of functions (mostly logconcave) has been exploited in several papers. Mostly integration was used to
dampen boundary effects; we use it in a different way. Instead of a sequence of bodies, we construct a
sequence of functions f0(cid:1)f1(cid:1)···(cid:1)fm that “co(cid:1)nnect” a(cid:1)function f0 whose integral is easy to find to
the characteristic function fm of K. The ratios ( fi−1)/( fi) can be estimated by sampling from the
distribution whose density function is proportional to fi and averaging the function fi−1/fi over the
sample points. (This method was briefly described in [16], but it was considered as a generalization of
thevolumecomputationalgorithmratherthanatoolforimprovement.)
If the functions f are characteristic functions of the convex bodies K , then this is just the standard
i i
algorithm.Thecrucialgaincomesfromthefactthatthenumberofsamplepointsneededineachphaseis
smallerifthefi aresmooth.Weaddanewcoordinatex0 andusefunctionsoftheformf(x) = e−x0/T,
where x0 is the first coordinate of √x (we will come back√to the preprocessing of K that is needed). For
∗ ∗
thischoice,wewillonlyneedO ( n)phases,andO ( n)samplepointsineachphase.Thus,weget
394 L.Lovász,S.Vempala/JournalofComputerandSystemSciences72(2006)392–417
samples from a density proportional to e−x0/T with monotonically increasing values ofT (in simulated
annealing,T,the“temperature”,isdecreased).
Ontwopointsthisnewapproachbringsinnewdifficulties.Thefirstisrelatedtothefactthatwehaveto
samplefromdistributionsthatarenotuniformoverK.Variousmethodsforsamplingconvexbodieshave
beenextendedtologconcavedistributions,andindeedourdensityfunctionsarelogconcave;buttheydo
not satisfy any smoothness conditions, and so we have to use recent results [18,19] that give sampling
algorithmswithO∗(n3)steps(oraclecalls)persamplepoint,withoutanysmoothnessassumption.
Theotherdifficultyisthatthesesamplingalgorithmsneeda“warmstart”,i.e.,theycannotbestarted
fromafixedpointbutfromarandompointthatisalreadyalmostuniformlydistributed,inthesensethat
the ratio of the target density and the starting density is bounded at every point. In the standard version
ofthevolumealgorithm,thiscouldbeguaranteedbyusingthesamplepointsgeneratedinthepreceding
phase as starting points for the new phase. In our case this cannot be done, since the ratio of densities
is not bounded. Instead, we use the hit-and-run random walk which has a much milder dependence on
starting density. In [19], it was shown that the complexity of sampling by this walk depends only the
logarithmoftheL normofthestartingdensity;i.e.,itsufficesthatthisnormispolynomiallybounded.
2
Themainresultofthepapercanbestatedpreciselyasfollows.
Theorem1.1. ThevolumeofaconvexbodyK,givenbyamembershiporacle,andaparameterRsuch
thatB ⊆ K ⊆ RB,canbeapproximatedtowithinarelativeerrorofε withprobability1−(cid:1)using
(cid:2) (cid:3)
n4 n n
O log9 +n4log8 logR = O∗(n4)
ε2 ε(cid:1) (cid:1)
oraclecalls.
Theoracleneededisaweakmembershiporacle[8].ThenumberofarithmeticoperationsisO∗(n6),on
numberswithapolylogarithmicnumberofdigits.Asinallpreviousalgorithms,itisafactorofO∗(n2)
morethantheoraclecomplexity.Inthenextsection,wedescribethevolumealgorithm.Fortheanalysis
(Section 2.4 gives an outline), we will need some tools about logconcavity and√probability (Section 3).
Inthedescriptionofthevolumealgorithm,wewillassumethatB ⊆ K ⊆ O∗( n)B.InSection5,we
showhowtoachievethisbyanalgorithmthat“rounds”agivenconvexbody.
2. Thevolumealgorithm
Inthissection,wedescribethemainvolumealgorithm.Wewillassumethattheconv√exbodyofinterest,
K ⊆ Rn, contains the unit ball B and is contained in the ball DB, where D = O( nln(1/ε)). If this
is not true for the given K, it can be achieved by a pre-processing step (finding and applying a suitable
affinetransformation)whichisdescribedinSection5.Toavoidsometrivialdifficulties,weassumethat
n(cid:2)16.
ThemainpartofthealgorithmconsistsofamodificationofK,calledthe“pencil"construction,described
in Section 2.2, followed by a multi-phase Monte-Carlo estimation.The algorithm uses a subroutine for
convexbodysamplingasablackbox,describednext.
L.Lovász,S.Vempala/JournalofComputerandSystemSciences72(2006)392–417 395
2.1. Sampling
Inouralgorithm,weuseasablackboxasamplingalgorithm(orsamplerforshort),whichsamplesfrom
adistributionsupportedonaconvexbodyK,whosedensityf(x)isproportionaltoagivenexponential
functione−aTx,i.e.,
e−aTx
f(x) = (cid:1) . (1)
e−aTydy
K
Let the corresponding measure be (cid:2) .The algorithm needs a starting point X ∈ K and a bound on the
f
followingnormmeasuringthedistancebetweenthestartingdensity(cid:3)andthetargetdensity(cid:2) :
f
(cid:5) (cid:6) (cid:5) (cid:6)
(cid:4) (cid:4)
2
d(cid:3) d(cid:3) d(cid:3)(X)
(cid:6)(cid:3)/(cid:2)f(cid:6) = d(cid:2) d(cid:3) = d(cid:2) d(cid:2)f = E(cid:3) d(cid:2) (X) .
K f K f f
Convexbodysampler:
• Input:aconvexbodyK ∈ Rn givenbyamembershiporacle,avectora ∈ Rn,astartingpointX ∈ K
drawnfromsomedistribution(cid:3),aboundMontheL normof(cid:3)w.r.t.(cid:2) ,andanaccuracyparameter
2 f
(cid:1) > 0;
• Output:apointY ∈ K drawnfromadistributionthathasvariationdistanceatmost(cid:1)from(cid:2) .
f
A sampler we can use is given in [19], using the hit-and-run algorithm. We make the following as-
sumptionsabouttheinput:
(A1)EverylevelsetLoffcontainsaballwithradius(cid:2) (L)r2.
(cid:1) f
(A2) f(x)|x|2dx = R2.
K
(A3)ThestartingpointXisarandompointfromadistribution(cid:3)whoseL -normwithrespectto(cid:2) is
2 f
atmostM.
A distribution that satisfies (A1) and (A2) is said to be (r,R)-rounded. If r = (cid:1)(1) and R = O∗(1),
thenwesaythatfiswell-rounded.
Under these assumptions, the main result of [19] says that the total variation distance of the output
distributionfrom(cid:2) islessthan(cid:1).Furthermore,thenumberofcallsonthemembershiporacleis
f
(cid:2) (cid:3)
R2 Mn
O n2 ln5 .
r2 (cid:1)2
√
Intheanalysis,wewillshowthat(A1)issatisfiedwithr = 1/10,(A2)issatisfiedwithR = O( nln 1)
ε
and(A3)withM(cid:1)8.Thus,thecomplexityisO(n3ln7 n)perrandompoint.
ε(cid:1)
Forthespecialcasewhena = 0in(1),i.e.,fistheuniformdistribution,wecansimplify(A1)tothe
conditionthatKcontainsaballofradiusr.Inthiscase,thesamplerisabitmoreefficient—thenumber
2Strictlyspeaking,[19]onlyneedsthisforthelevelsetofprobability1/8.
396 L.Lovász,S.Vempala/JournalofComputerandSystemSciences72(2006)392–417
Fig.1.ThepencilconstructionwhenKisapentagon.Thecross-sectionisaballnearthetipandKatthebase.
ofcallstothemembershiporacleis
(cid:2) (cid:3)
R2 M
O n2 ln3 .
r2 (cid:1)
2.2. Thepencilconstruction
LetKbethegivenbodyinRn andε > 0.LetCdenotetheconeinRn+1 definedby
(cid:7) (cid:9)
(cid:8)n
C = x ∈ Rn+1 : x (cid:2)0, x2(cid:1)x2 ,
0 i 0
i=1
wherex = (x ,x ,... ,x )T.WedefineanewconvexbodyK(cid:7) ∈ Rn+1asfollows(recallthatB ⊆ K ⊆
0 1 n
DB):
(cid:10) (cid:11)
K(cid:7) = [0,2D]×K ∩C.
Inotherwords,K(cid:7) isan(n+1)-dimensional“pencil”whosecross-sectionisK,whichissharpenedand
itstipisattheorigin.NotethatbythedefinitionofD,thepartofK(cid:7) inthehalfspacex (cid:2)D isinsideC
0
andsoitisacylinderoverK.Also,sinceKcontainsaunitball,thepartofK(cid:7) inthehalfspacex (cid:1)1isa
0
cone C over the unit ball. See Fig. 1 for an illustration. It is trivial to implement a membership oracle
B
(cid:7)
forK .
ThevolumeofthepencilK(cid:7)isatleasthalfthevolumeofthecylinder[0,2D]×K.Hence,ifweknow
thevolumeofK(cid:7),itiseasytoestimatethevolumeofKbygeneratingO(1/ε2)samplepointsfromthe
uniformdistributionon[0,2D]×K andthencountinghowmanyofthemfallintoK(cid:7).NotethatK(cid:7) is
containedinaballofradius2D.
2.3. Themulti-phaseMonte-Carlo
Nowwedescribethe“morphing”partofthealgorithm.Foreachrealnumbera > 0,let
(cid:4)
Z(a) = e−ax0dx,
K(cid:7)
wherex isthefirstcoordinateofx.Fora(cid:1)ε/D,aneasycomputationshowsthat
0
(1−ε)vol(K(cid:7))(cid:1)Z(a)(cid:1)vol(K(cid:7)),
L.Lovász,S.Vempala/JournalofComputerandSystemSciences72(2006)392–417 397
soitsufficestocomputeZ(a)forsuchana.Ontheotherhand,fora(cid:2)2nthevalueofZ(a)isessentially
thesameastheintegraloverthewholecone,whichiseasytocompute:
(cid:4) (cid:4)
∞
Z(a)(cid:1) e−ax0dx = e−attn(cid:4) dt = n!(cid:4) a−(n+1)
n n
C 0
(where(cid:4) = vol(B))and
n
(cid:4) (cid:4) (cid:4)
1 ∞
Z(a)(cid:2) e−ax0dx = e−attn(cid:4) dt > (1−ε) e−attn(cid:4) dt
n n
CB 0 0
bystandardcomputation(assumingε > (3/4)n).
So,ifweselectasequencea > a > ··· > a forwhicha = 2nanda (cid:1)ε/D,thenwecanestimate
0 1 m 0 m
(cid:7)
vol(K )by
m(cid:12)−1
Z(a ) = Z(a ) Z(ai+1).
m 0
Z(a )
i=0 i
ThealgorithmwillestimateZ(a )byestimatingtheratios
m
R = Z(ai+1). (2)
i
Z(a )
i
We will estimate the R ’s using sampling. Let (cid:2) be the probability distribution over K(cid:7) with density
i i
proportionaltoe−aix0,i.e.,
d(cid:2) (x) e−aix0
i = .
dx Z(a )
i
LetXbearandomsamplepointfrom(cid:2)i,letX0 beitsfirstcoordinateandY = e(ai−ai+1)X0.Itiseasyto
verifythatYhasexpectationR :
(cid:4) i
E(Y)= e(ai−ai+1)x0d(cid:2) (x)
i
(cid:4)K(cid:7)
= e(ai−ai+1)x0 e−aix0 dx
K(cid:7) (cid:4) Z(ai)
= 1 e−ai+1x0dx = Z(ai+1).
Z(ai) K(cid:7) Z(ai)
So,toestimatetheratioR ,wedrawrandomsamplesX1,... ,Xk from(cid:2) ,andcomputetheaverageof
i i
thecorrespondingY’s.
(cid:8)k
Wi = 1 e(ai−ai+1)(Xij)0. (3)
k
j=1
Sample points that are (approximately) from (cid:2) are easy to get: select a random positive real number
0
X from the exponential distribution with density proportional to e−2nx, and a uniform random point
0
(Y ,... ,Y )fromtheunitballB.IfX = (X ,X Y ,... ,X Y ) ∈/ K(cid:7),tryagain;else,returnX.
1 n 0 0 1 0 n
398 L.Lovász,S.Vempala/JournalofComputerandSystemSciences72(2006)392–417
Inordertogetappropriatesamplepointsfrom(cid:2) (i > 0)efficiently(i.e.,satisfyassumptions(A1)and
i
(A2)ofthesampler)√,wehavetomakeasimpleaffinetransformation,namelyascalingalongthex0axis.
Let(cid:5) = max(1,a / n)and
i i
(cid:7)
(cid:5) x ifj = 0,
(T x) = i 0
i j
x otherwise.
j
Thiswillensurethatthedistributionswesamplearewell-rounded.Thealgorithmshownintheboxtakes
as input the dimension n of K, a sampling oracle for (cid:2) (i = 1,... ,m), and an accuracy parameter ε.
i
Forcallingtheoracle,theroundnessparametersandthewarmstartmeasurewillalwaysbeboundedby
(cid:7)
thesamevalues,andsowedonotmentionthembelow.TheoutputZisanestimateofthevolumeofK ,
correcttowithina1± ε factor,withhighprobability.
2
Volumealgorithm:
(cid:10) (cid:11)
√ √ i
V1. Setm = 2(cid:9) nln n(cid:10), k = 512 nln n, (cid:1) = ε2n−10 and a = 2n 1− √1
ε ε2 ε i n
for i = 1,... ,m.
V2. Fori = 1,... ,m,dothefollowing.
• Run the sampler k times for convex body T K(cid:7), with vector (a /(cid:5) ,0,... ,0) (i.e.,
i i i
exponential function e−aix0/(cid:5)i), error parameter (cid:1), and (for i > 0) starting points
T X1 ,...,T Xk .ApplyT−1 totheresultingpointstogetpointsX1,... ,Xk.
i i−1 i i−1 i i i
• Usingthesepoints,compute
(cid:8)k
Wi = 1 e(ai−ai+1)(Xij)0.
k
j=1
V3. Return
Z = n!(cid:4) (2n)−(n+1)W ...W
n 1 m
(cid:7)
astheestimateofthevolumeofK .
2.4. Outlineofanalysis
The analysis of the algorithm will verify the following claims. The first asserts that the estimate
computedbythealgorithmisaccurate,whilethesecondandthirdaddresstheconditionsrequiredbythe
sampler.
1. The variance of the function e(ai−ai+1)x0 relative to the distribution (cid:2) is small enough so that k
i
sample points suffice to estimate its mean (Lemma 4.1) and the dependence between samples is
smallenoughthattheoutputisaccuratewithlargeprobability(Lemma4.2).
L.Lovász,S.Vempala/JournalofComputerandSystemSciences72(2006)392–417 399
2. Randomsamplesfromonephaseprovideagoodstartforthenextphase(Lemma4.4),i.e.,thewarm
startmeasure(L norm)Misboundedand(A3)issatisfied.
2
3. TheconvexbodyTiK(cid:7) andexponentialfunctione−aix0/(cid:5)i satisfy(A1)and(A2)(Lemmas4.5,4.6),
theroundnessassumptionsrequiredbytheconvexbodysampler.
4. Theoverallcomplexityis
√ √
O∗( n)phases ×O∗( n)samplesperphase ×O∗(n3)queriespersample = O∗(n4).
Together,theseclaimsyieldTheorem1.1.Theirproofsneedsometechnicaltools,collectedinthenext
section(theymightbeusefulelsewhere).
Below, we are going to assume that n(cid:2)15 and ε > (3/4)n, which only excludes uninteresting cases
butmakestheformulassimpler.
3. Analysistools
3.1. Logconcavity
Anonnegativefunctionf : Rn → R+ issaidtobelogconcaveifforanyx,y ∈ Rn andany(cid:6) ∈ [0,1],
f((cid:6)x +(1−(cid:6))y)(cid:2)f(x)(cid:6)f(y)1−(cid:6).
ThefollowingtheoremprovedbyDinghas[4],Leindler[13]andPrékopa[21,20]summarizesfundamental
propertiesoflogconcavefunctions.
Theorem3.1. Allmarginalsaswellasthedistributionfunctionofalogconcavefunctionarelogconcave.
Theconvolutionoftwologconcavefunctionsislogconcave.
Thenextlemmaisnewandwillplayakeyroleintheanalysis.
Lemma3.2. LetK ⊆ Rn beaconvexbodyandf : K → R,alogconcavefunction.Fora > 0,define
(cid:4)
Z(a) = f(ax)dx.
K
ThenanZ(a)isalogconcavefunctionofa.
Proof. Let
(cid:7)
1 ift > 0and(1/t)x ∈ K,
G(x,t) =
0 otherwise.
ItiseasytocheckthatG(x,t)islogconcave,andsothefunction
F(x,t) = f(x)G(x,t)
400 L.Lovász,S.Vempala/JournalofComputerandSystemSciences72(2006)392–417
isalsologconcave.ByTheorem3.1,itsmarginalintisalogconcavefunctionoft.Butthismarginalis
just
(cid:4) (cid:4)
f(x)G(x,t)dx = tn f(tx)dx. (cid:3)
Rn K
Thenextlemmaisfrom[18].
Lemma3.3. Let X be a random variable with a logconcave distribution, such that E(X) = X¯ and
E(|X−X¯|2) = (cid:3)2.Then
(cid:13) (cid:14)
P |X−X¯| > t(cid:3) (cid:1)e−t+1.
ThefollowinglemmaisalongthelinesofTheorem4.1in[10]anditsproofisthesame.
Lemma3.4. LetKbeaconvexbodyinRn whosecentroidistheorigin.Foraunitvectorv,let
−a = minvTx, b = maxvTx and (cid:3)2 = E ((vTx)2).
K
x∈K x∈K
Then
(cid:15)
n+2 (cid:16)
(cid:3) (cid:1)a,b(cid:1)(cid:3) n(n+2).
n
3.2. Probability
FortworandomvariablesX,Y,wewillusethefollowingmeasureoftheirindependence:
(cid:17) (cid:17)
(cid:2)(X,Y) = sup(cid:17)P(X ∈ A, Y ∈ B)−P(X ∈ A)P(Y ∈ B)(cid:17),
A,B
where A and B range over measurable subsets of the ranges of X and Y. We say that X and Y are (cid:2)-
independentwhere(cid:2) = (cid:2)(X,Y).Bytheidentity
(cid:17) (cid:17) (cid:17) (cid:17)
(cid:17)P(X ∈ A, Y ∈ B)−P(X ∈ A)P(Y ∈ B)(cid:17) = (cid:17)P(X ∈ A, Y ∈ B)−P(X ∈ A)P(Y ∈ B)(cid:17), (4)
whereAdenotesthecomplementarysetofA,itsufficestoconsidersetsA,B withP(X ∈ A)(cid:2)1/2and
P(Y ∈ B)(cid:2)1/2.
Thefollowingaresomebasicpropertiesof(cid:2).
Lemma3.5. Iffandgaretwomeasurablefunctions,then
(cid:2)(f(X),g(Y))(cid:1)(cid:2)(X,Y).
L.Lovász,S.Vempala/JournalofComputerandSystemSciences72(2006)392–417 401
Proof. Trivialfromtheequation
(cid:17) (cid:17)
(cid:17)P(f(X) ∈ A, g(Y) ∈ B)−P(f(X) ∈ A)P(f(Y) ∈ B)(cid:17)
(cid:17) (cid:17)
= (cid:17)P(X ∈ f−1(A), Y ∈ g−1(B))−P(X ∈ f−1(A))P(Y ∈ g−1(B))(cid:17). (cid:3)
Thenextlemmaisavariationofalemmain[11].
Lemma3.6. LetX,Y berandomvariablessuchthat0(cid:1)X(cid:1)a and0(cid:1)Y(cid:1)b.Then
(cid:17) (cid:17)
(cid:17)E(XY)−E(X)E(Y)(cid:17)(cid:1)ab(cid:2)(X,Y).
Proof. TheLHScanbewrittenas
(cid:17)(cid:4) (cid:4) (cid:17)
(cid:17) a b (cid:17)
(cid:17) (cid:17)
(cid:17) P(X(cid:2)x,Y(cid:2)y)dxdy(cid:17)
0 0
whichisclearlyatmost(cid:2)ab.
(cid:7) (cid:7)
Lemma3.7. LetX,Y,X ,Y berandomvariables,andassumethatthepair(X,Y)isindependentfrom
(cid:7) (cid:7)
thepair(X ,Y ).Then
(cid:2)((X,X(cid:7)),(Y,Y(cid:7)))(cid:1)(cid:2)(X,Y)+(cid:2)(X(cid:7),Y(cid:7)).
Proof. Set(cid:2) = (cid:2)(X,Y)and(cid:2)(cid:7) = (cid:2)(X(cid:7),Y(cid:7)).LetR,R(cid:7),S,S(cid:7) betherangesofX,Y,X(cid:7),Y(cid:7),respectively,
andletA ⊆ R×R(cid:7),B ⊆ S ×S(cid:7) bemeasurablesets.Wewanttoshowthat
(cid:17) (cid:17)
(cid:17)P((X,X(cid:7)) ∈ A, (Y,Y(cid:7)) ∈ B)−P((X,X(cid:7)) ∈ A)P((Y,Y(cid:7)) ∈ B)(cid:17)(cid:1)(cid:2)+(cid:2)(cid:7). (5)
Forr ∈ Rands ∈ S,letA = {r(cid:7) ∈ R(cid:7) : (r,r(cid:7)) ∈ A},B = {s(cid:7) ∈ S(cid:7) : (s,s(cid:7)) ∈ B},f(r) = P(X(cid:7) ∈ A ),
r s r
g(s) = P(Y(cid:7) ∈ B )andh(r,s) = P(X(cid:7) ∈ A ,Y(cid:7) ∈ B ).Then
s r s
P((X,X(cid:7)) ∈ A) = E(f(X)), P((Y,Y(cid:7)) ∈ B) = E(g(Y))
and
P((X,X(cid:7)) ∈ A, (Y,Y(cid:7)) ∈ B) = EX,Y(PX(cid:7),Y(cid:7)(X(cid:7) ∈ AX, Y(cid:7) ∈ BY)) = EX,Y(h(X,Y))
(cid:7) (cid:7)
(hereweusethat(X,Y)isindependentof(X ,Y )).Wecanwritetheleft-handsideof(5)as
E(h(X,Y))−E(f(X))E(g(Y))
(cid:18) (cid:19) (cid:18) (cid:19)
= E(h(X,Y)−f(X)g(Y)) + E(f(X)g(Y))−E(f(X))E(g(Y)) . (6)