Analysis of Coupled Scalar Systems by Displacement Convexity Rafah El-Khatib∗, Nicolas Macris∗, Tom Richardson† and Ruediger Urbanke∗ ∗School of Computer and Communication Sciences EPFL, Lausanne, Switzerland Emails: {rafah.el-khatib,nicolas.macris,ruediger.urbanke}@epfl.ch † Qualcomm, USA Email: tomr@qti.qualcomm.com Abstract—Potential functionals have been introduced recently under rather mild assumptions. For this purpose we will use as an important tool for the analysis of coupled scalar systems thepotentialintherepresentationof[1]whichallowstoobtain (e.g. density evolution equations). In this contribution we inves- more transparent, general, and simpler proofs. 7 tigate interesting properties of this potential. Using the tool of 1 displacement convexity we show that, under mild assumptions This manuscript is organized as follows. Section II intro- 0 on the system, the potential functional is displacement convex. ducesthemodelandthevariationalformulation.InSectionIII 2 Furthermore, we give the conditions on the system such that we proverearrangementinequalitiesthatallow us to naturally n the potential is strictly displacement convex in which case the considerthepotentialasafunctionalofcumulativedistribution a minimizer is unique. functions. The potential is shown to be displacement convex J 3 I. INTRODUCTION inSectionIV.Strictdisplacementconvexityandunicityofthe minimizer are proved in Section V. 1 Spatially coupled systems have been used recently in various frameworks, such as compressive sensing, statistical ] II. SETUP AND VARIATIONALFORMULATION T physics, coding, and random constraint satisfaction problems I (see [1] and the references therein for a review of the litera- The natural setting for displacement convexity is the con- . s ture).Theyhavebeenshowntoexhibitexcellentperformance, tinuum case which can be thought of as an approximation of c often optimal, under low complexity message passing algo- the discrete system in the regime of large spatial length and [ rithms. For example,spatially coupledcodesachieve capacity windowsize.Thecontinuumlimithasalreadybeenintroduced 1 under such algorithms [1]. intheliteratureasaconvenientmeanstoanalyzethebehavior v The performance of these systems is assessed by the solu- of the original discrete model [1], [3], [8], [9]. 7 6 tionsof(coupled)DensityEvolution(DE)updateequations.In Consider a spatially coupled system with an averaging 7 general, these equationscan be viewed as the stationary point window w : R → R which is always assumed to be 3 equationsofafunctionalthatistypicallycalled“thepotential”. bounded, non negative, even, integrable and normalized such 0 Ithasalreadybeenrecognizedthatthisvariationalformulation that dxw(x) = 1 (as we will see, sometimes further . R 1 is apowerfultoolto analyzeDE updatesundersuitable initial assumptions will be necessary depending on the statements). 0 conditions[2].Therearevariouspossibleformulationsofthis We dRenote by ⊗ the standard convolution on R and express 7 potentialfunctional[1], [2], [3], [4]. In this paperwe will use the “fixed point DE equations of a scalar continuous system” 1 the representation of [1] for scalar systems. as follows: : v In a previous contribution [5], [6] we showed that the i X potential (in the form [2]) associated to a spatially coupled g(x)=hg((f ⊗w)(x)), (1) r Low-DensityParity-Check(LDPC) codewhose single system f(x)=hf((g⊗w)(x)), (2) a is the (ℓ,r)-regular Gallager ensemble, with transmission over the BEC(ǫ), has a convex structure called “displacement where x ∈ R is the spatial position. We will often use the convexity”. This structure is well known in the theory of shorthand notation fw =f ⊗w and gw =g⊗w; further, we optimal transport [7]. In fact the potential we consider in [5], will often refer to the functionsf, g as profiles and to h , h f g [6] is not convex in the usual sense but it is in the sense of as update functions. displacementconvexity.Thisinitselfisaninterestingproperty. We will also adopt a convenientnormalizationfor all these When displacement convexity is strict one deduces that the functions. As will become clear in the example below it minimum of the potential is unique — assuming it exists — is always possible to adopt this normalization in specific and thus so is the solution of the DE equation. applications. First, we assume that the profiles are bounded. The main purpose of the present note is to prove that a Specifically,f,g :R→[0,1].Next,weassumethattheupdate general class of scalar systems also exhibits the property of functions h and h are non-decreasing bounded functions f g displacementconvexity,andevenstrictdisplacementconvexity h :[0,1]→[0,1] normalized such that h (0)=h (0)=0 f,g f g profiles f and g, h (x) f 0.8 W(f,g)= dx I (x)−f(x)gw(x) (4) f,g RZ n o 0.6 where h−1(x) g g(x) f(x) 0.4 I (x)= duh−1(u)+ dvh−1(v). f,g g f Z0 Z0 In order for the integral over R in (4) to exist we have to 0.2 impose some conditions on the profiles f and g. These will thus be taken in the spaces (here ǫ>0) x 0 0 0.2 0.4 0.6 0.8 S ={f :R→[0,1], (5) f lim x1+ǫf(x)=0, lim x1+ǫ(f(x)−1)=0}, Fig. 1. Anexample ofthe systems weconsider. TheEXIT-like curves are x→−∞ x→+∞ hf (in red) and h−g1 (in blue). The signed area A(hf,hg;1) from (3) is S ={g :R→[0,1], (6) the sum of the light gray areas (positively signed) and the dark gray areas g (negatively signed), anditisequalto0. lim x1+ǫg(x)=0, lim x1+ǫ(g(x)−1)=0}. x→−∞ x→+∞ The fixed point profile solutions of the DE equations (1), (2) and h (1) = h (1) = 1. We will think of them as EXIT- canbeseenastheleftsideofthe(symmetric)decodingwaves. f g like curves(u,h (u)) and (h (v),v) for u,v ∈[0,1] (see the In this paper we assume that these solutions belong to the f g generic plot). spacesS andS .Thisisachievedundersomemildconditions f g Consider the signed area between the two curves, namely on the slopes of the update functions hf and hg at the corner points, and if w decays fast enough. In particular this is true u A(h ,h ;u)= du′(h−1(u′)−h (u′)). (3) for the example of the BEC(ℓ,r) with a finitely supported w, f g g f Z0 wherethelimitingvaluesareapproachedatleastexponentially fast. This is a functional of h , h and a function of u ∈ [0,1]. f g We considerthe case where A(hf,hg;u)>0 for allu∈]0,1[ III. REARRANGEMENTS andA(h ,h ;1)=0.Thisisequivalenttothestrictlypositive f g Displacementconvexityisdefinedonaspaceofprobability gap condition of [1]. In [1] the condition was stated in terms measures. For measures on the real line it is most convenient of the function of u and v to view displacement convexity on a space of cumulative u v φ(h ,h ;u,v)= du′h−1(u′)+ dv′h−1(v′) −uv. distributionfunctions(cdf’s).Inthissectionweusethetoolof f g g f increasing rearrangements to show that such rearrangements Z0 Z0 of f and g can only decrease the potential. theconditionbeingthatφispositivefor(u,v)∈[0,1]2except We first give a brief introduction to the notion of non- at (0,0) and (1,1) where it takes the value 0. A(hf,hg;u) is decreasingrearrangement,see[10].Consideraprofilep:R→ obtained by minimizing φ(h ,h ;u,v) over v. f g [0,1] such that lim p(x) = 0 and lim p(x) = 1. Then, x→−∞ x→+∞ Example Take a spatially coupled LDPC code whose single the increasing rearrangement1 of p is the non-decreasing system is the (ℓ,r)-regular Gallager ensemble, with trans- function p¯ so that this function has the same limits and mission over the BEC(ǫ). Let u (resp. v) be the erasure the mass of each level set is preserved. More formally, let probabilities emitted by check (resp. variable) nodes. Let us represent p in layer cake form as p(x) = p(x)dt = the functions h˜f (resp. h˜g) give the usual erasure probabil- +∞dtχ (x), where χ (x) is the indicator functi0on of the ities emitted on variable (resp. check) node sides. Explicitly, 0 t t R level set E = {x|p(x) > t}. For each value t ∈ [0,+∞[, h˜ (u) = ǫuℓ−1 and h˜ (v) = 1−(1−v)r−1, and the usual R t f g the level set E can be written as the union of a bounded DE equations are v = h˜ (u), u = h˜ (v). We consider the t f g set A and a half line ]a ,+∞[. We define the rearranged set t t ffisupxneeccditaioplnocsianshtesǫua=cnhdǫMthhAPa.taLrAeet(h˜d(ueffiM,An˜hPe,gdv;uMaAMsPA)Ph)be(=uth)e0=.unTh˜ihqeu(eunnoornmu-)at/rlivivzieadl ET¯hten=,]pa¯(tx−)=|At0|+,+∞∞dt[χ¯atn(dx)χ.¯t the indicator function of E¯t. f g f f MAP MAP Proposition 3.1: Assume that the window function w is and hg(v) = h˜g(vMAPv)/uMAP. The coupled DE fixed point symmetric decRreasing2. Let f and g be in Sf and Sg re- equations are spectively, and let f¯ and g¯ be their respective increasing u g(x)=1− 1−fw(x)v r−1, f(x)= gw(x) ℓ−1. rearrangements. Then, W(f,g)≥W(f¯,g¯). MAP MAP It is not diffic(cid:0)ult to check th(cid:1)at the DE equa(cid:0)tions ((cid:1)1)-(2) 1Notethatanincreasingrearrangementisnotnecessarilystrictlyincreasing. 2We say that a function is symmetric decreasing if it is even and non- are stationarypointequationsofa potentialfunctionalofboth increasing onthepositive halfline. To prove Proposition 3.1, we make use of the Riesz rear- Now consider fˆ∗ and gˆ∗ the symmetric decreasing rearrange- rangement inequality in one dimension [11]. ments of fˆ, and gˆ, respectively.Each of the above terms may Let p be a non-negative measurable function on R. Then only decrease upon rearrangements. Indeed each of the first the symmetric decreasingrearrangement p∗ of p is definedas two terms is a functional of a single monotone function and an even function, that is decreasing on [0,+∞[ and such that thus remains unchanged by rearrangement: U (gˆ) = U (gˆ∗) 1 1 the level sets {x|p(x) ≥ t} and {x|p∗(x) ≥ t} have equal and U (fˆ) = U (fˆ∗) (see for example [11] p. 80 (3.3)). 2 2 Lebesgue measure. Note that the decrease on [0,+∞[ is not The term (10) decreases upon rearrangement as a direct necessarily strict. application4 of Lemma 3.2. We thus conclude that Lemma 3.2 (Riesz’s Inequality): Let f , f , and f be any 1 2 3 1 measurablenon-negativefunctionsontherealline,andf∗,f∗, W(fˆ,gˆ)≥ lim W(fˆ∗,gˆ∗). and f∗ be their symmetric decreasing rearrangements.T1hen23, 2R→+∞ 3 To obtain W(f,g) ≥ W(f¯,g¯) it remains to remark that R2dxdyf1(x)f2(x−y)f3(y) 21limR→+∞W(fˆ∗,gˆ∗) = W(f¯,g¯). This is achieved by re- ZZ versing the steps (7)-(9). ≤ dxdyf∗(x)f∗(x−y)f∗(y). From now on we therefore restrict the functional to the 1 2 3 ZZR2 spaces of non-increasing profiles. Proofof Proposition3.1: Consider the expressionof the potential in (4). In order to make use of the Riesz inequality IV. DISPLACEMENT CONVEXITY we first “symmetrize” the profiles f and g, and rewrite the A generic functional F(p) on a space X (of “profiles” functionalin termsof symmetricprofiles. Choose R>0 very say) is said to be convex in the usual sense if for any pair large but fixed. Eventually we will take R → +∞. Denote p ,p ∈ X, and for all λ ∈ [0,1], the inequality F((1 − by fˆthe profile such that fˆ(x) = f(x), x < R and fˆ(x) = 0 1 λ)p +λp ) ≤ (1−λ)F(p )+λF(p ) holds. Displacement fˆ(2R−x), x > R, and by gˆ the function such that gˆ(x) = 0 1 0 1 convexity, on the other hand, is defined as convexity under g(x), xR. an alternative interpolation called displacement interpolation. We now write the potential in (4) in terms of the sym- The right setting for displacement convexity is a space of metrized profiles fând gˆ. We have probabilitymeasures.For measuresover the realline one can R conveniently define the displacement interpolation in terms W(f,g)= lim dx I (x)−f(x)gw(x) f,g of the cdf’s associated to the measures. This is the simplest R→+∞Z−∞ n o setting and the one that we adopt here. R R = lim dxI (x)− dxf(x)gw(x) . (7) Wethinkoftheincreasingprofilesf andg ascdf’sofsome f,g R→+∞(cid:26)Z−∞ Z−∞ (cid:27) underlying measures over the real line. Consider two pairs For the firstterm in the bracketsaboveitis straightforwardto (f ,g ) and (f ,g ), and define two (pushforward) maps T 0 0 1 1 f see that and T as g R 1 dxI (x)= dxI (x). (8) T (x)=f−1(f (x)), T (x)=g−1(g (x)). Z−∞ f,g 2ZR fˆ,gˆ f 1 0 g 1 0 For the second term slight care must be taken because of the Consider the linear interpolation between points on R, convolution. We find x =(1−λ)x+λT (x), x =(1−λ)x+λT (x). f,λ f g,λ g R dx dyf(x)w(x−y)g(y) (9) The displacement interpolants(f ,g ) are defined so that the λ λ Z−∞ ZR following equalities hold for all λ∈[0,1] and x∈R 1 1 = dx dyfˆ(x)w(x−y)gˆ(y)+o( ). 2ZR ZR R2 fλ(xf,λ)=f0(x), gλ(xg,λ)=g0(x). Note that fând gˆ are integrable over R. Then we can write We now state the main result of this section. W(f,g)= 1 lim (U (gˆ)+U (fˆ)+U (fˆ,gˆ)) Proposition 4.1: The potential W(f,g) is displacement 2R→+∞ 1 2 3 convex which means that for all λ∈[0,1] where W(f ,g )≤(1−λ)W(f ,g )+λW(f ,g ). λ λ 0 0 1 1 gˆ(x) U1(gˆ)= dx duh−g1(u), We define the two following quantities ZR Z0 fˆ(x) x x U (fˆ)= dx dvh−1(v), Ω(x)= dzw(z), V(x)= dzΩ(z), (11) 2 f ZR Z0 Z−∞ Z−∞ U (fˆ,gˆ)=− dx dyfˆ(x)w(x−y)gˆ(y). (10) and call V the kernel for reasons which will appear shortly. 3 R R Z Z 4We apply this inequality for f1 = f, f2 = w, f3 = g. As assumed in 3Ifthelefthandsideisinfinitesoistherighthandsideandtheinequality Proposition 3.1 w is symmetric decreasing window so that for us w(x) = issatisfied. w∗(x)forallx∈R. Before proving the proposition let us first note In the last line we used a change of variables u = g (x). 0 Using a similar analysis for the other terms in (13) we find dx(f(x)−f(+∞))gw(x)= dx(fw(x)−f(+∞))g(x) W (f ,g )−W (f ,g ) R R 1 λ λ 1 0 0 Z Z = dx dy(f(y)−f(+∞))w(y−x)g(x) =λ dg0(x)(x−Tg(x))h−g1(g0(x)) ZR ZR ZR =− df(y)V(y−x)dg(x), ZRZR +df0(x)(x−Tf(x))h−f1(f0(x))+dg0(x)(x−Tg(x)) . ! where we have used integration by parts and g(−∞) = f(−∞)=0 for the last step. WethusconcludethatW1(fλ,gλ)islinear,andhenceconvex, Proof of Proposition 4.1: Using the last identity we in λ. rewrite the potential in (4) as follows. WenowconsiderthedoubleintegraltermW2(f,g)in(12). Using again a change of variables write g(x) f(x) W(f,g)= dx duh−1(u)+ dvh−1(v) RZ (Z0 g Z0 f W2(fλ,gλ)=ZZR2dfλ(y)V(y−x)dgλ(x) = df (y)V (1−λ)(y−x) −f(+∞)g(x) + df(y)V(y−x)dg(x). 0 R2 ) R2 ZZ (cid:16) ZZ +λ(T (y)−T (x)) dg (x). (12) f g 0 (cid:17) We now express the potential as the sum W(f,g) = This is convex in λ because the kernel V is (see (11)). W (f,g) + W (f,g), where W (f,g) consists of the last 1 2 2 V. UNICITY OFMINIMIZER double integral in (12). In this section we prove that the potential is strictly dis- We first consider W (f,g) and write 1 placement convex under the strictly positive gap condition. gλ(x) This implies that it admits a unique minimizer. W (f ,g )−W (f ,g )= dx duh−1(u) (13) 1 λ λ 1 0 0 g Under this condition and assuming that w is even and ZR (Z0 regular5, the existence of increasing fixed point solutions was g0(x) fλ(x) f0(x) − duh−1(u)+ dvh−1(v)− dvh−1(v) established in [1]. It was also shown in [1] that existence g f f Z0 Z0 Z0 of such a fixed point implies a positive gap condition6. It − fλ(+∞)gλ(x)+f0(+∞)g0(x) . was shown that if A(hf,hg;u) = 0 for some u ∈]0,1[ then ) there may be an infinite family of fixed point solutions not (cid:16) (cid:17) equivalentundertranslation.Theproofofunicityreliesonthe We remark that potential function formulationso the regularity conditions(5) dx gλ(x)duh−1(u)− g0(x)duh−1(u) and(6)arerequired.Theycanbeshowntobenecessaryunder g g ZR Z0 Z0 ! mild assumptionson the scalar recursionstability of the fixed 1 points (0,0) and (1,1) and on the decay of w. = dx du Θ(g (x)−u)−Θ(g (x)−u) h−1(u), λ 0 g Fromtheresultsinthepreceedingsectionitfollowsthatall ZR Z0 (cid:16) (cid:17) (14) increasing fixed points must have the same potential and this potential is minimal. where Θ is the Heaviside step function. One can check by Let f ,g and f ,g both be non-decreasing fixed points. considering the two cases g (x) > g (x) and g (x) > g (x) 0 0 1 1 λ 0 0 λ We claim that they must be translates of each other, i.e., y− that T (y)is constantdf -almosteverywhere(a.e.)andx−T (x) f 0 g 1 takes the same constant value dg -a.e. Note that one of these du Θ(g (x)−u)−Θ(g (x)−u) =|g (x)−g (x)|. 0 λ 0 λ 0 conditions implies the other since both pairs are fixed points. Z0 (cid:12) (cid:12) This obs(cid:12)ervation allows us to use Fubi(cid:12)ni’s theorem to swap We will show that y−Tf(y) is constantdf0-a.e. The method (cid:12) (cid:12) ofproofistoshowthatif thisisnotthecase thenW (f ,g ) the integrals in (14). We then write (14) as 2 λ λ is strictly convex at λ = 0 which contradicts the minimality 1 of f ,g . duh−1(u) dx Θ(g (x)−u)−Θ(g (x)−u) 1 1 g λ 0 The proof relies on results from [1] that relate the strictly Z0 1 ZR (cid:16) (cid:17) positive gap condition to the positivity of certain integrals of = duh−g1(u) dx Θ(x−gλ−1(u))−Θ(x−g0−1(u)) spatial fixed points, which will be shown to imply the strict Z01 ZR (cid:16) (cid:17) convexityof W2(fλ,gλ) if y−Tf(y) is not constant df0-a.e. = duh−1(u)(g−1(u)−g−1(u)) g 0 λ 5Regularityofwmeansthatitisstrictlypositiveonaninterval(−W,W), Z0 W ≤+∞and0offof[−W,W]. =λ dg0(x)(x−Tg(x))h−g1(g0(x)) 6ThestrictlypositivegapconditionrequiresthatA(hf,hg;u)>0whereas ZR thepositive gapcondition requires onlythatA(hf,hg;u)≥0. Since we have no further need for explicit use of f ,g we x+ ∈ A+, and x− ∈ A−, such that |x − x+| < W and 1 1 f f f f g f will simplify notation and refer to f0,g0 as f,g. |xg −x−f| 0 (by (16)) we have A ∩(z −W,z +W) 6= ∅ and g where the integrals are Lebesgue-Stieltjes integrals. If x′ 0 φ 1 2 for all x1,x2 where f(x1),g(x2) ∈]0,1[. Let the support of T1 ={(x,y):x≥z,y ≤z,x−y 0. Note that implies ξ (w;f,g;x,x) = 0 so dg(A ) > 0 for all x with 1 φ x x+ = minA+ ∩ [z,z + W] so it follows that there exists g(x)∈]0,1[ and, similarly df(Ax)>0 for all x with f(x)∈ f f ]0,1[. These conditions imply f˙w(x) > 0 and g˙w(x) > 0 xg ∈]z−W,z]suchthat0≤x+f −xg x−f −z > denotes differentiation. −W so we obtain |x−f −xg| < W. The argument assuming We express ξφ in a more useful form for our current dfdg(T2)>0 is similar. purpose. We assume that x = x since we need the result Acknowledgments. R. E. thanks Vahid Aref for many inter- 1 2 only for this case. We claim that (15) is equal to esting discussions. df(x)dg(y)1 Ω(−|x−y|). (17) REFERENCES {(x−x1)(y−x1)≤0} ZZ [1] S. Kudekar, T. Richardson, and R. L. Urbanke, “Wave-Like So- To see this note that lutions of General One-Dimensional Spatially Coupled Systems,” http://arxiv.org/abs/1208.5273, 2012. +∞ [2] A.Yedla, Y.-Y.Jian,P.S.Nguyen,andH.D.Pfister,“ASimpleProof dxw(x) df(y)(g(x +)−g(y−x)) 1 of Threshold Saturation for Coupled Scalar Recursions,” 7th Symp. on Z0 Z]x1,x1+x] TurboCodesanditerative Information Processing,pp.51–55,2012. +∞ [3] S.H.Hassani,N.Macris,andR.Urbanke,“CoupledGraphicalModels = dxw(x) df(y) dg(z) and Their Thresholds,” in Proc. Inf. Theory Workshop IEEE (Dublin), Z0 Z]x1,x1+x] Z]y−x,x1] 2010,pp.1–5. +∞ = dxw(x) dg(z)df(y)1 1 [4] S. Kumar, A. J. Young, N. Macris, and H. D. Pfister, “A Proof {0≤y−z≤x} {(z−x1)(y−x1)≤0} of Threshold Saturation for Spatially-Coupled LDPC Codes on BMS Z0 ZZ Channels,” http://arxiv.org/abs/1301.6111, 2013. = dg(z)df(y)1 Ω(−(y−z))1 [5] R. El-Khatib, N. Macris, and R. Urbanke, “Displacement Convexity – {0≤y−z} {(z−x1)(y−x1)≤0} AUsefulFrameworkfortheStudyofSpatially CoupledCodes,”Proc. ZZ Inf.TheoryWorkshopIEEE(Sevilla), pp.1–5,2013. where for the last step we integrate by parts, using [6] ——, “Displacement Convexity – A Useful Framework for the Study d (−Ω(−x)) = w(x). The other term can be handled sim- ofSpatially Coupled Codes,”http://arxiv.org/abs/1304.6026, 2013. dx ilarly and we obtain (17). [7] C.Villani, TopicsinOptimalTransportation. AMS,2003,vol.58. [8] S. H. Hassani, N. Macris, and R. L. Urbanke, “Chains of mean field Let A denote the support of dg and let A denote the g f models,”J.Stat. Mech:TheorandExp,vol.P02011,2012. support of df. Given a real valued s define measures df+ [9] D. L. Donoho, A. Javanmard, and A. Montanari, “Information- as df+(A) = df(A∩{x : x− T (x) ≥ s}) and df− as theoretically optimal compressed sensing via spatial coupling and ap- f df−(A)=df(A∩{x:x−T (x)

Analysis of Coupled Scalar Systems by Displacement Convexity

Similar Analysis of Coupled Scalar Systems by Displacement Convexity