Decay of Correlations in Low Density Parity Check Codes: Low Noise Regime

0.11 MB·English

by  Shrinivas Kudekar

Decay of Correlations in Low Density Parity Check
Codes: Low Noise Regime
Shrinivas Kudekar and Nicolas Macris
Laboratory of theory of communications
School of Computer and Communication Sciences, EPFL, Lausanne.
Email: {shrinivas.kudekar,nicolas.macris}@epfl.ch
Abstract—Considertransmission over abinaryadditivewhite temperature spin glass) and we are able to treat it through
9
gaussian noise channel using a fixed low-density parity check cluster expansions. There exist a host of such expansions[4],
0
code. We consider the posterior measure over the code bits and
butwewishtostressthatthesimplestonesdonotapplytothe
0 the corresponding correlation between two codebits, averaged
2 over the noise realizations. We show that for low enough noise presentsituationforatleasttworeasons.Thefirst,isthatthere
n variance this average correlation decays exponentially fast with existarbitrarilylargeportionsofthedualsystemwhichareina
a thegraphdistancebetweenthecodebits.Oneconsequenceofthis lownoise(orlowtemperature)phasewithpositiveprobability
J resultisthatforlowenoughnoisevariancetheGEXITfunctions (thisisrelatedtotheGriffithsingularityphenomenon[6]).The
(further averaged over a standard code ensemble) of the belief
3 second,isthattheweightsofthedualproblemarenotpositive
propagation and optimal decoders are the same.
2 so that the method in [3] does not work. It turns out that a

clusterexpansionoriginalydevisedbyBerretti[5]isverywell
] I. INTRODUCTION
T suited to overcome all these problems.
I We consider transmission over a binary additive white Our analysis can also be carried through for a class of
s. gaussian noise channel (BIAWGN) using low-density parity other channels including the BSC and BEC, but we do
c check codes (LDPC) and the optimal MAP decoder. We are not give the details here. The case of the BEC is special
[
interestedin the behaviorofthe correlationbetweentwo code because under duality the Gibbs weight remains positive and

1 bits as a function of their graph distance. In [1] we treated the communication problem using LDPC codes on BEC(ǫ)
v this problem, for the regime of high noise, for a special code transforms to a real communication problem using LDGM
0
ensemble containing a sufficiently large fraction of degree codes on the BEC(1−ǫ) [7].
3
one variable nodes. In the present contribution we attack the In the last section we sktech an application of our main
6
3 problem in the low noise regime. result to the MAP-GEXIT function (in other words the first
. The behavior of correlations between relevant degrees of derivativeoftheinput-outputentropywithrespecttothenoise
1
0 freedom is of central interest in the analysis of Gibbs mea- parameter). We prove that in the low noise regime where
9 sures, and various approaches have been developed to tackle theaveragecorrelationdecays(fastenough)theMAP-GEXIT
0 such problems. The Gibbs measures associated with the opti- function can be exactly computed from the density evolution
:
v mal decoder of LDPC codes confrontus with new challenges analysis. These curvesremain non-trivialall the way down to
i which invalidate the direct use of the standard methods. For zero noise as long as there are degree one variable nodes(e.g
X
example it is easy to see that the standard Dobrushin type Poisson LDPC codes). This proves that a non-trivial replica
r
a methods [2] fail due to the presence of hard constraints. In solution is the exact expression for the input-output entropy
the high noise regime we were able to convert the problem of a class of LDPC codes (containing a fraction of degree
(at least in the special case of [1]) to a spin glass containing one variable nodes) on the BIAWGN channel. Previously the
a mixture of soft and hard constaints for which appropriate replica expression was only known to be a one-sided bound
clusterexpansionscanbeapplied.Theseexpansionshavebeen [8], [9] for generalensembles and channels. The equality had
applied to the simpler case of low density generator matrix been obtained previously for some ensembles on the BEC
codes (LDGM) in [3] for the high noise regime, which boils using duality [10] and the interpolation method [11].
down to a high temperature spin glass.
II. DECAY OF CORRELATIONS
The low noise regime which is our interest here is a truly
low temperature spin glass problem for which all the above Let xn be a binary codeword of length n from a fixed
LDPC code with bounded, but otherwise arbitrary variable
methods fail. The general idea of our strategy is to apply a
duality transformation to the LDPC Gibbs measure. It turns and check node degrees. In the sequel we call lmax,rmax the
maximalvariableandcheckdegrees.Thenoisevarianceofthe
outthatthedualproblemdoesnotcorrespondtoawelldefined
communications problem, and in fact it does not even corre- BIAWGN channelis ǫ2 and yn denotesthe receivedmessage.
Assuming without loss of generality that the channel input
spond to a well defined Gibbs measure because the “weight”
is the all zero codeword, the output can be mapped onto the
takespositiveaswellasnegativevalues.Neverthelessthedual
problemhastheflavorofahighnoiseLDGMsystem(orhigh half-log-likelihoodratio li = 12lnppYY||XX((yyii||01)) where pY|X(y|x)
is the channel’s transition matrix. The channel outputs are where the Fourier (or Hadamard) transform is,
i.i.d with distribution p (y|0) which induces a distribution
c(l) = 1 e (l ǫ−Y2)|2X/2ǫ−2. Mapping the codewords xn f(τn)= f(σn)eiπ4 Pnj=1(1−τj)(1−σj)
√2πǫ−2 − − σn X1,+1 n
to spin configurations σn with σi = (−1)xi, the posterior b ∈{− }
measure becomes (for a uniform prior) to the partition function ZP of an LDPC code C. The dual
code C is an LDGM with codewords given by τn where
n m ⊥
1 1
pXn|Yn(xn|yn)= ZP Yi=1eliσicY=12(1+Yi∈cσi) τi =Yc iuc (3)
∈
In this expression c is a product over all the parity check and uc are the m information bits (i and c will always
constraints of the cQode and is a product over variable refer to the variable and check nodes of the original LDPC
i c
nodesattachedtothechecknQode∈c.ThepartitionfunctionZP Tanner graph and c ∈ i means that c is connected to i). A
is simply the normalizing factor straigthforwardapplicationof the Poisson formulathen yields
n m an extended form of the MacWilliams identity,
1
Z = eliσi (1+ σ )
P σn X1,+1 niY=1 cY=12 Yi c i ZP = 1 ePnj=1ljZG (4)
∈{− } ∈ |C⊥|
Theaverageofanarbitraryfunctionf(σn)withrespecttothe
where
abovemeasureis denotedas hf(σn)i where the subscriptP
P n
refers to parity check (later we use various other brackets). ZG = (1+e−2li uc)
This is still a random quantity which depends on the channel
um X1,+1 miY=1 Yc i
output realization. Further averages with respect to the noise ∈{− } ∈
are denoted by E [hf(σn)i ]. Of course it does not make This expression formaly looks like the partition function of
ln P
sense to permute the expectation E and the bracket h−i an LDGM code (hence the subscript G) with “channel log-
ln P
because of the normalizing factor Z in the denominator. likelihoods” gi such that tanhgi = e−2li. This is truly the
P
casefortheBEC(ǫ)wherel =0,+∞andhenceg =+∞,0
Our main result is on the average correlation between any i i
which still correspond to a BEC(1 − ǫ). The logarithm of
two codebits defined by
partition functions is related to the input-output entropy and
CP(i,j)=Eln[|hσiσjiP −hσiiPhσjiP|] one recovers (taking the ǫ derivative) the well known duality
relation between EXIT functions of a code and its dual on
the BEC [7]. For other channels however this is at best a
Theorem 1 (Decay of Correlations): Consider
formal(but useful) analogy since the weights can be negative
transmission over a BIAWGN channel with noise variance ǫ2
or equivalently the g can assume complex values.
using an arbitrary fixed LDPC code. Set k = (l r )1/2. i
max max
We will need a duality formula for the correlations them-
Let dist(i,j) denote the graph distance between the codebits
selves. We introduce a bracket h−i which is not a true
i,j.Thereexiststrictlypositivepurelynumericalconstantsǫ , G
0
probabilistic expectation (but it is linear)
c ,c suchthatforǫ2 ≤ǫ2k 2(lnk) 1 anddist(i,j)>4l
1 2 0 − − max
we have 1 n
CP(i,j)≤c1e−ǫc22kdist(i,j) (1) hf(um)iG = Z f(um) (1+e−2li uc)
G um X1,+1 m iY=1 Yc i
Remark: By graph distance we mean the smallest possible ∈{− } ∈
The denominator may vanish, but it can be shown that when
number of edges on a path connecting i and j.
this happens the numerator also does so, and in a way that
Infactwewillderive(anduseinsectionV)aslightlymore
ensuresthefinitenessoftheratio(thiswillbecomequiteclear
general estimate. Suppose that the bits x are transmitted at
i in all our subsequent calculations). Taking logarithm of (4)
different noise levels ǫ ≤ǫ. Then
i and then the derivative with respect to l we find
i
CP(i,j)≤c1e−c(ǫ12i+ǫ12j)e−ǫc22kdist(i,j) (2) hσ i = 1 − hτiiG (5)
i P
tanh2l sinh2l
where c>0 is a strictly positive number. In particular if bits i i
xi or xj are perfectly received we recover CP(i,j)=0. and differentiating once more with respect to lj, j 6=i
hτ τ i −hτ i hτ i
III. DUALITY FORMULAS hσ σ i −hσ i hσ i = i j G i G j G (6)
i j P i P j P
sinh2l sinh2l
Ageneraltheoryofdualityforcodesongraphscanbefound i j
in [14] and references therein. Here we derive by elementary We stress that in (5), (6), τ and τ are given by products of
i j
means formulasthat are usefulto us. Let C be a binaryparity information bits (3). The left hand side of (5) is obviously
checkcodeandC its dual.We applythePoisson summation bounded. It is less obvious to see this directly on the right
⊥
formula hand side and here we just note that the pole at l = 0 is
i
1
f(σn)= f(τn) harmless since, for li =0, the bracket has all its “weight“ on
|C |
σXn ⊥ τnX⊥ configurations with τi = 1. Similar remarks apply to (6). In
∈C ∈C b
any case, we will beat the poles by using the following trick. ∂Γ∩∂i6=φ and∂Γ∩∂j 6=φ, (iii) thereis a walk connecting
For any 0<s<1 and |x|≤1 we have |x|≤|x|s, thus ∂i and ∂j such that all its variable nodes are in Γ. Finaly,
CP(i,j)≤21−sEln[|hσiσjiP −hσiiPhσjiP|s] ZG(Xˆc)= (1+e−2li uc)
Xuc alYlis.t. Yc i
and using (6) and Cauchy-Schwarz c Xˆc∂i Xˆ=φ ∈
∈ ∩
Using | a |2s ≤ |a |2s for 0 < 2s < 1 and then
C (i,j)≤21 sE[(sinh2l) 2s] (7) i i i i
P − − Cauchy-ScPhwarz, we finPd
×E [|hτ τ i −hτ i hτ i |2s]1/2
ln i j G i G j G 1
C (i,j;s)≤ T (Xˆ)T (Xˆ)
G 1 2
The following bound 2
XXˆ
E[(sinh2l)−2s]≤ c e−c′s(1ǫ−22s) (8) where
|1−2s| T (Xˆ)2 =E |K (Xˆ)|4s (10)
1 ln i,j
ontheprefactorturnsouttobeimportantinouranalysis.Here (cid:2) (cid:3)
and
0<s< 12 and c>0, c′ >0 are purely numerical constants. T (Xˆ)2 =E ZG(Xˆc) 8s (11)
2 ln
Z
IV. PROOF OF MAIN THEOREM (cid:2)(cid:16) G (cid:17) (cid:3)
Bound on T (Xˆ). Trivially bounding the spins in (9) by 1
1
From inequalities (7), (8) of the previous section we see
we deduce (in the first inequality we need 4s< 1 and in the
that it suffices to prove that
second 8s<1)
CG(i,j;s)=Eln[|hτiτjiG−hτiiGhτjiG|2s] T1(Xˆ)2 ≤4|Xˆ| (24sEln[e−8sl]+Eln[e−16sl])|Γ|
Γ coXmpatible
decays. As explained in the introduction, the main tool used
withXˆ
hvaenretagiseaofcdluesatleinrgesxipmaunlstiaonneooufslByewrrietthtith[5e]G(rtihffiatthhsaisngthuelaraidty- ≤4|Xˆ| 2(4s+1)|Γ|e−8s(1ǫ−28s)|Γ|
Γ coXmpatible
phenomenonand at the same time does not use the positivity
withXˆ
of the weights). Here we can only explain the resulting
Nowletussets= 1 andtakeǫ2 ≤(10ln2) 1forsimplicity.
expansion, adapted to our setting, without giving the full 16 −
The bound becomes
derivation (a good starting point is [6]). We have
1 Z (Xˆc) 2 T1(Xˆ)2 ≤4|Xˆ| e−8ǫ12|Γ|
hτiτjiG−hτiiGhτjiG = Ki,j(Xˆ) G Γ coXmpatible
2XXˆ (cid:16) ZG (cid:17) withXˆ
If Γ is compatible with Xˆ we necessarily have |∂Γ|≥|Xˆ|−
where
|∂i|− |∂j| an since |∂Γ| ≤ |Γ|l , we get |Γ| ≥ (|Xˆ|−
max
Ki,j(Xˆ), (τi(1)−τi(2))(τj(1)−τj(2)) Ek 2lmax)/lmax. Also, the maximum number of variable nodes
u(cc1X),Xuˆ(c2)Γ cwoXimthpXaˆtible kY∈Γ wathmicohstha2v|Xeˆ|arnmaxinpteorssseibctlieonchwoiicthesXˆfoirsΓ|X.ˆT|rhmeasxe.rTehmuasrkthseirmepalrye
∈
and T1(Xˆ)2 ≤2(2+rmax)|Xˆ|e−8ǫ12(|Xˆ|−2lmax)/lmax
Ek =τk(1)e−2lk +τk(2)e−2lk +τk(1)τk(2)e−4lk (9) BoundonT2(Xˆ).Theratio(11)isnoteasilyestimateddirectly
because the weights in Z are not positive. However we can
Here u(c1) and u(c2) are two independent copies of the in- usethedualitytransformaGtion(4)togetanewratioofpartition
formation bits (these are also known as real replicas) and functions with positive weights,
τ(α) = u(α). To explain what are Xˆ and Γ we keep
rekferringQtoc∈ckheckksandvariablesin the originalLDPC Tanner ZG(Xˆc) = exp l |C⊥(Xˆc)|ZP(Xˆc)
i
graph language: checks are indexed by c and variables by i. ZG (cid:18) aXllis.t (cid:19) |C⊥| ZP
Given a subset S of variable or ckeck nodes of the Tanner ∂i Xˆ=φ
∩ 6
graph let ∂S be the subset of neighboring nodes. The sum with
over Xˆ is carried over clusters of check nodes such that: (i)
1
Xˆ is”connectedviahyperedges”(thismeansthatXˆ =∂X for Z (Xˆc)= eliσi (1+ σ )
P i
2
some connected subset X of variable nodes; X is connected Xσi aYllis.t cYXˆc iYcand
if any pair of variable nodes can be joined by a path all of ∂i∩Xˆ=φ∂i∩Xˆ=φ ∈ ∂i∈∩Xˆ=φ
whose variable nodes lie in X) and (ii) Xˆ contains both the which is the partition function correspondingto the subgraph
∂i and ∂j. Γ is a set of variable nodes (all distinct). We say (of the full Tanner graph) induced by checks of Xˆc and
that Γ is compatible with Xˆ if: (i) ∂Γ∪∂i∪∂j = Xˆ, (ii) variablenodesis.t∂i∩Xˆ =φ. MoreoverC (Xˆc) is the dual
⊥
of the later code C(Xˆc) definedon the subgraph.By standard where the variable node o is selected uniformly at random
propertiesoftherankofamatrix,therankoftheparitycheck (the result is independent of the node due to symmetry). In
matrixofC(Xˆc),whichisobtainedbyremovingrows(checks) this formula E [hσ i] is the MAP soft-bit estimate.
ln o
and columns (variables) from the parity check matrix of C, is In fact one can verify that the density evolution analysis
smaller than the rank of the parity check matrix of C. Thus is equivalent to performing statistical mechanical sums on a
|C(Xˆc)|≥|C| and |C (Xˆc)|≤|C |. Moreover tree whose leaves are the spins (variable nodes) with free
⊥ ⊥
boundary conditions (channel outputs as initial conditions).
(cid:18)exp li(cid:19)ZP(Xˆc)≤ZP More precisely if we call Nd(o) the neighborhoodof depth d
aXllis.t of o for d even (that is all the nodes of the Tanner graph that
∂i∩Xˆ6=φ are at a distance ≤ d from o) and consider the LDPC Gibbs
To see this one must recognize that the left hand side is the measure h−i restricted to the subgraph N (o), we can
sum of terms of ZP corresponding to σn such that σi = +1 verify by expNlidc(iot)calculation that d
for ∂i∩Xˆ 6=φ (and all terms are ≥0). These remarksimply
for (11) ELDPC,l[tanh(l+∆(d))]=ELDPC,ln[hσoiNd(o)|Nd(o)is a tree]
T2(Xˆ)2 ≤1 Now for d fixed, N (o) is a tree with probability 1−O(γd)
d n
Now we can conclude the proof of theorem 1. From where γ depends only on the maximum node degrees, so
the bounds on (10) and (11) we get for ǫ2 < (l (2 +
max γd
rmax)16ln2)−1 ELDPC,ln[hσoiNd(o)]=ELDPC,l[tanh(l+∆(d))]+O( n ) (13)
CG(i,j;s= 1 )≤ e−32lm1axǫ2(|Xˆ|−2lmax) Thus in view of (12) the theorem will follow if we can show
16
XXˆ that
The clusters Xˆ connect ∂i and ∂j and thus have sizes Eln[hσoiP]=Eln[hσoiNd(o)]+O(e−ξǫd2) (14)
|Xˆ|≥ 12dist(i,j). Moreoverthe numberof clustersof a given with ξ > 0 and O(e−ξǫd2) uniform in n and depending only
size grows at most like (lmaxrmax)|Xˆ|. Working out the final on lmax,rmax. Indeed, if (14) holds, combining with (13) we
bounds,andputtingtheminasymmetricalform,thenetresult get
is that for dist(i,j) > 4l we can find a purely numerical
max
constant ǫ0 such that for ǫ2 <ǫ20k−2(lnk)−1 ELDPC,ln[hσoiP]=ELDPC,l[tanh(l+∆(d))]+O(γd)
n
CG(i,j;s= 116)≤c1e−ǫc22kdist(i,j) +O(e−ξǫd2)
where k = (lmaxrmax)21 and c1 and c2 a strictly positive and the theorem follows by taking first the limit n → +∞
numbers. Using this bound with (7) and (8) concludes the and then d→+∞.
proof of (1) and (2). Formula (14) follows directly from the next two lemmas.
Let C (o) denote the circle of variable nodes at distance =d
V. EXACTNESS OF DENSITYEVOLUTION from do. Call h−i+ the LDPC Gibbs measure associated
In this section we illustrate an application of the theorem to the graph N (oN)d(wo)ith σ = +1 ”boundary condition” for
d j
to the GEXIT functionof standard irregularLDPC ensembles j ∈C (o). First we will show
d
with degrees bounded by l ,r . Let h = 1H(Xn|Yn)
be the input-output entropy.mTaxhemMaxAP-GEXnIT fnunction is in Lemma 1 (Cutting a piece of the Tanner graph): For ǫ2 ≤
general defined as ǫ21k−2(lnk)−1
d(ǫd2)ELDPC[hn] Eln[hσoiP]=Eln[hσoi+Nd(o)]+O(e−ξǫd2)
−
where ξ > 0 and O(e−ξǫd2) depend only on lmax,rmax. In
Theorem 2 (Exactness of Density Evolution): One canfind particular they are independent of n.
a strictlypositivenumberǫ (ingeneralsmaller thantheǫ of
1 0
theorem 1) such that for ǫ2 ≤ǫ2k 2(lnk) 1 The second step is to show that for ǫ small enough the
1 − −
soft estimate of the bit at o is independent from boundary
d 1
lim E [h ]= ( lim E [tanh(l+∆(d))]−1) conditions.
n→∞dǫ−2 LDPC n 2 d→∞ LDPC,l
Lemma 2 (Independence from Boundary Conditions):
where ∆(d) is the soft bit-estimate given by the density
Under the same conditions than in lemma 1
evolution analysis of the BP decoder.
The proof of this theorem rests on the simple formula [12], Eln[hσoiNd(o)]=Eln[hσoi+Nd(o)]+O(e−ξǫd2)
[13] valid for the BIAWGN channel Proof of Lemma 1. We first introduce new interpolating
d 1 Gibbs measures. Label the variable nodes in C (o) in some
E [h ]= (E [hσ i ]−1) (12) d
d(ǫ−2) C n 2 LDPC,ln o P arbitrary order Cd(o) = {1,2,...,N} and assume these bits
aretransmittedthrougha BIAWGNchannelwith noisevector for ǫ ≤ ǫ , is non zero for ǫ ≥ ǫ while it remains
MAP MAP
νN = (ν ,...,ν ) with 0 ≤ ν ≤ ǫ (here ν2 is the noise continuous at ǫ (the curve may have a discontinuity at
1 N k k MAP
variance). Set νj = (0,...,0,ν ,ǫ,...,ǫ) for j = 1,...,N. higher noise value ǫ ). Although in this case the statement of
j c
The interpolating Gibbs measures h−iνPbj are defined on the theorem 2 may be valid for some range of ǫ above ǫMAP, our
full Tanner grabph with noise vectors νj for bits in Cd(0) proofonlyworksonlybelowǫMAP.Thiscanbeexplicitlyseen
aνnNd=no(i0s,e..ǫ.,0fo)r=al0l other bits. A crucibal remark is that for fwrohmichLeimmpmlya2thaantdotuhrepfaroctoEf lonn[lhyσowi+Nordk(os)|iNnda(or)anisgea twreeer]e=th1e
GEXITfunctionvanihes.Ouranalysisisnotpowerfulenough
Eln[hσoiPνN=0]=Eln[hσoi+Nd(o)] (15) tocaptureanyinterestingbehaviorfortheGEXITfunctionfor
ǫ <<ǫ<ǫ .
Proceedingsimilarlyto[3]weapplyiterativelythefundamen- MAP c
Finally, consider the case of ensembles with some fraction
tal theorem of calculus,
of degree one nodes and a GEXIT function that does not
Eln[hσoiP]=Eln[hσoiνPN=0]+XjN=1Z0ǫdνjddνjEln[hσoiνPbj] vnwauintihtiysPhaotailsslsootmhneedwǫecga)ry.eAednodwisetnxraitbmouptǫiloe→nisf0ogr(ivvweainrtihabbpyloeLsnsDiobPdlCyes.aeNndsoiestmceobtnhlteais-t
For the BIAWGN channel we have the remarkable formula here E [hσ i+ |N (o)is a tree] 6= 1 because the tree still
[13] containlsnleaoveNsd((aot) disdtance < d from o) with free boundary
d conditions.In this case theorem2 really capturesa non trivial
E [hσ iνj]=E (hσ σ iνj −hσ iνjhσ iνj)2
d(νj−2) ln o Pb ln(cid:2) o j Pb o Pb j Pb (cid:3) bchehanavnieolsr opfrehveioGusEXreIsTulctsur[v1e0]f,or[1s1m]atlhlaǫt. hItadexbteenednsotbotaoitnheedr
Then using (15) we obtain the sum rule only for the BEC. This also provesthat the replica solution is
E [hσ i ]=E [hσ i+ ] indeed correct for channels other than the BEC.
ln o P ln o Nd(o)
N ǫ dν VII. ACKNOWLEDGMENT
−2 Z ν3jELDPC,ln (hσoσjiνPbj −hσoiPνbjhσjiPνbj)2 S.K acknowledges the support from the Fonds National
Xj=1 0 j (cid:2) (cid:3) Suisse pour la Recherche Scientifique, grant no 200020-
Now we apply the generalized form of theorem 1, namely eq 113412.WewouldliketothankCyrilMe´assonfordiscussions
(2) (with possibly different numerical constants) on duality and Hamed Hassani for a discussion that led to a
simplification of the proof of theorem 2.
Eln[hσoσjiPνbj −hσoiνPbjhσjiνPbj]≤c1e−νcj2e−ǫc22kd
REFERENCES
c
Note that the prefactor e−νj2 is important in order to get [1] S. Kudekar, N. Macris, ”Decay of correlations: an application to low
convergent integrals in the sum rule. For the number of densityparitycheckcodes”,5thInternationalSymposiumonTurboCodes
boundary terms we have N ≤ kd which leads to the result andRelatedTopics,pp.13-18(Lausanne2008)
[2] H.O.Georgii,”Gibbsmeasuresandphasetransitions”,deGruyterStudies
of the lemma for ǫ2 ≤ǫ21k−2(lnk)−1. inMathematics 9(1988).
[3] S.Kudekar,N.Macris,”Proofofreplicaformulasinthehighnoiseregime
Proof of Lemma 2. The proof is similar to that of Lemma 1 forcommunication usingLDGMcodes”,InformationTheoryWorkshop,
with h−i replaced by h−i . pp.416-4120(Porto2008)
P Nd(o) [4] D. C. Brydges, “A short course on cluster expansions”, in les Houches
VI. DISCUSSION summerschool,SessionXLIII,1984(K.OsterwalderandR.Stora,eds).
[5] A. Berretti, ”Some properties of random Ising models”, Journal of
Consider code ensemblessuch that the MAP-GEXIT curve Statistical Physics,vol38pp.483-496(1985)
hasonlyonediscontinuityatǫ andvanishesforǫ<ǫ . [6] J.Fro¨hlich,”Mathematicalaspectsofdisorderedsystems”,inlesHouches
MAP MAP summerschool,SessionXLIII,1984(K.OsterwalderandR.Stora,eds).
Because of the perturbative nature of the cluster expansion
[7] A. Ashikmin, G. Kramer, S. ten Brink, ”Extrinsic information transfer
our estimates for theorem 1 only work much below ǫ . functions: model and erasure channel property” IEEE Trans. Inform.
MAP
What is the exact range of validity for the decay of the Theory,vol50,pp.2657-2673(2004)
[8] A. Montanari, ”Tight bounds for LDPC and LDGM codes under MAP
theorem is an open question. Let now ǫBP be the Belief decoding”, IEEETrans.Inform.Theory,vol51pp.3221-3246(2005)
Propagation threshold. We know that theorem 2 cannot be [9] S. Kudekar, N. Macris, ”Sharp bounds for MAP decoding of general
valid for ǫ <ǫ<ǫ since in this range the BP and MAP irregular LDPCcodes”,ISIT pp.2259-2263, (Seattle 2006)
BP MAP [10] C.Me´asson,A.Montanari,R.Urbanke,”Asymptoticrateversusdesign
estimatesdiffer.InviewofthesumruleintheproofofLemma
rate”, ISIT pp.1541-1545(Nice2007)
1thismeansthatforthisrangethedecayofcorrelations(even [11] S. Kudekar, S. Korada, N. Macris, ”Exact solution for the conditional
if exponential) cannot overcome the exponential growth of entropyofPoissonianLDPCcodesoverthebinaryerasurechannel”,ISIT
pp.1016-1021(Nice2007)
the number of nodes in Cd(o). An interesting question is to [12] N. Macris, ”Griffiths-Kelly-Sherman correlation inequalities: a useful
determineifthesmallestǫ forwhichthishappenshasaclear toolinthetheoryoferrorcorrectingcodes”,IEEETrans.Inform.Theory,
algorithmicsignificancean∗difitisinanywayrelatedtoǫ . vol53pp.664-683(2007)
BP [13] N.Macris,”SharpboundsongeneralizedEXITfunctions”IEEETrans.
Consider now the case of cycle codes, or of codes with
Inform.Theory,vol53pp.2365-2375(2007)
sufficientfractionof degreetwo variablenodes(andnonodes [14] D. G. Forney, “Codes on graphs: normal realizations”, IEEE Trans.
of degree one),such that the GEXIT functionis equalto zero Inform.Theory,vol47pp.520-548(2001)