The Reliability Function for the Additive White Gaussian Noise Channel at Rates above the Capacity

0.17 MB·

by  Yasutada Oohama

1
The Reliability Function for the Additive White
Gaussian Noise Channel at Rates above the Capacity
Yasutada Oohama
Abstract—WeconsidertheadditivewhiteGaussiannoisechan- II. THE CAPACITY OF THEADDITIVE WHITEGAUSSIAN
nels.Weprovethattheerrorprobabilityofdecodingtendstoone NOISE CHANNELS
exponentiallyforratesabovethecapacityandderivetheoptimal
Let and be real lines and let X , and Y
exponent function. We shall demonstrate that the information X Y ∈ X ∈ Y
be real valued random variables. The set corresponds to a
spectrumapproach isquiteusefulforinvestigatingthisproblem. X
channel input and the set corresponds to a channel output.
7 Y
We write a conditional probability density function(p.d.f.) of
1
Keywords—AdditivewhiteGaussiannoisechannels,Strongcon- Y on given X as
0
verse theorem, Information spectrum approach
2
W = W(y x) .
{ | }(x,y)∈X×Y
b
e I. INTRODUCTION LetN beaGussianrandomvariablewithmean0andvariance
F It is well known that discrete memoryless channels(DMCs) σ2. The additive Gaussian noise channel is defined by
6 have a property that the error probability of decoding goes Y =X +N, X N,
to one as the block length n of transmitted codes tends to ⊥
] infinity at rates above the channel capacity. This property is where X N stands for that X and N are independent. In
T called the strong converse property. In this paper we study this case W⊥ = W(y x) = pN(y x) is a probability density
I thestrongconversepropertyforadditivewhiteGaussiannoise function(p.d.f.)given|by −
.
s channels(AWGNs). Han [1] proved that we have the strong
[c converse property for AWGNs. In [1], Han introduced a W(y|x)=pN(y−x)= √21πσ2e−(y−σ2x)2.
new method to study several coding problems of information

3 theory.Thismethodiscalledtheinformationspectrummethod. LetXn =(X ,X , ,X )bearandomvectortakingvalues
1 2 n
v Based on the information spectrum method, Han [1] gave a in n. We write a·n··element of n as xn = x x x .
1 2 n
7 X X ···
simple proof of the strong property for AWGNs. Similar notations are adopted for other random variables. Let
5
In this paper for AWGNs we study an asymptotic behavior Yn n bearandomvariableobtainedasthechanneloutput
3 forthecorrectprobabilityofdecodingtovanishatratesabove by c∈onYnecting Xn to the input of channel. The additive white
6
the capacity. To our knowledge we have had no work on Gaussian noise(AWGN) channel we study in this paper is
0
. this subject. In the case of DMCs, Arimoto [2] proved that defined by
1
the correct probability of decoding vanishes exponentially at
0 Y =X +N ,X N , for t=1,2, ,n, (1)
7 ratesabovethe capacityand derivean explicitlower boundof t t t t ⊥ t ···
1 this exponent. Dueck and Ko¨rner [3] determined the optimal where Nt, t = 1,2 ,n are independent Gussian random
: exponent. The equivalence of the bound of Arimoto [2] to variables having the··s·ame distribution as N. We write a
v the optimal bound of Dueck and Ko¨rner [3] was proved conditional probability p.d.f. of Yn on given Xn as
i by Oohama [4]. Derivations of the optimal exponent using
X
r informationspectrummethodwasinvestigatedbyNagaoka[5], Wn ={Wn(yn|xn)}(xn,yn)∈Xn×Yn.
a Hayashi and Nagaoka [6], and Oohama [7]. By the definition (1) of the AWGN channel, we have
In this paper we determine the optimal exponent function
on the correct probability of decoding at rates above capacity n
forAWGNs.Toobtainthisresultweusealowerboundofthe Wn(yn xn)= pN(yt xt).
| −
exponent function derived by Oohama [7]. This lower bound t=1
Y
is not computablesince it has an expression of the variational TheAWGNchannelisspecifiedwithσ2.LetK beuniformly
n
problem. We solve this problem by using its min-max struc- distributedrandomvariablestakingvaluesinmessagesets .
n
ture, obtaining an explicit formula of this lower bound. This The random variable K is a message sent to the receiverK. A
n
formula coincides with an upper bound obtained by another sender transforms K into a transmitted sequence Xn using
n
argument, thereby establishing the optimal exponent. Those an encoder function and sends it to the receiver. In this paper
arguments are quite simple and elementary. we assume that the encoder function ϕ(n) is a deterministic
encoder. In this case, ϕ(n) is a one-to-one mapping from
n
Y. Oohamais withDept. ofCommunication Engineering andInformatics, into n. The decoding function at the receiver is denotedKby
University of Electro-Communications, 1-5-1 Chofugaoka Chofu-shi, Tokyo ψ(n).XThis function is formally defined by ψ(n) : n .
182-8585,Japan. n
Y → K
Let c : [0, ) be a cost function. The average cost on
X → ∞
2
output of ϕ(n) must not exceed Γ. This condition is given by if for any δ > 0, there exist a positive integer n = n (ε,δ)
0 0
ϕ(n)(K ) (n), where and a sequence of pairs (ϕ(n),ψ(n)):ϕ(n)(K ) (n) ∞
n ∈SΓ { n ∈SΓ }n=1
such that for any n n (ε,δ),
SΓ(n) =△ xn ∈Xn : n1 n c(xt)≤Γ . P(n)(ϕ(n),ψ(≥n) W0 ) ε, 1 log R δ. (3)
(cid:26) Xt=1 (cid:27) e,m | ≤ n |Kn|≥ −
Weconsiderthecasewheretheinputconstrainc(X)isapower The supremumofallε-achievableratesunderΓ is denotedby
constraint give by c(x) = x2. The average error probabilities C (ε,Γσ2). We set
m,AWGN
of decoding at the receiver is defined by |
C (Γσ2)= inf C (ε,Γσ2)
P(n) =P(n)(ϕ(n),ψ(n) W)=△ Pr ψ(n)(Yn)=K m,AWGN | ε∈(0,1) m,AWGN |
e e | { 6 n}
=1 Pr ψ(n)(Yn)=K . which is called the maximum capacity of the AWGN. Let
n d
− { } G
be a set of all d dimensional Gaussian distributions. Set
For k , set (k) =△ yn : ψ(n)(yn) = k . The families
∈ Kn D { } 1 Γ
doefcsoedtsin{gDr(ekg)io}nk∈,KPne(ni)sccaanllebdetwheritdteencoadsing regions. Using the C(Γ|σ2)= EppXXmX∈aG2x1≤:ΓI(pX,W)= 2log(cid:18)1+ σ2(cid:19). (4)
P(n) =P(n)(ϕ(n),ψ(n) W)
e e | where I(pX,W) stands for a mutual information between X
= 1 dynWn yn ϕ(n)(k) and Y when input distribution of X is pX. The following is
|Kn|kX∈KnZyn∈/D(k) (cid:16) (cid:12) (cid:17) a wTehlelokrenmow1n:rFesourlta.ny AWGN, we have
n (cid:12)
1 (cid:12)
= dyn W (yt xt(k)), C (Γσ2)=C (Γσ2)=C(Γσ2).
|Kn|kX∈KnZyn∈/D(k) tY=1 | m,AWGN | AWGN | |
Han [1] established the strong converse theorem for
where x (k) = [ϕ(n)(k)] , t = 1,2, ,n are the t-th
t t AWGNs. His result is as follows.
componentsofxn =xn(k)=ϕ(n)(k)and·|·K·n|isacardinality Theorem 2 (Han [1]): If R>C(Γσ2), we have
of the set . Set |
n
K
lim P(n)(ϕ(n),ψ(n) W)=1
P(n) =P(n)(ϕ(n),ψ(n) W)=△ 1 P(n)(ϕ(n),ψ(n) W) n→∞ e |
c c | − e |
The quantity P(cn) is called the average correct probability of for any {(ϕ(n),ψ(n)):ϕ(n)(Kn)∈SΓ(n)}∞n=1 satisfying
decoding. This quantity has the following form
1
liminfM R.
P(n) =P(n)(ϕ(n),ψ(n) W) n n→∞ n ≥
c c |
= 1 dynWn yn ϕ(n)(k) . The following corollary immediately follows from this the-
orem.
|Kn|kX∈KnZyn∈D(k) (cid:16) (cid:12) (cid:17) Corollary 1: For each fixed ε (0,1) and any AWGN
(cid:12)
For given ε (0,1), R is ε-achievable u(cid:12)nder Γ if for any specified with σ2, we have ∈
∈
δ > 0, there exist a positive integer n = n (ε,δ) and a
sequence of pairs {(ϕ(n),ψ(n)) : ϕ(n)(K0n) ∈ SΓ0(n)}∞n=1 such Cm,AWGN(ε,Γ|σ2)=CAWGN(ε,Γ|σ2)=C(Γ|σ2).
that for any n n (ε,δ),
≥ 0 To examine an asymptotic behavior of Pc(n)(ϕ(n),ψ(n) W)
1 forlargenatR>C(Γσ2),wedefinethefollowingquanti|ties:
P(en)(ϕ(n),ψ(n)|W)≤ε, nlog|Kn|≥R−δ. (2) |
G(n)(R,Γσ2)
The supremum of all ε-achievable R under Γ is denoted by |
CAWGN(ε,Γσ2). We set =△ min 1 logP(n)(ϕ(n),ψ(n) W),
CA|WGN(Γ|σ2)=△ ε∈i(n0f,1)CAWGN(ε,Γ|σ2), (ϕ1(/(nϕn)(()nKl)on,gψ)M∈(nSn)Γ()≥n:R),(cid:18)−n(cid:19) c |
which is called the channel capacity. The maximum error G∗(R,Γσ2)=△ lim G(n)(R,Γσ2).
probability of decoding is defined by as follows: | n→∞ |
On the above exponent functions, we have the following
P(n) =P(n)(ϕ(n),ψ(n) W)
e,m e,m | property.
=△ maxPr ψ(n)(Yn)=k K =k . Property 1:
n
k∈Kn { 6 | } a) By definition we have that for each fixed n 1,
Based on this quantity, we define the maximum capacity as G(n)(R,Γσ2) is a monotone increasing functio≥n of
follows. For a given ε (0,1), R is ε-achievable under Γ, R 0 and|satisfies G(n)(R,Γσ2) R.
∈ ≥ | ≤
3
b) The sequence G(n)(R,Γσ2) of exponent func- To find an explicit formula of G(R,Γσ2), set
n≥1
{ | } |
tions satisfies the following subadditivity property:
G(n+m)(R,Γ|σ2) Ω(µ,λ)(σ2)=△ qmX∈aGx1mQinΩ(µ,λ)(qX,Q|σ2),
nG(n)(R,Γ|σ2)+mG(m)(R,Γ|σ2), (5) Ω(µ,λ)(σ2)=△ max min Ω(µ,λ)(qX,Qσ2),
≤ n+m qX Q∈G1 |
ftoroimnfwhicGh(wne)(hRa,vΓetσha2t).G∗(R,Γ|σ2)existsandisequal G(µ,λ)(R,Γ|σ2)=△ λ(R−µΓ1)+−λΩ(µ,λ)(σ2),
n≥1
c) tFoonrefidxeecdreRas>ing0,futhne|ctfiounncotifonΓG. F∗(oRr,fiΓx|eσd2)Γis>a mΓono=- G(µ,λ)(R,Γσ2)=△ λ(R−µΓ)−Ω(µ,λ)(σ2),
0 | 1+λ
min c(x), the function G∗(R,Γσ2) a monotone
increxa∈sXing function of R and satisfies| G(R,Γσ2)=△ sup G(µ,λ)(R,Γσ2),
| |
µ,λ≥0
G∗(R,Γσ2) R. (6)
| ≤ G(R,Γσ2)=△ sup G(µ,λ)(R,Γσ2).
d) The function G∗(R,Γσ2) is a convex function of | µ,λ≥0 |
|
(R,Γ).
It is obvious that
Proof of this property is found in [7].
G(R,Γσ2) G(R,Γσ2) G(R,Γσ2).
III. MAIN RESULTS | ≤ | ≤ |
In this section we state our main results. We first present a For λ [0,1), define
∈
result on an upper bound of G∗(R,Γσ2). Define
| J(µ,λ)(q σ2)
G (R,Γσ2) X|
DK | 1 1−λ
=△ EqqXXmY[X∈in2G]2≤:Γ(cid:8)[R−I(qX,qY|X)]++D(qY|X||W|qX)(cid:9), =△ logZdyZdxqX(x)e√−2(yπ2−σσx2)22 e−µλx21−λ ,
 
where [t]+ =max 0,t and  
{ } q (Y X) G(AµR,λ)(R,Γ,qX|σ2)=△ λ(R−µΓ)−J(µ,λ)(qX|σ2), 
I(qX,qY|X)=Eq(cid:20)log Yq|XY(Y|) (cid:21), G(AµR,λ)(R,Γ|σ2)=△ mqXinG(AµR,λ)(R,Γ,qX|σ2).
q (Y X)
Y|X
D(qY|X||W|qX)=Eq log W(Y X| ) . Furthermore, set
(cid:20) | (cid:21)
Then we have the following theorem. G (R,Γσ2)=△ sup G(µ,λ)(R,Γσ2)
Theorem 3: For any R>0, AR | µ≥0, AR |
λ∈[0,1)
Proof of thisGt∗h(eRo,reΓm|σ2is)≤givGeDnKi(nRA,Γp|pσe2n)d.ix A. We next = µs≥up0, mqXinGA(λR)(R,Γ,qX|σ2)
λ∈[0,1)
derive a lower bound of G∗(R,Γσ2). To this end we define
|
several quantities. Define = sup λ(R µΓ) maxJ(µ,λ)(q σ2) ,
X
Ω(µ,λ)(qX,Qσ2) λµ∈≥[00,1,)(cid:20) − − qX | (cid:21)
|
=△ log dxdyqX(x)e−(1+λ2)σ(y2−x)2e−µλx2 , GAR(R,Γ|σ2)=△ µs≥up0, G(AµR,λ)(R,Γ|σ2)
 (√2πσ2)1+λQλ(y)  λ∈[0,1)
Z Z
Ω(µ,λ)(σ2)=△ mqXaxmQinΩ(µ,λ)(qX,Q|σ2),  = λµ∈s≥u[0p0,1,)qXm∈inG1GA(λR)(R,Γ,qX|σ2)
G(µ,λ)(R,Γσ2)=△ λ(R−µΓ)−Ω(µ,λ)(σ2), = sup λ(R µΓ) max J(µ,λ)(q σ2) .
| 1+λ µ≥0, (cid:20) − −qX∈G1 X| (cid:21)
G(R,Γσ2)=△ sup G(µ,λ)(R,Γσ2). λ∈[0,1)
| |
µ,λ≥0
According to Oohama [7], we have the following lemma.
Accordingto Oohama [7], we have the following theorem.
Lemma 1 (Oohama [7]): For any q ,
X
Theorem 4 (Oohama [7]): For any AWGN W, we have
G∗(R,Γσ2) G(R,Γσ2). (7) minΩ(µ,λ)(qX,Qσ2)=(1+λ)J(µ,1+λλ)(qX σ2).
| ≥ | Q | |
4
The above minimization is attained by the probability density Furthermore, we have
function Q having the form
G(R,Γσ2)= max G(µ,λ)(R,Γσ2)
1 | µ,λ≥0 |
Q(y)=κ qX(x)e√−2(yπ2−σσx2)22 1+λe−µλx2dx1+λ , (8) = µ∈mλ[0≥,a10x+,λ]L(µ,λ)(R,Γ|σ2). (13)
Z   2σ2
 
  Proof of this proposition is given in Section IV. We next
where  
drive a relation between G (R,Γσ2) and G (R,Γσ2).
AR DK
κ−1 = dy dxq (x) e−(y2−σx2)2 e−µλx2 1−1λ 1−λ TΓo|σt2h)i.sFeonrdµw>e d0e,rwivee dtwefionpearametric| expressions of GDK|(R,
Z Z X  √2πσ2   G(µ)(R,Γσ2)
  DK |
=exp J(µ,1+λλ)(qX|σ2)   (9) =△ qXmYi∈nG2 R−I(qX,qY|X) ++D(qY|X||W|qX)
n o
is a constant for normalization. (cid:8)(cid:2) µ Γ E [X(cid:3)2] . (14)
− − qX
From Lemma 1, we have the following proposition.
For µ,λ 0, we define(cid:0) (cid:1)(cid:9)
Proposition 1: For any µ,λ 0, we have
≥
≥
GG((µµ,,λλ))((RR,,ΓΓ||σσ22))==GG((AAµµRR,,11++λλλλ))((RR,,ΓΓ||σσ22)),. G=△(DµqK,XλmY)(i∈RnG,2Γ|λσ2R) −I(qX,qY|X) −µΓ+µEqX[X2]
(cid:8) (cid:2)+D(q W q (cid:3)) . (15)
In particular, we have Y|X X
|| |
G(R,Γσ2)=G (R,Γσ2), According to Oohama [7], we have th(cid:9)e following lemma.
AR
| | Lemma 2 (Oohama [7]): For any R>0,
G(R,Γσ2)=G (R,Γσ2).
AR
| |
G (R,Γσ2)=maxG(µ)(R,Γσ2). (16)
LetqX,θ ∈G1 bethep.d.f.oftheGaussiandistributionwith DK | µ≥0 DK |
mean 0 and variance θ. We set
For any µ 0, any R>0, we have
△ (1+λ)θ 1+λ ≥
ξ =ξ(µ,λ,θ)= = , (10)
1+2µλθ θ−1+2µλ G(µ)(R,Γσ2)= max G(µ,λ)(R,Γσ2). (17)
ζ(µ,λ)(η σ2) DK | 0≤λ≤1 DK |
|
△ λ η 1 λ The two equalities (16) and (17) imply that
= log 1+ + log 1 2µη ,
L(µ2,λ)(R(cid:16),Γσ2σ)2(cid:17) 2 (cid:18) − 1+λ · (cid:19) GDK(R,Γ|σ2)= mµ≥a0x, G(DµK,λ)(R,Γ|σ2). (18)
| λ∈[0,1]
λ(R µΓ) ζ(µ,λ) 1 σ2 σ2
△ − − 2µ − 1+λ We can show the following proposition stating that the two
=
1+λ(cid:16) (cid:12) (cid:17) quantities G (R,Γσ2) and G (R,Γσ2) match.
(cid:12) AR DK
λ λ 1 (cid:12) λ 1 Proposition 3: For| any µ,λ 0, we h|ave the following:
= (R µΓ) log + ≥
1+λ − − 1+λ2 1+λ 2σ2µ
1 1log 1 1+ 2µλ(cid:18)σ2 . (cid:19)(11) G(AµR,1+λλ)(R,Γ|σ2)=G(D1Kµ+λλ,1+λλ)(R,Γ|σ2). (19)
−1+λ2 1+λ 1+λ In particular, we have
(cid:18) (cid:20) (cid:21)(cid:19)
The following proposition is a key result of this paper. G (R,Γσ2)=G (R,Γσ2). (20)
Proposition 2: For everyλ 0 and for everyµ [0,1+λ], AR | DK |
≥ ∈ 2σ2
we have Proof of this proposition is given in Section IV. From
Theorems 3, 4 and Propositions 2, 3, we immediately obtain
G(µ,λ)(R,Γσ2)=G(µ,λ)(R,Γσ2)=G(µ,λ)(R,Γσ2)
the following theorem.
| | |
=G(µ,1+λλ)(R,Γσ2)=L(µ,λ)(R,Γσ2). (12) Theorem 5:
AR | |
(µ, λ ) G∗(R,Γσ2)
TheinputGaussiandistributionq attainingG 1+λ ( |
R,Γσ2) satisfies the following: X,θ ∈G1 AR =G(R,Γ|σ2)=GAR(R,Γ|σ2)=GDK(R,Γ|σ2)
| = max L(µ,λ)(R,Γσ2).
(1+λ)θ 1 σ2 λ≥0, |
ξ =ξ(θ)= = . µ∈[0,1+λ]
1+2µλθ 2µ − 1+λ 2σ2
5
We finally solve the maximization problem defining G( Proof: By an elementary computation we have
R,Γσ2) to derive an explicit formula of this exponent func-
tion.|We set q (x) e−(y2−σx2)2 1+λe−µλx2
ρ=△ λ ,ν =△ 2λµσ2 =ρ (2µσ2). (21) X,θ  √2πσ2 
 
1+λ 1+λ ·
Then we have = e−x2θ2e−(1+λ2)σ(y2−x)2 e−µλx2
1+λ ν √2πθ (√2πσ2)1+λ
λ 0,µ 0, 0 ρ<1, (22)
L(≥µ,λ)(R,∈Γ(cid:20)|σ2)2=σ2ρR(cid:21)−⇔ν2 ·≤σΓ21−+ν21l≤og(1+ν) = e−(√1+2λ2π)ξ(σθξ2+(√σ22)(cid:16)πxσ−2ξ)+1ξ+σ2λy(cid:17)2e−(21(+ξ+λσ)y22), (26)
ρ 1 which together with Lemma 1 and (8) in this lemma yields
+ logν+ h(ρ), (23)
2 2 that Ω(µ,λ)(qX,θ,Qσ2) takes the minimum value at the p.d.f.
|
where h() is the binary entropy function. Since by (23), we Q=Q(y) given by
canregard· L(µ,λ)(R,Γσ2)asaquantitywithparametorρand 1
ν, we denote it by L(ρ|,ν)(R,Γ|σ2). Then we have Q(y)=κ dxq (x) e−(y2−σx2)2 1+λe−µλx2 1+λ
G(R,Γσ2)= max L(ρ,ν)(R,Γσ2). (24)  X,θ  √2πσ2  
| ρ∈[ ν ,1) | Z  
1+ν  
 1 
Solving the optimization problem (24) defining G(R,Γσ2), −(1+λ)(ξ+σ2)x− ξ y 21+λ − y2
we obtain the following result. | =κ dxe 2ξσ2 (cid:16) ξ+σ2 (cid:17) e 2(ξ+σ2)
Theorem 6: Let ν be the unique positive solution of  √2πθ  √2πσ2
0 Z
ν (1+ν )= σ2.  ξσ2 2(11+λ) ξ 21
0 0 Γ =κ 1+
(1+λ)θ(ξ+σ2) σ2
Then, the exponent function G(R,Γσ2) has the following (cid:20) (cid:21) (cid:20) (cid:21)
parametric form: | e−(21(+ξ+λσ)y22)
=Q (y). (27)
Γ × 2π(ξ+σ2) ξ+σ2
1+ (1+ν)
R= 1log σ2 , From (27), we can see that κ must satisfy
2 Γ  p
G(R,Γσ2)= νΓ1−1σl2ogν(11+ν)Γ ν(1+ν) , (25) κ−1 =(cid:20)(1+λ)ξθσ(2ξ+σ2)(cid:21)2(11+λ) ×(cid:20)1+ σξ2(cid:21)12 . (28)
| −2σ2 − 2 − σ2 The minimum value is
(cid:20) (cid:21)
Proof of this thefoorremsomisegiνve∈n[i0n,νS0e)c.tion IV.  (=b)(11l+ogλ)J(µ,1+λλξ)σ(2qX|σ2)(=a)+(11++λλ)lloogg(cid:0)κ1−+1(cid:1)ξ
IV. PROOFS OF RESULTS 2 (cid:20)(1+λ)θ(ξ+σ2)(cid:21) 2 (cid:18) σ2(cid:19)
In this section we prove Propositions 2, 3 and Theorem 6 (c) 1 ξσ2 1 2λµ 1+λ ξ
= log + log 1+
stated in Section III. 2 ξ+σ2 ξ − 1+λ 2 σ2
(cid:20) (cid:18) (cid:19)(cid:21) (cid:18) (cid:19)
Forθ 0,weletξ =ξ(µ,λ,θ)bethesamequantityasthat λ ξ 1 λ
defined b≥y (10). Let Q be a Gaussian distribution = log 1+ + log 1 2µξ .
withmean0andvariancξe+ξσ+2 σ∈2.GA1pplyingLemma1toq 2 (cid:18) σ2(cid:19) 2 (cid:18) − 1+λ · (cid:19)
X,θ
∈ Step (a) follows from (9) in Lemma 1. Step (b) follows from
, we have the following.
1
G Lemma 3: The functionΩ(µ,λ)(q ,Qσ2) takes the mini- (28).Step(c)followsfromthatbythedefinitionofξ,wehave
X,θ
|
mum value at Q=Q . The value is 1 1 2λµ
ξ+σ2 = .
minΩ(µ,λ)(q ,Qσ2)=Ω(µ,λ)(q ,Q σ2) (1+λ)θ ξ − 1+λ
X,θ X,θ ξ+σ2
Q | | Hence we have
=(1+λ)J(µ,1+λλ)(qX,θ|σ2)=ζ(µ,λ)(ξ(µ,λ,θ)|σ2). Ω(µ,λ)(σ2)=maxζ(µ,λ)(ξ(µ,λ,θ)σ2)
θ≥0 |
Furthermore, we have
= max ζ(µ,λ)(ξ σ2)= max ζ(µ,λ)(ξ σ2),
Ω(µ,λ)(σ2)= max ζ(µ,λ)(ξ σ2), θ≥0, | 0≤ξ<1+λ |
0≤ξ<12+µλλ | ξ=1(+1+2µλλ)θθ 2µλ
λ(R µΓ) max ζ(µ,λ)(ξ σ2) λ(R µΓ) max ζ(µ,λ)(ξ σ2)
G(µ,λ)(R,Γσ2)= − −0≤ξ<12+µλλ | . G(µ,λ)(R,Γσ2)= − −0≤ξ<12+µλλ | ,
| 1+λ | 1+λ
6
completing the proof. of equalities:
lemOmnaa. lower bound of G(R,Γ|σ2), we have the following exp Ω(µ,λ)(qX,Qη+σ2|σ2)
η(µL,eλm)msao4t:haStuppose that µ ∈ [0,12+σ2λ]. We choose η = = n dxdyq (x)e−(y2−σx2)2o e−(y2−σx2)2 λe−µλx2
X √2πσ2 √2πσ2Q (y)
ZZ  η+σ2 
η =η(µ,λ)= 1 σ2 . (29) (=a) 1+ ση2 λ2 dxqX(x)  
2µ − 1+λ (cid:0)√2πσ2(cid:1) Z
2
−1·(1+λ)η+σ2 y− η+σ2 x
Then for any qX, we have × dye 2 σ2(η+σ2) η+1σ+2λ !
Z
1+ η λ2 σ2(η+σ2)
= σ2 dxq (x) 2π
Qm∈iGn1Ω(µ,λ)(qX,Q|σ2)≤Ω(µ,λ)(qX,Qη+σ2|σ2) (cid:0)√2πσ2(cid:1) Z X ·s · (1+λ)η+σ2
=ζ(µ,λ) η σ2 =ζ(µ,λ) 1 σ2 σ2 = 1+ η λ2 η+σ2 .
| 2µ − 1+λ σ2 s(1+λ)η+σ2
(cid:18) (cid:12) (cid:19) (cid:16) (cid:17)
1 (cid:0) λ(cid:1) 1 (cid:12) Step (a) follows from (30). Hence we have
= log + (cid:12)
2 1+λ 2σ2µ
(cid:18) (cid:19) Ω(µ,λ)(q ,Q σ2)
1 1 2µλσ2 X η+σ2|
+ log 1+ . λ η 1 η+σ2
2 1+λ 1+λ = log 1+ + log
(cid:18) (cid:20) (cid:21)(cid:19) 2 σ2 2 (1+λ)η+σ2
(cid:16) (cid:17) (cid:18) (cid:19)
λ η 1 λη
= log 1+ + log 1
This implies that 2 σ2 2 − (1+λ)η+σ2
(cid:16) (cid:17) (cid:18) (cid:19)
(a) λ η 1 λ
= log 1+ + log 1 2µη
2 σ2 2 − 1+λ ·
Ω(µ,λ)(σ2)≤mqXaxΩ(µ,λ)(qX,Qη+σ2|σ2) =ζ(µ,λ) (cid:16)η σ2 (=b(cid:17))ζ(µ,λ) (cid:18)1 σ2 σ2 .(cid:19)
1 σ2 | 2µ − 1+λ
=ζ(µ,λ) σ2 , (cid:18) (cid:12) (cid:19)
2µ − 1+λ Step (a) follow(cid:0)s from(cid:1) that by (29), we have (cid:12)
(cid:18) (cid:12) (cid:19) (cid:12)
G(µ,λ)(R,Γσ2) L(µ,λ(cid:12))(R,Γσ2). 1 µ
| ≥ (cid:12) | = .
(1+λ)η+σ2 1+λ
Step (b) follows from (29).
Proof: By an elementary computation we have Proof of Proposition 2: We assume that λ 0 and µ
[0,1+λ]. In this case by Lemma 3, we have ≥ ∈
2σ2
e√−2(yπ2−σσx2)22 √2πeσ−2(Qy2−ησx+2)2σ2(y)λe−µλx2 G(µλ,λ()R(R−,Γµ|Γσ)2)−ζ(µ,λ) 21µ − 1σ+2λ σ2
 
≤ 1+λ(cid:16) (cid:12) (cid:17)
= 1√+2πσησ22λ2 e−(12+σ2λ)(y−x)2+2(ηy+2σ2)−µλx2 =L(µ,λ)(R,Γ|σ2), (cid:12)(cid:12) (31)
(cid:0) (cid:1) which together with Lemma 4 yields the equality (12). We
2
1+ η λ2 −1·(1+λ)η+σ2 y− η+σ2 x next prove (13). We set
= σ2 e 2 σ2(η+σ2) η+1σ+2λ !
(cid:0)√2πσ2(cid:1) A=△ max G(µ,λ)(R,Γσ2),
λ≥0, |
λ 1 −2µ x2 µ∈[0,1+λ]
e η+1σ+2λ ! 2σ2
× B =△ max G(µ,λ)(R,Γσ2).
2
1+ η λ2 −1·(1+λ)η+σ2 y− η+σ2 x λ≥0, |
(=a) σ2 e 2 σ2(η+σ2) η+1σ+2λ ! . (30) µ∈[12+σ2λ,∞)
√2πσ2
(cid:0) (cid:1) Then we have
G(R,Γσ2)= max G(µ,λ)(R,Γσ2)=max A,B . (32)
Step (a) follows from (29). Then we have the following chain | λ,µ≥0 | { }
7
By the equality (12) already proved, we have Using (q , q ), we obtain the following:
Y X|Y
λ
I(q ,q )+µE [X2]
A= max G(µ,λ)(R,Γσ2) 1+λ − X Y|X qX
λ≥0, | +D(qY(cid:8)|X W qX) (cid:9)
µ∈[0,1+λ] || |
2σ2 λ
= max L(µ,λ)(R,Γσ2). (33) =−1+λD(qX|Y||qX|qY)+D(qY,qX|Y||qX,W)
λ≥0, |
µ∈[0,12+σ2λ] +1µ+λλE(qY,qX|Y)[X2]
− λ
q 1+λ(xy)
HweenhcaeviettshueffifcoelslotwoinpgrocvheaBin≤ofAin.eWquhaelnitiλes≥: 0 and µ≥ 12+σ2λ, = dxdyqY(y)qX|Y(x|y)log X−|Yλ | 
Z Z  qX1+λ(x) 
+ dxdyqY(y)qX|Y(xy)  
G(µ,λ)(R,Γσ2) G(µ,λ)(R,Γσ2) Z Z |
| ≤ |
q (xy)q (y)
λ(R µΓ) Ω(µ,λ)(σ2) (a) λ(R µΓ) log X|Y | Y
= − 1+−λ ≤ 1+−λ × (qX(x)W(y x)e−1µ+λλx2)
|
(≤b) λ(R1−+12λ+σ2λΓ) (=c)L(12+σ2λ,λ)(R,Γ|σ2)(≤d)A. (34) = 1+1 λ dxdyqY(y)qX|Y(x|y)
Z Z
q (xy)
X|Y
Step (a) follows from that by Lemma 3, ×logqX(x) W(y x)e|−1µ+λλx2 1+λ
|
n o
Ω(µ,λ)(σ2) max λlog 1+ η + dyqY(y)logqY(y) 
≥ η≥0 2 σ2 Z
(cid:20) 1
(cid:16) (cid:17)
+1log 1 λ 2µη 0. (35) = 1+λD(qX|Y||qˆX|Y|qY)+D(qY||qˆY)
2 (cid:18) − 1+λ · (cid:19)(cid:21)≥ J(µ,1+λλ)(qX σ2), (36)
− |
where qˆ =qˆ (xy) is a conditional p.d.f. given by
X|Y X|Y
Step (b) follows from λ 0, µ 1+λ. Step (c) follows from |
TthheudsewfienihtiaovneoBf L(µA,λ,)(cRo≥m,Γp|lσet2in)≥.gS2tthσee2pp(rdo)offo.llows from (33). qˆX|Y(x|y)= Λ(1y)qX(x) W(y|x)e−1µ+λλx2 1+λ, (37)
Proof of Propo≤sition 3: For qX ∈G1, we set Λ(y)=△ dxqX(x)nW(y|x)e−1µ+λλx2o1+λ,
Z n o
and qˆ = qˆ (y) is a p.d.f. given by
△ Y { Y }y∈Y
(q )= q :q =(q ,q ) .
G1 X { Y|X XY X Y|X ∈G2} Λ(y)1+1λ
qˆ (y)= . (38)
Y 1
dyΛ(y)1+λ
For a given joint Gaussian p.d.f. q = q , we
XY 2 Hence,by(36)andthenon-negativityofdivergence,weobtain
introduce the conditional p.d.f. q and the p.d.f.∈q Gby R
X|Y Y
G(1µ+λλ,1+λλ)(R,Γ,q σ2) G(µ,1+λλ)(R,Γ,q σ2)
DK X| ≥ AR X|
qX(x)qY|X(y|x)=qY(y)qX|Y(x|y). for any qX ∈G1. Hence we have
G(1µ+λλ,1+λλ)(R,Γσ2) G(µ,1+λλ)(R,Γσ2).
DK | ≥ AR |
The above q is called a backward channel. Set
X|Y We next prove that for any λ 0,
≥
( µλ , λ ) (µ, λ )
G 1+λ 1+λ (R,ΓW) G 1+λ (R,ΓW).
G(µ1+λλ,1+λλ)(R,Γ,q σ2) DK | ≤ AR |
DK X| Let qX,θ be a Gaussian distribution with mean 0 and variance
=△ qY|Xm∈Gin1(qX)(cid:26)1+λλ(R−µΓ−I(qX,qY|X) Pθ.roWpoesiatisosnum2e, wtheathaqvXe,θ attains GA(µR,1+λλ)(R,Γ|W). Then by
+µE [X2])+D(q W q ) . (1+λ)θ 1 σ2
qX Y|X|| | X ξ =ξ(µ,λ,θ)= = . (39)
(cid:27) 1+2µλθ 2µ − 1+λ
8
From (39), we have Using (42), we also have
dy W(y x)e−1µ+λλx2 1+λΛ(y)−1+λλ
|
1 1+λ ξ Z n σ2 2(1−+λoλ) 1 1+λ
µ= ,θ = . (40) =
2(1+λ)ξ+σ2 (1+λ) 2µλξ (1+λ)ξ+σ2 √2πσ2
− (cid:20) (cid:21) (cid:18) (cid:19)
√2πσ2 λ dye−121σ+2λ(y−x)2+2(ξ+λσ2)y2−µλx2
×
(cid:16) (cid:17) Z −λ
From (40), we have σ2 2(1+λ) 1
=
(1+λ)ξ+σ2 √2πσ2
(cid:20) (cid:21)
−1(1+λ)ξ+σ2 y−(1+λ)(ξ+σ2)x 2
dye 2 σ2(ξ+σ2) (1+λ)ξ+σ2
(cid:18) (cid:19)
×
[(1+λ)ξ+σ2]ξ Z
θ = . (41) 1 λ(1+λ) x2−µλx2
(1+λ)(ξ+σ2) e2(1+λ)ξ+σ2
×
−λ
(a) σ2 2(1+λ) 1
=
(1+λ)ξ+σ2 √2πσ2
(cid:20) (cid:21)
Computing Λ(y) for this qX,θ, we have dye−12(σ1+2(λξ)+ξ+σ2σ)2 y−(1(1++λλ)()ξξ++σσ22)x 2
(cid:18) (cid:19)
×
Z
1
σ2 2(1+λ) ξ+σ2
= . (44)
(1+λ)ξ+σ2 σ2
Λ(y)= dxq W1+λ(y x)e−µλx2 (cid:20) (cid:21) r
X,θ | Step (a) follows fromthe first equality of (40). From (43) and
Z
1 1 1+λ (44), we have
= dx
√2πθ (cid:18)√2πσ2(cid:19) Z dy W(y x)e−1µ+λλx2 1+λΛ(y)−1+λλ = dyΛ(y)1+1λ.
e−12 xθ2+(1+λ)(x−σ2y)2+2µλx2 Z n | o Z (45)
(cid:20) (cid:21)
× Now we define the function V =V(y x) by
(a) 1 1 1+λ |
= dx qˆ (y)qˆ (xy)
√2πθ (cid:18)√2πσ2(cid:19) Z V(y x)= Y X|Y | . (46)
| q (x)
−1+λ ξ+σ2 x− ξ y 2+ y2 X
e 2 ξσ2 ξ+σ2 ξ+σ2
× (cid:20) (cid:16) (cid:17) (cid:21) By (37) and (38), V(y x) has the following form:
|
= 1 1+λ 1 ξσ2 e−1+2λ·ξ+y2σ2 Λ(y)1+1λ
(cid:18)√2πσ2(cid:19) sθ(1+λ)ξ+σ2 V(y|x)= dyΛ(y)1+1λ
(=b)(cid:18)√21πσ2(cid:19)1+λs(1+λσ)2ξ+σ2e−1+2λ·ξ+y2σ2. (42) × Λ(1Ry)qX(x) W(y|x)e−1µ+λλx2 1+λ· qX1(x)
n o
W(y x)e−1µ+λλx2 1+λΛ(y)−1+λλ
= | . (47)
n o 1
dyΛ(y)1+λ
Step (a) follows from (26). Step (b) follows from (41). Using
(42), we have the following: Taking integral ofRboth sides of (47) on the variable y, we
obtain
dyV(y x)
|
Z
dyΛ(y)1+1λ = (1+λσ)2ξ+σ2 2(11+λ) = dy W(y|x)e−1µ+λλx2 1+λΛ(y)−1+λλ (=a)1.
Z 1 (cid:20) −1+λ· y2 (cid:21) R n dyΛ(y)1o+1λ
dye 2 ξ+σ2
× √2πσ2 Step (a) follows from (R45). The aboveequality implies that V
Z
1 is a conditional density function. Furthermore, note that from
σ2 2(1+λ) ξ+σ2
= . (43) (46),
(cid:20)(1+λ)ξ+σ2(cid:21) r σ2 qX(x)V(y x)=qˆY(y)qˆX|Y(xy).
| |
9
Then, choosing q = qˆ ,q = qˆ in (36), we have, for Hence from (49), we have that if the pair (ρ∗,ν∗) satisfies
Y Y X|Y X|Y
λ 0, ρ∗ [ν∗/(1+ν∗),1) and
≥ ∈
G(DλµK,1+λλ)(R,Γ|W) ∂∂Fρ =−R− 12logν∗+ 12log1 ρ∗ρ∗ =0,
≤+1D+(λV{(WR−qXµ)Γ)−I(qX,V)+µEqX[c(X)]} ∂∂Fν (cid:12)(cid:12)(cid:12)(cid:12)(ρ,ν)=(ρ∗,ν∗) = 21 σΓ2 + 1+1ν∗ − νρ∗∗ =−0, 
|| | (cid:12)(ρ,ν)=(ρ∗,ν∗) (cid:20) (cid:21)
= 1+λλ(R−µΓ)−J(µ,1+λλ)(qX|W)=G(AµR,1+λλ)(R,Γ|W), then, it(cid:12)(cid:12)(cid:12)attains the minimum of F(ρ,ν) under ρ [ν/((153+)
∈
ν),1). From (53), we have
completing the proof.
Proof of Theorem 6: We set 1 1 ρ∗
R= logν∗+ log , (54)
F =F(ρ,ν)= L(ρ,ν)(R,Γσ2) −2 2 1 ρ∗
= ρR+ ν Γ− + 1log(1+| ν) ρlogν 1h(ρ). (48) ρ∗ = ν∗ +ν∗ Γ −ν∗ . (55)
− 2 · σ2 2 − 2 − 2 1+ν∗ σ2 ≥ 1+ν∗
Then we have Furthermore, for ν∗ [0,ν ), we have
0
∈
G(R,Γσ2)=( 1) min F(ρ,ν). 1 Γ
| − ·ρ∈[1+νν,1) 1−ρ∗ = 1+ν∗ −ν∗σ2
Computing the gradient of F, we obtain Γ Γ
1 ν∗(1+ν∗) 1 ν (1+ν )
∂F 1 1 ρ = − σ2 > − 0 0 σ2 =0. (56)
= R logν+ log , 1+ν∗ 1+ν∗
∂ρ − − 2 2 1 ρ
∂F = 1 Γ + 1 ρ . −  (49) (Fρro∗,mν∗()54c)e-r(t5a6in)l,ywaettacinans Gse(eR,thΓatσ2fo)r=νF∗ (∈ρ∗,[0ν,∗ν)0.)F,rothme (p5a4i)r
∂ν 2 σ2 1+ν − ν |
(cid:20) (cid:21) and (55), we have
Let ∂2F ∂2F  1+ Γ (1+ν∗)
∂ρ2 ∂ρ∂ν R= 1log σ2 . (57)
B = ∂2F ∂2F  2 1 Γ ν∗(1+ν∗)
∂ρ∂ν ∂ν2  − σ2
 
  Furthermore, we have
be the Hessian matrix of F. Computing B, we have
∂2F 1 G(R,Γ|σ2)=−F(ρ∗,ν∗)
= >0, (50) ν∗Γ 1 1 1
∂ρ2 2ρ(1 ρ) = ρ∗R log(1+ν∗)+ ρ∗logν∗+ h(ρ∗)
− − 2σ2 − 2 2 2
∂2F 1
∂ρ∂ν =−2ν, = ρ∗ R+ 1logν∗+ 1log1−ρ∗
2 2 ρ∗
∂∂2νF2 = 12 −(1+1ν)2 + νρ2 ν(cid:20)∗Γ 1log(1+ν∗) 1log(1(cid:21) ρ∗)
(cid:20) (cid:21) −2σ2 − 2 − 2 −
(a) 1 1 + 1 = 1 >0. (51) (=a) ν∗Γ 1log 1 Γ ν∗(1+ν∗) . (58)
≥ 2 −(1+ν)2 (1+ν)ν 2(1+ν)2ν −2σ2 − 2 − σ2
(cid:20) (cid:21) (cid:20) (cid:21)
Step (a) follows from ρ [ν/(1+ν),1). Computing B , we Step (a) follows from (54) and (55). Thus we obtain the
∈ | |
have parametric expression of G(R,Γσ2) given by (57) and (58).
|
∂2F ∂2F ∂2F 2
B =
| | ∂ρ2 ∂ν2 − ∂ρ∂ν
(cid:18) (cid:19)
1 ρ 1 1
= APPENDIX
ρ(1 ρ) ν2 − (1+ν)2 − ν2
1− (cid:20)ρ2 1 (cid:21) A. Proof of Theorem 3
=
ρ(1 ρ) ν2 − (1+ν)2 We fix δ [0,1/2). We consider a Gaussian channel with
− (cid:20) (cid:21) input X and∈output Y, having the form
1 ρ 1 ρ 1 (a)
= + 0. (52)
ρ(1 ρ) ν 1+ν ν − 1+ν ≥ Y =αX +S,X S,S (0,ξ).
− (cid:20) (cid:21)(cid:20) (cid:21) ⊥ ∼N
Step (a) followsfromρ [ν/(1+ν),1). From(50), (51), and We consider a Gaussian random pair (X,Y) obtained by
∈
(52), we can see that F(ρ,ν) is a convex function of (ρ,ν). lettingX beGaussianrandomvariablewithX (0,θ).We
∼N
10
assume that θ Γ. Let q be a probability density function Proof: We have the following chain of inequalities:
XY
≤
of(X,Y).FortheGaussianchannelspecifiedbyq ,wecan
Y|X
constructann-lengthblockcode(φ(n),ψ(n))withmessageset nD(q W q )
Y|X X
|| |
satisfying:
Kan) P(n)(φ(n),ψ(n) q ) 1 δ. (=a) 1 qn (yn φ(n)(k))log qYn|X(yn|φ(n)(k))
b) alclcodewordsφ|(nY)(|Xk),k≥ −nsatisfy φ(n)(k) 2 nθ. |Kn|kX∈KnynX∈Yn Y|X | Wn(yn|φ(n)(k))
c) n1 log|Kn|≥min{R,I(q∈X,KqY|X)−δ|}|. || ≤ (b) 1 β (k)log βn(k) +β (k)log βn(k)
By the condition b), we can obtain the following result. n n
Lemma 5: For every k n, we have ≥ |Kn|kX∈Kn" αn(k) αn(k)#
∈K
Z ···Z dynqYn|X(yn|φ(n)(k))log qWYn|nX((yynn||φφ((nn))((kk)))) = kX∈Knβ|Kn(nk|)log αβ||KKnn((nnkk||)) + β|Kn(nk|)log αβ||KKnn((nnkk||))
n (c) β β 
n n
|≤n{zD(q}Y|X||W|qX). (59) ≥ βnlogαn +βnlogαn ≥−h(βn)−βnlogαn
Proof: By a direct computation we have (d)
h(1 δ) (1 δ)logα . (62)
n
≥ − − − −
qn (yn φ(n)(k))
dynqn (yn φ(n)(k))log Y|X | Step (a)followsfromLemma5. Steps(b)and(c)followfrom
··· Y|X | Wn(yn φ(n)(k))
Z Z | the log-sum inequality. Step (d) follows from that
n
|= 12{z(1−} α)2||φ(n)σ(2k))||2 + n2 σξ2 −1+logσξ2 βn =Pc(n)(φ(n),ψ(n)|qY|X)≥1−δ
(cid:20) (cid:21)
(a) n θ n ξ σ2 and δ (0,1/2]. From (62), we obtain
(1 α)2 + 1+log ∈
≤ 2 − σ2 2 σ2 − ξ
=nD(qY|X||W|qX). (cid:20) (cid:21) (60) αn =Pc(n)(φ(n),ψ(n)|W)
nD(q W q )+h(1 δ)
Step (a) follows from the condition b). exp Y|X|| | X −
≥ − 1 δ
For k n, we set (cid:18) − (cid:19)
∈K =exp n[(1 δ)−1D(q W q )+η (δ)] ,
α (k)=△ Wn( (k)φ(n)(k))= Wn(yn φ(n)(k)), {− − Y|X|| | X n }
n
D | |
ynX∈D(k) completing the proof.
β (k)=△ qn ( (k)φ(n)(k))= qn (yn φ(n)(k)), Proof of Theorem 3: We first consider the case where R
n Y|X D | Y|X | I(q ,q ) δ. In this case we choose ϕ(n) = φ(n). The≤n
X Y|X
ynX∈D(k) we have −
α (k)=△ 1 α (k)=qn ( (k)φ(n)(k)),
n − n Y|X D | P(n)(ϕ(n),ψ(n) W)=P(n)(φ(n),ψ(n) W)
β (k)=△ 1 β (k)=qn ( (k)φ(n)(k)). c | c |
n − n Y|X D | (=a)exp n[R+δ I(q ,q )]+
Furthermore, set {− − X Y|X
n[(1 δ)−1D(q W q )+η (δ)]
αn =△ kX∈Kn |K1n|αn(k)=P(cn)(φ(n),ψ(n)|W), (≥b)−exp{−−n[R−I(qXY,|XqY|||X)|]+X n }
β =△ 1 β (k)=P(n)(φ(n),ψ(n) q ). −n[(1−δ)−1D(qY|X||W|qX)+δ+ηn(δ)]}. (63)
n n c | Y|X
n
kX∈Kn |K | Step (a) follows fromthe condition R+δ I(q ,q ) 0.
X Y|X
− ≤
The quantity P(n)(φ(n),ψ(n) W) has a lower bound given by Step (b) follows from that
c
|
the following Lemma.
Lemma 6: For any δ [0,1/2), we have [R+δ I(qX,qY|X)]+ [R I(qX,qY|X)]++δ.
∈ − ≤ −
1
P(n)(φ(n),ψ(n) W)= Wn( (k)φ(n)(k)) We next consider the case where R > I(q ,q ) δ.
c | D | X Y|X −
|Kn|kX∈Kn ConsiderthenewmessagesetKnsatisfying|Kn|=e⌊nR⌋.For
exp n[(1 δ)−1D(q W q )+η (δ)] . (61) new message set , we define ϕ(n)(k) such that ϕ(n)(k) =
≥ {− − Y|X|| | X n } Kn
φ(n)(k) if k . For k b , web define ϕ(n)(k)
n n n
Haebrienawryeesnettrηonpy(δf)u=△nctn1io(n1.−δ)−1h(1−δ) and h(·) stands for asarbmiteradryecosedqeur∈eψn(cKneb)oafsXthnatho∈afvtiKhnbeg m−theesKstaygpeesqeXt . Wn.eTuhseen twhee
K