Channel Resolvability Theorems for General Sources and Channels

0.22 MB·

by  Hideki Yagi

Channel Resolvability Theorems
for General Sources and Channels
Hideki Yagi
Dept. of Computer and Network Engineering
University of Electro-Communications
Tokyo, Japan
Email: [email protected]
7
Abstract—In the problem of channel resolvability, where a [4], [10]. The established general formula provides a new
1
given output probability distribution via a channel is approxi- expressionfortheδ-spectralsup-entropyrate,whichisawell-
0
matedbytransformingtheuniformrandomnumbers,character-
2 known informationquantity in informationspectrum methods
izingtheasymptoticallyminimumrateofthesizeoftherandom
[3].Theanalysisisalso extendedto thesecond-orderchannel
n numbers, called the channel resolvability, has been open. This
a paper derives formulas for the channel resolvability for a given resolvability,whichis definedas the asymptoticallyminimum
J general sourceandchannelpair.Wealsoinvestigate thechannel second-orderrateofthesizeofuniformrandomnumberswith
9 resolvability in an optimistic sense. It is demonstrated that the respect to a fixed first-order resolvability rate.
derivedgeneralformulasrecaptureasingle-letterformulaforthe
2
stationary memoryless source and channel. When the channel

] is the identity mapping, the established formulas reduce to an II. PROBLEMFORMULATION: CHANNEL RESOLVABILITY
T alternative form of the spectral sup-entropy rates, which play a
I key role in information spectrum methods. The analysis is also Let and be finite or countably infinite alphabets.
s. extended to the second-order channel resolvability. Let XnXdenoteYa sequence of n random variables taking
c
valuesin n with probabilitydistribution P . In this paper,
[ I. INTRODUCTION X Xn
we identify P with Xn, and both expressions are used
Xn
1 Finding the asymptotically minimum rate of the size of the interchangeably. We call X = Xn ∞ a general source.
v uniform random numbers (channel resolvability) which can Also, let Wn : n n denot{e a s}tno=ch1astic mapping, and
62 approximatea giventargetoutputdistributionviaa channelis wecallW ={WXn}→∞n=1Yageneralchannel.Wedonotimpose
3 called the problem of channel resolvability. When the varia- anyassumptionssuchasstationarityorergodicityoneitherX
8 tional distance between the target output distribution and the or W. We denote by Y = Yn ∞ the output process via
0 approximateddistributionis requiredto be asymptoticallynot W due to input process X.{ }n=1
.
1 greaterthanδ [0,1),theproblemiscalledtheproblemofδ-
∈ We review the problem of channel resolvability [3] using
0 channelresolvability.Thoughtheseproblemswereintroduced
7 byHanandVerdu´ [4]morethantwo decadesago,thegeneral thevariationaldistanceasanapproximationmeasure.LetUMn
1 formula for the channel resolvability has not been known denote the uniform random number of size Mn, which is
: a random variable uniformly distributed over 1,...,M .
v in general. A few cases where the channel resolvability has { n}
Xi been characterized are the worst input case with δ = 0 by CUonsidveiar aapdperotexrimmiantiisntgicthmeatpaprignegt dϕistr:ibu1t,io.n..P,MYn by usinng
Hayashi [5] and the case of the stationary memoryless source Mn n { n} → X
r and Wn. We denote by P the approximated output distri-
a and channel by Watanabe and Hayashi [11]. Recently, much Y˜n
bution via Wn due to the input X˜n :=ϕ (U ) (cf. Fig. 1).
attention has been paid to the channel resolvability because n Mn
Precisionofthe approximationis measuredbythe variational
this technique can be used to guarantee the strong secrecy in
distance between P and P .
physical-layer security systems [1], [5]. Thus, it is desirable Yn Y˜n
Definition 1 (Variational Distance): Letting P and P
tocharacterizethechannelresolvabilityforagivenpairofthe Z Z˜
be probability distributions on a countably infinite set ,
input distribution and the general channel.
Z
Inthispaper,wecharacterizetheδ-channelresolvabilityfor
1
a general source and a general channel with any δ ∈ [0,1). d(PZ,PZ˜):= 2 |PZ(z)−PZ˜(z)| (1)
By taking the maximum over all possible general sources, z∈Z
X
we can naturally obtain the general formula for the worst
is called the variational distance between P and P . ✷
input case. We also investigate the δ-channel resolvability Z Z˜
in an optimistic sense. When we restrict ourselves to the It is easily seen that 0 ≤ d(PZ,PZ˜) ≤ 1, where the left
noiseless channel (identity mapping), the problem of channel inequality becomes equality if and only if PZ =PZ˜.
resolvability reduces to the problem of source resolvability For any given sequence of random variables {Zn}∞n=1, we
introduce quantities which play an important role in informa-
ThisresearchissupportedbyJSPSKAKENHIGrantNumberJP16K06340. tion spectrum methods [3].
target distribution We define
S∗(δ X,W)
|
Approximation is measured :=inf R: R is optimistically δ-achievable at X ,
{ }
by referredto as the optimistic δ-channelresolvability (at X). ✷
The following channel resolvability theorem is implicitly
proved by Hayashi [5] for general sources and channels.
approximated distribution
Theorem1(Hayashi[5]):Letδ [0,1)befixedarbitrarily.
∈
For any general source X = Xn ∞ and any general
Fig.1. Channel Resolvability System channel W = Wn ∞ , { }n=1
{ }n=1
S(δ X,W) I (X;Y), (9)
δ
| ≤
Definition 2 (ε-Limit Superior in Probability): For ε
[0,1], ∈ S∗(δ|X,W)≤I∗δ(X;Y), (10)
where we define
εp-limsupZn:=inf α:limsupPr Zn>α ε , (2) 1 Wn(Yn Xn)
n→∞ n→∞ { }≤ I (X;Y):=δp-limsup log | , (11)
(cid:26) (cid:27) δ n→∞ n PYn(Yn)
εp∗-limsupZ :=inf α:liminfPr Z >α ε . (3)
n→∞ n n→∞ { n }≤ I∗(X;Y):=δp∗-limsup 1 logWn(Yn|Xn). (12)
n o δ n→∞ n PYn(Yn)
For ε = 0, the right-hand sides of (2) and (3) are simply
✷
denoted by p-limsupZ and p∗-limsupZ , respectively. ✷
n n Unfortunately, Theorem 1 does not provide a lower bound
n→∞ n→∞
on the δ-channel resolvability. For the worst input case, in
The problem of channel resolvability has been introduced
contrast, a lower bound has also been given by Hayashi [5].
by Han and Verdu´ [4].
Theorem 2 (Hayashi [5]): For any general channel W =
fixDedefianribtiitornari3ly.(δA-CrheasnonlvealbRileitsyolrvaatbeilRity): L0eitsδsa∈id[t0o,1b)ebδe- {Wn}∞n=1,
≥
achievable at X if there exists a deterministic mapping ϕn : supI2δ(X;Y)≤supS(δ|X,W)≤supIδ(X;Y), (13)
1,...,M n satisfying X X X
n
{ }→X supI∗ (X;Y) supS∗(δ X,W) supI∗(X;Y). (14)
1 2δ ≤ | ≤ δ
limsup logM R, (4) X X X
n
n→∞ n ≤ In particular,
linm→s∞upd(PYn,PY˜n)≤δ, (5) supS(0|X,W)=supI(X;Y), (15)
X X
where Y˜n denotes the output via Wn due to the input X˜n = supS∗(0X,W)=supI∗(X;Y), (16)
ϕ (U ). We define X | X
n Mn
where we define
S(δ|X,W):=inf{R: R is δ-achievable at X}, (6) 1 Wn(Yn Xn)
I(X;Y):=p-limsup log | , (17)
which is called the δ-channel resolvability (at X). ✷ n→∞ n PYn(Yn)
Equation (5) requires d(PYn,PY˜n) ≤ δ +γ for all large I∗(X;Y):=p∗-limsup 1 logWn(Yn|Xn). (18)
n, where γ > 0 is an arbitrary constant. We may consider n→∞ n PYn(Yn)
a slightly weaker constraint, which requires d(PYn,PY˜n) ≤ ✷
δ + γ for infinitely many n The following problem is the
weaker version of the δ-channel resolvability, introduced by III. MAIN THEOREMS:δ-CHANNELRESOLVABILITY
[9] in the context of partial resolvability. Now, we give the general formulas for the δ-channel re-
Definition 4 (Optimistic δ-Channel Resolvability): Let δ solvability at a specific input X and its optimistic version.
∈
[0,1) be fixed arbitrarily. A resolvability rate R 0 is
≥
said to be optimistically δ-achievable at X if there exists a Theorem3:Letδ [0,1)be fixedarbitrarily.Foranyinput
∈
deterministic mapping ϕ : 1,...,M n satisfying process X and any general channel W,
n n
{ }→X
1 S(δ X,W)= inf I(Xˆ;Yˆ), (19)
linm→s∞upnlogMn ≤R, (7) | Xˆ∈Bδ(X,W)
S∗(δ X,W)= inf I(Xˆ;Yˆ), (20)
linm→i∞nfd(PYn,PY˜n)≤δ. (8) | Xˆ∈Bδ∗(X,W)
where Yˆ = Yˆn ∞ denotes the output process via W due in [6, Theorem 6]. Comparing the two characterizations, the
to the input{proce}sns=X1ˆ = Xˆn ∞ , and we define following inequality is obvious for all δ [0,1):
{ }n=1 ∈
Bδ(X,W):= Xˆ ={Xˆn}∞n=1 :linm→s∞upd(PYn,PYˆn)≤δ , Xˆ∈Biδn(Xf ,W)I(Xˆ;Yˆ)≤Xˆ∈Biδn(Xf ,W) εs≥u0p, Dε(W||Z|Xˆ).
(cid:26) (cid:27) Z∈B˜ε(Yˆ)
B∗(X,W):= Xˆ = Xˆn ∞ :liminfd(P ,P ) δ . (25)
δ n=1 n→∞ Yn Yˆn ≤
(Proof) The nproof i(cid:8)s giv(cid:9)en in Sec. IV. o✷ because D0(W||Yˆ|Xˆ) = I(Xˆ;Yˆ). Also, we have for all
δ [0,1):
Remark 1: The right-hand sides of (19) and (20) are ∈
nonincreasing functions of δ. Furthermore, these are right- inf I(Xˆ;Yˆ) inf sup D (W Z Xˆ).
ε
continuous in δ [0,1). ✷ Xˆ∈B∗(X,W) ≤Xˆ∈B∗(X,W) ε≥0, || |
Remark 2: A∈s is mentioned in Theorem 1, Hayashi [5, δ δ Z∈B˜ε(Yˆ)
(26)
Theorem4]hasimplicitlyshownthatanyrateR>I (X;Y)
δ
isδ-achievableataspecificinputX.Therefore,weobtainthe These relationships are of use to prove Theorems 3 and 4. ✷
following relation between the right-hand side of (19) and δ-
spectral sup-mutual information rate I (X;Y): IV. PROOF OF THEOREMS 3AND 4
δ
inf I(Xˆ;Yˆ) I (X;Y) (δ [0,1)) (21) A. Finite-Length Bounds
δ
Xˆ∈Bδ(X,W) ≤ ∈ As we take an information spectrum approach to prove the
and analogously general formulas in Theorems 3 and 4, we will use finite-
length upper and lower bounds on the variational distance,
inf I(Xˆ;Yˆ) I∗(X;Y) (δ [0,1)). (22)
Xˆ∈B∗(X,W) ≤ δ ∈ which hold for each blocklength n.
δ Intheproofofthedirectpart,we usethe followinglemma.
We canfindexamplesofX andW forwhichtheinequalities
in (21) and (22) are strict. This statement is also true even in Lemma 1 (Finite-Length Upper Bound [5]): Let Vn be an
the case δ =0. ✷ arbitrary input random variable, and its corresponding output
viaWn isdenotedbyZn.Then,foranygivenpositiveinteger
Although the formulas established in Theorem 3 are suffi-
cienttocharacterizeS(δ X,W)andS∗(δ X,W),itrequires Mn, there exists a mapping ϕn : 1,2,...,Mn n such
{ }→X
| | that
atedioustasktoderiveasingle-letterformulaforthestationary
memoryless source and channel pair. We give alternative
d(P ,P )
formulas in the following theorem: Zn Y˜n
Theorem4: Letδ [0,1)be fixedarbitrarily.For anyinput 1 Wn(Zn Vn) 1 enc
∈ Pr log | >c + , (27)
process X and any general channel W, ≤ (cid:26)n PZn(Zn) (cid:27) 2rMn
S(δ X,W)= inf sup Dε(W Z Xˆ), (23) wherec 0isanarbitraryconstantandY˜ndenotestheoutput
| Xˆ∈Bδ(X,W)Z∈εB≥˜ε0(,Yˆ) || | via Wn≥due to input X˜n =ϕn(UMn). ✷
In the proof of the converse part, we use the following
S∗(δ X,W)= inf sup Dε(W Z Xˆ), (24) lemma.
| Xˆ∈Bδ∗(X,W)Z∈εB≥˜ε0(,Yˆ) || | arbLietrmarmyapr2ob(aFbiinliittye-dLiesntrgibthutiLoonwoenr Bno.uTnhde):n,LfoertaPnZynunbifeoramn
where Yˆ = Yˆn ∞ denotes the output process via W due randomnumberU ofsizeM anYdadeterministicmapping
to input proc{ess X}ˆn==1 Xˆn ∞ , and we define ϕ : 1,2,...,MMn n wenhave
{ }n=1 n { n}→X
Dε(W||Z|Xˆ):=εp-linm→s∞upn1 logWPnZ(Ynˆ(nYˆ|Xnˆ)n), d(PZn,PY˜n)≥Pr(n1 logWPnZ(Yn˜(nY˜|Xn˜)n) ≥c)− Mennc, (28)
B˜ε(Y):=(cid:26)Z ={Zn}∞n=1 : linm→s∞upd(PYn,PZn)≤ε(cid:27). twohXe˜ren,X˜annd=c iϕsna(nUMarnb)it,raY˜ryncdoennsotatenst sthaetisofyuitnpgutMvina Wenncd.ue
≤
(Proof) The proof is given in Sec. IV. ✷ (Proof) First, we define
Remark 3: Theorems 3 and 4 provide two formulas for the
enc
δ-channel resolvability S(δ X,W). Although the characteri- T := y n :P (y) P (y) . (29)
| n ∈Y Y˜n ≥ M Zn
zationin(23)ismorecomplicated,thisexpressioncanbeseen (cid:26) n (cid:27)
as a counterpart of the alternative formula for the channel Then, by the definition of the variational distance, it is easily
capacity given by Hayashi and Nagaoka [7, Theorem 1] verified that
established for quantumchannels. The correspondingformula
d(P ,P ) P (T ) P (T ), (30)
fortheδ-channelcapacityoverclassicalchannelscanbefound Zn Y˜n ≥ Y˜n n − Zn n
wherethesecondtermontheright-handsidecanbeevaluated Let V = Vn ∞ be a general source satisfying V
{ }n=1 ∈
as B (X,W) and
δ
M M
PZn(Tn)=yX∈TnPZn(y)≤ ennc yX∈TnPY˜n(y)≤ ennc. (31) I(V;Z)≤Xˆ∈Biδn(Xf ,W)I(Xˆ;Yˆ)+γ, (40)
To evaluate the first term on the right-hand side of (30), we whereZ = Zn ∞ denotestheoutputprocessviaW dueto
{ }n=1
borrow an idea given in [11]. Since the input process V. Setting M =en(I(V;Z)+2γ), it follows
n
from (39) and (40) that
Mn 1
PY˜n(y)=Xi=1 MnWn(y|ϕn(i)) (y ∈Yn), (32) linm→s∞upn1 logMn =I(V;Z)+2γ ≤R. (41)
denoting Wn (y)=Wn(y ϕ (i)), we have
ϕn(i) | n Lemma 1 with c = I(V;Z)+γ guarantees the existence
P (T ) of a deterministic mapping ϕ : 1,2,...,M n with
Y˜n n n { n} → X
the uniform random number U satisfying
Mn 1 enc Mn
= i=1 MnWϕnn(i)(cid:26)PY˜n(Y˜n)≥ MnPZn(Y˜n)(cid:27) linm→s∞upd(PZn,PY˜n)
X
1 Wn(Zn Vn)
Mn 1 Mn 1 enc limsupPr log | >I(V;Z)+γ
= M Wϕnn(i) M Wϕnn(j)(Y˜n)≥ M PZn(Y˜n). ≤ n→∞ (cid:26)n PZn(Zn) (cid:27)
n n n
Xi=1 Xj=1  =0, (42)
Here, noticing that
 
where Y˜n denotes the output via Wn due to the input X˜n =
1
M Wϕnn(i)(y)≥encPZn(y) ϕn(UMn). Then, the triangle inequality leads to
n
=⇒ M1 Wϕnn(j)(y)≥encPZn(y), (33) linm→s∞upd(PYn,PY˜n)
n
j
we obtain the folloXwing lower bound: ≤linm→s∞upd(PYn,PZn)+nl→im∞d(PZn,PY˜n)≤δ, (43)
Mn 1 where the last inequality is due to the fact V ∈ Bδ(X,W)
P (T ) Wn Wn (Y˜n) encP (Y˜n) . and (42). Combining (41) and (43) concludes that R is δ-
Y˜n n ≥ M ϕn(i) ϕn(i) ≥ Zn
Xi=1 n n o achievable, and hence (35) holds.
(34) To prove (36), for any given γ >0 setting
Thus, plugging (31) and (34) into (30), we obtain (28). ✷
R= inf I(Xˆ;Yˆ)+3γ, (44)
B. Proof of Theorems 3 and 4 Xˆ∈Bδ∗(X,W)
The relations shown in (25) and (26) imply that to prove we show that R is optimistically δ-achievable. Let V =
Theorems 3 and 4, it suffices to show Vn ∞ bea generalsourcesatisfyingV B∗(X,W)and
{ }n=1 ∈ δ
S(δ X,W) inf I(Xˆ;Yˆ), (35) I(V;Z) inf I(Xˆ;Yˆ)+γ, (45)
| ≤Xˆ∈Bδ(X,W) ≤Xˆ∈Bδ∗(X,W)
S∗(δ X,W) inf I(Xˆ;Yˆ) (36) where Z = Zn ∞ denotes the output process via W
| ≤Xˆ∈Bδ∗(X,W) due to input{V.}An=lo1ng the same line to prove (35), it
in the direct (achievability) part and is easily verified that there exists a deterministic mapping
ϕ : 1,2,...,M n satisfying (41) and (42). Then,
S(δ X,W) inf sup D (W Z Xˆ), (37) n { n} → X
| ≥Xˆ∈Bδ(X,W) ε≥0, ε || | the triangle inequality leads to
Z∈B˜ε(Yˆ)
liminfd(P ,P )
S∗(δ X,W) inf sup D (W Z Xˆ) (38) n→∞ Yn Y˜n
ε
| ≥Xˆ∈B∗(X,W) ε≥0, || | liminfd(P ,P )+ lim d(P ,P ) δ, (46)
δ Z∈B˜ε(Yˆ) ≤ n→∞ Yn Zn n→∞ Zn Y˜n ≤
in the converse part. where the last inequality is due to the fact V B∗(X,W).
∈ δ
1) Direct part: First, fix γ >0 arbitrarily. Setting Combining (41) and (46) concludes that R is optimistically
δ-achievable, and hence (36) holds. ✷
R= inf I(Xˆ;Yˆ)+3γ, (39)
Xˆ∈Bδ(X,W) 2)Conversepart: Weshallprove(37)and(38)toestablish
we show that R is δ-achievable, which means (35). the converse part of Theorems 3 and 4.
Let R be δ-achievable. Then, there exists a mapping ϕ : The following relation can be obtained from Theorems 3 and
n
1,2,...,M n satisfying (4) and (5). Let γ > 0 be 5, which gives a new characterization for H (X) and
n δ
{ } → X
fixed arbitrarily. From (4), we have
H∗(X):=δp∗-limsup 1 log 1 . (55)
1 logM R+γ (47) δ n→∞ n PXn(Xn)
n
n ≤
for all sufficiently large n. Fixing an ε [0,1) arbitrarily, we Theorem 6: For any general source X,
ochuotpoustevainayWZ d∈ueB˜tεo(Y˜in)p,uwt Xh˜ere=Y˜{X˜=n∈{=Y˜ϕnn}∞n(U=M1 nd)e}n∞no=te1s. tBhye Hδ(X)=Xˆ∈iB˜nδf(X)H(Xˆ), (56)
using Lemma 2 with c= 1 logM +γ and (47), we have
n n H∗(X)= inf H(Xˆ) (57)
δ
d(PZn,PY˜n) Xˆ∈B˜δ∗(X)
for all δ [0,1), where
1 Wn(Y˜n X˜n) ∈
Pr log | >R+2γ e−nγ (48)
≥ (n PZn(Y˜n) )− B˜δ(X):= Xˆ = Xˆn ∞n=1 : linm→s∞upd(PXn,PXˆn)≤δ ,
(cid:26) (cid:27)
for all sufficiently large n. Since Z B˜ (Y˜), we obtain (cid:8) (cid:9)
∈ ε B˜∗(X):= Xˆ = Xˆn ∞ : liminfd(P ,P ) δ .
1 Wn(Y˜n X˜n) δ n=1 n→∞ Xn Xˆn ≤
linm→s∞upPr(nlog PZn(Y˜|n) >R+2γ)≤ε. (49) Equations(5n6)and(cid:8)(57)i(cid:9)ndicatethatH (X)andH∗(X)oca✷n
δ δ
Since ε [0,1) and Z B˜ (Y˜) have been fixed arbitrarily, be viewed as “smoothed” 0-spectral sup-entropy rates. These
ε
(49) imp∈lies ∈ equationscanalso beprovendirectlyfromthepropertyof the
∗
δ-spectralsup-entropyratesH (X)andH (X),respectively.
R+2γ sup D (W Z X˜). (50) δ δ
ε
≥ ε≥0, || | VI. APPLICATIONOF GENERAL FORMULAS TO
Z∈B˜ε(Y˜) MEMORYLESSSOURCE AND CHANNEL
Since γ > 0 is arbitrary and X˜ B (X,W) follows from Now, let us consider a special case, where and are
δ
∈ X Y
(8), we obtain finite sets and for each n = 1,2, , both Xn and Wn are
···
memoryless with joint probability
R inf sup D (W Z Xˆ), (51)
ε
Xwˆhe=re YXˆˆn=.≥{TYˆhXˆnu∈}s∞nB,=wδ(1Xed,oWebnt)oaZtien∈sε(B≥˜t3hε07(e,Y)ˆ.o)utput vi|a|W| due to input fPoXrnx(x=)W(nx(1y,.|x..),=xn(cid:26)) QQ∈nini==X11nPPXXan21d((xxyii))WW=21(((yyyii1||xx,.ii))..,ffyoonrr)eovd∈ednYnnn,
Bδ∗T(hXe{,pWro})o,fcoofm(p3le8t)inigstahneaplorgoooufsofbtyheuscionngvetrhsee pfaacrtts.X˜ ✷∈ rWwehspe=erect{XivWejlnya}.n∞ndT=hW1eajrse(ojucro=cme1p,Xle2t)el=dyecnh{oaXteranac}tes∞nor=iuz1recdaenbadyndPthXae1cWchha1anninfneenll,
V. SOURCE RESOLVABILITY: REVISITED is odd and by PX2W2 if n is even and are known as one of
WhenthechannelWnisanidentitymapping,theaddressed the simplest examplesfor which S(δ|X,W) and S∗(δ|X,W)
do not coincide in general [3]. Let Y denote the output via
problem reduces to the problem of source resolvability [3], j
where the target distribution is the general source Xn itself. Wj due to input Xj for j = 1,2. The alternative formulas
(23) and (24) are of use to prove the converse parts.
In this case, we denote S(δ X,W) simply by S(δ X).
| | Theorem 7: For any δ [0,1),
For this problem, Steinberg and Verdu´ [10] have shown the ∈
followingtheorem,whichgeneralizestheresolvabilitytheorem S(δ X,W)= max inf I(Xˆ ;Yˆ ), (58)
j j
established by Han and Verdu´ [4] for δ =0: | j=1,2Xˆj∈B0(Xj,Wj)
Theorem 5 (HanandVerdu´ [4], SteinbergandVerdu´ [10]):
S∗(δ X,W)= min inf I(Xˆ ;Yˆ ), (59)
For any target general source X, | j=1,2Xˆj∈B0(Xj,Wj) j j
S(δ|X)=Hδ(X) (δ ∈[0,1)), (52) where Yˆj denotes the output via Wj due to the input Xˆj,
where I(Xˆj;Yˆj) denotes the mutual information between Xˆj and
Yˆ , and we define B (X ,W ):= Xˆ :P =P .
Hδ(X):=δp-linm→s∞upn1 logPXn1(Xn) (53) (PjrIotosfh)ouTlhdebperonooftiicse0dgivthejantitnhjeAc.on(cid:8)stanjt δ dYojes noYˆtja(cid:9)ppear i✷n
is the δ-spectral sup-entropy rate for X. ✷ formulas (58) and (59). This result indicates that the strong
When the channel Wn is an identity mapping, we have converse holds for the memoryless source and channel pair.
I(X;Y)=H(X) because Precisely, for any
1 logWn(Yn|Xn) = 1 log 1 a.s. (54) R< min inf I(Xˆj;Yˆj), (60)
n PYn(Yn) n PXn(Xn) j=1,2Xˆj∈B0(Xj,Wj)
anymappingϕ : 1,...,M n satisfying(7)produces Hence, only the case R = S(δ X,W) is of our interest.
n n
{ }→X |
tY˜hen vdaernioattieosntahledoisuttapnuctevdia(PWYnn,dPuY˜ent)o→inp1ut(Xn˜n→=∞ϕ),(Uwher)e. sSoimlvialbairlliyt,y,wthheencadseiscRus=sinSg∗(tδheXo,pWtim)isisticou(rδp,Rrim)-acrhyaninnteelrerset-.
For an i.i.d. sourceX with X =X =X and a sntatioMnanry | ✷
1 2
memoryless channel W with W = W = W , we obtain the Now, we establish the general formulas for the second-
1 2
following corollary from Theorem 7, which has been proved orderresolvability.Thefollowingtwotheoremscanbeproven
by Watanabe and Hayashi [11]. analogously to Theorems 3 and 4 in the first-order case.
Corollary 1 (Watanabe and Hayashi [11]): For any i.i.d. Theorem 8: Let δ [0,1) and R 0 be fixed arbitrarily.
∈ ≥
input source X and any stationary memoryless channel W, For any input process X and any general channel W,
S(δ X,W)=S∗(δ X,W)= inf I(Xˆ;Yˆ) (61) T(δ,RX,W)= inf I(RXˆ;Yˆ), (67)
| | Xˆ∈B0(X,W) | Xˆ∈Bδ(X,W) |
for every δ [0,1), where Yˆ denotes the output via W T∗(δ,RX,W)= inf I(RXˆ;Yˆ), (68)
induced by in∈put Xˆ. ✷ | Xˆ∈Bδ∗(X,W) |
where Yˆ = Yˆn ∞ denotes the output process via W due
VII. SECOND-ORDER CHANNEL RESOLVABILITY to the input{proce}sns=X1ˆ = Xˆn ∞ , and we define
{ }n=1
We turn to considering the second-order resolution rates 1 Wn(Yn Xn)
[11]. First, we define the second-order achievability. I(RX;Y):=p-limsup log | nR .
Definition 5 ((δ,R)-Channel Resolvability): Let δ [0,1) | n→∞ √n(cid:18) PYn(Yn) − (cid:19)
∈ ✷
and R 0 be fixed arbitrarily. A resolvability rate L is said
≥ We give alternative formulas in the following theorem,
to be (δ,R)-achievable at X if there exists a deterministic
whichcorrespondtoTheorem4onthefirst-orderresolvability
mapping ϕ : 1,...,M n satisfying
n { n}→X rates:
1 Theorem 9: Let δ [0,1) and R > 0 be fixed arbitrarily.
linm→s∞up√n(logMn−nR)≤L, (62) For any input process∈X and any general channel W,
linm→s∞upd(PYn,PY˜n)≤δ, (63) T(δ,R|X,W)=Xˆ∈Biδn(Xf ,W) εs≥u0p, Jε(R|W,Z,Xˆ),
where Y˜n denotes the output via Wn due to the input X˜n = Z∈B˜ε(Yˆ) (69)
ϕ (U ). We define
n Mn T∗(δ,RX,W)= inf sup J (RW,Z,Xˆ),
ε
T(δ,RX,W):=inf L: L is (δ,R)-achievable at X , | Xˆ∈B∗(X,W) ε≥0, |
| { } δ Z∈B˜ε(Yˆ)
which is called the (δ,R)-channel resolvability (at X). ✷ (70)
As in the first-order case, we addressthe relaxed constraint
where Yˆ = Yˆn ∞ denotes the output process via W due
on the variational distance. to input proc{ess X}ˆn==1 Xˆn ∞ , and we define
Definition 6 (Optimistic (δ,R)-Channel Resolvability): Let { }n=1
δ [0,1) and R 0 be fixed arbitrarily. A resolvability J (RW,Z,Xˆ)
∈ ≥ ε |
rate L is said to be optimistically (δ,R)-achievable at X if
there existsa deterministicmappingϕ : 1,...,M n 1 Wn(Yˆn Xˆn)
n { n}→X :=εp-limsup log | nR .
satisfying n→∞ √n PZn(Yˆn) − !
1 ✷
limsup (logM nR) L, (64)
n→∞ √n n− ≤ When the channel is an identity mapping, the problem
addressed here reduces to finding the second-order δ-source
liminfd(P ,P ) δ, (65)
n→∞ Yn Y˜n ≤ resolvability[8].Inthiscase,wedenoteT(δ,RX,W)simply
|
by T(δ,RX). Nomura and Han [8] have established the fol-
where Y˜n denotes the output via Wn due to the input X˜n = |
lowing fundamental theorem, which generalizes the theorem
ϕ (U ). We define
n Mn on the first-order δ-source resolvability given by [4], [10]:
T∗(δ,RX,W) Theorem 10 (Nomura and Han [8]): For any target general
| source X,
:=inf L: L is optimistically (δ,R)-achievable at X ,
{ } S(δ,RX)=Hδ(RX) (δ [0,1)), (71)
| | ∈
called the optimistic (δ,R)-channel resolvability (at X). ✷
where
Remark 4: By definition, it is easily verified that
1 1
H (RX):=δp-limsup log nR .
T(δ,RX,W)= +∞ for R<S(δ|X,W) (66) δ | n→∞ √n(cid:18) PXn(Xn) − (cid:19)
| for R>S(δ X,W). ✷
(cid:26) −∞ |
Since the channel Wn is the identity mapping, we have whereV = Vn ∞ andZ = Zn ∞ . On the otherhand,
{ }n=1 { }n=1
I(RX;Y) = H(RX). The following relation can be ob- because it obviously holds that V B (X,W), we have
δ
| | ∈
tained from Theorems 8 and 10, which gives a new represen-
inf I(Xˆ;Yˆ) I(V;Z). (79)
tation for Hδ(R|X) and Xˆ∈Bδ(X,W) ≤
H∗(RX):=δp∗-limsup 1 log 1 nR . Sinceγ >0isanarbitraryconstant,(75),(78)and(79)imply
δ | n→∞ √n(cid:18) PXn(Xn) − (cid:19) inf I(Xˆ;Yˆ) inf I(Xˆ ;Yˆ ). (80)
1 1
Xˆ∈Bδ(X,W) ≤Xˆ1∈B(X1,W1)
Theorem 11: For any general source X,
n
(ii) For an arbitrary fixed γ > 0, let X be n i.i.d. samples
Hδ(RX)= inf H(RXˆ), (72) from source P satisfying X B(X2,W ) and (75) with
| Xˆ∈B˜δ(X) | X2 n 2 ∈ 2 2
j = 2. Also, let X be n i.i.d. samples from source P
H∗(RX)= inf H(RXˆ) (73) satisfying I(X ;Y )1= 0, where Y denotes the output vXi1a
for all δ [0,1)δ an|d R Xˆ0∈,B˜wδ∗h(Xer)e we d|efine H(RXˆ) = Wan1d (dVuen,toZnin)p=1ut(XX11n2.,YSen2t)(fVorn,eZvenn)1n=. T(hXenn1,,Ywen1)obfotarinodd n
H0(R|Xˆ)∈and H∗(R|Xˆ)≥=H∗0(R|Xˆ). | ✷ E 1 logWn(Zn|Vn) I(X ;Y ) for all n.
Equation (73) as well as (72) can be provendirectly from the (cid:26)n PZn(Zn) (cid:27)≤ 2 2
definitionofthequantitiesonbothsides.Aswasshownin(56)
Again, by the weak law of large numbers, we have
and(57)inthefirst-ordercase,so-calledsmoothingoperations
∗
appear here; both H (RX) and H (RX) are characterized 1 Wn(Zn Vn)
bB˜y∗(HX(R) |cXeˆn)teorefdaagteXδne,rar|lesspoeucrtcieveXlˆy.δin t|he δ-ball B˜δ(X) and nl→im∞Pr(cid:26)nlog PZn(Z|n) >I(X2;Y2)+γ(cid:27)=0(,81)
δ
APPENDIX A indicating that
PROOF OF THEOREM7
I(V;Z) I(X ;Y )+γ. (82)
2 2
≤
1) Direct part:
On the other hand, it holds that V B∗(X,W) because
Without loss of generality, we assume that ∈ δ
inf I(Xˆ1;Yˆ1) inf I(Xˆ2;Yˆ2). (74) linm→i∞nfd(PYn,PZn)≤likm→i∞nfd(PY2k,PZ2k)=0. (83)
Xˆ1∈B(X1,W1) ≥Xˆ2∈B(X2,W2)
Then, we have
n
(i) First, fixγ >0 arbitrarily.Forj =1,2,letX beni.i.d.
j inf I(Xˆ;Yˆ) I(V;Z). (84)
samples from source PXj satisfying Xj ∈B(Xj,Wj) and Xˆ∈Bδ∗(X,W) ≤
I(X ;Y ) inf I(Xˆ ;Yˆ )+γ, (75) Since γ > 0 is an arbitrary constant, (75) with j = 2, (82)
j j j j
≤Xˆj∈B(Xj,Wj) and (84) imply
(wVhner,eZYn)j=de(nXotn1e,sYthn1e) foourtpoudtdvniaaWndj(dVune,Zton)in=put(XXn2j,.YSn2e)t Xˆ∈Biδ∗n(Xf ,W)I(Xˆ;Yˆ)≤Xˆ2∈Bi(nXf2,W2)I(Xˆ2;Yˆ2). (85)
for even n. Since the random variable
2) Converse part:
1 logWn(Zn|Vn) = 1 k logW(Zi|Vi) (76) As was argued in [11], we shall use the method of types
n PZn(Zn) n i=1 PZi(Zi) [2]. The following notation is introduced.
is a sum of independent randomXvariables, where Vn = • Let Px denote the type of x ∈ Xn, i.e., Px(a) denotes
the number of occurrence of symbol a in x.
v(Val1u,eV2sa,t.i.s.fi,eVsn) and Zn = (Z1,Z2,...,Zn), its expected • Let Pxy denote the joint type of (x,y)∈∈XXn×Yn.
E(cid:26)n1 logWPnZ(nZ(nZ|Vn)n)(cid:27)≤I(X1;Y1)+γ for all n. •• dLDieesttfirnPibexuWtthioe(nbs)oetn:s=Yof.Pε-atyPpxic(aal)Wseq(ub|ean)cedsenasote the marginal
The weak law of large numbers guarantees TYn,ε:={y∈Yn :|Py(b)−PY(b)|≤ε, ∀b∈Y},
(86)
1 Wn(Zn Vn)
nl→im∞Pr(cid:26)nlog PZn(Z|n) >I(X1;Y1)+γ(cid:27)=0, TWn,ε(x):={y∈Yn :|Pxy(a,b)−Px(a)W(b|a)|≤ε,
(77)
∀(a,b)∈X ×Y}, (87)
which indicates that AY(ε):={P ∈P(X):|PW(b)−PY(b)|≤2|X|ε,
I(V;Z) I(X1;Y1)+γ, (78) ∀b∈Y}. (88)
≤
Now, we are in a position to prove the converse part of We invoke the method of squeezing a subsequence of good
Theorem 7. We again assume (74) without loss of generality. types in the information spectrum approach as in [12]. Equa-
In view of Theorems 3 and 4, we shall show tion (95) implies that there exists some d >0:d >d >
n 1 2
{
δ satisfying
inf sup D (W Z Xˆ) ···→ }
ε
Xˆ∈Bδ(X,W) ε≥0, || | 1 Wn(Yˆn Xˆn)
inZf∈B˜ε(YˆI)(Xˆ ;Yˆ ), (89) dn ≥Pr(nlog PYn(Yˆ|n) >R+2γ) (97)
1 1
≥Xˆ1∈B(X1,W1) for all n=1,2, . Since
···
and 1 Wn(Yˆn Xˆn)
Pr log | >R+2γ
inf sup Dε(W Z Xˆ) (n PYn(Yˆn) )
Xˆ∈B∗(X,W) ε≥0, || |
δ
Z∈B˜ε(Yˆ) = P (x)Wn 1 logWn(Yˆn|x) >R+2γ ,
inf I(Xˆ2;Yˆ2). (90) x∈Xn Xˆn x (n PYn(Yˆn) )
≥Xˆ2∈B(X2,W2) X
where we use Wn to denote Wn( x) for simplicity, (97)
(i) To show (89), we first fix an arbitrary indicates that therexexists some x ·| n satisfying
n
∈X
R>Xˆ∈Biδn(Xf ,W)Z∈εsB≥˜uε0p(,Yˆ)Dε(W||Z|Xˆ), (91) dn ≥Wxnn(n1 logWPnY(nYˆ(nYˆ|xn)n) >R+2γ). (98)
and we shall show that R is not smaller than the right-hand It is important to use the fact following from (98) that there
side of (89). For simplicity, we define existsa sequenceofoddnumbers n <n < and
1 2
{ ···→∞}
P ( ) such that
1 if n is odd γ ∈P X
j(n)= (92)
2 if n is even. limsupd δ, lim P =P , (99)
(cid:26) i→∞ ni ≤ i→∞ xni γ
Then, we can write Xn = Xn and Wn = Wn and the
j(n) j(n) where P denotes the type of x n (cf. [12]). The
corresponding output is Yn = Yn . Letting γ > 0 be arbi- xn n ∈ X
j(n) existenceofsuchaconvergentpointP ( )followsfrom
atrnadrilsyetfitxheed,fowlleowdeinfignepγro′b:a=bi|lXity|γd,iτstnri:b=utiPorn{Yonn ∈nT:Ynj(n),γ′} thefactthatP(X)isacompactsetforγfi∈nitPeXX.Fornotational
Y simplicity, we use k to denote (odd number) k = n1,n2,
···
P (y)1 y Tn so that (98) and (99) can be rewritten as
PYn(y):= Yn {τn∈ Yj(n),γ′} (y ∈Yn), (93) 1 Wk(Yˆk x )
d Wk log | k >R+2γ (100)
where 1{E} is the indicator function for the event E. Then, k ≥ xk(k PYk(Yˆk) )
from the property of the set of γ′-typical sequences Tn ,
Yj(n),γ′ and
we have τ 1 as n and hence
n
→ →∞
limsupd δ, lim P =P , (101)
nl→im∞d(PYn,PYn)=0. (94) k→∞ k ≤ k→∞ xk γ
respectively. The following lemma is of use.
Now, we can see that by (91) there exists an Xˆ
Bδ(X,W) satisfying ∈ (10L0e)mamnad3(1:0A1s)swumithe tshoamtexkδ∈X[0,k1()ka=ndnP1,n2,··(·))s,awtishfieeres
γ
∈ ∈P X
R> sup Dε(W Z Xˆ) γ k = n1,n2,··· denotes either odd or even numbers. If k
ε≥0, || | − denotes odd numbers, then
Z∈B˜ε(Yˆ)
P A (2γ), (102)
D (W Y Xˆ) γ, (95) γ ∈ Y1
δ
≥ || | −
whereas if k denotes even numbers, then
where Yˆ = Yˆn ∞ denotes the output via W due to input
XYˆ =B{˜Xˆ(nYˆ}∞n){=w1h,i}acnnhd=f1toolldoewrisvefr(o9m5)XwˆehaBve(uXse,dWth)atafnadct(t9h4a)t Pγ ∈AY2(2γ). (103)
with∈theδtriangle inequality: ∈ δ (Proof) Let k = n1,n2,··· denote odd numbers. Suppose
that P A (2γ). From the right inequality in (101) we
γ 6∈ Y1
linm→s∞upd(PYˆn,PYn) o[1b1ta,iLnePmxmka6∈2]AhYa1v(eγ)shfoowrnaltlhalatrigfeyk∈. WTWakt1a,nγa(bxek)a,ntdheHnayahi
≤nl→im∞d(PYn,PYn)+linm→s∞upd(PYn,PYˆn)≤δ. (96) y 6∈TYk1,γ′. (104)
Further, if y Tk , then P (y) = 0 by definition, and the weak law of large numbers and under the conditional
thus 6∈ Y1,γ′ Yk probability distribution Wk , yielding
xk
k1 logWPkY(ky(|yx)k) >R+2γ, (105) likm→s∞upWxkk(k1 logWPkY(kYˆ(kYˆ|xk)k) >R+3γ)
Therefore, for all y k we have
0 if D(W P P )<R+3γ,
∈Y = 1|| Y1| γ (111)
1 if D(W P P )>R+3γ
1 Wk(y x ) (cid:26) 1|| Y1| γ
1 y Tk (x ) 1 log | k >R+2γ .
∈ W1,γ k ≤ (k PYk(y) ) and from the left inequality in (101) and (108), we obtain
(cid:8) (cid:9)
R+3γ D(W P P ). (112)
Since the set of γ-typical sequences TWk1,γ(xk) satisfies ≥ 1|| Y1| γ
Since γ >0 is arbitrary, taking the limit γ 0 for both sides,
Wxkk Yˆk ∈TWk1,γ(xk) →1 (k →∞), (106) we obtain ↓
n o
this inequality and (100) leads to R limD(W P P )
≥ γ↓0 1|| Y1| γ
kl→im∞dk =1, (107) =D(W1||PYˆ1|PXˆ1)=I(Xˆ1,Yˆ1) (113)
which is a contradiction, and hence (102) holds. with some Xˆ B(X ,W ), where Yˆ denotes the output
1 1 1 1
In the case of even numbers k = n ,n , , (103) can be via W due to ∈input Xˆ . Here, we have used the fact that
1 2 1 1
proven analogously. ··· ✷ P A (γ) by Lemma 3 and A (γ) B(X ,W ) as
γ ∈ Y1 Y1 → 1 1
Since P (y) P (y)/τ for all y k, we can bound γ 0. Thus, we have
Yk ≤ Yk k ∈Y ↓
the right-hand side of (100) from below as
R inf I(Xˆ ;Yˆ ), (114)
1 1
1 Wk(Yˆk x ) 1 1 ≥Xˆ1∈B(X1,W1)
d Wk log | k >R+2γ+ log
k ≥ xk(k PYk(Yˆk) k τn) completing the proof of (89).
(ii) To show (90), we first fix an arbitrary
1 Wk(Yˆk x )
≥Wxkk(k log PYk(Yˆ|k)k >R+3γ) (k ≥k0), R>Xˆ∈Bi∗n(Xf ,W) εs≥u0p, Dε(W||Z|Xˆ). (115)
(108) δ Z∈B˜ε(Yˆ)
where the second inequality holds for all large odd numbers Recall that we can write Xn = Xn and Wn = Wn
j(n) j(n)
k. Since the random variable and the corresponding output is Yn = Yn with definition
j(n)
(92). Let γ > 0 be arbitrarily fixed. We define γ′ := γ,
1 logWk(Yˆk|xk) = 1 k logW1(Yˆi|xk,i) (109) τn :=Pr{Yn ∈TYnj(n),γ′}andsetPYn againasin(93).T|Xhe|n,
k PYk(Yˆk) k i=1 PY1(Yˆi) from the property of the set of γ′-typical sequences TYnj(n),γ′,
X we have (94).
is a sum of conditionallyindependentrandomvariables given Now, for any general source Xˆ = Xˆn ∞ it is easily
Xˆk = x = (x ,x ,...,x ), its expected value can be verified that { }n=1
k k,1 k,2 k,k
evaluated as
sup D (W Z Xˆ) sup D∗(W Z Xˆ), (116)
1 Wk(Yˆk x ) ε≥0, ε || | ≥ ε≥0, ε || |
E(k log PYk(Yˆ|k)k Xˆk =xk) Z∈B˜ε(Yˆ) Z∈B˜ε∗(Yˆ)
(cid:12)
(cid:12) where Yˆ = Yˆn ∞ denotes the output via W due to input
1 k (cid:12) W1(bxk,i) Xˆ and we d{efine}n=1
= W (bx )log |
1 k,i
k | P (b)
Xi=1bX∈Y Y1 D∗(W Z Xˆ):=εp∗-limsup 1 logWn(Yˆn|Xˆn),
1 W1(bxk,i) ε || | n→∞ n PZn(Yˆn)
= kP (a) W (bx )log |
k xk 1 | k,i P (b)
aX∈X bX∈Y Y1 B˜ε∗(Y):= Z ={Zn}∞n=1 : linm→i∞nfd(PYn,PZn)≤ε
=:D(W1||PY1|Pxk), (110) since for all εn>0 and Z it holds that o
whereD(W P P )istheconditionaldivergencebetween
1|| Y1| xk D (W Z Xˆ) D∗(W Z Xˆ). (117)
W1 and PY1 given Pxk ∈ P(X). Then, we can invoke ε || | ≥ ε || |
We can see that by (115) and (116) there exists an Xˆ REFERENCES
∈
B∗(X,W) satisfying
δ [1] M.R.BlochandJ.N.Laneman,“Strongsecrecyfromchannel resolv-
R> sup D∗(W Z Xˆ) γ ability,” IEEETrans.Inf.Theory, vol.59,no.12,pp.8077–8098,Dec.
ε || | − 2013.
ε≥0,
Z∈B˜∗(Yˆ) [2] I. Csisza´r and J. Ko¨rner, Information Theory: Coding Theorems for
ε Discrete Memoryless Systems, 2nd ed., Cambridge University Press,
D∗(W Y Xˆ) γ (118) Cambridge, U.K.,2011.
≥ δ || | − [3] T. S. Han, Information-Spectrum Methods in Information Theory,
Springer, 2003.
wwhheicrhetofodlleorwivsef(r1o1m8)wXˆehaveBu∗s(eXdt,hWat)facatntdha(t9Y4)∈wBi˜thδ∗(Ytˆhe) [4] ITE.ESE. THraannsa.nIdnf.ST.hVeeorrdyu´,,v“oAl.p3p9ro,xniom.a3ti,opnp.th7e5o2r–y77o1f,oMutapyut19st9a3ti.stics,”
∈ δ [5] M.Hayashi,“Generalnonasymptoticandasymptoticformulasinchan-
triangle inequality:
nelresolvability andidentification capacity andtheirapplication tothe
wiretapchannel,” IEEETrans.Inf.Theory,vol.52,no.4,Apr.2006.
linm→i∞nfd(PYˆn,PYn) [6] M. Hayashi, “Information spectrum approach to second-order coding
rateinchannel coding,”IEEETrans.Inf.Theory,vol.55,no.11,Nov.
≤nl→im∞d(PYn,PYn)+linm→i∞nfd(PYn,PYˆn)≤δ. (119) [7] 2M0.0H9.ayashiandH.Nagaoka,“Generalformulasforcapacityofclassical-
Equation (118) implies that there exists some d > 0 ∞ quantum channels,” IEEETrans.Inf.Theory, vol.49,no.7,pp.1753–
{ n }n=1 1768,July2003.
satisfying [8] R. Nomura and T. S. Han, “Second-order resolvability, intrinsic ran-
domness,andfixed-lengthsourcecodingformixedsources:information
liminfdn δ (120) spectrum approach,” IEEETrans.Inf.Theory,vol.59,no.1,pp.1–16,
n→∞ ≤ Jan.2013.
and [9] Y. Steinberg, “New converses in the theory of identification via chan-
nels,”IEEETrans.Inf.Theory,vol.44,no.3,pp.984–997,May1998.
1 Wn(Yˆn Xˆn) [10] Y. Steinberg and S. Verdu´, “Simulation of random processes and rate-
dn ≥Pr(nlog PYn(Yˆ|n) >R+2γ) (121) Jdaisnt.or1t9io9n6.theory,” IEEE Trans. Inf. Theory, vol. 42, no. 1, pp. 63–86,
[11] S. Watanabe and M. Hayashi, “Strong converse and second-order
for all n=1,2, . Also, (120) indicates that at least one of asymptotics of channel resolvability,” Proc. IEEE Int. Symp. on Inf.
···
the following inequalities holds: Theory,Jun.2014.
[12] H. Yagi, T. S. Han, and R. Nomura, “First- and second-order coding
liminfd δ or liminfd δ. (122) theorems formixedmemoryless channels with general mixture,” IEEE
2k+1 2k
k→∞ ≤ k→∞ ≤ Trans.Inf.Theory,vol.68,no.8,pp.4395–4412, Aug.2016
[13] H. Yagi and T. S. Han, “Variable-length resolvability for general
First, we assume that
sources,”submittedtoIEEEInt.Symp.onInf.Theory,Jan.2017.
liminfd δ (123)
k˜→∞ k˜ ≤
for odd k˜ = 1,3, . Similarly to the derivation of (100)
···
and (101), (118) indicates that there exists some x n, a
n
∈X
sequence k =n ,n , , where n ,n , are odd numbers,
1 2 1 2
··· ···
and P ( ) such that
γ
∈P X
1 Wk(Y˜k x )
d Wk log | k >R+2γ . (124)
k ≥ xk(k PYk(Y˜k) )
and
lim d δ, lim P =P . (125)
k→∞ k ≤ k→∞ xk γ
From Lemma 3, we have
P A (2γ). (126)
γ ∈ Y1
Then, we can invoke the weak law of large numbersas in the
derivation of (114) to yield
R inf I(Xˆ ;Yˆ )
1 1
≥Xˆ1∈B(X1,W1)
inf I(Xˆ ;Yˆ ), (127)
2 2
≥Xˆ2∈B(X2,W2)
where we have used (74) for the last inequality. Thus, we
obtain (90).
In the case where (123) holds for even k˜ = 2,4, , we
can show (127) in the analogous way. ··· ✷