On the complexity of computing the capacity of codes that avoid forbidden difference patterns

0.14 MB·English

by  Vincent D. Blondel

1
On the complexity of computing the capacity of
codes that avoid forbidden difference patterns
Vincent D. Blondel, Raphae¨l Jungers, and Vladimir Protasov
Abstract—We consider questions related to the computation avoid the forbiddenpatterns in D. If the set D is empty, then
ofthecapacityofcodesthatavoid forbiddendifferencepatterns. there are no forbidden patterns and δ (D) = 2n. When D
The maximal number of n-bit sequences whose pairwise differ- n
is nonempty, then δ (D) grows exponentially with the word
06 einnccreesasdeos nexoptocnoennttaiianllysowmitehgniv.eTnhfeorebxipdodneenntdiifsfetrheencceappaactitteyrnosf length n and is asynmptotically equal to 2cap(D)n where the
0 the forbidden patterns, which is given by the logarithm of the scalar 0 cap(D) 1 is the capacity of the set D. The
≤ ≤
2 joint spectral radius of a set of matrices constructed from the capacity is thus a measure of how constraining a set D is;
n forbiddendifferencepatterns.Weprovideanewfamilyofbounds the smaller the capacity, the more constraining the forbidden
that allows for the approximation, in exponential time, of the
a difference patterns are.
J capacitywitharbitraryhighdegreeofaccuracy.Wealsoprovide As an illustration consider the set of forbidden patterns
a polynomial time algorithm for the problem of determining if
0 the capacity of a set is positive, but we prove that the same D = + ,++ . Differences between two words in C =
1 { − }
problem becomes NP-hard when the sets of forbidden patterns u 0u 0 0u :u 0,1 will have a “0” in any succes-
{ 1 2 ··· k i ∈{ }}
are defined over an extended set of symbols. Finally, we prove sionoftwocharactersandwillthereforenotcontainanyofthe
]
T theexistenceofextremalnormsforthesetsofmatricesarisingin forbidden patterns. From this it follows that δ 2 n/2 and
thecapacitycomputation.Thisresultmakesitpossibletoapplya n ≥ ⌈ ⌉
I so cap(D) 1/2. One can show in fact that cap(D) = 1/2.
. specific (even though non polynomial) approximation algorithm. ≥
s Weillustratethisfactbycomputingexactlythecapacityofcodes Thisfollowsfromthenextpropositiontogetherwiththesimple
c
that were only known approximately. observation that the capacity of the set D = + ,++ is
[
{ − }
identical to the capacity of the set D = + ,++, +, .
Index Terms—Capacity of codes, joint spectral radius.
1 Proposition 1: Thecapacityoftheset{ +−, m i−sgiv−en−b}y
v { −}
(m 1)/m.
36 I. INTRODUCTION −Proof: Let Ckm be a code of length km avoiding D.
0 In certain coding applications one is interested in binary In any given window of length m, the set of words appearing
1 codes whose elements avoid a set of forbidden patterns. In cannot contain both u and u¯ (we use u¯ to denote the word
0 order to minimize the error probability of some particular obtainedbyinvertingtheonesandthezerosinu).Thisimplies
6
magnetic-recording systems, a more complicated problem that there are at most 2m 1 different words in any given
0 −
/ ariseswhenitisdesirabletofindcodewordswhosedifferences window of size m. Let us now consider words in Ckm as
s avoid forbidden patterns. a concatenation of k words of length m. There are at most
c
v: andLelett{u0,,v1}n d0e,n1oten.thTehesedtifofferwenocrdesuofvleinsgathwnorodvoefrl{e0n,g1t}h c2o(mns−id1)ekr twhoercdosdien Ckm and so cap(D) ≤ (m−1)/m. Now
i ∈{ } −
X n over 1,0,+1 (as a shorthand we shall use ,0,+
r instead {o−f 1,0,}+1 ). This difference is obtaine{d−from u} Ckm ={z10z20···0zk :zi ∈{0,1}m−1}. (1)
a and v by sy{m−bol-by-s}ymbolsubtraction so that, for example, Thiscodesatisfiestheconstraints,andsothebound(m 1)/m
0110 1011= +0 .Considernowa finite set D ofwords is reached. −
− − −
over ,0,+ ;wethinkofD asa setofforbiddendifference Thecomputationofthe capacityisnotalwaysthateasy.As
{− }
patterns. A set (or code) C 0,1 n is said to avoid the set an example it is proved in [15] that the capacity of +++
D if none of the difference⊆s o{f wo}rds in C contain a word isgivenbylog ((1+(19+3√33)1/3+(19 3√33)1{/3)/3)=}
fromD assubword,thatis,noneofthedifferencesu v with .8791... and t2he same reference provides−numerical bounds
−
u,v C can be written as u v =xdy for d D and some forthecapacityof 0+ + forwhichnoexplicitexpression
(poss∈ibly empty) words x and−y over ,0,+∈. is known. { − }
{− }
Weareinterestedinthelargestcardinality,whichwedenote The capacity of codes that avoid forbidden difference pat-
by δn(D), of sets of words of length n whose differences terns was first introduced and studied by Moision, Orlitsky
and Siegel. In [15], these authors provide explicit values for
V. Blondel and R. Jungers are with the Department of Mathematical
Engineering, Universite´ catholique de Louvain, Avenue Georges the capacity of particular sets of forbidden patterns and they
Lemaitre 4, B-1348 Louvain-la-Neuve, Belgium, {jungers, provethat,ingeneral,thecapacityofaforbiddensetDcanbe
blondel}@inma.ucl.ac.be. Their work is supported by a
obtainedasthelogarithmofthejointspectralradiusofasetof
grant ARC”Communaut franaise deBelgique”. Raphae¨l Jungers isaFNRS
fellow (Belgian FundforScientific Research). matrices with binary entries. The joint spectral radius, which
V.ProtasoviswiththedepartmentofMechanicsandMathematics,Moscow we formally define below, is a quantity that quantifies the
State University, Vorobyovy Gory, Moscow, 119992, Russia, vladimir
maximalasymptoticgrowthrateofproductsofmatricestaken
[email protected]
05-01-00066andbythegrant304.2003.1fortheleadingscientific schools. from a set. This quantity is notoriously difficult to compute
2
and approximate. It is known in particular that the problem computationproblemsthat are polynomialtime solvable from
of computing,or evenapproximating,the jointspectralradius those that are not. We do however not provide in this paper
of two matrices with binary entries is a problem that is NP- an answer to the question, which was the original motivation
hard [20] and that the problem of determining if the joint for the research reporter here, as to whether or not one
spectral radius of two matrices with nonnegative entries is can compute the capacity of sets of forbidden patterns over
greater than one is undecidable [6]. Moreover, the size of the ,0,+ in polynomial time. This interesting question that
{− }
matrices constructed in order to compute the capacity is not was already raised in [15], remains unsettled.
polynomial in the size of the forbidden set D : if the length
of the forbidden words is m, the dimension of the matrices II. CAPACITY AND JOINTSPECTRAL RADIUS
is 2m 1 2m 1. Hence, even constructing the matrices is
− × − Let D be a set of forbidden patterns over the alphabet
an operation that cannot be performed in polynomial time.
,0,+ and consider for any n 1 the largest cardinality,
However,as pointed out in [15], the matrices that arise in the {− } ≥
denotedbyδ (D),ofsetsofwordsoflengthnwhosepairwise
n
contextofcapacitycomputationhaveaparticularstructureand
differencesavoidtheforbiddenpatternsin D. Thecapacityof
so these negativeresults do notrule outthe possibility for the
D is defined by
capacity to be computable in polynomial time.
We provide several results in this paper, all are related to cap(D)= lim log2δn(D). (2)
the capacity computation and its complexity. n n
→∞
Wefirstprovidenewboundsthatrelatethecapacityofaset Moision et al. show in [15] how to represent codes as
of forbiddenpatternsD with the valuesδn(D), the maximum products of matrices. Associated to any set D of forbidden
sizeofanycodeoflengthnavoidingD.Theseboundsdepend patterns,theyconstructafinitesetΣ(D)ofmatricesforwhich
on parameters that express the number and positions of zeros
δ =max A ...A :A Σ(D) . (3)
in the patterns of D. These new bounds allow us to compute m−1+n {k 1 nk i ∈ }
the capacity of any set to any given degree of accuracy
In this expression, the matrix norm used is the sum of
by numerically evaluating δ (D) for increasing values of
n the absolute values of the matrix entries. The matrices they
n. The approximation algorithm resulting from these bounds
construct are of dimension 2m 1 2m 1 and have binary
− −
has exponential growth but provides an a-priori guaranteed ×
entries.Forsakeofconciseness,wedonotreproduceherethe
precision, and so with this algorithm the computationaleffort
explicit construction of Σ(D) but refer instead the interested
requiredtocomputethecapacitytoagivendegreeofaccuracy
reader to [15] for more details. Combining (3) and (2) we
can be evaluated before the calculations are actually started.
deduce that
Asanexample,itfollowsfromtheboundsweprovidethatthe
log δ (D)
capacityofasetofforbiddenpatternthatdoesnotcontainany cap(D) = lim 2 n
0s can be computed with an accuracy of 90% by evaluating n→∞ n
log δ (D)
δn(D) for n=10 (see Corollary 2 below). = lim 2 m−1+n
In a subsequent section, we provide explicit necessary and n m 1+n
→∞ −
log max A ...A
sufficientconditionsforasettohavepositivecapacityandwe = lim 2 Ai∈Σk 1 nk
use this condition for producing a polynomial time algorithm n n
→∞
thatdecideswhetherornotthecapacityofasetispositive.As = log2nlim AmiaxΣkA1...Ank1/n
explainedabove,thecapacityofasetisgivenbythelogarithm →∞ ∈
of the joint spectral radius of a set of matrices constructed The quantity limn→∞maxAi∈ΣkA1...Ank1/n appearing
from the forbidden patterns. Our polynomial time algorithm in the last identity is a joint spectral radius. For any compact
therefore provides a procedure for checking whether or not set of matrices A, the joint spectral radius of A is defined by
the joint spectral radius of these matrices is larger than one.
ρ(A)=limsup sup A ...A 1/n.
This problem is known to be NP-hard for general matrices. k 1 nk
n Ai A
We then consider the situation where in addition to the →∞ ∈
Hence we have the fundamental relation :
forbidden symbols ,0 and + the forbidden patterns in D
−
may also include the symbol , where stands for both the cap(D)=log ρ(Σ(D)).
± ± 2
symbol + and . We prove that in this case the problem of
− The joint spectral radius of a set of matrices is a quantity
computing the capacity, or even determining if this capacity
that was introduced by Rota and Strang [19] and that has
is positive, becomes NP-hard.
receivedintenseresearchattentioninthelastdecade.Formore
Finally, we show an algebraic property of the sets of
references on the joint spectral radius, consult the survey [5].
matrices constructed in order to compute the capacity: there
exists always an extremal norm for this set. This theoretical
result makes it possible to apply specific algorithms in order III. UPPERAND LOWERBOUNDS
to compute the joint spectral radius. These methods, although Inthissection,wederiveboundsthatrelatethecapacityofa
non-polynomial,can be more efficient than general ones, and setD with δ (D). ConsidersomesetD offorbiddenpatterns
n
we use them for computing the capacity of some specific and denote by r (respectively r ) the maximal k for which
1 2
forbidden sets. 0k is the prefix (respectivelysuffix) of some pattern in D. No
These results allow us to better delineate the capacity pattern in D begins with more than r zeros and no pattern
1
3
in D ends with more than r zeros. We also denote by r the of size δ = 2m 1. It happens that for any given length n
2 n −
maximal number of consecutive zeros in any pattern in D; this size is maximum. Otherwise, there must be two different
obviously, r max(r ,r ). In the next theorem we provide words u and v whose prefixes of length k coincide. In order
1 2
≥
upper and lower bounds on the capacity in terms of δ (D). to avoid theforbiddenpattern,the k+1-thsymbolsmustalso
n
As n increases these bounds converge to the same identical be equal, and so on. But then both words are equal, and we
limit, cap(D), and so the boundscan be used to approximate have reached a contradiction.
the capacity to any desired degree of accuracy. Corollary 1: LetD begivenandletr,r andr bedefined
1 2
Theorem 1: For any n r +r we have as above. Then
1 2
≥ cap(D) log2δn 1 max(r +r ,r+1)+cap(D).
log2δn(D)−(r1+r2) cap(D) log2δn(D). (4) Proof:≤Simnply≤usne the fac1t tha2t the capacity is always
n+r+1 (r +r ) ≤ ≤ n
Proof: L−et us1 first2consider the upper bound. The fol- less than one.
lowingequationisstraightforward,givenanypositiveintegers Wemayspecializethesegeneralboundstosetsofparticular
k,n, and any set of forbidden patterns D: interest.
Corollary 2: LetD begivenandletr,r andr bedefined
1 2
δkn ≤δnk. (5) as above. Then
Indeed, considering any word of length kn as the concatena- 1) If cap(D) = 0 the size of any code avoiding D is
tion of k subwords of length n, for each of these subwords bounded above by the constant 2r1+r2.
we have at most δ possibilities. Taking the 1 th power of 2) If the patterns in D contain no zeros, then
n kn
both sides of this inequality and taking the limit k , we
obtain : →∞ ncap(D)≤log2δn(D)≤(n+1)cap(D).
3) If none of the patterns in D start or end with a zero,
2cap(D) δ1/n
≤ n then ncap(D) log δ (D) (n+r+1)cap(D).
ρn δ . ≤ 2 n ≤
n Example 1: We do not know the capacity of the set D =
≤
Now let us consider the lower bound. The optimal code of +0+0+ with a good precision, but by Corollary 4 below,
{ }
lengthncontainsatleast 2−r1−r2δn(D) wordsthatcoincide we know that it is positive. In this case r1 = r2 = 0, r = 1,
⌈ ⌉ and hence,
in the first r bits and in the last r bits (because there are in
1 2
total 2r1+r2 differentwords of length r1+r2). Denote the set ρn ≤δn ≤4ρn.
of strings of all these words from (r +1)st bit to (n r )th Example 2: Consider the set D = ++ . A approxi-
bitby C′. This set containsat least 12−r1−r2δn(D) d−iffe2rent mate computation of the joint spectral{radiu−s}leads to ρ =
words of length n r r . Then ⌈for any l 1 th⌉e code 20.8113... <1.755, and thus, by Corollary 2,
1 2
− − ≥
C =(cid:8)u10r+1u20r+1···0r+1ul0r+1, uk ∈C′, k =1,...,l(cid:9) 20.8113n ≤δn ≤ 1.755 20.8113n.
avoids D. The cardinality of this code is at least
2−r1−r2δn(D) l and the length of its words is N = IV. POSITIVE CAPACITY CAN BE DECIDED IN POLYNOMIAL
⌈ ⌉ TIME
l(n r r +r+1). Therefore, for any l we have
1 2
− −
As previously seen by a direct argument, the capacity of
δN(D) ≥ ⌈2−r1−r2δn(D)⌉l. the set 0m−1+ is equal to zero. In this section we provide
{ }
Takingthe power1/N of both sides of this inequality,we get a systematic way of deciding when the capacity of a set is
equaltozero.Wefirstprovideasimplepositivitycriterionthat
1/N
hδN(D)i 2−r1−r2δn(D) 1/(n−r1−r2+r+1), can be verified in finite time and then exploit this criterion
≥ ⌈ ⌉ for producing a positivity checking algorithm that runs in
which as N yields polynomial time. In the sequel we shall use the notation D
→∞ −
to denote the set of elements that are the opposites to the
ρ ≥ ⌈2−r1−r2δn(D)⌉1/(n−r1−r2+r+1). elements of D, for example if D = + 0,0 then
{− − −}
Now after elementary simplifications we arrive at the lower D= + 0,0++ .
− { − }
bound on cap(D). Theorem 2: LetD bea setof forbiddenpatternsoflengths
These bounds allow to design an algorithm in order to atmostm.Thencap(D)>0ifandonlyifthereexistsaword
computethecapacityonlybyevaluatingthesuccessivevalues on the alphabet +, ,0 that does not contain any word of
{ − }
δ . For an efficient (although non-polynomial) method of D D as subword and that has a prefix 0m and a suffix
n
∪−
computation of δ , see [13]. +0m 1.
n −
Both bounds in this theorem are sharp in the sense that Proof: If the capacity is positive, then for sufficiently
they are both attained for particular sets D. The upper bound large n there is a code avoiding D of size 22m 1 + 1.
−
≥
is attained for the set D = and the lower bound is attained, This code has at least two words u,v with the same m-bit
∅
for instance, for the set D = 0m 1+ . Indeed, in this case prefixand(m 1)-bitsuffix(becausethereareintotal22m 1
− −
{ } −
r = r = m 1,r = 0 and cap(D) = 0, while δ = 2m 1 different words of the length 2m 1). Taking the difference
1 2 n −
− −
for n m 1. Here is a direct proof of this equality, drawn u v and removing,if necessary, severallast zeroswe get an
≥ − −
from[15]:Clearly,foralln>m 1,wecanconstructacode admissible string with a prefix 0m and a suffix 0m 1. If the
−
− ±
4
first bit in the suffix is +, then the proof is completed, if it is constructedgraphispolynomialin thesize andthenumberof
, then we apply the same reasoning to v u. theforbiddenpatterns.Letusnowdenoteq0m thestatereached
− −
Conversely,suppose there exists a feasible string d of +, after entering the word 0m. This state is well defined since
−
and 0 of some length n. Clearly, n 2m. Let u ,u 0m doesn’t contain any forbidden pattern, and hence no state
1 0
≥ ∈
0,1 n 2m+1bebinarywords,forwhich0m(u u )0m 1 = reachedafterenteringanyprefixofthestring0m wasremoved
− 1 0 −
{ } −
d. Then for any l ≥1 the code fromtheprimaryautomaton.We also denoteq+0m−1 the state
corresponding to the suffix +0m 1 for the entered text (i.e.
C = u 0m 1 u 0m 1, i 0,1 , k =1,...,l (6) −
(cid:8) i1 − ··· il − k ∈{ } (cid:9) the acceptingstate correspondingto the pattern+0m−1 in the
avoids D. The cardinality of this code is 2l. Hence δ (D) is Aho-Corasickautomaton).Wehavethefollowingcriterionfor
n
unboundedandweconcludefromCorollary1thatthecapacity zero-capacity:
is positive. Theorem 3: The capacity of a set D is positive if and only
Corollary 3: If every word in D contains at least two ifthereisa pathfromq0m toq+0m−1 inthegraphconstructed
nonzero symbols, then cap(D)>0. above.
Proof: For any such set the word d = 0m +0m 1 is Proof: If cap(D) > 0, by Theorem 2, there exists a
−
admissible, and by Theorem 2 the capacity is positive. wordd,beginningwithmzeros,andendingwith+0m 1,that
−
Corollary 4: If D consists of one forbidden pattern p of avoids D D. Hence, entering this word in the automaton,
∪−
lengthm, then its capacityis zero if andonly if p hasat least thefinitestate willbe(welldefinedandwillbe)thevertexla-
m 1 consecutive zeros. beled+0m 1, because the verticesremovedfromthe original
−
−
Proof: If a pattern p is 0m or +0m 1, then obviously automaton of Aho-Corasick do not make any problem, since
−
therearenoadmissiblestrings,andbyTheorem2thecapacity we do not reach the vertices labeled with forbidden patterns.
is zero. The same holds for 0m 1, since this is the negation On the other hand, a path in the constructed graph represents
−
−
of+0m 1 andfor0m 1 becauseofthesymmetry.Inallthe anacceptableword,sinceitdoesn’tpassthroughanyremoved
− −
±
other cases the admissible string exists and so cap(D) > 0. vertex, and hence no suffix of any prefix of this word will be
Indeed, if p has a unique nonzero bit, then the word d = in the forbidden set.
0m++0m 1 is admissible, if it has at least two nonzerobits, Moreover, a shortest path will give the shortest acceptable
−
then the proof follows from Corollary 3. word, since the length of the path is equal to the length of
We now prove the polynomial-time solvability of the the represented word.
problem of determining whether the capacity of a set D Corollary 5: The problem of determining whether or not
is positive. The proof is constructive and is based on the the capacity of a given set of forbidden patterns is positive
so-called Aho-Corasick automaton that checks whether a can be solved in polynomial time.
given text contains as a subsequence a pattern out of a given Proof: Aho shows in [1] that the automaton is con-
set [1]. Let P be a given set of patterns. The transition graph structible in polynomial time. The determination of the state
of the Aho-Corasick automaton for the set P is defined as q0m and the computation of the shortest path are obviously
follows : First, construct the retrieval tree, or trie, of the set polynomially feasible.
P. The trie of P is the directed tree of which each vertex Corollary 6: If for a set D of forbidden patterns there are
has a label representing a prefix of a pattern in P, and all admissiblewords,thenthelengthofashortestadmissibleword
prefixes are represented, including the patterns themselves. does not exceed 2M +2m, where m is the maximal length
The rootof the tree has the empty string as label. Edgeshave of all patterns in D and M is the sum of the lengths of each
a label too, which is a symbol of the used alphabet. There forbidden pattern.
is an edge labeled with the symbol a from a vertex s to a Proof: The number of vertices of the graph does not
vertex t if t is the concatenation sa. exceed 2M +m+1. Indeed, for each pattern of length l in
In order to have an automaton, we complete the trie by D D we add to the automatonatmost l states, since there
∪−
adding edges so that for each vertex s, and each symbol are no more than l prefixes of this pattern. We still add the
a, there is an edge labeled a leaving s. This edge points pattern +0m 1 (maximum m new states), and the root. If
−
{ }
to the vertex of the trie of which the label is the longest there is a path connecting two given vertices, this path can
suffix of the concatenation sa. Note that this vertex can be chosen so that its length (in terms of number of vertices)
be the root (that is, the empty string) if no vertex in the will not exceed the total number of vertices (if it does not
trie is a suffix of sa. Finally, the accepting states of the pass through the same vertex twice). Every edge of this path
automaton are the vertices whose labels are patterns of adds one bit to the admissible string. The initial length of the
P. This automaton accepts words that contain a pattern in string is m (we start from 0m), therefore the total length of
P andhaltswheneverthispatternisasuffixoftheenteredtext. the admissible word is at most 2M +2m.
Proposition 2: If the capacity is positive, then cap(D) >
If 0k D or +0k D, k m, then, by Theorem 2, 1/(2M +m), where M is the total number of characters in
∈ ∈ ≤
cap(D)=0. If this is not the case, we construct the graph of D and m is their maximal length.
the automaton of Aho-Corasick for the set P =D ( D) Proof: If cap(D)>0, then there is an admissible string
∪ − ∪
+0m 1 . We then remove any vertex labeled with a pattern oflengthn 2M+2m(Corollary6).Considerthecodegiven
−
{ } ≤
inP (i.e.,astatereachedwhenasuffixofthetextenteredisin by equation (6). Its size is 2l and the length of its words is at
thesetP)exceptthevertexlabeled +0m 1 .Thesizeofthe most
−
{ }
5
Such a set D has always a length polynomial in the number
N = l(2M +2m m)=l 2M +m of clauses and the number of variables.
l
− (cid:0) (cid:1)
We now prove that there is a solution to the instance of Not-
Therefore
All-Equal3SATifandonlyifcap(D)>0.First, supposethat
cap(D) = lim log2δNl there exists a satisfying truth assignment for x and denote it
≥ llim l 2loMg2+2mll→∞= N2lM1+m. bcoyd{eωo1f,l.e.n.g,tωhmk}(.mA+ss1o)cicaotendtationinagny2kkw≥ord1swasefcoollnoswtrsu:ct a
→∞ (cid:0) (cid:1)
C = 0ω0ω0ω0 0ω0ω,0ω0ω0ω0 0ω0ω¯,
k(m+1)
{ ··· ···
0ω0ω0ω0 0ω¯0ω,...,0ω¯0ω¯0ω¯0 0ω¯0ω¯ ,
V. POSITIVE CAPACITY IS NP-HARD FOREXTENDEDSETS ··· ··· }
We nowconsiderthe situationwhereforbiddenpatternsare where ω =ω1 ωm.
···
allowed to contain the symbol. The symbol is to be Any difference between two words in this code is a word
understood in the follow±ing sense: whenever it o±ccurs in a of the form 0z10z20 0zk where for every 1 i k,
··· ≤ ≤
forbidden pattern, both the occurrences of + and of are zi is either a sequence of m 0’s or a word of length m
forbiddenatthatparticularlocation.So,forexample,avo−iding over ,+ . Because ω satisfies the instance of Not-All-
{− }
the forbidden set 0 + is equivalent to avoiding the set Equal 3SAT, these words avoid the set D constructed above.
0+++,0++ {,0±++±,}0 + . All results obtained for Moreover, the cardinality of Ck(m+1) is 2k and hence
{ − − − −}
fcoorubnidtedrepnarptsattienrntsheovseirtu{a−ti,o0n,+w}hehraevethteherfeofrobrieddtheneirpnaattteurrnasl cap(D)≥klim log22k(mk+1) = m1+1 >0. (8)
are defined over the alphabet ,0,+, . In particular, the →∞
{− ±} For the reverse implication, assume now that cap(D) >
results of Section 3 do transfer verbatim and the bounds
0. The capacity is positive, and so one can find two words
derived in Theorem 1 are valid exactly as stated there. We
whose differences contain a 0 and a +. But then since this
nowproveacomplexityresultof capacitycomputationinthis
difference must avoid the first part of the forbidden pattern,
set-up.
for a code C large enough, there must exist two words in the
Theorem 4: The problem of determining if the capacity of
code whose differencecontainsa word over ,+ of length
a set of forbidden patterns over 0,+, , is equal to zero {− }
{ − ±} m. But this sequence avoids also the second part of D, and
is NP-hard.
thus it represents an acceptable solution to our instance of
Proof: The proof proceeds by reduction from the Not-
Not-All-Equal 3SAT.
All-Equal 3SAT problem that is known to be NP-complete
Note that a similar proof can be given if we replace the
(see [10]). In the Not-All-Equal 3SAT problem, we are given
symbol“ ” in the statement of the theorem by a symbolthat
mbinaryvariablesx1,...,xm andnclausesthateachcontain ±
represents either +, , or 0.
three literals (a literal can be a variable or its negation), and −
we search a truth assignment for the variables such that each
clause has at least one true literal and one false literal. VI. EXTREMAL NORMS AND COMPUTING OF THE
Supposethatwe are givena set of clauses. We constructa set CAPACITY
of forbidden patterns D such that cap(D) > 0 if and only if A classical way to estimate a joint spectral radius consists
the instance of Not-All-Equal 3SAT has a solution. The first of computing successive upper bounds on it by applying the
part of D is given by: following well known inequality
(0 0),(0 0),...,(0 m 10) . (7) ρn(Σ) max A ...A :A Σ
− 1 n i
{ ± ±± ± } ≤ {k k ∈ }
Words over ,0,+ that avoid these patterns are exactly thatholdsforanynorm,andanylengthnoftheproducts.One
{− }
those words for which any two consecutive zeros are either couldthenhopethatforawell-chosennorm,thejointspectral
adjacent or have at least m symbols + or between them. radiuswouldbealreadyobtainedforn=1,thatis,fortheset
−
We use these m symbols as a way of encoding possible truth ofmatricesitself.Thisistheconceptofextremalnorm,which
assignments for the variables. wenowdefineproperly:Anorm inRd iscalledextremal
k·k
We then add to D two patterns for every clause. These forafamilyofoperatorsA ,...,A if A ρ(A ,...,A )
1 r i 1 r
k k≤
patternsareoflengthmandareentirelycomposedofsymbols for all i = 1,...,r. The unit ball M of this norm is called
,exceptforthepositionscorrespondingtothethreevariables the extremal convex body.
±
of the clause, which we set to + if the clause contains the The notion extremal is justified by the fact that ρ is the
variable itself, or to if the clause contains the negation of smallestpossiblevaluesuchthatthenormsofalltheoperators
−
the variable. We also add the opposite of this pattern; this A ,...,A do not exceed this value. The above inequality,
1 r
last pattern is not necessary for the proof but preserves the which holds for any norm, for extremal norms becomes an
symmetry of the construction. equality (for all n 1). If M is the unit ball corresponding
≥
Forexample,iftheinstanceofNot-All-Equal3SATconsists to an extremal norm (an extremal body), A M ρM. On
i
⊂
of the two clauses (x ,x¯ ,x ) and (x¯ ,x ,x ), the corre- the other hand, any convex body M (convex compact with
1 3 4 2 4 5
spondingsetDwillbeD = (0 0),(0 0),(0 0),(0 nonempty interior centrally symmetric with respect to the
{ ± ±± ±±± ±
0),(+ + ),( + ),( ++),( + ) . origin) possessing this property generates an extremal norm.
±±± ±− ± −± −± ±−± ± ±−− }
6
It suffices to take the Minkowski norm defined by this body: After N = 3√dlncc21 steps the algorithm terminates. The
x = inf λ > 0, 1x M . Thus, there is a natural one- h ε i
k k { λ ∈ } value ρ = v 1/(N+1) gives the desirable approximation
to-one correspondence between extremal norms and extremal ∗ (cid:0) N+1(cid:1)
ofthejointspectralradius.Herev isthebiggestdistancefrom
bodies. k
the origin to the vertices of the polytope P , c ,c are lower
Theexistenceofanextremalnormcansimplifymanyprob- k 1 2
andupperboundsofthevalues ρ k ρ (Σ, ), k N .For
lems related to the jointspectral radius, see [2], [16] and [18] { − · k k·k ∈ }
the adjacency matrices the values c ,c can be taken directly
for details. However, not every set of matrices possesses an 1 2
from Theorem 1.
extremalnorm.Thecorrespondingcounterexamplesaresimple
and well-known. Sufficient conditions for the existence of an Each step requires us taking the convex hull of two poly-
extremalnormcanbefoundin[2],[16],[17].Forthematrices topes having at most q(ε) vertices and requires the approxi-
arising in the context of the capacity computation, however, mationofonepolytopewith2q(ε)verticesbyapolytopewith
theseconditionsarenotalwayssatisfied.Nevertheless,itturns q(ε) vertices with accuracy ε. Both operations are known to
out that in the case of capacity computation, the matrices do be polynomial w.r.t. 1 [16] (the dimension d is fixed). The
ε
in fact always possess an extremal norm. computationalcomplexityofthisalgorithmisC ε−d+21,where
·
Theorem 5: For any set D of forbidden patterns the set C is some constant and d = 2m 1. Therefore the algorithm
−
Σ(D) possesses an extremal norm. is applicable for small values of m, say, for m 6.
≤
Proof: Let Σ(D) be the set of matrices corresponding
Thecomplexityofthisalgorithmisexponentialwithrespect
to D. For a given point x 0 let
≥ tom,astheoneproposedinSectionIIIthatapproximatesthe
capacity by successive estimations of δ . The advantages of
(x) = ρ nA ...A x, A Σ(D) n 0 n
O n − 1 n i ∈ ≥ o onealgorithmovertheotherappearsinnumericalcomputation
of the capacity. In many cases the approximationof invariant
be the normalized orbit of the point x under the action of
bodies by polytopes can lead to the exact value of the
all possible products of the operators in Σ. The product of
joint spectral radius. Suppose that by numerical observations
lengthzeroisdefinedas theidentityoperator,so theset (x)
O we conjecture that ρ is attained by some product Π =
contains x. Now define a set M as follows n
A ...A , i.e. ρ = ρ(Π )1/n. If during the calculations
i1 in n
M = ConvnO(ej), O(−ej), j=1,...,2m−1o, witeocficnudrsathpaotlyρto=peρP(Πsu)c1h/nt.haAtsAthPe p⊂olyρt(oΠpne)1P/nwPe, tthaekne
n
wheree1,...,e2m−1 arethecanonicalbasisvectorsinR2m−1, P =Pk =Conv{A¯jv, A¯jv, j =0,...,k} for some integer
and Conv denotes the convex hull. The set M is obviously k, where v is the eigenvector of Πn corresponding to the
largestby modulo eigenvalue(we assume that this is real and
convex, centrally symmetric with respect to the origin, and
unique).
possesses a nonempty interior (because it contains the cross-
polytope with the vertices ej, j = 1,...,2m−1 whose Let us illustrate this method by computingthe exact values
{± }
interior is nonempty). Moreover, Theorem 1 implies that the of the capacity for several codes. In Examples 3 and 4 we
set (x)isboundedforanyx,thereforeM isbounded.SoM find the values of capacities that were approximated in [15].
O
isaconvexbodythatpossessesthepropertyAiM ρM, i= Example 5 deals with a code with m=4.
⊂
1,...,r. Therefore it generates an extremal norm, and the
Example 3: cap( 0++ ) = log ρ(A ) = log √5+1 =
theorem follows. { } 2 0 2(cid:0) 2 (cid:1)
0.69424191.... The eigenvector is v = (2,√5 1,2,√5
− −
1)T. The algorithm terminates after five steps, the polytope
The very existence of an extremal norm for a set of matrices
P =P has 32 vertices.
makes it possible to apply a geometric algorithm for comput- 5
ingajointspectralradiuswithagivenrelativeprecisionε.We Example 4: cap({0 + −}) = log2ρ(A0) = log2(cid:0)√52+1(cid:1).
nowbrieflydescribethisalgorithm;foralltechnicaldetailswe Thealgorithmterminatesafterfoursteps,v =(2,√5 1,√5
− −
referthereaderto[16].Forthesakeofsimplicityweconsider 1,2)T, P =P , the polytope has 40 vertices.
4
the case of two matrices, the case of an arbitrary number of Example 5: cap( + + + ) = log √3+2√5+1 =
matrices is treated in the same way. { −} 2(cid:0) 2 (cid:1)
log ρ(A A )=0.90053676.... The algorithm terminates
The algorithm. Suppose operators A ,A acting in Rd 2p 0 1
0 1 after eleven steps, the polytope P =P has 528 vertices.
11
possess an extremal norm; one needs to find a number ρ
∗
ρ∗ ρ Asillustrated in manyapplicationsitis quiteoftenthe case
such that (cid:12) − (cid:12) < ε, where ε > 0 is a given accuracy.
(cid:12) ρ (cid:12) thatthejointspectralradiusisattainedbysomefiniteproduct.
Consider a sequence of convex polytopes P produced as
k We say in these cases that the set of matrices possess the
{ }
follows. P0 =n(x1,...,xd) Rd, xi 1o is a cross- finitenessproperty. It was conjecturedthatall sets of matrices
∈ P| |≤
polytope. For any k 0 the polytope P is an arbitrary have the finiteness property: this is the well knownfiniteness
k+1
≥
polytope possessing the following properties: it is symmetric conjecture which has been disproved in [8], [7], and [12].
with respect to the origin, has at most q(ε) = Cdε1−2d Nevertheless, we conjecture here that the sets of matrices
vertices, where C is an effective constant depending only constructed in order to compute a capacity do always possess
d
on d, and (1 ε)A¯P P A¯P , where A¯X = the finiteness property.Numericalresults in [15], [11], and in
k k+1 k
− ⊂ ⊂
Conv A X,A X . this paper seem to support this conjecture.
0 1
{ }
7
VII. CONCLUSION [11] R. Jungers. NP-Completeness of the computation of the capacity of
constraints oncodes.Master’sthesis,UniversitCatholique deLouvain-
One way to compute the capacity of a set of forbidden
La-Neuve,2005.
patterns is to compute the joint spectral radius of a set of [12] VictorKozyakin.Adynamicalsystemsconstructionofacounterexample
matrices. In practice, this leads to a double difficulty : first, to the finiteness conjecture. Proc. 44th IEEE Conference on Decision
andControlandECC2005,toappear, 2005.
the size of the matrices is exponential in the size of the set
[13] Bruce E. Moision and Alon Orlitsky. On codes with local joint con-
of forbidden patterns, and second, the joint spectral radius straints. Preprint. Submitted,2004.
is in general NP-hard to compute. Actually, it is even NP- [14] Bruce E.Moision, Alon Orlitsky, and Paul H. Siegel. Bounds On The
RateOfCodesWhichForbidSpecifiedDifferenceSequences. InProc.
hard to decide whether a joint spectral radius of two matrices
1999 IEEE Global Telecommun. Conf. (GLOBECOM ’99), December
is greater than one. We show in this paper that the simpler 1999.
problem of checking the positivity of the capacity of a set [15] Bruce E. Moision, Alon Orlitsky, and Paul H. Siegel. On Codes that
AvoidSpecifiedDifferences.IEEETransactionsonInformationTheory,
defined on +, ,0 is polynomially decidable but that the
47,2001.
{ − }
same problem becomes NP-hard when defined over the al- [16] VladimirProtasov.Thejointspectralradius andinvariant setsoflinear
phabet +, ,0, .We alsoprovideboundsthatallowfaster operators,Fundamentalnayaiprikladnayamatematika,2(1996),No1,
{ − ±} pp.205–231.
computation of the capacity. Finally we prove the existence
[17] Vladimir Protasov. On the asymptotics of the partition function, Sb.
of extremal norms for the sets of matrices arising in the Math.191(2000),no.3-4,381–414, 2(1996),No1,pp.205–231.
capacity computation and present a geometrical algorithm [18] Vladimir Protasov. The Geometric Approach for Computing the Joint
Spectral Radius Proc.44th IEEEConference onDecision andControl
for capacity computation which we illustrate with several
andECC2005,toappear, 2005.
numerical examples. Even if this latter result allows to use [19] G.C. Rota and C. Strang. A note on the joint spectral radius. Indag.
algorithms that have proved to be quite efficient in practice, Math.,22,pp.379-381,1960.
[20] J.Tsitsiklis andV.Blondel. TheLyapunov exponent andjointspectral
we should keep in mind that the approach that consists of
radiusofpairsofmatricesarehard-whennotimpossible-tocompute
computing the joint spectral radius of the matrices defined in and to approximate. Mathematics of Control, Signals, and Systems.
[15] cannot lead to a polynomial algorithm to compute the 10:31,40,1997.
capacity because of the exponential size of the matrices.
ACKNOWLEDGMENT
WewouldliketothankNogaAlonandAlexanderRazborov,
bothfromthe Institutefor AdvancedStudy (Princeton,USA),
for providing an initial proof of a result that is essentially
equivalenttothestatementofTheorem3.Theproofpresented
hereisdifferentandisbasedontheAho-Corasickautomaton.
We also express our thanks to a student of Moscow State
University, E.Shatokhin, for implementing the algorithm in-
troduced in this paper and for the numerical computation of
the Examples 3 to 5.
REFERENCES
[1] AlfredV.Aho.Algorithmsforfindingpatterns instrings.InHandbook
of theoretical computer science (vol. A): algorithms and complexity,
pages 255-300.MITPress,Cambridge, MA,USA,1990.
[2] Nikita Barabanov. Lyapunovindicators ofdiscrete inclusions. PartI,II
andIII,TranslationfromAvtomatiTelemekh.2(1988)40-46,3(1988)
24-29,5(1988)17-24.
[3] M.A.BergerandY. Wang,Bounded Semigroups ofMatrices, Journal
ofLinearAlgebra anditsApplications, vol.166,pp.21-27,1992.
[4] Vincent D. Blondel and Yurii Nesterov. Computationally efficient ap-
proximations of the joint spectral radius. SIAM Journal of Matrix
Analysis,27:1,pp.256-272, 2005.
[5] Vincent D. Blondel and John N. Tsitsiklis. A survey of computational
complexityresultsinsystemsandcontrol.Automatica,36(9):1249-1274,
2000.
[6] Vincent D. Blondel and John N. Tsitsiklis. The boundedness of all
products of a pair of matrices is undecidable, Systems and Control
Letters,41:2,pp.135-140,2000.
[7] Vincent D. Blondel, J. Theys, and A. A. Vladimirov. An elementary
counterexample to the finiteness conjecture, SIAM Journal on Matrix
Analysis,24:4,pp.963-970, 2003.
[8] T.Bousch and J.Mairesse. Asymptotic height optimization for topical
IFS, Tetris heaps, and the finiteness conjecture. Journal of the Mathe-
matical AmericanSociety, vol15,no.1,pp.77-111,2002.
[9] I. Daubechies and J.C. Lagarias. Sets of matrices all infinite products
ofwhichconverge. LinearalgebraAppl.,161,pp.227-263,1992.
[10] M.R.GareyandD.S.Johnson.ComputersandIntractability–AGuide
totheTheoryofNP-Completeness. Freeman,SanFrancisco, 1979.