Codes With Hierarchical Locality

0.47 MB·English

by  Birenjith Sasidharan

1
Codes With Hierarchical Locality
Birenjith Sasidharan, Gaurav Kumar Agarwal, and P. Vijay Kumar
Department of Electrical Communication Engineering, Indian Institute of Science, Bangalore.
Email: biren, agarwal, [e-mail]
Abstract
In this paper, we study the notion of codes with hierarchical locality that is identified as another approach to local recovery
from multiple erasures. The well-known class of codes with locality is said to possess hierarchical locality with a single level.
In a code with two-level hierarchical locality, every symbol is protected by an inner-most local code, and another middle-level
code of larger dimension containing the local code. We first consider codes with two levels of hierarchical locality, derive an
upper bound on the minimum distance, and provide optimal code constructions of low field-size under certain parameter sets.
Subsequently, we generalize both the bound and the constructions to hierarchical locality of arbitrary levels.
Index Terms
Codes with locality, locally recoverable codes, hierarchical locality, multiple erasures, distributed storage.
5
1
I. INTRODUCTION
0
2 Animportantdesirableattributeinadistributedstoragesystemistheefficiencyincarryingoutrepairoffailednodes.Among
many others, two important metrics to characterize efficiency of node repair are repair bandwidth, i.e., the amount of data
n
a download in the case of a node failure and repair degree, i.e., the number of helper nodes accessed for node repair. While
J regenerating codes [1] aim to minimize the repair bandwidth, codes with locality [2] seek to minimize the repair degree. The

7 focus of the present paper is on codes with locality.
2

] A. Codes with Locality
T
An [n,k,d] linear code C can possibly require to access k symbols to recover one lost symbol. The notion of locality of
I
. code symbols was introduced in [2], with the aim of designing codes in such a way that the number of symbols accessed
s
c to repair a lost symbol is much smaller than the dimension k of the code. The code C is said to have locality r if the i-th
[ code symbol c ,1≤i≤n can be recovered by accessing r <<k other code symbols. In [2], authors proved an upper bound
i
on the minimum distance of codes with locality, and showed that an existing family of pyramid codes [3] can achieve the
1
v bound. In [4], authors extended the notion to (r,δ)-locality, where each symbol can be recovered locally even in the presence
3 of an additional (δ−2) erasures. In [2], authors introduced categories of information-symbol and all-symbol locality. In the
8 former, local recoverability is guaranteed for symbols from an information set, while in the latter, it is guaranteed for every
6
symbol. Explicit constructions for codes with all-symbol locality are provided in [5], [6], respectively based on rank-distance
6
and Reed-Solomon (RS) codes. Improved bounds on the minimum distance of codes with all-symbol locality are provided in
0
. [7],[8],alongwithcertainoptimalconstructions.Familiesofcodeswithall-symbollocalitywithsmallalphabetsize(lowfield
1
size) are constructed in [9]. Locally repairable codes over binary alphabet are constructed in [10]. A new approach of local
0
regeneration, where in repair is both local and in addition bandwidth-efficient within the local group, achievable by making
5
1 use of a vector alphabet is considered in [4], [11], [12].
: Recently, many approaches are proposed in literature [4], [7], [9], [13], [14] to address the problem of recovering from
v
i multiple erasures locally. The notion of (r,δ)-locality introduced in [4] is one such. In [13], an approach of protecting a single
X
symbol by multiple support-disjoint local codes of the same length is considered. An upper bound on the minimum distance
r is derived, and existence of optimal codes is established under certain constraints. A similar approach is considered in [9]
a
also. In [9], authors allow multiple recovering sets of different sizes, and also provide constructions requiring field-size only
in the order of block-length. Quite differently, authors of [7] consider codes allowing sequential recovery of two erasures,
motivated by the fact that such a family of codes allow a larger minimum distance. An upper bound on the minimum distance
and optimal constructions for restricted set of parameters are provided.
B. Our Contributions
Inthepresentpaper,westudythenotionofhierarchicallocalitythatisidentifiedasanotherapproachtolocalrecoveryfrom
multiple erasures. In consideration of practical distributed storage systems, Duminuco et al. in [15] had proposed the topology
This research is supported in part by the National Science Foundation under Grant No. 1422955 and in part by the joint UGC-ISF Research Grant No.
1676/14.BirenjithSasidharanwouldliketoacknowledgethesupportofTCSResearchScholarProgrammeFellowship.
2
X1 X2 X3 X4 X5 Px
P1 P2
Y1 Y2 Y3 Y4 Y5 Py
Fig. 1: Illustration of [16,12,4]-code used in Windows Azure.
[24,14,6]
[12,8,3] [12,8,3]
[4,3,2] [4,3,2] [4,3,2] [4,3,2] [4,3,2] [4,3,2]
Fig. 2: Illustration of [24,14,6]-code having 2-level hierarchical locality.
ofhierarchicalcodesearlier.TheycomparedhierarchicalcodeswithRScodesintermsofrepair-efficiencyusingrealnetwork-
traces of KAD and PlanetLab networks. Their work was focused on collecting empirical data for performance improvements,
rather than undertaking a theoretical study of such a topology. In the present paper, we study codes with hierarchical locality,
firstconsideringthecaseoftwo-levelhierarchy.Wederiveanupperboundontheminimumdistanceandprovideoptimalcode
constructions under certain parameter-sets. This is further generalized to a setting of h-hierarchy in a straightforward manner.
II. CODESWITHHIERARCHICALLOCALITY
The Windows Azure Storage solution employs a [16,12,4]-pyramid code with a locality parameter r = 6. In the code,
as illustrated in Fig. 1, every code symbol except the global parities P ,P can be recovered accessing r = 6 other code
1 2
symbols.Whilethecodeperformswellinsystemswheresinglenode-failureremainsthedominantevent,itrequirestoconnect
to k =12 symbols to recover a failed under certain erasure-patterns consisting of 2 node-failures. We consider an example of
[24,14,6]-code from the family of codes with hierarchical locality in an attempt to reduce such an overhead. The structure of
thecodeisdepictedinFig.2asatreeinwhicheachnoderepresentsaconstituentcode.Thecodecontainstwosupport-disjoint
[n = 12,r = 8,d = 3] codes, each of them in turn comprised of three support-disjoint [n = 4,r = 3,d = 2] codes.
1 1 2 2 2 1
Makinguseof[4,3,2]-code,allsingle-erasurescanberepairedaccessingr =3symbols,whichishalfthenumberofsymbols
2
required in the Windows Azure code in a similar situation. We can recover a lost symbol connecting to r =8 symbols in the
1
case of erasure-pattern involving 2 erasures. This is in contrast to the Windows Azure code where we had to download the
entire message of 12 symbols. While the Windows Azure code offers a storage overhead of 1.3x with a minimum distance of
4, our code has a larger overhead of 1.7x with a better minimum distance d=6. The example of [24,14,6]-code can indeed
be constructed, and it will be shown that the minimum distance is optimal among the class of codes.
A. Preliminaries
Definition 1: [4] An [n,k,d] linear code C is a code with (r,δ)-locality if for every symbol c ,1 ≤ i ≤ n, there exists a
i
punctured code C such that c ∈Supp(C ) and the following conditions hold: 1) dim(C )≤r, 2) d (C )≥δ.
i i i i min i
Codes with locality were first defined in [2] for the case of δ =2, and the class was generalized for arbitrary δ in [4]. In the
definition given in [4], the authors imposed constraints on the length and the d of C . We replace the constraint on length
min i
with a constraint on dim(C ), and it may be noted that it does not introduce any loss in generality. The code C associated
i i
with the i-the symbol is referred to as its local code. If it is sufficient to have local codes only for symbols belonging to some
fixed information set I, such codes are referred to as codes with information-symbol (r,δ)-locality. The general class in Def. 1
is also referred to as codes with all-symbol (r,δ)-locality, in order to differentiate them from the former. In this paper, unless
otherwise mentioned, we consider codes with all-symbol locality.
Definition 2: An [n,k,d] linear code C is a code with hierarchical locality having locality parameters [(r ,δ ),(r ,δ )] if
1 1 2 2
for every symbol c ,1≤i≤n, there exists a punctured code C such that c ∈Supp(C ) and the following conditions hold:
i i i i
1) dim(C )≤r , 2) d (C )≥δ , 3) C is a code with (r ,δ )-locality.
i 1 min i 1 i 2 2
The punctured code C associated with c is referred to as its middle code. Since the middle code is a code with locality,
i i
each of its symbols will in turn be associated with a local code.
B. An Upper Bound On the Minimum Distance
Theorem 2.1: Let C be an [n,k,d]-linear code with hierarchical locality having locality parameters [(r ,δ ),(r ,δ )]. Then
2 2 1 1
(cid:18)(cid:24) (cid:25) (cid:19) (cid:18)(cid:24) (cid:25) (cid:19)
k k
d≤n−k+1− −1 (δ −1)− −1 (δ −δ ). (1)
r 2 r 1 2
2 1
Proof: We extend the techniques introduced in [2] in proving the theorem. A punctured code C of C having dimension
s
k−1, is identified first. Then we will use the fact that
3
d ≤ n−|Supp(C )|. (2)
s
The Algorithm 1 (see flow chart in App. A) in is used to find C with a large support. In each iteration indexed by j, the
s
algorithm identifies a middle code from C, that accumulates additional rank. Then it picks up local codes from within the
middle code that accumulate additional rank. Clearly, the algorithm terminates as the total rank is bounded by k. Let i and
end
j respectively denote the final values of the variables i and j before the algorithm terminates. Let a denote the incremental
end i
rank and s denote the incremental support while adding a local code L . Then we have s ≥ a +(δ −1), 1 ≤ i ≤ i ,
i i i i 2 end
since we have a >0 in every iteration. The set S denotes the support of L , and V denotes the space Column-Space(G| ),
i i i i Si
where G is the generator matrix of the code. If no more local codes can be added from the middle code M , then the support
j
of the last local code added from M is removed and an additional support T of M is added to Ψ. Let i(j) denote the index
j j j
of the last local code added from M . Since the middle code has a minimum distance of δ , and every rank accumulating
j 1
local code brings at least one new information symbol, it follows that
t :=|T | ≥ a +(δ −1),
j j i(j) 1
= a +(δ −1)+(δ −δ ) 1≤j ≤j .
i(j) 2 1 2 end
Algorithm 1 For the proof of Thm. 2.1
1: Let j =0,i=0,W =φ,Ψ=φ.
2: while (∃ a middle code Mj ∈C such that rank(G|Ψ∪Mj)>rank(G|Ψ)) do
3: while (∃ a local code Li ∈Mj such that Vi (cid:40)W) do
4: W =W +Vi
5: Ψ=Ψ∪Si
6: i=i+1
7: end while
8: Ψ=(Ψ\Si−1)∪Tj
9: j =j+1
10: end while
The rank accumulates to k after adding the last local code L . We would also have visited j middle codes by then.
iend end
Hence,
(cid:24) (cid:25) (cid:24) (cid:25)
k k
i ≥ , j ≥ . (3)
end r end r
2 1
After adding L local codes, we would have accumulated rank that is less than or equal to (k−1). Hence we can always
pick s :=(k−ie1nd)−−1 (cid:80)iend−1a columns from L so that the total rank accumulated becomes (k−1). Note that s ≥0. The
e i=1 i iend e
resultant punctured code is identified as C . Let E ={i(j)|1≤j ≤j }. Then
s end
ie(cid:88)nd−1 je(cid:88)nd−1
|Supp(C )| ≥ s +s + t . (4)
s i e j
i∈/E,i=1 j=1
In (4), the last term (cid:80)jend−1t includes a sum of only j −1 terms because we could have possibly accumulated a rank of
j=1 j end
(k−1) after adding L , i.e., s =0. Thus we have,
iend−1 e
ie(cid:88)nd−1 ie(cid:88)nd−1 je(cid:88)nd−1
|Supp(C )| ≥ s +(k−1)− a + t
s i i j
i∈/E,i=1 i=1 j=1
ie(cid:88)nd−1 ie(cid:88)nd−1
≥ (a +(δ −1))+(k−1)− a
i 2 i
i∈/E,i=1 i=1
je(cid:88)nd−1
+ (a +(δ −1)+(δ −δ ))
i(j) 2 1 2
j=1
ie(cid:88)nd−1 je(cid:88)nd−1
= (δ −1)+(k−1)+ (δ −δ )
2 1 2
i=1 j=1
4
Substituting values of i and j from (3) and using (2), we obtain the bound.
end end
It may be noted that the theorem holds good even for codes with information-symbol hierarchical locality.
C. Code Constructions For Information-Symbol Locality
Astraightforwardextensionofpyramidcodes[3]ispossibletoconstructoptimalcodeswithinformation-symbolhierarchical
locality. They achieve the bound in (1) if r |r |k. In this section, we illustrate the construction with an example, assuming
2 1
δ =2. The two-level hierarchical code described here extends naturally to multiple-level hierarchy and yields optimal codes
2
if r |r |···|r |k.
h h−1 1
The construction is built on a systematic MDS code with parameters [k+d−1,k,d] with a generator matrix G . Let
mds
G = [I |Q ]
mds k×k k×(d−1)
k = αr +βr +γ, 0≤βr +γ <r
1 2 2 1
r = µr +ν, 0≤ν <r .
1 2 2
Partition Q as
 
Q
1
 Q2 
Q =  . Q(cid:48) ,
 . 
 . 
Q
α+1
where Q ,1≤i≤α is of size r ×(δ −1), Q is of size (βr +γ)×(δ −1) and Q(cid:48) is of size (k×(d−δ )). Further
i 1 1 α+1 2 1 1
partition Q ,1≤i≤α as
i
 
R
i1
 Ri2 
Qi =  .. Ri(cid:48) ,
 . 
R
i,µ+1
where R ,1≤j ≤µ is of size r ×1, R is of size ν×1 and R(cid:48) is of size (r ×(δ −2)). At the same time Q is
ij 2 i,µ+1 i 1 1 α+1
partitioned as
 
R
α+1,1
 Rα+1,2 
Qα+1 =  .. Rα(cid:48)+1 ,
 . 
R
α+1,β+1
where R ,1≤j ≤β is of size r ×1, R is of size γ×1 and R(cid:48) is of size ((βr +γ)×(δ −2)). Next, we
α+1,j 2 α+1,β+1 α+1 2 1
can construct matrices
 
R
i1
Qˆi =  Ri2 ... Ri(cid:48) 
R
i,µ+1
 
R
α+1,1
Qˆα+1 =  Rα+1,2 ... Rα(cid:48)+1 
R
α+1,β+1
 Qˆ 
1
Qˆ =  Qˆ2 ... Q(cid:48) 
Qˆ
α+1
Let J be defined as
 
I
r2
J =  ... .
 
 Ir2 
I
ν
5
where I is repeated µ times. Finally we construct the generator matrix G of the pyramid code as,
r2
 
J
 ... 
 
 J 
G =  Ir2 Qˆ ,
 ... 
 
 Ir2 
I
γ
with J being repeated α times, and I repeated β times. The resultant code has length
r2
(cid:18)(cid:24) (cid:25)(cid:24) (cid:25) (cid:19) (cid:18)(cid:24) (cid:25) (cid:19)
k r k
n = k+d−1+ 1 −1 + −1 (δ −2).
r r r 1
1 2 1
Clearly,minimumdistanceofthecodecorrespondingtoGisgreaterthanorequaltothatofG ,whichisd.Also,Gsatisfies
mds
the property of information-symbol hierarchical locality by construction. Using the bound in (1), the code is optimal if
(cid:24) (cid:25)(cid:24) (cid:25) (cid:24) (cid:25)
k r k
1 = .
r r r
1 2 2
D. Code Constructions For All-Symbol Locality
Weassumeadivisibilityconditionn |n |n.Theconstructionisdescribedinthreeparts.Thefirstpartinvolvesidentification
2 1
of a suitable finite field F , a partition of F∗ and a set of polynomials in F [X] that satisfy certain conditions. We require
pm pm pm
that every polynomial evaluates to a constant within one subset in the partition, and evaluates to zero in all the remaining
subsets. In the second part, we construct a code polynomial c(X) from the message symbols with the aid of the suitably
chosen polynomials. The code polynomial c(X) is formed in such a way that the locality constraints are satisfied. This part
also involves precoding of message symbols in such a way that the dimension of the middle codes and the global code are
kepttothedesiredvalues.Finally,thethirdpartinvolvesevaluationofthecodepolynomialatnpointsofF∗ ,thatarechosen
pm
in the first part.
1) Identification of F , a partition of F∗ and a set of polynomials: Let the finite field F be such that n | pm−1,
pm pm pm 1
and n<pm. Existence of such a pair (p,m) is shown in App. B. We define the integers
n n
n =pm−1,µ = 1,µ = 0,µ = 1.
0 0 1 n 2 n
1 2
Let α denote the primitive element of F , and hence F∗ = {1,α,α2,...,αpm−2}. Set β =α and β ,β be elements of
pm pm 0 1 2
order n and n respectively. Then we have the following subgroups:
1 2
H = F∗
0 pm
H = {1,β ,β2,...,βn1−1}
1 1 1 1
H = {1,β ,β2,...,βn2−1.
2 2 2 2
We can further write
H = H (cid:93)β H (cid:93)···(cid:93)βµ1−1H
0 1 0 1 0 1
H = H (cid:93)β H (cid:93)···(cid:93)βµ2−1H .
1 2 1 1 1 2
Having set up a subgroup chain, we proceed to define a family of subsets of H . These subsets are indexed by a tuple (i,t)
0
withi∈{0,1,2}.Foragivenvalueofi,ttakesvaluesfromthesetT ={(t ,t ,t )|1≤t ≤µ for j ≤i;t =0 for j >i}.
i 0 1 2 j j j
For a given tuple (i,t), let us define a coset A of the subgroup H as follows:
(i,t) i
i−1
γ = (cid:89)βtj+1−1, A =γ H .
(i,t) j (i,t) (i,t) i
j=0
The set of possible indices has a tree-structure with each index (i,t) associated with a unique vertex of the tree. A vertex (i,t)
belongs to the i-th level of the tree, and the 3-tuple t describes the unique path from the vertex to the root of the tree. The
parent of a vertex (i,t) is denoted by π(i,t), and the set of its siblings, i.e, other vertices having the same parent, is denoted
by ψ(i,t). The tree structure of {A |i∈{0,1,2},t∈T } is depicted in Fig. 3. Since each vertex at i-th level is associated
(i,t) i
with a coset of H , we refer to this tree as the coset-tree.
i
6
Fig. 3: Illustration of the coset-tree.
Next, we define the polynomials p (X),q (X)∈F [X] as,
(i,t) (i,t) pm
(cid:89) (cid:89)
p (X)= (X−θ), q (X)= p (X).
(i,t) (i,t) (i,s)
θ∈A(i,t) s∈ψ(i,t)
The polynomial p (X) is the annihilator of A . Furthermore, the polynomials in {q (X) | 1 ≤ t ≤
(i,t) (i,t) (i,t) i
µ ,t is fixed for j (cid:54)= i} are relatively prime collectively. Thus for i = 1,2 there exists {a (X) | 1 ≤ t ≤
i j (i,t) i
µ ,t is fixed for j (cid:54)=i} such that
i j
(cid:88)µi a (X)q (X)=1 mod (Xni−1 −γni−1 ), (5)
(i,t) (i,t) (i−1,s)
ti=1
tjfixedforj(cid:54)=i
where s=π(i,t). Next, we define E (X)=a (X)q (X), and determine a valid candidate for E (X) in the next
(i,t) (i,t) (i,t) (i,t)
Lemma such that (5) holds. Subsequently in Lem. 2.3, we will list down certain useful properties of these polynomials. The
proof is relegated to Appendix.
Lemma 2.2:
(cid:89) (cid:20) Xni −(γ(i,s))ni (cid:21)
E (X)= . (6)
(i,t) (γ(i,t))ni −(γ(i,s))ni
s∈ψ(i,t)
Lemma 2.3: Let i∈{1,2}, t,s∈T and t∈ψ(i,s). Let τ =(τ ,τ ,τ )=π(i,t). Then
i 0 1 2
E (X) = g(Xni) for some polynomial (7)
(i,t)
g(.), deg(g)=µ −1
i
(cid:26)
1 θ ∈A (X)
E (θ) = (i,t) (8)
(i,t) 0 θ ∈γ H \A
(i−1,τ) i−1 (i,t)
E (X)E (X) ≡ 0 mod (Xni−1 −γni−1 ) (9)
(i,s) (i,t) (i−1,τ)
E2 (X)=E (X) mod (Xni−1 −γni−1 ). (10)
(i,t) (i,t) (i−1,τ)
Proof: The property (7) is clear from the definition of E (X). The properties (8), (9) are clear from the proof of
(i,t)
Lemma 2.2. Hence (10) follows by (5).
2) Construction of c(X): We start with associating message polynomials of degree (r −1) with certain leaves of the
2
coset-tree. The total number of leaves of the coset-tree equals µ µ . However, we will only consider a suitable subtree of the
1 2
coset-tree such that the number of leaves equals µ¯ µ where µ¯ = n . The required subtree is obtained by removing the last
1 2 1 n1
(µ −µ¯ ) branches emanating from the root of the tree. Every leaf that is retained in the subtree has an index (2,t) where t
1 1
belongs to the set
T(cid:48) ={t∈T |1≤t ≤µ¯ }.
2 2 1 1
This subtree is referred to as the relevant coset-tree. A vertex from the i-th level, i>0 of the relevant coset-tree will have an
index (i,t) where t∈T(cid:48) ={t∈T |1≤t ≤µ¯ }.
i i 1 1
Consider a set U ={ut(X)=ut,0+ut,1X+...+ut,r2−1Xr2−1 |t∈T2(cid:48)} of message polynomials of size µ¯1µ2. The code
polynomialc(X)isbuiltfromU inaniterativemanner.Ineveryiteration,wetakeasinputasetofpolynomialscorresponding
to vertices of the i-th level of the relevant coset-tree, and output another set of polynomials corresponding to vertices of
the (i−1)-th level. As noted earlier, each leaf of the relevant coset-tree is uniquely mapped to a polynomial in U. In the
end, we will identify a polynomial c (X) associated with the root of the relevant coset-tree. The code polynomial
(0,(1,0,0))
c(X)=c (X). It may be noted that the polynomials in U is made up of µ¯ µ r message symbols in total. However,
(0,(1,0,0)) 1 2 2
the desired dimension k can be less than µ¯ µ r . Hence in every iteration, a precoding of message symbols is carried out
1 2 2
causing a reduction in the number of independent message symbols. The dimension would be reduced to the desired value k
at the end of the final iteration.
7
Letusnowstarttheiterationbysettingc (X)=u (X) ∀t∈T(cid:48).Evaluationsofc (X)atn pointsinA (X)give
(2,t) t 2 (2,t) 2 (2,t)
risetoan[n ,r ]-codeword.Recognizingthiscorrespondence,werefertoc (X),t∈T(cid:48) asasecondlevelcodepolynomial.
2 2 (2,t) 2
In the next iteration, for every t∈T(cid:48)
1
(cid:88)
d (X)= c (X)E (X).
(1,t) (2,s) (2,s)
s:π(2,s)=(1,t)
By(7)inLemma2.3,thecoefficientofX(cid:96) iszeroinE (X)whenever(cid:96)(cid:54)=0(modn ).Henceforeveryt∈T(cid:48),thereareof
(2,t) 2 1
µ r monomialsind (X).Evaluationsofd (X)atn pointsinA (X)giverisetoan[n ,µ r ]-codeword.Sincethe
2 2 (1,t) (1,t) 1 (1,t) 1 2 2
desired dimension of the middle code is r , we precode the message symbols such that the coefficients of (r µ −r ) highest
1 2 2 1
degree monomials in d (X) vanishes to zero. The polynomials c (X) thus obtained corresponds to an [n ,r ]-middle
(1,t) (1,t) 1 1
code, and hence referred to as a first level code polynomial. We can write
(cid:88)
c (X) = P (c (X))E (X), t∈T(cid:48),
(1,t) 1 (2,s) (2,s) 1
s:π(2,s)=(1,t)
where P (·) denotes the precoding transformation at the first level. In the next iteration, we compute d (X) and
1 (0,(1,0,0))
subsequently precode the message symbols by P (·) to reduce the dimension from µ¯ r to k to obtain the zeroth level code
0 1 1
polynomial c (X):
(0,(1,0,0))
(cid:88)
d (X) = c (X)E (X)
(0,(1,0,0)) (1,s) (1,s)
s:π(1,s)=(0,(1,0,0))
s∈T(cid:48)
1
(cid:88)
c (X) = P (c (X))E (X),
(0,(1,0,0)) 0 (1,s) (1,s)
s:π(1,s)=(0,(1,0,0))
s∈T(cid:48)
1
The code polynomial c(X) is identified as
c(X)=c (X). (11)
(0,(1,0,0))
3) Evaluation of c(X): The codeword c = (c(θ) | θ ∈ A) is obtained by evaluating the polynomial c(X) at n points
taken from
(cid:91)
A= A .
(1,t)
t∈T(cid:48)
1
Thiscompletesthedescriptionoftheconstruction.Bytheconstruction,itisclearthatthedimensionandtheminimumdistance
of the code are given by
k = |{(cid:96)|coefficient of X(cid:96) in c(X)(cid:54)=0}|
d ≥ n−deg(c(X)).
Remark 1: A principal construction in [9] for codes with all-symbol locality, relies on a partitioning of the roots of unity
contained in a finite field into a subgroup and its cosets. The construction then identifies polynomials that are constant on each
coset and makes use of these polynomials in the construction. The approach adopted here is along similar lines.
Example 1: In this example, we construct a code with [n,k] = [24,14] having locality parameters (n ,r ) = (12,8) and
1 1
(n ,r )=(4,3), satisfying the divisibility condition. We can choose the finite field F =F . Let α be a primitive element
2 2 pm 52
of F . We have n =n=24, µ =µ¯ =2, and µ =3. We set
52 0 1 1 2
H = F∗
0 52
H = {1,β ,β2,...,β11}
1 1 1 1
H = {1,β ,β2,β3,
2 2 2 2
where β =α, β =α2 and β =α6. The relevant coset-tree can be computed as
0 1 2
A =H , A =H , A =β H
(0,(1,0,0)) 0 (1,(1,1,0)) 1 (1,(1,2,0)) 0 1
A =βt1−1βt2−1H , 1≤t ≤2,1≤t ≤3.
(2,(1,t1,t2)) 0 1 2 1 2
LetusdefinetheindexsetsT ={(t ,t ,t )|t =1,1≤t ≤2,t =0},T ={(t ,t ,t )|t =1,1≤t ≤2,1≤t ≤3}.
1 0 1 2 0 1 2 2 0 1 2 0 1 2
For every t=(1,t ,0)∈T , we set s as the unique element in {1,2}\{t } and then we have
1 1 1 1
(cid:18) X12−(βs1−1)12 (cid:19)
E (X) = 0
(1,t) (βt1−1)12−(βs1−1)12
0 0
:= a X12+b .
t t
8
Similarly for every t=(1,t ,t )∈T , we set {s ,s }={1,2,3}\{t } and then we have
1 2 2 1 2 2
(cid:18) X4−(βt1−1βs1−1)4 (cid:19)(cid:18) X4−(βt1−1βs2−1)4 (cid:19)
E (X) = 0 1 0 1
(2,t) (βt1−1βt2−1)4−(βt1βs1−1)4 (βt1−1βt2−1)4−(βt1βs2−1)4
0 1 0 1 0 1 0 1
:= e X8+f X4+g .
t t t
There are |T |=6 message polynomials denoted by {u (X)=u +u X+u X2 |t∈T }, each of degree (r −1)=2.
2 t t,0 t,1 t,2 2 2
The second level code polynomial for each t∈T corresponding to a [4,3]-local code is taken to be c (X) = u (X). In
2 (2,t) t
the next step, the first level code polynomial {c (X)} is constructed as
1,s
s(cid:88)2=2
c (X) = P (c (X))(e X8+f X4+g ).
1,t 1 (2,s) s s s
s:s1=t1,s2=1
for each of t∈T . By virtue of the precoding P (·), the term X10 vanishes and the resultant polynomial c (X) corresponds
1 1 1,t
to a [12,8]-middle code. Subsequently, the zeroth level code polynomial is constructed as
s(cid:88)1=2
c (X) = P (c (X))(a X12+b ).
0,(1,0,0) 0 (1,s) s s
s:s1=1,s2=0
Without precoding P (·), we would have obtained a polynomial of degree 21 having 16 monomials. Precoding wipes out the
0
terms {X21,X20}, and the resultant polynomial c (X) =: c(X) of degree 18 is the code polynomial consisting of 14
0,(1,0,0)
monomials. Thus k =14, and d≥6. The codeword c is given by c=(c(θ)|θ ∈H ).
0
It is of interest to look at the exponents of monomials in polynomials of each level. From each level, we pick a candidate
polynomial c(X), c (X):=c (X), c (X):=c (X).
1 (1,(1,1,0)) 2 (2,(1,1,2))
(cid:1)3 2 1 0 (cid:26)1(cid:26)1 (cid:26)1(cid:26)0 9 8 (cid:1)7 6 5 4 (cid:1)3 2 1 0
(cid:26)2(cid:26)3 (cid:26)2(cid:26)2 (cid:26)2(cid:26)1 (cid:26)2(cid:26)0 (cid:26)1(cid:26)9 18 17 16 (cid:26)1(cid:26)5 14 13 12
(cid:26)1(cid:26)1 (cid:26)1(cid:26)0 9 8 (cid:1)7 6 5 4 (cid:1)3 2 1 0
Fig.4:Illustrationoftheexponentsofmonomialsinc (X),c (X)andc(X)inorder.Thecanceledexponentsarethosewhose
2 1
coefficients are fixed to zero by precoding.
The illustration in Fig. 4 gives an equivalent simplistic description of the [24,14,6] code. This works in general. Let Exp(f)
represent the ordered set of exponents of the monomials in a polynomial f(X). By ordered set, we mean that the elements
of the set are listed in the descending order. For example, Exp(X3 +αX +1) = {3,1,0}. For an ordered finite set S of
non-negative integers and a positive integer r ≤|S|, we define Trunc(S,r) as the set comprising of the last r elements of the
set. Then we have that
Exp(c ) = {r −1,r −2,...,0} = Trunc(Z ,r )
2 2 2 n2 2
µ(cid:91)2−1
Exp(c ) = Trunc( (jn +Exp(c )),r )
1 2 2 1
j=0
µ¯(cid:91)1−1
Exp(c) = Trunc( (jn +Exp(c )),k),
1 1
j=0
where Z ={0,1,...,n−1}. The set Exp(c) is an equivalent simplistic description of the code. In terms of Exp(c), we can
n
write the parameters of the code as k =|Exp(c)|, d≥n−max(Exp(c)).
E. Locality Properties Of the Code
In this section, we will show that the code satisfies locality constraints. Consider the case c(y) is lost. We need to recover
it accessing r other symbols {c(y ),c(y ),...,c(y )} that along with c(y) are part of an [n ,r ] punctured code. Without
1 1 2 r1 1 1
loss of generality, let us assume that y ∈A . Using (9), (10) in Lemma 2.3, we can write
(1,(1,1,0))
c(X)E (X) = P (c (X))E (X).
(1,(1,1,0)) 0 (1,(1,1,0)) (1,(1,1,0))
Evaluations at r out of n points in A will help reconstruct P (c (X)), since deg(P (c (X))) ≤
1 1 (1,(1,1,0)) 0 (1,(1,1,0)) 0 (1,(1,1,0))
(r −1). Then we can recover c(y) = P (c (y))E (y). The same argument can be used inductively to show
1 0 (1,(1,1,0)) (1,(1,1,0))
thateachsymbolwithinan[n ,r ]-middlecodecanberecoveredbyr outofsomen symbols.Thisestablishestheexistence
1 1 2 2
of [n ,r ]-local codes.
2 2
9
F. Optimality Of the Code
Theorem 2.4: The [n,k,d]-code with [n ,r ,δ ]-middle codes and [n ,r ,δ ]-local codes constructed in Sec. II-D achieves
1 1 1 2 2 2
the optimal minimum distance if r |r |k.
2 1
Proof: Let c(X) denote the code polynomial. The proof follows from counting Z in two different ways. We can write
n
n = |Z | = |Exp(c)|+|Z \Exp(c)|. (12)
n n
We have that |Exp(c)| = k. On the other hand, since r | r | k, we can count the number of exponents that are truncated
2 1
while forming c as,
(cid:18) (cid:19)
k
|Z \Exp(c)| = −1 (δ −1)
n r 2
2
(cid:18) (cid:19)
k
+ −1 (δ −δ )+(d−1)
r 1 2
1
Substituting back in (12), we conclude that the code is optimal.
Theorem 2.5: The [n,k,d]-code with [n ,r ,δ ]-middle codes and [n ,r ,δ ]-local codes constructed in Sec. II-D achieves
1 1 1 2 2 2
the optimal minimum distance if the following conditions hold:
1) d=n +δ
2(cid:108) (cid:109)2 (cid:108) (cid:109)
2) n = k , n = k +1
n1 r1 n2 r2
Proof: Let c(X) denote the code polynomial. The proof is analogous to that of Thm. 2.4. The only difference lies in the
count of |Z \Exp(c)|. Since d=n +δ , we obtain that,
n 2 2
(cid:18) (cid:19) (cid:18) (cid:19)
n n
|Z \Exp(c)|= −1 (δ −1)+ −1 (δ −δ )
n n 2 n 1 2
2 1
+((d−1)−(δ −1))
2
(cid:18) (cid:19) (cid:18) (cid:19)
n n
= −2 (δ −1)+ −1 (δ −δ )+(d−1).
n 2 n 1 2
2 1
Hence the code is optimal if the second condition in the theorem holds.
While Thm. 2.4, Thm. 2.5 provide optimality conditions that can be generalized to hierarchical locality of arbitrary levels, a
subject of discussion in Sec. III, the next theorem characterizes the conditions for optimality for two-level hierarchy without
imposing any restrictions.
Theorem 2.6: The [n,k,d]-code with [n ,r ,δ ]-middle codes and [n ,r ,δ ]-local codes constructed in Sec. II-D achieves
1 1 1 2 2 2
the optimal minimum distance if
(cid:24) (cid:18)(cid:24) (cid:25) (cid:19) (cid:25) (cid:24) (cid:25) (cid:18)(cid:24) (cid:25) (cid:19)(cid:24) (cid:25)
k k r k k r
− −1 1 = − −1 1 . (13)
r r r r r r
2 1 2 2 1 2
Proof: Let c(X) denote the code polynomial. The proof is analogous to that of Thm. 2.4. It is possible to count the size
of Z \Exp(c) as
n
(cid:18)(cid:18)(cid:24) (cid:25) (cid:19) (cid:19)(cid:18)(cid:24) (cid:25) (cid:19)
r k
1 −1 (δ −1)+(δ −1) −1
r 2 1 r
2 1
(cid:16) (cid:16)(cid:108) (cid:109) (cid:17) (cid:17) 
k− k −1 r
+ r1r 1 −1(δ2−1)+(d−1).
 2 
The expression can be recast into the form
(cid:18)(cid:24) (cid:25) (cid:19)
k
−1 (δ −δ )+(d−1)+
r 1 2
1
(cid:18)(cid:24) (cid:18)(cid:24) (cid:25) (cid:19) (cid:25) (cid:18)(cid:24) (cid:25) (cid:19)(cid:24) (cid:25) (cid:19)
k k r k r
− −1 1 + −1 1 −1 (δ −1),
r r r r r 2
2 1 2 1 2
thus leading to a value of d as in
(cid:18)(cid:24) (cid:25) (cid:19)
k
d = n−k+1− −1 (δ −δ )
r 1 2
1
(cid:18)(cid:24) (cid:18)(cid:24) (cid:25) (cid:19) (cid:25) (cid:18)(cid:24) (cid:25) (cid:19)(cid:24) (cid:25) (cid:19)
k k r k r
− − −1 1 + −1 1 −1 (δ −1).
r r r r r 2
2 1 2 1 2
Comparing against the upper bound in (1), we conclude that the code is optimal if (13) holds.
10
III. GENERALIZATIONTOh-LEVELHIERARCHY
In Sec. II, we considered codes with hierarchical locality where the hierarchy had two levels. Here we extend the notion to
h-level hierarchy where h is an arbitrary number.
Definition 3: An [n,k,d] linear code C is a code with h-level hierarchical locality having locality parameters
[(r ,δ ),(r ,δ ),...(r ,δ )] if for every symbol c ,1≤i≤n, there exists a punctured code C such that c ∈Supp(C ) and
1 1 2 2 h h i i i i
the following conditions hold:
1) dim(C )≤r ,
i 1
2) d (C )≥δ ,
min i 1
3) C is a code with (h−1)-level hierarchical locality having locality parameters [(r ,δ ),(r ,δ ),...(r ,δ )].
i 2 2 3 3 h h
Code with 1-level hierarchical locality is defined to be code with locality.
The punctured code C associated with c is referred to as local code of level-1. In fact, each symbol is associated with
i i
a bunch of local codes, each of level-i, i = 1,2,...,h. In the previous section, we studied codes with 2-level hierarchical
locality.
A. An Upper Bound on The Minimum Distance
Theorem 3.1: Let C be an [n,k,d]-linear code with h-level hierarchical locality having locality parameters
[(r ,δ ),(r ,δ ),...,(r ,δ )]. Then
1 1 2 2 h h
(cid:96)=h−1(cid:18)(cid:18)(cid:24) (cid:25) (cid:19) (cid:19)
(cid:88) k
d ≤ n−k+1− −1 (δ −δ )
r (cid:96) (cid:96)+1
(cid:96)
(cid:96)=1
(cid:18)(cid:24) (cid:25) (cid:19)
k
− −1 (δ −1). (14)
r h
h
Proof:TheproofisastraightforwardextensionofthatofThm.2.1.Thealgorithm2,(seeflowchartinFig.5(b))identifies
a (k−1)-dimensional punctured code C of C, having large support. Then we will use the Fact in (2). The algorithm identifies
s
a level-1 code that accumulates rank, and subsequently visits a level-2 code from within that, and continues recursively upto
reaching a level-(h−1) code that accumulates rank. Then it picks up all the level-h codes that accumulate rank. If no more
level-h codes can be picked up, it steps back one level up, and finds a new level-(h−1) code that accumulates rank. This
can be viewed as a depth-first search for rank-accumulating level-h codes. At each level, incremental support is added to the
variable Ψ. Vaguely speaking, the incremental support that is added at each level depends on the minimum distance of the
code at that level. Let a denote the incremental rank and s denote the incremental support while adding a level-h code
ih ih
Algorithm 2 For the proof of Thm. 3.1
1: Let i1 =0,i2 =0,...,ih =0;Ψ=Φ,(cid:96)=1;
2: M0 =C,M1 =Φ,M2 =Φ,...,Mh =Φ
3: while (rank(G|Ψ)<k) do
4: if (∃ a level-(cid:96) code Li(cid:96) ∈M(cid:96)−1 such that rank(G|Ψ∪Supp(Li(cid:96)))>rank(G|Ψ)) then
5: M(cid:96) =Li(cid:96)
6: if ((cid:96) equals h) then
7: Ψ=Ψ∪Supp(Mh)
8: Slast =Supp(Mh)
9: ih =ih+1
10: else
11: (cid:96)=(cid:96)+1
12: end if
13: else
14: (cid:96)=(cid:96)−1;
15: Ψ=(Ψ\Slast)∪T(cid:96)
16: Slast =T(cid:96)
17: i(cid:96) =i(cid:96)+1
18: end if
19: end while
L . By the algorithm, a >0. The set S denotes the support of the level-h code L . The set T denotes the incremental
ih ih ih ih (cid:96)
support of the level-(cid:96) code M , along with the columns from the last code that accumulated rank. It will contain at least
(cid:96)
(d −1) columns in addition to the incremental rank. Let i denote the final value of the variables i ,1≤(cid:96)≤h before the
(cid:96) (cid:96)end (cid:96)
algorithm terminates. Since a > 0, we can get non-trivial lower bounds on s and t := |T | quite similar to the proof of
ih ih (cid:96) (cid:96)