Distributed K-means over Compressed Binary Data

0.38 MB·

by  Elsa Dupraz

1
Distributed K-means over Compressed Binary Data
Elsa DUPRAZ
Telecom Bretagne; UMR CNRS 6285 Lab-STICC, Brest, France
Abstract—We consider a network of binary-valued sensors would like to perform K-means directly over the compressed
with a fusion center. The fusion center has to perform K-means data. Distributed K-means over compressed data raises three
clusteringonthebinarydatatransmittedbythesensors.Inorder
questions: (i) How should the data be compressed so that
toreducetheamountofdatatransmittedwithinthenetwork,the
the fusion center can perform K-means without having to
sensorscompresstheirdatawithasourcecodingschemebasedon
LDPCcodes.WeproposetoapplytheK-meansalgorithmdirectly reconstruct all the measurements? (ii) How good is clustering
over the compressed data without reconstructing the original overcompresseddatacomparedtoclusteringovertheoriginal
7 sensors measurements, in order to avoid potentially complex data? (iii) Is the rate needed to perform K-means lower than
1 decoding operations. We provide approximated expressions of
the rate needed to reconstruct all the data?
0 the error probabilities of the K-means steps in the compressed
Regarding the first two questions, [9], [10] considered real-
2 domain. From these expressions, we show that applying the K-
means algorithm in the compressed domain enables to recover valued measurement vectors compressed from Compressed
n
the clusters of the original domain. Monte Carlo simulations Sensing (CS) techniques, and proposed to apply the K-means
a
J illustrate the accuracy of the obtained approximated error algorithm directly in the compressed domain. CS consists of
probabilities, and show that the coding rate needed to perform computing M random linear combinations of the N com-
2
K-means clustering in the compressed domain is lower than the
1 ponents of the measurement vectors, with M ≤ N. The
rate needed to reconstruct all the measurements.
resultsof[9]showthatifM islargeenough,thecompression

] preserves the Euclidian distances with high probability, which
T
I. INTRODUCTION enablestoperformK-meansinthecompresseddomain.Other
I
. Networks of sensors have long been employed in various than K-means, detection and parameter estimation can be
s
c domains such as environmental monitoring, electrical energy applied over compressed real-valued data [11]–[13], but also
[ management, and medicine [1]. In particular, inexpensive over binary data compressed with LDPC codes [14], [15].

1 binary-valued sensors are successfully used in a wide range However,noneoftheseworksconsidertheK-meansalgorithm
v of applications, such as traffic control in telecommunication over compressed binary data.
3 systems [2], self-testing in nanonelectric devices [3], or activ- Regarding the third question, information theory has re-
0
ity recognition on home environments [4]. In this paper, we cently addressed data analysis tasks such as distributed hy-
4
consider a network of J binary-valued sensors that transmit pothesis testing [16], [17] or similarity queries [18], [19] over
3
0 their data to a fusion center. compressed data. These works provide analytic expressions
. In such network, the sensors potentially collect a large of the minimum rates that need to be transmitted in order to
1
amount of data. The fusion center may realize complex data perform the considered tasks, although these general analytic
0
7 analysis tasks by aggregating the sensors measurements and expressions can be difficult to evaluate for particular source
1 by exploiting the diversity of the collected data. Clustering is models. However, to the best of our knowledge, the K-means
v: a particular data analysis task that consists of separating the algorithm has not been studied yet with information theory.
i data in a given number of classes with similar characteristics. In this paper, we consider binary measurement vectors and
X
One of the most popular clustering methods is the K-means weassumethatthecompressionisrealizedfromLDPCcodes.
r algorithm [5] due to its simplicity and its efficiency. The K- We want to determine whether the K-means algorithm can
a
means algorithm groups the J measurement vectors into K be applied directly over the compressed data in order to
clusterssoastominimizetheaveragedistancebetweenvectors recovertheclustersoftheoriginaldata.Inthefollowing,after
inaclusterandtheclustercenter.Ifthemeasurementsarereal- describingthesourcecodingsystem(SectionII),weproposea
valued, K-means usually considers the Euclidian distance [5], formulation of the K-means algorithm over binary data in the
while in case of binary measurements, K-means relies on the compressed domain (Section III). We then carry a theoretical
Hamming distance [6]. analysis of the performance of the K-means algorithm in
In our context, the J sensors should send their measure- the compressed domain (Section IV). We in particular derive
ments to the fusion center in a compressed form in order approximated error probabilities of each of the two steps of
to greatly reduce the amount of data transmitted within the the K-means algorithm. The theoretical analysis shows that
network. Low Density Parity Check (LDPC) codes, initially applying the K-means algorithm in the compressed domain
introduced in the context of channel coding [7], have been permits to recover the clusters of the original domain. Monte
shown to be very efficient for distributed compression in Carlo simulations confirm the accuracy of the obtained ap-
a network of sensors [8]. However, the standard distributed proximatederrorprobabilities,andshowthattheeffectiverate
compressionframework[8]considersthatthefusioncenterhas needed to perform K-means over compressed data is lower
toreconstructallthemeasurementsfromallthesensors.Here, thantheratethatwouldbeneededtoreconstructallthesensors
inordertoavoidpotentiallycomplexdecodingoperations,we measurements (Section V).
2
II. SYSTEMDESCRIPTION by UM ⊆{0,1}M. The compressed domain UM depends on
the considered code H.
In this section, we first introduce our notations and as-
sumptionsforthebinarymeasurementvectorscollectedbythe As in [8], we assume that the vectors uj are transmitted
sensors. We then present the source coding technique based reliably to the fusion center. We consider this assumption in
on LDPC codes that is used in the system. order to focus on the source coding aspects of the problem,
and we do not describe the channel codes that should be used
in the system in order to satisfy this assumption. The source
A. Source Model
coding technique described by (2) was initially proposed
The network is composed by J sensors and a fusion center. in [8] in a context where the fusion center has to reconstruct
Each sensor j ∈ 1,J performs N binary measurements all the measurement vectors x . However, reconstructing all
j
xj,n ∈{0,1} that a(cid:74)re sto(cid:75)red in a vector xj of size N. the sensor measurements usually requires complex decoding
Consider K different clusters Ck where each cluster is operations and may need a higher coding rate r than simply
associated to a centroid θk of length N. The binary compo- applying K-means over the compressed data. Hence, in the
nents θk,n of θk are independent and identically distributed following,weproposetoapplytheK-meansalgorithmdirectly
(i.i.d.). with P(θk,n = 1) = pc. We assume that each overthecompressedvectorsuj,withouthavingtoreconstruct
measurement vector xj belongs to one of the K clusters. the original vectors xj.
The cluster assignment variables e are defined as e = 1
j,k j,k
if x ∈ C , e = 0 otherwise. Let Θ = {θ ,··· ,θ }
j k j,k 1 K III. K-MEANSALGORITHM
and E = {e ,··· ,e } be the sets of centroids and of
1,1 J,K
cluster assignment variables, respectively. Within cluster Ck, The K-means algorithm for clustering binary vectors xj in
each vector x ∈C is generated as the original domain XN was initially proposed in [6] and it
j k
makesuseoftheHammingdistance.Inthissection,werestate
xj =θk⊕bj, (1) the K-means algorithm of [6] in the compressed domain UM.
TheHammingdistancebetweentwovectorsa,b∈UM inthe
where ⊕ represents the XOR componentwise operation, and
compressed domain is defined as d(a,b)=(cid:80)M a ⊕b .
bj is a vector of size N with binary i.i.d. components such Denote ψ =HTθ and Ψ={ψ ,...,ψ } thme=c1ommpressmed
that P(b = 1) = p. In the following, we assume that the k k 1 K
j,n
versionsofthecentroidsθ .ApplyingtheK-meansalgorithm
cluster assignment variables e , the centroids θ , and the k
j,k k
in the compressed domain corresponds to minimizing the
parameter p and p are unknown. This model is equivalent
c
objective function
to the model presented in [20] for K-means clustering with
binary data. Some instances of the K-means algorithm have J K
(cid:88)(cid:88)
been proposed to deal with an unknown number of clusters F(Ψ,E)= e d(u ,ψ ). (3)
j,k j k
K [5]. However, here, as a first step, K is assumed to be j=1k=1
known in order to focus on the compression aspects of the
withrespecttothecompressedcentroidsψ andtothecluster
k
problem.
assignment variables e .
j,k
Each sensor has to transmit its data to the fusion center
We initialize the K-means algorithm with K compressed
that should perform K-means on the received data in order centroids ψ(0) that may be either selected at random among
to recover the cluster assignments E and the centroids Θ. k
the set of input vectors u , or obtained from the K-means++
j
We now describe the source coding technique that is used
procedure[21].DenotebyLthenumberofiterationsoftheK-
in our system in order to greatly reduce the amount of data
means algorithm. In the following, exponent (cid:96) always refers
transmitted to the fusion center.
to a quantity obtained at the (cid:96)-th iteration of the algorithm.
At iteration (cid:96)∈ 1,L , K-means proceeds in two steps. First,
B. Source Coding with Low Density Parity Check Codes fromthecentroid(cid:74)sψ((cid:75)(cid:96)−1) obtainedatiteration(cid:96)−1,itassigns
k
In [8], it is shown that LDPC codes are very efficient to each vector uj to a cluster as
performdistributedsourcecodinginanetworkofsensors,and (cid:40) 1 if d(u ,ψ((cid:96)−1))= min d(u ,ψ((cid:96)−1)),
in [14], [15] it is shown that they allow parameter estimation ∀j,∀k,e((cid:96)) = j k k(cid:48)∈ 1,K j k(cid:48)
overthecompresseddata.DenotebyH thebinaryparitycheck j,k 0 otherwise. (cid:74) (cid:75)
matrixofsizeN×M (M <N)ofanLDPCcode.Denoteby (4)
d (cid:28)M thenumberofnon-zerocomponentsinanyrowofH, Second, the algorithm updates the centroids as follows:
v
and denote by dc (cid:28) N the number of non-zero components  J
in any column of H. In our system, each sensor j transmits  1 if (cid:80)e((cid:96))u ≥ 1J((cid:96)),
∀j,∀n, ψ((cid:96)) = j,k j,n 2 k (5)
to the fusion center a binary vector uj of length M, obtained k,n  0 othej=rw1ise.
as
u =HTx . (2)
j j where J((cid:96)) is the number of vectors assigned to cluster k at
k
Foreachsensor,thecodingrateisgivenbyr = M = dv.The iteration(cid:96).Theclusterassignmentstep(4)assignseachvector
set of all the possible vectors x is called the oriNginalddcomain u to the cluster with the closest compressed centroid ψ((cid:96)).
j j k
anditisdenotedasXN ={0,1}N.Thesetofallthepossible The centroid computation step (5) is a majority voting oper-
vectors u is called the compressed domain and it is denoted ation which can be shown to minimize the average distances
j
3
between the centroid ψ((cid:96)) and all the vectors u assigned to can be expressed as P = P(A ≥ A ) which can be
k j a,k(cid:48) k(cid:48) k
cluster k at iteration (cid:96). approximated as
Following the same reasonning as for K-means in the
M M
original domain [6], it is easy to show that when applying P ≈(cid:88)(cid:88)P (A =u)P (A =v). (10)
a,k(cid:48) k k(cid:48)
K-means in the compressed domain, the objective function
u=0v=u
F(Ψ((cid:96)),E((cid:96))) is decreasing with (cid:96) and at least converges to
In order to get (10) we implicitly assume that the random
a local minimum. However, this property does not guarantee
variables A and A are independent, and hence (10) is only
that the cluster assignment variables e((cid:96)) obtained from the k k(cid:48)
j,k an approximation of Pa,k(cid:48). Assuming that the ak,m and ak(cid:48),m
algorithm in the compressed domain will correspond to the
are all independent, we get P (A =u) ≈ B (u,q ) and
k M 1
correct cluster assignments in the original domain. In order to
P (A =v) ≈ B (v,q ). To finish, the error probability of
k(cid:48) M 2
justify that the K-means algorithm applied in the compressed (cid:80)
the cluster assignment step is given by P = P =
domaincanrecoverthecorrectclustersoftheoriginaldomain, (K−1)P since P does not dependaon k(cid:48).k(cid:48)(cid:54)=k a,k(cid:48)
a,k(cid:48) a,k(cid:48)
we now propose a theoretical analysis of the two steps of the
It can be seen from (8) that the approximated error prob-
algorithm.
ability P does not depend on the considered cluster k. The
a
expression(8)isonlyanapproximationoftheerrorprobability
IV. K-MEANSPERFORMANCEEVALUATION of the cluster assignment step, since it assumes that the
In this section, in order to assess the performance of the K- components of the vector HTb are independent, which is
j
means algorithm in the compressed domain, we evaluate each not true in general. However, it is shown in [14], [22] that
stepofthealgorithmindividually.Weprovideanapproximated this assumption is reasonable for parameter estimation over
expression of the error probability of the cluster assignment LDPC codes. In Section V, we verify the accuracy of the
stepinthecompresseddomain,assumingthatthecompressed approximation by comparing the values of (8) to the error
clustercentroidsψ areperfectlyknown.Inthesameway,we probabilities measured from Monte Carlo simulations.
k
provide an approximated expression of the error probability
of the centroid estimation step in the compressed domain,
B. Error Probability of the Centroid Computation Step
assuming that the cluster assignment variables e are per-
j,k The following proposition evaluates the error probability of
fectlyknown.Althoughevaluatedinthemostfavorablecases,
the centroid computation step in the compressed domain.
these error probabilities will enable us determine whether it
is reasonable to apply K-means in the compressed domain in Proposition 2. Let Ψˆ be the estimated compressed centroids
k
order to recover the clusters of the original domain. obtained after applying the centroid estimation step (5) to the
The expressions of the error probabilities we derive rely on true cluster assignment variables e . The error probability
j,k
two functions B and f defined as P = P(ψˆ (cid:54)= ψ ) for cluster k can be approximated
M c,k k,m k,m
(cid:18) (cid:19) as
M
BM(m,p)= m pm(1−p)M−m, (6) P ≈ (cid:88)Jk B (j,p ) (11)
c,k Jk d
1 1
f(d,p)= − (1−2p)d. (7) j=(cid:100)Jk(cid:101)
2 2 2
where J is the number of vectors in cluster k, and p =
k d
A. Error Probability of the Cluster Assignment Step f(dc,p).
The following proposition evaluates the error probability Proof: From the model defined in Section II, a codeword
of the cluster assignment step (4) applied to the compressed u (j ∈C ), can be expressed as
j k
centroids ψ .
k u =HT(θ ⊕b )=ψ ⊕a , (12)
j k j k j
Proposition 1. Let eˆ be the cluster assignments obtained
j,k where a =HTb is such that P(a =1)=p . Let A =
whenapplyingtheclusterassignmentstep(4)tothetruecom- j j j,m d j
(cid:80)Jk a . The error probability of the centroid computation
pressed centroids ψk. The error probability Pa = P(eˆj,k = j=1 j,m
step can be evaluated as
0|x ∈C ) can be approximated as
j k
(cid:88)M (cid:88)M P =P (cid:18)A ≥ Jk(cid:19)≈ (cid:88)Jk B (j,p ). (13)
Pa ≈(K−1) BM(m1,q1)BM(m2,q2) (8) c,k j 2 Jk d
m1=0m2=m1 j=(cid:100)J2k(cid:101)
with q =f(d ,p), q =f(d ,(1−p)f(2,p )). Theapproximationcomesfromthefactthat(11)assumesthat
1 c 2 c c
the a are independent.
Proof: We first evaluate the error probabilities P = j,m
a,k(cid:48) It can be seen from (11) that the approximated error prob-
P(eˆ =1|x ∈C ), ∀k(cid:48) (cid:54)=k. Let
j,k(cid:48) j k ability P only depends on the considered cluster k through
c,k
a =u ⊕ψ =HTb , the number J of vectors in cluster C . The expression (8)
k j k j k k
a =u ⊕ψ =HT(θ ⊕θ ⊕b ). (9) is only an approximation of the error probability of the
k(cid:48) j k(cid:48) k k(cid:48) j
centroidassignmentstepforthesamereasonsasforthecluster
anddefineA =(cid:80)M a ,A =(cid:80)M a .According assignment step. We will also verify the accuracy of this
k m=1 k,m k(cid:48) m=1 k(cid:48),m
to the cluster assignment step (4), the error probability P approximation in Section V.
a,k(cid:48)
4
1e+0 1e+0
1e+0
1e-1 1e-1 1e-1
1e-2 1e-2 1e-2
Pe1e-3 Pe1e-3 Pe1e-3
1e-4 1e-4
1e-4
1e-5 M=250, theoretic 1e-5 M=250, theoretic
1e-6 MM==255000,, pthreaoctriectaicl 1e-6 MM==255000,, pthreaoctriectaicl 1e-5 M=250
M=500, practical M=500, practical M=500
1e-7 1e-7 1e-6
0.05 0.1 0.15 0.2 0.25 0.3 0.35 0 0.05 0.1 0.15 0.2 0 0.05 0.1 0.15 0.2
p p p
(a) (b) (c)
Fig. 1. (a) Comparison of error probability predicted by theory with error probability measured in practice for the cluster assignment step
(b) Comparison of error probability predicted by theory with error probability measured in practice for the centroid estimation step
(c) ErrorprobabilityoftheK-meansalgorithmwithL=10iterations
V. SIMULATIONRESULTS From [24], the minimum coding rate (per symbol normalized
by the number of sources J) the fusion center should receive
In this section, we evaluate through simulations the perfor-
to reconstruct all the x is given by R = 1H(X ,··· ,X ),
mance of the K-means algorithm in the compressed domain. j d J 1 J
where H(X ,··· ,X ) is the joint entropy of the sources
We first consider each step of the algorithm individually, and 1 J
(X ,··· ,X ). Since no closed-form expression of R exist,
weverifytheaccuracyoftheapproximatederrorprobabilities 1 J d
we evaluate the rate R from Monte-Carlo simulations.
obtained in Section IV. We then assess the performance of d
We run over Nt = 10000 simulations the K-means al-
the full algorithm and we evaluate the rate needed to perform
gorithm in the compressed domain initialized with the K-
K-means over compressed data. Throughout the section, we
means++ procedure and L = 10 iterations. In order to
set J = 200, K = 4, p = 0.1 and we consider two LDPC
c
evaluate the performance of the algorithm, we measure the
codes of length N = 1000 with d = 2. The two codes are
v
error probability of the cluster assignments decided by the
constructedfromtheProgressiveEdgeGrowthalgorithm[23];
algorithminthecompresseddomainwithrespecttothecorrect
the first code is of rate r = 1/4 with M = 250 and d = 8,
c
clustersintheoriginaldomain.Figure1(c)representstheerror
and the second one is of rate r = 1/2, with M = 500 and
probabilitieswithrespecttopobtainedforthetwoconsidered
d = 4. We set d = 2 for the two considered codes, since
c v
codes. As expected, the error probability is increasing with p
it can be shown from (8) and (11) that the error probabilities
and is decreasing with the coding rate r.
P and P are increasing with d (d is necessarily greater
a c,k v v
We then compare the rate needed to perform K-means over
than 2).
compressed data to the rate needed to reconstruct all the
A. Accuracy of the error probability approximations sensors measurements. For pc = 0.1 and p = 0.1, we get
R = 0.68 bits/symbol, and, for p = 0.05 and p = 0.1, we
We compare the approximated expressions P (4) and d c
a
obtainR =0.43bits/symbol.TheresultsofFigure1(c)show
P (11) with the effective error probabilities measured from d
c,k
thatforp =0.1andp=0.1,thecodeofrater =1/2<0.68
Monte Carlo simulations for each step of the algorithm for c
enables to perform K-means with an error probability lower
the two constructed codes over Nt = 10000 simulations.
than 10−6. For p = 0.1 and p = 0.05, the code of rate
Figure 1(a) represents the obtained error probabilities for the c
r = 1/4 < 0.43 also enables to perform K-means with a
cluster assignment step, while Figure 1(b) represents the cen-
low error probability P = 10−5. This shows that the rate
troidestimationstep.Weseethatforthetwoconsideredcodes, e
needed to perform K-means is lower than the rate needed to
the theoretic error probabilities P and P are close to the
a c,k
reconstruct all the sensors measurements, which justifies the
measurederrorprobabilitiesforthetwostepsofthealgorithm,
use of the method presented in the paper.
which shows the accuracy of the proposed approximations.
Figure 1(a) also illustrates that the cluster assignment step in
thecompresseddomaincanindeedrecoverthecorrectclusters VI. CONCLUSION
of the original domain, since it is possible to reach error In this paper, we considered a network of sensors which
probabilities from 10−3 to 10−7. The same conclusion holds transmit their compressed binary measurements to a fusion
for Figure 1(b) for the centroid estimation step. center. We proposed to apply the K-means algorithm directly
over the compressed data, without reconstructing the sensor
B. K-means algorithm and rate evaluation
measurements. From a theoretical analysis and Monte Carlo
In addition to analyzing the performance of the K-means simulations,weshowedtheefficiencyofapplyingK-meansin
algorithminthecompresseddomain,wealsowanttocompare the compressed domain. We also showed that the rate needed
the coding rate r needed to perform K-means to the rate that to perform K-means on the compressed vectors is lower than
would be needed to reconstruct all the sensors measurements. the rate needed to reconstruct all the measurements.
5
REFERENCES
[1] J.Yick,B.Mukherjee,andD.Ghosal,“Wirelesssensornetworksurvey,”
Computernetworks,vol.52,no.12,pp.2292–2330,2008.
[2] L. Y. Wang, J.-F. Zhang, and G. G. Yin, “System identification using
binary sensors,” IEEE Transactions on Automatic Control, vol. 48,
no.11,pp.1892–1907,2003.
[3] E. Colinet and J. Juillard, “A weighted least-squares approach to
parameter estimation problems based on binary measurements,” IEEE
TransactionsonAutomaticControl,vol.55,no.1,pp.148–152,2010.
[4] F.J.Ordo´n˜ez,P.deToledo,andA.Sanchis,“Activityrecognitionusing
hybrid generative/discriminative models on home environments using
binarysensors,”Sensors,vol.13,no.5,pp.5460–5477,2013.
[5] A. K. Jain, “Data clustering: 50 years beyond K-means,” Pattern
recognitionletters,vol.31,no.8,pp.651–666,2010.
[6] Z. Huang, “Extensions to the K-means algorithm for clustering large
datasetswithcategoricalvalues,”Dataminingandknowledgediscovery,
vol.2,no.3,pp.283–304,1998.
[7] T. Richardson and R. Urbanke, “The capacity of low-density parity-
check codes under message-passing decoding,” IEEE Transactions on
InformationTheory,vol.47,no.2,pp.599–618,2001.
[8] Z. Xiong, A. Liveris, and S. Cheng, “Distributed source coding for
sensor networks,” IEEE Signal Processing. Magazine, vol. 21, no. 5,
pp.80–94,2004.
[9] C.Boutsidis,A.Zouzias,M.W.Mahoney,andP.Drineas,“Randomized
dimensionalityreductionforK-meansclustering,”IEEETransactionson
InformationTheory,vol.61,no.2,pp.1045–1062,2015.
[10] F. Pourkamali-Anaraki and S. Becker, “Preconditioned data sparsifica-
tionforbigdatawithapplicationstoPCAandK-means,”arXivpreprint
arXiv:1511.00152,2015.
[11] M. Davenport, P. T. Boufounos, M. B. Wakin, R. G. Baraniuk et al.,
“Signal processing with compressive measurements,” IEEE Journal of
SelectedTopicsinSignalProcessing,vol.4,no.2,pp.445–460,2010.
[12] A.Bourrier,R.Gribonval,andP.Perez,“Compressivegaussianmixture
estimation,”inIEEEInternationalConferenceonAcoustics,Speechand
SignalProcessing,2013,pp.6024–6028.
[13] A. Zebadua, P. O. Amblard, E. Moisan, and O. Michel, “Compressed
andquantizedcorrelationestimators,”ToappearinIEEETransactions
onSignalProcessing,2016.
[14] V. Toto-Zarasoa, A. Roumy, and C. Guillemot, “Maximum likelihood
bscparameterestimationfortheslepian-wolfproblem,”IEEECommu-
nicationsLetters,vol.15,no.2,pp.232–234,2011.
[15] S.Wang,L.Cui,L.Stankovic,V.Stankovic,andS.Cheng,“Adaptive
correlation estimation with particle filtering for distributed video cod-
ing,”IEEETransactionsonCircuitsandSystemsforVideoTechnology,
vol.22,no.5,pp.649–658,May2012.
[16] M. S. Rahman and A. B. Wagner, “On the optimality of binning
for distributed hypothesis testing,” IEEE Transactions on Information
Theory,vol.58,no.10,pp.6282–6303,2012.
[17] G. Katz, P. Piantanida, and M. Debbah, “Distributed binary detection
withlossydatacompression,”arXivpreprintarXiv:1601.01152,2016.
[18] E. Tuncel and D. Gunduz, “Identification and lossy reconstruction in
noisy databases,” IEEE Transactions on Information Theory, vol. 60,
no.2,pp.822–831,2014.
[19] A. Ingber, T. Courtade, and T. Weissman, “Compression for quadratic
similarityqueries,”IEEETransactionsonInformationTheory,vol.61,
no.5,pp.2729–2747,2015.
[20] T. Li, “A general model for clustering binary data,” in Proceedings
of the eleventh ACM SIGKDD international conference on Knowledge
discoveryindatamining,2005,pp.188–197.
[21] D.ArthurandS.Vassilvitskii,“K-means++:Theadvantagesofcareful
seeding,” in Proceedings of the eighteenth annual ACM-SIAM sym-
posium on Discrete algorithms. Society for Industrial and Applied
Mathematics,2007,pp.1027–1035.
[22] E. Dupraz, A. Roumy, and M. Kieffer, “Source coding with side
informationatthedecoderanduncertainknowledgeofthecorrelation,”
IEEE Transactions on Communications, vol. 62, no. 1, pp. 269–279,
2014.
[23] X.-Y.Hu,E.Eleftheriou,andD.M.Arnold,“Regularandirregularpro-
gressiveedge-growthtannergraphs,”IEEETransactionsonInformation
Theory,vol.51,no.1,pp.386–398,2005.
[24] D.SlepianandJ.K.Wolf,“Noiselesscodingofcorrelatedinformation
sources,”IEEETransactionsonInformationTheory,vol.19,no.4,pp.
471–480,1973.