Distributed source coding : theory and practice

363 Pages·2017·12.016 MB·English

by  Cheng, Samuel, Fang, Yong, Wang, Shuang

(cid:2)
DistributedSourceCoding
(cid:2) (cid:2)
(cid:2)
(cid:2)
Distributed Source Coding
TheoryandPractice
ShuangWang,YongFang,andSamuelCheng
(cid:2) (cid:2)
(cid:2)
(cid:2)
Thiseditionfirstpublished2017
©2017JohnWiley&SonsLtd
RegisteredofficeJohnWiley&SonsLtd,TheAtrium,SouthernGate,Chichester,WestSussex,
PO198SQ,UnitedKingdom
Fordetailsofourglobaleditorialoffices,forcustomerservicesandforinformationabouthowto
applyforpermissiontoreusethecopyrightmaterialinthisbookpleaseseeourwebsiteat
www.wiley.com.
TherightofShuangWang,YongFang,andSamuelChengtobeidentifiedastheauthorsofthis
workhasbeenassertedinaccordancewiththeCopyright,DesignsandPatentsAct1988.
Allrightsreserved.Nopartofthispublicationmaybereproduced,storedinaretrievalsystem,or
transmitted,inanyformorbyanymeans,electronic,mechanical,photocopying,recordingor
(cid:2) otherwise,exceptaspermittedbytheUKCopyright,DesignsandPatentsAct1988,withoutthe (cid:2)
priorpermissionofthepublisher.
Wileyalsopublishesitsbooksinavarietyofelectronicformats.Somecontentthatappearsin
printmaynotbeavailableinelectronicbooks.
Designationsusedbycompaniestodistinguishtheirproductsareoftenclaimedastrademarks.
Allbrandnamesandproductnamesusedinthisbookaretradenames,servicemarks,
trademarksorregisteredtrademarksoftheirrespectiveowners.Thepublisherisnotassociated
withanyproductorvendormentionedinthisbook.
LimitofLiability/DisclaimerofWarranty:Whilethepublisherandauthorhaveusedtheirbest
effortsinpreparingthisbook,theymakenorepresentationsorwarrantieswithrespecttothe
accuracyorcompletenessofthecontentsofthisbookandspecificallydisclaimanyimplied
warrantiesofmerchantabilityorfitnessforaparticularpurpose.Itissoldontheunderstanding
thatthepublisherisnotengagedinrenderingprofessionalservicesandneitherthepublishernor
theauthorshallbeliablefordamagesarisingherefrom.Ifprofessionaladviceorotherexpert
assistanceisrequired,theservicesofacompetentprofessionalshouldbesought
LibraryofCongressCataloging-in-Publicationdataappliedfor
ISBN:9780470688991
AcataloguerecordforthisbookisavailablefromtheBritishLibrary.
CoverdesignbyWiley
Coverimage:Topnumbersimage:a_Taiga/Gettyimages
Bottommolecularimage:KTSDESIGN/SCIENCEPHOTOLIBRARY/Gettyimages
Setin10/12ptWarnockProbySPiGlobal,Chennai,India
10 9 8 7 6 5 4 3 2 1
(cid:2)
(cid:2)
v
Contents
Preface xiii
Acknowledgment xv
AbouttheCompanionWebsite xvii
1 Introduction 1
1.1 WhatisDistributedSourceCoding? 2
1.2 HistoricalOverviewandBackground 2
1.3 PotentialandApplications 3
(cid:2) 1.4 Outline 4 (cid:2)
PartI TheoryofDistributedSourceCoding 7
2 LosslessCompressionofCorrelatedSources 9
2.1 Slepian–WolfCoding 10
2.1.1 ProofoftheSWTheorem 15
AchievabilityoftheSWTheorem 16
ConverseoftheSWTheorem 19
2.2 AsymmetricandSymmetricSWCoding 21
2.3 SWCodingofMultipleSources 22
3 Wyner–ZivCodingTheory 25
3.1 ForwardProofofWZCoding 27
3.2 ConverseProofofWZCoding 29
3.3 Examples 30
3.3.1 DoublySymmetricBinarySource 30
ProblemSetup 30
AProposedScheme 31
VerifytheOptimalityoftheProposedScheme 32
3.3.2 QuadraticGaussianSource 35
(cid:2)
(cid:2)
vi Contents
ProblemSetup 35
ProposedScheme 36
VerifytheOptimalityoftheProposedScheme 37
3.4 RateLossoftheWZProblem 38
BinarySourceCase 39
RatelossofGeneralCases 39
4 LossyDistributedSourceCoding 41
4.1 Berger–TungInnerBound 42
4.1.1 Berger–TungScheme 42
CodebookPreparation 42
Encoding 42
Decoding 43
4.1.2 DistortionAnalysis 43
Pr( )→0∶ 44
1
Pr(C ∩ )→0∶ 44
1 2
Pr(C ∩C ∩ )→0∶ 44
1 2 3
Pr( )→0 44
4
Pr( ),Pr( )→0 44
5 6
4.2 IndirectMultiterminalSourceCoding 45
(cid:2) 4.2.1 QuadraticGaussianCEOProblemwithTwoEncoders 45 (cid:2)
ForwardProofofQuadraticGaussianCEOProblemwithTwo
Terminals 46
ConverseProofofQuadraticGaussianCEOProblemwithTwo
Terminals 48
4.3 DirectMultiterminalSourceCoding 54
4.3.1 ForwardProofofGaussianMultiterminalSourceCodingProblem
withTwoSources 55
Case1:(1− )≤𝜌2(1− ) 59
2 1
Case2:(1− )>𝜌2(1− ) 62
2 1
4.3.2 ConverseProofofGaussianMultiterminalSourceCodingProblem
withTwoSources 63
BoundsforR andR 64
1 2
CollaborativeLowerBound 66
𝜇-sumBound 67
PartII Implementation 75
5 Slepian–WolfCodeDesignsBasedonChannelCoding 77
5.1 AsymmetricSWCoding 77
5.1.1 BinningIdea 78
5.1.2 Syndrome-basedApproach 79
(cid:2)
(cid:2)
Contents vii
HammingBinning 80
SWEncoding 80
SWDecoding 80
LDPC-basedSWCoding 81
5.1.3 Parity-basedApproach 82
5.1.4 Syndrome-basedVersusParity-basedApproach 84
5.2 Non-asymmetricSWCoding 85
5.2.1 GeneralizedSyndrome-basedApproach 86
5.2.2 ImplementationusingIRACodes 88
5.3 AdaptiveSlepian–WolfCoding 90
5.3.1 Particle-basedBeliefPropagationforSWCoding 91
5.4 LatestDevelopmentsandTrends 93
6 DistributedArithmeticCoding 97
6.1 ArithmeticCoding 97
6.2 DistributedArithmeticCoding 101
6.3 DefinitionoftheDACSpectrum 103
6.3.1 Motivations 103
6.3.2 InitialDACSpectrum 104
6.3.3 Depth-iDACSpectrum 105
(cid:2) 6.3.4 SomeSimplePropertiesoftheDACSpectrum 107 (cid:2)
6.4 FormulationoftheInitialDACSpectrum 107
6.5 ExplicitFormoftheInitialDACSpectrum 110
6.6 EvolutionoftheDACSpectrum 113
6.7 NumericalCalculationoftheDACSpectrum 116
6.7.1 NumericalCalculationoftheInitialDACSpectrum 117
6.7.2 NumericalEstimationofDACSpectrumEvolution 118
6.8 AnalysesonDACCodeswithSpectrum 120
6.8.1 DefinitionofDACCodes 121
6.8.2 CodebookCardinality 122
6.8.3 CodebookIndexDistribution 123
6.8.4 RateLoss 123
6.8.5 DecoderComplexity 124
6.8.6 DecodingErrorProbability 126
6.9 ImprovedBinaryDACCodec 130
6.9.1 PermutatedBDACCodec 130
Principle 130
ProofofSWLimitAchievability 131
6.9.2 BDACDecoderwithWeightedBranching 132
6.10 ImplementationoftheImprovedBDACCodec 134
6.10.1 Encoder 134
Principle 134
Implementation 135
(cid:2)
(cid:2)
viii Contents
6.10.2 Decoder 135
Principle 135
Implementation 136
6.11 ExperimentalResults 138
EffectofSegmentSizeonPermutationTechnique 139
EffectofSurviving-PathNumberonWBTechnique 139
ComparisonwithLDPCCodes 139
ApplicationofPBDACtoNonuniformSources 140
6.12 Conclusion 141
7 Wyner–ZivCodeDesign 143
7.1 VectorQuantization 143
7.2 LatticeTheory 146
7.2.1 WhatisaLattice? 146
Examples 146
DualLattice 147
IntegralLattice 147
LatticeQuantization 148
7.2.2 WhatisaGoodLattice? 149
PackingEfficiency 149
(cid:2) CoveringEfficiency 150 (cid:2)
NormalizedSecondMoment 150
KissingNumber 150
SomeGoodLattices 151
7.3 NestedLatticeQuantization 151
Encoding/decoding 152
CosetBinning 152
QuantizationLossandBinningLoss 153
SWCodedNLQ 154
7.3.1 TrellisCodedQuantization 154
7.3.2 PrincipleofTCQ 155
GenerationofCodebooks 156
GenerationofTrellisfromConvolutionalCodes 156
MappingofTrellisBranchesontoSub-codebooks 157
Quantization 157
Example 158
7.4 WZCodingBasedonTCQandLDPCCodes 159
7.4.1 StatisticsofTCQIndices 159
7.4.2 LLRofTrellisBits 162
7.4.3 LLRofCodewordBits 163
7.4.4 MinimumMSEEstimation 163
7.4.5 RateAllocationofBit-planes 164
7.4.6 ExperimentalResults 166
(cid:2)
(cid:2)
Contents ix
PartIII Applications 167
8 Wyner–ZivVideoCoding 169
8.1 BasicPrinciple 169
8.2 BenefitsofWZVideoCoding 170
8.3 KeyComponentsofWZVideoDecoding 171
8.3.1 Side-informationPreparation 171
BidirectionalMotionCompensation 172
8.3.2 CorrelationModeling 173
ExploitingSpatialRedundancy 174
8.3.3 RateController 175
8.4 OtherNotableFeaturesofMiscellaneousWZVideoCoders 175
9 CorrelationEstimationinDVC 177
9.1 BackgroundtoCorrelationParameterEstimationinDVC 177
9.1.1 CorrelationModelinWZVideoCoding 177
9.1.2 OfflineCorrelationEstimation 178
PixelDomainOfflineCorrelationEstimation 178
TransformDomainOfflineCorrelationEstimation 180
9.1.3 OnlineCorrelationEstimation 181
(cid:2) PixelDomainOnlineCorrelationEstimation 182 (cid:2)
TransformDomainOnlineCorrelationEstimation 184
9.2 RecapofBeliefPropagationandParticleFilterAlgorithms 185
9.2.1 BeliefPropagationAlgorithm 185
9.2.2 ParticleFiltering 186
9.3 CorrelationEstimationinDVCwithParticleFiltering 187
9.3.1 FactorGraphConstruction 187
9.3.2 CorrelationEstimationinDVCwithParticleFiltering 190
9.3.3 ExperimentalResults 192
9.3.4 Conclusion 197
9.4 LowComplexityCorrelationEstimationusingExpectation
Propagation 199
9.4.1 SystemArchitecture 199
9.4.2 FactorGraphConstruction 199
JointBit-planeSWCoding(RegionII) 200
CorrelationParameterTracking(RegionI) 201
9.4.3 MessagePassingontheConstructedFactorGraph 202
ExpectationPropagation 203
9.4.4 PosteriorApproximationoftheCorrelationParameterusing
ExpectationPropagation 204
MomentMatching 205
9.4.5 ExperimentalResults 206
9.4.6 Conclusion 211
(cid:2)
(cid:2)
x Contents
10 DSCforSolarImageCompression 213
10.1 Background 213
10.2 RelatedWork 215
10.3 DistributedMulti-viewImageCoding 217
10.4 AdaptiveJointBit-planeWZDecodingofMulti-viewImageswith
DisparityEstimation 217
10.4.1 JointBit-planeWZDecoding 217
10.4.2 JointBit-planeWZDecodingwithDisparityEstimation 219
10.4.3 JointBit-planeWZDecodingwithCorrelationEstimation 220
10.5 ResultsandDiscussion 221
10.6 Summary 224
11 SecureDistributedImageCoding 225
11.1 Background 225
11.2 SystemArchitecture 227
11.2.1 CompressionofEncryptedData 228
11.2.2 JointDecompressionandDecryptionDesign 230
11.3 PracticalImplementationIssues 233
11.4 ExperimentalResults 233
11.4.1 ExperimentSetup 234
(cid:2) 11.4.2 SecurityandPrivacyProtection 235 (cid:2)
11.4.3 CompressionPerformance 236
11.5 Discussion 239
12 SecureBiometricAuthenticationUsingDSC 241
12.1 Background 241
12.2 RelatedWork 243
12.3 SystemArchitecture 245
12.3.1 FeatureExtraction 246
12.3.2 FeaturePre-encryption 248
12.3.3 SeDSCEncrypter/decrypter 248
12.3.4 Privacy-preservingAuthentication 249
12.4 SeDSCEncrypterDesign 249
12.4.1 Non-asymmetricSWCodeswithCodePartitioning 250
12.4.2 ImplementationofSeDSCEncrypterusingIRACodes 251
12.5 SeDSCDecrypterDesign 252
12.6 Experiments 256
12.6.1 DatasetandExperimentalSetup 256
12.6.2 FeatureLengthSelection 257
12.6.3 AuthenticationAccuracy 257
AuthenticationPerformancesonSmallFeatureLength(i.e.,
N =100) 257
PerformancesonLargeFeatureLengths(i.e.,N ≥300) 258
(cid:2)
(cid:2)
Contents xi
12.6.4 PrivacyandSecurity 259
12.6.5 ComplexityAnalysis 261
12.7 Discussion 261
A BasicInformationTheory 263
A.1 InformationMeasures 263
A.1.1 Entropy 263
A.1.2 RelativeEntropy 267
A.1.3 MutualInformation 268
A.1.4 EntropyRate 269
A.2 IndependenceandMutualInformation 270
A.3 VennDiagramInterpretation 273
A.4 ConvexityandJensen’sInequality 274
A.5 DifferentialEntropy 277
A.5.1 GaussianRandomVariables 278
A.5.2 EntropyPowerInequality 278
A.6 Typicality 279
A.6.1 JointlyTypicalSequences 282
A.7 PackingLemmasandCoveringLemmas 284
A.8 Shannon’sSourceCodingTheorem 286
(cid:2) A.9 LossySourceCoding—Rate-distortionTheorem 289 (cid:2)
A.9.1 Rate-distortionProblemwithSideInformation 291
B BackgroundonChannelCoding 293
B.1 LinearBlockCodes 294
B.1.1 SyndromeDecodingofBlockCodes 295
B.1.2 HammingCodes,PackingBound,andPerfectCodes 295
B.2 ConvolutionalCodes 297
B.2.1 ViterbiDecodingAlgorithm 298
B.3 Shannon’sChannelCodingTheorem 301
B.3.1 AchievabilityProofoftheChannelCodingTheorem 303
B.3.2 ConverseProofofChannelCodingTheorem 305
B.4 Low-densityParity-checkCodes 306
B.4.1 AQuickSummaryofLDPCCodes 306
B.4.2 BeliefPropagationAlgorithm 307
B.4.3 LDPCDecodingusingBP 312
B.4.4 IRACodes 314
C ApproximateInference 319
C.1 StochasticApproximation 319
C.1.1 ImportanceSamplingMethods 320
C.1.2 MarkovChainMonteCarlo 321
MarkovChains 321
(cid:2)