Introduction to data compression

765 Pages·2012·22.652 MB·English

by  Sayood, Khalid

F O U R T H E D I T I O N
Introduction to
Data Compression
The Morgan Kaufmann Series in Multimedia Information and Systems
Series Editor, Edward A. Fox, Virginia Polytechnic University
Introduction to Data Compression, Third Edition
Khalid Sayood
Understanding Digital Libraries, Second Edition
Michael Lesk
Bioinformatics: Managing Scientific Data
Zoe Lacroix and Terence Critchlow
How to Build a Digital Library
Ian H. Witten and David Bainbridge
Digital Watermarking
Ingemar J. Cox, Matthew L. Miller, and Jeffrey A. Bloom
Readings in Multimedia Computing and Networking
Edited by Kevin Jeffay and HongJiang Zhang
Introduction to Data Compression, Second Edition
Khalid Sayood
Multimedia Servers: Applications, Environments, and Design
Dinkar Sitaram and Asit Dan
Managing Gigabytes: Compressing and IndexingDocuments and Images, SecondEdition
Ian H. Witten, Alistair Moffat, and Timothy C. Bell
Digital Compression for Multimedia: Principles and Standards
Jerry D. Gibson, Toby Berger, Tom Lookabaugh, Dave Lindbergh, and Richard L. Baker
Readings in Information Retrieval
Edited by Karen Sparck Jones and Peter Willett
F O U R T H E D I T I O N
Introduction to
Data Compression
Khalid Sayood
University of Nebraska
AMSTERDAM dBOSTON dHEIDELBERG dLONDON
NEW YORK dOXFORD dPARIS dSAN DIEGO
SAN FRANCISCO dSINGAPORE dSYDNEY dTOKYO
MorganKaufmannisanimprintofElsevier
AcquiringEditor:AndreaDierna
DevelopmentEditor:MeaganWhite
ProjectManager:DanielleS.Miller
Designer:EricDeCicco
MorganKaufmannisanimprintofElsevier
225WymanStreet,Waltham,MA02451,USA
(cid:2)2012Elsevier,Inc.Allrightsreserved.
Nopartofthispublicationmaybereproducedortransmittedinanyformorbyanymeans,electronicormechanical,
includingphotocopying,recording,oranyinformationstorageandretrievalsystem,withoutpermissioninwritingfromthe
publisher.Detailsonhowtoseekpermission,furtherinformationaboutthePublisher’spermissionspoliciesandour
arrangementswithorganizationssuchastheCopyrightClearanceCenterandtheCopyrightLicensingAgency,canbefound
atourwebsite:www.elsevier.com/permissions.
ThisbookandtheindividualcontributionscontainedinitareprotectedundercopyrightbythePublisher(otherthanasmay
benotedherein).
Notices
Knowledgeandbestpracticeinthisfieldareconstantlychanging.Asnewresearchandexperiencebroadenour
understanding,changesinresearchmethodsorprofessionalpractices,maybecomenecessary.Practitionersandresearchers
mustalwaysrelyontheirownexperienceandknowledgeinevaluatingandusinganyinformationormethodsdescribed
herein.Inusingsuchinformationormethodstheyshouldbemindfuloftheirownsafetyandthesafetyofothers,including
partiesforwhomtheyhaveaprofessionalresponsibility.
Tothefullestextentofthelaw,neitherthePublishernortheauthors,contributors,oreditors,assumeanyliabilityforany
injuryand/ordamagetopersonsorpropertyasamatterofproductsliability,negligenceorotherwise,orfromanyuseor
operationofanymethods,products,instructions,orideascontainedinthematerialherein.
LibraryofCongressCataloging-in-PublicationData
Sayood,Khalid.
Introductiontodatacompression/KhalidSayood.–4thed.
p.cm.
ISBN978-0-12-415796-5
1.Datacompression(Telecommunication)2. Codingtheory. I.Title.
TK5102.92.S392012
005.74’6–dc23
2012023803
BritishLibraryCataloguing-in-PublicationData
AcataloguerecordforthisbookisavailablefromtheBritishLibrary.
ISBN:978-0-12-415796-5
PrintedintheUnitedStatesofAmerica
12 13 14 15 10 9 8 7 6 5 4 3 2 1
To Fu¨sun
Thispageintentionallyleftblank
Contents
Preface xvii
1 Introduction 1
1.1 Compression Techniques 3
1.1.1 Lossless Compression 4
1.1.2 Lossy Compression 5
1.1.3 Measures of Performance 5
1.2 Modeling and Coding 6
1.3 Summary 10
1.4 Projects and Problems 10
2 Mathematical Preliminaries for Lossless Compression 13
2.1 Overview 13
2.2 A Brief Introduction to Information Theory 13
2.2.1 Derivation of Average Information w 20
2.3 Models 25
2.3.1 Physical Models 25
2.3.2 Probability Models 25
2.3.3 Markov Models 26
2.3.4 Composite Source Model 29
2.4 Coding 29
2.4.1 Uniquely Decodable Codes 30
2.4.2 Prefix Codes 33
2.4.3 The Kraft-McMillan Inequality w 34
2.5 Algorithmic Information Theory 37
2.6 Minimum Description Length Principle 38
2.7 Summary 39
2.8 Projects and Problems 40
3 Huffman Coding 43
3.1 Overview 43
3.2 The Huffman Coding Algorithm 43
3.2.1 Minimum Variance Huffman Codes 47
3.2.2 Canonical Huffman Codes 50
3.2.3 Length-Limited Huffman Codes 52
3.2.4 Optimality of Huffman Codes w 55
viii C O N T E N T S
3.2.5 Length of Huffman Codes w 56
3.2.6 Extended Huffman Codes w 58
3.2.7 Implementation of Huffman Codes 61
3.3 Nonbinary Huffman Codes w 65
3.4 Adaptive Huffman Coding 67
3.4.1 Update Procedure 68
3.4.2 Encoding Procedure 71
3.4.3 Decoding Procedure 73
3.5 Golomb Codes 75
3.6 Rice Codes 76
3.6.1 CCSDS Recommendation for Lossless Compression 77
3.7 Tunstall Codes 79
3.8 Applications of Huffman Coding 81
3.8.1 Lossless Image Compression 81
3.8.2 Text Compression 83
3.8.3 Audio Compression 85
3.9 Summary 86
3.10 Projects and Problems 87
4 Arithmetic Coding 91
4.1 Overview 91
4.2 Introduction 91
4.3 Coding a Sequence 93
4.3.1 Generating a Tag 94
4.3.2 Deciphering the Tag 101
4.4 Generating a Binary Code 102
4.4.1 Uniqueness and Efficiency of the Arithmetic Code 103
4.4.2 Algorithm Implementation 106
4.4.3 Integer Implementation 111
4.5 Adaptive Arithmetic Coding 119
4.6 Binary Arithmetic Coding 120
4.6.1 The QM Coder 125
4.6.2 The MQ Coder 125
4.6.3 The M Coder 126
4.7 Comparison of Huffman and Arithmetic Coding 127
4.8 Applications 130
4.9 Summary 131
4.10 Projects and Problems 131
5 Dictionary Techniques 135
5.1 Overview 135
5.2 Introduction 135
5.3 Static Dictionary 136
5.3.1 Digram Coding 137
5.4 Adaptive Dictionary 139
C O N T E N T S ix
5.4.1 The LZ77 Approach 139
5.4.2 The LZ78 Approach 143
5.5 Applications 150
5.5.1 File Compression–UNIX compress 151
5.5.2 Image Compression–The Graphics Interchange Format
(GIF) 151
5.5.3 Image Compression–Portable Network Graphics (PNG) 152
5.5.4 Compression over Modems–V.42 bis 153
5.6 Beyond Compression—Lempel-Ziv Complexity w 156
5.7 Summary 158
5.8 Projects and Problems 159
6 Context-Based Compression 163
6.1 Overview 163
6.2 Introduction 163
6.3 Prediction with Partial Match (ppm) 165
6.3.1 The Basic Algorithm 165
6.3.2 The Escape Symbol 170
6.3.3 Length of Context 172
6.3.4 The Exclusion Principle 173
6.4 The Burrows-Wheeler Transform 174
6.4.1 Move-to-Front Coding 177
6.5 Associative Coder of Buyanovsky (ACB) 178
6.6 Dynamic Markov Compression 179
6.7 Summary 182
6.8 Projects and Problems 182
7 Lossless Image Compression 183
7.1 Overview 183
7.2 Introduction 183
7.2.1 The Old JPEG Standard 184
7.3 CALIC 186
7.4 JPEG-LS 190
7.5 Prediction Using Conditional Averages 192
7.6 Multiresolution Approaches 193
7.6.1 Progressive Image Transmission 193
7.7 Facsimile Encoding 198
7.7.1 Run-Length Coding 199
7.7.2 CCITT Group 3 and 4–Recommendations T.4 and T.6 200
7.7.3 JBIG 203
7.7.4 JBIG2-T.88 209
7.8 MRC-T.44 211
7.9 Summary 213
7.10 Projects and Problems 214