Information Theoretic Analysis of Watermarking Systems

193 Pages·2001·1.134 MB·English

by  Cohen

Information Theoretic Analysis of Watermarking Systems
by
Aaron Seth Cohen
S.B., Electrical Engineering and Computer Science
Massachusetts Institute of Technology (1997)
M.Eng., Electrical Engineering and Computer Science
Massachusetts Institute of Technology (1997)
Submitted to the Department of Electrical Engineering and Computer Science
in partial ful(cid:12)llment of the requirements for the degree of
Doctor of Philosophy
at the
MASSACHUSETTS INSTITUTE OF TECHNOLOGY
September 2001
(cid:13)c Massachusetts Institute of Technology 2001. All rights reserved.
Author............................................................................
Department of Electrical Engineering and Computer Science
August 31, 2001
Certi(cid:12)ed by........................................................................
Amos Lapidoth
Associate Professor of Electrical Engineering
Thesis Supervisor
Accepted by.......................................................................
Arthur C. Smith
Chairman, Department Committee on Graduate Students

Information Theoretic Analysis of Watermarking Systems
by
Aaron Seth Cohen
Submitted to the Department of Electrical Engineering and Computer Science
on August 31, 2001, in partial ful(cid:12)llment of the
requirements for the degree of
Doctor of Philosophy
Abstract
Watermarking modelsa copyright protection mechanismwhere anoriginaldata sequence is
modi(cid:12)ed before distribution to the public in order to embed some extra information. The
embedding should be transparent (i.e., the modi(cid:12)ed data should be similar to the original
data) and robust (i.e., the information should be recoverable even if the data is modi(cid:12)ed
further). Inthisthesis, we describethe information-theoreticcapacity ofsuch a system as a
functionofthestatisticsofthedatatobewatermarkedandthedesiredleveloftransparency
and robustness. That is, we view watermarking from a communication perspective and
describe the maximum bit-rate that can be reliably transmitted from encoder to decoder.
We make the conservative assumptionthat there isa maliciousattacker who knows how
the watermarking system works and who attempts to design a forgery that is similar to
the original data but that does not contain the watermark. Conversely, the watermarking
system must meet its performance criteria for any feasible attacker and would like to force
theattackertoe(cid:11)ectivelydestroythedatainordertoremovethewatermark. Watermarking
canthusbeviewedasadynamicgamebetweenthesetwoplayerswhoaretryingtominimize
and maximize, respectively, the amount of information that can be reliably embedded.
We compute the capacity for several scenarios, focusing largely on Gaussian data and
a squared di(cid:11)erence similarity measure. In contrast to many suggested watermarking tech-
niques that view the original data as interference, we (cid:12)nd that the capacity increases with
the uncertainty in the original data. Indeed, we (cid:12)nd that out of all distributions with the
same variance, a Gaussian distribution on the original data results in the highest capacity.
Furthermore, for Gaussian data, the capacity increases with its variance.
One surprising result is that with Gaussian data the capacity does not increase if the
original data can be used to decode the watermark. This is reminiscent of a similar model,
Costa’s \writing on dirty paper", in which the attacker simply adds independent Gaussian
noise. Unlikewithamoresophisticatedattacker, weshowthatthecapacitydoesnotchange
for Costa’s model if the original data is not Gaussian.
Thesis Supervisor: Amos Lapidoth
Title: Associate Professor of Electrical Engineering

Acknowledgments
First, I would like to thank my advisor, Prof. Amos Lapidoth, for all of his advice and
encouragement. Even from long distance, he has graciously helped me overcome the many
hurdles I have encountered throughout the writing of this thesis.
I would also like to thank the members of my thesis committee, Prof. Bob Gallager and
Prof. Greg Wornell, for their insightful comments. The many comments by Neri Merhav
and Anelia Somekh-Baruch have also been extremely helpful.
My briefvisit to the ISI in Zu(cid:127)rich was quite pleasant, thanks mainly to the friendlyand
accommodating students there. I would speci(cid:12)cally like to thank Renate Agotai for taking
care of all of the important details and Ibrahim Abou Faycal for being a great traveling
companion.
Theintellectualenvironmentcreatedbythefaculty, sta(cid:11)andstudentsofLIDShasmade
it a wonderful place to pursue a graduate education. I would especially like to thank my
many oÆce-mates in35-303 includingRandy Berry, AnandGanti, HishamKassab, Thierry
Klein, Emre Koksal, Mike Neely, EdmundYeh and Won Yoon; the oÆce was always a good
place for getting feedback about research or (more typically) discussing such important
topics as sports or the stock market.
Of course, I could not have done any of this without my parents, Michael and Jackie,
who have always encouraged me to vigorously pursue my goals. I would like to thank them
and the rest of my family for the support they have given me over the years.
My(cid:12)nalandmost importantacknowledgment goes tomylovely wifeDinaMayzlin. She
has motivated and inspired me, and she has made the many years we have spent together
at MIT worthwhile.
This research was supported in part by an NSF Graduate Fellowship.
6
Contents
1 Introduction 13
1.1 Outline of Thesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
1.2 Notation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
2 Watermarking Model and Results 19
2.1 Precise De(cid:12)nition of Watermarking . . . . . . . . . . . . . . . . . . . . . . . 19
2.2 Capacity Results for Watermarking . . . . . . . . . . . . . . . . . . . . . . . 22
2.2.1 Scalar Gaussian Watermarking Game . . . . . . . . . . . . . . . . . 23
2.2.2 Additive Attack Watermarking Game . . . . . . . . . . . . . . . . . 28
2.2.3 Average Distortion Constraints . . . . . . . . . . . . . . . . . . . . . 29
2.2.4 Vector Gaussian Watermarking Game . . . . . . . . . . . . . . . . . 30
2.2.5 Discrete Alphabets, No Covertext . . . . . . . . . . . . . . . . . . . 32
2.2.6 Binary Watermarking Game . . . . . . . . . . . . . . . . . . . . . . 34
2.3 Prior Work on Watermarking . . . . . . . . . . . . . . . . . . . . . . . . . . 37
2.3.1 Practical Approaches to Watermarking . . . . . . . . . . . . . . . . 37
2.3.2 Information-Theoretic Watermarking . . . . . . . . . . . . . . . . . . 38
2.3.3 Similar Models: Steganography and Fingerprinting . . . . . . . . . . 39
2.3.4 Communication Games . . . . . . . . . . . . . . . . . . . . . . . . . 40
2.4 Assumptions in Watermarking Model. . . . . . . . . . . . . . . . . . . . . . 41
2.4.1 Is Capacity Meaningful? . . . . . . . . . . . . . . . . . . . . . . . . . 41
2.4.2 Randomization for Encoder/Decoder . . . . . . . . . . . . . . . . . . 41
2.4.3 Randomization for Attacker - Deterministic is SuÆcient . . . . . . . 42
2.4.4 Distortion Constraints . . . . . . . . . . . . . . . . . . . . . . . . . . 43
2.4.5 Statistics of Covertext . . . . . . . . . . . . . . . . . . . . . . . . . . 44
7
2.5 Uncertainty in the Watermarking Model . . . . . . . . . . . . . . . . . . . . 45
2.5.1 Types of State Generators . . . . . . . . . . . . . . . . . . . . . . . . 45
2.5.2 Communication with Side Information . . . . . . . . . . . . . . . . . 47
2.5.3 Arbitrarily Varying Channels . . . . . . . . . . . . . . . . . . . . . . 50
2.5.4 Extended Writing on Dirty Paper. . . . . . . . . . . . . . . . . . . . 52
3 Mutual Information Games 57
3.1 De(cid:12)nition and Main Result . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
3.2 Proof of Mutual Information Game Result . . . . . . . . . . . . . . . . . . . 60
3.2.1 Optimal Attack Channel . . . . . . . . . . . . . . . . . . . . . . . . . 60
3.2.2 Optimal Watermarking Channel . . . . . . . . . . . . . . . . . . . . 61
3.2.3 Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
3.3 Game Theoretic Interpretation . . . . . . . . . . . . . . . . . . . . . . . . . 66
3.4 Other Mutual Information Games . . . . . . . . . . . . . . . . . . . . . . . 68
4 The Scalar Gaussian Watermarking Game 71
4.1 Deterministic Attacks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
4.1.1 Deterministic Additive Attack . . . . . . . . . . . . . . . . . . . . . 72
4.1.2 Deterministic General Attack . . . . . . . . . . . . . . . . . . . . . . 73
4.2 Achievability for Private Version . . . . . . . . . . . . . . . . . . . . . . . . 73
4.2.1 Coding Strategy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
4.2.2 Analysis of Probability of Error . . . . . . . . . . . . . . . . . . . . . 76
4.3 Achievability for Public Version . . . . . . . . . . . . . . . . . . . . . . . . . 80
4.3.1 Coding Strategy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80
4.3.2 Probability of Error . . . . . . . . . . . . . . . . . . . . . . . . . . . 84
4.3.3 Distribution of Chosen Auxiliary Codeword . . . . . . . . . . . . . . 87
4.3.4 Analysis for Additive Attack Watermarking Game . . . . . . . . . . 88
4.3.5 Analysis for General Watermarking Game . . . . . . . . . . . . . . . 90
4.4 Spherically Uniform Covertext is SuÆcient. . . . . . . . . . . . . . . . . . . 92
4.5 Converse for Squared Error Distortion . . . . . . . . . . . . . . . . . . . . . 94
4.5.1 Attacker . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95
4.5.2 Analysis of Distortion . . . . . . . . . . . . . . . . . . . . . . . . . . 98
4.5.3 Analysis of Probability of Error . . . . . . . . . . . . . . . . . . . . . 100
8
4.5.4 Discussion: The Ergodicity Assumption . . . . . . . . . . . . . . . . 104
5 The Vector Gaussian Watermarking Game 105
5.1 Diagonal Covariance is SuÆcient . . . . . . . . . . . . . . . . . . . . . . . . 105
5.2 De(cid:12)nitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106
5.3 Outline of Proof . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109
5.4 Achievability for the Private Version . . . . . . . . . . . . . . . . . . . . . . 111
5.5 Achievability for the Public Version. . . . . . . . . . . . . . . . . . . . . . . 117
5.5.1 Codebook Generation . . . . . . . . . . . . . . . . . . . . . . . . . . 117
5.5.2 Encoding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118
5.5.3 Decoding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119
5.5.4 Probability of Error . . . . . . . . . . . . . . . . . . . . . . . . . . . 120
5.6 Optimization Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123
5.6.1 Proof of Lemma 5.3 . . . . . . . . . . . . . . . . . . . . . . . . . . . 123
5.6.2 Proof of (5.17) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125
5.7 The Optimal Attack and Lossy Compression . . . . . . . . . . . . . . . . . 128
5.7.1 Compression Attack . . . . . . . . . . . . . . . . . . . . . . . . . . . 128
5.7.2 Designing for a Compression Attack . . . . . . . . . . . . . . . . . . 130
6 Watermarking with Discrete Alphabets 133
6.1 No Covertext . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134
6.1.1 De(cid:12)nitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134
6.1.2 Achievability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135
6.1.3 Converse . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137
6.2 Binary Covertext . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138
6.2.1 Private Version . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139
6.2.2 Public Version . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140
7 Conclusions 145
7.1 Future Research . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147
7.1.1 Gaussian Sources with Memory . . . . . . . . . . . . . . . . . . . . . 147
7.1.2 Discrete Memoryless Covertext . . . . . . . . . . . . . . . . . . . . . 148
7.1.3 Deterministic Code Capacity for Public Version . . . . . . . . . . . . 148
9
7.1.4 Multiple Rate Requirements . . . . . . . . . . . . . . . . . . . . . . . 150
A De(cid:12)nitions for Gaussian covertext 151
B Technical Proofs 155
B.1 Proof of Theorem 2.3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155
B.2 Proof of Lemma 2.1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156
B.3 Proof of Lemma 3.1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159
B.4 Proof of Lemma 3.2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 160
B.5 Proof of Lemma 3.3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162
B.6 Proof of Lemma 4.3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 167
B.7 Proof of Lemma 4.6 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169
B.8 Proof of Lemma 4.8 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 170
B.9 Proof of Lemma 4.10 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172
B.10 Proof of Lemma 4.12 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 174
B.11 Proof of Lemma 4.13 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177
B.12 Proof of Lemma 5.6 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 180
B.13 Proof of Lemma 5.10 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182
Bibliography 185
10