Minimax Theory of Image Reconstruction

271 Pages·1993·6.06 MB·English

by  A. P. Korostelev, A. B. Tsybakov (auth.)

Editorial Policy
for the publication of monographs
In what follows all references to monographs are applicable also to multiauthorship
volumes such as seminar notes.
§ 1. Lecture Notes aim to report new developments - quickly, informally, and al a
high level. Monograph manuscripts should be reasonably self-contained and rounded
off. Thus they may, and often will, present not only results of the author but also
related work by other people. Furthermore, the manusc ripts should provide sufficient
motivation, examples, and applications. This clearly distinguishes Lecture Noles
manuscripts fromjoumal articles which normally are very concise. Art ic les intended
for a journal but too long to be accepted by most journals usually do not have this
" lecture notes" character. For similar reasons it is unusual for Ph.D. theses to be
accepted for the Lecture Notes series.
§ 2. Manuscriptsor plans for Lecture Notes volumes should be submitted (preferably
in duplicate) either to one of the series editors or to Springer- Verlag, New York. These
proposals are then refereed. A final decision concerning publication can only be made
on the basis of the complete manuscript, but a preliminary decision can often be
based on partial information: a fairly detailed outline describing the planned contents
of each chapter, and an indication of the estimated length, a bibliog raphy, and one
or two sample chapters - or a first draft of the manuscript. The editors will try to
make the preliminary decision as definite as they can on the basis of the available
information.
§ 3. Final manuscripts should be in Eng lish. They should contain at least 100 pages
of scientific text and should include
- a table of contents;
- an informative introduction, perhaps with some historical remarks: it should be
accessible to a reader not particularly familiar with the topic treated;
- a subject index: as a rule this is genuinely helpful for the reader.
Lecture Notes in Statistics 82
Edited by 1. Berger, S. Fienberg, 1. Gani, K. Krickeberg,
I. OIkin, and B. Singer
A. P. Korostelev
A. B. Tsybakov
Minimax Theory of
Image Reconstruction
Springer-Verlag
New York Berlin Heidelberg London Paris
Tokyo Hong Kong Barcelona Budapest
A. P. Korostelev A. B. Tsybakov
Institute for Systems Studies Institute for Problems of Information Transmission
Prospect 6O-letija Octjabuja 9 Ermolovoy Street 19
117312 Moscow 101447 MoscowGSP-4
Russia Russia
Mathematics Subject Classification: 68UIO, 62005
Litn.ry ofCansIUI CatalopS·in-Publication Dau.
Korom1ev. A. P. (Aldr.Wldr Petrovich)
Minimax theory cl image reOOll$uuctioD I A. Korostdev, A.
Tsybakov)
p. an. -- (Lcc:w~ n«ea in datiltiCI ; 82)
Includes bibliographical relermce.s and indexes.
ISBN-\3: 978.Q-387-94028-1
I. Imaae procenina-Digiul techniqUC5. 2. Image proa:llina-
-Statistical mclhoch. 3. Irnaae tualJtruction. 4. o..ebytbev
approximation.. L Tsybakov, A. (A. B.) n. TIde. m. Series:
I...cau~ notel in rwistiCI (Sprinacr-Vedag) ; v. 82.
TAI637.K67 1993
621.361-«20 93-18028
Printed on acid-free paper.
e 1993 Springer-Verlag New York, Inc.
Reprint ofthc original edition 1993
All ri&hU racrvcd. Thi. WQfk may nOl. be trandaJcd or copied in wbok orin pmt wizhcxa the wriltm
pamissioD. do the puNiahcr (SprinleJ-Verlq New York, Inc. , 175 Pi!tlI Aw:nlle, New York, NY
10010, USA), ex~ £01" brid excelJU in aJmCC:tian with ~cws or sdIolarty anaI)'Iit. Ute in
cmncaion with any form of information lIOnCe md rellZvaI. tlecbonic .a..pwion. computer
aoftwue, 01" by .imilar 01" dil limilar tmIbodoIOI)' now blown or hcrWter developed is fCJlbiddcn
The IIJC of ,mera1 dctcripIivc MmCI, trade _, ndmwtl, eIC., in thil publiadian, even if !he
formcr~not upcciaIJyidentifi.cd. i. DOC 10 be Iaten .. a riF dw ItICh _, at WKknlood by die
Trade Mariti .uI MeJdwldite Matt. Aa. may ccordiJI&ly be UIed &eely by ..y gne.
Camenl ready copy pnwided by the aulhon.
981654321
ISBN-\3: 978.Q-387-94028- 1 c-ISBN-1 3: 978-1-4612-2712-0
001: IO.IOO7f978-14612-2712-O
PREFACE
There exists a large variety of image reconstruction methods
proposed by different authors (see e.g. Pratt (1978), Rosenfeld
and Kak (1982), Marr (1982)). Selection of an appropriate method
for a specific problem in image analysis has been always
considered as an art. How to find the image reconstruction method
which is optimal in some sense? In this book we give an answer
to this question using the asymptotic minimax approach in the
spirit of Ibragimov and Khasminskii (1980a,b, 1981, 1982),
Bretagnolle and Huber (1979), Stone (1980, 1982). We assume that
the image belongs to a certain functional class and we find the
image estimators that achieve the best order of accuracy for the
worst images in the class. This concept of optimality is rather
rough since only the order of accuracy is optimized. However, it
is useful for comparing various image reconstruction methods. For
example, we show that some popular methods such as simple
linewise processing and linear estimation are not optimal for
images with sharp edges. Note that discontinuity of images is an
important specific feature appearing in most practical situations
where one has to distinguish between the "image domain" and the
"background" .
The approach of this book is based on generalization of
nonparametric regression and nonparametric change-point
techniques. We discuss these two basic problems in Chapter 1.
Chapter 2 is devoted to minimax lower bounds for arbitrary
estimators in general statistical models. These are the main
tools applied in the book. In Chapters 1 and 2 the exposition is
mostly tutorial. They present a general introduction to
nonparametric estimation theory. To prove the theorems some
nonstandard methods are chosen which are, in our opinion, simple
and transparent. Chapters 3-9 contain mostly the new results, and
the reader who is familiar with nonparametric estimation
background may proceed to them directly.
The working example that we study in detail is the
two-dimensional binary image of "boundary fragment" type. Roughly
speaking, it is a small piece of discontinuous image containing
the discontinuity curve (the boundary). Imposing some smoothness
restrictions on the boundary we find the minimax rates and
Preface
vi
optimal estimators for boundary fragments. This is the main
message of Chapters 3 and 4. Various extensions are discussed in
Chapter 5. Some proofs in Chapter 5 and in the following chapters
are not detailed. Simple but technical steps of the proofs are
sometimes left to the reader.
Chapter 6 deals with the simplified image reconstruction
procedures, namely with linewise and linear processing. It is
shown that linewise processing can be organized in such a way
that it has the optimal rate of convergence in minimax sense.
Linear procedures, however, are proved to be suboptimal.
In Chapters 7-9 we discuss some further issues related to
the basic image reconstruction problem, namely: the estimation of
support of a density, the estimation of image functionals, image
estimation from indirect observations, the stochastic tomography
setup. For all these problems we derive the minimax rates of
convergence and construct the optimal estimators.
One of the points raised in the book is the choice of design
in image reconstruction. This point is often ignored since in
practice the simplest regular grid design has no competitors. We
show that the choice of design is important in image analysis:
some randomized designs allow to improve substantially the
accuracy of estimation as compared to the regular grid design.
We also consider in brief some parametric imaging problems
(Section 1.9, Section 8.2). For parametric case we refer to the
continuous-"time" models where the image is supposed to be a
solution of a stochastic differential equation. This makes the
proofs more concise. Readers who are not familiar with stochastic
differential equations may easily skip this part of the book.
Our attitude in this book is to prove the results under the
simplest assumptions which still allow to keep the main features
of a particular problem. For example, we often assume that the
random errors are independent identically distributed Gaussian.
Generalizations are mainly given without proofs.
Some words about the notation. We use the small letters c,
c ' i=1,2, ... , and letter ;\ (possibly, with indices) to denote
i
positive constants appearing in the proofs. This notation is kept
only inside a chapter, so that in different chapters c may be
i
different. The constants CO' C are reserved for the lower and
l
Preface vii
upper bounds of minimax risks respectively. They are different in
different theorems.
The work on this book was strongly influenced by the ideas
of I.A.Ibragimov and R.Z.Khasminskii and stimulated by the
discussions at the seminar of M.B.Pinsker and R.Z.Khasminskii in
the Institute for Problems of Information Transmission in Moscow.
We would like to thank E.Marnrnen and A.B.Nemirovskii for helpful
personal discussions and suggestions. We are grateful to W.Hardle
B.Park, M.Rudemo and B.Turlach who made important remarks that
helped much to improve the text.
A.P.Korostelev
A.B.Tsybakov
CONTENTS
CHAPTER 1. NONPARAMETRIC REGRESSION AND CHANGE-POINT PROBLEMS 1
1.1. Introduction 1
1.2. The nonparametric regression problem 1
1.3. Kernel estimators 3
1.4. Locally-polynomial estimators 6
1.5. Piecewise-polynomial estimators 10
1.6. Bias and variance of the estimators 13
1.7. Criteria for comparing the nonparametric
estimators 25
1.8. Rates of the uniform and L - convergence 28
1
1.9. The change-point problem 32
CHAPTER 2. MINIMAX LOWER BOUNDS 46
2.1. General statistical model and minimax rates of
convergence 46
2.2. The basic idea 51
2.3. Distances between distributions 54
2.4. Examples 59
2.5. The main theorem on lower bounds 64
2.6. Assouad's lemma 67
2.7. Examples: uniform and integral metrics 73
2.8. Arbitrary design 82
CHAPTER 3. THE PROBLEM OF EDGE AND IMAGE ESTIMATION 88
3.1. Introduction 88
3.2. Assumptions and notation 91
3.3. Lower bounds on the accuracy of estimates 98
CHAPTER 4. OPTIMAL IMAGE AND EDGE ESTIMATION FOR BOUNDARY
FRAGMENTS 107
4.1. Optimal edge estimation 107
4.2. Preliminary lemmas 110
4.3. Proof of Theorem 4.1.1 114
4.4. Optimal image estimation 118
4.5. Proof of Theorem 4.4.5 123
x Contents
CHAPTER 5. GENERALIZATIONS AND EXTENSIONS 128
5.1. High-dimensional boundary fragments. Non-Gaussian
noise 128
5.2. General domains in high dimensions: a simple
and rough estimator 137
5.3. Optimal estimators for general domains in two
dimensions 142
5.4. Dudley's classes of domains 148
5.5. Maximum likelihood estimation on c-net 153
5.6. Optimal edge estimators for Dudley's classes 155
5.7. On calculation of optimal edge estimators for
general domains 159
CHAPTER 6. IMAGE RECONSTRUCTION UNDER RESTRICTIONS ON
ESTIMATES 163
6.1. Naive linewise processing 163
6.2. Modified linewise procedure 166
6.3. Proofs 172
6.4. Linear image estimators 177
CHAPTER 7. ESTIMATION OF SUPPORT OF A DENSITY 182
7.1. Problem statement 182
7.2. A simple and rough support estimator 184
7.3. Minimax lower bounds for support estimation 185
7.4. Optimal support estimation for boundary fragments 188
7.5. Optimal support estimation for convex domains
and for Dudley's classes 195
CHAPTER 8. ESTIMATION OF THE DOMAIN'S AREA 198
8.1. Preliminary discussion 198
8.2. Domain's area estimation in continuous parametric
models 201
8.3. Theorem on the lower bound 206
8.4. Optimal estimator for the domain's area 208
8.5. Generalizations and extensions 212
8.6. Functionals of support of a density 216
Contents xi
CHAPTER 9. IMAGE ESTIMATION FROM INDIRECT OBSERVATIONS 223
9.1. The blurred image model 224
9.2. High-dimensional blurred image models 229
9.3. Upper bounds in non-regular case 232
9.4. The stochastic problem of tomography 235
9.5. Minimax rates of convergence 239
REFERENCES 243
AUTHOR INDEX 252
SUBJECT INDEX 254