Probability, Statistical Optics, and Data Testing: A Problem Solving Approach

458 Pages·1991·8.185 MB·English

by  Professor B. Roy Frieden Ph. D. (auth.)

Springer Series in Information Sciences 10
Editor: T. S. Huang
Springer Series in Information Sciences
Editors: Thomas S. Huang Teuvo Kohonen Manfred R. Schroeder
Managing Editor: H.K.V. Lotsch
1 Content-Addressable Memories 14 Mathematics of Kalman-Bucy Filtering
By T. Kohonen 2nd Edition By P.A. Ruymgaart and T.T. Soong
2nd Edition
2 Fast Fourier Transform and
Convolution Algorithms 15 Fundamentals of Electronic Imaging
By H. 1. Nussbaumer 2nd Edition Systems Some Aspects of Image
Processing By W.F. Schreiber
3 Pitch Determination of Speecb Signals
2nd Edition
Algorithms and Devices By W. Hess
16 Radon and Projection Transform
4 Pattern Analysis and Understanding Based Computer Vision
By H. Niemann 2nd Edition Algorithms, A Pipeline Architecture, and
Industrial Applications By 1.L.C. Sanz,
5 Image Sequence Analysis
E.B. Hinkle, and A.K. lain
Editor: T.S. Huang
17 Kalman Filtering with Real-Time
6 Picture Engineering
Applications By C.K. Chui and G. Chen
Editors: King-sun Fu and T.L. Kunii
18 Linear Systems and Optimal Control
7 Number Theory in Science and
By C.K. Chui and G. Chen
Communication With Applications in
Cryptography, Physics, Digital 19 Harmony: A Psychoacoustical
Information, Computing, and Self Approach By R. Parncutt
Similarity By M.R. Schroeder
20 Group Theoretical Methods
2nd Edition
in Image Understanding
8 Self-Organization and Associative By Ken-ichi Kanatani
Memory By T. Kohonen 21 Linear Prediction Theory
3rdEdition A Mathematical Basis for Adaptive
Systems By P. Strobach
9 Digital Picture Processing
An Introduction By L.P. Yaroslavsky 22 Psychoacoustics Facts and Models
By E. Zwicker and H. Fast!
10 Probability, Statistical Optics, and
Data Testing A Problem Solving 23 Digital Image Restoration
Approach By B.R. Frieden Editor: A. K. Katsaggelos
2nd Edition
24 Parallel Algorithms
11 Physical and Biological Processing of in Computational Science
Images Editors: 0.1. Braddick and By D. W. Heermann and A. N. Burkitt
A.c. Sleigh
25 Radar Array Processing
12 Multiresolution Image Processing and Editors: S. Haykin, 1. Litva, T. 1. Shepherd
Analysis Editor: A. Rosenfeld
26 Signal Processing and Systems Theory
13 VLSI for Pattern Recognition and Selected Topics
Image Processing Editor: King-sun Fu By C. K. Chui and G. Chen
B. Roy Frieden
Probability,
Statistical Optics, and
Data Testing
A Problem Solving Approach
Second Edition
With 110 Figures
Springer-Verlag
Berlin Heidelberg New York
London Paris Tokyo
Hong Kong Barcelona
Professor B. Roy Frieden, Ph. D.
Optical Sciences Center, The University of Arizona
Tucson, AZ 85721, USA
Series Editors:
Professor Thomas S. Huang
Department of Electrical Engineering and Coordinated Science Laboratory,
University of Illinois, Urbana, IL 61801, USA
Professor Teuvo Kohonen
Laboratory of Computer and Information Sciences,
Helsinki University of Technology,
SF-02150 Espoo 15, Finland
Professor Dr. Manfred R. Schroeder
Drittes Physikalisches Institut, Universitiit Gottingen, Biirgerstrasse 42-44,
W-3400 Gottingen, Fed. Rep. of Germany
Managing Editor: Helmut K. V. Lotsch
Springer-Verlag, Tiergartenstrasse 17,
W-6900 Heidelberg, Fed. Rep. of Germany
ISBN-13: 978-3-540-53310-8 e-ISBN-13: 978-3-642-97289-8
DOl: 10.1007/978-3-642-97289-8
This work is subject to copyright. All rights are reserved, whether the whole or part of the material is
concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting,
reproduction on microfilms or in other ways, and storage in data banks. Duplication of this publication
or parts thereof is only permitted under the provisions of the German Copyright Law of September 9,
1965, in its current version and a copyright fee must always be paid. Violations fall under the prosecution
act of the German Copyright Law.
© Springer-Verlag Berlin Heidelberg 1983, 1991
The use of registered names, trademarks, etc. in this publication does not imply, even in the absence of
a specific statement, that such names are exempt from the relevant protective laws and regulations and
therefore free for general use
54/3140 - 54321 0 - Printed on acid-free paper
To Sarah and Miriam
Preface
This new edition incorporates corrections of all known typographical errors in
the first edition, as well as some more substantive changes. Chief among the
latter is the addition of Chap. 17, on methods of estimation. As with the rest
of the text, most applications and examples cited in the new chapter are from
the optical perspective. The intention behind this new chapter is to empower
the optical researcher with a yet broader range of research tools. Certainly a
basic knowledge of estimation methods should be among these. In particular,
the sections on likelihood theory and Fisher information prepare readers for
the problems of optical parameter estimation and probability law estimation.
Physicists and optical scientists might find this material particularly useful,
since the subject of Fisher information is generally not covered in standard
physical science curricula.
Since the words "statistical optics" are prominent in the title of this book,
their meaning needs to be clarified. There is a general tendency to overly
emphasize the statistics of photons as the sine qua non of statistical optics. In
this text a wider view is taken, which equally emphasizes the random medium
that surrounds the photon, be it a photographic emulsion, the turbulent atmo
sphere, a vibrating lens holder, etc. Also included are random interpretations
of ostensibly deterministic phenomena, such as the Hurter-Driffield (H and D)
curve of photography. Such a "random interpretation" sometimes breaks new
ground, as in Chap. 5, where it is shown how to produce very accurate ray
trace-based spot diagrams, using the statistical theory of Jacobian transforma
tion.
This edition, like the first, is intended to beftrst andforemost an introduc
tory text on methods of probability and statistics. Emphasis is on the linear
(and sometimes explicitly Fourier) theory of probability calculation, chi
square and other statistical tests of data, stochastic processes, the information
theories of both Shannon and Fisher, and estimation theory. Applications to
statistical optics are given, in the main, so as to give a reader with some
background in either optics or linear communications theory a running start.
As a pedagogical aid to understanding the mathematical tools that are devel
oped, the simplest possible statistical model is used that fits a given optical
phenomenon. Hence, a semiclassical model of radiation is used in place of the
full-blown quantum optical theory, a poker-chip model is used to describe film
granularity, etc. However, references are given to more advanced models as
well, so as to steer the interested reader in the right direction. The listing
"Statistical models ..." in the index gives a useful overview of the variety of
models used. The reader might, for example, be amazed at how much
"mileage" can be obtained from a model as simple as the checkerboard model
of granularity (Chaps. 6 and 9).
VIII Preface
The reader who needs an in-depth phenomenological viewpoint of a
specific topic, with statistical optics as the emphasis and probability theory
subsidiary, can consult such fine books as Photoelectron Statistics by B. Saleh
[Springer Ser. Opt. Sci., Vol. 6 (Springer, Berlin, Heidelberg 1978)] and Statis
tical Optics by J. W. Goodman (Wiley, New York 1985). We recommend these
as useful supplements to this text, as well.
The deterministic optical theory that is used in application of the statistical
methods presented is, for the most part, separately developed within the text,
either within the main body or in the exercises (e.g., Exercise 4.3.13). Some
times the development takes the form of a sequence of exercises (e.g., Exercises
6.1.21-23). The simplest starting point is used - recalling our aims - such as
Huygens' wave theory as the basis for diffraction theory. In this way, readers
who have not previously been exposed to optical theory are introduced to it in
the text.
The goal of this book remains as before: To teach the powerful problem
solving methods of probability and statistics to students who have some back
ground in either optics or linear systems theory. We hope to have furthered this
purpose with this new edition.
Tucson
December 1990 B. Roy Frieden
Preface to the First Edition
A basic skill in probability is practically demanded nowadays in many bran
ches of optics, especially in image science. On the other hand, there is no
text presently available that develops probability, and its companion fields
stochastic processes and statistics, from the optical perspective. [Short of a
book, a chapter was recently written for this purpose; see B. R. Frieden (ed.):
The Computer in Optical Research, Topics in Applied Physics, Vol. 41
(Springer, Berlin, Heidelberg, New York 1980) Chap. 3]
Most standard texts either use illustrative examples and problems from
electrical engineering or from the life sciences. The present book is meant
to remedy this situation, by teaching probability with the specific needs of
the optical researcher in mind. Virtually all the illustrative examples and
applications of the theory are from image science and other fields of optics.
One might say that photons have replaced electrons in nearly all considera
tions here. We hope, in this manner, to make the learning of probability a
pleasant and absorbing experience for optical workers.
Some of the remaining applications are from information theory, a con
cept which complements image science in particular. As will be seen, there
are numerous tie-ins between the two concepts.
Students will be adequately prepared for the material in this book if they
have had a course in calculus, and know the basics of matrix manipulation.
Prior formal education in optics is not strictly needed, although it obvi
ously would help in understanding and appreciating some of the applications
that are developed. For the most part, the optical phenomena that are treated
are developed in the main body of text, or in exercises. A student who has had
a prior course in linear theory, e.g. out of Gaskill's book [1. D. Gaskill: Linear
Systems, Fourier Transforms, and Optics (Wiley, New York 1978)], or in
Fourier optics as in Goodman's text [J. W. Goodman: Introduction to Fou
rier Optics (McGraw-Hill, New York 1968)], is very well prepared for this
book, as we have found by teaching from it. In the main, however, the ques
tion is one of motivation. The reader who is interested in optics, and in prob
lems pertaining to it, will enjoy the material in this book and therefore will
learn from it.
We would like to thank the following colleagues for helping us to under
stand some of the more mystifying aspects of probability and statistics, and
for suggesting some of the problems: B. E. A. Saleh, S. K. Park, H. H. Bar
rett, and B. H. Soffer. The excellent artwork was by Don Cowen. Kathy Seeley
and her editorial staff typed up the manuscript.
Tucson
April 1982 Roy Frieden
Contents
1. Introduction .
1.1 What Is Chance, and Why Study It? 3
1.1.1 Chance vs Determinism . 3
1.1.2 Probability Problems in Optics 5
1.1.3 Statistical Problems in Optics 5
2. The Axiomatic Approach . 7
2.1 Notion of an Experiment; Events 7
2.l.l Event Space; The Space Event 8
2.1.2 Disjoint Events. 9
2.1.3 The Certain Event 9
Exercise 2.1 9
2.2 Definition of Probability 9
2.3 Relation to Frequency of Occurrence 10
2.4 Some Elementary Consequences 10
2.4.1 Additivity Property. 11
2.4.2 Normalization Property 11
2.5 Marginal Probability. 12
2.6 The "Traditional" Definition of Probability 12
2.7 Illustrative Problem: A Dice Game 13
2.8 Illustrative Problem: Let's (Try to) Take a Trip 14
2.9 Law of Large Numbers 15
2.10 Optical Objects and Images as Probability Laws 16
2.11 Conditional Probability 17
Exercise 2.2 18
2.12 The Quantity of Information 19
2.13 Statistical Independence 21
2.13.1 Illustrative Problem: Let's (Try to) Take a Trip
(Continued) 22
2.14 Informationless Messages . 23
2.15 A Definition of Noise 23
2.16 "Additivity" Property of Information 24
2.17 Partition Law . 25
2.18 Illustrative Problem: Transmittance Through a Film 25
2.19 How to Correct a Success Rate for Guesses 26
Exercise 2.3 27
2.20 Bayes'Rule 27
2.21 Some Optical Applications 28
XII Contents
2.22 Information Theory Application 30
2.23 Application to Markov Events 30
2.24 Complex Number Events 32
Exercise 2.4 . . . . 32
3. Continuous Random Variables. 37
3.1 Definition of a Random Variable . . . . . . 37
3.2 Probability Density Function, Basic Properties 37
3.3 Information Theory Application: Continuous Limit 39
3.4 Optical Application: Continuous Form ofImaging Law 40
3.5 Expected Values, Moments . . . . . . . . . . . . 40
3.6 Optical Application: Moments of the Slit Diffraction Pattern 41
3.7 Information Theory Application 43
3.8 Case of Statistical Independence 44
3.9 Mean of a Sum . . . . . . . 44
3.10 Optical Application . . . . . 45
3.11 Deterministic Limit; Representations of the Dirac 0-Function 46
3.12 Correspondence Between Discrete and Continuous Cases 47
3.13 Cumulative Probability .... . . . . . . . . . . . . 48
3.14 The Means of an Algebraic Expression: A Simplified Approach 48
3.15 A Potpourri of Probability Laws 50
3.15.1 Poisson. 50
3.15.2 Binomial . 51
3.15.3 Uniform .. 51
3.15.4 Exponential 52
3.15.5 Normal (One-Dimensional) 53
3.15.6 Normal (Two-Dimensional) 53
3.15.7 Normal (Multi-Dimensional) . 55
3.15.8 Skewed Gaussian Case; Gram-Charlier Expansion 56
3.15.9 Optical Application. 57
3.15.10 Geometric Law 58
3.15.11 Cauchy Law. . . . . 59
3.15.12 Sinc2 Law . 59
Exercise 3.1 60
4. Fourier Methods in Probability 70
4.1 Characteristic Function Defined 70
4.2 Use in Generating Moments 71
4.3 An Alternative to Describing RV x 71
4.4 On Optical Applications 71
4.5 Shift Theorem 72
4.6 Poisson Case . 72
4.7 Binomial Case 73