Maximum Entropy and Bayesian Methods: Cambridge, England, 1988

521 Pages·1989·30.938 MB·English

by  E. T. Jaynes (auth.), J. Skilling (eds.)

Maximum Entropy and Bayesian Methods
Fundamental Theories of Physics
An International Book Series on The Fundamental Theories of Physics:
Their Clarification. Development and Application
Editor: ALWYN V AN DER MERWE
University ofD enver. U.SA.
Editorial Advisory Board:
ASIM BARUT, University of Colorado. U.S.A.
HERMANN BONDI, University of Cambridge. U.K.
BRIAN D. JOSEPHSON, University of Cambridge. U.K.
CLIVE KILMIS1ER, University ofL ondon. U.K.
GUNTER LUDWIG, Philipps-Universitiit. Marburg. F R.G.
NATHAN ROSEN, Israel Institute of Technology. Israel
MENDEL SACHS, State University ofN ew York at Buffalo. U.s.A.
ABDUS SALAM, International Centre for Theoretical Physics, Trieste. Italy
HANS-JORGEN TREDER, Zentralinstitut fur Astrophysik der Akademie der
Wissenschaften. GD.R.
Maximum Entropy
and
Bayesian Methods
Cambridge, England, 1988
edited by
1. Skilling
Department ofA pplied Mathematics and Theoretical Physics,
University of Cambridge, Cambridge, u.K.
Springer-Science+Business Media, B. V.
Library of Congress Cataloging in Publication Data
Maximum Entropy Workshop (8th: 1988: St. John's College)
Maximum entropy and Bayesian methods, Cambridge, U.K., 1988
edited by J. Skilling.
p. cm. -- (Fundamental theories of physiCS)
Inc 1u des index.
Proceedings of the 8th MaxEnt Workshop held at St. John's College,
Cambridge, England, August 1-5, 1988,
1. Entropy (Information theory)--Congresses. 2. Bayesian
statistical decision theory--Congresses, I. Skilling, J. (John)
II. Title. III. Series.
0370.M385 1988
001.53'9--dc19 89-2480
printed on acid free paper
This work relates to Department of Navy Grant NOOOl4-88-J-1126 issued by the Office of
Naval Research. The United States Government has a royalty-free license throughout the
world in all copyrightable material contained herein."
ISBN 978-90-481-4044-2 ISBN 978-94-015-7860-8 (eBook)
DOI 10.1007/978-94-015-7860-8
Softcover reprint of the hardcover 1st edition 1989
All Rights Reserved
@ 1989 by Springer Science+Business Media Dordrecht
Originally published by Kluwer Academic Publishers in 1989.
No part of the material protected by this copyright notice may be reproduced or
utilized in any form or by any means, electronic or mechanical
including photocopying, recording or by any information storage and
retrieval system, without written permission from the copyright owner.
Dedication
To the ideal of rational inference
PREFACE
The "8th MaxEnt Workshop", to give it its short name, was
held in St. John's College, Cambridge, England, on August 1 - 5,
1988, and this Volume of 55 papers records the Proceedings.
History repeats itself in many ways. All ancient
civilisations evolved some core of basic practical mathematics,
but it was the Greeks who insisted upon superior intellectual
standards. It was the Greeks who invented the world of axioms
and theorems. They invented and formalised the central idea of
logical proof, so that assent to axioms perforce requires assent
to their consequences, however long the chain of reasoning
involved. Conversely, any attack on the consequences becomes an
attack upon the axioms, which are usually much simpler to
discuss. This power and beauty swept cruder mathematics aside
for ever.
An echo of this occurs in our own century. We live in a
complicated world, and our procedures for learning from
observations are codified as the subject of statistics. Often
enough, we are concerned with difficult problems, and a variety
of more or less ad hoc practical techniques has evolved to deal
with them. Yet there is an inner logic to the practice of
inference, which leads inevitably to the use of quantified
probabilities, manipulated by Bayes' theorem and assigned by the
principle of maximum entropy (MaxEnt). Those who live by this
logic are called Bayesians. Because of their inner certainty of
methodology, they are sometimes perceived as religious
fundamentalists - but that does not in itself mean that they are
wrong.
The Bayesian/MaxEnt church is alive and well, and has its
own calendar of saints (and devils). Foreshadowed by the MIT
meeting of 1978, the first formal assembly was held ~n Laramie,
Wyoming in June 1981. It was my particular pleasure to attend
that first "Workshop on Maximum Entropy and Bayesian Methods in
Applied Statistics", organised by Ray Smith and Tom Grandy, who
have since become lasting friends of mine. MaxEnt was then
beginning to grow beyond the confines of statistical thermo
dynamics, where it had enjoyed a certain degree of protection
afforded by the abstract nature of the subject, by the
conveniently large value of Avogadro's number, and by the
demonstrable success of its predictions. Bayes' theorem,
likewise, was beginning to break free of the suffocating weight
of "orthodox" statistics, substantially aided by the brilliant
logic and polemic of Edwin Jaynes. Each summer since 1981 has
seen a further meeting in the series. Each has been notable for
some new inspiration and application, for the particular strength
of rational thought is that it works. More and more quickly,
inference problems in all sorts of disciplines are being brought
within the purview of Bayesian/MaxEnt analysis.
The 1988 meeting, held in St. John's College, Cambridge,
had a particularly appropriate venue. Sir Harold Jeffreys, who
vii
viii PREFACE
cared deeply about rational inference throughout a long working
life, is the Senior Fellow. Prof. Edwin T. Jaynes, whose
influence on the subject has been so profound, is also connected
with St. John's, having been Overseas Fellow in 1983/4. On a
lesser plane, my own introduction to MaxEnt was a lunch-time
conversation in College with my mentor, friend and colleague,
Steve Gull.
Being the first of the workshops to be held in Europe,
this meeting attracted over 100 delegates, from industry and from
defence establishments as well as from academia. A central topic
such as inference can be expected to touch a number of other
subjects, but even the organisers were surprised by the variety
of topics which were offered and presented, from philosophy to
floods, from biology to astronomy, and with references ranging
from New Left Publications to Acta Crystallographica.
Profound thanks are due to our financial sponsors, who
provided the funds needed to invite distinguished overseas
speakers whilst keeping the fees low enough for the academic
pocket. The United States Navy Office of Naval Research
maintained its valued connection with the workshop series through
its grant N00014-88-J-1126, and industrial support was provided
by E.I. DuPont Company Central Research and Development, ICI
Chemicals and Polymers Group, Glaxo Group Research Limited,
British Petroleum pic, and Maximum Entropy Data Consultants
Limited. Thanks are also due to St. John's College, which
provided such appropriate and attractive facilities, and whose
staff were unfailingly generous with their time and effort. Not
least, I wish to thank in particular my wife and son, Jennifer
and Martin Skilling, for their secretarial and organisational
help, which contributed so much to the smooth running of the
meeting. Thank you, all.
The authors of the papers published here also deserve my
editorial thanks for producing their papers so well and so
promptly. In the interests of quick publication, the workshop is
continuing the recent practice of using camera-ready copy.
Because interest continues to grow, the workshops are currently
being formally organised on a continuing basis, with a permanent
organising committee, and their Proceedings are henceforward to
be published by Kluwer Academic Publishers under the generic
title
"Maximum Entropy and Bayesian Methods (location) (year)".
It is hoped that each successive volume will continue to capture
something of the excitement and vitality of current research.
Lastly, I wish as Editor to dedicate this Volume, not to
any particular individual, but to that transcending ideal to
which we try to aspire - the ideal of rational inference.
St. John's College John Skilling
Cambridge
January 1989
CONTENTS
Preface
V11
Tutorial
E.T. Jaynes
Clearing up mysteries -The original goal
C.R. Smith, G. Erickson
From rationality and consistency to Bayesian probability 29
1. Skilling
Classic maximum entropy 45
S.F. Gull
Developments in maximum entropy data analysis 53
W.T. Grandy, Jr.
The three phases of statistical dynamics 73
A.l.M. Garrett
Bell's theorem, inference and quantum transactions 93
Philosophy
A.J.M. Garrett
Probability, philosophy and science: a briefing for Bayesians 107
Statistical thermodynamics & Quantum mechanics
R.D. Levine
The statistics of quantum mechanical wavefunctions 117
R. Balian
Justification of the maximum entropy criterion in quantum mechanics 123
CONTENTS
J.P. Dougherty
Approaches to non-equilibrium statistical mechanics 131
D.A. Drabold, A.E. Carlsson, P.A. Fedders
Applications of maximum entropy to condensed matter physics 137
R. Collins, T. Ogawa and T. Ogana
Problems of maximum-entropy formalism in the statistical geometry of
simple liquids 143
Physical measurement techniques
G.l. Daniell, 1.A. Potton
Liquid structure factor detem1ination by neutron scattering -some dangers
of maximum entropy 151
R.l. Papoular, A.K. Livesey
Quasielastic neutron scattering data evaluation using the maximum entropy
method 163
R. T. Constable, R.M. Henkelman
Maximum entropy reconstruction in magnetic resonance imaging 175
N.A. Farrow, F.P. Ottensmeyer
Solution of autocorrelation function constrained maximum entropy problems
using the method of simulated annealing 181
A.K. Livesey, 1-C. Brochon, P. Licinio
Solution of Laplace transform equations (sum of exponentials) by maximum
entropy 191
A. Mohammad-Djafari, G. Demoment
Maximum entropy and Bayesian approach in tomographic image reconstruction
and restoration 195
Crystallography
S. Steenstrup, S.W. Wilkins
Maximum-entropy-based approaches to X-ray structure determination and
data processing 203
R.K. Bryan
Maximum entropy in crystallography 213