A Mathematical Theory of Arguments for Statistical Evidence

159 Pages·2003·13.101 MB·English

by  Paul-André Monney (auth.)

A Mathematical Theory
of Arguments for Statistical Evidence
Contributions to Statistics
w.
V. FedorovlW.G. Muller/I.N. Vuchkov HardlelM.G. Schimek (Eds.)
(Eds.) Statistical Theory and Computational
Model-Oriented Data Analysis, Aspects of Smoothing,
XIU248 pages, 1992 VIIII265 pages, 1996
J. Antoch (Ed.) S. Klinke
Computational Aspects of Model Choice, Data Structures for Computational Statistics,
VIII285 pages, 1993 VIIII284 pages, 1997
W. G. MulierlH. P. Wynn/A. A. Zhigljavsky A. C. AtkinsonlL. PronzatolH. P. Wynn
(Eds.) (Eds.)
Model-Oriented Data Analysis, MODA 5 - Advances in Model-Oriented
XIIII287 pages, 1993 Data Analysis and Experimental Design,
XIV/300 pages, 1998
P. MandllM. Huskov;} (Eds.)
Asymptotic Statistics, M. Moryson
X/474 pages, 1994 Testing for Random Walk Coefficients in
Regression and State Space Models,
P. DirschedllR. Ostermann (Eds.) XV/ 317 pages, 1998
Computational Statistics,
VII/553 pages, 1994 S. Biffignandi (Ed.)
Micro- and Macrodata of Firms,
C. P. KitsosIW. G. Muller (Eds.) XUn76 pages, 1999
MODA 4 - Advances in Model-Oriented
w.
Data Analysis, HardlelHua Liang/J. Gao
XIV/297 pages, 1995 Partially Linear Models,
Xl203 pages, 2000
H. Schmidli
Reduced Rank Regression,
Xl179 pages, 1995
Paul-Andre Monney
A Mathematical Theory
of Arguments
for Statistical Evidence
Springer-Verlag Berlin Heidelberg GmbH
Series Editors
Werner A. Muller
Martina Bihn
Author
Paul-Andre Manney
Purdue University
Department of Statistics
1399 Mathematical Sciences Building
West Lafayette, IN 47907-1399
USA
[email protected]
ISSN 1431-1968
ISBN 978-3-7908-1527-6 ISBN 978-3-642-51746-4 (eBook)
DOI 10.1007/978-3-642-51746-4
Cataloging-in-Publication Data applied for
Die Deutsche Bibliothek - CIP-Einhcitsaufnahme
Monney, Paul-Andre: A mathematical theory of arguments for statistical evidence / Paul-Andre Monney. -
Heidelberg: Physica-VerI.. 2003
(Contributions to statistics)
This work is subject to copyright. All rights are reserved, whether the whole or part of the material is
concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting,
reproduction on microfilm or in any other way, and storage in data banks. Duplication of this publication or
parts thereof is permitted only under the provisions of the German Copyright Law of September 9, 1965, in
its current version, and permission for use must always be obtained from Physica-Verlag. Violations are
liable for prosecution under the German Copyright Law.
© Springer-Verlag Berlin Heidelberg 2003
Originally published by Springer-Verlag Berlin Heidelberg New YO fk 2003.
The use of general descriptive names, registered names, trademarks, etc. in this publication does not imply, even
in the absence of a specific statement, that such names are exempt from the relevant protective laws and
regulations and therefore free for general use.
Softcover Design: Erich Kirchner, Heidelberg
SPIN 10890114 88/2202-5 4 3 2 I 0 - Printed on acid-free and non-aging paper
To Brenda
Preface
The subject of this book is the reasoning under uncertainty based on sta
tistical evidence, where the word reasoning is taken to mean searching for
arguments in favor or against particular hypotheses of interest. The kind of
reasoning we are using is composed of two aspects. The first one is inspired
from classical reasoning in formal logic, where deductions are made from a
knowledge base of observed facts and formulas representing the domain spe
cific knowledge. In this book, the facts are the statistical observations and
the general knowledge is represented by an instance of a special kind of sta
tistical models called functional models. The second aspect deals with the
uncertainty under which the formal reasoning takes place. For this aspect,
the theory of hints [27] is the appropriate tool. Basically, we assume that
some uncertain perturbation takes a specific value and then logically eval
uate the consequences of this assumption. The original uncertainty about
the perturbation is then transferred to the consequences of the assumption.
This kind of reasoning is called assumption-based reasoning. Before going
into more details about the content of this book, it might be interesting to
look briefly at the roots and origins of assumption-based reasoning in the
statistical context.
In 1930, R.A. Fisher [17] defined the notion of fiducial distribution as
the result of a new form of argument, as opposed to the result of the older
Bayesian argument. The idea of functional model was implicitly present in the
fiducial argument, as shown by the analysis of Dawid and Stone [10]. Several
other authors, like e.g. Bunke [7] and Plante [35], have considered the fiducial
argument from a functional perspective. On the other hand, Dempster [14]
presented a theory that was broad enough to include both the fiducial ar
gument and the standard Bayesian methods. A key concept introduced by
Dempster was the notion of upper and lower probabilities induced by a mul
tivaluecl mapping [12]. Some of these ideas were later used by Shafer [38] and
resulted in a mathematical theory of evidence using belief and plausibility
functions. Furthermore, Kohlas & Monney [27] reinterpreted the Dempster
Shafer theory of evidence which resulted in the theory of hints.
In view of this historical development, it is not surprising that the theory
of hints is the appropriate tool to represent the result of assumption-based
reasoning on functional models and their generalization. Each hint about
VIII Preface
the unknown value of the parameter permits to evaluate some hypothesis
of interest by computing its degree of support and its degree of plausibility.
The degree of support measures the strength of the arguments speaking in
favor of the hypothesis, whereas the degree of plausibility measures the de
gree of compatibility between the hypothesis and the available evidence. The
plausibility function gives a clear and natural interpretation to the likelihood
function. Along with the concept of hint, the notion of generalized functional
model and its analysis by assumption-based reasoning is central to this book.
In the first two chapters, the theory of generalized functional models for
a discrete parameter is developed. This theory is used to define a generalized
notion of weight of evidence. Obviously, such a concept has many possible
fields of applications, ranging from law (e.g. in investigating the statistical
evidence of discrimination), to business (e.g. in evaluating various competing
economic hypotheses) and medicine (e.g. in evaluating the merits of treat
ments based on clinical trials).
The second part of the book is dedicated to the study of special linear
functional models called Gaussian linear models. Since these models involve
continuous spaces, we need to develop a theory of continuous hints, which
is done in chapter 3. In the following chapters, several kinds of Gaussian
linear systems are analyzed, each time resulting in a Gaussian hint for the
parameter. In particular, the analysis of the classical linear regression model
is performed.
As there are usually several different sources of information for some re
lated questions, the problem of how to pool several Gaussian hints is ad
dressed. Then, by properly defining the concept of a marginal hint, it is
shown that Gaussian hints form a valuation system in the sense of Shenoy
and Shafer [44] for which the so-called local propagation axioms are satis
fied. This allows for an efficient computation of the inferred hint about the
parameter vector of interest. It should be mentioned that valuation systems
and their axioms constitute one of the most important achievements in un
certain reasoning and Gaussian hints form a further instance of this general
framework. The local propagation algorithm is presented in chapter 7.
Finally, in chapter 8, an application of our theory to a problem in control
theory is considered. More precisely, it is shown that the celebrated Kalman
filter [23] can be derived from our approach by locally propagating Gaussian
hints in a Markov tree.
It is my pleasure to acknowledge a number of people who have helped to
bring this book into existence. I am deeply indebted to Prof. J. Kohlas for
his many suggestions and ideas which are at the origin of this work. I also
want to thank him for his constant support and interest in my research.
I also wish to express my gratitude to Prof. H.W. Brachinger and Prof.
A.P. Dempster for their support and encouragement. I am thankful to Prof.
P.P. Shenoy and Prof. P. Smets for the many interesting and stimulating
discussions we had during the time they spent at the University of Fribourg.
Preface IX
My present and past friends and colleagues of the Seminar of Statistics,
the Department of Quantitative Economics and the Department of Informa
tics of the University of Fribourg also deserve my gratitude for their help and
for the friendly working atmosphere I have enjoyed during so many years. I
also want to thank the Swiss Federal Office for Education and Science for
their financial support.
Most of all, my gratitude goes to my wife Brenda, who provided me with
her constant support, enthusiasm and encouragement during the preparation
of this book.
Fribourg, July 2002 Paul-Andre Manney
Contents
1. The Theory of Generalized Functional Models . . . . . . . . . . . . 1
l.1 The Theory of Hints on Finite Frames .................... 1
1.1.1 Definition of a Hint and its Associated Functions. . . . . 2
1.1.2 Combination of Hints. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
l.l.3 Mass functions and gem-functions. . . . . . . . . . . . . . . . . . 7
1.1.4 Dempster Specialization Matrices ................. " 10
1.1.5 A Representation of the Dempster Specialization Matrix 11
1.l.6 Combining Several Hints. . . . . . . . . . . . . . . . . . . . . . . . .. 12
l.2 Application............................................ 13
l.3 The Combination of Closed Hints. . . . . . . . . . . . . . . . . . . . . . . .. 14
l.4 The Theories of Bayes and Fisher ........................ 18
l.4.1 Jessica's Pregnancy Test .......................... 18
l.4.2 The Solution of Bayes. . . . . . . . . . . . . . . . . . . . . . . . . . . .. 18
l.4.3 The Solution of Fisher. . . . . . . . . . . . . . . . . . . . . . . . . . .. 19
1.5 The Definition and Analysis of a Generalized Functional Model 21
1.6 Generalized Functional Models and Hints. . . . . . . . . . . . . . . . .. 24
l. 7 Examples of Generalized Functional Models .............. " 26
1.7.1 Jessica's Pregnancy Test .......................... 26
1.7.2 Policy Identification (I) ........................... 28
1.7.3 Policy Identification (II) . . . . . . . . . . . . . . . . . . . . . . . . . .. 31
1.8 Prior Information. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 33
2. The Plausibility and Likelihood Functions . . . . . . . . . . . . . . .. 39
2.1 The Likelihood Ratio as a Weight of Evidence .............. 39
2.2 The Weight of Evidence for Composite Hypotheses. . . . . . . .. 40
2.3 Functional and Distribution Models. . . . . . . . . . . . . . . . . . . . . .. 43
2.3.1 The Distribution Model of the Problem ........... " 43
2.3.2 A GFM Obtained by Conditional Embedding ...... " 44
2.3.3 A GFM Inspired by Dempster's Structures of the First
Kind ............................................ 46
2.4 Evidence About a Survival Rate. . . . . . . . . . . . . . . . . . . . . . . . .. 49
2.5 Degrees of Support as Weights of Evidence ................ 55
XII Contents
3. Hints on Continuous Frames and Gaussian Linear Systems 59
3.1 Continuous Hints. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 59
3.2 Gaussian Hints ....................................... " 62
3.3 Precise Gaussian Hints. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 64
3.4 Gaussian Linear Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 68
4. Assumption-Based Reasoning with Classical Regression Mo-
dels... . ...... .. .... .. . .. . . ... . . .. ..... . . . ... ... .. . ...... .. 71
4.1 Classical Linear Regression Models as Special Gaussian Li-
near Systems .......................................... 71
4.2 The Principle of the Inference. . . . . . . . . . . . . . . . . . . . . . . . . . .. 72
4.3 The Result of the Inference. . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 76
4.4 Changing Basis ........................................ 79
4.5 Permissible Bases and Admissible Matrices ................ 82
4.6 Different Representations of the Result of the Inference. . . . .. 86
4.6.1 A Representation Derived from Linear Dependencies.. 86
4.6.2 A Representation Derived from a Special Class of Ad
missible Matrices. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 88
4.7 Full Rank Matrices T2 such that T2A = O. . . . . . . . . . . . . . . . .. 90
4.7.1 A Matrix Based on Linear Dependencies.. .. .... . . .. 90
4.7.2 A Matrix Based on the Householder Method. . . . . . . .. 91
4.7.3 A Matrix Based on Classical Variable Elimination. . .. 91
4.7.4 A Matrix Based on a Generalized Inverse of A ....... 93
5. Assumption-Based Reasoning with General Gaussian Linear
Systems .................................................. 97
5.1 The Principle of the Inference. . . . . . . . . . . . . . . . . . . . . . . . . . .. 97
5.2 The Gaussian Hint Inferred from a Gaussian Linear System
of the First Kind . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 98
5.3 The Gaussian Hint Inferred from a Gaussian Linear System
of the Second Kind ... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 99
5.4 The Gaussian Hint Inferred from a Gaussian Linear System
of the Third Kind ...................................... 104
5.5 The Gaussian Hint Inferred from a Gaussian Linear System
of the Fourth Kind ..................................... 105
5.6 Canonical Proper Potentials ............................. 106
6. Gaussian Hints as a Valuation System .................... 109
6.1 Shenoy-Shafer's Axiomatic Valuation Systems .............. 109
6.2 Marginalization of Gaussian Hints ........................ 110
6.2.1 Marginalization of a Precise Gaussian Hint .......... 111
6.2.2 Marginalization of a Non-Precise Gaussian Hint ...... 112
6.3 Vacuous Extension of Gaussian Hints ..................... 117
6.4 Transport of Gaussian Hints ............................. 117
6.5 Projection and Composition of Potentials .................. 118