Some Basic Theory for Statistical Inference: Monographs on Applied Probability and Statistics

119 Pages·2017·9.16 MB·English

by  E. J. G. Pitman

MONOGRAPHS ON
APPLIED PROBABILITY AND STATISTICS
General Editors
M.S. BARTLETT, F.R.S. andD.R. COX, F.R.S.
SOME BASIC THEORY
FOR STATISTICAL INFERENCE

Some Basic Theory
for Statistical Inference
E.J.G. PITMAN
M.A. D.Sc. F.A.A.
Emeritus Professor ofM athematics
University ofTasmania
LONDON
CHAPMAN AND HALL
Boca Raton London New York
A Halsted Press Book
CRC Press is an imprint of the
JOHNT ayWlor I&L FEranYcis G&ro upS, aOn NinSfo,r mNa EbuWsines sYORK
First published 1979 by CRC Press
Taylor & Francis Group
6000 Broken Sound Parkway NW, Suite 300
Boca Raton, FL 33487-2742
Reissued 2018 by CRC Press
© 1979 E.J.G. Pitman
CRC Press is an imprint of Taylor & Francis Group, an Informa business
No claim to original U.S. Government works
This book contains information obtained from authentic and highly regarded sources. Reasonable efforts
have been made to publish reliable data and information, but the author and publisher cannot assume
responsibility for the validity of all materials or the consequences of their use. The authors and publishers
have attempted to trace the copyright holders of all material reproduced in this publication and apologize to
copyright holders if permission to publish in this form has not been obtained. If any copyright material has
not been acknowledged please write and let us know so we may rectify in any future reprint.
Except as permitted under U.S. Copyright Law, no part of this book may be reprinted, reproduced, transmit-
ted, or utilized in any form by any electronic, mechanical, or other means, now known or hereafter invented,
including photocopying, microfilming, and recording, or in any information storage or retrieval system,
without written permission from the publishers.
Trademark Notice: Product or corporate names may be trademarks or registered trademarks, and are used
only for identification and explanation without intent to infringe.
Library of Congress Cataloging in Publication Data
Pitman, Edwin J. G.
Some basic theory for statistical inference.
(Monographs on applied probability and statistics)
Includes bibliographical references.
1. Mathematical statistics. I. Title.
QA276.P537 519.5 78-11921
ISBN 0–470–26554–X
A Library of Congress record exists under LC control number: 78011921
Publisher’s Note
The publisher has gone to great lengths to ensure the quality of this reprint but points out that some imper-
fections in the original copies may be apparent.
Disclaimer
The publisher has made every effort to trace copyright holders and welcomes correspondence from those
they have been unable to contact.
ISBN 13: 978-1-315-89767-7 (hbk)
ISBN 13: 978-1-351-07677-7 (ebk)
Visit the Taylor & Francis Web site at http://www.taylorandfrancis.com and the
CRC Press Web site at http://www.crcpress.com
CONTENTS
Preface page vii
Chapter one 1
Basic Principles of the Theory of Inference,
The Likelihood Principle, Sufficient Statistics
Chapter two 6
Distance between Probability Measures
Chapter three 11
Sensitivity of a Family of Probability Measures
with respect to a Parameter
Chapter Jour 24
Sensitivity Rating, Conditional Sensitivity,
The Discrimination Rate Statistic
Chapter five 29
Efficacy, Sensitivity, The Cramer-Rao Inequality
Chapter six 50
Many Parameters, The Sensitivity Matrix
Chapter seven 56
Asymptotic Power of a Test, Asymptotic
Relative Efficiency
Chapter eight 63
Maximum Likelihood Estimation
Chapter nine 79
The Sample Distribution Function
Appendix: Mathematical Preliminaries 98
References 107
Index 109
v

PREFACE
This book is largely based on work done in 1973 while I was
a Senior Visiting Research Fellow, supported by the Science
Research Council, in the Mathematics Department of Dundee
University, and later while a visitor in the Department of Statistics
at the University of Melbourne. In both institutions, and also
at the 1975 Summer Research Institute of the Australian Mathe-
matical Society, I gave a series of talks with the general title
'A New Look at Some Old Statistical Theory'. That title indicates
fairly well my intentions when I started writing this book.
I was encouraged in my project by some remarks of Professor
D.V. Lindley (1972) in his review of The Theory of Statistical
Inference by S. Zacks:
One point that does distress me about this book-and let
me hasten to say that this is not the fault of the author-is
the ugliness of some of the material and the drabness of most
of it ... The truth is that the mathematics of our subject has
little beauty to it. Is it wrong to ask that a subject should be a
delight for its own sake? I hope not. Is there no elegant proof
of the consistency of maximum likelihood, or do we have to
live with inelegant conditions?
I share Lindley's dissatisfaction with much statistical theory.
This book is an attempt to present some of the basic mathematical
results required for statistical inference with some elegance
as well as precision, and at a level which will make it readable
by most students of statistics. The topics treated are simply
those that I have been able to do to my own satisfaction by
this date.
I am grateful to those who, at Dundee, Melbourne, or Sydney,
were presented with earlier versions, and who helped with their
questions and criticisms. I am specially grateful to Professor
E.J. Williams, with whom I have had many discussions, and
who arranged for and supervised the typing; to Judith Adams
and Judith Benney, who did most ofthe typing, and to Betty La by,
who drew the diagrams.
E.J.G.P.
vii

CHAPTER ONE
BASIC PRINCIPLES OF
THE THEORY OF INFERENCE
THE LIKELIHOOD PRINCIPLE
SUFFICIENT STATISTICS
In developing the theory of statistical inference, I find it helpful
to bear in mind two considerations. Firstly, I take the view that
the aim of the theory of inference is to provide a set of principles,
which help the statistician to assess the strength of the evidence
supplied by a trial or experiment for or against a hypothesis,
or to assess the reliability of an estimate derived from the result
of such a trial or experiment. In making such an assessment
we may look at the results to be assessed from various points
of view, and express ourselves in various ways. For example,
we may think and speak in terms of repeated trials as for confi-
dence limits or for significance tests, or we may consider the
effect of various loss functions. Standard errors do give us some
comprehension of reliability; but we may sometimes prefer
to think in terms of prior and posterior distributions. All of
these may be helpful, and none should be interdicted. The theory
of inference is persuasive rather than coercive.
Secondly, statistics being essentially a branch of applied
mathematics, we should be guided in our choice of principles
and methods by the practical applications. All actual sample
spaces are discrete, and all observable random variables have
discrete distributions. The continuous distribution is a mathe-
matical construction, suitable for mathematical treatment, but
not practically observable. We develop our fundamental concepts,
principles and methods, in the study of discrete distributions. In
the case of a discrete sample space, it is easy to understand and
appreciate the practical or experimental significance and value of
conditional distributions, the likelihood principle, the principles
of sufficiency and conditionality, and the method of maximum
likelihood. These are then extended to more general distributions
by means of suitable definitions and mathematical theorems.