Finite Mixture Distributions

147 Pages·1981·3.691 MB·English

by  B. S. Everitt, D. J. Hand (auth.)

MONOGRAPHS ON
APPLl~[) PROBABILITY AND STATISTICS
FINITE MIXTURE DISTRIBUTIONS
MONOGRAPHS ON
APPLIED PROBABILITY AND STATISTICS
General Editor
D.R. COX, FRS
Also available in the series
Probability, Statistics and Time
M.S. Bartlett
The Statistical Analysis of Spatial Pattern
M.S. Bartlett
Stochastic Population Models in Ecology and Epidemiology
M.S. Bartlett
Risk Theory
R.E. Beard, T. Pentikilinen and E. Pesonen
Point Processes
O.R. Cox and V. Isham
Analysis of Binary Data
D.R. Cox
The Statistical Analysis of Series of Events
D.R. Cox and P.A.W. Lewis
Queues
O.R. Cox and W.L. Smith
Stochastic Abundance Models
E. Engen
The Analysis of Contingency Tables
B.S. Everitt
Finite Mixture Distributions
B.S. Everitt and O.J. Hand
Population Genetics
W.J. Ewens
Classification
A.D. Gordon
Monte Carlo Methods
J.M. Hammersley and D.C. Handscomb
Identification of Outliers
D.M. Hawkins
Multivariate Analysis in Behavioural Research
A.E. Maxwell
Applications of Queueing Theory
G.F. Newell
Some Basic Theoryfor Statistical Inference
E.J.G. Pitman
Statistical Inference
S.D. Silvey
Models in Regression and Related Topics
P. Sprent
Sequential Methods in Statistics
G.B. Wetherill
Finite Mixture Distributions
B. S. EVERITT
Head of Biometrics Unit, Institute of Psychiatry
and
DJ. HAND
Lecturer, Biometrics Unit, Institute of Psychiatry
LONDON NEW YORK
CHAPMAN AND HALL
First published in 1981 by
Chapman and Hall Ltd
II New Fetter Lane, London EC4P 4EE
Published in the USA by
Chapman and Hall
in association with Methuen, Inc.
733 Third Avenue, New York. NY 10017
© 1981 B.S. Everitt and 0.1. Hand
Softcover reprint of the hardcover 1st edition 1981
ISBN-13: 978-94-009-5899-9
All rights reserved. No part of
this book may be reprinted, or reproduced
or utilized in any form or by any electronic,
mechanical or other means, now known or hereafter
invented, including photocopying and recording,
or in any information storage and retrieval
system, without permission in writing
from the Publisher.
British Library Cataloguing in Publication Data
Everitt. B S
Finite mixture distributions. - (Monographs
on applied probability and statistics).
I. Distribution (Probability theory)
I. Title II. Series
5195'32 QA273.6 80-41131
ISBN-13: 978-94-009-5899-9 e-ISBN-13: 978-94-009-5897-5
001: 10.1007/978-94-009-5897-5
Contents
Preface page xi
1 General introduction 1
1.1 Introduction 1
1.2 Some applications of finite mixture distributions 2
1.3 Definition 4
1.4 Estimation methods 7
1.4.1 Maximum likelihood 8
1.4.2 Bayesian estimation 11
1.4.3 Inversion and error minimization 13
1.4.4 Other methods 20
1.4.5 Estimating the number of components 22
1.5 Summary 23
2 Mixtures of normal distributions 25
2.1 Introduction 25
2.2 Some descriptive properties of mixtures of normal
distributions 27
2.3 Estimating the parameters in normal mixture
distributions 30
2.3.1 Method of moments estimation 31
2.3.2 Maximum likelihood estimation 36
2.3.3 Maximum likelihood estimates for grouped
data 45
2.3.4 Obtaining initial parameter values for the
maximum likelihood estimation algorithms 47
2.3.5 Graphical estimation techniques 48
2.3.6 Other estimation methods 51
2.4 Summary 57
3 Mixtures of exponential and other continuous
distributions 59
3.1 Exponential mixtures 59
VIll CONTENTS
3.2 Estimating exponential mixture parameters 60
3.2.1 The method of moments and generalizations 60
3.2.2 Maximum likelihood 73
3.3 Properties of exponential mixtures 79
3.4 Other continuous distributions 80
3.4.1 Non-central chi-squared distribution 81
3.4.2 Non-central F distribution 81
3.4.3 Beta distributions 82
3.4.4 Doubly non-central t distribution 83
3.4.5 Planck's distribution 83
3.4.6 Logistic 84
3.4.7 Laplace 84
3.4.8 Weibull 84
3.4.9 Gamma 86
3.5 Mixtures of different component types 86
3.6 Summary 87
4 Mixtures of discrete distributions 89
4.1 Introduction 89
4.2 Mixtures of binomial distributions 89
4.2.1 Moment estimators for binomial mixtures 90
4.2.2 Maximum likelihood estimators for mixtures
of binomial distributions 94
4.2.3 Other estimation methods for mixtures of
binomial distributions 97
4.3 Mixtures of Poisson distributions 97
4.3.1 Moment estimators for mixtures of Poisson
distributions 98
4.3.2 Maximum likelihood estimators for a Poisson
mixture 100
4.4 Mixtures of Poisson and binomial distributions 102
4.5 Mixtures of other discrete distributions 102
4.6 Summary 105
5 Miscellaneous topics 107
5.1 Introduction \07
5.2 Determining the number of components in a mixture 107
5.2.1 Informal diagnostic tools for the detection of
mixtures \07
5.2.2 Testing hypotheses on the number of
components in a mixture 116
CONTENTS ix
5.3 Probability density function estimation 118
5.4 Miscellaneous problems 124
5.5 Summary 127
References 129
Index 139
Preface
Finite mixture distributions arise in a variety of applications ranging
from the length distribution of fish to the content of DNA in the
nuclei of liver cells. The literature surrounding them is large and
goes back to the end of the last century when Karl Pearson published
his well-known paper on estimating the five parameters in a mixture
of two normal distributions. In this text we attempt to review this
literature and in addition indicate the practical details of fitting such
distributions to sample data. Our hope is that the monograph will be
useful to statisticians interested in mixture distributions and to re
search workers in other areas applying such distributions to their
data. We would like to express our gratitude to Mrs Bertha Lakey
for typing the manuscript.
Institute oj Psychiatry B.S. Everitt
University of London D.l Hand
1980
CHAPTER I
General introduction
1.1 Introduction
This monograph is concerned with statistical distributions which can
be expressed as superpositions of (usually simpler) component
distributions. Such superpositions are termed mixture distributions
or compound distributions. For example, the distribution of height in
a population of children might be expressed as follows:
h(height) = fg(height: age)f(age)d age (1.1)
where g(height: age) is the conditional distribution of height on age,
and/(age) is the age distribution of the children in the population.
The probability density function of height has been expressed as an
infinite superposition of conditional height density functions, and is
thus a mixture density. As a further example, we might express the
height density function in the form
h(height) = h J (height: boy) p(boy) + h2 (height; girl) p (girl) (1.2)
where p(boy) and p(girl) are, respectively, the probabilities that a
member of the population is a boy or a girl, and h1 and h2 are the
height density functions for boys and girls. Thus the density function
of height has been expressed as a superposition of two conditional
density functions.
Density functions of the forms (1.1) and (1.2) have received increas
ing attention in the statistical literature recently, partly because of
interest in their mathematical properties, but mainly because of the
considerable number of areas in which such density functions are
encountered. In this text we are primarily concerned with mixtures
such as (1.2). involving a finite number of components, and which,
not surprisingly, are termed finite mixtures. The problems of central