Mathematical Statistics with Mathematica®

489 Pages·2002·8.93 MB·English

by  Colin Rose, Murray D. Smith (auth.)

Springer Texts in Statistics
Advisors:
George Casella Stephen Fienberg Ingram Olkin
Springer
New York
Berlin
Heidelberg
Hong Kong
Londoll
Milan
Paris
Tokyo
Springer Texts in Statistics
Alfred: Elements of Statistics for the Life and Social Sciences
Berger: An Introduction to Probability and Stochastic Processes
Bilodeau and Brenner: Theory of Multivariate Statistics
Blom: Probability and Statistics: Theory and Applications
Brockwell and Davis: Introduction to Times Series and Forecasting,
Second Edition
Chow and Teicher: Probability Theory: Independence, Interchangeability,
Martingales, Third Edition
Christensen: Advanced Linear Modeling: Multivariate, Time Series, and
Spatial Data; Nonparametric Regression and Response Surface
Maximization, Second Edition
Christensen: Log-Linear Models and Logistic Regression, Second Edition
Christensen: Plane Answers to Complex Questions: The Theory of Linear
Models, Third Edition
Creighton: A First Course in Probability Models and Statistical Inference
Davis: Statistical Methods for the Analysis of Repeated Measurements
Dean and Voss: Design and Analysis of Experiments
du Toif, Steyn, and Stumpf Graphical Exploratory Data Analysis
Durrett: Essentials of Stochastic Processes
Edwards: Introduction to Graphical Modelling, Second Edition
Finkelstein and Levin: Statistics for Lawyers
Flury: A First Course in Multivariate Statistics
Jobson: Applied Multivariate Data Analysis, Volume I: Regression and
Experimental Design
Jobson: Applied Multivariate Data Analysis, Volume II: Categorical and
Multivariate Methods
Kalbfleisch: Probability and Statistical Inference, Volume I: Probability,
Second Edition
Kalbfleisch: Probability and Statistical Inference, Volume II: Statistical Inference,
Second Edition
Karr: Probability
Keyfitz: Applied Mathematical Demography, Second Edition
Kiefer: Introduction to Statistical Inference
Kokoska and Nevison: Statistical Tables and Formulae
Kulkarni: Modeling, Analysis, Design, and Control of Stochastic Systems
Lange: Applied Probability
Lehmann: Elements of Large-Sample Theory
Lehmann: Testing Statistical Hypotheses, Second Edition
Lehmann and Casella: Theory of Point Estimation, Second Edition
Lindman: Analysis of Variance in Experimental Design
Lindsey: Applying Generalized Linear Models
(continued after index)
Colin Rose Murray D. Smith
Mathematical Statistics
with Mathematica®
With 134 Figures
Springer
Colin Rose Murray D. Smith
Theoretical Research Institute Discipline of Econometrics and Business Statistics
Sydney NSW 2023 University of Sydney
Australia Sydney NSW 2006
[email protected] Australia
[email protected]
Editorial Board
George Casella Stephen Fienberg Ingram OIkin
Department of Statistics Department of Statistics Department of Statistics
University of Florida Carnegie Mellon University Stanford University
Gainesville, FL 32611-8545 Pittsburgh, PA 15213-3890 Stanford, CA 94305
USA USA USA
Library of Congress Cataloging-in-Publication Data
Rose, Colin.
Mathematical statistics with Mathematica / Colin Rose, Murray D. Smith.
p. em. - (Springer texts in statistics)
Includes bibliographical references and index.
Additional material to this book can be downloaded from http://extras.springer.com.
ISBN-13: 978-1-4612-7403-2 e-ISBN-13: 978-1-4612-2072-5
001: 10.1007/978-1-4612-2072-5
I. Mathematical statistics-Data processing. 2. Mathematica (Computer file) I. Smith,
Murray D. II. Title. III. Series.
QA276.4 .R67 2001
519.5'0285-dc2l 00-067926
Printed on acid-free paper.
Mathematica is a registered trademark of Wolfram Research, Inc.
© 2002 Springer-Verlag New York, Inc.
Softcover reprint of the hardcover 1st edition 2002
This Work consists ofa printed book and two CD-ROMs packaged with the book, all of which are protected by
federal copyright law and international treaty. The book may not be translated or copied in whole or in part
without the written permission of the publisher (Springer-Verlag New York, Inc., 175 Fifth Avenue, New York,
NY 10010, USA), except for brief excerpts in connection with reviews or scholarly analysis. For copyright in
formation regarding the CD-ROMs, please consult the printed information below the CD-ROM package in the
back of this publication. Use of this Work in connection with any form of informati9n storage and retrieval,
electronic adaptation, computer software, or by similar or dissimilar methodology now known, or hereafter de
veloped, other than as expressly granted in the CD-ROM copyright notice and disclaimer information, and as
expressly limited by applicable law, is strictly forbidden.
The use in this publication of trade names, trademarks, service marks and similar terms, even if they are not
identified as such, is not to be taken as an expression of opinion as to whether or not they are subject to pro
prietary rights. Where those designations appear in the book and Springer-Verlag was aware of a trademark
claim, the designations follow the capitalization style used by the manufacturer.
Production managed by Steven Pisano; manufacturing supervised by Joe Quatela.
Photocomposed pages prepared from the authors' Mathematica files.
9 8 7 6 543 2
SPIN 10931462
Springer-Verlag New York Berlin Heidelberg
A member ofB erteismannSpringer Science+ Business Media GmbH
Contents
Preface xi
Chapter 1 Introduction
1.1 Mathematical Statistics with Mathematica 1
A A New Approach 1
B Design Philosophy 1
C If You Are New to Mathematica 2
1.2 Installation, Registration and Password 3
A Installation, Registration and Password 3
B Loading mathStatica 5
C Help 5
1.3 Core Functions 6
A Getting Started 6
B Working with Parameters 8
C Discrete Random Variables 9
D Multivariate Random Variables 11
E Piecewise Distributions 13
1.4 Some Specialised Functions 15
1.5 Notation and Conventions 24
A Introduction 24
B Statistics Notation 25
C Mathematica Notation 27
Chapter 2 Continuous Random Variables
2.1 Introduction 31
2.2 Measures of Location 35
A Mean 35
B Mode 36
C Median and Quantiles 37
2.3 Measures of Dispersion 40
2.4 Moments and Generating Functions 45
A Moments 45
B The Moment Generating Function 46
C The Characteristic Function 50
D Properties of Characteristic Functions (and mgf's) 52
vi CONTENTS
E Stable Distributions 56
F Cumulants and Probability Generating Functions 60
G Moment Conversion Formulae 62
2.5 Conditioning, Truncation and Censoring 65
A Conditional I Truncated Distributions 65
B Conditional Expectations 66
C Censored Distributions 68
D Option Pricing 70
2.6 Pseudo-Random Number Generation 72
A Mathematica's Statistics Package 72
B Inverse Method (Symbolic) 74
C Inverse Method (Numerical) 75
D Rejection Method 77
2.7 Exercises 80
Chapter 3 Discrete Random Variables
3.1 Introduction 81
3.2 Probability: 'Throwing' a Die 84
3.3 Common Discrete Distributions 89
A The Bernoulli Distribution 89
B The Binomial Distribution 91
C The Poisson Distribution 95
D The Geometric and Negative Binomial Distributions 98
E The Hypergeometric Distribution 100
3.4 Mixing Distributions 102
A Component-Mix Distributions 102
B Parameter-Mix Distributions 105
3.5 Pseudo-Random Number Generation 109
A Introducing DiscreteRNG 109
B Implementation Notes 113
3.6 Exercises 115
Chapter 4 Distributions of Functions of Random Variables
4.1 Introduction 117
4.2 The Transformation Method 118
A Univariate Cases 118
B Multivariate Cases 123
C Transformations That Are Not One-to-One; Manual Methods 127
4.3 The MGF Method 130
4.4 Products and Ratios of Random Variables 133
4.5 Sums and Differences of Random Variables 136
A Applying the Transformation Method 136
B Applying the MGF Method 141
4.6 Exercises 147
CONTENTS Vll
Chapter 5 Systems of Distributions
5.1 Introduction 149
5.2 The Pearson Family 149
A Introduction 149
B Fitting Pearson Densities 151
C Pearson Types 157
D Pearson Coefficients in Terms of Moments 159
E Higher Order Pearson-Style Families 161
5.3 Johnson Transformations 164
A Introduction 164
B SL System (Lognormal) 165
C Su System (Unbounded) 168
D SB System (Bounded) 173
5.4 Gram-Charlier Expansions 175
A Definitions and Fitting 175
B Hermite Polynomials; Gram-Charlier Coefficients 179
5.5 Non-Parametric Kernel Density Estimation 181
5.6 The Method of Moments 183
5.7 Exercises 185
Chapter 6 Multivariate Distributions
6.1 Introduction 187
A Joint Density Functions 187
B Non-Rectangular Domains 190
C Probability and Prob 191
D Marginal Distributions 195
E Conditional Distributions 197
6.2 Expectations, Moments, Generating Functions 200
A Expectations 200
B Product Moments, Covariance and Correlation 200
C Generating Functions 203
D Moment Conversion Formulae 206
6.3 Independence and Dependence 210
A Stochastic Independence 210
B Copulae 211
6.4 The Multivariate Normal Distribution 216
A The Bivariate Normal 216
B The Trivariate Normal 226
C CDF, Probability Calculations and Numerics 229
D Random Number Generation for the Multivariate Normal 232
6.5 The Multivariate t and Multivariate Cauchy 236
6.6 Multinomial and Bivariate Poisson 238
A The Multinomial Distribution 238
B The Bivariate Poisson 243
6.7 Exercises 248
viii CONTENTS
Chapter 7 Moments of Sampling Distributions
7.1 Introduction 251
A Overview 251
B Power Sums and Symmetric Functions 252
7.2 Unbiased Estimators of Population Moments 253
A Unbiased Estimators of Raw Moments of the Population 253
B h-statistics: Unbiased Estimators of Central Moments 253
C k-statistics: Unbiased Estimators of Cumulants 256
D Multivariate h-and k-statistics 259
7.3 Moments of Moments 261
A Getting Started 261
B Product Moments 266
C Cumulants of k-statistics 267
7.4 Augmented Symmetries and Power Sums 272
A Definitions and a Fundamental Expectation Result 272
B Application 1: Understanding Unbiased Estimation 275
C Application 2: Understanding Moments of Moments 275
7.5 Exercises 276
Chapter 8 Asymptotic Theory
8.1 Introduction 277
8.2 Convergence in Distribution 278
8.3 Asymptotic Distribution 282
8.4 Central Limit Theorem 286
8.5 Convergence in Probability 292
A Introduction 292
B Markov and Chebyshev Inequalities 295
C Weak Law of Large Numbers 296
8.6 Exercises 298
Chapter 9 Statistical Decision Theory
9.1 Introduction 301
9.2 Loss and Risk 301
9.3 Mean Square Error as Risk 306
9.4 Order Statistics 311
A Definition and OrderStat 311
B Applications 318
9.5 Exercises 322
Chapter 10 Unbiased Parameter Estimation
10.1 Introduction 325
A Overview 325
B SuperD 326
CONTENTS ix
10.2 Fisher Information 326
A Fisher Information 326
B Alternate Form 329
C Automating Computation: Fisherlnformation 330
D Multiple Parameters 331
E Sample Information 332
10.3 Best Unbiased Estimators 333
A The Cramer-Rao Lower Bound 333
B Best Unbiased Estimators 335
10.4 Sufficient Statistics 337
A Introduction 337
B The Factorisation Criterion 339
10.5 Minimum Variance Unbiased Estimation 341
A Introduction 341
B The Rao-Blackwell Theorem 342
C Completeness and MVUE 343
D Conclusion 346
10.6 Exercises 347
Chapter 11 Principles of Maximum Likelihood Estimation
11.1 Introduction 349
A Review 349
B SuperLog 350
11.2 The Likelihood Function 350
11.3 Maximum Likelihood Estimation 357
11.4 Properties of the ML Estimator 362
A Introduction 362
B Small Sample Properties 363
C Asymptotic Properties 365
D Regularity Conditions 367
E Invariance Property 369
11.5 Asymptotic Properties: Extensions 371
A More Than One Parameter 371
B Non-identically Distributed Samples 374
11.6 Exercises 377
Chapter 12 Maximum Likelihood Estimation in Practice
12.1 Introduction 379
12.2 FindMaximum 380
12.3 A Journey with F indMaximum 384
12.4 Asymptotic Inference 392
A Hypothesis Testing 392
B Standard Errors and t-statistics 395
x CONTENTS
12.5 Optimisation Algorithms 399
A Preliminaries 399
B Gradient Method Algorithms 401
12.6 The BFGS Algorithm 405
12.7 The Newton-Raphson Algorithm 412
12.8 Exercises 418
Appendix
A.l Is That the Right Answer, Dr Faustus? 421
A.2 Working with Packages 425
A.3 Working with =, ---t, == and:= 426
A.4 Working with Lists 428
A.5 Working with Subscripts 429
A6 Working with Matrices 433
A7 Working with Vectors 438
A8 Changes to Default Behaviour 443
A9 Building Your Own mathStatica Function 446
Notes 447
References 463
Index 469