Table Of Content

MULTIPARAMETRIC STATISTICS This page intentionally left blank MULTIPARAMETRIC STATISTICS BY Vadim I. Serdobolskii Moscow State Institute of Electronics and Mathematics Moscow, Russia AMSTERDAM•BOSTON•HEIDELBERG•LONDON NEWYORK•OXFORD•PARIS•SANDIEGO SANFRANCISCO•SINGAPORE•SYDNEY•TOKYO Elsevier Radarweg29,POBox211,1000AEAmsterdam,TheNetherlands LinacreHouse,JordanHill,OxfordOX28DP,UK Firstedition2008 Copyright(cid:2)c 2008ElsevierB.V.Allrightsreserved Nopartofthispublicationmaybereproduced,storedinaretrievalsystemor transmittedinanyformorbyanymeanselectronic,mechanical,photocopying, recordingorotherwisewithoutthepriorwrittenpermissionofthepublisher PermissionsmaybesoughtdirectlyfromElsevier’sScience&TechnologyRights DepartmentinOxford,UK:phone(+44)(0)1865843830;fax(+44)(0)1865 853333;email:[email protected] onlinebyvisitingtheElsevierwebsiteathttp://elsevier.com/locate/permissions, andselectingObtaining permission to use Elsevier material Notice Noresponsibilityisassumedbythepublisherforanyinjuryand/ordamageto personsorpropertyasamatterofproductsliability,negligenceorotherwise,or fromanyuseoroperationofanymethods,products,instructionsorideascontained inthematerialherein.Becauseofrapidadvancesinthemedicalsciences,in particular,independentverificationofdiagnosesanddrugdosagesshouldbemade Library of Congress Cataloging-in-Publication Data AcatalogrecordforthisbookisavailablefromtheLibraryofCongress British Library Cataloguing in Publication Data AcataloguerecordforthisbookisavailablefromtheBritishLibrary ISBN:978-0-444-53049-3 ForinformationonallElsevierpublications visitourwebsiteatwww.books.elsevier.com PrintedandboundinTheUnitedKingdom 08 09 10 11 12 10 9 8 7 6 5 4 3 2 1 ON THE AUTHOR Vadim Ivanovich Serdobolskii is a professor at Applied Mathematics Faculty, Moscow State Institute of Electronics and Mathematics. HegraduatedfromMoscowStateUniversityasphysicisttheoretician. In 1952, received his first doctorate from Nuclear Research Insti- tute of Moscow State University in investigations in resonance theory of nuclear reactions. In 2001, received the second (advanced) doctoral degree from the Faculty of Calculation Mathematics and Cybernetics of Moscow State University in the development of asymptotical theory of statistical analysis of high-dimensional observations. He is the author of the monograph “Multivariate Statistical Anal- ysis. A High-Dimensional Approach,” Kluwer Academic Publishers, Dordrecht, 2000. Internet page: serd.miem.edu.ru E-mail: [email protected] v This page intentionally left blank CONTENTS Foreword xi Preface xiii Chapter 1. Introduction: the Development of Multiparametric Statistics 1 The Stein effect . . . . . . . . . . . . . . . . . . 4 The Kolmogorov Asymptotics . . . . . . . . . . 10 Spectral Theory of Increasing Random Matrices . . . . . . . . . . . . . . . . . . . . . . 12 Constructing Multiparametric Procedures. . . . 17 Optimal Solution to Empirical Linear Equations . . . . . . . . . . . . . . . . . . . . . 19 Chapter 2. Fundamental Problem of Statistics 21 2.1. Shrinkage of Sample Mean Vector . . . . . . . . . . 23 Shrinkage for Normal Distributions . . . . . . . 24 Shrinkage for a Wide Class of Distributions . . . 29 Conclusions . . . . . . . . . . . . . . . . . . . . 32 2.2. Shrinkage of Unbiased Estimators . . . . . . . . . . 33 Special Shrinkage of Normal Estimators . . . . . 33 Shrinkage of Arbitrary Unbiased Estimators . . 35 Limit Quadratic Risk of Shrinkage Estimators . 41 Conclusions . . . . . . . . . . . . . . . . . . . . 43 2.3. Shrinkage of Infinite-Dimensional Vectors . . . . . 45 Normal distributions . . . . . . . . . . . . . . . 46 Wide Class of Distributions . . . . . . . . . . . 50 Conclusions . . . . . . . . . . . . . . . . . . . . 54 2.4. Unimprovable Component-Wise Estimation . . . . 56 Estimator for the Density of Parameters . . . . 59 Estimator for the Best Estimating Function . . 63 vii viii CONTENTS Chapter 3. Spectral Theory of Large Sample Covariance Matrices 71 3.1. Spectral Functions of Large Sample Covariance Matrices . . . . . . . . . . . . . . . . . . . . . . . . 75 Gram Matrices . . . . . . . . . . . . . . . . . . . 75 Sample Covariance Matrices . . . . . . . . . . . 83 Limit Spectra . . . . . . . . . . . . . . . . . . . 88 3.2. Spectral Functions of Infinite Sample Covariance Matrices . . . . . . . . . . . . . . . . . . . . . . . . 97 Dispersion Equations for Infinite Gram Matrices . . . . . . . . . . . . . . . . . . . . . . 98 Dispersion Equations for Sample Covariance Matrices . . . . . . . . . . . . . . . . . . . . . . 103 Limit Spectral Equations . . . . . . . . . . . . . 105 3.3. Normalization of Quality Functions . . . . . . . . . 114 Spectral Functions of Sample Covariance Matrices . . . . . . . . . . . . . . . . . . . . . . 116 Normal Evaluation of Sample-Dependent Functionals . . . . . . . . . . . . . . . . . . . . . 117 Conclusions . . . . . . . . . . . . . . . . . . . . 124 Chapter 4. Asymptotically Unimprovable Solution of Multivariate Problems 127 4.1. Estimators of Large Inverse Covariance Matrices . 129 Problem Setting . . . . . . . . . . . . . . . . . . 130 Shrinkage for Inverse Covariance Matrices . . . 131 Generalized Ridge Estimators . . . . . . . . . . 133 Asymptotically Unimprovable Estimator . . . . 138 Proofs for Section 4.1 . . . . . . . . . . . . . . . 140 4.2. Matrix Shrinkage Estimators of Expectation Vectors 147 Limit Quadratic Risk for Estimators of Vectors 148 Minimization of the Limit Quadratic Risk . . . 154 Statistics to Approximate Limit Risk . . . . . . 159 Statistics to Approximate the Extremum Solution . . . . . . . . . . . . . . . . . . . . . . 162 CONTENTS ix 4.3. Multiparametric Sample Linear Regression. . . . . 167 Functionals of Random Gram Matrices . . . . . 171 Functionals in the Regression Problem . . . . . 181 Minimization of Quadratic Risk . . . . . . . . . 186 Special Cases. . . . . . . . . . . . . . . . . . . . 190 Chapter 5. Multiparametric Discriminant Analysis 193 5.1. Discriminant Analysis of Independent Variables . . 195 A Priori Weighting of Variables . . . . . . . . . 197 Empirical Weighting of Variables . . . . . . . . 200 Minimum Error Probability for Empirical Weighting . . . . . . . . . . . . . . . . . . . . . 203 Statistics to Estimate Probabilities of Errors . . 207 Contribution of a Small Number of Variables . . 209 Selection of Variables by Threshold . . . . . . . 211 5.2. Discriminant Analysis of Dependent Variables . . . 220 Asymptotical Setting . . . . . . . . . . . . . . . 221 Moments of Generalized Discriminant Function 224 Limit Probabilities of Errors . . . . . . . . . . . 227 Best-in-the-Limit Discriminant Procedure . . . . 231 The Extension to a Wide Class of Distributions 234 Estimating the Error Probability . . . . . . . . 236 Chapter 6. Theory of Solution to High-Order Systems of Empirical Linear Algebraic Equations 239 6.1. The Best Bayes Solution . . . . . . . . . . . . . . . 240 6.2. Asymptotically Unimprovable Solution . . . . . . . 246 Spectral Functions of Large Gram Matrices . . . 248 Limit Spectral Functions of Gram Matrices . . . 251 Quadratic Risk of Pseudosolutions . . . . . . . . 254 Minimization of the Limit Risk . . . . . . . . . 258 Shrinkage-Ridge Pseudosolution . . . . . . . . . 262 Proofs for Section 6.2 . . . . . . . . . . . . . . . 266