Table Of Content

Springer Monographs in Mathematics Malempati M. Rao Stochastic Processes – Inference Theory Second Edition Springer Monographs in Mathematics More information about this series at http://www.springer.com/series/3733 Malempati M. Rao Stochastic Processes – Inference Theory Second Edition Malempati M. Rao Department of Mathematics University of California Riverside, CA, USA ISSN 1439-7382 ISSN 2196-9922 (electronic) ISBN 978-3-319-12171-0 ISBN 978-3-319-12172-7 (eBook) DOI 10.1007/978-3-319-12172-7 Springer Cham Heidelberg New York Dordrecht London Library of Congress Control Number: 2014953897 Mathematics Subject Classification (2010): 60Gxx, 60H05, 60H30, 60J25, 62J02, 62MXX 1 st Edition: © Kluwer Academic Publishers 2000 2nd Edition: © Springer International Publishing Switzerland 2014 This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. Exempted from this legal reservation are brief excerpts in connection with reviews or scholarly analysis or material supplied specifically for the purpose of being entered and executed on a computer system, for exclusive use by the purchaser of the work. Duplication of this publication or parts thereof is permitted only under the provisions of the Copyright Law of the Publishers’s location, in its current version, and permission for use must always be obtained from Springer. Permissions for use may be obtained through RightsLink at the Copyright Clearance Center. Violations are liable to prosecution under the process respective Copyright Law. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. While the advice and information in this book are believed to be true and accurate at the date of publication, neither the authors nor the editors nor the publisher can accept any legal responsibility for any errors or omissions that may be made. The publisher makes no warranty, express or implied, with respect to the material contained herein. Printed on acid-free paper Springer is part of Springer Science+Business Media (www.springer.com) To Professors Ulf Grenander and Tom S. Pitcher whose fundamental and deep contributions shaped stochastic inference Preface to the Second Edition This new edition of ‘Stochastic Inference’ preserves the treatment of the orignal format as detailed in preface of this book, which is ap- pended, but contains the following new material. After making the few typographical corrections, noted earlier, the main new addition, given in reasonable detail, is the topic of ‘Regression analysis for processes’. Scanning the literature of this very popular and important subject whichwasgivenaformaldefinition,perhapsforthefirsttime,abstract- ingfromseveralexamples, byH.Cramérinhiswell-knownandoriginal volume Mathematical Methods of Statistics (1946). It states that the regression of a random variable (or vector) Y, on another such variable or vector X, when their joint probability distribution is known and Y has one moment, as the conditional expectation E(Y|X) = gY(X). It is somewhat surprizing that the foundations of the subject for random processes (or random measures) have not been treated (as far as I could find in extensive search) in the literature, although ‘Regression Analysies’ for numerous problems were treated, mostly for Gaussian processes. However the problem is deeper and is considered in this edition. To begin with the subject, one has to consider the difficulty (and even nonavailability) of a standard procedure for obtaining the gY(X) obtained above. After noting the nontriviality of the computing problem, detailed in my earlier book, “Conditional Measures and Applica- tions” (second edition, 2006), it is perhaps appropriate that I should discuss the foundations of this subject. Consequently Chapter VI of this volume is expanded to include this topic in some reasonable detail. TheproblemoflinearityofRegressionistreatedinthisgenerality. The problem of linearity of gY(·), first discussed by M. Kanter in this gen- erality, identifying it as an aspect of symmetric stable class (a key part of infinitely divisible family), which thus includes the Gaussian case which is so important for many applications. An aspect of the problem was considered by E. Lukacs and R. G. Laha (1964) to characterizing some processes such as Gamma, Poisson and the like, devoting a modest chapter in their book. Most of the books on the subject assume that g (X) is linear and obtain computational procedures in related Y evaluations of constants in the model. In this new edition, I have expanded Chapter VI, with three addi- tional sections on Regression and some new applications. These are devoted to characterizing the linearity of regression function gY(X), as vii viii Preface to the Second Edition a part of the (symmetric) stable processes (with extended treatment by C. D. Hacken) as well as my extensions for random measures and integrals. Alsospecializationsandapplicationsoflinearregressionwith (linear) constraints and related problems are discussed. The work as presented, along with included Problems Section, shows their use in some key econometric applications – well-treated by J. S. Chipman (2011) leading to further analysis to ’ridge regression’ in the subject. After this Chapters VII-IX retain the treatment as before, centering onfiltering,predictionandnonparametricestimation-allinthecontext of processes. These include harmonizable classes. In all the cases, the subject is illustrated by examples from different processes. As noted before, I still have some computer problems, and I hope that these are not distracting the reader. As noted in the earlier Preface, I had also been in correspondence with Tom Pitcher since his Pacific Journal paper (1965) which was extended considerably by Robert Rosenberg (1970) in his Carnegie- Mellon thesis. The correspondence continued even when I moved to UCR,bythenhemovedtoU.ofHawaii,andwenevermetalthoughour correspondencecontinued. Ihopedtopresentacopyofthisbookwitha surprize dedication along with Grenander, but I learned he has passed away by then. His continued encouraging correspondence and deep mathematical contributions are signified in my inclusive dedication of this volume. The new edition of this work is again done at UCR with departmen- tal help, especially the techincal computer assistance by James Mar- berry. I hope that the new edition with added material will further stimulate workers in the subject and help expand the area of Stochas- tic Inference. Riverside, CA M. M. Rao August, 2014 Preface to the First Edition The material accumulated and presented in this volume can be ex- plained easily. At the start of my graduate studies in the early 1950s, I came across Grenander’s (1950) thesis, and was much attracted to the entire subject considered there. I then began preparing for the necessary mathematics to appreciate and possibly make some contributions to the area. Thus after a decade of learning and some publications on the way, I wanted to write a modest monograph complementing Grenander’s fundamental memoir. So I took a sabbatical leave from my teaching position at the Carnegie-Mellon University, encouraged by an Air Force Grant for the purpose, and followed by a couple of years more learning opportunity at the Institute for Advanced Study to complete the project. As I progressed, the plan grew larger needing asubstantialbackgroundmaterialwhichwasmadeintoanindependent initial volume in (1979). In its preface I said: “My intention was to presentthefollowingmaterialasthefirstpartofabooktreatingtheIn- ference Theory of stochastic processes, but the latter account has now receded to a distant future,” namely for two more decades! Meanwhile, amuchenlargedsecondeditionofthatearlyworkhasappeared(1995), andnowIamabletopresentthemainpartoftheoriginalplan. Infact, while this effort took on the form of a life’s project, and developing all the necessary backup material during the long gestation period, I have written some seven books and directed several theses on related topics that helped me appreciate the main subject much better. It is now termed ‘stochastic inference’ as an abbreviation as well as a homage to Grenander’s “Stochastic processes and statistical inference”. Let me explain the method adapted in preparing this work. At the outset, it became clear that there can be no compromise with the mathematics of inference theory. One observes that, broadly speak- ing, inference has theoretical, practical, philosophical, and interpreta- tive aspects. But these components are also present in other scientific studies. However, for inference theory all these parts are founded on soundmathematicalprinciples,aviolationofwhichleadstounintended controversies. Thus the primary concern here is on mathematical ram- ifications of the subject, and the work is illustrated with a number of important examples, many of independent interest. It is noted that, as a basis of the classical statistical inference, two original sources are visible. The crucial idea on hypothesis testing is founded in the formulation of the Neyman-Pearson lemma which itself has a firm backing of the calculus of variations. All later developments ix