Table Of ContentECONOMETRICS OF INFORMATION AND EFFICIENCY
THEORY AND DECISION LIBRARY
General Editors: W. Leinfellner (Vienna) and G. Eberlein (Munich)
Series A: Philosophy and Methodology of the Social Sciences
Series B: Mathematical and Statistical Methods
Series C: Game Theory, Mathematical Programming and Operations Research
Series 0: System Theory, Knowledge Engineering and Problem Solving
SERIES B: MATHEMATICAL AND STATISTICAL METHODS
VOLUME 25
Editor: H. J. Skala (Paderborn); Assistant Editor: M. Kraft (Paderborn); Editorial Board:
J. Aczel (Waterloo, Ont.), G. Bamberg (Augsburg), H. Drygas (Kassel), W. Eichhorn
(Karlsruhe), P. Fishburn (Murray Hill, NJ.), D. Fraser (Toronto), W. Janko (Vienna), P. de
Jong (Vancouver), T. Kariya (Tokyo), M. Machina (La Jolla, Calif.), A. Rapoport (Toronto),
M. Richter (Kaiserslautern), B. K. Sinha (Cattonsville, Md.), D. A. Sprott (Waterloo, Ont.),
P. Suppes (Stanford, Calif.), H. Theil (Gainesville, Fla.), E. Trillas (Madrid), L. A. Zadeh
(Berkeley, Calif.).
Scope: The series focuses on the application of methods and ideas of logic, mathematics and
statistics to the social sciences. In particular, formal treatment of social phenomena, the
analysis of decision making, information theory and problems of inference will be central
themes of this part of the library. Besides theoretical results, empirical investigations and the
testing of theoretical models of real world problems will be subjects of interest. In addition
to emphasizing interdisciplinary communication, the series will seek to support the rapid
dissemination of recent results.
The titles published in this series are listed at the end of this volume.
ECONOMETRICS OF
INFORMATION AND
EFFICIENCY
by
JATI K. SENGUPTA
Professor of Economics and Operations Research,
University of California
Springer-Science+Business Media, B.V.
Library of Congress Cataloging-in-Publication Data
Sengupta, J.
Econometrics of information and efficiency / Jati K. Sengupta.
p. cm. -- (Theory and decision library. Series B,
Mathematical and statistical methods; v. 25)
Includes bibl iographical references (p. ) and index.
1. Information theory in economics--Econometric models.
2. Entropy (Information theory)--Econometric models. 3. Efficiency,
Industrlal--Econometric models. I. Title. 11. Series.
HB133.S4S 1993
338'.OS'015195--dc20 93-4584
ISBN 978-90-481-4288-0 ISBN 978-94-015-8202-5 (eBook)
DOl 10.1007/978-94-015-8202-5
Printed on acid-free paper
All Rights Reserved
© 1993 Springer Science+Business Media Dordrecht
Originally published by K1uwer Academic Publishers in 1993.
Softcover reprint of the hardcover 1s t edition 1993
No part of the material protected by this copyright notice may be reproduced or
utilized in any form or by any means, electronic or mechanical,
including photocopying, recording or by any information storage and
retrieval system, without written permission from the copyright owner.
To
Mother
CONTENTS
Preface ix
1 ThITRODUCTIONTOThWORMATION 1
1.1 Infonnation and Data Analysis 2
1.2 Infonnation and Estimation 4
1.3 Information Theory and Choice of Models 6
1.4 Economics of Information 7
1.5 Information and Efficiency 9
1.6 Outline of the Book 11
2 APPLIED ThWORMA TION TIIEORY 13
2.1 Tools of Information Theory 13
A. Conditional entropy 15
B. Mutual information 15
C. Divergence measure 17
D. Multivariate distance 18
2.2 Maximum Entropy Principle 21
2.3 Infonnation Theory Approach to Estimation 25
2.4 Mutual Information and Prediction 31
2.5 U se of Entropy in Economic Models 35
A. Production function models 35
B. Transition probability models 44
C. Structural change and economic growth 56
2.6 Applications to Stochastic Processes 63
3 ThWORMA TION TI-IEORY IN ECONOMETRlCS 71
3.1 Data and Information 71
3.2 Nonparametric Estimation and Entropy Theory 75
3.3 Entropy. Likelihood and Bayesian Estimation 79
3.4 Information Gain and Divergence 82
3.5 Applications in Production Frontier Estimation 94
3.6 Entropy-based Measure of Economic Inequality 102
4 APPLICATIONS IN CONTROL TIIEORY AND OPERATIONS
RESEARCH 106
4.1 Dynamic Models and Information Theory 106
4.2 Entropy Minimizing Control 116
A. A model of LEQG 117
B. Risk sensitive DEA models 120
4.3 Entropy and Game Theory 130
4.4 Information Theory and Risk Analysis 135
A. Entropy in SLP models 137
B. Entropy in DEA models 144
4.5 Entropy in Models of Queueing and Transportation 146
vii
TABLE OF CONTENTS (Continued)
4.6 Efficiency Measurement Under Inexact Information 150
A. hnprecision in DEA models 151
B. Fuzzy regression in DEA models 153
C. Fuzzy models in games 157
5 ECONOMIC THEORIES OF INFORMATION 160
5.1 Information in Selection Models 160
A. Selection samples 161
B. Truncation models in market returns 163
5.2 Informational Basis of Rational Expectations 171
A. Estimation of dynamic efficiency 180
B. A model of asymmetric information 185
5.3 Information and Game Theory 188
5.4 Semiparametric Estimation of Efficiency 195
A. Estimation of dynamic frontier 195
B. Comparing efficiency distributions 207
5.5 Market Models of Search 218
5.6 Decisions Under Incomplete Information 222
A. Quadratic decision model 223
B. Risk-sensitive production frontier 225
6 ECONOMETIUCSANDENTROPYTHEORY 230
6.1 Problems in Econometric Modelling 232
A. Stochastic complexity analysis 234
B. Model selection criteria 235
6.2 Trends in Entropy Theory 237
A. Optimal search and entropy 241
B. Duality in information theory 242
REFERENCES 246
INDEX 254
viii
Preface
Econometrics as an applied discipline is intended to use information in a most
efficient manner. Yet the information theory and the entropy approach developed by
Shannon and others have not played much of a role in applied econometrics. This volume
is intended to bridge the gap.
Broadly viewed the information theory analyzes the uncertainty of a given set of data
and its probabilistic characteristics. Whereas the economic theory of information
emphasizes the value of information to agents in a market, the entropy theory stresses the
various aspects of imprecision of data and their interactions with the SUbjective decision
processes. The tools of information theory such as the maximum entropy principle, mutual
information and the minimum discrepancy are useful in several areas of statistical inference
e.g., Bayesian estimation, expected maximum likelihood principle, the fuzzy statistical
regression. This volume analyzes the applications of these tools of information theory to the
most commonly used models in econometrics.
One model which has been consistently used as a benchmark is the model for
estimating productive efficiency originally proposed by Farrell, who applied a data-based
nonparametric method of estimating a production frontier. Various tools of information
theory are applied here to show its great potentiality.
The interface with the economic theories of information has also been discussed in
terms of the following models in particular: (a) market models of search, (b) informational
role of market prices under rational expectations, (c) informational basis of two-person
game theory models and (e) the optimal decision models under various types of imprecision
of information.
This volume includes a large part of my research work during the past five years and
I am particularly indebted to my student Ed Dumas for his many innovative discussions on
entropy.
Finally, I deeply appreciate the loving support of my wife and two children. This
work would never have been completed without their constant support and encouragement.
Iati K. Sengupta
Professor of Economics and
Operations Research
University of California
Santa Barbara, California
ix
1
CHAPIER 1
Introduction to Information
Information is central to all applied studies in economics and other sciences. It has many
facets. As empirical data it provides the basis for testing an economic model or theory. It
is also intimately connected with decision making under conditions of risk and uncertainty.
Hence the choice of optimal policy under an uncertain environment depends on the type of
information structures e.g., is it partial or total, incomplete or complete and imprecise or
precise?
In communication theory in engineering the central problem is to analyze the process
of information transmission through a noisy channel. A channel is the link between the
source which sends a certain message coded before transmission and the destination where
the message is decoded. In the case of telegram, the channel is a wire, while in the case of
a message sent from a spacecraft, the channel is the whole universe. Due to the presence of
noise, which represents any kind of distorting influence which is random in its effect, the
information passing through a channel gets randomly distorted or modified. The theory of
information transmission in noisy channels seeks to analyze the implications of different
statistical laws applying to the information source and the probabilities of the different types
of distortion introduced by the channel.
The economics of information looks at the demand for and value of both public and
private information, as it affects the behavior of the agents in the market. Thus at the
microlevel the economics of information analyzes the implications of asymmetric
information structures e.g., the seller may have complete information on the product it
sells, while the buyers may have incomplete information, since the search process is costly.
At the macrolevel one may analyze e.g., the concept of informational efficiency of the
capital market. This raises such questions as: (a) To what extent a securities market is
informationally efficient in the sense of its prices fully reflecting all available information?
(b) What is the role of the market information signals in the formation and change of the
equilibrium price vector in a market where the traders are rational economic agents in a
competitive framework? and (c) What would be the optimal decision rules for the rational
investors, when the returns from different groups of securities fluctuate over time? Clearly
these issues require that we analyze the informational basis of the market price and returns
data.
In statistics and econometrics information theory plays a basic role. In the parametric
estimation theory, one usually applies the maximum likelihood (ML) principle to a set of
mutually independent samples to estimate the population parameters. However this
assumes that the samples are all drawn from a specific distribution e.g., normal or gamma.
If no such assumption about a specific form of distribution is made, then the ML principle
2
cannot be applied. Infonnation theory based on a measure of infonnation known as
entropy can be applied here so as to derive a best approximation to the unknown
distribution. Again, in Bayesian methods of estimation one could measure infonnation
gains, when the prior infonnation is modified by the sequence of observed samples into the
posterior infonnation structure, once again measuring infonnation by the concept of
entropy. Furthennore, the infonnation-theoretic measures of distance between alternative
distributions, also called divergence statistics have been employed to discriminate between
two or more hypotheses. Recently, nonparametric methods of estimation which are data
based and not dependent on any specific fonn of the underlying distribution are
increasingly applied in econometric studies. Entropy-based infonnation theory is ideally
suitable for this framework, since it is based on the frequencies or probability densities,
which can be easily estimated by the histogram or kernel estimates of sample proportions.
Thus the use of information in different facets is basic to applied quantitative models
and we would concentrate on the econometric models including the models of operations
research, where the discipline of operations research is viewed as the econometrics of the
enterprise.
1. 1 Information and data analysis
Empirical data used in econometrics are very often sample observations reflecting the
behavior of agents. To make predictions based on these samples is one of the major tasks
of the econometric models. Two types of models are fonnulated in the usual econometric
approach. One is structural modeling, where economic theory is utilized in order to
develop the specification of the equations to be used for forecasting purposes. The other
type is a purely forecasting model, very often applied in time series models known as
ARIMA (autoregressive integrated moving average) models where time andlor the lagged
values of the variables to be forecast are used as predictors.
Besides this predictive purpose sample data are used for nonnative and simulation
purposes. The latter framework is most frequently applied in engineering and other
physical sciences, where a small scale prototype model is studied, sometimes under
controlled experiments to gain more insight into the large scale model which is more
appropriate in real life. The nonnative framework arises in econometric models, whenever
the agents are assumed to be rational and their aggregate behavior is reflected in the
observed data. A typical example is the estimation of a production frontier, rather than a
production function and the underlying data on inputs and output are provided by a cross
section of finns, not all of which are efficient.
To describe the empirical data and to use it in both prescriptive and normative models
is the major task of the infonnational approach to econometrics. While the standard
econometrics concentrates only on the statistical estimation aspect, the infonnation theory
approach, also called the entropy approach due to its emphasis on the concept of entropy
