Uncertainty And Information: Foundations of Generalized Information Theory

518 Pages·2005·9.14 MB·English

by  George J. Klir

TEAM LinG
UNCERTAINTY AND
INFORMATION

UNCERTAINTY AND
INFORMATION
Foundations of Generalized
Information Theory
George J. Klir
Binghamton University—SUNY
A JOHN WILEY & SONS, INC., PUBLICATION
Copyright © 2006 by John Wiley & Sons,Inc.All rights reserved
Published by John Wiley & Sons,Inc.,Hoboken,New Jersey
Published simultaneously in Canada
No part of this publication may be reproduced,stored in a retrieval system,or transmitted in
any form or by any means,electronic,mechanical,photocopying,recording,scanning,or other-
wise,except as permitted under Section 107 or 108 of the 1976 United States Copyright Act,
without either the prior written permission of the Publisher,or authorization through payment
of the appropriate per-copy fee to the Copyright Clearance Center,Inc.,222 Rosewood Drive,
Danvers,MA 01923,[phone],fax [phone],or on the web at www.copyright.com.
Requests to the Publisher for permission should be addressed to the Permissions Department,
John Wiley & Sons,Inc.,111 River Street,Hoboken,NJ 07030,[phone],fax (201)
748-6008,or online at http://www.wiley.com/go/permission.
Limit of Liability/Disclaimer of Warranty:While the publisher and author have used their best
efforts in preparing this book,they make no representations or warranties with respect to the
accuracy or completeness of the contents of this book and specifically disclaim any implied
warranties of merchantability or fitness for a particular purpose.No warranty may be created
or extended by sales representatives or written sales materials.The advice and strategies con-
tained herein may not be suitable for your situation.You should consult with a professional
where appropriate.Neither the publisher nor author shall be liable for any loss of profit or any
other commercial damages,including but not limited to special,incidental,consequential,or
other damages.
For general information on our other products and services or for technical support,please
contact our Customer Care Department within the United States at [phone],outside the
United States at [phone] or fax [phone].
Wiley also publishes its books in a variety of electronic formats.Some content that appears in
print may not be available in electronic formats.For more information about Wiley products,
visit our web site at www.wiley.com.
Library of Congress Cataloging-in-Publication Data:
Klir,George J.,1932–
Uncertainty and information :foundations of generalized information theory / George J.Klir.
p. cm.
Includes bibliographical references and indexes.
ISBN-13:978-0-471-74867-0
ISBN-10:0-471-74867-6
1. Uncertainty (Information theory) 2. Fuzzy systems. I. Title.
Q375.K55 2005
033¢.54—dc22
2005047792
Printed in the United States of America
10 9 8 7 6 5 4 3 2 1
A book is never finished.
It is only abandoned.
—Honoré De Balzac

CONTENTS
Preface xiii
Acknowledgments xvii
1 Introduction 1
1.1. Uncertainty and Its Significance / 1
1.2. Uncertainty-Based Information / 6
1.3. Generalized Information Theory / 7
1.4. Relevant Terminology and Notation / 10
1.5. An Outline of the Book / 20
Notes / 22
Exercises / 23
2 Classical Possibility-Based Uncertainty Theory 26
2.1. Possibility and Necessity Functions / 26
2.2. Hartley Measure of Uncertainty for Finite Sets / 27
2.2.1. Simple Derivation of the Hartley Measure / 28
2.2.2. Uniqueness of the Hartley Measure / 29
2.2.3. Basic Properties of the Hartley Measure / 31
2.2.4. Examples / 35
2.3. Hartley-Like Measure of Uncertainty for Infinite Sets / 45
2.3.1. Definition / 45
2.3.2. Required Properties / 46
2.3.3. Examples / 52
Notes / 56
Exercises / 57
3 Classical Probability-Based Uncertainty Theory 61
3.1. Probability Functions / 61
3.1.1. Functions on Finite Sets / 62
vii
viii CONTENTS
3.1.2. Functions on Infinite Sets / 64
3.1.3. Bayes’ Theorem / 66
3.2. Shannon Measure of Uncertainty for Finite Sets / 67
3.2.1. Simple Derivation of the Shannon Entropy / 69
3.2.2. Uniqueness of the Shannon Entropy / 71
3.2.3. Basic Properties of the Shannon Entropy / 77
3.2.4. Examples / 83
3.3. Shannon-Like Measure of Uncertainty for Infinite Sets / 91
Notes / 95
Exercises / 97
4 Generalized Measures and Imprecise Probabilities 101
4.1. Monotone Measures / 101
4.2. Choquet Capacities / 106
4.2.1. Möbius Representation / 107
4.3. Imprecise Probabilities:General Principles / 110
4.3.1. Lower and Upper Probabilities / 112
4.3.2. Alternating Choquet Capacities / 115
4.3.3. Interaction Representation / 116
4.3.4. Möbius Representation / 119
4.3.5. Joint and Marginal Imprecise Probabilities / 121
4.3.6. Conditional Imprecise Probabilities / 122
4.3.7. Noninteraction of Imprecise Probabilities / 123
4.4. Arguments for Imprecise Probabilities / 129
4.5. Choquet Integral / 133
4.6. Unifying Features of Imprecise Probabilities / 135
Notes / 137
Exercises / 139
5 Special Theories of Imprecise Probabilities 143
5.1. An Overview / 143
5.2. Graded Possibilities / 144
5.2.1. Möbius Representation / 149
5.2.2. Ordering of Possibility Profiles / 151
5.2.3. Joint and Marginal Possibilities / 153
5.2.4. Conditional Possibilities / 155
5.2.5. Possibilities on Infinite Sets / 158
5.2.6. Some Interpretations of Graded Possibilities / 160
5.3. Sugeno l-Measures / 160
5.3.1. Möbius Representation / 165
5.4. Belief and Plausibility Measures / 166
5.4.1. Joint and Marginal Bodies of Evidence / 169
CONTENTS ix
5.4.2. Rules of Combination / 170
5.4.3. Special Classes of Bodies of Evidence / 174
5.5. Reachable Interval-Valued Probability Distributions / 178
5.5.1. Joint and Marginal Interval-Valued Probability
Distributions / 183
5.6. Other Types of Monotone Measures / 185
Notes / 186
Exercises / 190
6 Measures of Uncertainty and Information 196
6.1. General Discussion / 196
6.2. Generalized Hartley Measure for Graded Possibilities / 198
6.2.1. Joint and Marginal U-Uncertainties / 201
6.2.2. Conditional U-Uncertainty / 203
6.2.3. Axiomatic Requirements for the U-Uncertainty / 205
6.2.4. U-Uncertainty for Infinite Sets / 206
6.3. Generalized Hartley Measure in Dempster–Shafer
Theory / 209
6.3.1. Joint and Marginal Generalized Hartley Measures / 209
6.3.2. Monotonicity of the Generalized Hartley Measure / 211
6.3.3. Conditional Generalized Hartley Measures / 213
6.4. Generalized Hartley Measure for Convex Sets of Probability
Distributions / 214
6.5. Generalized Shannon Measure in Dempster-Shafer
Theory / 216
6.6. Aggregate Uncertainty in Dempster–Shafer Theory / 226
6.6.1. General Algorithm for Computing the Aggregate
Uncertainty / 230
6.6.2. Computing the Aggregated Uncertainty in Possibility
Theory / 232
6.7. Aggregate Uncertainty for Convex Sets of Probability
Distributions / 234
6.8. Disaggregated Total Uncertainty / 238
6.9. Generalized Shannon Entropy / 241
6.10. Alternative View of Disaggregated Total Uncertainty / 248
6.11. Unifying Features of Uncertainty Measures / 253
Notes / 253
Exercises / 255
7 Fuzzy Set Theory 260
7.1. An Overview / 260
7.2. Basic Concepts of Standard Fuzzy Sets / 262
7.3. Operations on Standard Fuzzy Sets / 266