Learning and Coordination: Enhancing Agent Performance through Distributed Decision Making

193 Pages·1994·6.268 MB·English

by  Steven H. Kim (auth.)

Learning and Coordination
International Series on
MICROPROCESSOR-BASED AND
INTELLIGENT SYSTEMS ENGINEERING
VOLUME 13
Editor
Professor S. G. Tzafestas, National Technical University, Athens, Greece
Editorial Advisory Board
Professor C. S. Chen, University of Akron, Ohio, US.A.
Professor T. Fokuda, Nagoya University, Japan
Professor F. Harashima, University of Tokyo, Tokyo, Japan
Professor G. Schmidt, Technical University of Munich, Germany
Professor N. K. Sinha, McMaster University, Hamilton, Ontario, Canada
Professor D. Tabak, George Mason University, Fairfax, Virginia, US.A.
Professor K. Valavanis, University of Southern Louisiana, Lafayette, U S.A.
Learning and
Coordination
Enhancing Agent Performance through
Distributed Decision Making
by
STEVEN H. KIM
Lightwelllnc.,
Charlottesville, Virginia, U.SA
SPRINGER-SCIENCE+BUSINESS MEDIA, B.V.
Library of Congress Cataloging-in-Publication Data
Kim. Steven H.
Learning and coordination : enhancing agent performance through
distributed decision making / by Steven H. Kim.
p. cm. -- (International series on microprocessor-based and
intell igent systems engineering ; v. 13)
Inc 1u des index.
ISBN 978-94-010-4442-4 ISBN 978-94-011-1016-7 (eBook)
DOI 10.1007/978-94-011-1016-7
1. Neural networks (Computer science) 2. Machine learning.
3. Artificial intelligence. 1. Title. II. Series.
OA76.87.K56 1994
003' .7--dc20 94-30344
ISBN 978-94-010-4442-4
Printed on acid-free paper
AH Rights Reserved
© 1994 Springer Science+Business Media Dordrecht
OriginaHy published by Kluwer Academic Publishers in 1994
Softcover reprint of the hardcover 1s t edition 1994
No part ofthe material protected by this copyright notice may be reproduced or
utilized in any form or by any means, electronic or mechanical,
including photocopying, recording or by any information storage and
retrieval system, without written permission from the copyright owner.
To the memory of my grandfather
YongJooKim
Contents
Preface .............................................................................................................................. xi
Chapter 1 Introduction and Framework ............................................................... 1
Introduction .................................................................................................................. 1
General Framework for Learning Systems .................................................................. 2
Application to Production-Rule Systems ..................................................................... 5
Structure ...................................................................................................... 5
Operation .................................................................................................... 6
Interpretation for Classifier Systems .......................................................... 8
Application to Neural Networks ................................................................................. 10
Explicit vs. Implicit Representation ........................................................................... 11
Software Structure ...................................................................................................... 11
illustrative Applications ............................................................................................. 12
Mobile Robot ............................................................................................. 12
Production Process Control ...................................................................... 13
Discussion .................................................................................................................. 14
Scope of the Book ...................................................................................................... 16
References .................................................................................................................. 17
Chapter 2 Learning Speed in Neural Networks .................................................. 21
Introduction ................................................................................................................ 21
Neural Nets ................................................................................................................. 22
Learning Rate as a Performance Metric ................................................... 23
Types of Learning ...................................................................................... 23
Activation Rules ........................................................................................ 24
Output Rules .............................................................................................. 26
Examples ofA ctivation Rules .................................................................... 27
Multiplicative Links ................................................................................... 32
Weight Training ........................................................................................ 34
Training Procedures ................................................................................. 39
Deterministic, Closed-form Training ........................................................ 41
Open-form Training .................................................................................. 45
Probabilistic Algorithms ........................................................................... 47
Pattern Associator ..................................................................................... 54
Graceful Degradation ............................................................................... 54
Biological Systems ..................................................................................................... 56
Vision System ............................................................................................ 56
Anatomy of Vision ..................................................................................... 56
viii Contents
Architecture of Vision ............................................................................... 57
Implications for Artificial Systems ............................................................ 59
A Hybrid Organization for Artificial Networks ........................................ 60
Efficiency in Weights ................................................................................. 61
Effect on Computational Speed ................................................................. 63
Efficiency in Operation ............................................................................. 63
Conclusion .................................................................................................................. 64
References .................................................................................................................. 64
Chapter 3 Principles of Coordination .................................................................. 69
Introduction ................................................................................................................ 69
Limitations of Centralized Control ............................................................................. 70
Decentralization based on Explicit Valuation ............................................................ 71
Games and Strategies ................................................................................................. 72
Deadlock among Independent Systems ...................................................................... 76
Concurrent Design ...................................................................................................... 77
Cooperative Systems .................................................................................................. 78
Characteristic Functions '" .......................................................................................... 79
Imputations ................................................................................................................. 81
Essential and Inessential Games ................................................................................. 82
Strategic Equivalence ................................................................................................. 84
Zero Game .................................................................................................................. 87
Binary Game ............................................................................................................... 88
Geometric Interpretation " .......................................................................................... 90
Games with Few Players ............................................................................................ 91
Dominance of Imputations ......................................................................................... 95
Core of a Game ......................................................................................................... 104
Examples of Cores .................................................................................................... 108
Solution Principles ................................................................................................... 110
Assumptions of the Shapley Solution ...................................................................... 113
Properties of Shapley Vector .................................................................................... 116
Application to Supervisor and Workers ................................................................... 127
Learning systems and Nested Agents ....................................................................... 128
Conclusion and discussion ....................................................................................... 129
References ................................................................................................................ 131
Chapter 4 Case Study in Coordination .............................................................. 135
Introduction .............................................................................................................. 135
Analytical models ..................................................................................................... 136
Map of the Environment .......................................................................... 137
Time to Respond to an Event ................................................................... 142
Simulation Model ..................................................................................................... 147
Simulation Specifics ................................................................................................. 153
Deployment Rules .................................................................................................... 154
Conclusion ................................................................................................................ 164
References ................................................................................................................ 164
Chapter 5 Conclusion .......................................................................................... 167
Summary .................................................................................................................. 167
Implications .............................................................................................................. 169
Contents ix
Appendix Dynamic Models in Statistical Physics ............................................. 171
Emergence ................................................................................................................ 171
Statistical Mechanics ................................................................................................ 172
Dual State System .................................................................................... 172
Multi-state System ................................................................................... 173
Modal Configuration ............................................................................... 175
Temperature Parameter .......................................................................... 177
Use of the Exponential Distribution ......................................................................... 180
References ................................................................................................................ 182
Index .............................................................................................................................. 185
Preface
Learning is a hallmark of intelligence. As such, the subject fascinates us all,
whether we are participants at a technical conference, students of intelligent systems, or
guests at a cocktail party.
The psychologist might define learning as a relatively permanent change in
behavior as a result of experience. But this perspective ignores the positive factor
implicit in the term learning. In everyday parlance, we associate learning with an
improvement in performance rather than neutral or even degraded behavior. In
operational terms, then, we may define learning as the enhancement ofpeiformance over
time.
When a system is a product of human fabrication rather than the natural
environment, its performance is often specified in terms of a set of functional
requirements. These requirements indicate the level of performance expected of the
system. In an uncertain world subject to stochastic disturbances, the fulfillment of
functional requirements is a probabilistic rather than deterministic affair. In fact all
systems-whether of the learning variety or not-operate in an uncertain environment.
But there is a second aspect of uncertainty: that of the system itself. The
components of the system, both in terms of hardware and software, may be subject to
stochastic influences.
This book examines a number of models which exhibit learning behavior,
ranging from the human brain to artificial systems. The stochastic nature and learning
behavior of intelligent systems are explored.
Intelligent systems must often interact in the attainment of their goals. For such
systems, the ability to coordinate their activities prudently may prove to be more
instrumental in fulfilling objectives than their portfolio of individual capabilities. As
with individual strategies, the formulation of effective policies for coordination relies as
well on probabilistic arguments. The resulting models and tools can then be used as a
basis for improving collective performance with experience.
Given the universality of the subjects - namely, learning behavior and
coordinative strategies in uncertain environments - this book should be of interest to
students and researchers in diverse disciplines. These fields range from all areas of
xii Preface
engineering to the computing disciplines; from the life sciences to the physical sciences;
and frpm the management arts to societal studies.
I would like to thank Elias Towe and Mark Novick for reviewing the manuscript.
Much of the word processing and proofreading were ably performed by Tasha Seitz.
The thoughtful comments from these contributors were invaluable in improving the
readability of the book.
S.K.