Advances in Information Systems Science

365 Pages·1970·14.633 MB·English

by  V. A. Kovalevsky (auth.), Julius T. Tou (eds.)

Advances in
Information
Systems Science
Volume 3
Contributors
1. K. Hawkins Robot Research Company
La 10Ua, Calijornia
v.
A. Kovalevsky Institute oj Cybernetics
The Ukrainian Academy oj Sciences
Kiev, USSR
Saburo Muroga Department oj Computer Science
The University oj Illinois
Urbana, Illinois
Richard 1. Pankhurst The University Mathematical Laboratory
Cambridge, England
Satosi Watanabe Department oj Physics
The University oj Hawaii
Honolulu, Hawaii
Volume 3
Advances in
Information
Systems Science
Edited by
Julius T. Tou
Center for Informatics Research
University of Florida
Gainesville, Florida
PLENUM PRESS· NEW YORK-LONDON ·1970
Library of Congress Catalog Card Number 69-12544
ISBN-13: 978-1-4615-8245-8 e-ISBN-13: 978-1-4615-8243-4
DOl: 10.1007/978-1-4615-8243-4
© 1970 Plenum Press, New York
Softcover reprint oft he hardcover 1st edition 1970
A Division of Plenum Publishing Corporation
227 West 17th Street, New York, N. Y.100ll
United Kingdom edition published by Plenum Press, London
A Division of Plenum Publishing Company, Ltd.
Donington House, 30 Norfolk Street, London W.C. 2, England
All rights reserved
No part of this publication may be reproduced in any form
without written permission from the Dublisher
Articles Planned for Future Volumes
C. Gordon Bell An Introduction to the
and Michael Gold (USA) Structure of Time-Sharing Computers
L. Bolliet(France) Compiler Writing Techniques and Systems
R. M. Graham (USA) Basic Techniques
in System Programming
M. A. Harrison (USA) Relations between Grammars
and Automata
James L. Massey Error Correcting Codes
and O. N. Garcia (USA) in Computer Arithmetic
V. K. Smirnov (USSR) A ugmentation of Computer
Internal Language Level
Robert Tabory (USA) Theoretical Foundations and
Techniques of Automatic Syntax Analysis
Peter Wegner (USA) Data Structures in
Programming Languages
Preface
Information systems science embraces a broad spectrum of topics. It is vir
tually impossible to provide comprehensive and in-depth discussion, other
than simple recitals of recent results, of every important topic in each volume
of this annual review series. Since we have chosen the former approach,
each volume will only cover certain aspects of recent advances in this bur
geoning field. The emphasis in this volume, the third of a continuing series,
is focussed upon pattern recognition, pictorial information manipulation,
and new approaches to logical design of information networks.
In Chapter 1, V. A. Kovalevsky presents a tutorial survey of practical
and theoretical developments in pattern recognition. He categorizes the
basic developments in three different directions. The first direction is charac
terized by an empirical treatment with highly specialized recognition
schemes. In the second direction, the major efforts are centered upon the cre
ation of learning systems capable of improving recognition performance on
the basis of past experience. The majority of the work in the third direction
is devoted to the study of the basic structure of complex patterns, the con
struction of mathematical models for pattern recognition, and the analysis
of complex pictorial representations. The author elucidates the "heuristics"
approach and the "science" approach to pattern recognition problems. This
chapter together with Chapter 2 of this volume supplements the chapter on
Engineering Principles of Pattern Recognition in Volume 1 to provide a
more complete treatment of this subject.
Feature extraction and compression in pattern recognition was given a
cursory discussion in Volume 1. Now, we have a whole chapter devoted
entirely to this subject matter. S. Watanabe classifies the pattern recognition
problems into mechanical sorting, pattern recognition, generative sorting,
and clustering. In Chapter 2, the author explains the concept of features so
as to make it useful in solving pattern recognition problems and discusses
some basic methodologies for the purpose of extracting features from a spec
ified set of data. The concept of feature extraction is illustrated by a con
crete example from the post office work.
A picture is worth more than a thousand words. Pictures have been
recognized as important information communication media. Processing and
vii
viii Preface
manipulation of pictorial information are among the key issues in the design
of sophisticated information systems. In Chapter 3, J. K. Hawkins reviews
the principles and techniques for image processing. The author discusses
image processing in its most general sense, covering image enhancement
operations, data compression for bandwidth reduction, and image classifi
cation processes. This chapter is concerned not only with the theory and
techniques of image processing but also with methods of implementation
and useful devices which are of great value to practicing engineers. This
material adds another dimension to the pattern recognition work discussed
in this series.
Interactive information processing has been recognized as the key to the
understanding and synthesis of sophisticated information systems. The es
sence in the development of interactive information processing is the achieve
ment of effective man-computer interaction. In view of the fact that graph
ics makes possible a high degree of interaction, Chapter 4 is devoted to
computer graphics. R. J. Pankhurst introduces the reader to this subject.
He begins with a survey of the interactive graphic devices and systems, be
fore discussing data structures, programming languages, and software,
which are fundamental for the study of computer graphics.
The design of information networks often makes of composite gates
of the TTL and MOS integrated circuitry, requires the diagnosis of a
faulty network and the correction of a certain number of errors, and is
subject to the fan-in and fan-out restrictions. However, conventional logical
design methods known in switching theory fail to be effective when the above
important engineering aspects are taken into consideration. On the other
hand, integer programming seems to offer a promising technique for solving
such problems. In Chapter 5, S. Muroga provides a comprehensive discus
sion of the integer programming approach to the logical design of optimal
digital networks subject to important practical restrictions and engineering
requirements.
The contributors to this volume come from the Soviet Union, England,
Japan, and the United States. The editor wishes to express heartfelt thanks
for their cooperation and for the timely completion of their manuscripts.
In fact, many more contributed to the book than those whose names appear
in the contents. Much credit is due to the editor's colleagues for their in
valuable advice and to invited reviewers for their constructive criticsm.
Julius T. Tou
Gainesville, Florida
April 1970
Contents
Chapter 1 Pattern Recognition: Heuristics or Science?
V. A. Kovalevsky
1. Introduction. . . . . . . . . . . . . . 1
2. Principal Directions in Pattern Recognition 3
2.1. Basic Concepts. . . . . . . . 3
2.2. Heuristic Recognition Methods. . . 7
2.3. Perceptrons . . . . . . . . . . . 10
2.4. Learning As Approximation to a Decision Function 12
2.5. The Method of Stochastic Approximation . .' . . . 15
2.6. Methods Based on Assumptions About the Properties of
the Observed Signals . 18
2.7. Applied Results. . . . . . . . . . . . 21
3. Parametric Models of Signals . . . . . . . . 21
3.1. Distributions with Interfering Parameters 24
3.2. The Problem of Recognition of Complex Signals 25
3.3. The Statistical Problems of Supervised and Nonsupervised
Learning . . . . . . . . . . . . . . . 28
3.4. Parametric Models with Reference Patterns . 30
4. The Method of Permissible Transformations 35
4.1. Formalization of the Concept of Resemblance 35
4.2. Permissible Transformations . . . . . 36
4.3. Correlation Method. . . . . . . . . 38
4.4. Effectiveness of the Correlation Method 41
5. Methods of Analyzing Complex Pictures . . 42
5.1. Formal Syntactic Rules for Constructing Complex Pictures 43
5.2. Description of Complex Pictures in the Presence of Noise
(the Method of Reference Sequences) . . . . . . . .. 48
5.3. Examples of the Use of the Reference-Sequences Method 51
6. Conclusions. 56
References . 58
ix
x Contents
Chapter 2 Feature Compression
Satosi Watanabe
1. The Role of "Features" in Pattern Recognition 63
1.1. Four Kinds of Pattern Recognition and Featu'res . 63
1.2. Component and Composition-Structure Analysis 65
1.3. Pattern Recognition As Induction 67
1.4. Decision Procedure and Features 70
1.5. Selection of Variables . . . . . . 73
1.6. Distance and Feature . . . . . . 75
2. A Concrete Example of Feature Compression-Handwritten
ZIP Code Reader . . . . . . . 76
2.1. Nature of the Problem. . . 76
2.2. Compression of Invariants . 77
2.3. Local Features . . . . . 79
2.4. Horizontal Zone Feature. . 81
2.5. Global Features . . . . . 82
2.6. Feature Compression As Structural Analysis 83
3. Discriminatory Feature Compression-SELFIC 85
3.1. Rotations in Representation Space 85
3.2. Minimum-Entropy Principle . . . . . . 89
3.3. Basic Theorem of SELFIC. . . . . . . 93
3.4. Discriminatory Feature Space and SELFIC 96
3.5. Object-Predicate Reciprocity. . . . . . 99
4. Characteristic Feature Compression-CLAFIC 100
4.1. Class-Feature Space. . . . . . . . . . 100
4.2. Subspace Model Versus Zone Model . . 101
4.3. Decision Procedures by Projection and by Entropy 103
5. Implications of Subspace Model-Fuzzy Class. . 104
5.1. Modular Nondistributive Predicate Lattice. 104
5.2. Implications of the New Logic 107
5.3. Fuzzy Class 109
References . . . . . . . . . . . 11 0
Chapter 3 Image Processing Principles and Techniques
J. K. Hawkins
1. Introduction. . . . . . 113
1.1. Central Problems. 114
Contents xi
1.2. Processing for Data Compression. 116
1.3. Processing for Enhancement 117
104. Processing for Classification 118
2. Filter Theory Applied to Images . 120
2.1. Spatial Frequency Filtering. 121
2.2. Matched Filtering. . . . . 128
3. Statistical Decision Theory . . . 133
3.1. Decision Theory Formalisms. 134
3.2. Special Cases. . . . . . . . 136
3.3. Commentary on Applications. 139
4. Adaptive Network Approaches 141
5. Image Features ...... . 146
5.1. Approximating Functions 147
5.2. Random Features. . 150
5.3. Feature Adaptation . 153
504. Shape Features. . . 155
5.5. Textural Features. . 158
5.6. Serially Derived Features 160
5.7. Picture Linguistics . 163
5.8. Distance Features. . . . 165
6. Implementations: Staging. . . 167
6.1. Realizable Decision Functions 167
6.2. Number of Stages. . . 171
7. Implementations: Parallelism . . . 174
7.1. All-Serial Methods ..... 176
7.2. Parallel Operator, Serial Image Processing 177
7.3. Serial Operator, Parallel Image Processing 178
704. All-Parallel Methods . . . . 179
8. Electro optical Devices . . . . . . 180
8.1. Point and Aperture Scanners. 181
8.2. Image Parallel Devices. . . 187
9. Digital Computers. . . . . . . 189
9.1. The Fast Fourier Transform 189
9.2. Parallel Computers 191
10. Optical Techniques. . . 195
10.1. Coherent Optics . 196
10.2. Incoherent Optics. 200