Intelligent Multimedia Multi-Agent Systems: A Human-Centered Approach

333 Pages·2000·2.37 MB·English

by  Rajiv Khosla

INTELLIGENT
MULTIMEDIA
MULTI-AGENT
SYSTEMS
A Human-Centered Approach
THE KLUWER INTERNATIONAL SERIES
IN ENGINEERING AND COMPUTER SCIENCE
INTELLIGENT
MULTIMEDIA
MULTI-AGENT
SYSTEMS
A H uman-Centered Approach
Rajiv Khosla
Ishwar K. Sethi
Ernesto Damiani
Springer Science+Business Media, LLC
Library of Congress Cataloging-in-Publication
Khosla, Rajiv, 1959-
Intelligent multimedia multi-agent systems : a human-eentered approach I Rajiv
Khosla, Ishwar K. Sethi, Ernesto Damiani.
p. cm. --(The Kluwer international series in engineering and computer science ; SECS 582)
Includes bibliographical references and index.
ISBN 978-1-4419-5008-6 ISBN 978-1-4757-3196-5 (eBook)
DOI 10.1007/978-1-4757-3196-5
1. Multimediasystems. 2. Intelligent agents (Computer software) 3. Human-computer
interaction. I. Sethi, Ishwar K., 1948-II. Damiani, Ernesto. III. Title. IV. Series.
QA7 6.575 .K52 2000
006.7--dc21
00-135009
Copyright® 2000 by Springer Science+Business Media New Y ork
Originally published by Kluwer Academic Publishers in 2000
Softcover reprint of the bardeover 1st edition 2000
All rights reserved. No part ofthis publication may be reproduced, stored in a retrieval system or
transmitted in any form or by any means, mechanical, photo-copying, recording, or otherwise,
without the prior written permission of the publisher, Springer Science+Business Media, LLC.
Printed on acid-free paper.
TABLE OF CONTENTS
PREFACE .......................................................................................................> 0/
ACKNOWLEDGEMENTS ............................................................................ XVIII
PART 1: (Chapters 1-5)
Motivation, Technologies, Enabling Theories and Human Centered
Virtual Machine
1
SUCCESSFUL SYSTEMS OR SUCCESSFUL TECHNOLOG IE S? ........... 1
1.1. INTRODUCTION ................................................................................. 1
1.2. TECHNOLOGY-CENTEREDNESS VS HUMAN-CENTEREDNESS. 1
1.3. HUMAN-CENTERED APPROACH ..................................................... 5
1.4. SUMMARY .......................................................................................... 7
REFERENCES ................................................................................................. 7
2
TECHNOLOGIES ......................................................................................... 9
2.1 INTRODUCTION ................................................................................. 9
2.2. EXPERT SYSTEMS ............................................................................ 9
2.2.1. SYMBOLICKNOWLEDGEREPRESENTATION ...............................•................. 10
2.2.2. RULEBASEDARCHITECTURE .......•............................................................... 13
2.2.3. RULE AND FRAME (OBJECT) BASED ARCHITECTURE ................................... 13
2.2.4. MODEL BASED ARCHITECTURE .................................................................... 14
2.2.5. BLACKBOARD ARCHITECTURE ..................................................................... 15
2.2.6. SOME LIMITATIONS OF EXPERT SYSTEM ARCHITECTURES ........................... 15
vt Table of Contents
2.3. CASE BASED REASONING SYSTEMS ........................................... 16
2.4. ARTIFICIAL NEURAL NETWORKS .................................................. 16
2.4.1. PERCEPTRON .................................................................................................. 17
2.4.2. MULTILAY ER PERCEPTRONS .......................................................................... 20
2.4.3. RADIAL BASIS FUNCTION NET ....................................................................... 22
2.4.4. KOHONEN NETWORKS ................................................................................... 24
2.5. FUZZY SYSTEMS .............................................................................. 26
2.5 .1. Fuzzy SETS ................................................................................................... 26
2.5.2. FUZZIFICATION OF INPUTS ............................................................................. 27
2.5 .3. Fuzzy INFERENCING AND RULE EVALUATION .............................................. 28
2.5.4. DEFUZZIFICATIONOF OUTPUTS ..................................................................... 29
2.6. GENETIC ALGORITHMS .................................................................. 30
2.7. INTELLIGENT FUSION, TRANSFORMATION AND COMBINATION
............................................................................................................ 32
2.8. OBJECT-ORIENTED SOFTWARE ENGINEERING ......................... 33
2.8.1. INHERITANCE AND COMPOSABILITY .............................................................. 33
2.8.2. ENCAPSULATION ........................................................................................... 34
2.8.3. MESSAGEPASSING ........................................................................................ 34
2.8.4. POLYMORPHISM ............................................................................................. 34
2.9. AGENTSAND AGENT ARCHITECTURES ....................................... 34
2.1 0. MULTIMEDIA ..................................................................................... 36
2.11. THE EXTENSIBLE MARKUP LANGUAGE ....................................... 37
2.11.1. XML NAMESPACES .................................................................................... 42
2.11.2. XML-BASED AGENT SYSTEMS DEVELOPMENT .......................................... 43
2.12. SUMMARY ......................................................................................... 44
RE FE RENCES ................................................................................................ 45
3
PRAGMATIC CONSIDERATIONS AND ENABLING THEORIES ............. 51
3.1. INTRODUCTION ................................................................................ 51
3.2. PRAGMATIC CONSIDERATIONS FOR HUMAN-CENTERED
SOFTWARE DEVELOPMENT ........................................................... 51
3 .2.1. INTELLIGENT SYSTEMS AND HUMAN-CENTEREDNESS .................................. 52
3.2.2. SOFTWARE ENGINEERING AND HUMAN-CENTEREDNESS ............................... 55
3.2.3. MULTIMEDIA DATABASES AND HUMAN-CENTEREDNESS .............................. 57
Table of Contents vii
3.2.4. ELECTRONIC COMMERCE AND HUMAN-CENTEREDNESS ............................... 59
3.2.5. DATA MINING AND HUMAN-CENTEREDNESS ................................................ 61
3.2.6. ENTERPRISE MODELING AND HUMAN-CENTEREDNESS ................................. 61
3 .2. 7. HUMAN-COMPUTER INTERACTION AND HUMAN-CENTEREDNESS ................. 63
3.3. ENABLING THEORIES FOR HUMAN-CENTERED SYSTEMS ....... 63
3.3.1. SEMIOTIC THEORY-LANGUAGE OF SIGNS .................................................. 64
3.3.1.1. Rhematic Knowledge ............................................................................ 67
3.3.1.2. Dicent Knowledge ................................................................................. 68
3.3.1.3. Argumentative Knowledge .................................................................... 68
3.3.2. COGNITIVE SCIENCE THEORIES .................................................................... 69
3.3.2.1. Traditional Approach ............................................................................ 69
3.3.2.2. Radical Approach .................................................................................. 70
3.3.2.3. Situated Cognition ................................................................................ 71
3.3.2.4. Distributed Cognition ........................................................................... 73
3.3.3. ACTIVITY THEORY ....................................................................................... 75
3.3.4. WORKPLACE THEORY .................................................................................. 77
3.4. DISGUSSION .................................................................................... 78
3.5. SUMMARY ........................................................................................ 80
REFERENCES ............................................................................................... 81
4
HUMAN-CENTERED SYSTEM DEVELOPMENT FRAMEWORK ........... 87
4.1. INTRODUCTION ............................................................................... 87
4.2. OVERVIEW ....................................................................................... 87
4.3 EXTERNALAND INTERNAL PLANES OF HUMAN-CENTERED
FRAMEWORK ................................................................................... 88
4.4. COMPONENTS OF THE HUMAN-CENTERED SYSTEM
DEVELOPMENT FRAMEWORK ....................................................... 91
4.5. ACTIVITY-CENTERED ANALYSIS COMPONENT .......................... 92
4.5 .1. PROBLEM DEFINITION AND SCOPE ............................................................... 92
4.5.2. PERFORMANCE ANALYSIS OF SYSTEM COMPONENTS ................................... 94
4.5.3. CONTEXT ANALYSIS OF SYSTEM COMPONENTS ........................................... 95
4.5.3.1. WorkActivity Context ........................................................................... 95
4.5.3.2. Direct Stakeholder Context ................................................................... 95
4.5.3.3. Product Context .................................................................................... 96
4.5.3.4. Data Context ......................................................................................... 96
4.5.3.5. Tool Context ......................................................................................... 96
4.5.4. ALTERNATIVE SYSTEM GOALSAND TASKS ................................................... 97
Table of Contents
VIII
4.5.5. HUMAN-TASK-TOOL DIAGRAM ......•...•...•..........•........................................... 97
4.5.6. TASKPRODUCTTRANSITIONNETWORK ....................................................... 98
4.6. PROBLEM SOLVING ONTOLOGY COMPONENT ........................... 98
4.6.1. STRENGTHS AND WEAKNESSES OF EXISTING PROBLEM SOLVING
ÜNTOLOGIES .....••.....................••........•..•.......•.•••......•....•.........••.....................•......... 99
4.7. SUMMARY ....................................................................................... 102
REFERENCES: ............................................................................................. 10 3
5 HUMAN-CENTERED VIRTUAL MACHINE .............................................. 105
5.1. INTRODUCTION .............................................................................. 10 5
5.2. PROBLEM SOLVING ONTOLOGY COMPONENT ......................... 105
5.2.1. DEFINITION OF TERMS USED ....................................................................... 106
5.2.2. PROBLEM SOLVING ADAPTERS .................................................................... 109
5.2.2.1. Preprocessing Phase Adapter: ............................................................ 109
5.2.2.2. Decomposition Phase Adapter ............................................................ lll
5.2.2.3. Control Phase Adapter ........................................................................ ll4
5.2.2.4. Decision Phase Adapter ...................................................................... ll8
5.2.2.5. Postprocessing Phase Adapter ............................................................ 122
5.3. HUMAN-CENTERED CRITERIA AND PROBLEM SOLVING
ONTOLOGY ..................................................................................... 124
5.4. TRANSFORMATION AGENT COMPONENT ................................. 124
5.5. MULTIMEDIA INTERPRETATION COMPONENT .......................... 127
5.5.1. DATA CONTENT ANALYSIS ......................................................................... 128
5.5.2. MEDIA, MEDIA EXPRESSION AND ÜRNAMENTATION SELECTION ................ 129
5.5.3. MEDIA PRESENTATION DESIGN AND COORDINATION ................................. 132
5.6. EMERGENT CHARACTERISTICS OF HCVM ................................ 132
5.6.1. ARCHITECTURAL CHARACTERISTICS .......................................................... 132
5.6.1.1. Human-Centeredness .......................................................................... J32
5.6.1.2. Task Orientation vs Technology Orientation ....................................... 133
5. 6.1.2. Flexibility ............................................................................................. 133
5.6.1.3. Versatility ............................................................................................. J33
5.6.1.4. Forms ofKnowledge ........................................................................... l33
5.6.1.5. Learningand Adaptation ..................................................................... 133
5.6.1.6. Distributed Problem Solving and Communication- Collaboration and
Competition ....................................................................................................... 134
5. 6.1. 7. Component Based Software Design .................................................... 134
5.6.2. MANAGEMENT CHARACTERISTICS .............................................................. 134
Table of Contents ix
5.6.2.1. Cast, Deve/opment Time and Reuse: .................................................. 134
5.6.2.2. Sealability and Maintainability: .......................................................... 134
5.6.2.3. Intelligibility: ....................................................................................... 135
5.6.3. DOMAIN CHARACTERISTICS ....................................................................... 135
5.7. SUMMARY ...................................................................................... 135
REFERENCES: ............................................................................................ 136
PART II: (Chapters 6-8)
HCVM Applications in Health lnformatics, Face Detection,
Net Euchre and Sales Recruitment
6 INTELLIGENT MULTIMEDIA MUL Tl-AGENT CLINICAL DIAGNOSIS AND
TREATMENT SUPPORT SYSTEM ................................................ 137
6.1. INTRODUCTION ............................................................................. 137
6.2. ACTIVITY-CENTERED ANALYSIS- DRUG PRESCRIPTION
MONITORING ACTIVITY ................................................................ 137
6.2.1 PROBLEM DEFINITION AND SCOPE ............................................................... 138
6.2.2. PERFORMANCE AND CONTEXT ANALYSIS .................................................. 139
6.2.2.1. Performance Analysis ofDPMA ......................................................... 139
6.2.2.2. Context Analysis ofthe Drug Prescription Monitaring Activity ......... 143
6.2.3. ALTERNATESYSTEMGOALSANDTASKS ................................................... 147
6.2.4. HUMAN-TASK-TOOLDIAGRAM .................................................................. 147
6.2.5. TASK PRODUCT TRANSITION NETWORK ..................................................... 148
6.3. APPLICATION OF HCVM PROBLEM SOLVING ONTOLOGY IN
DPMA .............................................................................................. 150
6.3.1. HUMAN-CENTEREDDOMAINMODEL ......................................................... 150
6.3.1 .. 1 Mapping Decomposition Adapter to DPMA Tasks ............................. 150
6.3.1.2. Mapping Contra/ Phase Adapters to DPMA Tasks ............................ 152
6.3.1.3. Mapping Decision Phase Adapter to DPMA Tasks ............................ 154
6.3.1.4. Mapping Postprocessing Phase Adapter to DPMATasks ................... 156
6.4. INTELLIGENT DIAGNOSIS AND TREATMENT SUPPORT
PROBLEM SOL VING AGENTS ...................................................... 158
6.4.1. ETIOLOGY SELECTION DECOMPOSITION AGENT ......................................... 159
6.4.2. UPPER RESPIRAT ORY CONTROL AGENT ..................................................... 162
6.4.3. SUGGESTEDIPREDICTED DIAGNOSIS DECISION AGENT .............................. 164
6.4.5. DIAGNOSIS AND TREATMENT VALIDATION POSTPROCESSING AGENT ........ 168
6.5. DIAGNOSIS SUPPORT MULTIMEDIA INTERPRETATION
COMPONENT ................................................................................. 168
6.5.1. SYMPTOM CONTENT ANALYSIS ......................................................•........... 168
6.5.2 MEDIA, MEDIA EXPRESSION AND 0RNAMENTATION SELECTION ................. 171
Table of Contents
X
6.5.3. MULTIMEDIAAGENTS ............................................................ 172
6.6. SUMMARY ....................................................................................... 174
ACKNOWLEDGMENTS ................................................................................ 174
REFERENCES .............................................................................................. 174
7
A MULTI-AGENT SYSTEM FOR FACE DETECTION AND ANNOTATION
.......................................................................................................... 175
7 .1. I NTRODUCTION .............................................................................. 175
7.2. IMAGE ANNOTATION ..................................................................... 176
7.3. SOME EXISTING FACE DETECTION SYSTEMS ......................... 176
7.3.1. FACE DETECTION BY GRAY-LEVEL.. .......................................................... 176
7.3.2. FACE DETECTION BY COLDRAND MOTION ................................................ 178
7.4. TASK-PRODUCT NETWORK OFFACE DETECTION &
ANNOTATION ACTIVITY ................................................................ 178
7 .4.1. DETECTION OF SKIN-TONE REGION ............................................................ 179
7.4.2. CLEANING OF SKIN-TONE REGIONS .................................................... 180
7.4.3. LOCATION OF CANDIDAT E FACE REGIONS .................................................. 180
7.4.4. FACIAL FEATURE EXTRACTION, FACE DETECTION AND ANNOTATION ........ 181
7.5. HCVM PROBLEM SOLVING ONTOLOGY AND FACE DETECTION
AND ANNOTATION ACTIVITY (FDAA) .......................................... 181
7. 5.1. DECOMPOSITION PHASE ADAPTER DEFINITION IN FDAA ......................... 181
7.5.2. CONTROL PHASE ADAPTER DEFINITIONS IN FDAA ................................... 183
7.5 .3. DECISION PHASE ADAPTER DEFINITION IN FDAA ..................................... 185
7.6. FACE DETECTION AGENTSAND THEIR IMPLEMENTATION ......... 186
7 .6.1. COLOR IMAGE PREPROCESSING AND SKIN REGION DECOMPOSITION AGENT
.............................. ··················································· ............... 186
7 .6.2. FACE CANDIDAT E REGION CONTROL AGENT.. ........................................... 189
7. 6.2.1. Morphologica/ Operations for Cleaning Skin Tone Regions ............. 189
7.6.2.2. Filtering Cleaned Regions by Size and Shape .................................... 190
7.6.2.3. Improving the Detection Rate ............................................................. 192
7.6.3. FACE DETECTION ANDANNOTATION DECISION AGENT.. ........................... 193
7.7. EXPERIMENTAL RESULTS AND FUTURE WORK ...................... 195
7.8. SUMMARY ..................................................................................... 196
REFERENCES .............................................................................. 196