Data Fusion for Sensory Information Processing Systems

257 Pages·1990·4.992 MB·English

by  James J. Clark, Alan L. Yuille (auth.)

DATA FUSION FOR SENSORY
INFORMATION PROCESSING SYSTEMS
THE KLUWER INTERNATIONAL SERIES
IN ENGINEERING AND COMPUTER SCIENCE
ROBOTICS: VISION, MANIPULATION AND SENSORS
Consulting Editor
Takeo Kanade
Other books in the series:
ROBOTIC GRASPING AND FINE MANIPULATION, M. Cutkosky
ISBN: 0-89838-200-9
SHADOWS AND SILHOUETfES IN COMPUTER VISION, S. Shafer
ISBN: 0-89838-167-3
PERCEPlUAL ORGANIZATION AND VISUAL RECOGNITION, D. Lowe
ISBN: 0-89838-172-X
ROBOT DYNAMICS ALGORTIHMS, F. Featherstone
ISBN: 0-89838-230-0
TIlREE-DIMENSIONAL MACHINE VISION, T. Kanade (editor)
ISBN: 0-89838-188-6
KINEMATIC MODEliNG, IDENTIFICATION AND CONTROL OF
ROBOT MANIPULATORS, H.W. Stone
ISBN: 0-89838-237-8
OBJECT RECOGNITION USING VISION AND TOUCH, P. Allen
ISBN: 0-89838-245-9
INTEGRATION, COORDINATION AND CONTROL OF MULTI-SENSOR
ROBOT SYSTEMS, H.F. Durrant-Whyte
ISBN: 0-89838-247-5
MOTION UNDERSTANDING: Robot and Human Vision, W.N. Martin
and J. K. Aggrawal (editors)
IsBN: 0-89838-258-0
BAYESIAN MODELING OF UNCERTAINTY IN LOW-LEVEL VISION,
R.Szeliski
ISBN 0-7923-9039-3
VISION AND NAVIGATION: TIlE CMU NAV LAB, C. Thorpe (editor)
ISBN 0-7923-9068-7
TASK-DIRECI'ED SENSOR FUSION AND PLANNING: A Computational
Approach, G. D. Hager
ISBN: 0-7923-9108-X
COMPUTER ANALYSIS OF VISUAL TEXTURES, F. Tomita and S. Tsuji
ISBN: 0-7923-9114-4
DATA FUSION FOR SENSORY
INFORMATION PROCESSING SYSTEMS
by
James J. Clark
Alan L. Yuille
Division of Applied Sciences
Harvard University
Cambridge, Massachusetts
~.
"
SPRINGER SCIENCE+BUSINESS MEDIA, LLC
Library of Congress Cataloging-in-Publication Data
Clark, James Joseph, 1957-
Data fusion for sensory information processing systems / James J.
Clark, Alan L. Yuille.
p. cm. - Kluwer international series in engineering and
computer science ; SECS 105)
Includes bibliographical references (p. ) and index.
ISBN 978-1-4419-5126-7 ISBN 978-1-4757-2076-1 (eBook)
DOI 10.1007/978-1-4757-2076-1
1. Computer vision. 2. Image processing. I. Yuille, A. L. (Alan
L.) II. Title. III. Series.
TA1632.C58 1990
006.3'7-dc20 90-4770
CIP
Copyright © 1990 by Springer Science+Business Media New York
Originally published by Kluwer Academic Publishers in 1990
Softcover reprint of the hardcover 1st edition 1990
All rights reserved. No part of this publication may be reproduced, stored in a retrieval
system or transmitted in any form or by any means, mechanical, photo-copying, recording,
or otherwise, without the prior written permission of the publisher,
Springer Science+Business Media, LLC.
Contents
Preface xiii
1 Introduction: The Role of Data Fusion In Sensory
Systems 1
1.1 INFORMATION ACQUISITION: INVERTING THE
WORLD-IMAGE MAPPING . . . . . . . . . . . . .. 1
1.2 THE NEED FOR CONSTRAINTS. . . . 5
1.3 DETERMINATION AND EMBEDDING OF
CONSTRAINTS . . . . . . . . . . . . . . . . . . . . 9
1.4 THE NEED FOR DATA FUSION .... 13
1.5 SUMMARy........................ 15
2 Bayesian Sensory Information Processing 17
2.1 BAYES RULE ............. 18
2.2 THE IMAGE FORMATION MODEL 19
v
vi
2.3 THE PRIORS ...................... 23
2.3.1 THE SYSTEM MODEL. . . . . . . . . . . .. 24
f .. . . . . . . ..
2.4 BAYESIAN ESTIMATORS FOR 25
2.5 BAYESIAN DETECTION AND EXTRACTION
SYSTEMS ......................... 28
2.6 THE BAYESIAN CONTROVERSY . . . . . . . . .. 30
2.7 CHAPTER S'UMMARY . . . . . . . . . . . . . . . .. 37
3 Information Processing U sing Energy Function
Minimization ' 39
3.1 MARKOV RANDOM FIELDS . . . . . . . . . . . .. 44
3.2 ENERGY FUNCTIONS WITH MATCHING
ELEMENTS ....................... 46
3.3 STATISTICAL MECHANICS AND MEAN FIELD
THEORY ......................... 50
3.3.1 W.T.A. WITH THE MEAN FIELD
APPROXIMATION . . . . . . . . . . . . . .. 51
3.3.2 W.T.A. WITHOUT THE MEAN FIELD
APPROXIMATION . . . . . . . . . . . . . .. 53
3.3.3 AVERAGING OUT FIELDS . . . . . . . . .. 54
3.4 THE FORM OF THE SMOOTHNESS CONSTRAINT 55
3.5 ALTERNATIVE FORMS OF CONSTRAINT. . . .. 57
VB
3.5.1 PARAMETRIC CONSTRAINTS ...... " 57
3.5.2 MINIMAL DESCRIPTION LENGTH CODING 64
3.5.3 MULTIPLE SETS OF PRIORS . 66
3.6 CHAPTER SUMMARY ......... . 68
4 Weakly vs. Strongly Coupled Data Fusion: A
Classification of Fusional Methods 71
4.1 A CLASSIFICATION 0F FUSIONAL METHODS. 72
4.2 WEAKLY COUPLED DATA FUSION ......... 73
4.2.1 CLASS I WEAKLY COUPLED DATA FUSION 73
4.2.2 CLASS II WEAKLY COUPLED DATA
FUSION ...................... 75
4.2.3 CLASS III WEAKLY COUPLED DATA
FUSION ............... ....... 76
4.3 STRONGLY COUPLED DATA FUSION
ALGORITHMS ............... ....... 78
4.3.1 STRONG COUPLING BY PRIOR CONSTRAINT
ADAPTION ................... 80
4.3.2 STRONG COUPLING BY ADAPTION OF
THE IMAGE FORMATION MODEL ..... 80
4.3.3 RECURRENT STRONG COUPLING ..... 81
4.3.4 COUPLED MRF METHODS AS STRONGLY
COUPLED DATA FUSION ........... 82
Vlll
4.4 BAYESIAN IMPLEMENTATION OF DATA FUSION 83
4.5 EXAMPLES OF WEAKLY COUPLED DATA
FUSION IN THE VISION LITERATURE. . . . . .. 87
4.6 EXAMPLES OF STRONGLY COUPLED FUSION
IN THE VISION LITERATURE . . . . . . . .. 91
4.7 SUMMARy............ · 103
5 Data Fusion Applied to Feature Based Stereo
Algorithms 105
5.1 INTRODUCTION .......... 105
5.2 THE BAYESIAN APPROACH TO STEREO VISION 108
5.2.1 THE MATCHING PROBLEM ......... 108
5.2.2 THE FIRST LEVEL: MATCHING FIELD AND
DISPARITY FIELD ............... 109
5.2.3 THE SECOND LEVEL: ADDING
DISCONTINUITY FIELDS . · 112
5.2.4 THE THIRD LEVEL: ADDING INTENSITY
TERMS ...................... 113
5.2.5 THE BAYESIAN FORMULATION OF THE
STEREO ALGORITHM ............ 114
5.3 STATISTICAL MECHANICS AND MEAN FIELD
THEORY. . . . . . . . . . . . . . . . ...... 116
5.3.1 AVERAGING OUT FIELDS · 117
ix
5.3.2 DETERMINISTIC SOLUTIONS OF THE MEAN
FIELD EQUATIONS .............. 125
5.4 COMPARISONS WITH OTHER THEORIES ..... 128
5.4.1 THE MARR-POGGIO COOPERATIVE
STEREO ALGORITHM ............ 128
5.4.2 DISPARITY GRADIENT LIMIT THEORIES 130
5.5 COMPARISONS WITH PSYCHOPHYSICAL DATA 131
5.6 CHAPTER SUMMARY . . . . . . . . . . . . . . . . . 134
6 Fusing Binocular and Monocular Depth Cues 137
6.1 STRONG FUSION STEREO WITH
MONOCULAR CUES .................. 137
6.2 PREVIOUS ATTEMPTS AT STRONG COUPLING
FOR STEREO ...................... 138
6.2.1 A GENERAL FRAMEWORK ......... 142
6.2.2 SOFT AND HARD CONSTRAINTS ..... 144
6.3 SUMMARy ........................ 146
7 Data Fusion in Shape From Shading Algorithms 147
7.1 AN ALGEBRAIC APPROACH TO FUSING
SPECULAR AND LAMBERTIAN REFLECTANCE
DATA ........................... 148
x
7.2 A CLASS III WEAKLY COUPLED FUSION
IMPLEMENTATION .................. 158
7.3 A STRONGLY COUPLED APPROACH TO
POLYCHROMATIC SHAPE FROM SHADING ... 168
7.4 FUSION OF IMAGE FORMATION MODELS .... 172
7.5 CHAPTER SUMMARY . . . . . . . . . . . . . . . . . 179
8 Temporal Aspects of Data Fusion 181
8.1 A TEMPORAL COHERENCE EDGE DETECTOR. 182
8.1.1 DETERMINATION OF THE CONDITIONAL
DENSITIES .................... 185
8.1.2 BAYESIAN EDGE DETECTION DECISION
PROCESS .................... 189
8.2 A STRONGLY COUPLED TEMPORAL COHER-
ENCE EDGE DETECTOR ............... 197
8.3 TEMPORAL SAMPLING ................ 201
8.3.1 COMPUTATIONAL CONSTRAINTS ..... 204
8.4 ACTIVE DETERMINATION OF CONSTRAINTS . 206
8.5 SUMMARy........................ 215