Visual Tracking and Data Fusion for Automatic Video Surveillance

133 Pages·2009·30.85 MB·English

SAPIENZA Universita` di Roma
Dottorato di Ricerca in Ingegneria Informatica
XXII Ciclo
Visual Tracking and Data Fusion for Automatic
Video Surveillance
Domenico Daniele Bloisi
Dipartimento di Informatica e Sistemistica
Sapienza Universit`a di Roma
September 2009

SAPIENZA Universita` di Roma
Dottorato di Ricerca in Ingegneria Informatica
XXII Ciclo
Visual Tracking and Data Fusion for Automatic
Video Surveillance
Domenico Daniele Bloisi
Thesis Commitee Reviewers
Prof. Luca Iocchi (Advisor) Prof. Rita Cucchiara
Prof. Andrea Vitaletti Prof. Erik Granum
Copyright (cid:13)c 2009
by Domenico Daniele Bloisi
AUTHOR’S ADDRESS:
Domenico Daniele Bloisi
Dipartimento di Informatica e Sistemistica “Antonio Ruberti”
Sapienza Universit`a di Roma
Via Ariosto 25, 00185 Roma, Italy.
E-MAIL: [email protected]
WWW: http://www.dis.uniroma1.it/∼bloisi
Contents
1 Introduction 1
1.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.2 Outline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.3 Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.4 Publications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.5 Collaborations . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
I Automatic Video Surveillance 5
2 Video Surveillance Systems 7
2.1 General Scheme . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
2.1.1 Object Detection . . . . . . . . . . . . . . . . . . . . . . . 9
2.1.2 Tracking Object of Interest . . . . . . . . . . . . . . . . . 10
2.1.3 Data Fusion . . . . . . . . . . . . . . . . . . . . . . . . . . 11
2.1.4 Event Understanding. . . . . . . . . . . . . . . . . . . . . 12
2.2 System Classification . . . . . . . . . . . . . . . . . . . . . . . . . 13
2.2.1 Background Nature . . . . . . . . . . . . . . . . . . . . . 13
2.2.2 Number of Objects to Track. . . . . . . . . . . . . . . . . 14
2.2.3 Size of the Monitored Area . . . . . . . . . . . . . . . . . 14
2.2.4 Evaluation Method . . . . . . . . . . . . . . . . . . . . . . 15
2.3 Ideal System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
2.4 Examples of Real Systems . . . . . . . . . . . . . . . . . . . . . . 17
2.4.1 ARGOS . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
2.4.2 PBKU . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
2.4.3 SAMURAI . . . . . . . . . . . . . . . . . . . . . . . . . . 19
3 Systems Overview 21
3.1 People Tracking Systems . . . . . . . . . . . . . . . . . . . . . . . 21
3.1.1 Single Camera . . . . . . . . . . . . . . . . . . . . . . . . 21
3.1.2 Stereo Camera . . . . . . . . . . . . . . . . . . . . . . . . 22
3.1.3 Multiple Cameras . . . . . . . . . . . . . . . . . . . . . . 22
3.1.4 Classification . . . . . . . . . . . . . . . . . . . . . . . . . 23
3.2 Traffic Monitoring Systems . . . . . . . . . . . . . . . . . . . . . 23
3.3 Video-surveillance Systems on Water . . . . . . . . . . . . . . . . 24
3.4 Analysis of the Literature . . . . . . . . . . . . . . . . . . . . . . 26
i
II Methods Overview 27
4 Background Modelling 29
4.1 Background Subtraction . . . . . . . . . . . . . . . . . . . . . . . 29
4.2 Buffered Based Methods . . . . . . . . . . . . . . . . . . . . . . . 30
4.2.1 Median Filter . . . . . . . . . . . . . . . . . . . . . . . . . 31
4.2.2 Minimum Maximum Filter . . . . . . . . . . . . . . . . . 31
4.2.3 Mediod Filter . . . . . . . . . . . . . . . . . . . . . . . . . 31
4.2.4 Eigenbackgrounds . . . . . . . . . . . . . . . . . . . . . . 31
4.3 Non-Parametric Methods . . . . . . . . . . . . . . . . . . . . . . 32
4.3.1 Kernel Density Estimation . . . . . . . . . . . . . . . . . 32
4.3.2 CONDENSATION . . . . . . . . . . . . . . . . . . . . . . 33
4.4 Predictive Methods . . . . . . . . . . . . . . . . . . . . . . . . . . 33
4.4.1 Kalman Filter. . . . . . . . . . . . . . . . . . . . . . . . . 33
4.4.2 Wiener Filter . . . . . . . . . . . . . . . . . . . . . . . . . 33
4.4.3 Hidden Markov Models . . . . . . . . . . . . . . . . . . . 34
4.5 Recursive Methods . . . . . . . . . . . . . . . . . . . . . . . . . . 34
4.5.1 Approximated Median Filter . . . . . . . . . . . . . . . . 35
4.5.2 Single Gaussian (Running Gaussian Average) . . . . . . . 35
4.5.3 Mixture of Gaussians. . . . . . . . . . . . . . . . . . . . . 35
4.5.4 Adaptive Mixture of Gaussians . . . . . . . . . . . . . . . 36
4.6 Speed and Memory Requirements . . . . . . . . . . . . . . . . . . 37
5 Tracking 39
5.1 Bayesian Tracking . . . . . . . . . . . . . . . . . . . . . . . . . . 39
5.2 The Kalman Filter . . . . . . . . . . . . . . . . . . . . . . . . . . 41
5.3 Particle Filter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
5.3.1 HJS Filter . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
6 Data Fusion 47
6.1 Multiple Camera Data Fusion . . . . . . . . . . . . . . . . . . . . 47
6.2 Camera Installation . . . . . . . . . . . . . . . . . . . . . . . . . 47
6.3 Camera Rectification . . . . . . . . . . . . . . . . . . . . . . . . . 48
6.4 Overlapping Views . . . . . . . . . . . . . . . . . . . . . . . . . . 49
6.5 Non-overlapping Views . . . . . . . . . . . . . . . . . . . . . . . . 49
6.6 Fusion Techniques for Combining Soft and Hard Data . . . . . . 50
III Visual Tracking and Data Fusion 53
7 Independent Multimodal Background Subtraction 55
7.1 Background Modelling Issues . . . . . . . . . . . . . . . . . . . . 56
7.2 A Fast and Robust Approach to Background Modelling . . . . . 56
7.3 Shadow Suppression . . . . . . . . . . . . . . . . . . . . . . . . . 58
7.4 Noise Removal and Blob Processing . . . . . . . . . . . . . . . . 61
ii
8 Image Segmentation 63
8.1 Image Segmentation for Water Scenario . . . . . . . . . . . . . . 63
8.1.1 Optical flow analysis . . . . . . . . . . . . . . . . . . . . . 64
8.1.2 Clustering through Rek-means . . . . . . . . . . . . . . . 66
8.2 Image Segmentation for Crowded Environments . . . . . . . . . . 69
8.2.1 The Height Image Algorithm . . . . . . . . . . . . . . . . 71
8.3 Image Segmentation for Calibrated Cameras. . . . . . . . . . . . 72
8.3.1 Number of People Estimation . . . . . . . . . . . . . . . . 75
9 Tracking 78
9.1 Multi-hypothesis Kalman Filter Tracking . . . . . . . . . . . . . 78
9.1.1 Data Association . . . . . . . . . . . . . . . . . . . . . . . 79
9.1.2 Track management . . . . . . . . . . . . . . . . . . . . . . 79
10 Fusion Strategies 82
10.1 System Architecture . . . . . . . . . . . . . . . . . . . . . . . . . 82
10.2 Information from User . . . . . . . . . . . . . . . . . . . . . . . . 83
10.3 Data Fusion Layer . . . . . . . . . . . . . . . . . . . . . . . . . . 85
10.4 Implementation and Example . . . . . . . . . . . . . . . . . . . . 88
IV Experimental Evaluation 92
11 Experimental Results 94
11.1 ARGOS: Application and User Interface . . . . . . . . . . . . . . 94
11.1.1 Track selection for the end user . . . . . . . . . . . . . . . 95
11.1.2 Statistics and Event Detection . . . . . . . . . . . . . . . 96
11.2 ARGOS: Experimental Evaluation . . . . . . . . . . . . . . . . . 97
11.3 Rek-means: Experimental Evaluation. . . . . . . . . . . . . . . . 101
11.4 IMBM: Experimental Evaluation . . . . . . . . . . . . . . . . . . 103
11.5 PBKU: Experimental Evaluation . . . . . . . . . . . . . . . . . . 105
11.6 NPE: Experimental Evaluation . . . . . . . . . . . . . . . . . . . 106
11.6.1 Segmentation Results . . . . . . . . . . . . . . . . . . . . 106
12 Conclusions 108
12.1 Future Directions . . . . . . . . . . . . . . . . . . . . . . . . . . . 109
iii

Acknowledgements
Iwishtothankthemanypeoplewhohavecollaboratedwithmeandcontributed
to various parts of this research. Special thanks go to my advisor Prof. Luca
Iocchiforhishelpandconstantsupportthroughoutthisthreeyears. Hegaveme
the chance to face a challenging research topic and to apply my research to real
and pioneer systems. I appreciate his vast knowledge and skill in many areas.
I would like to thank Dr. Paolo Remagnino for his support and suggestions
during my visiting scholarship in London. I would also like to thank Riccardo
Leone, Archimedes Logica, and the City Council of Venice for their support of
this research. I would like to express my gratitude to my family for supporting
andencouragingmeduringthisexcitingadventure. ManythankstoAndreafor
hissincerefriendship. ThebiggestthankgoestoErika,forbeingtheluckofmy
life.
v