Handbook on Soft Computing for Video Surveillance

342 Pages·2012·5.36 MB·English

by  Maddalena

Computer Science H Chapman & Hall/CRC
a CRYPTOGRAPHY AND NETWORK SECURITY
n
d
b
Information on integrating soft computing techniques into video surveillance is o Handbook on
widely scattered among conference papers, journal articles, and books. Bringing o
k
this research together in one source, Handbook on Soft Computing for Video

o
Surveillance illustrates the application of soft computing techniques to different n Soft Computing for
tasks in video surveillance. Worldwide experts in the field present novel solutions
S
to video surveillance problems and discuss future trends. o
f
After an introduction to video surveillance systems and soft computing tools, t Video Surveillance
C
the book gives examples of neural network–based approaches for solving
o
video surveillance tasks and describes summarization techniques for content m
identification. Covering a broad spectrum of video surveillance topics, the p
u
remaining chapters explain how soft computing techniques are used to detect
t
moving objects, track objects, and classify and recognize target objects. The book in
also explores advanced surveillance systems under development. g

f
o
Features
r
• Describes soft computing tools useful in video surveillance, such as neural
V
networks, genetic algorithms, probabilistic reasoning, and the combination of i
d
fuzzy and rough sets e
o
• Includes an introduction to video surveillance systems for beginners

S
• Presents methods and algorithms for detecting moving objects in video
u
streams, tracking objects in video sequences, human action modeling and r
v
recognition from video sequences, automated video analysis, and detecting e
video shot boundaries il
l
• Provides examples of state-of-the-art surveillance systems, including a multi- a
n
camera, multi-robot system and a system using multiple audio and video c
sensors e

Incorporating both existing and new ideas, this handbook unifies the basic
concepts, theories, algorithms, and applications of soft computing. It
demonstrates why and how soft computing methodologies can be used in P
a
various video surveillance problems. M l
a •
d P Edited by
d e
a t
le ro Sankar K. Pal
n s
a i
n
o Alfredo Petrosino
K12673
Lucia Maddalena
K12673_Cover.indd 1 12/13/11 12:21 PM
Handbook on
Soft Computing for
Video Surveillance
K12673_FM.indd 1 12/14/11 11:35 AM
CHAPMAN & HALL/CRC
CRYPTOGRAPHY AND NETWORK SECURITY
Series Editor
Douglas R. Stinson
Published Titles

Jonathan Katz and Yehuda Lindell, Introduction to Modern
Cryptography
Antoine Joux, Algorithmic Cryptanalysis
M. Jason Hinek, Cryptanalysis of RSA and Its Variants
Burton Rosenberg, Handbook of Financial Cryptography and Security
Shiu-Kai Chin and Susan Older, Access Control, Security, and Trust:
A Logical Approach
Sankar K. Pal, Alfredo Petrosino, and Lucia Maddalena, Handbook on
Soft Computing for Video Surveillance
Forthcoming Titles

Maria Isabel Vasco, Spyros Magliveras, and Rainer Steinwandt,
Group Theoretic Cryptography
K12673_FM.indd 2 12/14/11 11:35 AM
Chapman & Hall/CRC
CRYPTOGRAPHY AND NETWORK SECURITY
Handbook on
Soft Computing for
Video Surveillance
Edited by
Sankar K. Pal
Indian Statistical Institute
Kolkata, India
Alfredo Petrosino
University of Naples Parthenope
Naples, Italy
Lucia Maddalena
National Research Council
Naples, Italy
K12673_FM.indd 3 12/14/11 11:35 AM
CRC Press
Taylor & Francis Group
6000 Broken Sound Parkway NW, Suite 300
Boca Raton, FL 33487-2742
© 2012 by Taylor & Francis Group, LLC
CRC Press is an imprint of Taylor & Francis Group, an Informa business
No claim to original U.S. Government works
Version Date: 20120104
International Standard Book Number-13: 978-1-4398-5685-7 (eBook - PDF)
This book contains information obtained from authentic and highly regarded sources. Reasonable efforts
have been made to publish reliable data and information, but the author and publisher cannot assume
responsibility for the validity of all materials or the consequences of their use. The authors and publishers
have attempted to trace the copyright holders of all material reproduced in this publication and apologize to
copyright holders if permission to publish in this form has not been obtained. If any copyright material has
not been acknowledged please write and let us know so we may rectify in any future reprint.
Except as permitted under U.S. Copyright Law, no part of this book may be reprinted, reproduced, transmit-
ted, or utilized in any form by any electronic, mechanical, or other means, now known or hereafter invented,
including photocopying, microfilming, and recording, or in any information storage or retrieval system,
without written permission from the publishers.
For permission to photocopy or use material electronically from this work, please access www.copyright.
com (http://www.copyright.com/) or contact the Copyright Clearance Center, Inc. (CCC), 222 Rosewood
Drive, Danvers, MA 01923, 978-750-8400. CCC is a not-for-profit organization that provides licenses and
registration for a variety of users. For organizations that have been granted a photocopy license by the CCC,
a separate system of payment has been arranged.
Trademark Notice: Product or corporate names may be trademarks or registered trademarks, and are used
only for identification and explanation without intent to infringe.
Visit the Taylor & Francis Web site at
http://www.taylorandfrancis.com
and the CRC Press Web site at
http://www.crcpress.com
Contents
Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii
About the Editors . . . . . . . . . . . . . . . . . . . . . . . . . . xi
List of Contributors . . . . . . . . . . . . . . . . . . . . . . . . . xiii
1 Introduction to Video Surveillance Systems Tomi D. R¨aty 1
2 TheRoleofSoftComputingin Image Analysis:Rough-Fuzzy
Approach AlessioFerone,SankarK.Pal,andAlfredoPetro-
sino . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
3 Neural Networks in Video Surveillance: A Perspective View
Lucia Maddalena and Alfredo Petrosino . . . . . . . . . . . . 59
4 Video Summarization and Significance of Content: A Review
Rajarshi Pal, Ashish Ghosh, and Sankar K. Pal . . . . . . . 79
5 Background Subtraction for Visual Surveillance: A Fuzzy
Approach Thierry Bouwmans . . . . . . . . . . . . . . . . . 103
6 Sensor and Data Fusion: Taxonomy, Challenges, and Appli-
cations LawrenceA.Klein,LyudmilaMihaylova,andNour-
Eddin El Faouzi . . . . . . . . . . . . . . . . . . . . . . . . . . 139
7 Independent Viewpoint Silhouette-Based Human Action
ModelingandRecognition CarlosOrrite,FranciscoMart´ınez-
Contreras, El´ıas Herrero, Hossein Ragheb, and Sergio A. Ve-
lastin . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185
8 Clustering for Multi-Perspective Video Analytics: A Soft
Computing-Based Approach Ayesha Choudhary, Santanu
Chaudhury, and Subhashis Banerjee . . . . . . . . . . . . . . 211
9 An Unsupervised Video Shot Boundary Detection Tech-
nique Using Fuzzy Entropy Estimation of Video Content
Biswanath Chakraborty, Siddhartha Bhattacharyya, and
Paramartha Dutta . . . . . . . . . . . . . . . . . . . . . . . . . 237
v
vi
10 Multi-Robot and Multi-Camera Patrolling Christopher
King, Maria Valera, Raphael Grech, Robert Mullen, Paolo
Remagnino, Luca Iocchi, Luca Marchetti, Daniele Nardi,
Dorothy Monekosso, and Mircea Nicolescu . . . . . . . . . . 255
11 A Network of Audio and Video Sensors for Monitoring Large
Environments Claudio Piciarelli, Sergio Canazza, Christian
Micheloni, and Gian Luca Foresti . . . . . . . . . . . . . . . . 287
Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 317
Preface
Video surveillance is the area of computer science devoted to real-time acqui-
sition, processing, and management of videos coming from cameras installed
in public and private areas, in order to automatically understand events hap-
peningatthemonitoredsites,eventuallysendingupanalarm.Becauseofthe
rapidly increasing number of surveillance cameras, it has become a key tech-
nologyforsecurityandsafety,withapplicationsrangingfromthefightagainst
terrorism and crime, to private and public safety (e.g., in private buildings,
transport networks, town centers, schools, and hospitals), and to the efficient
management of transport networks and public facilities (e.g., traffic lights
and railroad crossings). Video surveillance is an extremely interdisciplinary
area, embracing the study of methods and algorithms for computer vision
and pattern recognition, but also hardware for sensors and acquisition tools,
computer architectures, wired and wireless communication infrastructures,
andmiddleware.Fromanalgorithmicstandpoint,thegeneralproblemcanbe
broken down into several steps, including motion detection, object classifica-
tion,tracking,activityunderstanding,andsemanticdescription,eachofwhich
posesitsownchallengesandhurdlesforsystemdesigners.Moreover,thescope
of video surveillance is being extended to offline multimedia analysis systems
related to security and safety, thus entailing disciplines such as content-based
video retrieval for visualdata similarity retrieval and video mining for knowl-
edgeextraction;typicalapplicationsareinforensicvideoanalysisandhuman
behavior analysis.
Soft computing is a consortium of methodologies (working synergistically,
not competitively) that, in one form or another, reflects its guiding princi-
ple:exploitthetoleranceforimprecision,uncertainty,approximatereasoning,
and partial truth to achieve tractability, robustness, low-cost solution, and
close resemblance to human-like decision making. This provides flexible in-
formation processing capability for representation and evaluation of various
real-lifeambiguousanduncertainsituations,andthereforeresultsinthefoun-
dation for the conception and design of high MIQ (Machine IQ) systems. At
thisjuncture,theprincipalconstituentsofsoftcomputingarefuzzysets,neu-
rocomputing, genetic algorithms, probabilistic reasoning, and rough sets.
Recently,softcomputingtoolshaveshownenormouspromiseinmanydif-
ferent tasks of video surveillance. For example, fuzzy logic has proved to be
a powerful tool that allows one to handle the imprecision and uncertainty
inherent in the background subtraction approach to moving object detection,
as well as in the tracking process, and in visual sensor networks; approximate
reasoning has proved beneficial for action recognition; fuzzy-rough interpre-
tation of video data can help in dealing with the approximate, incomplete,
and vague characteristics of surveillance videos for the analysis of usual and
viii
unusualevents;neuralnetworksallowself-organizinglearningofasceneback-
groundmodelformovingobjectdetection,trackingofimagefeaturesinvideo
imagesequences,andlearningofusualandunusualhumanbehaviors.Several
conference papers and journal articles on integrating soft computing tech-
niques into video surveillance have been published in the past decade. Arti-
clesdescribingthechallengingissuesofsoftcomputing,namelysynergistically
integrating the constituting components to achieve application specific mer-
its, have also been reported. However, this scattering of information causes
inconvenience for researchers, applied scientists, and practitioners.
With this volume we aim to bring together research work concerning the
application of soft computing techniques to different tasks of video surveil-
lance, investigating novel solutions, and discussing future trends of existing
literature in this field. It includes both review and new material written by
worldwide experts describing, in a unified way, the basic concepts, theories,
algorithms, and applications that demonstrate why and how soft computing
methodologies can be used in different video surveillance problems.
Thebookconsistsofelevenchapters.Chapter1presentsanintroductionto
videosurveillancesystems,providinganicereferenceforbeginnersinthisarea.
It includes a fairly extensive and updated survey on such systems, tracing a
briefhistoryoftheirevolutionandtheactualstateoftheart.Italsohighlights
thechallengesthatmodern-daysystemsstillfacedespiteachievingsignificant
advancements, and covers several sub-topics concerning video sensors, data
fusion, and artificial intelligence techniques.
Fortheconvenienceofreaders,abriefdescriptionofdifferentsoftcomput-
ingtoolsisprovidedinChapter2,coveringthebasicsoffuzzysets,roughsets,
neuralnetworks,geneticalgorithms,andprobabilisticreasoning.Specialfocus
isprovidedonthecombinationandthehybridizationofroughandfuzzysets,
and their role for several image processing tasks, which represent the starting
point of many algorithms employed in video surveillance.
Chapter3presentssomeexamplesofneuralnetwork–basedapproachesto
the solution of video surveillance tasks provided in the literature, including
moving object detection and tracking, crowd and traffic density estimation,
anomaly detection, and behavior understanding. A specific neural-based ap-
proach is further described in order to give evidence of the advantages of its
adoptionformovingobjectdetection,alsoshowingpossibleusesinthecontext
of other video surveillance tasks, such as stopped object detection, activity
recognition, and anomaly detection.
Chapter 4 focuses on summarization, which helps generate movie trailers,
sports and news video highlights, and keeps the records of interesting events
forfutureinspection,playinganimportantroleinthecontextofvideosurveil-
lance,whereprocessinghugechunksofvideodataforpotentialriskdemandsa
hugeamountofresources.Itreviewsexistingsummarizationtechniquesinthe
contextoftheirrelationtosignificantcontentidentification,andaddressesthe
suitabilityofthesetechniquesforsurveillancevideosummarizationalongwith
the issue of personalization. Relevance of soft computing is also mentioned.
ix
Chapters5through11coverabroadspectrumofvideosurveillancetopics
thatadoptsoftcomputingtechniques.Theyareorganizedfollowingtheusual
articulation of a video surveillance system, starting with the task of moving
object detection, to tracking, to classification and the recognition of target
objects.
Chapter5concernsthedetectionofmovingobjectsinvideostreams,which
is the first relevant step in information extraction in many computer vision
applications, and specifically in video surveillance applications. It presents a
fairly extensive and updated survey of research on background subtraction
that exploits fuzzy techniques in order to handle imprecision and uncertainty
inherentinallthestepsforproblemsolution,rangingfrombackgroundmodel-
ing,toforegrounddetection,tobackgroundmaintenance.Someoftheexisting
methods are thoroughly compared and future research directions are envis-
aged.
Theproblemoffusionofdataacquiredfrommultiplesensors,togetherwith
related methods and challenges, is considered in Chapter 6, with some appli-
cationstotrackingobjectsinvideosequences.Dempster–ShaferandBayesian
inference–based data fusion algorithms are considered, and the impact of the
fusion of video sequences from different sensors for enhancing the tracking
process in the presence of changeable illumination, shadows, and other am-
biguous conditions, is specifically addressed. Open issues and other possible
applications are also briefly mentioned.
Chapter 7 tackles the problem of human action modeling and recognition
fromvideosequencesofhumansilhouettesindependentoftheviewpoint.The
modeling step relies on Kohonen self-organizing maps, trained from 2D mo-
tiontemplatesrecordedindifferentviewpointsandvelocities,thusintegrating
spatial and temporal templates into a common framework and reducing their
high dimensionality. Efficacy of the corresponding recognition system, adopt-
ingresamplingtoincreasethenumberoftrainingsequences,isevaluatedboth
on virtual and real datasets.
The task of multi-perspective automated analysis of video data for learn-
ing usual event patterns, as well as detecting unusual events, is addressed in
Chapter8,basedonclusteringusingbothindividualattributesandcombina-
tionofseveralattributes,suchastime,size,shape,andpositionofobjects.A
fuzzy–rough interpretation of the video data for automated video analysis is
provided for the analysis of events from surveillance videos, which is helpful
in dealing with the approximate, incomplete, and vague characteristics of the
video data.
Chapter9concernsthetaskofdetectingvideoshotboundariestoidentify
subsequences consisting of different video contexts, which is a prerequisite in
severalapplications,includingvideosurveillance,targettracking,androbotic
maneuvering. The significance of a fuzzy entropy measure in determining the
change in video context is highlighted, leading to an automatic unsupervised
detection method.
Although many of the aforesaid chapters present specific applications