Wide Area Surveillance: Real-time Motion Detection Systems

245 Pages·2014·8.58 MB·English

by  Ahmed Elgammal (auth.)

Augmented Vision and Reality 6
Vijayan K. Asari Editor
Wide Area
Surveillance
Real-time Motion Detection Systems
Augmented Vision and Reality
Volume 6
Series Editors
Riad I. Hammoud, Kokomo, IN, USA
Lawrence B. Wolff, New York, NY, USA
For furthervolumes:
http://www.springer.com/series/8612
Vijayan K. Asari
Editor
Wide Area Surveillance
Real-time Motion Detection Systems
123
Editor
Vijayan K.Asari
Electrical andComputer Engineering
Universityof Dayton
Dayton,OH
USA
ISSN 2190-5916 ISSN 2190-5924 (electronic)
ISBN 978-3-642-37840-9 ISBN 978-3-642-37841-6 (eBook)
DOI 10.1007/978-3-642-37841-6
SpringerHeidelbergNewYorkDordrechtLondon
LibraryofCongressControlNumber:2013950723
(cid:2)Springer-VerlagBerlinHeidelberg2014
Thisworkissubjecttocopyright.AllrightsarereservedbythePublisher,whetherthewholeorpartof
the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations,
recitation,broadcasting,reproductiononmicrofilmsorinanyotherphysicalway,andtransmissionor
informationstorageandretrieval,electronicadaptation,computersoftware,orbysimilarordissimilar
methodology now known or hereafter developed. Exempted from this legal reservation are brief
excerpts in connection with reviews or scholarly analysis or material supplied specifically for the
purposeofbeingenteredandexecutedonacomputersystem,forexclusiveusebythepurchaserofthe
work. Duplication of this publication or parts thereof is permitted only under the provisions of
theCopyright Law of the Publisher’s location, in its current version, and permission for use must
always be obtained from Springer. Permissions for use may be obtained through RightsLink at the
CopyrightClearanceCenter.ViolationsareliabletoprosecutionundertherespectiveCopyrightLaw.
The use of general descriptive names, registered names, trademarks, service marks, etc. in this
publicationdoesnotimply,evenintheabsenceofaspecificstatement,thatsuchnamesareexempt
fromtherelevantprotectivelawsandregulationsandthereforefreeforgeneraluse.
While the advice and information in this book are believed to be true and accurate at the date of
publication,neithertheauthorsnortheeditorsnorthepublishercanacceptanylegalresponsibilityfor
anyerrorsoromissionsthatmaybemade.Thepublishermakesnowarranty,expressorimplied,with
respecttothematerialcontainedherein.
Printedonacid-freepaper
SpringerispartofSpringerScience+BusinessMedia(www.springer.com)
Preface
Wide area surveillance refers to an automated monitoring process that involves
data acquisition, analysis, and interpretation for understanding object behaviors.
Automated surveillance systems are mostly used for military, law enforcement,
and commercial applications. Sensors of different types and characteristics in
surface-basedoraerial-basedplatformsareusedfortheacquisitionofdataoflarge
areas sometimes covering several square miles. Intelligent visual surveillance is
becoming more popular in applications such as human identification, activity
recognition, behavior analysis, anomaly detection, alarming, etc. Detection,
tracking, and identification of moving objects in a wide area surveillance envir-
onment have been an active research area in the past few decades. Object motion
analysis and interpretation are integral components for activity monitoring and
situationalawareness.Real-timeperformanceofthesedataanalysistasksinavery
wide field of view is an important need for monitoring in security and law
enforcement applications.
This edited book, Wide Area Surveillance: Real-time Motion Detection
Systems, aims to present a few selected state-of-the-art research and development
outcomespertainingtoreal-worldapplications.ItispartofaSpringerBookseries
titled Augmented Vision and Reality which has been initiated by Dr. Riad Ham-
moud. This book includes a wide variety of active research topics which are very
relevanttowideareasurveillanceforsceneanalysisandunderstanding.Themajor
research areas addressed in the 10 chapters of this book are background elim-
ination, shadow detection, moving object detection and tracking, human activity
recognition and crowd monitoring, person identification, networked camera con-
trols, and target recognition in multi-spectral and hyper-spectral imagery.
The first part of the book covers research on wide area surveillance data ana-
lysis for background subtraction, moving cast shadow detection, and moving
object detection and tracking. The chapter on Background Subtraction by Ahmed
ElgammalofRutgersUniversitypresentsdetaileddescriptionofvariousstatistical
approaches to model scene background. An extensive review of the concept and
the practice in background subtraction is provided in this chapter to establish the
need for this important research activity in wide area surveillance and scene
understanding. Several basic statistical background subtraction models such as
v
vi Preface
parametric Gaussian models and nonparametric models are presented. It also
discussestheissueofshadowsuppressionforhumanmotionanalysisapplications
and different approaches and tradeoffs for background maintenance. Many recent
developments in background subtraction paradigm are also addressed in this
chapter.
The chapter on Moving Cast Shadows Detection by Ariel Amato, Ivan Huerta,
Mikhail G. Mozerov, F. Xavier Roca, and Jordi González of Universitat Autò-
noma de Barcelona reviews several shadow detection methods as well as their
taxonomiesrelatedtochangedetection,movingobjectdetection,scenematching,
and visual surveillance in a background subtraction context. Detection of moving
cast shadows is an important research activity in a broad range of vision-based
surveillance applications as it helps in improving the performance of long-range
objectclassificationtasks.Severalshadowdetectionmethodsarepresentedinthis
chapter to work with static images and with image sequences. The detected sha-
dow regions in an image can be exploited to obtain geometric and semantic cues
about shape and position of its casting object and localization of the light source.
Theshadowsinvideosequencescanbeexploitedfordetectingchangesinascene
and object matching. The effect of shadows on the shape and color of the target
object which may affect the performance of scene analysis and interpretation is
also discussed in this chapter. The chapter on Object Detection and Tracking in
Wide Area Motion Imagery by Varun Santhaseelan and Vijayan K. Asari of
UniversityofDaytonpresentsnewmethodologiesformovingobjectdetectionand
feature-basedobjecttrackinginlowresolutionvideothatcanworkwellinvarying
lighting conditions as well as for very small objects like pedestrians in an aerial
image.Thebasicphilosophyemployedinthedevelopmentofthenewalgorithmis
thattheentireinformationavailableinanobjectofinteresthastobeutilizedinthe
detection and tracking processes. A dense version of a localized histogram of
gradients on the difference images forms the feature set for representing a low
resolutionobjectregion. Theeffectivenessofanonlinearimageenhancementand
super-resolution algorithms in relationship with tracking in shadows is also pre-
sented in this chapter.
Thesecondpartofthebookcoversresearchanddevelopmentofalgorithmsfor
human activity recognition, autonomous crowd monitoring, and human identifi-
cationinawideareasurveillanceenvironment.ThechapterontheRecognitionof
Complex Human Activities in a Crowd Context by Wongun Choi and Silvio
Savarese of University of Michigan examines the problem of classification of
collective human activities from video sequences. The presence of the coherent
behavioramongindividualsinaspatialandtemporalneighborhoodisdefinedasa
collectiveactivityinthischapter.Collectiveactivitiessuchasqueuinginalineor
talkingaredefinedbyobservingtheinteractionsofnearbyindividualsintimeand
space. Several recent methods for analyzing collective activities through the
concept of crowd context are presented in this chapter. Various solutions for
modeling the crowd context are also discussed along with demonstration of the
Preface vii
flexibility and scalability of the new framework on several datasets of collective
human activities. The chapter on Autonomous Cognitive Crowd Monitoring by
Simone Chiappino, Lucio Marcenaro, Pietro Morerio, and Carlo Regazzoni of
University of Genoa presents an event-based dynamic Bayesian network that can
switch among alternative Bayesian filtering and control lower level modules to
capture adaptive reactions of human operators. An intelligent video surveillance
system should be able to represent complex situations that describe actions and
activities of humans in a dynamically varying background. Human behavior
analysis is an important research activity in cognitive crowd monitoring. A cog-
nitive decision making process includes an automatic support to human decisions
based on object detection, tracking, and situation assessment in the environment.
The new event-based switched dynamic Bayesian network presented in this
chapter can be used to represent and anticipate possible actions and activities
within the intelligent video surveillance context. This can also interact with an
active visual simulator of crowd situations.
The chapter on Unified Representation of Human Faces for Recognition of
Individuals in Surveillance Videos by Le An, Bir Bhanu, and Songfan Yang of
University of California, Riverside uses a novel image representation of a unified
faceimagewhichissynthesizedfrommultiplecameravideofeeds.Lowqualityof
the probe data in terms of resolution, noise, blurriness, and varying lighting
conditionsandthevariationsinfaceposesmaketherecognitionofindividualsina
video surveillance environment more difficult and less accurate in real-world
surveillance video data captured in a multi-camera network. The unified face
image representation warps the probe frames from different cameras toward a
template frontal face which then generate a frontal view of the subject that
incorporates information from different cameras. The unified face image
representation framework presented in this chapter is a generalized approach
which can be adapted to any multi-camera video-based face recognition system
that uses any face feature descriptors and classifiers. The chapter on
Person Re-identification in Wide Area Camera Networks by Shishir K. Shah and
Apurva Bedagkar-Gala of University of Houston considers the context of con-
sistentpeopletrackingovermultiplecamerasinordertofacilitatetheestimationof
theglobaltrajectoryofapersonoverthecameranetwork.Personre-identification
is a fundamental task in wide area surveillance for multi-camera tracking and the
subsequent analysis of long-term activities and behaviors of people in the scene.
Theappearanceofanindividualinasurveillanceenvironmentmaybeaffectedby
varying background lighting, poses, and orientations, and camera to person dis-
tance. These cause the deterioration of accuracies in feature extraction and clas-
sification processes. This chapter discusses these issues and presents a re-
identification model that works well in such challenging conditions. A multi-
parametric model and its effectiveness for person re-identification are also pre-
sented in this chapter.
The third part of the book covers research on navigation aid for unmanned air
systems,automatictargetrecognitiononmulti-spectralandhyper-spectralimagery,
and distributed sensor data processing. The chapter on Opportunities and
viii Preface
Challenges of Terrain Aided Navigation Systems for Aerial Surveillance by
UnmannedAerialVehiclesbySamilTemelandNumanUnaldiofTurkishAirForce
Academypresentsthedevelopmentofaterrainaidednavigationsystemanditsuse
asatest-bedforthedesignofanautonomousnavigationsystem.Unmannedaerial
vehiclesarenowbecomingpopularinmilitaryandciviliansurveillanceapplications
as they provide more accurate, inexpensive, and durable information than ground
surveillance systems. Unmanned aerial surveillance systems gather data using
various sensors equipped on them. Most of the current unmanned aerial systems
depend on satellite-based navigation systems which are likely to be jammed in
militaryfields. Theterrainaidednavigationsystem presentedinthischapter pro-
vides position estimates relative to known terrains by utilizing the height values
fromthesurfacewiththehelpofactiverangesensorswhicharethenmatchedwithin
a terrain digital elevation map. This chapter also summarizes some of the design
objectivesforunmannedaerialvehicles-basedsurveillanceposts.
The chapter on Automatic Target Recognition in Multi-Spectral and Hyper-
Spectral Imagery by Mohammad S. Alam and Adel Sakla of University of South
Alabama discusses the one-dimensional spectral fringe-adjusted joint transform
correlation-based technique for detecting very small targets involving only a few
pixels in multi-spectral and hyper-spectral imagery. The joint transform correla-
tion of the spectral signatures from the unknown hyper-spectral imagery with the
reference signature can detect bothsingle and multiple desired targets in constant
timewhileaccommodatingthein-planeandout-of-planedistortions.Theproposed
joint transform correlation technique is also applied to the discrete wavelet
transform coefficients of the multi-spectral and hyper-spectral data in order to
improvethedetectionperformance.Thischapteralsopresentstheeffectivenessin
performance of the proposed method in some real-life hyper-spectral image data
cubes.ThechapteronDistributedEstimationandControlinCameraNetworksby
A. Kamal, C. Ding, A. Morye, J. A. Farrell, A. Roy-Chowdhury of University of
California,Riversidereviewssomeofthestate-of-the-arttechniquesindistributed
computervisionalgorithmsrelatedtodistributedestimationanddistributedcontrol
scenarios.Distributedprocessingisanecessityinseveralapplicationdomainssuch
asnationalsecurity,homemonitoring,andenvironmentalmonitoringwherelarge-
scale camera networks are deployed. The limitations of setting up a substantial
communication infrastructure beforehand andthe possible mobilityof the sensors
may also necessitate a distributed processing environment. The basic consensus
algorithms and analysis of their applicability in camera networks are discussed in
the distributed estimation section and presents some modifications that would
consider the constraints posed by vision sensors. A review of game-theoretic
cooperative control algorithms is presented in the distributed control section and
proposes how they can be adapted for active sensing in a camera network that
leads to an integrated sensing and control paradigm.
The Editor and the Authors of this book believe that it provides a good source
of information and references for academic and industrial researchers, profes-
sionals in intelligence, surveillance and reconnaissance (ISR) community, and
graduate and senior undergraduate students for conducting wide area image
Preface ix
analysis and scene understanding research. The Editor thanks all the Authors for
their contributions, Dr. Riad Hammoud for proposing this project, and the
reviewers for their invaluable service in making this project a great success.
October 2012 Vijayan K. Asari
Contents
Background Subtraction: Theory and Practice. . . . . . . . . . . . . . . . . . 1
Ahmed Elgammal
Moving Cast Shadows Detection Methods for Video
Surveillance Applications. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
Ariel Amato, Ivan Huerta, Mikhail G. Mozerov,
F. Xavier Roca and Jordi Gonzàlez
Moving Object Detection and Tracking in Wide
Area Motion Imagery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
Varun Santhaseelan and Vijayan K. Asari
Recognizing Complex Human Activities via Crowd Context . . . . . . . . 71
Wongun Choi and Silvio Savarese
Event-Based Switched Dynamic Bayesian Networks
for Autonomous Cognitive Crowd Monitoring . . . . . . . . . . . . . . . . . . 93
Simone Chiappino, Lucio Marcenaro, Pietro Morerio
and Carlo Regazzoni
Unified Face Representation for Individual Recognition
in Surveillance Videos. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123
Le An, Bir Bhanu and Songfan Yang
Person Re-identification in Wide Area Camera Networks. . . . . . . . . . 137
Shishir K. Shah and Apurva Bedagkar-Gala
Opportunities and Challenges of Terrain Aided Navigation Systems
for Aerial Surveillance by Unmanned Aerial Vehicles. . . . . . . . . . . . . 163
Samil Temel and Numan Unaldi
xi