Intelligent video surveillance systems : an algorithmic approach

209 Pages·2019·9.73 MB·English

by  Kolekar

Intelligent Video
Surveillance Systems
An Algorithmic Approach

Intelligent Video
Surveillance Systems
An Algorithmic Approach
Maheshkumar H. Kolekar
CRC Press
Taylor & Francis Group
6000 Broken Sound Parkway NW, Suite 300
Boca Raton, FL 33487-2742
© 2018 by Taylor & Francis Group, LLC
CRC Press is an imprint of Taylor & Francis Group, an Informa business
No claim to original U.S. Government works
Printed on acid-free paper
Version Date: 20180529
International Standard Book Number-13: 978-1-4987-6711-8 (Hardback)
This book contains information obtained from authentic and highly regarded sources. Reasonable
efforts have been made to publish reliable data and information, but the author and publisher cannot
assume responsibility for the validity of all materials or the consequences of their use. The authors and
publishers have attempted to trace the copyright holders of all material reproduced in this publication
and apologize to copyright holders if permission to publish in this form has not been obtained. If any
copyright material has not been acknowledged please write and let us know so we may rectify in any
future reprint.
Except as permitted under U.S. Copyright Law, no part of this book may be reprinted, reproduced,
transmitted, or utilized in any form by any electronic, mechanical, or other means, now known or
hereafter invented, including photocopying, microfilming, and recording, or in any information
storage or retrieval system, without written permission from the publishers.
For permission to photocopy or use material electronically from this work, please access
www.copyright.com (http://www.copyright.com/) or contact the Copyright Clearance Center, Inc.
(CCC), 222 Rosewood Drive, Danvers, MA 01923, 978-750-8400. CCC is a not-for-profit organization
that provides licenses and registration for a variety of users. For organizations that have been granted
a photocopy license by the CCC, a separate system of payment has been arranged.
Trademark Notice: Product or corporate names may be trademarks or registered trademarks, and
are used only for identification and explanation without intent to infringe.
Library of Congress Cataloging-in-Publication Data
Names: Kolekar, Maheshkumar H., 1972- author.
Title: Intelligent video surveillance systems : an algorithmic approach /
Maheshkumar H Kolekar.
Description: First edition. | Boca Raton, Florida : CRC Press/Taylor &
Francis Group, [2019] | Includes bibliographical references and index.
Identifiers: LCCN 2018008209| ISBN 9781498767118 (hardback : acid-free paper)
| ISBN 9781315153865 (ebook)
Subjects: LCSH: Video surveillance. | Image analysis--Data processing. |
Artificial intelligence.
Classification: LCC TK6680.3 .K65 2019 | DDC 621.389/28--dc23
LC record available at https://lccn.loc.gov/2018008209
Visit the Taylor & Francis Web site at
http://www.taylorandfrancis.com
and the CRC Press Web site at
http://www.crcpress.com
Dedicated to my father Hanmant Baburao Kolekar, who believed
in me and motivated me to pursue a career in teaching.

Contents
List of Figures xv
List of Tables xix
Foreword xxi
Preface xxv
I Basics of Image and Video Processing 1
1 Basics of Image Processing 3
1.1 Introduction to Digital Image Processing . . . . . . . . . . . 3
1.1.1 Why Digital Image Processing? . . . . . . . . . . . . . 3
1.1.2 What Is Digital Image? . . . . . . . . . . . . . . . . . 4
1.1.3 What Is Digital Image Processing? . . . . . . . . . . . 4
1.2 Digital Image Processing System . . . . . . . . . . . . . . . . 5
1.2.1 Image Acquisition . . . . . . . . . . . . . . . . . . . . 5
1.2.2 Storage . . . . . . . . . . . . . . . . . . . . . . . . . . 7
1.2.3 Processing . . . . . . . . . . . . . . . . . . . . . . . . . 7
1.2.4 Communication . . . . . . . . . . . . . . . . . . . . . . 7
1.2.5 Display . . . . . . . . . . . . . . . . . . . . . . . . . . 7
1.3 Digital Image Processing Methods . . . . . . . . . . . . . . . 8
1.3.1 Image Enhancement . . . . . . . . . . . . . . . . . . . 8
1.3.2 Image Restoration . . . . . . . . . . . . . . . . . . . . 8
1.3.3 Image Segmentation . . . . . . . . . . . . . . . . . . . 9
1.3.4 Image Compression. . . . . . . . . . . . . . . . . . . . 9
1.3.5 Image Reconstruction . . . . . . . . . . . . . . . . . . 10
1.3.5.1 Analytical Reconstruction. . . . . . . . . . . 10
1.3.5.2 Iterative Reconstruction . . . . . . . . . . . . 11
1.3.6 Image Morphing . . . . . . . . . . . . . . . . . . . . . 11
1.3.7 Image Recognition . . . . . . . . . . . . . . . . . . . . 12
1.3.8 Image Mosaicing . . . . . . . . . . . . . . . . . . . . . 12
1.3.9 Image Watermarking . . . . . . . . . . . . . . . . . . . 13
1.3.10 Image Registration . . . . . . . . . . . . . . . . . . . . 14
1.4 Digital Image Segmentation . . . . . . . . . . . . . . . . . . 14
1.4.1 Classification of Image Segmentation Techniques . . . 15
vii
viii Contents
1.4.2 Edge Detection . . . . . . . . . . . . . . . . . . . . . . 15
1.4.2.1 Classification of Edges . . . . . . . . . . . . . 16
1.4.2.2 Gradient Operator . . . . . . . . . . . . . . . 16
1.4.2.3 Laplacian Operator . . . . . . . . . . . . . . 18
1.4.2.4 Marr Hildreth Edge Detector . . . . . . . . . 19
1.4.2.5 Isolated Point Detection. . . . . . . . . . . . 21
1.4.2.6 Line Detection . . . . . . . . . . . . . . . . . 21
1.4.2.7 Canny Edge Detector . . . . . . . . . . . . . 21
1.4.3 Edge Linking . . . . . . . . . . . . . . . . . . . . . . . 23
1.4.3.1 Local Processing . . . . . . . . . . . . . . . . 24
1.4.3.2 Regional Processing . . . . . . . . . . . . . . 25
1.4.3.3 Global Processing Using Hough Transform . 25
1.4.4 Thresholding . . . . . . . . . . . . . . . . . . . . . . . 27
1.4.4.1 Multiple Thresholding . . . . . . . . . . . . . 27
1.4.4.2 Global Thresholding . . . . . . . . . . . . . . 28
1.4.4.3 Local Thresholding . . . . . . . . . . . . . . 30
1.4.5 Region Growing . . . . . . . . . . . . . . . . . . . . . 30
1.4.6 Region Splitting and Merging . . . . . . . . . . . . . . 30
1.4.7 Watershed-Based Segmentation . . . . . . . . . . . . . 31
1.4.7.1 Use of Markers . . . . . . . . . . . . . . . . . 32
1.5 Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
1.5.1 Television Signal Processing . . . . . . . . . . . . . . . 32
1.5.2 Satellite Image Processing . . . . . . . . . . . . . . . . 33
1.5.3 Medical Image Processing . . . . . . . . . . . . . . . . 34
1.5.4 Robot Control . . . . . . . . . . . . . . . . . . . . . . 34
1.5.5 Visual Communications . . . . . . . . . . . . . . . . . 35
1.5.6 Law Enforcement . . . . . . . . . . . . . . . . . . . . . 35
1.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
2 Basics of Video Compression and Motion Analysis 37
2.1 Video Compression . . . . . . . . . . . . . . . . . . . . . . . 37
2.1.1 What Is Video Compression? . . . . . . . . . . . . . . 37
2.1.2 Why Video Compression? . . . . . . . . . . . . . . . . 37
2.1.3 Types of Video Compression . . . . . . . . . . . . . . 38
2.1.3.1 Lossless . . . . . . . . . . . . . . . . . . . . . 38
2.1.3.2 Lossy . . . . . . . . . . . . . . . . . . . . . . 39
2.1.4 Latency . . . . . . . . . . . . . . . . . . . . . . . . . . 39
2.1.5 MPEG Compression . . . . . . . . . . . . . . . . . . . 40
2.1.5.1 Reduction of the Resolution . . . . . . . . . 40
2.1.5.2 Motion Estimation . . . . . . . . . . . . . . . 41
2.1.5.3 Discrete Cosine Transform . . . . . . . . . . 44
2.1.5.4 Quantization . . . . . . . . . . . . . . . . . . 44
2.1.5.5 Entropy Coding . . . . . . . . . . . . . . . . 44
2.1.6 Video Compression Standards . . . . . . . . . . . . . . 45
2.2 Motion Segmentation . . . . . . . . . . . . . . . . . . . . . . 46
Contents ix
2.2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . 46
2.2.1.1 Issues in Motion Segmentations . . . . . . . 46
2.2.1.2 MainAttributesofaMotionSegmentationAl-
gorithm . . . . . . . . . . . . . . . . . . . . . 47
2.2.2 Motion Segmentation Algorithms . . . . . . . . . . . . 48
2.2.2.1 Image Difference . . . . . . . . . . . . . . . . 48
2.2.2.2 Statistical Theory . . . . . . . . . . . . . . . 48
2.2.2.3 Optical Flow . . . . . . . . . . . . . . . . . . 49
2.2.2.4 Layers . . . . . . . . . . . . . . . . . . . . . . 49
2.2.2.5 Factorization Technique . . . . . . . . . . . . 50
2.3 Optical Flow Methods . . . . . . . . . . . . . . . . . . . . . . 50
2.3.1 Estimation of Optical Flow . . . . . . . . . . . . . . . 50
2.3.1.1 Horn-Schunck Optical Flow Estimation . . . 51
2.3.1.2 Lucas Kanade Optical Flow Estimation . . . 53
2.4 Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
2.4.1 Surveillance and Security . . . . . . . . . . . . . . . . 56
2.4.2 Content-Based Video Indexing and Retrieval . . . . . 56
2.4.3 Automatic Highlight Generation of Sports Videos. . . 57
2.4.4 Traffic Monitoring . . . . . . . . . . . . . . . . . . . . 57
2.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
3 Background Modeling 59
3.1 What Is Background Modeling? . . . . . . . . . . . . . . . . 59
3.2 Background Modeling Techniques . . . . . . . . . . . . . . . 59
3.2.1 Non-Statistical Background Modeling Methods . . . . 61
3.2.1.1 Background Modeling Independent of Time . 61
3.2.1.2 Improved Basic Background Modeling . . . . 61
3.2.1.3 Long-Term Average Background Modeling . 62
3.2.2 Statistical Background Modeling Methods . . . . . . . 63
3.2.2.1 Example of GMM . . . . . . . . . . . . . . . 63
3.2.2.2 GMM Model . . . . . . . . . . . . . . . . . . 63
3.2.2.3 Expectation Maximization GMM Algorithm 65
3.2.2.4 GMM-Based Background Detection . . . . . 66
3.3 Shadow Detection and Removal . . . . . . . . . . . . . . . . 67
3.3.1 Shadow Removal for Traffic Flow Detection . . . . . . 69
3.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
II Object Tracking 73
4 Object Classification 75
4.1 Shape-Based Object Classification . . . . . . . . . . . . . . . 76
4.2 Motion-Based Object Classification . . . . . . . . . . . . . . 76
4.2.1 Approaches . . . . . . . . . . . . . . . . . . . . . . . . 76
4.2.2 Applications . . . . . . . . . . . . . . . . . . . . . . . 77
4.3 Viola Jones Object Detection Framework . . . . . . . . . . . 77