Table Of Content

SPRINGER BRIEFS IN COMPUTER SCIENCE Yongzhen Huang Tieniu Tan Feature Coding for Image Representation and Recognition SpringerBriefs in Computer Science Series editors Stan Zdonik, Brown University, Providence, USA Shashi Shekhar, University of Minnesota, Minneapolis, USA Jonathan Katz, University of Maryland, College Park, USA Xindong Wu, University of Vermont, Burlington, USA Lakhmi C. Jain, University of South Australia, Adelaide, Australia David Padua, University of Illinois Urbana-Champaign, Urbana, USA Xuemin (Sherman) Shen, University of Waterloo, Waterloo, Canada Borko Furht, Florida Atlantic University, Boca Raton, USA V.S. Subrahmanian, University of Maryland, College Park, USA Martial Hebert, Carnegie Mellon University, Pittsburgh, USA Katsushi Ikeuchi, University of Tokyo, Tokyo, Japan Bruno Siciliano, Università di Napoli Federico II, Napoli, Italy Sushil Jajodia, George Mason University, Fairfax, USA Newton Lee, Newton Lee Laboratories, LLC, Tujunga, USA More information about this series at http://www.springer.com/series/10028 Yongzhen Huang Tieniu Tan (cid:129) Feature Coding for Image Representation and Recognition 123 YongzhenHuang Tieniu Tan Instituteof Automation Instituteof Automation ChineseAcademy ofSciences ChineseAcademy ofSciences Beijing Beijing China China ISSN 2191-5768 ISSN 2191-5776 (electronic) SpringerBriefs inComputer Science ISBN 978-3-662-44999-8 ISBN 978-3-662-45000-0 (eBook) DOI 10.1007/978-3-662-45000-0 LibraryofCongressControlNumber:2014956903 SpringerHeidelbergNewYorkDordrechtLondon ©TheAuthor(s)2014 Thisworkissubjecttocopyright.AllrightsarereservedbythePublisher,whetherthewholeorpart of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilarmethodologynowknownorhereafterdeveloped. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publicationdoesnotimply,evenintheabsenceofaspecificstatement,thatsuchnamesareexempt fromtherelevantprotectivelawsandregulationsandthereforefreeforgeneraluse. Thepublisher,theauthorsandtheeditorsaresafetoassumethattheadviceandinformationinthis book are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors give a warranty, express or implied, with respect to the material contained hereinorforanyerrorsoromissionsthatmayhavebeenmade. Printedonacid-freepaper Springer-VerlagGmbHBerlinHeidelbergispartofSpringerScience+BusinessMedia (www.springer.com) To my wife and parents —Yongzhen Huang To my wife and son —Tieniu Tan Preface Encoding local features of images (also known as feature coding) is a key issue in computer vision andpattern recognitionand essential tomany visualtasks suchas object/scene classification and image/video retrieval. It has been widely studied, and a large number of algorithms have been proposed in the past several years. However, there has not been a monograph that carefully summarizes various feature coding algorithms and discusses how to apply them successfully, possibly due to the variety of their motivations and representations, the difficulty of exploiting their relations and characteristics, and the difference in experimental comparison rules. With an attempt to address these problems, this monograph provides a comprehensive study in the following aspects: (cid:129) Introduces various feature coding methods, including their motivations and mathematical representations; (cid:129) Exploits their relations, based on which a taxonomy is proposed to reveal how they evolve and develop; (cid:129) Summarizesthemaincharacteristicsofcurrentfeaturecodingalgorithms,eachof which is shared by several coding strategies; (cid:129) Discusses the applications of feature coding in different visual tasks, and con- siders the influence of some key factors in feature coding with sufficient experimental studies; (cid:129) Providessuggestionsofenhancingandemployingfeaturecodinginpractice,and points out potential directions for future work. Wehopethatthismonographprovidesausefulreferencetowardfeaturecoding for researchers, practitioners, and students working on problems where feature coding is an issue, such as object classification, scene categorization, and image retrieval. November 2014 Yongzhen Huang Tieniu Tan vii Acknowledgments We express our deep gratitude to those who have helped us in writing this book. The large part of experiments in this monograph would not have been possible without the hard work of Zifeng Wu. We thank Liang Wang, Jian Liang, Dong Wang,andChunshuiCaofordiscussingorproofreadingtheearlierversionsofthe book. Our thanks also go to Lanlan Chang and Jian Li at Springer for their kind help and patience during the preparation of this book. We acknowledge the financial support by National Basic Research Program of China (2012CB316300) and National Natural Science Foundation of China (61135002, 61203252). ix Contents 1 Introduction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1.1 Background. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1.2 Brief Introduction to Feature Coding . . . . . . . . . . . . . . . . . . . . 2 1.3 Organization of the Monograph. . . . . . . . . . . . . . . . . . . . . . . . 5 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 2 Taxonomy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 2.1 Taxonomy Based on Representation. . . . . . . . . . . . . . . . . . . . . 9 2.2 Taxonomy Based on Motivation . . . . . . . . . . . . . . . . . . . . . . . 12 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 3 Representative Feature Coding Algorithms . . . . . . . . . . . . . . . . . . 15 3.1 Definition of Variables. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 3.2 Formulation of Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 3.2.1 Voting-Based Coding. . . . . . . . . . . . . . . . . . . . . . . . . . 15 3.2.2 Fisher Coding. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 3.2.3 Reconstruction-Based Coding . . . . . . . . . . . . . . . . . . . . 17 3.2.4 Local Tangent-Based Coding . . . . . . . . . . . . . . . . . . . . 21 3.2.5 Saliency-Based Coding. . . . . . . . . . . . . . . . . . . . . . . . . 23 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 4 Evolution of Feature Coding. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 4.1 From “Voting” to “Fisher Coding”. . . . . . . . . . . . . . . . . . . . . . 27 4.2 From “Voting” to “Reconstruction” . . . . . . . . . . . . . . . . . . . . . 29 4.3 From “Reconstruction” to “Saliency”. . . . . . . . . . . . . . . . . . . . 30 4.4 From “Reconstruction” to “Local Tangent”. . . . . . . . . . . . . . . . 32 4.5 Evolution Map . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34 xi xii Contents 5 Experimental Study of Feature Coding . . . . . . . . . . . . . . . . . . . . . 37 5.1 Experimental Setting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37 5.2 Basic Results. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39 5.3 Analysis of Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39 5.3.1 Influence of Coding Algorithm . . . . . . . . . . . . . . . . . . . 40 5.3.2 Influence of Codebook Size . . . . . . . . . . . . . . . . . . . . . 41 5.3.3 Influence of Number of Training Samples . . . . . . . . . . . 43 5.3.4 Influence of Implementation. . . . . . . . . . . . . . . . . . . . . 43 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45 6 Enhancement via Integrating Spatial Information . . . . . . . . . . . . . 47 6.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47 6.2 A Unified Framework . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 6.2.1 Unified Form of Multiple Pooling. . . . . . . . . . . . . . . . . 49 6.2.2 Multiple Spatial Pooling. . . . . . . . . . . . . . . . . . . . . . . . 51 6.2.3 Relations with Other Methods. . . . . . . . . . . . . . . . . . . . 52 6.3 Experimental Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54 6.3.1 Basic Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54 6.3.2 Comparison with SPM. . . . . . . . . . . . . . . . . . . . . . . . . 55 6.3.3 Efficiency Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . 57 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58 7 Enhancement via Integrating High Order Coding Information. . . . 59 7.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59 7.2 Exploring High Order Information. . . . . . . . . . . . . . . . . . . . . . 60 7.2.1 Construct Codebook Graph. . . . . . . . . . . . . . . . . . . . . . 60 7.2.2 Describe Codebook Graph . . . . . . . . . . . . . . . . . . . . . . 62 7.3 Experimental Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64 7.3.1 Accuracy Analysis. . . . . . . . . . . . . . . . . . . . . . . . . . . . 65 7.3.2 Efficiency Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . 69 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70 8 Conclusion. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71 8.1 Summary of Feature Coding Algorithms. . . . . . . . . . . . . . . . . . 71 8.2 Open Problems and Future Work. . . . . . . . . . . . . . . . . . . . . . . 72 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74