Visual tracking of articulated and flexible objects

145 Pages·2016·19.32 MB·English

Visual tracking of articulated and flexible objects
Daniel Wesierski
To cite this version:
Daniel Wesierski. Visual tracking of articulated and flexible objects. Other [cs.OH]. Institut National
des Télécommunications, 2013. English. ￿NNT: 2013TELE0007￿. ￿tel-00939073￿
HAL Id: tel-00939073
https://theses.hal.science/tel-00939073
Submitted on 30 Jan 2014
HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est
archive for the deposit and dissemination of sci- destinée au dépôt et à la diffusion de documents
entific research documents, whether they are pub- scientifiques de niveau recherche, publiés ou non,
lished or not. The documents may come from émanant des établissements d’enseignement et de
teaching and research institutions in France or recherche français ou étrangers, des laboratoires
abroad, or from public or private research centers. publics ou privés.
DOCTORANT EN CO-ACCREDITATION
TELECOM SUDPARIS ET L’UNIVERSITE EVRY VAL D’ESSONNE
Sp´ecialit´e: Informatique
E´cole doctorale : Sciences et Ing´enierie
Pr´esent´ee par
Daniel Wesierski
Pour obtenir le grade de
DOCTEUR DE TELECOM SUDPARIS
Visual tracking of articulated and flexible objects
Soutenue le 25 mars 2013
devant le jury compos´e de:
Fr´ed´eric Jurie Professeur `a l’Universit´e de Caen Rapporteur
R´emi Ronfard Charg´e de Recherche de 1`ere classe HDR `a l’INRIA et au LJK Rapporteur
Catherine Achard Maˆıtre de Conf´erences HDR `a l’UPMC Examinatrice
Malik Mallem Professeur `a l’Universit´e d’E´vry-Val-d’Essonne Examinateur
Bernadette Dorizzi Professeur `a T´el´ecom SudParis Directrice de th`ese
Patrick Horain Ing´enieur d’E´tude `a T´el´ecom SudParis, Docteur ing´enieur Encadrant
Th`ese 2013TELE0007
ii
Contents
Abstract vii
Acknowledgments ix
1 Introduction 1
1.1 Problem formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.2 Challenges . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.3 Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
1.3.1 Tracking objects of various shapes and motions . . . . . . . . 7
1.3.2 Efficient convolution routine for Haar-like features . . . . . . 8
1.4 Organization of the thesis . . . . . . . . . . . . . . . . . . . . . . . . 9
1.5 Publications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
2 State of the art 11
2.1 Image Fundamentals . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
2.1.1 Feature primitives . . . . . . . . . . . . . . . . . . . . . . . . 12
2.1.2 Feature transforms . . . . . . . . . . . . . . . . . . . . . . . . 13
2.2 Tracking in low dimensions . . . . . . . . . . . . . . . . . . . . . . . 16
2.2.1 Basic idea behind tracking . . . . . . . . . . . . . . . . . . . . 16
2.2.2 Gradient-based tracking methods . . . . . . . . . . . . . . . . 18
2.2.3 Fast motion . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
2.2.4 Appearance change . . . . . . . . . . . . . . . . . . . . . . . . 22
2.2.4.1 Stability vs plasticity dilemma . . . . . . . . . . . . 23
2.2.4.2 On-line learning . . . . . . . . . . . . . . . . . . . . 24
iii
iv Contents
2.2.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
2.3 Tracking in higher dimensions . . . . . . . . . . . . . . . . . . . . . . 27
2.3.1 Models of objects . . . . . . . . . . . . . . . . . . . . . . . . . 27
2.3.1.1 Shape and appearance . . . . . . . . . . . . . . . . . 28
2.3.1.2 Complex Motion . . . . . . . . . . . . . . . . . . . . 30
2.3.2 Model matching algorithms . . . . . . . . . . . . . . . . . . . 31
2.3.2.1 Local methods . . . . . . . . . . . . . . . . . . . . . 32
2.3.2.2 Semi-global methods . . . . . . . . . . . . . . . . . . 33
2.3.2.3 Global methods . . . . . . . . . . . . . . . . . . . . 34
2.3.3 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
3 Tracking objects with global spatio-temporal constraints 37
3.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
3.2 Chain-graph models . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
3.2.1 Appearance term . . . . . . . . . . . . . . . . . . . . . . . . . 42
3.2.2 Spatial term . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
3.2.3 Temporal term . . . . . . . . . . . . . . . . . . . . . . . . . . 44
3.3 Model-based object categories . . . . . . . . . . . . . . . . . . . . . . 46
3.4 Tracking algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
3.4.1 Initialization . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
3.4.2 Matching . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
3.4.3 Update . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
3.5 Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
3.5.1 Implementation details . . . . . . . . . . . . . . . . . . . . . . 53
3.5.2 Qualitative Analysis . . . . . . . . . . . . . . . . . . . . . . . 54
3.5.3 Quantitative Analysis . . . . . . . . . . . . . . . . . . . . . . 55
3.6 Conclusions and Discussion . . . . . . . . . . . . . . . . . . . . . . . 60
4 Model configurations for tracking various objects 63
4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
4.2 Tracking straight objects . . . . . . . . . . . . . . . . . . . . . . . . . 65
4.3 Tracking planar objects . . . . . . . . . . . . . . . . . . . . . . . . . 74
Contents v
4.4 Tracking heavily deformable objects . . . . . . . . . . . . . . . . . . 78
4.5 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86
5 Efficient Convolution of Haar-like Features 87
5.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88
5.2 Parsing Haar-like features . . . . . . . . . . . . . . . . . . . . . . . . 90
5.2.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
5.2.2 Problem formulation . . . . . . . . . . . . . . . . . . . . . . . 93
5.2.2.1 Model . . . . . . . . . . . . . . . . . . . . . . . . . . 93
5.2.2.2 Example . . . . . . . . . . . . . . . . . . . . . . . . 96
5.2.3 Proposed parsing algorithm . . . . . . . . . . . . . . . . . . . 97
5.2.3.1 Decomposing features into smaller kernels . . . . . . 99
5.2.3.2 Ensembles of trees . . . . . . . . . . . . . . . . . . . 100
5.2.3.3 Choosing the best ensemble . . . . . . . . . . . . . . 102
5.2.3.4 Implementation: B-channel buffer . . . . . . . . . . 103
5.3 Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104
5.4 Conclusions and Discussion . . . . . . . . . . . . . . . . . . . . . . . 107
6 Conclusions 109
6.1 Contributions of the thesis . . . . . . . . . . . . . . . . . . . . . . . . 109
6.2 Potential of the tracking approach . . . . . . . . . . . . . . . . . . . 110
6.3 Limitations of the approach . . . . . . . . . . . . . . . . . . . . . . . 111
6.4 Future work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112
List of figures 115
List of tables 119
Bibliography 121

Abstract
Humans can visually track objects mostly effortlessly. However, it is hard for a
computer to track a fast moving object under varying illumination and occlusions,
inclutter, andwithvaryingappearanceincameraprojectivespaceduetoitsrelaxed
rigidity or change in viewpoint. Since a generic, precise, robust, and fast tracker
could trigger many applications, object tracking has been a fundamental problem of
practical importance since the beginnings of computer vision.
The first contribution of the thesis is a computationally efficient approach to
tracking objects of various shapes and motions. It describes a unifying tracking
system that can be configured to track the pose of a deformable object in a low
or high-dimensional state-space. The object is decomposed into a chained assem-
bly of segments of multiple parts that are arranged under a hierarchy of tailored
spatio-temporalconstraints. Therobustnessandgeneralityoftheapproachiswidely
demonstrated on tracking various flexible and articulated objects.
Haar-like features are widely used in tracking. The second contribution of the
thesis is a parser of ensembles of Haar-like features to compute them efficiently. The
features are decomposed into simpler kernels, possibly shared by subsets of features,
thus forming multi-pass convolutions. Discovering and aligning these kernels within
andbetweenpassesallowsformingrecursivetreesofkernelsthatrequirefewermem-
ory operations than the classic computation, thereby producing the same result but
more efficiently. The approach is validated experimentally on popular examples of
Haar-like features.
vii

Acknowledgments
Foremost,Iwouldliketothankmysupervisor,PatrickHorain,whohasbeenguiding
me from the very first day of the thesis. He has always had time for me and
my thinking benefited much from discussions with him on computer vision. Our
interaction have progressed my skills considerably.
My sincere thanks go to my thesis director, Berndatte Dorizzi, and to J´erˆome
Boudy coordinating my work along the ambitious roadmap of the EU-funded Com-
panionAble project. Their cheerfulness and readiness to help in all matters have
been very impressing to me. Moreover, it was a big fun to interact with the Com-
panionAble “crew” during our project meetings across whole Europe. In particular,
I would like to thank the Technical University of Ilmenau for inviting me for a short
stay at their robotics lab - it was a great, interesting experience.
Nice atmosphere at the department have also embraced me thanks to Badr-
Eddine Benkelfat and his kind leadership as well as to Patricia Fixot and her smile,
openness, and patience of the Super Souris power, who always helped me with
administrative work. My friends at the lab, Maher, David, Manoj, Daria, Sesh,
J´erˆome, Paulo, Toufik, andMohammed, meritmyspecialthanks, too. Ihaveshared
interesting work and great time and jokes with them.
At all times, my fianc´ee, Aneczka, has been beside me with love, support, and
encouragement. Her unmatchable positive attitude to life has been brightening my
path throughout this intensive period of the thesis.
Finally, I wish to thank my parents, for everything, and dedicate this thesis to
them.
ix