Visual Servoing via Advanced Numerical Methods

407 Pages·2010·6.38 MB·English

by  Gian Luca Mariottini

Lecture Notes
in Control and Information Sciences 401
Editors:M.Thoma,F.Allgöwer,M.Morari
Graziano Chesi and Koichi Hashimoto (Eds.)
Visual Servoing via Advanced
Numerical Methods
ABC
SeriesAdvisoryBoard
P.Fleming,P.Kokotovic,
A.B.Kurzhanski,H.Kwakernaak,
A.Rantzer,J.N.Tsitsiklis
Editors
Prof.GrazianoChesi Prof.KoichiHashimoto
UniversityofHongKong TohokuUniversity
DepartmentofElectrical Dept.SystemInformationSciences
andElectronicEngineering 6-6-01Aramaki-AzaAoba
PokfulamRoad Sendai
HongKong Aoba-ku
ChowYeiChingBuilding 980-8579Japan
China,People’sRepublic E-mail:[email protected]
E-mail:[email protected]
ISBN978-1-84996-088-5 e-ISBN978-1-84996-089-2
DOI 10.1007/978-1-84996-089-2
LectureNotesinControlandInformationSciences ISSN0170-8643
LibraryofCongressControlNumber:Appliedfor
(cid:2)c 2010Springer-VerlagBerlinHeidelberg
(cid:2) (cid:2)
MATLAB andSimulink areregisteredtrademarksofTheMathWorks,Inc.,3AppleHill
Drive,Natick,MA01760-2098,USA.http://www.mathworks.com
Thisworkissubjecttocopyright.Allrightsarereserved,whetherthewholeorpartofthematerialis
concerned,specificallytherightsoftranslation,reprinting,reuseofillustrations,recitation,broadcasting,
reproductiononmicrofilmorinanyotherway,andstorageindatabanks.Duplicationofthispublication
orpartsthereofispermittedonlyundertheprovisionsoftheGermanCopyrightLawofSeptember 9,
1965,initscurrentversion,andpermissionforusemustalwaysbeobtainedfromSpringer.Violations
areliableforprosecutionundertheGermanCopyrightLaw.
Theuseofgeneral descriptive names,registered names,trademarks, etc. inthis publication does not
imply,evenintheabsenceofaspecificstatement,thatsuchnamesareexemptfromtherelevantprotective
lawsandregulationsandthereforefreeforgeneraluse.
Typeset&CoverDesign:ScientificPublishingServicesPvt.Ltd.,Chennai,India.
Printedonacid-freepaper
543210
springer.com
To my family Shing Chee, Isabella and Sofia
(GC).
To my family Ritsuko, Riho, Daisuke, Kanta,
Teppei and Shogo (KH).
This book is dedicated in memory of Dr.
Ryosuke Mori (1972–2008).
Preface
Robots able to imitate human beings have been at the core of stories of
science fiction as well as dreams of inventors for a long time. Among the
various skills that Mother Nature has provided us with and that often go
forgotten,the abilityofsightiscertainlyoneofthe mostimportant.Perhaps
inspiredby talesofIsaacAsimov,comics andcartoons,andsurelyhelped by
the progress of electronics in recent decades, researchers have progressively
made the dream of creating robots able to move and operate by exploiting
artificial vision a concrete reality.
Technically speaking, we would say that these robots position themselves
and their end-effectors by using the view provided by some artificial eyes
as feedback information. Indeed, the artificial eyes are visual sensors such
as cameras that have the function to acquire an image of the environment.
Such animage describes if and howthe robotis movingtowardthe goaland
hence constitutes feedback information. This procedure is known in robotics
with the term visual servoing, and it is nothing else than an imitation of the
intrinsic mechanism that allows human beings to realize daily tasks such as
reaching the door of the house or grasping a cup of coffee.
Hence, there is no need to say that visual servoing is one of the hottest
areas of robotics. Indeed, visual servoing can be, and is, exploited in numer-
ous and various applications, such as industry, surveillance, transportation,
exploration, surgery, and the replacement of human beings in dangerous en-
vironments. This book aims to provide a collection of the latest advances in
visual servoing, which have been achieved through the development of dedi-
catednumericalmethodsthatthe recentprogressinvideodevices,computer
hardware, and optimization techniques, have made possible.
Organization of the Book
The book is organized in three main parts, reflecting three main classes of
issues in visual servoing.
VIII Preface
The first part, Vision, comprises contributions whose main focus is on
vision issues that characterize visual servoing tasks. Specifically, Mariottini
et al. introduce visual servoing via catadioptric stereo with planar mirrors.
Swensen et al. analyze convergence properties of featureless visual servoing.
Namiki et al. propose a high-speed vision system for grasping and manipu-
lation. Hager considers human-machine cooperation via vision-basedmotion
constraints. Collewet and Marchand propose luminance as a new feature for
visual servoing. Bachta et al. address vision issues in beating heart surgery.
The second part, Estimation and Path-planning, collects contributions
whosemainfocusisonestimationissuesandpath-planningstrategies.Specif-
ically,Mezouaraddressestrajectoryplanningviavariationalcalculous.Malis
et al. consider the estimation of homography dynamics. Chesi et al. investi-
gate the effect of image measurement errors on the positioning error. Danes
et al. analyze visual servoing through rational systems and linear matrix in-
equalities (LMIs). Kazemi et al. present a review of path-planning methods.
Dani and Dixon consider structure and motion estimation with single cam-
era. Tahri et al. address visual servoing and pose estimation with cameras
obeying the unified model.
Lastly, the third part, Control, focuses on controlissues of visual servoing
tasks. Specifically, Ma and Hutchinson present gradient projection methods
for taking into account constraints in image-based visual servoing (IBVS).
Tarbouriech and Soueres address multi-constraint satisfaction in IBVS via
LMIs. Hadj-Abdelkader et al. present a control scheme with central cam-
eras. Lamiraux and Lefebvre describe deformation-based trajectory control
fornonholonomicrobots.Beloet al.considereye-in-handunicycle-likerobots
viaposition-basedvisualservoingandIBVS.Iwataniet al. addressocclusion
avoidance in unmanned helicopter control. Allibert et al. propose the use of
nonlinear predictive control in visual servoing.
Acknowledgements
We thank the authors of the chapters for their precious contributions that
have made possible to realize this great collection of advanced numerical
methods for visual servoing. And we also thank the Springer Editors for
their patience and suggestions that have helped us in the difficult task of
editing a book.
This book was typeset by the authors using LATEX.
Hong Kong Graziano Chesi
Sendai Koichi Hashimoto
September 2009
Contents
Part I: Vision
1 Catadioptric Stereo with Planar Mirrors:
Multiple-view Geometry and Camera Localization....... 3
Gian Luca Mariottini, Stefano Scheggi, Fabio Morbidi,
Domenico Prattichizzo
1.1 Introduction .......................................... 3
1.1.1 Motivation and Related Works .................... 3
1.1.2 Contributions................................... 4
1.1.3 Organization.................................... 5
1.2 Planar Mirrors and Perspective Projection................ 5
1.2.1 The Virtual Point and the Reflection
Transformation ................................. 5
1.2.2 The Virtual Camera and the Projection
Equivalence..................................... 7
1.2.3 Reflective Epipolar Geometry ..................... 7
1.3 Single-View and Multiple-view Geometry for PCS
Sensors............................................... 9
1.3.1 Single-View Geometry ........................... 9
1.3.2 Multiple-view Geometry.......................... 10
1.4 Mirror Calibration and Image-Based Camera
Localization .......................................... 12
1.4.1 Mirror Calibration............................... 12
1.4.2 Image-Based Camera Localization ................. 13
1.5 Simulation and Experimental Results .................... 16
1.5.1 Simulations..................................... 16
1.5.2 Experiments .................................... 18
1.6 Conclusions and Future Work........................... 20
References ................................................ 20
X Contents
2 Empirical Characterization of Convergence Properties
for Kernel-Based Visual Servoing ........................ 23
John P. Swensen, Vinutha Kallem, Noah J. Cowan
2.1 Featureless Visual Servoing ............................. 23
2.2 Kernel-BasedVisual Servoing ........................... 25
2.3 Empirical Validation................................... 29
2.3.1 Analysis of the Domain of Attraction .............. 30
2.3.2 Experimental Results ............................ 32
2.3.3 Final Error in Pixels............................. 36
2.4 Conclusions........................................... 37
References ................................................ 37
3 High-Speed Visual Feedback Control for Grasping and
Manipulation ............................................ 39
Akio Namiki, Taku Senoo, Satoru Mizusawa, Masatoshi Ishikawa
3.1 Introduction .......................................... 39
3.2 High-Speed Batting.................................... 40
3.2.1 Hybrid Trajectory Generator ..................... 40
3.2.2 System Configuration............................ 41
3.2.3 Batting Algorithm............................... 43
3.2.4 Experiment..................................... 45
3.3 Tool Manipulation by Visual Servoing.................... 48
3.3.1 Passive Joint in the Contact Point................. 49
3.3.2 Visual Servo Control for Tool Manipulation......... 49
3.3.3 Experiments .................................... 51
3.4 Conclusion ........................................... 52
References ................................................ 52
4 Human-Machine Cooperative Manipulation with
Vision-Based Motion Constraints ........................ 55
Gregory D. Hager
4.1 Introduction .......................................... 55
4.2 Virtual Fixtures....................................... 56
4.2.1 Virtual Fixtures as a Control Law ................. 57
4.2.2 Virtual Fixtures as Geometric Constraints.......... 58
4.2.3 Choosing the PreferredDirection.................. 59
4.3 Vision-Based Virtual Fixtures........................... 61
4.3.1 Controlling the Viewer: Pure Translation ........... 61
4.3.2 Controlling the Viewer: General Case .............. 64
4.3.3 More General Camera Configurations .............. 65
4.4 Examples of Use ...................................... 66
4.5 Conclusion ........................................... 68
References ................................................ 69
Contents XI
5 Luminance: A New Visual Feature for Visual Servoing... 71
Christophe Collewet, Eric Marchand
5.1 Introduction .......................................... 71
5.2 Luminance as a Visual Feature.......................... 73
5.3 Visual Servoing Control Law............................ 77
5.3.1 Visual Servoing as an Optimization Problem........ 77
5.3.2 Shape of the Cost Function....................... 79
5.3.3 Control Law .................................... 80
5.4 Experimental Results .................................. 81
5.4.1 Positioning Tasks under Temporal Luminance
Constancy...................................... 81
5.4.2 Positioning Tasks under Complex Illumination ...... 86
5.5 Conclusion and Future Works........................... 88
References ................................................ 88
6 Visual Servoing for Beating Heart Surgery .............. 91
Wael Bachta, Pierre Renaud, Ezio Malis, Koichi Hashimoto,
Jacques Gangloff
6.1 Introduction .......................................... 91
6.2 Motion Compensation Approaches....................... 93
6.2.1 Heart-Tool Synchronization....................... 94
6.2.2 Heart Immobilization ............................ 95
6.3 Heart Motion Prediction ............................... 97
6.3.1 Linear Parameter Varying Method................. 97
6.3.2 Amplitude Modulation Method ................... 98
6.4 Robust Real-Time Visual Measurement .................. 100
6.4.1 Problem Formulation ............................ 101
6.4.2 Algorithm ...................................... 102
6.4.3 High-Speed Implementation ...................... 103
6.4.4 Results......................................... 103
6.5 Control Strategies ..................................... 105
6.5.1 Heart-Tool Synchronization....................... 105
6.5.2 Heart Immobilization ............................ 108
6.6 Perspectives .......................................... 111
References ................................................ 112
Part II: Estimation and Path-Planning
7 A Variational Approach to Trajectory Planning in
Visual Servoing.......................................... 117
Youcef Mezouar
7.1 Introduction .......................................... 117
7.2 Preliminaries.......................................... 119
7.2.1 Brief Review of SO(3) ........................... 119