Inertial stabilization, estimation and visual servoing for aerial surveillance Czech Technical ...

169 Pages·2013·33.34 MB·English

Inertial stabilization, estimation and visual servoing for
aerial surveillance
by
Ing. Martin Rˇez´aˇc
supervised by Ing. Zdenˇek Hur´ak, Ph.D.
Dissertation
Presented to the Faculty of Electrical Engineering of
Czech Technical University in Prague
in Partial Fulfillment of the Requirements
for the Degree of
Doctor
in Ph.D. programme
Electrical Engineering and Information Technology
in the branch of study
Control Engineering and Robotics
Czech Technical University in Prague
October 2013

To my parents

Acknowledgement
First and the foremost, I would like to thank to Zdenˇek Hur´ak for being a caring and
enthusiastic supervisor. He was always ready for discussions, giving me not only a useful
technical advise and a hint to a relevant literature but also a wealth of encouragement. His
AdvancedAlgorithmsforControlandCommunications(AA4CC)groupattheDepartment
of Control Engineering, Faculty of Electrical Engineering, Czech Technical University in
Prague was a stimulating place for several years.
I am grateful for the opportunity to work in a broader industrial project consor-
tium formed by a team from Czech Air Force and Air Defense Technological Institute
(VTU´LaPVO) lead by Jiˇr´ı Nohy´l, a team from ESSA company lead by Milan Bartoˇs, a
team from Center for Machine Perception (CMP) lead by V´aclav Hlav´aˇc and our AA4CC
team lead by Zdenˇek Hur´ak. This gave my research activities a true multidisciplinary engi-
neeringflavor. Negotiatingmajormodificationsofthemechanicaldesignwithamechanical
engineer, planning the flight tests with the Air Force people, matching the performance of
the camera system to that of the inertial stabilization loop with a computer vision expert,
all these were very enriching.
In order to keep my own research in algorithms and methods relevant from an
engineeringpointofview,Ireliedalotonbothindoorandoutdoorexperimentswithvarious
hardware platforms. I am happy to acknowledge numerous assistance from my colleagues
at the A44CC group. Above all, the successful accomplishment of my work was totally
conditioned by the excellent work of the key developer of electronics — Jaroslav Zˇoha.
I also feel privileged that I have witnessed a few of our graduate students growing into
experienced control systems developers, namely Jan Sal´aˇsek, Ondˇrej Mikul´ın and Jarom´ır
Dvoˇr´ak. Collaboration with these young engineers was a pleasure.
Although not immediately involved in the related research, Jiˇr´ı Zem´anek — a doc-
toral student at the AA4CC group — brought a lot of inspiration to me in diverse areas of
engineering and science. I really enjoyed sharing a part of PhD student’s life with him.
Finally, I would like to express the gratitude to my parents and my girlfriend Lenka
for their love and patience not only during the time I was finishing this thesis.
Martin Rˇeza´cˇ
Czech Technical University in Prague
October 2013
v

Inertial stabilization, estimation and visual servoing for
aerial surveillance
Ing. Martin Rˇez´aˇc
Czech Technical University in Prague, 2013
Supervisor: Ing. Zdenˇek Hur´ak, Ph.D.
This thesis addresses a few technical problems that are all related to the central topic of
inertial stabilizationof the opticalaxis of anairborne camera system (so-calledline-of-sight
stabilization). First,thetaskofaugmentationoftheclassicalinertialstabilizationloopwith
an automatic visual tracking loop is solved in a systematic way invoking the concepts from
the young discipline of visual servoing. Second, the issue of computer-vision-induced delays
was tackled and an intuitive engineering solution was formulated as a special instance of a
reset control, a modified Smith predictor and a multirate control, which allowed to propose
more efficient solutions. Third, the advanced mechanical configuration with two motors ac-
tuatingtherotationoftheopticalpayloadaroundacommonaxis(so-calleddual-stagecon-
figuration) was studied and recent numerical optimization tools for structured -optimal
H∞
controller design were used to provide controllers with a performance superior to the one
obtained with classically tuned PID loops. Definition and selection of these research topics
were motivated by a collaboration with an industrial partner within a series of projects.
Therefore the thesis also documents a few routine engineering results for completeness.
Namely, mathematical modeling of the kinematics and dynamics of common mechanical
configurations such as double-gimbal and dual-stage configurations, inertial estimation, de-
sign of a feedforward disturbance rejection controller and input command shaping filters.
Allthealgorithmswerealwaysimplementedinoneoftheavailablebenchmarksystemsand
tested either in a laboratory or even during helicopter flights.
vii
Contents
Acknowledgement v
Abstract vii
Chapter 1 Introduction 1
1.1 PROBLEM STATEMENT . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 SHORT HISTORY OF THE RELATED PROJECTS SOLVED AT CTU . . 3
1.3 STATE OFTHEART IN INERTIAL STABILIZATION FOR AERIAL AP-
PLICATIONS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.3.1 Academic publications . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.3.2 Commercial products . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.4 CONTRIBUTION OF THE THESIS . . . . . . . . . . . . . . . . . . . . . . . 5
1.5 SOME OTHER MEANS OF PRESENTATION . . . . . . . . . . . . . . . . . 7
1.6 OUTLINE OF THE THESIS . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
Chapter 2 Mechanical configurations 9
2.1 MAIN PRINCIPLES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
2.1.1 Mass stabilization of the camera . . . . . . . . . . . . . . . . . . . . . 9
2.1.2 Fast steering mirrors . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
2.2 DOUBLE GIMBAL PLATFORM (AZ-EL) . . . . . . . . . . . . . . . . . . . 14
2.2.1 Benchmark systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
2.2.2 Assumptions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
2.2.3 Notation for coordinate frames and their rotations . . . . . . . . . . . 15
2.2.4 Forward recursion of Newton-Euler method, determining velocities
and accelerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
2.2.5 Backward recursion of Newton-Euler method, determining torques . . 20
2.2.6 Nonlinear differential equations for dynamics of Az-El system . . . . . 21
2.2.7 Rigid body dynamics modeling . . . . . . . . . . . . . . . . . . . . . . 22
2.2.8 Modeling the friction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
2.2.9 Experimental identification results . . . . . . . . . . . . . . . . . . . . 23
2.3 INTRODUCING DUAL STAGES . . . . . . . . . . . . . . . . . . . . . . . . 25
2.3.1 Short survey of dual-stage stabilization literature . . . . . . . . . . . . 25
viii
2.4 SINGLE-AXIS DUAL-STAGE PLATFORM. . . . . . . . . . . . . . . . . . . 27
2.4.1 Notation and coordinate frames . . . . . . . . . . . . . . . . . . . . . . 27
2.4.2 Model of dynamics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
2.5 FOUR-GIMBAL DUAL-STAGE PLATFORM . . . . . . . . . . . . . . . . . 32
2.5.1 Coordinate frames and their rotations . . . . . . . . . . . . . . . . . . 33
2.5.2 Forward recursion of Newton-Euler method . . . . . . . . . . . . . . . 34
2.5.3 Backward recursion of Newton-Euler method, determining the torques 36
2.6 CONCLUSION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
Chapter 3 Single-axis control structure 39
3.1 CASCADE CONTROL STRUCTURE . . . . . . . . . . . . . . . . . . . . . . 39
3.2 CURRENT LOOP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
3.2.1 Current controller design . . . . . . . . . . . . . . . . . . . . . . . . . 42
3.3 ANGULAR VELOCITY (RATE) LOOP . . . . . . . . . . . . . . . . . . . . 44
3.3.1 Inertial angular rate sensors (gyroscopes) . . . . . . . . . . . . . . . . 45
3.4 (ANGULAR) POSITION LOOP . . . . . . . . . . . . . . . . . . . . . . . . . 46
3.4.1 Role of the feedforward and trajectory shaping . . . . . . . . . . . . . 48
3.4.2 Reference shaping (or trajectory planning) while the target moves . . 48
3.4.3 Trajectory shaping vs. tracking . . . . . . . . . . . . . . . . . . . . . . 49
3.4.4 Trajectory shaping by using saturation and rate limiter . . . . . . . . 49
3.5 SATURATING THE CONTROLLER SIGNAL . . . . . . . . . . . . . . . . . 50
3.6 BIAS PRESENT IN GYRO RATE SIGNAL . . . . . . . . . . . . . . . . . . 50
3.7 CONCLUSION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
Chapter 4 Line-of-sight inertial Stabilization 53
4.1 DOUBLE GIMBAL INERTIAL STABILIZATION . . . . . . . . . . . . . . . 54
4.1.1 LOS stabilization experiment with H240 platform. . . . . . . . . . . . 56
4.2 SINGLE-AXIS DUAL-STAGE INERTIAL STABILIZATION . . . . . . . . . 58
4.2.1 Feedback control configuration . . . . . . . . . . . . . . . . . . . . . . 58
4.2.2 Design of a structured MIMO low-order controller . . . . . . . . . . . 59
4.2.3 Controllers designed using HIFOO and Hinfstruct . . . . . . . . . . . 62
4.2.4 Experimental verification . . . . . . . . . . . . . . . . . . . . . . . . . 64
4.2.5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
4.3 FOUR-JOINT DUAL-STAGE STABILIZATION . . . . . . . . . . . . . . . . 68
4.3.1 Stabilization of inertial angular rates . . . . . . . . . . . . . . . . . . . 69
4.3.2 Closing the outer loops — the full dual-stage stabilization . . . . . . . 70
4.3.3 Experimental results of the LOS stabilization . . . . . . . . . . . . . . 72
4.4 DISTURBANCE REJECTION BY ACCELERATION FEEDFORWARD . . 75
4.4.1 Projection of the base acceleration into other gimbals . . . . . . . . . 76
4.4.2 Experimental identification of elevation unbalance . . . . . . . . . . . 77
4.4.3 Feedforward disturbance rejection . . . . . . . . . . . . . . . . . . . . 79
4.4.4 Simulations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80
4.4.5 Experimental results . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80
ix
4.5 CONCLUSIONS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
Chapter 5 Visual tracking on top of inertial stabilization 84
5.1 IMAGE TRACKER DESCRIPTION . . . . . . . . . . . . . . . . . . . . . . . 85
5.2 MODELING THE DYNAMICS FOR POINTING AND TRACKING . . . . 87
5.2.1 Perspective projection . . . . . . . . . . . . . . . . . . . . . . . . . . . 87
5.2.2 Camera motion and the interaction matrix . . . . . . . . . . . . . . . 88
5.2.3 Linearization at distinguished operating points . . . . . . . . . . . . . 90
5.2.4 Analysis of achievable bandwidth for pointing and tracking . . . . . . 91
5.3 DECOUPLED POINTING AND TRACKING . . . . . . . . . . . . . . . . . 91
5.4 FEEDBACKLINEARIZATIONBASEDVISUALPOINTINGANDTRACK-
ING . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93
5.4.1 Simple proportional image-based pointing and tracking . . . . . . . . 94
5.4.2 Establishing the camera inertial rate using two motors . . . . . . . . . 95
5.4.3 Summary of controller structure for pointing and tracking . . . . . . . 97
5.4.4 Practical considerations for setting the image dynamics . . . . . . . . 98
5.5 NUMERICAL SIMULATIONS . . . . . . . . . . . . . . . . . . . . . . . . . . 98
5.6 LABORATORY EXPERIMENTS . . . . . . . . . . . . . . . . . . . . . . . . 100
5.7 EXPERIMENT ON A HELICOPTER . . . . . . . . . . . . . . . . . . . . . . 102
5.8 CONCLUSIONS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103
Chapter 6 Delay compensation in visual servoing (for aerial surveillance) 105
6.1 INTRODUCTION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105
6.1.1 Definition of the problem . . . . . . . . . . . . . . . . . . . . . . . . . 105
6.1.2 Notation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107
6.2 INTUITIVE WAY OF DELAY COMPENSATION . . . . . . . . . . . . . . . 107
6.2.1 Updating at the slow sampling rate (Case B) . . . . . . . . . . . . . . 107
6.2.2 Updating at the fast sampling rate (Case C) . . . . . . . . . . . . . . 109
6.3 MODIFIED SMITH PREDICTOR . . . . . . . . . . . . . . . . . . . . . . . . 110
6.4 MULTIRATE ESTIMATION APPROACHED VIA LIFTING TECHNIQUE 113
6.4.1 Lifting of (the inputs of) the discrete-time integrator . . . . . . . . . . 116
6.4.2 Design of a reduced observer for a lifted delayed integrator . . . . . . 117
6.5 SIMULATIONS. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118
6.6 EXPERIMENTAL RESULTS . . . . . . . . . . . . . . . . . . . . . . . . . . . 119
6.7 IMPLEMENTATION ISSUES ON REAL SYSTEM . . . . . . . . . . . . . . 121
6.7.1 Extension for camera gimbal . . . . . . . . . . . . . . . . . . . . . . . 121
6.7.2 Bias present in gyro rate signal . . . . . . . . . . . . . . . . . . . . . . 125
6.7.3 Bias estimation based on tracking image background . . . . . . . . . 125
6.8 CONCLUSIONS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126
x