Analysis and Design of Machine Learning Techniques: Evolutionary Solutions for Regression, Prediction, and Control Problems

162 Pages·2014·2.839 MB·English

by  Patrick Stalph

Analysis and Design of Machine
Learning Techniques
Patrick Stalph
Analysis and Design
of Machine Learning
Techniques
Evolutionary Solutions for
Regression, Prediction, and
Control Problems
Patrick Stalph
Tübingen , Germany
PhD Th esis, University of Tübingen, 2013

ISBN 978-3-658-04936-2 ISBN 978-3-658-04937-9 (eBook)
DOI 10.1007/978-3-658-04937-9

Th e Deutsche Nationalbibliothek lists this publication in the Deutsche Nationalbiblio-
grafi e; detailed bibliographic data are available in the Internet at http://dnb.d-nb.de.
Library of Congress Control Number: 2014931388

Springer Vieweg
© Springer Fachmedien Wiesbaden 2014
Th is work is subject to copyright. All rights are reserved by the Publisher, whether the whole
or part of the material is concerned, specifi cally the rights of translation, reprinting, reuse of
illustrations, recitation, broadcasting, reproduction on microfi lms or in any other physical
way, and transmission or information storage and retrieval, electronic adaptation, computer
soft ware, or by similar or dissimilar methodology now known or hereaft er developed. Ex-
empted from this legal reservation are brief excerpts in connection with reviews or scholarly
analysis or material supplied specifi cally for the purpose of being entered and executed on
a computer system, for exclusive use by the purchaser of the work. Duplication of this pub-
lication or parts thereof is permitted only under the provisions of the Copyright Law of the
Publisher’s location, in its current version, and permission for use must always be obtained
from Springer. Permissions for use may be obtained through RightsLink at the Copyright
Clearance Center. Violations are liable to prosecution under the respective Copyright Law.
Th e use of general descriptive names, registered names, trademarks, service marks, etc. in
this publication does not imply, even in the absence of a specifi c statement, that such names
are exempt from the relevant protective laws and regulations and therefore free for general
use. While the advice and information in this book are believed to be true and accurate at
the date of publication, neither the authors nor the editors nor the publisher can accept any
legal responsibility for any errors or omissions that may be made. Th e publisher makes no
warranty, express or implied, with respect to the material contained herein.

Printed on acid-free paper

Springer Vieweg is a brand of Springer DE.
Springer DE is part of Springer Science+Business Media.
www.springer-vieweg.de
Acknowledgments
First, I’d like to thank my supervisor, Martin Butz, for his support in general
and, particularly, his constructive criticism, when it came to scientific writing.
Furthermore,Iwanttothankthemembersofthedepartmentofcognitivemod-
elling, formerly called COBOSLAB, for all those inspiring discussions.
I also want to thank Moritz Stru¨be, David Hock, Andreas Alin, Stephan
Ehrenfeld,andJanKneisslerforhelpfulreviews. Mostimportantly,I’mgrateful
for my wife being so patient with me.
Patrick Stalph
Abstract
Manipulating or grasping objects seems like a trivial task for humans, as these
are motor skills of everyday life. However, motor skills are not easy to learn –
babies require several month to develop proper grasping skills. Learning motor
skills is also an active research topic in robotics. However, most solutions are
optimized for industrial applications and, thus, few are plausible explanations
for learning in the human brain.
Thefundamentalchallenge,thatmotivatesthisresearchwork,originatesfrom
the cognitive science: How do humans learn their motor skills? This work
makesaconnectionbetweenroboticsandcognitivesciencesbyanalyzingmotor
skill learning in well defined, analytically tractable scenarios using algorithmic
implementations that could be found in human brain structures – at least to
some extent. However, the work is on the technical side of the two research
fields and oriented towards robotics.
The first contribution is an analysis of algorithms that are suitable for motor
skill learning and plausible from a biological viewpoint. The identification of
elementary features inherent to those algorithms allows other researchers to
develop powerful, yet plausible learning algorithms for similar scenarios. The
algorithm of choice, a learning classifier system – originally envisioned as a
cognitive system –, is a genetic-based machine learning method that combines
reinforcement learning and genetic algorithms. Two alternative algorithms are
also considered.
The second part of this thesis is a scalability analysis of the learning clas-
sifier system and shows the limitations on the one hand, but on the other
hand highlights ways to improve the scaling on certain problems. Nonetheless,
high-dimensional problem may still requirean unacceptable amount of training
time. Therefore a new, informed search operator is developed to guide evolu-
tion through high-dimensional search spaces. Both the need for specialization
but also generalization capabilities are integrated and thus the learning time is
reduced drastically on high-dimensional problems.
The third part of this work discusses the basics of robot control and its chal-
lenges. A complete robot control framework is developed for a simulated, an-
thropomorphicarmwithsevendegreesoffreedom. Theaforementionedlearning
VIII Abstract
algorithms are embedded in a cognitive-plausible fashion. Finally, the frame-
work is evaluated in a more realistic scenario with the anthropomorphic iCub
robot, where stereo cameras are used for visual servoing. Convincing results
confirm the applicability of the approach, while the biological plausibility is
discussed in retrospect.
Contents
1 Introduction and Motivation 1
1.1 How to Learn Motor Skills? . . . . . . . . . . . . . . . . . . . . . 2
1.2 The Robotics Viewpoint . . . . . . . . . . . . . . . . . . . . . . . 4
1.3 From the View of Cognitive Science . . . . . . . . . . . . . . . . 5
1.4 Requirements for Plausible and Feasible Models . . . . . . . . . . 6
1.5 Scope and Structure . . . . . . . . . . . . . . . . . . . . . . . . . 7
I Background 9
2 Introduction to Function Approximation and Regression 11
2.1 Problem Statement . . . . . . . . . . . . . . . . . . . . . . . . . . 11
2.2 Measuring Quality . . . . . . . . . . . . . . . . . . . . . . . . . . 12
2.3 Function Fitting or Parametric Regression . . . . . . . . . . . . . 13
2.3.1 Linear Models with Ordinary Least Squares . . . . . . . . 13
2.3.2 Online Approximation with Recursive Least Squares . . . 15
2.4 Non-Parametric Regression . . . . . . . . . . . . . . . . . . . . . 17
2.4.1 Interpolation and Extrapolation . . . . . . . . . . . . . . 17
2.4.2 Gaussian Process Regression . . . . . . . . . . . . . . . . 19
2.4.3 Artificial Neural Networks . . . . . . . . . . . . . . . . . . 20
2.5 Local Learning Algorithms . . . . . . . . . . . . . . . . . . . . . 24
2.5.1 Locally Weighted Projection Regression . . . . . . . . . . 25
2.5.2 XCSF – a Learning Classifier System. . . . . . . . . . . . 26
2.6 Discussion: Applicability and Plausibility . . . . . . . . . . . . . 26
3 Elementary Features of Local Learning Algorithms 29
3.1 Clustering via Kernels . . . . . . . . . . . . . . . . . . . . . . . . 30
3.1.1 Spherical and Ellipsoidal Kernels . . . . . . . . . . . . . . 32
3.1.2 Alternative Shapes . . . . . . . . . . . . . . . . . . . . . . 34
3.2 Local Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
3.3 Inference as a Weighted Sum . . . . . . . . . . . . . . . . . . . . 36
3.4 Interaction of Kernel, Local Models, and Weighting Strategies . . 37
X Contents
4 Algorithmic Description of XCSF 41
4.1 General Workflow. . . . . . . . . . . . . . . . . . . . . . . . . . . 41
4.2 Matching, Covering, and Weighted Prediction . . . . . . . . . . . 43
4.3 Local Model Adaptation . . . . . . . . . . . . . . . . . . . . . . . 43
4.4 Global Structure Evolution . . . . . . . . . . . . . . . . . . . . . 45
4.4.1 Uniform Crossover and Mutation . . . . . . . . . . . . . . 47
4.4.2 Adding new Receptive Fields and Deletion. . . . . . . . . 48
4.4.3 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
4.5 Relevant Extensions to XCSF . . . . . . . . . . . . . . . . . . . . 50
4.5.1 Subsumption . . . . . . . . . . . . . . . . . . . . . . . . . 50
4.5.2 Condensation and Compaction . . . . . . . . . . . . . . . 52
II Analysis and Enhancements of XCSF 55
5 How and Why XCSF works 57
5.1 XCSF’s Objectives . . . . . . . . . . . . . . . . . . . . . . . . . . 57
5.2 Accuracy versus Generality . . . . . . . . . . . . . . . . . . . . . 58
5.3 Coverage and Overlap . . . . . . . . . . . . . . . . . . . . . . . . 59
5.4 Three Phases to Meet the Objectives . . . . . . . . . . . . . . . . 60
6 Evolutionary Challenges for XCSF 63
6.1 Resource Management and Scalability . . . . . . . . . . . . . . . 64
6.1.1 A Simple Scenario . . . . . . . . . . . . . . . . . . . . . . 64
6.1.2 Scalability Theory . . . . . . . . . . . . . . . . . . . . . . 66
6.1.3 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
6.1.4 Empirical Validation . . . . . . . . . . . . . . . . . . . . . 68
6.1.5 Structure Alignment Reduces Problem Complexity . . . . 69
6.1.6 Summary and Conclusion . . . . . . . . . . . . . . . . . . 71
6.2 Guided Mutation to Reduce Learning Time . . . . . . . . . . . . 72
6.2.1 Guiding the Mutation . . . . . . . . . . . . . . . . . . . . 73
6.2.2 Experimental Validation . . . . . . . . . . . . . . . . . . . 77
6.2.3 Experiment 2: A 10D Sine Wave . . . . . . . . . . . . . . 79
6.2.4 What is the Optimal Guidance Probability? . . . . . . . . 81
6.2.5 Summary and Conclusion . . . . . . . . . . . . . . . . . . 82
III Control Applications in Robotics 85
7 Basics of Kinematic Robot Control 87
7.1 Task Space and Forward Kinematics . . . . . . . . . . . . . . . . 88
Contents XI
7.2 Redundancy and Singularities . . . . . . . . . . . . . . . . . . . . 90
7.2.1 Singularities. . . . . . . . . . . . . . . . . . . . . . . . . . 91
7.3 Smooth Inverse Kinematics and the Nullspace . . . . . . . . . . . 92
7.3.1 Singular Value Decomposition. . . . . . . . . . . . . . . . 93
7.3.2 Pseudoinverse and Damped Least Squares . . . . . . . . . 94
7.3.3 Redundancy and the Jacobian’s Nullspace . . . . . . . . . 96
7.4 A Simple Directional Control Loop . . . . . . . . . . . . . . . . . 98
8 Learning Directional Control of an Anthropomorphic Arm 101
8.1 Learning Velocity Kinematics . . . . . . . . . . . . . . . . . . . . 102
8.1.1 Learning on Trajectories . . . . . . . . . . . . . . . . . . . 104
8.1.2 Joint Limits . . . . . . . . . . . . . . . . . . . . . . . . . . 105
8.2 Complete Learning and Control Framework . . . . . . . . . . . . 105
8.3 Simulation and Tasks . . . . . . . . . . . . . . . . . . . . . . . . 107
8.3.1 Target Generation . . . . . . . . . . . . . . . . . . . . . . 107
8.4 Evaluating Model Performance . . . . . . . . . . . . . . . . . . . 108
8.5 Experiments. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110
8.5.1 Linear Regression for Control . . . . . . . . . . . . . . . . 111
8.5.2 RBFN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114
8.5.3 XCSF . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115
8.5.4 LWPR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117
8.5.5 Exploiting Redundancy: Secondary Constraints . . . . . . 118
8.5.6 Representational Independence . . . . . . . . . . . . . . . 119
8.6 Summary and Conclusion . . . . . . . . . . . . . . . . . . . . . . 122
9 Visual Servoing for the iCub 125
9.1 Vision Defines the Task Space . . . . . . . . . . . . . . . . . . . . 125
9.1.1 Reprojection with Stereo Cameras . . . . . . . . . . . . . 127
9.2 Learning to Control Arm and Head . . . . . . . . . . . . . . . . . 130
9.3 Experimental Validation . . . . . . . . . . . . . . . . . . . . . . . 131
10 Summary and Conclusion 137
10.1 Function Approximation in the Brain? . . . . . . . . . . . . . . . 137
10.2 Computational Demand of Neural Network Approximation . . . 138
10.3 Learning Motor Skills for Control . . . . . . . . . . . . . . . . . . 139
10.3.1 Retrospective: Is it Cognitive and Plausible? . . . . . . . 141
10.3.2 On Optimization and Inverse Control . . . . . . . . . . . 142
10.4 Outlook . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142
Bibliography 145
A A one-dimensional Toy Problem for Regression 155