Hybrid Abductive Inductive Learning

180 Pages·2005·1.15 MB·English

University of London
Imperial College of Science, Technology and Medicine
Department of Computing
Hybrid Abductive Inductive Learning
Oliver Ray
Submitted in part fulfilment of the requirements for the degree of
Doctor of Philosophy in Computing of the University of London and
the Diploma of Imperial College, December 2005
Abstract
This thesis introduces a new Machine Learning technique called Hybrid Abductive
Inductive Learning (HAIL) that integrates Abductive Logic Programming (ALP) and
Inductive Logic Programming (ILP) in order to automate the learning of first-order
theories from examples and prior knowledge. A semantics is proposed called Kernel
Set Subsumption (KSS) that generalises the well-known inference method of Bottom
Generalisation by deriving hypotheses with more than one clause. A corresponding
proofprocedureis presented, called HAIL,which extends theALPprocedureof Kakas
and Mancarella and integrates it within a generalisation of Muggleton’s widely-used
ILP system Progol5. HAIL is shown to overcome some of the limitations of Progol5
— including a previously unsuspected incompleteness — and to enlarge the class of
learning problems soluble in practice.
i
Acknowledgements
I am especially grateful to my supervisors Alessandra Russo and Krysia Broda for
their generous support, guidance and patience throughout this research. Special
thanks also go to Antonis Kakas and Stephen Muggleton for several stimulating and
fruitful discussions. I am grateful to Dov Gabbay and Christopher Hogger for their
inspiration and encouragement. I would also like to mention Francesca Toni, Simon
Colton and Artur Garcez for their helpful advice and Gerson Zaverucha for his useful
comments regardingancestor resolution. Discussions with GordonPlotkin andRobin
Smith helped to improve the historical accuracy of the work. I thank my family for
their many sacrifices that made this work possible. Sasivimol Kittivoravitkul was my
constantcompanionthroughthehighsandlowsofthisresearch. Igladlyacknowledge
the financial support of the EPSRC and the Department of Computing at Imperial
College. IextendmyappreciationtotheexaminersPeterFlachandMurrayShanahan
fortheirclosereadingofthethesisandvaluablesuggestionsforimprovement. Thanks
are also due to the anonymous reviewers of [75, 74, 73, 72] for their useful comments.
(These papers are all the original work of the author and were published during the
course of this research.) Finally, I dedicate this thesis to an old friend, Wilfred Leng,
who sadly died just as the work was nearing completion.
ii
Preface
Thisthesisadvancestheclaimthatabductionandinductioncanbeusefullyintegrated
in the context of scientific knowledge discovery. However, while the motivation is
practical, many of the results are technical and contribute to the formal area of
computational logic. The mathematical emphasis adopted in this thesis affords the
benefits of a rigorous semantics and automated logic programming procedures. For
thesakeofconcreteness, thisworkfocusesontwohighlyregardedproofprocedures—
theabductiveprocedureof KakasandMancarella andMuggleton’s inductivelearning
system Progol5 — which are discussed in the first half of the thesis.
Thoughthecontributionsofthisthesiscouldbepresentedwithlessregardtothese
existing systems, the present structure has two advantages. First, it allows a formal
account of Progol5 to be developed that is lacking from the literature. This account
is based on a detailed examination of the Progol5 source code [57], a close reading
of the relevant published work, including [58, 61, 55], and personal correspondence
with Stephen Muggleton, who also provided some original notes [60]. But, more
importantly, an analysis of this procedure adds a strong motivation to the thesis by
revealing an incompleteness of Progol5 with an evident abductive solution.
Thisobservationprovidesthebasisforamoregeneralhybridapproachthatplaces
equal emphasis on its abductive and inductive components. Consequently, the main
challenge addressed in this thesis is that of integrating abductive and inductive logic
programming in a logically principled and practically viable way. A biological case
study is justifiably included to complement the more theoretical aspects of this thesis
and to demonstrate their practical utility.
iii
Contents
1 Introduction 1
1.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.2 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
1.3 Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
1.4 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
2 Preliminaries 13
2.1 Mathematical Notation . . . . . . . . . . . . . . . . . . . . . . . . . . 13
2.2 Classical Logic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
2.2.1 Syntax . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
2.2.2 Semantics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
2.3 Clausal Logic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
2.3.1 Syntax . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
2.3.2 Semantics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
2.3.3 Resolution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
2.3.4 C-derivations . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
2.4 Logic Programming . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
3 Abductive Logic Programming 30
3.1 Abductive Logic Programming (ALP) Task . . . . . . . . . . . . . . . 30
3.2 Generalised Stable Model (GSM) Semantics . . . . . . . . . . . . . . . 33
3.3 Kakas-Mancarella (KM) Proof Procedure . . . . . . . . . . . . . . . . 35
3.4 Bibliographic Remarks . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
iv
CONTENTS v
4 Inductive Logic Programming 44
4.1 Inductive Logic Programming (ILP) Task . . . . . . . . . . . . . . . . 44
4.2 Bottom Generalisation (BG) Semantics . . . . . . . . . . . . . . . . . 50
4.3 Progol5 Proof Procedure . . . . . . . . . . . . . . . . . . . . . . . . . . 52
4.4 Bibliographic Remarks . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
5 Analysis of the Progol5 StartSet 58
5.1 Formalisation of the StartSet Procedure . . . . . . . . . . . . . . . . . 58
5.1.1 Contrapositive Reasoning of StartSet . . . . . . . . . . . . . . . 59
5.1.2 Concrete Specification of StartSet . . . . . . . . . . . . . . . . 68
5.1.3 Abstract Specification of StartSet. . . . . . . . . . . . . . . . . 72
5.2 Proof of the Soundness of StartSet . . . . . . . . . . . . . . . . . . . . 73
5.3 Proof of the Incompleteness of StartSet . . . . . . . . . . . . . . . . . 77
5.4 Characterising the Incompleteness of StartSet . . . . . . . . . . . . . . 83
5.5 Solving the Incompleteness of StartSet . . . . . . . . . . . . . . . . . . 86
5.5.1 Ancestor Resolution . . . . . . . . . . . . . . . . . . . . . . . . 87
5.5.2 Abduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
6 Hybrid Abductive Inductive Learning 95
6.1 Hybrid Abductive Inductive Learning (HAIL) . . . . . . . . . . . . . . 96
6.2 Kernel Set Subsumption (KSS) Semantics . . . . . . . . . . . . . . . . 101
6.3 HAIL Proof Procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . 110
6.3.1 HAIL computations . . . . . . . . . . . . . . . . . . . . . . . . 110
6.3.2 HAIL Implementation . . . . . . . . . . . . . . . . . . . . . . . 114
6.3.3 Contextual Transform . . . . . . . . . . . . . . . . . . . . . . . 120
7 Case Study 127
7.1 Regulation of Lactose Metabolism in E. coli . . . . . . . . . . . . . . . 127
7.2 Application of HAIL . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132
7.3 Application of Progol5 . . . . . . . . . . . . . . . . . . . . . . . . . . . 136
7.4 Comparison of HAIL and Progol5 . . . . . . . . . . . . . . . . . . . . . 142
CONTENTS vi
8 Related Work 145
8.1 Formal Characterisation of Abduction & Induction . . . . . . . . . . . 145
8.2 Conceptual Integration of Abduction & Induction. . . . . . . . . . . . 148
8.3 Procedural Integration of Abduction & Induction . . . . . . . . . . . . 150
9 Conclusions and Future Work 155
9.1 Summary and Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . 155
9.2 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157
List of Figures
1.1 Peirce’s ‘Syllogistic Theory’ . . . . . . . . . . . . . . . . . . . . . . . . 3
1.2 Peirce’s ‘Inferential Theory’ . . . . . . . . . . . . . . . . . . . . . . . . 4
3.1 Graphical representation of Example 3.3.3 . . . . . . . . . . . . . . . . 40
3.2 Textual representation of Example 3.3.3 . . . . . . . . . . . . . . . . . 40
5.1 CProgol5.0 Grammar Experiment — Revisited . . . . . . . . . . . . . 80
5.2 Derivations R −R in the ‘only if’ direction of Theorem 5.4.1 . . . . 85
1 5
5.3 Derivations S −S in the ‘if’ direction of Theorem 5.4.1 . . . . . . . . 86
1 5
5.4 Solution to Example in Lemma 5.3.1 using Ancestor Resolution . . . . 90
5.5 Solution to Example in Fig. 5.1 using Ancestor Resolution . . . . . . . 90
5.6 Solution to Example in Lemma 5.3.1 using Abduction . . . . . . . . . 92
5.7 Solution to Example in Fig. 5.1 using Abduction . . . . . . . . . . . . 92
6.1 Overview of HAIL Learning Cycle . . . . . . . . . . . . . . . . . . . . 100
6.2 Prolog Description of HAIL Learning Cycle . . . . . . . . . . . . . . . 115
6.3 Pseudocode Description of HAIL Learning Cycle . . . . . . . . . . . . 115
6.4 Prolog Description of HAIL Abductive Phase . . . . . . . . . . . . . . 117
6.5 Prolog Description of HAIL Deductive Phase . . . . . . . . . . . . . . 117
6.6 Prolog Description of HAIL Inductive Phase . . . . . . . . . . . . . . . 117
7.1 Transcriptional regulation of the LAC Operon . . . . . . . . . . . . . . 129
7.2 HAIL/Progol5 input file . . . . . . . . . . . . . . . . . . . . . . . . . . 131
7.3 HAIL Contextual Transform for Fig. 7.2 . . . . . . . . . . . . . . . . . 133
vii
LIST OF FIGURES viii
7.4 Successful HAIL/KM computation for Fig. 7.3 . . . . . . . . . . . . . 135
7.5 Progol5 Contrapositives for Fig. 7.2 . . . . . . . . . . . . . . . . . . . 137
7.6 Failed Progol5/StartSet SLD-tree for Fig. 7.5 . . . . . . . . . . . . . . 139
7.7 SuccessfulProgol5/StartSetSLD-treeforFig.7.5usingadditionalback-
ground facts conc(lac,hi,e1), exp(lac(z),e1) and sugar(glu) . . . . . . 141
7.8 Comparison of HAIL/KM and Progol5/StartSet Search Spaces . . . . 143
Chapter 1
Introduction
Abduction and induction are two forms of reasoning that alongside deduction have
played a prominent role in the study of logic and philosophy of science. Historically,
these three reasoning forms have their roots in the work of Aristotle, but were placed
in their modern context by C.S. Peirce around the turn of the twentieth century.
According to Peirce, in the same way deduction epitomises analytic (or necessary)
inference, so abduction and induction exemplify synthetic (or hypothetical) reasoning.
The key distinction is that whereas deduction aims to make explicit the consequences
already implicit in some existing body of knowledge, abduction and induction aim to
discover genuinely new knowledge from empirical data relating to some phenomenon
or phenomena of interest.
In essence, abduction is the process of explanation — of reasoning from effects
to possible causes; while induction is the process of generalisation — or reasoning
from samples to wider populations. Although the logical foundations of abduction
and induction are less clear than for deduction, significant progress has been made in
the areas of Artificial Intelligence (AI) [80] and Machine Learning (ML) [53] where
recentattemptstoformaliseandautomatetheseformsofreasoninghaveledtogreater
understanding and convincing real-world applications, such as [42, 11, 39, 32]. Much
of this success stems from work in the fields of Abductive Logic Programming (ALP)
[34] and Inductive Logic Programming (ILP) [63].
1