Foundations of Inductive Logic Programming

406 Pages·1997·10.216 MB·English

by  Shan-Hwei Nienhuys-Cheng, Roland de Wolf (auth.)

Lecture Notes in Artificial Intelligence 1228
Subseries of Lecture Notes in Computer Science
Edited by .J G. Carbonell and J. Siekmann
Lecture Notes in Computer Science
Edited by G. Goos, J. Hartmanis and J. van Leeuwen
Springer
Berlin
Heidelberg
New York
Barcelona
Budapest
Hong Kong
London
Milan
Paris
Santa Clara
Singapore
Tokyo
Shan-Hwei Nienhuys-Cheng Ronald de Wolf
snoitadnuoF
of evitcudnI
Logic gnimmargorP
r e g n~ i r p S
Series Editors
Jaime G. Carbonell, Carnegie Mellon University, Pittsburgh, ,AP USA
J6rg Siekmann, University of Saarland, Saarbrticken, Germany
Authors
Shan-Hwei Nienhuys-Cheng
Ronald de Wolf
Erasmus University of Rotterdam, Department of Computer Science
P.O. Box 1738, 3000 DR Rotterdam,The Netherlands
E-mail: cheng @cs.few.eur.nl
Cataloging-in-Publication Data applied for
Die Deutsche Bibliothek - CIP-Einheitsaufnahme
Nienhuys-Cheng, Shan-Hwei:
Foundations of inductive logic programming / S.-H. Nienhuys-Cheng
; R. de Wolf. - Berlin ; Heidelberg ; New York ; Barcelona ;
Budapest ; Hong Kong ; London ; Milan ; Paris ; Santa Clara ;
Singapore ; Tokyo : Springer, 1997
(Lecture notes in computer science ; 1228 : Lecture notes in artificial
intelligence)
ISBN 3-540-62927-0 kart.
CR Subject Classification (1991): 1.2,F.4.1, D.1.6
ISBN 3-540-62927-0 Springer-Verlag Berlin Heidelberg New York
This work is subject to copyright. All rights are reserved, whether the whole or part of the material is
coneerned~ specifically the rights of translation, reprinting, re-use of illustrations, recitation, broadcasting,
reproduction on microfilms or in any other way, and storage in data banks. Duplication of this publication
or parts thereof is permitted only under the provisions of the German Copyright Law of September 9, 1965,
in its current version, and permission for use must always be obtained from Springer -Verlag. Violations are
liable for prosecution under the German Copyright Law.
(cid:14)9 Springer-Verlag Berlin Heidelberg 1997
Printed in Germany
Typesetting: Camera ready by author
SPIN 10549682 06/3142 - 5 4 3 2 1 0 Printed on acid-free paper
Foreword
One of the most interesting recent developments within the field of auto-
mated deduction is inductive logic programming, an area that combines logic
programming with machine learning. Within a short time this area has grown
to an impressive field, rich in spectacular applications and full of techniques
calling for new theoretical insights.
This is the first book that provides a systematic introduction to the theo-
retical foundations of this area. It is a most welcome addition to the literature
concerning learning, resolution, and logic programming.
The authors offer in this book a solid, scholarly presentation of the sub-
ject. By starting their presentation with a self-contained account of the res-
olution method and of the foundations of logic programming they enable
the reader to place the theory of inductive logic programming in the right
historical and mathematical perspective. By presenting in detail the theoret-
ical aspects of all components of inductive logic programming they make it
clear that this field has grown into an important area of theoretical computer
science.
The presentation given by the authors also allows us to reevaluate the
role of some, until now, isolated results in the field of resolution and yields
an interesting novel framework that sheds new light on the use of first-order
logic in computer science.
I would like to take this opportunity to congratulate the authors on the
outcome of their work. I am sure this book will have an impact on the future
of inductive logic programming.
March 1997
Krzysztof R. Apt
CWI and University of Amsterdam
The Netherlands
Contents
About the Book (cid:141)
I Logic 1
1 Propositional Logic 3
1.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.2 Syntax . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.3 Semantics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.3.1 Informally . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.3.2 Interpretations . . . . . . . . . . . . . . . . . . . . . . 7
1.3.3 Models . . . . . . . . . . . . . . . . . . . . . . . . . . 9
1.4 Conventions to Simplify Notation . . . . . . . . . . . . . . . . 15
1.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
2 First-Order Logic 17
2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
2.2 Syntax . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
2.3 Semantics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
2.3.1 Informally . . . . . . . . . . . . . . . . . . . . . . . . . 22
2.3.2 Interpretations . . . . . . . . . . . . . . . . . . . . . . 24
2.3.3 Models . . . . . . . . . . . . . . . . . . . . . . . . . . 29
2.4 Conventions to Simplify" Notation . . . . . . . . . . . . . . . . 33
2.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
3 Normal Forms and Herbrand Models 35
3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
3.2 Prenex Conjunctive Normal Form . . . . . . . . . . . . . . . . 36
3.3 Skolem Standard Form . . . . . . . . . . . . . . . . . . . . . . 39
3.3.1 Clauses and Universal Quantification .......... 39
3.3.2 Standard Form . . . . . . . . . . . . . . . . . . . . . . 40
3.4 Herbrand Models . . . . . . . . . . . . . . . . . . . . . . . . . 45
3.5 Results Concerning Herbrand Models . . . . . . . . . . . . . . 48
vlll CONTENTS
3.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
3.A Alternative Notation for Standard Forms ........... 51
4 Resolution 55
4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
4.2 What Is a Proof Procedure? ................... 57
4.3 Substitution and Unification ................... 59
4.3.1 Substitution . . . . . . . . . . . . . . . . . . . . . . . . 59
4.3.2 Unification . . . . . . . . . . . . . . . . . . . . . . . . 63
4.4 An Informal Introduction to Resolution ............ 65
4.5 A Formal Treatment of Resolution ............... 68
4.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
Subsumption Theorem and Refutation Completeness 75
5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
5.2 Deductions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
5.3 The Subsumption Theorem ................... 78
5.3.1 The Subsumption Theorem for Ground 2 and C . . . 78
5.3.2 The Subsumption Theorem when C is Ground .... 79
5.3.8 The Subsumption Theorem (General Case) ...... 82
5.4 Refutation Completeness ..................... 84
5.4.1 From the Subsumption Theorem to Refutation Com-
pleteness . . . . . . . . . . . . . . . . . . . . . . . . . 84
5.4.2 From Refutation Completeness to the Subsumption
Theorem . . . . . . . . . . . . . . . . . . . . . . . . . 84
5.5 Proving Non-Clausal Logical Implication ............ 87
5.6 How to Find a Deduction .................... 87
5.7 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90
5.A Alternative Definitions of Resolution .............. 91
Linear and Input Resolution 93
6.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93
6.2 Linear Resolution . . . . . . . . . . . . . . . . . . . . . . . . . 94
6.3 Refutation Completeness ..................... 95
6.4 The Subsumption Theorem ................... 98
6.5 The Incompleteness of Input Resolution ............ 100
6.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103
SLD-Reso!ution 105
7.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105
7.2 SLD-Resolution . . . . . . . . . . . . . . . . . . . . . . . . . . 106
7.3 Soundness and Completeness .................. 108
7.3.1 Refutation Completeness ................ 108
7.3.2 The Subsumption Theorem ............... 109
7.4 Definite Programs and Least Herbrand Models ........ 111
CONTENTS Ix
7.5 Correct Answers and Computed Answers ........... 1!3
7.6 Computation Rules ........................ 119
7.7 SLD-Trees ............................. 122
7.8 Undecidability .......................... 125
7.9 Summary ............................. 126
8 SLDNF-Resolution 127
8.1 Introduction ............................ 127
8.2 Negation as Failure ........................ 130
8.3 SLDNF-Trees for Normal Programs ............... 133
8.4 Floundering, and How to Avoid It ............... 141
8.5 The Completion of a Normal Program ............. 145
8.6 Soundness with Respect to the Completion .......... 150
8.7 Completeness ........................... 153
8.8 Prolog ............................... 154
8.8.1 Syntax ........................... 154
8.8.2 Prolog and SLDNF-Trees ................ 155
8.8.3 The Cut Operator .................... 157
8.9 Summary ............................. 159
II Inductive Logic Programming 161
What Is Inductive Logic Programming? 163
9.1 Introduction ............................ 163
9.2 The Normal Problem Setting for ILP .............. 165
9.3 The Nonmonotonic Problem Setting .............. 172
9.4 Abduction ............................. 173
9.5 A Brief History of the Field ................... 174
9.6 Summary ............................. 177
10 The Framework for Model Inference 179
10.1 Introduction ............................ 179
10.2 Formalizing the Problem ..................... 180
10.2.1 Enumerations and the Oracle .............. 180
10.2.2 Complete Axiomatizations and Admissibility ..... 182
10.2.3 Formal Statement of the Problem ............ 184
10.3 Finding a False Clause by Backtracing ............. 186
10.4 Introduction to Refinement Operators ............. 191
10.5 The Model Inference Algorithm ................. 192
10.6 Summary ............................. 195
X CONTENTS
11 Inverse Resolution 197
11.1 Introduction ............................ 197
11.2 The V-Operator ......................... 198
11.3 The W-Operator ......................... 203
11.4 Motivation for Studying Generality Orders ........... 205
11.5 Summary ............................. 205
12 Unfolding 207
12.1 Introduction ............................ 207
12.2 Unfolding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 209
12.3 UDS Specialization . . . . . . . . . . . . . . . . . . . . . . . . 213
12.4 Summary ............................. 217
13 The Lattice and Cover Structure of Atoms 219
13.1 Introduction ............................ 219
13.2 Quasi-Ordered Sets ........................ 220
13.3 Quasi-Ordered Sets of Clauses .................. 225
13.4 Atoms as a Quasi-Ordered Set ................. 225
13.4.1 Greatest Specializations ................. 227
13.4.2 Least Generalizations .................. 227
13.5 Covers ............................... 232
13.5.1 Downward Covers .................... 232
13.5.2 Upward Covers ...................... 234
13.6 Finite Chains of Downward Covers ............... 234
13.7 Finite Chains of Upward Covers ................. 237
13.8 Size . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 240
13.9 Summary ............................. 241
14 The Subsumption Order 243
14.1 Introduction ............................ 243
14.2 Clauses Considered as Atoms .................. 243
14.3 Subsumption ........................... 245
14.4 Reduction ............................. 247
14.5 Inverse Reduction ......................... 249
14.6 Greatest Specializations ..................... 251
14.7 Least Generalizations ...................... 252
14.8 Covers in the Subsume Order .................. 256
14.8.1 Upward Covers ...................... 256
14.8.2 Downward Covers .................... 257
14.9 A Complexity Measure for Clauses ............... 260
14.9.1 Size as Defined by Reynolds ............... 260
14.9.2 A New Complexity Measure ............... 261
14.10 Summary ............................. 262
CONTENTS ~x
15 The Implication Order 265
15.1 Introduction ............................ 265
15.2 Least Generalizations ...................... 266
15.2.1 A Sufficient Condition for the Existence of an LGI . . 267
15.2.2 The LGI is Computable ................. 274
15.3 Greatest Specializations ..................... 275
15.4 Covers in the Implication Order ................. 277
15.5 Summary ............................. 278
16 Background Knowledge 279
16.1 Introduction ............................ 279
16.2 Relative Subsumption ...................... 281
16.2.1 Definition and Some Properties ............. 281
16.2.2 Least Generalizations .................. 285
16.3 Relative Implication ....................... 287
16.3.1 Definition and Some Properties ............. 287
16.3.2 Least Generalizations .................. 288
16.4 Generalized Subsumption .................... 289
16.4.1 Definition and Some Properties ............. 289
16.4.2 Least Generalizations .................. 294
16.5 Summary ............................. 297
17 Refinement Operators 299
17.1 Introduction ............................ 299
17.2 Ideal Refinement Operators for Atoms ............. 300
17.3 Non-Existence of Ideal Refinement Operators ......... 303
17.4 Complete Operators for Subsumption ............. 305
17.4.1 Downward ......................... 305
17.4.2 Upward .......................... 306
17.5 Ideal Operators for Finite Sets ................. 310
17.5.1 Downward ......................... 311
17.5.2 Upward .......................... 315
17.6 Optimal Refinement Operators ................. 316
17.7 Refinement Operators for Theories ............... 317
17.8 Summary ............................. 319
18 PAC Learning 321
18.1 Introduction ............................ 321
18.2 PAC Algorithms . . . . . . . . . . . . . . . . . . . . . . . . 322
18.3 Sample Complexity ........................ 324
18.4 Time Complexity ......................... 326
18.4.1 Representations ...................... 326
18.4.2 Polynomial Time PAC Learnability ........... 328
18.5 Some Related Settings ...................... 329
18.5.1 Polynomial Time PAC Predictability .......... 329