Inductive Synthesis of Functional Programs: Universal Planning, Folding of Finite Programs, and Schema Abstraction by Analogical Reasoning

365 Pages·2003·1.48 MB·English

by  Ute Schmid (auth.)

INDUCTIVE SYNTHESIS OF
FUNCTIONAL PROGRAMS
Learning Domain-Specific Control Rules
and Abstract Schemes
UTE SCHMID
Fakultät IV – Elektrotechnik und Informatik
Technische Universita¨t Berlin
Deutschland
Habilitationsschrift –Mai2001
ii INDUCTIVESYNTHESISOFFUNCTIONALPROGRAMS
Abstract In this book a novel approach to inductive synthesis of recursive functions is
proposed,combininguniversalplanning,foldingoffiniteprograms,andschema
abstractionbyanalogicalreasoning. Inafirststep,anexampledomainofsmall
complexity is explored by universal planning. For example, for all possible
lists over four fixed natural numbers, their optimal transformation sequences
intothesortedlistarecalculatedand representedasaDAG.Inasecondstep,
theplanistransformedintoafiniteprogramterm. Plantransformationmainly
relies on inferring the data type underlying a given plan. In a third step, the
finiteprogramisfoldedinto(asetof)recursivefunctions. Foldingisperformed
bysyntacticalpattern-matchingandcorrespondstoinducingaspecialclassof
context-freetreegrammars. Itisshown thattheapproachcanbesuccessfully
appliedtolearndomain-specificcontrolrules. Controlrulelearningisimportant
to gain the efficiency of domain-specific planners without the need to hand-
code the domain-specific knowledge. Furthermore, an extension of planning
basedonapurelyrelationaldomaindescriptionlanguagetofunctionapplication
is presented. This extension makes planning applicable to a larger class of
problemswhichareofinterestforprogramsynthesis.
As a last step, a hierarchy of program schemes (patterns) is generated by
generalizing over already synthesized recursive functions. Generalization can
beconsideredasthelaststepofproblemsolvingbyanalogyorprogrammingby
analogy. Somepsychologicalexperimentswereperformedtoinvestigatewhich
kindofstructuralrelationsbetweenproblemscanbeexploitedbyhumanproblem
solvers. Anti-unificationispresentedasanapproachtomappingandgeneralizing
programs structures. It is proposed that the integration of planning, program
synthesis,andanalogicalreasoningcontributestocognitivescienceresearchon
skillacquisitionbyaddressingtheproblemofextractinggeneralizedrulesfrom
someinitialexperience. Such(control)rulesrepresentdomain-specificproblem
solvingstrategies.
AllpartsoftheapproachareimplementedinCommonLisp.
Keywords: InductiveProgramSynthesis,RecursiveFunctions,UniversalPlanning,Function
ApplicationinPlanning,PlanTransformation,DataTypeInference,Foldingof
ProgramTerms,Context-freeTreeGrammars,ControlRuleLearning,Strategy
Learning,AnalogicalReasoning,Anti-Unification,SchemeAbstraction
ForMyParentsand
InMemoryofMy
Grandparents

Acknowledgments
Researchisakindofworkonecanneverdocompletelyalone. Overtheyears
Iprofitedfromtheguidanceofseveralprofessorswhosupervisedorsupported
mywork,namelyFritzWysotzki,KlausEyferth,BerndMahr,JaimeCarbonell,
ArnoldUpmeyer, Peter Pepper, and Gerhard Strube. I learned a lot from dis-
cussions withthem, with colleagues, and students, such as JochenBurghardt,
Bruce Burns, the group of Hartmut Ehrig, Pierre Flener, Hector Geffner, Pe-
ter Geibel, Peter Gerjets, Jürgen Giesl, Wolfgang Grieskamp, Maritta Heisel,
Ralf Herbrich, Laurie Hiyakumoto, Petra Hofstedt, Rune Jensen, Emanuel
Kitzelmann, Jana Koehler, Steffen Lange, Martin Mühlpfordt, Brigitte Pien-
tka, Heike Pisch, Manuela Veloso, Ulrich Wagner, Bernhard Wolf, Thomas
Zeugmann(sorrytoeveryoneIforgot). IamverygratefulforthetimeIcould
spendatCarnegieMellonUniversity. ThankstoKlausEyferthandBerndMahr
who motivated meto go, to GerhardStrube forhis support,to FritzWysotzki
who accepted my absence from teaching, and, of course to Jaime Carbonell
whowasmyveryhelpfulhost. MyworkprofitedmuchfromtheinspirationI
gotfromtalks,classes,anddiscussions,andfromtheveryspecialatmosphere
suggesting that everything is all right as long as “the heart is in the work”. I
thankallmydiplomastudentswhosupportedtheworkreportedinthisbook–
Dirk Matzke, Rene Mercy, Martin Mühlpfordt, Marina Müller, Mark Müller,
Heike Pisch, Knut Polkehn, Uwe Sinha, Imre Szabo, Janin Toussaint, Ulrich
Wagner, Joachim Wirth, and Bernhard Wolf. Additional thanks to some of
them and Peter Pollmanns for proof-reading parts of the draft of this book. I
owe a lot to Fritz Wysotzki for giving me the chance to move from cognitive
psychology to artificial intelligence, for many interesting discussions and for
criticallyreadingandcommentingthedraftofthisbook. Finally,thankstomy
colleaguesandfriendsBerryClaus,RobinHörnig,BarbaraKaup,andMartin
Kindsmüller, to my family, and my husband Uwe Konerding for support and
highqualityleisuretimeandtoallauthorsofgoodcrimenovels.
v

Contents
Acknowledgments v
ListofFigures xv
ListofTables xxi
1. INTRODUCTION 1
PartI Planning
2. STATE-BASEDPLANNING 15
1 StandardStrips 15
1.1 ABlocks-WorldExample 16
1.2 BasicDefinitions 16
1.3 BackwardOperatorApplication 21
2 ExtensionsandAlternativestoStrips 22
2.1 ThePlanningDomainDefinitionLanguage 23
2.1.1 TypingandEqualityConstraints 24
2.1.2 ConditionalEffects 26
2.2 SituationCalculus 27
3 BasicPlanningAlgorithms 30
3.1 InformalIntroductionofBasicConcepts 31
3.2 ForwardPlanning 32
3.3 FormalPropertiesofPlanning 35
3.3.1 ComplexityofPlanning 35
3.3.2 Termination,Soundness,Completeness 37
3.3.3 Optimality 38
3.4 BackwardPlanning 39
3.4.1 GoalRegression 39
3.4.2 IncompletenessofLinearPlanning 42
4 PlanningSystems 44
4.1 ClassicalApproaches 44
4.1.1 StripsPlanning 44
4.1.2 DeductivePlanning 45
4.1.3 PartialOrderPlanning 45
vii
viii INDUCTIVESYNTHESISOFFUNCTIONALPROGRAMS
4.1.4 TotalOrderNon-LinearPlanning 46
4.2 CurrentApproaches 46
4.2.1 GraphplanandDerivates 46
4.2.2 ForwardPlanningRevisited 48
4.3 ComplexDomainsandUncertainEnvironments 48
4.3.1 IncludingDomainKnowledge 48
4.3.2 PlanningforNon-DeterministicDomains 49
4.4 UniversalPlanning 50
4.5 PlanningandRelatedFields 52
4.5.1 PlanningandProblemSolving 52
4.5.2 PlanningandScheduling 54
4.5.3 ProofPlanning 54
4.6 PlanningLiterature 55
5 AutomaticKnowledgeAcquisitionforPlanning 56
5.1 Pre-PlanningAnalysis 56
5.2 PlanningandLearning 56
5.2.1 LinearMacro-Operators 56
5.2.2 LearningControlRules 57
5.2.3 LearningControlPrograms 58
3. CONSTRUCTINGCOMPLETESETSOFOPTIMALPLANS 61
1 IntroductiontoDPlan 61
1.1 DPlanPlanningLanguage 62
1.2 DPlanAlgorithm 64
1.3 EfficiencyConcerns 65
1.4 ExampleProblems 66
1.4.1 TheClearblockProblem 66
1.4.2 TheRocketProblem 67
1.4.3 TheSortingProblem 67
1.4.4 TheHanoiProblem 69
1.4.5 TheTowerProblem 69
2 OptimalFullUniversalPlans 71
3 Termination,Soundness,Completeness 74
3.1 TerminationofDPlan 74
3.2 OperatorRestrictions 75
3.3 SoundnessandCompletenessofDPlan 77
4. INTEGRATINGFUNCTIONAPPLICATIONINPLANNING 79
1 Motivation 79
2 ExtendingStripstoFunctionApplications 82
3 ExtensionsofFPlan 88
3.1 BackwardOperatorApplication 88
3.2 IntroducingUser-DefinedFunctions 90
4 Examples 91
4.1 PlanningwithResourceVariables 91
4.2 PlanningforNumericalProblems 94
4.3 FunctionalPlanningforStandardProblems 96
4.4 MixingADD/DELEffectsandUpdates 97
4.5 PlanningforProgrammingProblems 98
Contents ix
4.6 ConstraintSatisfactionandPlanning 99
5. CONCLUSIONSANDFURTHERRESEARCH 101
1 ComparingDPlanwiththeStateoftheArt 101
2 ExtensionsofDPlan 102
3 UniversalPlanningversusIncrementalExploration 103
PartII InductiveProgramSynthesis
6. AUTOMATICPROGRAMMING 107
1 OverviewofAutomaticProgrammingResearch 108
1.1 AIandSoftwareEngineering 108
1.1.1 Knowledge-BasedSoftwareEngineering 108
1.1.2 ProgrammingTutors 109
1.2 ApproachestoProgramSynthesis 110
1.2.1 MethodsofProgramSpecification 110
1.2.2 SynthesisMethods 113
1.3 PointerstoLiterature 117
2 DeductiveApproaches 118
2.1 ConstructiveTheoremProving 118
2.1.1 ProgramSynthesisbyResolution 119
2.1.2 PlanningandProgramSynthesiswithDeductive
Tableaus 122
2.1.3 FurtherApproaches 124
2.2 ProgramTransformation 124
2.2.1 TransformationalImplementation: TheFold
UnfoldMechanism 125
2.2.2 Meta-Programming: TheCIPApproach 129
2.2.3 ProgramSynthesisbyStepwiseRefinement:
KIDS 132
2.2.4 ConcludingRemarks 133
3 InductiveApproaches 134
3.1 FoundationsofInduction 134
3.1.1 BasicConceptsofInductiveLearning 134
3.1.2 GrammarInference 140
3.2 GeneticProgramming 145
3.2.1 BasicConcepts 145
3.2.2 IterationandRecursion 148
3.2.3 EvaluationofGeneticProgramming 151
3.3 InductiveLogicProgramming 152
3.3.1 BasicConcepts 152
3.3.2 BasicLearningTechniques 155
3.3.3 DeclarativeBiases 158
3.3.4 LearningRecursivePrologPrograms 160
3.4 InductiveFunctionalProgramming 163
3.4.1 Summers’ Recurrence Relation Detection
Approach 163
3.4.2 ExtensionsofSummers’Approach 171
x INDUCTIVESYNTHESISOFFUNCTIONALPROGRAMS
3.4.3 Biermann’sFunctionMergingApproach 173
3.4.4 Generalization to N and Programming by
Demonstration 175
4 FinalComments 177
4.1 InductiveversusDeductiveSynthesis 177
4.2 InductiveFunctionalversusLogicProgramming 178
7. FOLDINGOFFINITEPROGRAMTERMS 181
1 TerminologyandBasicConcepts 182
1.1 TermsandTermRewriting 182
1.2 PatternsandAnti-Unification 185
1.3 RecursiveProgramSchemes 187
1.3.1 BasicDefinitions 187
1.3.2 RPSsasTermRewriteSystems 189
1.3.3 RPSsasContext-FreeTreeGrammars 190
1.3.4 InitialProgramsandUnfoldings 192
2 SynthesisofRPSsfromInitialPrograms 197
2.1 FoldingandFixpointSemantics 197
2.2 CharacteristicsofRPSs 197
2.3 TheSynthesisProblem 200
3 SolvingtheSynthesisProblem 201
3.1 ConstructingSegmentations 201
3.1.1 Segmentationfor‘Mod’ 202
3.1.2 SegmentationandSkeleton 204
3.1.3 ValidHypotheses 205
3.1.4 Algorithm 210
3.2 ConstructingaProgramBody 212
3.2.1 Separating Program Body and Instantiated
Parameters 212
3.2.2 ConstructionofaMaximalPattern 213
3.2.3 Algorithm 215
3.3 DealingwithFurtherSubprograms 215
3.3.1 InducingtwoSubprogramsfor‘ModList’ 215
3.3.2 DecompositionofInitialTrees 219
3.3.3 EquivalenceofSub-Schemata 220
3.3.4 ReductionofInitialTrees 222
3.4 FindingParameterSubstitutions 223
3.4.1 FindingSubstitutionsfor‘Mod’ 223
3.4.2 BasicConsiderations 225
3.4.3 DetectingHiddenVariables 230
3.4.4 Algorithm 231
3.5 ConstructinganRPS 233
3.5.1 ParameterInstantiationsfortheMainProgram
233
3.5.2 PuttingallPartsTogether 233
3.5.3 ExistenceofanRPS 235
3.5.4 Extension: EqualityofSubprograms 237
3.5.5 FaultTolerance 238
4 ExampleProblems 238