Artificial Intelligence, Second Edition, Python Code

221 Pages·2017·1.01 MB·English

1
Python code for Artificial
Intelligence: Foundations of
Computational Agents
David L. Poole and Alan K. Mackworth
Version0.9.5ofNovember22,2022.
http://aipython.org http://artint.info
©DavidLPooleandAlanKMackworth2017-2022.
AllcodeislicensedunderaCreativeCommonsAttribution-NonCommercial-
http://creativecommons.org/licenses/
ShareAlike4.0InternationalLicense. See:
by-nc-sa/4.0/deed.en US
Thisdocumentandallthecodecanbedownloadedfrom
http://artint.info/AIPython/ http://aipython.org
orfrom
Theauthorsandpublisherofthisbookhaveusedtheirbesteffortsinprepar-
ing this book. These efforts include the development, research and testing of
the theories and programs to determine their effectiveness. The authors and
publishermakenowarrantyofanykind,expressedorimplied,withregardto
these programs or the documentation contained in this book. The author and
publisher shall not be liable in any event for incidental or consequential dam-
ages in connection with, or arising out of, the furnishing, performance, or use
oftheseprograms.
http://aipython.org
Version0.9.5 November22,2022

Contents
Contents 3
1 PythonforArtificialIntelligence 9
1.1 WhyPython? . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
1.2 GettingPython . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
1.3 RunningPython . . . . . . . . . . . . . . . . . . . . . . . . . . 10
1.4 Pitfalls . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
1.5 FeaturesofPython . . . . . . . . . . . . . . . . . . . . . . . . . 11
1.5.1 Lists,Tuples,Sets,DictionariesandComprehensions . . 11
1.5.2 Functionsasfirst-classobjects . . . . . . . . . . . . . . . . 12
1.5.3 GeneratorsandCoroutines . . . . . . . . . . . . . . . . . 14
1.6 UsefulLibraries . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
1.6.1 TimingCode . . . . . . . . . . . . . . . . . . . . . . . . . 15
1.6.2 Plotting: Matplotlib . . . . . . . . . . . . . . . . . . . . . 16
1.7 Utilities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
1.7.1 Display . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
1.7.2 Argmax . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
1.7.3 Probability . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
1.7.4 DictionaryUnion . . . . . . . . . . . . . . . . . . . . . . . 19
1.8 TestingCode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
2 AgentArchitecturesandHierarchicalControl 21
2.1 RepresentingAgentsandEnvironments . . . . . . . . . . . . . 21
2.2 Paperbuyingagentandenvironment . . . . . . . . . . . . . . 22
2.2.1 TheEnvironment . . . . . . . . . . . . . . . . . . . . . . . 22
2.2.2 TheAgent . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
3
4 Contents
2.2.3 Plotting . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
2.3 HierarchicalController . . . . . . . . . . . . . . . . . . . . . . . 25
2.3.1 Environment . . . . . . . . . . . . . . . . . . . . . . . . . 25
2.3.2 Body . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
2.3.3 MiddleLayer . . . . . . . . . . . . . . . . . . . . . . . . . 28
2.3.4 TopLayer . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
2.3.5 Plotting . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
3 SearchingforSolutions 33
3.1 RepresentingSearchProblems . . . . . . . . . . . . . . . . . . 33
3.1.1 ExplicitRepresentationofSearchGraph . . . . . . . . . . 34
3.1.2 Paths . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
3.1.3 ExampleSearchProblems . . . . . . . . . . . . . . . . . . 37
3.2 GenericSearcherandVariants . . . . . . . . . . . . . . . . . . . 43
3.2.1 Searcher . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
3.2.2 FrontierasaPriorityQueue . . . . . . . . . . . . . . . . . 44
3.2.3 A∗ Search . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
3.2.4 MultiplePathPruning . . . . . . . . . . . . . . . . . . . . 47
3.3 Branch-and-boundSearch . . . . . . . . . . . . . . . . . . . . . 49
4 ReasoningwithConstraints 53
4.1 ConstraintSatisfactionProblems . . . . . . . . . . . . . . . . . 53
4.1.1 Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
4.1.2 Constraints . . . . . . . . . . . . . . . . . . . . . . . . . . 54
4.1.3 CSPs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
4.1.4 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
4.2 ASimpleDepth-firstSolver . . . . . . . . . . . . . . . . . . . . 66
4.3 ConvertingCSPstoSearchProblems . . . . . . . . . . . . . . . 67
4.4 ConsistencyAlgorithms . . . . . . . . . . . . . . . . . . . . . . 69
4.4.1 DirectImplementationofDomainSplitting . . . . . . . . 72
4.4.2 DomainSplittingasaninterfacetographsearching . . . 74
4.5 SolvingCSPsusingStochasticLocalSearch . . . . . . . . . . . 75
4.5.1 Any-conflict . . . . . . . . . . . . . . . . . . . . . . . . . . 78
4.5.2 Two-StageChoice . . . . . . . . . . . . . . . . . . . . . . . 79
4.5.3 UpdatablePriorityQueues . . . . . . . . . . . . . . . . . 81
4.5.4 PlottingRuntimeDistributions . . . . . . . . . . . . . . . 83
4.5.5 Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
4.6 DiscreteOptimization . . . . . . . . . . . . . . . . . . . . . . . 84
4.6.1 Branch-and-boundSearch . . . . . . . . . . . . . . . . . . 86
5 PropositionsandInference 89
5.1 RepresentingKnowledgeBases . . . . . . . . . . . . . . . . . . 89
5.2 Bottom-upProofs(withaskables) . . . . . . . . . . . . . . . . . 92
5.3 Top-downProofs(withaskables) . . . . . . . . . . . . . . . . . 94
5.4 DebuggingandExplanation . . . . . . . . . . . . . . . . . . . . 95
http://aipython.org
Version0.9.5 November22,2022
Contents 5
5.5 Assumables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
6 DeterministicPlanning 103
6.1 RepresentingActionsandPlanningProblems . . . . . . . . . . 103
6.1.1 RobotDeliveryDomain . . . . . . . . . . . . . . . . . . . 104
6.1.2 BlocksWorld . . . . . . . . . . . . . . . . . . . . . . . . . 106
6.2 ForwardPlanning . . . . . . . . . . . . . . . . . . . . . . . . . . 108
6.2.1 DefiningHeuristicsforaPlanner . . . . . . . . . . . . . . 111
6.3 RegressionPlanning . . . . . . . . . . . . . . . . . . . . . . . . 113
6.3.1 DefiningHeuristicsforaRegressionPlanner . . . . . . . 115
6.4 PlanningasaCSP . . . . . . . . . . . . . . . . . . . . . . . . . . 116
6.5 Partial-OrderPlanning . . . . . . . . . . . . . . . . . . . . . . . 119
7 SupervisedMachineLearning 127
7.1 RepresentationsofDataandPredictions . . . . . . . . . . . . . 128
7.1.1 CreatingBooleanConditionsfromFeatures . . . . . . . . 131
7.1.2 EvaluatingPredictions . . . . . . . . . . . . . . . . . . . . 133
7.1.3 CreatingTestandTrainingSets . . . . . . . . . . . . . . . 134
7.1.4 ImportingDataFromFile . . . . . . . . . . . . . . . . . . 135
7.1.5 AugmentedFeatures . . . . . . . . . . . . . . . . . . . . . 138
7.2 GenericLearnerInterface . . . . . . . . . . . . . . . . . . . . . 140
7.3 LearningWithNoInputFeatures . . . . . . . . . . . . . . . . . 141
7.3.1 Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . 143
7.4 DecisionTreeLearning . . . . . . . . . . . . . . . . . . . . . . . 145
7.5 CrossValidationandParameterTuning . . . . . . . . . . . . . 149
7.6 LinearRegressionandClassification . . . . . . . . . . . . . . . 152
7.6.1 BatchedStochasticGradientDescent . . . . . . . . . . . . 158
7.7 Boosting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159
7.7.1 GradientTreeBoosting . . . . . . . . . . . . . . . . . . . . 162
8 NeuralNetworksandDeepLearning 165
8.1 Layers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165
8.2 FeedforwardNetworks. . . . . . . . . . . . . . . . . . . . . . . 168
8.3 ImprovedOptimization . . . . . . . . . . . . . . . . . . . . . . 170
8.3.1 Momentum . . . . . . . . . . . . . . . . . . . . . . . . . . 170
8.3.2 RMS-Prop . . . . . . . . . . . . . . . . . . . . . . . . . . . 171
8.4 Dropout . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172
8.4.1 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . 173
9 ReasoningwithUncertainty 179
9.1 RepresentingProbabilisticModels . . . . . . . . . . . . . . . . 179
9.2 RepresentingFactors . . . . . . . . . . . . . . . . . . . . . . . . 180
9.3 ConditionalProbabilityDistributions . . . . . . . . . . . . . . 181
9.3.1 LogisticRegression . . . . . . . . . . . . . . . . . . . . . . 182
9.3.2 Noisy-or . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183
http://aipython.org
Version0.9.5 November22,2022
6 Contents
9.3.3 TabularFactors . . . . . . . . . . . . . . . . . . . . . . . . 183
9.4 GraphicalModels . . . . . . . . . . . . . . . . . . . . . . . . . . 184
9.4.1 ExampleBeliefNetworks . . . . . . . . . . . . . . . . . . 186
9.5 InferenceMethods . . . . . . . . . . . . . . . . . . . . . . . . . 191
9.6 RecursiveConditioning . . . . . . . . . . . . . . . . . . . . . . 192
9.7 VariableElimination . . . . . . . . . . . . . . . . . . . . . . . . 197
9.8 StochasticSimulation . . . . . . . . . . . . . . . . . . . . . . . . 201
9.8.1 Samplingfromadiscretedistribution . . . . . . . . . . . 201
9.8.2 SamplingMethodsforBeliefNetworkInference . . . . . 202
9.8.3 RejectionSampling . . . . . . . . . . . . . . . . . . . . . . 203
9.8.4 LikelihoodWeighting . . . . . . . . . . . . . . . . . . . . 204
9.8.5 ParticleFiltering . . . . . . . . . . . . . . . . . . . . . . . 205
9.8.6 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . 206
9.8.7 GibbsSampling . . . . . . . . . . . . . . . . . . . . . . . . 208
9.8.8 PlottingBehaviourofStochasticSimulators . . . . . . . . 209
9.9 HiddenMarkovModels . . . . . . . . . . . . . . . . . . . . . . 211
9.9.1 ExactFilteringforHMMs . . . . . . . . . . . . . . . . . . 213
9.9.2 Localization . . . . . . . . . . . . . . . . . . . . . . . . . . 215
9.9.3 ParticleFilteringforHMMs . . . . . . . . . . . . . . . . . 217
9.9.4 GeneratingExamples . . . . . . . . . . . . . . . . . . . . 219
9.10 DynamicBeliefNetworks . . . . . . . . . . . . . . . . . . . . . 220
9.10.1 RepresentingDynamicBeliefNetworks . . . . . . . . . . 220
9.10.2 UnrollingDBNs . . . . . . . . . . . . . . . . . . . . . . . . 224
9.10.3 DBNFiltering . . . . . . . . . . . . . . . . . . . . . . . . . 224
10 LearningwithUncertainty 227
10.1 K-means . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 227
10.2 EM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 231
11 Causality 237
11.1 DoQuestions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 237
12 PlanningwithUncertainty 239
12.1 DecisionNetworks . . . . . . . . . . . . . . . . . . . . . . . . . 239
12.1.1 ExampleDecisionNetworks . . . . . . . . . . . . . . . . 241
12.1.2 RecursiveConditioningfordecisionnetworks . . . . . . 246
12.1.3 Variableeliminationfordecisionnetworks . . . . . . . . 250
12.2 MarkovDecisionProcesses . . . . . . . . . . . . . . . . . . . . 252
12.2.1 ValueIteration . . . . . . . . . . . . . . . . . . . . . . . . 255
12.2.2 ShowingGridMDPs . . . . . . . . . . . . . . . . . . . . . 256
12.2.3 AsynchronousValueIteration. . . . . . . . . . . . . . . . 259
13 ReinforcementLearning 263
13.1 RepresentingAgentsandEnvironments . . . . . . . . . . . . . 263
13.1.1 SimulatinganenvironmentfromanMDP . . . . . . . . . 264
http://aipython.org
Version0.9.5 November22,2022
Contents 7
13.1.2 MonsterGame . . . . . . . . . . . . . . . . . . . . . . . . 265
13.1.3 EvaluationandPlotting . . . . . . . . . . . . . . . . . . . 267
13.2 QLearning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 269
13.2.1 TestingQ-learning . . . . . . . . . . . . . . . . . . . . . . 271
13.3 Q-leaningwithExperienceReplay . . . . . . . . . . . . . . . . 272
13.4 Model-basedReinforcementLearner . . . . . . . . . . . . . . . 274
13.5 ReinforcementLearningwithFeatures . . . . . . . . . . . . . . 277
13.5.1 RepresentingFeatures . . . . . . . . . . . . . . . . . . . . 277
13.5.2 Feature-basedRLlearner . . . . . . . . . . . . . . . . . . 280
13.5.3 ExperienceReplay . . . . . . . . . . . . . . . . . . . . . . 283
14 MultiagentSystems 285
14.1 Minimax . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 285
14.1.1 Creatingatwo-playergame . . . . . . . . . . . . . . . . . 285
14.1.2 Minimaxandα-βPruning . . . . . . . . . . . . . . . . . . 288
14.2 MultiagentLearning . . . . . . . . . . . . . . . . . . . . . . . . 290
15 RelationalLearning 297
15.1 CollaborativeFiltering . . . . . . . . . . . . . . . . . . . . . . . 297
15.1.1 AlternativeFormulation . . . . . . . . . . . . . . . . . . . 300
15.1.2 Plotting . . . . . . . . . . . . . . . . . . . . . . . . . . . . 300
15.1.3 CreatingRatingSets . . . . . . . . . . . . . . . . . . . . . 301
16 VersionHistory 305
Bibliography 307
http://aipython.org
Version0.9.5 November22,2022

Chapter 1
Python for Artificial Intelligence
1.1 Why Python?
We use Python because Python programs can be close to pseudo-code. It is
designedforhumanstoread.
Pythonisreasonablyefficient. Efficiencyisusuallynotaproblemforsmall
examples. If your Python code is not efficient enough, a general procedure
to improve it is to find out what is taking most the time, and implement just
that part more efficiently in some lower-level language. Most of these lower-
level languages interoperate with Python nicely. This will result in much less
programming and more efficient code (because you will have more time to
optimize) than writing everything in a low-level language. You will not have
todothatforthecodehereifyouareusingitforcourseprojects.
1.2 Getting Python
http://python.org/ http://matplotlib.
YouneedPython3( )andmatplotlib(
org/)thatrunswithPython3. ThiscodeisnotcompatiblewithPython2(e.g.,
withPython2.7).
http://python.org/
Download and istall the latest Python 3 release from .
Thisshouldalsoinstallpip3. Youcaninstallmatplotlibusing
pip3 install matplotlib
in a terminal shell (not in Python). That should “just work”. If not, try using
pip pip3
insteadof .
python python3
The command or should then start the interactive python
quit()
shell. YoucanquitPythonwithacontrol-Dorwith .
9
10 1. PythonforArtificialIntelligence
To upgrade matplotlib to the latest version (which you should do if you
installanewversionofPython)do:
pip3 install --upgrade matplotlib
We recommend using the enhanced interactive python ipython (http://
ipython.org/
). Toinstallipythonafteryouhaveinstalledpythondo:
pip3 install ipython
1.3 Running Python
We assume that everything is done with an interactive Python shell. You can
either do this with an IDE, such as IDLE that comes with standard Python
distributions,orjustrunningipython3(orperhapsjustipython)fromashell.
Here we describe the most simple version that uses no IDE. If you down-
aipython
load the zip file, and cd to the “ ” folder where the .py files are, you
:
should be able to do the following, with user input following . The first
-i
ipython3commandisintheoperatingsystemshell(notethatthe isimpor-
tanttoenterinteractivemode),withuserinputinbold:
ipython -i searchGeneric.py
Python 3.6.5 (v3.6.5:f59c0932b4, Mar 28 2018, 05:52:31)
Type 'copyright', 'credits' or 'license' for more information
IPython 6.2.1 -- An enhanced Interactive Python. Type '?' for help.
Testing problem 1:
7 paths have been expanded and 4 paths remain in the frontier
Path found: a --> b --> c --> d --> g
Passed unit test
In [1]: searcher2 = AStarSearcher(searchProblem.acyclic_delivery_problem) #A*
In [2]: searcher2.search() # find first path
16 paths have been expanded and 5 paths remain in the frontier
Out[2]: o103 --> o109 --> o119 --> o123 --> r123
In [3]: searcher2.search() # find next path
21 paths have been expanded and 6 paths remain in the frontier
Out[3]: o103 --> b3 --> b4 --> o109 --> o119 --> o123 --> r123
In [4]: searcher2.search() # find next path
28 paths have been expanded and 5 paths remain in the frontier
Out[4]: o103 --> b3 --> b1 --> b2 --> b4 --> o109 --> o119 --> o123 --> r123
In [5]: searcher2.search() # find next path
No (more) solutions. Total of 33 paths expanded.
http://aipython.org
Version0.9.5 November22,2022