Practical Artificial Intelligence Programming With Java

222 Pages·2008·1.24 MB·English

Practical Artificial Intelligence
Programming With Java
Third Edition
Mark Watson
Copyright 2001-2008 Mark Watson. All rights reserved.
This work is licensed under a Creative Commons
Attribution-Noncommercial-No Derivative Works
Version 3.0 United States License.
November 11, 2008

Contents
Preface xi
1 Introduction 1
1.1 OtherJVMLanguages . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 WhyisaPDFVersionofthisBookAvailableFreeontheWeb?. . . 1
1.3 BookSoftware . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.4 UseofJavaGenericsandNativeTypes . . . . . . . . . . . . . . . . 2
1.5 NotesonJavaCodingStylesUsedinthisBook . . . . . . . . . . . 3
1.6 BookSummary . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
2 Search 5
2.1 RepresentationofSearchStateSpaceandSearchOperators . . . . . 5
2.2 FindingPathsinMazes . . . . . . . . . . . . . . . . . . . . . . . . 6
2.3 FindingPathsinGraphs. . . . . . . . . . . . . . . . . . . . . . . . 13
2.4 AddingHeuristicstoBreadthFirstSearch . . . . . . . . . . . . . . 22
2.5 SearchandGamePlaying . . . . . . . . . . . . . . . . . . . . . . . 22
2.5.1 Alpha-BetaSearch . . . . . . . . . . . . . . . . . . . . . . 22
2.5.2 AJavaFrameworkforSearchandGamePlaying . . . . . . 24
2.5.3 Tic-Tac-ToeUsingtheAlpha-BetaSearchAlgorithm . . . . 29
2.5.4 ChessUsingtheAlpha-BetaSearchAlgorithm . . . . . . . 34
3 Reasoning 45
3.1 Logic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
3.1.1 HistoryofLogic . . . . . . . . . . . . . . . . . . . . . . . 47
3.1.2 ExamplesofDifferentLogicTypes . . . . . . . . . . . . . 47
3.2 PowerLoomOverview . . . . . . . . . . . . . . . . . . . . . . . . 48
3.3 RunningPowerLoomInteractively . . . . . . . . . . . . . . . . . . 49
3.4 UsingthePowerLoomAPIsinJavaPrograms . . . . . . . . . . . . 52
3.5 SuggestionsforFurtherStudy . . . . . . . . . . . . . . . . . . . . 54
4 SemanticWeb 57
4.1 RelationalDatabaseModelHasProblemsDealingwithRapidlyChang-
ingDataRequirements . . . . . . . . . . . . . . . . . . . . . . . . 58
4.2 RDF:TheUniversalDataFormat . . . . . . . . . . . . . . . . . . . 59
4.3 ExtendingRDFwithRDFSchema . . . . . . . . . . . . . . . . . . 62
4.4 TheSPARQLQueryLanguage . . . . . . . . . . . . . . . . . . . . 63
4.5 UsingSesame . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
iii
Contents
4.6 OWL:TheWebOntologyLanguage . . . . . . . . . . . . . . . . . 69
4.7 KnowledgeRepresentationandREST . . . . . . . . . . . . . . . . 71
4.8 MaterialforFurtherStudy . . . . . . . . . . . . . . . . . . . . . . 72
5 ExpertSystems 73
5.1 ProductionSystems . . . . . . . . . . . . . . . . . . . . . . . . . . 75
5.2 TheDroolsRulesLanguage . . . . . . . . . . . . . . . . . . . . . 75
5.3 UsingDroolsinJavaApplications . . . . . . . . . . . . . . . . . . 77
5.4 ExampleDroolsExpertSystem: BlocksWorld . . . . . . . . . . . 81
5.4.1 POJOObjectModelsforBlocksWorldExample . . . . . . 82
5.4.2 DroolsRulesforBlocksWorldExample. . . . . . . . . . . 85
5.4.3 JavaCodeforBlocksWorldExample . . . . . . . . . . . . 88
5.5 ExampleDroolsExpertSystem: HelpDeskSystem . . . . . . . . . 90
5.5.1 ObjectModelsforanExampleHelpDesk . . . . . . . . . . 91
5.5.2 DroolsRulesforanExampleHelpDesk . . . . . . . . . . . 93
5.5.3 JavaCodeforanExampleHelpDesk . . . . . . . . . . . . 95
5.6 NotesontheCraftofBuildingExpertSystems. . . . . . . . . . . . 97
6 GeneticAlgorithms 99
6.1 Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
6.2 JavaLibraryforGeneticAlgorithms . . . . . . . . . . . . . . . . . 101
6.3 FindingtheMaximumValueofaFunction . . . . . . . . . . . . . . 105
7 NeuralNetworks 109
7.1 HopfieldNeuralNetworks . . . . . . . . . . . . . . . . . . . . . . 110
7.2 JavaClassesforHopfieldNeuralNetworks . . . . . . . . . . . . . 111
7.3 TestingtheHopfieldNeuralNetworkClass . . . . . . . . . . . . . 114
7.4 BackPropagationNeuralNetworks . . . . . . . . . . . . . . . . . 116
7.5 AJavaClassLibraryforBackPropagation. . . . . . . . . . . . . . 119
7.6 AddingMomentumtoSpeedUpBack-PropTraining . . . . . . . . 127
8 MachineLearningwithWeka 129
8.1 UsingWeka’sInteractiveGUIApplication . . . . . . . . . . . . . . 130
8.2 InteractiveCommandLineUseofWeka . . . . . . . . . . . . . . . 132
8.3 EmbeddingWekainaJavaApplication . . . . . . . . . . . . . . . 134
8.4 SuggestionsforFurtherStudy . . . . . . . . . . . . . . . . . . . . 136
9 StatisticalNaturalLanguageProcessing 137
9.1 Tokenizing,Stemming,andPartofSpeechTaggingText . . . . . . 137
9.2 NamedEntityExtractionFromText . . . . . . . . . . . . . . . . . 141
9.3 UsingtheWordNetLinguisticDatabase . . . . . . . . . . . . . . . 144
9.3.1 TutorialonWordNet . . . . . . . . . . . . . . . . . . . . . 144
9.3.2 ExampleUseoftheJAWSWordNetLibrary . . . . . . . . 145
9.3.3 Suggested Project: Using a Part of Speech Tagger to Use
theCorrectWordNetSynonyms . . . . . . . . . . . . . . . 149
iv
Contents
9.3.4 Suggested Project: Using WordNet Synonyms to Improve
DocumentClustering . . . . . . . . . . . . . . . . . . . . . 150
9.4 AutomaticallyAssigningTagstoText . . . . . . . . . . . . . . . . 150
9.5 TextClustering . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152
9.6 SpellingCorrection . . . . . . . . . . . . . . . . . . . . . . . . . . 156
9.6.1 GNUASpellLibraryandJazzy . . . . . . . . . . . . . . . 157
9.6.2 PeterNorvig’sSpellingAlgorithm . . . . . . . . . . . . . . 158
9.6.3 ExtendingtheNorvigAlgorithmbyUsingWordPairStatistics162
9.7 HiddenMarkovModels . . . . . . . . . . . . . . . . . . . . . . . . 166
9.7.1 TrainingHiddenMarkovModels. . . . . . . . . . . . . . . 168
9.7.2 UsingtheTrainedMarkovModeltoTagText . . . . . . . . 173
10 InformationGathering 177
10.1 OpenCalais . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177
10.2 InformationDiscoveryinRelationalDatabases . . . . . . . . . . . 181
10.2.1 CreatingaTestDerbyDatabaseUsingtheCIAWorldFact-
BookandDataonUSStates . . . . . . . . . . . . . . . . . 182
10.2.2 UsingtheJDBCMetaDataAPIs. . . . . . . . . . . . . . . 183
10.2.3 UsingtheMetaDataAPIstoDiscernEntityRelationships . 187
10.3 DowntotheBareMetal: In-MemoryIndexandSearch . . . . . . . 187
10.4 IndexingandSearchUsingEmbeddedLucene . . . . . . . . . . . . 193
10.5 IndexingandSearchwithNutchClients . . . . . . . . . . . . . . . 197
10.5.1 NutchServerFastStartSetup . . . . . . . . . . . . . . . . 198
10.5.2 UsingtheNutchOpenSearchWebAPIs . . . . . . . . . . . 201
11 Conclusions 207
v
Contents
vi
List of Figures
2.1 A directed graph representation is shown on the left and a two-
dimensionalgrid(ormaze)representationisshownontheright. In
bothrepresentations,theletterRisusedtorepresentthecurrentpo-
sition(orreferencepoint)andthearrowheadsindicatelegalmoves
generatedbyasearchoperator. Inthemazerepresentation,thetwo
grid cells marked with an X indicate that a search operator cannot
generatethisgridlocation. . . . . . . . . . . . . . . . . . . . . . . 7
2.2 UMLclassdiagramforthemazesearchJavaclasses . . . . . . . . 8
2.3 Usingdepthfirstsearchtofindapathinamazefindsanon-optimal
solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
2.4 Usingbreadthfirstsearchinamazetofindanoptimalsolution . . . 14
2.5 UMLclassdiagramforthegraphsearchclasses . . . . . . . . . . . 15
2.6 Usingdepthfirstsearchinasamplegraph . . . . . . . . . . . . . . 21
2.7 Usingbreadthfirstsearchinasamplegraph . . . . . . . . . . . . . 21
2.8 Alpha-betaalgorithmappliedtopartofagameoftic-tac-toe . . . . 23
2.9 UMLclassdiagramsforgamesearchengineandtic-tac-toe . . . . . 30
2.10 UMLclassdiagramsforgamesearchengineandchess . . . . . . . 35
2.11 Theexamplechessprogramdoesnotcontainanopeningbooksoit
playstomaximizethemobilityofitspiecesandmaximizematerial
advantage using a two-move lookahead. The first version of the
chessprogramcontainsafewheuristicslikewantingtocontrolthe
centerfoursquares. . . . . . . . . . . . . . . . . . . . . . . . . . . 36
2.12 Continuing the first sample game: the computer is looking ahead
twomovesandnoopeningbookisused. . . . . . . . . . . . . . . . 37
2.13 Secondgamewitha21/2movelookahead. . . . . . . . . . . . . . 41
2.14 Continuingthesecondgamewithatwoandahalfmovelookahead.
Wewilladdmoreheuristicstothestaticevaluationmethodtoreduce
thevalueofmovingthequeenearlyinthegame. . . . . . . . . . . 42
3.1 OverviewofhowwewillusePowerLoomfordevelopmentandde-
ployment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
4.1 LayersofdatamodelsusedinimplementingSemanticWebapplica-
tions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
4.2 JavautilityclassesandinterfaceforusingSesame . . . . . . . . . . 68
vii
ListofFigures
5.1 UsingDroolsfordevelopingrule-basedsystemsandthendeploying
them. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
5.2 Initialstateofablocksworldproblemwiththreeblocksstackedon
topofeachother. ThegoalistomovetheblockssothatblockCis
ontopofblockA. . . . . . . . . . . . . . . . . . . . . . . . . . . . 82
5.3 BlockChasbeenremovedfromblockBandplacedonthetable. . . 82
5.4 BlockBhasbeenremovedfromblockAandplacedonthetable. . . 84
5.5 ThegoalissolvedbyplacingblockContopofblockA. . . . . . . 85
6.1 Thetestfunctionevaluatedovertheinterval[0.0,10.0]. Themaxi-
mumvalueof0.56occursatx=3.8 . . . . . . . . . . . . . . . . . . 100
6.2 Crossoveroperation . . . . . . . . . . . . . . . . . . . . . . . . . . 101
7.1 Physicalstructureofaneuron. . . . . . . . . . . . . . . . . . . . . 110
7.2 Two views of the same two-layer neural network; the view on the
right shows the connection weights between the input and output
layersasatwo-dimensionalarray. . . . . . . . . . . . . . . . . . . 117
7.3 SigmoidandderivativeoftheSigmoid(SigmoidP)functions. This
plotwasproducedbythefilesrc-neural-networks/Graph.java. . . . 118
7.4 Capabilities of zero, one, and two hidden neuron layer neural net-
works. The grayed areas depict one of two possible output values
based on two input neuron activation values. Note that this is a
two-dimensional case for visualization purposes; if a network had
teninputneuronsinsteadoftwo,thentheseplotswouldhavetobe
ten-dimensionalinsteadoftwo-dimensional. . . . . . . . . . . . . . 119
7.5 Examplebackpropagationneuralnetworkwithonehiddenlayer. . . 120
7.6 Examplebackpropagationneuralnetworkwithtwohiddenlayers. . 120
8.1 RunningtheWekaDataExplorer . . . . . . . . . . . . . . . . . . . 131
8.2 RunningtheWekaDataExplorer . . . . . . . . . . . . . . . . . . . 131
viii
List of Tables
2.1 RuntimesbyMethodforChessProgram . . . . . . . . . . . . . . . 44
6.1 Randomchromosomesandthefloatingpointnumbersthattheyencode106
9.1 Mostcommonlyusedpartofspeechtags . . . . . . . . . . . . . . . 139
9.2 Samplepartofspeechtags . . . . . . . . . . . . . . . . . . . . . . 167
9.3 Transitioncountsfromthefirsttag(showninrow)tothesecondtag
(shown in column). We see that the transition from NNP to VB is
common. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169
9.4 Normalize data in Table 9.3 to get probability of one tag (seen in
row)transitioningtoanothertag(seenincolumn) . . . . . . . . . . 171
9.5 Probabilities of words having specific tags. Only a few tags are
showninthistable. . . . . . . . . . . . . . . . . . . . . . . . . . . 172
ix
ListofTables
x