Automata and Computability: A Programmer’s Perspective

349 Pages·2019·17.72 MB·English

by  Ganesh Gopalakrishnan

Automata and Computability
A Programmer’s Perspective

Automata and Computability
A Programmer’s Perspective
Ganesh Lalitha Gopalakrishnan
Cover Photo Credit: NASA, ESA
This image combines an image taken with Hubble Space Telescope in the optical (taken in spring 2014) and observations of its auroras in the ultraviolet,
taken in 2016. URL: https://www.spacetelescope.org/images/heic1613a/
CRC Press
Taylor & Francis Group
6000 Broken Sound Parkway NW, Suite 300
Boca Raton, FL 33487-2742
© 2019 by Taylor & Francis Group, LLC
CRC Press is an imprint of Taylor & Francis Group, an Informa business
No claim to original U.S. Government works
Printed on acid-free paper
Version Date: 20190206
International Standard Book Number-13: 978-1-138-55242-5 (Hardback)
This book contains information obtained from authentic and highly regarded sources. Reasonable efforts have been made to publish reliable data and
information, but the author and publisher cannot assume responsibility for the validity of all materials or the consequences of their use. The authors and
publishers have attempted to trace the copyright holders of all material reproduced in this publication and apologize to copyright holders if permission
to publish in this form has not been obtained. If any copyright material has not been acknowledged please write and let us know so we may rectify in any
future reprint.
Except as permitted under U.S. Copyright Law, no part of this book may be reprinted, reproduced, transmitted, or utilized in any form by any electronic,
mechanical, or other means, now known or hereafter invented, including photocopying, microfilming, and recording, or in any information storage or
retrieval system, without written permission from the publishers.
For permission to photocopy or use material electronically from this work, please access www.copyright.com (http://www.copyright.com/) or contact
the Copyright Clearance Center, Inc. (CCC), 222 Rosewood Drive, Danvers, MA 01923, 978-750-8400. CCC is a not-for-profit organization that provides
licenses and registration for a variety of users. For organizations that have been granted a photocopy license by the CCC, a separate system of payment
has been arranged.
Trademark Notice: Product or corporate names may be trademarks or registered trademarks, and are used only for identification and explanation
without intent to infringe.
Visit the Taylor & Francis Web site at
http://www.taylorandfrancis.com
and the CRC Press Web site at
http://www.crcpress.com
Contents
Foreword xiii
Preface xv
Acknowledgments xix
1 WhatMachinesThink 3
1.1 ProblemsWithoutAlgorithms . . . . . . . . . . . . . . . . . . 4
1.2 HowtoDefineaComputer? . . . . . . . . . . . . . . . . . . . . 5
1.3 PracticalApplication: SyntaxDefinition/Checking . . . . . . 7
1.4 SimplifiedTuringMachinesasParsers. . . . . . . . . . . . . 11
1.5 AutomataandComputabilityforLifelongLearning . . . . . 13
2 DefiningLanguages: PatternsinSetsofStrings 15
2.1 Symbol,Alphabet,String,Language . . . . . . . . . . . . . . 15
2.1.1 Symbol . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
2.1.2 Alphabet. . . . . . . . . . . . . . . . . . . . . . . . . . . 15
2.1.3 StringorWord . . . . . . . . . . . . . . . . . . . . . . . 16
2.1.4 VariousNotionsofZeroandOne . . . . . . . . . . . . 17
2.2 Language . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
2.2.1 LanguageConcatenation . . . . . . . . . . . . . . . . . 20
2.2.2 TheZeroandOneforLanguageConcatenation . . . 21
2.2.3 ZeroandOneofLanguageConcatenationinPython 21
2.2.4 ExponentiationofaLanguage . . . . . . . . . . . . . . 22
2.2.5 PythonEncodingofLanguageExponentiation . . . . 23
2.2.6 UnionandIntersectionofLanguages . . . . . . . . . 24
2.3 UsefulResults,SlipperyRoads . . . . . . . . . . . . . . . . . 25
3 KleeneStar:BasicMethodofDefiningRepetitiousPatterns 27
3.1 ThreeWaystoDescribeStar . . . . . . . . . . . . . . . . . . . 27
3.2 AdditionalDefinitionsandPropertiesofStar . . . . . . . . . 28
3.3 LanguageComplementation . . . . . . . . . . . . . . . . . . . 31
3.4 OtherLanguageOperations . . . . . . . . . . . . . . . . . . . 32
3.4.1 SymmetricDifference,Subtraction . . . . . . . . . . . 32
3.4.2 ReverseofaLanguage . . . . . . . . . . . . . . . . . . 32
vi
3.5 String/LanguageHomomorphisms . . . . . . . . . . . . . . . 33
3.5.1 TakingStarRepeatedly. . . . . . . . . . . . . . . . . . 35
3.6 EnumeratingStringsinaLanguage . . . . . . . . . . . . . . 35
4 BasicsofDFA 41
4.1 DFAEverywhere . . . . . . . . . . . . . . . . . . . . . . . . . . 41
4.2 ElementsofaDFA . . . . . . . . . . . . . . . . . . . . . . . . . 42
4.3 FormalStructureofDFA . . . . . . . . . . . . . . . . . . . . . 43
4.4 TheLanguageofaDFA . . . . . . . . . . . . . . . . . . . . . . 44
4.4.1 DFAasStringClassifers . . . . . . . . . . . . . . . . . 44
4.4.2 BasicsofDesigningaDFA . . . . . . . . . . . . . . . . 45
4.5 FormalDefinitionofDFALanguageAcceptance . . . . . . . 45
4.6 “Lasso”ShapeofDFAandthePumpingLemma . . . . . . . 47
4.6.1 GeneralStatementofthePumpingLemma. . . . . . 48
4.7 ProvingaLanguagetoBeNon-Regular . . . . . . . . . . . . 50
4.7.1 WhyAllSplitsofx,y,z?. . . . . . . . . . . . . . . . . . 51
4.8 GrosslyAbusingthePumpingLemma . . . . . . . . . . . . . 52
4.8.1 InabilitytoProvewiththisPumpingLemma. . . . . 53
4.9 Regularity-PreservingTransformationsAidProofs . . . . . 54
5 DesigningDFA 57
5.1 UnderstandingtheLanguagetoBeRealized . . . . . . . . . 57
5.1.1 TheLanguageofEqualChanges . . . . . . . . . . . . 57
5.1.2 BestPracticestoCorrectDFADesignandVerification 58
5.2 ExamplesofDesigningDFA . . . . . . . . . . . . . . . . . . . 59
5.2.1 TheLanguageofBlocksof3 . . . . . . . . . . . . . . . 59
5.2.2 DFAfor“Endswith0101” . . . . . . . . . . . . . . . . 60
5.2.3 DFAfor“MSB/LSB-firstBinaryNumberisDivisible
by3” . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
5.2.4 DFAfor“Third-lastbitisa1” . . . . . . . . . . . . . . 62
5.3 Automd: AMarkdownLanguageforAllMachines . . . . . . 64
5.3.1 MarkdownforDFA . . . . . . . . . . . . . . . . . . . . 64
6 OperationsonDFA 67
6.1 ComplementationofDFA . . . . . . . . . . . . . . . . . . . . . 67
6.2 UnionandIntersectionofDFA . . . . . . . . . . . . . . . . . 67
6.3 LanguageEquivalenceandIsomorphism . . . . . . . . . . . 71
6.4 DFAMinimizationandMyhill-NerodeTheorem . . . . . . . 72
6.4.1 FullyWorked-outExampleofDFAMinimization . . 72
6.4.2 SalientCodeExcerpts. . . . . . . . . . . . . . . . . . . 75
6.5 ExamplesofLanguageDesignandManipulation . . . . . . 76
6.5.1 UseofUnion,Minimization,andLanguage
Equivalence . . . . . . . . . . . . . . . . . . . . . . . . . 77
6.5.2 UseofDeMorgan’sLaw. . . . . . . . . . . . . . . . . . 78
vii
7 NondeterministicFiniteAutomata 81
7.1 OverviewofNFA . . . . . . . . . . . . . . . . . . . . . . . . . . 81
7.2 FormalDescriptionofNFA . . . . . . . . . . . . . . . . . . . . 83
7.3 LanguageofanNFA:ExampleDriven . . . . . . . . . . . . . 84
7.3.1 SimulationsoftheNFAofFigure7.5 . . . . . . . . . 84
7.3.2 SimulationsoftheNFAofFigure7.3 . . . . . . . . . 85
7.4 LanguageofanNFA:viaEclosure . . . . . . . . . . . . . . . 85
7.4.1 DefiningEclosure . . . . . . . . . . . . . . . . . . . . . 86
7.4.2 Definitionofδandδˆ. . . . . . . . . . . . . . . . . . . . 86
7.5 NFAtoDFAConversionthroughSubsetConstruction . . . 87
7.6 Brzozowski’sDFAMinimizationAlgorithm . . . . . . . . . . 90
7.6.1 ReversalofaDFAYieldsanNFA . . . . . . . . . . . . 90
7.7 ACompleteIllustrationofBrzozowski’sMinimization . . . 92
8 RegularExpressionsandNFA 95
8.1 RegularExpressions . . . . . . . . . . . . . . . . . . . . . . . . 95
8.2 REtoNFAConversion: Examples,AlgorithmSketch . . . . 96
8.3 AMoreExtensiveExample . . . . . . . . . . . . . . . . . . . . 100
8.4 RegularExpressions: Ubiquitous,yetError-Prone . . . . . . 101
8.5 AnatomyoftheREtoNFAConverter . . . . . . . . . . . . . 103
8.6 Example: DesigninganError-CorrectingDFA . . . . . . . . 106
8.6.1 Error-correcting RE for “within Hamming Distance
of2” . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106
8.6.2 NFA-basedDesignof“withinHammingDistanceof2”108
8.7 MinimalDFAandIsomorphism . . . . . . . . . . . . . . . . . 108
8.8 DFAUltimatePeriodicitytoSolvethePostageStamp
Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111
8.8.1 Ultimately periodic sets and lengths of members of
aregularlanguage. . . . . . . . . . . . . . . . . . . . . 111
8.8.2 StampProblemandUltimatePeriodicityviaJove . . 112
8.8.3 Applyingtonumbersthatarenotrelativelyprime . 113
8.8.4 Solvingforthreestamps . . . . . . . . . . . . . . . . . 113
8.8.5 LengthsofstringsacceptedbyDFA . . . . . . . . . . 113
9 NFAtoREConversion 115
9.1 NFAtoREConversionAlgorithm . . . . . . . . . . . . . . . . 115
9.2 IllustrationonPedagogicalExample . . . . . . . . . . . . . . 116
9.3 IllustrationonNon-trivialExample . . . . . . . . . . . . . . 118
9.4 CheckingtheConversion . . . . . . . . . . . . . . . . . . . . . 121
9.5 DFA,NFA,andREAreEquallyPowerful . . . . . . . . . . . 122
9.6 ImplementationofNFAtoRE . . . . . . . . . . . . . . . . . . 123
9.7 ClosureResultsPertainingtoRegularLanguages . . . . . . 123
10 Derivative-BasedRegularExpressionMatching 127
10.1 IntroductiontoREDerivatives . . . . . . . . . . . . . . . . . 127
viii
10.2 Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129
10.2.1 DerivativeRules . . . . . . . . . . . . . . . . . . . . . . 129
10.2.2 NullabilityRules . . . . . . . . . . . . . . . . . . . . . . 132
10.3 ImplementationofDerivative-BasedStringMatching. . . . 134
10.3.1 Derivatives: ClosingThoughts . . . . . . . . . . . . . 134
11 Context-FreeLanguages
andGrammars 137
11.1 Context-FreeLanguageExamples . . . . . . . . . . . . . . . 137
11.2 Context-FreeGrammarsandParseTrees . . . . . . . . . . . 138
11.2.1 ElementsofContext-FreeGrammars . . . . . . . . . 139
11.2.2 ParseTrees,LanguageofaCFG . . . . . . . . . . . . 139
11.3 AvoidingMistakesinDesigningCFGs . . . . . . . . . . . . . 140
11.3.1 CompletenessandConsistency . . . . . . . . . . . . . 141
11.4 TheDesignofCFG,andtheHill/ValleyPlot
forArguingConsistencyandCompleteness . . . . . . . . . . 142
11.5 AmbiguousGrammars,andDisambiguation . . . . . . . . . 144
11.5.1 Disambiguation . . . . . . . . . . . . . . . . . . . . . . 145
11.5.2 DisambiguationIsCrucial!. . . . . . . . . . . . . . . . 147
11.5.3 ImpossibilityResults . . . . . . . . . . . . . . . . . . . 147
11.6 InherentlyAmbiguousLanguages. . . . . . . . . . . . . . . . 147
11.7 ExpressingDFAviaCFGs . . . . . . . . . . . . . . . . . . . . 149
11.7.1 PurelyRightLinearGrammars . . . . . . . . . . . . . 150
11.7.2 Closure,PurelyLeftLinear,andMixedLinearity . . 151
11.8 HistoricalImportanceoftheTheoryofParsing . . . . . . . . 152
11.8.1 CombatingInherentAmbiguity . . . . . . . . . . . . . 153
11.9 APumpingLemmaforCFLs . . . . . . . . . . . . . . . . . . . 154
11.9.1 ApplicationoftheCFLPumpingLemma . . . . . . . 157
11.10 TheComplementofaNon-CFLCanBeaCFL . . . . . . . 157
11.10.1 Growing“Inside-Out” . . . . . . . . . . . . . . . . . . 158
12 PushdownAutomata 161
12.1 PushdownAutomatonBasics . . . . . . . . . . . . . . . . . . 161
12.2 FormalDescriptionofPDA . . . . . . . . . . . . . . . . . . . . 164
12.2.1 Acceptance,DeterministicPDA . . . . . . . . . . . . . 165
12.3 ExploringthePDAforL UsingJove. . . . . . . . . . . . 165
Dyck
12.4 PDABehaviorThroughExamples. . . . . . . . . . . . . . . . 167
12.4.1 Rerunningpda6withLargerStackAllowed . . . . . 171
12.5 TowardMorePracticalPDA . . . . . . . . . . . . . . . . . . . 173
12.6 CFGtoPDAConversion . . . . . . . . . . . . . . . . . . . . . 175
12.6.1 Disambiguation . . . . . . . . . . . . . . . . . . . . . . 179
12.7 PracticalKnowledgeImpartedbyJove: ThreeParsers . . . 181
13 TuringMachines 183
13.1 BriefHistoryofTuringMachines . . . . . . . . . . . . . . . . 183
ix
13.2 UniversalComputingDevices . . . . . . . . . . . . . . . . . . 184
13.3 FormalDefinitionofTuringMachines . . . . . . . . . . . . . 186
13.4 ExamplesofSimpleTMs . . . . . . . . . . . . . . . . . . . . . 189
13.4.1 ASimpleDTMthatFlipsBits . . . . . . . . . . . . . . 189
13.4.2 TMsthatcheckifastringcontains101 . . . . . . . . 189
13.5 ADTMforw#wandanNDTMforww . . . . . . . . . . . . . 193
13.5.1 ADTMRecognizingw#w . . . . . . . . . . . . . . . . . 193
13.5.2 ANondeterministicTMRecognizingww . . . . . . . 193
13.5.3 NondeterminismdoesnotincreaseaTM’s
ExpressivePower . . . . . . . . . . . . . . . . . . . . . 194
13.6 Example: ATMthatWorksontheCollatzProblem . . . . . 200
13.6.1 MarkdownfortheCollatzProblemTMwith
Comments. . . . . . . . . . . . . . . . . . . . . . . . . . 200
13.7 TheChomskyHierarchy . . . . . . . . . . . . . . . . . . . . . 203
13.7.1 RecursivelyEnumerableandRecursiveLanguages . 204
13.8 AnAlternateNotationforInstantaneousDescriptions . . . 205
14 InterplaybetweenFormalLanguages 209
14.1 WhyStudyImpossibilityResults?. . . . . . . . . . . . . . . . 209
14.1.1 Definitions: ProcedureandAlgorithm . . . . . . . . . 209
14.1.2 FormalLanguagesAssociatedwithTuringMachines 210
14.1.3 AllayingConfusion: Languagevs. LanguageFamily 210
14.2 OneExampleofaRecursiveandanRELanguage . . . . . . 211
14.2.1 ARecursiveLanguage . . . . . . . . . . . . . . . . . . 211
14.2.2 PrerequisitestoDefiningtheNotionofRE . . . . . . 212
14.2.3 CombiningSemi-decidersforLandL . . . . . . . . . 213
14.2.4 The“nowimp”Clause. . . . . . . . . . . . . . . . . . . 213
14.2.5 ARecursivelyEnumerableLanguage . . . . . . . . . 213
14.3 OtherExamplesofRecursiveandRELanguages . . . . . . 215
14.3.1 RELanguagesthatarenotRecursive . . . . . . . . . 216
14.3.2 WhyisitcalledRecursivelyEnumerable? . . . . . . 217
14.3.3 Alternateproofof A beingRE . . . . . . . . . . . . 218
TM
14.3.4 SomeMoreRELanguages . . . . . . . . . . . . . . . . 218
14.4 SummaryofDecidability/Semi-DecidabilityResults . . . . . 219
14.4.1 ExistenceofNon-RElanguages . . . . . . . . . . . . . 220
14.5 REandRecursiveSets: MoreHigh-LevelProofSketches . . 221
14.5.1 LanguageofDFAD whereL(D)=Σ∗ isRecursive. . 221
14.5.2 LanguageofCFGG whereL(G)=(cid:59)isRecursive . . 222
14.5.3 LanguageofLBAthathaltoninputwisRecursive . 223
14.5.4 LanguageofTuringmachineswhosefirstoutputis‘3’224
15 PostCorrespondence,and
OtherUndecidabilityProofs 227
15.1 PostCorrespondence: “Drosophila”forDecidability . . . . . 227