Introduction to Computing: Explorations in Language, Logic, and Machines

2011·4.4 MB·English

by  David Evans

Introduction to
Computing
Explorations in
Language, Logic, and Machines
David Evans
University of Virginia
Forthelatestversionofthisbookandsupplementarymaterials,visit:
http://computingbook.org
Version:August19,2011
Attribution-Noncommercial-ShareAlike3.0UnitedStatesLicense
Contents
1 Computing 1
1.1 Processes,Procedures,andComputers . . . . . . . . . . . . . . . . 2
1.2 MeasuringComputingPower . . . . . . . . . . . . . . . . . . . . . 3
1.2.1 Information . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.2.2 RepresentingData . . . . . . . . . . . . . . . . . . . . . . . 8
1.2.3 GrowthofComputingPower . . . . . . . . . . . . . . . . . 12
1.3 Science,Engineering,andtheLiberalArts . . . . . . . . . . . . . . 13
1.4 SummaryandRoadmap . . . . . . . . . . . . . . . . . . . . . . . . 16
PartI:DefiningProcedures
2 Language 19
2.1 SurfaceFormsandMeanings . . . . . . . . . . . . . . . . . . . . . 19
2.2 LanguageConstruction . . . . . . . . . . . . . . . . . . . . . . . . . 20
2.3 RecursiveTransitionNetworks . . . . . . . . . . . . . . . . . . . . . 22
2.4 ReplacementGrammars . . . . . . . . . . . . . . . . . . . . . . . . 26
2.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
3 Programming 35
3.1 ProblemswithNaturalLanguages . . . . . . . . . . . . . . . . . . . 36
3.2 ProgrammingLanguages . . . . . . . . . . . . . . . . . . . . . . . . 37
3.3 Scheme . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
3.4 Expressions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
3.4.1 Primitives . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
3.4.2 ApplicationExpressions . . . . . . . . . . . . . . . . . . . . 41
3.5 Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
3.6 Procedures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
3.6.1 MakingProcedures . . . . . . . . . . . . . . . . . . . . . . . 45
3.6.2 SubstitutionModelofEvaluation . . . . . . . . . . . . . . . 46
3.7 Decisions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
3.8 EvaluationRules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
3.9 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
4 ProblemsandProcedures 53
4.1 SolvingProblems . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
4.2 ComposingProcedures . . . . . . . . . . . . . . . . . . . . . . . . . 54
4.2.1 ProceduresasInputsandOutputs . . . . . . . . . . . . . . 55
4.3 RecursiveProblemSolving . . . . . . . . . . . . . . . . . . . . . . . 56
4.4 EvaluatingRecursiveApplications. . . . . . . . . . . . . . . . . . . 64
4.5 DevelopingComplexPrograms . . . . . . . . . . . . . . . . . . . . 67
4.5.1 Printing. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
4.5.2 Tracing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
4.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
5 Data 75
5.1 Types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
5.2 Pairs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
5.2.1 MakingPairs . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
5.2.2 TriplestoOctuples . . . . . . . . . . . . . . . . . . . . . . . 80
5.3 Lists . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
5.4 ListProcedures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
5.4.1 ProceduresthatExamineLists. . . . . . . . . . . . . . . . . 83
5.4.2 GenericAccumulators . . . . . . . . . . . . . . . . . . . . . 84
5.4.3 ProceduresthatConstructLists . . . . . . . . . . . . . . . . 86
5.5 ListsofLists . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90
5.6 DataAbstraction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92
5.7 SummaryofPartI . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102
PartII:AnalyzingProcedures
6 Machines 105
6.1 HistoryofComputingMachines . . . . . . . . . . . . . . . . . . . . 106
6.2 MechanizingLogic . . . . . . . . . . . . . . . . . . . . . . . . . . . 108
6.2.1 ImplementingLogic . . . . . . . . . . . . . . . . . . . . . . 109
6.2.2 ComposingOperations . . . . . . . . . . . . . . . . . . . . . 111
6.2.3 Arithmetic . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114
6.3 ModelingComputing . . . . . . . . . . . . . . . . . . . . . . . . . . 116
6.3.1 TuringMachines . . . . . . . . . . . . . . . . . . . . . . . . 118
6.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123
7 Cost 125
7.1 EmpiricalMeasurements . . . . . . . . . . . . . . . . . . . . . . . . 125
7.2 OrdersofGrowth . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129
7.2.1 BigO . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130
7.2.2 Omega . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133
7.2.3 Theta . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134
7.3 AnalyzingProcedures . . . . . . . . . . . . . . . . . . . . . . . . . . 136
7.3.1 InputSize . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136
7.3.2 RunningTime . . . . . . . . . . . . . . . . . . . . . . . . . . 137
7.3.3 WorstCaseInput . . . . . . . . . . . . . . . . . . . . . . . . 138
7.4 GrowthRates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139
7.4.1 NoGrowth:ConstantTime . . . . . . . . . . . . . . . . . . 139
7.4.2 LinearGrowth . . . . . . . . . . . . . . . . . . . . . . . . . . 140
7.4.3 QuadraticGrowth . . . . . . . . . . . . . . . . . . . . . . . . 145
7.4.4 ExponentialGrowth. . . . . . . . . . . . . . . . . . . . . . . 147
7.4.5 FasterthanExponentialGrowth . . . . . . . . . . . . . . . . 149
7.4.6 Non-terminatingProcedures . . . . . . . . . . . . . . . . . 149
7.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149
8 SortingandSearching 153
8.1 Sorting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153
8.1.1 Best-FirstSort . . . . . . . . . . . . . . . . . . . . . . . . . . 153
8.1.2 InsertionSort . . . . . . . . . . . . . . . . . . . . . . . . . . 157
8.1.3 QuickerSorting . . . . . . . . . . . . . . . . . . . . . . . . . 158
8.1.4 BinaryTrees . . . . . . . . . . . . . . . . . . . . . . . . . . . 161
8.1.5 Quicksort. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166
8.2 Searching . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 167
8.2.1 UnstructuredSearch . . . . . . . . . . . . . . . . . . . . . . 168
8.2.2 BinarySearch . . . . . . . . . . . . . . . . . . . . . . . . . . 168
8.2.3 IndexedSearch . . . . . . . . . . . . . . . . . . . . . . . . . 169
8.3 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 178
PartIII:ImprovingExpressiveness
9 Mutation 179
9.1 Assignment. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179
9.2 ImpactofMutation . . . . . . . . . . . . . . . . . . . . . . . . . . . 181
9.2.1 Names,Places,Frames,andEnvironments . . . . . . . . . 182
9.2.2 EvaluationRuleswithState . . . . . . . . . . . . . . . . . . 183
9.3 MutablePairsandLists . . . . . . . . . . . . . . . . . . . . . . . . . 186
9.4 ImperativeProgramming . . . . . . . . . . . . . . . . . . . . . . . . 188
9.4.1 ListMutators. . . . . . . . . . . . . . . . . . . . . . . . . . . 188
9.4.2 ImperativeControlStructures . . . . . . . . . . . . . . . . . 191
9.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193
10 Objects 195
10.1 PackagingProceduresandState . . . . . . . . . . . . . . . . . . . . 196
10.1.1 Encapsulation . . . . . . . . . . . . . . . . . . . . . . . . . . 196
10.1.2 Messages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197
10.1.3 ObjectTerminology . . . . . . . . . . . . . . . . . . . . . . . 199
10.2 Inheritance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 200
10.2.1 ImplementingSubclasses . . . . . . . . . . . . . . . . . . . 202
10.2.2 OverridingMethods . . . . . . . . . . . . . . . . . . . . . . 204
10.3 Object-OrientedProgramming . . . . . . . . . . . . . . . . . . . . 207
10.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 209
11 Interpreters 211
11.1 Python . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 212
11.1.1 PythonPrograms . . . . . . . . . . . . . . . . . . . . . . . . 213
11.1.2 DataTypes . . . . . . . . . . . . . . . . . . . . . . . . . . . . 216
11.1.3 ApplicationsandInvocations . . . . . . . . . . . . . . . . . 219
11.1.4 ControlStatements . . . . . . . . . . . . . . . . . . . . . . . 219
11.2 Parser . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221
11.3 Evaluator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 223
11.3.1 Primitives . . . . . . . . . . . . . . . . . . . . . . . . . . . . 223
11.3.2 IfExpressions . . . . . . . . . . . . . . . . . . . . . . . . . . 225
11.3.3 DefinitionsandNames . . . . . . . . . . . . . . . . . . . . . 226
11.3.4 Procedures . . . . . . . . . . . . . . . . . . . . . . . . . . . . 227
11.3.5 Application . . . . . . . . . . . . . . . . . . . . . . . . . . . 228
11.3.6 FinishingtheInterpreter . . . . . . . . . . . . . . . . . . . . 229
11.4 LazyEvaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 229
11.4.1 LazyInterpreter . . . . . . . . . . . . . . . . . . . . . . . . . 230
11.4.2 LazyProgramming . . . . . . . . . . . . . . . . . . . . . . . 232
11.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 235
PartIV:TheLimitsofComputing
12 Computability 237
12.1 MechanizingReasoning . . . . . . . . . . . . . . . . . . . . . . . . 237
12.1.1 Go¨del’sIncompletenessTheorem . . . . . . . . . . . . . . . 240
12.2 TheHaltingProblem . . . . . . . . . . . . . . . . . . . . . . . . . . 241
12.3 Universality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 244
12.4 ProvingNon-Computability . . . . . . . . . . . . . . . . . . . . . . 245
12.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 251
Indexes 253
Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 253
People. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 256
ListofExplorations
1.1 GuessingNumbers . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
1.2 TwentyQuestions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
2.1 PowerofLanguageSystems . . . . . . . . . . . . . . . . . . . . . . . 29
4.1 SquareRoots. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
4.2 Recipesforπ. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
4.3 RecursiveDefinitionsandGames . . . . . . . . . . . . . . . . . . . . 71
5.1 Pascal’sTriangle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
5.2 PegboardPuzzle. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93
7.1 MultiplyingLikeRabbits . . . . . . . . . . . . . . . . . . . . . . . . . 127
8.1 SearchingtheWeb . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177
12.1 VirusDetection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 246
12.2 BusyBeavers. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 249
ListofFigures
1.1 Usingthreebitstodistinguisheightpossiblevalues. . . . . . . . . . . 6
2.1 Simplerecursivetransitionnetwork. . . . . . . . . . . . . . . . . . . . 22
2.2 RTNwithacycle. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
2.3 Recursivetransitionnetworkwithsubnetworks. . . . . . . . . . . . . 24
2.4 AlternateNounsubnetwork. . . . . . . . . . . . . . . . . . . . . . . . . 24
2.5 RTNgenerating“Aliceruns”. . . . . . . . . . . . . . . . . . . . . . . . . 25
2.6 Systempowerrelationships. . . . . . . . . . . . . . . . . . . . . . . . . 30
2.7 ConvertingtheNumberproductionstoanRTN. . . . . . . . . . . . . 31
2.8 ConvertingtheMoreDigitsproductionstoanRTN. . . . . . . . . . . . 31
2.9 ConvertingtheDigitproductionstoanRTN. . . . . . . . . . . . . . . 32
3.1 RunningaSchemeprogram. . . . . . . . . . . . . . . . . . . . . . . . . 39
4.1 Aproceduremapsinputstoanoutput. . . . . . . . . . . . . . . . . . . 54
4.2 Composition. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
4.3 CircularComposition. . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
4.4 RecursiveComposition. . . . . . . . . . . . . . . . . . . . . . . . . . . 58
4.5 CorneringtheQueen. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
5.1 PegboardPuzzle. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93
6.1 Computingandwithwine. . . . . . . . . . . . . . . . . . . . . . . . . . 110
6.2 Computinglogicalor andnot withwine . . . . . . . . . . . . . . . . . 111
6.3 Computingand3bycomposingtwoandfunctions. . . . . . . . . . . 112
6.4 TuringMachinemodel. . . . . . . . . . . . . . . . . . . . . . . . . . . . 119
6.5 RulesforcheckingbalancedparenthesesTuringMachine.. . . . . . . 121
6.6 CheckingparenthesesTuringMachine. . . . . . . . . . . . . . . . . . 121
7.1 Evaluationoffiboprocedure. . . . . . . . . . . . . . . . . . . . . . . . 128
7.2 VisualizationofthesetsO(f),Ω(f),andΘ(f). . . . . . . . . . . . . . 130
7.3 OrdersofGrowth. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131
8.1 Unbalancedtrees. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165
9.1 Sampleenvironments. . . . . . . . . . . . . . . . . . . . . . . . . . . . 182
9.2 Environmentcreatedtoevaluate(bigger 34). . . . . . . . . . . . . . . 184
9.3 Environmentafterevaluating(defineinc(make-adder 1)). . . . . . . 185
9.4 Environmentforevaluatingthebodyof(inc149). . . . . . . . . . . . . 186
9.5 Mutablepaircreatedbyevaluating(set-mcdr! pair pair). . . . . . . . 187
9.6 MutableListcreatedbyevaluating(mlist 123). . . . . . . . . . . . . . 187
10.1 Environmentproducedbyevaluating: . . . . . . . . . . . . . . . . . . 197
10.2 Inheritancehierarchy. . . . . . . . . . . . . . . . . . . . . . . . . . . . 201
10.3 Counterclasshierarchy. . . . . . . . . . . . . . . . . . . . . . . . . . . 206
12.1 Incompleteandinconsistentaxiomaticsystems. . . . . . . . . . . . . 239
12.2 UniversalTuringMachine. . . . . . . . . . . . . . . . . . . . . . . . . . 245
12.3 Two-stateBusyBeaverMachine. . . . . . . . . . . . . . . . . . . . . . 249
ImageCredits
Mostoftheimagesinthebook,includingthetilesonthecover,weregenerated
bytheauthor.
Someofthetileimagesonthecoverarefromflickrcreativecommonslicenses
imagesfrom:ellbrown,JohnsonCameraface,cogdogblog,Cyberslayer,dmealif-
fe,Dunechaser,MichaelFitz,WolfieFox,glingl,jurvetson,KayVee.INC,michaeld-
beavers,andOneras.
The Van Gogh Starry Night image from Section 1.2.2 is from the Google Art
Project. TheApolloGuidanceComputerimageinSection1.2.3wasreleasedby
NASAandisinthepublicdomain.ThetrafficlightinSection2.1isfromiStock-
Photo, andtherotarytrafficsignalisfromtheWikimediaCommons. Thepic-
tureofGraceHopperinChapter3isfromtheComputerHistoryMuseum. The
playing card images in Chapter 4 are from iStockPhoto. The images of Gauss,
Heron,andGraceHopper’sbugareinthepublicdomain. TheDilbertcomicin
Chapter4islicensedfromUnitedFeatureSyndicate,Inc. ThePascal’striangle
imageinExcursion5.1isfromWikipediaandisinthepublicdomain.Theimage
ofAdaLovelaceinChapter6isfromtheWikimediaCommons,ofapaintingby
MargaretCarpenter.TheodomoterimageinChapter7isfromiStockPhoto,asis
theimageofthefrustratedstudent.ThePythonsnakecharmerinSection11.1is
fromiStockPhoto.TheDynabookimagesattheendofChapter10arefromAlan
Kay’spaper. ThexkcdcomicattheendofChapter11isusedunderthecreative
commonslicensegenerouslyprovidedbyRandallMunroe.
Preface
ThisbookstartedfromthepremisethatComputerScienceshouldbetaughtas
aliberalart,notanindustrialskill.Ihadtheprivilegeoftaking6.001fromGerry
SussmanwhenIwasafirstyearstudentatMIT,andthatcourseawakenedme
to the power and beauty of computing, and inspired me to pursue a career as
ateacherandresearcherinComputerScience. WhenIarrivedasanewfaculty
memberattheUniversityofVirginiain1999, Iwasdistraughttodiscoverthat
the introductory computing courses focused on teaching industrial skills, and
withsomuchofthecoursetimedevotedtoexplainingthetechnicalcomplexi-
tiesofusingbloatedindustriallanguageslikeC++andJava,therewasverylittle,
if any, time left to get across the core intellectual ideas that are the essence of
computingandthereasoneveryoneshouldlearnit.
WiththehelpofaUniversityTeachingFellowshipandNationalScienceFoun-
dation grants, I developed a new introductory computer science course, tar-
getedespeciallytostudentsintheCollegeofArts&Sciences. Thiscoursewas
firstofferedinSpring2002,withthehelpofanextraordinarygroupofAssistant
Coaches. Because of some unreasonable assumptions in the first assignment,
halfthestudentsquicklydroppedthecourse,butasmall,intrepid,groupofpi-
oneeringstudentspersisted,anditisthankstotheireffortsthatthisbookexists.
Thatcourse,andthenextseveralofferings,usedAbelson&Sussman’soutstand-
ing Structure and Interpretation of Computer Programs (SICP) textbook along
withDouglasHofstadter’sGo¨del,Escher,Bach:AnEternalGoldenBraid.
Spring2002CS200PioneerGraduates
Backrow,fromleft:PortmanWills(AssistantCoach),SpencerStockdale,ShawnO’Hargan,
JeffTaylor,JacquesFournier,KatieWinstanley,RussellO’Reagan,VictorClayYount.
Front:GraceDeng,RachelDada,JonErdman(AssistantCoach).
IamnotaloneinthinkingSICPisperhapsthegreatesttextbookeverwrittenin
any field, so it was with much trepidation that I endeavored to develop a new
textbook. I hope the resulting book captures the spirit and fun of computing
exemplified by SICP, but better suited to an introductory course for students
withnopreviousbackgroundwhilecoveringmanytopicsnotincludedinSICP
such as languages, complexity analysis, objects, and computability. Although
thisbookisdesignedaroundaonesemesterintroductorycourse,itshouldalso
besuitableforself-studystudentsandforpeoplewithsubstantialprogramming
experiencebutwithoutsimilarcomputerscienceknowledge.
Iamindebtedtomanypeoplewhohelpeddevelopthiscourseandbook.West-
leyWeimerwasthefirstpersontoteachusingsomethingresemblingthisbook,
andhisthoroughandinsightfulfeedbackledtoimprovementsthroughout.Greg
Humphreys,PaulReynolds,andMarkSherriffhavealsotaughtversionsofthis
course, and contributed to its development. I am thankful to all of the Assis-
tantCoachesovertheyears,especiallySarahBergkuist(2004),AndrewConnors
(2004),RachelDada(2003),PaulDiOrio(2009),KingaDobolyi(2007),JonErd-
man(2002),EthanFast(2009),DavidFaulkner(2005),JacquesFournier(2003),
RichardHsu(2007),RachelLathbury(2009),MichaelLew(2009),StephenLiang
(2002),DanMarcus(2007),RachelRater(2009),SpencerStockdale(2003),Dan
Upton(2005),PortmanWills(2002),KatieWinstanley(2003and2004),andRe-
beccaZapfel(2009). WilliamAiello, AnnaChefter, ChrisFrost, JonathanGrier,
ThadHughes,AlanKay,TimKoogle,JerryMcGann,GaryMcGraw,RadhikaNag-
pal, ShawnO’Hargan, MikePeck, andJudithShatinalsomadeimportantcon-
tributionstotheclassandbook.
Mydeepestthanksaretomywife,Nora,whoisaconstantsourceofinspiration,
support,andwonder.
Finally, my thanks to all past, present, and future students who use this book,
withoutwhomitwouldhavenopurpose.
HappyComputing!
DavidEvans
Charlottesville,Virginia
August2011
Spring2003
Spring2004 Spring2005