Once upon an Algorithm. How Stories explain Computing

306 Pages·2017·8.971 MB·english

by  Martin Erwig

Once Upon an Algorithm
How Stories Explain Computing
Martin Erwig
TheMITPress
Cambridge,Massachusetts
London,England
©2017MassachusettsInstituteofTechnology
PrintedandboundintheUnitedStatesofAmerica
LibraryofCongressCataloging-in-PublicationData
Names: Erwig,Martin,author.
Title: Onceuponanalgorithm: howstoriesexplaincomputing/MartinErwig.
Description: Cambridge,MA:MITPress, [2017] |Includesbibliographical
referencesandindex.
Identifiers: LCCN2016053945|ISBN9780262036634(hardcover: alk. paper)
Subjects: LCSH:Computeralgorithms–Popularworks.
Classification: LCCQA76:9:A43E782017|DDC005:1–dc23LCrecordavailableat
https://lccn.loc.gov/2016053945
Contents
Preface vii
Acknowledgments xi
Introduction 1
PartI ALGORITHMS
ComputationandAlgorithms—HanselandGretel
1 APathtoUnderstandingComputation 19
2 WalktheWalk: WhenComputationReallyHappens 33
RepresentationandDataStructures—SherlockHolmes
3 TheMysteryofSigns 49
4 Detective’sNotebook: AccessoryaftertheFact 63
ProblemSolvingandItsLimitations—IndianaJones
5 TheSearchforthePerfectDataStructure 83
6 SortingoutSorting 103
7 MissionIntractable 121
PartII LANGUAGES
LanguageandMeaning—OvertheRainbow
8 ThePrismofLanguage 141
9 FindingtheRightTone: SoundMeaning 159
ControlStructuresandLoops—GroundhogDay
10 Weather,Rinse,Repeat 175
11 HappyEndingNotGuaranteed 189
Recursion—BacktotheFuture
12 AStitchinTimeComputesFine 205
13 AMatterofInterpretation 225
TypesandAbstraction—HarryPotter
14 TheMagicalType 245
15 ABird’sEyeView: AbstractingfromDetails 263
Glossary 287
Notes 303
Index 313
Preface
When people ask about my work, the conversation quickly turns to the question of
what computer science is. To say computer science is the science of computers is mis-
leading(thoughstrictlyspeakingnotincorrect)becausemostpeoplewilltakecomputer
tomeanPCorlaptopandconcludethatcomputerscientistsspendtheirtimeconstruct-
ing hardware. On the other hand, defining computer science as the study of computa-
tion is simply kicking the can down the road, since it immediately raises the question
ofwhatcomputationis.
Overtheyears,Ihavecometorealizethatteachingbyjustintroducingconceptafter
concept doesn’t work very well; it is simply too abstract. Nowadays, I typically start
bydescribingcomputerscienceasthestudyofsystematicproblemsolving. Everybody
knows what a problem is, and everybody has seen solutions as well. After explain-
ing this view through an example, I often get the chance to introduce the concept of
an algorithm, which allows me to point out important differences between computer
science and mathematics. Most of the time, I don’t need to talk about programming
languages, computers, and related technical matters, but even if it comes to that, the
concreteproblemmakesiteasytoillustratetheseconcepts. OnceUponanAlgorithmis
anelaborationofthisapproach.
Computer science is a relatively new member of the science club, and sometimes
it seems it has not yet earned the respect granted to serious scientific disciplines like
physics, chemistry, and biology. Think of a movie scene involving a physicist. You
willprobablyseesomeonediscussingcomplicatedformulaswrittenonablackboardor
supervisinganexperimentwearingalabcoat. Thephysicistisshownasareputablesci-
entistwhoseknowledgeistreasured. Nowimagineasimilarsceneinvolvingacomputer
scientist. Here you will likely see some nerdy guy sitting in a dark, messy room and
staringatacomputerscreen. Heisfranticallytypingonakeyboard,probablytryingto
breaksomecodeorpassword. Inbothscenes,animportantproblemisbeingsolved,but
while the physicist might provide some plausible explanation of how it can be solved,
the solution of the computer problem remains mysterious, often magical, and much
toocomplextobeexplainedtoanonspecialist. Ifcomputerscienceisunexplainableto
laypeople,whywouldanyoneevertrytoknowmoreaboutitorunderstandit?
The subject of computer science is computation, a phenomenon that affects every-
body. Iamnottalkingonlyaboutcellphones,laptops,ortheinternet. Considerfolding
apaperairplane,drivingtowork,cookingameal,orevenDNAtranscription,aprocess
thatoccursinyourcellsmillionsoftimeswhileyouarereadingthissentence. Theseare
allexamplesofcomputation—asystematicwayofproblemsolving—eventhoughmost
peoplewouldnotperceivethemassuch.
Science provides us with a basic understanding of how the natural world works,
and it gives us the scientific method for reliably establishing that knowledge. What
applies to science in general holds for computer science, too, especially since we en-
counter computation in so many different forms and so many different situations. A
basicunderstandingofcomputationthusprovidesbenefitssimilartoabasicknowledge
of physics, chemistry, and biology in making sense of the world and tackling many
real-worldproblemsmoreeffectively. Thisaspectofcomputationisoftenreferredtoas
computationalthinking.
A major goal of this book is to emphasize the general nature of computation and
thus the wide applicability of computer science. My hope is that this will spark a
broaderinterestincomputerscienceandadesiretolearnmoreaboutit.
I first identify computation in everyday activities and then explain corresponding
computer science concepts through popular stories. The everyday situations are taken
from a typical workday: getting up in the morning, having breakfast, commuting to
work, episodesattheworkplace,adoctor’sappointment, ahobbyactivityintheafter-
noon, having dinner, and reflecting on the day’s events in the evening. Each of these
fifteenvignettesintroducesabookchapter. Thechaptersthenexplainthecomputation
concepts using seven popular stories. Each story spans two or three chapters and deals
withaspecifictopicofcomputerscience.
The book has two parts: algorithms and languages. These are the two main pillars
on which the concept of computation rests. Table 1 summarizes the stories and the
computerscienceconceptstheyillustrate.
We all appreciate a good story. Stories console us, give us hope, and inspire us.
They tell us about the world, make us aware of the problems we face, and sometimes
suggest solutions. Stories can also provide guidance for our lives. When you think
aboutwhatstorieshavetoteachus,youprobablythinkaboutlove,conflict,thehuman
condition. ButIalsothinkaboutcomputation. WhenShakespeare’sJulietasks,“What’s
in a name?” she is on to an important question about representation. Albert Camus’s
Table1
Story Chapters Topics
PartI
HanselandGretel 1,2 ComputationandAlgorithms
SherlockHolmes 3,4 RepresentationandDataStructures
IndianaJones 5,6,7 ProblemSolvingandItsLimitations
PartII
OvertheRainbow 8,9 LanguageandMeaning
GroundhogDay 10,11 ControlStructuresandLoops
BacktotheFuture 12,13 Recursion
HarryPotter 14,15 TypesandAbstraction
TheMythofSisyphusraisesthequestionofhowtofacetheabsurdityoflifeandalsohow
todetectanever-endingcomputation.
Storieshavemultiplelayersofmeaning. Theyoftenincludeacomputationallayer.
Once Upon an Algorithm is an effort to unveil this layer and offer readers a new per-
spective on stories and computation. I hope that stories will be appreciated for their
computationalcontentandthatthisnovelpointofviewwillsparkinterestincomputer
science.
Introduction
Computation plays a prominent role in society now. But unless you want to become
a computer scientist, why should this prompt you to learn more about it? You could
simplyenjoythetechnologypoweredbycomputationandtakeadvantageofitsbenefits.
Youlikelyalsodon’tstudyavionicstomakeuseofairtravelorseekamedicaldegreeto
benefitfromtheachievementsofmodernhealthcare.
However, the world we live in consists of more than man-made technology. We
still have to interact with nonautonomous objects ruled by physical laws. Therefore,
it is beneficial to understand the basics of mechanics simply to predict the behavior
of objects and to safely navigate one’s environment. A similar case can be made for
thebenefitsofstudyingcomputationanditsrelatedconcepts. Computationisnotonly
revealedincomputersandelectronicgadgetsbutalsooccursoutsideofmachines. Inthe
following I briefly discuss some of the major computer science principles and explain
whytheymatter.
Computation and Algorithms
Iinviteyoutoperformthefollowingsimpleexercise. Forthisyouneedaruler,apencil,
and a piece of(quadrille-ruled) paper. First, draw a horizontal line that is 1 inch long.
Then draw a vertical line of the same length, perpendicular to the first, starting at one
ofitsends. Finally,connectthetwoopenendsofthelinesyoudrewwithadiagonalto
form a triangle. Now measure the length of the diagonal just drawn. Congratulations;
youhavejustcomputedthesquarerootof2. (Seefigure1.)
What does this exercise in geometry have to do with computation? As I explain in
chapters 1 and 2, it is the execution of an algorithm by a computer that brings about
a computation. In this example, you acted as a computer thpat executed an algorithm
to draw and measure lines, which led to the computation of 2. Having an algorithm
iscrucialbecauseonlythencandifferentcomputersperformacomputationrepeatedly
andatdifferenttimes. Animportantaspectofcomputationisthatitrequiresresources
2 OnceUponanAlgorithm
measure
1
in
draw draw ch
1 inch
Figure1 Computingthesquarerootof2usingapencilandaruler.
(such as pencil, paper, and a ruler) and takes time to carry out. Here again, having an
algorithmicdescriptionisimportantbecauseithelpsinanalyzingtheresourcerequire-
mentsforcomputations.
Chapters1and2explain
• whatalgorithmsare,
• thatalgorithmsareusedforsystematicproblemsolving,
• thatanalgorithmneedstobeexecutedbyacomputer(human,machine,etc.) to
produceacomputation,
• thattheexecutionofalgorithmsconsumesresources.
Whydoesitmatter?
Recipesareexamplesofalgorithms. Wheneveryoumakeasandwich, bakeachocolate
cake,orcookyourfavoritedishbyfollowingtheinstructionsofarecipe,youareeffec-
tively executing an algorithm to transform the raw ingredients into the final product.
Therequiredresourcesincludetheingredients,tools,energy,andpreparationtime.
Knowledge about algorithms makes us sensitive to questions about the correctness
of a method and what its resource requirements are. It helps us identify opportunities
forimprovingprocessesinallareasoflifethroughthe(re)organizationofstepsandma-
terials. Forinstance,inthegeometricsquarerootcomputation,wecouldhaveomitted
drawingthediagonallineandjustmeasuredthedistancebetweenthetwounconnected
endpoints.
In cooking the improvement can be as simple and obvious as saving trips to the
fridge by planning ahead or gathering ingredients ahead of time. You could plan how
to use your oven or stove more efficiently and save time by parallelizing steps, such as
preheating the oven or washing the salad while the potatoes are cooking. These tech-
Introduction 3
niquesalsoapplytomanyotherareas,fromsimpleassemblyinstructionsforfurniture
toorganizationalprocessesforrunninganofficeormanagingafactoryfloor.
In the realm of technology, algorithms control basically all the computations in
computers. One prominent example is data compression, without which the trans-
mission of music and movies over the internet would be nearly impossible. A data
compressionalgorithmidentifiesfrequentpatternsandreplacesthemwithsmallcodes.
Datacompressiondirectlyaddressestheresourceproblemofcomputationbyreducing
thespacerequiredtorepresentsongsandmoviesandthusalsothetimeittakestoload
themovertheinternet. AnotherexampleisGoogle’spage-rankalgorithm, whichdeter-
minestheorderinwhichsearchresultsarepresentedtoauser. Itworksbyassessingthe
importance of a web page through counting how many links point to it and weighing
howimportantthoselinksare.
Representation and Data Structures
One might expect that numerical computations are carried out with the help of num-
bersbecauseweusetheHindu-Arabicnumerpalsystem,andmachinesworkwith0sand
1s. Sothegeometricmethodforcomputing 2withthehelpoflinesmightbesurpris-
ing. The example demonstrates, however, that one and the same thing(for example, a
quantity) canberepresentedindifferentways(numbersymbolsorlines).
The essence of a computation is the transformation of representation. I explain in
chapter 3 what representations are and how they are employed in computations. Since
manycomputationsdealwithlargeamountsofinformation,Iexplaininchapter4how
collections of data can be efficiently organized. What complicates this question is the
factthatanyparticulardataorganizationmaysupportsomeformsofaccessingthedata
efficientlybutnotothers.
Chapters3and4discuss
• differentformsofrepresentation,
• differentwaysoforganizingandaccessingdatacollections,
• theadvantagesanddisadvantagesofdifferentdataorganizations.
Whydoesitmatter?
Measurements of ingredients in recipes can be given by weight or by volume. These
aredifferentformsofrepresentation,whichrequiredifferentcookingutensils(scalesor
measuringcups)forproperlyexecutingtherecipe/algorithm. Asfordataorganization,
4 OnceUponanAlgorithm
thewayyourfridgeorpantryisorganizedhasanimpactonhowfastyoucanretrieveall
theingredientsrequiredfortherecipe. Orconsiderhowthequestionofrepresentation
applies to the recipe itself. It may be given as a textual description, through a series of
pictures, or even in the form of a YouTube video. The choice of representation often
makesabigdifferenceintheeffectivenessofalgorithms.
In particular, the question of how to arrange a collection of items or people has
many applications, for instance, how to organize your desk or garage to help you find
items more quickly or how to organize the book shelves in a library. Or consider the
differentwaysyouwaitinline: inagrocerystore(standinginaqueue) oratadoctor’s
office(sittinginawaitingroomhavingpickedanumber)orwhenboardinganairplane
(multiplequeues).
In the realm of technology, spreadsheets are among the most successful program-
ming tools. The organization of data in tabular form has contributed to much of
their success because it supports the quick and easy formulation of sums over rows
and columns and represents the data and computation results for them in one place.
On the other hand, the internet, one of the most transformative inventions of the late
twentiethcentury,organizeswebpages,computers,andtheconnectionsbetweenthem
as a network. This representation supports flexible access to information and efficient
transmissionofdata.
Problem Solving and Its Limitations
Analgorithmisamethodforsolvingproblems,beitfindingthesquarerootofanumber
orbakingacake. Andcomputerscienceisadisciplinethatisconcernedwithsystematic
problemsolving.
Of the many problems that can be solved through algorithms, two deserve to be
discussed in detail. In chapter 5, I explain the problem of searching, one of the most
frequentlyusedcomputationsondata. Chapter6thenexplainstheproblemofsorting,
which illustrates a powerful problem-solving method and also the notion of intrinsic
problemcomplexity. Inchapter7,Idescribetheclassofso-calledintractableproblems.
Algorithms exist for these problems, but take too long to execute, so the problems are
notsolvableinpractice.
Chapters5,6,and7clarify
• whysearchingcanbedifficultandtimeconsuming,
• methodstoimprovesearching,
• differentalgorithmsforsorting,