A Programmer’s Introduction to Mathematics

376 Pages·2018·32.099 MB·English

by  Dr. Jeremy Kun

Copyright©2018JeremyKun
Allrightsreserved. Thisbookoranyportionthereofmaynotbereproducedorusedin
anymannerwhatsoeverwithouttheexpresswrittenpermissionofthepublisherexcept
fortheuseofbriefquotationsinabookreview.
All images used in this book are either the author’s original works or in the public
domain. In particular, the only non-original images are in the chapter on group theory,
specifically the textures from The Grammar of the Orient, M.C. Escher’s Circle Limit IV,
andtwodiagramsinthepublicdomain,sourcedfromWikipedia.
Firstedition,2018.
pimbook.org
Tomywife,Erin.
My unbounded, uncountable thanks goes out to the many people who read drafts at
various stages of roughness and gave feedback, including (in alphabetical order by first
name),AaronShifman,AdamLelkes,AlexWalchli,AliFathalian,ArunKoshy,BenFish,
CraigStuntz,DevinIvy,ErinKelly,FredRoss,IanSharkey,JasperSlusallek,Jean-Gabriel
Young,JoãoRico,JohnGranata,JulianLeonardoCuevasRozo,KevinFinn,LandonKavlie,
LouisMaddox,MatthijsHollemans,OliviaSimpson,PabloGonzálezdeAledo,PaigeBai-
ley,PatrickRegan,PatrickStein,RodrigoZhou,StephanieLabasan,TempleKeller,Trent
McCormick.
SpecialthankstoDevinIvyforathoroughtechnicalreviewoftwokeychapters.
Contents
OurGoal i
1 LikeProgramming,MathematicshasaCulture 1
2 Polynomials 5
2.1 Polynomials,Java,andDefinitions . . . . . . . . . . . . . . . . . . . . . . 5
2.2 ALittleMoreNotation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
2.3 Existence&Uniqueness . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
2.4 RealizingitinCode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
2.5 Application: SharingSecrets . . . . . . . . . . . . . . . . . . . . . . . . . 24
2.6 CulturalReview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
2.7 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
2.8 ChapterNotes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
3 OnPaceandPatience 35
4 Sets 39
4.1 Sets,Functions,andTheir-Jections . . . . . . . . . . . . . . . . . . . . . 40
4.2 CleverBijectionsandCounting . . . . . . . . . . . . . . . . . . . . . . . 48
4.3 ProofbyInductionandContradiction . . . . . . . . . . . . . . . . . . . . 51
4.4 Application: StableMarriages . . . . . . . . . . . . . . . . . . . . . . . . 54
4.5 CulturalReview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
4.6 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
4.7 ChapterNotes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
5 VariableNames,Overloading,andYourBrain 63
6 Graphs 69
6.1 TheDefinitionofaGraph . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
6.2 GraphColoring . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
6.3 RegisterAllocationandHardness . . . . . . . . . . . . . . . . . . . . . . 73
6.4 PlanarityandtheEulerCharacteristic . . . . . . . . . . . . . . . . . . . . 75
6.5 Application: theFiveColorTheorem . . . . . . . . . . . . . . . . . . . . 77
6.6 ApproximateColoring . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82
6.7 CulturalReview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
6.8 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84
6.9 ChapterNotes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
7 TheManySubculturesofMathematics 89
8 CalculuswithOneVariable 95
8.1 LinesandCurves . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96
8.2 Limits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101
8.3 TheDerivative . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107
8.4 TaylorSeries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111
8.5 Remainders . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116
8.6 Application: FindingRoots . . . . . . . . . . . . . . . . . . . . . . . . . . 118
8.7 CulturalReview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125
8.8 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125
9 OnTypesandTailCalls 129
10 LinearAlgebra 135
10.1 LinearMapsandVectorSpaces . . . . . . . . . . . . . . . . . . . . . . . . 136
10.2 LinearMaps,FormallyThisTime . . . . . . . . . . . . . . . . . . . . . . 141
10.3 TheBasisandLinearCombinations . . . . . . . . . . . . . . . . . . . . . 143
10.4 Dimension . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147
10.5 Matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149
10.6 ConjugationsandComputations . . . . . . . . . . . . . . . . . . . . . . . 155
10.7 OneVectorSpacetoRuleThemAll . . . . . . . . . . . . . . . . . . . . . 157
10.8 GeometryofVectorSpaces . . . . . . . . . . . . . . . . . . . . . . . . . . 159
10.9 Application: SingularValueDecomposition . . . . . . . . . . . . . . . . . 164
10.10CulturalReview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179
10.11Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179
10.12ChapterNotes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 181
11 LiveandLearnLinearAlgebra(Again) 185
12 EigenvectorsandEigenvalues 191
12.1 EigenvaluesofGraphs . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193
12.2 LimitingtheScope: SymmetricMatrices . . . . . . . . . . . . . . . . . . 195
12.3 InnerProducts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 198
12.4 OrthonormalBases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202
12.5 ComputingEigenvalues . . . . . . . . . . . . . . . . . . . . . . . . . . . . 205
12.6 TheSpectralTheorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 207
12.7 Application: Waves . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 210
12.8 CulturalReview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 225
12.9 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 226
12.10ChapterNotes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 229
13 RigorandFormality 231
14 MultivariableCalculusandOptimization 237
14.1 GeneralizingtheDerivative . . . . . . . . . . . . . . . . . . . . . . . . . . 237
14.2 LinearApproximations . . . . . . . . . . . . . . . . . . . . . . . . . . . . 240
14.3 MultivariableFunctionsandtheChainRule . . . . . . . . . . . . . . . . 245
14.4 ComputingtheTotalDerivative . . . . . . . . . . . . . . . . . . . . . . . 246
14.5 TheGeometryoftheGradient . . . . . . . . . . . . . . . . . . . . . . . . 250
14.6 OptimizingMultivariableFunctions . . . . . . . . . . . . . . . . . . . . . 251
14.7 TheChainRule: aRepriseandaProof . . . . . . . . . . . . . . . . . . . . 260
14.8 GradientDescent: anOptimizationHammer . . . . . . . . . . . . . . . . 263
14.9 GradientsofComputationGraphs . . . . . . . . . . . . . . . . . . . . . . 264
14.10Application: AutomaticDifferentiationandaSimpleNeuralNetwork . . 267
14.11CulturalReview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 283
14.12Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 283
14.13ChapterNotes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 286
15 TheArgumentforBig-ONotation 289
16 Groups 299
16.1 TheGeometricPerspective . . . . . . . . . . . . . . . . . . . . . . . . . . 301
16.2 TheInterfacePerspective . . . . . . . . . . . . . . . . . . . . . . . . . . . 305
16.3 Homomorphisms: StructurePreservingFunctions . . . . . . . . . . . . . 307
16.4 BuildingBlocksofGroups . . . . . . . . . . . . . . . . . . . . . . . . . . 310
16.5 GeometryastheStudyofGroups . . . . . . . . . . . . . . . . . . . . . . 312
16.6 TheSymmetryGroupofthePoincaréDisk . . . . . . . . . . . . . . . . . 321
16.7 TheHyperbolicIsometryGroupasaGroupofMatrices . . . . . . . . . . 327
16.8 Application: DrawingHyperbolicTessellations. . . . . . . . . . . . . . . 328
16.9 CulturalReview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 344
16.10Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 344
16.11ChapterNotes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 349
17 ANewInterface 351
AbouttheAuthorandCover 361
Index 363

Our Goal
Thisbookhasastraightforwardgoal: toteachyouhowtoengagewithmathematics.
Let’sunpackthis. By“mathematics,” Imeantheuniverseofbooks, papers, talks, and
blog posts that contain the meat of mathematics: formal definitions, theorems, proofs,
conjectures, and algorithms. By “engage” I mean that for any mathematical topic, you
have the cognitive tools to actively progress toward understanding that topic. I will
“teach” you by introducing you to—or having you revisit—a broad foundation of topics
and techniques that support the restof mathematics. I say “with” because mathematics
requiresactiveparticipation.
We will define and study many basic objects of mathematics, such as polynomials,
graphs, and matrices. More importantly, I’ll explain how to think about those objects
as seasoned mathematicians do. We will examine the hierarchies of mathematical ab-
straction,alongwithmanyofthesofterskillsandinsightsthatconstitute“mathematical
intuition.” Along the way we’ll hear the voices of mathematicians—both famous histor-
ical figures and my friends and colleagues—to paint a picture of mathematics as both a
messy amalgam of competing ideas and preferences, and a story with delightfully sur-
prising twists and connections. In the end, I will show you how mathematicians think
aboutmathematics.
Sowhywouldsomeonelikeyou1 wanttoengagewithmathematics? Manysoftware
engineers, especially the sort who like to push the limits of what can be done with pro-
grams, eventually come to realize a deep truth: mathematics unlocks a lot of cool new
programs. These are truly novel programs. They would simply be impossible to write
(ifnotinconceivable!) withoutmathematics. Thatincludesprogramsinthisbookabout
cryptography, data science, and art, but also to many revolutionary technologies in in-
dustry, such as signal processing, compression, ranking, optimization, and artificial in-
telligence. As importantly, a wealth of opportunity makes programming more fun! To
quoteRandallMunroeinhisXKCDcomicForgotAlgebra,“TheonlythingsyouHAVEto
know are how to make enough of a living to stay alive and how to get your taxes done.
All the fun parts of life are optional.” If you want your career to grow beyond shuffling
dataaroundtomeetarbitrarybusinessgoals,youshouldlearnthetoolsthatenableyou
towriteprogramsthatcaptivateanddelightyou. Mathematicsisoneofthosetools.
Programmersareinaprivilegedpositiontoengagewithmathematics. Asaprogram-
1Hopefullyyou’reaprogrammer;otherwise,thetitleofthisbookmusthavesurelycausedapanicattack.
i
ii
mer, you eat paradigms for breakfast and reshape them into new ones for lunch. Your
comfort with functions, logic, and protocols gives you an intuitive familiarity with ba-
sic topics such as boolean algebra, recursion, and abstraction. You can rely on this to
makemathematicslessforeign,progressingallthefastertomorenuancedandstimulat-
ing topics. Contrast this to most educational math content aimed at students with no
backgroundandfocusingonroteexercisesandpassingtests. Asabonus,programming
allows me to provide immediate applications that ground the abstract ideas in code. In
each chapter of this book, we’ll fashion our mathematical designs into a program you
couldn’thavewrittenbefore,todazzlingeffect. ThecodeisavailableonGithub,2 witha
directoryforeachchapter.
Alltold,thisbookisnotatextbook. Iwon’tdrillyouwithexercises,thoughdrillshave
theirplace. Wewon’tbuildupanyparticularfieldofmathematicsfromscratch. Though
we’ll visit calculus, linear algebra, and many other topics, this book is far too short to
cover everything a mathematician ought to know about these topics. Moreover, while
much of the book is appropriately rigorous, I will occasionally and judiciously loosen
rigorwhenitfacilitatesabetterunderstandingandrelievestedium. Iwillnotewhenthis
occurs,andwe’lldiscusstheroleofrigorinmathematicsmorebroadly.
Indeed, rather than read an encyclopedic reference, you want to become comfortable
withtheprocessoflearningmathematics. Inpartthatmeansbecomingcomfortablewith
discomfort, with the struggle of understanding a new concept, and the techniques that
mathematicians use to remain productive and sane. Many people find calculus difficult,
or squeaked by a linear algebra course without grokking it. After this book you should
haveacorenuggetofunderstandingofthesesubjects,alongwiththecognitivetoolsthat
willenableyoudiveasdeeplyasyoulike.
Asanecessaryconsequence,inthisbookyou’lllearnhowtoreadandwriteproofs. The
simplestandbroadesttruthaboutmathematicsisthatitrevolvesaroundproofs. Proofs
areboththeprimaryvehicleofinsightandthefundamentalmeasureofjudgment. They
are the law, the currency, and the fine art of mathematics. Most of what makes math-
ematics mysterious and opaque—the rigorous definitions, the notation, the overloading
ofterminology,themountainsoftheory,andtheunspokenobligationsonthereader—is
duetothecentralityofproofs. Adominantobstacletolearningmathisanunfamiliarity
withthisculture. InthisbookI’llshowyouwhyproofsaresoimportant,coverthebasic
methods, anddisplayexamplesofproofs ineachchapter. Tobesure, youdon’thaveto
understand every proof to finish this book, and you will probably be confounded by a
few. Embrace your humility. I hope to convince you that each proof contains layers of
insight that are genuinely worthwhile, and that no single person can see the complete
pictureinasinglesitting. Asyougrowintomathematics,theactofreadingevenprevi-
ouslyunderstoodproofsprovidesbothrenewedandincreaseedwisdom. Solongasyou
identifythevaluegainedbyyourstruggle,yourtimeiswellspent.
I’llalsoteachyouhowtoreadbetweenthemathematicallinesofatext,andunderstand
theimplicitdirectionsandculturalcuesthatlittertextbooksandpapers. Asweproceed
2pimbook.org
iii
throughthechapters,we’llgraduallybecomemoreterse,andyou’llhavemanyopportu-
nitiestopracticeparsing,interpreting,andunderstandingmath. Allofthetopicsinthis
bookareexplainedbyhundredsofothersources,andeachchapter’sexercisesincludeex-
plorationsofconceptsbeyondthesepages. Inaddition,I’lldiscusshowmathematicians
approachproblems,andhowtheirprocessinfluencesthecultureofmath.
You will not learn everything you want to know in this book, nor will you learn ev-
erythingthisbookhastoofferinonesitting. Thosealreadyfamiliarwithmathmayfind
earlychaptersoffensivelyslowanddetailed. Thosegenuinelynewtomathmayfindthe
later chapters offensively fast. This is by design. I want you to be exposed to as much
mathematicsaspossible,tolearnthedefinitionsofcentralmathematiclideas,tobeintro-
ducedtonotations,conventions,andattitudes,andtohaveampleopportunitytoexplore
topicsthatpiqueyourinterest.
Anumberoftopicsareconspicuouslymissingfromthisbook,mynegligenceofwhich
approaches criminal. Except for a few informal cameos, we ignore complex numbers,
probabilityandstatistics,differentialequations,andformallogic. Inmyhumbleopinion,
none of these topics is as fundamental for mathematical computer science as those I’ve
chosen to cover. After becoming comfortable with the topics in this book, for example,
probability will be very accessible. The chapter on eigenvalues will include a miniature
introductiontodifferentialequations. Thechapterongroupswillbrieflysummarizecom-
plex numbers. Probability will echo in your brain when we discuss random graphs and
machinelearning. Moreover,manytopicsinthisbookareprerequisitesfortheseotherar-
eas. And,ofcourse,asasinglehumanself-publishingthisbookonnightsandweekends,
Ihaveonlysomuchtime.
The first step on our journey is to confirm that mathematics has a culture worth be-
comingacquaintedwith. We’lldothiswithacomparativetourofthecultureofsoftware
thatweunderstandsowell.