A Programmer’s Introduction to Mathematics

398 Pages·2020·32.89 MB·English

by  Jeremy Kun

Sold to
[email protected]
A Programmer’s Introduction to Mathematics
JeremyKun
Copyright©2020JeremyKun
Allrightsreserved. Thisbookoranyportionthereofmaynotbereproducedorusedin
anymannerwhatsoeverwithouttheexpresswrittenpermissionofthepublisherexcept
fortheuseofbriefquotationsinabookreview.
All images used in this book are either the author’s original works or in the public
domain. In particular, the only non-original images are in the chapter on group theory,
specificallythetexturesfromOwenJones’sdesignmasterpiece,TheGrammaroftheOrna-
ment (1856),M.C.Escher’sCircleLimitIV (1960),andtwodiagramsinthepublicdomain,
sourcedfromWikipedia.
Secondedition,2020.
pimbook.org
Tomywife,Erin.
My unbounded, uncountable thanks goes out to the many people who read drafts at
various stages of roughness and gave feedback, including (in alphabetical order by first
name),AaronShifman,AdamLelkes,AlexWalchli,AliFathalian,ArunKoshy,BenFish,
CraigStuntz,DevinIvy,ErinKelly,FredRoss,IanSharkey,JasperSlusallek,Jean-Gabriel
Young,JoãoRico,JohnGranata,JulianLeonardoCuevasRozo,KevinFinn,LandonKavlie,
LouisMaddox,MatthijsHollemans,OliviaSimpson,PabloGonzálezdeAledo,PaigeBai-
ley,PatrickRegan,PatrickStein,RodrigoZhou,StephanieLabasan,TempleKeller,Trent
McCormick.
An extra thanks to the readers who submitted errata at pimbook.org for the first
edition, including Abhinav Upadhyay, Abhishek Bhatia, Alejandro Baldominos, Andrei
Paleyes, Arman Yessenamanov, Arthur Allshire, Arunoda Susiripala, Bilal Karim Reffas,
Brian Cloutier, Brian van den Broek, Britton Winterrose, Cedric Bucerius, Changyoung
Koh, Charlie Mead, Chris G, Chrislain Razafimahefa, Darin Brezeale, David Bimmler,
David Furcy, David Shockley, David Wu, Devin Conathan, Don-Duong Quach, Fidel
Barrera-Cruz, Francis Huynh, Glen De Cauwsemaecker, Harry Altman, Ivan Katanic,
Jaime, Jan Moren, Jason Hooper, K. Alex Mills, Kenytt Avery, Konstantin Weitz, Lean-
droMottaBarros,LukeA.,MarcoCraveiro,Matthijs,MaximilianSchlund,MejiAbidoye,
Michael Cohen, Michaël Defferrard, Nicolas Krause, Nikita V., Oliver Sampson, Ondrej
Slamecka, Patrick Stingley, Rich Yonts, Rodrigo Ariel Sota, Ryan Troxler, Seth Yastrov,
Simon Skrede, Sriram Srinivasan, Steve Dwyer, Steven D. Brown, Tim Wilkens, Timo
Vesalainen,TylerSmith,WojciechKryscinski,andZorro.
Specialthanksto JohnPeloquinfor histhoroughtechnical reviewforthe secondedi-
tion,andtoDevinIvyfortechnicalreviewofpartsofthefirstedition.
Contents
OurGoal i
Chapter1. LikeProgramming,MathematicshasaCulture 1
Chapter2. Polynomials 5
2.1 Polynomials,Java,andDefinitions . . . . . . . . . . . . . . . . . . . . . . 5
2.2 ALittleMoreNotation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
2.3 Existence&Uniqueness . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
2.4 RealizingitinCode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
2.5 Application: SharingSecrets . . . . . . . . . . . . . . . . . . . . . . . . . 24
2.6 CulturalReview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
2.7 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
2.8 ChapterNotes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
Chapter3. OnPaceandPatience 35
Chapter4. Sets 39
4.1 Sets,Functions,andTheir-Jections . . . . . . . . . . . . . . . . . . . . . 40
4.2 CleverBijectionsandCounting . . . . . . . . . . . . . . . . . . . . . . . 48
4.3 ProofbyInductionandContradiction . . . . . . . . . . . . . . . . . . . . 51
4.4 Application: StableMarriages . . . . . . . . . . . . . . . . . . . . . . . . 54
4.5 CulturalReview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
4.6 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
4.7 ChapterNotes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
Chapter5. VariableNames,Overloading,andYourBrain 63
Chapter6. Graphs 69
6.1 TheDefinitionofaGraph . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
6.2 GraphColoring . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
6.3 RegisterAllocationandHardness . . . . . . . . . . . . . . . . . . . . . . 73
6.4 PlanarityandtheEulerCharacteristic . . . . . . . . . . . . . . . . . . . . 75
6.5 Application: theFiveColorTheorem . . . . . . . . . . . . . . . . . . . . 78
6.6 ApproximateColoring . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
6.7 CulturalReview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
6.8 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
6.9 ChapterNotes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87
Chapter7. TheManySubculturesofMathematics 89
Chapter8. CalculuswithOneVariable 95
8.1 LinesandCurves . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96
8.2 Limits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100
8.3 TheDerivative . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107
8.4 TaylorSeries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111
8.5 Remainders . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117
8.6 Application: FindingRoots . . . . . . . . . . . . . . . . . . . . . . . . . . 119
8.7 CulturalReview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125
8.8 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125
Chapter9. OnTypesandTailCalls 129
Chapter10. LinearAlgebra 135
10.1 LinearMapsandVectorSpaces . . . . . . . . . . . . . . . . . . . . . . . . 136
10.2 LinearMaps,FormallyThisTime . . . . . . . . . . . . . . . . . . . . . . 141
10.3 TheBasisandLinearCombinations . . . . . . . . . . . . . . . . . . . . . 143
10.4 Dimension . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147
10.5 Matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149
10.6 ConjugationsandComputations . . . . . . . . . . . . . . . . . . . . . . . 155
10.7 OneVectorSpacetoRuleThemAll . . . . . . . . . . . . . . . . . . . . . 158
10.8 GeometryofVectorSpaces . . . . . . . . . . . . . . . . . . . . . . . . . . 159
10.9 Application: SingularValueDecomposition . . . . . . . . . . . . . . . . . 164
10.10CulturalReview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179
10.11Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179
10.12ChapterNotes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182
Chapter11. LiveandLearnLinearAlgebra(Again) 185
Chapter12. EigenvectorsandEigenvalues 191
12.1 EigenvaluesofGraphs . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193
12.2 LimitingtheScope: SymmetricMatrices . . . . . . . . . . . . . . . . . . 195
12.3 InnerProducts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 198
12.4 OrthonormalBases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202
12.5 ComputingEigenvalues . . . . . . . . . . . . . . . . . . . . . . . . . . . . 205
12.6 TheSpectralTheorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 207
12.7 Application: Waves . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 209
12.8 CulturalReview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 225
12.9 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 226
12.10ChapterNotes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 229
Chapter13. RigorandFormality 233
Chapter14. MultivariableCalculusandOptimization 239
14.1 GeneralizingtheDerivative . . . . . . . . . . . . . . . . . . . . . . . . . . 239
14.2 LinearApproximations . . . . . . . . . . . . . . . . . . . . . . . . . . . . 241
14.3 Vector-valuedFunctionsandtheChainRule . . . . . . . . . . . . . . . . 246
14.4 ComputingtheTotalDerivative . . . . . . . . . . . . . . . . . . . . . . . 248
14.5 TheGeometryoftheGradient . . . . . . . . . . . . . . . . . . . . . . . . 251
14.6 OptimizingMultivariableFunctions . . . . . . . . . . . . . . . . . . . . . 253
14.7 GradientDescent: anOptimizationHammer . . . . . . . . . . . . . . . . 261
14.8 GradientsofComputationGraphs . . . . . . . . . . . . . . . . . . . . . . 262
14.9 Application: AutomaticDifferentiationandaSimpleNeuralNetwork . . 265
14.10CulturalReview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 281
14.11Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 281
14.12ChapterNotes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 284
Chapter15. TheArgumentforBig-ONotation 291
Chapter16. Groups 301
16.1 TheGeometricPerspective . . . . . . . . . . . . . . . . . . . . . . . . . . 303
16.2 TheInterfacePerspective . . . . . . . . . . . . . . . . . . . . . . . . . . . 307
16.3 Homomorphisms: StructurePreservingFunctions . . . . . . . . . . . . . 309
16.4 BuildingBlocksofGroups . . . . . . . . . . . . . . . . . . . . . . . . . . 312
16.5 GeometryastheStudyofGroups . . . . . . . . . . . . . . . . . . . . . . 314
16.6 TheSymmetryGroupofthePoincaréDisk . . . . . . . . . . . . . . . . . 324
16.7 Application: DrawingHyperbolicTessellations. . . . . . . . . . . . . . . 329
16.8 CulturalReview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 345
16.9 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 345
16.10ChapterNotes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 350
Chapter17. ANewInterface 353
AppendixA. Notation 363
AppendixB. ASummaryofProofs 365
B.1 Propositionalandfirst-orderlogic . . . . . . . . . . . . . . . . . . . . . . 365
B.2 Methodsofproof . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 367
B.3 Howdoesoneactuallyprovethings? . . . . . . . . . . . . . . . . . . . . 368
AppendixC. AnnotatedResources 373
C.1 FundamentalsandFoundations . . . . . . . . . . . . . . . . . . . . . . . 373
C.2 Polynomials . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 374
C.3 GraphTheoryandCombinatorics . . . . . . . . . . . . . . . . . . . . . . 375
C.4 CalculusandAnalysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 375
C.5 LinearAlgebra . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 376
C.6 Optimization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 377
C.7 AbstractAlgebra(Groups,etc.) . . . . . . . . . . . . . . . . . . . . . . . . 377
C.8 Topology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 378
C.9 ComputerScience,Theory,andAlgorithms . . . . . . . . . . . . . . . . . 378
C.10 FunandRecreation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 380
AbouttheAuthorandCover 381
Index 383

Our Goal
Thisbookhasastraightforwardgoal: toteachyouhowtoengagewithmathematics.
Let’sunpackthis. By“mathematics,” Imeantheuniverseofbooks, papers, talks, and
blog posts that contain the meat of mathematics: formal definitions, theorems, proofs,
conjectures, and algorithms. By “engage” I mean that for any mathematical topic, you
havethecognitivetoolstomakeprogresstowardunderstandingthattopic. Iwill“teach”
youbyintroducingyouto—orhavingyourevisit—abroadfoundationoftopicsandtech-
niquesthatsupporttherestofmathematics. Isay“with”becausemathematicsrequires
activeparticipation.
We will define and study many basic objects of mathematics, such as polynomials,
graphs, and matrices. More importantly, I’ll explain how to think about those objects
as seasoned mathematicians do. We will examine the hierarchies of mathematical ab-
straction,alongwithmanyofthesofterskillsandinsightsthatconstitute“mathematical
intuition.” Along the way we’ll hear the voices of mathematicians—both famous histor-
ical figures and my friends and colleagues—to paint a picture of mathematics as both a
messy amalgam of competing ideas and preferences, and a story with delightfully sur-
prising twists and connections. In the end, I will show you how mathematicians think
aboutmathematics.
Sowhywouldsomeonelikeyou1 wanttoengagewithmathematics? Manysoftware
engineers, especially the sort who like to push the limits of what can be done with pro-
grams,eventuallyrealizeadeeptruth: mathematicsunlocksalot ofcoolnewprograms.
Thesearetrulynovelprograms. Theywouldsimplybeimpossibletowrite(ifnotincon-
ceivable!) without mathematics. That includes programs in this book about cryptogra-
phy,datascience,andart,butalsotomanyrevolutionarytechnologiesinindustry,such
as signal processing, compression, ranking, optimization, and artificial intelligence. As
importantly, a wealth of opportunity makes programming more fun! To quote Randall
Munroe in his XKCD comic Forgot Algebra, “The only things you HAVE to know are
how to make enough of a living to stay alive and how to get your taxes done. All the
fun parts of life are optional.” If you want your career to grow beyond shuffling data
around to meet arbitrary business goals, you should learn the tools that enable you to
writeprogramsthatcaptivateanddelightyou. Mathematicsisoneofthosetools.
Programmers are in a privileged position to engage with mathematics. Your comfort
1Hopefullyyou’reaprogrammer;otherwise,thetitleofthisbookmusthavesurelycausedapanicattack.
i