Numeric Program Analysis Techniques with Applications to Array Analysis and Library ...

248 Pages·2007·1.25 MB·English

NUMERIC PROGRAMANALYSISTECHNIQUESWITH APPLICATIONSTO ARRAY
ANALYSISAND LIBRARY SUMMARIZATION
by
DenisGopan
A dissertationsubmittedin partialfulfillmentof
therequirementsforthedegreeof
DoctorofPhilosophy
(ComputerSciences)
at the
UNIVERSITY OFWISCONSIN–MADISON
2007
c Copyrightby DenisGopan 2007
(cid:13)
AllRightsReserved
i
Tomy daughter,Yunna.
ii
ACKNOWLEDGMENTS
First of all, I would like to thank my adviser Thomas Reps. Under his guidance, I learned a
great deal about program analysis and software verification, as well as about other related areas
of computer science. More importantly, Tom has taught me how to address problems and how to
expressmythoughtsinwritinginawaycomprehensibletoothers. Ionlyhopethatatleastasmall
fraction ofTom’sdedicationto research had rubbed offon me.
I would like to thank Ras Bodik, who was my academic adviser early in my graduate-student
career, and Mooly Sagiv, with whom I had a pleasure of collaborating on several projects. Both
Ras and Mooly played important roles in my development as a scientist. Also, I am thankful to
Bertrand Jeannet, who taught me to appreciate the more formal (and the more arcane) aspects of
abstract interpretation; to Ethan Munson, who encouraged me to apply to graduate school; and to
Michael Gontar, who was both my companion and my guide when I took the first steps into the
area ofComputerScience.
I would like to thank the members of my Ph.D. committee, Somesh Jha, Ben Liblit, Marvin
Solomon, and Amos Ron, for their comments on my thesis and for the insightful questions they
raised duringmydefense.
Duringmygraduatestudies,Iwasfortunatetohaveanumberofamazingofficemateswhohad
aprofoundinfluenceonmebothinsideandoutsideofmyresearch. Iamthankfultoeachandevery
oneofthem: Shai Rubin,Glen Ammons,Darryl Roy, Michael Brim, Nick Kidd, AlexeyLoginov,
and Evan Driscoll. Also, I would like to thank the members of programming languages research
groupandsecurityresearchgroupatWisconsin: SusanHorwitz,CharlesFischer,SomeshJha,Ben
Liblit, Glen Ammons, Gogul Balakrishnan, Mihai Christodorescu, Vinod Ganapathy, Nick Kidd,
Raghvan Komondoor, Akash Lal, Junghee Lim, Alexey Loginov, David Melski, Anne Mulhern,
iii
Manoj Plakal, Shai Rubin, Cindy Rubio, Hao Wang, Suanhsi Yong, and others. I am thankful
to Mooly’s students Tal Lev-Ami, Greta Yorsh, and others in Tel Aviv for very interesting (but,
unfortunately,very rare)discussions.
I would like to thank my daughter Yunna and my wife Julia for their unconditional love and
moralsupport. Theyprovidedtheinspirationfor meto completethisdissertation.
I amindebted tomy parentsTatyanaand Alex fortheirconstant supportandtheirbeliefin my
abilities. Ifit was not for them,I wouldhavenevermadeit thisfar. I am especially grateful to my
grandfather Anatoly whose dream was to see me become a researcher. I wish he had lived long
enough tosee theday. I also wouldliketothank my sistersOlgaand Ellen,my aunt Svetlana, my
grandma Lisa, my cousin Vitaliy, my uncle and aunt Lev and Nataliya, and other members of my
family: theyallplayed an importantroleinshapingmypersonality.
Last, but not least, I would like to thank my friends, Shura and Nata, Rost, Sasho, Taras and
Tamara,AlexandMaya,VadimandAnya,LevandErica,DmitriandIra,SergeiandJulia,Alexey
and Wendy, Liya, Igor Solunsky, Zurab and Julia, Alexey Samsonov, Rita and Seich, and many,
many others, for their support, encouragement, and for providing welcome distractions from my
research work.
I am sure that I am forgetting to mention some people who have contributed in some way to
thecompletionofmydissertation. To thosepeople,pleaseknowthatIam grateful.
Funding. My dissertation research was supported by a number of sources, including the Of-
fice of Naval Research, under grant N00014-01-1-0796, the UW Graduate School, under a Cisco
Systems Distinguished Graduate Fellowship, and the National Science Foundation, under grant
CCF-0540955. Iam gratefulfortheirsupport.
DISCARD THISPAGE
iv
TABLE OF CONTENTS
Page
LIST OFTABLES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viii
LIST OFFIGURES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix
ABSTRACT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi
1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.1 Program AnalysisBasics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 AFew Words onTerminology . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.3 NumericProgram Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.4 ThesisContributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.4.1 Contributionsat aGlance . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.4.2 Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
1.4.3 SummarizingAbstractions . . . . . . . . . . . . . . . . . . . . . . . . . . 8
1.4.4 Array Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
1.4.5 Guided StaticAnalysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
1.4.6 Interprocedural Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
1.4.7 Library Summarization . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
1.5 ThesisOrganization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
2 Numeric Program Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
2.1 NumericPrograms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
2.1.1 Numerical Expressionsand Conditionals . . . . . . . . . . . . . . . . . . 18
2.1.2 Support forNondeterminism . . . . . . . . . . . . . . . . . . . . . . . . . 18
2.1.3 EvaluationofExpressionsand Conditionals . . . . . . . . . . . . . . . . . 19
2.1.4 Concrete SemanticsofaProgram . . . . . . . . . . . . . . . . . . . . . . 19
2.1.5 CollectingSemantics ofaProgram . . . . . . . . . . . . . . . . . . . . . . 20
2.1.6 TextualRepresentation ofaProgram . . . . . . . . . . . . . . . . . . . . . 20
2.2 Program Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
2.2.1 Abstract SemanticsofaProgram . . . . . . . . . . . . . . . . . . . . . . . 22
v
Page
2.2.2 Abstract CollectingSemantics . . . . . . . . . . . . . . . . . . . . . . . . 22
2.3 IterativeComputation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
2.3.1 KleeneIteration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
2.3.2 Widening . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
2.3.3 Narrowing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
2.3.4 ChaoticIteration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
2.4 AbstractDomainInterface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
2.5 NumericAbstractions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
2.5.1 ThePolyhedral AbstractDomain. . . . . . . . . . . . . . . . . . . . . . . 30
3 Summarizing abstractions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
3.1 Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
3.1.1 Extended ConcreteSemantics . . . . . . . . . . . . . . . . . . . . . . . . 34
3.1.2 Summarization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
3.1.3 Standard Abstract DomainsRevisited . . . . . . . . . . . . . . . . . . . . 36
3.2 Summarizingabstraction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
3.3 Newoperations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
3.3.1 Theadd operation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
3.3.2 Thedropoperation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
3.3.3 Thefoldoperation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
3.3.4 Theexpand operation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
3.4 AbstractSemantics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
3.4.1 Assignmenttransitions: x φ(w ,...,w ) . . . . . . . . . . . . . . . . . 47
1 k
←
3.4.2 Assumetransitions: assume(ψ(w ,...,w )) . . . . . . . . . . . . . . . . 51
1 k
3.4.3 Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
3.5 SymbolicConcretization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
3.6 SupportforMultipleValues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
3.7 NumericextensionofTVLA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
3.8 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
4 AnalysisofArray Operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
4.1 OverviewofArray Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
4.2 Concretesemantics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
4.2.1 Concrete Program States . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
4.2.2 Array Transitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
4.3 Array Abstraction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
4.3.1 Array Partitioning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
4.3.2 NumericAbstraction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76
vi
Appendix
Page
4.3.3 Beyond summarizingdomains . . . . . . . . . . . . . . . . . . . . . . . . 77
4.4 Array Copy Revisited . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80
4.5 Implementationofan Array-AnalysisTool . . . . . . . . . . . . . . . . . . . . . . 85
4.5.1 OverviewofTVLA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
4.5.2 Modelingarrays . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87
4.6 ExperimentalEvaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88
4.6.1 Array initialization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89
4.6.2 Partial array initialization . . . . . . . . . . . . . . . . . . . . . . . . . . . 89
4.6.3 Insertionsort . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
4.6.4 AnalysisMeasurements . . . . . . . . . . . . . . . . . . . . . . . . . . . 92
4.7 Related work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93
5 Guided StaticAnalysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96
5.1 Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
5.1.1 StaticAnalysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
5.2 OverviewofGuidedStaticAnalysis . . . . . . . . . . . . . . . . . . . . . . . . . 100
5.3 GuidedStaticAnalysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105
5.4 FrameworkInstantiations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107
5.4.1 Wideningin loopswithmultiplephases . . . . . . . . . . . . . . . . . . . 107
5.4.2 Wideningin loopswithnon-deterministicallychosenbehavior . . . . . . . 110
5.5 DisjunctiveExtension . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113
5.6 LookaheadWidening . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116
5.6.1 ApproximationofLoopPhases . . . . . . . . . . . . . . . . . . . . . . . 116
5.6.2 Practical Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . 119
5.6.3 RevisitingtheRunningExample . . . . . . . . . . . . . . . . . . . . . . . 122
5.6.4 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122
5.7 ExperimentalEvaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126
5.7.1 Lookahead-WideningExperiments . . . . . . . . . . . . . . . . . . . . . . 127
5.7.2 Guided-Static-AnalysisExperiments . . . . . . . . . . . . . . . . . . . . . 130
5.8 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132
5.8.1 Controlledstate-spaceexploration . . . . . . . . . . . . . . . . . . . . . . 133
5.8.2 Wideningprecision . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133
5.8.3 Powerset extensions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135
6 Numeric Program AnalysiswithWeightedPushdown Systems . . . . . . . . . . . . 136
6.1 Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139
6.1.1 Program States . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139
6.1.2 Concrete SemanticsoftheCall Transition . . . . . . . . . . . . . . . . . . 141
vii
Appendix
Page
6.2 OverviewofWeighted PushdownSystems . . . . . . . . . . . . . . . . . . . . . . 142
6.2.1 PushdownSystems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142
6.2.2 Weighted PushdownSystems . . . . . . . . . . . . . . . . . . . . . . . . 144
6.2.3 WPDS inProgram Analysis . . . . . . . . . . . . . . . . . . . . . . . . . 146
6.2.4 SolvingtheGeneralized SuccessorProblem . . . . . . . . . . . . . . . . . 148
6.3 NumericProgram Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149
6.4 Widening . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156
7 Low-Level Library AnalysisandSummarization . . . . . . . . . . . . . . . . . . . . 159
7.1 OverviewoftheAnalysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161
7.1.1 AnalysisGoals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161
7.1.2 AnalysisArchitecture. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162
7.1.3 Thesummaryobtainedformemset . . . . . . . . . . . . . . . . . . . . . . 166
7.2 Intermediate-RepresentationRecovery . . . . . . . . . . . . . . . . . . . . . . . . 168
7.2.1 Variableand TypeDiscovery. . . . . . . . . . . . . . . . . . . . . . . . . 169
7.3 Numeric-ProgramGeneration . . . . . . . . . . . . . . . . . . . . . . . . . . . . 170
7.3.1 Numeric-Program Variables . . . . . . . . . . . . . . . . . . . . . . . . . 171
7.3.2 Basic Translationofx86Instructions . . . . . . . . . . . . . . . . . . . . 172
7.3.3 ValueDependenceGraph . . . . . . . . . . . . . . . . . . . . . . . . . . . 176
7.3.4 Memory-Safety Checks and AllocationBounds . . . . . . . . . . . . . . . 179
7.3.5 IntegerDivisionandRemainderComputations . . . . . . . . . . . . . . . 181
7.3.6 SymbolicMemoryConstants . . . . . . . . . . . . . . . . . . . . . . . . . 183
7.3.7 Numeric-Program Generation . . . . . . . . . . . . . . . . . . . . . . . . 186
7.4 NumericAnalysisandSummary Generation . . . . . . . . . . . . . . . . . . . . . 188
7.4.1 VariablePacking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189
7.4.2 Error Triggers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 190
7.4.3 Summary Transformers . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192
7.5 ExperimentalEvaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193
7.5.1 Case Study: MemoryFunctions . . . . . . . . . . . . . . . . . . . . . . . 195
7.5.2 Case Study: Stream Functions . . . . . . . . . . . . . . . . . . . . . . . . 197
7.6 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 203
8 Conclusions and Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 206
APPENDIX Proofs ofSeveral Lemmas andTheorems . . . . . . . . . . . . . . . . 224