ebook img

Static Analysis of x86 Executables PDF

199 Pages·2010·1.08 MB·English
Save to my drive
Quick download
Download
Most books are stored in the elastic cloud where traffic is expensive. For this reason, we have a limit on daily download.

Preview Static Analysis of x86 Executables

Dissertation zur Erlangung des Grades eines Doktors der Naturwissenschaften (Dr. rer. nat.) Static Analysis of x86 Executables Statische Analyse von Programmen in x86 Maschinensprache Dipl.-Inf. Johannes Kinder geb. in Mu¨nchen Eingereicht am 24. September 2010 Referent: Prof. Dr. Helmut Veith Korreferentin: Prof. Dr.-Ing. Mira Mezini Pru¨fungstermin: 17. November 2010 Fachbereich Informatik Technische Universita¨t Darmstadt Darmstadt – 2010 – D17 Erkl¨arung zur Dissertation Hiermit versichere ich, die vorliegende Dissertation ohne Hilfe Dritter nur mit den angegebenen Quellen und Hilfsmitteln angefertigt zu haben. Alle Stellen, die aus Quellen entnommen wurden, sind als solche kenntlich gemacht. Diese Arbeithatingleicherodera¨hnlicherFormnochkeinerPru¨fungsbeho¨rdevorge- legen. Darmstadt,den24.September2010 JohannesKinder Acknowledgments Firstandforemost,Iwouldliketothankmyadvisor,HelmutVeith,forhiscon- tinuing support and his valuable guidance in all aspects of academic life. He gave me considerable freedom in developing my own research agenda and al- ways trusted in my abilities. His uncomplicated way of leading our group al- lowedeveryonetodotheirbestandmadeiteasytofocusonresearchandteach- ingwithoutunnecessaryoverhead. Furthermore, I thank my colleagues for fruitful scientific discussions and the cheerfulhoursbothonandoffcampus. Evenintoughtimes,therewasalwaysan exceptionalspiritofcompanionshipandmutualsupport. Inparticular,Iwould liketothankFlorianZulegerforhiscontributionstoourworkoncontrolflowre- construction;AndreasHolzerforourfrequentdiscussionsabouttheCPAframe- work;him,VisarJanuzaj,andtheuntiringMichaelTautschnigforproofreading onshortnotice. Finally,IwanttothankmyparentsSusanneandHelmutKinderfortheirsup- portandtheirfirmbeliefinme,andAnne-SophieDo¨rnbrackforbeingmyemo- tional stronghold in the never-ending series of highs and lows that is graduate research. Darmstadt,November2010 JohannesKinder 3 Contents Abstract 13 Zusammenfassung (German Abstract) 15 1 Introduction 17 1.1 BenefitsofBinaryAnalysis . . . . . . . . . . . . . . . . . . . . . . . 18 1.1.1 AlternativetoSourceCodeAnalysis . . . . . . . . . . . . . 19 1.1.2 AnalysiswithoutAccesstoSourceCode . . . . . . . . . . 22 1.2 ChallengesinBinaryAnalysis . . . . . . . . . . . . . . . . . . . . . 24 1.3 TraditionalDisassemblyandAnalysis . . . . . . . . . . . . . . . . 30 1.4 OverviewontheProposedMethod . . . . . . . . . . . . . . . . . . 34 1.5 Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39 2 An Intermediate Language for Executable Analysis 41 2.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 2.2 Syntax . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43 2.2.1 Expressions . . . . . . . . . . . . . . . . . . . . . . . . . . . 44 2.2.2 BasicStatements . . . . . . . . . . . . . . . . . . . . . . . . 45 2.2.3 AbstractStatements . . . . . . . . . . . . . . . . . . . . . . 46 2.3 Types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47 2.4 Semantics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48 2.5 ILPrograms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51 2.6 RelatedWork . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53 5 Contents 3 Control Flow Analysis for Low Level Programs 57 3.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57 3.2 AWorkedExample . . . . . . . . . . . . . . . . . . . . . . . . . . . 60 3.3 ControlFlowSemantics . . . . . . . . . . . . . . . . . . . . . . . . 63 3.4 ControlFlowReconstructionbyAbstractInterpretation . . . . . . 65 3.4.1 TheResolveOperator . . . . . . . . . . . . . . . . . . . . . 66 3.4.2 AConstraintSystemforControlFlowAutomata . . . . . . 67 3.5 AlgorithmsforControlFlowReconstruction . . . . . . . . . . . . 74 3.5.1 GenericFixedPointAlgorithm . . . . . . . . . . . . . . . . 74 3.5.2 WorklistAlgorithm . . . . . . . . . . . . . . . . . . . . . . . 78 3.6 RelatedWork . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82 4 Bounded Address Tracking 87 4.1 PrecisionRequirements . . . . . . . . . . . . . . . . . . . . . . . . 88 4.2 PartitionedMemoryModel . . . . . . . . . . . . . . . . . . . . . . 90 4.3 AbstractDomainofAddressValuations . . . . . . . . . . . . . . . 92 4.4 AbstractSemantics . . . . . . . . . . . . . . . . . . . . . . . . . . . 95 4.4.1 BoundedPathSensitivity . . . . . . . . . . . . . . . . . . . 96 4.4.2 AbstractExpressionEvaluation . . . . . . . . . . . . . . . . 98 4.4.3 AbstractPostOperator . . . . . . . . . . . . . . . . . . . . . 101 4.5 AbstractionofNondeterminism . . . . . . . . . . . . . . . . . . . . 104 4.6 ImplementationIssues . . . . . . . . . . . . . . . . . . . . . . . . . 106 4.6.1 RepresentingByte-AddressableAbstractMemory . . . . . 106 4.6.2 RegisterAliasing . . . . . . . . . . . . . . . . . . . . . . . . 108 4.7 RelatedWork . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110 5 Disassembly and Static Analysis with Jakstab 113 5.1 GeneralArchitecture . . . . . . . . . . . . . . . . . . . . . . . . . . 113 5.1.1 SinglePassDisassemblyandAnalysis . . . . . . . . . . . . 114 5.1.2 SecondaryPost-ReconstructionAnalysis . . . . . . . . . . 116 5.1.3 ProgramRepresentation . . . . . . . . . . . . . . . . . . . . 117 5.1.4 ExecutionEnvironment . . . . . . . . . . . . . . . . . . . . 120 6 Contents 5.2 ModularImplementationofDifferentAnalyses . . . . . . . . . . . 123 5.2.1 ConfigurableProgramAnalysis . . . . . . . . . . . . . . . 124 5.2.2 ModificationstotheWorklistAlgorithm . . . . . . . . . . 127 5.2.3 BalancingSoundnessandCoverage . . . . . . . . . . . . . 129 5.2.4 CompositeAnalysiswithSelectiveMerging . . . . . . . . 132 5.3 AbstractDomainsinJakstab . . . . . . . . . . . . . . . . . . . . . . 134 5.3.1 LocationAnalysis . . . . . . . . . . . . . . . . . . . . . . . . 135 5.3.2 BoundedAddressTracking . . . . . . . . . . . . . . . . . . 136 5.3.3 ConstantPropagation . . . . . . . . . . . . . . . . . . . . . 139 5.3.4 StridedIntervalAnalysis . . . . . . . . . . . . . . . . . . . 141 5.3.5 CallStackAnalysis . . . . . . . . . . . . . . . . . . . . . . . 144 5.3.6 ForwardExpressionSubstitution . . . . . . . . . . . . . . . 145 5.3.7 LiveVariableAnalysis . . . . . . . . . . . . . . . . . . . . . 148 5.4 CodeTransformations . . . . . . . . . . . . . . . . . . . . . . . . . 149 5.5 RelatedWork . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151 6 Experiments 155 6.1 AnalyzingUntrustedDriverBinaries . . . . . . . . . . . . . . . . . 155 6.1.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156 6.1.2 WindowsDriverModel . . . . . . . . . . . . . . . . . . . . 156 6.1.3 OSAbstractionandDriverHarness . . . . . . . . . . . . . 157 6.1.4 ExperimentalSetup . . . . . . . . . . . . . . . . . . . . . . . 161 6.1.5 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161 6.1.6 AnalysisofCOTSDriverBinaries . . . . . . . . . . . . . . . 168 6.2 Disassembly . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 170 6.2.1 ProcedureEntryPointHeuristic . . . . . . . . . . . . . . . 170 6.2.2 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171 7 Conclusions 175 Bibliography 179 Curriculum Vitae 199 7 List of Figures 1.1 Exampleofpossibleprocedurelayoutsinanexecutable. . . . . . . 25 1.2 Exampleofoverlappinginstructionsinx86machinecode. . . . . 28 1.3 Executiontraceoftheexampleforoverlappinginstructions. . . . 28 1.4 Challengesinbinaryanalysisandtheproposedsolutions. . . . . 35 3.1 Controlflowreconstructionexample. . . . . . . . . . . . . . . . . 61 3.2 GenericControlFlowReconstructionAlgorithm. . . . . . . . . . . 75 3.3 WorklistControlFlowReconstructionAlgorithm. . . . . . . . . . 79 3.4 Addinganunknownnode((cid:62))withunlabelededgesleadstoad- ditionalpossiblevaluesfor attheindirectjump. . . . . . . . . . 83 x 4.1 Diagramofthelatticeofabstractaddressesandvalues Aˆ. . . . . . 93 4.2 Examplecodefragmentandfinalstatespace. . . . . . . . . . . . . 98 5.1 Unifieddisassemblyandanalysisarchitecture. . . . . . . . . . . . 114 5.2 SecondaryanalysisperformedonthereconstructedCFA. . . . . . 116 5.3 ThethreelevelsofprogramrepresentationinJakstab. . . . . . . . 118 5.4 DynamiclinkinginWindowsPEfiles. . . . . . . . . . . . . . . . . 120 5.5 CPA+algorithmfordeterminingthesetofreachablestates. . . . . 125 5.6 TheJakstabalgorithm,acontrolflowresolvingversionoftheCPA+ algorithm. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128 6.1 Simplifiedcodefrom floppy.cwithabstractVSA/ESPstates. . . 166 9 ListofFigures 6.2 Resultsofanalyzing322driverbinariesfromastandardWindows XP machine (a) using standard settings and (b) when ignoring weakupdates. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169 6.3 AverageresolverateofIDAProandJakstab(inheuristicmode). . 172 10

Description:
of procedures. These building blocks are combined into an extensible program analysis archi- tecture, which is implemented in a novel binary analysis tool. The tool the software is compiled into binary format and shipped, however, users further From a specification of the instruction semantics,.
See more

The list of books you might like

Most books are stored in the elastic cloud where traffic is expensive. For this reason, we have a limit on daily download.