Mathematical Methods of Engineering Analysis ErhanC¸inlar RobertJ.Vanderbei February2,2000 Contents SetsandFunctions 1 1 Sets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 Subsets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 SetOperations . . . . . . . . . . . . . . . . . . . . . . . . . 2 DisjointSets . . . . . . . . . . . . . . . . . . . . . . . . . . 3 ProductsofSets . . . . . . . . . . . . . . . . . . . . . . . . 3 2 FunctionsandSequences . . . . . . . . . . . . . . . . . . . . . . . . 4 Injections,Surjections,Bijections . . . . . . . . . . . . . . . 4 Sequences. . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 3 Countability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 4 OntheRealLine . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 PositiveandNegative . . . . . . . . . . . . . . . . . . . . . . 9 Increasing,Decreasing . . . . . . . . . . . . . . . . . . . . . 9 Bounds . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 SupremumandInfimum . . . . . . . . . . . . . . . . . . . . 9 Limits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 ConvergenceofSequences . . . . . . . . . . . . . . . . . . . 11 5 Series . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 RatioTest,RootTest . . . . . . . . . . . . . . . . . . . . . . 16 PowerSeries . . . . . . . . . . . . . . . . . . . . . . . . . . 17 AbsoluteConvergence . . . . . . . . . . . . . . . . . . . . . 18 Rearrangements. . . . . . . . . . . . . . . . . . . . . . . . . 19 MetricSpaces 23 6 EuclideanSpaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 InnerProductandNorm . . . . . . . . . . . . . . . . . . . . 23 EuclideanDistance . . . . . . . . . . . . . . . . . . . . . . . 24 7 MetricSpaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 Usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 DistancesfromPointstoSetsandfromSetstoSets . . . . . . 26 Balls . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 8 OpenandClosedSets . . . . . . . . . . . . . . . . . . . . . . . . . . 29 ClosedSets . . . . . . . . . . . . . . . . . . . . . . . . . . . 30 Interior,Closure,andBoundary . . . . . . . . . . . . . . . . 30 i OpenSubsetsoftheRealLine . . . . . . . . . . . . . . . . . 31 9 Convergence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34 Subsequences . . . . . . . . . . . . . . . . . . . . . . . . . . 35 ConvergenceandClosedSets . . . . . . . . . . . . . . . . . 36 10 Completeness . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37 CauchySequences . . . . . . . . . . . . . . . . . . . . . . . 37 CompleteMetricSpaces . . . . . . . . . . . . . . . . . . . . 38 11 Compactness . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40 CompactSubspaces . . . . . . . . . . . . . . . . . . . . . . 40 ClusterPoints,Convergence,Completeness . . . . . . . . . . 41 CompactnessinEuclideanSpaces . . . . . . . . . . . . . . . 42 FunctionsonMetricSpaces 45 12 ContinuousMappings . . . . . . . . . . . . . . . . . . . . . . . . . . 45 ContinuityandOpenSets . . . . . . . . . . . . . . . . . . . 46 ContinuityandConvergence . . . . . . . . . . . . . . . . . . 46 Compositions . . . . . . . . . . . . . . . . . . . . . . . . . . 47 Real-ValuedFunctions . . . . . . . . . . . . . . . . . . . . . 48 Rn-ValuedFunctions . . . . . . . . . . . . . . . . . . . . . . 48 13 CompactnessandUniformContinuity . . . . . . . . . . . . . . . . . 50 UniformContinuity. . . . . . . . . . . . . . . . . . . . . . . 51 14 SequencesofFunctions . . . . . . . . . . . . . . . . . . . . . . . . . 53 CauchyCriterion . . . . . . . . . . . . . . . . . . . . . . . . 54 ContinuityofLimitFunctions . . . . . . . . . . . . . . . . . 56 15 SpacesofContinuousFunctions . . . . . . . . . . . . . . . . . . . . 57 ConvergenceinC . . . . . . . . . . . . . . . . . . . . . . . . 57 LipschitzContinuousFunctions . . . . . . . . . . . . . . . . 58 Completeness . . . . . . . . . . . . . . . . . . . . . . . . . . 60 Functionals . . . . . . . . . . . . . . . . . . . . . . . . . . . 60 DifferentialandIntegralEquations 63 16 ContractionMappings . . . . . . . . . . . . . . . . . . . . . . . . . 63 FixedPointTheorem . . . . . . . . . . . . . . . . . . . . . . 64 17 SystemsofLinearEquations . . . . . . . . . . . . . . . . . . . . . . 69 MaximumNorm . . . . . . . . . . . . . . . . . . . . . . . . 69 ManhattanMetric . . . . . . . . . . . . . . . . . . . . . . . . 70 EuclideanMetric . . . . . . . . . . . . . . . . . . . . . . . . 70 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . 71 18 IntegralEquations . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71 FredholmEquation . . . . . . . . . . . . . . . . . . . . . . . 71 VolterraEquation . . . . . . . . . . . . . . . . . . . . . . . . 76 GeneralizationoftheFixedPointTheorem . . . . . . . . . . 77 19 DifferentialEquations . . . . . . . . . . . . . . . . . . . . . . . . . . 78 ii ConvexAnalysis 83 20 ConvexSetsandConvexFunctions . . . . . . . . . . . . . . . . . . . 83 21 Projection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86 22 SupportingHyperplaneTheorem . . . . . . . . . . . . . . . . . . . . 90 MeasureandIntegration 91 23 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91 24 Algebras . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93 MonotoneClassTheorem . . . . . . . . . . . . . . . . . . . 94 25 MeasurableSpacesandFunctions . . . . . . . . . . . . . . . . . . . 96 MeasurableFunctions . . . . . . . . . . . . . . . . . . . . . 96 BorelFunctions . . . . . . . . . . . . . . . . . . . . . . . . . 97 CompositionsofFunctions . . . . . . . . . . . . . . . . . . . 97 NumericalFunctions . . . . . . . . . . . . . . . . . . . . . . 97 PositiveandNegativePartsofaFunction . . . . . . . . . . . 98 IndicatorsandSimpleFunctions . . . . . . . . . . . . . . . . 98 ApproximationsbySimpleFunctions . . . . . . . . . . . . . 99 LimitsofSequencesofFunctions . . . . . . . . . . . . . . . 100 MonotoneClassesofFunctions . . . . . . . . . . . . . . . . 100 Notation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101 26 Measures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103 ArithmeticofMeasures. . . . . . . . . . . . . . . . . . . . . 104 Finite,σ-finite,Σ-finitemeasures . . . . . . . . . . . . . . . 104 SpecificationofMeasures . . . . . . . . . . . . . . . . . . . 105 ImageofMeasure . . . . . . . . . . . . . . . . . . . . . . . 106 AlmostEverywhere . . . . . . . . . . . . . . . . . . . . . . 106 27 Integration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108 DefinitionoftheIntegral . . . . . . . . . . . . . . . . . . . . 109 IntegraloveraSet . . . . . . . . . . . . . . . . . . . . . . . 110 Integrability . . . . . . . . . . . . . . . . . . . . . . . . . . . 110 ElementaryProperties . . . . . . . . . . . . . . . . . . . . . 110 MonotoneConvergenceTheorem . . . . . . . . . . . . . . . 111 LinearityofIntegration . . . . . . . . . . . . . . . . . . . . . 113 Fatou’sLemma . . . . . . . . . . . . . . . . . . . . . . . . . 113 DominatedConvergenceTheorem . . . . . . . . . . . . . . . 114 iii Sets and Functions This introductory chapter is devoted to general notions regarding sets, functions, se- quences,andseries.Theaimistointroduceandreviewthebasicnotation,terminology, conventions,andelementaryfacts. 1 Sets Asetisacollectionofsomeobjects. Givenaset,theobjectsthatformitarecalledits elements. GivenasetA,wewritex∈AtomeanthatxisanelementofA. Tosaythat x ∈ A,wealsousephraseslikexisinA,xisamemberofA,xbelongstoA,andA includesx. Tospecifyaset,onecaneitherwritedownallitselementsinsidecurlybrackets(if this is feasible), or indicate the properties that distinguish its elements. For example, A = {a,b,c}isthesetwhoseelementsarea,b,andc,andB = {x : x > 2.7}isthe setofallnumbersexceeding2.7. Thefollowingaresomespecialsets: ∅: Theemptyset. Ithasnoelements. N={1,2,3,...}: Setofnaturalnumbers. Z={0,1,−1,2,−2,...}: Setofintegers. Z ={0,1,2,...}: Setofpositiveintegers. + Q={m :m∈Z,n∈N}: Setofrationals. n R=(−∞,∞)={x:−∞<x<+∞}: Setofreals. [a,b]={x∈R:a≤x≤b}: Closedintervals. (a,b)={x∈R:a<x<b}: Openintervals. R =[0,∞)={x∈R:x≥0}: Setofpositivereals. + 1 2 SETSANDFUNCTIONS Subsets AsetAissaidtobeasubsetofasetB ifeveryelementofAisanelementofB. We write A ⊂ B or B ⊃ A to indicate it and use expressions like A is contained in B, B containsA, tothesameeffect. Thesets A and B arethesame, andthen we write A=B,ifandonlyifA⊂BandA⊃B. WewriteA6=BwhenAandBarenotthe same. ThesetAiscalledapropersubsetofB ifAisasubsetofB andAandB are notthesame. The empty set is a subset of every set. This is a point of logic: let A be a set; the claim is that ∅ ⊂ A, that is, that every element of ∅ is also an element of A, or equivalently, there is no element of ∅ that does not belong to A. But the last is obviouslytruesimplybecause∅hasnoelements. SetOperations LetAandBbesets.Theirunion,denotedbyA∪B,isthesetconsistingofallelements that belong to either A or B (or both). Their intersection, denoted by A∩B, is the setofallelementsthatbelongtobothAandB. ThecomplementofAinB,denoted by B \A, is the set of all elements of B that are not in A. Sometimes, when B is understoodfromcontext,B\AisalsocalledthecomplementofAandisdenotedby Ac. Regardingtheseoperations,thefollowinghold: Commutativelaws: A∪B = B∪A, A∩B = B∩A. Associativelaws: (A∪B)∪C = A∪(B∪C), (A∩B)∩C = A∩(B∩C). Distributivelaws: A∩(B∪C) = (A∩B)∪(A∩C), A∪(B∩C) = (A∪B)∩(A∪C). TheassociativelawsshowthatA∪B∪CandA∩B∩Chaveunambiguousmeanings. Definitionsofunionsandintersectionscanbeextendedtoarbitrarycollectionsof sets. LetI beaset. Foreachi ∈ I,letA beaset. TheunionofthesetsA ,i ∈ I,is i i thesetAsuchthatx∈Aifandonlyifx∈A forsomeiinI.Thefollowingnotations i areusedtodenotetheunionandintersectionrespectively: [ \ A , A . i i i∈I i∈I 1. SETS 3 WhenI =N={1,2,3,...},itiscustomarytowrite ∞ ∞ [ \ A , A . i i i=1 i=1 Allofthesenotationsfollowtheconventionsforsumsofnumbers. Forinstance, n 13 [ \ A =A ∪···∪A , A =A ∩A ∩···∩A i 1 n i 5 6 13 i=1 i=5 stand, respectively, for the union over I = {1,...,n} and the intersection over I = {5,6,...,13}. DisjointSets Two sets are said to be disjoint if their intersection is empty; that is, if they have no elements in common. A collection {A : i ∈ I} of sets is said to be disjointed if A i i andA aredisjointforalliandj inI withi6=j. j ProductsofSets LetAandBbesets. Theirproduct,denotedbyA×B,isthesetofallpairs(x,y)with xinAandyinB. ItisalsocalledtherectanglewithsidesAandB. IfA ,...,A aresets,thentheirproductA ×···×A isthesetofalln-tuples 1 n 1 n (x ,...,x )wherex ∈ A ,...,x ∈ A . Thisproductiscalled,variously,arect- 1 n 1 1 n n angle,orabox,orann-dimensionalbox. IfA =···=A =A,thenA ×···×A 1 n 1 n isdenotedbyAn. Thus,R2 istheplane,R3 isthethree-dimensionalspace,R2 isthe + positivequadrantoftheplane,etc. Exercises: 1.1 Let E be a set. Show the following for subsets A,B,C, and A of E. i Here,allcomplementsarewithrespecttoE;forinstance,Ac =E\A. 1. (Ac)c =A 2. B\A=B∩Ac 3. (B\A)∩C =(B∩C)\(A∩C) 4. (A∪B)c =Ac∩Bc 5. (A∩B)c =Ac∪Bc 6. (S A )c =T Ac i∈I i i∈I i 7. (T A )c =S Ac i∈I i i∈I i 1.2 Letaandbberealnumberswitha<b. Find ∞ ∞ [ 1 1 \ 1 1 [a+ ,b− ], [a− ,b+ ] n n n n n=1 n=1 4 SETSANDFUNCTIONS 1.3 Describethefollowingsetsinwordsandpictures: 1. A={x∈R2 :x2+x2 <1} 1 2 2. B ={x∈R2 :x2+x2 ≤1} 1 2 3. C =B\A 4. D =C×B 5. S =C×C 1.4 Let A be the set of points (x,y) ∈ R2 lying on the curve y = 1/xn, n T 0<x<∞. Whatis A ? n≥1 n 2 Functions and Sequences Let E and F be sets. With each element x of E, let there be associated a unique elementf(x)ofF. Thenf iscalledafunctionfromE intoF,andf issaidtomapE intoF. Wewritef :E 7→F toindicateit. Letf beafunctionfromE intoF. For xinE, thepointf(x)inF iscalledthe imageofxorthevalueoff atx. Similarly,forA⊂E,theset {y ∈F :y =f(x)forsomex∈A} iscalledtheimageofA. Inparticular,theimageofEiscalledtherangeoff. Moving intheoppositedirection,forB ⊂F, 2.1 f−1(B)={x∈E :f(x)∈B} iscalledtheinverseimageofBunderf. Obviously,theinverseofF isE. Termslikemapping,operator,transformationaresynonymsfortheterm“function” withvaryingshadesofmeaningdependingonthecontextandonthesetsEandF. We shallbecomefamiliarwiththemintime. Sometimes, wewritex 7→ f(x)toindicate themappingf; forinstance, themappingx 7→ x3 +5fromRintoRisthefunction f :R7→Rdefinedbyf(x)=x3+5. Injections,Surjections,Bijections Letf beafunctionfromEintoF. Itiscalledaninjection,orissaidtobeinjective,or issaidtobeone-to-one,ifdistinctpointshavedistinctimages(thatis,ifx6=yimplies f(x) 6= f(y)). It is called a surjection, or is said to be surjective, if its range is F, in which case f is said to be from E onto F. It is called a bijection, or is said to be bijective,ifitisbothinjectiveandsurjective. ThesetermsarerelativetoE andF. Forexamples,x7→ex isaninjectionfromR intoR,butisabijectionfromRinto(0,∞). Thefunctionx7→sinxfromRintoRis neitherinjectivenorsurjective,butitisasurjectionfromRonto[−1,1]. 2. FUNCTIONSANDSEQUENCES 5 Sequences A sequence is a function from N into some set. If f is a sequence, it is custom- ary to denote f(n) by something like x and write (x ) or (x ,x ,...) for the se- n n 1 2 quence(insteadoff). Then,thex arecalledthetermsofthesequence. Forinstance, n (1,3,4,7,11,...)isasequencewhosefirst,second,etc. termsarex = 1,x = 3,... 1 2 . IfAisasetandeverytermofthesequence(x )belongstoA,then(x )issaidto n n beasequenceinAorasequenceofelementsofA,andwewrite(x )⊂Atoindicate n this. A sequence (x ) is said to be a subsequence of (y ) if there exist integers 1 ≤ n n k <k <k <···suchthat 1 2 3 x =y n kn for each n. For instance, the sequence (1,1/2,1/4,1/8,...) is a subsequence of (1,1/2,1/3,1/4,1/5,...). Exercises: 2.1 Letf beamappingfromE intoF. Showthat 1. f−1(∅)=∅, 2. f−1(F)=E, 3. f−1(B\C)=f−1(B)\f−1(C), 4. f−1(S B )=S f−1(B ), i∈I i i∈I i 5. f−1(T B )=T f−1(B ), i∈I i i∈I i forallsubsetsB,C,B ofF. i 2.2 Show that x 7→ e−x is a bijection from R onto (0,1]. Show that x 7→ + logx is a bijection from (0,∞) onto R. (Incidentally, logx is the loga- rithmofxtothebasee, whichisnowadayscalledthenaturallogarithm. Wecallitthelogarithm. Letotherscalltheirlogarithms“unnatural.”) 2.3 Letf bedefinedbythearrowsbelow: 1 2 3 4 5 6 7 ··· ↓ ↓ ↓ ↓ ↓ ↓ ↓ 0 −1 1 −2 2 −3 3 ··· This defines a bijection from N onto Z. Using this, construct a bijection fromZontoN. 2.4 Letf :N×N7→Nbedefinedbythetablebelowwheref(i,j)istheentry th th inthei rowandthej column. Usethisandtheprecedingexerciseto constructabijectionfromZ×ZontoN. 6 SETSANDFUNCTIONS ... j 1 2 3 4 5 6 ··· i ... 1 1 3 6 10 15 21 2 2 5 9 14 20 3 4 8 13 19 4 7 12 18 5 11 17 6 16 . . . 2.5 FunctionalInverses. Letf beabijectionfromE ontoF. Then,foreach y in F there is a unique x in E such that f(x) = y. In other words, in thenotationof(2.1), f−1({y}) = {x}foreachy inF andsomeunique xinE. Inthiscase, wedropsomebracketsandwritef−1(y) = x. The resultingfunctionf−1 isabijectionfromF ontoE;itiscalledthefunc- tionalinverseoff. Thisparticularusageshouldnotbeconfusedwiththe general notation of f−1. (Note that (2.1) defines a function f−1 form F intoE,whereF isthecollectionofallsubsetsofF andE isthecollection ofallsubsetsofE.) 3 Countability TwosetsAandB aresaidtohavethesamecardinality,andthenwewriteA ∼ B,if there exists a bijection from A onto B. Obviously, having the same cardinality is an equivalencerelation;itis 1. reflexive: A∼A, 2. symmetric: A∼B ⇒B ∼A, 3. transitive: A∼BandB ∼C ⇒A∼C. Asetissaidtobefiniteifitisemptyorhasthesamecardinalityas{1,2,...,n}for someninN; intheformercaseithas0elements,inthelatterexactlyn. Itissaidto becountableifitisfiniteorhasthesamecardinalityasN;inthelattercaseitissaidto haveacountableinfinityofelements. In particular, N is countable. So are Z, N×N in view of exercises 2.3 and 2.4. Notethataninfinitesetcanhavethesamecardinalityasoneofitspropersubsets. For instance, Z ∼ N, R ∼ (0,1], R ∼ R ∼ (0,1); see exercise 2.2 for the latter. + + Incidentally,R ,R,etc. areuncountable,asweshallshowshortly. + A set is countable if and only if it can be injected into N, or equivalently, if and only if there is a surjection from N onto it. Thus, a set A is countable if and only if thereisasequence(x )whoserangeisA. Thefollowinglemmafollowseasilyfrom n theseremarks.