Table Of Content

Generalized Hyper Markov Laws For Directed Acyclic Graphs Emanuel Ben-David Bala Rajaratnam 2 1 StanfordUniversity StanfordUniversity 0 2 n a Abstract J 1 Recent theoretical work ([15, 9]) in graphical models have introduced classes of flexible multi-parameter Wishart distributions for high dimensional Bayesian infer- ] T ence. A parallel analysis for DAGs or Bayesian networks, arguably one of the most S widelyusedclassesofgraphicalmodels,ishowevernotavailable. ForGaussianDAG h. models the parameter of interest is the Cholesky space of lower triangular matrices t withfixedzeroscorresponding tothemissingarrowsofadirectedacyclicgraph . In a G m thispaper weconstruct afamilyofDAGWishart distributions thatformarichconju- [ gate family of priors with multiple shape parameters for Gaussian DAG models, and proceedtoundertakeatheoreticalanalysisofthisclasswiththegoalofposteriorinfer- 3 v ence. Wefirst prove that our family of DAG Wishart distributions satisfies the strong 1 directed hyper Markov property. Operating on the Cholesky space we derive closed 7 form expressions for normalizing constants, posterior moments, Laplace transforms 3 4 and posterior modes, and demonstrate the use of the DAG Wishart class in posterior 9. analysis. Wethenconsider submanifolds oftheconeofpositivedefinitematrices that 0 correspond to covariance and concentration matrices of Gaussian DAG models. In 1 generalthesespacesarecurvedmanifoldsandthustheDAGWishartshavenodensity 1 : w.r.t Lebesgue measure. Hence tools for posterior inference on these spaces are not v immediately available. We tackle the problem in three parts, with each part building i X ontheprevious one,untilacompletesolution isavailable forALLDAGs. r In Part I we note that when is perfect, associated covariance and concentration a G spaces are open cones and hence weproceed toderive theinduced DAGWishart distribution on these cones. A comprehensive analysis is however only possible for the classofperfectDAGs. InPartIIweformallyestablish thatforanynon-perfect DAG, covariance and concentration spaces have Lebesgue measure zero in any Euclidean vector space containing them, and hence the DAG Wishart family introduced above does not have a density w.r.t. Lebesgue measure for non-perfect. We therefore G propose a unified approach for all Gaussian DAG models by appealing to the theory ofHausdorff measuretheory. Firstwederivethefunctional formoftheDAGWishart densityw.r.tHausdorffmeasure. Wedemonstratehoweverthatevenforthesimplestof graphs, the Hausdorff density isnotamenable toposterior analysis. InpartIIIwede- finenewspacesthatareprojections ofcovarianceandconcentration DAGspacesonto Euclidean space that yield natural isomorphisms. We exploit this bijection to derive the densities of DAG Wishart and DAG inverse Wishart distributions w.r.t Lebesgue 1 measure, and thus avoid recourse to Hausdorff densities. We demonstrate that this third approach is extremely beneficial and is readily amenable for high dimensional posterior analysis. We derive hyper Markov properties and posterior moments for DAGWishart andinverseWishart distributions corresponding toarbitrary DAGs,and notjustfortheclassofperfectDAGs. 1 Introduction Graphical models yield compact representations of the joint probability distribution of a multivariate random vector, and they have therefore proved to be very useful in discov- ering structure, especially in high-dimensional data. These models use nodes of graphs to represent components of a random vector, and edges between these nodes, to capture the relationships between these variables. In general these graphs can have three types of edges: directed, undirected or bi-directed. Undirected graphs are often used to represent association through conditional independences whereas bi-directed graphs are often used torepresentmarginalindependences. Directedacyclicgraphicalmodels(DAGs),orsome- times referred to as Bayesian networks1, are often used to represent causal relationships among random variables. Graphical Markov models corresponding to DAGs have useful statistical properties, especially in high dimensional settings. The joint probability density function (pdf) of a DAG model factorizes according to the graph into the product of the conditional pdfs for each variable given its parents, and can thus lead to a substantial reduction in the dimensionalityof the parameter space. Directed acyclic graphical models havealsofoundwidespreaduseinthebiomedicalsciences,socialsciencesandincomputer science. Estimating the covariance or inverse covariance corresponding to such DAGs is thereforean importantarea ofresearch, especiallyin highdimensionalsettings. FromatheoreticalstatisticsperspectiveDAGmodelscorrespondtocurvedexponential families, and are distinctly different from the standard undirected or concentration graph models, which correspond to natural exponential families. Unlike in the natural exponential family setup, a default prior, as given by the Diaconis-Yvilsaker (DY) framework, is notavailableforageneralDAG(see[9]forathoroughdiscussion). Aconnectionbetween DAGsandundirectedorconcentrationgraphscanhoweverbeusedtoderiveadefaultprior forasubclasscalledperfectgraphs. Indeedifthegraphis“perfect”2,suchDAGsaresaidto beMarkovequivalenttodecomposableconcentrationgraphmodels,i.e.,theybothcapture the same set of conditional independences. This connection can be exploited in the sense thatinferentialtoolsforperfectDAGscan beborrowedfromthedecomposableconcentra- tion graph setting. More specifically, in their pioneering work, Dawid and Lauritzen [7] developedtheDYpriorforthisclassofmodelswhenthegraphisdecomposable. Inpartic- ular,theyintroducedthehyper-inverseWishartastheDYconjugatepriorforconcentration graphmodels. Thisworkhasbeenextendedbytherecentmethodologicalcontributionsby LetacandMassam[15]whodeveloparichfamilyofmulti-parameterconjugatepriorsthat subsumes the DY class. Both the hyper inverse Wishart priors and the “Letac-Massam” 1SometimesalsocalledrecursiveMarkovmodels. 2Aconceptthatwillbeformallydefinedlater. 2 priors have attractive properties which enable Bayesian inference, with the latter allow- ing multiple shape parameters and hence suitable in high-dimensional settings. Bayesian procedures corresponding to these Letac-Massam priors have been derived in a decision theoretic framework in therecent work of Rajaratnam et al. [19]. A parallel theory for the classofGaussiancovariancegraphmodels,graphicalmodelswhichencodemarginalinde- pendencies, has been recently developed by Khare and Rajaratnam ([9], [10]). With a few exceptions(see[21]forinstance),alloftheabovemethodologicalcontributionsareforde- composable graph models. Furthermore, the results for undirected or concentration graph models cannot be carried over to DAGs when is no longer perfect. This is because the G MarkovequivalencepropertybetweenDAGsandundirectedgraphicalmodelsbreaksdown in the sensethat the DAG model,and the undirected graphical model, capture different set ofconditionalindependencieswhen G isnotperfect (orequivalentlynon-decomposable). The literatureon graphical modelsin general, and DAGs in particular, is extensiveand thus we do not undertake a literature review of the work in this area. We shall however briefly reviewwork that isdirectly relevanttothispaper when notationand terminologyis introduced. The principal objective of this paper is to develop a framework for flexible high dimensional Bayesian inference for Gaussian DAG models, i.e., for the class of Gaussian distributionsthat havethedirected Markovproperty. Thepreviouslyestablishedclasses of generalized multi-parameterWishartdistributionsdevelopedbyLetacandMassam[15]in theconcentrationgraph setting,and byKhare and Rajaratnam ([9]in thecovariancegraph model setting, are not directly applicable to the general DAG setting, though they provide usefulinsights,as willbedemonstratedinthispaper. For Gaussian DAG models the parameter of interest, denoted by Θ , is the space of G lowertriangularmatriceswithfixedzeroscorrespondingtothemissingarrowsofadirected acyclic graph . We introduce a rich class of generalized multi-parameter DAG Wishart G distributionsonΘ wasproposed,andstudiedwiththeexplicitgoalofBayesianinference G in high dimensional settings. This family extends the classical Wishart distribution in the sense as the latter becomes a special case of our family of DAG Wishart distributions. A comprehensive analysis of this family of generalized Wishart distributions was possible for arbitrary DAGs when working on Θ . Indeed analytic expressions for posterior mo- G ments, Laplace transforms, posterior modes and hyper Markov properties are established. The hyper Markov property in turn enables the explicit computation of expected values and Laplace transforms in the Cholesky parameterization. Unlike their concentration and covariance graph counterparts, we show that sampling from our Wishart distribution for an arbitrary DAG model does not require recourse to MCMC. Once more we note that for concentration graph models, sampling from the posterior can be done in closed form only for decomposablemodels. Forcovariancegraph model, samplingfrom the posteriorwith- out recourse to MCMC can be done only for homogeneous graphs. We also show that our DAGWishartdistributionscan bederivedinanequivalentwayusingthegeneral approach in [7] under the so-called global independence assumption. The latter approach does not howeverimmediatelygiveameanstospecifyhyper-parametersthatwillcorrespondtoour DAG Wishartdistributions. Wealsoprovideadiscussionofthefact that ourDAG Wishart distributions are in general different from the Letac-Massam priors. However, when the 3 underlying graph is homogeneous, the Letac-Massam W priors are a special case of P G our distributions. We also provide a discussion of the fact thaGt our DAG Wishart distributions are in general different from the priors introduced in Khare and Rajaratnam([9] for covariancegraph models. AfterintroducingourclassofflexibleHyperMarkovlawsweexplicitlytackletheques- tionofderivingtoolsforBayesianinferenceforGaussianDAGmodels. Inordertodothis we then consider submanifoldsof the cone of positivedefinite matrices that correspond to covariance and concentration matrices of Gaussian DAG models. In general these spaces are curved manifolds and thus the DAG Wisharts have no density w.r.tLebesgue measure. Hencetoolsforposteriorinferenceonthesespacesarenotimmediatelyavailable. Wepro- ceedtotacklethisprobleminthreeparts,witheachpartbuildingonthepreviousone,until acompletesolutionis availableforALLDAGs. In Part I we derive DAG Wishart densities for perfect DAGs. It was noted above that the space of covariance and concentration matrices corresponding to Gaussian DAGs are ingeneral curvedsub-manifoldsofEuclideanspace. When isperfect however,theseare G open cones and the induced DAG Wishart density on these cones can be derived. We then proceed to derive Laplace transforms and expected values in this setting. Computation of expectedvaluesofcovarianceandconcentrationmatricescorrespondingtoDAGmodelsis nolongerpossiblewiththisapproach,exceptwhen isperfect,asthespaceonwhichthese G matrices live are in general curved manifolds. We note that a comprehensive framework for Bayesian inference that goes beyond “perfect” graphs is however critical for practical applications. The induced Wishart and inverse Wishart distributions on concentration and covariance spaces for general DAGs require more sophisticated tools and is the subject of Parts IIand III. In Parts II we undertake the endeavor of deriving the induced Wishart and inverse WishartdensitiesoncovarianceandconcentrationspacescorrespondingtoarbitraryDAGs. We first establishthat for any non-perfect DAG, covarianceand concentration spaces have Lebesgue measure zero in any Euclidean vector space containing it, and hence the DAG P Wishartfamilyπ introducedinourpreviousworkdoesnothaveadensityw.r.t. Lebesgue UG,α measure. We propose to overcome this in two novel ways. First we derive the functional form of the density of πP w.r.t Hausdorff measure by developing the appropriate tools UG,α which allow us to work on concentration spaces corresponding to DAGs. This approach entailsworkingwithcurvedmanifoldsandHausdorffmeasuresonarbitrary metricspaces. Wethenproceedtodemonstratethatevenforthesimplestofgraphs,theHausdorffdensity isnot amenableto posterioranalysis. In Part III we define new spaces that are projections of covariance and concentration DAGspacesontoEuclideanspacethatyieldnaturalisomorphisms. Inparticular,thesenew spaces termed as the space of “incomplete” covariance and concentration spaces correspond to functionally independent elements of the covariance and concentration matrix of GaussianDAGmodels. Givenincompletematricesfromthesespaces,itisalwayspossible to “complete” them in polynomial time, so that the completion corresponds to covariance and concentration matrices of Gaussian DAG models. We exploit these bijections to derive the densities of DAG Wishart and DAG inverse Wishart distributions w.r.t. Lebesgue measure and thus avoid recourse to Hausdorff densities. We demonstrate that the latter 4 approach isnovel,extremelybeneficial, and isreadilyamenableforhighdimensionalpos- terioranalysis. We then proceed to establishhyperMarkov properties and deriveposterior moments for DAG Wishart and inverse Wishart distributions corresponding to arbitrary DAG models and not just for the class of perfect DAGs. In doing so we succeed in developing a unified framework for all Gaussian DAG models - that is suitable for both perfect andnon-perfect DAGs. Ourapproach alsoallowsusto formallydemonstratethattheclass ofinverseDAGWishartsintroducedinthispapernaturallycontainsanimportantsub-class ofinverseWishartdistributionsforthatwasintroducedbyKhareandRajaratnam[9]inthe contextofGaussiancovariancegraph models. Table1summarizesthepropertiesofthevariousmulit-parameterWishartdistributions that havebeen recently introducedto themathematicalstatisticsliteratureforusein Gaus- siangraphicalmodels. ItisclearfromthistablethattheWishartdistributionsintroducedin thispaperareapplicableinallgenerality-andnotjustwhenthegraphisperfect, orequivalently, decomposable, and in this sense very powerful. The ability to specify the induced Wishartdistributionsand posteriormomentsforarbitrary graphsisespecially useful. DAG UG COVG ALL P H ND D H ND D H Conjugacy property " " " % " " % " " Normalizingconstant in closedform " " " % " " % % " Posteriormoments in closedfrom " " " % " " % % " Posteriormode in closedfrom " " " % " " % % " HyperMarkov properties " " " % " " % % " Tractablesamplingfrom thedistribution " " " " " " % " " Table 1: Properties of Wishart distributions for the three classes of Gaussian graphical models. Abbreviations. ND: Non-decomposable,D/P: Decomposable/Perfect, H:Homogeneous. This paper is structured as follows. Section 2 introduces required preliminaries, notation and Section 3 formally defines Gaussian DAG models and parameterization corre- spondingtoGaussianDAGmodels. Theintroduction,preliminariesand parameterizations for DAG models are discussed in some detail to make the paper self-contained and for es- tablishingconsistent notation. These sections can be skipped by a reader familiar with the subject matter. In Section 5, the class of generalized Wishart distributions for Gaussian DAG models are formally constructed. Conjugacy to the class of Gaussian DAG models and necessary and sufficient conditions for integrability are established. Furthermore, 5 comparisonto conjugatepriors in concentration graph and covariancegraph modelsis un- dertaken. Section6establisheshyperMarkovpropertiesforourfamilyofpriors. InSection 7 we evaluateLaplace transforms, posteriormoments and posteriormodes for ourclass of distributionscorrespondingtotheCholeskyparameterizationwhenGisanarbitraryDAG. AnalysisofourDAGWishartdistributionsoncorrespondingcovarianceandconcentra- tionspaces witha view to developingtoolsforhigh dimensionalBayesian inference using the class of DAG Wishart distributions in three Parts. Part I considers the class of perfect DAGsandderivesposteriorquantities. PartI(Section8)derivestheinducedDAGWishart densities on covariance and concentration spaces for perfect DAGs. We the proceed to showthattheexpectedvaluesofthecovarianceandconcentrationmatrixcanbecomputed easilyforperfect DAGs. Part II (Section 9)introducesderivesthedensityofourpriors w.r.t. Hausdorffmeasure when is arbitrary, i.e., no longer perfect. Part III (Sections 10 and Section 11) defines G functionally independent projection of spaces of concentration and covariance matrices that correspond to arbitrary DAG models and proceed to derive the induced measure of our class of Wishart distributions on these spaces. Consequently we proceed to establish hyperMarkovpropertiesandderivetheexpectedvalueofthecovarianceandconcentration matrix for DAG models. We also demonstrate that when is no longer perfect the class G of DAG Wishart distributions do not belong to the class of general exponential families. Section 12concludes bysummarizingtheresultsinthepaper. 2 Preliminaries In this section, we give the necessary notation, background and preliminaries required in subsequentsections. 2.1 Graph theoretic notation and terminology In this subsection, we introducesomenecessary graph theoreticnotation and terminology. Ournotationpresented herecloselyfollowsthenotationestablishedin[13], [6]. Agraph isapairofobjects(V,E),whereV isafinitesetrepresentingthevertices(or G nodes)of ;and E isasubsetofV V consistingoftheedges. Anedge(i, j) E iscalled G × ∈ directed if (j,i) < E. We write this as i j and say that i is a parent of j, and that j is −→ a child of i. The set of parents of a vertex j is denoted by pa(j), and the set of children of a vertex i is denoted by ch(i). The family of j, denoted by fa(j), is fa(j) = pa(j) j . ∪ { } Two distinct vertices i and j are said to be adjacent if (i, j) or (j,i) are in E, i.e., if there is any type of edge, directed or undirected, between these two vertices. We write i j if ∼ thereisan undirectededge3 betweenverticesi and jand saythati isaneighborof j, j isa neighborofi,ori and jare neighbors. Theset ofneighborsofiis denotedbyne(i). More generally, for A V we define pa(A), ch(A), ne(A) and bd(A) as the collection ⊂ 3Notethatinenumeratingthenumberofedgesofagraph,eachundirectededge,thoughconsistingoftwo pairs,countsonlyonce. 6 of the parents, children, neighbors, and boundary respectively, of the members of A, but excludingany vertexin A: pa(A) = pa(i) A, ch(A) = ch(i) A, ne(A) = ne(i) A, i A i A i A ∪∈ \ ∪∈ \ ∪∈ \ An undirected graph, “UG”, is a graph with all of its edges undirected, whereas a directed graph,“DG”, isagraph withallofitsedgesdirected. Weshallusethesymbol to G denoteageneral graph,andmakeclearwithinthecontextinwhichitisused,whether is G undirectedordirected. We say that the graph = (V ,E ) is a subgraph of = (V,E), denoted by , ′ ′ ′ ′ G G G ⊂ G if V V and E E. In addition, if and E = V V E, we say that is an ′ ′ ′ ′ ′ ′ ′ ⊂ ⊂ G ⊂ G × ∩ G induced subgraph of . We shall consider only induced subgraphs in what follows. For a G subset A V, the induced subgraph = (A,A A E) is said to be the graph induced A ⊂ G × ∩ by A. A graph is called complete if every pair of vertices are adjacent. A clique of G G is an induced complete subgraph of that is not a subset of any other induced complete G subgraphsof . Moresimply,asubset A V iscalledacliqueiftheinducedsubgraph A G ⊂ G isacliqueof . G A path of length k 1 from vertex i to j is a finite sequence of distinct vertices v = 0 ≥ i,...,v = j in V and edges (v ,v ),...,(v ,v ) E. We say that the path is directed if k 0 1 k 1 k − ∈ at least one of the edges is directed. We say i leads to j, denoted by i j, if there is a 7−→ directed path from i to j. A graph = (V,E) is called connected if for any pair of distinct G verticesi, j V thereexistsapathbetweenthem. Ann-cyclein isapathoflengthnwith ∈ G the additional requirement that the end points are identical. A directed n-cycle is defined accordingly. A graph is acyclic if it does not have any cycles. An acyclic directed graph, denotedby DAG (orADG),is adirected graphwith nocyclesoflengthgreater than1. Theundirectedversionofagraph = (V,E),denotedby u = (V,Eu),istheundirected G G graphobtainedbyreplacingallthedirectededgesof byundirectedones. Animmorality G in a directed graph is an induced subgraph of the from i k j. Moralizing an G −→ ←− immorality entails adding an undirected edge between the pair of parents that have the same children. Then the moral graph of , denoted by m = (V,Em), is the undirected G G graph obtained by first moralizing each immorality of and then making the undirected G version of the resulting graph. Naturally there are DAGs which have no immoralities and thisleads tothefollowingdefinition. Definition 2.1. A DAG is said to be “perfect” if it has no immoralities; i.e., the parents G of all vertices are adjacent, or equivalently if the set of parents of each vertex induces a completesubgraphof . G Given a directed acyclic graph (DAG), the set of ancestors of a vertex j, denoted by an(j), is the set of those vertices i such that i j. Similarly, the set of descendants of 7−→ a vertex i, denoted by de(i), is the set of those vertices j such that i j. The set of 7−→ non-descendants of i is nd(i) = V (de(i) i ). A set A V is called ancestral when A \ ∪{ } ⊂ contains the parents of its members. The smallest ancestral set containing the subset B of V is denotedby An(B). 7 2.2 Decomposable graphs Anundirectedgraph issaidtobedecomposableifnoinducedsubgraphcontainsacycle G of length greater than or equal to four. The reader is referred to Lauritzen [13] for all the commonnotionsofdecomposablegraphsthatwewillusehere. Onesuchimportantnotion is that of a perfect order of the cliques. Every decomposable graph admits a perfect order ofitscliques. Let(C , ,C )beonesuchperfectorderofthecliquesofthegraph . The 1 k ··· G historyforthegraph is givenby H = C and 1 1 H = C C C , j = 2,3, ,k, j 1 2 j ∪ ∪···∪ ··· and the(minimalvertex)separators ofthegraph aregivenby S = H C , j = 2,3, ,k. j j 1 j − ∩ ··· Let R = C H for j = 2,3, ,k. j j j 1 \ − ··· Let k k 1 denote the numberof distinctseparators and ν(S) denotethe multiplicityof ′ ≤ − S,i.e.,thenumberof jsuchthatS = S. Generally,wewilldenotebyC thesetofcliques j ofagraph and byS itsset ofseparators. G G 2.3 Markov properties for directed acyclic graphs Let V be a finite set of indices and (X) a collection of random variables, where each X i i V i ∈ is a random variable on the probability space . Let the probability space be defined i X X as the product space = . Now let = (V,E) be a DAG. For simplicity, and i V i X ×∈ X G without loss of generality, we always assume that the given DAG is connected and the G edgeset E containsall theloops(i,i),i V4. Wesay thataprobabilitydistribution P on ∈ X has the recursive factorization property w.r.t. , denoted by DF (the directed factorization G property), if there are σ-finite measures µ on and nonnegative functions ki(x,x ), i i i pa(i) X referred to askernels, defined on such that fa(i) X ki(y,x )dµ(y) = 1, i V, i pa(i) i i Z ∀ ∈ and P has adensity p, w.r.t. theproductmeasureµ = µ, givenby i V i ⊗∈ p(x) = ki(x,x ). i pa(i) Yi V ∈ In this case, each kernel ki(x,x ) is in fact a version of p(x x ), the conditional dis- i pa(i) i pa(i) | tribution of X given X . An immediate consequence of this definition is the following i pa(i) lemma. 4Forconveniencewedrawthegraphswithouttheirloops. 8 Lemma 2.1. (from [13]) If P admits a recursive factorization w.r.t. the directed graph , G thenitalsoadmitsafactorizationw.r.t. theundirectedgraph m,and,consequently,obeys G theglobalMarkovproperty5 w.r.t. m. G Proof. Note that for each vertex i V the set fa(i) is a complete subset of m. Thus if ∈ G we define ψ (x ) = ki(x,x ), then p(x) = p(x x ) = ki(x,x ) = fa(i) fa(i) i pa(i) i V i pa(i) i V i pa(i) ψ (x ). Therefore, P admits a factorization ∈w.r.t. | m and by pr∈oposition 3.8 in i V fa(i) fa(i) Q Q [13∈]italso obeystheglobalMarkovpropertyw.r.t. m. G (cid:3) Q G Another direct implication of the DF property is that if P admits a recursive factorization w.r.t. , then, for each ancestral set A, the marginal distribution P admitsa recursive A G factorization w.r.t. the induced graph . Combining this result with Lemma 2.1 we ob- A G tain the following: P admits a recursive factorization w.r.t. then A BS [P], whenever G ⊥ | A and B are separated by S in ( )m. We call this property the directed global An(A B S) G ∪ ∪ Markov property, DG, and any distribution that satisfies this property is said to be a directed Markov field over . For DAGs the directed Markov property plays the same role G as the global Markov property does for undirected graphs, in the sense that it provides an optimalruleforrecoveringtheconditionalindependencerelationsencodedbythedirected graph. We now introducebelow anotherMarkov property for DAGs. A distribution P on is X saidto obeythedirectedlocal Markovproperty(DL)w.r.t. ifforeach i V G ∈ i nd(i) pa(i). ⊥ | Now for a given DAG consider the so-called “parent graph” defined as follows: The G parent graph of is a DAG isomorphic to and obtained by relabeling the vertex par G G G set V as 1,2,..., V , in such a way that pa(i) i + 1,..., V for each vertex i V. It | | ⊂ { | |} ∈ is easily shown that for any given DAG it is possibleto relabel the vertices so that parents always have a higher numbering that their respective children though such an ordering is not unique in general. For a given parent ordering we say that P obeys the parent ordered Markovproperty(PO) w.r.t. ifforeveryvertexi wehave G i i+1,..., V pa(i) pa(i). ⊥ { | |}\ | It can be shown that if P has a density w.r.t. µ, then P obeys one of the directed MarkovpropertiesDF,DG,DL,POifandonlyifitobeysallofthem,i.e.,thefourMarkov propertiesforDAGs areequivalentundermildconditions[13]. 3 Gaussian directed acyclic graphical models In this section we focus on multivariate Gaussian distributions which obey the directed Markov property w.r.t. a DAG . From now on and unless otherwise stated, we shall G always assume without loss of generality that = (V,E) is given in a parent ordering. G 5see[13]fordefinition. 9 A Gaussian Bayesian network over (or Gaussian DAG over ), denoted by N ( ), is G G G thestatisticalmodel that consistsofall multivariateGaussian distributionsN (µ,Σ) which m follow the directed Markov property w.r.t. where µ Rm and Σ PD (R), the set of m G ∈ ∈ m mreal positivedefinitematrices. × 3.1 Linear recursive properties of Gaussian DAGs Let x = (x ,...,x )t be a random vector in Rm with the multivariatedistributionN (0,Σ). 1 m m Considerthesystemoflinearrecursiveregressionequations: x +β x +β x + +β x = ǫ orequivalently x = β x β x β x +ǫ 1 12 2 13 3 1m m 1 1 12 2 13 3 1m m 1 ··· − − −···− x +β x + +β x = ǫ x = β x β x +ǫ 2 23 3 2m m 2 2 23 3 2m m 2 ··· . . − −···− . . . . x = ǫ x = ǫ , m m m m where β is the partial regression coefficient of x (j > i) in the regression of x on ij j i − its predecessors x ,...,x ,...,x . Now β is zero if and only if i y i + 1,..., V i+1 j m ij { | |} \ pa(i) pa(i) . Hence the partial regression coefficient β is zero if there does not exist an ij | arrow from j to i, i.e., j < pa(i), j > i. In addition,theresidualsǫ are normallydistributed i and mutually independent with mean zero and variance σ2 . We can rewrite the first ipa(i) system of equations in the form of a linear system Bx = ǫ, w|here B is the upper triangular matrix 1 β ... β x ǫ 12 1m 1 1 0 1 ... β x ǫ From thisweobtaBin:= 00 ...... .0.. 12...m, x = x...m2 and ǫ = ǫ...m2. Var[Bx] = Var[ǫ] BΣBt = diag(σ2 ,...,σ2 ,σ2 ) =: D ⇒ 1pa(1) m 1pa(m 1) mm | − | − Σ = B 1D(Bt) 1 − − ⇒ Σ 1 = BtD 1B. (3.1) − − ⇒ Thus,ifwedefineL = Bt,thenΣ 1 = LD 1Lt istheso-calledmodifiedCholeskydecompo- − − sition of Σ 1, in terms of the lower triangular matrix L and the diagonal matrix D 1. Now − − consider a DAG denoted by = (V,E). In [25] it has been shown that N (0,Σ) obeys the m G directed Markov property w.r.t. if and only if L = 0 whenever there is no arrow from i ij G to j, i.e., i < pa(j). Equation (3.1) above therefore gives a very convenient description of theGaussianBayesian networkN ( ). Weexplorethismodelinmoredetailbelow. G 10