NORMAL LINEAR REGRESSION MODELS WITHRECURSIVE GRAPHICAL MARKOV STRUCTURE

by

Steen A. AnderssonMichael D. Perlman

TECHNICAL REPORT No. 312

March 1996

Department of Statistics

Box 354322

University of Washington

Seattle, Washington 98195 USA

NORMAL LINEAR REGRESSION MODELSWITH RECURSIVE GRAPHICAL MARKOV STRUCTURE*

30 1996

Abstract

A multivariate normal statistical model defined the Markov nriW\O,,-t ioc i!otrwm,;ni,'r!

an admits a recursive factorization of its likelihood functioninto the conditional LFs, each factor the form of a classicalt ivariate linear model MANOVA Here these models are ex-tended in a natural way to normal linear regression models whose LFs continue toadmit such recursive factorizations, from which maximum likelihood estimators andlikelihood ratio (LR) test statistics can be derived by classical linear methods. Thecentral distribution of the LR test statistic for testing one such multivariate normallinear regression model against another is derived, and the relation of these regres­sion models to block-recursive linear structural equations models is established. Itis shown how a collection of nonnested dependent normal linear regression models

seemingly unrelated regressions) can be combined into a single multivariate nor­mal linear regression model by imposing a parsimonious set of graphical Markovconditional independence) restrictions.

1. Introduction.

Graphical Markov models use graphs, either undirected, directed, or mixed, to rep­resent multivariate statistical dependencies. Statistical variables are represented by thenodes of the graph, local dependencies are specified by postulating that each vari-able is conditionally independent of all other variables given (for undirectedgraphs), or conditionally independent of its norulesceruianis (for directedgraphs). Although local dependencies may be relatively simple, complex multivariatedependencies are determined by the global structure of aspectsof graphical Markov models are surveyed the books Whittaker (1990), Edwards(1995), Lauritzen (1996), and Cox and Wermuth (1996) and paper by Coxand Wermuth (1993).

combineregression, tocussmg primarily on Oaus-

ltivariate normal previously concen-very simple structure , MANOVA

structure, see 6. the regression mean-value) subspace(e.g., Lauritzen and Wermuth (1989, Section 6». For such simple linear regression mod-

it is well-known that the joint likelihood function (LF) factors according to the graphinto the product of conditional LFs corresponding to lower-dimensional linear regressionmodels (in fact, this is true regardless of the mean-value assumptions), and, furthermore,that the joint parameter space factors into the product of parameter spaces associatedwith these lower-dimensional models.

We shall address the following question: under the Markov covariance structure de­termined by an ADG D, what is the largest class of linear regression mean-value)subspaces L for which the joint parameter space continues to factor according to D intothe product of the parameter associated with the family of conditional LFs? Ouranswer, presented in Section 6, is the class of D-imear subspaces, whose structure is char-acterized Sections 6 and illustrated a of in Section 13. Like

classical Mi"~NOVA models, ADG mooerssense conditional and associated parameter

classical multivariate normal linear regression modelcan be methods(MLE) and likelihood test statistics.

For the MarkovADG

and studied in Sections 7 and 8.normai linear ADG model againstof the LR test ti\'o,\,lti\,l<:

ADG models and block-recursive11. Section 12 we show

Zellner's (SUR) can be comomedinto a single parsimonious normal linear ADG model, then extend this to case nrh"'r£'

it is desired to test one such SUR model against another. Section 13 contains a ofexamples illustrating the ideas.

Since the class of normal ADG models includes the class normal conditionalindependence (LeI) models introduced in [AP] (1993) (cf. 1, 9.3, 10.1, 10.3,11.1, and 12.2), the in this paper may be regarded as extensions of those in(1993, 1994, 1995a)

2. Acyclic digraphs (ADGs).

A directed graph (digraph) D is a pair (V, R), where V is a finite set of vertices andR ~ (V x V) \ 6. is a binary relation (the set of directed edges) on V such that (u, v) E Rimplies (u, v) ¢ R. Here, 6. - 6.(V) is the diagonal {(v, v) E V}; thus loops and multipleedges are excluded from D. Vve use the customary arrow u -+ v in our figures to indicatethat (u, v) E R, but in the text this relation is indicated by u -<D v, or simply by u -< vwhen D is understood. The corresponding reflexive relation R := R U 6. is denoted by::::S = ::::SD· Thus u::::S v means that u -< v or u = v.

\Ve write u < v - u <D v if u -< v or there exist VI, ... , Vk E V, k > 1, such thatu -< VI -< ... -< Vk -< v. The relation < is transitive. The corresponding reflexive relationis denoted by ::; = ::;D.

An acyclic directed graph (ADG) is a directed graph D (V, R) with the propertythat v I- v for all v E V. Here the relation ::;D is a partial ordering on V, i.e., it isreflexive. antisymmetric, and transitive. Everv ADG D admits a never-decreasing listing/ ,; v ~

(not necessarily unique) of its vertices: V = {VI, ... , v,} where i < j =} Vj i Vi.

For an ADG D (V, R) and v E V, define paCv) := {u E V -< , the parents ofv; an(v) := {u E V!u < »}, the ancestors of v; de(v): {u E < u}, the descendantsv; and nd(v) := {71. E Vlv i u} = V \ (de(v) U {v}), the nondescendants of u. Note thatpa(v) ~ nd(v) and that , detu), and nd(v) depend on the relation -< only throughthe < set ~ V is if <;:; A v E

ancestral set depends on -< <. set AD.

"'r»" .......... diglraplrts (ADGs).

(VR) an ADG. We consider multivariate fJi-"JlJ<J'lJ,LHU)

X x(Xv E , whereexistence of i-VF,<Ai.'C»i.

COJ1VE:l1!C;ntly represented by a random VC,,", ",,"'AC

Iv E A), so x = zvA, respectively.

n""lr,,!'l<::p disjoint subsets A, B, C of V, we write

(3. A BIC[P]

) .

to indicate x B are conditionally independentI C if A = or B while A B 0

following elementar property of conditional independenceB, C, Fare CllS,]OlIlL subsets of V 1 then

(3.2) A B I (P] and All C I F U B [P] ~ B C F

Definition 3.1. A probability measure P on X is (local) Ir-Markovian if

(3.3) v (ndD(v) \ paD(v») I paD(v) [P] V v E V

Lauritzen ei al (1990, Proposition 4) define the global Markov property determined byD and show it is equivalent to the local Markov property for ADGs; thus the globalproperty need not be considered separately here. If {VI, ... , vq } is a never-decreasing listingof V, it follows from Proposition 5 of Lauritzen et al (1990) that P is D-Markovian iff

V1n 'P'l j,m 2, ... , q.

Themodel

if!J :

== P(D; X) determined by D simply, the ADGdonned to family of all D-Markovian distributions P on X.

",hPrp R~ R (i.e.,!J fewer edges D), acyclic property

c

is

nrooer ADG homomorphismnon-degenerate probability distribution,

C P(D).

Proof.never-decreasing listing of

combined sequence Wl1, ' .. ,

define the ADG E _is a

Wmk if l < k,

Wmk if n =1= m and V n <o vm ·

m 1, ... ,1'

S;;; peE). w«s S;;; S (E has fewershowing that P(D) = peE).

,,"H',CIC-lLh) are immediate from the construct.ion

Sinceshall complete proof

The followingand k 1, ... ,

(3.8) ).

(3.9) { } . '-l( I'Wml, ... ,Wm(k-l) UIf; nc D

hence

(3.10) \ (ndv

Suppose E P(D).10) to obtain

m 1, ... , r and k

so~ . . ,

L ... ,P E peE), so (3.1 m=l, ... ,randk

12)

The two relations obtained(3.2) to

k qm 1 k combine according to

(3.13)

Combineprocess

UkJ'J'CL"".'cAA from (3.12) k qm - 2,

(3.14)

Since {Wrnl,"" } = and paE(Wml) paD by (3.8), (3.14) (3.3),hence P E P(D).

(ii) Let y =(Yw IW E lV) E x (Y11 , IW E l-V) =X be a random variate such thatW 1= w', ui"; are mutually independent and independent of (Yw" Yw"), while Yw' and Yw"are dependent with non-degenerate distributions; denote the distribution of y by P. Byhypothesis, either ui" E ndE(w') \ paE(w') or w' E ndE(w") \ paE(w"). Since

w' -JL w" I paE(w') [P]

andw' !paE(w") [P]'

it follows from (3.3) that Prj. peE).Next, either -<D 'IjJ(w"). In the case, it ~ULlUV\'''' irnme-

diately from (3.6) and of P that for each v E ,V, \paD(v) are mutually independent, hence P trivially satisfies (3.3), so P E P(D).argument holds the case for v = then E paD

).

for some non-negative measurable functions k" on XvxXpa.(v) such that I k V( . ,

1 E Xpa.(v) and all v E V.

LJ-.lVlarkCiv18,n if1990, 1)).

nr,,,ri,,uot measure Jl on X.In thisif P a l)-I'eCUlrsi"e factorization.

conditional density p(-Ixpa(v)) Xv

of P assumes the following form:

(3.15) p(X) = IT( ) I v E V), X E X.

Remark 3.1. If (k V E V) is a family functions satisfying the above ,",VJ.H.li'J.lVUu, then

j5(X): IT( ) Iv E , XEX,

defines a probability density wrt Ji on X. To verify that JpdJl = 1,decreasing listing VI, ... , v, of V and integrate p iteratively over X V rn wrtThus by Proposition 3.2, P := p. Ji is D-Markovian.

choose a never­,m r, ... ,1.

4. Normal ADG models.

For the remainder of this paper, D (V, R) shall denote an ADG as in the precedingsection, but we now add the assumption that X; = RIv, v E V, where the L, are nonemptyfinite index sets, so that X = RI with 1:= u(Ivlv E V). The normal ADO model NI(D)is defined! to be the restriction of the Markov model P(D; R I ) to the class of nonsingularmultivariate normal distributions on RI. In this section we characterize P(D; I), the set ofIxI positive definite covariance matrices corresponding to the normal ADG model NI(D).

For any subset J ~ I and vector x - (Xiii E 1) E R I , define XJ Ii E J) E R J .

For any subset A ~ V, define I A u(Ivlv E A) and XA := XIA E RIA. Notex XI =Xv and define x0. O.

For any subsets J, K ~ I, M(J x K) denote/ the vector space of all real J x Kmatrices, P(J) the cone of all positive definite JxJ matrices, and set M(J) lVI( JxJ).Denote the JxJ identity matrix by 1J. For E E P(1), let E J K denote JxK subrnatrixof E 1 -1 J, C

2;-1K E

E I:

-<v~ u-<v>-.

E E P(I), we have

(4. ( )EP EP V>-f,

>- f, E[vf := ~ivl-i('"y,. E l\!I([v] x -/< v= (E[vr)t, )t,

E P(-/< v, E-.<vl := l:-.<vr E(E-,<vl y. Furthermore, r;pt'r1p

E x-<

(4.2)

and recall that IE[v].1

"E[vJ. := E[v],-.<vr E P([v]).

i-==":"'-;-, where lEI := det(E).

Definition 4.1. For E E P (I), the family of matrices

) Iv E V) E x (M([v] x -<v>-) x P([v]) Iv E V) TI(D; I)

is called the family of D-pornmeters of

Let jVI(~, E) denote the normal distribution on R I with mean vector ~ E R 1 andcovariance matrix E E P(I). Note that "N'I(~, E) E P(D; R I

) V~ E R I{:::=? .IVI (O, E) E

P(D;RI ) .

Definition 4.2. The sunset P(D; I) ~ P(I) is defined as follows:

l: E P(D; I) < ;. /....!J(O, E) E P(D; R I) ;

that is, E E P(D; I) x EV

Every E E P(D; I) is uniquely deterrnineo by its D-parameters xt:

1.

is hii'Grotiu£>

Proof. v E E II(D; l), apply rternark 3.1

to see

E

( -(vi-, A v)

is a prooaonitvc·is

(4.5)

wrt 11 : measure onnnciti"", semidefinite quadratic on, necessarily Q

Q(:c) tr(L;-l for some unique L; E P(l) and c- 1 -

card(I) and IL;l := det(L;). Set z = ° to ontain

= IT(IAu I I v E V),

which combines with yield

It follows from Proposition 3.2 that JVI(O, E) = p. fl E P(D; R 1) , hence L; E P(D; I).Furthermore, by (3.15) and the well-known fact that

(4.7)

when z rv NI(O, I;), we R: = E[vi- 1 and Au E[vb v E V, 1\D is surjective.Next suppose I;, E P(D; l) satisfy "D(I;) = "D(E'). By , JVI(O,NI(O, L;') determine the same family of conditional distributions, N I L;)

l'[I(O, by (3.15). Therefore L; = E', so "D is injective. completes proof.

\..-V'VCU.lCH.l\..-,-, set P(D; I) was defined indirectly to a fJL'-JLJOAJJ.J.J.uv.l\..-

Relations following proposition

II( vE

,I v E v) JrD in

follows from Proposition

(1993), it is seenmooers introduced in rAP]

[AMPT] (199Sa,b)that of all LeI models is a

those determined by trcnsitiue ADGs (also

Similarly, it is well-known Markov model determined aundirected graph coincides with some ADG model, but not conversely - see (AMP] (1996).Therefore, under assumption multivariate normality, the class of decomposablecovariosice selection models (Dempster (1972), Lauritzen (1996) is a proper subclass of

class of normal ADG models.

5. Reconstruction of the covariance matrix from its D-parameters.

It will be shown in Section 7 that for the normal ADG model N[(D), the maximumlikelihood estimate (MLE) t of the covariance matrix :E E P(D; I) is obtained in termsof the MLEs of its D-parameters JrD(:E). order to recover t, an explicit representationof the inverse JrDl of the mapping 3) is needed. This representation is now described

following Reconstruction a generalization of the algorithm given in [AP]I., rvrvcv C)\,1\::1\::1"::>, ~.

Let Vl,'" , V r be a never-decreasing H,;:)uHi}", of the elements in V. notational C011-

ve:mE~nc:e abbreviate rn, , -< V rn >- by -< m >-, and >- by >- .:E accorumg to

I

l}-para:me:teJ:'s r:D

x x-<m rn

o.

Since -< 2 >­Because -< 3 >- C

carried out.is a subrnatrix ... '".''-'1'''' and next may

3a:

It is important to noteE-<3>-0[3] are now determineddetermined. Since

Steps 1, 2, andcomplete matrix ---1"·1'-'1·"-1'-'101

the remaining (3] x ( \ -< 3 >-) submatrix ofmined from E[lJ0[2J by means of Proposition 4.2(ii):

fl1Llf21tjf3], denoted by I:[3} , is rlPtpt·_

Step 3b :

where E-<3} is the -< 3>- x ( ([1]U(2]) \ -< 3 >-) subrnatrix of E[I]0[2J'

After m - 1 such steps, the submatrix E[l]u"'U[m-lj is fully determined and in turnmay be used to obtain E[lju"'U[mj as follows. First note that the never-decreasing natureof VI, ... ,Vr implies that

(5.2)

(5.3) ([1]U· .. U

<rn»- ~ [1](j.· ·U(m 1],

1])\ -<m>- ~ f<m>f .

Thus, if we denote the [m] x ( . '" U[m 1])\ -< m >-) subrnatrix of I:[lju"

and the -< m >- x ( ([1]tJ ... eJ - 1}) \ -< m >-) subrnatrix by , it follows fromthat both I:-<m>- and are submatrices of 11. so that the next

J'

carried out to fully determine

Step m:

product of con­>U'-'AU>VV'U mrerence can

oecr ion 4 is

Definition 6.1. A linear subspace L <;;;; l\tl(I x isl\tI(I)L <;;;; L 01'1 equivalently (since 11 E :LVf(I», if M(I)L

It can be shown that L is a MANOVA subspace x

1) L = Nl(IxN)P

for some (necessarily unique) N x N orthogonal P =PL (P' P,p 2 P). For fixed I, this establishes a 1-1 corespondence all MANOVA subspacesL <;;;; M(I x N) and all N x N projection matrices P. Note dim(L) = III . tr(PL).

In tensor product notation, Nl(Ix N) = R 1 R N , and L is a MANOVA subspaceM(Ix N) iff

(6.2)

for some (necessarily unique) linear subspace K _ /{L <;;;; R N . For fixed I, this establishes a1-1 corespondence between all MANOVA subspaces L <;;;; M and all linear subspacesK <;;;; R N . It follows from (6.1) and (6.2) that /{L = R N PL, or equivalently, that Ki, is

row space of PL:

Remark 6. Any linear subspace L <;;;; NI(I x of

L L6ZIp E :LVI (I x T) }

subsoace. where T is a finite index set Z E NI(TxN) is a design matrix.ur h,e>rp (Zzt\- denotes

\ !

E

y:= E l\t1(Ix

Lx pel) is unique and

iVLLJL)" are

the maximum of nkeunoon tunction is

tnerrnore, the MLE (~, t) is a complete and sufficient statistic for the MANOVA model

classical MANOVA model NlxN(L), the matrix I; E PCl) is un­If, instead, the assumption that I; E P(D; I) is imposed, then the class of

subspaces can be replaced by the larger class D-linear subspace» L (see Def-We shall see that the resulting normal ADG model

E LxP(D; I))

J x N submatrix of ~,

two subspaces

retains most amenable features of~ E lVI(lxN) and any subset J s;: I,

subspace L ~ M(lxN) and

:= {~-<v>-I~ E L} s;: >- x

L-x c

Definition 6.3.

(

EX

E M(Ix

x

x P(I),

x x-<V>- xP(

) v E

v V)

is Ir-posometers (~, E).

Proposition 6.1. If L is l)-SUIJsY:fac1c, the mapping

1 1£D: L X P(D; I) ---+ x ( x x-<V>-)xP( Iv E V) =: nrz, D; I)

Thus, every (~, E) E Lx P(D; I) is uniquely determined by its D-parameters.

Proof. SinceIT(L, D; I) = (x(L[vllv E V») x IT(D;!) ,

the inclusionl£D(LxP(D;I) ~ IT(L,D;I)

follows from (4.3) and conditions (i) and (iii). To see that 1£D is injective, suppose that(~,E) l£D«(,E') for (~,E),«(,E') E LxP(D;I). Then E = E' by Proposition 4.1,

hence

(6.12) Ell.

Ch<JOE;e a never-decreasing listing VI, ... ,Vr of V apply (6.12) successively= VI,.·., ,together with (5.2), to obtain ~l = ~~, ... , = ~~j hence ~ ~'.

To see that is surjective, consider any «/1v , R v , i V E V) E n(L, D; I). Againa lh 1 ••• ,Vr of V and augment the general Step m of the

lte:const:ructJon Algorithm Section 5) with following additional relation:

Step

7. Maximum likelihood estimation in a normal linear ADG model.

By Proposition 4.2, likelihood function (LF)normal ADG model N1xN(L, D) in

(7.(LxP(D;I)) x l\1(Ix -+

((~, ,y) f-+ }1-tr2

For each v E V, let Pv . PL\vj E M(N) denote matrix corresponding tothe MANOVA subspace L[v] and set Qv := QLrvl = IN - P~,. By the orthogonalityand QVl the final expression in (7.1) has the following further factorization:

II( -n/2 { 1 ( -1IAvl .exp -2"tr Av (Y[vJPv JLv RvY--,<v>-Pv)("

.exp { - ~tr(A;l (Y[v]Qv RvY--,<v>-Qv)(' .. )t) } Iv E V),

where ((/Lv, R v, Av)lv E V) - 1r(~, 2::) are the D-parameters of (~, 2::) E Lx P(D; 1). ByProposition 6.1, the parameter space Lx P (D; I) factors into the produet of the ranges ofthe D-parameters.

It now follows readily from well-known results for the MAN OVA model that the MLE(~(y),f:,(y)) of (~,2::) is unique and exists for a.e. Y E l\1(IxN) [Lebesgue] if and only if

(7.2) n :::: max{ p., + I~ v>: I Iv E V},

",ho.,.o p., := tr(Pv ) = dim(Kv ) with K; row(Pu ) . In case,( (fiv, ic; Av) v E V) of MLE (~, t) are determined thesion estimators:

R v

D-parameters

D) is a curved exponential family in ge][leI:al, so a cornptete

ernpirical generalized Vdll<:lHI,-,t:.

to deriveis the MLE of 'E, obtained UH[JH~"U".Y

normal ADG model NIxoection 9 to central

testing one linear ADG model Gf.','Huul. «(LLUU1'ul

a never-decreasing listingwe , -< Vm >- by -< m >-, and Av m

Markov under model N1xN(L, D)1J, ... , same as the conditional distribution

, ,

conditional distribution of nA r nL:[r]. given Y[r-l],' .. ,distribution Y-<rr' By well-known results for the MANOVA conditionaldistribution is the Wishart distribution W(L:[r]., fr') -- PT - 1--< r >-freedom and expectation frL:[r]., where Pr := tr(Pr) with PT : P",.. this conditional

distribution does not depend on , ... , U[I], 'E[r]. is independent ).

For Tn T - 1, ... ,1, 'E[m]. depends on Y E M(Ix N) , ... , . By

repeating the preceding argument, we see that the conditional distribution of ni=[m]. givenY[m-lj, ... , is same as its conditional distribution Wishartdistribution vV'('E[mj., with l-. := n - Pm -1--< m >-1 degrees expectation

fm'E[m]., P·rn := tr(Pm) with Pm := PV rn ' Thus 'E[r].,"" 'E 1],···, Y[I])

are mutually independent.\Ve COIlCHme

8. Distribution

1\J are mutually moepenoent,

W(I:[mJ" m 1..... T.

above and equationempirical generalized variance

I..nderson ( , p.

E(

E

E V) is a disjointan family

C""'6/··t"'L~ ADG homomorphism

E

E vV), so that (3.6) and (3.7) hold X R I, X, RIvProposition 3.1(i), NI(E) is a submodet of NI(D), or equivalently,

<;; P(D; I). If '1/) is a proper homomorphism, it follows from a proof;::'111111<11. to that of Proposition 3.1(ii) that in fact NI(E) is a submodel of NI(D),

(E;!) c P(D; I).For the remainder this section, assume that L is a D-subspace of IVI(I x N) and

is an E-subspace of x N) such that 1\1 <;; L. Then NIxN(1i1, E) is a submodel ofNIxN(L, D), so we may consider the problem of testing NIxN(lv1, E) vs, NIxN(L, D).More specifically, based on an observation y rv Ns x N (~, L: 1N ), we may test

(9.2) Hu» : (~, L:) E A1 x peE; I) VB. HL,D: (~, L:) E Lx P(D;!).

Remark 9.1. It follows from Definition 6.2, (9.1), and the order-preserving propertyof 1}J that every D-subspace L of 1\1(I x N) is also an E-subspace of M(I x N), but theconverse is not valid." Therefore, the general problem (9.2) includes the following twotesting problems as special cases:

(9.3) HL,E: (~,L:) E LxP(E;I) VB. HL,D: (CL:) E LxP(D;I),

H " «M,E· <;,

In order to test

E AI x peE;!) VB. HL,E: (~, L:) E Lx I).

(9.5) HM,D: (~, E AlxP(D;/) VB. HL,D: (~, E LxP(D;I),

it M is a lJ-:m bspace M(/xN).

i.e., n 2': + I:5 v::: I v E }.

are unique a.e. y E M(IxN) under HL.D,

case the LR statistic A= A(Y) for testing HM,E vs. HL,D existsby

Proposition 9.

t(y)also under HivE E·,

a.e. y and is

Proof. The existenceshow the following:

uniqueness a.e. of t follows from (7.2). For W E vV we

(9.7)

(9.8)

these two relations imply that I :5 W ::: I < 1:5 'l/J(w) ::: I and Pw :::; P'Ij;( to), respectively.Therefore

n 2': max{pv + l:5v:::l! v E V}= max{ + I:5'¢(w)::: II wE vV}2': max{ Pw + I:5 w >- I Iw E vV},

implying the existence and uniqueness a.e. of to, also by (7.2). The two expressions for .\follow from (7.4).

To establish (9.7), first that (9.1) implies that

Iw ~ ,W E vV,

To begin the derivation of central distributionwhen (7.2) holds, choose a never-decreasmz llstmg

a never-decreasing nsting

se(iUE:nc:e WII, ... ,

, which in turn implies

(9.11)1, ... .tn 1, j=l, ... , { j = 1, ... ,k - 1 }

For each m 1, ... , r k 1, ... ,

I •...• k-l} .

As in Section 5, we shall usually abbreviate Vrn , , -< Vrn >-, [vrn >- by m, , -< rrt >-,[m>-, respectively, and Wrnk, [wrnkL -<Wmk>-, [Wmk>-, (Wmk) by mk, [mk], <mk >, [mk>-,(mk), respectively.

From (9.6), )., can be expressed as follows:

(9.12)

where

(9.13)

2/n - II -A - (1]mlm-1, ... .r},

T/rn := IT( II. ' I )."-'ulm,kl'! k = 1, ... ,Qm

Furthermore, by eqn. (2.21) [AP] (1995b), for m = 1, ... .r we have

(9.14) T/m = II(wmk I k = 1, ... ,qm),

where

15)

E

E

E

It follows from Proposition 6.1 with D replaced by(fLrnk, Rrn k , Arnk ) under H A4 ,E is

H.LI'/nkl X lVI([mk} x -< x

Therefore, the range of (fLnlk + R1n k Y -<rn k 'r E' is

where, for a.e. Y-<rnk'rE E M(-<mk>-E xN), A1(Y-<rnkhJ ~

subspace''x is MANOVA

(9.18)

Next, let E = (~~ 8) be the ADG constructed from E and D in the proof of Propo­sition 3.1; recall from (3.8) that

(9.19) -<mk>-f; = -<m>-U[m1}U"'U[m(k I)} ~

By (3.4) and (3.2), under hypothesis HL,D

is the following (recall Footnote 3):conditional

(9.20)

where now

E

- E -<

- E

It follows from Proposiuonunder HL,D is

x l\1([mkJx <mk

+

a.e . .<I-crnk;""

L(Y-<mkrE) x P([mkJ),

E l\1(-<mk>-iJ xN), L(Y-cmkrE) ~ J\!I([mk]x is the MANOVA

23)

L(Y-<mkrE):= L[mk) {RrnkY-<rnkrE I Rmk E M([mk] x -<mk>-iJ)}'

now that W l1 tk in (9.15) can be expressed as

It[mkj.-<mkrE IWrfLk = A .'

II:O[rnkJ.-<mkr EI

By (3.5) with fJ,D replaced by E,E, paE(mk) ~ pajJ;(mk) , hence -<mk>-E ~ -<mk>-jJ;'~ L implies that lvf[rnkJ ~ L[mkj, hence lvI(Y-<rnkrE) ~ L(Y-<mkrE)' Thus for

each fixed Y(mk) E M( (mk) x N), we may consider the conditional problem of testing thenormal MANOVA models (d. (6.5»

24)

It follows from (6.8) that the LR statistic Amk = Amk(Y(mk» for this conditional testingproblem is given by

25)j A I1"l.mkl

IAomkl'

'"hewc. AOm k and Am k are respective MLEs of Amk under conditional mCiQe18N[mkjxN(lv1(Y-<mkrE» (9.16» and N[mkjxN(L(Y-<mkrE» (9.20». Straightforward

",,"UE,U""J algebra using 7) and the relation QL[mk! = QLm however,

]Jrn:=

)

)

Thus Wrnk is a (conditionally) ancillary statistic and the MLE A.Ornk is a runcnonthe (conditionally) complete sufficient statistic (d. (6.7», so by Basu's -UvHUUCk,

to{rnkj. are (conditionally) independent. With (9.27), this implies that

(9.28)

by (3.2),

(9.29) L;O[rnk]. Jl Y(rnk)'

Since A.rn(k-l) and AOrn(k-l) are functions of Y(rnk) , so are Wrn(k-l) , and to[rn(k- . Thuswe may use an inductive argument to obtain the following main result:

Proposition 9.2. Under the null hypothesis HM,E in (9.2), the 21VV! statistics

( (Wrnk, to[rnkJ') Im = 1, ... , T, k = 1, ... ,qrn)

are mutually independent, and unconditional distributions are the same asconditional distributions in 27). Therefore, by (9.12) - (9.15), the LR statistic A

MLEs (to[w]. E vV) are mutually independent under HM,E.

Now recall from (9.6)

1==I I1(1 IwE vV);

it A is

[AP] (199Sa, pp.somewhat more accurate.

Remark 9.2. If we set LLR A in (9.6) satrsnes

classical Hadamard-Fischer deternunantal HIC;YeU<:Lll'L,,Y

two ADGs let ---j. DI = U(Iv E V)E E P(l),

(9.31)\

V E V) <

In particular j by taking D toinequality

only one "",rt",v this

(9.32)

Remark 9.3. Since the class of normal LeI models is a subset of the class of normal ADGmodels (see Remark 4.1), the results in this section may be regarded as extensions ofin [AP] (1995a) where, furthermore, no non-zero mean-value subspaces were considered.These results also extend results concerning testing one decomposable covariance uGIG",L1U1J

model against another (ef. Porteous (1989), Andersen et al (1995, §7.6.1), Eriksen (1996),Lauritzen (1996)) where, again, general mean-value subspaces were not considered.

10. Two characterizations D-linear subspaces.

In this section we study further the structure of D-subspaces. Dis an ADG and I = U(lvlv E is an associated partition of the finite index set I

Iv

To begin, note that in Definition 6.2, condition is equivalent

EV U <D x

invari­matrrces determined

a specified

Next, we present an algebraic characterizationance under a linear class M(D; of geIlerailijeci otock-triangularby D (Proposition 10.1). This Cll(Llo,I~Le,Llz,i:LLlL)ll

regression subspace is aD-subspace - see ~e(;tlo,n

For any A E Nl(I) and u; v E the x subrnatrix of anddefine A[v] := A[vv]' Each A E Nl(I) can be partitioned according to the decompositionI = U( [v] !v E V) as follows:

= ( I u, v E V).

Define

(10.1 )

(10.2)

M(D; I) := {A E M(I) IM 1(D; I) := {A E l\1(D; I) I

v E V, v i u =} A[uv]

E V, I}.

a},

For v E V and A E M(I), partition A-<v>- E M(:s v ~ x -< v~) according to thedecomposition :S v~= [v] U -< v >-: - -

where A[v] E 1\1(['0]), E M'0:>-). By (10.1), if E M(D;I)mapping is bijective:

E M(-<EV.

E lYI( -< v >- x -<following

M(D; I) --+ x (N1(

A~ (

x--<E

1.

(10.5) M(D;I) = M([vJ) l\!I( x -< v Iv E t1) ,

(10.7)

holds iff both the rouowme two conditions hold

l\1((vDL ~

M([v] x -<v>-)L ~

E V:

is straightforward to show that conditions (i) and (ii) of Definition 6.2 are togetherequivalent to (10.6), and that when (i) and (ii) hold, condition (iii) is equivalent to (10.7).

completes the proof.

It follows from Proposition 10.1 that the of D-subspaces is closed under op-erations of intersection and summation. The application of Proposition 10.1 to identifyD-subspaces is illustrated Section 13.

Remark 10.2. Clearly, M(D; I) contains II and is a linear space, i.e., closed underaddition and scalar multiplication, but is not necessarily closed under matrix multiplica­tion, i.e., l\!I(D; I) is not necessarily a matrix algebra. The matrix algebra generated byl\1(D; I), denoted by M(D; I), is the set of all finite sums of finite products of matrices inM(D; I). It is easy to see that condition (lOA) is equivalent to the condition

(10.8) M(D; I)L ~ L.

Lemma 10.1. The following four conditions are equivalent:

(i) D is transitive.

(ii) M(D; I) is closed under matrix multiplication, Le., M(D; I) is a matrix algebra;

(iii) l\1(D; I) is closed under matrix inversion;

(iv)M(D; I)is closed under Jordan multiplication: A, B E M(D; I) :=;. AB+BA E l\!I(D; I).

Proof. (i) ::::} (ii): If D is transitive, it follows from (10.1) and standard relation

(10.9) (AB)

(iv) =>but v u: Define

-<u

:= {

:= {~,

ifif

if (w', v') (w, v),if (w', v') = (w, u),

x are chosen such that AB O.0, but AB rt. M(D; I) since (AB)

where A E M x and B E M((10. and (10.9), A, B E M(D; I), BA

AB + BA rt. IVi(D; I).

Lemma 10.2. M(D; I) = M(T(D); I).

Proof. From the definitions of IVi(D; I) and T(D), M(D; I) <;: lYI(T(D); I). By "-'''-'LULLLU

10.1 applied to T(D), l\t1(T(D); I) is an algebra, hence M(D; I) <;: M(T(D); I). Since

M(T(D); I) = span(Eij liE [u],j E [v], V ::; u),

whereEij := (6ii,6jjl ! 'i',)' E I) E lYl(I),

in order to establish the opposite inclusion it suffices to show that Eij E lYI( D; I) ifi E ,j E [v], v ~ 1J,. In this case, v =Wk -< ... -< Wo =u for some Wo, ... ,Wk E V, h; 2: O.If h; < 1 then Eij E 1\1(D; I). If h; 2: 2, then

where lo := i, lk := j i and lv is chosen arbitrarily in vE1,,-tll/ E M(D; I) for u = 1, ... .k, Eij E M(D; I) as required.

1, ... , h; 1. Since

From Proposition 10.1, Remark 10.2, and Lemma 10.2, we may again conclude thatL is a D-subspace iff L is a T(D)-subspace.

Remark 10.3. As in Remark 10.1, it can be seen that M(T(D); I) IV1(K(D», where,K, NI(K) is <lenned

o>,ynnl linear ADG models and block-recursive nrrear structural eq:uai~lons.

)(- . . v E

By PropositionsHUH'c;U by

unique elementThis completes

By Proposition 11.1, Y _from model NIxN(L, D)

E E x

(11.1) y

for some A E 1\11(D;I) and some ~ E L, where Z (z[vllv E V) E R 1x N is an unob-servable stochastic variable such Z "-' N(O, r IN) for some r := diag(r[.vJiv E V),r[v] E P ), v E V. From Proposition 10.1, Remark 10.2, and the that any matrixalgebra containing II (thus M(D; is closed under matrix inversion, it follows that thisrepresentation is equivalent to the reiatron

(11.2) Ay = /L + Z

for some A E 1\11(D; I) and some /L E L. In turn, (11.2) is equivalent to the following sys­tem of block-recursive linear structural equations with block-recursive regression subspaces:

(11.3) /L[v] + , v E V,

1\ ;r (r '" /"'"' T T ! . Tr\' .,...

E d1dvJ x --< v , e v , some IV E v ) E L.

this is determined by D through the definition of Y-<v,;- and tnroucnrestriction on (P[vJ!v E V) E L. Thus, normal linear ADG model NlxN

nterpreted as a block-recursive linear extends models considered nC"""L'>"'

Wermuth (1980), Kiiveri et al (1984), [AP] (1993, Remark 3.5), andConversely, consider an arbitrary of block-recursive linear structural equations

uE E

Remark 11.1. The representation P(D;

relationFinally,

(11.3) and

Proposition 11.1 is eouivaient to

1.5) P(D; I) E M(D; ,A nonsingular}.

of P(D; I)-I. We assert here thattransitive:

following representation of P(D; I) holds iff D is

(11.6) P(D; I) {AAt A E IVI(D; I), A nonsingular},

(Also see Remark 2.4 of [AP] (1993).)If D is transitive, then by 10.1 ((i) :::::>- (iii)), (11.5) is equivalent to (11.6).

Conversely, if (11.6) holds, then for any nonsingular A E M(D; I), AAt E P(D; I). By(11.5), AtA E P(D; I) 1, hence A-l(A-I)t E P(D; I). Thus, again by (11.6), there exists

E M(D; I) such that A-I (At)-I = BBt, hence AB(AB)t = II, Le., I' := AB is anorthogonal matrix. Since T E M'(D; I) M(T(D); I) (Lemma 10.2) and M(T(D); I)is an algebra of block-triangular matrices (see Remark 10.3 and apply Remark 2.1 oflAP] (1993)), T must be block-diagonal, Le., u =F v :::::>- f[1W] = 0 (u, v E V). Therefore

1 = Bf-1 E M(D; I); thus M(D; I) is closed under matrix inversion, so D is transitiveLemma 10.1 ((iii) :::::>- (i)).

12. The maximal normal linear ADG model determined by a multivariateregression subspace.

In a specific application a multivariate nh,~pnJ::lt.lnn space IVI(IxN) and covariancestructure of the form 'E IN, one encounter a linear subspaceL <;;;; IVI(Ix

L E

IS

I

E beE is a family of nonnested subspaces of R N -

matrix E is unrestricted, this Zellner's (model, which does not MLEs un IDC'C'

\Ve now how to setform such that is a

normal ADG model, hence admits explicit MLEs as in 7. More precisely,we show how to construct the unique maximal ADG D =D(L) such that L is a D­subspace. resulting normal linear ADG model NlxN(L, D(L)) is maximal in thesense that if L is also an E-subspace for another ADG NlxN(L, E) is a submodelof NlxN(L, D(L)).

For each v E V, set K; = K L v (recall (6.2)). Without loss of generality weassume that the K; are distinct" subspaces of R N . Define D(L) to be the TADG withvertex set V and transitive binary relation -<L defined as follows: (compare to (iv) above):

(12.4) Vu, v E V, U -<L V {::::=;> Ku, c K v .

By (12.1)-(12.4), L is a D(L)-subspace of M(I x N). In fact, D(L) is the maximal ADGwith this property:

Proposition 12.1. Let E _ (vV, S) be another ADG with associated partitioning I =U(Iwlw E w) such that L is also an E-subspace of J\1(IxN). Then there exists a surjectiveADG homomorphism 'IjJ: E ---+ D(L) that satisfies (9.1), hence N1xN(L, E) is a submodelof N1xN(L, D(L)).

Proof. Define

.:T: {J ~ I' J ¥= 0, L J is a MAN OVA subspace of M(JxN)}.

Clearly L, E .:T E "i/. Note that if J E .:T and J' C J, then J' E .:T and K L Jf = K L r

Since K v , v E V, are distinct, each .J E J therefore satisfies J ~ L, for exactly one v E V.Thus the subsets Iv, v E V, are the maximal elements of.:T.

Since L is an E-subspace, L[w] is a MANOVA subspace ofM([w]x for each w E 1iV(recall our notation [w] :- Iw ) , hence E J, so C for exactly one v E V. Define

mapping 'IjJ : IV ---+ V as follows: = v iff ~ Iv. Clearly 'lj; is surjective andsatisnes (9.1) K , w E 1iV. if w ~

L is an follows

Remark 12.2. In (1994, Section 6) it was shown how to construct K(L),ring of subsets of I such that L is a K(L)-subspace of M(Ix . The normal LeImodel N IxN (L, K(L )) was shown there to be maximal in sense if L is an .IVi-subspace for a ring .IVi of subsets of I, then NIxlV(L, .IVi) is a submodel of NIxNBy 1 correspondence between LeI models and TADG models by [AMPT](1995b, Theorem 4.1), this result also follows from the stronger in Proposition 12.1- in fact, K(L) = K(D(L)) and NIxN(L, K(L)) NIxN(L, D(L)).

vVe now generalize this construction of the maximal ADG D(L) as follows. Supposethat we have not one but two multivariate linear regression subspaces L, M <;;; M] I xof the form given by (12.1)-(12.3):

L = x(Lv!v E V)

M = x (A1w E vV) ,

where V, vV are index sets that determine two partitionings

I U(Ivlv E V) = U(Iwlw E W), L, t; 0,

and where (Lviv E V), (l\t!wlw E ltV) are two families of possibly nonnested MANOVAsubspaces with

-: - _"";W'" / T !\ T\M w <::= lVl\Jw X 1'1) .

Suppose we wish to determine a parsimonious set of covariance restrictions of ADGMarkov form such that both resulting multivariate normal linear regression models arenormal ADG models. That is, we wish to construct the maximal ADG D =D(L,

that both Land A1 are D-subspaces. The resulting normal linear ADG modelsNlxN(L, D(L, ) and NIxN(lV!, D(L, are maximal sense thatif L N xN(

I'IlJ =:

2.L = x(

AI - x (j\;f[v,wj IE )

EV(L,M»),

E Af), L[v,wj and are MANOVA suospaces ofM([v, xN)

K = K L u

KM[v,wi = K Mw'

before, we may assume that (KL"Iv E V) and (I<Mw Iw E l¥) are families of distinctsubspaces of , so that ((KLv,KMw ) I (v, w) E V(L,NI) ) is a family of distinct pairs of

b 'Rv .su spaces at 1, l.e.,

(12,9)

Now define D(L, AI) to be the TADG with vertex set VeL, NI) and transitive binaryrelation -< L M defined as follows:,(12.10)

V(v,W), (v',w') E V(L, AI), (v,w) -<L,M (v',w') {:::=:;> (I<Lv,I<Mw) C (I<LvIlKMv/)'

where (I<L v' I<J'vfw) C (KL v' ,I<Jyfw t ) is defined to mean that I<Lv ~ I<Lv' and I<Mw ~ I<lv1wl

at one proper inclusion. It follows from (12.6)-(12.8) and 2.10) that both Land are D(L, A1)-subspaces of M(I x N). We now show that D(L,lv!) is the uniquemaximal ADG with this property.

Proposition 12.2. Let E (U, S) be another ADG, with associated partitioning Iu(Iu E U) such that Land M are also E-subspaces of M(I x Then there exists a<::'"'1Pf'+"'P ADG homomorphism E -+ D(L,l\1) such

2.1

x IS

E ,

the maximal

K

if U u'

K r;;.K

K C

L and AI are E-subspaces (recall Remark 6.2(iii)"), so 1j;(u) -:5.L,M 1j;(u/) by (12.10)and (12.12). Thus </J is an ADG homomorphism. The final assertion again follows fromProposition 3.1.

This construction of D(L, AI) can be extended in an obvious way to the case of threeor more regression subspaces of the form given by (12.1)-(12.3).

Remark 12.3. For a fixed ADG D, Definition 6.2, Remark 6.2, and Proposition 10.1provide several characterizations of the set all D-subspaces. Two ADGs D, D' with

same vertex set V are called J1;farko» equivalent if they determine the same Markovmodel, i.e., ifP(D; X) = P(D'; X) for all X. A discussion Markov equivalence of ADGs,including necessary and sufficient graphical condition for Markov equivalence, can befound in [AMP} (1995). If D and D' are Markov equivalent but not identical, the set of

D-·subslPa<:::es will not be identical to the set of all D'-subspaces. For example,notation similar to that of Example 4 in Section 13, D := 1 ---+ 2 and D ' := 1 +--- 2 both

same (vacuous) Markov condition, but L XL[2] is aD-subspace (resp.,r;;. R N

1 r;;. K2 , ic, :2 K 2).

13. Examples.

First, nine examples of normal linear ADG models N/xN(L, D) as definedSections 4, 6, and 7. In each example index sets I and are {I, 2,

{I, ... ,n} (n 2: 4), so that n. independent u_v;'"-,,,

observations J':ll ... ,Xn , with same UniGICIWn covariance matrrx

y:= (Xl, ... ,Xn ) E M(IxN)

and ~ := E(y). For each model N/xN(L, D), we the ADG D = (V, R),associated partitioning I Iv E V), and Markov conditionsthe D-linear regression subspace L ~ NI(IxN), representation L = x Iv E

product of MANOVA subspaces, the linear subspace K; ~ R N associated with(cf. (6.2», and P» =dim(I<u); the D-parameters 1fD(~, E) =(({lu, Ru, Au) v E V)necessary and sufficient condition (7.2) for the a.e. existence of the MLE (~, t) undermodel. In Examples 4-9 we also apply the algebraic characterization in Proposition 10.1to verify that L is aD-subspace.

Next, we illustrate the construction (12.4) of the maximal ADG D(L) determined bya linear subspace L ~ M(IxN) given by (12.1)-(12.3). Then we select nested pairs of themodels in Examples 2-9 to illustrate the general testing problem (9.2) for normal linearADG in Section 9.

Finally, in Example 10 we show that in a 4-variate two-way MANOVA model withno interactions, under a suitable ADG Markov assumption it is possible simultaneously totest the hypotheses of no row effects for one variable, no column effects for a second, andno row or column effects for a third.

In Examples 1-9, the D-subspace L is taken to be of the form

3.1) L == L(B) := {/32 r3 E B}

(compare to

fJ24) E M(IxT)

{

2 E x

Example DR 0. For convenience we abbreviate V1234

"- 1,2,

the Markov condition is vacuous, so P(D; = P(I),= M(IxT) in (13.1), i.e., j3 is unrestricted, so

. 2:: IS

(13.3)

(13.4) K = row(Z) =span(zl, Z2, Z3,

Then £[1234) = L, a MANOVA subspace, and corresponding subspace K1234 .~

Thus, trivially, L is a D-subspace by Definition 6.2 and NlxN(L, D) is a normal linear ADGmodel, namely the MANOVA model NlxN(L). Since -< 1234>-= for «, 2::) E L x P(I)the D-parameters 1f(e, 2::) are

(13.5) f-L1234 = ~, R 1234 = 0, 1\1234

Since P1234 := tr(P1234) = 4 and I :::; 1234 ~ I = condition (7.2) for the existence of theMLEs e, 2:: is ti 2: 8.

Example 2. Here we consider the same model N IxN(£, D) as in Example 1 but witha different (finer) parameterization. Let V:= {V1,V23,V4} == {1,23,4} and let D be thefollowing ADG:

4

/"1 ---+ 23

IJ,u'hfinn [ as

1\J13.rkc)V condition is vacuous, so Prepresent L equrvaientty as

{1 {2, . {4}.

B, K as in nxarnpte

L

2(iii)". SinceD-parameters

Al = E{1}

A23 = E{2,3}'{1}

A4 E{4}.{1,2,3}'{I

R1 = 0

{I} l}~{I}E-1 c

{l,2,3}<,,{1,2,3}

I}

4 I, _-< 1 >-_ il

1 I -<?3 >- I, I -- - I I -< 4 >- I = 4, condition (7.2)

model in Examples 1 and 2 properly contains each of the normal linearADG models in Examples 3-9.

Example 3. the ADG D and the associated partitioning of I be as in Example2, so again P(D; l) = P'(?'), Now define

=o},

so that B is the set of all (J E M(IxT) of the form

o o o \o I

o I'/

L is (13. and (13.8), where now

1)

3, P4 =remains n 2

{I, n".'titiAn I c:nn.n!v

I

Example 4. Let D

From DefinitionI are 1 2 and 4

the tA!!"'Ulnrr ADG:

1 +-- 2

4

the conditions determined by D and the above partitioning of1 I 2,3, which determine P(D; I) according to Definition 4.2. Define

B=

so that B is the set of all (3 E M (I xT) of the form

CI 0 0 0

)(3 = 0 (322 0 0/332 0

fC141 ,042 /343

Now

(13.14)

where

L= x

(13.15)

so it follows from Dennitron-< >- = -< 2 >- = -< 3 >- {1, 2}, and -< 4 >-D-parameters Ti(~, are

A3 ~{3}.{1,2

=~

oR2 =0

R3 = }{1,2}

R4 = ~{4}

~{3}{ 1,2} ~{/2}~{1,2}

~

}

P,2 }

/L3 = ~{3}

/L4 = ~{4}

(13.18)

Since PI = ])2 = 1, ])3 = 3, ])4 = 4 and I j 1::: I I::; 2::: i L j:3 >-condition (7.2) becomes n 2: 7.

To illustrate the use of Proposition 10.1 to verify that L is aD-subspace,that the linear class M (D; I) defined in (10.1) here consists of all A E

A = (a~l a~2 ~ ~)a31 a32 a33 0 .

o a42 a43 a44

Then M(D; I)B <; B, so M(D; I)L <; L (recall (13.1)), hence L is a D-subspace by (10.4).

Example 5. Let D be the following ADG:

1-3 +-- 2

14

The Markov conditions determined by Dare 1 2 and 4in Example L[i], Ki, and Pi, i = 1,2,3, remain same asBy (13.17), it from Definition 6.2 and Remark 6.2(iii)" L is aD-subspace.Again -< 1 >- = -< 2>- = 0 and -< 3 >- = {1,2} but now -< 4>- . so D-parameters( (/Li, Hi, Ai) Ii, 2, 3) remain the same as in Example 2 now

.(13.

Example 6. D following ADG:

I'"

Markov conditions determined by Dare 3 1 2 2 1,

B

so that B is the set of all /3 E rv1(Ix T) of the

/3 = (0'1 0'1 0'1 0'1).;1 "(2;3

th 152 153

Then L is given by (13.14) and (13.15) where now

3.21)

J(1 = span(zl + Z2 + Z3 +J(2 = spanlzj , Z2 + Z3 +J(3 = span(zl, Z2, Z3 +J(4 = span(zl, Z2, Z3, Z4) .

Since

(13.22)

it follows from Definition 6.2 and Remark 6.2(iii)"-< 2 >- = {l}, < 3 >- = {2}, and -< 4>-= {1,3}, for (~,

n(E, I;) are

-,OULhOiJULL. Since -< 1>- =D- parameters

)'"-'{

= :E{2}'{1}

R I = 0

R2 = I:{2 I} {l}

=2:

11,1 = ~{l}

fi2 ~{2} :E{2}{1}I:{1\~{1}

J:'jx:alnple 7.

c so

D I,HlrlDO' ADG:

conditions Dare 3 1 I 2 and 4 1,2 I 3, B, L,.K ,i = 1,2,3, the same as in Example 6, so again L is aD-subspace.

-< 1 >- = -< 2 >- = {I}, -< 3 >- = {2} but now -< 4>- {3}, D-parameters,Ri , Ai) I i = I, 3) remain the same as in Example 6 but now

24)

Pi = i, i = 1,2,3,4 and I :5 1 ~ I = 1, I :5 2 ~ I = I :5;3 ~ I = I :5 4 ~ I = 2, (7.2)becomes n 2: 6.

The transitive closures T(D) are identical for the ADGs D in Examples 6 and 7, I.e.,T(D) is the TADG

(13.25)

in both examples. Therefore, the families of D-subspaces coincide in Examples 5 and 6.

Example 8. Let D be following ADG:

Markov conditions determined Dare2 311

(13.

Since

(·13 ')'7\._1 )

can

K 1 C 2 C K 4

K 1 C K 3 C K4 ,

(13.28)

it follows from Definition 6.2 and 6.2(iii)" that L is a D-subspace. Again -< 1 >- 0,-<2>-= {I} but now -<3>-= {I}, -<4>- {2,3}, the D-parameters «fLi,Ri,Ai) Ii = 1,l.vl11<:MJ.1J. the same as in Examples 6 and 7 but now

tL3 = ~{3} - E{3}{1}I:ll\~{l} R3 = E{3}{l}E l{} A3 I:{3Hl}

fL4 = ~{4} - E{4}{2,3} El2\}~{2,3} R4 = E{ 4H2,3} El2~3} 1\4 = E ·{2,3}·

Since PI = 1, P2 = P.3 = 2, P4 = 4 and I =sI): 1= 1, I =s2): I = I =s3): I = 2,1-<4): I 3,(7.2) is n ~ 7.

In this example, the linear class M(D; I) in (10.1) consists of all A E ]\;1(1) of theform

o )oo .

a44

Here again 1\1(D; I)E ~ E, so IVI(D; I)L ~ L by (13.1) and Proposition 10.1 implies thatL is aD-subspace.

Example 9. D be Ir"Ulna ADG:

I'*'

2

4

}.

Again L is

29)

(13.30)

Because

K 1 C K2

K 1 C K3 C K4 ,

it follows from Definition 6.2 and 6.2(iii)" L is a D-subspace. Since -< 1 >­-<2>-= {l}, -<3>-= {I} but -<4>-= {3}, the D-parameters ( ,Ri,Ai ) i'i 1,remain the same as in Example 8 but here

- -1(13.31) J.l4 = ~{4} - 2:{4}{3} {3}~{3}, ReI = 2:{4}{3}2:{3}' k i = 2:{4}'{3}

Since PI = 1, P2 = P3 = 2, P4 = 3 and i ::: 1 >- I = 1, I ::: 2 ~ I I:::3 ~ I = I ::: 4 ~ I 2,(7.2) is n :2: 5.

In this example, the linear class M(D; I) in (10.1) consists of all A E M(I) of theform

o )oo .

a44

Again M(D; I)B ~ B so M(D; I)L L, hence L is a D-subspace by Proposition 10.1.

In order to illustrate the construction of the maximal ADG D(L) assocated with agiven regression subspace L <;;;; J'vI(I x N), first take L to be the subspace occurring inExamples 4 and 5. There we saw that L can be expressed in the form given by (13.14)­(13.16). By (12.4) and (13.17), D(L) is the TADG in (13.20) with associated Markovcondition 1 2, which is less restrictive than the Markov conditions determined byADGs D in Examples 4 and 5. Thus normal linear ADG model N1xN(L, D(L)) isless restrictive than the models N1xN(L, D) in these two examples, as guaranteedProposition 12.1.

Next L to, (13.15), (13.2

J".,l.O'!JH with vacuous

conditionwith Markov conditionmined by the ADG D in Example

Lastly, L be the subspace occurrmg(13.29). Then by

(13.33) 12 4

Markov conditions 2 3,4 I 1 4conditions determined by the ADG D in .L..JA'CHHF'V

Markov

We now exhibit nested pairs of the models in 1-9 in order to thesubmodel relation in the general problem (9.2). Denote the ADG D = (V, R)and the D-subspace L occurring in Example i, i 1, .. j 9 by D, = ,Ri ) Ls,respectively, and let N, denote the normal linear ADG model NlxN(Li, D i ) . We shallverify the following relations among these models:

(13.34)

N 5 C N 4 C N3 C N 2 N 1

N 7 C N6 C N 2

N g C N8 C N2 .

From their definitions,L5 = L4 C L3 C £2 = £1

£7 = L6 C L2

£g C £8 C L 2 .

Since D3 = D2 , we have N 3 C N 2 . To show that N(Di ) C N(D j ) and therefore thatN, C N j in the remaining cases in (13.34), we must exhibit a proper surjective ADGhomomorphism u = :D, ---t Dj that (9.1) with VV = ,V Vj.

For (i,j) = (2,1 we have V2 = {1,23,4} and \ll = {1234}; thus we may define1) = '¢(23) = := 1234. Trivially, D2 ---t D1 is a surjective ADG homomorphism

satisfies (9.1), hence N(D2) <;;;: N(Dll, but .ij; is not proper and in N(D2) =N(Dl), so N2 = Nj , For (i,j) = 2), and 2) we 1~ {I, 2, 3,4} and

= {I, 23, . thus := 1, 23, Init is verified ---t Di ADG nornomorpmsm

N(Di ) C N(D j

1!ixalnple 10.structure.) in 4, let I = {1, 2,3,4} and now let

IRI , so that we observe

y i E J, r E R, CEO) E RlxRxC =(

r E R, c E C) E RRxC. Assume that 7'mE(y) has the usual additive

(13.35)

where air, (3ic E R. Equivalently, E(y) E L, where

L:= R I K c M(IxRxC)

is the MANOVA subspace with

r E R, c E C,

As in Example 1, let D be the trivial ADG with V = {1234} and associated trivialpartitioning 11234 [1234] := {I, 2,3, 4}, so that L is a D-subspace and NIxRxC(L, D) isa normal linear ADG model, in fact a normal MANOVA model with unrestricted covariancestructure. Since pj, = IRI+ICI-1 and 1=51234>-1 = 4, condition (7.2) for the a.e, existenceof the MLE is IRIICI ~ IR! !CI 3.

Suppose that we wish simultaneously to test the hypotheses that has no row orcolumn effects, X(2) has no column effects, and has no row effects, i.e.,(13.36)

E(Xl r c ) =

This combined hypothesis can

x x x Cl\!I(I

expressed equivalently as

x

jJ-SuIDSiJ)ac:e ofSince «, C K 2 C

with Markov condition 2proper surrectrve ADG

I as in ]::;x,ampleTADG in (13.32)

t 1, 3,by, so we can test exacuy

(9.3) and (9.4)). Pi dim(Ki ) , i 1,2, 4.P3= -1,P4=IRI+ -1 and, forE, 1::::;1::: =1,1::::;2>-1 1::::;3:::1=2,1::::;4:::

condition in Proposition 9.1 for a.e. existence of the LR statistic for testing HM,E VS.

HL,E is IRI 2: IR! lei + 3, the same as for testing H 1'v1,E vs. HL,D. Thus, whencondition holds, by imposing the parsimonious constraint 2 3 1 on the covariance

structure we can test (13.35) vs. (13.36) by exact classical methods.

Acknowledgement. vVe wish to thank Svante Eriksen and Steffen Lauritzen for helpfulcomments and discussions.

References

Almond, R. (1995). Graphical Belief 1\1Iodelling. Springer, New York.

Andersen, H. H., Hejbjerre, M., Sorensen, D., Eriksen, P. S. (1995). Linear and GraphicalModels for the Multiooricie Complex Normal Distribution. Springer, New York.

Anderson, T. VV. (1984). An Introduction to ~Multivariate Statistical Analysis, 2nd ed.Wiley, New York.

Andersson, S. A. (1990). The lattice structure of orthogonal linear models and orthogonalvariance component models. Scand. J. Statist. 17 287-319.

A d S A 1\'1' d -U ~ D l' D f1nnr:\ A ~l,~~~~j.~>·;~"f;An Markovn ersson, . ., Iva 19an, ., anu .I. er luan, . \1;;1;;10). ft CH<:1.l.<:1Lvc;uuuv"JU ~

equivalence classes for acyclic digraphs. Submitted for publication.

mdersson, S. A., Madigan, D., and Perlman, M. D. (1996). On the Markov equivalencezrapns, undirected graphs, and acyclic digraphs. To appear in Scand. J. Statist.

Andersson, S. A., Madigan, D., Triggs, C. M. (1995a). On the relation

48 25-46.

W. Anderson, K.-T. 1.

andfor conditional mdependence mooers,

Introductioii to Lattices and Order.

Linear dependencies represented by chain graphs247-277.

Cox, D. R. and Wermuth. N.(with discussion). Statist.

Cox, D. R. and Wermuth,Interpretation. Chapman

Davey, B. A. and Priestley, H. A. (1990).U. Press, Cambridge.

Biometrics 28 157-175.

Non-singularity of generalized sample covariance

Dawid, A. P. (1980). Conditional mdependence for statrstrcat operations. Ann. Statist. 81272-1317.

Dempster, A. P. (1972). Covariance selection.

Eaton, M. L. and Perlman, M. D. (1973).matrices. Ann. Statist. 1 710-717.

Edwards, D. (1995). Introduction to Graphical Modelling. Springer, New York.

Eriksen, P. S. (1996). Tests in covariance selection models. To appear in Scand. J. Statist.

Kiiveri, H., Speed, T. P., and Carlin, J. B. (1984). Recursive causal models. J. Austral.Math. Soc. Ser. A 36 30-52.

Lauritzen, S. L. (1996). Graphical Models. Oxford University Press, Oxford.

Lauritzen, S. L., Dawid, A. P., B.., and Leimer, H.-G. (1990).properties of directed Markov fields. Networks 20 491-505.

Lauritzen, S. L. and Spiegelhalter, D. J. (1988). Local computations with probabilitiesgraphical structures and their application to expert (with discussion.) J.Statist. Soc. Series B 50 157-224.

Lauritzen, S. L. and Wermuth, N. (1989). Graphical models for U<)C'V,""L(»vJlVU betweenables, some of which are some quantitatrve

Management Sci.

R.(

35 527-550.

D. J., Dawid, A. P., Lauritzen,PV"r.P1ct """",tpn-'Q (with discussion

recursive models for contingency

Wermuth, (1980). Linear recursive equations, covariance selection, and path analysis.J. Amer. Statist. Assoc. 75 963-972.

Wermuth, . and Lauritzen, S. L. (1983). Graphicaltables. Biometrika 70 537-552.

Whittaker, J. L. (1990). Graphical <Models in Appl'ied Statistics. Wiley, NewYork.

Zellner, A. (1962). An efficient method of estimating seemingly unrelated regression equa­tions and tests for aggregation bias. 1. Amer. Statist. Assoc. 57 348-368.