A SIMPLE TEST FOR THE EQUALITY OF CORRELATION MATRICES · tion (4.6)) has provided a...
Transcript of A SIMPLE TEST FOR THE EQUALITY OF CORRELATION MATRICES · tion (4.6)) has provided a...
A SIMPLE TEST FOR THE EQUALITYOF CORRELATION MATRICES
by
Kinley Larntz and Michael D. Perlman
TECHNICAL REPORT No. 63
May 1985
Department of Statistics
University of Washington
Seattle, Washington 98195
A SIMPLE TESTFOR THE EQUAIJ1Y OF CORRELATIONMATRICES
by
Kinley Larntz
School of StatisticsUniversity of Minnesota
St. Paul, Minnesota 55101
Michael D. Perlman
Department of StatisticsUniversity of Washington
Seattle, Washington 98195
May 1985
This research report was supported in part by National Science Founda-
lion Grant No. MCS 83-01807.
A SIMPLE TESTFOR THE EQUAlJTY OF CORRELATIONMATRICES
by
Kinley Larntz
School of StatisticsUniversity of Minnesota
St. Paul, Minnesota 55101
Michael D. Perlman
Department of StatisticsUniversity of Washington
Seattle. Washington 98195
May 1985
1. INTRODUCTION
1.1. The testing problem
Suppose R(l) •...• R(k) are sample correlation matrices from k: independent P:
variate normal populations (k :;?; 2). Each R(J3) == (r/p) is based on a random sam-
ple of size np. Denote the f3th population correlation matrix by p(J3)
this paper we consider the problem of the hvpothesis
(piP). In
. e'» = P) ,
where P is an~~~~Q nonsingular correlation matrix, against the ,,""""'OY''''
alternative.
- 2 -
that "unfortunately there is presently no criterion available to test the
H o. As pointed out Jennrich p. 904). computation of the likelihood ratio test
(LRT) statistic would require numerical maximization of a function of
p + P (p - 1) /2 == pZ/2 variables under H 0 • a difficult task unless the dimension p
is very small. In 1967 Kuilback [7] proposed the test which rejects H 0 for large values
of the statistic
k R:z= N (3 log ! R I'f3=! i
where N (3 = n(3 - 1 and R = :z= N (3R«(3)/:Z= N (3. Kullback asserted that under H 0 •
T 1 has an asymptotic xZ distribution with (k - 1)P (p - 1 )/2 degrees of freedom.
However. Aitken ([lJ. Section 5) and Jennrich ([4]. Section 2) have shown that this
assertion is incorrect, the asymptotic distribution of T 1 under H 0 in fact being a
linear combination of independent ~ variates with the weights depending on the unk-
nown value of the common correlation matrix P. One might attempt to estimate
these weights from the data. as Aitken suggests. but this would entail difficult compute-
tions and would alter the significance level in an unknown way for finite sample size.
1.2. Jennrich's test procedure
In 1970 Jennrich proposed a test statistic Tz which suffers from neither of the
above drawbacks -- the computation of T 2 is only of moderate ri1ffirnltv (involving the
inversion of two p x P and Tz does possess an asymptotic t distribution
under For .,UlJ.I-'lJ.....;l~), we confine attention to the case k: 2 now, r",'hll'nina to
the case ~ 3 in the final section. Let denote the p - 1
column vector consistmg of the elements of R (f3) 1 ~ i < j ~ p)
c As 00 random
- 3
nonsingular covariance matrix r = r(p) which depends on P in a known way (see
Section 3). Also, the matrix R = nI3R({3) is a consistent estimator of P under
n«, so r r (R) is a consistent estimator of r. Thus, Jennrich' s test statistic
has an asymptotic X2 distribution with p (p - 1 )/2 degrees of freedom under He,
and He is rejected for large values of T2 . Fortunately, inversion of the
[p (p 1 )/2] x [p (p - 1 )/2 J matrix r is not necessary, for Jennrich ([4], equa-
tion (4.6)) has provided a computationally simpler formula for T2 which only requires
inversion of two p x p matrices.
Although Jennrich' s test has much better computational and distributional proper-
ties than the LRT and Kullback's test, it is basically a large sample test and can perform
poorly for small samples, as we show in Section 3. The main purpose of this paper is to
propose a test statistic Ts which can be readily computed by hand, and which deter-
mines a test with reasonable small sample properties and with power comparable to
that of Jennrich's test for large samples. Furthermore, our test remains valid even if
one or more of the population correlation matrices p(i) are singular, a situation which
may occasionally arise in application and under which Jennrich's statistic
undefined.
'T' ..l. 2 1S
Section 3 contains a summary of our recommendations. In the final section we
discuss the application of our procedures in the case k: 3 , and also discuss the one-
we show how our procedure can be aopuec to obtain simultarie-
ous confidence intervals for the population correlation coefficients in the
- 4 -
2. OTIIER TESf PROCEDURES
2.1. A new test procedure
The basic idea is to apply the Fisher z -transforrnation to each sample correlation
coefficient rit) , obtaining
Z(8) =tJ
and to consider the p (p - 1) /2-dimensional random column vector z (f3) consisting of
the arranged in lexicographic order. Let
d 1/2 C~(l) - ~(2)) is asymptotically normal under H 0 with zero mean vector and
covariance matrix DrD , where D D (P) is the [p (p - 1)/2] x [p (p - 1)/2]
diagonal matrix whose (ij, ij)th entry is (1 - Pi])-l. The significant aspect of the
matrix DrD is that only its off-diagonal elements involve the unknown matrix P , the
diagonal elements of D r D each being 1 By a result of Sidak [13J, therefore, one has
that for any constants bi j > 0,
f
Prob 1d 1/2 zi~l) - Zi~2)l
lJ
)
- 5 -
In particular, if we define the test statistic T3 by
d V 2 maxl::ii<j::ip
then for any b > 0
Prob
asymptotically under H.
;2 bS :2: [<P(b) - <p(-b)]p(P-l)/2
we propose the (possibly conservative) level ex. test
under which Ho is rejected if Ts > bet., where b a > 0 is chosen such that
(2.1)
In effect, this procedure simultaneously tests each of the p (p - 1) /2 univariate
hypotheses p ~p = p ~J) , treating the differences z ~p - z ~J) as if they were indepen-
dent, and relying on Sidak's result to provide an upper bound for the significance level,
Our procedure has several potential advantages over Jennrich's test especially for
small sample size: (a) the value of Ts is easily computable with only a hand calcula-
tor; (b) the z-transform of a sample correlation coefficient r converges in distribu-
tion to normality faster than r itself; (c) larger sample sizes are needed to guarantee
the approximate Yr",lti"",..;;",to normality required of the vectors r(f3) by Jennrich's pro-
cedures T2 (and required of the vectors z(f3) by and T5, two variants of
Jennrich's procedure to be introduced in Section 2.3) than are needed for the approxi-
of each z sample
sizes for1"' _
1 - to be an accurate estimate of ) under n, same is
true for T4 and T 5 ). These may explain the breakdown
the sample SIzes dimension p Sec-
- 6 -
parameters and does not break dm'ITI for small sample sizes. In
may be applied when np < p + 1 for one or more of the "U.Uq..lH.n:l
this procedure
which case R (P)
is singular ), and even when, under H 0 ' the common population correlation matrix P
is singular, in which case the statistics
hence I' will be singular.
2.2. An example
, T4 ' and T 5 are not defined since Rand
Jennrich ([ 4J, Section 6) applied his test statistic T2 to two 12 x 12 sample
correlation matrices R (1) and R (2) computed from test scores of a group of 40
retarded children and a group of 89 normal children respectively
(p = 12, nl = 40, nz = 89). The matrices R(1) and R(2) appear at the bottom
of page 910 of [4]. He found that T 2 = 74.8 which, when referred to the X2 distribu-
tion with 12 ( 11) /2 = 66 degrees of freedom, corresponds to an attained significance
level of .22, hence is not significant at the 5 per cent level. Jennrich attributes this
lack of significance to "the relatively small sample size, 40, of the sample of retarded
children" and to "the similarity of the observed correlation matrices." Inspection
shows, however, that R(l) and R(2) are not very similar. In fact, the average of the 66
absolute difference is .255, and only 4 of the 66 signed differences are
negative. The maximum absolute difference of the z -transforms is
.706, occurring for (i,j) = (9,12), whence we find T 3 = 3.59. Setting b a = 3.59
in (2. and to a table of the normal distribution, one finds that this
corresponds to an attained level of ex = .02 . when to
Jennrich's our procedure is far more sensitive than Jennrichs nrocedure
another indication of the satisractory power nrnnprr.iF'" of
small" sampie sizes.
- 7 -
2.3. Two variants of Jennrich's procedure
The test statistics and are similar to Jennrich' s T2. but are based on the
vectors z{{3) rather than rep) is an attempt to exploit the more rapid convergence of
the z -transtorrns to normality. Still. T 4, and T 5 share with T2 the potential draw-
backs (c) and (d) discussed earlier. The statistics T 4, and T5 are defined by the for-
mulae
where r rUf) s fj = D(R), r = r(R*), fJ = D(R*), and R* = (rtj ) is an
alternate estimator of P under H o defined in terms of z(l) and z(2) by the relation
where Zij = ~p ( np - 3) z ~J) / L;p ( np - 3). The statistics T 4, and T 5 differ only in
the way the asymptotic covariance matrix D r D of d 1/2 (z (1) - Z (2)) is estimated.-- -
Under H 0, both T 4, and T5 have asymptotic X2 distributions with p (p - 1) /2
degrees of freedom. It should be kept in mind that as the sample sizes approach 00,
the tests based on T 2 , s and are asymptotically exact under H 0 , whereas the
test based on will be asyrnptotrcany conservative under unless P = J .
2.4. Summary of Monte Carlo study
In we conducted a Monte Carlo of the actual levels and
""'iin""" of p
- 8 -
n I = nz = 10, 20, 40, 100, and 200.. The results indicated that for small sample
only has an actual significance level which can be trusted to lie in the mf"ll-"t"
of the nominal value 0:, the levels of Tz, T4 ,and T5 deviating greatly from 0: for
some values of the matrix P under H 0.. As n increases, the rejection rates for Tz ,
T4 , and T5 appear to approach the nominal value, whereas the slightly conservative
nature of Ts becomes apparent. Thus, for small samples sizes Ts comes closest to
the nominal level, whereas any of T z , T 4 , or T 5 could be selected for the larger sam
ple sizes. As expected, for a given sample size, the rejection rates tend to be less accu
rate and more variable as p increases.
As to power, no one of the four tests is uniformly better than the others. Prelim
inary investigations showed that the powers of T z , T4, and T 5 are comparable for
moderate to large sample sizes, so in [8] emphasis was placed on comparing the powers
of T z and Ts under a variety of alternatives. When p = 3, we found that in those
cases (75 out of 210 pairs of alternatives (p(I), p(Z) ) ) where the powers of Ta and Ta
differed significantly, more often the advantage lay with T z (59 cases) than "with Ts
(16 cases) (due to the arbitrary natures of the population correlation matrices selected
for the study, the importance of these numbers should not be overemphasized). In
addition, when the power of Tz was significantly larger than that of Ts , the magnitude
of the difference was occasionally quite substantial, whereas this is not the case when
Ts dominates Tz. Part of the reason for the often lower power of Ts is, of course,
its conservative nature, but this does not prevent the conclusion that when p is small
and the sizes are moderate or is test
from considerations of ease of implementation, where has a considerable advan-
When the dimension p 5 or 10, the relative status of T z and
in power often --
and 10 , we found
- 9 -
pairs of alternatives ( , p(2)) where the estimated power of
was np,:,r!v 1 , while that of was much lower, often nearly O. Each of these
pairs of alternatives involved some very large correlation coefficients pij,0.9 or
There were no cases where the estimated power of was nearly 1 while
that of Ta was substantially lower. It should be mentioned that unlike T2 , the statis
tics T 4 and T5 did not exhibit instances of extremely low power.
3. RECOMMENDATIONS
Because the nominal significance levels of the tests based on T 2 , T 4, and T 5
have been found to be unreliable for small sample sizes, Ts is the clear choice in this
situation. Based on the cases studied, we propose the following rule-of-thumb: when
the ratio of sample size to dimension does not exceed 4, i.e., when
min ( n 1, n2) /p :::;; 4, then Ta is recomm ended. (As p increases, it is likely that
the constant 4 should be increase also.) Thus, for the values of (p,n) studied in
[8], Ts should be used in the following cases: (p,n) = (3,10), (5,10), (5,20),
(10, 10), ( 10, 20), ( 10, 40). (Note too that in the example of Section 2.2.,
min(nl,n2)/p = 40/12:::;; 4,and Ts was found to be more sensitive than Tz .)
For moderate and large sample sizes the choice is less clear-cut. Certainly T3 is
the easiest procedure to implement, and it readily provides a set of simultaneous
confidence intervals for all differences ,uU) - ,u~J)' 1:::;; i, j :::;; p, where ,u~f)
denotes the z -transform of Plf) -- also see Section 4. For sample sizes we sug-
that any of T5 should be since are exact
is conservative and more "''''Arc,rf, d for most alternatives. Of course, for
the convenience ofmoderate sample
outweich a r elativelv small loss in power.
may
we remark that as the sample sizes
- 10 -
4. THE k-SA.\£PLE AND ONE-SAMPLE CASES
When the number k: of populations is 3 or more, the hypothesis H 0 can be tested
by means of Jennrich's extension of T2 , given in equation (7.2) of Alternatively,
our procedure can be extended to the k-sample case as described next, and should be
expected to retain its advantageous small-sample properties. For fixed i < j let
t t , - (fl{\) l1(k) z .. - (zO) z(J;) and let H·, denote the univariater:=..1.J - \ ~tJ " .. , 1'""'1-) J _ tJ - c: 1.) " .. , 1.) ,..l l.J v
hypothesis
(1)H ij : f-Lij
11(1<;)r-tJ ' l~i<j~p.
AE min ( n l' ... , nk) -> 00, the random vector }£ij has an asymptotic k -variate normal
distribution with mean vector and covariance matrix
b.. = diag [(n1 - 3)-1, ... , (nk - 3)-1]. The hypothesis H i j can be tested by means
of the statistic
S ij
r k 12
I~ (n{J - 3) z ff) j1{J=1. ,
k
~ (n{J 3){J=1
(see Rao [l1J, p. 434-5), which reduces to
k(n - 3) ~ (z
{J-1
when n ri , where Zij z Under H ij' S ij has as
distribution. Since is the intersection of the p /2
univariate H ij , we introduce the statistic
- 11 -
and propose the test which rejects H a if T; > Xk~t,e(a)' where Xk-l,e is the upper t;
point of the Xk2_ t distribution and t; (Cf.) = (1 - rx )2/p(P-t) .
[5],
Corollary 5' of Khatri
Prob ITs 1 - ex
asymptotically under H a ' so this procedure yields a (possibly conservative) level ex
test.
Our proposed procedure for the k-sample p-variate problem may be regarded as
a generalization of the Pearson-Wilks procedure for testing equality of the correlation
coefficients from k: independent bivariate normal populations (see [10] or [11], pp.
434-5). For moderate sample sizes, the Xk2_ t approximation can be improved if S ij is
modified as indicated in the formula for H on page 436 of Rao [1].
Next. we consider briefly the one-sample case. Let R = (r ij) be a sample corre-
lation matrix based on a random sample of size n from a p -variate normal distribu-
tion with population correlation matrix P = (Pij)' Consider the problem of testing
the hypothesis H o: P = Po, where Po = (Pij,O) is a specified correlation matrix. If
Po is the identity matrix, H a is the standard hypothesis of independence of a set of
p-variates and may be tested by means of Bartlett's modified LRT [9]. See also M. Hills
[3J for a graphical method. For other values of Po, Jennrich's method may be applied
as follows. Let r denote the p (p - 1) /2-dimensional column vector consisting of
the elements r ij (i < j) of R s and similarly define E!.- and 120 in terms of the ele-
ments of P and As n -;. 00 r- is normal with zero
mean vector and covariance matrix 1f;
P in a known way. Thus the statistic
1f; P) . where 1/1 depends on the elements of
- 12-
has an asymptotic X2 distribution under H 0 with P (p 1) /2 riPicrrF'p" of freedom,
-where 1/Jo = 1/J(Po) , and Jennrich proposes rejection of Ho for large values of 12.
He gives a computationally simpler formula for T2 in equation (7.1) of
Our method applied to the one-sample testing problem leads to the statistic
(n 3 )1/2 m.ax Z ij - fJij ,0 ,t;:;i<j :O;p
where Z ij is the z -transform of r ij and fJij ,0 is the z -transforrn of P ij ,0 . By
Sidak's Theorem [13] the test which rejects H o if 1§ > b a • where b a is determined
by (1.1), is a (possibly conservative) level ex test of H o . Our procedure may be
expected to behave slightly better than Jennrichs for small samples since T3 relies.....,
only on the appropriate univariate normality of the z ij whereas 12 relies on the rnul-
tivariate normality of the vector I-. However. the difference should not be expected to
be as pronounced here as in the k-sample case (k :f:: 2), since the statistic 12 does
not require the estimation of a covariance matrix, unlike T2. The procedure based on
T3 has also been discussed by Morrison ([9], pp. 119-120); he uses the somewhat
weaker Bonferoni bound rather than the Sidak's inequality,
On the other hand, our procedure docs have the advantage that it readily yields a
set of simultaneous confidence intervals for the p (p - 1) /2 population correlation
coefficients Pij' If we let fJij denote the z-transform of Pij , asymptotically the pro-
bability is at least 1 ex that the inequalities
(l~i<j~p)
are simultaneously satisfied. These me.quanties are then inverted to obtain simult.ane-
ous confidence bounds the Jennrichs procedure
perform
dimension p .
(as in the k-sample
- 13 -
if the sample size n is small relative to the
Finally, we remark that in both the one-sample and k: -sarnple case, our method
can be applied much more readily than Jennrich's procedure if it is desired to restrict
attention to some arbitrary subset L of the p (p - 1) /2 indices (i, j) and test the
hypothesis
P OltJ
(in the k-sample case) or
D-, tJ
(in the one-sample case).
Pij,O
P (~ )tJ ' (i,j)c:L
(i,j)c:L
- 14-
References
[1] Pitkin, M. 1969. Some tests for correlation matrices. Biometrika. 56: 443-6.
[2] Bartlett, M. S., 1933. On the theory of statistical regression. Proceedings of the
Royal Society, Edinburgh 53: 260-83.
Hills, M., 1964. On looking at large correlation matrices. Biometrika. 56: 249
253.
[4] Jennrich, R. 1., 1965. An asymptotic X2 test for the equality of two correlation
matrices. Journal of the American Statistical Association. 65: 904-12.
[5] Khatri, C. G.. 1967. On certain inequalities for normal distributions and their
applications to simultaneous confidence bounds. Annals of Mathematical Statis
tics. 38: 1853-67.
[6J Knuth, D. Eo, 1969. The Art of Computer Programming, Vol. 2: Semiriurnericoi
Algorithms. Addison-Wesley. Reading, 1vfA.
[7J Kullback, S., 1967. On testing correlation matrices. Applied Statistics. 16: 80
5.
[8] Larntz, K.. and M. D. Perlman. 1985. A simple test for the equality of correlation
matrices. Unpublished rp,">("\lrt
St. Paul. MN.
Department of Statistics. University of Minnesota,
Morrison,
York.
. 1976. Multivariate Statistical Methods, 2nd ed. New
E. S., and S. S. 1933. Methods of statistical anaIV'SIS "lnnrnnri;,rp for
k samples of two variables. Biometrika. 25: 353-78.
1 c. 1973. Linear Statistical and its A;z:rpliccLti,ons, 2nd ed, John
and NI.
- 15 -
[12] Seal, H. L., 1964. Multivariate Statistical Analysis for Biologists. John Wiley and
Sons, Inc., New Nr'.
[13] Sidak, Z., 1967. Rectangular confidence regions for the means of multivariate nor
mal distributions. Journal of the American Statistical Association. 62: 626-33.