Curran,West&Finch (1996)
-
Upload
share-wimby -
Category
Documents
-
view
225 -
download
0
Transcript of Curran,West&Finch (1996)
-
7/25/2019 Curran,West&Finch (1996)
1/14
Psycholog ica l Met hods Copy righ t 1996 by the Am erican Psycholog ica l Assoc ia t io n , Inc .
1996. VoL 1, No. l, 16-29 1082-989X/96/ 3.00
T h e Rob u s tn es s o f T es t S ta t i s t i c s to Non n orm al i ty
and Spec i f icat ion Error in Co nf irmatory Factor nalys i s
Patrick J Curran
Uni ve r s i t y o f Ca l i f o r n ia , L os A nge l es
Stephen G West
Ar i zona S t a t e Uni ve r s i t y
John F Finch
T e x a s A M U n i v e r si t y
M o n t e C a r l o c o m p u t e r s i m u l at io n s w e r e u s e d t o i n v e s ti g a te t h e p e r f o r m a n c e
o f t h r e e X2 es t s t a ti s ti c s in con f i r ma t or y f ac t o r ana lys i s ( CF A) . No r mal t h eor y
m a x i m u m l i k e l i h o o d )~2 ( M L ) , B r owne ' s a sympt o t i c d i s t r i bu t i on f r ee X2
( ADF ) , and t he S a t o r r a - Ben t l e r r e sca l ed X2 ( S B) wer e exam i ned un der va r y-
ing condi t ions of sample s ize , model speci f icat ion, and mul t ivar ia te di s t r ibu-
t ion . F or p r ope r l y spec i f ied mode l s , M L a nd S B showed no ev i dence o f b ia s
unde r nor m a l d i s t r ibu t i ons ac r os s a ll s ampl e s izes, wher eas A DF was b i a sed
a t a l l bu t t he l a r ges t s amp l e s izes. M L was i nc r eas i ng ly ove r es t i ma t ed w i t h
i nc r eas i ng nonnor mal i t y , bu t bo t h S B ( a t a l l s ampl e s i zes ) and ADF ( on l y
a t l a r ge s amp l e s izes ) showed no ev i dence o f b i a s . F or mi s spec i f ied mode l s ,
M L was aga i n i n f l a t ed w i t h i nc r eas i ng nonnor mal i t y , bu t bo t h S B and ADF
wer e un de r es t i ma t ed w i t h inc r eas ing nonno r mal i t y . I t app ea r s t ha t t he power
of t he S B and A D F t e s t s ta t i st i c s t o de t ec t a mo de l m i s speci f ica t ion i s a t t enu-
a t ed g i ven nonnor mal l y d i s t r i bu t ed da t a .
C o n f i r m a t o r y f a c to r a n al y si s ( C F A ) h a s b e c o m e
a n i n c r e a si n g ly p o p u l a r m e t h o d o f in v e s t ig a t in g
t h e s t r u c t u r e o f d a t a s e t s i n p s y c h o l o g y . I n c o n t r a s t
t o t r a d it i o n a l e x p l o r a t o r y f a c t o r a n a ly s i s t h a t d o e s
n o t p l a c e s t r o n g a p r i o r i r e s t r i c t i o n s o n t h e s t r u c -
t u r e o f t h e m o d e l b e i n g t e s te d , C F A r e q u i r e s t h e
i n v e s t i g a to r t o s p e c i fy b o t h t h e n u m b e r o f fa c t o r s
P a t r i ck J . Cur r an , Gr adua t e S choo l o f E duca t i on ,
Uni ve r s i t y o f Ca l i fo r n i a , L os Ange l es ; S t ephen G . W es t ,
De par t m ent o f P sycho l ogy , Ar i zon a S t a t e Uni ve r s i ty ;
John F . Fi nch , De par t m ent o f P sycho l ogy , T exas
A M Uni ve r s i t y .
W e t h a n k L e o n a A i k e n , P e t e r B e n t l e r, D a v i d K a p l a n ,
D a v i d M a c K i n n o n , B e n g t M u t h 6 n , M a r k R e i s e r , a n d
Al be r t S a t o r r a f o r t he i r va l uab l e i npu t . T h i s wor k was
pa r t i a l l y suppor t ed by Na t i ona l I ns t i t u t e on A l coho l
Abuse and Al coho l i sm P os t doc t o r a l F e l l owsh i p F 32
AA05402-01 and G r an t P 50M H 39246 dur i ng t he wr it i ng
of thi s ar t ic le .
Cor r esp onde nce co nce r n i ng t h i s a r t ic l e shou l d be ad-
dr es sed t o P a t r i ck J. Cur r an , G r adu a t e S choo l o f E duca -
t ion , U ni ve r s i ty o f Ca l i f o r n i a, L os A nge l es , 2318 M oor e
Hal l , Los Angeles , Cal i fornia 90095-1521. E lect ronic
mai l may be s en t v i a I n t e r ne t t o i dhbp j c@mvs .
oac.ucla .edu.
a n d t h e s p e c i f ic p a t t e r n o f l o a d i n g s o f e a c h o f t h e
m e a s u r e d v a r i a b l e s o n t h e u n d e r l y i n g s e t o f f a c-
t o rs . I n ty p i c a l s i m p l e C F A m o d e l s , e a c h m e a s u r e d
v a r i a b l e i s h y p o t h e s i z e d t o l o a d o n o n l y o n e f a c to r ,
a n d p o s i ti v e , n e g a ti v e , o r z e r o ( o r t h o g o n a l ) c o r r e -
l a t i o n s a r e s p e c i fi e d b e t w e e n t h e f a c t o r s . S u c h
m o d e l s c a n p r o v i d e s tr o n g e v i d e n c e a b o u t t h e c o n -
v e r g e n t a n d d i s c r im i n a n t v a l i d i ty o f a s e t o f m e a -
s u r e d v a r i a b l e s a n d a l l o w t e s t s a m o n g a s e t o f
t h e o r i e s o f m e a s u r e m e n t s t r u c tu r e . M o r e c o m p l i -
c a t e d C F A m o d e l s m a y s p e c i fy m o r e c o m p l e x p a t-
t e r n s o f f a c t o r lo a d i n gs , c o r r e l a t i o n s a m o n g e r r o r s
o r s p e c i f ic fa c t o r s , o r b o t h . I n a l l c a s e s , C F A m o d -
e l s s e t r e s t ri c t i o n s o n t h e f a c t o r l o a d i n g s , t h e c o r r e -
l a t i o n s b e t w e e n f a c t o r s , a n d t h e c o r r e l a t i o n s b e -
t w e e n e r r o r s o f m e a s u r e m e n t t h a t p e r m i t t e s ts o f
t h e f it o f th e h y p o t h e s i z e d m o d e l t o t h e d a t a .
T h e r e a r e t w o g e n e r a l c l a ss e s o f a s s u m p t i o n s
t h a t u n d e r l i e t h e s t a t i s t i c a l m e t h o d s u s e d t o e s t i -
m a t e C F A m o d e l s : d i s t r i b u t i o n a l a n d s t r u c t u r a l
( S a t o r r a , 1 9 9 0 ) . N o r m a l t h e o r y m a x i m u m l i k e l i -
h o o d ( M L ) e s t i m a t i o n h a s b e e n u s e d t o a n a l y z e
t h e m a j o r it y o f C F A m o d e l s . M L m a k e s t h e dis tr i
b u t i o n a l a s s u m p t i o n t h a t t h e m e a s u r e d v a r i a b l e s
h a v e a m u l t i v a r i a te n o r m a l d i s t ri b u t io n i n t h e p o p -
16
-
7/25/2019 Curran,West&Finch (1996)
2/14
CHI-SQUARE TEST STATISTICS 17
ulation. However, the majority of data collected
in behavioral research do not follow univariate
normal distributions, let alone a multivariate nor-
mal distribution (Micceri, 1989). Indeed, in some
important areas of research such as drug use, child
abuse, and psychopathology, it would not be rea-
sonable to even expect that the observed data
would follow a normal distribution in the popula-
tion. In addition to the distributional assumption,
ML (and all methods of estimation) makes the
structur l
assumption that the structure tested in
the sample accurately reflects the structure that
exists in the population. If the sample structure
does not adequately conform to the corresponding
population structure, severe distortions in all as-
pects of the final solution can result.
Although the chi-square test statistic can be
used to measure the extent of the violation of the
structural assumpt ion (Lawley Maxwell, 1971),
the accuracy of this test statistic can be compro-
mised given violation of the distributional assump-
tion (Satorra, 1990). Violations of both the distri-
butional and structural assumptions are common
(and often unavoidable) in practice and can poten-
tially lead to seriously misleading results. It is thus
important to fully understand the effects of the
multivariate nonnormality and specification error
on maximum likelihood estimation and other al-
ternative estimators used in CFA.
Methods of Estimation
By far the most common method used to esti-
mate confirmatory factor models is normal theory
ML. Nearly all of the major software packages use
ML as the standard default estimator (e.g., EQS,
Bentler, 1989; LISREL, Jtireskog S6rbom,
1993; PROC CALLS, SAS Institute, Inc., 1990;
RAMONA, Browne, Mels, Coward, 1994).
Under the assumptions of multivariate normality,
proper specification of the model, and a suffi-
ciently large sample size (N), ML provides asymp-
totically (large sample) unbiased, consistent, and
efficient parameter estimates and standard errors
(Bollen, 1989). An important advantage of ML is
that it allows for a formal statistical test of model
fit. (N - 1) multiplied by the minimum of the ML
fit function is distributed as a large sample chi-
square with 1/2(p)(p + 1) -t degrees of freedom,
where p is the number o f observed variables and
t is the number of freely estimated parameters
(Bollen, 1989).
One potential limitation of ML estimation is
the strong assumption of multivariate normality.
Given the presence of non-zero third- and (partic-
ularly) fourth-order moment s (skewness and kur-
tosis, respectively),1 the resulting ML parameter
estimates are consistent but no t efficient, and the
minimum of the ML fit function is no longer dis-
tributed as a large sample central chi-square. In-
stead, (N - 1) multiplied by the minimum of the
ML fit function generally produces an inflated
(positively biased) estimate of the referenced chi-
square distribution (Browne, 1982; Satorra, 1991).
Hence, using the normal theory chi-square statistic
as a measure of model fit under conditions of non-
normality will lead to an inflated Type I er ror rate
for model rejection. Consequently, in practice a
researcher may mistakenly reject or opportunisti-
cally modify a model because the distribution of
the observed variables is not multivariate normal
rather than because the model itself is not correct
(see MacCallum, 1986; MacCallum, Roznowski,
Necowitz, 1990).
Several different approaches have been pro-
posed to address the problems with ML estimation
under conditions of multivariate nonnormality.
One example is the development of alternative
methods of estimation that do not assume multi-
variate normality. One such estimator that is cur-
rently available in structural modeling programs
such as EQS (Bentler, 1989), LISCOMP (Muthrn,
1987), LISRE L (Jr reskog SOrbom, 1993), and
RAMONA (Browne et al., 1994) is Browne's
(1982, 1984) asymptotic distribution free (ADF)
method of estimation. The derivation of the ADF
estimator was not based on the assumption of mul-
tivariate normality so that variables possessing
non-zero kurtoses theoretically pose no special
problems for estimation. ADF provides asymptot-
ically consistent and efficient parameter estimates
and standard errors, and (N - 1) times the mini-
mum of the fit function is distributed as a large
sample chi-square (Browne, 1984). One practical
disadvantage of ADF is that it is computationally
1The multivariate normal distribution is actually char-
acterized by skewness equal to 0 and kurtosis equal to 3.
However, it is common practice to subtract the constant
value of 3 from the kurtosis estimate so that the normal
distribution is characterized by zero skewness and zero
kurtosis. We will similarly refer to the normal distribu-
tion as defined by zero skewness and zero kurtosis.
-
7/25/2019 Curran,West&Finch (1996)
3/14
18 CURRAN, WEST, AND FINCH
very demanding. Browne (1984) anticipated that
models with greater than 20 variables could not
be feasibly estimated with ADF. A second possible
disadvantage is that initial findings suggest that
this estimator performs poorly at the small to mod-
erate sample sizes that typify much of psychologi-
cal research.
A second approach that has been developed
for computing a more accurate test statistic under
conditions of nonnormal ity is to adjust the normal
theory ML chi-square estimate for the presence
of non-zero kurtosis. Because the normal theory
chi-square does not follow the expected chi-square
distribution under conditions of nonnormality, the
normal theory chi-square must be corrected, or
rescaled, to provide a statistic that more closely
approximates the referenced chi-square distribu-
tion (Browne, 1982, 1984). One variant of the re-
scaled test statistic that is currently only available
in EQS (Bentler , 1989) is the Satorra-Bent ler chi-
square (SB X2; Satorra, 1990, 1991; Satorra
Bentler, 1988). The SB X2 corrects the normal the-
ory chi-square by a constant k, a scalar value that
is a function of the model implied residual weight
matrix, the observed multivariate kurtosis, and the
model degrees of freedom. The greater the degree
of observed multivariate kurtosis, the greater
downward adjustment that is made to the inflated
normal theory chi-square.
Review of Monte Carlo Studies
Muth6n and Kaplan (1985) studied a properly
specified four indicator single-factor model under
five distributions ranging from normal to severely
nonnormal for one sample size (1,000). For univar-
iate skewness greater than 2.0, the ML X2 was
clearly inflated whereas the ADF X2remained con-
sistent. Muthrn and Kaplan (1992) extended these
findings by adding more complex model specifica-
tions, an addi tional sample size (500), and increas-
ing the number of replications to 1,000 per condi-
tion. The normal theory chi-square was extremely
sensitive to both nonnormality and model com-
plexity (defined as the number o f parameters esti-
mated in the model). The ADF X2 appeared to be
very sensitive to model complexity, with extreme
inflation of the model chi-square as the tested
model became increasingly complex. The ADF X2
was also particularly inflated at the smaller sam-
ple size.
Satorra and Bent ler (1988) per formed a Monte
Carlo simulation using a proper ly specified four in-
dicator single-factor model to evaluate the behav-
ior of the SB X2 est statistic. The unique variances
of the four indicators were calculated with univari-
ate skewness of 0 and a homogenous univariate kur-
tosis of 3.7. The models were estimated using ML,
unweighted least squares (ULS), and ADF, based
on 1,000 replications of a single sample size of 300.
The normal theory ML X2and the SB X2performed
similarly to one another. On average, the ML X2
slightly underestimated the expected value of the
model chi-square while the SB g 2 slightly overesti-
mated the expected value. However, the ML
X z
had
a larger variance than did the SB X2. The ADF X2
resulted in the highest average value, although it
also attained the lowest variance.
Chou, Bentler , and Sa torra (1991) similarly used
a Monte Carlo simulation to examine the ML,
ADF, and SB X2 est statistics for a properly speci-
fied model under varying conditions of normality.
A two-factor six indicator CFA model was repli-
cated 100 times per condi tion based on two sample
sizes (200 and 400) and six multivariate distribu-
tions. Two versions of the model were estimated,
one in which all of the necessary parameters were
freely estimated, and one in which the factor load-
ings were fixed to the population values. Consis-
tent with previous research, the ML X2was inflated
under nonnormal conditions. The SB
X
outper-
formed both the ML and ADF X2 test statistics in
nearly all conditions.
Finally, Hu, Bentler, and Kano (1992) performed
a major simulation study based on a three-factor
confirmatory factor model with five indicators per
factor. Six sample sizes were used (ranging from 150
to 5,000) with 200 replications per condition. Seven
different symmetric distributions were considered,
ranging from normal to severely nonnormal (high
kurtosis). The normal theory estimators (maximum
likelihood and generalized least squares) provided
inflated chi-square values as nonnormality in-
creased. The ADF test statistic was relatively unaf-
fected by distribution but was only reliable at the
largest sample size (5,000). Finally, the SB X2 per-
formed the best of all test statistics, although mod-
els were rejected at a higher frequency than was ex-
pected at small sample sizes.
In summary, Monte Carlo simulation studies
have consistently supported the theoretical predic-
tion that the normal theory ML X2 test statistic is
-
7/25/2019 Curran,West&Finch (1996)
4/14
C H I - S Q U A R E T E S T S T A T I S T I C S 1 9
s i g n if i c a n tl y i n f l a te d a s a f u n c t i o n o f m u l t i v a r i a t e
n o n n o r m a l i t y . T h e A D F X 2 is t h e o r e t i c a ll y a s y m p -
t o ti c a ll y r o b u s t t o m u l t i v a r i a te n o n n o r m a l i t y , b u t
i ts b e h a v i o r a t s m a l l e r ( a n d m o r e r e a l is t i c ) s a m p l e
s i ze s i s p o o r . T h e S B X 2 h a s n o t b e e n f u l l y e x a m -
i n e d , b u t i n i t i a l f i n d i n g s i n d i c a t e t h a t i t o u t p e r -
f o r m s b o t h t h e M L a n d A D F t e s t s ta t is ti c s u n d e r
n o n n o r m a l d i s t r ib u t i o n s, a l t h o u g h i t d o e s t e n d t o
o v e r r e j e c t m o d e l s a t s m a l l e r s a m p l e s iz e s.
A l t h o u g h t h e r a m i f i c a t i o n s o f v io l a t i n g t h e d i s tr i -
b u t i o n a l a s s u m p t i o n s i n C F A i s b e c o m i n g b e t t e r
u n d e r s t o o d , m u c h l e ss is k n o w n a b o u t v i o l at i o n s o f
t h e s t r u c t u r a l a s s u m p t i o n . R e c e n t w o r k h a s a d -
d r e s s e d t h e e f f e c t s o f m o d e l m i s s p e c i f i c a ti o n o n t h e
c o m p u t a t i o n o f p a r a m e t e r e s t i m a t e s a n d st a n d a r d
e r r o r s ( K a p l a n , 1 9 8 8 , 19 8 9 ) a s w e l l a s p o s t h o c
m o d e l m o d i f i c a t i o n ( M a c C a l l u m , 1 9 8 6 ) a n d t h e
n e e d f o r a l t e r n a t i v e i n d i c e s o f f it ( M a c C a l l u m ,
1 9 90 ). H o w e v e r , l i tt le is k n o w n a b o u t t h e b e h a v i o r
o f c h i - s q u a r e t e s t s ta t is t ic s u n d e r s i m u l t a n e o u s v i o -
l a t i o n s o f b o t h t h e d i s t r i b u t i o n a l a n d t h e s t r u c t u r a l
a s s u m p t i o n s . I n d e e d , w e a r e n o t a w a r e o f a s i n g le
e m p i r i c al s tu d y t h a t h a s e x a m i n e d t h e A D F a n d S B
A 2 t e s t s t a t is t ic s u n d e r t h e s e t w o c o n d i t i o n s . G i v e n
t h e l o w p r o b a b i l i t y t h a t t h e s t r u c t u r e t e s t e d i n a
s a m p l e p r e c i s e l y c o n f o r m s t o t h e s t r u c t u r e t h a t e x -
i s ts in t h e p o p u l a t i o n , i t is c r i t i c a l t h a t a b e t t e r u n -
d e r s t a n d i n g b e g a i n e d o f t h e b e h a v i o r o f t h e t e s t
s t at i st i cs u n d e r t h e s e m o r e r e a l is t i c c o n d i t io n s .
T h e P r e s e n t S t u d y
A s e ri e s o f M o n t e C a r l o c o m p u t e r s i m u l a t i on s
w e r e u s e d t o s t u d y t h e e f f e c t s o f s a m p l e s i z e , m u l t i -
v a r i a t e n o n n o r m a l i t y , a n d m o d e l s p e c i fi c a ti o n o n
t h e c o m p u t a t i o n o f t h r e e c h i - s q u a r e t e s t s t at i st i cs
t h a t a r e c u r r e n t l y w i d e l y a v a i l a b l e t o t h e p r a c t i c i n g
r e s e a r c h e r : M L , S B , a n d A D F . 2 F o u r s p e c i f i c a t io n s
o f a n o b l i q u e t h r e e - f a c t o r m o d e l w i t h t h r e e i n d ic a -
t o r s p e r f a c t o r w e r e c o n s i d e r e d . T h e f ir st t w o m o d -
e l s w e r e
correc t l y spec i f i ed
s u c h t h a t t h e s t r u c t u r e
e s t i m a t e d i n t h e s a m p l e p r e c i s e l y c o r r e s p o n d e d t o
t h e s t r u c t u r e t h a t e x i s t e d i n t h e p o p u l a t i o n . T h e
s e c o n d t w o m o d e l s w e r e m i s s p e c i f i e d s u c h t h a t t h e
s t r u c t u r e t e s t e d i n t h e s a m p l e d i d
n o t
c o r r e s p o n d
t o t h e s t r u c t u r e t h a t e x i s t e d i n t h e p o p u l a t i o n .
M e t h o d
M o d e l S p e c i f i c a t i o n
F o u r s p e c if i ca t io n s o f a n o b l i q u e t h r e e - f a c t o r
m o d e l w i t h t h r e e i n d i c a t o r s p e r f a c t o r w e r e e x a m -
i n e d. T h e b a s ic c o n f i r m a t o r y f a c t o r m o d e l i s p r e -
s e n t e d i n F i g u r e 1 . T h e p o p u l a t i o n p a r a m e t e r s
c o n s i s t e d o f f a c t o r l o a d i n g s ( e a c h A = . 70 ) , u n i q u e -
n e s s e s ( e a c h O ~ = . 51 ) , i n t e r f a c t o r c o r r e l a t i o n s
( e a c h ~b = . 3 0 ), a n d f a c t o r v a r i a n c e s ( a l l s e t t o 1 . 0 ) .
M o d e l 1 M o d e l 1 w a s p r o p e r l y s p e c i f i e d s u c h
t h a t t h e m o d e l t h a t w a s e s t i m a t e d i n t h e s a m p l e
d i r e c tl y c o r r e s p o n d e d t o t h e m o d e l t h a t e x i s t e d in
t h e p o p u l a t i o n . T h u s , b o t h t h e s a m p l e a n d t h e
p o p u l a t i o n m o d e l s c o r r e s p o n d e d t o t h e s o l id l in e s
p r e s e n t e d i n F i g u r e 1 .
M o d e l 2
M o d e l 2 c o n t a i n e d t w o f a c t o r lo a d -
i n g s t h a t w e r e e s t i m a t e d i n t h e s a m p l e b u t d i d
n o t
e x i s t i n t h e p o p u l a t i o n . T h u s , i n F i g u r e 1 , t h e
d o u b l e d a s h e d l in e s r e p r e s e n t t h e t w o f a c t o r lo a d -
i n g s t h a t l i n k e d I t e m 5 t o F a c t o r 3 a n d I t e m 8 t o
F a c t o r 2 , an d t h e e x p e c t e d v a l u e o f t h e s e p a r a m e -
t e r s w a s 0 . T h i s i s a m i s s p e c i f i c a t i o n o f i nc lus ion
N o t e t h a t f r o m t h e s t a n d p o i n t o f st a t is t ic a l th e o r y ,
e s t i m a t io n o f p a r a m e t e r s w i th a n e x p e c t e d v a l u e
o f 0 i n th e p o p u l a t i o n d o e s n o t b i a s t h e s a m p l e
r e s u lt s . M o d e l 2 is t h u s c o n s i d e r e d t o b e a p r o p e r l y
s p e c i f ie d m o d e l .
M o d e l 3
M o d e l 3 e x c l u d e d t w o l o a d in g s f r o m
t h e s a m p l e t h a t
d i d
e x i s t i n t h e p o p u l a t i o n . T h u s ,
i n F i g u r e 1 , t h e s i n g l e d a s h e d l i n e s r e p r e s e n t t h e
t w o e x c l u d e d f a c t o r l o a d i n g s ( b o t h p o p u l a t i o n
A s = . 35 ) t h a t l i n k e d I t e m 6 t o F a c t o r 3 a n d I t e m
7 t o F a c t o r 2 . T h e v a l u e o f A = .3 5 w a s c h o s e n t o
r e f l e c t a s m a ll t o m o d e r a t e f a c t o r l o a d i n g t h a t
m i g h t b e c o m m o n l y e n c o u n t e r e d i n p ra c t ic e . T h is
i s a m i s s p e c i f i c a t i o n o f
exc lus ion
M o d e l 4
F i n a l l y , M o d e l 4 w a s t h e c o m b i n a -
t i o n o f M o d e l s 2 a n d 3 . L i k e M o d e l 2 , tw o f a c t o r
l o a d i n g s w e r e e s t i m a t e d i n t h e s a m p l e t h a t d i d
n o t
e x i s t i n t h e p o p u l a t i o n ( t h e d o u b l e d a s h e d
l i n e s l i n k i n g I t e m 5 t o F a c t o r 3 a n d I t e m 8 t o
F a c t o r 2 ) . A d d i t i o n a l l y , l i k e M o d e l 3 , t w o f a c t o r
l o a d in g s w e r e e x c l u d e d f r o m t h e s a m p l e t h a t
d i d
e x i s t i n t h e p o p u l a t i o n ( t h e s i n g le d a s h e d l i n e s
t h a t l i n k e d I t e m 6 t o F a c t o r 3 a n d I t e m 7 t o
F a c t o r 2 ) . T h i s i s a m i s s p e c if i c a t io n o f b o t h
i nc lus ion
a n d
exc lus ion
2 Note tha t G LS i s a lso ava i l ab le in s t anda rd packages
and is re la t ively widely used. H ow ever , GL S is a norm al
theory e s t im a tor tha t i s a sym pto t i ca l ly equ iva len t to
ML , and p reviou s s tudies (e .g. , Mu th6n & Kaplan , 1985,
1992) have shown the behav ior o f ML and GLS to be
ve ry similar.
-
7/25/2019 Curran,West&Finch (1996)
5/14
2 0
C U R R A N W E S T A N D F I N C H
. 5 1 . 5 1 . 5 1 . 5 1 . 5 1 . 5 1 . 5 1 . 5 1 . 5 1
. . 7 0 \ . 70 1 . 7 0 / ' ~ . - ' ' ' ~ / ' \ 7 0 1 7 0
1 70
a o a o
. 3 0
Figure 1 Nine - indicator three- facto r obl ique co nf irmato ry factor analys is m odel with
populat ion parameter values . Solid l ines represent parameters that were shared be-
tween the sample and the populat ion; s ingle dashed l ines represent parameters that
ex i s ted in the popula t ion ) t = . 35) bu t wer e om i t ted f rom the s am ple ; dou ble dashed
lines repres ent param eters that did not exis t in the popu lat ion ) t = 0) but were
es t imated in the sample .
Cond i t ions
Multivariate distr ibutions
T h r e e p o p u l a t i o n
d i s t r i b u t i o n s w e r e c o n s i d e r e d f o r a l l f o u r m o d e l
s p e c i fi c a t io n s s e e F i g u r e 2 ). D i s t r i b u t i o n 1 w a s
m u l t i v a r i a t e n o r m a l w i t h u n i v a r i a t e s k e w n e s s a n d
k u r t o s e s e q u a l t o 0 . D i s t r i b u t i o n 2 w a s m o d e r a t e l y
n o n n o r m a l w i t h u n i v a r ia t e s k e w n e s s o f 2 .0 a n d
k u r t o s e s o f 7 .0 . F i n a ll y , D i s t r i b u t i o n 3 w a s s e v e r e l y
n o n n o r m a l w i t h u n i v a r i a t e s k e w n e s s o f 3 . 0 a n d
k u r t o s e s o f 2 1 . 0 . T h e s e l e v e l s o f n o n n o r m a l i t y
w e r e c h o s e n t o r e p r e s e n t m o d e r a t e a n d s e v e r e
n o n n o r m a l i t y b a s e d o n o u r e x a m i n a t i o n o f t h e
l e v e ls o f s k e w n e s s a n d k u r t o s e s i n d a t a s e ts f r o m
s e v e ra l c o m m u n i t y - b a s e d m e n t a l h e a l t h a n d s u b -
s t a n c e a b u s e s t u d i e s .
Sam ple s i z e F o u r s a m p l e s i z e s w e r e c o n s i d -
e r e d f o r a l l m o d e l s p e c i f i c a t i o n s : 1 0 0 , 2 0 0 , 5 0 0 ,
and 1 , 000 .
Replications A l l m o d e l s w e r e r e p l i c a t e d 2 0 0
t i m e s p e r c o n d i t i o n .
Data generation T h e r a w d a t a w e r e g e n e r a t e d
u s in g b o t h t h e P C a n d m a i n f r a m e v e r s io n o f E Q S
V e r s i o n 3 ; B e n t l e r , 1 9 8 9 ) . D e t a i l s o f t h e d a t a g e n -
e r a t i o n p r o c e d u r e a r e p r e s e n t e d i n t h e A p p e n d i x .
M e a s u r e s
T h r e e c h i - s q u a r e t e s t s t a t i s t i c s w e r e s t u d i e d :
n o r m a l t h e o r y M L , A D F , a n d t h e S B s c al e d X2.
A l l t h r e e t e s t s t a t i s t i c s w e r e c o m p u t e d b y E Q S
V e r s i o n 3 . 0 ). N o t e t h a t t h e M L a n d A D F s t at is t ic s
p r o v i d e d b y E Q S s h o u l d b e i d e n t i c a l t o t h a t
a v a i la b l e t h r o u g h c u r r e n t v e r s io n s o f L I S C O M P ,
L I S R E L , a nd R A M O N A .
R e s u l t s
Expec ted Va lue o f T es t S ta t i s t i c s
T h e e x p e c t e d v a l u e s o f t h e c h i - s q u a r e t e s t s ta t is -
t i c s f o r M o d e l s 1 a n d 2 w e r e s i m p l y t h e m o d e l
d e g r e e s o f f r e e d o m f o r al l t h r e e e s t i m a t o r s a c r o s s
a ll d i s t r ib u t i o n s a n d s a m p l e s i z es 2 4 .0 fo r M o d e l
1 a n d 2 2 . 0 f o r M o d e l 2 ) . B e c a u s e M o d e l s 3 a n d 4
w e r e m i s s p e c if i e d , t h e e x p e c t e d v a l u e s o f t h e s e
t e s t s t a t i s t i c s c o u l d n o t b e c o m p u t e d d i r e c t l y . I n -
s t ea d , t h e e x p e c t e d v a l u es w e r e c o m p u t e d a s la r ge
s a m p l e e m p i r i c a l e s t i m a t e s t h a t d i f f e r e d a s a f u n c -
t i o n o f m e t h o d o f e s t im a t i o n , m u l t i v a r i a t e d i s tr i-
b u t i o n , a n d s a m p l e s i z e. F u r t h e r d e t a i ls r e g a r d i n g
t h e c o m p u t a t i o n s o f t h e se e s t i m a t e s a r e p r e s e n t e d
i n th e A p p e n d i x .
M o n t e C a r l o R e s u l t s
T a b l e s 1 , 2 , 3 , a n d 4 p r e s e n t t h e m e a n o b s e r v e d
v a l u e , t h e e x p e c t e d v a l u e , t h e p e r c e n t a g e o f b i a s,
-
7/25/2019 Curran,West&Finch (1996)
6/14
C H I - S Q U A R E T E S T S T A T I S T I C S 2 1
~J
t -
Q
O
U .
5 [ ] 0 0 -
4 0 0 0
3 0 0 0
200O
1000
:
- 4 . 5 - 3 5
- 2 . 5 - 1 5
f ~
I I
I I
I I
I I
I . , |
, I
\
4 , 5 0 . 5
1 . 5 2 . 5 3 . 5 4 . 5
S t a n d a r d i z e d V a l u e
Figure 2 . P lo t s o f norm a l ( so lid l ine) , m od era te ly non-
norm al (do t t ed l ine ) , and s eve re ly nonnorm al (dashed
l ine ) em pi rica l d is t r ibu t ions based on a rand om sam ple
of N = 10,000.
a n d t h e p e r c e n t a g e o f m o d e l s r e j e c t e d a t p < .0 5
f o r t h e M L , S B , a n d A D F X 2 t e s t s t at i st i cs f o r
a l l f o u r m o d e l s p e c i f i c a t i o n s ? F o r t h e c o r r e c t l y
s p e c i f ie d m o d e l s ( M o d e l s 1 a n d 2 ) , t h e e x p e c t e d
r e j e c t i o n r a t e w a s 5 ; a n d g i v e n 2 0 0 r e p l i c a t i o n s
a n d a = . 05 , t h e 9 5 c o n f i d e n c e i n t e r v a l f o r t h e
p e r c e n t a g e o f r e j e c t e d m o d e l s d e f i n e d a n a p p r o x i -
m a t e u p p e r a n d l o w e r b o u n d o f 2 a n d 8 , r e s p e c-
t i v el y . R e j e c t i o n r a t e s f o r t h e o b t a i n e d c h i - s q u a r e
v a l u e s f a l li n g w i t h i n t h e s e b o u n d s a r e c o n s i s t e n t
w i t h t h e n u l l h y p o t h e s i s t h a t t h e e s t i m a t o r i s u n b i -
a s ed . M o d e l r e j e c t io n r a t e s w e r e n o t a s m e a n i n g f u l
f o r m i s s p e c i f ie d m o d e l s ( M o d e l s 3 a n d 4 ) , s o r e l a -
t i ve b ia s w a s c o m p u t e d ( t h e o b s e r v e d v a l u e m i n u s
t h e e x p e c t e d v a l u e d i v i d e d b y t h e e x p e c t e d v a l u e ) .
B i a s i n e x c e s s o f 1 0 w a s c o n s i d e r e d s i g n if i c a n t
( K a p l a n , 1 9 8 9 ) .
M o d e l S p e c i f i c a t i o n 1 .
T a b l e 1 p r e s e n t s t h e r e -
s u l ts f o r M o d e l 1 . R e c a l l t h a t M o d e l 1 w a s p r o p e r l y
s p e c i f ie d s u c h t h a t t h e m o d e l e s t i m a t e d i n t h e s a m -
p l e d i re c t l y c o r r e s p o n d e d t o t h e m o d e l t h a t e x i s t e d
i n th e p o p u l a t i o n . T h e e x p e c t e d v a l u e f o r a l l t h r e e
te s t s t a t i s t i c s wa s E (X 2) = 24 . 0 .
U n d e r m u l t i v a r ia t e n o r m a l i t y , t h e M L X 2 r e -
j e c t e d t h e e x p e c t e d n u m b e r o f m o d e l s a c r o ss a ll
s a m p l e s iz e s ( a p p r o x i m a t e l y 5 ) . C o n s i s t e n t w i t h
b o t h t h e o r y a n d p r e v i o u s s i m u l a t io n r e s e a r c h , t h e
M L X 2 b e c a m e i n c r e a s i n g l y p o s i t iv e l y b i a s e d a s
t h e d i s tr i b u ti o n b e c a m e i n c r e a s i ng l y n o n n o r m a l .
T h i s i n f l a t i o n w a s e x a c e r b a t e d w i t h i n c r e a s i n g
s a m p l e s iz e . F o r e x a m p l e , n e a r l y h a l f o f t h e c o r -
r e c t l y s p e c i f ie d m o d e l s w e r e r e j e c t e d f o r N = 1 ,0 0 0
u n d e r t h e s e v e r e ly n o n n o r m a l c o n d i ti o n . U n d e r
m u l t i v a r i a t e n o r m a l i t y , t h e A D F X 2 w a s i n f l a t e d
a t s m a l l s a m p l e s i ze s , f o r e x a m p l e , r e j e c t i n g 4 3
o f t h e c o r r e c t l y s p e c if i e d m o d e l s a t N = 1 0 0. T h e
p e r f o r m a n c e o f t h e A D F i m p r o v e d w i t h in c r e a si n g
s a m p l e s i ze , b u t e v e n u n d e r m u l t i v a ri a t e n o r m a l i t y
a t N = 1 ,0 0 0, 1 0 o f t h e c o r r e c t m o d e l s w e r e
r e j e c t e d . T h e A D F X 2 w a s a l s o p o s i t i v e l y b i a s e d
w i t h i n c r e a s i n g n o n n o r m a l i t y , b u t t h i s b i a s w a s
a t t e n u a t e d w i t h i n c r e a s i n g s a m p l e s iz e . F i n a l ly , t h e
S B X 2 w a s v e r y w e l l b e h a v e d a t n e a r l y a l l s a m p l e
s i z e s a c r o s s a l l d i s t r i b u t i o n s . F o r e x a m p l e , a t a
s a m p l e s iz e o f N = 2 0 0 u n d e r s e v e r e n o n n o r m a l i t y ,
t h e S B X 2 r e j e c t e d 7 o f th e p r o p e r l y s p e c i f ie d
m o d e l s ( c o m p a r e d t o 2 5 f o r A D F a n d 3 6 f o r
M L ) . U n d e r t h e s e c o n d i t io n s , t h e p e r f o r m a n c e o f
t h e S B X 2 r e p r e s e n t e d a d i s t in c t i m p r o v e m e n t o v e r
t h e M L X 2 u n d e r c o n d i t io n s o f n o n n o r m a l i t y . I n -
t e r e st i n g ly , t h e S B a n d A D F p e r f o r m e d s i m i la r ly
a t s a m p l e s o f N = 5 0 0 a n d N = 1 , 00 0 .
M o d e l S p e c i f ic a t io n 2 .
T h e r e s u l ts f r o m M o d e l
2 a r e p r e s e n t e d i n T a b l e 2 . R e c a l l t h a t M o d e l 2
e s t i m a t e d t w o f a c t o r l o a d i n g s i n t h e s a m p l e t h a t
d i d n o t e x i s t i n t h e p o p u l a t i o n . B e c a u s e t h e e r r o r
i s t h e a d d i t i o n o f t w o t r u l y n o n e x i s t e n t p a r a m e -
t e r s , t h i s c a n b e c o n s i d e r e d a p r o p e r l y s p e c i f i e d
m o d e l , a n d t h e e x p e c t e d v a l u e o f th e m o d e l c h i -
s q u a r e w a s e q u a l t o t h e m o d e l d e g r e e s o f f r e e -
d o m f o r a l l e s t i m a t o r s a c r o s s a l l s a m p l e s i z e s ,
E X 2) = 22.0.
O v e r a l l , t h e r e s u l t s f r o m M o d e l 2 c l o s e ly fo l -
l o w e d t h o s e o f M o d e l 1 . U n d e r m u l t i v a r i a te n o r -
m a l i ty , t h e r e j e c ti o n r a t e s f o r b o t h t h e M L a n d
S B w e r e s l i g h tl y h i g h e r t h a n e x p e c t e d a t N = 1 0 0
b u t w e r e u n b i a s e d a t N = 2 0 0 a n d g r e a t e r . U n d e r
m u l t i v a r ia t e n o r m a l i t y , th e A D F a g a in r e j e c t e d a
v e r y h ig h n u m b e r o f m o d e l s a t t h e t w o s m a l l e r
s a m p l e s i z e s b u t w a s u n b i a s e d a t t h e t w o l a r g e r
s a m p l e s iz e s. A s w i t h M o d e l 1 , t h e M L X 2 w a s
3 Al l im p roper so lu tions (non converg ed so lu t ions and
so lu t ions tha t converged bu t re su l t ed in ou t -o f -bound
param ete rs , e . g . , Heywood cases ) were d ropped f rom
subsequent analyses . Collaps ing across a l l condit ions ,
90 of the rep l i ca t ions were p rope r fo r M L and 83
w e r e p r o p e r f o r A D F .
-
7/25/2019 Curran,West&Finch (1996)
7/14
2 2 C U R R A N , W E S T , A N D F I N C H
T a b l e 1
Observed C hi-Square Expected Chi-Square Percentage Bias and Percentage of Rejected Models
for Model Specif ication 1
No rma l M o d e ra t e l y n o n n o rma l Se v ere ly n o n n o rm a l
Ob se rv e d Ex p e c t e d % % Ob se rv e d Ex p e c t e d % % Ob se rv e d Ex p e c t e d % %
Size X2 X2 X2 Bias Rejec t X2 X2 Bias Rejec t X2 X2 Bias Rejec t
100 M L 25.01 24.0 4.0 5.5 29.35 24.0 22.0 20.0 33.54 24.0 40.0 30.0
SB 25.87 24.0 8.0 7.5 26.06 24.0 9.0 8.5 27.26 24.0 14.0 13.0
A D F 36.44 24.0 52.0 43.0 38.04 24.0 59.0 49.0 44.82 24.0 87.0 68.0
200 ML 24.78 24.0 3.0 6.5 30.15 24.0 26.0 25.0 34.40 24.0 43.0 36.0
SB 25.22 24.0 5.0 8.5 25.44 24.0 6.0 8.0 25.80 24.0 8.0 6.5
A D F 29.19 24.0 22.0 19.0 29.27 24.0 22.0 19.0 31.29 24.0 30.0 25.0
500 ML 23.94 24.0 0.0 3.5 31.26 24.0 30.0 24.0 35.55 24.0 48.0 40.0
SB 24.10 24.0 0.0 5.0 25.44 24.0 6.0 6.9 24.85 24.0 4.0 8.5
A D F 25.92 24.0 8.0 11.0 26.42 24.0 10.0 6.7 26.83 24.0 12.0 8.5
1000 ML 25.05 24.0 4.0 7.0 30.78 24.0 28.0 24.0 37.40 24.0 56.0 48.0
SB 25.16 24.0 5.0 8.0 24.77 24.0 3.0 7.5 25.01 24.0 4.0 7.0
A D F 25.79 24.0 7.0 9.5 25.36 24.0 6.0 7.5 25.47 24.0 6.0 7.2
Note.
U n i v a r i a t e s k e w n e s s a n d k u r t o s e s w e r e ( 0 ,0 ) , (2 , 7) , a n d ( 3 ,2 1) f o r n o r m a l , m o d e r a t e l y n o n n o r m a l , a n d s e v e r e l y n o n n o r m a l
d i s t r ib u t i o n s , r e s p ec t iv e l y . M L = m a x i m u m l i k e l i ho o d ; S B = S a t o r r a - B e n t l e r r e s c a le d ; A D F = a s y m p t o t i c d i s t r ib u t i o n
f re e .
i n c r e a s i n g l y p o s i t i v e l y b i a s e d w i t h i n c r e a s i n g n o n -
n o r m a l i t y , a n d t h i s i n f l a t i o n w a s e x a c e r b a t e d w i t h
i n c r e a s i n g s a m p l e s iz e . I n c o m p a r i s o n , t h e S B X 2
s h o w e d m i n i m a l b i a s w i t h i n c r e a s i n g n o n n o r m a l -
i ty , a l t h o u g h t h e o b s e r v e d r e j e c t i o n r a t e s a t t h e
s m a l l e s t s a m p l e s i ze w e r e s l i gh t l y l a r g e r t h a n e x -
p e c t e d . E v e n u n d e r s e v e r e n o n n o r m a l i t y , t h e S B
X 2 a g a i n s h o w e d l i t t l e e v i d e n c e o f b i a s , e s p e c i a l l y
a t s a m p l e s i z e s o f N - - 2 0 0 o r g r e a t e r . F i n a l l y ,
t h e A D F X 2 w a s p o s i t i v e l y b i a s e d w i t h i n c r e a s i n g
n o n n o r m a l i t y a t t h e s m a l l e r s a m p l e s i z es b u t w a s
u n b i a s e d a t s a m p l e s i z e s o f N = 5 0 0 a n d N =
1 ,0 0 0 , e v e n u n d e r s e v e r e n o n n o r m a l i t y .
M o d e l S p e c i f i c a t i o n 3 .
M o d e l S p e c i f i c a t i o n 3
e x c l u d e d t w o f a c t o r l o a d i n g s i n t h e s a m p l e ( )t =
. 35 ) t h a t t r u l y e x i s t e d i n th e p o p u l a t i o n . T h e s e
r e s u l t s a r e p r e s e n t e d i n T a b l e 3 . R e c a l l t h a t d u e
t o t h e e x c l u s i o n o f e x i s t i n g p a r a m e t e r s , t h e r e w a s
a d i f f e r e n t e x p e c t e d v a l u e f o r e a c h t e s t s t a ti s t i c .
A l s o , b e c a u s e t h e m o d e l w a s m i s s p e c i f i e d i n t h e
T a b l e 2
Observed C hi-Square Expected C hi-Square Percentage Bias and Percentage of Rejected Models
for Model Specif ication 2
No rma l M o d e ra t e l y n o n n o rm a l Se ve rel y n o n n o rma l
Ob se rv e d Ex p e c t e d % % Ob se rv e d Ex p e c t e d % % Ob se rv e d Ex p e c t e d % %
Size X2 g 2 X2 Bias Rejec t X2 2 2 Bias Rejec t X2 X2 Bias Rejec t
100 M L 23.42 22.0 6.0 9.6 26.89 22.0 22.0 22.2 29.82 22.0 36.0 34.4
SB 24.19 22.0 9.0 12.1 23.80 22.0 8.0 8.5 25.07 22.0 14.0 11.1
A D F 31.0 22.0 41.0 30.5 34.95 22.0 59.0 40.8 46.45 22.0 111.0 63.2
200 ML 22.48 22.0 2.0 6.0 27.70 22.0 26.0 26.5 31.77 22.0 44.0 36.7
SB 22.86 22.0 3.0 7.0 23.75 22.0 8.0 8.5 24.10 22.0 10.0 8.5
A D F 26.43 22.0 20.0 17.5 26.06 22.0 18.0 14.5 28.52 22.0 30.0 22.0
500 M L 21.89 22.0 0.0 6.0 26.68 22.0 21.0 19.0 31.86 22.0 45.0 32.5
SB 22.02 22.0 0.0 7.0 21.90 22.0 0.0 5.5 23.33 22.0 6.0 6.5
A D F 23.13 22.0 5.0 7.0 23.04 22.0 5.0 8.0 24.02 22.0 9.0 5.5
1000 M L 22.25 22.0 0.0 5.0 26.05 22.0 18.0 14.5 33.37 22.0 52.0 41.5
SB 22.31 22.0 0.0 3.5 21.14 22.0 4~0 4.0 22.74 22.0 3.0 7.0
A D F 22.09 22.0 0.0 6.0 23.32 22.0 6.0 9.5 23.41 22.0 6.0 7.5
Note.
U n i v a r i a t e s k e w n e s s a n d k u r t o s e s w e r e ( 0 ,0 ) , ( 2, 7 ), a n d ( 3 ,2 1) f o r n o r m a l , m o d e r a t e l y n o n n o r m a l , a n d s e v e r e l y n o n n o r m a l
d i s t ri b u t io n s , r e s p e c ti v e ly . M L = m a x i m u m l i k e l i h o o d; S B = S a t o r r a B e n t l e r r es c a l ed ; A D F = a s y m p t o t i c d i s t r ib u t i o n
free .
-
7/25/2019 Curran,West&Finch (1996)
8/14
C H I - S Q U A R E T E S T S T A T I S T I C S 2 3
T a b l e 3
Observed C hi-Square Expected C hi-Square Percentage Bias and Percentage of Rejected Models
for Model Specification 3
Normal Moderately nonnormal Severely nonnormal
Observed Expected Observed Expected Observed Expected
Size X2 2,2 X2 Bia s Reject X2 X2 Bias Reject A, X2 Bias Reject
100 M L 38.45 37.62 2.0 54.3 46.50 37.75 23.0 78.9 52.87 37.77 40.0 81.4
SB 39.63 37.65 5.0 57.9 37.63 33.99 11.0 49.1 38.10 30.85 24.0 47.1
AD F 52.04 33.74 54.0 80.3 60.88 29 .68 105 .0 86.3 81.31 27 .29 19 8.0 95.3
200 M L 51.07 51.38 0.0 88.1 59.72 51.66 16.0 93.7 68.58 51.68 33.0 95.4
SB 51.65 51.44 0.0 89.9 47.04 44.09 7.0 82.4 44.66 37.77 18.0 78.9
A D F 50.17 43.59 15.0 85.6 47.93 35.42 36.0 84.8 47.16 30.61 54.0 75.2
500 ML 92.27 92.66 0.0 100.0 99.39 93.37 6.0 1 0 0 . 0 109.87 93.42 18 .0 100.0
SB 92.52 92.80 0.0 100 .0 75.04 74.40 1.0 100 .0 63.46 58.52 8.0 97.2
A D F 76.89 73.11 5.0 100 .0 60.10 52.63 14.0 99.5 53.67 40.58 32.0 96.7
1000 ML 161.46 161.35 0.0 1 0 0 .0 171.07 162.88 5.0 10 0. 0 180.90 162.98 11.0 100.0
SB 161.66 161.74 0.0 1 0 0 .0 126 .23 124.89 2.0 1 0 0 .0 101.25 93.11 9.0 100.0
A D F 127.71 122.32 4.0 10 0.0 90.23 81.31 11 .0 1130.0 76.10 57.20 33 .0 100.0
Note. Univ ariate skewness and k urtoses were (0,0) , (2,7), and (3,21) for normal, mo derately nonnorma l, and severely nonnorm al
distr ibutions, respectively. ML = maximum likel ihood; SB = Satorra -Ben tler rescaled; A DF = asymp totic distr ibution free.
s a m p l e , t h e p e r c e n t a g e o f r e j e c t e d m o d e l s w a s n o
l o n g e r a m e a n i n g f u l g u i d e w i t h w h i c h t o ju d g e t h e
b e h a v i o r o f t h e t e s t s t a ti s t ic s . T h u s , t h e f o l l o w i n g
r e s u l ts w i ll n o w b e p r e s e n t e d i n t e r m s o f t h e r e l a -
t i v e b i a s i n t h e t e s t s t a t i s t i c s . 4
U n d e r m u l t i v a r i a t e n o r m a l i t y , t h e e x p e c t e d v a l -
u e s f o r t h e M L a n d S B X 2 w e r e n e a r l y i d e n t i c a l
a c r o s s a l l f o u r s a m p l e s i ze s . T h i s i s f u r t h e r s u p p o r t
t h a t f o r n o r m a l d i s t r ib u t i o n s , n o s c a l i n g c o r re c t i o n
i s r e q u i r e d f o r t h e M L X 2, a n d t h e S B X 2 t h u s
s i m p l i f i e s t o t h e M L X 2. A d d i t i o n a l l y , n e i t h e r t h e
M L o r S B t e s t s t a t i s t ic s h o w e d a p p r e c i a b l e b i a s
u n d e r n o r m a l i t y a c r o s s a l l f o u r s a m p l e s i z e s. I n
c o m p a r i s o n , t h e e m p i r i c a l e s t i m a t e o f t h e e x -
p e c t e d v a l u e o f t h e A D F X 2 w a s s m a l l e r t h a n t h a t
o f t h e M L o r S B t e s t s t a ti s t ic s . R e c a l l t h a t t h e
e x p e c t e d v a l u e f o r a l l t h r e e t e s t s ta t i s ti c s w e r e
e q u a l f o r t h e p r o p e r l y s p e c i fi e d m o d e l s . T h e l o w e r
e x p e c t e d v a l u e o f th e A D F f o r m i s sp e c i f ie d m o d -
e ls e v e n u n d e r m u l t i v a r i a t e n o r m a l i t y s u g g e s t s th a t
t h i s t e s t s t a t is t i c m a y h a v e l e s s p o w e r t o r e j e c t t h e
n u l l h y p o t h e s i s c o m p a r e d w i t h t h e M L o r S B X 2.
U n l i k e t h e M L a n d S B , t h e A D F w a s s ig n i f i c a n tl y
p o s i t i v e ly b i a s e d a t t h e t w o s m a l l e r s a m p l e s iz e s.
F o r e x a m p l e , a t N = 1 00 t h e a v e r a g e o b s e r v e d
A D F X 2 w a s 5 4 l a r g e r t h a n t h e e x p e c t e d v a l u e .
T h i s b i a s d r o p p e d t o 1 5 a t N = 2 0 0 a n d w a s
n e g l i g i b l e a t th e t w o l a r g e r s a m p l e s i z e s.
T h e f in d in g s b e c o m e m o r e c o m p l i c a t e d g i ve n
n o n n o r m a l d i s t r i b u t io n s . T h e e x p e c t e d v a l u e o f
t h e M L X 2 w a s t h e s a m e a c r o s s a l l t h r e e d i s t r i b u -
t io n s . A s w i t h t h e p r e v i o u s m o d e l s , t h e M L X 2
s h o w e d i n c r e a s i n g l e v e l s o f p o s i t i v e b i a s w i t h i n -
c r e a s i n g n o n n o r m a l i t y . A p a r t i c u l a r l y i n t e r e s t i n g
f i n d in g p e r t a i n e d t o t h e e x p e c t e d v a l u e s o f th e S B
a n d A D F t e s t s t at i st i cs u n d e r n o n n o r m a l i t y . B o t h
t h e e x p e c t e d a n d t h e o b s e r v e d v a l u e s o f t h e S B
a n d A D F t e s t s t a ti s t i c s d e c r e a s e d w i t h i n c r e a s i n g
n o n n o r m a l i t y . F o r e x a m p l e , a t s a m p l e s iz e N =
2 0 0 , th e e x p e c t e d v a l u e f o r t h e S B X 2 w a s a p p r o x i -
m a t e l y 5 1 u n d e r n o r m a l i t y , 4 4 u n d e r m o d e r a t e
n o n n o r m a l i t y , a n d 3 8 u n d e r s e v e r e n o n n o r m a l i t y .
T h e A D F t e s t s t a t is t i c s h o w e d a s i m i l a r p a t t e r n .
T h e d i r e c t i n t e r p r e t a t i o n o f t h i s f i n d i n g is t h a t i t
i s i n c r e a s i n g l y d i f f i c u lt to d e t e c t a m i s s p e c i f i c a t i o n
w i t h i n t h e m o d e l g i v e n t h e a d d e d v a r i a b i l i t y d u e
t o t h e n o n n o r m a l d i s t r i b u t io n o f t h e d a ta . T h u s ,
t h e p o w e r o f S B a n d A D F t e s t s t at i st ic s d e c r e a s e d
w i t h i n c r e a s in g n o n n o r m a l i t y .
U n d e r m o d e r a t e n o n n o r m a l i t y , t h e S B X 2 w a s
s l i g h t l y b i a s e d a t N = 1 00 ( 1 1 ) b u t w a s u n b i a s e d
a t s a m p l e s i z e s o f N = 2 0 0 a n d a b o v e . I n c o m p a r i -
s o n , a l so u n d e r m o d e r a t e n o n n o r m a l i t y , t he A D F
X 2 s h o w e d e x t r e m e b i a s a t t h e s m a l l e r s a m p l e s iz e s
4 M o d e l s 1 a n d 2 c o u l d h a v e s i m i l a r l y b e e n e v a l u a t e d
u s i n g th e p e r c e n t a g e o f b ia s , a n d t h e s a m e c o n c l u s i o n s
w o u l d h a v e b e e n d r a w n . T h e p e r c e n t a g e o f r e j e c te d
m o d e l s w a s c h o s e n i n s t e a d f o r M o d e l s 1 a n d 2 g i v e n
t h e m o r e d i r e c t i n t e r p r e t a b i l i t y o f th e f i n di n g s .
-
7/25/2019 Curran,West&Finch (1996)
9/14
2 4 C U R R A N , W E S T , A N D F I N C H
T a b l e 4
Observed Ch i-Square Expected C hi-Square Percentage Bias and Percentage of Rejected Models
for Model Specif ication 4
Normal Moderately nonnormal Severely nonnormal
Observed Expected % % Observed Expected % % Observed Expected % %
Size X2 X2 X Bias Reject X ~v Bias Reject X2 X Bia s Reject
100 ML 34.03 32.52 5.0 48.7 40.37 32.52 24.0 63.4 45.15 32.45 39.0 78.6
SB 34.97 32.56 7.0 51.8 33.94 29.76 14.0 45.5 34.09 27.33 25.0 50.3
AD F 47.99 30.66 56.0 82.1 48.67 27.19 79.0 85.4 63.25 25 .06 152.0 91.8
200 ML 43.75 43.14 1.0 81.1 49.84 43.14 16.0 91.0 58.14 43.01 35.0 88.1
SB 44.34 43.23 3.0 81.6 39.55 37.59 5.0 67.9 38.48 32.71 18.0 60.4
A D F 46.42 39.41 18.0 85.5 41.84 32.43 29.0 73.4 46.05 28.16 64.0 84.5
500 ML 75.29 75.04 0.0 100.0 81.35 75.04 8.0 100.0 91.71 74.68 23. 0 100.0
SB 75.62 75.23 0.0 100 .0 62.09 61.09 2.0 97.0 55.94 48.85 15.0 92.6
A D F 69.62 65.66 6.0 100 .0 53.84 48.15 12.0 94.2 49.13 37.45 31.0 93.2
1000 ML 128.71 128.20 0.0 1 0 0 .0 133.8 6 128.20 4.0 1 0 0 .0 144 .56 127.46 13.0 100.0
SB 129.16 128.56 0.0 1 0 0 .0 10 0.4 8 100.26 0.0 100.0 83.44 75.76 1 0.0 100.0
AD F 111.39 109.41 2.0 100.0 80.91 74.35 9.0 100.0 68.25 52.92 30 .0 100.0
Note.
Univ ariate skewness and kurtoses were (0,0) , (2,7), and (3,21) for normal, mode rately nonnormal, and severely nonnormal
distr ibutions, respectively. ML = maximum likel ihood; SB = Satorra-B entler rescaled; AD F = asymptotic distr ibution free.
a n d r e m a i n e d b i a s e d e v e n a t N = 1 ,0 0 0 ( 1 1 % ) .
U n d e r s e v e r e n o n n o r m a l i t y , t h e S B X 2 s h o w e d
s u b s t a n t i a l b i a s a t t h e t w o s m a l l e r s a m p l e s i z e s
( e .g . , 1 8 % a t N = 2 0 0 ) b u t w a s o n l y m o d e r a t e l y
b i a s e d a t t h e l a r g e r s a m p l e s i z e s ( e .g . , 9 % a t N =
1 ,0 0 0 ). T h e A D F s h o w e d v e r y h i g h l e v e l s o f r e l a -
t i v e b i a s a c r o s s a l l f o u r s a m p l e s i z e s u n d e r s e v e r e
n o n n o r m a l i t y a n d w a s o v e r e s t i m a t e d b y 3 3 % e v e n
a t t h e l a r g e s t s a m p l e s i z e N = 1 , 00 0 .
Model Spec i f i ca t i on 4 .
M o d e l S p e c i f ic a t i o n 4
c o n t a i n e d b o t h e r r o r s o f i n cl u s i o n a n d e x c l u s io n .
T w o c r o s s - lo a d i n g s e x i st e d in t h e p o p u l a t i o n t h a t
w e r e n o t e s t i m a t e d i n th e s a m p l e ( A = . 35 ), a n d
t w o c r o s s - l o a d i n g s w e r e e s t i m a t e d i n t h e s a m p l e
t h a t d i d n o t e x i s t i n t h e p o p u l a t i o n ( A = 0 ) . T h e s e
r e s u l t s a r e p r e s e n t e d i n T a b l e 4 .
T h e f i n d i n g s f r o m M o d e l S p e c i f i c a t i o n 4 fo l -
l o w e d t h e s a m e g e n e r a l p a t t e r n a s w a s o b s e r v e d
f o r M o d e l S p e c i f ic a t i o n 3 . T h e p r i m a r y d i f f e r e n c e
w a s t h a t t h e e x p e c t e d v a l u e s a n d r e j e c t i o n r a te s
i n M o d e l 4 w e r e l o w e r c o m p a r e d w i t h t h o s e o f
M o d e l 3 . T h i s r e s u l t m a y i n i ti a l ly a p p e a r c o u n t e r -
i n t u it i v e gi v e n th a t M o d e l 4 c o m b i n e d e r r o r s o f
b o t h i n c l u si o n a n d e x c l u s io n . H o w e v e r , u n l i k e
M o d e l 3 , M o d e l 4 c o n t a i n e d t h e s i m u l t a n e o u s e s t i-
m a t i o n o f t h e t w o t r u ly n o n e x i s t e n t p a t h s a n d t h e
e x c l u s io n o f t h e t w o t r u ly e x i s t e n t p a th s . T h e m e a n
e s t i m a t e d f a c t o r l o a d i n g s f o r t he t w o a d d i t i o n a l
p a t h s i n M o d e l 2 ( w h e r e n o o t h e r p a t h s w e r e e x -
c l u d e d ) w a s A = 0 ( th e p o p u l a t i o n e x p e c t e d v a l u e ) .
H o w e v e r , t h e m e a n f a c t o r l o a d i n g s f o r t h e s e s a m e
t w o a d d i t i o n a l p a th s i n M o d e l 4 w a s h = - . 4 0 .
T h u s , t h e s e a d d i t i o n a l fr e e p a r a m e t e r s s e r v e d t o
a b s o r b t h e m i s sp e c i f i c a t io n , a n d M o d e l 4 r e -
s u i t e d i n a b e t t e r f it t o t h e d a t a t h a n d i d M o d e l 3 .
A s w i t h M o d e l 3 , u n d e r m u l t i v a r i a t e n o r m a l i t y ,
t h e e x p e c t e d v a l u e s o f t h e M L a n d S B X 2 w e r e
e q u a l t o o n e a n o t h e r w h e r e a s t h e e x p e c t e d v a l u e
o f th e A D F g 2 w a s s m a l l e r . N e i t h e r t h e M L o r S B
X 2 s h o w e d a n y s i g n i f i c a n t b ia s u n d e r t h e n o r m a l
d i s t r i b u t i o n a c r o s s a n y o f t h e f o u r s a m p l e s i z e s.
T h e l a r g e s t b i a s w a s f o r t h e S B X 2 a t N = 1 0 0
( 7 % ), b u t t h e m a g n i t u d e o f b i a s d r o p p e d t o n e a r
0 a t s a m p l e s iz e s o f N = 2 0 0 a n d a b o v e . I n c o m p a r i -
s o n , t h e A D F X 2 w a s a g a i n s i g n i f ic a n t l y o v e r e s t i -
m a t e d a t th e t w o s m a l l e r s a m p l e s i z es b u t w a s
u n b i a s e d a t s a m p l e s i ze s o f N = 5 0 0 a n d a b o v e .
T h e e x p e c t e d v a l u e o f t h e M L X 2 w a s a g a i n
e q u a l a c r o s s d i s t r i b u t i o n s , a n d t h e M L X 2 w a s i n -
c r e a s i n g ly p o s i t i v e l y b i a s e d w i t h i n c r e a s i n g n o n -
n o r m a l i t y . L i k e M o d e l 3 , t h e e x p e c t e d v a l u e s f o r
t h e S B a n d A D F X 2 d e c r e a s e d w i t h i n c r e a s i n g
n o n n o r m a l i t y . T h e S B g 2 s h o w e d i n c r e a s i n g p o s i-
t i v e b i a s w i t h i n c r e a s i n g n o n n o r m a l i t y . T h i s b i a s
b e c a m e n e g l i g i b le a t N = 2 00 u n d e r m o d e r a t e
n o n n o r m a l i t y ( 5 % ) b u t w a s s t i ll s l i g h t ly b i a s e d
e v e n a t N = 1 ,0 0 0 u n d e r s e v e r e n o n n o r m a l i t y
( 1 0 % ) . F i n a l l y , t h e A D F X 2 w a s a g a i n s t r o n g l y
b i a s e d , w i t h i n c r e a s i n g n o n n o r m a l i t y e v e n a t th e
l a r g e s t s a m p l e s i z e .
-
7/25/2019 Curran,West&Finch (1996)
10/14
C H I - S Q U A R E T E S T S T A T IS T I C S 2 5
D i s c u s s i o n
T h e f ir st t w o m o d e l s w e r e t h e o r e t i c a l l y p r o p -
e r l y s p ec i f ie d . M o d e l 1 w a s e s t i m a t e d i n t h e
s a m p l e p r e c i s e l y a s i t e x i s t e d i n t h e p o p u l a t i o n ,
w h e r e a s M o d e l 2 i n c l u d e d t w o p a r a m e t e r s i n t h e
s a m p l e t h a t d i d n o t e x i s t i n t h e p o p u l a t i o n . T h e
f i n d i n g s f o r t h e M L X 2 f r o m M o d e l s 1 a n d 2
c l o s e l y r e p l i c a t e b o t h p r e v i o u s t h e o r e t i c a l p r e d i c -
t i o n s a n d e m p i r i c a l f i n d i n g s . F o r e x a m p l e , t h e
M L X 2 s h o w e d n o e v i d e n c e o f b i a s a c r o s s a l l
s a m p l e s i z e s u n d e r m u l t i v a r i a t e n o r m a l d i s t r i b u -
t i o n s b u t w a s s i g n i f ic a n t ly i n f l a t e d w i t h i n c r e a s i n g
n o n n o r m a l i t y . T h u s , a c o r r e c t m o d e l w a s s i g n if i-
c a n t l y m o r e l i k e l y t o b e e r r o n e o u s l y r e j e c t e d
b a s e d o n t h e M L X 2 g i v e n d e p a r t u r e s f r o m a
m u l t i v a r i a t e n o r m a l d i s t r i b u t i o n t h u s r e s u l t i n g
i n a n i n c r e a s e d T y p e I e r r o r r a t e ) .
T h e A D F a n d S B t e s t s ta ti st ic s h a v e b e e n p r o -
P O se d as a l t e r n at i v e s t o t h e n o r m a l t h e o r y M L t e st
s ta t is ti c w h e n t h e o b s e r v e d d a t a d o n o t m e e t t h e
m u l t i v a r i a t e n o r m a l i t y a s s u m p t i o n . C o n s i s t e n t
w i t h p r e v i o u s r e s e a r c h o n p r o p e r l y s p e c i fi e d m o d -
e l s, t h e A D F X 2 w a s s u b s t a n t i a ll y i n f l a t e d a t
s m a l l e r s a m p l e s i z es , e v e n u n d e r m u l t i v a r i a t e n o r -
m a l d i s t r i b u t io n s . A l t h o u g h t h is s m a l l s a m p l e s i ze
i n f l a t i o n w a s e x a c e r b a t e d w i t h i n c r e a s i n g n o n n o r -
m a l i t y , t h e A D F w a s u n b i a s e d a t s a m p l e s i z e s o f
N = 5 0 0 a n d a b o v e , r e g a r d l e s s o f d i st r i b u t i o n .
T h e S B X 2 p e r f o r m e d q u i t e w e l l a c r o ss n e a r l y a l l
s a m p l e s i z e s a n d a l l d i s t r i b u t i o n s a n d s h o w e d n o
e v i d e n c e o f bi as e v e n u n d e r s e v e r e l y n o n n o r m a l
d i s t r i b u t i o n s a t s a m p l e s i z e s o f N = 2 0 0 o r m o r e .
T h e s e a r e v e r y h e a r t e n i n g f i n d in g s f o r t h e p r a c t i c-
i n g r e s e a r c h e r w h o e n c o u n t e r s n o n n o r m a l d a t a
a s a w a y o f li f e e . g. , i n t h e s t u d y o f a d o l e s c e n t
s u b s t a n c e u s e o r p s y c h o p a t h o l o g y ) . N o t o n l y w a s
t h e S B X 2 a c c u r a t e u n d e r e v e n s e v e r e l y n o n n o r m a l
d i s t r i b u ti o n s , b u t t h e S B X 2 s i m p l i fi e d t o t h e M L
X 2 u n d e r c o n d i t i o n s o f m u l t i v a ri a t e n o r m a l i ty . A s -
s u m i n g a p r o p e r l y s p e c i f ie d m o d e l , t h e S B X 2 a p -
p e a r s t o b e a v e r y u s e f ul m e a s u r e o f f it g i v e n m o d -
e r a t e l y si z ed s a m p l e s a n d n o n n o r m a l d a t a.
W h e r e a s m a n y o f t h e re s u lt s f r o m M o d e l s 1
a n d 2 w e r e p r e d i c t e d f r o m t h e o r y a n d p r e v i o u s
r e s e a r c h , t h e f i n d i n g s f r o m M o d e l s 3 a n d 4 w e r e
n o t . R e c a l l t h a t M o d e l s 3 a n d 4 w e r e t w o v a r i a t i o n s
o f a m i ss p e c if i ed m o d e l w h e r e t h e m o d e l e s t i m a t e d
i n t h e s a m p l e d i d n o t c o n f o r m t o t h e m o d e l t h a t
e x i s t e d i n t h e p o p u l a t i o n . S t u d y i n g t h e b e h a v i o r
o f t h e t e s t s t a t is t ic s u n d e r t h e s e c o n d i t i o n s i s o f
p a r t i c u l a r i n t e r e s t g i v e n t h e h i g h l i k e l i h o o d t h a t
t h e m o d e l e s t i m a t e d i n t h e s a m p l e d o e s n o t p r e -
c i s e ly c o n f o r m t o t h e m o d e l t h a t e x i st s i n t h e p o p u -
l a t io n . T h e r e s u l t s f o r t h e M L X 2 w e r e a s e x p e c t e d :
T h e M L t e s t st a ti s ti c s h o w e d n o e v i d e n c e o f b i as
a t a n y s a m p l e s i z e u n d e r m u l t i v a r i a t e n o r m a l i t y
b u t w a s i n c r e a s i n g l y in f l a t e d g i v e n i n c r e a si n g n o n -
n o r m a l i t y . A s i n M o d e l s 1 a n d 2 , t h e S B X 2 a l so
s h o w e d n o e v i d e n c e o f b ia s a t a n y s a m p l e s iz e
g i v e n m u l t i v a r i a t e n o r m a l i t y , a n d t h u s s i m p l i f i e d
t o t h e M L X :. I n t e r e s ti n g l y , t h e e x p e c t e d v a l u e
f o r t h e A D F X 2 u n d e r m o d e l m i s s p ec i fi c a ti o n w a s
m u c h s m a l l er t h a n t h a t o f t h e M L a n d S B , e v e n
u n d e r m u l t i v a r i a t e n o r m a l i t y . T h i s s u g g e s t s t h a t ,
c o m p a r e d t o t h e M L a n d S B , t h e A D F t e st st a ti s ti c
m a y b e a l es s p o w e r f u l t e s t o f t h e n u l l h y p o t h e s i s .
T h i s c o n c l u s i o n i s t e n t a t i v e , a n d m o r e w o r k i s
n e e d e d t o b e t t e r u n d e r s t a n d t h i s f i n d i n g . L i k e
M o d e l s 1 a n d 2 , t h e A D F w a s p o s i ti v e ly b ia s e d
u n d e r m u l t i v a r i a t e n o r m a l d i s t r i b u t i o n s a t t h e t w o
s m a l l e r s a m p l e s i z e s b u t s h o w e d n o b i a s a t t h e t w o
l a r g e r s a m p l e s i z e s .
T h e m o s t s u r p r i s i n g f i n d i n g s r e l a t e d t o t h e b e -
h a v i o r o f th e S B a n d A D F t e s t s t a ti s ti c s u n d e r t h e
s i m u l t a n e o u s c o n d i t i o n s o f m i s s p e c i fi c a t i o n a n d
m u l t i v a r ia t e n o n n o r m a l i t y M o d e l s 3 a n d 4 ). T h e
e x p e c t e d v a l u e s o f t h e s e t e s t s t a t i s t i c s m a r k e d l y
d e c r e a s e d w i t h i n c r e a s i n g n o n n o r m a l i t y . T h a t i s ,
a ll e ls e b e i n g e q u a l , t h e S B a n d A D F t e s t s ta t is t ic s
w e r e l es s l ik e l y to d e t e c t a s p e c i f ic a t i o n e r r o r g i v e n
i n c re a s in g d e p a r t u r e s f r o m a m u l t i v a r ia t e n o r m a l
d i s t r i b u t i o n . T h e m o r e s e v e r e t h e n o n n o r m a l i t y ,
t h e g r e a t e r t h e c o r r e s p o n d i n g l o ss o f p o w e r . T h i s
r e s u l t w a s u n e x p e c t e d , a n d w e a r e n o t a w a r e o f
a n y p r e v i o u s d i s c u s s i o n s o f t h i s f i n d i n g .
A l t h o u g h t h e s p e c i f ic r e a s o n f o r t h i s l os s o f
p o w e r i s c u r r e n t l y n o t k n o w n , w e t h e o r i z e t h a t i t
i s d u e t o t h e i n c lu s i on o f t h e f o u r t h - o r d e r m o m e n t s
k u r t o s e s ) in t h e c o m p u t a t i o n o f th e S B a n d A D F
t e s t s t a t i s t i c s , i n f o r m a t i o n t h a t i s i g n o r e d b y t h e
n o r m a l t h e o r y M L X : . R e c a l l t h a t a n o r m a l d i s t r i -
b u t i o n i s c o m p l e t e l y d e s c r i b e d b y t h e f i r st t w o
m o m e n t s , t h e m e a n a n d t h e v a r i a n ce . A s t h e d i s tr i-
b u t i o n b e c o m e s i n c r e a s i n g l y n o n s y m m e t r i c , i s
c h a r a c t e r i z e d b y t h ic k e r o r t h i n n e r t ai ls c o m p a r e d
w i t h t h e n o r m a l c u r v e ) , o r b o t h , a d d i ti o n a l p a r a m -
e t e r s a r e n e e d e d t o d e s c r i b e t h i s m o r e c o m p l e x
d i s t r i b u t io n . B e c a u s e M L i s a n o r m a l t h e o r y e s ti -
m a t o r , i t is a ss u m e d t h a t t h e f o u r t h - o r d e r m o -
m e n t s a r e e q u a l t o 0 , m u l t i v a r i a t e k u r t o s i s i s i g -
n o r e d , a n d t h e e x p e c t e d v a l u e o f t h e M L X 2 i s
-
7/25/2019 Curran,West&Finch (1996)
11/14
2 6 C U R R A N , W E S T , A N D F I N C H
e q u a l a c r o s s a l l d i s t r i b u t i o n s . 5 I n c o n t r a s t , t h e
A D F a n d S B d o n o t as s u m e m u l t i v a ri a t e n o r m a l -
i ty , t h e f o u r t h - o r d e r m o m e n t s a r e n o t a s s u m e d t o
e q u a l 0 , a n d m e a s u r e s o f m u l t i v a r i a t e k u r t o s i s a r e
e x p li c it ly in c o r p o r a t e d i n t o t h e c o m p u t a t i o n o f t h e
t e s t s t a t i s t i c s . A s a r e s u l t , t h e e x p e c t e d v a l u e s o f
t h e A D F a n d S B d i r e c tl y d e p e n d u p o n t h e p a r t ic u -
l a r c h a r a c t e r is t i c s o f t h e m u l t i v a r i a t e d i s t r i b u t i o n
u n d e r c o n s i d e r a ti o n .
T h e i n c l u s i o n o f m u l t i v a r i a t e k u r t o s i s i n t o t h e
c o m p u t a t i o n o f t h e S B a n d A D F t e s t s t a ti s ti c s
p r o v i d e s t h e c r i ti c a l i n f o r m a t i o n n e c e s s a r y t o f u l l y
d e s c r ib e t h e m o r e c o m p l e x n o n n o r m a l d i s t ri b u -
t i o n . H o w e v e r , t h i s a d d e d i n f o r m a t i o n r e s u l t i n g
f r o m t h e m o r e c o m p l e x d i s t r ib u t i o n a l so r e d u c e s
t h e a b i l it y o f t h e S B a n d A D F t o i d e n t i f y a g i v e n
m o d e l m i s s p e c i f i c a t i o n . O t h e r w i s e s t a t e d , w e c a n
t h i n k o f a h y p o t h e t i c a l s i gn a l t o n o i s e r a t i o i n
w h i c h t h e t e s t s t at i st i c is a t t e m p t i n g t o i d e n t i f y t h e
p r e s e n c e o f t h e s i g n a l (i . e. , t h e m i s s p e c i f i c a t i o n )
a g a i n s t t h e b a c k g r o u n d n o i s e ( i . e . , t h e s a m p l i n g
v a r ia b i li t y o f t h e d a ta ) . C o m p a r e d w i t h t h e n o r m a l
d i s t r i b u t io n , t h e n o n n o r m a l d i s t r i b u t i o n i s c h a r a c -
t e r i z e d b y a d d i t i o n a l n o i s e ( i n t h e f o r m o f n o n -
z e r o k u r t o s is ) t h a t m a k e s i t c o r r e s p o n d i n g l y m o r e
d i f fi c u lt t o i d e n t i f y t h e p r e s e n c e o f t h e s ig n a l. T h u s ,
a n y p a r t i c u l a r s i g n al i s e a s i e r t o d e t e c t g i v e n m u l t i -
v a r i a t e n o r m a l i t y t h a n i s t h e v e r y s a m e s i g na l g i v e n
t h e m u l t i v a r i a t e n o n n o r m a l d i s t r i b u t i o n s c o n s i d -
e r e d h e r e . T h e p o w e r o f t h e A D F a n d S B t e s t
s t a t i s t i c s ( a n d a n y t e s t s t a t i s t i c t h a t i n c o r p o r a t e s
i n f o r m a t i o n f r o m f o u r t h - o r d e r m o m e n t s ) t o d e t e c t
a g i v e n m i s s p e c i f i c a t i o n is t h u s d e c r e a s e d a s m u l t i -
v a r i a t e n o n n o r m a l i t y i n c r e a s es . 6 T h i s i n t e r p r e t a -
t i o n is o n l y sp e c u l a t i v e , a n d w e a r e c u r r e n t l y w o r k -
i n g o n d i s c e r n i n g p r e c i s e l y w h y t h i s lo s s o f p o w e r
u n d e r n o n n o r m a l i t y e x i s t s .
T h e r e a r e t w o i m p o r t a n t i m p l i c at i o ns o f t h e s e
f i n d in g s f o r t h e p r a c t i c i n g r e s e a r c h e r . F i r s t , t h e S B
X
w i ll a l m o s t a l w a y s b e s m a l l e r t h a n t h e M L X 2
u n d e r c o n d i t io n s o f m u l t i v a r i a te n o n n o r m a l i t y .
H o w e v e r , t h e l o w e r S B
X
d o e s n o t n e c e s s a r i l y
i m p l y t h a t t h e m o d e l i s a b e t t e r f i t t o t h e d a t a
b e c a u s e u n d e r n o n n o r m a l i t y t h e r e i s a s i m u l ta n e -
o u s d e c r e a s e i n t h e a b i l it y o f t h e S B X 2 t o d e t e c t
a m o d e l m i s s p e c i f ic a t i o n . T h e S B X 2 s s m a l l e r t h a n
t h e M L X 2 b e c a u s e o f t w o ( i n s e p a r a b l e ) r e a s o n s :
a c o r r e c t i o n f o r t h e i n f la t i o n t o t h e n o r m a l t h e o r y
M L X 2 a n d a d e c r e a s e i n s t a t i st i ca l p o w e r t o d e t e c t
a m i s s p e c i f i c a t i o n . T h e M L
X
a n d S B X 2 s h o u l d
t h u s b e i n t e r p r e t e d w i t h t h i s i n m i n d .
A s e c o n d i m p l i c a t i o n o f t h e s e f i n d i n g s i s t h a t i f
a r e s e a r c h e r i s p l a n n i n g a s t u d y t h a t w i l l n o t b e
c h a r a c t e r i z e d b y a m u l t i v a r i a t e n o r m a l d i s t r i b u -
t i o n , f u r t h e r s t e p s m u s t b e t a k e n t o c o m p e n s a t e
f o r t h e d e c r e a s e d s t a t i s t i c a l p o w e r t h a t r e s u l t s a s
a f u n c t i o n o f t h e n o n n o r m a l d a t a ( i .e ., p l a n t o
i n c l u d e a d d i t i o n a l s u b j e c t s i n t h e s t u d y ) . F o r e x -
a m p l e , t h e p o w e r e s t i m a t i o n m e t h o d s d e v e l o p e d
b y S a t o r r a a n d S a r i s ( 1 9 8 5 ) o n l y a p p l y t o n o r m a l
t h e o r y e s t i m a to r s . U s i n g th i s m e t h o d t o c o m p u t e
t h e r e q u i r e d s a m p l e s iz e n e e d e d t o a c h i e v e a g i v e n
l e v e l o f s t a ti s ti c a l p o w e r w i l l b e u n d e r e s t i m a t e d i f
t h e h y p o t h e s i z e d m o d e l w a s m i s s p e c i f i e d a n d
t e s t e d b a s e d o n d a t a t h a t d o n o t f o l l o w a m u l t i v a r i -
a t e n o r m a l d i s t r i b u t i o n .
R e c o m m e n d a t i o n s
O n t h e b a s is o f th e p r e v i o u s r e s u l ts , w e h a v e
s e v e r a l r e c o m m e n d a t i o n s f o r t h e p r a c t i c i n g r e -
s e a r c h e r . F i r s t, w e h a v e n o t i d e n t i f i e d a t w h a t p o i n t
t h e d a t a a p p r e c i a b l y d e v i a t e f r o m m u l t i v a r i a t e
n o r m a l i t y . S i m i l a r t o p r e v i o u s r e s e a r c h e r s ( e . g . ,
M u t h r n a n d K a p l a n , 1 9 8 5 , 1 9 9 2 ), w e f o u n d s ig -
n i f i ca n t p r o b l e m s a r i si n g w i t h u n i v a r i a t e s k e w n e s s
o f 2 .0 a n d k u r t o s e s o f 7 .0 . F u r t h e r r e s e a r c h i s
n e e d e d t o b e t t e r u n d e r s t a n d m o r e p r e c i se l y w h e n
n o n n o r m a l i t y b e c o m e s p r o b l e m a t i c , b u t i t s e e m s
c l e a r th a t o b t a i n e d u n i v a r i a te v a l u e s a p p r o a c h i n g
a t l e a s t 2 .0 a n d 7 . 0 f o r sk e w n e s s a n d k u r t o s e s a r e
s u s p e c t. S e c o n d , w e a g r e e w i t h p r e v i o u s r e s e a r c h -
e r s ( e . g ., H u e t a l ., 1 9 9 2 ; M u t h r n K a p l a n , 1 9 9 2 )
t h a t t h e A D F X 2 n o t b e u s e d w i t h s m a l l s a m p l e
s i z e s . A l t h o u g h w e f o u n d a d e q u a t e b e h a v i o r a t
s a m p l e s a s s m a l l a s N = 5 0 0 , o t h e r r e s e a r c h e r s
h a v e f o u n d p r o b l e m s w i t h t h e A D F X 2 a t s a m p l es
a s l a r g e a s N = 5 , 00 0 w h e n t e s t i n g m o r e c o m p l e x
m o d e l s ( H u e t a l. , 1 9 9 2) . T h e r e a r e s o m e e p i d e m i -
o l o g i c a l a n d c a t c h m e n t a r e a s t u d i e s t h a t d o h a v e
t h e s e l a r g e s a m p l e s i z e s a v a i l a b l e , a n d i n t h e s e
c a s es t h e A D F i s a p r o m i s in g m e t h o d o f e s t im a -
t i o n , p a r t i c u l a r l y f o r s m a l l e r m o d e l s . R e c e n t r e -
s e a r c h h a s a l s o s h o w n t h e p o s s i b i l i ty o f u s in g b o o t -
s t r ap p i n g t e c h n i q u e s to c o m p u t e m o r e s t a b l e A D F
5 Note tha t a l though the ob ta ined M L 2 values in-
c reased wi th inc reas ing nonnorm al i ty , the expec ted M L
X2 values we re e qual across dis tr ibut ion.
6 We thank b o th Albe r t S a tor ra and P e te r B ent le r ,
whom each independent ly sugges ted th i s s am e a rgu-
me nt as a potent ia l exp lanat ion for the ob tained resul ts .
-
7/25/2019 Curran,West&Finch (1996)
12/14
C H I - S Q U A R E T E S T S T A T I S T IC S 2 7
X 2 e s t i m a t e s ( Y u n g a n d B e n t l e r , 1 9 94 ); h o w e v e r ,
m o r e w o r k i s n e e d e d t o e x p l o r e t h e u t il it y o f th i s
a p p r o a c h i n a p p l i e d r e s e a r c h s e t t i n g s .
F i n a l l y , r e l a t i v e t o t h e M L X 2 a n d t h e A D F X 2,
t h e S B X 2 b e h a v e d e x t r e m e l y w e l l i n n e a r l y e v e r y
c o n d i t i o n a c r o s s s a m p l e si z e , d i s t r i b u t i o n , a n d
m o d e l s p e c i f i c a ti o n . A d d i t i o n a l l y , t h e S B X 2 h a d
t h e d e s i r a b l e p r o p e r t y o f s im p l i f y in g t o t h e M L X 2
u n d e r m u l t i v a r i a t e n o r m a l i t y . W e t h u s r e c o m -
m e n d r e p o r t i n g b o t h t h e M L X 2 a n d t h e S B X 2
w h e n n o n n o r m a l d a t a i s s u s p e c t e d w i t h th e c l e a r
r e a l iz a t i o n t h a t t h e l o w e r S B v a l u e m a y b e r e -
f le c t in g d e c r e a s e d p o w e r a n d n o t s i m p l y t h a t th e
m o d e l i s a b e t t e r f it t o t h e d a t a b a s e d o n t h e S B
X 2. M o d e l f i t s h o u l d t h u s b e e v a l u a t e d w i t h a p p r o -
p r i a t e c a u t io n . T h e r e a r e a fe w d i s a d v a n t a g e s t o
u s i n g t h e S B X 2 i n p r a c t ic e . O n e i s t h a t t h e c o m p u -
t a t i o n o f t h e S B X 2 r e q u i r e s r a w d a t a , w h i c h m i g h t
p o s e a p r o b l e m f o r s o m e r e s e a r c h e r s . S e c o n d , t h e
S B X 2 i s c u r r e n t l y o n l y a v a i l a b l e i n E Q S . T h i s
p o s e s a p r a c ti c a l p r o b l e m f o r r e s e a r c h e r s w h o a r e
e i t h e r n o t t r a i n e d i n o r d o n o t h a v e a c c e s s t o E Q S .
e f e r e n c e s
Bent ler , P . M. (1989) . E QS: S t ruc t ura l equa t i ons pro-
gram manual . L os Ange l es : BM DP S t a t i s t i ca l
S of t ware .
Bol len, K. A. (1989) . Structural equations with latent
variables. New York : W i l ey .
Browne , M . W . (1982) . Cov ar i ance s t ruc t u res . I n D . M .
Hawki ns (E d . ) , Topics in appl ied mul t ivariate analysis
(pp . 72-141) . Cambr i dge : Cambr i dge Uni ve r s i t y
Press .
Brow ne , M . W . (1984). A sym pt o t i c d i s tr i bu t ion f r ee
me t hods i n t he ana l ys i s o f cova r i ance s t ruc t u res . Brit-
i sh Jour nal o f Mathem at ical and Stat is tical Psychol -
ogy, 37, 127-141.
Brow ne, M. W., Mels , G. , Cow ard, M. (1994). Path
Ana l ys i s: RA M ON A. In L . W i l k i nson M . A . H i l l
(Eds. ) , Systat : Ad va nce d appl icat ions (pp. 163-224) .
E v a n s t o n , I L : S Y S T A T .
Cho u, C. P . , Ben t ler , P . M. , Sato rra , A. (1991) . Scaled
t e s t s ta t is t ic s and rob us t s t anda rd e r ro r s f o r non-no r -
ma l da t a i n cova r i ance s t ruc t u r e ana l ys i s : A M ont e
Car l o s t udy . B r i ti sh Journa l o f M at hemat i ca l and S ta -
t ist ical Psychology, 44, 347-357 .
F l e i shman , A . I . ( 1978). A me t hod fo r s i mul a t ing non-
norm al d i s t ri bu t ions . Psychometr ika, 43, 521-532 .
Hu , L . , Ben t ler , P . M. , Ka no, Y. (1992). Ca n tes t
s t a ti s ti c s i n cova r i ance s t ruc t u r e ana l ys is be t r us t ed?
Psycholog ical Bulle tin , 112, 351-362 .
J /Sreskog, K. G. , SSrbo m, D. (1993). L I S R E L - V I I I :
A guide to the prog ram and appl icat ions . M ooresv i l l e ,
IN: Scientific Software.
Kapl an , D . ( 1988). T h e i mpac t o f spec i f ica t ion e r ro r on
t he e s t i ma t i on , t e s ti ng , and i m prov em ent o f s tr uc t u r a l
equa t i on mode l s . Mult ivariate Behavioral Research,
23, 69-86 .
Kapl an , D . ( 1989). A s t udy o f t he sam pl i ng va r i ab i li t y o f
t he z -va l ues o f pa r am et e r e s t i ma t es f rom m i sspec if i ed
s t ruc t u r a l equa t i on mode l s . Multivariate Behavioral
Research, 24, 41-57 .
Law ley, D. N. , Maxw el l , A. E . (1971). Facto r analysis
as a statist ical method. L ondon: But t e rwor t h .
MacCal lum, R. (1986) . Speci f icat ion searches in covar i -
ance s t ruc t u r e mode l i ng . Psychological Bulletin,
100, 107-120.
M acCa l l um, R . ( 1990) . T he need fo r a l t e rna t i ve mea-
sures o f f i t i n cova r i ance s t ruc t u r e mode l i ng . Mult ivar-
iate Behavioral Research, 25, 157-162.
M acCa l l um, R . C . , Rozno wski , M . , Necowi t z , L . B .
(1990) . Model modif icat ions in covar iance s t ructure
ana l ys i s : T he p rob l em of cap i t a l i za t i on on chance .
Psycholog ical Bulle tin , 111, 490-504 .
Miccer i , T . (1989) . The unicorn, the normal curve, and
o t he r i mprobab l e c r ea t u r es . Psychological Bulletin,
105, 156-166.
Mu th6n, B. (1987) . L IS CO M P : A nal ys i s o f l inear s truc-
t ura l equa t i ons w i t h a comprehens i ve measurement
mode l . Mooresvi l le , IN: Scient i f ic Sof tware .
M ut h6n , B ., Kap l an , D . ( 1985). A compa r i son o f som e
met hodol og i es fo r t he f ac t o r ana l ys i s o f non-normal
L i ke r t va r i ab l e s . B r i ti sh Journa l o f Mat hema t i ca l and
Statist ical Psychology, 38, 171-189.
M ut h6n , B ., Kap l an , D . ( 1992). A com par i son o f som e
met hodol og i es fo r t he f ac t o r ana l ys i s o f non-normal
L i ke r t va r i ab le s : A n o t e on t he s ize o f t he mode l .
Bri t i sh Journal of Mathemat ical and Stat i s t ical Psy-
chology, 45, 19-30 .
Sar is , W. E . , S t ronk hors t , H. (1984). Causal model l ing
in nonexperimental research. A m s t e r d a m : S o c i o m e t -
r i c Resea rch F ounda t i on .
SAS Inst i tute , Inc . (1990) . SA S/ S TA T user 's guide: Vols .
1 and 2. Cary , NC: Aut hor .
Satorra , A. (1990) . Robustness i ssues in s t ructural equa-
t i on mode l i ng : A r ev i ew of r ecen t deve l opme nt s .
Qu ality Quantity, 24, 367-386 .
Satorra , A. (1991, June) . A sympt o t i c robus t i n f e rences
in the analys is o f mea n and covariance s t ructure . B a r -
ce l ona , S pa i n : Uni ve r s i t a t P ompeu F abra .
-
7/25/2019 Curran,West&Finch (1996)
13/14
2 8 C U R R A N , W E S T , A N D F I N C H
S a t o r r a , A . , B e n t l e r , P . M . ( 1 9 8 8 ). S c a l i n g c o r r e c t i o n s
f o r c h i - s q u a r e s t a t i s ti c s in c o v a r i a n c e s t r u c t u r e a n a l y -
sis . A S A Proceedings o f the Business and Econom ic
Section 3 0 8 - 3 1 3 .
S a t o r r a , A . , S a r i s, W . E . ( 1 98 5 ). T h e p o w e r o f t h e
l i k e l i h o o d r a t i o t e s t i n c o v a r i a n c e s t r u c t u r e a n a l y s i s .
Psychometrika 50 8 3 - 9 0 .
V a l e , C . D . , M a u r e l l i , V . A . (1 9 8 3 ) . S i m u l a t i n g m u l t i -
v a r i a t e n o n n o r m a l d i s t r i b u t i o n s . Psychometrika 48
4 6 5 - 4 7 1 .
Y u n g , Y . - F . , B e n t l e r , P . M . ( 1 99 4 ). B o o t s t r a p - c o r -
r e c t e d A D F t e s t s t a ti s t ic s i n c o v a r i a n c e s t r u c t u r e a n a l -
ysis . British Journal of Mathematical and Statistical
Psychology 47 6 3 - 8 4 .
A p p e n d i x
T e c h n i c a l D e t a i l s
D a t a G e n e r a t io n
E Q S g e n e r a t e s t h e r a w d a t a w i t h n o n - z e r o s k e w n e s s
a n d k u r t o s i s u si n g th e f o r m u l a e d e v e l o p e d b y F l e i s h m a n
( 1 97 8 ) i n a c c o r d a n c e w i t h t h e p r o c e d u r e s d e s c r i b e d b y
V a l e a n d M a u r e l l i (1 9 83 ) . T h e r a w d a t a w e r e g e n e r a t e d
b a s e d u p o n t h e c o v a r i a n c e m a t r i x i m p l i e d b y t h e m o d e l
p a r a m e t e r s f o r e a c h o f t h e t h r e e m o d e l s . T h e a v a i l a b i l i t y
o f th e r a w d a t a w a s n e c e s s a r y f o r th e c o m p u t a t i o n o f
t h e A D F a n d S B X2 tes t s t a t i s t i c s .
T h e s a m p l e s w e r e c r e a t e d u s i n g t h e m o d e l i m p l i e d
p o p u l a t i o n c o v a r i an c e m a t r i x ~ ( 0) . T h e m e a s u r e m e n t
e q u a t i o n s c o n s i s t e d o f th e p o p u l a t i o n p a r a m e t e r v a l u e s
t h a t d e f i n e d t h e p a r t i c u l a r m o d e l . E Q S g e n e r a t e d t h e
p o p u l a t i o n c o v a r i a n c e m a t r i x b a s e d o n t h e s e m e a s u r e -
m e n t e q u a t i o n s . T h e s a m p l e r a w d a t a w e r e c r e a t e d u s i n g
a r a n d o m n u m b e r g e n e r a t o r i n c o n j u n c t i o n w i t h t h e
c h a r a c t e r i s t ic s o f t h e p o p u l a t i o n c o v a r i a n c e m a t r i x . T h e
r a w d a t a w e r e g e n e r a t e d u n d e r t w o c o n s t r a i n t s : ( a ) t h e
e x p e c t e d v a l u e o f S s h o u l d e q u a l t h e p o p u l a t i o n c o v a r i -
a n c e m a t r i x ~ ( 0 ) , a n d ( b ) t h e e x p e c t e d v a l u e o f t h e
i n d i c e s o f s k e w n e s s a n d k u r t o s i s s h o u l d e q u a l t h e v a l u e s
s p e c i fi e d f o r e a c h m e a s u r e d v a r i a b l e .
V e r i f i c a ti o n o f D a t a G e n e r a t i o n
T o v e r i f y t h a t E Q S p r o p e r l y g e n e r a t e d t h e r a w d a t a
i n a c c o r d a n c e w i t h t h e d e s i r e d l e v e l s o f s k e w n e s s a n d
k u r t o s e s , t h r e e s e t s o f ra w d a t a o f s a m p l e s i z e N =
6 0 , 0 0 0 w e r e g e n e r a t e d . T h e t h r e e d a t a s e t s w e r e p r o -
d u c e d u s i n g t h e s a m e p r o c e d u r e s t h a t c r e a t e d t h e m u l t i -
v a r i a t e n o r m a l , m o d e r a t e l y n o n n o r m a l , a n d s e v e r e l y
n o n n o r m a l d i s t r i b u t i o n s f o r t h e s i m u l a t i o n s . T h e l a r g e
s a m p l e s i z e p r o v i d e s a m o r e a c c u r a t e e s t i m a t e o f th e
c o e f f ic i e n ts o f s k e w n e s s a n d k u r t o s i s f o r t h e g e n e r -
a t e d d a t a .
U n i v a r i a t e s k e w n e s s a n d k u r t o s e s w e r e c o m p u t e d f o r
t h e t h r e e s a m p l e s o f N = 6 0 ,0 0 0 u s i n g S A S P R O C
U N I V A R I A T E . F o r t h e n o r m a l l y d i s t ri b u t e d c o n d i t io n
( s k e w n e s s = 0 , k u r t o s i s = 0 ) , t h e m e a n u n i v a r i a t e sk e w -
n e s s f o r th e n i n e v a r i a b l e s w a s . 00 1 a n d t h e m e a n k u r t o -
s is w a s . 0 04 . F o r t h e m o d e r a t e l y n o n n o r m a l d i s t r i b u t i o n
( s k e w n e s s = 2 .0 , k u r t o s i s = 7 .0 ) , t h e m e a n s k e w n e s s
w a s 1 .97 3 a n d t h e m e a n k u r t o s i s w a s 6 .64 8. F i n a l l y , f o r
t h e s e v e r e l y n o n n o r m a l d i s t r i b u t i o n (s k e w n e s s = 3 .0 ,
k u r t o s i s =