1298 Agronomy Journa l • Volume105 , I s sue5 • 2013

Biometry, Modeling & Statistics

AnalyzingPartiallyNestedDesignswithIrregularNesting: ACautionaryCaseStudy

DawnM.VanLeeuwen,ZiliYou,andBerndLeinauer*

Published in Agron. J. 105:1298–1306 (2013)doi:10.2134/agronj2013.0039Available freely online through the author-supported open access option.Copyright © 2013 by the American Society of Agronomy, 5585 Guilford Road, Madison, WI 53711. All rights reserved. No part of this periodical may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying, recording, or any information storage and retrieval system, without permission in writing from the publisher.

A “partially nested design” (Sahai and Ageel, 2000) arises when a two-way design includes one factor

that can be represented as having a nested structure. Textbook presentations of partially nested designs have focused on the case with equal numbers of nested factor levels within each nesting factor level (Milliken and Johnson, 2008, p. 342; Kuehl, 2000, p. 251–254; Sahai and Ageel, 2000). When a nested factor is fixed, however, such as when cultivars are nested within species, it is more likely that there will be an irregular structure, with different numbers of nested factor levels within the different nesting factor levels.

Many researchers prefer to use software defaults and options rather than write contrast statements to obtain the desired tests. For a two-way factorial treatment structure, the effects model is often applied even though a cell means model might instead be used (Kuehl, 2000; Milliken and Johnson, 2008), with contrasts specified to test main effects and the interaction effect (Milliken and Johnson, 2008). When selecting an effects model, software such as SAS PROC GLM or PROC MIXED (SAS Institute, 2011) automatically produces these tests, while using the cell means model formulation would require the user to write contrast statements. Motivated by a desire to obtain

tests of interest through software defaults rather than having to write contrast statements, Piepho et al. (2006) illustrated obtaining analyses for complex treatment structures such as a 3 ´ 2 + 1 design by using models incorporating dummy vari-ables and both nested and crossed factors. They also noted that the choice between Type I and Type III hypotheses (or sums of squares [SS]) requires “special attention.” While the decision among hypothesis types continues to be controversial (Kutner, 1974; Speed et al., 1978; Aitkin, 1978; Herr, 1986; Shaw and Mitchell-Olds, 1993; Nelder, 1994; Aitkin, 1995; Rodriguez et al., 1995; Searle, 1995; Hector et al., 2010), a general con-sensus favoring the use of Type III hypotheses is reflected in the default options of many software packages for conducting ANOVA (Langsrud, 2003), including SAS PROC MIXED (SAS Institute, 2011).

While much of the debate on SS and hypothesis types has focused on factorial (crossed) treatment structures, for strictly nested treatment designs, Yandell (1997, p. 348) and Piepho et al. (2006) recommended the partial or adjusted Type III SS over Type I because Type I SS test hypotheses that reflect inequalities in cell frequencies. For the strictly nested design (Piepho et al., 2006; Yandell, 1997) and in crossed or factorial designs with no missing cells, Type III hypotheses correspond to contrasts among least squares means (LSMEANS) (Shaw and Mitchell-Olds, 1993; Westfall et al., 2011, p.78). Per-haps because of a perceived relationship between LSMEANS and Type III hypotheses, Yandell (1997, p. 373) tentatively recommended Type III SS for analyzing unbalanced designs that include fixed nested effects but also suggested combin-ing nested factors into one factor. The perception that Type III tests somehow correspond to LSMEANS comparisons is challenged, however, by the fact that in missing cells factorial

ABSTRACTModels with partially nested fixed effect structures arise when two-way structures include a factor that can be partitioned according to a nested structure. In such cases, it is likely that the nesting will have an irregular structure with unequal numbers of nested factor levels among nesting factor levels. If a priori hypotheses correspond to the nested structure, these might be tested using the two-way model and writing contrast statements. Alternatively, a more complex partially nested model might be used in an attempt to obtain the desired tests via model respecification. Comparing analyses based on the two-way model and on the partially nested model established that the partially nested model correctly partitions sums of squares for the nested structure but that Type III non-nested factor main effect hypotheses and sums of squares differed. Additionally non-nested factor least squares means differed between the two models, and the partially nested model Type III non-nested factor main effect hypothesis coefficients did not correspond to a comparison of the least squares means from either model. For the equal replications case, Type I hypotheses from the partially nested model produced the desired analysis but Type III hypotheses did not. For the unequal replications case, researchers might avoid writing contrast statements by running both models and selecting appropriate Type III tests and estimates from each analysis.

D.M. VanLeeuwen and Z. You, Dep. of Economics, Applied Statistics and International Business, New Mexico State Univ., Las Cruces, NM 88003; and B. Leinauer, Dep. of Extension Plant Sciences, New Mexico State Univ., Univ. Ave. MSC 3AE, P.O. Box 30001, Las Cruces, NM 88003. Received 25 Jan. 2013. *Corresponding author (leinauer@nmsu.edu).

Abbreviations: REML, restricted maximum likelihood; SS, sums of squares.

Published July 1, 2013

Agronomy Journa l • Volume105, Issue5 • 2013 1299

designs they are not equivalent; Type III hypotheses are com-puted even when LSMEANS are not estimable.

This study explores analyses for balanced crossed fixed effect structures where there is interest in partitioning variability due to one factor into components corresponding to an irregularly nested structure among that factor’s levels. Both the two-way and the partially nested models are considered and, for these models, Type I and Type III SS and LSMEANS are compared. For the non-nested factor main effect, using the partially nested model produces Type III tests that do not correspond to comparing LSMEANS. Furthermore, the Type III non-nested factor main effect test from the partially nested model is not equivalent to the test using the two-way fixed effect structure model. Additionally, the two models produce different non-nested factor main effect LSMEANS. These phenomena are explored, and an appropriate analysis scheme for the partially nested design is identified.

THEORYWe considered a two-way linear model with factors A and C

with fixed portion specified as

( ) ( )= ¼

=m+a +q + a

= ¼ = ¼

q

1,2, , ; 1,2, , c; 1, 2, ,itl i t it

i a t lE y

r [1]

where E(yitl) represents the expectation of yitl, ai is the main effect due to factor A level i, qt is the main effect due to factor C level t, and (aq)it is the A ´ C interaction effect. A complete model requires also defining the variance structure; however, specifying only the fixed portion of a model allows consider-ation of fixed effect hypothesis formation for a broad class of mixed models including the usual fixed model for two-way factorial experiments, split-plot designs, or repeated measures designs with a treatment factor and a repeated factor. While these models may involve a single error term, multiple random effects, or correlations among repeated measures from the same subject, respectively, these model elements contribute to the variance structure and affect test statistics, estimates, and stan-dard errors but do not affect fixed effect hypotheses.

If levels of factor C are grouped or nested according to an additional third factor B, the fixed portion of a model explicitly recognizing this nesting and using the factors B and C(B) can be specified as

( ) ( ) ( )=m+a +b + ab +g

= ¼ == = ¼

ag

¼¼

+( ) ( )

1,2, , ; 1,2, , ; 1,2, , ; 1 ,2, ,

ijkl i j k jij ik j

j

i a j bk m b l

E y

r [2]

where ai is the same as in Model 1 (Eq. [1]), bj is the fixed main effect due to factor B level j, (ab)ij is the fixed effect for the A ´ B interaction, gk( j) is the fixed effect for C(B) for the factor C level k within B level j, and (ag)ik( j) is the fixed effect for A ´ C(B).

The second model is obtained by partitioning the parameters for C into B and C(B) parameters. Thus, Model 2 (Eq. [2]) is a reparameterization of Model 1 (Eq. [1]), where

( ) ( ) ( )

-=

q =b +g aq = ab + ag

= +å

( ) ( )

11

,

for

t j k j it ij ik j

j

hh

t k m

with m0 = 0 defined.For Model 2 (Eq. [2]), the Type III factor A main effect hypoth-

esis produced using SAS software is the following (a – 1) compari-sons corresponding to an (a – 1) degree of freedom contrast:

( ) ( ) ( )

( ) ( )

( )

=

= =

é ùa -a + ab - ab +ê úë û

é ùag - ag =ê úë û

= -

å

åå

01

( ) ( )1 1

H :

0

for all 1, ..., 1

j

b

i a j ij ajj

mbj

ik j ak jj k j

K

Km

i a

where Kb = 1 – -=S 1

1bj jK and K1, …, Kb–1 are the solution to

the following system of equations:

- -

é ù é ù é ùê ú ê ú ê úê ú ê ú ê úê ú ê ú ê úê ú ê ú ê ú=ê ú ê ú ê úê ú ê ú ê úê ú ê ú ê úê ú ê ú ê úê ú ê ú ê úë û ë û ë û

L

M O M M ML M M

1 1

2 2

1 1

b b b

b b b

b b b

b b b b b

c c c K cc c c K c

c c cc c c K c

[3]

where

2 24

4 2 2, 1, , 1

jj b

j b b j

j b

cm m

m m m mj b

m m

æ ö÷ç ÷ç= + + ÷ç ÷ç ÷è ø+ +

= = ¼ -

= +

+=

22

2 2

bb

b

b

cm

mm

SAS Institute (2011, Ch. 15) provides a three-step algorithm for computing Type III hypotheses. This algorithm produces the usual Type III hypotheses for Model 1 (Eq. [1]) effects. For Model 2 (Eq.[ 2]), the derivation of the Eq. [3] characterization of the Type III factor A main effect hypothesis for the special case with a = 2, b = 3, m1 = 4, m2 = 3, and m3 = 2 is presented. The algorithm uses the general form of estimable functions

1300 Agronomy Journa l • Volume105, Issue5 • 2013

and, based on these, uses the following steps when forming the A main effect test:

1. Set coefficients for effects that do not contain A to zero.

2. Simplify the A contrast by replacing complicated or com-pound expressions in the A parameter block with new symbols.

3. In the A main effect contrast, replace coefficients in pa-rameter blocks corresponding to interaction effects con-taining A with linear functions of the coefficients appear-ing in the A parameter block. Solve by finding values for unknowns in the linear functions that make the A main effect hypothesis orthogonal to hypotheses for interaction effects containing A.

The E option in PROC MIXED produces the Type I, II, and III hypothesis coefficients but not the general form of the esti-mable functions. Therefore, PROC GLM was used to obtain the general form of estimable functions for the fixed portion of the model (Table 1). The A effect was contained in the A ´ B and A ´ C(B) effects but was not contained in the intercept, B, or C(B) effects. Consequently, when computing the A main effect hypothesis, Step 1 of the algorithm required setting coef-ficients in the general form of the estimable functions (Table 1) for m , bj, and gk( j) to zero (i.e., set L1 = L4 = L5 = L13 = L14 = L15 = L17 = L18 = L20 = 0; see Table 1). Step 2 required creating new symbols to replace compound expressions that appeared in the A main effect block, but only L2 and –L2 appear in this block and there were no compound expressions in the A main effect block. Consequently, L2 was retained and Step 3 was applied to the column produced at the end of Step 1.

Step 3 required that L7, L9, L22, L24, L26, L30, L32, and L36 be replaced with linear functions of L2. At Step 3, therefore, L7 was replaced with K1L2 and L9 with K2L2. Because L22, L24, L26, and L7 – L22 – L24 – L26 must sum to K1L2 and because the m1 = 4 C factor levels within B factor level 1 were interchangeable, these four terms were replaced by 1/4 K1L2, or more generally by (1/m1)K1L2. Similarly, substitutions were made using (1/m2)K2L2 for L30, L32, and L9 – L30 – L32 and (1/m3)(1 – K1 – K2)L2 for L36 and L2 – L7 – L9 – L36 (see the Step 3 column of Table 1). Step 3 required solving for K1 and K2 to produce an A main effect hypothesis contrast that was orthogonal to the A ´ B and A ´ C(B) contrasts formed ear-lier in the algorithm (see Table 1).

Requiring the A main effect hypothesis to be orthogonal to the A ´ C(B) tests placed no constraints on K1 or K2. As an example, setting L2 = 1 in the contrast established for the A main effect and setting L22 = 1 in the first A ´ C(B) contrast, then multiplying the transpose of the A contrast coefficient vector by this first A ´ C(B) contrast coefficient vector yielded

( ) ( )

( ) ( )

-+ -

-+ - + =

1 11 1

1 11 1

1 11 1

1 11 01

K Km m

K Km m

A similar relationship exists between the A main effect contrast coefficients and each one of the other five A ´ C(B)

Agronomy Journa l • Volume105, Issue5 • 2013 1301

1302 Agronomy Journa l • Volume105, Issue5 • 2013

contrasts. Consequently, any factor A main effect hypothesis contrast vector having the form produced at the end of Step 2 will be orthogonal to the A ´ C(B) hypothesis contrasts.

The defining constraints derive from the requirement that the A main effect hypothesis be orthogonal to the A ´ B contrasts. Using the first A ´ B contrast (defined by L7), the following equation was derived. The initial representation sepa-rates portions of the equation arising from the A ´ B and the A ´ C(B) parameter blocks:

( )( ) ( )

( )( )

æ öæ ö ÷÷ çç ÷÷ çç ÷÷ çç ÷ ÷

+ - - -

- - - ´

+ -

- -

é ù=

ç çè ø è

´

´ û

ø

ë

1 1 1 2

1 2

22

1 1 31 3

1 2

1

1 A B

1 1 2 2

1

0 A C B

K K K KK K

m K mm m

K K

becomes

( )

( )

+ - ´

é ù+ + + - = ´ë û

1 2

1 21 3 3 3

4 2 2 A B2 2 2 2

0 A C B

K K

K Km m m m

Combining the portions deriving from the A ´ B and the A ´ C(B) parameter blocks produces

æ ö æ ö æ ö÷ ÷ ÷ç ç ç÷ ÷ ÷+ + + + - + =ç ç ç÷ ÷ ÷ç ç ç÷ ÷ ÷ç ç çè ø è ø è ø1 2

1 3 3 3

2 2 2 24 2 2 0K K

m m m m

And we see that the coefficient for K1 is equal to c1from Eq. [3] and the coefficient for K2 is equal to c3 = cb in Eq. [3]. Similarly, using the second A ´ B contrast (defined by L9) produced

æ ö æ ö æ ö÷ ÷ ÷ç ç ç÷ ÷ ÷+ + + + - + =ç ç ç÷ ÷ ÷ç ç ç÷ ÷ ÷ç ç çè ø è ø è ø1 2

3 2 3 3

2 2 2 22 4 2 0K K

m m m m

which matches the general form given in Eq. [3].Adding (2 + 2/m3) to both sides of both equations and

expressing the system of equations using matrices and vectors produces Eq. [3] with b = 3.

MATERiAlS And METHOdSThese data use a subset of data documented by Sevostianova

et al. (2011). For this reduced data set, researchers wanted to know the effect of irrigation (A), species (B), cultivar within species [C(B)], and their interactions on turfgrass quality. A split-plot design was used with the whole-plot factor A com-pletely randomized. Two irrigation methods were tested, with

three replications (whole plots) for each method. Cultivar (C) was the split-plot factor, with nine levels. These nine cultivars were grouped into four species: bermudagrass [Cynodon dac-tylon (L.) Pers.] (four cultivars), seashore paspalum (Paspalum vaginatum Sw.) (two cultivars), inland saltgrass [Distichlis spicata (L.) Greene] (two cultivars), and zoysiagrass (Zoysia japonica Steud.) (one cultivar).

The model corresponding to a balanced split-plot design might be used to analyze these data.

Model 1:

( )=m+a +q

= =

+ +

=

+ aq e

1,2,..., ; 1,2, ..., ; 1,2, ...,itl i t il itlit

id

c ly

a t r [4]

This model has fixed effect parameters as defined for Model 1 where ai is the fixed whole-plot factor effect while qt is the fixed subplot factor main effect. The model includes the whole-plot random error dil (equal to replication within irrigation [R(A)]), with dil ~ independent and identically distributed (i.i.d.)N(0,sd

2), and independent subplot random errors eitl ~ i.i.d. N(0,s2).

Using a model that explicitly recognizes B and C(B) pro-duces the following model based on Eq. [4]:

Model 2:

( ) ( )

( ) ( )

1,2, ..., ; 1,2, ..., ;1,2, ..., ; 1,2, ...,

ijkl i j k jij

il ijkl

j

ik j

i a j bl

y

d

k m r

=m+a +b + ab +g

+ a

= =

g + +e

= =

[5]

In our case, a = 2, b = 4, c = 9, r = 3, m1 = 4, m2 = 2, m3 = 2, and m4 = 1.

Preplanned hypotheses corresponded to assessing whether the effects involving cultivar were due to species alone or were related to differences among cultivars within species. In the first model, these hypotheses must be tested using contrast statements, while the second model represents an attempt to obtain the desired tests as tests of effects incorporated into the model.

Statistical Analyses Compared

Sums of squares from three analyses conducted using SAS version 9.3 PROC MIXED software (SAS Institute, 2011) were compared. All three analyses incorporated a random effect for the whole-plot error.

Analysis 1Type III SS for the balanced split-plot Model 1 (Eq. [4])

using contrasts to partition the SS for cultivars (8 df) into SS for species (3 df) and SS for cultivar within species (5 df) components were obtained. For this balanced and equal repli-cations design, Type I SS are equal to Type III SS so only the Type III are reported. In the code below, Irr corresponds to

Agronomy Journa l • Volume105, Issue5 • 2013 1303

factor A (irrigation), Cult corresponds to factor C (cultivar), and Rep(Irr) uniquely identifies whole-plot units.

PROC MIXED METHOD=TYPE3;CLASS Rep Cult Irr;MODEL Quality = Irr|Cult;RANDOM REP(IRR);

CONTRAST ‘Species main’ Cult 1 1 1 1 0 0 0 0 –4, Cult 0 0 0 0 2 2 –2 –2 0, C ult 0.8 0.8 0.8 0.8 –1 –1

–1 –1 0.8;

CONTRAST ‘Cultivar within species’ Cult 1 –1 0 0 0 0 0 0 0,

Cult 1 1 –2 0 0 0 0 0 0, Cult 1 1 1 –3 0 0 0 0 0, Cult 0 0 0 0 0 0 1 –1 0, Cult 0 0 0 0 1 –1 0 0 0 ;

CONTRAST ‘Species*Irr’Irr*Cult 1 –1 1 –1 1 –1 1 –1 0 0 0 0 0 0 0 0 –4 4,Irr*Cult 0 0 0 0 0 0 0 0 2 –2 2 –2 – 2 2 –2 2 0 0,Irr*Cult 0 .8 –0.8 0.8 –0.8 0.8 –0.8 0.8 –0.8 –1 1

–1 1 –1 1 –1 1 0.8 –0.8;

CONTRAST ‘Irr*(Cultivar within species)’Irr*Cult 1 –1 –1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0,Irr*Cult 1 –1 1 –1 –2 2 0 0 0 0 0 0 0 0 0 0 0 0,Irr*Cult 1 –1 1 –1 1 –1 –3 3 0 0 0 0 0 0 0 0 0 0,Irr*Cult 0 0 0 0 0 0 0 0 0 0 0 0 1 –1 –1 1 0 0,Irr*Cult 0 0 0 0 0 0 0 0 1 –1 –1 1 0 0 0 0 0 0;

RUN; QUIT;

Analysis 2Type I SS for Model 2 with the B and C(B) effects (Eq. [5])

incorporated into the model to test species and cultivar within species components were obtained. Both the METHOD = TYPE1 option on the call to PROC MIXED and the HTYPE = 1 option on the MODEL statement were used. To obtain contrast coefficients for the Type I hypothesis test, the E1 option on the MODEL statement was used.

PROC MIXED METHOD=TYPE1;CLASS Rep Species Cultivar Irr;MODEL Quality = I rr|Species|Cultivar(Species)/E1

HTYPE = 1;RANDOM REP(IRR);RUN; QUIT;

Analysis 3Type III SS for Model 2 (Eq. [5]) with the B and C(B) effects

incorporated into the model used to test species and cultivar within species components were obtained. For this analysis, METHOD = TYPE3 and HTYPE = 3 and E3 were speci-fied. In addition, LSMEANS (least squares means) for both Models 2 and 3 (Eq. [4] and [5]) were obtained and compared.

Coefficients used in computing LSMEANS can be obtained using an E option on the LSMEANS statement.

RESUlTSAll three analyses produced SS for B and C(B) that summed to

the Model 1 C main effect SS (Table 2). Similarly, SS for A ´ B and A ´ C(B) summed to the Model 1 A ´ C SS (Eq. [4]). Thus the contrasts used with the balanced split-plot model to partition SS for C into B and C(B) components were statistically orthogo-nal. This was expected because the data had equal replications.

The only discrepancy among the three SS sets was that the Model 2 Type III SS (Eq. [5]) for the A main effect did not match either the original balanced split-plot Model 1 (Eq. [4]) Type III (or Type I) SS or the Model 2 (Eq. [5]) Type I SS (Table 2). Thus while the Type III SS for Model 2 (Eq. [5]) correctly partitioned both the C and A ´ C SS from Model 1 (Eq. [4]) into compo-nents corresponding to B and C(B), the Model 2 Type III SS failed to produce the expected SS for the A main effect.

Irrigation LSMEANS for Models 1 and 2 (Eq. [4] and [5]) differed. For the first model (Eq. [4]), the irrigation LSMEANS were 6.11 and 5.96 for Levels 1 and 2, respectively. These LSMEANS reflected equal cultivar weightings. For Model 2 (Eq. [5]), however, the LSMEANS were 5.98 and 6.04 and reflected equal species weightings (and so cultivar weight-ings ranging from 0.0625–0.25).

For Model 1 (Eq. [4]), the Type III factor A main effect test and for Model 2 (Eq. [5]) the Type I factor A main effect test gave equal weight to each cultivar, while the Model 2 (Eq. [5]) Type III test with species weights K1 = 24/79, K2 = K3 = 20/79, 1 – K1 – K2 – K3 = 15/79 did not give equal weight to either species or cultivars (Table 3). In the Type III test, species with higher numbers of cultivars received more weight than species with fewer cultivars, but unlike the Type I test, that spe-cies weighting was not proportional to the number of cultivars within the species. The Type III weighting for an individual cultivar ranged from 6/79 (= 0.076) to 15/79 (= 0.190) vs. the Type I equal weighting of 1/9 (= 0.111).

The Type III coefficients are the solution to the following equations formed using Eq. [3]:

Table 2. Results of sums of squares (SS) analyses 1 to 3. italicized data denote that the df and SS were calculated using contrasts.

Source

Model 1 Type iii†

Model 2 Type i

Model 2 Type iii

df SS df SS df SSFixedeffects A (irrigation) 1 0.2963 1 0.2963 1 0.0092 C(cultivar) 8 54.4259 – – – – B(species) 3 49.7384 3 49.7384 3 49.7384 C(B) 5 4.6875 5 4.6875 5 4.6875

A ´ C 8 4.3704 – – – –

A ´B 3 3.7662 3 3.7662 3 3.7662

A ´C(B) 5 0.6042 5 0.6042 5 0.6042Randomeffects Replicate(A) 4 2.2407 4 2.2407 4 2.2407 Residual 32 11.5926 32 11.5926 32 11.5926

†Model1dfandSSobtainedusingthebasicbalancedsplit-plotModel1.

1304 Agronomy Journa l • Volume105, Issue5 • 2013

é ù é ù é ùê ú ê ú ê úê ú ê ú ê ú=ê ú ê ú ê úê ú ê ú ê úê ú ê ú ê úë û ë û ë û

1

2

3

6.5 4 4 44 7 4 44 4 7 4

KKK

The equations have contributions from coefficients on both A ´ B and A ´ C(B) parameter blocks. Using only contribu-tions from the A´B parameter block produced the following system of equations:

é ù é ù é ùê ú ê ú ê úê ú ê ú ê ú=ê ú ê ú ê úê ú ê ú ê úê ú ê ú ê úë û ë û ë û

1

2

3

4 2 2 22 4 2 22 2 4 2

KKK

The solution to these equations weights species equally (i.e., K1 = K2 = K3 = 1 – K1 – K2 – K3 = 1/4). Dividing these coef-ficients evenly among the cultivar levels within a species would produce a hypothesis that compares the Model 2 (Eq. [5]) irrigation LSMEANS but that does not match any hypothesis tested using either Model 1 or 2 (Eq. [4] or [5]).

If only the portion of the equations corresponding to the A ´ C(B) parameter block were used to solve for K1, K2, and K3, then the system of equations becomes

é ù é ù é ùê ú ê ú ê úê ú ê ú ê ú=ê ú ê ú ê úê ú ê ú ê úê ú ê ú ê úë û ë û ë û

1

2

3

2.5 2 2 22 3 2 22 2 3 2

KKK

and coefficients weighting cultivars equally are derived (K1 = 4/9, K2 = 2/9, K3 = 2/9, and 1 – K1 – K2 – K3 = 1/9). A hypothesis based on this set of coefficients is equivalent to the original Model 1 (Eq. [4]) Type III and Type I and Model 2 (Eq. [5]) Type I A main effect hypotheses.

In light of this, one might expect that the Model 2 (Eq. [5]) irrigation main effect Type III coefficient values would fall between the coefficients corresponding to equal species and

equal cultivar weightings. For the first species, this relationship holds because the Type III coefficient fell between the equal species and equal cultivar weightings (1/4 < 24/79 < 4/9). A similar relationship but with the signs reversed held for the fourth species (1/4 > 15/79 > 1/9). For the second and third species, however, the Type III coefficient was higher than either the equal species or equal cultivar weightings (20/79 > 1/4 and 20/79 > 2/9). Consequently, the Type III irrigation main effect hypothesis cannot be interpreted as corresponding to a hypothesis that somehow balances or interpolates between the two weighting schemes.

diSCUSSiOnFor factorial designs with no missing cells, Type III hypoth-

eses have been advocated because they test logical hypotheses that reflect cell means or model parameters only and are invariant to the order in which the model effects are listed, while Type I hypotheses are biased by the cell frequencies and are order dependent (Goodnight, 1976; Speed et al., 1978; Pendleton et al., 1986; Maxwell and Delaney, 1990, p. 289; Rodriguez et al., 1995; Quinn and Keough, 2002, p. 243). In the case of missing cells, however, the Type III hypotheses may not be interpretable (Pendleton et al., 1986; Yandell, 1997, p. 179; Milliken and Johnson, 2008, p. 258), and analyses using contrasts tailored to preplanned hypotheses under a cell means model have been advocated widely (Urquhart and Weeks, 1978; Freund, 1980; Milliken and Johnson, 2008). Our find-ings demonstrate that respecifying a crossed fixed effect model structure by replacing one factor with nested and nesting fac-tors corresponding to an irregularly nested structure causes counterintuitive changes in Type III hypotheses for model factors that do not involve either the nested or nesting factors. Thus, while Type III hypotheses have become standard in many applications, caution should be exercised when analyzing par-tially nested designs with irregular nesting or any design that has unusual or irregular features and is not well studied.

For the equal replications data considered here, variance component estimates were the same whether analyses were based on Type I or Type III methods of moments estimation or on restricted maximum likelihood (REML). The SAS PROC MIXED software defaults specify REML estimation and Type III fixed effect hypotheses. For the current case study, the REML-based analysis using the recommended Kenward–Rog-ers method of computing denominator degrees of freedom (ddfm = kr) (Littell et al., 2006) produced the same test statistics and p values as the Type III analysis. Reporting SS allowed easy assessment of whether the Type III analysis for Model 2 (Eq. [5]) partitioned the Model 1 (Eq. [4]) cultivar SS into orthogonal components for species and cultivar within species. For designs with unequal replications, SS corresponding to correct tests will not produce orthogonal partitions, and REML estimation with the Kenward–Rogers denominator df method should be used.

This example illustrates that when using a design with irregu-lar features, one might be able to recognize likely problems by identifying effects that contain the tested effect and associated parameter blocks that will be involved in Type III hypothesis for-mation. If these effects or parameter blocks considered separately would lead to different hypotheses, then it is likely that the Type III hypothesis will not make sense. Both models (Eq. [4] and [5])

Table 3. Coefficients of the hypothesis test for the irrigation main effect.

Effect Type i Type iiiai (–1)i–1 (–1)i–1

(ab)i1 (–1)i–1(4/9) (–1)i–1(24/79)(ab)i2 (–1)i–1(2/9) (–1)i–1(20/79)(ab)i3 (–1)i–1(2/9) (–1)i–1(20/79)(ab)i4 (–1)i–1(1/9) (–1)i–1(15/79)

=S41k (ag)ik(1) (–1)i–1(1/9) (–1)i–1(6/79)

=S21k (ag)ik(2) (–1)i–1(1/9) (–1)i–1(10/79)

=S21k (ag)ik(3) (–1)i–1(1/9) (–1)i–1(10/79)

(ag)ik(4) (–1)i–1(1/9) (–1)i–1(15/79)

Agronomy Journa l • Volume105, Issue5 • 2013 1305

are overparameterized models that, for our example case, used 30 and 42 parameters to represent just 18 means. The higher level of overparameterization in Model 2 (Eq. [5]) contributed to computing counterintuitive Type III hypotheses; combining cross-products for parameter blocks linked to both the A ´ B and A ´ C(B) effects when setting up the equations for com-puting Type III coefficients produced less intuitive hypotheses than using equations based on either parameter block alone. The Model 1 Type III irrigation main effect hypothesis weighted cul-tivars equally and corresponded to the test that would have been derived using Model 2 if one used only those portions of the equation associated with the A ´ C(B) effects.

For the equal replications case considered here, the Model 2 (Eq. [5]) Type I factor A main effect test, which is equivalent to the Model 1 (Eq. [4]) Type III factor A main effect test, is optimal (Siefert, 1979; Mathew and Sinha, 1988). This test is an exact test with the expected four denominator dfs and uses the R(A) mean square (i.e., the whole-plot error term) in the denomi-nator (Kuehl, 2000; Steel and Torrie, 1980). The Model 2 (Eq. [5]) Type III factor A main effect test, however, is an approxi-mate test, with the denominator using a linear combination of the whole-plot and residual error terms (0.9017 MS[R(A)] + 0.0983 MS[Residual]), and error df = 4.58 calculated using Sat-terthwaite’s (1946) approximation. In addition, even if the con-vention of interpreting main effect tests only if interactions are not significant is used, the Model 2 (Eq. [5]) factor A main effect Type III test may be biased by undetected interaction effects.

The partially nested design explored here provides illu-mination on the relationship between Type III tests and LSMEANS and might be used to dispel misconceptions that may have arisen based on the behavior observed in factorial and purely nested designs. First, Type III tests are not based on LSMEANS even though they may appear to be in crossed designs with no missing cells (Shaw and Mitchell-Olds, 1993; Westfall et al., 2011, p.78) and purely nested designs (Yandell, 1997; Piepho et al., 2006). In Model 2, the LSMEANS for irrigation are obtained by giving equal weight to each species. Neither the Type I nor the Type III irrigation main effect hypotheses correspond to comparing the Model 2 (Eq. [5]) irrigation LSMEANS. On the other hand in Model 1 (Eq. [4]), the LSMEANS for irrigation weight each cultivar equally, and both the Model 1 Type III and the Model 2 Type I tests compare the two Model 1 LSMEANS. For the equal replica-tions case considered here, the equal cultivar weightings of the Model 1 irrigation LSMEANS produces estimates with lower variance than the Model 2 LSMEANS estimates.

For factorial treatment structures, non-estimability of any LSMEAN is a symptom of missing cells and should serve as a red flag that Type III tests should not be used. For the partially nested model, however, all LSMEANS are estimable. There-fore, the current example dispels another misconception and illustrates that Type III hypotheses may be problematic even when all LSMEANS are estimable. Consequently, relying on non-estimability messages is not an adequate screen to ensure that the Type III hypotheses are appropriate. Software compa-nies could assist data analysts by including warning messages whenever Type III hypothesis coefficients do not correspond to comparing LSMEANS. Such a warning would apply equally to the case where there are missing cells in a factorial treatment

structure and the current partially nested design. For the par-tially nested design, however, LSMEANS do exist and could be the basis for a test that might be used as an alternative to the Type III tests. As occurred here, however, that may not be the optimal test, so that in cases where Type III tests do not correspond to comparing LSMEANS, modifying the hypoth-esis-generating algorithm to use individual parameter blocks, particularly the highest order parameter block, rather than combining coefficients across parameter blocks might identify desirable alternative tests.

Another interesting phenomenon is that the statement

contrast ‘Irrigation main effect’ Irr 1 -1;

calculates the correct test in Model 1 (Eq. [4]) but does not pro-duce any test in Model 2 (Eq. [5]). For Model 2, the SAS conven-tion of distributing the coefficients evenly among the contained effects (SAS Institute, 2011) does not produce an estimable func-tion because cultivars are not distributed equally among species.

The partially and irregularly nested model provides an additional caution regarding the use of software designed for fixed effects models, such as PROC GLM, when analyzing mixed models. When analyzing mixed models using software designed for fixed effect models, the typical concerns are that standard error estimates may be incorrect and fixed effect estimates may be inefficient; however, random effects are often nested and if there is any irregularity in the design, including random effects as fixed effects can influence Type III hypoth-esis formation in unintended and undesirable ways. Dallal (1992) observed this phenomenon when he compared Type III SS obtained using PROC GLM for two equivalent models with proportional but unequal cell frequencies. The first was the usual 2 ´ 2 factorial model, while the second specified the residual term as a nested effect in the model. While both models are essentially the same, the SAS Type III SS differed. Several statisticians responded to Dallal’s study (DeLong, 1994; Fleury, 1994; Kutner, 1994; Lam, 1994; Lazaro, 1994; Rogers, 1994; Searle,1994), and DeLong (1994, p. 141) cor-rectly observed that the discrepancy was due to the second model requiring the B contrast to be orthogonal to the A ´ B and C(A ´ B) contrasts while the first model did not specify C(A ´ B) as a model effect and so did not require the B con-trast to be orthogonal to the C(A ´ B) contrasts. In addition, the second model’s A ´ B contrast involved both the A ´ B and C(A ´ B) parameter blocks, and, as occurred in the cur-rent example, this may have contributed to the discrepancy.

For a partially nested design with equal numbers of nested fac-tor levels within each nesting factor level (i.e., with regular nest-ing), the Type III analysis for the partially nested model would not present any of the difficulties observed for the irregularly nested case. The irrigation LSMEANS equal weighting for each species would translate into equal weighting for all cultivars, and when deriving the Type III hypothesis coefficients, portions of the equation deriving from the A ´ B and A ´ C(B) interac-tions would result in coefficients weighting both species and cul-tivars equally. Combining both of these portions and solving the resulting equation would produce a Type III factor A main effect hypothesis weighting both cultivars and species equally.

1306 Agronomy Journa l • Volume105, Issue5 • 2013

To avoid writing contrast statements, Piepho et al. (2006) suggested running multiple models to obtain the desired quantities. For the current case study data, one might be satis-fied with the Model 2 (Eq. [5]) Type I tests but prefer to use a combination of Model 1 (Eq. [4]) and Model 2 LSMEANS. If the design has unequal cell frequencies, however, the Type I hypotheses might change. An analysis robust to unequal cell frequencies would require using REML estimation (the PROC MIXED default) with Kenward–Rogers denominator degrees of freedom specified (the ddfm = kr model statement option) and Type III hypotheses from both Models 1 and 2 (Eq. [4] and [5]). Model 1 Type III tests would produce the correct fac-tor A main effect test and LSMEANS, and Model 2 Type III tests would be used to obtain tests and LSMEANS for effects containing the B and C(B) factors. The alternative would be to use the suggestion of Yandell (1997) to combine nested factors into a single factor. In our example case, that translates into using Model 1 and writing contrasts to test hypotheses involv-ing effects containing the B or C(B) factors and estimate state-ments to obtain species means.

Researchers should be cautious when choosing to increase model complexity in an effort to obtain tests through model respecification instead of using a simpler model and writing contrast statements. A reformulated model may produce unin-tended changes in tests and LSMEANS for factors with speci-fications that appear to be unchanged. Consequently, when respecifying a model, one should take care to look for changes to tests and LSMEANS even among factors whose specifica-tion did not change.

ACknOwlEdgMEnTS

Financial support of the study was provided by New Mexico State University’s Agricultural Experiment Station and by the Office for Facilities and Services.

REFEREnCESAitkin, M. 1978. The analysis of unbalanced cross-classifications. J. R. Stat. Soc.

Ser. A 141:195–223.Aitkin, M. 1995. Comments on J.A. Nelder: ‘The statistics of linear models:

Back to basics’. Stat. Comput. 5:85–86. doi:10.1007/BF00143934Dallal, G.E. 1992. The computer analysis of factorial experiments with nested

factors. Am. Stat. 46:240.DeLong, D. 1994. Invited response to Dallal, G.E. (1992), ‘The computer

analysis of factorial experiments with nested factors’. Am. Stat. 48:141. doi:10.2307/2684270

Fleury, P.J. 1994. Invited response to Dallal, G.E. (1992), ‘The computer analy-sis of factorial experiments with nested factors’. Am. Stat. 48:145–147. doi:10.2307/2684273

Freund, R.J. 1980. The case of the missing cell. Am. Stat. 34:94–98. doi:10.2307/2684112

Goodnight, J.H. 1976. General linear models procedure. In: J. Horwich and E.R. Horwich, editors, SAS.ONE, Proceedings of the First International SAS User’s Conference, Kissimee, FL. 26–28 Jan. 1976. SAS Inst., Cary, NC. p. 1–39.

Hector, A., S. von Felten, and B. Schmid. 2010. Analysis of variance with unbal-anced data: An update for ecology and evolution. J. Anim. Ecol. 79:308–316. doi:10.1111/j.1365-2656.2009.01634.x

Herr, D.G. 1986. On the history of ANOVA in unbalanced, factorial designs: The first 30 years. Am. Stat. 40:265–270.

Kuehl, R.O. 2000. Design of experiments: Statistical principles of research design and analysis. 2nd ed. Brooks/Cole Publ., Belmont, CA.

Kutner, M.H. 1974. Hypothesis testing in linear models (Eisenhart Model I). Am. Stat. 28:98–100.

Kutner, M.H. 1994. Invited response to Dallal, G.E. (1992), ‘The computer analysis of factorial experiments with nested factors’. Am. Stat. 48:147. doi:10.2307/2684274

Lam, M. 1994. Invited response to Dallal, G.E. (1992), ‘The computer analy-sis of factorial experiments with nested factors’. Am. Stat. 48:142–144. doi:10.2307/2684271

Langsrud, Ø. 2003. ANOVA for unbalanced data: Use Type II instead of Type III sums of squares. Stat. Comput. 13:163–167. doi:10.1023/A:1023260610025

Lazaro, C.G. 1994. Invited response to Dallal, G.E. (1992), ‘The computer analy-sis of factorial experiments with nested factors’. Am. Stat. 48:140–141.

Littell, R.C., G.A. Milliken, W.W. Stroup, R.D. Wolfinger, and O. Schaben-berger. 2006. SAS for mixed models. 2nd ed. SAS Inst., Cary, NC.

Mathew, T., and B.K. Sinha. 1988. Optimum tests for fixed effects and variance components in balanced models. J. Am. Stat. Assoc. 83:133–135. doi:10.1080/01621459.1988.10478577

Maxwell, S.E., and H.D. Delaney. 1990. Designing experiments and analyzing data: A model comparison perspective. Brooks/Cole Publ., Pacific Grove, CA.

Milliken, G.A., and D.E. Johnson. 2008. Analysis of messy data. Vol. I. Designed experiments. 2nd ed. CRC Press, Boca Raton, FL.

Nelder, J.A. 1994. The statistics of linear models: Back to basics. Stat. Comput. 4:221–234. doi:10.1007/BF00156745

Pendleton, O.J., M. Von Tress, and R. Bremer. 1986. Interpretation of the four types of analysis of variance tables in SAS. Commun. Stat. Theory Meth. 15:2785–2808. doi:10.1080/03610928608829273

Piepho, H.P., E.R. Williams, and M. Fleck. 2006. A note on the analysis of designed experiments with complex treatment structure. HortScience 41:446–452.

Quinn, G.P., and M.J. Keough. 2002. Experimental design and data analysis for biologists. Cambridge Univ. Press, Cambridge, UK.

Rodriguez, R., R. Tobias, and R. Wolfinger. 1995. Comments on J.A. Nelder: ‘The statistics of linear models: Back to basics’. Stat. Comput. 5:97–101. doi:10.1007/BF00143938

Rogers, W.H. 1994. Invited response to Dallal, G.E. (1992), ‘The computer analysis of factorial experiments with nested factors’. Am. Stat. 48:144–145. doi:10.2307/2684272

Sahai, H., and M.I. Ageel. 2000. The analysis of variance: Fixed, random and mixed models. Birkhäuser, New York.

SAS Institute. 2011. SAS/STAT 9.3 user’s guide. SAS Inst., Cary, NC.Satterthwaite, F.E. 1946. An approximate distribution of estimates of variance

components. Biom. Bull. 2:110–114. doi:10.2307/3002019Searle, S.R. 1994. Analysis of variance computing package output for unbalanced

data from fixed effects models with nested factors. Am. Stat. 48:148–153.Searle, S.R. 1995. Comments on J.A. Nelder: ‘The statistics of linear models:

Back to basics’. Stat. Comput. 5:103–107. doi:10.1007/BF00143939Sevostianova, E., B. Leinauer, R. Sallenave, D. Karcher, and B. Maier. 2011. Soil

salinity and quality of sprinkler and drip irrigated warm-season turfgrasses. Agron. J. 103:1773–1784. doi:10.2134/agronj2011.0163

Shaw, R.G., and T. Mitchell-Olds. 1993. ANOVA for unbalanced data: An over-view. Ecology 74:1638–1645. doi:10.2307/1939922

Siefert, B. 1979. Optimal testing for fixed effects in general balanced mixed clas-sification models. Statistics 10:237–255.

Speed, F.M., R.R. Hocking, and O.P. Hackney. 1978. Methods of analysis of lin-ear models with unbalanced data. J. Am. Stat. Assoc. 73:105–112. doi:10.1080/01621459.1978.10480012

Steel, R.G.D., and J.H. Torrie. 1980. Principles and procedures of statistics. 2nd ed. McGraw-Hill, New York.

Urquhart, N.S., and D.L. Weeks. 1978. Linear models in messy data: Some prob-lems and alternatives. Biometrics 34:696–705. doi:10.2307/2530391

Westfall, P.H., R.D. Tobias, and R.D. Wolfinger. 2011. Multiple comparisons and multiple tests using SAS. 2nd ed. SAS Inst., Cary, NC.

Yandell, B.S. 1997. Practical data analysis for designed experiments. Chapman and Hall, London.