S2

Evidence of a Continuum Structure of Academic Self-Determination: A Two-Study Test Using a Bifactor-ESEM Representation of Academic Motivation

David Litaliena, Alexandre J.S. Morinb, Marylène Gagnéc, Robert J. Vallerandde, Gaëtan F. Losierf, and Richard M. Ryaneg

aFaculté des sciences de l’éducation, Université Laval, Canada

bConcordia University, Canada

cBusiness School, University of Western Australia

dDepartment of Psychology, Université du Québec à Montréal, Canada

eInstitute for Positive Psychology and Education, Australian Catholic University, Australia

fDepartment of Psychology, Université de Moncton, Canada

gUniversity of Rochester, United States

This is the prepublication version of the following manuscript:

Litalien, D., Morin, A. J. S., Gagné, M., Vallerand, R. J., Losier, G. F., & Ryan, R. M. (Accepted, 29 June 2017). Evidence of a continuum structure of academic self-determination: A two-study test using a bifactor-ESEM representation of academic motivation. Contemporary Educational Psychology.

© 2017. This paper is not the copy of record and may not exactly replicate the authoritative document published in Contemporary Educational Psychology.

Acknowledgements

The first author’s work was supported by a research grant from the Fond de Recherche du Québec – Société et Culture.

Corresponding author:

David Litalien, Faculté des sciences de l'éducation, Université Laval

2320, rue des Bibliothèques, Office 974

Québec (Québec), G1V 0A6, Canada

E-mail: David.Litalien@fse.ulaval.ca

Phone: (+1) 418-656-2131, Ext. 8699

Fax: (+1) 418-656-7343

Abstract

Self-determination theory postulates various types of motivation can be placed on a continuum according to their level of relative autonomy, or self-determination. We analyze this question through the application of a bifactor-ESEM framework to the Academic Motivation Scale, completed by undergraduate (N = 547; Study 1) and graduate (N = 571; Study 2) students. In both studies, the results showed that bifactor-ESEM was well-suited to modeling the continuum of academic motivation, and provided a simultaneous assessment of the global level of self-determination and of the specific motivation factors. Global academic self-determination positively predicted satisfaction with studies and vitality. It also negatively predicted dropout intentions and ill-being. Specific motivation types additionally predicted outcomes over and above the global factor.

Keywords: bifactor-ESEM, self-determination theory, academic motivation scale, continuum

Highlights

· Bifactor-ESEM is used to test self-determination theory’s continuum hypothesis

· The continuum structure of academic motivation was supported in two samples

· Both global self-determination and specific motivations predicted relevant outcomes

· Bifactor-ESEM disaggregates global and specific motivations

· It may represent an alternative to computed scores and higher-order representations

Evidence of a Continuum Structure of Academic Self-Determination: A Two-Study Test Using a Bifactor-ESEM Representation of Academic Motivation

1. Introduction

Self-determination theory (SDT; Deci & Ryan, 1985; Ryan & Deci, 2017) has been widely used by researchers to achieve a better understanding of students’ motivation (Deci & Ryan, 2016; Ryan & Deci, 2009). At the core of SDT is the assumption that human motivation can take many forms, differing from one another based on their degree of relative autonomy, or self-determination. Autonomous forms of motivation are experienced when individuals engage in behaviors for reasons that are perceived as self-endorsed and volitional. In contrast, controlled forms of motivation are experienced when individuals engage in behaviors for reasons that are perceived as resulting from internal or external pressures, reflecting a lower sense of volition. In the academic context, results from numerous studies supported the SDT assumption that more autonomous forms of motivations will be associated with more positive educational outcomes (for a review, see Guay, Lessard, & Dubois, 2016). SDT assumes that autonomous and controlled types of motivations can take many forms, which are expected to fall along a continuum of self-determination (Ryan & Connell, 1989).

Despite abundant research conducted to better understand the underlying structure of academic motivation, there is still no consensus on how this underlying continuum should be represented. Researchers relying on relatively recent statistical developments have suggested alternative ways to test this continuum hypothesis either through the examination of correlations among properly defined motivation factors (Guay, Morin, Litalien, Valois, & Vallerand, 2015), through the direct estimation of a global level of self-determination (Chemolli & Gagné, 2014), or a combination of both (Howard, Gagné, Morin, Wang, & Forest, 2016). Following from Howard et al.’s (2016) work conducted on work motivation, we propose a novel approach to assess the underlying continuum structure of academic motivation relying on the bifactor exploratory structural equation modeling (bifactor-ESEM) framework (Morin, Arens, & Marsh, 2016a; Morin, Arens, Tran, & Caci, 2016b). This representation has the advantage of providing a simultaneous assessment of the global level of academic self-determination and of the more specific types of motivation.

1.1 The self-determination continuum

At one extreme of the self-determination continuum, the most autonomous form of regulation is intrinsic motivation, which refers to the act of performing an activity for its own sake, out of interest and enjoyment (Ryan & Deci, 2002; 2017). In the academic area, some measures, notably those developed by Vallerand and colleagues (e.g., Vallerand et al., 1992), further divide this type of regulation into three dimensions: intrinsic motivation to know (for the pleasure of learning, exploring, and understanding something new), intrinsic motivation to accomplish (for the pleasure of trying to surpass oneself or to accomplish something), and intrinsic motivation to experience stimulation (for sensory pleasure, excitement, or enjoyment) (e.g., Carbonneau, Vallerand, & Lafrenière, 2012; Guay et al., 2015). Various types of extrinsic motivation may also occur when individuals engage in an activity as a mean to an end which is different from the activity itself (Deci & Ryan, 2012a; Ryan & Deci, 2017). Identified regulation, recognized as an autonomous form of extrinsic motivation, occurs when behaviors are accepted, valued, and considered to be personally important. Introjected regulation, an internalized type of controlled motivation, happens when individuals are driven to act by internal pressures based in contingent self-worth, or by the avoidance of guilt or shame. External regulation occurs when individuals adopt a behavior in order to obtain an externally controlled reward or to avoid punishment. Finally, amotivation refers to a relative lack of motivation, an absence of reason or willingness to enact specific behaviors (Deci & Ryan, 2012a, 2016; Ryan & Deci, 2017).

Although these various motivation types are proposed to be important in their own right and distinct from one another in terms of antecedents, phenomenology and functional consequences, according to the SDT continuum hypothesis they are also expected to differ from one another in their level of relative autonomy or self-determination (e.g., Ryan & Connell, 1989; Deci & Ryan, 2016). Although amotivation is not considered in all depictions of the continuum structure of motivation, in others it occupies the lowest point of the continuum (e.g., Cox, Ullrich-French, Madonia, & Witty, 2011; Guay, Ratelle, Roy, & Litalien, 2010; Howard et al., 2016). In the SDT research literature, providing empirical support for this hypothetical continuum structure has traditionally involved demonstrating the quasi-simplex[footnoteRef:2] (ordered) pattern of correlations described by Ryan and Connell (1989). When correlations follow such a quasi-simplex pattern, concepts that are hypothesized to be more similar to one another should be more strongly correlated than more distinct concepts, which should be more weakly, or even negatively, correlated. Thus, SDT’s continuum hypothesis leads researchers to predict stronger positive correlations between theoretically adjacent forms of regulation (e.g., introjected and external) than between more distal forms (e.g., intrinsic motivation and external regulations), which would be either uncorrelated or even negatively related. [2: In comparison to a perfect simplex structure, the quasi-simplex assume that the variables of interest contain measurement error (Jöreskog, 1970).]

In the academic domain, one of the oldest and most widely used instruments to assess motivation under the SDT framework is the Academic Motivation Scale (AMS; Vallerand et al., 1992). Guay et al. (2015) recently conducted a review of studies based on this scale to examine whether the correlations obtained among the various AMS motivation subscales followed the expected quasi-simplex pattern. They mentioned that evidence for such a continuum, specifically within the AMS, remained inconclusive, with some studies lending support to the continuum hypothesis, and others failing to provide support for various reasons (e.g., poor fit, higher correlations between more distant types of motivation). Nevertheless, recent statistical research evidence suggests that exploratory structural equation modeling (ESEM, Model A in Figure 1; allowing for the free estimation of cross-loadings between items and non-target factors) tends to provide more exact estimates of factor correlations than more traditional confirmatory factor analyses (CFA, Model B in Figure 1; for a recent review, see Asparouhov, Muthén & Morin, 2015). Based on this evidence, Guay et al. (2015) sought to clarify the idea that motivation types followed an underlying continuum structure through the application of ESEM to responses provided to the AMS by two independent samples of students. First, they found that correlations between motivational factors generally conformed to a quasi-simplex in both samples. Second, in line with their expectations, Guay et al. (2015) found that ESEM factor correlations tended to be much more aligned with SDT’s hypothesized continuum of motivation than the results from CFA factor correlations (for similar results obtained among doctoral students, see Litalien, Guay, & Morin, 2015). However, despite obtaining interesting results from the estimation of cross-loadings, the ESEM approach does not provide a global index of academic self-determination.

Rather than examining factor correlations to see whether they followed a quasi-simplex pattern, Chemolli and Gagné (2014) argued that the presence of a single factor model underlying answers to motivation instruments would provide a more direct test of the SDT continuum hypothesis. They suggested that should this representation be supported by the data, evidence in favor of the continuum structure of motivation would come from the observation of factor loadings on this single global factor[footnoteRef:3] ranging from negative for the least self-determined forms of motivation to positive for the most self-determined forms of motivation. Conducting a Rasch (1960) analysis, Chemolli and Gagné (2014) failed to find support for the continuum structure of motivation either in the academic (using the AMS) or the work domain (using the Multidimensional Work Motivation Scale; Gagné et al., 2015). Yet a key limitation of this approach is that the Rasch method is designed to identify a single overarching motivation factor while neglecting to consider that the various motivation types are still expected to retain a meaningful level of specificity over and above this global factor, according to SDT. This limitation may explain Chemolli and Gagné’s (2014) failure to support the continuum structure. In order to simultaneously estimate a global level of academic self-determination while accounting for the more specific types of motivation, we adopted a bifactor-ESEM approach (Morin et al., 2016a; Morin et al., 2016b). This technique is used to as a way to integrate in a single model of academic motivation the Chemolli and Gagné (2014) and Guay et al. (2015) perspectives. Recently, Howard et al. (2016) adopted a similar approach in a single study conducted in the field of work motivation, and found stronger support for the continuum structure of work motivation than what had been found so far with different methods, while also finding evidence of meaningful specificity located at the subscale level. In the present series of studies, we assess the generalizability of this approach for the study of academic motivation. [3: The term “global” is used to refer to the presence of an overarching construct relative to more specific constructs, in line with the suggested bifactor representation. Unless otherwise stated, it does not refer to the hierarchical levels of motivation (global, contextual, and situational) as discussed by Vallerand (1997) given that this whole study is about the contextual “domain” of academic motivation. ]

1.2 The Bifactor Exploratory Structural Equation Modeling (ESEM) Framework

CFA is generally considered as a gold standard in the investigation of the psychometric properties of measurement instruments, although many well-established measures do not consistently meet acceptable goodness-of-fit criteria when analyzed using this approach (Marsh, Morin, Parker, & Kaur, 2014). This observation has led some to question the appropriateness and realism of the restrictive independent cluster model (ICM) assumption of CFA, which forces cross-loadings between items and non-target factors to be constrained to zero (Marsh et al., 2014; Morin, Marsh, & Nagengast, 2013). Morin et al. (2016a, 2016b) noted that this requirement was never part of classical test theory, which recognizes that items may reflect more than one source of construct-relevant multidimensionality (i.e., true score variance). Morin et al. (2016a, 2016b) pointed out that typical measures used in psychological research often tap into two distinct sources of construct-relevant psychometric multidimensionality, associated with the assessment of conceptually-related constructs (e.g., interrelated types of motivation; Guay et al., 2015; Litalien et al., 2015) and global overarching constructs (e.g., an overarching continuum of motivation).

Models incorporating cross-loadings between items and non-target factors, such as exploratory factor analyses (EFA), have been proposed as a more appropriate way to model responses to measurement instruments assessing conceptually-related constructs (Marsh et al., 2014; Morin et al., 2013). Furthermore, whereas classical approaches to EFA display limitations in comparison to CFA, the newly developed ESEM framework has incorporated EFA within the overarching structural equation modeling (SEM) framework (Asparouhov & Muthén, 2009), thus solving most of these limitations (e.g., goodness-of-fit assessment, tests of measurement invariance, estimation of predictive relations between latent EFA factors). Similarly, target rotation makes it possible to specify EFA/ESEM models with cross-loadings in a purely confirmatory manner, “targeting” cross-loadings to be as close to zero as possible (Asparouhov & Muthén, 2009; Browne, 2001). Perhaps more importantly, rapidly accumulating evidence from both statistical simulation studies and studies of simulated data showed that when cross-loadings are present in the population model (even as low as .100), forcing them to be zero in a CFA model results in biased estimates of the factor correlations, whereas relying on an EFA/ESEM model when no cross-loadings are present in the population model still results in unbiased estimates of the factor correlations despite the loss in terms of parsimony (for a review, see Asparouhov et al., 2015). In relation to tests of the continuum structure of motivation, which are typically conducted either based on an examination of factor correlations, this limitation of CFA appears quite critical.

The second source of construct-relevant psychometric multidimensionality identified by Morin et al. (2016a, 2016b) deals with the assessment of overarching constructs. The typical approach to capture this source of construct-relevant psychometric multidimensionality relies on the estimation of higher-order factor models, which explain the covariance among first-order factors through the estimation of one or more higher-order factors (Rindskopf & Rose, 1988). Despite its appeal, this approach relies on a restrictive implicit assumption – a proportionality constraint – that could explain why higher-order models often show poor fit indices (Gignac, 2016; Morin et al., 2016a; Reise, 2012). This proportionality constraint means that the ratio of the variance attributed to the higher-order factor versus uniquely attributed to the first-order factor is a constant for all items associated with a single first-order factor (Morin et al., 2016a; Reise, 2012). Furthermore, in such models, the higher-order factors do not explain additional variance besides that already explained by the first-order factors.

Conversely, bifactor models (Holzinger & Swineford, 1937; see Figure 2) allow for the estimation of overarching constructs without relying on this restrictive implicit assumption and for the separate assessment of variance uniquely attributable to specific and global factors (see Gignac, 2016; Rijmen, 2010; Schmid & Leiman, 1957; Yung, Thissen, & McLeod, 1999, for comparisons between higher-order and bifactor model). In the bifactor approach, the covariance among a set of n items can be explained by a set of f orthogonal factors including one Global (G) factor and f-1 (total number of factors minus one G-factor) orthogonal Specific (S) factors. As each item is used to simultaneously define the G-factor and one S-factor, the covariance is divided into a G-factor underlying all items, and f-1 S-factors corresponding to the covariance not explained by the G-factor. As such, the G-factor estimated as part of a bifactor model provides a direct way to test for the presence of a global overarching construct underlying responses to all items, while also acknowledging that important distinctions exist at the subscale level. As noted by Howard et al., (2016, p. 7) “This clean partitioning is made possible by the orthogonality of the factors, which forces all of the variance shared among all items to be absorbed into the G-factor, and the S-factors to represent what is shared among a specific subset of items but not the others.” It has thus been argued that unless researchers can theoretically justify the presence of the proportionality constraints and of indirect associations between the indicators and the global factors, bifactor models should be preferred (Gignac, 2016).

Some psychological scales like the AMS are expected to include both sources of construct-relevant multidimensionality, as they assess both conceptually-adjacent constructs (as estimated by Guay et al., 2015) and the presence of an overarching construct (as estimated by Chemolli & Gagné, 2014). In such case, a bifactor-ESEM approach (see Model C in Figure 2), including both cross-loadings among specific dimensions (i.e., ESEM) and a global factor (i.e., bifactor), appears particularly relevant (Morin et al., 2016a, 2016b). Statistically, the ability to include an ESEM and a bifactor component in a single model appears critical given the evidence from statistical studies and studies of simulated data showing that unmodeled cross-loadings tend to result in inflated estimates of factor correlations in CFA, or of the global factor in bifactor-CFA (see Model D in Figure 2), while an unmodeled global factor tends to result in inflated factor correlations in CFA or inflated cross-loadings in ESEM (e.g., Morin et al., 2016a; Murray & Johnson, 2013). For this reason, whenever there are reasons to expect the presence of both sources of construct-relevant psychometric multidimensionality, Morin et al. (2016a, 2016b) recommend the systematic comparison of CFA, ESEM, bifactor-CFA, and bifactor-ESEM models in order to clearly identify both sources of multidimensionality. In particular, bifactor-ESEM provides a single easily interpretable estimate of SDT’s overarching continuum of self-determination (the G-factor), while acknowledging the specificity, unrelated to this continuum, remaining at the subscale level (the S-factors), and controlling for the cross-loadings likely to be present.

Over and above this ongoing debate regarding how to model the underlying continuum proposed by SDT, there also seem to be practical advantages to the bifactor-ESEM approach. In particular, although SDT suggests that each of the proposed motivation types (i.e., intrinsic, identified, introjected, external, amotivation) is important to consider in its own right, SDT research has seldom been able to simultaneously include all motivation types in a single predictive model, possibly because of the high levels of factor correlations typically obtained in CFA studies (Guay et al., 2015), which may induce multicollinearity. Thus, in practice, SDT research has tended to rely either on a single global indicator of participants’ relative levels of autonomy (i.e., the relative autonomy index – RAI; e.g., Litalien et al., 2013; Ricard & Pelletier, 2016; Ryan & Connell, 1989) or on two higher-order factors representing autonomous and controlled forms of motivation (e.g., Gillet, Gagné, Sauvagère, & Fouquereau, 2013). Without contesting the value of this prior research, it is important to note that these simplified representations provide, at best, only a partial test of the SDT proposition that there are specific meaningful subtypes of motivation that also fall along a continuum. Because bifactor models are orthogonal, they provide a way to directly assess the added-value of all specific S-factors over and above that of the global G-factor in terms of prediction.

In organizational psychology, Howard et al. (2016) recently explored the continuum structure of work motivation through a bifactor-ESEM approach. Their results supported the continuum hypothesis, and allowed them to demonstrate meaningful relations between a global self-determination at work factor and a series of covariates. Thus, this global factor was positively predicted by affective commitment and needs satisfaction. Moreover, the explained variance from the covariates were also increased by the simultaneous assessment of the specific types of motivation. Although showing promising results, this study was specific to the work context and only explored a limited number of predictors using one sample. In the present series of two studies, we aim to replicate and extend the results obtained by Howard et al. (2016) to the academic domain. We thus illustrate the value of the bifactor-ESEM approach by a systematic investigation of the relations between the AMS G- and S- factors and a series of predictors and outcomes typically considered in SDT research.

1.3 The Present Research

In the present series of studies, our objective was to further assess the ability of the SDT continuum hypothesis to reflect participants’ answers to the AMS through the application of a bifactor-ESEM framework. This framework, recently tested in organizational psychology by Howard et al. (2016), allowed us to simultaneously integrate the conceptually-adjacent constructs perspective of academic motivation advocated by Guay and colleagues (Guay et al., 2015; Litalien et al., 2015) and the overarching construct perspective sponsored by Chemolli and Gagné (2014) within a single model. More precisely, we investigated whether the AMS items measuring specific types of academic motivation also loaded onto a global factor with loadings ranging from negative to positive according to their expected position along the continuum. This global factor should provide an estimate of the overall level of self-determined academic motivation, while the specific factors should more precisely represent the unique features of students’ academic motivation, over and above this global level of academic self-determination. In a first study, we applied the bifactor-ESEM framework to answers provided to the AMS by a sample of undergraduate students to test SDT’s continuum hypothesis. In order to test the criterion-related validity of the resulting G- and S- motivational factors, we also examined their relations with two wellbeing outcomes (subjective vitality and ill-health). Wellbeing, performance, and engagement have been proposed and commonly used as specifiable motivational outcomes within the SDT framework as applied to a variety of domains, including education (e.g., Deci & Ryan, 2008; Ryan & Deci, 2000; 2017). A second study was conducted to assess the generalizability of our findings to a sample of graduate students, and to extend tests of criterion-related validity to a series of academic outcomes (academic achievement, dropout intentions, and satisfaction with studies). In this second study, we also considered predictors of these motivational factors assumed to be central to the internalization process proposed by SDT (i.e., satisfaction of the needs for autonomy, competence and relatedness), and previously assessed by Howard et al. (2016). In the current research, we aim to replicate and extend the work of these authors (1) by investigating the structure of a widely-used instrument to assess motivation in the academic context, (2) by comparing the results between two samples of students from various level, (3) by assessing a range of outcomes that have been previously associated with self-determined motivation, and (4) by assessing the three types of intrinsic motivations measured by the AMS.

2. Study 1

In this first study, we tested the SDT continuum hypothesis through the application of the bifactor-ESEM framework to undergraduate students’ answers to the AMS. To assess the criterion-related validity of the resulting G- and S-factors, we assessed their relations with two outcome variables related to participants’ wellbeing: levels of vitality and ill-being. Research generally shows that more autonomous forms of regulation tend to produce more positive outcomes than the more controlled forms of regulations, such as higher levels of vitality (e.g., Niemiec et al., 2006; Ryan & Frederick, 1997) and wellbeing (or lower levels of ill-being; e.g., Litalien et al., 2015; Litalien & Guay, 2015; Vallerand, Fortier, & Guay, 1997). For this reason, we hypothesized relations involving global levels of academic self-determination to replicate these previous results. We also expected the remaining specific motivation S-factors to relate to outcomes in a manner that follows their position on this continuum, with more autonomous types of regulation predicting more positive outcomes and the less autonomous types predicting less positive, and in some cases, negative outcomes. Additionally, we investigated the added value of simultaneously considering both components of participants’ motivation. To this end, we compared models in which only the overall levels of self-determined motivation (G-factor) was associated with the outcomes to models in which the specific types of motivation (S-factors) were also allowed to be associated with the outcome variables.

2.1 Method

2.1.1 Participants and Procedures. A total of 547 undergraduate students from an English-speaking Canadian university participated voluntarily to this first study in exchange for extra credit towards an introductory organizational behaviour course. Mean age was 22.8 years (SD = 4.8 years), 58.6% were female, and 63.3% were Canadian citizens. Students were mostly in their first (38.1%) or second year (45.1%) at university and were more likely to have parents who completed at least a college degree (68.0% and 62.5% of fathers and mothers, respectively).

2.2 Measures

2.2.1 Academic Motivation Scale. The 28 items from the AMS (Vallerand et al., 1989, 1992, 1993) were used to assess seven dimensions (4 items per dimension) of students’ motivation toward school activities: (a) intrinsic motivation to know (α = .860; e.g., “Because I experience pleasure and satisfaction while learning new things”); (b) intrinsic motivation to experience stimulation (α = .843; e.g., “For the pleasure that I experience when I feel completely absorbed by what certain authors have written”); (c) intrinsic motivation to accomplish (α = .848; e.g., “For the satisfaction I feel when I am in the process of accomplishing difficult academic activities”); (d) identified regulation (α = .781; e.g., “Because I believe that a few additional years of education will improve my competence as a worker”); (e) introjected regulation (α = .834; e.g., “To prove to myself that I am capable of completing my college degree”); (f) external regulation (α = .847; e.g., “In order to have a better salary later on”); (g) amotivation (α = .883; e.g., “I can’t see why I go to university and frankly, I couldn’t care less”). Participants were asked to rate each item using a seven-point Likert scale (1 = does not correspond at all, 7 = corresponds exactly). For additional information on the psychometric properties of the AMS, readers are referred to Guay et al. (2015) review.

2.2.2 Outcomes: Vitality and Ill-being. The seven items from the Subjective Vitality Scale (SVS; Ryan & Frederick, 1997; α = .84 to .86 in their three samples) were used to assess students’ level of vitality experienced in the last six months (e.g., “I have been feeling very alert and awake”; α = .871 in the current study). Fourteen items from the General Health Questionnaire (Goldberg & Hillier, 1979) were used to assess students’ ill-being in the last six months. These items were taken from the anxiety and somatic symptoms subscales to represent a general ill-being factor (e.g., “I have been feeling ill”, “I have been feeling constantly under strain”; α = .902 in the current study), on which higher score represents higher ill-being and poorer health. Previous studies also provided support for the reliability of the specific subscales and the general ill-being factor (e.g., Vallejo, Jordán, Díaz, Comeche, & Ortega, 2007; paper version, α = .83, .84, and .90 for anxiety, somatic symptoms, and general ill-being, respectively). Participants rated each item using a seven-point Likert scale (1 = not at all true, 7 = definitely true).

2.3 Analyses

Analyses were conducted using the robust Maximum Likelihood estimator (MLR) available in Mplus 7.3 (Muthén, & Muthén, 2014). Annotated Mplus input codes for estimating the various models are reported in the online supplements. CFA models were specified according to ICM assumptions, with items allowed to load onto their a priori factor, and all cross-loadings constrained to be exactly zero. ESEM was specified via oblique target rotation, with item loadings on their a priori factors freely estimated, and cross-loadings “targeted” to be as close to zero as possible. Bifactor-CFA models were specified as orthogonal, with each item specified as loading on the self-determination G-factor as well as on their a priori motivation S-factors. Finally, bifactor-ESEM was specified via orthogonal bifactor target rotation: All items were used to define the self-determination G-factor, while the seven motivation S-factors were defined with the same pattern of target and non-target loadings as in ESEM.

Outcomes of the G- and S- factors were then integrated into the bifactor-ESEM model[footnoteRef:4]. In a first model, only the G-factor was allowed to predict the outcomes through the ESEM-within-CFA method described by Morin et al. (2013, 2016a). This method allows for the estimation of relations between a subset of factors from an ESEM or bifactor-ESEM model (i.e., here only the G-factor) and the outcomes, while the relations between the remaining factors (i.e., here the S-factors) and the outcomes are constrained to be zero. In the second model (relying on a regular bifactor-ESEM model), both the G-factor and the S-factors were allowed to freely predict the outcomes. Both models were compared based on goodness-of-fit, but also on standardized regression coefficients and percentage of explained variance (R2) in relation to the assessed outcomes. [4: The complexity of the Bifactor-ESEM model used to represent the motivation factors made it impossible to integrate the outcomes to these models as latent variables. Still, in order to achieve at least a partial level of correction for measurement errors (e.g., Skrondal & Laake, 2001; Morin, Meyer, Creusier, & Biétry, 2016), these outcomes were represented by factors scores saved from a preliminary measurement models (Vitality: χ2 = 13.989, df = 14, p > .05; CFI = 1.000; TLI = 1.000; RMSEA = .000; Ill-Being: χ2 = 205.679, df = 63, p ≤ .01; CFI = .938; TLI = .911; RMSEA = .065). Vitality was estimated as a simple CFA factor, while ill-health was estimated as the G-factor from a bifactor model including two S-factors representing anxiety and somatization. ]

Model fit was assessed using several goodness-of-fit indices and information criteria: the comparative fit index (CFI), the Tucker-Lewis index (TLI), the root mean square error of approximation (RMSEA) with its 90% confidence interval, the Akaike Information Criteria (AIC), the Consistent AIC (CAIC), the Bayesian Information Criteria (BIC), and the sample-size adjusted BIC (ABIC). Values greater than .90 and .95 for the CFI and TLI respectively support adequate and excellent fit of the data to the model while values smaller than .08 or .06 for the RMSEA support acceptable and excellent fit (Hu & Bentler, 1999; Marsh, Hau, & Wen, 2004; Marsh, Hau, & Grayson, 2005). The information criteria (AIC, CAIC, BIC, ABIC) are used to compare alternative models, with lower values suggesting a better fitting model. These guidelines have been established for CFA, and used in previous ESEM applications (e.g., Marsh et al., 2009, 2014; Morin et al., 2013, 2016a).

2.4 Results

The descriptive statistics of all items are reported in Table S1 of the online supplements. The goodness-of-fit of the four alternative models is reported in Table 1. These results showed that whereas the fit of the ICM-CFA model fell within the range of acceptable values, the fit of the bifactor-CFA models fell below acceptable values according to the CFI and TLI. In contrast, and despite the fact that they result in slightly higher values on the BIC and CAIC, both the ESEM and bifactor-ESEM models provided an excellent degree of fit to the data, resulting in a significant improvement of fit in comparison with the ICM-CFA model (ΔCFI = +.043 to .054; ΔTLI = +.031 to .047; ΔRMSEA = -.012 to -.019; lower AIC and ABIC). Although both of these models provided an excellent fit to the data, the bifactor-ESEM solution resulted in a substantial improvement of fit relative to ESEM (ΔCFI = +.011; ΔTLI = +.016; ΔRMSEA = -.007; lower AIC and ABIC).

The superiority of the ESEM/bifactor-ESEM solution relative to the ICM-CFA/bifactor-CFA solutions in goodness-of-fit strongly suggests the presence of cross-loadings. Factor correlations are thus expected to be higher in ICM-CFA compared to ESEM as this is the only way through which these cross-loadings can be expressed. In contrast, given the orthogonality of the bifactor-CFA model, these cross-loadings can only be expressed through an inflated estimate of the G-factor, which is unlikely to be enough to compensate for this potential source of misfit if the cross-loadings reflect another source of multidimensionality than the presence of an underlying global construct. This might explain the suboptimal level of fit of the bifactor-CFA solution. Thus, because the bifactor-CFA model did not show adequate fit to the data, and following Morin et al. (2016a) recommendations suggesting that decisions regarding model selection should be based on an examination of parameter estimates in addition to goodness-of-fit information, we first turn to a comparison of ICM-CFA and ESEM solutions, before moving on to the bifactor-ESEM solution.

Parameter estimates from the ICM-CFA and ESEM solutions are reported in Table 2 (factor loadings, cross-loadings, and uniquenesses) and Table 3 (factor correlations). Both models revealed factors that are generally well-defined by satisfactory factor loadings (ICM-CFA: Mλ = .76; ESEM: Mλ = .63) corresponding to a priori expectations. As expected, the ESEM solution also revealed multiple cross-loadings, which remained relatively small (|λ| = .00 to .34; M = .08) and generally lower than the main target loadings. One exception was observed, showing that the first item of intrinsic motivation to accomplish loaded weakly on its a priori factor (.25) and equivalently on the intrinsic motivation to know factor (.27), suggesting that this specific item may not be as strongly specific to one type of intrinsic motivation as expected. This item label (“For the pleasure I experience while surpassing myself in my studies”) does in fact appears to tap into both the accomplishment (“surpassing myself”) and knowledge (“in my studies”) dimensions.

As expected, factor correlations were substantially lower in ESEM (|r| =.06 to .57; M = .32) than ICM-CFA (|r| = .07 to .88; M = .45). The pattern of correlations was similar between both models and partly supports the SDT continuum hypothesis, showing stronger correlations between theoretically adjacent factors and lower correlations among more distant ones. Amotivation was negatively associated with most motivation factors, but unrelated to intrinsic motivation to experience stimulation. Overall, correlations were slightly stronger among autonomous, rather than controlled, forms of motivation. Two exceptions are worth mentioning: Introjected regulation was more strongly associated with intrinsic motivation to accomplish than with identified regulation, and identified regulation was more strongly associated with external regulation than introjected regulation.

As mentioned above, the bifactor-ESEM model proved to be the best fitting model, and is of particular theoretical interest as it provides a direct estimate of the SDT continuum. The results associated with this model are reported in Table 4, and generally support the presence of an underlying continuum of self-determination. Indeed, item loadings on the G-factor were generally high and positive for the items associated with the intrinsic motivation S-factors (λ = .61 to .73 for knowledge, .47 to .64 for stimulation, .62 to .83 for accomplishment), moderate for identified (λ = .37 to .49) and introjected (λ = .38 to .60) regulations, small for external motivation (λ = .13 to .33), and negative for amotivation (λ = -.31 to -.40). The S-factors were also generally well-defined by relatively high loadings (|λ| = .24 to .78; M = .52), and weaker cross-loadings (|λ| = .00 to .36; M = .08), although these S-factors remained slightly more weakly defined that their ESEM counterparts due to the extraction of the variance explained by the G-factor from the items. In particular, items 3 and 4 of the intrinsic motivation to accomplish dimension only showed weak loadings on their a priori factor (λ = .15 and -.03), and strong loadings on the G-factor (λ = .74 and .83), suggesting that these items are more efficient at tapping into global self-determination than specific intrinsic motivation. Overall, the more autonomous forms of motivation appear to retain less specificity once the global continuum factor is included in the model, whereas the more controlled forms of motivation, as well as amotivation, seem to retain more specificity, consistent with the labelling of the global factor reflecting the overall level of academic self-determination.

Because of both its greater level of fit to the data, as well as its greater level of theoretical consistency, the bifactor-ESEM solution was thus retained as the final model. The criterion-related validity of this model was then tested through the direct inclusion of the outcomes to the model. Results from these analyses are reported in Table 5. These results showed that when the G-factor is considered as the sole predictor of the outcomes, it significantly predicted higher scores of vitality and lower scores of ill-being, as expected. Interestingly, these results remained stable in the more complete predictive model in which all factors were allowed to predict the outcomes. However, this more exhaustive model resulted in substantial increases in percentage of explained variance (5.1% to 13.3% for vitality and 1.5% to 21.0% for ill-being). Regarding the S-factors, their relations with the outcome variables proved to be partly in line with our expectations. External regulation and amotivation S-factors negatively predicted vitality and positively predicted ill-being. In addition, ill-being was also negatively predicted by the S-factor for intrinsic motivation to know and positively by the S-factor for introjected regulation. However, the S-factors for intrinsic motivation to experience stimulation and accomplishment, as well as identified regulation, did not predict vitality nor ill-being, and those for intrinsic motivation to know and introjected regulation did not predict vitality.

3. Study 2

Results from Study 1 confirmed the hypothesized self-determination continuum among undergraduate students and supported the criterion-related validity of the model using general wellbeing indicators (vitality and ill-being). A second study was conducted among a new independent sample of graduate students in order to assess the extent to which the results would generalize to a more advanced academic level. We also rely on Study 2 to extend the results of Study 1 by considering a new set of outcomes relevant to the academic context (academic achievement, dropout intentions, and satisfaction with one’s studies). As for wellbeing, research shows that more autonomous types of regulations tend to produce more positive academic outcomes, such as academic achievement (Black & Deci, 2000), satisfaction with studies (Litalien et al., 2015; Vallerand et al., 1993), and lower academic dropout intentions (Litalien et al., 2015; Litalien & Guay, 2015; Vallerand, Fortier, & Guay, 1997). Based on these results, we hypothesize that relations involving global levels of self-determination will replicate these previous results in showing positive associations with desirable academic outcomes. Similarly, we expect the specific motivation S-factors to relate to academic outcomes in a manner that follows their position on this continuum, with more autonomous types predicting more positive academic outcomes and less autonomous types predicting more negative academic outcomes.

In Study 2, we also consider the relations between the G- and S- factors and a set of core SDT predictors related to the satisfaction of the basic psychological needs for competence, autonomy and relatedness. At the core of SDT is the assumption that individuals possess a natural tendency toward integration and internalization (to strive toward more autonomous forms of motivation), and that this tendency will depend social environments’ ability to support and satisfy basic psychological needs for autonomy, competence, and relatedness (Deci & Ryan, 1985, 2012b). Autonomy refers to “experiencing a sense of choice, willingness, and volition as one behaves” (Deci, Ryan, & Guay, 2013, p. 113). The satisfaction of the need for competence relates to the feeling of being effective in one’s interactions with the environment and being able to exercise one’s capacities. The satisfaction of the need for relatedness refers to the quality of interpersonal relationships, to the satisfaction of the “need to be close to, trusting of, caring for, and cared for by others” (Deci & Ryan, 2012b, p. 421). SDT particularly posits the centrality of the satisfaction of the need of autonomy in individual growth (Deci & Ryan, 2000). Empirical research strongly supports the importance of the satisfaction of these three needs, showing that it predicts the internalization of motivation (e.g., higher levels on the more autonomous forms of motivation, and lower levels on the more controlled forms of motivation; for reviews see Ryan & Deci, 2000; Ryan, Deci, & Vansteenkiste, 2016). We thus hypothesize that the satisfaction of those needs, especially for autonomy, should positively predict the global academic self-determination factor. Overall, the predictive associations between needs satisfaction and the specific motivation factors should also reflect the continuum and show positive to negative regression coefficients from the more to the less self-determined types of motivation.

3.1 Methods

3.1.1 Participants and Procedures. Graduate students from every program of a French-speaking Canadian university were invited to participate voluntarily to a mail survey, with no financial incentive. A total of 571 graduate students participated, mean age was 33.0 years (SD = 8.0 years) and 53.8% were females.

3.2 Measures

3.2.1 Academic Motivation Scale. The French version of the AMS (Échelle de Motivation en Éducation; Vallerand, Blais, Briere, & Pelletier, 1989) was used to assess graduate students’ motivation toward school activities. Nine items out of 28 are specific to student’s academic-level and were slightly modified to fit graduate students’ experiences. For instance, the item “Because I think that a college education will help me better prepare for the career I have chosen” was adjusted to “Because I think that graduate studies will help me better prepare for the career I have chosen”. Similar adaptations of the scale have been used in previous studies (e.g., Ahmed & Bruinsma, 2006; Losier, 1994). Cronbach’s alphas were similar to those obtained among English-speaking undergraduate students from Study 1, ranging from .781 for identified regulation to .893 for introjected regulation.

3.2.2 Academic Outcomes: Dropout Intentions, Satisfaction with Studies, and Achievement. Seven items were used to assess dropout intentions (Losier, 1994; e.g., “Sometimes I consider dropping out of my program”; α = .864). The five items of the Satisfaction with Studies Scale (Échelle de Satisfaction dans les Études; Vallerand & Bissonnette, 1990; α = .71 to .85 in their 5 studies) were used to assess students’ satisfaction with their studies. This instrument contains five items (e.g., “I am satisfied with my studies”; α = .864 in the current study). For both scales, participants rated their level of agreement using a seven-point Likert scale (1 = strongly disagree, 7 = strongly agree). Academic achievement was assessed via self-reported cumulative grade point average, ranging from 1.8 to 4.3 on a 4.3 scale (M = 3.6, SD = 0.5).

3.2.3 Predictors: Basic Psychological Needs Satisfaction. Twelve items were used to assess students’ perceptions of satisfaction on each of the three basic psychological needs. Satisfaction of the need for competence was assessed using an adaptation of the Perceptions of Competence in Life Domains scale (4 items; e.g., “I think that I am a good student”; Losier, Vallerand, & Blais, 1993). Cronbach’s alpha for this subscale varied from .66 to .71 in Losier et al. (1993; 3 studies) and was .701 in the current study. Satisfaction of the need for autonomy need was assessed using an adaptation of the Perceived Autonomy in Life Domains scale (4 items; e.g., “I feel a freedom of action at university”; Blais, Vallerand, & Lachance, 1990). Cronbach’s alpha for this subscale varied from .70 to .72 in Losier (1994; 2 studies) and was .662 in the present study. Finally, satisfaction of the need for relatedness was assessed via a four-item scale developed by Losier (1994; e.g., “Overall, I feel connected to the people I am studying with [other students and faculty]”). Cronbach’s alpha for this subscale varied from .81 to .84 in Losier (1994; 2 studies) and was .835 in the current study. On each items of these subscales, participants rated their level of agreement using a seven-point Likert scale (1 = strongly disagree, 7 = strongly agree).

3.3 Analyses

The analyses conducted in Study 2 are identical to those conducted in Study 1. In addition, the predictor variables (i.e., three basic psychological needs satisfaction) were also directly integrated into the bifactor-ESEM model, using a sequence similar to that used for the outcomes[footnoteRef:5]. In a first model, predictors were only allowed to predict the G-factor through the ESEM-within-CFA method (Morin et al., 2013, 2016a). In a second model, the predictors were allowed to predict both the G-factor and the S-factors in a regular bifactor-ESEM model. Both models were then compared based on standardized regression coefficients and goodness-of-fit as in Study 1. [5: As in Study 1, covariates (predictors and outcomes) were incorporated to this model as factor scores saved from a single preliminary measurement model (χ2 = 554.219, df = 216, p ≤ .01; CFI = .929; TLI = .909; RMSEA = .052), which proved to be particularly important as a way to incorporate a partial control for the lower levels of reliability associated to some of the need satisfaction measures (e.g., Skrondal & Laake, 2001; Morin, Meyer et al., 2016). In this model, the need satisfaction measures were represented as ESEM factors, whereas the outcomes were represented as CFA factors. A method factor was incorporated to the model to control for the methodological artefact associated with the negative wording of some items (Marsh, Scalas, & Nagengast, 2010).]

3.4 Results

Goodness-of-fit results, reported in the bottom of Table 1, replicated those from Study 1 in supporting the superiority of the bifactor-ESEM solution. Turning first our attention to the ICM-CFA and ESEM solutions (see Table 6 and the bottom of Table 3), our results revealed well-defined factors for both the ICM-CFA (λ = .54 to .89; M = .79) and ESEM (λ = .31 to .98; M = .74) solutions, with evidence of cross-loadings that remained smaller than target loadings in ESEM (|λ| = .00 to .38; M = .06). Only the first item of intrinsic motivation to experience stimulation loaded more strongly on intrinsic motivation (.38) to accomplishment than on its a priori factor (.31). Factor correlations were slightly lower in ESEM (|r| = .02 to .63; M = .27) than ICM-CFA (|r| = .01 to. 75; M = .31). These correlations mainly supported the SDT continuum hypothesis, being higher between adjacent constructs, smaller between more distal construct, and sometimes negative with amotivation. However, as in Study 1, introjected regulation was more strongly associated with intrinsic motivation to accomplish than with identified regulation, and identified regulation was more strongly associated with external regulation than with introjected regulation.

The results from the best-fitting bifactor-ESEM solution are reported in Table 7. As in Study 1, these results showed that the G-factor corresponds to an underlying continuum of self-determination, being characterized by high and positive loadings for the items associated with the intrinsic motivation S-factors (λ = .56 to .79 for knowledge, .48 to .68 for stimulation, .67 to .71 for accomplishment), moderate and positive loadings for identified (λ = .28 to .42) and introjected (λ = .28 to .55) regulations, small loadings for external regulation (λ = .05 to .28), and negative loadings for amotivation (λ = -.09 to -.22). The S-factors were also well-defined by relatively high factor loadings (|λ| = .21 to .83; M = .57), and weaker cross-loadings (|λ| = .00 to .30; M = .08), although once again the more autonomous forms of motivation appeared to retain less specificity once the G-factor reflecting global self-determination was taken into account. In line with the ESEM results, the first item of intrinsic motivation to experience stimulation loaded equivalently on intrinsic motivation (.30) to accomplishment than on its a priori factor (.29), suggesting that this specific item may not be as strongly specific to one type of intrinsic motivation as expected.

The criterion-related validity of the bifactor-ESEM solution was investigated by the inclusion of the predictors and outcomes in the model. Results from these analyses are reported in the middle (outcomes) and bottom (predictors) of Table 5. As expected, the results showed that the G-factor significantly predicted lower scores on dropout intentions and higher scores on satisfaction with studies. However, it did not significantly predict achievement. As in Study 1, including relations between the motivation S-factors and the academic outcomes resulted in substantial increases in explained variance (0.9% to 10.2% for academic achievement; 9.7% to 49.7% for dropout intentions; 17.0% to 36.9% for satisfaction with studies). Academic achievement was positively predicted by the S-factor for intrinsic motivation to experience stimulation and negatively predicted by S-factors for introjected regulation, external regulation, and amotivation. Dropout intentions were negatively predicted by identified regulation and positively by amotivation S-factors. Finally, satisfaction with studies was positively predicted by intrinsic motivation to know and by identified regulation S-factors, but negatively predicted by the amotivation S-factor.

Among the predictors, only the satisfaction of the need for autonomy positively predicted the G-factor, with or without the inclusion of the associations with the S-factors. When included, the associations between the predictors and the types of motivation were partly in line with our expectations. Thus, the satisfaction of the need for competence positively predicted S-factors for intrinsic motivation to accomplish and negatively introjected regulation, whereas the satisfaction of the need for autonomy positively predicted the identified regulation S-factor, and negatively the S-factors for external regulation and amotivation. Conversely, the satisfaction of the need for autonomy negatively predicted the S-factor for intrinsic motivation to accomplish, and the satisfaction of the need for need for competence positively predicted the external regulation S-factor. The satisfaction of the need for relatedness did not predict any type of motivation among this sample of graduate students.

4. General Discussion

Based on newly developed bifactor-ESEM framework, the present series of studies integrated and built on previous perspectives regarding the nature and existence of SDT’s hypothesized continuum of self-determination. More specifically, bifactor-ESEM analyses (Morin et al., 2016a, 2016b) were conducted in two studies to test the factorial structure of the AMS, a well-established measure of academic motivation. In Study 1, we used a sample of undergraduate students and investigated wellbeing outcomes (vitality and ill-being). We conducted Study 2 to replicate and extend the results from Study 1 with a new sample of graduate students, while considering a series of predictors and academic outcomes (academic achievement, dropout intentions, and satisfaction with studies). Bifactor-ESEM simultaneously takes into account the construct-relevant psychometric multidimensionality present in AMS ratings due to the presence of conceptually-related (Guay et al., 2015; Litalien et al., 2015) and overarching (Chemolli & Gagné, 2014) constructs.

Our results, which were replicated in both studies, supported the need to incorporate cross-loadings, showing the superiority of an ESEM, versus ICM-CFA, representation of participants’ responses to the AMS. The motivation factors were all well-defined in the ESEM solution, and the estimated cross-loadings remained relatively small in comparison to the target loadings (see Howard et al., 2016, for similar results with work motivation). Consistent with the statistical research showing that ESEM tends to provide reduced, and more exact, estimates of factor correlations (Asparouhov et al., 2015), the ESEM solution resulted in substantially smaller estimates of factor correlations between AMS subscales. Importantly, this difference suggests that relying on ICM-CFA might potentially induce unnecessary multicollinearity in motivation ratings, which might affect the estimation of predictive associations between the motivation factors and other variables. This potential multicollinearity could also explain why few published studies using the AMS include all motivation subscales in predictive models, rather relying on a single RAI score (e.g., Litalien et al., 2013; Ricard & Pelletier, 2016) or two higher-order factors of autonomous and controlled motivations (e.g., Gillet et al., 2013). Perhaps more importantly, the ESEM factor correlations obtained in both studies were in line with the SDT continuum hypothesis: Stronger and positive between conceptually adjacent factors, and smaller or negative between more distal factors.

Although results did not support the adequacy of the bifactor-CFA model, they did support the superiority of the bifactor-ESEM representation of the data, highlighting the importance of explicitly acknowledging the existence of an overarching self-determination construct in the model. Furthermore, the G-factor included in this model provided a direct estimate of global levels of academic self-determination, and proved to be in line with the existence of a continuum structure of motivation specified by SDT, with factor loadings ranging from strongly positive for items tapping into more autonomous forms of motivation to moderately negative for amotivation items. Despite the extraction of the G-factor, factor loadings on the S-factors suggest that they kept some specificity, although the more controlled forms of motivation appeared to retain higher levels of specificity than the more autonomous forms of motivation. Thus, in accordance with our expectations and results from Howard et al. (2016), the bifactor-ESEM framework provides a way to achieve a disaggregation of students’ global levels of self-determined academic motivation from the specific nature of their individual motivation types.

In addition to testing alternative approaches to modelling the SDT continuum, the current studies also provided some additional information on the AMS. The AMS is among the oldest (Vallerand et al., 1989, 1992, 1993) and most widely used measure of academic motivation based on SDT, and has shown substantial predictive validity across studies. Yet, at times it also has yielded some findings that deviate from SDT hypotheses. Two of these were highlighted here. First, the present findings have shown a moderately strong correlation between the intrinsic motivation to accomplish and the introjected regulation subscales. Intrinsic motivation is not typically focused on ends or outcomes but rather on processes. Both the definition of the intrinsic motivation to accomplish construct (“engaging in an activity for the pleasure experienced when attempting task mastery”; Carbonneau et al., 2012, p. 1147) and the subscale items (e.g., “For the satisfaction I feel when I am in the process of accomplishing difficult academic activities”) suggest such a focus on processes, and research has supported the validity of this type of intrinsic motivation (see Carbonneau et al., 2012). It would thus appear that the high correlation between the introjected regulation and the intrinsic motivation to accomplish subscales may be due to the theme of accomplishment that is also present in the introjected regulation items: “completing my college degree”, “succeed in college”, and “succeed in my studies”. This shared content might help to explain why introjected regulation was found to be more strongly associated with intrinsic motivation to accomplish than with identified regulation in both the ESEM and CFA solutions. High correlations between these two types of regulation have also been found in other studies using the AMS (Barkoukis, Tsorbatzoudis, Grouios, & Sideridis 2008; Fairchild, Horst, Finney, & Barron, 2005; Guay et al., 2015; Vallerand et al., 1993). Future research is needed in order to determine if shared content is indeed responsible for this higher than expected correlation found between these two subscales and how best to address this issue.

Secondly, past research using the AMS has also found at times a stronger than expected correlation between the identified and external regulation subscales (Fairchild et al., 2005; Guay et al., 2015; Vallerand et al., 1993). Similarly, our ESEM and CFA results showed that identified regulation was more strongly associated with external regulation than with introjected regulation. We note that some items of the external regulation subscale (e.g., “in order to have a better salary later on”) reflect extrinsic aspirations (e.g., Kasser & Ryan, 1996) rather than external regulations per se (i.e., being controlled or pressured by others). Research has specifically shown that extrinsic aspirations (e.g., wanting to be financially successful) may not exclusively reflect controlled (or external) regulation (e.g., Sheldon, Ryan, Deci & Kasser, 2004). In addition, items of both the external (e.g., “Because with only a high-school degree I would not find a high-paying job later on”) and identified regulation (e.g., “Because eventually it will enable me to enter the job market in a field that I like”) subscales focus on future job issues. Thus, alternatively, shared content may explain these higher than expected correlations between the identified and external regulation subscales. Future research thus appears necessary in order to determine which of these two hypotheses is correct.

Despite these caveats, our results generally support a continuum structure underlying the types of motivation represented within the AMS. Based on current results, the AMS G-factor appears to represent a global level of academic self-determination, ranging from strongly positive for the items related to the more autonomous forms of motivation, to moderately positive for the items related to introjected regulation, to small and positive for the external regulation items, to negative for the amotivation items. The pattern of S-factor loadings shows that they provide relevant information over and above that provided by the G-factor. This pattern of loadings on the G-factor and the S-factors are aligned with results obtain by Howard et al. (2016) in the work area.

Furthermore, the bifactor-ESEM representation also provides a way to directly test the relations between all motivation factors and relevant covariates without suffering from multicollinearity, as well as to directly assess the added predictive value of the specific types of motivation over and above students’ global levels of self-determination. In this approach, the G-factor provides an explicit expression of SDT’s motivation continuum that can be used in testing this continuum’s associations with predictors and outcomes. Yet in addition, the contribution of the S-factors to these predictions can be assessed, over and above this global factor.

It should be noted that the interpretation of the S-factors differs from how one typically interprets first-order factors. Whereas the latter reflect the total covariance between a subset of items, the S-factors reflect the residual covariance between a subset of items once the shared covariance between all items (from all subsets included in the model) has been extracted and reflected by the G-factor. For instance, a S-factor of introjected regulation will provide a measure controlling for participants’ global academic self-determination level across various motivation types, whereas a first-order factor of introjected regulation will also include this global level of academic self-determination. This introjected regulation S-factor may thus reflect elements of the introjection process, but much of the self-determination-related variance central to the phenomenology of introjection has been removed. Although this approach can statistically parse sources of variance to test for the continuum and to account for unique variances of the constructs falling along it, the correlations of the separate S-factor scores with other variables must be interpreted both carefully and cautiously. Such residualized scores are not, in and of themselves, fully representative of the original construct from which they are derived so that a complete interpretation must take into account both the S- and G- components.

The importance of considering both G and S-factors is further illustrated by the examination of their relations to outcomes relative to models ignoring part of the information. Results from both studies showed that the inclusive predictive model (including the free estimation of the relations between outcomes and the G- and S- factors) was able to explain substantially more variance in the outcomes when compared to a model in which only the G-factor was allowed to predict the outcomes. As expected, the self-determination G-factor positively predicted positive outcomes (vitality and satisfaction with studies) and negatively predicted negative outcomes (ill-being and dropout intentions). However, it was not significantly associated with academic achievement. Once the effect of global academic self-determination (G-factor) was considered, specific types of regulation significantly added to these predictions in a manner that proved to be mainly in line SDT: Students presenting higher levels on the S-factors reflecting more autonomous forms of motivation were more likely to experience positive outcomes (e.g., vitality, satisfaction), whereas students with higher levels on the S-factors reflecting more controlled forms of motivations or amotivation were more likely to experience negative outcomes (ill-being, dropout intentions). Particularly noteworthy was the observation that global levels of self-determination (G-factor) did not predict academic achievement among graduate students, whereas lower levels on the introjected regulation, external regulation and amotivation S-factors, or higher levels on the intrinsic motivation to experience stimulation S-factor proved to be significantly associated with higher levels of achievement. These results show the importance of considering specific motivation types in explanatory models. Also noteworthy is that, once global levels of academic self-determination (G-factor) are accounted for, the introjected regulation S-factor positively predicted ill-being, suggesting that not controlling for global levels of self-determination might have masked the negative effects of introjected regulation in some previous studies (Gagné et al., 2015).

Relations between predictors related to the satisfaction of the needs for autonomy, competence, and relatedness, and the various AMS G- and S- factors were also assessed in Study 2. These additional results showed that only the satisfaction of the need for autonomy predicted the global level of self-determination (G-factor) among graduate students. The importance of this particular need is not surprising, as it has been posited to lie at the core of the internalization process (Deci & Ryan, 2000; Ryan et al., 2016) and appears to play a particularly important role among graduate students (Overall, Deane, & Peterson, 2011). However, relations involving the specific motivation factors once again bring additional insights on these relations. For instance, the satisfaction of the need for competence positively predicted levels on the intrinsic motivation to accomplish and external regulation S-factors, but negatively predicted the introjected regulation S-factor. As mentioned earlier, the external regulation subscale of the AMS taps into career aspirations, and more precisely into the desire to attain highly paid prestigious jobs. Because these types of jobs are likely to be competitive, high feelings of academic competence could also be associated with the endorsement of these items.

The satisfaction of the need for autonomy also predicted motivation S-factors. Three of these associations were in the expected direction (positive for the identified regulation S-factor and negative for both the external regulation and the amotivation S-factors). Surprisingly, students with higher levels of satisfaction of the need for autonomy were likely to present lower levels on the intrinsic motivation to accomplish S-factor. However, once global levels of academic self-determination were extracted from the ratings of intrinsic motivation to accomplish, it is likely that what remains in this S-factor could be more strongly related to the quest for accomplishment (which could be driven by introjection or identification), whereas the intrinsic pleasure associated to this subscale is likely to be absorbed in the global self-determination factor. These results highlight how a refined analysis of S-factors can help to pinpoint construct validity issues, in this case with the AMS, not otherwise apparent.

Overall, both studies supported the presence of a global academic self-determination continuum underlying participants’ ratings of the AMS. More importantly, the results highlighted the utility of a method allowing SDT researchers to simultaneously consider specific motivation types alongside global levels of academic self-determination.

4.1 Limitations

The current research has limitations that are worth noting. First, both studies included Canadian participants, which was also the case from the previous studies on which the present investigation was built (Chemolli & Gagné, 2014; Guay et al., 2015; Litalien et al., 2015). A thorough test of the factorial validity of the AMS, and particularly of the extent to which the current results replicate, should be conducted within a wider range of cultural contexts. Here our results were similar across studies, one focused on English-speaking undergraduates, and the other French-speaking graduates.

A second limitation is related to the sample of graduate students used in Study 2, as no information was available to identify whether they pursued doctoral or master studies. The results could have differed between these two levels of education. For instance, the need for relatedness could have played a different role across these levels, given that doctoral students appear to be particularly likely to experience social isolation (Kolmos, Kofoed, & Du, 2008).

Third, although we rely on previous studies and on a strong theoretical background to support the proposed sequence of predictors and outcomes, both studies were cross-sectional, precluding tests of the directionality of the associations. Longitudinal research is thus needed to corroborate the present results in terms of directionality. That said, our intent was to demonstrate a modeling approach to the continuum, the effects of which have been widely researched elsewhere (Ryan & Deci, 2017).

Fourth, it is important to keep in mind that the current results are based on the AMS, which is specific to the academic domain. The SDT is a much broader framework covering multiple domains of motivation (sport, work, etc.) within which measures based on this theory can be applied. There are also additional measures of academic motivation within the SDT literature (e.g., Ryan & Connell, 1989). Thus, the generalizability of the current results should be more thoroughly investigated across domains. In particular, the hierarchical model of human motivation (Vallerand, 1997) suggests that motivation should be examined across various situational (e.g., varying across specific situations), contextual (e.g., academic motivation), and global (motivational tendencies that generalize across domains) levels. Future research could look at the generalizability of our results to measures taken at each of these distinct levels of analysis.

Fifth, we assessed graduate students’ academic achievement through self-reports, which generally tends to represent, at best, a weak proxy of students’ true levels of academic achievement (Kuncel, Credé, & Thomas, 2005). However, it is noteworthy that Kuncel et al. (2005) also mentioned that self-reported grades tend to be a far better indicator of actual grades among college students with high cognitive ability, a population among which the correlation between self-reported and actual grades reaches .90, and which seems to match the characteristics of our own sample of graduate students.

4.2. Directions for Future Research

Following Morin et al.’s (2016a) recommendations and based on the results from the present research and others (e.g., Howard et al., 2016), we believe that researchers should consider a bifactor-ESEM representation of participants’ responses when employing measures that assess conceptually-related constructs assumed to also form global overarching constructs, such as the AMS. In order to compare and select the appropriate model to represent these types of measurement scales, Morin et al.’s (2016a) suggested a systematic two-step procedure, which was used to guide the current study. In a first step, whenever a multidimensional measure is assumed to tap into conceptually-related constructs, a first-order ICM-CFA model should be compared to an ESEM model to assess the presence of potential construct-relevant psychometric multidimensionality due to the conceptually-related nature of the constructs. The selection of the most appropriate model should be based on fit indices, parameter estimates, and the related theory. In particular, the observation of reduced factor correlations in the ESEM relative to the ICM-CFA model should be considered as a strong source of evidence in favor of the ESEM solution based on statistical evidence showing that ESEM will provide unbiased estimates of factor correlations irrespective of whether cross-loadings are really present in the underlying population model, whereas ICM-CFA factor correlations will be biased when cross loadings should be included in the model (Asparouhov et al., 2015). The observation of unexplainably large cross-loadings should lead to a re-assessment of the appropriateness of the items in question. However, minor cross-loadings should still be kept in the model based on evidence showing that ignoring cross loadings as low as .100 may lead to biased parameter estimates (Asparouhov et al., 2015).

The second step should then be conducted whenever the measure is assumed to tap into some type of global overarching constructs (Morin et al., 2016a). In this situation, the ICM-CFA or ESEM model retained in the first step needs to be compared to its bifactor counterpart (bifactor-CFA or bifactor-ESEM, respectively), once again based on a consideration of fit indices, parameter estimates, and theoretical expectations. Here, observing a G-factor well-defined by strong factor loadings, at least some well-defined S-factors, and possibly reduced cross-loadings in relation to the ESEM solution would support the need to rely on a bifactor solution (Morin et al., 2016a).

In the present research, as well as in previous studies (Guay et al., 2015; Howard et al., 2016; Litalien et al., 2015), the result showed that an ESEM representation of the data was necessary to achieve an optimal level of differentiation among the various latent factors representing the motivation types. Furthermore, in accordance with Howard et al.’s (2016) results, we found that the continuum representation of self-determination, central to the SDT conception of human motivation, was best captured by a bifactor representation of the data. As such, our results support the idea that a bifactor-ESEM approach provides an appropriate representation of the continuum of academic motivation proposed by SDT. A particular strength of this approach is that it provides a simultaneous assessment of the global quantity of academic self-determination (i.e., the continuum) and of the specific quality (or unique features) of the academic motivation orientation characterizing the participants over and above their global level of self-determination (the motivation types) that can be simultaneously used in more complex predictive models. As mentioned above, researchers interested in applying a bifactor-ESEM representation to SDT’s continuum should kept in mind that the interpretation of the S-factors differs from how one typically interprets first-order factors, and that the construct validity of these S-factors would benefit from further investigation (Ryan & Deci, 2017).

Despite the clear advantages of this methodological approach, some caveats remain. For instance, statistical evidence showing that bifactor and ESEM models might provide a more accurate depiction of many of the psychological constructs of interest to educational psychologists (e.g., Asparouhov et al., 2015; Marsh et al., 2014; Morin et al., 2016a) is only a first step that will need to be complemented by additional statistical developments and changes in practices. For research purposes, this simply serves to reinforce prior calls for an increased focus on latent variable models, which are not only corrected for measurement errors, but also provide a more accurate depiction of the key constructs of interest (Borsboom, 2006; Marsh & Hau, 2007). Statistical research even shows that these types of models are far less demanding than what was previously thought in terms of sample size (e.g., de Winter et al., 2009), and that even in these cases, factor scores saved from preliminary measurement models may help to preserve the underlying nature of the latent constructs (e.g., Morin et al., 2016c). However, in this context, there is currently no clear recommendations on how to proceed, reinforcing the need for further statistical research in this area.

5. Conclusion

In the current studies, we provide a methodological demonstration of the usefulness of a newly developed bifactor-ESEM framework in testing the SDT continuum hypothesis of human motivation. Using the AMS to assess motivation in undergraduate and graduate samples, our results supported the presence of a continuum structure of motivation and the utility of the proposed framework. In particular, the analyses yielded a direct, and latent, representation of this continuum for use in predictive analyses. These results extend previous research of the continuum structure, as prior studies had used techniques that only partially assessed and controlled for both sources of psychometric multidimensionality likely to be present in the AMS (Chemolli & Gagné, 2014; Guay et al., 2015).

The bifactor-ESEM framework appears particularly well-suited to validate the multidimensional structure of motivation proposed by SDT. Yielding a precise and reliable latent indicator of the general level of self-determination (G-factor) while allowing for the simultaneous consideration of the contribution of additional specific (S-) factors, this strategy may provide an alternative to higher order factor approaches commonly used in SDT. Applying bifactor-ESEM to other measures of self-regulation and in different domains of motivation would thus be an important next step.

References

Ahmed,W., & Bruinsma, M. (2006). A structural model of self-concept, autonomous motivation and academic performance in cross-cultural perspective. Electronic Journal of Research in Educational Psychology, 4, 551-576.

Asparouhov, T., & Muthén, B. (2009). Exploratory structural equation modeling. Structural Equation Modeling, 16, 397-438.

Asparouhov, T., Muthén, B., & Morin, A.J.S. (2015). Bayesian Structural equation modeling with cross-loadings and residual covariances: Comments on Stromeyer et al. Journal of Management, 41, 1561-1577.

Barkoukis, V., Tsorbatzoudis, H., Grouios, G., & Sideridis, G. (2008). The assessment of intrinsic and extrinsic motivation and amotivation: Validity and reliability of the Greek version of the Academic Motivation Scale. Assessment in Education: Principles, Policy & Practice, 15, 39-55.

Black, A. E., & Deci, E. L. (2000). The effects of instructors' autonomy support and students' autonomous motivation on learning organic chemistry: A self-determination theory perspective. Science Education, 84, 740-756.

Blais, M. R., Vallerand, R. J., & Lachance, L. (1990). L’échelle des Perceptions d’Autonomie dans les Domaines de Vie [The Perceived Autonomy in Life Domains Scale]. Unpublished manuscript, University of Quebec at Montreal, Montreal, Quebec, Canada. Retrieved from http://www.er.uqam.ca/nobel/r26710/LRCS/scales/epadv.doc

Borsboom, D. (2006). The attack of the psychometricians. Psychometrika, 71, 425-440.

Browne, M. (2001) An overview of analytic rotation in exploratory factor analysis. Multivariate Behavioral Research, 36, 111-150.

Cox, A., Ullrich-French, S., Madonia, J., & Witty, K. (2011). Social physique anxiety in physical education: Social contextual factors and links to motivation and behavior. Psychology of Sport and Exercise, 12, 555-562.

Carbonneau, N., Vallerand, R. J., & Lafrenière, M.-A. K. (2012). Toward a tripartite model of intrinsic motivation. Journal of Personality, 80, 1147-1178.

Chemolli, E., & Gagné, M. (2014). Evidence against the continuum structure underlying motivation measures derived from self-determination theory. Psychological Assessment, 26, 575-585.

Deci, E. L., & Ryan, R. M. (1985). Intrinsic motivation and self-determination in human behavior. New York: Plenum Press.

Deci, E. L., & Ryan, R. M. (2000). The “what” and “why” of goal pursuits: Human needs and the self-determination of behavior. Psychological Inquiry, 11, 227-268.

Deci, E. L., & Ryan, R. M. (2008). Self-determination theory: A macrotheory of human motivation, development, and health. Canadian Psychology, 49, 182-185.

Deci, E. L., & Ryan, R. M. (2016). Optimizing students’ motivation in the era of testing and pressure: A self-determination theory perspective. In W. C. Liu, J. C. K. Wang, & R. M. Ryan (Eds.), Building autonomous learners: Perspectives from research and practice using self-determination theory (pp. 9–29). Singapore: Springer Singapore.

Deci, E. L., & Ryan, R. M. (2012a). Motivation, personality, and development within embedded social contexts: An overview of self-determination theory. In R. M. Ryan (Ed.), The Oxford handbook of human motivation (pp. 85–107). New York, NY: Oxford University Press.

Deci, E. L., & Ryan, R. M. (2012b). Self-determination theory. In P. A. M. Van Lange, A. W. Kruglanski, & E. T. Higgins (Eds.), Handbook of theories of social psychology (pp. 416-437). Thousand Oaks, CA: Sage.

Deci, E. L., Ryan, R. M., & Guay, F. (2013). Self-determination theory and actualization of human potential. In D. McInerney, H. Marsh, R. Craven, & F. Guay (Eds.), Theory driving research: New wave perspectives on self-processes and human development (pp. 109-133). Charlotte, NC: Information Age.

de Winter, J.C.F., Dodou, D., & Wieringa, P.A. (2009). Explorarory factor analysis with mall sample sizes. Multivariate Behavioral Resrearch, 44, 147-181.

Gagné, M., Forest, J., Vansteenkiste, M., Crevier-Braud, L., Van den Broeck, A., Kristin-Aspeli, A., … Westbye, C. (2015). The multidimensional work motivation scale: Validation in seven languages and nine countries. Journal of Work and Organizational Psychology, 24, 178-196.

Gillet, N., Gagné, M, Sauvagère, S., & Fouquereau, E. (2013). Predicting employees’ satisfaction and turnover intentions using self-determination theory. European Journal of Work and Organizational Psychology, 22, 450-460.

Gignac, G. E. (2016). The higher-order model imposes a proportionality constraint: That is why the bifactor model tends to fit better. Intelligence, 55, 57-68.

Goldberg, D. P. & Hillier, V. F. (1979) A scaled version of the General Health Questionnaire. Psychological Medicine, 9, 139-145.

Guay, F., Lessard, V., & Dubois, P. (2016). How can we create better learning contexts for children? Promoting students’ autonomous motivation as a way to foster enhanced educational outcomes. In W. C. Liu, J. C. K. Wang, & R. M. Ryan (Eds.), Building autonomous learners: Perspectives from research and practice using self-determination theory (pp. 83–106). Singapore: Springer Singapore.

Guay, F., Morin, A. J. S., Litalien, D., Valois, P., & Vallerand, R. J. (2015). Application of exploratory structural equation modeling to evaluate the academic motivation scale. Journal of Experimental Education, 83, 51-82.

Guay, F., Ratelle, C., Roy, A., & Litalien, D. (2010). Academic self-concept, autonomous academic motivation, and academic achievement: Mediating and additive effects. Learning and Individual Differences, 20, 644-653.

Fairchild, A. J., Horst, S. J., Finney, S. J., & Barron, K. E. (2005). Evaluating existing and new validity evidence for the Academic Motivation Scale. Contemporary Educational Psychology, 30, 331-358.

Howard, J. L., Gagné, M., Morin, A. J. S., & Forest, J. (2016). Using bifactor exploratory structural equation modeling to test for a continuum structure of motivation. Journal of Management. Advance online publication. doi: 10.1177/0149206316645653

Hu, L., & Bentler, P. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6, 1-55.

Jöreskog, K. G. (1970). Estimation and testing of simplex models. British Journal of Mathematical and Statistical Psychology, 23, 121-145.

Kolmos, A., Kofoed, L. B., & Du, X. Y. (2008). PhD students' work conditions and study environment in university- and industry-based PhD programmes. European Journal of Engineering Education, 33, 539-550.

Kasser, T., & Ryan, R. M. (1996). Further examining the American dream: Differential correlates of intrinsic and extrinsic goals. Personality and Social Psychology Bulletin, 22, 280-287.

Litalien, D., & Guay, F. (2015). Dropout intentions in PhD studies: A comprehensive model based on interpersonal relationships and motivational resources. Contemporary Educational Psychology, 41, 218-231.

Litalien, D., Guay, F., & Morin, A. J. S. (2015). Motivation for PhD studies: Scale development and validation. Learning and Individual Differences, 41, 1-13.

Litalien, D., Lüdtke, O., Parker, P., & Trautwein, U. (2013). Different pathways, same effects: Autonomous goal regulation is associated with subjective well-being during the post-school transition. Motivation and Emotion, 37, 444-456.

Losier, G.F. (1994). Une analyse motivationnelle de la persévérance aux études avancées [analysis of graduate student persistence]. Unpublished doctoral dissertation, Université du Québec à Montréal, Montréal.

Losier, G. F., Vallerand, R. J., & Blais, M. R. (1993). Construction et validation de l'Échelle des Perceptions de Compétence dans les Domaines de Vie. Science et comportement, 23, 1-16.

Marsh, H. W., & Hau, K. T. (2007). Applications of latent-variable models in educational psychology: The need for methodological-substantive synergies. Contemporary Educational Psychology, 32, 151-170.

Marsh, H., Hau, K.-T., & Grayson, D. (2005). Goodness of fit evaluation in structural equation modeling. In A. Maydeu-Olivares & J. McArdle (Eds.), Contemporary psychometrics. A Festschrift for Roderick P. McDonald. Mahwah NJ: Erlbaum.

Marsh, H., Hau, K.-T., & Wen, Z. (2004). In search of golden rules: Comment on hypothesis testing approaches to cutoff values for fit indexes and dangers in overgeneralizing Hu & Bentler’s (1999). Structural Equation Modeling, 11, 320-341.

Marsh, H., Morin, A., Parker, P., & Kaur, G. (2014). Exploratory structural equat