Linear Structural Relations (LISREL and R) - The Personality Project
reliablewebinfo.files.wordpress.com · Web viewThe statistical model that was tested was tested...
Transcript of reliablewebinfo.files.wordpress.com · Web viewThe statistical model that was tested was tested...
Attributes of Elite Performers
Stacy Gnacinski, Allie Nickel, & Brian Pochinski
UW-Milwaukee
1
Conceptual Model
For hundreds of years, researchers have been tasked with the challenge of identifying the
best and most reliable predictors of success in achievement domains. Achievement in one such
domain, sport, has been particularly difficult to study as researchers have recognized the complex
and multidimensional nature of sport talent and achievement. That said, a robust body of
psychology literature exists to support theoretical constructs which may be related to success in elite
sport. Specifically, the conceptual model under investigation emerged from an extensive review of
the talent identification and development in sport literature (Abbott & Collins, 2004). Many
scholars have purported that an athlete’s ability to self-regulate his or her behaviors is linked to
success in sport (Collins & Durand-Bush, 2014). Self-regulation is often described as a cyclical
process that includes behaviors such as self-evaluation (i.e., the assessment of task correctness or
accuracy immediately after the task), self-reflection (i.e., desire to improve skills and use of
evaluation to inform future performance), and effort (e.g., concentration and dedication to tasks,
indicating hard work). These three sub –components of self-regulation can be measured via the
Self-Regulation Scale (Toering et al., 2011).
It has also been suggested that psychological constructs such as grit, challenge, commitment,
and task orientation are linked to success in sport and other achievement domains (e.g., academics,
military academies, etc.). Grit, defined as passion and perseverance for long-term goals, emerged in
the literature as an underrepresented component of long-term success as a result of traditional
emphasis on the influence of intelligence on long-term success (Duckworth et al., 2007).
Challenge, or the view that challenges are opportunities to display skills rather than a threat or
exposure of weaknesses, has been long recognized in the literature as an important mental strategy
which facilitates peak performance during high-pressure situations and across long periods of
training (Clough et al., 2007). Commitment, or the continued and deliberate effort toward goal
2
achievement, has been long cited in the sport literature as a critical component to elite performance
success (Clough et al., 2002). Finally, task orientation, a component of achievement goal theory
defined as a continuous focus on the process of skill mastery, has been extensively studied and
linked to goal achievement (Duda, 2005). Taken together, the hypothetical overlap between the
operational definitions of these four constructs is ‘passion to achieve’.
Figure 1. Conceptual Model
Purpose. The purpose of this project was to determine the theoretical relationship between
self-regulation (i.e., self-evaluation, self-reflection, effort) and passion for goal achievement (i.e.,
grit, challenge, commitment, task orientation). The conceptual model to be tested is depicted in
Figure 1. The proposed paths leading to self-regulation (i.e., self-evaluation, self-reflection, and
effort) are derived directly from self-regulation theories. The self-evaluation, self-reflection, and
effort subscales are not thought to be theoretically related, thus, no covariance arcs have been
included in the conceptual model. The paths leading to passion to achieve (i.e., grit, challenge,
commitment, and task) are derived from the larger body of sport psychology literature. Again, no
theoretical or empirical evidence exists to support the inclusion of covariance arcs between these
indicators. As such, no covariance arcs were included in the conceptual model depicted above. The
3
Task
Commitment
Grit
ChallengePassion to Achieve
Effort
Self-Reflection
Self-Evaluation
Self-Regulation
main path under investigation in the current project is the relationship between the latent variables
of ‘self-regulation’ and ‘passion to achieve’. These were included as latent variables as there are no
ways to directly measure these psychological constructs. Since there are no measures in existence
to assess ‘self-regulation’ and ‘passion to achieve’.
Statistical Model
The statistical model that was tested was tested using LISREL 9.1 Student Edition. The
matrices that were included in the statistical model were the Phi matrix, the Lambda X matrix, the
Theta Delta matrix, the Psi matrix, the Lambda Y matrix, the Theta Epsilion matrix, and the
Gamma matrix. Below is a detailed outline of which parameters within each of the matrices were
freely estimated and/or fixed. In the current model, there are 17 parameters to be estimated and 30
unique elements to the model. As such, there are 13 degrees of freedom which is confirmed in the
LISREL output. Per the identification rule of confirmatory factor analysis (CFA), the unit has
been defined for each statistical model by fixing the estimated parameters of Lambda X11 and
Lambda Y11 to equal 1.0. Also, the degrees of freedom are not less than zero, and thus the
identification rule has not been violated.
4
Task
Commitment
Grit
ChallengePassion to Achieve
Effort
Self-Reflection
Self-Evaluation
Self-Regulation
Phi Matrix Psi Matrix Gamma Matrix
φ11 ψ11 γ11
Lambda X Matrix Lambda Y Matrix
1.0 1.00 λ21 λ21
λ31 λ31
λ41
Theta Delta Matrix Theta Epsilon Matrix
Θδ1 Θδ1
Θδ2 Θδ2
Θδ3 Θδ3
Θδ4
Data
Self-regulation. To assess self-evaluation, self-reflection, and effort, the Self-Regulation
Scale (SR-S) was administered. The validity of this scale has been confirmed in previous research
(Toering et al., 2011) and the reliability of the subscales of interest have been confirmed via in-
house Cronbach’s alpha reliability calculations (αs = .81-.87). All subscales are scored on a 5-point
Likert-type scale. The self-evaluation subscale consists of 8 total items, the self-reflection subscale
consists of 9 total items, and the effort subscale consists of 9 total items. The average of all items
was scored and reported.
Passion to achieve. To assess passion to achieve, several separate psychological surveys
were administered. These surveys will be reviewed in detail below.
Grit. To assess grit, the 12-item Grit Scale was administered. The validity of the scale has
been confirmed in previous research (Duckworth et al., 2007). The reliability of the scale has been
confirmed via an in-house Cronbach’s alpha reliability calculation (α = .81). This subscale is
scored on a 5-point Likert-type scale and consists of 12 total items. The average of all items was
scored and reported.
5
Commitment and challenge. To assess commitment and challenge, the Mental Toughness-
48 (MT-48) survey was administered. The validity of this measure has been confirmed in previous
research (Clough et al., 2002). While this mental toughness assessments include several subscales,
only the commitment and challenge subscales were administered for the current investigation. The
reliability of these scales has been confirmed via in-house Cronbach’s alpha reliability calculations
(α = .74 - .77). These subscales are scored on a 5-point Likert-type scale. The commitment
subscale consists of 11 total items, and the challenge subscale consists of 8 total items. The average
of all items was scored and reported.
Task orientation. To assess task orientation, the task subscale of the Task and Ego
Orientation Sport Questionnaire (TEOSQ) was administered. The validity of this scale has been
confirmed in previous research (Duda, 2005). The reliability of the task orientation subscale has
been confirmed via an in-house Cronbach’s reliability calculation (α = .84). This scale is scored on
a 5-point Likert type scale and the mean of all items was scored and reported.
Participants. Participants in the current study included elite and pre-elite athletes aspiring
for professional or World Cup/Olympic success in their careers. At the time of data collection(s),
all participants were still participating in their respective sports. Sports represented within the
sample include: aerial skiing, professional ice-hockey, golf, tennis, moguls, slopestyle, soccer, and
football. A total of 107 participants were included in the current investigation.
Missing Data. This project is part of a much larger longitudinal applied project within one
of the author’s advisor’s research laboratory. As such, these data have been collected over time
with clients or potential clients of the advisor. Since these data were not collected with the
intentions of conducting research or manuscript publication, all items were manipulated to be
forced-response. This ensured that each athlete had a full and complete set of data in which we
could interpret and provide developmental feedback. As such, there are no missing data in the
6
current data set. If missing data were present, we would assume the data to be missing at random,
as missing completely at random almost never happens in true applied research and there is no
reason to believe data would be missing not at random (responses omitted are related to the item).
Normality Assumption. All variables, with the exception of the self-reflection subscale,
demonstrated normality in the distribution of data. The skewness and kurtosis statistics were
reported in Table 1 below the means and standard deviations of all subscales. The histograms
displayed in Figure 2 further confirm the normality of the univariate distributions, thereby
supporting the assumption of multivariate normality.
Variable 1 2 3 4 5 6 7Communication 1.000
Challenge 0.575 1.000Task 0.417 0.534 1.000Grit 0.701 0.543 0.358 1.000
Self-Evaluation 0.343 0.447 0.261 0.423 1.000Self-Reflection 0.075 0.252 0.137 0.134 0.206 1.000
Effort 0.598 0.553 0.547 0.665 0.558 0.193 1.000M 4.12 3.96 4.40 3.93 4.43 4.13 3.35SD 0.41 0.43 0.42 0.54 0.67 0.67 0.48
Skewness 0.044 0.426 0.419 0.411 -0.093 -2.145 0.484Kurtosis -0.647 1.606 0.095 -0.205 0.190 7.231 -0.531
Table 1. Scale correlation and descriptive statistics.
7
Figure 2. Histograms: Univariate data.
8
Results
Method of Estimation. The method of estimation selected for the analysis was maximum
likelihood. The freely estimated parameters were the observed variance of self-regulation (φ11), the
error in the passion to achieve and maintain performance excellence (ψ11), and the errors in
measurement for commitment to achieve (θε1), response to challenge (θε2), task orientation (θε3), grit
(θε4), self-evaluation (θδ1), self-reflection (θδ2), and effort (θδ3). The paths that were freely estimated
were from passion to achieve and maintain performance excellence to response to challenge (λy2,1),
from passion to achieve and maintain performance excellence to task orientation (λ y3,1), from
passion to achieve and maintain performance excellence to grit (λy4,1), from self-regulation to self-
reflection (λx2,1), and from self-regulation to effort (λx3,1). The path from self-regulation to passion
to achieve and maintain performance excellence was also freely estimated (γ11). The variances for
the paths from passion to achieve and maintain performance excellence to commitment to achieve
and self-regulation to self-evaluation were set to zero (used as marker variables). The
characteristics of the data do not question the appropriateness of maximum likelihood as an
estimation method.
Fit Criteria. The tests used to determine model fit were as follows: chi-square test (χ2),
Non-normed fit index (NNFI), Comparative fit index (CFI), Root Mean Square Error of
Approximation (RMSEA), the and goodness of fit index (GFI). Definitions for each test will be
laid out in turn in the following section. These tests were selected as indicators of fit because one
tests the null hypothesis (χ2), two are incremental (NNFI and CFI), one is based on the residuals
matrix (RMSEA), and one is an absolute fit index (GFI). Together, these tests gave us an idea of
how well the suggested model fit the data.
The chi-square test χ2: The chi-square test is used to test if the observed covariance matrix,
from the data, fits an estimated covariance matrix, calculated from the model. Here, accepting the
9
null hypothesis suggests that the estimated matrix and the observed matrix were highly similar (p’s
> 0.05). The measure is susceptible to type I error if there are small sample sizes (rejecting the null
hypothesis when it is true).
Non-normed fit index NNFI: Incremental measure of goodness of fit for a statistical model,
which takes into account the size of the correlations in the data and the number of parameters in the
model. Values greater than .9 imply that the model has good fit.
Comparative fit index CFI: Incremental measure of goodness of fit for a statistical model,
which compares the relative improvement of the suggested model over the baseline model. Values
greater than .9 imply that the model has good fit.
Root Mean Square Error of Approximation RMSEA: Measures error of approximation,
where values closer to zero indicate better fit (i.e., less error in the approximation). Values less than
.10 imply that the model has good fit. The confidence interval should be less than or include 0.05.
Goodness of fit index GFI: Calculates the proportion of variance that is accounted for by the
estimated population covariance. Shows how closely the model comes to replicating the observed
covariance matrix. Values greater than .95 imply that the model has good fit.
Indicators of Overall Fit. The chi-squared value obtained from the analysis was χ2(13) =
31.669, p = 0.0027, N = 107. The significant p value suggests that the model fit was poor.
Specifically, the estimated covariance matrix did not fit the observed covariance matrix well,
suggesting that the model is indeed a poor way to explain the relationships among our variables.
Although the chi-squared value suggested poor model fit, the incremental fit indices suggested that
the model fit was not necessarily poor. Small sample sizes could have led to overestimated chi-
squared values. The NNFI for the model was 0.928, which was greater than the cutoff value of .9,
suggesting good fit. The CFI for the model was 0.956, which was greater than the cutoff value
of .9, suggesting good fit. Some of the other fit indices were close to, but greater (or less) than the
10
required minimums (maximums) (RMSEA = 0.116 (0.0649; 0.168), should be less than .10; and
GFI = 0.925, should be greater than 0.95). Overall, the fit indices suggest that the model fit is
acceptable but not great.
Model Comparisons. The reported model was compared to one other a priori model, which
will be discussed further in the alternative models section. The current model was found to be
superior to the alternative model as indicated by the chi-square difference test (Δχ2), the Akaike
information criterion (AIC), and the Bayesian information criterion (BIC). The Δχ2 was significant
(Δχ2 = 46.846, df = 19), suggesting that the more parsimonious model had a better fit than the
alternative model. When ΔAIC was calculated the difference was large (ΔAIC = 256.617)
suggesting that there is essentially no empirical evidence for the alternative model over the
suggested, and more parsimonious model. When ΔBIC was calculated the difference was large
(ΔBIC = 277.99) suggesting that there is very strong evidence against the alternative model relative
to the suggested model. Taken together, the absolute fit statistics suggest that the estimated model
is far superior to the alternative model in its ability to account for the relationships among the data.
Comparisons between absolute fit indices provided a good indication of which model provided
superior fit because the alternative model was not significantly more complex than the estimated
model.
Parameter Estimates. The standardized parameter estimates are reported below. The
unstandardized estimates will be included, along with the standard errors, in parentheses following
the numerical description of the standardized estimate. Estimates are considered significant if the
critical ratio value is greater than z = 1.96.
φ11 = 1.00 (0.36; -0.112) Observed variance in self-regulation accounted for by the model.
ψ11 = 0.265 (0.167; -0.07) Error in estimation of Passion to achieve and maintain
performance excellence; proportion of observed variance in Passion to achieve and maintain
11
performance excellence not accounted for by the model; model explains 73.5% of the total variance
in Passion to achieve and maintain performance excellence. Model explains a significant
proportion of the variance in passion to achieve and maintain performance excellence (z = 2.392).
θε1 = 0.367 (0.367; 0.067) Error in estimation of commitment to achieve; proportion of
observed variance in commitment to achieve not accounted for by the model; model explains 63.3%
of the total variance in commitment to achieve. Model explains a significant proportion of the
variance in commitment to achieve (z = 5.471).
θε2 = 0.483 (0.483; 0.078) Error in estimation of response to challenge; proportion of
observed variance in response to challenge not accounted for by the model; model explains 51.7%
of the total variance in response to challenge. Model explains a significant proportion of the
variance in response to challenge (z = 6.201).
θε3 = 0.668 (0.668; 0.098) Error in estimation of task orientation; proportion of observed
variance in task orientation not accounted for by the model; model explains 33.2% of the total
variance in task orientation. Model explains a significant proportion of the variance in task
orientation (z = 6.824).
θε4 = 0.341 (0.341; 0.065) Error in estimation of grit; proportion of observed variance in grit
not accounted for by the model; model explains 65.9% of the total variance in grit. Model explains a
significant proportion of the variance in grit (z = 5.240).
θδ1 = 0.640 (0.640; 0.096) Error in estimation of self-evaluation; proportion of observed
variance in self-evaluation not accounted for by the model; model explains 36% of the total
variance in self-evaluation. Model explains a significant proportion of the variance in self-
evaluation (z = 6.645).
θδ2 = 0.952 (0.952; 0.130) Error in estimation of self-reflection; proportion of observed
variance in self-reflection not accounted for by the model; model explains 4.8% of the total variance
12
in self-reflection. Model explains a significant proportion of the variance in self-reflection (z =
7.305).
θδ3 = 0.135 (0.135; 0.098) Error in estimation of effort; proportion of observed variance in
effort not accounted for by the model; model explains 86.5% of the total variance in effort. Model
does not explain a significant proportion of the variance in effort (z = 1.371).
λy1,1 = 0.796 (1.00; variance set at 1, used as marker variable) Relationship between passion
to achieve and maintain performance excellence and commitment to achieve. A one standard unit
increase in passion to achieve and maintain performance excellence predicts a 0.796 unit increase in
commitment to achievement. Passion to achieve and maintain performance excellence was a
significant predictor of commitment to achievement.
λy2,1 = 0.719 (0.904; -.119) Relationship between passion to achieve and maintain
performance excellence and response to challenge. A one standard unit increase in passion to
achieve and maintain performance excellence predicts a 0.719 unit increase in response to
challenge. Passion to achieve and maintain performance excellence was a significant predictor of
response to challenge (z = 7.61).
λy3,1 = 0.576 (0.724; -0.122) Relationship between passion to achieve and maintain
performance excellence and task orientation. A one standard unit increase in passion to achieve and
maintain performance excellence predicts a 0.576 unit increase in task orientation. Passion to
achieve and maintain performance excellence was a significant predictor of task orientation (z =
5.919).
λy4,1 = 0.812 (1.020; -0.117) Relationship between passion to achieve and maintain
performance excellence and grit. A one standard unit increase in passion to achieve and maintain
performance excellence predicts a 0.812 unit increase in grit. Passion to achieve and maintain
performance excellence was a significant predictor of grit (z = 8.718).
13
λx1,1 = 0.600 (1.00; variance set at 1, used as marker variable) Relationship between self-
regulation and self-evaluation. A one standard unit increase in self-regulation predicts a 0.600 unit
increase in self-evaluation. Self-regulation was a significant predictor of self-evaluation.
λx2,1 = 0.218 (0.364; -0.173) Relationship between self-regulation and self-reflection. A one
standard unit increase in self-regulation predicts a 0.218 unit increase in self-reflection. Self-
regulation was a significant predictor of self-reflection (z = 2.106).
λx3,1 = 0.930 (1.551; -0.26) Relationship between self-regulation and effort. A one standard
unit increase in self-regulation predicts a 0.930 unit increase in effort. Self-regulation was a
significant predictor of effort (z = 5.961).
γ11 = 0.858 (1.137; -0.198) Relationship between self-regulation and passion to achieve and
maintain performance excellence. A one standard unit increase in self-regulation predicts a 0.858
unit increase in passion to achieve and maintain performance excellence. Self-regulation was a
significant predictor of passion to achieve and maintain performance excellence (z = 5.755).
The estimates returned by the analysis were all logical or plausible, including the variances.
For example, even though the model accounts for a numerically large proportion of the variance in
effort, it isn’t a significant percentage of the variance. This seems logical, as effort is quite a
broadly defined variable and it is unlikely that the handful of variables selected for out model could
possibly explain all of the variance in something like effort. Our beta values were also logical in
that significant increases in our predictor variables led to significant increases in the predicted
variables. This was expected based on the previously discussed theory.
Alternative Models
Although it is preferred that alternative models are specified a priori, SEM does not always
allow such a luxury. While our alternative model did have some theoretical support, the data proved
to be of poor fit. Thus, we made post hoc modifications in an attempt to find a better fitting model. .
14
Fit statistics for both the target model and the alternative model can be found in Table 2. Because
the development of this model is still in the early stages, we decided to choose the better fitting
model for our criteria for choosing our model.
The better fitting model turned out to be the target model. While the chi-square was
significant for both the target model (df = 13, χ2 = 31.669, p = 0.0027) and the alternative model (df
= 32, χ2 = 78.515, p = 0.0000), the target model was less significant. Thus, while both models were
not good, the target model was not as bad. Furthermore, the CFI value was better for the target
model (CFI = 0.956) than the alternative model (CFI = 0.928). A CFI value greater than 0.950 is
considered to be a good fit, thus, the target model is the better fitting model. Similarly, the target
model (NNFI = 0.928) had a superior NNFI compared to the alternative model (NNFI = 0.899).
While an NNFI of 0.950 is preferred for a good fitting model, an NNFI of 0.900 is deemed a fair fit.
The RMSEA were close to good fit for both models, but neither was good. The target model was
0.116 (0.0649; 0.168) compared to a RMSEA of 0.117 (0.0842; 0.149) for the alternative model.
The GFI of the target model (GFI = 0.925) was superior to the alternative model (GFI = 0.884).
Thus, based on numerous fit indices, target model appears to be a better fit than the alternative
model.
Target model Goodness of Fit Statistics
Degrees of Freedom for (C1)-(C2) 13 Maximum Likelihood Ratio Chi-Square (C1) 31.669 (P = 0.0027) Browne's (1984) ADF Chi-Square (C2_NT) 30.407 (P = 0.0041) Estimated Non-centrality Parameter (NCP) 18.669 90 Percent Confidence Interval for NCP (5.862; 39.154) Minimum Fit Function Value 0.296 Population Discrepancy Function Value (F0) 0.174 90 Percent Confidence Interval for F0 (0.0548; 0.366) Root Mean Square Error of Approximation (RMSEA) 0.116 90 Percent Confidence Interval for RMSEA (0.0649; 0.168) P-Value for Test of Close Fit (RMSEA < 0.05) 0.0207
15
Expected Cross-Validation Index (ECVI) 0.576 90 Percent Confidence Interval for ECVI (0.457; 0.768) ECVI for Saturated Model 0.523 ECVI for Independence Model 4.301 Chi-Square for Independence Model (21 df) 446.227 Normed Fit Index (NFI) 0.928 Non-Normed Fit Index (NNFI) 0.928 Parsimony Normed Fit Index (PNFI) 0.575 Comparative Fit Index (CFI) 0.956 Incremental Fit Index (IFI) 0.956 Relative Fit Index (RFI) 0.884 Critical N (CN) 93.680 Root Mean Square Residual (RMR) 0.0536 Standardized RMR 0.0536 Goodness of Fit Index (GFI) 0.925 Adjusted Goodness of Fit Index (AGFI) 0.838 Parsimony Goodness of Fit Index (PGFI) 0.429
Alternative model Goodness of Fit Statistics
Degrees of Freedom for (C1)-(C2) 32 Maximum Likelihood Ratio Chi-Square (C1) 78.515 (P = 0.0000) Browne's (1984) ADF Chi-Square (C2_NT) 69.897 (P = 0.0001) Estimated Non-centrality Parameter (NCP) 46.515 90 Percent Confidence Interval for NCP (24.261; 76.461) Minimum Fit Function Value 0.734 Population Discrepancy Function Value (F0) 0.435 90 Percent Confidence Interval for F0 (0.227; 0.715) Root Mean Square Error of Approximation (RMSEA) 0.117 90 Percent Confidence Interval for RMSEA (0.0842; 0.149) P-Value for Test of Close Fit (RMSEA < 0.05) 0.000871 Expected Cross-Validation Index (ECVI) 1.164 90 Percent Confidence Interval for ECVI (0.956; 1.444) ECVI for Saturated Model 1.028 ECVI for Independence Model 6.743 Chi-Square for Independence Model (45 df) 701.554 Normed Fit Index (NFI) 0.887 Non-Normed Fit Index (NNFI) 0.899 Parsimony Normed Fit Index (PNFI) 0.631 Comparative Fit Index (CFI) 0.928
16
Incremental Fit Index (IFI) 0.930 Relative Fit Index (RFI) 0.841 Critical N (CN) 73.214 Root Mean Square Residual (RMR) 0.0647 Standardized RMR 0.0647 Goodness of Fit Index (GFI) 0.884 Adjusted Goodness of Fit Index (AGFI) 0.801 Parsimony Goodness of Fit Index (PGFI) 0.515
Table 2: fit statistics for the target model and the alternative model.
As previously mentioned, our a priori model turned out to be of bad fit. Thus, we were
forced to make post hoc modifications to our model. We chose to make our a priori model more
parsimonious. SEM should always pay special attention to the more parsimonious models. After
making some minor adjustments to simplify our model, we found that our modified target model
was a better. Thus, we feel that our post hoc modifications are justified based on parsimony and the
model fit statistics. Based on the extensive literature on our topic, we do not believe equivalent
models will be an issue.
Statistical Power
Cross-validation
Because post-hoc modifications were made, cross validating the data is crucial. The EVCI
allows for cross validation with the lower value being the better model. The EVCI of the target
model is 0.576 (0.457; 0.768) and the ECVI of the alternative model is 1.164 (0.956; 1.444). Thus,
when examining the target model and alternative model, the target model is better.
Directionality
The results of the current investigation support a model where the latent variable self-
regulation predicts the latent variable passion to achieve. Self-regulation consists of the observed
17
variables self-evaluation, self-reflection, and effort. Passion to achieve consists of the observed
variables grit, challenge, commitment, and task. The observed variables for self-regulation are
strongly supported by self-regulation theory. The observed variables for Passion to achieve come
from the sport psychology literature.
Limitations
While the current investigation supports a model where self-regulation predicts passion to
achieve, there are important limitations that must be discussed. The sample size in the current
investigation was only 107. Larger sample sizes are often preferred in SEM for more accurate
modeling. For example, the TLI/NNFI performs poorly with small sample sizes (Hoyle and Panter,
1995). Thus, interpreting model fit based on the TLI/NNFI should be done with caution. However,
the IFI performs better with small sample sizes than the TLI/NNFI. The IFI for the target model is
0.956 whereas the IFI for the alternative model is 0.930. Thus, again the fit statistics support the
target model over the alternative model.
Another issue with the small sample size relates to our post hoc modifications. It has been
suggested that post hoc modifications should not be made when sample sizes are below 400 (Hoyle
and Panter, 1995). Adding participants in future investigations could help overcome the issue of our
small sample size and justify our post hoc modifications. The addition of new participants could
also help increase power.
While most of the data did not violate the assumption of normality, the data for self-
reflection and challenge was non-normal. The data for self-reflection was negatively skewed (-
2.145) and had a positive kurtosis (7.231). Challenge also has a positive kurtosis of 1.606. Thus,
there is the possibility that that non-normality had some impact on the results.
18
References
Hoyle, R. H., & Panter, A. T. (1995). Writing about structural equation models. In R. H. Hoyle
(Ed.), Structural equation modeling: Concepts, issues, and applications (pp. 158-176).
Thousand Oaks, CA: Sage Publications.
19