MONTH YEAR
Final Report: Evaluation of
Anchorage School District’s
Second Order Change Project
Subgrant to the Cook Inlet Tribal
Council’s Strategic Prevention
Framework State/Tribal Incentive Grant
Kimberly Kendziora, Ph.D.
Evaluation Project Director
Principal Research Analyst
Education Program
American Institutes for Research Final Report: Evaluation of ASD’s Second Order Change Project—i
Contents
Page
List of Exhibits ........................................................................................................................ iii
Executive Summary ................................................................................................................. iv
Acknowledgements ................................................................................................................... vi
Introduction............................................................................................................................... 1
Context ............................................................................................................................ 1
Second Order Change Project .......................................................................................... 1
Methods ..................................................................................................................................... 4
Participants ...................................................................................................................... 4
Training for Teachers, Principals, and Other Staff ........................................................... 4
Resolving Conflict Creatively Program ................................................................ 4
National Coalition Building Institute .................................................................... 5
Life Space Crisis Intervention .............................................................................. 5
Response Ability Pathways .................................................................................. 5
Conscious Discipline ............................................................................................ 5
Discipline with Dignity ........................................................................................ 6
Breaking Ranks in the Middle/Middle School Matters.......................................... 6
Additional Classes ............................................................................................... 6
Measures ......................................................................................................................... 6
Data Analysis .................................................................................................................. 9
Results ..................................................................................................................................... 10
Effectiveness of Training ............................................................................................... 10
Interaction Style ............................................................................................................ 10
Authoritarianism ............................................................................................................ 13
Mindfulness ................................................................................................................... 14
Student School Climate and Connectedness Survey ....................................................... 15
Participants ........................................................................................................ 15
Student-Reported Respectful Climate ................................................................. 16
American Institutes for Research Final Report: Evaluation of ASD’s Second Order Change Project—ii
Student-Reported Caring Adults ......................................................................... 17
Student-Reported Risk Behaviors ....................................................................... 21
Staff School Climate and Connectedness Survey ........................................................... 23
Participants ........................................................................................................ 23
Staff-Reported Respectful Climate ..................................................................... 24
Staff-Reported Risk Behaviors ........................................................................... 24
Discussion ................................................................................................................................ 26
Training and Coaching .................................................................................................. 26
Improved Staff Interactional Styles ................................................................................ 31
School Outcomes: Respectful Climate and Caring Adults .............................................. 31
Student Outcomes: Drug and Alcohol Use ..................................................................... 32
Conclusion .................................................................................................................... 32
References................................................................................................................................ 33
American Institutes for Research Final Report: Evaluation of ASD’s Second Order Change Project—iii
List of Exhibits Exhibit 1. Logic Model for the Second Order Change Project......................................................2
Exhibit 2. Participants in ASD Second Order Change Professional Development Events .............4
Exhibit 3. The Interpersonal Circumplex Model ..........................................................................7
Exhibit 4. Average Pre- to Post-test Item Differences by Training Type .................................... 10
Exhibit 5. Circumplex Scale of Interpersonal Values: Teachers & Other School Staff ............... 11
Exhibit 6. HLM Results: Friendliness, Teacher and Other School Staff ..................................... 11
Exhibit 7. HLM results: Distance, Teachers and Other School Staff .......................................... 11
Exhibit 8. Circumplex Scale of Interpersonal Values: Principals ............................................... 12
Exhibit 9. HLM Results: Friendliness, Principals ...................................................................... 12
Exhibit 10. HLM Results: Distance, Principals .......................................................................... 12
Exhibit 11. Circumplex Scale of Interpersonal Values: Findings for Teachers & Staff and
Principals by Year ..................................................................................................................... 13
Exhibit 12. Authoritarianism Over Time ................................................................................... 14
Exhibit 13. HLM Results: Authoritarianism, All Respondents ................................................... 14
Exhibit 14. Mindfulness Scores by Group ................................................................................. 15
Exhibit 15. HLM Results: Mindfulness, Teachers and Other School Staff ................................. 15
Exhibit 16. HLM Results: Mindfulness, Principals .................................................................... 15
Exhibit 17. Anchorage Student Participants by Year ................................................................. 15
Exhibit 18. Student-Reported Respectful Climate Over Time by School Type ........................... 17
Exhibit 19. Student-Reported Caring Adults Over Time by School Type................................... 18
Exhibit 20. Elementary SCCS Scale Scores: Statewide, Anchorage, and Anchorage Native
Alaskan Students ....................................................................................................................... 19
Exhibit 21. Middle School SCCS Scale Scores: Statewide, Anchorage, and Anchorage Native
Alaskan Students ....................................................................................................................... 20
Exhibit 22. High School SCCS Scale Scores: Statewide, Anchorage, and Anchorage Native
Alaskan Students ....................................................................................................................... 21
Exhibit 23. Student-Reported Drug and Alcohol Use Over Time by School Type ...................... 22
Exhibit 24. Student-Reported Drug & Alcohol Use: Anchorage Middle Schools and Statewide
Grades 5–12 .............................................................................................................................. 22
Exhibit 25. Student-Reported Drug & Alcohol Use: Anchorage High Schools and Statewide
Grades 5–12 .............................................................................................................................. 23
Exhibit 26. Anchorage Staff Participants by Year ...................................................................... 23
Exhibit 27. Staff-Reported Respectful Climate Over Time by School Type ............................... 24
Exhibit 28. Staff-Reported Student Drug and Alcohol Use Over Time by School Type ............. 25
Exhibit 29. Logic Model for the Second Order Change Project .................................................. 26
American Institutes for Research Final Report: Evaluation of ASD’s Second Order Change Project—iv
Executive Summary ASD’s Second Order Change project aimed to promote better outcomes for Anchorage youth—
particularly less alcohol and drug use, and particularly for Native Alaskan students—by
improving the social and emotional interactional skills of adults in the education system.
Data for this evaluation showed that the inputs for this project were delivered with both quantity
and quality. Over three years, almost 70 professional development opportunities were offered,
and these reached nearly 2,000 educators. In a district with 3,500 teachers and roughly 7,000
total staff members, these efforts represent a respectable start.
Not only were a large number of professional development opportunities offered, but they were
very well-received by the attendees. Comparison of responses to knowledge, practice, and
attitude items from pre to post-class showed some significant improvements for every class, and
for some classes, significant gains on a majority of items. These changes are particularly notable
given that many of the classes were of relatively short duration; only a few days or weeks.
More enduring changes for participants in the training were tested using an annual follow-up
survey that measured interpersonal values, authoritarianism, and mindfulness. This is a much
more stringent test of effects than the pre- post-surveys. The Circumplex Scales of Interpersonal
Values showed that participants in Second Order Change activities significantly reduced their
distance with students (for teachers and other school staff) and with staff (for principals).
Although they did not improve in friendliness overall, they were already at quite high levels at
baseline. Although there were no changes in authoritarianism, mindfulness did improve
significantly for teachers and other staff across both waves of follow-up. Principals improved
also, but the statistical tests were not significant.
Based on anonymous student surveys, we found that student-reported respectful climate
improved steadily over time in ASD schools. Respectful climate includes being treated fairly and
respectfully, and reporting that “teachers here are nice people.” The results showed that the
trends in respectful climate did not appear to be “interrupted” by the onset of the Second Order
Change grant starting in the 2009–10 school year. Instead, there was already a trend toward
improvement that appeared to continue. It appears that in middle schools, improvement was
somewhat greater than at baseline, however.
The caring adults scale reflects a more personal sense of connection to the adults at school This
scale also appeared to improve somewhat for middle schools (and to a lesser extent in
elementary schools), but scores for high schools were flat. The timing of improvements overlaps
with the period of the Second Order Change grant. Although there are many plausible competing
explanations for this, the data are consistent with an effect of the program.
Findings for Native Alaskan students on these school variables showed that these students
reported the highest scores for respectful climate at school, but until 2012 trailed other groups in
reporting the presence of caring adults.
Rates of both student-reported and staff-reported drug and alcohol use have been declining
sharply over the past six years. The trends are too steady to show an effect of the Second Order
American Institutes for Research Final Report: Evaluation of ASD’s Second Order Change Project—v
Change project. Reports of drug and alcohol use by Native Alaskan students have consistently
been lower than those for students overall.
The Second Order Change project achieved modest but significant effects on an already high-
scoring group of staff. Because participants volunteered for the courses, there was likely some
bias toward more interest in and more capacity for social and emotional skills among
participants.
Outcomes for schools and students were already moving in a positive direction before the
beginning of the Second Order Change project, which makes it difficult to attribute effects
simply to the grant. More likely, the effects observed in this evaluation are a cumulative effect of
years of ASD’s investments in the promotion of social and emotional learning for all students;
combined with broader cultural trends toward less drug and alcohol use nationwide (rates of
alcohol use and all drugs except marijuana are decreasing nationally, CDC, 2012).
Although there is not an apparent causal connection between the Second Order Change project
and improved youth indicators, the work ASD has done through this effort has kept it on a course
toward continuously more positive outcomes for youth.
American Institutes for Research Final Report: Evaluation of ASD’s Second Order Change Project—vi
Acknowledgements
The American Institutes for Research (AIR) thanks the many individuals who contributed to this
project to examine outcomes of the Anchorage School District’s (ASD’s) Second Order Change
project.
First, I thank Ann McKay Bryson, whose leadership of this initiative over the past two years has
been both visionary and tireless. I have learned so much from her and had a wonderful time
doing so. I also thank Michael Kerosky, her predecessor as the leader of the project, for his
commitment to positive youth development and his work building the resilience of all youth in
Anchorage. I also thank him for the bunny boots.
I would also like to thank the staff members of the Anchorage School District who sat with me to
talk about their work, and who provided me with documents and data. The enthusiasm that many
of them feel for building a strong foundation for social and emotional learning in ASD is
palpable.
I would also like to recognize the contributions of other staff members at AIR who contributed to
this effort. In particular, Elizabeth Spier and Olivia Padilla for their work on the School Climate
and Connectedness Survey and Brian Lundgren for his research support with the many pre- and
post-tests. Finally, I thank David Osher for introducing me to ASD, and for his many years of
masterful mentorship.
Kimberly Kendziora
Project Director, Evaluation of the ASD Second Order Change Project
AIR
American Institutes for Research Final Report: Evaluation of ASD’s Second Order Change Project—1
Introduction
Context Anchorage is the urban hub of Alaska and home to more than 290,000 people (including about
40 percent of all Native Alaskans). The city is challenged by rapidly changing demographics: a
large inflow of immigrants from Asia, Latin America, the Pacific Islands, Eastern Europe, and
Africa, as well as a growing influx of Alaska Natives from remote communities. Anchorage is
also home to two major military bases, and Anchorage schools serve a large number of students
from military families.
The Anchorage School District (ASD) is the 93rd-largest district in the United States, serving
about 50,000 students in 100 schools in a 100-square-mile enrollment area. The student body is
culturally, economically, and intellectually diverse. Of the district students, 48.1 percent are
White, 12.6 percent are Multi-racial, 10.2 percent are Asian, 9.9 percent are Latino/Hispanic, 9.0
percent are Alaska Native/American Indian, 6.3 percent are African American, and 4.0 percent
are Native Hawaiian or other Pacific Islander. ASD students speak 91 different languages at
home. The five most common foreign languages spoken at home are Spanish, Hmong, Samoan,
Tagalog, and Yup’ik.
For the past 15 years, the Anchorage School District (ASD) has been actively promoting,
teaching and implementing a positive youth development-based approach to the prevention of
risk behaviors. Rather than focusing on risk factors, this approach acknowledges that youth do
not just need to move away from risky behaviors, they need to actively develop competencies
that enable them to successful and make good life choices (Pittman & Cahill, 1991). The
common refrain among those engaged in youth development is: “problem free is not fully
prepared” (Pittman, Irby, Tolman, Yohalem, & Ferber, 2003).
In the late 1990s, ASD’s approach was organized around the Search Institute’s Developmental
Assets Framework (Benson, 2003; Scales, Benson, Leffert, & Blyth, 2000). In 2006, ASD
embraced a social and emotional learning framework and formally adopted standards that specify
what all students should know and be able to do with respect to self-awareness, self-
management, social awareness, and relationship skills. Beginning in 2006, ASD began annual
measurement of school climate and student connectedness using a survey that included both risk
behavior and social and emotional learning scales.
Second Order Change Project After about a decade of work to reduce risk and improve resilience, ASD noticed that a persistent
challenge in getting to the outcomes they envisioned related less to student behavior and more to
the behavior of adults in the system. ASD decided to work on improving the quality of adults’
interactions with students in schools. The rationale was that if a student’s interactions with the
adults in a school could be truly strengths-based, then the quality of all facets of his/her learning,
feelings about self, and interaction with others would improve.
American Institutes for Research Final Report: Evaluation of ASD’s Second Order Change Project—2
ASD hypothesized that there are a good number of adults in schools that do their best to offer
high quality interactions to each student, and notice/reflect and self‐correct on the interactions
that do not work. However, there are many adults who are not fully conscious of how their own
behavior affects students, and who could do more to model, recognize, and reinforce positive
social and emotional skills.
Consequently, ASD’s project under the Cook Inlet Tribal Council’s block grant targeted what
they termed “second order” systems change for prevention and intervention programming.
Rather than focusing on students directly, this project would focus on adults and help them
change the way they interact with young people.
The logic model for the Anchorage School District component of the initiative (Exhibit 1Exhibit
1) shows the hypothesized steps between training teachers, principals, and other school staff to
improve the ways they interact with students and students’ risk behaviors, including substance
use. Ultimately, the desired outcome is reduced substance use, particularly for Native Alaskan
students.
Exhibit 1. Logic Model for the Second Order Change Project
A recent meta-analysis of hundreds of studies of school-based SEL interventions for students
(Durlak, Weissberg, Dymnicki, Taylor, & Schellinger, 2011) has established that student SEL is
malleable (the effect size for this outcome domain was a strong 0.61). A smaller body of research
has shown that outside of therapeutic settings, adults’ emotional competencies can also be
improved through intervention. In two studies, groups participated in a workshop focused on
social and emotional skills. Scores on the post assessment for the first sample ranged from .02
standard deviations to .75 standard deviations higher than the pre-assessment scores, with an
average effect of .33, which corresponds to approximately an 11% improvement. In a second
sample, the effect size was .70, or approximately a 24% improvement (Sala, 2002).
A large body of research has documented a strong relationship between “school climate” and
student health outcomes, including alcohol, tobacco, and other drug use (Catalano et al., 2004;
American Institutes for Research Final Report: Evaluation of ASD’s Second Order Change Project—3
Kuperminc et al., 1997; 2001; Loukas & Robinson, 2004; Maddox & Prinz, 2003; Roeser &
Eccles, 1998; Roeser et al., 2000; Simons-Morton et al., 1999; Welsh, 2001) . Mayberry,
Espelage, and Koenig (2009) reported that a positive school climate and a positive sense of
community were associated with less adolescent substance use. Coker and Borders (2001)
reported that relationships with positively influencing adults (e.g., teachers) in the eighth grade
fostered the formation of positive relationships with peers and inhibited binge drinking behavior
in the tenth grade. Henry and Slater (2007) showed that regardless of a student’s own level of
school attachment, students who attended schools where the pupils overall were attached to
school were less likely to use alcohol, had lower intentions to use alcohol, and perceived that
fewer of their peers used alcohol. Research using data from the National Longitudinal Study of
Adolescent Health showed that school connectedness (which included indicators such as
students’ perceptions that teachers care about them and their feeling close to others at school)
was associated with lower levels of violence, tobacco, marijuana, and alcohol use, and delay in
sexual intercourse (Bonny et al., 2000; Resnick et al., 1997).
LaRusso, Romer, and Selman (2008) used structural equation modeling to examine the specific
mechanisms through which school climate relates to improved student health outcomes. They
found that students who perceived their teachers to be supportive and sensitive to their needs
were more likely to experience favorable climates of respect and to feel a sense of social
belonging in their school. In addition, students’ experience of a respectful climate was related to
lower rates of school drug use, fewer friendships with risky peers, and stronger perceptions of
healthy school norms. Overall, schools that promote respect in relations between students and
adults were more likely to engender healthy norms of behavior than schools that focus on control
of behavior without regard for student needs and perspectives.
The findings from the LaRusso et al. (2008) study also reinforced the idea that the way teachers,
as well as other school authority figures, relate to students and respond to their problems can be a
powerful factor in producing a favorable school climate for adolescents. Empirical support for
respect as a pathway from climate to student health outcomes also includes Ryan and Patrick’s
(2001) finding that teacher encouragement of mutual respect was the strongest predictor of
changes in academic efficacy and self-regulation in middle schools, and Welsh’s (2001) report
that perceptions of respect for students was the most important predictor of perceived safety and
risk behavior in middle school. Such findings suggest that teachers who cultivate a climate of
respect in high schools would do a better job of creating health-enhancing norms of behavior
than teachers who primarily focus on enforcing rules. In addition, respectful climates may
engender less risk taking, not only by encouraging adherence to healthy school norms of
behavior, but also by discouraging attraction to peers who engage in health-risk behavior
(Loukas et al. 2006). Studies based on the Monitoring the Future Project, a nationally
representative study of health risk behavior, additionally support the importance of school norms
for risk behavior (Kumar et al., 2002). Results from those data showed that individual cigarette
use, heavy drinking, and marijuana use were more likely when school-level norms reflected
approval of substance use.
American Institutes for Research Final Report: Evaluation of ASD’s Second Order Change Project—4
Methods
Participants As of September 30, 2012, ASD’s Second Order Change grant provided professional
development to 1,946 individuals. Some of them participated in multiple events, so the total
enrollment in all PD offerings was 2,670. Participants worked at a total of 99 schools, which
represents 98% of all ASD schools, plus many departments in the Administration Building.
Exhibit 2. Participants in ASD Second Order Change Professional Development Events
Role Number
Pk–12 classroom teachers 1,257
Principals & assistant principals 480
Supervisors, directors and cabinet 69
Specialist teachers (e.g., Music, Art, ELL, Gifted, Librarians) 96
Special education teachers 86
Counselors 115
Teacher experts / content specialists 127
Teacher assistants / tutors 94
Security 35
Administrative assistants & clerical staff 38
Noon (playground) supervisors 83
Nurses 28
Other (e.g. custodians, union representatives, graduation support coaches, psychologists, parents & community members)
162
Training for Teachers, Principals, and Other Staff The training activities spanned a variety of programs that varied in the strength of their scientific
evidence but which all represent best practices in strengthening teacher-student relationships and
building respectful school climates. Trainings included the following programs.
Resolving Conflict Creatively Program
Resolving Conflict Creatively Program (RCCP) is a universal, school-based intervention
involving violence prevention and intergroup understanding that was developed collaboratively
by Educators for Social Responsibility and the New York City Board of Education. The main
goal of RCCP is to change the mental and behavioral strategies that lead children to engage in
aggression and violence by teaching them constructive conflict resolution strategies and
promoting positive intergroup relations. Specific program objectives are to (a) make children
aware of the different choices they have besides passivity or aggression for dealing with
conflicts, (b) help children develop skills for making those choices real in their own lives, (c)
encourage children’s respect for their own culture and those of others, (d) teach children how to
identify and stand against prejudice, and (e) make children aware of their role in creating a more
peaceful world. The intervention has two major components: (a) training and coaching of
teachers to support them in implementing a curriculum in conflict resolution and intergroup
understanding (teacher training and coaching) and (b) the delivery of that curriculum via
American Institutes for Research Final Report: Evaluation of ASD’s Second Order Change Project—5
classroom instruction for children provided by the trained teachers (classroom instruction).
Additional features of RCCP include peer mediation, principals’ training, and parent training
(see Aber, Brown, Chaudry, Jones, & Samples, 1996; Brown, Roderick, Lantieri, & Aber, 2004).
RCCP is listed as an “effective” program in the Office of Juvenile Justice and Delinquency
Prevention’s 2007 Model Programs guide.
National Coalition Building Institute
The National Coalition Building Institute (NCBI) is a non-profit leadership development
network dedicated to the elimination of racism and other forms of oppression. Rooted in an
understanding of individual, community, and systemic change, NCBI leaders work with public
and private organizations to further cultural competence, collaboration and partnerships, and
effective relationships within and across group identities (see Brown & Mazza, 1991, 1996).
Although there have not been research studies done on the NCBI program, the principles around
which it is organized are consistent with best practices in human relations training (Karp &
Sammour, 2000; Roberson, Kulik, & Pepper, 2001).
Life Space Crisis Intervention
Life Space Crisis Intervention (LSCI) is a competency-based approach to intervening with
students who are experiencing emotional pain, psychological distress, or behavioral disruption in
their personal lives (Forthun, McCombie, & Freado, 2006). LSCI is used to help classroom
teachers, guidance counselors, administrators, and other school staff to promote positive
development and reduce the likelihood of negative consequences due to poor decision making.
The skills encourage educators to focus on the specific pattern of self-defeating behavior rather
than get bogged down in attempts to determine who is at fault and who should be disciplined.
The overall goal of LSCI is to encourage more adaptive problem-solving behaviors among
students (Long, Fecser, & Wood, 2001). Forthun, Payne, and McCombie (2009) documented
positive effects of LSCI on both student behaviors and educators’ attitudes.
Response Ability Pathways
Response Ability Pathways (RAP) methods are grounded in resilience, brain science, and
positive psychology (Brendtro, Brokenleg, Van Bockern, 1990) and offer methods for positive
behavior support and creating climates of respect in schools. Rather than enforcing obedience,
RAP sets high expectations for youth to take responsibility and show respect for themselves and
others. RAP intends to develop the strength and resilience of young persons.
Conscious Discipline
Conscious Discipline (CD) is a comprehensive classroom management program and a social-
emotional curriculum. It is designed to make changes in the lives of adults first, who, in turn,
effect improved outcomes for students. CD teaches adult the “Seven Powers for Self Control”
that help them to change their perception of and relationship with conflict, and be proactive
instead of reactive during conflict. Additionally, “Seven Basic Skills of Discipline” help adults
respond to conflict in such a way as to transform conflict into teaching moments for students.
Hoffman, Hutchinson, and Reiss (2009) found that early childhood teachers who participated in
a Conscious Discipline program reported improved perceptions of school climate and better
knowledge and use of classroom management techniques.
American Institutes for Research Final Report: Evaluation of ASD’s Second Order Change Project—6
Discipline with Dignity
Discipline with Dignity (Curwin & Mendler, 1988) is a flexible program for effective school and
classroom management that teaches responsible thinking, cooperation, mutual respect, and
shared decision-making. It is based on teaching students responsibility rather than on
punishment. The notion is that more students change their behavior when they are given
instruction on how to behave in the future, rather than simply being admonished for behaviors in
the past. In addition, educators can benefit from watching their own behavior; in dealing with
students, other faculty, and administrators, teachers should model the behavior they want from
students. For example, when teachers are angry with students, they need to express that anger in
the same way they want students to express anger with their classmates.
Breaking Ranks in the Middle/Middle School Matters
Breaking Ranks in the Middle (BRIM) training (NASSP, 2006) offers participants the tools and
strategies to address and deal with the unique challenges facing middle level leadership including
personalization, advisories, teaming, transition from elementary and transition to high school,
use of data and other critical topics. The training program’s design includes an interactive format
for small and large group discussions, problem-solving assignments, as well as analysis and
reflections necessary for meaningful comprehension and learning. The goal of this training
program is to facilitate reform efforts by school leadership teams for the improvement of student
learning and development of successful schools.
Additional Classes
The ASD Social and Emotional Learning Department created several of their own courses to
better meet local needs. Among these were:
• Social and Emotional Learning Inquiry Class
• Building Resilience in Trauma-Impacted Youth and Families
• Secondary Principals Listening Circle
• Reinforcing Social and Emotional Learning
• SEL Training for Noon Duty Staff
Measures
Training-Specific Pre- and Post-Course Inventories. For each of the courses offered, the
evaluator worked with the course instructor and ASD SEL Department staff to create brief
assessments of the knowledge, behaviors, and attitudes that each course aimed to affect. These
inventories were administered using the Survey Monkey platform. Email addresses of all course
registrants were supplied by the instructor and/or the SEL department, and invitations to
complete the online measures were sent via email. Non-respondents received up to three email
reminders to complete the measure.
Over the three years of the project, 59 separate online instruments were created. A total of 2,726
(non-unique) respondents completed the various pre and post measures.
Adult Interactional Style Inventory (AISI). The AISI was composed of the three component
surveys listed below. Registrants in any Second Order Change course (at any time over the
course of the three year project period) were invited to take the AISI at pre-test. All participants
who completed an AISI at least once were invited to participate in a 1-year follow up during
American Institutes for Research Final Report: Evaluation of ASD’s Second Order Change Project—7
February 2011, and a second follow-up in February 2012. Participants in the follow up AISI
waves of measurement were offered an incentive: two iPads were raffled off to respondents with
completed surveys at each wave of follow up measurement.
The three measures composing the AISI are as follows:
Interpersonal style was measured using Locke’s Circumplex Scales of Interpersonal
Values (Locke, 2000). The interpersonal circumplex is structural model for representing
interpersonal dispositions. It is typically defined with reference to the orthogonal
dimensions of dominant/yielding and friendly/distant (see Exhibit 3Exhibit 3). The
interpersonal circumplex has been used to describe, organize, and compare interpersonal
adjectives, personality measures, interpersonal transactions, interpersonal problems, and
personality disorders. The Circumplex Scales of Interpersonal Values have been found to
be reliable and valid in multiple samples (Locke, 2000).
Exhibit 3. The Interpersonal Circumplex Model
Authoritarianism. The commonly used circumplex dimensions have also been used to
identify types of parenting (Baumrind, 1966), with “authoritative” parenting being in the
dominant and friendly quadrant, “authoritarian” parenting being in the dominant and
distant quadrant, and “permissive” parenting being in either of the yielding quadrants
(with “neglectful” parents in the distant/yielding quadrant and “indulgent” parents in the
yielding/friendly quadrant; Lamborn, Mounts, Steinberg, & Dornbush, 1991). Thus, we
sought to strengthen our assessment of approaches to students with a measure of
authoritarianism. The Parental Modernity Scale (originally developed by Shaffer &
Edgerton, 1985) is used to measure authoritarian (traditional) beliefs and has been
adapted for use by teachers in multiple studies, including the NICHD Study of Early
Child Care and Youth Development and the national study of Early Head Start. In
unpublished research, Dr. Bridget Hamre of the Center for the Advanced Study of
Teaching and Learning (CASTL) at the University of Virginia stated that it is “a measure
of authoritarian/authoritative teaching that we’ve used in lots of studies that actually
American Institutes for Research Final Report: Evaluation of ASD’s Second Order Change Project—8
relates quite highly to observed [teaching]. Some of the items seem quite dated, but it
works.”
Mindfulness is a construct common across multiple training programs offered through
the Strengths-Based Prevention project, as it is an underlying strategy for adults’ own
self-control and social-emotional competence in interactions with students. Jennings and
Greenberg (2009) developed a measure of teachers’ mindfulness that is hypothesized to
underlie the maintenance of supportive teacher-student relationships, effective classroom
management, and successful social and emotional learning program implementation.
Although this measure does not have its own evidence base yet, it is in use in two
research projects funded by the US Department of Education’s Institute for Education
Sciences.
Alaska School Climate and Connectedness Survey (SCCS). The schools’ learning environment
was assessed using the Alaska School Climate and Connectedness Survey (SCCS) (Kendziora,
Osher, & Spier, 2005). This survey was originally developed as part of a statewide study for the
Alaska Association of School Boards. The student version measures three school climate scales:
High Expectations, School Safety, and School Leadership; as well as four connectedness scales
(Respectful Climate, Peer Climate, Caring Adults, and Community Involvement). An additional
SEL scale was written to directly assess the 15 Anchorage Social and Emotional Learning
Standards, and further items collect perceptions of the frequency of Delinquent Behaviors and
Drug and Alcohol use. The staff version of the survey supplies an overall climate factor that is
the mean of six scales: School Leadership, Respectful Climate, Staff Attitudes, Parent and
Community Involvement, Student Involvement, and School Safety, plus reverse-coded Student
Delinquency and Drug and Alcohol Use scores.
The SCCS was first administered in 2005 to 38 schools in 12 districts across Alaska, including
eight schools in Anchorage (with a valid sample size of 456 staff and 4,092 students). In 2006
and 2007, Anchorage administered the survey district-wide (with all staff and students in grades
5–12), significantly increasing the sample size. In all, 148 schools in 15 districts across Alaska
completed the survey, with a sample of 3,453 school staff and 24,732 students. In 2007, some
districts took the survey in the fall as baseline for a multi-site digital learning initiative, and in
the spring 150 schools in 14 districts (3,315 school staff and 22,411 students) completed the
survey. The history of administering this survey district-wide in Anchorage for the past two
years provides a helpful baseline for assessing change in school climate (and students’ SEL
skills) as a function of the intervention, allowing us to examine each principal’s building in a
single-subject time series analysis.
After years of development, the SCCS has been refined and has demonstrated solid reliability.
The average internal consistency across the staff scales is 0.87 (range=0.78 to 0.94) and for the
student scales is 0.77 (range=0.64 to .87). Additionally, the SCCS has shown not only strong
cross-sectional correlations with school-wide student achievement and schools’ Adequate Yearly
Progress status, but also positive change in school climate and connectedness is related to
significant gains in student scores on statewide achievement tests (Spier, Cai, Kendziora, &
Osher, 2007).
American Institutes for Research Final Report: Evaluation of ASD’s Second Order Change Project—9
The SCCS includes a 3-item scale in which students rate their perceived frequency of drug and
alcohol use at school or school events (including seeing students under the influence of drugs,
alcohol, or inhalants; α =.76). This scale score will be complemented by results from the Youth
Risk Behavior Survey, which is administered in Anchorage every two years (including 2005,
2007, and 2009). These data are also available by subgroup.
Native Alaskan Subgroup. The SCCS includes a demographic section in which basic student
data are recorded (e.g., grade, gender, ethnic group). All data will be reported separately for
Native Alaskan students as long as there are 10 or more students per group. Results will be
suppressed when there are 9 or fewer students to protect student privacy.
Data Analysis Basic descriptive statistics were used to examine outcomes on all measures administered.
Scoring was as follows:
Pre and post tests for training classes: means across all respondents were presented for
each item in the reports to ASD. Results in this report summarize only the paired t-tests
that were done comparing the pre- and post-training event responses for only those
individuals who took the instrument both times. Pre- and post-test responses were
matched using email addresses.
The 64 questions on the interaction style survey were scored according to the
developer’s instructions (available at http://www.class.uidaho.edu/klocke/csiv.htm).
Although personality researchers may be interested in scores for each X, Y, and diagonal
axis, for this report we focused on scores for the four major poles: dominant, friendly,
yielding, and distant. Separate forms were created for teachers (using the phrase, “with
my students”), other school staff (“with students”), and principals (“with my staff”).
Authoritarianism was computed as the mean of the 16 items; scored on a scale of 1 to 5
(strongly disagree to strongly agree). The same form was used for all roles.
Mindfulness was scored as the mean of the 22 items, with responses on a scale of 1
(never true) to 6 (almost always true). Separate forms were created for teachers (using the
phrase, “with my students”), other school staff (“with students”), and principals (“with
my staff”).
School Climate and Connectedness Survey scales were scored and reported by Dr.
Elizabeth Spier as part of a separate task. Data from her report that show the means for
each relevant scale over time (2006 to 2012) are shared here. Results from both the
student and the staff surveys are presented. In addition, means for just the Native Alaskan
subgroup were computed and are presented in this report.
Beyond the descriptive statistics just mentioned, a simple two-level hierarchical linear model
(HLM) was used to estimate the change over time in outcomes for participants in Second Order
Change training events. Time was the level one variable, and respondents were level two. Only
the 480 respondents who completed the measure on at least two occasions were included in the
HLM analyses.
American Institutes for Research Final Report: Evaluation of ASD’s Second Order Change Project—10
Results
Effectiveness of Training For the large majority of training events, participants were asked to complete online pre- and
post-tests. After the post-course data were completed, the evaluator prepared reports that
described the respondents, presented descriptive data for pre- and post-test scores, and shared
results of paired t-tests for just those respondents who took the measure at both pre and post. A
total of 47 reports were submitted over the course of the project (some of these had only post-test
feedback and no significance testing).
Pre-post testing showed at least some significant improvement in every instance. The average
proportion of items showing significant improvement from pre to post is listed in Exhibit 4 by
class type. In addition, comments provided by course participants tended to be quite positive.
Exhibit 4. Average Pre- to Post-test Item Differences by Training Type
Course
Number of pre-
post item set
analyses
Number of
items in each
analysis
Average percentage
of items showing
significant change
Resolving Conflict Creatively Program, Connected & Respected
18 12 55%
Life Space Crisis Intervention 4 10 80%
Response Ability Pathways 1 17 53%
Breaking Ranks in the Middle 3 12 61%
Discipline with Dignity 1 17 53%
Conscious Discipline 1 11 73%
SEL Inquiry 2 4 88%
Trauma-Informed Education 3 8 91%
Noon Duty Staff 1 24 46%
Interaction Style
The logic model for the Second Order Change project posited that participating in training events
sponsored through the grant should result in participants being more friendly and less distant
with student (and for principals, with their staffs). We did not hypothesize that there would be
any effects for dominance. The results were partially borne out: for teachers and other school
staff, friendliness did not increase over time, but distance did decrease significantly. Means for
all respondents are shown in Exhibit 5.
American Institutes for Research Final Report: Evaluation of ASD’s Second Order Change Project—11
Exhibit 5. Circumplex Scale of Interpersonal Values: Teachers & Other School Staff
The results of the hierarchical linear models testing the intercepts and slopes of friendliness and
distance for teachers and other school staff are shown in Exhibits 6 and 7.
Exhibit 6. HLM Results: Friendliness, Teacher and Other School Staff
Fixed Effect Coefficient
Standard
error T-ratio
Approximate
d.f. P-value
Intercept 3.416008 0.056109 60.882 413 0.000
Slope over time 0.003104 0.025023 0.124 855 0.902
Exhibit 7. HLM results: Distance, Teachers and Other School Staff
Fixed Effect Coefficient
Standard
error T-ratio
Approximate
d.f. P-value
Intercept 1.817814 0.047035 38.648 413 0.000
Slope over time -0.048631 0.020924 -2.324 855 0.020*
The results for principals are shown in Exhibit 8. There were many fewer principals who
participated, and so there was less power to detect effects if they existed. Although friendliness
did increase and distance did decrease, these changes were not statistically significant, as shown
in Exhibits 9–10.
1.00
1.50
2.00
2.50
3.00
3.50
4.00
2010 (n=407) 2011 (n=535) 2012 (n=321)
Scale
scores
(ran
ge =
1 t
o 5
)
Friendly
Dominant
Yielding
Distant*
American Institutes for Research Final Report: Evaluation of ASD’s Second Order Change Project—12
Exhibit 8. Circumplex Scale of Interpersonal Values: Principals
Exhibit 9. HLM Results: Friendliness, Principals
Fixed Effect Coefficient
Standard
error T-ratio
Approximate
d.f. P-value
Intercept 3.269360 0.107779 30.334 63 0.000
Slope over time 0.040728 0.053688 0.759 144 0.449
Exhibit 10. HLM Results: Distance, Principals
Fixed Effect Coefficient
Standard
error T-ratio
Approximate
d.f. P-value
Intercept 1.688724 0.095052 17.766 63 0.000
Slope over time
-0.007957 0.048978 -0.162 144
0.872
We did test the change for the other scores over time as well, and found that there were no
changes in dominance or yielding.
One of the advantages of a circumplex model is the opportunity to represent findings in a two-
dimensional space. Exhibit 11 shows the space defined by the dimensions of yielding-dominance
and distance-friendliness. Results show that teachers clustered in the upper right quadrant in each
of the three years, moving slightly up (more dominant) and to the right (more friendly) each year.
Principals’ results were much less stable, with relatively wide variation in dominance each year,
and some movement toward greater friendliness over time.
1.00
1.50
2.00
2.50
3.00
3.50
4.00
2010 (n=52) 2011 (n=51) 2012 (n=19)
Scale
scores
(ran
ge =
1 t
o 5
)
Friendly
Dominant
Yielding
Distant
American Institutes for Research Final Report: Evaluation of ASD’s Second Order Change Project—13
Exhibit 11. Circumplex Scale of Interpersonal Values: Findings for Teachers & Staff and Principals by Year
Authoritarianism No significant change over time was observed for authoritarianism (see Exhibits 11 and 12).
Authoritarian styles are generally both dominant and distant; the fact that we saw decreases in
distance and, as hypothesized, no changes in dominance are consistent with no overall shift for
this measure.
-0.50
-0.40
-0.30
-0.20
-0.10
0.00
0.10
0.20
0.30
0.40
0.50
-2.50 -1.50 -0.50 0.50 1.50 2.50
Yie
ldin
g <
--->
Dom
inan
t
Distant <---> Friendly
Teachers & Staff 2010
(n=407)
Teachers & Staff 2011
(n=535)
Teachers & Staff 2012
(n=320)
Principals 2010 (n=52)
Principals 2011 (n=51)
Principals 2012 (n=19)
American Institutes for Research Final Report: Evaluation of ASD’s Second Order Change Project—14
Exhibit 12. Authoritarianism Over Time
Exhibit 13. HLM Results: Authoritarianism, All Respondents
Fixed Effect Coefficient
Standard
error T-ratio
Approximate
d.f. P-value
Intercept 2.230836 0.049143 45.395 477 0.000
Slope over time 0.000651 0.022008 0.030 1042 0.977
Mindfulness Mindfulness is perhaps a less familiar construct than the others, but improvements in reflective
practice were an important hypothesized outcome for the Second Order Change participants.
Sample items on the mindfulness measure include:
When I am teaching it seems I am running on automatic without much awareness of what
I’m doing.
I try to be understanding and patient towards those aspects of my class I don’t always
like.
Results (Exhibits 13–15) showed significant improvement for teachers and other school staff, but
once again, not for principals. Once again the numbers of principals was much lower than the
other groups, which gives us less power to detect effects where they might exist. Descriptively,
we see that mindfulness improved for all groups over the course of the project.
2.00
2.05
2.10
2.15
2.20
2.25
2.30
2.35
2.40
2.45
2.50
2010 (n=473) 2011 (n=612) 2012 (n=359)
Mean
Score (
1 t
o 5
scale
)
Note: changes are not statistically significant
American Institutes for Research Final Report: Evaluation of ASD’s Second Order Change Project—15
Exhibit 14. Mindfulness Scores by Group
Exhibit 15. HLM Results: Mindfulness, Teachers and Other School Staff
Fixed Effect Coefficient
Standard
error T-ratio
Approximate
d.f. P-value
Intercept 4.401024 0.046804 94.031 419 0.000
Slope over time 0.128839 0.020804 6.193 916 0.000
Exhibit 16. HLM Results: Mindfulness, Principals
Fixed Effect Coefficient
Standard
error T-ratio
Approximate
d.f. P-value
Intercept 4.727399 0.122670 38.537 63 0.000
Slope over time -0.004282 0.060753 -0.070 138 0.944
Student School Climate and Connectedness Survey
Participants
All students in grades 5 through 12 in participating ASD schools were invited to take part in the
survey. Participating schools included elementary, middle, high, charter and alternative schools.
See Exhibit 17 for information regarding the numbers of schools and participants each year.
Exhibit 17. Anchorage Student Participants by Year
Year Number of Schools Students Grades 5–12 with Valid Surveys
2006 92 20,812
2007 92 18,162
2008 95 18,508
2009 96 18,813
2010 96 19,212
4.40
4.45
4.50
4.55
4.60
4.65
4.70
4.75
4.80
4.85
2010 2011 2012
Min
dfu
lness
scores
(ran
ge =
1 t
o 6
)
Teachers* (n by
year=259, 325, 212)
Other School Staff* (n
by year=147, 233, 124)
Principals (ns) (n by
year=68, 65, 33)
American Institutes for Research Final Report: Evaluation of ASD’s Second Order Change Project—16
2011 95 13,531
2012 90 15,363
In 2012, the overall participation rate among Anchorage students in grades 5 through 12 was 53
percent.
Because surveys were completed anonymously, we are not able to compare scores from one year
to the next using traditional repeated-measures statistics (although there is likely substantial
overlap in respondents from one year to the next). We can, however, with a certain degree of
confidence, ascertain whether observed differences in scores from year to year represent real
shifts in responses among participants rather than chance fluctuation. We calculated an effect
size (Cohen’s d) to determine whether the differences in two mean scores and differences in the
distribution of scores around those means (such as scale scores for Caring Adults from 2010 and
from 2011) were substantial enough for us to be confident that there really has been a significant
change in scores. Where differences were significant, changes are marked with an asterisk.
Student-Reported Respectful Climate
This scale reflects students’ feelings about fairness of rules and respect for students’
contributions. This scale is also included as part of the Overall Connectedness summary scale.
The items comprising this scale (and the scale’s reliability) are as follows (α = .84):
44. Teachers here are nice people
46. My teachers treat me with respect
47. When students break rules, they are treated fairly
50. My teachers are fair
53. Our school rules are fair
54. It pays to follow the rules at my school
American Institutes for Research Final Report: Evaluation of ASD’s Second Order Change Project—17
Students selected responses between 1 and 5, where 1 indicates a lack of a respectful climate and
5 indicates a highly respectful climate for students. Exhibit 18 shows that respectful climate has
climbed steadily over time in all school types.
Exhibit 18. Student-Reported Respectful Climate Over Time by School Type
Student-Reported Caring Adults
This scale reflects students’ feelings about how close they feel to adults in the school. This scale
is also included as part of the Overall Connectedness summary scale. The items comprising this
scale (and the scale’s reliability) are as follows (α = .72):
15. There is at least one adult at this school whom I feel comfortable talking to about things
that are bothering me
16. At school, there is a teacher or some other adult who will miss me when I’m absent
17. There are a lot of chances for students in my school to talk with teachers one on one
26. I can name at least five adults who really care about me
27. Other adults at school besides my teachers know my name
Students selected responses between 1 and 5, where 1 indicates a lack of caring adult
relationships and 5 indicates a high number of caring adult relationships at the school. Exhibit 19
shows that scores for Caring Adults have improved somewhat in 2011 and 2012 in elementary
and middle schools, but have been flat in high schools.
3.6
1
3.1
7
3.0
9
3.4
3
3.5
6
3.7
0
3.2
9
3.1
7
3.6
1
3.7
2
3.7
0
3.2
6
3.1
9 3.7
2
3.6
6
3.7
9
3.3
5
3.2
4 3.7
7
3.7
5
3.8
2
3.4
2
3.2
6
3.6
4
3.7
4
3.8
9
3.5
9
3.3
7
3.7
9
3.8
4
3.9
1
3.6
5
3.3
4 3.9
2
3.9
8*
1
2
3
4
5
ASD Elementary
School Average
ASD Middle School
Average
ASD High School
Average
ASD Charter School
Average
ASD Alternative
School Average
2006 2007 2008 2009 2010 2011 2012
American Institutes for Research Final Report: Evaluation of ASD’s Second Order Change Project—18
Exhibit 19. Student-Reported Caring Adults Over Time by School Type
In addition to examining trends for all students over time, we also examined respectful climate
and caring adult scores for three groups of students:
1. A weighted statewide averagei of students in grades 5–12 who completed the survey
(participation details are included in the endnote)
2. ASD results for students in grades 5–12
3. ASD Native Alaskan students in grades 5–12
Exhibit 20 shows that for students in grade 5 (and 6, where 6 is in an elementary school), ASD
(the lines with circle markers) tended to have higher respectful climate and caring adult scores
than the state overall. Native Alaskan student in ASD have reported the highest scores for
respectful climate at school, but until 2012 trailed other groups in reporting the presence of
caring adults. In 2012, Native Alaskan students scored the same as ASD students overall.
3.7
3
3.3
3
3.3
5
3.5
6
3.5
8
3.6
9
3.3
4
3.3
7
3.6
9
3.7
5
3.6
9
3.3
3
3.3
6
3.7
3
3.7
4
3.7
2
3.3
1
3.3
4
3.6
9
3.7
4
3.7
5
3.3
8
3.3
6
3.7
6
3.7
3
3.8
2
3.5
4
3.4
1
3.7
6
3.7
7
3.8
3
3.5
5
3.3
8
3.8
0
3.8
3
1
2
3
4
5
ASD Elementary
School Average
ASD Middle School
Average
ASD High School
Average
ASD Charter School
Average
ASD Alternative
School Average
2006 2007 2008 2009 2010 2011 2012
American Institutes for Research Final Report: Evaluation of ASD’s Second Order Change Project—19
Exhibit 20. Elementary SCCS Scale Scores: Statewide, Anchorage, and Anchorage Native Alaskan Students
Exhibit 21 shows that for middle grade students, the patterns are similar, with ASD out-scoring
the state for both scales, and ASD Native Alaskan students reporting the highest levels of
respectful climate at school, but more moderate levels of caring adults.
3.00
3.25
3.50
3.75
4.00
2006 2007 2008 2009 2010 2011 2012
SC
CS
Scale
Scores
(1 t
o 5
scale
)
Statewide Respectful
Climate
Statewide Caring
Adults
ASD Respectful
Climate
ASD Caring Adults
ASD Native Alaskans
Respectful Climate
ASD Native Alaskans
Caring Adults
American Institutes for Research Final Report: Evaluation of ASD’s Second Order Change Project—20
Exhibit 21. Middle School SCCS Scale Scores: Statewide, Anchorage, and Anchorage Native Alaskan Students
Among high school students (Exhibit 22), the patterns are different. At earlier grade levels,
respectful climate scores tended to be higher than caring adult scores. The reverse is true in high
school: caring adult scores tend to be higher than respectful climate at school. ASD no longer
out-scores the state for these variables. Within ASD, Native Alaskan high school students did
still report the highest levels of respectful climate, but had lower ratings for caring adults
compared to students overall.
3.00
3.25
3.50
3.75
4.00
2006 2007 2008 2009 2010 2011 2012
SC
CS
Scale
Score (
1 t
o 5
scale
)
Statewide Respectful
Climate
Statewide Caring Adults
ASD Respectful Climate
ASD Caring Adults
ASD Native Alaskans
Respectful Climate
ASD Native Alaskans
Caring Adults
American Institutes for Research Final Report: Evaluation of ASD’s Second Order Change Project—21
Exhibit 22. High School SCCS Scale Scores: Statewide, Anchorage, and Anchorage Native Alaskan Students
Student-Reported Risk Behaviors
This scale reports the number of times students reported observing other students’ drug and
alcohol use. The items comprising these scales (and the scales’ reliabilities) are as follows.
Response categories are:
1 = 0 times
2 = 1–2 times
3 = 3–6 times
4 = 7–12 times
5 = More than 12 times
Student Drug and Alcohol Use (α =.73)
56. Under the influence of drugs (marijuana, coke, crack)
57. Under the influence of alcohol (beer/wine/liquor)
62. Under the influence of inhalants (sniffing glue, paints, or aerosol sprays)
Student-reported levels of drug and alcohol use were quite low overall, with notable jumps in
frequency across elementary, middle, and high school (Exhibit 23). Rates have been decreasing
steadily over time.
3.00
3.25
3.50
3.75
4.00
2006 2007 2008 2009 2010 2011 2012
SC
CS
SC
ale
Score (
1 t
o 5
scale
)
Statewide Respectful
Climate
Statewide Caring Adults
ASD Respectful Climate
ASD Caring Adults
ASD Native Alaskans
Respectful Climate
ASD Native Alaskans
Caring Adults
American Institutes for Research Final Report: Evaluation of ASD’s Second Order Change Project—22
Exhibit 23. Student-Reported Drug and Alcohol Use Over Time by School Type
Exhibit 24 shows that for middle school students, statewide rates of drug and alcohol use have
been similar in ASD and statewide, but rates for Native Alaskan students have consistently been
lower. Both for Native Alaskan students and ASD students overall, rates of drug and alcohol use
appear to have been decreasing more sharply than in the state overall.
Exhibit 24. Student-Reported Drug & Alcohol Use: Anchorage Middle Schools and Statewide Grades 5–12
1.1
5
1.6
1
2.5
2
1.4
2 2.0
1
1.2
7
1.6
6 2.2
1
1.4
3
1.5
2
1.2
5
1.6
4 2.1
3
1.3
4
1.4
9
1.2
2
1.5
2 2.0
6
1.2
8
1.4
5
1.2
2
1.5
2 2.0
3
1.2
5
1.4
9
1.1
7
1.3
7 1.9
1
1.2
3
1.4
1
1.1
6
1.3
3 1.9
6
1.1
0*
1.4
3
1
2
3
4
5
ASD Elementary
School Average
ASD Middle School
Average
ASD High School
Average
ASD Charter School
Average
ASD Alternative
School Average
2006 2007 2008 2009 2010 2011 2012
1
1.2
1.4
1.6
1.8
2
2.2
2.4
2006
2007
2008
2009
2010
2011
2012
Scale
Score (
1=
never;
2=
seen
1-2
tim
es
pa
st y
ea
r)
ASD Middle Schools Drug
& Alcohol Use
ASD Middle School Native
Alaskans Drug & Alcohol
Use
Statewide Grades 5-12 Drug
& Alcohol Use
American Institutes for Research Final Report: Evaluation of ASD’s Second Order Change Project—23
Exhibit 25 shows that for high school students, rates of drug and alcohol use have actually
increased statewide between 2010 and 2012, with students across the state “catching up” to the
relatively higher rates of drug and alcohol use among ASD high school students. In ASD, after
years of decreasing usage, rates ticked up slightly in 2012. Rates for Native Alaskan students did
show a steady and quite marked decline over time, with no increase in recent years.
Exhibit 25. Student-Reported Drug & Alcohol Use: Anchorage High Schools and Statewide Grades 5–12
Staff School Climate and Connectedness Survey
Participants
All staff in participating Anchorage School District schools was invited to take part in the
survey. Participating schools included regular primary schools, charter schools, and alternative
schools. See Exhibit 26 for more information regarding the numbers of schools and participants
each year.
Exhibit 26. Anchorage Staff Participants by Year
Year Number of Schools Total Staff with Valid Surveys Teachers with Valid Surveys
2006 92 2,995 1,681
2007 92 2,905 1,644
2008 95 2,968 1,557
2009 96 3,437 1,845
2010 96 3,070 1,693
2011 95 2,675 1,525
2012 94 2,744 1,522
1
1.2
1.4
1.6
1.8
2
2.2
2.4
2.6
2.8
20
06
20
07
20
08
20
09
20
10
20
11
20
12
Sca
le S
core
(1=
nev
er;
2=
seen
1-2
tim
es p
ast
yea
r)
ASD High Schools Drug &
Alcohol Use
ASD High School Native
Alaskans Drug & Alcohol
Use
Statewide Grades 5-12
Drug & Alcohol Use
American Institutes for Research Final Report: Evaluation of ASD’s Second Order Change Project—24
In 2012, the overall participation rate among Anchorage staff was 49 percent.
Staff-Reported Respectful Climate
This scale reflects staff members’ feelings about how students treat each other and how well
students and staff members treat one another. The items comprising this scale (and the scale’s
reliability) are as follows (α =.86):
6. At this school, students and teachers get along really well
7. Students in this school help each other, even if they are not friends
11. Teachers and students treat each other with respect in this school
14. Students in this school treat each other with respect
16. The students in this school don’t really care about each other (reverse scored)
Staff members selected responses between 1 and 5, where 1 indicates a low level of respect and 5
indicates a high level of respect. According to staff reports, ratings for respectful climate have
been relatively flat overall, with a slight uptick for middle schools in 2011 and 2012.
Exhibit 27. Staff-Reported Respectful Climate Over Time by School Type
Staff-Reported Risk Behaviors
School staff responded to the same set of items pertaining to Risk Behaviors that the students
were presented with:
Response categories are:
1 = 0 times
2 = 1–2 times
3 = 3–6 times
4 = 7–12 times
5 = More than 12 times
3.8
5
3.5
9
3.6
1
3.9
8
3.6
7
3.8
9
3.5
9
3.6
3 4
.26
3.8
0
3.8
2
3.7
1
3.5
7 4
.11
3.9
1
3.9
0
3.6
8
3.7
2 4.1
4
3.8
2
3.8
8
3.6
6
3.6
9 4.1
3
3.8
5
3.9
5
3.8
5
3.7
2
3.8
0
4.1
3
3.9
3
3.8
2
3.7
5 4.2
1
3.9
2
1
2
3
4
5
Anchorage
Elementary Average
Anchorage Middle
School Average
Anchorage High
School Average
Anchorage Charter
School Average
Anchorage
Alternative School
Average
2006 2007 2008 2009 2010 2011 2012
American Institutes for Research Final Report: Evaluation of ASD’s Second Order Change Project—25
Student Drug and Alcohol Use (α =.64)
39. Under the influence of drugs (marijuana, coke, crack)
40. Under the influence of alcohol (beer/wine/liquor)
45. Under the influence of inhalants (sniffing glue, paints, or aerosol sprays)
According to staff report, there is virtually zero use of drugs or alcohol in elementary schools
(note that students reported higher rates of usage). Staff reports also show a generally downward
trend over time, but not as dramatic a drop as seen in the student-reported data (Exhibit 28).
Exhibit 28. Staff-Reported Student Drug and Alcohol Use Over Time by School Type
1.0
2
1.2
4
1.5
0
1.0
3 1.4
4
1.0
2
1.2
5
1.5
9
1.0
7 1.5
0
1.0
1 1.4
5
1.1
9
1.0
9
1.3
5
1.0
1
1.1
3
1.4
4
1.0
2
1.3
4
1.0
1
1.1
6
1.4
7
1.0
4
1.2
8
1.0
0
1.1
5
1.4
8
1.1
9
1.0
4
1.0
1
1.1
7
1.4
9
1.0
4
1.3
0*
1
2
3
4
5
Anchorage
Elementary Average
Anchorage Middle
School Average
Anchorage High
School Average
Anchorage Charter
School Average
Anchorage
Alternative School
Average
2006 2007 2008 2009 2010 2011 2012
American Institutes for Research Final Report: Evaluation of ASD’s Second Order Change Project—26
Discussion
ASD’s Second Order Change project was both an ambitious and a forward-looking project. The
project embedded a vision of promoting social and emotional skills for all—students and staff
alike into an ambitious program of ongoing professional development. In this discussion, the
evaluation will examine the logic model for this project one element at a time and relate the
results to whether the model was borne out. The model is repeated in Exhibit 29 for the reader’s
convenience.
Exhibit 29. Logic Model for the Second Order Change Project
Training and Coaching
Data for this evaluation showed that the inputs for this project were delivered with both quantity
and quality. Over three years, almost 70 professional development opportunities were offered,
and these reached nearly 2,000 educators. In a district with 3,500 teachers and roughly 7,000
total staff members, these efforts represent a respectable start.
Not only were a large number of professional development opportunities offered, but they were
very well-received by the attendees. Comparison of responses to knowledge, practice, and
attitude items from pre to post-class showed some significant improvements for every class, and
for some classes, significant gains on a majority of items. These changes are particularly notable
given that many of the classes were of relatively short duration; only a few days or weeks.
Comments from attendees were generally quite positive. Examples from three classes selected at
random are presented here in their entirety, edited only for clarity.
Participants in the Oceanview SEL Inquiry class wrote:
American Institutes for Research Final Report: Evaluation of ASD’s Second Order Change Project—27
The class was more than I ever anticipated. It is my hope and desire what I learned in this
class will remain permanently part of my daily “radar.” I believe the students I serve will
be positively impacted and this is my goal but the self-benefit has been above and
beyond. How lucky I am to have such an opportunity!
The inquiry method and the length of time of the class offered an opportunity for deeper
self-reflection at a level a shorter period of time would not have provided.
I enjoyed the collegial interactions. It was a great opportunity to reflect on my practice
and needs that individual students in my classroom have.
It was a great resource for a first year teacher!
Participants in a Middle School Matters course wrote the following:
Was there anything about the class that you found particularly helpful or useful to you in your
work?
I enjoyed getting to hear about how things run at other schools; how other teams operate,
etc. Having time to discuss issues that come up.
I thought the book we used was VERY insightful. I also enjoyed working with and
collaborating with the members in the class.
All of it; the book, discussions, expertise of the instructor.
Do you have any other comments about the class you took?
I wish it lasted longer! I would love to continue the dialog.
I think that Jan Davis is exceptional at introducing classes and topics that benefit teachers
and students. I look forward to doing another follow up class again with this material and
exploring the book a little more in depth. I hope to use the information at the start of next
school year and collaborate even more with the members of my team and with Jan.
I am glad I took it and would like to take another!
Participants in the Trauma-informed Educational Practices class reported the following
comments:
Was there anything about the Trauma-Informed Effective Educational Practices class that you
wish had been different, or that could have been done better?
No, it was excellent
More “move around time” and visual aids.
It would have been nice to have some small group breakout sessions to process the
information, but there probably wasn't enough time.
I think a little more time on interventions would be helpful. Not sure, in survey, what is
meant by "I know more about interventions that I can use when (my emphasis) a student
has a strong emotional reaction". During the outburst, what can you do? When the child
American Institutes for Research Final Report: Evaluation of ASD’s Second Order Change Project—28
is in a calmer state, one can take some measures, and those were presented. But when a
child is melting down? Nada.
A more comfortable room/seats/setting, but no complaints about the content of the
presentation.
More practice or role playing of interventions
It would be nice to have a list of some samples of books, games, websites, and/or
activities we can use with students, or maybe a list of catalogues or websites for the
orders.
It is always nice to leave with specific tools to try.
I personally cannot do a full day of heavy PowerPoint presentation. Breaking it up with
more engaged activity and dialogue would help a lot! I did appreciate the few small
group discussions that occurred.
No; the session was purposeful and interesting.
More strategies, offer strategies earlier in the class as people start to lose focus after
lunch. Maybe thread strategies in with brain/background info.
Since it was the first class with basic information, I think it was excellent. A follow-up
session may include small group work with each small group examining the possible
triggers and interventions either for case examples or from our own personal cases (using
fictitious names).
I was very pleased with it as it was.
I program would not let me click my strongly agree answers to the post questionnaire...all
my answers would have been strongly agree.
I would like to know if there is any research in regards to cultural responses to trauma.
Are there children who are brought up to respond to trauma differently?
I wish that all teachers had to participate in a shortened version of the "class" before
students started. I also would like to explore how RCCP, Conscious Discipline, etc. fit
in. Schools need strategies or to develop procedures to help students and support school
personnel that work with these students so we can keep them in school and out of special
education with a label if possible. The class was helpful and not a waste if time.
The workshop was informative and practical.
It would have been great if Kagan Cooperative Learning strategies were used to get us to
process more interactively. They could have also gotten us up and moving.
I would have liked to have more intervention strategies or teacher practices that would
help teach these students.
Perhaps pairing the clinical presentation with an experienced classroom teacher with
concrete examples of interventions.
Too much sitting
I think the presenter used the time very well. Perhaps a second session focusing on
specific interventions that could be applied in the school setting would be helpful, with
role playing?
Was there anything about the class that you found particularly helpful or useful to you in
your work?
Absolutely—further validates need for SEL!! Gained some strong background
knowledge to support the students and teachers of these students.
Regulating students through art, movement...giving students wait time.
American Institutes for Research Final Report: Evaluation of ASD’s Second Order Change Project—29
Scenarios and talk time with table.
The idea that being cool and calm and aware of how I am reacting is sometimes more
important than what the child is doing. Behavior is a form of communication, not an
insult.
I especially liked the ideas for regulations of emotions that could be used in a classroom
setting.
Validation. Earnestness of presenter and audience. Sharing of techniques. A few leads on
resources, and more effective ways to work with known resources. The peacock feather
exercise.
Validating earlier suspicions regarding dealing with symptomatic behaviors, and had
always wondered what root causes had been underlying.
Interventions, Interventions, Interventions!
The detailed explanation and description of the emotional and behavioral reactions of
traumatized children
Handouts with data and outlines are always useful.
The peacock feather activity.
I found this very helpful in thinking about the students who present these challenging
behaviors may be trauma impacted students. It reinforced what I knew about best
practices about having predictable routines.
I found it helpful to remember that all students are different and therefore have different
needs. What works for one student may very well not work for another one.
Great information
The reminder that having students with extreme needs related to their life experience
creates additional stresses and challenges for teachers is helpful. This will encourage me
to search ways to support teachers, encourage them, and find training to build their
knowledge base about ways to help or respond to kids who have experienced complex
multiple traumas.
The open interaction was great. I thought of many things I will do different when dealing
with many kids......not just the multi traumatic.
The case studies/amalgamations of children in our own community.
Increased awareness and strategies that are helpful
Yes, it was all helpful in understanding kids' (and adults') interactions and behaviors.
Everything—the case histories were very helpful.
I thought it was all useful and hopeful!
The importance of communicating the effects of trauma on children to staff if it is
suspected in a child’s past.
The information increased ones sensitivity and awareness regarding this very important
issue impacting the lives of our students.
More specific strategies might have been helpful. There were some specific strategies
and lots of general strategies. For participants to have actually practiced some of the
strategies with each other would have been helpful.
Yes, it was all very relevant and I found all of it very useful for myself and how I
perceive these students.
The information provided during the class will help me as I work with parents on our
Student Assistance Team.
American Institutes for Research Final Report: Evaluation of ASD’s Second Order Change Project—30
Case examples and examples of successful interventions.
I found the information about how reactions can be misinterpreted and easily mishandled
very useful. I know I will be looking more closely at my students and how I respond to
them and paying close attention to triggers.
It always nice to have more information about the students in the district. Also, it's good
to be part of healthy and concerned dialogue about the trauma in our students' lives.
Do you have any other comments about the class you took?
Excellent class - Helen did an outstanding job! Thanks for coordinating this Ann!! :-)
Helen is excellent. Thank you for inviting me!
Enjoyed, it was very informational.
Very interesting.
Great instructor. Nice setting. Important information.
I think this workshop should have an intro aspect, for those unfamiliar with trauma and
its effect, and a Part II with interventions and community-building development
techniques for those who already understand the problems. The pre-test could spot which
educators belong in which part. I would really have liked more emphasis on what to DO
with our traumatized little people to help them.
Needs to be broadcast to all districts and to all classrooms. I would even suggest that
such information be presented in High School health classes for seniors, who will
someday be parents; with the hopes of a preventative strategy.
Great information, I wish more people could hear it.
It is always good to know how important that we need to work together as a team to help
students.
Helen was a very engaged and dynamic person who brought great examples of cases to
illustrate points. I think this was a very powerful topic and very useful in my work with
students and staff.
Very helpful class
Thank you for making this information available to us.
Need more on this topic
Very well organized. Lots of handouts. PowerPoint copy is great; the graphs were not
labeled though. Presenter very knowledgeable and engaging. All ASD staff should
receive this training!
This info needs to be part of August orientations/trainings for teachers and maybe all staff
interacting with children and/or parents.
It was extremely beneficial, and the presenter was very good.
I hope that there are future opportunities for other ASD staff to participate in this class.
Great presentation.
I would like to have a list of online resources.
The hand outs were good.
It is definitely great and useful information to have.
The class was informative and well worth the time.
Great, knowledgeable, positive presenter. Lots of good information presented.
It was great! I learned so much.
American Institutes for Research Final Report: Evaluation of ASD’s Second Order Change Project—31
This class was very complete and concise. I learned so much more than I had expected.
Thank you!!
It was excellent and very important for educators.
Lots of good information
This information is important for all educators and others working with children. The
more we understand why children act the way they do, the better we can reach them and
provide for them as learners.
Improved Staff Interactional Styles Enjoying a class, writing positive comments about it, and even improving knowledge, practice,
and attitudes in the short term are good outcomes, but they do not shed light on any potentially
more enduring effects of having participated in such a professional development event. A much
more stringent test of effects is a one-year follow up to determine whether enduring changes in
interactional styles occurred.
The Circumplex Scales of Interpersonal Values showed that participants in Second Order
Change activities significantly reduced their distance with students (for teachers and other school
staff) and with staff (for principals). Although they did not improve in friendliness overall, they
were already at quite high levels at baseline (see Exhibit 11).
Although there were no changes in authoritarianism, mindfulness did improve significantly for
teachers and other staff across both waves of follow-up. Principals improved, but the statistical
tests were not significant.
School Outcomes: Respectful Climate and Caring Adults Based on anonymous student surveys, we found that student-reported respectful climate
improved steadily over time in ASD schools. Respectful climate includes being treated fairly and
respectfully, and reporting that “teachers here are nice people.” If adults in ASD were improving
their interactional style as predicted by the logic model, then respectful climate should improve.
The results showed that the trends in respectful climate did not appear to be “interrupted” by the
onset of the Second Order Change grant starting in the 2009–10 school year. Instead, there was
already a trend toward improvement that appeared to continue. It appears that in middle schools,
improvement was somewhat greater than at baseline, however.
The caring adults scale reflects a more personal sense of connection to the adults at school
(“There is at least one adult at this school whom I feel comfortable talking to about things that
are bothering me;” “At school, there is a teacher or some other adult who will miss me when I’m
absent;” “Other adults at school besides my teachers know my name.”). This scale also appeared
to improve somewhat for middle schools (and to a lesser extent in elementary schools), but
scores for high schools were flat. The timing of improvements overlaps with the period of the
Second Order Change grant. Although there are many plausible competing explanations for this,
the data are consistent with an effect of the program.
Findings for Native Alaskan students on these school variables showed that these students
reported the highest scores for respectful climate at school, but until 2012 trailed other groups in
American Institutes for Research Final Report: Evaluation of ASD’s Second Order Change Project—32
reporting the presence of caring adults. In 2012, Native Alaskan students scored the same as
ASD students overall. These improvements are consistent with hypothesized effects of the
Second Order Change project, but are too consistent with prior trend to attribute change to an
initiative that began in 2009–10.
Student Outcomes: Drug and Alcohol Use Rates of both student-reported and staff-reported drug and alcohol use have been declining
sharply over the past six years. The trends are too steady to show an effect of the Second Order
Change project. Reports of drug and alcohol use by Native Alaskan students have consistently
been lower than those for students overall.
Conclusion The Second Order Change project achieved modest but significant effects on an already high-
scoring group of staff. Because participants volunteered for the courses, there may be some bias
toward more interest in and more capacity for social and emotional skills among participants.
Outcomes for schools and students were already generally moving in a positive direction before
the beginning of the Second Order Change project, which makes it difficult to attribute effects
simply to the grant. More likely, the effects observed in this evaluation are a cumulative effect of
years of ASD’s investments in the promotion of social and emotional learning for all students;
combined with broader cultural trends toward less drug and alcohol use nationwide (rates of
alcohol use and all drugs except marijuana are decreasing nationally, CDC, 2012).
Although there is not an apparent causal connection between the Second Order Change project
and improved youth indicators, the work ASD has done through this effort has kept it on a course
toward continuously more positive outcomes for youth.
American Institutes for Research Final Report: Evaluation of ASD’s Second Order Change Project—33
References
Aber, J. L., Brown, J. L., Chaudry, N., Jones, S. M., & Samples, F. (1996). The evaluation of the
Resolving Conflict Creatively Program: An overview. American Journal of Preventive
Medicine, Supplement to 12(5), 82–90.
Bar-On, R. (1997). The Bar-On Emotional Quotient Inventory (EQ-i): Technical manual.
Toronto, Canada: Multi-Health Systems.
Bar-On, R. (2005). The impact of emotional intelligence on subjective well-being. Perspectives
in Education, 23, 41–61.
Baumrind, D. (1966). Effects of authoritative parental control on child behavior. Child
Development, 37, 887–907.
Benson, P. L. (2003). Developmental assets and asset-building community: Conceptual and
empirical foundations. In R. M. Lerner & P. L. Benson (Eds.), Developmental assets and
asset building communities: Implications for research, policy, and practice (pp. 19–46).
New York: Kluwer Academic.
Bonny, A. E., Britto, M. T., Klostermann, B. K., Hornung, R. W., & Slap, G. B. (2000). School
disconnectedness: Identifying adolescents at risk. Pediatrics, 106, 1017–1021.
Brendtro, L. K., Brokenleg, M., Van Bockern, S. (1990). Reclaiming youth at risk: Our hope for
the future. Bloomington, IN: National Educational Service.
Brown, C. R., & Mazza, G. J. (1991). Peer training strategies for welcoming diversity. In J. C.
Dalton (Ed.), Racism on campus: Confronting racial bias through peer interventions (pp.
39–51). San Francisco: Jossey-Bass.
Brown, C. R., & Mazza, G. J. (1996). Anti-racism, healing and community activism. The
Humanistic Psychologist, 24, 391–402.
Brown, J. L., Roderick, T., Lantieri, L., & Aber, J. L. (2004). The Resolving Conflict Creatively
Program: A school-based social and emotional learning program. In J. Zins, R. P.
Weissberg, M. Wang, & H. J. Walberg, (Eds.), Building academic success on social and
emotional learning: What the research says (pp. 151–169). New York: Teachers College
Press.
Catalano, R. F., Haggerty, K. P., Oesterle, S., Fleming, C. B., & Hawkins, J. D. (2004). The
importance of bonding to school for healthy development: Findings from the social
development research group. Journal of School Health, 74, 252–261.
Centers for Disease Control and Prevention (CDC). (2012). Youth risk behavior surveillance—
United States, 2011. Morbidity and Mortality Weekly Report, 61.
American Institutes for Research Final Report: Evaluation of ASD’s Second Order Change Project—34
Coker, J. K., & Borders, L. D. (2001). An analysis of environmental and social factors affecting
adolescent problem drinking. Journal of Counseling and Development, 79, 200–208.
Curwin, R. L. & Mendler, A. N. (1988). Discipline with dignity. Alexandria, VA: Association for
Supervision and Curriculum Development.
Darling-Hammond, L., LaPointe, M., Meyerson, D., Orr, M. T., & Cohen, C. (2007). Preparing
School Leaders for a Changing World: Lessons from Exemplary Leadership
Development Programs. Stanford, CA: Stanford University, Stanford Educational
Leadership Institute.
Davis, S., Darling-Hammond, L., LaPointe, M., & Meyerson, D. (2005). School leadership
study: Developing successful principals (Review of Research). Stanford, CA: Stanford
University, Stanford Educational Leadership Institute.
Durlak, J. A., Weissberg, R. P., Dymnicki, A. B., Taylor, R. D., & Schellinger, K. B. (2011). The
impact of enhancing students’ social and emotional learning: A meta-analysis of school-
based universal interventions. Child Development, 82, 405–432.
Forthun, L. F., McCombie, J. W., & Freado, M. (2006). A study of LSCI in a school setting.
Reclaiming Children and Youth, 15, 95–102.
Forthun, L. F., Payne, C., & McCombie, J. W. (2009). LSCI in a School Setting: Final Results.
Reclaiming Children and Youth, 18, 51–57.
Grskovic, J. A., & Goetze, H. (2005). An evaluation of the effects of Life Space Crisis
Intervention on the challenging behavior of individual students. Reclaiming Children and
Youth, 13, 231.
Henry, K. L., & Slater, M. D. (2007). The contextual effect of school attachment on young
adolescents’ alcohol use. Journal of School Health, 77, 67–74.
Hoffman, L. L., Hutchinson, C. J., & Reiss, E. (2009). On improving school climate: Reducing
reliance on rewards and punishment. International Journal of Whole Schooling, 5, 13–24.
Jennings, P. A. & Greenberg, M. T. (2009). The prosocial classroom: Teacher social and
emotional competence in relation to student and classroom outcomes. Review of
Educational Research, 79, 491–525.
Karp, H. B. & Sammour, H. Y. (2000). Workforce diversity: Choices in diversity training
programs & dealing with resistance to diversity. College Student Journal, 34, 451–458.
Kerosky, M. E., Chaney, K., & Kendziora, K. (2008). Anchorage School District 2007 Youth
Risk Behavior Survey (YRBS). Anchorage, AK: Anchorage School District, Office of Safe
and Drug Free Schools.
Kumar, R., O’Malley, P. M., Johnston, L. D., Schulenberg, J. E., & Bachman, J. G. (2002).
Effects of school-level norms on student substance use. Prevention Science, 3, 105–124.
American Institutes for Research Final Report: Evaluation of ASD’s Second Order Change Project—35
Kuperminc, G. P., Leadbeater, B. J., & Blatt, S. J. (2001). School social climate and individual
differences in vulnerability to psychopathology among middle school students. Journal of
School Psychology, 39, 141–159.
Kuperminc, G. P., Leadbeater, B. J., Emmons, C., & Blatt, S. J. (1997). Perceived school climate
and difficulties in the social adjustment of middle school students. Applied
Developmental Science, 1, 76–88.
Lamborn, S. D., Mounts, N. S., Steinberg, L. & Dornbush, S.M. (1991). Pattern of competence
and adjustment among adolescents from authoritative, authoritarian, indulgent, and
neglectful families. Child Development, 62, 1049–1065.
LaRusso, M., Romer, D., & Selman, R. (2008). Teachers as builders of respectful school
climates: Implications for adolescent drug use norms and depressive symptoms in high
school. Journal of Youth and Adolescence, 37, 386–398.
Locke, K. D. (2000). Circumplex Scales of Interpersonal Values: Reliability, validity, and
applicability to interpersonal problems and personality disorders. Journal of Personality
Assessment, 75, 249–267.
Long, N. J., Fecser, F. A., & Wood. M. M. (2001). Life Space Crisis Intervention: Talking with
students in conflict (2nd
ed.). Austin, TX: Pro-Ed.
Long, N.J., & Fecser, E (1997). Life Space Crisis Intervention Institute training manual.
Hagerstown, MD: Author.
Loukas, A., & Robinson, S. (2004). Examining the moderating role of perceived school climate
in early adolescent adjustment. Journal of Research on Adolescence, 14, 209–233.
Loukas, A., Suzuki, R., & Horton, K. D. (2006). Examining school connectedness as a mediator
of school climate effects. Journal of Research on Adolescence, 16, 491–502.
Mayberry, M. L., Espelage, D. L., & Koenig, B. (2009). Multilevel modeling of direct effects
and interactions of peers, parents, school, and community influences on adolescent
substance use. Journal of Youth and Adolescence, 38, 1038–1049.
McDaniel, T. R. (2008). Review of Conscious Discipline: Seven basic skills for brain smart
classroom management. Clearing House, 81, 282–283.
National Association of Secondary School Principals (NASSP) (2006). Breaking ranks in the
middle: Strategies for leading middle level reform. Reston, VA: Author.
Pittman, K. J., Irby, M., Tolman, J., Yohalem, N., & Ferber, T. (2003, March). Preventing
problems, promoting development, encouraging engagement: Competing priorities or
inseparable goals? Washington, DC: The Forum for Youth Investment, Impact Strategies,
Inc.
American Institutes for Research Final Report: Evaluation of ASD’s Second Order Change Project—36
Resnick, M. D., Bearman, P. S., Blum, R. W., Bauman, K. E., Harris, K. E., & Jones, J., Tabor
J., Beuhring, T., Sieving, R. E., Shew, M., Ireland, M., Bearinger, L. H., & Udry, J. R.
(1997). Protecting adolescents from harm: Findings from the national longitudinal study
on adolescent health. Journal of the American Medical Association, 278, 823–832.
Roberson, L., Kulik, C. T., & Pepper, M. B. (2001). Designing effective diversity training:
Influence of group composition and trainee experience. Journal of Organizational
Behavior, 22, 871–885.
Roeser, R. W., & Eccles, J. S. (1998). Adolescents’ perceptions of middle school: Relation to
longitudinal changes in academic and psychological adjustment. Journal of Research on
Adolescence, 8, 123–158.
Roeser, R. W., Eccles, J. S., & Sameroff, A. J. (2000). School as a context of early adolescents’
academic and social-emotional development: A summary of research findings. The
Elementary School Journal, 100, 443–471.
Ryan, A. M., & Patrick, H. (2001). The classroom social environment and changes in
adolescents’ motivation and engagement during middle school. America Educational
Research Journal, 38, 437–460.
Sala, F. (2002). Emotional Competence Inventory (ECI) Technical Manual. Boston: McClelland
Center for Research and Innovation, Hay Group.
Scales, P. C., Benson, P. L., Leffert, N., & Blyth, D. A. (2000). Contribution of developmental
assets to the prediction of thriving among adolescents. Applied Developmental Science, 4,
27–46.
Schaefer, E. S., & Edgerton, M. (1985). Parent and child correlates of parental modernity. In I. E.
Sigel (Ed.), Parental belief systems: The psychological consequences for children (pp.
287-318). Hillsdale, NJ: Erlbaum.
Simons-Morton, B. G., Crump, A. D., Haynie, D. L., & Saylor, K. E. (1999). Student-school
bonding and adolescent problem behavior. Health Education Research, 14, 99–107.
Welsh, W. N. (2001). Effects of student and school factors on five measures of school disorder.
Justice Quarterly, 18, 911–947.
i Table E-1. District and School SCCS Participation Across Alaska by Year
Year Participating
Districts Participating
Schools Valid Student
Surveys Valid Staff
Surveys
2006 15 148 24,732 3,453
American Institutes for Research Final Report: Evaluation of ASD’s Second Order Change Project—37
2007 14 150 22,411 3,315
2008 33 242 30,124 4,730
2009 24 225 26,949 5,177
2010 34 268 33,413 5,931
2011 26 250 22,481 4,982
Top Related