Project SAILSS: Final Evaluation Report
Anne Arundel County Public Schools
September 15, 2013
Project SAILSS: Final Evaluation Report
Anne Arundel County Public Schools
Submitted to:
Lori Snyder, NBCT
Office of Advanced Studies and Programs
Anne Arundel County Public Schools
2644 Riva Road
Annapolis, Maryland 21401
410-570-0315
Submitted by:
Sonia Jurich, MD, Ed.D., Senior Research Associate
Laura Taylor, M.A., Research Associate
RMC Research Corporation
1501 Wilson Boulevard, Suite 1250
Arlington, VA 22209
Phone: 703.558.4000
Fax: 703.558.4823
September 30, 2013
Sculpture created by students and Kevin Reese, artist in residence,
integrating visual arts and mathematics
as part of the SAILSS project
Location: Entrance to Willey H. Bates Middle School
Annapolis, Maryland
AACPS Project SAILSS – Final Evaluation Report Page | i
TABLE OF CONTENTS
TABLE OF CONTENTS .............................................................................................................. i
LIST OF TABLES ....................................................................................................................... iii
LIST OF FIGURES ..................................................................................................................... iv
EXECUTIVE SUMMARY .......................................................................................................... v
INTRODUCTION......................................................................................................................... 1
BACKGROUND ........................................................................................................................... 3
The Grantee ......................................................................................................................................3
The Project .......................................................................................................................................4
IMPLEMENTATION OUTPUTS .............................................................................................. 7
Preparing Teachers for Arts Integration ..........................................................................................7
1. Off-campus professional development activities ................................................................ 7
2. In-house professional development activities...................................................................... 8
3. Modeling Arts Integration: Artists in Residence and Arts Teachers ................................... 9
From Professional Development to Classroom Application .........................................................11
From the Classroom to the Neighborhood .....................................................................................12
From Neighborhood to the Community .........................................................................................13
INTERMEDIATE OUTCOMES .............................................................................................. 14
Changes in School Organization ....................................................................................................15
Changes in Teachers’ Perceptions .................................................................................................17
1. Arts integration as a teaching strategy ........................................................................... 17
2. Arts integration and student engagement ....................................................................... 18
3. Arts integration and school climate ................................................................................ 20
Changes in Teaching Practices ......................................................................................................23
Changes on Students’ Engagement ................................................................................................26
1. Perceptions about arts integration ...................................................................................... 26
2. Observed behavior ............................................................................................................. 29
AACPS Project SAILSS – Final Evaluation Report Page | ii
PROJECT IMPACT ................................................................................................................... 31
Student Demographics ...................................................................................................................31
Academic Performance ..................................................................................................................33
1. Descriptive analysis........................................................................................................ 33
2. Comparison of means ..................................................................................................... 36
3. Effect size analysis ......................................................................................................... 37
4. Cohort Analysis .............................................................................................................. 38
Monitoring Project Objectives .......................................................................................................38
CONCLUSIONS ......................................................................................................................... 43
REFERENCES ............................................................................................................................ 47
APPENDIX A: EVALUATION DESIGN ................................................................................ 48
AACPS Project SAILSS – Final Evaluation Report Page | iii
LIST OF TABLES
Table 1: Average results for statewide assessments for Bates and AACPS ................................... 3
Table 2: Analysis of standards targeted in arts integration lessons .............................................. 12
Table 3: Comparison of means – student support scale (2009 and 2012 surveys) ....................... 23
Table 4: Integrating arts with core content areas .......................................................................... 28
Table 5: Students’ perceptions that arts improve understanding in core content areas ................ 29
Table 6: Average yearly attendance per school from SY 2009 to SY 2012 ................................. 30
Table 7: Average attendance by school and grade (SY 2013) ...................................................... 30
Table 8: Student demographics by school (SY 2012-2013) ......................................................... 31
Table 9: Results from the 2013 MSA Reading and Mathematics ................................................ 33
Table 10: Changes in percentages of students at or above proficient in the MSAs between 2008
and 2013 .................................................................................................................................. 35
Table 11: Results for the 2013 MSA disaggregated by grade and demographics ........................ 35
Table 12: Results of the comparison of means analysis (2013 MSA) .......................................... 36
Table 13: Effect size analysis for MSA Reading and Mathematics (SY 2009 and SY 2013) ...... 37
Table 14: Results in the MSA Reading and Mathematics for Grade 8 students disaggregated by
years in SAILSS (2012 and 2013 Cohorts)............................................................................. 38
Table 15: Percent of students with 5% or greater increases on the MSA mathematics ................ 39
Table 16: Changes in the 2012 and 2013 MSA mathematics for students at advance levels ....... 40
Table 17: Number of discipline referrals and referrals per student (SY 2009 to SY 2013) ......... 41
Table 18: Number of discipline referrals and referrals per student by grade (SY 2013) .............. 41
Table 19: Number of suspensions and average suspension per student (SY 2013) ...................... 42
Table 20: Number of suspensions and average suspension per student by grade (SY 2013) ....... 42
AACPS Project SAILSS – Final Evaluation Report Page | iv
LIST OF FIGURES
Figure 1: Proposed logic model for the SAILSS Project ................................................................ 6
Figure 2: Announcement of the Arts Showcase in February 2012 ............................................... 13
Figure 3: SAILSS' model for arts integration ............................................................................... 14
Figure 4: Use of arts integration (2010 Faculty Survey) .............................................................. 18
Figure 5: Influence of arts integration on student engagement (2010 Faculty Survey) ................ 19
Figure 6: Changes in responses to the student support scale between 2009 and 2012 ................. 22
Figure 7: Art as motivation to learning (percentage of agree/strongly agree responses) ............. 27
Figure 8: Percentage of Bates students disaggregated by race/ethnicity from 2005-2013 ........... 32
Figure 9: Percentage of FARM-eligible students at Bates across grant years .............................. 33
Figure 10: Percentage of students at or above proficient on the 2013 MSA Reading .................. 34
Figure 11: Percentage of students at or above proficient at the 2013 MSA Mathematics ............ 34
AACPS Project SAILSS – Final Evaluation Report Page | v
EXECUTIVE SUMMARY
In the summer of 2008, Anne Arundel County Public Schools (AACPS) applied for and
was awarded a grant from the U.S. Department of Education, Office of Innovation and
Improvement, to implement the Supporting Arts Integrated Learning for Student Success
(SAILSS) project. The project reflected a partnership among AACPS, Maryland Hall for the
Creative Arts, the Annapolis Arts Council, Arts Education in Maryland Schools Alliance
(AEMS), and Towson University. The grant extended from school year (SY) 2008-2009 through
SY 2012-2013. SAILSS project proposed a comprehensive cross-curricular, arts integrated
model for middle school students in high poverty, high mobility areas. The model included (1)
focused, systemic school-wide professional development; (2) mentoring of core content area
teachers by art teachers and artists in residence to build capacity related to arts integration; (3)
focused use of arts integration strategies to teach standards where students showed the greatest
challenges; and (4) involvement of families and community. RMC Research Corporation
conducted the evaluation of the program through its five years.
Main findings from the five grant years included:
Implementation
o All Bates teachers and administrators attended workshops and/or conferences related
to arts integration both off-campus and in-house; attendees to off-campus activities
presented to school faculty upon their return.
o Six Bates teachers completed the two-year Post-Baccalaureate Certificate in Arts
Integration at Towson University.
o All Bates teachers were offered opportunities to work side-by-side with artists in
residence and arts teachers to develop and implement arts integration lessons.
o Teacher mentoring followed three models – long-term (artists in residence worked
with specific teachers or classrooms for a full semester); short-term (artists in
residence worked with teachers and/or classrooms on a time-limited project); and
peer-to-peer (content area and arts teachers worked together on specific lessons).
o A position of Arts Integration Specialist (AIS) was created to help in the planning
and implementation of professional development, ensure availability of resources,
and support teachers as they designed and implemented arts integration lessons.
AACPS Project SAILSS – Final Evaluation Report Page | vi
o Approximately 700 arts integration lessons and projects for grades 6-8 were
developed and piloted; the lessons focused on standards where students
demonstrated greater difficulties and included core content areas, elective courses,
special education, and English as a Second Language.
o Through the grant years, the school implemented quarterly showcases and yearly
festivals as vehicles to display student work and involve the community. The
presentations were attended by families, members of the Board of Education and
County Board, invited artists and partners.
o The project director, the AIS, and Bates teachers presented in state and national
conferences and workshops to disseminate the model.
Intermediate Outcomes
o Creation of an Arts integration Specialist position at the central office to help arts
integration across all AACPS schools.
o Change in teachers’ perceptions – higher acceptance of arts integration as a viable
instructional strategy for improving student engagement, student ability to make
connections with other disciplines and overall academic performance.
o Changes in teachers’ practices – arts integration became an integral part of the
strategies available to teachers – the “Bates way” to address student’s needs.
o Changes in students’ engagement in learning – through the years, responses to the
students’ surveys indicated increased student engagement and better understanding
of concepts, particularly in English and Mathematics, when learning through arts.
o Classroom observations showed the majority of students actively engaged and
collaborating with each other.
o Attendance – Although already high at the beginning of the grant, attendance rate (as
percentage of days enrolled) has grown and is now 96.4%; attendance increase at
Bates was slightly larger than that at the comparison school.
Long-term outcomes
o Results in the 2013 Maryland State Assessments (MSA) Reading indicated that
Bates’ students outscored their peers in the comparison school and state averages in
all three grade levels.
AACPS Project SAILSS – Final Evaluation Report Page | vii
o At the 2013 MSA Mathematics, students at Bates did better than students in the
comparison school in grades 6, 7 and 8. For grade 7, Bates’ students outscored the
district and the state percentages.
o When Bates data are disaggregated by student subgroup, FARM-eligible students
scored lower than their non-Farm peers. Likewise, students from the two largest
minority groups (African American and Hispanic/Latino) tended to score lower than
White students. Mean differences were statistically significant with 95% confidence
level.
o An effect size analysis was conducted on the mean scores for the MSAs Reading and
Mathematics between 2009 and 2013. For Bates, medium effect size was found in
grades 6 and 7 Reading, and grade 7 Mathematics, while a small effect size was found
for grade 8 Reading. Grade 8 Mathematics showed no program effect on test results.
The comparison school, which represents a traditional approach (without arts
integration) showed negligible or negative effect sizes for all grades in Mathematics
and grades 6 and 8 in Reading. A small effect size was found in grade 7 Reading.
Therefore, arts integration appears to be more effective in increasing student learning
as measured by state assessments, in contrast to teaching strategies that exclude arts
integration.
o A cohort analysis was conducted on the MSAs results for students who have been in
Bates for one, two and three years. Results from the analysis suggested that time in
the school had no influence on test results.
o The analysis of disciplinary referrals showed improvements in both the treatment
(Bates) and the comparison schools. Between 2009 and 2013, discipline referrals in
Bates declined by 80% (from 746 to 152), and by 82% at the comparison school
(from 674 to 122). However, within the same time period, the number of suspensions
per student declined by 71% in Bates and 52% at the comparison school. These
results suggest that although the numbers of disciplinary referrals have declined
equally at both schools, the severity of referrals declined further at Bates (thus the
smaller number of suspensions).
AACPS Project SAILSS – Final Evaluation Report Page | 1
INTRODUCTION
The Arts in Education—Model Development and Dissemination (AEMDD) grant
program was authorized under the No Child Left Behind Act of 2001 (Public Law 107-110), Title
V, Part D, Subpart 15, Sec. 5551. The program is managed by the Office of Innovation and
Improvement (OII) at the U.S. Department of Education (USDE). Using a competitive grant
system, AEMDD provides funds to local education agencies and nonprofit or governmental arts
organizations to “support the enhancement, expansion, documentation, evaluation, and
dissemination of innovative, cohesive models that demonstrate effectiveness” to attain its goals
(USDE, 2008, p. 222). These goals are: (1) integrate standards-based art education into the core
elementary and middle school curricula; (2) strengthen standards-based arts instruction in those
grades; and (3) improve students’ academic performance, including their skills in creating,
performing, and responding to the arts. Applicants are encouraged to form partnerships with
school districts, independent schools (including charters), arts organizations, community
organizations, and/or institutions of higher education.
In the summer of 2008, Anne Arundel County Public Schools (AACPS) applied for and
was awarded a four-year AEMDD grant (award number U351D080013). The AACPS project,
Supporting Arts Integrated Learning for Student Success (SAILSS), reflected a partnership
among AACPS, Maryland Hall for the Creative Arts, the Annapolis Arts Council, Arts
Education in Maryland Schools Alliance (AEMS), and Towson University. RMC Research
Corporation was included in the proposal as the external evaluator for the project. At the end of
fourth project year, AACPS was awarded a one year no-cost extension. Therefore, the project
extended from school year (SY) 2008-2009 (2009) through SY 2012-2013 (2013).
The SAILSS project proposed a comprehensive cross-curricular, arts-integrated approach
for middle school students in high poverty, high mobility areas. The vision for the project was
based on the assumption that art, in its various forms, engage students’ sensory, intellectual,
physical, social, aesthetic, and emotional aptitudes. Consequently, art has the potential to
improve the learning process for all students.
Founded upon this theory of change, SAILSS was implemented in one middle school in
Anne Arundel County, Maryland, with the goal of improving mathematics learning for students
from grades 6 through 8. The project relied upon a four-prong foundation that included: (1)
AACPS Project SAILSS – Final Evaluation Report Page | 2
professional development for all teachers in the school; (2) mentoring of core content area
teachers by art teachers and artists in residence to build capacity related to arts integration; (3)
focused use of arts integration strategies to teach standards where students showed the greatest
challenges; and (4) involvement of families and community.
This report reflects findings from evaluation activities conducted during the grant years,
from September 2008 through June 2013. The report focuses on themes rather than activities.
Descriptions of each evaluation activity implemented during the project years and their findings
have been detailed in the yearly evaluation reports, which are available at AACPS and/or RMC.
The report comprises five sections and an appendix, as such:
Background: Introduces the grantee and provides an overview of the project with a focus
on its theory of change.
Implementation Outputs: Summarizes the activities implemented and their immediate
outcomes (outputs).
Intermediate Outcomes: Describes changes in school organization, teachers’ perceptions
and school climate during the project years.
Project Impact: Analyzes the impact of the project on student behavior and academic
performances using a matched comparison design.
Conclusions: Provides an overview of the project implementation and discusses lessons
learned.
Appendix A: Describes the evaluation design and the methods for data collection and
analysis.
This report was written for a diverse audience that includes implementers, funders,
educators, and families. The purpose of the report is to provide readers with an overview of the
implementation of an arts integration project and its impact on teachers, students, and the school.
Graphics and tables are used to highlight results and to maintain the narrative flow. Technical
information is placed in the appendices.
AACPS Project SAILSS – Final Evaluation Report Page | 3
BACKGROUND
The Grantee
Anne Arundel County Public Schools (AACPS) is the 5th
largest school district in
Maryland and one of the 50th
largest in the country. In 2008, at the time of the grant application,
AACPS was serving approximately 74,000 students in more than 120 schools. AACPS schools
are organized in feeder systems such that a number of elementary schools send their students to
two or three middle schools, which then feed into one high school. The school involved in the
SAILSS project, Wiley H. Bates Middle School (herein referred to as Bates), is part of the
Annapolis High School feeder system, which serves the eastern part of the county. This area is
characterized by wide socio-economic and cultural diversity. The school, located in Annapolis,
the state capital, is surrounded by five federal housing projects. Four of the five elementary
schools that feed into Bates are Title I schools.
In 2008, Bates served 567 students in grades 6 through 8. Half of the student population
was African American, 30% Caucasian, 15% Hispanic, and 3% Asian. Students who qualified
for Free and Reduced Meals (FARM) program comprised 45% of the school population and 17%
received special education services. Of the 45 teachers in the school, 44% held advanced
professional certification, 44% held the standard professional certification, and 12% had
provisional certifications. The school had not made Adequate Yearly Progress (AYP) for two
consecutive years and was classified as a school in need of improvement. The percentage of
Bates students who passed the Maryland School Assessment (MSA) was on average 13
percentage points below the district levels for both reading and mathematics, as displayed in
Table 1.
Table 1: Average results for statewide assessments for Bates and AACPS
SCHOOL
YEAR CONTENT
GRADE 6 GRADE 7 GRADE 8
Bates District Bates District Bates District
2006-2007 Reading 65.0 79.4 55.2 74.6 46.5 73.5
Mathematics 63.2 78.9 54.3 71.5 51.9 69.2
2007-2008 Reading 64.5 83.2 71.6 83.6 57.1 76.0
Mathematics 67.9 82.0 61.7 78.9 67.6 72.6
Source: Maryland Report Card website, http://www.mdreportcard.org/
AACPS Project SAILSS – Final Evaluation Report Page | 4
The Project
Burnaford (2007) placed the origin of arts integration in the early 1900s, when educators
started to challenge the organization of the secondary school curriculum as a group of insulated
content areas. Proposals to reform secondary schools included reorganizing the curricula by
themes, rather than content areas; embedding interdisciplinary topics into content-specific
curricula; or adopting a full integration model. Gardner’s (1983) theory of multiple intelligences
was instrumental to the arts integration movement. In the early 1980s, Gardner advanced the
concept that human beings have different intelligences that account for their broader range of
potential. He identified eight types of intelligence: linguistic, mathematical, spatial, kinesthetic,
musical, interpersonal, intrapersonal, and naturalist. His theory highlighted the need to reach
students’ different intelligences by diversifying teaching methods (Armstrong, 2006). Art, with
its many forms (visual, dance, music, drama), addressed this need for diversification (Nelson,
2001). The integration of art into the academic content curricula seemed a logical approach to
address the variety of students’ intelligences that are reflected in their different learning styles.
SAILSS involved the design and implementation of a school-wide instructional model
that integrated the learning of different art forms into the core content areas. The project was
based on a triad of teacher-driven activities that included sustained and focused professional
development, curriculum writing, and curriculum implementation. Art-related enrichment
activities were also planned to bring the project beyond the school walls and involve the
community.
The SAILSS proposal described a number of professional development activities that
included three-week summer sessions in the first two years of the grant, access to Towson
University post-Baccalaureate Arts Integration Certification, and attendance at workshops,
conferences, and field trips. During the summer sessions, teachers were expected to develop and
pilot test lessons that integrated core content topics and various forms of art, including visual and
technical arts, dance, drama, and music. The lessons would incorporate interdisciplinary topics
and align with the AACPS Curriculum Frameworks and the Maryland State Voluntary
Curriculum. All Bates teachers were welcomed to the activities, although the emphasis was on
mathematics teachers. During the school year, arts teachers and artists in residence were to work
side by side with core content area teachers to implement arts integrated lessons and co-teach
lessons to expand the repertoire of arts integration techniques. Grant funds were geared toward
AACPS Project SAILSS – Final Evaluation Report Page | 5
tuition and fees, stipends for attendance at professional development and curriculum writing
activities outside of regular work hours, substitute teachers when activities coincided with school
hours, and purchase of resources needed to implement the arts integration lessons.
SAILSS emphasized student participation in art-related extracurricular activities, such as
drama club, creative writing club, orchestra, and others. Students were offered a one-week art
summer camp that included, among other activities, the piloting of arts integration lesson plans.
Student work was expected to be showcased at the school and at additional community locations.
Annual art celebrations and related activities were planned to involve families and other
community members.
Partnerships with arts organizations, such as the Maryland Hall for the Creative Arts, the
Annapolis Arts Council, and others, were an integral part of the project. The purpose of these
partnerships was to bring a variety of enrichment opportunities to school staff, students, and the
community, such as fieldtrips to art museums and exhibits, workshops and speakers.
The project, as proposed to the USDE, had five objectives that were reduced to four as
more analysis occurred. The four objectives included:
By the end of each academic year, 100% of the students (500 students) at Bates will be
engaged in instructional activities designed to reflect their diverse learning styles and
multiple intelligences, as measured by teacher activity logs.
At least 20% of all Bate students will participate yearly in extracurricular art-related
activities, countywide music and dance competitions, and art portfolios, starting in the
2009 academic year, as measured by enrollment logs.
At least 40% of all Bates students will increase their scores on the MSA in mathematics
by 5% yearly, starting with the 2009 MSA in mathematics.
At the end of each academic year, starting with 2009, the number of discipline referrals at
Bates will decline by at least 5%.
The external evaluation was embedded into the project as a feedback mechanism to
inform the decision-making process. Figure 1, on the next page, displays the project’s logic
model and the role of the evaluation within the model.
AACPS Project SAILSS – Final Evaluation Report Page | 6
Figure 1: Proposed logic model for the SAILSS Project
Supporting Arts Integration for Student Success (SAILSS) Project Logic Model
Resources
Par
tner
sE
du
cato
rsS
tud
ents
AACPS
Engagement and
Collaboration
Arts in Education
Partnership
Network
Facilities,
Equipment and
Materials
Educators
Skilled in Arts
Integration
Evidence-Based
Curriculum
Partners
Increased Commitment to
Arts in Education
Educators
Increased/Enhanced Arts
Integration Practices and
Techniques Among
Educators
Increased Educator
Perceptions of
Psychosocial Dimensions
in the School
Environment
Students
Arts Integration
Products and
Reflections
Educators
Curriculum
and Program
Development
Number
of Educators
Participating in
Professional
Development
Activities
Partners
Provide
Professional
Development
for Educators
Provide
Fieldtrips for
Educators and
Students
Educators
Attend
Professional
Development
Attend
Fieldtrips
to Art
Museums
Students
Perform
Classroom
Arts
Integration
Activities
Attend
Class
Fieldtrips
to Art
Museums
Partners
& Educators
Educators
Skilled and
Competent in
Arts
Integration
Practices and
Techniques
Educators
Applying
Integration
Practices and
Techniques in
the Classroom
Students
Students with
Positive
Attitudes
Toward
Learning
Activities Outputs Outcomes
Partners &
Educators
Partners &
Students
Number
of Students
Participating in Art
Activities Outside of
School
Impact
EvaluationFeedback Feedback
Students
Student Improved
Academic Performance
Decreased Discipline
Referrals and
Suspensions
AACPS Project SAILSS – Final Evaluation Report Page | 7
Classroom Observations Instrument 7
IMPLEMENTATION OUTPUTS
This section focuses on the immediate results from the implementation of SAILSS.
Information was provided by the AIS or collected by the evaluators through classroom
observations and interviews.
Preparing Teachers for Arts Integration
1. Off-campus professional development activities
SAILSS funds allowed Bates faculty to attend more than 20 off-campus PD activities
across the project years. The activities included visits to arts museums and exhibits accompanied
by workshops. Upon return, attendees were required to share the information with school faculty
during staff meetings or professional development days. Attendees included administrators,
department chairs and core content area and art teachers. Grant funds paid for fees, per diem and
substitute teachers (when appropriate) for more than 100 individual participations in off-campus
workshops and conferences. In the fourth project year alone 22 Bates teachers (unduplicated
numbers), in addition to interns, other school staff, and other AACPS staff, attended workshops
and conferences in the Washington Metropolitan area for a total of 64 attendees.
During the first project year (2008-09), off-campus activities focused on mathematics and
involved teachers and students from grades 6 to 8. Workshops and field trips included the
Walters Arts Museum, in Baltimore for a presentation on Mosaic, Mummies and Math; the
Baltimore Museum of Art for a tour accompanied of a lecture on Math at the Museum; and the
National Building Museum in Washington, D.C., to learn about Architecture and Geodesic
Domes.
For the second project year (2009-2010), attendees explored different opportunities that
included the Crossing Borders Conference at the University of Maryland, Baltimore Campus
(UMBC); a meeting with the Maryland Arts Integration Network (MAIN); the Neuro-education
Summit at the Johns Hopkins University; the AEMS Tri County Conference (which involved
faculty from Montgomery, Howard and Anne Arundel Counties); the Picturing America
Workshop at the Philadelphia Museum; and Exploring the Early Americas at the Library of
AACPS Project SAILSS – Final Evaluation Report Page | 8
Congress. Additionally, Bates hosted the 21st Century Institute, with the participation of the tri-
county consortium and MAIN.
The third project year (2010-2011) provided a mix of content-focused and systemic-
focused activities. Content-focused workshops included the Mathematics and Art, at the
Baltimore Museum of Art, and the Artifact Analysis at The Walters Art Museum, also in
Baltimore. Systemic perspectives were provided in the Changing Education through Arts
(CETA) workshops at the Kennedy Center, in Washington D.C, the Re-envisioning Education, at
the UMBC, and the AEMS Tri County Conference. The 21st Century Institute, initiated the
previous year under SAILSS’ auspices, became a self-sustained activity sponsored by AACPS
(from general funds) and partners. The Institute provides arts integration workshops to teachers
from Bates and other schools in and outside the county.
The fourth project year (SY 2011-2012) included attendance to the 21st Century Arts
Integration Institute, the Institute at the Freer Gallery, the Teaching Artist Institute III at the
Young Audience Arts for Learning Maryland (YAMD), and the Engaging with Arts Education at
Towson University. Workshops included Digital Viewing at the National Postal Museum, Story
of America Through Art at the Smithsonian American Art Museum, Cherry Blossoms lecture at
the Library of Congress, African Americans in the National Portrait Gallery at the National
Portrait Gallery, Washington, DC, and African American Artists at the Smithsonian American
Art Museum. Additionally, six Bates teachers completed the post-Baccalaureate certification in
arts integration, at Towson University. The cohort comprised the AIS, two reading teachers, two
English Language Arts teachers, and a mathematics teacher. The certification holders worked as
mentors to other teachers during the development and implementation of arts integration lessons.
2. In-house professional development activities
At the beginning of the second project year, Bates teachers were offered in-house
sessions provided by the AIS, artists in residence, and other arts integration experts. Four
sessions were provided per day to give all Bates teachers the opportunity to attend the
workshops. For the 2009-2010 project year, workshops included Introduction to Arts Integration,
Digital Documentation, Elements of Drama, Elements of Dance, Drama Games, Artful Thinking,
Artful Thinking and Higher Order Thinking Skills, and Bring your Best Arts Integration
Reflection. The Arts Integration Fridays continued in 2010-2011 with topics that included
Introduction to Arts Integration Resources, Digital Documentation Basic and Advanced, Priming
AACPS Project SAILSS – Final Evaluation Report Page | 9
with Arts Integration, Drama Games, Music in your Classroom, Photography and Resources, Art
and Drama in the Classroom, Islamic Art and Culture, and Beyond I See, I Think and I Wonder.
Moving to Thursdays for the fourth project year, the workshops included Principles of
Design, Dance, Rap and Poetry, Artful Thinking, Using Digital Cameras in Classroom, Art
Inspire Writing in the Curriculum, Technology and Arts Integration. In addition to these
activities provided by Bates teachers, presentations from the artists in residence included
Movement, Process Drama, Summary Songs, and Pottery and Text. Arts teachers at Bates
worked with core content area teachers to model arts integration lessons with topics such as Jazz
and Harlem Renaissance, Hip-Hop Geometry, Printing Spanish, Pop Art Spanish, Math Scale
Drawings, and Science Watercolor. All content area teachers were also involved in half-day
hands-on activities related to arts integration.
During the final project year, in-house teachers provided workshops on Orientation to
Arts Integration in the Classroom. Presentations from artists in residence included: Arianna Ross
presented on Storytelling, Ali Kruger discussed Drama Games, Tim Gregory and Laura Groo
presented on Arts Integration and Common Core, and Michael Lamason presented on Story
Boarding.
The off-campus activity provided lead teachers and interested teachers at Bates with
opportunities to hear from experts and update their knowledge of arts integration. Concomitantly,
in-house activities introduced all teachers to the basics of arts integration and provided them with
knowledge and support in order to plan and implement lessons integrating arts.
3. Modeling Arts Integration: Artists in Residence and Arts Teachers
SAILSS used three models of mentoring: long-term, short-term, and peer-to-peer. The
two first models involved artists in residence. In the long-term model, the artist in residence
worked for a couple of months or a semester side by side with teachers. The objective was to
provide enough time for teachers and students to build rapport with the artist, and for the artist to
know the school and gradually build teachers’ capacity in arts integration. In the short-term
model, the artists came to the school for a couple of weeks and worked directly with a classroom
or a group of classrooms on a focused, specific project. The peer-to-peer model involved Bates
arts teachers working together use the core content area teachers to create and model arts
integration lessons
AACPS Project SAILSS – Final Evaluation Report Page | 10
As an example of the long-term model, during SY 2008-09, Robert Levit worked year-
round with mathematics and art teachers to develop a variety of projects that involved
mathematics, music and visual arts. Examples of the short-term model for the same year
included: the artist David Cunningham and students created a large acrylic mural that adorns the
lobby of the school cafeteria on a project that involved mathematics and visual arts; Footworks
Percussive Dance Company provided a 10-day workshop to highlight the contributions of Irish
and African communities to the American culture – the workshop concluded with students
presenting to the community; Bob Benson, from Shiny Happy Things, worked with teachers to
create a mirror tree that integrated mathematics and visual arts. The tree is in permanent display
at the Bates Arts Center.
Bob Benson and Robert Levit returned for the second project year. Benson helped
mathematics teachers integrate glass sculpture into their lessons; while Levit created 13 lessons
integrating music and visual arts into mathematics. Other artists in residence who came to Bates
on short-term assignments included: Ali Oliver, who worked with the sixth grade classes to
integrate social studies and creative opera and drama; Ryan Sellers, who helped the seventh
grade English language arts class use mime and tableau to interpret myths; Christina Delgado,
who worked with the social studies classes on the use of photography to define culture; and
Amanda Pellerin, who integrated visual arts and Italian language into social studies classes.
During SY 2010-11, Robert Levit continued his collaboration with the mathematics
teachers for the entire school year. Kevin Reese, in collaboration with the AIS and mathematic
teachers, implemented a one-week residency with the students to integrate concepts of
engineering and visual arts. The result of this collaboration was the 15 foot sculpture of brushed
aluminum and wood that adorns the school entrance. As part of the in-house mentoring, the AIS
facilitated collaboration between art teachers – including visual arts, dance, drama and music –
and teachers from other content areas to create and implement arts integration lessons.
Six artists in residence worked with Bates teachers and students during SY 2011-12. For
two months, Kevin Martin helped integrate music into science and English lessons with themes
as Physics of Steel Drums and Steel Drums in Caribe Culture. Ali Oliver returned for a full
month to help social studies teachers and students create two operas on Qin Dynasty and
Mesoamerica. Other artists who stayed for two months working with core content area teachers
included Joan Gaither (visual arts), Tim Gregory (music and movement), and Sue Trainer
AACPS Project SAILSS – Final Evaluation Report Page | 11
(music). Footworks returned for a workshop on American Diversity with social studies teachers
and students for grades 7 and 8. Visual arts and dance teachers at Bates co-taught about a dozen
lessons that integrated arts into social studies, English, mathematics, and foreign languages.
From Professional Development to Classroom Application
Since the start of the grant in SY 2008-09, time for collaborative planning was scheduled
for the development of arts integrated lessons and projects that involved teachers from core
content areas and art teachers or artists in residence. The teachers used the time to model the
lessons with each other and pilot with students. On many occasions, the AIS and arts experts
taught the lessons alongside the core content area teacher. Three teachers from Bates served on
district-wide curriculum writing teams that were responsible for the development of arts
integration units for Grades 6 and 8. The units are available for use by all AACPS middle school
teachers. It is of note that the process, while geared toward core content area teachers, ended up
by involving teachers from elective courses and specialized areas.
Teachers were encouraged to use arts integration lessons to target those standards where
students traditionally faltered. Department chairs, working with the AIS, were instrumental in
supporting faculty in these activities and in providing them with time for planning and review.
The AIS collected data from the quarterly benchmark tests and reported on the outcomes for the
targeted standards. Table 2, on the next page, displays information on benchmark results related
to standards taught with arts integration (from a 2012 presentation).
About 700 arts integration lessons and projects were developed and piloted during the
five project years. Lessons reflected all arts forms (visual, music, dance, drama) and all content
areas, including core areas (English, mathematics, science and social studies), foreign languages,
and specialized areas, such as English as a Second Language (ESOL) and special education. Arts
integration lessons also represented all grade levels. During the fourth grant period alone, 180
lessons were developed and implemented. Examples of such lessons include: Mehendi Art
Negative/Positive Spatial and Celestial Bodies, a collaboration between grade 6 science teachers
and the dance teacher; Area of Trapezoids with Bearden Collage, a lesson that integrate seventh
grade mathematics and visual arts; Silhouettes and Writing: Guatemalan Worry Dolls,
collaboration between the ESOL teacher and visual arts teacher; Harlem Renaissance Literature
and the Jazz Connection, for students in Grade 8, integrating English, social studies, and music;
AACPS Project SAILSS – Final Evaluation Report Page | 12
Tableau – How to Show Angle Relationships and Find Missing Angles, integrating drama and
mathematics for students in grade 7.
Table 2: Analysis of standards targeted in arts integration lessons
Grade Arts integration
Activity Social studies Content Standard
Correct Answers
Pre Post
6 Monochromatic
Mapping/Value
2.C.1.a . Identify reasons why people migrate, such
as economic opportunity, climate, and political
reasons
32% 88%
7 Economies/Creative
Comparisons/Dance
4.B.1.a Examine how different economic systems,
traditional, command, market, and mixed, answer
the basic economic question of what, how, and for
whom to produce
31% 74%
8 Federalism/Visual
Arts
1.A.1.b Explain and summarize the principles of
federalism, and how they protect individual rights
and impact the functioning of government 22% 81%
8
War of 1812/
Musical
Expressions
5.C. 2.a. Explain why the US adopted a policy of
neutrality prior to the War of 1812.
5.C.2.b. Explain how the continuing conflict
between Great Britain and France influenced the
foreign policy of the United States
27% 86%
From the Classroom to the Neighborhood
One prong of SAILSS-sponsored activities had the objective of bringing families and
communities into the school to celebrate student work and create a sense of school pride. During
the five project years, the school promoted not only quarterly showcases and but also larger
celebrations, such as the Family Heritage and Multicultural Arts Integration Festival and the
International Arts Festival. All the activities involved displays of student work and student
presentations. Families, Board of Education members, politicians, and representatives of the
county and state administration were invited to attend, thus bringing much attention to the school
and arts integration. Figure 2 displays part of a brochure announcing the 2012 February
Quarterly Showcase.
AACPS Project SAILSS – Final Evaluation Report Page | 13
Figure 2: Announcement of the Arts Showcase in February 2012
From Neighborhood to the Community
As Bates’ faculty became familiar with arts integration, a process of dissemination of the
proposed model was set in motion. This included presentations to faculty in other AACPS
schools and outside the school district. As early as the second project year, the cohort of Bates
teachers who were working toward their Arts Integration Certification at Towson University
presented to AACPS and Towson students on Art Forms and Artful Thinking. They also
presented at the Kennedy Center on Laying a Foundation: Defining Arts Integration and
Documenting Arts Integration.
During the third project year, for instance, Bates faculty presented before a total of 130
participants which included AACPS leadership, both principals and administrators, and attendees
to the Arts Education in Maryland Schools (AEMS) Conference, the Arts Integration Conference
at the University of Maryland, Baltimore Campus (UMBC), and the World Language County
Coordinators at the Maryland State Department of Education. Topics presented included Arts
Integration: Overview, Artful Thinking, Art of Storytelling, A Literacy Quartet and Arts
Integration and World Languages.
In the fourth project year, the AIS dedicated over 20 hours to presentations regarding arts
integration at diverse AACPS schools, the College Board Forum in New York, meetings for the
Pennsylvania and Maryland Foreign Language Associations, conferences for the National
Science Teachers and the American Council on the Teaching of Foreign Languages, and others.
Join us on Monday, February 13, 2012 to celebrate
Quarterly Arts Showcase
Black History Month
African American Parent Involvement Day
Valentine’s Day
The staff, students, and PTA invite all parents and guardians to join their Bates students to view our
artistic accomplishments and celebrate Black History Month.
AACPS Project SAILSS – Final Evaluation Report Page | 14
Topics included Motivating with Arts Integration, How to Integrate the Arts, Artful Thinking, and
Thinking through Art.
In summary, SAILSS presented a model of arts integration that can be replicated in
schools in different settings. Figure 3 displays schematically the model of arts integration
proposed by SAILSS.
Figure 3: SAILSS' model for arts integration
Off-campus PD
Neighboring Community
Focused lessons
School District
Community at large
SYSTEMATIC SUSTAINED
FOCUSED
AACPS Project SAILSS – Final Evaluation Report Page | 15
Bates has quarterly
benchmark tests, and we are
able to see what standard
was taught by arts
integration and it shows
improvement. The county is
very interested in the
results. The data is getting
teacher buy-in and the
principal is backing the
project 100%... Arts
Integration Specialist
INTERMEDIATE OUTCOMES
Intermediate outcomes are here defined as conditions that are expected to lead to changes
in academic performance. They include changes in school organization, teachers’ perceptions
and practices, and student engagement in learning. Data for this section were collected from
yearly faculty and student surveys, classroom observations, and interviews with project and
school staff. For details on the evaluation design and methods of data collection refer to
Appendix A. Findings from each independent activity are detailed in the evaluation reports
provided to AACPS in previous years. This section presents an overview of these activities and
changes through time.
Initially, surveys were conducted at Bates and a comparison school (School A). However,
School A started to implement arts integration at the very onset of SAILSS and soon became an
arts magnet school, thus invalidating any comparison. A second comparison school (School B)
was selected at the beginning of the third grant year but comparisons were limited to analysis of
student performance data.
Changes in School Organization
Changes in the school organization during the SAILSS
period included the creation of the Arts integration Specialist
(AIS) position and the opening of a magnet program in
Performing and Visual Arts (PVA). Both changes occurred in the
second project year (2009-10).
The AIS position was created to coordinate the schools’
activities and mentor teachers as they integrated arts into their
lessons. The role encompassed a number of functions that
included working with the teachers during the planning and
development of the lessons, helping in the implementation of the lessons through modeling and
co-teaching, and supporting the evaluation activities.
The teacher hired for the AIS position was working toward her Arts Integration
Baccalaureate Certification (which she completed). The position was fully supported with
AACPS Project SAILSS – Final Evaluation Report Page | 16
SAILSS funds during the grant period, but AACPS was planning to sustain the position with
general funds after the grant expired. Eventually, the school district created a districtwide AIS
position to provide similar supports across its schools.
The PVA magnet program opened at Bates in September 2009. Technically, the program
is not an output from SAILSS, since it was planned before the grant was awarded. However,
according to PVA staff, SAILSS contributed to buy-in within and outside the school, and
facilitated the interaction between the magnet program and school wide activities.
PVA students come from feeder systems in the southern part of the county. As the name
indicates, the program focuses on visual and performing arts, and offers a variety of classes that
include instrumental music, dance, cinema, computer generated art, painting, sculpting, among
other activities. In addition to the core content areas (mathematics, English, science, social
studies), the program offers a course on World & Classical Languages with a focus on Italian.
Students must apply to the program and part of the application process includes an audition. The
students select one of four pathways: dance, instrumental music, visual arts and vocal arts.
Students who complete the three-year program gain high school credit in Algebra and foreign
languages. Qualified arts instructors and artists in residence provide studio instruction during the
Performance Plus sessions (Wednesdays and Thursdays from 3:30 to 4:30 pm). The program
also offers eight Saturday sessions that include field trips to museums and performances within
the Baltimore-Washington, DC area.
Interviews with teachers and parents suggest that the introduction of the magnet program
had initial challenges, as it created a divide between PVA and non-PVA students. SAILSS
helped this divide disappears, since all Bates students are involved with arts in core content
classes, and actively participate in showcases and festivals. SAILSS made “the playing field
more even,” commented a 7th
grade teacher.
During individual and group interviews, the teachers agreed that the transformation of
Bates into an arts integration school and the opening of the magnet program changed the way the
community sees the school. Bates is now perceived by the community as a school that provides
students with holistic and high quality educational opportunities. Parents who would previously
send their children to private schools are staying and the school has seen an influx of students
who are academically oriented. Applications for the magnet program are already exceeding the
slots and the school has been forced to institute a raffle system for acceptance. One teacher
AACPS Project SAILSS – Final Evaluation Report Page | 17
explained that she left a position in the central office to teach at Bates because the school “has
developed a reputation for providing students with a unique opportunity to learn through the
arts,” and she wanted to be part of it.
In 2012 the school received three awards: The Award for Excellence and Innovation in
the Arts, from the College Board; the Maryland Schools of Excellence in Arts Education Award,
from the Arts Education in Maryland Schools Alliance (AEMS), and the Schools that Work
roster from Edutopia.
Changes in Teachers’ Perceptions
Bates faculty perceptions were assessed through two surveys: a faculty survey and a
school climate survey. The faculty survey, created by the AIS, focused on arts integration as a
teaching technique and its impact on teaching and learning. Using a pre/post design, the survey
was conducted in the fall and spring semesters of the second and third project years. The school
climate survey was conducted at the end of each project year and examined changes in
perceptions regarding the students and the school.1 Survey findings were complemented by
individual and group interviews with school faculty from different content areas and grade
levels.
1. Arts integration as a teaching strategy
Figure 4, on the next page, displays results from the pre/post survey conducted in the
third project year (2010). The responses indicate the percentage of teachers who gave moderate
to considerable ratings to the items. The survey results suggest that Bates teachers perceived
themselves as being moderately to considerably knowledgeable about arts integration, and saw
arts integration as a viable instructional strategy for improving achievement. They also felt
comfortable with integrating arts in their lessons. Yet, fewer than half of the survey respondents
perceived major changes in their teaching practices as a result of arts integration. Results were
similar for the 2009 and 2010 surveys.
1 Response rates for each survey were above 70%; for the faculty survey, paired-sample t-tests were used to compare
responses for a group of teachers who participated in the pre and post- application that year. T-tests and ANOVA
were used for the climate surveys. Analyses of the survey templates and findings are detailed in yearly evaluation
reports presented to AACPS.
AACPS Project SAILSS – Final Evaluation Report Page | 18
Figure 4: Use of arts integration (2010 Faculty Survey)
During interviews, teachers commented that, as they started SAILSS, integrating arts into
their lessons “seemed daunting,” and the AIS was central to the success of the project. The AIS
was described as always supportive, planning lessons together, finding resources, modeling, and
helping them “in any way possible.” According to survey respondents, the two most important
obstacles to implementing arts integration were lack of time for planning and lack of training.
However, as SAILSS implementation matured, teachers became more familiar with arts
integration. One mathematics teacher commented during an interview,
We teach with the arts to find ways to get students interested and it becomes second
nature; it’s universal, it’s done everywhere in the school—there is no resistance.
2. Arts integration and student engagement
Teachers’ perceptions that student engagement increased in response to the use of arts
integration was a finding from surveys and interviews as early as the second grant year. The
consensus among teachers was that arts integration provided a natural way to differentiate
instruction and tended to engage most students, even those students who were traditionally
disengaged. Teachers from all grade levels commented during interviews that students responded
differently to arts. Students might resist arts activities that involved skills with which they did not
0%
20%
40%
60%
80%
100%
pre post pre post pre post pre post pre post
Knowledge aboutarts integration (AI)
Participation inprofessional
development aboutAI
Belief that AI is aviable instructional
strategy forimproving
achievement
Comfort level withintegrating the artsinto your classroom
Changes in teachingpractice as a result
of AIimplementation
57.4%
67.9%
48.0%
67.5%
81.3% 90.4%
77.1% 77.4%
47.9% 54.9%
Co
nsi
der
able
+Mo
der
ate
resp
on
ses
AACPS Project SAILSS – Final Evaluation Report Page | 19
feel comfortable or which they saw as foolish or “uncool,” explained a teacher. Performance arts
activities were seen as particularly successful with hard-to-engage and shy students.
Alternatively, some teachers claimed that performance activities were too disruptive and
perceived their students responding better to “quieter” strategies, such as painting with
watercolors.
Interviewees suggested that lessons should provide options for students who were
reluctant to participate in one specific activity, such as inviting them to perform instead of
drawing, or vice versa. Overall, teachers agreed that their comfort level with arts influenced how
they utilized arts integration in the classroom and how well the students received it. As one
teacher commented, “if the teacher enjoys it, then the students will too.”
The pre/post teacher survey conducted in 2009 and 2010 included questions regarding
teachers’ perceptions of the influence of arts integration on students’ engagement in instructional
activities (engagement), ability to make connections across different content areas (connections),
motivation to learn (motivation), academic performance, and behavior. Figure 5 displays
“moderate to considerable positive” ratings for the survey conducted in the third grant year.
Figure 5: Influence of arts integration on student engagement (2010 Faculty Survey)
0%
20%
40%
60%
80%
100%
Pre Post Pre Post Pre Post Pre Post Pre Post
Engagement Connections Motivation Academicperformance
Behavior
84.5%
97.8%
66.7%
93.5%
82.2%
95.7%
68.2%
89.1%
77.8%
86.6%
Mo
der
ate
to L
arge
Po
siti
ve E
ffec
t (%
Res
po
nse
s)
AACPS Project SAILSS – Final Evaluation Report Page | 20
I absolutely love arts
integration—it’s been
amazing…I was afraid and
uncomfortable at first, but then
the kids started teaching me.
We had an artist in residence
to teach steel drums. [A special
needs student] asked to be in
the final performance. He had
never participated in anything!
In another occasion, we
worked on describing scenes
and surroundings, and we
“stepped into a painting.”
Everybody was participating.
It was one of those moments
I’ll never forget. It gives the
students a feeling that they can
[learn]. Teacher, first year in
Bates
. A comparison of means test for the 2010 survey indicated robust increase between the
pre and post-application for engagement (t=2.524, df=31, p<0.017), connections (t=3.350, df=31,
p<0.002) and academic performance (t=2.048, df=31; p<.002). That is, within one year of using
arts integration, teachers were considerably more likely to perceive these techniques as
improving moderately to considerably students’ engagement, their ability to make connections
with other disciplines, and their academic performance. Teachers’ perception of the program
influence on student behavior was already high (78%) in the pre-survey.
3. Arts integration and school climate
Research indicates that teachers in art-rich schools demonstrated more interest in their
work, were more likely to become involved in professional development activities, and were
more innovative than teachers in art-poor schools (Burton, Horowitz, and Abeles, 1999). The
climate survey tested whether SAILSS would build a more congenial climate at Bates. The
survey, conducted in May of the first (2009), second (2010) and fourth (2012) project years, used
the School-Level Environment Questionnaire (SLEQ) developed by Fisher and Fraser (1990).2
The instrument includes eight scales: student support, affiliation, professional interest,
participation in decision-making, innovation, resource adequacy, work pressure and staff
freedom. A detailed discussion of the instruments and the
results through the years is found in the 2012 Evaluation
Report.
Overall, Bates faculty scored quite high in the
affiliation, professional interest, innovation and resource
adequacy scales since the first survey (May 2009). Faculty at
Bates felt part of a broader community that was supportive
and easily integrated new members (affiliation scale). They
also perceived the school as a place that encouraged
professional growth and collaboration (professional
development scale), and inspired creativity and innovation
(innovation scale). The majority of these perceptions
remained relatively constant across the years. However,
2 The SLEQ was used in this project with permission of the authors.
AACPS Project SAILSS – Final Evaluation Report Page | 21
perceptions related to the professional development scale, albeit already strongly positive in the
2009 survey, showed robust improvements between the first and the fourth project years,
particularly for three statements: Teachers are keen to learn from each other – 2009 mean was
3.68 vs. 4.06 in 2012 (t=2.176; df=83; p<0.032); Teachers avoid talking with each other about
teaching and learning – mean of 1.97 in 2009 and 1.67 in 2012 (t=2.272; df=83; p<0.026); and
Teachers frequently discuss teaching methods and strategies with each other – mean of 3.82 in
2009 and 4.31 in 2012 (t=3.192; df=83; p<0.002). Compared to the beginning of SAILSS,
teachers in 2012 were more likely to see the school as strongly embracing professional growth.
Two scales showed lower means between the first and the fourth project years: resources
adequacy and work pressure. The resources adequacy scale measures teachers’ perceptions about
availability of resources in the school. Fewer teachers tended to perceive the schools as having
appropriate resources in 2012, when compared to 2009. However, these results may suggest
rather than a decline in resources, an increase in the demand for them, as more teachers utilize a
variety of materials for arts integration.
The work pressure scale measures how teachers perceive the pressure in their daily
working routine. Although responses for most items in this scale were similar for the two
surveys, increased agreement was found in one statement – There is constant pressure to keep
working. Mean response for this item in 2009 was 3.50 and 3.96 in 2012 (t=2.326; df=83;
p<0.022). Feelings of pressure may be a result of the large number of new teachers in the school,
the increase in student population, or changes in demand from central office. Yet, it can also be a
result from the longer time required to plan arts integration lessons, at least for first year
teachers, a comment heard in the interviews.
The participatory decision making scale measures the faculty’ perceptions regarding their
level of participation in the governance of the school. Responses to items in this scale showed
some ambivalence. Regarding decisions about the running of the school, more teachers perceived
that they were involved in the process in 2012, when compared to 2009. Regarding decisions
about teaching, more teachers indicated that they need to ask permission to the department head
before making decisions about teaching in 2012 when compared to 2009. That is, results from
the fourth project year suggest that Bates teachers see themselves as having more autonomy
regarding school governance than regarding decisions about instruction. However, large
AACPS Project SAILSS – Final Evaluation Report Page | 22
percentages of “not sure” responses (above 10% for most statements) challenge any conclusion
based on the small numbers of respondents (55 teachers, 85% response rate).
The only scale that showed strong improvement in the climate survey between the first
and the fourth project years was the student support scale. Figure 6 displays the percentages of
agree/strongly agree responses in the student support scale for the 2009 and 2012 surveys.
Figure 6: Changes in responses to the student support scale between 2009 and 2012
Findings from the 2009 and 2010 climate surveys suggested that teachers at Bates had a
more pessimistic view of their students, when compared to a similar school that was being used
as comparison. In the 2009 survey, items that used adverse words to describe students (e.g.
disruptive, difficult, noisy, badly behaved, or needing strict discipline) tended to receive a higher
There are many disruptive, difficult students in theschool.
Most students are helpful and cooperative to teachers.
Most students are pleasant and friendly to teachers.
There are many noisy, badly behaved students.
Students get along well with teachers.
Most students are well-mannered and respectful to theschool staff
Very strict discipline is needed to control many of thestudents.
52.9%
70.5%
72.6%
39.2%
82.3%
66.7%
56.9%
83.4%
43.8%
39.6%
79.2%
43.8%
22.8%
85.5%
Agree/Strongly Agree Responses
2009
2010
2012
AACPS Project SAILSS – Final Evaluation Report Page | 23
percentage of agree and strongly agree responses from Bates teachers than statements that used
constructive words (e.g. pleasant, friendly, helpful, well-mannered, respectful). In the 2012
climate survey, perceptions about the students had changed considerably. For instance, in the
2009 survey 83% of the teachers agreed/strongly agreed that there were many disruptive,
difficult students in the school. The percentage of agreed/strongly agreed responses for this item
fell to 56% in 2012, a 33% decline. Alternatively, the agree/strongly agree responses for the
statement “students get along well with their teachers” increased from 44% to 83%, a 47%
increase.
As displayed in Table 3, all seven items pertaining to the student support scale elicited
robust differences in responses between the two surveys, with statistical significance established
at 95% confidence level. By the fourth project year, Bates teachers were likely to agree/strongly
agree that their students were helpful, cooperative, pleasant, well-mannered and respectful.
Table 3: Comparison of means – student support scale (2009 and 2012 surveys)
STATEMENTS
Mean* t-test
2009 2012 t df p
There are many disruptive, difficult students in the
school. 4.29 3.35 4.481 83 0.000
Most students are helpful and cooperative to teachers. 3.03 3.57 2.458 83 0.016
Most students are pleasant and friendly to teachers. 3.18 3.69 2.408 83 0.018
There are many noisy, badly behaved students. 3.91 3.14 3.503 83 0.001
Students get along well with teachers. 3.06 3.80 4.715 83 0.000
Most students are well-mannered and respectful to the
school staff 2.65 3.49 3.730 83 0.000
Very strict discipline is needed to control many of the
students. 4.18 3.57 3.092 83 0.003
*Higher means indicate greater numbers of agree or strongly agree responses adverse wording statements are
expected to have lower means (more disagree/strongly disagree responses)
Changes in Teaching Practices
Already during the first school visit, in April 2009, the evaluators observed that the
classrooms and halls of the school displayed a number of reproductions of works by renowned
artists as well as students’ arts integration projects. Of the 14 teachers observed, 11 reported that
they had documented the arts integration activities implemented since the beginning of the
school year. They were also planning to showcase the student work throughout the school and
AACPS Project SAILSS – Final Evaluation Report Page | 24
Students used to display
avoidance behavior in class,
but now they want to stay. We
are working on a science
project about water
conservation here and in
Kenya. We are comparing
water levels and using colors
to signify shortages. Then
students will weave together
the color patterns into Kente
cloth. Some students have
even taken it a step further—
they are developing their own
water conservation plan for
the school. It’s forcing kids to
think on a higher level.
Science teacher
use the work for instructional purposes in future classes. Art-related activities were integrated
into the core content area lessons in 12 of the 14 classrooms observed, including classes where
arts integration were not supposed to occur. The activities involved different forms of art. For
instance, in an English class students were asked to read a play and recreate scenes by using
musical instruments and ambient sounds. Students in a mathematics class were painting triangles
on the color wheel that corresponded to equations. An artist in residence was helping a social
studies teacher to explain the Irish famine through dance.
During the second project year, the evaluators visited 18
classrooms. At the time, student arts integration products were
displayed across the school, starting with the entrance lobby.
Arts – dance, drama, drawing, poetry, writing, photography –
were being integrated in 17 of the 18 classrooms observed. In
one English class, students were asked to discuss moods in the
colors of a painting and draw pictures that corresponded to the
poem they had finished reading, while in another English class,
students were asked to sing a poem and act out Latin roots of
the words learned. In two math classes, students learned how to
increase the proportions of a portrait and draw it to scale, and in
a third class, students were identifying patterns in artwork and
creating bracelets by following the pattern. In the Italian Language class, students were asked to
identify words that described different portraits, while in the Spanish Language class the students
created sentences to describe picture or role-played with partners on representing words.
Teachers continually invited students to participate in the lesson and, in all but one class,
learning in or through the arts was promoted. Teachers used modeling about 83% of the
classroom time, and made frequent connections with students’ previous learning and between
disciplines. Examples from the arts and other disciplines were used in 17 classes for illustration
or demonstration.
Two mini-case studies were conducted during the spring semester of 2010 to investigate
the implementation of arts integration at Bates and its impact on instruction. The observation
included four 7th
grade pre-algebra classes and two 6th
grade English classes. Two pre-algebra
and one English teacher taught the classes, with the agreement that they would use arts
AACPS Project SAILSS – Final Evaluation Report Page | 25
Arts gives this school life, energy and respect…
it engages kids in a common interest—everyone
is equal. Arts integration is the tipping point of
bringing a broader range of kids back to the
school. PTA member
integration in one class (AI class) but not in the other (NAI class). The evaluators dropped the
observations of the English class in the second visit, when it became clear that arts integration
techniques were being used in both classrooms. Regarding the pre-algebra classes, AI classes
used more project-based strategies than NAI classes and therefore, student collaboration was
more frequent. Additionally, AI classes were more likely to connect the information being
learned to students’ previous learning experience, when compared to NAI classes. No significant
differences in benchmarks results were found between AI and NAI classes.
During SY 2010-11, another mini-study was conducted with three different teachers in
the mathematics department: a first year teacher, a veteran teacher who was seen by project staff
as resistant to arts integration, and another veteran teacher who was supportive of the project.
Pre-algebra and algebra classes were observed. Across the observed classes, all three teachers
showed similar levels of arts integration (about 10% of their class time). Three different arts
integration activities were observed during the visits: identify solutions of equations in a
Kandinsky painting; make metaphorical statements involving inequality relationships in a
Japanese painting, and draw relationships from a Harlem Renaissance painting. The activities
aimed at encouraging students to think creatively while making statements informed by
mathematics and logic. The teachers made comments about the paintings and urged students to
consider the cultural impacts of the Harlem Renaissance, thus introducing social studies themes
into mathematics. Two of the observed teachers also used contemporary songs to help students
remember the difference with inequalities.
Observed differences in student
behavior and participation were related to
content topic rather than use of arts
integration. Students in Algebra classes tended
to be more engaged than those in Algebra Readiness (pre-algebra) regardless of the teacher.
However, the more teachers integrated arts into their lessons, the more collaboration, critical
thinking and connection to previous learning experiences were observed. Benchmarks results
were not statistically different across teachers when content area was taken into account;
improvements were greater for the Algebra students than the Algebra Readiness students
regardless of teacher (Details of the mini-case studies are included in the 2010 and 2011
evaluation reports).
AACPS Project SAILSS – Final Evaluation Report Page | 26
The findings in the mini-study were corroborated by a series of group interviews
conducted during the spring semester of 2011. Teachers from all grade levels and content areas
agreed that they were integrating a variety of arts forms into their lessons and perceived an
increase in student engagement. The reasons cited for integrating arts included increase
comprehension of mathematical concepts, novels, and historical events, encourage creativity, and
elicit higher-order thinking skills activities (associate, compare, explain). Examples of arts
integration strategies employed included: using a wheel as a time allotment; creating a tableau
for vocabulary; using dance to focus or to control stress; using music and sound waves to
understand frequencies; linking artwork to the period covered in social studies; using drawings to
reinforce the plot moments in a story; and using artwork to help students visualize historic
events.
Interview and surveys showed that artful thinking was the most frequently used strategy
for all grades and content areas. Teachers commented that artful thinking activities forced
students to think outside the box and helped shy students to come out of their shells. Arts
integration was used as entry event, as a theme throughout the lesson, as a complement to a
specific point in the lesson, or as final event.
Overall, teachers agreed that planning for arts integration took more time than planning
for regular lessons, but they seemed to be able to adjust. Additionally, teachers perceived that
some content areas, such as English, were easier to integrate than mathematics. Additionally,
mathematics teachers felt that the standards they had to cover were too broad and they were
afraid of losing instruction time when integration arts.
Changes on Students’ Engagement
1. Perceptions about arts integration
Bates students participated in three yearly surveys that probed their perceptions about arts
integration. 3 Survey responses indicate that, overall, Bates students are receptive to the use of
arts as a strategy for making learning more engaging. Figure 7, on the next page, displays the
3 Survey findings are described in details in the yearly evaluation reports. A total of 345 students in grades 6-8
participated in the 2009 survey (60% response rate) and 586 (79%) participated in the 2012 survey.
AACPS Project SAILSS – Final Evaluation Report Page | 27
percentages of students who agreed/strongly agreed with 15 statements related to art as a
motivator for learning in the surveys conducted during the first and fourth project years.
Figure 7: Art as motivation to learning (percentage of agree/strongly agree responses)
I ask more questions in class when arts are part of thelesson
Sharing my art work with others is important for me
Presenting my art work to others makes me proud of myschoolwork
I feel energetic after I participate in arts activities
Sharing my art work reminds me of what I have learned
Describing what my art work means reminds me of what Ihave learned
I can concentrate best when I am participating in artsactivities
I pay more attention in class when arts are part of thelesson
Arts activities motivate me to participate in class
Expressing my knowledge through arts activities is exciting
I enjoy learning through the arts
I would like to do more activities involving arts in myclasses
Working with my classmates during arts activities is fun
Using arts to learn other subjects is enjoyable
Arts activities make learning fun
40.7%
46.5%
57.4%
58.7%
59.0%
61.3%
68.7%
71.3%
74.4%
77.3%
81.9%
82.7%
83.5%
84.9%
88.2%
32.7%
46.9%
60.9%
51.6%
56.7%
59.9%
64.6%
70.9%
74.7%
73.9%
79.7%
75.2%
83.4%
79.3%
75.3%
2009 2012
AACPS Project SAILSS – Final Evaluation Report Page | 28
A comparison of means between the 2009 and 2012 surveys indicated [statistically
significant] differences in five items: Arts activities make learning fun (t=5.070; df=911;
p<.000); Using arts to learn other subjects is enjoyable (t=2.173; df=911;p<0.03); I would like to
do more activities involving arts (t=2.740; df=911; p<.006); I feel energetic after I participate in
arts activities (t=2.096; df=911; p<.03); and I ask more questions in class when arts are part of
the lesson (t=2.419; df=911; p<.01). Students were more likely to agree/strongly agree with these
items in the 2012 survey when compared to the 2009 survey.
Students were also asked how they perceived blending arts into core content areas. Once
more, students were more likely to perceive that art projects blend well with the lessons in the
core content areas in 2012, when compared to 2009. Between the two survey years, all four
content areas showed increases in positive perceptions, with mathematics having the largest
change (21%), followed by English (12%). A possible explanation for this finding is the fact that
teachers, being more familiar with arts integration, were improving the process of integrating arts
to support learning. Table 4 displays the percentage of agree/strongly agree responses for the
2009 and 2012 surveys and the percentage of change between the two surveys.
Table 4: Integrating arts with core content areas
CONTENT AREAS Agree/Strongly Agree Responses (%)
% Change 2009 2012
English 70.3 78.8 +12.1
Math 63.1 76.1 +20.6
Science 69.0 73.0 + 5.8
Social Studies 74.2 77.2 + 4.0
The survey also probed students’ perceptions about the influence of arts integration on
their understanding of content in the core content areas. Options included “yes,” “no” and “don’t
know.” Table 5 displays the percentage of “yes” responses in the 2009 and 2012 surveys.
Relative to the number of students responding to the survey, considerably more students in 2012
stated that arts integration helped them have a new understanding in all core content areas, when
compared to students in the 2009 survey. The percentage of change was large, particularly for
English (an increase of 48%) and Mathematics (41% increase).
AACPS Project SAILSS – Final Evaluation Report Page | 29
I was working one on one with a new student from
[Latin America]. He had said he did not know
English. However, when asked to examine a painting,
he formed a list of English words describing the
painting: rooster, chicken, red, blue, yellow, green,
orange... He knew English, but didn’t feel comfortable
speaking. Arts integration allows for open
communication. Talking about the arts before going
into a reading or writing assignment gets students
ready. ESOL teacher
Table 5: Students’ perceptions that arts improve understanding in core content areas
CORE CONTENT AREAS “Yes” responses (%)
% Change 2009 2012
English 43.1 63.7 +47.8
Math 47.4 66.9 +41.1
Science 49.1 62.5 +27.3
Social Studies 54.6 71.2 +30.4
2. Observed behavior
During site visits, student engagement in the lesson was assessed through student body
language and behavior such as actively listening or watching the educator; questioning,
exploring, or discussing the topic; collaborating with each other or the educator on a project;
engaging in activities that require critical/creative thinking. During the visits, more than 70% of
the students in all classes were observed
to be engaged in instruction.
Instructional approaches that involved
peer collaboration generated the highest
level of observed engagement. Students
also were engaged during lessons that
linked prior learning to the task at hand.
The observers were not able to capture
differences in students’ levels of engagement during arts integration versus non-arts integration
classrooms. One reason for this difficulty was that arts integration was being used in most
classes, even those that had been reported to the observers as not integrating arts.
Attendance was also reviewed as a proxy for student engagement, based on the
hypothesis that students who feel engaged in learning are more likely to come to school and
remain for the day than students who feel disengaged. A review of the percentage of days
attended relative to days enrolled was conducted, as displayed in Table 6. It must be noted that a
large increase cannot be expected when the baseline is already high. However, growth in average
attendance at Bates surpassed that at School B in SY 2013. Average attendance for Bates grew
by 3% between SY 2009 and SY 2013, while the growth for School B was 2%. Both schools
surpassed the Maryland satisfactory standard for attendance (94%).
AACPS Project SAILSS – Final Evaluation Report Page | 30
Table 6: Average yearly attendance per school from SY 2009 to SY 2012
School SY 2009 SY 2010 SY 2011 SY 2012 SY 2013
Bates 93.6 93.8 94.8 96.4 96.4
School B 93.9 94.4 94.3 95.8 95.7
Table 7 lists attendance disaggregated by grade in 2013. Attendance at School B was
higher for students in Grade 6 when compared to upper-grade students. At Bates, however,
attendance was relatively the same through all three grade levels.
Table 7: Average attendance by school and grade (SY 2013)
School Grade 6 Grade 7 Grade 8
Bates 96.5 96.5 96.2
School B 96.0 95.9 95.3
AACPS Project SAILSS – Final Evaluation Report Page | 31
PROJECT IMPACT
This section starts with a summary of student demographics and changes across grant
years. It follows an analysis of results in the Maryland School Assessment (MSA) Reading and
Mathematics across time. Finally, the five-year progress toward the grant objectives is reviewed.
Comparisons are made, when appropriate, with another AACPS middle school that had a student
population similar to Bates and was not implementing arts integration (School B). Data for this
section was provided by AACPS, unless otherwise noted.
Student Demographics
Table 8 displays the number of students in Bates and School B disaggregated by grade
and demographic characteristics. Demographics included race/ethnicity, gender, and eligibility
for special education services, English Language Proficiency (ELP) services, and Free and
Reduced Meals (FARM) program, used as a proxy for low socio-economic status.
Table 8: Student demographics by school (SY 2012-2013)
Characteristics Bates School B
Number % Number %
Grade
6 243 32.6 235 32.7
7 246 33.0 229 31.8
8 256 34.4 255 35.5
Total 745 100.0 719 100.0
Race/
Ethnicity
Black/African American 230 30.9 387 53.8
White 286 38.4 114 15.9
Hispanic 177 23.8 134 18.6
Multi-racial 29 3.9 42 5.8
Other 23 3.0 42 5.8
Gender Female 405 54.4 359 49.9
Male 340 45.6 360 50.1
Free and Reduced Meals (FARM) 358 48.1 396 55.1
Special Education 63 8.5 80 11.1
English Language Proficiency (ELP) 73 9.8 53 7.4
AACPS Project SAILSS – Final Evaluation Report Page | 32
As seen in the table, both schools have a high composition of students from minority
backgrounds and low socio-economic status. Between SY 2009 and SY 2013, School B saw a
67% increase in the percentage of FARM-eligible students (from 33% to 55%), while the
proportion of FARM-eligible students at Bates remained constant around 48%. School B has
also a higher percentage of African American students when compared to Bates, which has a
more diversified population.
Figure 8 displays changes in the racial/ethnic composition of Bates’ students across the
years. The project started in 2009; data from 2005 through 2008 are provided as baseline. As
seen in the graphic, although the student population at Bates is still mostly from minority
backgrounds, the minority composition has changed considerably across the years. Between
2005 and 2013, the percentages of students classified as African Americans declined by 52%
(from 60% to 31%), while the percentages of students classified as Hispanic/Latino and from
other minorities, particularly multi-racial, more than doubled (from 15% to 31%). Within this
same period, the school experienced a 41% increase in the numbers of white students relative to
the total student enrollment (from 27% to 38%).
Figure 8: Percentage of Bates students disaggregated by race/ethnicity from 2005-2013
*SAILSS started in 2009; 2005-2008 are baseline years
0%
20%
40%
60%
80%
100%
2005 (588) 2006 (558) 2007 (567) 2008 (530) 2009 (580) 2010 (612) 2011 (701) 2012 (748) 2013 (745)
African American White Hispanic Other
AACPS Project SAILSS – Final Evaluation Report Page | 33
Figure 9 displays changes in the percentage of students classified as FARM-eligible
during the grant period. As seen in the graphic, FARM-eligibility remained relatively constant
across the project years (about half of the student population)
Figure 9: Percentage of FARM-eligible students at Bates across grant years
Numbers in parenthesis = school enrollment per year
Table 9: Results from the 2013 MSA Reading and Mathematics
Grade
Reading
Bates School B AACPS Maryland
N % N % N % N %
6 239 85.4 222 83.8 5,583 88.8 60,389 84.1
7 229 85.2 208 79.8 5.615 88.9 61,149 85.0
8 240 82.1 233 79.1 5,633 85.4 60,079 81.0
Mathematics
6 238 73.1 224 58.9 5,580 76.0 60,399 77.1
7 230 79.6 211 54.5 5,616 79.2 61,139 72.6
8 240 64.6 232 48.7 5,636 66.3 60,011 67.0 N= number of students taking the MSA
%= percentage scoring at or above proficient
Source: 2013 Maryland Report Card, http://www.mdreportcard.org/
Academic Performance
1. Descriptive analysis
Maryland middle school students take the Maryland School Assessments (MSAs) in
Reading and Mathematics at grades 6, 7 and 8. The assessments are conducted in April or May
of each year. Table 9, above, lists the number of students who took the 2013 MSAs Reading and
49.50% 50.20%
47.20% 48.80% 48.10%
30%
35%
40%
45%
50%
55%
60%
2009 (580) 2010 (612) 2011 (701) 2012 (748) 2013 (745)
AACPS Project SAILSS – Final Evaluation Report Page | 34
Mathematics and the percentages who scored at or above proficient. For comparison, the table
displays information for Bates, School B, the district (AACPS) and the state (Maryland).
Figures 10 and 11 display the percentages of students who scored at or above proficiency
in the 2013 MSAs Reading and Mathematics. In reading, results for Bates were above those for
the comparison school and the state average for all three grade levels. In mathematics, students at
Bates also did better than students at School B in all three grades. For grade 7, Bates’ students
outscored the district and the state averages.
Figure 10: Percentage of students at or above proficient on the 2013 MSA Reading
Figure 11: Percentage of students at or above proficient at the 2013 MSA Mathematics
Data source: 2013 Maryland Report Card (http://www.mdreportcard.org)
85.4% 85.2% 82.1% 83.8%
79.8% 78.1%
88.8% 88.9% 85.4% 84.1% 85.0%
81.0%
0%
20%
40%
60%
80%
100%
Grade 6 Grade 7 Grade 8
Bates School B District State
73.1% 79.6%
64.6% 58.9% 54.5%
48.7%
76.0% 79.2%
66.3%
77.1% 72.6%
67.0%
0%
20%
40%
60%
80%
100%
Grade 6 Grade 7 Grade 8
Bates School B District State
AACPS Project SAILSS – Final Evaluation Report Page | 35
Table 10 compares differences in the percentages of students at or above proficient in the
MSAs Reading and Mathematics at two points in time: 2008, the year before SAILSS started at
Bates, and 2013, the final project year. The table shows a strong decline in the differences in
percentages for Bates and AACPS between the 2008 and 2013 MSAs. For grade 7 mathematics,
Bates results exceeded those for the district in 2013.
Table 10: Changes in percentages of students at or above proficient in the MSAs between
2008 and 2013
MSA COMPARISON GRADE 6 GRADE 7 GRADE 8
2008 2013 Bates District Bates District
Reading
AACPS 83.2 88.8 83.6 88.9 76.0 85.4
Bates 64.6 85.4 71.7 85.2 57.1 82.1
Difference) 18.6 3.4 11.9 3.7 18.9 3.3
Mathematics
AACPS 82.0 76.0 78.9 79.2 72.7 66.3
Bates 67.9 73.1 61.7 79.6 67.6 64.6
Difference 14.1 2.9 17.2 -0.4 5.1 1.7
Source: Maryland Report Card website, http://www.mdreportcard.org/
Table 11: Results for the 2013 MSA disaggregated by grade and demographics
Grade Demographics Reading Mathematics
N % N %
6
All 239 85.4 238 73.1
African American 70 84.3 68 67.6
Hispanic 63 68.3 62 54.0
Male 93 81.7 93 62.4
Female 146 87.7 145 80.0
FARM 123 76.4 122 59.8
Non-FARM 116 94.8 116 87.1
7
All 229 85.4 230 79.6
African American 68 70.6 68 67.6
Hispanic 53 79.2 53 64.2
Male 109 82.6 110 75.5
Female 120 87.5 120 83.3
FARM 96 67.7 96 60.4
Non-FARM * <95.0 134 93.3
8
All 240 82.1 240 64.6
African American 87 72.4 87 48.3
Hispanic 43 67.4 43 46.5
Male 119 79.8 119 60.5
Female 121 84.3 121 68.6
FARM 113 67.3 113 47.8
Non-FARM * <95.0 127 79.5
N = Number of students in the subgroup %= percentage at or above proficient
* Numbers were not included in the report
AACPS Project SAILSS – Final Evaluation Report Page | 36
Results in the 2013 MSAs for Bates were disaggregated by large student subgroups, here
defined as groups with 20 or more students. These included male and female students, and
students classified as African American, Hispanic, and FARM-eligible. Results are displayed in
Table 11, above. Differences in percentages of students who scored at or above proficient were
quite large for FARM and non-FARM students, and they tend to increase as students move to
upper grades. Similar trends are seen in results for minorities when compared to all students.
2. Comparison of means
A comparison of means analysis was conducted for students in the two schools using
Independent-samples t-tests with a 95% confidence level. As displayed in Table 12, mean scale
scores for students at Bates surpassed those for School B students at all grade levels and in both
assessments.
Table 12: Results of the comparison of means analysis (2013 MSA)
Grade MSA Mean t-test
Grade 6
MSA Reading Bates 417.51
t= 3.085; df= 457; p<.002 School B 407.77
MSA Mathematics Bates 421.14
t= 6.401; df= 460; p<.000 School B 402.67
Grade 7
MSA Reading Bates 426.55
t= 4.825; df= 434; p<.000 School B 401.57
MSA Mathematics Bates 426.78
t= 7.116; df= 448; p<.000 School B 410.70
Grade 8
MSA Reading Bates 419.90
t= 2.028; df= 469; p<.043 School B 413.79
MSA Mathematics Bates 419.46
t= 4.016; df= 474; p<.000 School B 407.38
A comparison of means was also conducted for test results between student subgroups
within the treatment school (Bates). Means for minority students (African American and
Hispanics) were considerably lower than means for white students in both reading and
mathematics at all three grade levels. Likewise, large differences were found when mean test
results for FARM and non-FARM students were compared, with FARM students showing
considerably lower means than their non-FARM peers. Mean test results for female students
were higher than those for male students, while differences attained statistical significance in all
AACPS Project SAILSS – Final Evaluation Report Page | 37
three grade levels for Reading, and grade 6 for Mathematics. At grades 7 and 8, mean test
results were similar for both genders. [Test results are not displayed for reasons of space].
3. Effect size analysis
Table 13 displays results of the effect size calculations. By comparing a school that is
implementing arts integration (Bates) with a similar school that is not implementing the program
(School B), the effect size analysis informs the program effectiveness in facilitating improvements
in student achievement. The effect size (Cohen’s d) was calculated as the difference between the
means for the treatment and comparison group, divided by the standard deviations for the
population. Cohen’s d is interpreted as such: 0.3 to 0.5 = small; 0.5 to 0.8 = medium; greater than
0.8 = large. The table does not presents results that are negative or below 0.3.
Table 13: Effect size analysis for MSA Reading and Mathematics (SY 2009 and SY 2013)
Grade School MSA Reading MSA Mathematics
Mean St. Dev. Cohen’s d Mean St. Dev. Cohen’s d
Bates
6 2009 396.05 49.40
0.50 405.76 45.98
0.39 2013 417.51 35.76 421.14 33.76
7 2009 390.67 80.57
0.60 389.61 83.17
0.60 2013 426.78 38.18 426.55 39.99
8 2009 400.38 59.66
0.42 414.01 52.08
2013 419.90 34.31 419.46 33.02
School B
6 2009 413.94 56.87
399.55 50.13
2013 407.77 31.55 402.67 27.77
7 2009 393.96 76.85
0.31 399.16 76.18
2013 410.70 30.61 401.57 33.86
8 2009 414.55 60.43
402.74 58.12
2013 413.79 30.86 407.38 32.55
*All results are statistically significant at α=.05; numbers in italics = small effect size; bold = medium effect size.
For Bates, medium effect size was found in grades 6 and 7 Reading, and grade 7
Mathematics, while a small effect size was found for grade 8 Reading. Grade 8 Mathematics
showed no program effect on test results. School B, which represents a traditional approach
(without arts integration) showed negligible or negative effect sizes for all grades in Mathematics
and grades 6 and 8 in Reading. A small effect size was found in grade 7 Reading.
AACPS Project SAILSS – Final Evaluation Report Page | 38
4. Cohort Analysis
A cohort analysis was also conducted with Bates students to examine potential effect of
time in the project, as displayed in Table 14. The 2012 Cohort represents students who were in
Grade 8 in 2012; the 2013 Cohort includes students who were in grade 8 in 2013. The students
were divided into subgroups according to their time at Bates: first year – students who entered
Bates that school year (2012 or 2013) and, therefore, had SAILSS only while in Grade 8; two
years – students who were exposed to the project in Grades 7 and 8; and three years – students
who had been at Bates since sixth grade and had SAILSS for three years (grades 6, 7 and 8). An
ANOVA was conducted using MSA results as the dependent variable, and years in Bates as the
factor. No relevant differences were found in mean scale scores for both the 2012 and 2013 MSAs
Reading and Mathematics across the three groups. In other words, time in Bates does not appear
to influence results in MSAs.
Table 14: Results in the MSA Reading and Mathematics for Grade 8 students
disaggregated by years in SAILSS (2012 and 2013 Cohorts)
2012 Cohort 1 Year 2 Years 3 Years
Numbers 28 21 168
Reading 417.2 397.7 415.9
Mathematics 406.6 402.3 421.9
2013 Cohort 1 Year 2 Years 3 Years
Numbers 16 26 200
Reading 415.4 423.2 419.8
Mathematics 404.1 417.2 421.3
Monitoring Project Objectives
Objective 1: At the end of the grant period, 100% of the students at Bates Middle School
will be engaged in instructional activities designed to reflect their diverse learning styles and
multiple intelligences, as measured by teacher activity logs.
Status: Attained
This objective was attained in the first project year and the involvement of students in
arts integration continued throughout the grant. Information from teacher logs, site visits and
interviews show that students are involved in arts integration activities regardless of teacher or
grade level.
AACPS Project SAILSS – Final Evaluation Report Page | 39
Objective 2: At least 20% of all Bates students will participate yearly in extracurricular arts
activities, countywide music and dance competitions, and arts portfolios, starting in the 2009
academic year, as measured by enrollment logs.
Status: Attained
The majority of Bates students are participating in arts-related activities, including arts
clubs, countywide music and dance competitions and arts portfolios. Close to 70% of the
students who participated in the 2012 survey acknowledged being involved in arts activities
outside of school time.
Objective 3: At least 40% of all Bates students will increase their scores on MSA for
mathematics by 5% yearly, starting with the 2009 MSA for mathematics.
Status: Not attained
To determine whether this objective was achieved, results from the MSA for mathematics
were compared to results from the same assessment the year before. If the resulting value was
greater than 0.05, the objective was met. The analysis must exclude students in Grade 6, as their
previous year’s MSA scores (when they attended elementary schools) were not available to the
evaluators. Table 15 displays the percentage of students in Grade 7 and 8 who increased their
MSA mathematics scores by 5% or more across the project years.
Table 15: Percent of students with 5% or greater increases on the MSA mathematics
Year Grade 7 Grade 8 Number of students
2009 20.0 17.9 108
2010 10.5 17.4 291
2011 13.7 30.2 329
2012 8.6 16.5 325
2013 5.1 5.0 418
A problem with the objective is that students who achieve high scores in the previous
year’s assessment will have difficulty increasing their scores by 5% from one year to another.
Another problem is that assessments have different levels of difficulty across grades and across
years. For instance, results for the 2013 MSA mathematics were lower overall, when compared
to the 2012 results, what may explain the decline in the percentage of students who increased
their scores by 5% between 2012 and 2013.
AACPS Project SAILSS – Final Evaluation Report Page | 40
To further explore these caveats, Table 16 displays changes in the 2013 MSA
Mathematics broke down by how the students performed in the 2012 assessment. As seen in the
table, among the 28 students who were in grade 6 in 2012 and did not achieved proficient in the
2012 assessment, 22 (78.6%) were not able to increase their scale scores by 5% between 2012
and 2013, while 6 (21.4%) did achieve it. For the 78 grade 6 students who achieved proficient in
2012, 77 (98.7%) could not increase their scale scores by 5% between 2012 and 2013, while 1
(1.3%) did. For the 91 students in grade 6 who attained advanced in the 2012 MSA, 88 (96.7%)
could not increase their scale scores by 5%, while 3 (3.3%) did. Similar results are displayed for
the 2012 grade 7 students. As a rule of thumb, the lower the scores in the previous assessment,
the greater was the probability of the requested 5% increase. As more Bates’ students attain
advanced levels in the state assessments, it becomes more difficult for the objective to be
achieved.
Table 16: Changes between the 2012 and 2013 MSA mathematics for students at advance
levels
2012 MSA 2013 MSA
Grade Level Grade Below 5%* 5% or more**
Total N % N %
6
Below proficient
7
22 78.6 6 21.4 28
Proficient 77 98.7 1 1.3 78
Advanced 88 96.7 3 3.3 91
Total 187 94.9 10 5.1 197
7
Below proficient
8
32 82.1 7 17.9 39
Proficient 129 97.0 4 3.0 133
Advanced 49 100.0 0 0.0 49
Total 210 95.0 11 5.0 221
*Scale scores did not change, declined, or changed by less than 5%
**Scale scores changed by 5% or more
Objective 4: At the end of each academic year, starting with 2009, the number of discipline
referrals at Bates will decline by at least 5%.
Status: Attained
Table 17 displays the total number of referrals for disciplinary actions and average
number of referrals per student for Bates from SY 2009 to SY 2013, and compares to numbers
from School B. As indicated in the table, the two schools experienced a large decrease in the
AACPS Project SAILSS – Final Evaluation Report Page | 41
number of referrals in the past five years. Discipline referrals in Bates declined by 80% (from
746 to 152), and by 82% at School B (from 674 to 122).
Table 17: Number of discipline referrals and referrals per student (SY 2009 to SY 2013)
School Referrals SY 2009 SY 2010 SY 2011 SY 2012 SY2013
Bates
Sum1 746 1,223 698 169 152
Average2 1.29 2.00 0.99 0.23 0.20
Enrollment3 580 612 701 748 745
School B
Sum1 674 538 429 170 122
Average2 0.85 0.70 0.57 0.24 0.17
Enrollment3 784 768 756 721 719
1 Number of disciplinary referrals
2 Average= number of disciplinary referrals divided by total enrollment
3Total number of students enrolled in the school year
Table 18 disaggregates information on number of discipline referrals per grade for SY
2013. Grade 8 students at Bates had fewer discipline referrals than students in grades 6 and 7. At
School B, variations across grades were small
Table 18: Number of discipline referrals and referrals per student by grade (SY 2013)
School Suspension Grade 6 Grade 7 Grade 8 All
Bates
Sum 55 54 43 152
Average 0.22 0.22 0.17 0.20
Total Enrollment 243 246 256 745
School B
Sum 43 33 46 122
Average 0.18 0.14 0.18 0.17
Total Enrollment 235 229 255 719
Information on suspensions complement the information on disciplinary referrals, since
the overall number of discipline issues in the school may decline, but the severity of the actions
may increase (resulting in greater number of suspensions). Table 19, on the next page, displays
suspensions at both schools since SY 2009. In contrast to the data on disciplinary referrals,
where decline was similar to both schools, the number of suspensions per student declined by
71% in Bates between 2009 and 2013, compared to a decline of 52% at School B.
AACPS Project SAILSS – Final Evaluation Report Page | 42
Table 19: Number of suspensions and average suspension per student (SY 2013)
School Suspensions SY 2009 SY 2010 SY 2011 SY 2012 SY 2013
Bates
Sum 249 225 233 99 71
Average 0.43 0.37 0.33 0.13 0.10
Enrollment 580 612 701 748 745
School B
Sum 183 255 177 98 88
Average 0.23 0.33 0.23 0.14 0.12
Enrollment 784 768 756 721 719
Table 20 displays suspensions disaggregated by grade for SY 2013. Differences in
numbers of students suspended by grades were small at both schools.
Table 20: Number of suspensions and average suspension per student by grade (SY 2013)
School Suspension Grade 6 Grade 7 Grade 8 All
Bates
Sum 21 29 21 71
Average 0.09 0.12 0.08 0.10
Total Enrollment 243 246 256 745
School B
Sum 35 21 32 88
Average 0.15 0.09 0.13 0.12
Total Enrollment 235 229 255 719
AACPS Project SAILSS – Final Evaluation Report Page | 43
CONCLUSIONS
From SY 2008-2009 through 2012-2013, AACPS implemented Project SAILSS in a
middle school located in an urban setting. The project proposed a model of arts integration
founded upon systemic, focused professional development and mentoring support for all school
faculty. Arts integration lessons were geared toward standards where students demonstrated the
greatest difficulties. The model expanded from its core – the school – to the community at large
through the involvement of families, partners, and community members, and dissemination of
the model, and its success, in conferences and publications.
Answering the evaluation questions
This report summarizes evaluation findings from all five project years to address the
initial evaluation questions. Two series of questions were proposed:
How does the model of Arts Integration proposed by SAILSS work to bring quality across
the curriculum?
As described in the previous evaluation reports, the project implementation happened
“with a bang.” From year one, content area teachers, working side-by-side with arts teachers and
artists in residence, developed and implemented arts integration lessons with students at all grade
levels. By the end of the project, all Bates teachers had received professional development on
arts integration and the project was expanding its reach to teachers in other AACPS schools.
Close to 700 arts integration lessons were developed. These lessons included all types of arts
(visual, drama, music) and content areas (English, Mathematics, Social Studies, Science, Foreign
Languages). Interviews with teachers in the fourth project year showed that arts integration had
become “the Bates way” of teaching. Attempts to observe classes without arts integration classes
failed as basically in all observed classes arts integration was being used at some point.
How does the model impact student achievement and attitudes toward learning?
Analyses of student academic performance in the statewide assessments showed a steady
increase in the percentage of students who scored at or above proficient in reading and
mathematics. When compared to a school that did not implement arts integration and had a
similar student population, Bates’ results were considerably higher, particularly in mathematics.
An analysis of effect size using Cohen’s d showed that the project had a medium effect on test
AACPS Project SAILSS – Final Evaluation Report Page | 44
results for grades 6 and 7 Reading and grade 7 Mathematics, and a small effect size for grade 8
Reading, with no effect on test results for grade 8 Mathematics.
Behavior has been improving across the years and the trend is seen in both schools.
Therefore, changes at Bates cannot be attributed solely to SAILSS. Indeed, AACPS has been
implementing a number of programs that focus on behavior management, such as Positive
Behavior Intervention Strategies (PBIS), Collaborative Decision Making (CDM) and others. The
role of these programs cannot be minimized when discipline is examined (School B implements
PBIS). Connections among these programs are certainly a winning strategy for AACPS.
Lessons learned
A reflection on SAILSS’ accomplishments highlights three important lessons that may
help stakeholders to implement future projects.
A project success depends on shared responsibility and collaboration
Strong support from administration and the presence of the Arts Integration Specialist
(AIS) were probably the two main factors responsible for overcoming the initial resistance to the
project from teachers, particularly veteran teachers. Even at the beginning of the project, the
school was already recognized as a place for learning, where the administration supported
innovation. Therefore, arts integration was another innovation that the administration was open
to pilot. The school district put its weight behind the project and in support of the school
administration. Teachers heard one voice and they knew they had to comply. The AIS position
was essential as both a voice for the project among peers and a mentor that created conditions for
success. The AIS was a lead teacher, highly respected among her peers, who went through the
Post-Baccalaureate Certification and became an arts integration expert. She worked closely with
all participants – content area teachers, arts teachers, and artist in residence – to ensure that the
grant’s objectives were not lost, but also to ensure teachers’ success as they developed arts
integration lessons. She also worked closely with the evaluators to ensure the availability of data
and even proposed further ways to collect data that informed decisions related to project
improvement.
An important strategy advanced by the AIS involved looking at standards to see changes
in benchmark results. As teachers saw the results of arts integration on benchmark tests and on
students’ engagement, buy-in was total and evaluators could no longer find classes that were not
integrating arts as a way to advance knowledge of new concepts. A major change in teachers’
AACPS Project SAILSS – Final Evaluation Report Page | 45
perceptions that can be attributed to the project was a heightened sense of camaraderie not only
among teachers but also among teachers, families and students. Perceptions regarding students as
disciplined and engaged increased significantly between the first and the last climate survey.
Teachers who are now entering Bates already know that arts integration is an integral component
of the teaching strategies available and are receptive to its use.
Changes in education take time and patience
Changes in education take time, particularly changes in student academic performance as
measured by standardized tests. There are so many factors beyond the control of the schools that
influence the lives of both students and faculty (economic crisis and personal histories to name a
few). At Bates, indicators of student behavior are showing progress and the school climate has
improved. MSA results have also improved and the project showed small to middle effect size on
the Reading and Mathematics assessment at almost all grades. It is true that teachers at Bates and
elsewhere in the state will face challenges these coming years, as Maryland changes its statewide
assessment. The new assessments are said to be more performance oriented, what may show
even stronger improvements under arts integration.
Improving student learning is an ongoing process
Despite the project successes, Bates still face areas of concern regarding student
academic performance, particularly the gap in performance across student subgroups. Students
from minority and low socio-economic background continue to lag behind their peers from
majority, higher socio-economic families. What works for what groups of students? This is a
question that teachers should strive to answer. As teachers continue looking at the impact of arts
integration on learning of specific standards, it is important to disaggregate results for student
subgroups and understand how the most at-risk students react to the lessons. It is also important
to accept that not everything that is done with so much love and dedication will work, or will
work equally for all students. Student feedback on the lessons will help to gauge interest and
engagement in learning. The benchmark analysis, disaggregated by subgroup, will provide
further evidence on what needs to be retaught through different strategies.
AACPS Project SAILSS – Final Evaluation Report Page | 46
Final words
Probably the greatest success of SAILSS was the creation of a model of Arts Integration
that shows evidence of working .in large middle schools within urban settings. The model is
simple and uses a lot of “in-house” resources, which facilitates its implementation by other
schools. Within Bates, SAILSS has attained a maturity that guarantees its continuity, even
without grant funds. SAILSS is “the Bates way” of teaching. Many of the project-supported
lessons are being used in other schools in the district. AACPS created a district-wide AIS
position to support those schools. Additionally, the partnership that gave impetus to SAILSS is
sustaining the summer institute. Consequently, federal funding was well spent in a project that
showed success, has clear signs that it will be sustained, and is already expanding beyond its
initial boundaries.
AACPS Project SAILSS – Final Evaluation Report Page | 47
REFERENCES
Armstrong, T. (2006). The best schools: How human development research should inform
educational practice. Alexandria, VA: Association for Supervision and Curriculum
Development.
Burnaford, G. (2007). Arts integration frameworks, research and practice: A literature review.
Washington, D.C.: Arts Education Partnership.
Burton, J., Horowitz, R., & Abeles, H. (1999). Learning in and through the arts: Curriculum
Implications. New York: Teachers College, Columbia University., Center for Arts Education
Research.
Fisher, D. L., & Fraser, B. J. (1990, April 16-20). Validity and use of the School-Level
Environment Questionnaire. Paper presented at the Annual Meeting of the American
Educational Research Association, Boston.
Gardner, H. (1983). Frames of mind: The theory of multiple intelligences. New York: Basic
Books.
Glaser, B. G. & Strauss, A. L. (1967). The discovery of grounded theory; strategies for
qualitative research. Chicago, Illinois: Aldine Publication, Co.
Nelson, C. A. (2001). The arts and education reform: Lessons from a four-year evaluation of
the A+ Schools Program, 1995-1999 (Executive summary of the series of seven policy
reports summarizing the four-year pilot of A+ Schools in North Carolina). Winston-Salem,
NC: Thomas S. Kenan Institute for the Arts.
Strauss, A.L. (1987). Qualitative analysis for social scientists. New York, New York: Cambridge
University Press.
U.S. Department of Education. (2008). Guide to U.S. Department of Education Programs.
Washington, D.C.: USED, Office of Communications and Outreach.
Yin, R.K. (2009). Case study research: design and methods. Los Angeles, California: Sage
Publications.
AACPS Project SAILSS – Final Evaluation Report Page | 48
APPENDIX A: EVALUATION DESIGN
Purpose
SAILSS’ external evaluation had a three-fold purpose: (1) provide stakeholders with
periodic assessments of the project’s progress toward goals, (2) collect evidence that informs a
continuous quality improvement process, and (3) assess changes in teacher practices and student
outcomes that are related to the project.
RMC approaches evaluation from a perspective of partnership. Evaluators work closely
with project staff to address their questions, and promptly provide information that can be used
for project’s improvement. To ensure timely and useful feedback, the evaluators and project staff
meet periodically to review the implementation and evaluation plans, adjust timelines, share
information, and discuss evaluation findings and recommendations.
Research Questions
The following questions guided the implementation and evaluation of the project:
How does the arts integration triad model of spiraled teacher professional development,
curriculum writing, and curriculum implementation work to bring quality integrated arts
instruction across the curriculum?
o What are the challenges and successes with arts integration development and
implementation using this model?
o What are the challenges and successes with arts integration growth and sustainability
using this model?
o What are the experiences and perceptions of teachers, students and artists in residence
as they engage in the integrated arts curriculum implementation?
o What are the experiences of the community, including parents and other community
stakeholders, as they engage with a school moving into arts integration?
o What are the perceived meaningful nuances and details of arts integration that support
or detract from implementation success (student, parent, teacher, artist, and
administration)?
How does the integration of arts into mathematics classes affect student achievement in,
and attitudes toward, mathematics?
AACPS Project SAILSS – Final Evaluation Report Page | 49
How does the integration of arts across the curriculum affect overall student achievement
and attitude towards learning?
Evaluation Design
The implementation of the project (formative evaluation) used a case study design (Yin,
2009) to address the first four research questions. The effects of the project on teacher practice
and student learning (outcome evaluation) were assessed through a quasi-experimental design
using a school with similar demographics but without arts integration as the comparison group.
The hypothesis was that students at the treatment school would show greater gains when
compared to students at the comparison school. Indicators used to assess gains were: MSA
mathematics and discipline referrals. Results in the MSA reading were also collected.
At the time of the grant proposal, AACPS chose the treatment and the comparison
schools on the basis of school size, student demographics, and the principals’ interest in the
project. Both schools had expressed their interest in becoming Arts Magnet schools. Bates was to
start the Arts Magnet program in SY 2009-2010, while the comparison school would start the
Magnet Program by the end of the grant. Yet, as SAILSS started, teachers at the comparison
school began attending professional development together with the treatment teachers and the
following year, an arts magnet program started at the school. Therefore, the school had to be
dropped from the study. In SY 2010-11, a new school was chosen for comparison based on
student demographics and size.
The plan was to involve the comparison school in the surveys, and this happened in the
two first years of the grant. When the initial comparison school had to be dropped, the project
was in its third grant year and the new comparison school was not fully committed to the project.
Therefore, comparisons were limited to the analyses of student data, which were collected from
central office.
Data Collection
The following instruments were used to collect data for the study:
Classroom Observation Form – This form was developed by Dr. Elda Franklin for the
North Carolina Blumenthal Performing Arts Center Education Institute, a past AEMDD
project. Blumenthal Performing Arts Center graciously granted permission to use the
form (Franklin, 2005). The instrument consists of a time line with five-minute
AACPS Project SAILSS – Final Evaluation Report Page | 50
observation intervals to assess student engagement and instruction. A rating scale for
assessing the implemented lesson plan for each indicator is also provided.
Arts integration: Classroom Observations for Middle Schools (AICOM) – This
instrument was developed by RMC to be used in conjunction with the Classroom
Observation Form. The form requires visual observation of the classroom environment,
an overview of the lesson in progress and a brief review of student Arts integration
projects.
Professional development log – collects information on who attended which professional
development activity. This log was collected by the AIS and provided to the evaluator on
a quarterly basis.
Arts integration activities log – provides information on curriculum development and
Arts integration activities implemented in the classrooms. Teachers kept a log of these
activities and provided it to the AIS at the end of the each school semester.
The School-level Environment Questionnaire (SLEQ) - the SLEQ measures teachers’
perceptions of psychosocial dimensions in the school environment. It consists of 56
items scored on a five-point scale and organized into eight categories: (1) student
support; (2) affiliation; (3) professional interest; (4) staff freedom; (5) participatory
decision making; (6) innovation; (7) resource adequacy, and (8) work pressure. The
development and validation of the SLEQ is described in Fisher & Fraser (1990), who
granted permission to RMC for use in this evaluation. Burton, Horowitz & Abeles
(1999) used the SLEQ in their research regarding arts-rich versus arts-poor schools. The
instrument was applied to all full time teachers with classroom responsibility (that is,
resource teachers and substitute were not surveyed).
Student Arts integration survey – this instrument was created by RMC after a literature
search indicated that no existing instrument addressed the project needs and interest. The
instrument, created with feedback from the project director, was applied annually.
Table A-1 summarizes the evaluation activities, their target population and focus, the
instruments or protocols used to collect data, and the timeline.
AACPS Project SAILSS – Final Evaluation Report Page | 51
Table A-1: Evaluation Activities, timeline, and instruments
Data Collection Focus Timeline Instrument
Professional
Development
Identify intensity of
treatment and participation Yearlong; all four years Project Director Log
Arts integration
Activities
Track development of
curriculum components and
activities
Yearlong; all four years Teacher Log
Classroom
observation
Assess fidelity and intensity
of implementation. Yearly
Blumenthal Performing
Arts Center Education
Institute (2003)
Form and Arts integration
Form (RMC)
Teacher Inventory Assess teachers’ perception
of school climate May (yearly) -
School-level
Environment
Questionnaires (SLEQ) –
Fisher & Fraser (1990)
Student survey
Gauge students’ interest on
learning and involvement on
Arts integration activities
April or May (yearly) -
Student Arts integration
Survey (Project -
developed)
Student
attendance/
behavior data
Indirect measures of
engagement in learning September, yearly - AACPS database
Student academic
performance data
Assess changes in student
performance on mathematics
(project objective/GPRA)
and reading (GPRA)
Quarterly AACPS Benchmarks
(Mathematics)
Yearly (released end of
summer)
Maryland - MSA results
(in Reading and
Mathematics)
Data management and analysis
Both qualitative and quantitative data were collected as part of the evaluation, and
methods addressing both types of data were utilized in order to elicit pertinent information for
program decision making. Triangulation was used to confirm, explain, or disqualify qualitative
and quantitative responses. Qualitative data were coded thematically using the Grounded Theory
approach (Glaser & Strauss, 1967), and summarized in narrative format. This method allows the
theory to emerge from the data, rather than follow from a hypothesis. The process starts with
breaking the information down into categories as small as recurring words and phrases (Strauss,
1987). Individual codes are derived from the data. They are then categorized under developed
themes and examined for patterns within and across documents to address the specific research
questions.
AACPS Project SAILSS – Final Evaluation Report Page | 52
Quantitative analyses explored the following outcomes: academic performance, as
measured by state assessments, and behavioral data. Changes in performance measures at the
two schools were analyzed longitudinally, using the four baseline years and the project years.
Results from the treatment school (Bates) were compared to those from the comparison school
(School B) using comparison of means tests and Cohen’s d analyses.
Communication of findings
In addition to oral communications made at meetings with project and school staff, the
evaluators completed a number of small reports to present findings from the surveys.
Additionally, yearly evaluation reports were completed and provided to the project director.
These reports detail implementation activities conducted for each project year, as well as results
from surveys, interviews and observations. The final report summarizes findings from a
longitudinal perspective.
IRB review and parental consent issues
All the study procedures were reviewed at RMC Research Corporation and kept in
agreement with its ethics procedures. AACPS does not have a formal IRB process. However, an
agreement was established between evaluators and AACPS that ensures fair and confidential
treatment for all study participants. RMC Research Corporation was granted permission to obtain
student-level data from AACPS, upon consultation with its lawyers. The data is kept in a secure
partition within RMC server and can be accessed only at that location by the evaluation team
members. No copies of the data or transfers over non-secure lines are allowed. Two evaluation
activities involved student participation: classroom observations and an annual survey.
Following AACPS procedures, as the student surveys are anonymous and do not collect data that
can identify students or are of a personal nature, parents were informed of the study but no opt-in
permission was required.
Top Related