Kris Williams, NTN Publications Specialist and School Development Coach

New Tech Collinwood: A Case Study of One School’s Use of Feedback

In early 2016, the New Tech Network (NTN) began publishing case studies to more vividly illustrate the key principles used to achieve student

outcomes that matter through its partnered work with schools and districts. While the intention of each study is to highlight high-quality examples of these principles in action, it should be acknowledged that every context is unique and no implementation

is ever fully complete or universally successful. Instead, the hope is that these examples from the field will help support others in their efforts to understand the impact and implications of putting the often-abstract principles of systemic change into practice. Inspired by the work of Marshall Ganz, NTN also believes that storytelling is a powerful tool when attempting to address complex challenges.

New Tech Collinwood is located on the east side of Cleveland, Ohio, in the South Collinwood neighborhood. It is a member of the Cleveland Metro-politan School District, which serves nearly 40,000 students across the city of Cleveland. The school site was orig-inally founded in 1926, evolving over the years into its current form with roughly 350 9th-12th grade students enrolled for the 2017-18 school year. The demographics of the school closely mirror the makeup of the neighbor-hood in which it sits, with 99% of the student body identifying as African American. Reflecting the high levels of poverty in the local area and across the district, 100% of New Tech Collinwood students qualify for free or reduced cost lunch.

In 2012, the Cleveland Metropolitan School District launched an initiative to increase the pace and effectiveness of efforts to support student achievement throughout the district. The initiative, called The Cleveland Plan, aims to cre-ate a “portfolio” of schools across the district that have been given greater autonomy in designing programs that meet the needs of all students through innovative and effective practices. As a district of choice, CMSD students can attend any school across the district to ensure that they are finding the best fit for their needs. As part of The Cleve-land Plan, Collinwood High School was identified as an “Investment School,” due it’s historically low achievement and high-need student population. As a primary strategy to address these

issues, Collinwood partnered with the New Tech Network in 2014 and was renamed “New Tech Collinwood” to highlight this partnership and their implementation of NTN practices.

New Tech Collinwood began imple-menting NTN instructional practices and the use of NTN’s LMS, Echo, at the 9th grade level in the 2014-15 school year, adding a new grade level of imple-mentation with each year that followed. Now in the 2017-18 school year, New Tech Collinwood has fully implement-ed the NTN model across the 9th-12th grades, with all 34 faculty trained in project and problem-based learning practices. Over the course of their im-plementation the school has weathered the typical challenges that accompany a transformation of practices at this scale, bumping up against ingrained structures and beliefs that can be diffi-cult to change along the way. However, these challenges of implementation were often amplified by the pervasive and urgent needs of their student pop-ulation. Student - and staff - doubts about whether they are capable of this kind of work were sometimes hard to keep at bay. However, even with these changes and challenges, the school has established a strong implementation of NTN practices. In addition to the visible levels of engagement in the classrooms and the rigorous community-connected projects that teachers are developing, a significant increase in daily attendance and student achievement results are just further evidence of their success.

THE STORY OF NEW TECH COLLINWOODThe focus of this case

study is to explore one school’s use of feedback practices to help im-prove instruction and increase student achieve-ment. NTN believes that learning how to provide and receive feedback for growth is a critical step for improvement at any level. NTN also believes that effective feedback practices can be help-ful tools for addressing issues of inequality in our schools. By using feed-back to accurately and constructively surface the gaps between indi-vidual student needs and instruction, each student and each teacher can be more equitably supported in a way that is respon-sive to their individual needs.

By embedding effec-tive feedback practices into structures for both adult and student learn-ing, New Tech Collinwood has made great strides in its efforts to support each of its students’ developmental needs. While the Collinwood staff would never suggest that they’ve successfully addressed all of their students’ needs or that their feedback processes are perfect, the results they’ve achieved amidst particularly challenging conditions made it feel important to share their approach with others. The information used to develop this study was collected through interviews with current and former staff, artifacts shared by staff members, and discussions with NTN coaches that have worked with the school.

INTRODUCTION

PAGE 2 2017

After being introduced to a variety of protocols at NTN’s initial trainings, the first group of implementing staff began using the Critical Friends Tuning Peer Review Protocol to structure feedback discussions focused on the development of project and problem-based curricu-lum. The peer review protocol, which is familiar to the majority of New Tech Network schools, often involves the sharing out of a curriculum idea or resource with colleagues (sometimes including students) for examina-tion and feedback. After sharing their project or curric-ulum idea, the presenter may answer a few clarifying questions before stepping back from the discussion to listen and record notes as their colleagues discuss what they heard. The first round of feedback is used to share out elements of the idea that they liked. In the second round of feedback, the participants share out anything that the idea has left them wondering about. Then, in the final feedback round, the participants share out any potential next steps that the presenter(s) might consid-er to strengthen their work. At the end of the protocol, the presenter rejoins the discussion to share how their thinking about the work has shifted as a result of the feedback. For New Tech Collinwood, this protocol became - and continues to be - a key component of their grade level meeting time as a staff.

As was the case with the general implementation process for the Collinwood staff, this approach to provid-ing feedback amongst faculty colleagues was not always easy. Perhaps the most significant factor in making staff collaboration difficult was that a large number of students were struggling to experience success in the projects teachers were designing that first year. Frustra-tion with those struggles resulted in defensiveness and an overall decrease in morale across the implementing staff. However, toward the end of that first year there were some signs of success starting to emerge. Through repetition and the staff ’s commitment to the approach, students were starting to understand and respond to the new expectations for learning in a PBL classroom. The faculty were also starting to fine tune their classroom supports a bit more, with support from their NTN coach, to help ensure that student needs were being met. Still, while their push-back on the new approach had lessened, the work that students were producing wasn’t showing enough growth as a group overall. The teachers and leaders at Collinwood recognized the need to figure out how to collectively improve their support for students, but the staff culture had been significantly strained and work was needed to ensure that future curriculum feed-back could be given and received constructively.

USING FEEDBACK TO SUPPORT EARLY IMPLEMENTATION

At the start of the second year of implementation, with the addition of a new grade level team and a new school principal, the faculty determined that they would need to resume their practice of providing each other with feedback using the peer feedback protocol to ensure that newly implementing teachers were receiv-ing guidance on their curriculum design. In reflecting on the previ-ous year, they decided to adopt a norm to “always follow the protocol” when providing feedback. They recognized that they hadn’t always been good at keeping to each of the protocol’s steps in the past and that this had sometimes caused problems when attempt-ing to provide and receive feedback. Thoughtful reflection enabled them to identify instances from the previ-

ous year in which the protocol had helped them to share effective practices across the grade level, regardless of who was getting feedback at the time. This seemed particularly important if they were going to meet the urgent demands for increased student achievement. Looking back on the

start of their second year, the staff credits the renewed commitment to a feedback

protocol as one of the keys to seeing some significant gains in student success over the course of that school year.

By the end of that year, the signs of a professional learning culture amongst the faculty started to emerge. Teachers were finding time to give feedback to each oth-

er, even outside of their structured learning time. More ambitious projects were being de-veloped as the faculty became more comfort-able with taking risks and raising expecta-tions for their students and for themselves. They were becom-ing hungry for other structures and prac-tices that could help them give even more constructive feedback to each other. This set

the stage for a big step forward as a staff, heading into the 2016-17 school year.

LEARNING FROM CHALLENGES

"I would say that our second year of implementation was probably our most challenging year. We went from nine very committed teachers who believed in the process to 22 teachers, some of which were very skeptical. We knew we had to do something to build staff culture while still working to improve student achievement. The use of staff collaborative time during Critical Friends Groups with strict adherence to protocols definitely helped to bridge the staff divide. It did not happen quickly, but by the end of year 2 we began to value the feedback we received from one another. We were starting to look more like a learning organization."

- Maria Carlson, former New Tech Collinwood principal, 2014-2017

PAGE 3 2017

USING FEEDBACK TO ADDRESS SPECIFIC STUDENT LEARNING NEEDSInspired by feedback from their

NTN coach and examples shared at NTN events such as NTAC and Lead-ership Summits, the 9th grade faculty team decided to add a new protocol to their recurring meeting structure: a modified version of the ATLAS Learn-ing From Student Work protocol, as designed by the School Reform Initia-tive. The key elements of this protocol include an initial step of simply dis-cussing what can be seen in the student work, without judgement or analysis. From there, protocol participants share any questions that the work raises for them. Following that step, they attempt to determine what the students know and can do, as evidenced by the work. Finally, they explore whether they can determine the objectives of the task and whether the work is aligned with those objectives. With this new tool in hand, they decided that they could at-tempt to address some of the more per-vasive student learning needs. Specifi-cally, student needs related to literacy often felt completely overwhelming, as most students arrived with reading and writing levels far below their grade level.

To better understand the struggles their students were going through with literacy, and to work toward some po-tential strategies for support, the team began to use the ATLAS protocol to an-alyze student writing samples. Stick-ing to their agreed-upon version of the protocol, the steps pushed them to see their student work from a variety of different helpful lenses. Needing to use evidence from the work itself (and not

what they assumed about the task or the student) to qualify their feedback, they were able to keep their discussions grounded in what the students were able to actually demonstrate through their work. Through the repeated use of this protocol, patterns began to emerge that eventually created a more precise understanding of their stu-dents’ literacy needs.

Over and over again, feedback generated during the ATLAS proto-col had revealed that students were struggling with some fundamental elements of argumentative writing: making a reasonable claim, supporting the claim with evidence, and analyzing the evidence. Out of this discovery, the TREEAT strategy (listed below) was developed with the goal of providing simple guidance to students, and teach-ers, on how to ensure that all of these fundamental elements are present in argumentative writing tasks. Almost

immediately after implementing this strategy, the feedback provided during the ATLAS protocols more explicitly informed instructional practice and students began showing signs of im-provement in their writing. By the end of the year the evidence of impact for this strategy was evident in almost all forms of student work, as the ability to explain and defend their thinking is a skill for college readiness in nearly every subject. The final validation of the school’s efforts eventually came in the form of improved ELA 1 scores on the Ohio State Test (OST) by over 20%. Not only was this a significant improve-ment for the school, but this growth had now placed the school within the top 50% of all schools in the district for achievement. This was a huge accomplishment, considering that the school had been targeted as a severely low-performing investment school just 3 years earlier.

TREAT STRATEGY

THE TREEAT STRATEGYDeveloped with the goal of

providing simple guidance to

students, and teachers, on how to

ensure that all of these fundamen

tal elements are present in

argumentative writing tasks.

PAGE 4 2017

Throughout the 2016-17 school year, as the real val-ue of giving and receiving feedback became more and more evident, the staff be-gan to explore other ways to more deeply engage in these types of practices together. With support from the cam-pus principal, the faculty participated in a book study to help guide them toward more productive and enrich-ing peer coaching practices. Together, they studied The Art of Coaching, by Elena Aguilar and explored the use of coaching lenses to help further improve how the give feedback to each other in their grade level teams. Not only did this provide a new set of tools for the team to use together, but it also opened up new opportunities for teacher-leadership. Teachers felt better equipped to make suggestions about how to im-prove practices more broadly across the school, to offer sup-port to newly implementing staff, and felt more empow-ered to help facilitate grade level meetings.

Additionally, this work began to influence how teachers supported students

in their approach to giving and receiving peer feedback. The 9th grade team decided to develop a project for all 9th graders that would help them learn about giving and receiv-ing feedback through the use of peer feedback surveys. The hope was that the data would help their students gain a more accurate understanding of how they’re perceived by their peers, while also giving them a guided opportunity to reflect on how this feedback could be used for improve-ment.

As a result of this proj-ect, teachers began to see their students understand-ing the relationship between feedback and growth. Not only were their students less reluctant to gather feedback about their own practices, but they’d also learned how to make the feedback they provide more constructive for others. Just as the teach-ers had experienced with their evolving use of feedback protocols, the guided use of strategies for more in-depth feedback and reflection had significantly increased the learning for the students.

DEEPENING FEEDBACK PRACTICES FOR STAFF AND STUDENTS

“All in all, I found some surprising results. They show me what I’m doing well at and things that I just need a little work. These results will help me build up on my strengths and reduce my weaknesses. But what is good is that the majority who took the survey think that I am a good person. But I really need to work on my social skills when it comes to collaborating with other students. This also includes working with other students and coming up with ideas for any upcoming assignments. These results matter because they will help me resolve the things I wasn’t so strong and improve in them.”

- 9th grade student at New Tech Collinwood

PAGE 5 2017

As we look back at the work that New Tech Collinwood has done over the past three years, many lessons can be taken away from their approach. It’s worth noting that the path has not been easy or clearly marked, as they’ve had to navigate a signif-icant set of challenges along the way. The first year of implementation can be a crucial time for setting the con-ditions that eventually lead to collective improve-ment efforts, so their difficult start could’ve significantly delayed their learning as a staff. Instead, they sought out bright spots, reflected on lessons learned, and used the challenges to help gal-vanize their professional learning culture.

After reflecting upon their learning, they became determined to make good use of just a few key protocols and to fully embed them into their professional devel-opment structures. There are a seemingly endless amount of available protocols and strategies for staff development, but the New Tech Collinwood staff decided to keep their focus on the tools that seemed to best fit their needs at the time: one protocol for peer feedback on curriculum, used when implementation was the main focus; anoth-er protocol for deeper analysis of student work to support the shift of focus from implementation to improvement. Providing an extended amount of time for all of the

faculty to fully understand and experience a new feedback process appears to have been a particularly thoughtful approach, especially when the staff culture is strained by a wide range of adaptive challenges. When trust is low or still developing amongst staff, a protocol can provide the structure needed to keep feedback safe and constructive. As

evidence of teacher practice and student success improved from the initial feedback protocols, the opportunities increased to add more complex and challenging feedback practices.

As New Tech Network has learned through its partnership with over 200 dif-ferent school contexts, the work to improve on a specific area of student need is almost always separate from – and must follow – the initial work to implement NTN’s instruc-tional model. This was true for Collinwood in their focused use of only the Critical Friends peer feedback protocol during the first phase of implementation. They also recognized that this initial phase isn’t lim-

ited to just their first year, but as each new grade level team joins the implementation effort. As the first team to implement NTN practices, the 9th grade group has been the first to move on to more complex forms of peer feedback, shifting from a focus on implementation to one of improvement. This has also put them in a position to provide

support to the other grade level teams still in the early stages of PBL imple-mentation.

Looking ahead to the remainder of the 2017-18 school year and beyond, the New Tech Collinwood team is continuing its use of both the Critical Friends tuning protocol and LASW to help each other grow. Teacher leadership

is expanding to allow for more facilitation opportunities during professional develop-ment, with feedback protocols now includ-ing more mixing of staff across grade level teams. The 9th grade team is also planning to invite cross-grade level feedback in order to improve the reflection project for the freshmen class. Given the urgency of needs across their student population, the staff understands that the work to improve their practices will never truly end. However, they hope that by sharing their story with others, more schools will see that the work required to exchange feedback on our efforts to sup-port all students is incredibly worthwhile.

LOOKING BACK AND LOOKING AHEAD

“Implementing New Tech has been incredibly difficult. People doubted, and still doubt, that ‘our kids’ can do New Tech due to low expectations for urban youth and doubts that they can learn to be critical thinkers, problem solvers, etc. Especially after the first year of NT when we didn’t get results, this was a huge sentiment in the building. We are the only pre-existing whole-campus New Tech school in Cleveland. If you come to Collinwood you have to take New Tech classes, regardless of ability. This openness to all students makes our work even more difficult, but also more worthwhile in the end. If we can implement NT with fidelity and success, that is truly a testament to the fact that any school can do it too!”

- Cara, teacher at New Tech Collinwood

1250 MAIN STREET, SUITE 100, NAPA, CA 94559 | P: 707-253-6951 | /NEWTECHNETWORK | © 2017 NEW TECH NETWORK

There are nearly 200 schools in 28 states and Australia in the New Tech Network.

Visit a school today!

http://www.newtechnetwork.org/engage/tours

New Tech Network, a national non-profit organization, is a leading design partner for comprehensive school

change. We work closely with districts and schools to create innovative learning environments. Through a

proven school model, a project-based learning platform, and powerful professional development, we coach

schools toward lasting change and ongoing improvement.

ABOUT US