DESIGN SQUAD FINAL PAPER Denise Brannan-Prischak...

Running head: DESIGN SQUAD FINAL PAPER 1

DESIGN SQUAD FINAL PAPER

Denise Brannan-Prischak Misty Franklin Stacy Mercer Krystal Reno Eric Standley

The University of Akron

December 7, 2014

DESIGN SQUAD FINAL PAPER 2

Analysis Phase

Introduction to the Learning Situation

The Design Squad consists of Denise Brannan-Prischak, Misty Franklin, Stacy Mercer,

Krystal Reno-Kelley, and Eric Standley. This is Denise’s first year at Ellet High School as a

Spanish teacher. Misty has been working for The University of Akron School of Law for 13

years as a College Program Specialist. This is Stacy’s fifth year as the middle school Title I

math tutor at Western Reserve Local School District and her second year teaching a seventh

grade technology class. Krystal graduated this past spring from The University of Akron with a

bachelor’s degree in AYA social studies, is currently a building substitute. Eric is a third grade

teacher at Hudson City Schools. He has been teaching for 16 years and this year he is the

technology content facilitator.

The Design Squad worked with one Subject Matter Expert (SME) Michelle Marquess-

Kearns, the principal at Ellet High School in Akron, Ohio. Ellet High School is one of seven

high schools in Akron. This is an urban school that enrolls about 1,150 students between 9-12

grades. The demographics of Ellet High School are: 74% White; 19% Black; 5% two or more

races; 2% Hispanic (Great Schools, 2014). According to Great Schools, 130 % of the poverty

level is $30,615 annual income for a family of four; 185% is $43,568. The median household

income in Akron city is $34,952 per year. About 56.4% of Ellet High School students qualify

for free or reduced lunch (Great Schools).

Identification of the Learning Problem

Formative assessments give teachers the opportunity to collect valuable data that will

help teachers identify concepts that students are having difficulty understanding, skills that have


not yet been mastered, and to create clear learning objectives so the necessary modifications and

adjustments can be made to facilitate effective teaching. The general goal of formative

assessment is to collect detailed information and data to drive instruction, thus improving and

enhancing student learning while it’s happening. Effective formative assessment needs to be

integrated into the overall learning process organically every day.

The learning problem was that the teachers were not using formative assessment in the

classroom, they had not been using it to adjust or drive instruction, but rather they had been

simply using it to assign points for grades. According to the SME, only about 50% of the 84

teachers at Ellet High School understand how to use formative assessment to guide instruction in

the classroom. For example, some teachers were using exit slips to evaluate learning, but may

not have the required time to assess the exit slips until a later time due to limited time constraints

and other prominent demands of their job.

Statement of the Learning Goal

The school goal, to meet state requirements, is to have 80-90% of teachers using the data

from formative assessment to adjust and drive instruction to improve individual student

performance. Hence, our learning goal was for the teachers at Ellet High School to effectually

use formative assessment to drive instruction.

The Design Squad created a high-level of interest concerning how technology could be

used for formative assessment through presenting and demonstrating some user-friendly websites

that would provide immediate, measurable data to the teachers and students OF the learning

while it is happening. Teachers would also have full access anywhere to the wiki and be able to

easily add to this organized online database of valuable, pertinent formative assessment


resources. Due to the diverse level of computer literacy, we also provided some tutorial videos

about how to navigate, edit, and use a wiki and how to productively use some of the formative

assessment websites, such as Socrative.

The instantaneous quantitative data that was gathered through using technology to

formatively assess the students helped the teachers analyze the teaching and learning process.

Moreover, teachers will be able to assess and adjust instruction effectively and efficiently,

consequently, raising the educational achievement of all students while also reducing learning

gaps.

Detailed Needs Analysis

The need of using formative assessment to adjust and drive instruction was a felt need by

the principal, Michelle Marquess-Kearns. According to Morrison et al (2011), “a felt need is a

desire or want that an individual has to improve either his or her performance or that of the target

audience. Felt needs express a gap between current performance or skill level and desired

performance or skill level’ (p. 34). Michelle Marquess-Kearns expressed a felt need for the

teachers to increase and improve their usage of formative assessment to adjust and drive

instruction.

Through the use of the 90 Day Action Plan developed by the SME, the felt need was to

increase the amount of teachers using formative assessment to adjust and drive instruction from

the current 50% to 80 – 90%. It was expressed that teachers must understand and use formative

assessment practices during the course of the lesson or unit when determining modification in

curriculum and instruction. The goal was for at least 85% of teachers to conduct daily formative

assessments during the course of a lesson or unit to enhance instruction and to provide


themselves and their students with concrete information on how to improve individual student

performance. Individual teachers and teacher teams continuously assess, adjust curriculum and

instruction, and assess again through the use of multiple sources of data until student targets are

met.

Detailed Learner Analysis

There are about 84 teachers and support staff that The Design Squad worked with at Ellet

High School in Akron, Ohio for this project. Many of the teachers are veteran teachers and will

be able to retire in a few years. The Design Squad decided to inquire about what the teachers

would declare concerning technology usage at school and home.

SurveyMonkey was used to create a survey inquiring about how the teachers are

currently using technology, the accessibility and availability of technology, and how often they

are using technology at school and home. Thirty-six of the 84 teachers (43%) responded to the

eight question survey; consisting of eight intervention specialists, six language arts, five college

and career education/technology, four social studies, four science, and nine other teachers.

Thirty-two teachers said that they use technology for educational purposes concerning their

teaching. Thirty teachers indicated that they use technology to learn. Thirty-two people use

technology for communication purposes; such as: voice calling, texting, email, Skype and

blogging. Twenty-seven said that they use technology for entertainment purposes and 17 said

that they use a digital calendar.

Twenty-one people said that they “Always/Often” use technology at school and nine

people said that they “Sometimes” use technology. All the people that completed the survey said

that they have access to a computer (desktop, laptop, netbook) at school. Four people said that

they use an interactive whiteboard (SMART board, multimedia projector, Promethean) to aid


their teaching. Four teachers said that they use a Class Response System (Clickers) with their

students and ten other 21st

century digital tools were also mentioned. Most people said that they

use technology between five and thirty hours at school, with a median of 12 hours per week.

Thirty-one people have access to a computer (desktops, laptops, netbook), 30 people have

internet accessible smartphones, twenty-one have a tablet, and twelve people have a smart TV.

Seventeen people say that they even play video games at home. Most people said that they use

technology between five and twenty-five hours at home, with a median of fourteen and a half

hours per week.

Overall, the staff at Ellet High School seem to have a considerable amount of technology

available and accessible at school and home and readily use technology for a variety of purposes.

Design Phase

Task Analysis

The task of the Design Squad was to find a solution for Ellet High School’s problem of

lack of formative assessments by the instructional staff in each classroom. According to the SME

about only 50% of the staff understood the proper uses and various tools to effectively carry out

formative assessment to drive instruction. After some discussion about formative assessment on

WebEx, The Design Squad then decided to primarily focus on the delivery of the web 2.0 tool

www.socrative.com as a possible solution for the formative assessment issue among many of the

staff members. Socrative would be a quick and easy delivery for staff to obtain student data to

drive instruction in their classroom. Backward design was then entertained as a way to approach

the problem through the use of this tool.


Identify Learning Theory

The Design Squad initially determined that the relevant learning theory to use is the

Cognitive/Constructivist learning theory in which the teachers are active participants in the

learning process through the use of hands-on workshop(s) and access to additional resources for

self-directed learning. After reviewing the original survey results, along with taking into

consideration the time constraints of the teachers at Ellet High School the Design Squad decided

to move forward with online training of Socrative, which allows teachers the ability to review the

training video and utilize the software providing the hands-on component, along with providing

additional resources through the use of a Wiki for continued self-directed learning.

Objectives

● Continuing with the backward design method and upon meeting with the SME, the

Design Squad identified the following high performance learning objectives:

● Demonstrate use of formative assessment to obtain data.


● Analyze formative assessment data.

The identification of the performance/learning objective also complement the overall

team objective identified by the Design Squad and the SME , which is to raise the 50% use and

knowledge of formative assessment of Ellet High school staff to at least 80-90% through the use

of the staff wiki containing web tools and promoting professional development through an online

forum

Instructional Strategies

The Design Squad used cognitive procedure as the instructional strategy. Using

formative assessment to adjust and drive instruction can be taught by a sequence of steps and

worked examples. The step-by-step Socrative demonstration provides the opportunity for a

worked example, along with the video on the use of Socrative supports this instructional strategy.

Additionally, the use of a Wiki to provide direct access to additional formative assessment sites

containing additional how-to videos to provide continued cognitive procedure to improve self-

directed learning.

Instructional Sequence

Since most teachers use formative instruction the instruction additionally becomes how to

use the data retrieved from the assessments to adjust and drive instruction. This falls within the

real-world temporal sequencing. This sequencing allows instruction to be taught using a timeline

of events and examples of “If this – then this,” which again provides a means of teaching an

orderly sequence.


Development Phase

Media Selection Rationale

Design Squad determined that Socrative was the primary suggestion for immediate

delivery and data collection. The Design Squad further culminated its design around a

Wikispaces created by the team members for Ellet High School as a bulletin board discussion

area for formative assessment. Therefore, this allows staff members to not only use the suggested

tools from Design Squad, but also use the Wiki as a forum to share effective and efficient uses of

formative assessments used in their classroom. The wiki then continues to serve as a place for

Ellet Staff members to improve the formative assessment from the wiki without much formal

training or formal face to face discussion. The intention of the Design Squad was to deliver a

method of instruction of formal assessment minimizing one on one trainings through on online

source while simultaneously raising the awareness and possible use of formative assessments.

Issues of Message Design and Instructional Resources

The content was designed to allow the Ellet High School instructional staff to:

● log-in to the ElletStaff wiki

● navigate through pages of the ElletStaff wiki to view classroom resources (web

tools)

● navigate through the pages of the ElletStaff wiki to view their respective

professional community

● post comments, questions, and ideas about formative assessment on the wiki

● use at least one idea taken from the ElletStaff wiki resource page as a formative

assessment tool to be implemented in the classroom


The Design Squad took a “learning sequencing” (Morrison, p. 124) approach when

planning and designing instruction for both the web 2.0 tools and the wiki. The team needed to

invite staff to the wiki via email knowing from the analysis phase that email would be an

effective way to deliver this instruction. The design of the ElletStaff wiki was then set up into

departments as a way to provide specific professional learning communities a way to learn, share

and discuss ideas involving formative assessment in an online forum.

The Design Squad then created a homepage on the wiki describing its purpose and

structure. A YouTube was also embedded into the wiki to instruct “basic” users of web tools

how to use and create a wiki of their own. Based on the results of the SurveyMonkey, the team

began discussing the staff’s “familiarity” of the subject matter such as exit tickets as a way of

providing formative assessment. Where an exit ticket would be something the student could

“exit” the room upon answering a quick question about a learning objective or key point taught

during class, the web tool Socrative could provide an online way to do so.


The building does have one school lab, utilization of this tool would occur online through

Ellet High School’s BYOD policy through a wireless connection. To make sure “difficulty” was

addressed first, a ten minute YouTube was created as a way for teachers to easily and quickly set

up a class using Socrative, was embedded into the wiki as a place to begin for teachers seeking

formative assessment ideas.

An initial “interest” of the tool Socrative was initiated by Design Squad’s own, Denise

Brannan-Prischack in the setting of a staff meeting. Being a member of the Ellet staff, Denise


introduced Socrative to pique the interest of the staff so they could then experience effective

formative assessment and how it could be utilized in this particular environment.

Assessment Plan

The assessment on the effectiveness of the design of the materials will require additional

follow up with the subject matter expert, who will have to assess teaching to determine if

teachers and staff have increased their use of formative assessment to direct and drive

instruction. However, initial feedback from teachers and staff was gathered through the use of a

follow up survey.

Implementation Phase

The implementation phase began on October 10 when Design Squad sent a technology

survey to the Ellet staff. The survey addressed issues such as background knowledge, and

technology access. The results of the survey told us that due to the size of the staff and the time

availability of the staff, the best way to give them continuous support would be to create

something online that the staff could have access to, 24/7 and learn at their own pace. With this

information it was suggested by Eric, that Socrative might be the perfect tool for the staff to use

as a formative assessment piece. On October 17, our SME observed Denise using Socrative in a

classroom setting. Design Squad met with the SME through WebEx on October 19 to get

feedback from the observation. The SME was impressed with Socrative and suggested that it be

demonstrated to the staff at the monthly staff meeting on October 28. After getting a lot of

positive feedback from the staff, Design Squad determined that the best course of action would

be to create a staff Wiki with a tutorial for Socrative. Misty added a table to the Wiki with other

formative assessment tools and at the suggestion of the Ellet staff, department pages were added

so that each department could add tools that pertained to their specific subject. It is the Design


Squad’s hope that the Ellet staff will take this wiki and maintain it as a sort of living document

that will adjust to their needs. On November 17 we released the Wiki to the Ellet staff.

Throughout this process the Design Squad maintained progress through WebEx meetings, email,

and Google Docs.

Implementation of evaluation plan

On November 19, 2014 the Design Squad released an evaluation survey via Survey

Monkey to all Ellet staff members. This target audience also received and responded to the initial

survey. Staff was invited to the survey via email and the survey remained open for three days.

Only seven members of Ellet staff completed the survey, compared to 36 who took the initial

needs analysis survey Of the seven responders five were teachers, one participant was an

intervention specialist, and the last participant was a Librarian.

Collection of Formative Evaluation Data

The purpose of the evaluation survey was to evaluate the resources provided by the

Design Squad. The quality of data collected by the survey is poor since many of the questions

referred to the Wiki which the participants had not used. None of the seven members of the Ellet

staff who participated in the survey stated that they had visited the Wiki. Due to the low number

of respondents, couple with these respondents not visiting the Wiki the data from the survey

cannot assist in determining the usefulness of the Wiki as a formative assessment resource.

Although, the data does not determine the usefulness of the Wiki’s content it does give quality

data to the Design Squad about the usage of the Wiki. A point to consider in the data analysis

between the initial survey and the evaluation survey is the fact that the Design Squad was unable

to identify if respondents took one or both of the surveys. This lack of information makes it hard

to determine if any progress occurred in the usage of formative assessment at Ellet High School.


Verbal feedback from staff members to Denise gave positive results about the

demonstration. Staff member responses included that “Wiki and Socrative were very cool.”

Other staff members asked for additional help with Socrative and later used it in their classroom.

Additional, another staff member requested the login in information for the Wiki after the survey

was collected.

Assessment and Interpretation of Formative Evaluation Data

The formative evaluation data identifies a flaw the Design Squad did not foresee, which

was the marketing of the resources. Of the seven who took the survey, results indicate that three

people were unaware of the Ellet Staff Wiki or were unaware of what a Wiki was. Three other

participants stated they had not had time to visit and one stated that they had forgotten.

Therefore, based on this information the Design Squad did not provide enough information about

the existence and benefits of the Wiki to the Ellet High School Staff.

Additionally, the survey did not ask the participants to identify whether or not they

attended the Socrative demonstration or training sessions with Denise at Ellet High School. In

theory, knowledge that more than seven members of Ellet staff attended the original

demonstration and several requested additional training, had these individuals taken the

evaluation survey the Design Squad may have received data to further evaluate the actual content

of the Wiki and success of the training. The verbal feedback provided to Denise leads the Design

Squad to believe that the training session did improve overall percent of teachers using tools for

formative assessment in their class.

Recommendations for Changes

Based on the evaluation results the Design Squad needs to reevaluate the manner in

which they communicate with staff members of Ellet High School. The Design Squad also


should improve the presentation of the Wiki by explaining to the staff both what a Wiki is and

why the resource will be helpful to their classroom. Another piece that could be improved by the

Design Squad is timing. The follow up survey was opened two days prior to Thanksgiving break

during a shortened school week. It is possible that better survey results could have been produced

by planning to release the follow up survey both closer to training and during a normal school

week.

Since verbal feedback was positive for the product created and survey feedback gave no

feedback on the Wiki there are no recommendations for changes to the Wiki.


References

Great Schools: Ellet High School. (n.d.). Retrieved December 8, 2014, from

http://www.greatschools.org/ohio/akron/80-Ellet-High-School/details/#Students

Morrison, G., Ross, R., & Kemp, J. (2006/2010). Designing effective instruction. (5thed.).

New York: Wiley.

DESIGN SQUAD FINAL PAPER Denise Brannan-Prischak...

Documents

Transcript of DESIGN SQUAD FINAL PAPER Denise Brannan-Prischak...