Assessing Change in Changing Times
-
Upload
edtechteacherorg -
Category
Education
-
view
531 -
download
0
description
Transcript of Assessing Change in Changing Times
Did we change?
Justin Reich
Doctoral Student, Harvard Graduate School of Education
Co-Director, EdTechTeacher.org
@bjfr
Blog: edtechresearcher.org
Why Change?
How do we change?
Did we change?
Using Data to Address Instruction
Identify a Problem of
Practice
Gather data about the problem
Examine Instruction
Develop an Action Plan
Act and Assess
Further Reading: Data Wise and Data Wise in Action
The lower school at the American School of Brussels is working on increasing connections between home and school and having parents more aware of and involved in children’s classroom experiences.
They are requiring all elementary teachers to create blogs to help build these connections. How could you assess, short term and long term, whether this initiative is working?
Scenario 1
InterviewsFocus Group
SurveysContent Analysis
Test ScoreAnalysis
How would you test French Fry Quality at
McDonalds?
The Fakename School District has a Wordpress install for classroom blogging.
Teachers have been asked to consider Will Richardson’s definition of blogging, emphasizing student autonomy, creativity, and attentiveness to a real audience
Bud Hunt wanted to write his MA thesis about blogging in the district. What should he do?
Scenario 2
What did Bud do?• He took a sample of all of his school’s blog
posts• He read them• He categorized them• He drew some conclusions about the
kinds of learning taking place based on that analysis
American School of Doha has a 6-12 one-to-one program. They are implementing a 1-1 program in the fifth grade.
The tech directors want to assess the efficacy of their intervention.
What questions do you have? What could we do?
Scenario 3
What does awesome look
like?
The Newmarket High School has just purchased 300 iPads to allow them to pilot a 1 to 1 program in the ninth grade.
Teachers have been asked to create “21st century learning environments,” and they can use the iPads however they wish.
How could you assess whether this pilot should expand to the entire high school?
Scenario 4
Using Data to Address Instruction
Identify a Problem of
Practice
Gather data about the problem
Examine Instruction
Develop an Action Plan
Act and Assess
Further Reading: Data Wise and Data Wise in Action
Six Steps for Research1. Ask a Question
2. Define Indicators
3. Identify a Sample
4. Identify a Method
5. Gather Data
6. Analyze the Results
Ask a QuestionGood questions ask about how technology changes people’s experience of learning.
Two helpful prompts:
What is the learning goal before technology?
Is technology supposed to change the learning goal?
Define Indicators• Patrick Larkin said he wants students to
be more engaged. What are indicators of engagement?
• Chris Lehmann said he wants students to be able to ask good questions. What are indicators that students ask good questions?
Identify a Sample• Whole Population• Convenience Sample- whoever you can
get• Purposive Sample- a few, well-chosen
people• Random sample- often the best method
Identify a Method• Online Survey• Interview (phone or in person)• Classroom observation (online or in person)• Teaching Materials Review (online or in
print)• Looking at Student Work (online or in print) • Web Analytics• Test Score Analysis
Gather Data• Structured Data Collection Allows for
Structured Data Analysis– Ask all of your random sample of parents the
same four questions, and you can compare their answers
• Unstructured Data Collection Allows for Surprising Insight
Analyze the Results• Carefully crunch the numbers OR• Just look at stuff and see what you think
• How will the data make your school better?
Rules• Write your name on the back of your cards• Gain points by presenting research plans• Start Research with a Question Card• All cards only go face up, cards only connect on lines• Extend Research with Sample, sampling, method, and
other cards• All current cardholders must agree to add a new card• Cards only add constraints, anything else is allowed• Call Justin when you present, total points times
multiplier is score
Presentations• What is the question?• What are the indicators?• What methods will allow you to assess
those indicators?• How will the data gathered help you
answer to the question?• What could you do with your results to
improve the school?
Final Thoughts• Identify learning goals connected to
technology investments and interventions• Measure progress towards those goals• Draw random samples rather than
assessing everyone– (and actually make them random)
• Leverage partnerships with universities and other organizations
• Use assessment data to reflect upon and improve instruction