1

Evaluation Framework and Communication of Results

2

Evaluation Questions and Construction of Evaluation Framework

• Evaluation Questions—A Review

• Formative

• Summative

Question: From WHERE do we derive our Evaluation Questions?

3

Evaluation Framework

• What are the key constructs that will be measured?

• What changes am I hoping to find?

• What do I need to know about the strategies and content used in professional development?

• Who has the information I need to collect?

• How will I gather the information I need from each source?

• What tools or processes will I use to gather the information I need?

4

Evaluation Framework Components

• Program Goals

• Measurable Objectives

• Information/Data Needed

• Data Source

• Data Collection Strategy

• Data Analysis Plan

• Timeline

5

Creation of Evaluation Framework

Question: What do we consult to develop the Evaluation Framework?

6

Putting the Pieces Together

Theory of Change Logic Model Evaluation Framework

7

Questions to ask of your Theory of Change:

• What key concepts will be measured?

• How will those key concepts be measured?

• How will data from those measures be analyzed to construct the answer to your evaluation questions?

8

Evaluation Design:

• Experimental

• Quasi-experimental

• Descriptive

• Naturalistic

• Case studies

• Mixed-method

Communicating Results

9

• Allow your data to tell your story• Guiding questions or goals provide the

basis of your narrative (NOT the survey)

• Be clear about what was studied• Be concise! (Always include an

executive summary)• Be objective!

Creating Tables

10

• Write table titles that report exactly what is in your table

• Label every column and every row• Avoid using too many numbers in the

table• Report group sizes• Report whole number percentages• Present data in some sort of order (i.e.,

low to high)

11

Quantitative Data: Example

Descriptive Statistics

Figure 1. Total Years of Teaching Experience for Teachers who completed the Teacher Survey

Frequency Distribution

13

Daily/Weekly Seldom/NeverActivity % (n) % (n)

Manage student information 100% (88) 24% (21)

Communicate with colleagues 95% (84) 15% (13)

Develop instructional materials 84% (73) 23% (20)

Communicate with parents 87% (73) 27% (23)

Research to augment a lesson plan 74%

(65) 58% (51)

Post homework assignments 72% (63) 0% (0)

Access student assignments 65% (57) 48% (42)

Communicate with students 52% (45) 3% (3)

Distribute and collect tests 36% (32) 17% (15)

Table 6Teacher’s Frequency of Technology Use in the Classroom by Activity

Data…

14

Figure 2. Frequency of Student Use of Laptop Computers by Activity

15

Tamara OttoResearch and Evaluation

Center for Technology in Educationtamaraotto@jhu.edu