Regional Educational Laboratory at EDC relnei.org Logic Models to Support Program Design,...
-
Upload
angelica-lyons -
Category
Documents
-
view
214 -
download
1
Transcript of Regional Educational Laboratory at EDC relnei.org Logic Models to Support Program Design,...
![Page 1: Regional Educational Laboratory at EDC relnei.org Logic Models to Support Program Design, Implementation, and Evaluation Session I: Learning about Logic.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e055503460f94af1ac3/html5/thumbnails/1.jpg)
Regional Educational Laboratory at EDCrelnei.org
Logic Models to Support Program Design, Implementation, and Evaluation
Session I: Learning about Logic Models
Hosted by the [insert alliance here]Moderator:Presenter:
DateTime
1
![Page 2: Regional Educational Laboratory at EDC relnei.org Logic Models to Support Program Design, Implementation, and Evaluation Session I: Learning about Logic.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e055503460f94af1ac3/html5/thumbnails/2.jpg)
Regional Educational Laboratory at EDCrelnei.org
Introductions
2
Workshop facilitator:
Participants:
Name and affiliation
![Page 3: Regional Educational Laboratory at EDC relnei.org Logic Models to Support Program Design, Implementation, and Evaluation Session I: Learning about Logic.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e055503460f94af1ac3/html5/thumbnails/3.jpg)
Regional Educational Laboratory at EDCrelnei.org
Agenda
3
Introduction and goals
Introducing the cases
What is a logic model?
Elements of a logic model
The logic in a logic model
Next steps
![Page 4: Regional Educational Laboratory at EDC relnei.org Logic Models to Support Program Design, Implementation, and Evaluation Session I: Learning about Logic.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e055503460f94af1ac3/html5/thumbnails/4.jpg)
Regional Educational Laboratory at EDCrelnei.org
Session I – Goals
• Introduce logic models as an effective tool for program and policy design, implementation, and evaluation
• Practice the elements of a logic model
• Provide guidance on the appropriate steps for building a logic model for a program or initiative
4
![Page 5: Regional Educational Laboratory at EDC relnei.org Logic Models to Support Program Design, Implementation, and Evaluation Session I: Learning about Logic.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e055503460f94af1ac3/html5/thumbnails/5.jpg)
Regional Educational Laboratory at EDCrelnei.org
Case Examples
College-Ready and Blended Learning Programs
Activity I.1: Discussion of Cases
Consider one of the cases: •What are the goals of the program or policy?
•What might we want to know about it?
5
![Page 6: Regional Educational Laboratory at EDC relnei.org Logic Models to Support Program Design, Implementation, and Evaluation Session I: Learning about Logic.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e055503460f94af1ac3/html5/thumbnails/6.jpg)
Regional Educational Laboratory at EDCrelnei.org
What Is a Logic Model?
• Where are you going?
• How will you get there?
• What will tell you that you have arrived?
6
![Page 7: Regional Educational Laboratory at EDC relnei.org Logic Models to Support Program Design, Implementation, and Evaluation Session I: Learning about Logic.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e055503460f94af1ac3/html5/thumbnails/7.jpg)
Regional Educational Laboratory at EDCrelnei.org
What Is a Logic Model?
A logic model:
•Provides a simplified picture of the relationships between the program inputs and the desired outcomes of the program
•Is a framework for: o Planningo Implementationo Monitoringo Evaluation
•Is a graphic and explicit representation of relationships, assumptions, and rationale
7
![Page 8: Regional Educational Laboratory at EDC relnei.org Logic Models to Support Program Design, Implementation, and Evaluation Session I: Learning about Logic.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e055503460f94af1ac3/html5/thumbnails/8.jpg)
Regional Educational Laboratory at EDCrelnei.org
What Is a Logic Model?
A logic model is not:
• A strategic plan or a fully developed plan for designing or managing a program or policy
• An evaluation design or an evaluation method
8
![Page 9: Regional Educational Laboratory at EDC relnei.org Logic Models to Support Program Design, Implementation, and Evaluation Session I: Learning about Logic.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e055503460f94af1ac3/html5/thumbnails/9.jpg)
Regional Educational Laboratory at EDCrelnei.org
What Is a Logic Model?
Types of logic models:
•Theory approach model: Conceptual, emphasizes theory of change (program design)
•Activities approach model: Activities and relationships, detailed steps (program management and implementation)
•Outcomes approach model: Connects resources and activities with results and outcomes, may break up outcomes and impacts over time segments (program evaluation)
9
![Page 10: Regional Educational Laboratory at EDC relnei.org Logic Models to Support Program Design, Implementation, and Evaluation Session I: Learning about Logic.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e055503460f94af1ac3/html5/thumbnails/10.jpg)
Regional Educational Laboratory at EDCrelnei.org
What Is a Logic Model?
The simplest form of a logic model:
INPUTS OUTPUTS OUTCOMES
10
![Page 11: Regional Educational Laboratory at EDC relnei.org Logic Models to Support Program Design, Implementation, and Evaluation Session I: Learning about Logic.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e055503460f94af1ac3/html5/thumbnails/11.jpg)
Regional Educational Laboratory at EDCrelnei.org
What Is a Logic Model?
The simplest form of a logic model:
INPUTS OUTPUTS OUTCOMES
Inputs: What is invested in the program (e.g., money, people, time, and space)
Outputs: What is done in the program (e.g., program strategies and activities)
Outcomes: What results from the program (i.e., short- and long-term outcomes)
11
![Page 12: Regional Educational Laboratory at EDC relnei.org Logic Models to Support Program Design, Implementation, and Evaluation Session I: Learning about Logic.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e055503460f94af1ac3/html5/thumbnails/12.jpg)
Regional Educational Laboratory at EDCrelnei.org
What Is a Logic Model?
12
Inputs Outputs Outcomes
Existing technology infrastructure
Technology integration staff person for three schools
Teachers’ enthusiasm in three schools
Technology integration grant
Infrastructure audit completed
Six days of summer teacher professional development completed
Six blended-learning classrooms established
Teachers’ reported use of diverse instruction strategies increases.
Student engagement increases.
Student achievement on districtwide assessments improves.
Case: Blended-Learning Program
![Page 13: Regional Educational Laboratory at EDC relnei.org Logic Models to Support Program Design, Implementation, and Evaluation Session I: Learning about Logic.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e055503460f94af1ac3/html5/thumbnails/13.jpg)
Regional Educational Laboratory at EDCrelnei.org
What Is a Logic Model?
13
Inputs Outputs Outcomes
Activity I.2: Inputs – Outputs – Outcomes
![Page 14: Regional Educational Laboratory at EDC relnei.org Logic Models to Support Program Design, Implementation, and Evaluation Session I: Learning about Logic.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e055503460f94af1ac3/html5/thumbnails/14.jpg)
Regional Educational Laboratory at EDCrelnei.org
What Is a Logic Model?
14
Inputs Outputs Outcomes
Staff
Volunteer mentors
School space and resources
Teacher time
Course for parents
Mentoring for students
Guidance meetings
Student meetings
Parent involvement increases.
College applications increase.
College acceptances increase.
College attendance increases.
Case: College-Ready Program
![Page 15: Regional Educational Laboratory at EDC relnei.org Logic Models to Support Program Design, Implementation, and Evaluation Session I: Learning about Logic.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e055503460f94af1ac3/html5/thumbnails/15.jpg)
Regional Educational Laboratory at EDCrelnei.org
Elements of a Logic Model
15
Problem Statement
Resources
(inputs)
Strategies
andActivities
Outputs Short-Term
Outcomes
Long-Term
Outcomes
Impacts
Assumptions
![Page 16: Regional Educational Laboratory at EDC relnei.org Logic Models to Support Program Design, Implementation, and Evaluation Session I: Learning about Logic.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e055503460f94af1ac3/html5/thumbnails/16.jpg)
Regional Educational Laboratory at EDCrelnei.org
Elements of a Logic Model
The elements of a logic model:
•Problem statement
•Short- and long-term outcomes
•Impacts
•Outputs
•Strategies and activities
•Resources (inputs)
•Assumptions
16
![Page 17: Regional Educational Laboratory at EDC relnei.org Logic Models to Support Program Design, Implementation, and Evaluation Session I: Learning about Logic.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e055503460f94af1ac3/html5/thumbnails/17.jpg)
Regional Educational Laboratory at EDCrelnei.org
Elements of a Logic Model:Problem Statement
Problem statement: The problem or challenge that the program or policy is designed to address
Questions to ask in defining the problem:
•What is the problem or issue?
•Why is this a problem?
•For whom does this problem exist?
•Who has a stake in the problem?
•What is known about the problem (through previous work, research, etc.)?
17
![Page 18: Regional Educational Laboratory at EDC relnei.org Logic Models to Support Program Design, Implementation, and Evaluation Session I: Learning about Logic.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e055503460f94af1ac3/html5/thumbnails/18.jpg)
Regional Educational Laboratory at EDCrelnei.org
Elements of a Logic Model:Problem StatementProblem statement: The problem or challenge that the program or policy is designed to address
Case: Blended-Learning Program
18
• Students are not actively engaged in their learning.
• Courses are sometimes monotonous.
• Students have limited one-on-one attention from adults.
• Students’ courses are not personalized.
• Students are all expected to work at the same pace.
![Page 19: Regional Educational Laboratory at EDC relnei.org Logic Models to Support Program Design, Implementation, and Evaluation Session I: Learning about Logic.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e055503460f94af1ac3/html5/thumbnails/19.jpg)
Regional Educational Laboratory at EDCrelnei.org
Elements of a Logic Model:Problem Statement
Problem statement: The problem or challenge that the program or policy is designed to address
Activity I.3: Problem Statement
19
• Articulate a targeted and specific problem
• Avoid a problem statement that restates the program as a need
![Page 20: Regional Educational Laboratory at EDC relnei.org Logic Models to Support Program Design, Implementation, and Evaluation Session I: Learning about Logic.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e055503460f94af1ac3/html5/thumbnails/20.jpg)
Regional Educational Laboratory at EDCrelnei.org
Elements of a Logic Model:Outcomes
Outcomes: What difference does it make?
20
![Page 21: Regional Educational Laboratory at EDC relnei.org Logic Models to Support Program Design, Implementation, and Evaluation Session I: Learning about Logic.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e055503460f94af1ac3/html5/thumbnails/21.jpg)
Regional Educational Laboratory at EDCrelnei.org
Elements of a Logic Model:Outcomes
Outcomes: What difference does it make?
Short-term Long-term Impacts
21
![Page 22: Regional Educational Laboratory at EDC relnei.org Logic Models to Support Program Design, Implementation, and Evaluation Session I: Learning about Logic.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e055503460f94af1ac3/html5/thumbnails/22.jpg)
Regional Educational Laboratory at EDCrelnei.org
Elements of a Logic Model:Outcomes
Outcomes: What difference does it make?
Short-term Long-term Impacts
22
Most immediate and measurable results for participants that can be attributed to
strategies and activities
Most immediate and measurable results for participants that can be attributed to
strategies and activities
![Page 23: Regional Educational Laboratory at EDC relnei.org Logic Models to Support Program Design, Implementation, and Evaluation Session I: Learning about Logic.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e055503460f94af1ac3/html5/thumbnails/23.jpg)
Regional Educational Laboratory at EDCrelnei.org
Elements of a Logic Model:Outcomes
Outcomes: What difference does it make?
Short-term Long-term Impacts
23
Most immediate and measurable results for participants that can be attributed to
strategies and activities
Most immediate and measurable results for participants that can be attributed to
strategies and activities
More distant, though anticipated, results of participation in
strategies and activities
More distant, though anticipated, results of participation in
strategies and activities
![Page 24: Regional Educational Laboratory at EDC relnei.org Logic Models to Support Program Design, Implementation, and Evaluation Session I: Learning about Logic.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e055503460f94af1ac3/html5/thumbnails/24.jpg)
Regional Educational Laboratory at EDCrelnei.org
Elements of a Logic Model:Outcomes
Outcomes: What difference does it make?
Short-term Long-term Impacts
24
Most immediate and measurable results for participants that can be attributed to
strategies and activities
Most immediate and measurable results for participants that can be attributed to
strategies and activities
More distant, though anticipated, results of participation in
strategies and activities
More distant, though anticipated, results of participation in
strategies and activities
Desired outcomes of long-term
implementation of strategies and
activities, dependent on conditions
beyond the scope of the program
Desired outcomes of long-term
implementation of strategies and
activities, dependent on conditions
beyond the scope of the program
![Page 25: Regional Educational Laboratory at EDC relnei.org Logic Models to Support Program Design, Implementation, and Evaluation Session I: Learning about Logic.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e055503460f94af1ac3/html5/thumbnails/25.jpg)
Regional Educational Laboratory at EDCrelnei.org
Elements of a Logic Model:Outcomes
Outcomes: What difference does it make?
Case: College-Ready Program
Short-term Long-term Impacts
25
Increased contact with parents or
guardians
Increased contact with parents or
guardians
Improved attendance and academic performance
Improved attendance and academic performance
Increased percentage of
students graduating from postsecondary
institutions
Increased percentage of
students graduating from postsecondary
institutions
![Page 26: Regional Educational Laboratory at EDC relnei.org Logic Models to Support Program Design, Implementation, and Evaluation Session I: Learning about Logic.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e055503460f94af1ac3/html5/thumbnails/26.jpg)
Regional Educational Laboratory at EDCrelnei.org
Elements of a Logic Model:Outcomes
26
Outcomes: What difference does it make?
Activity I.4: Focus on Outcomes
![Page 27: Regional Educational Laboratory at EDC relnei.org Logic Models to Support Program Design, Implementation, and Evaluation Session I: Learning about Logic.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e055503460f94af1ac3/html5/thumbnails/27.jpg)
Regional Educational Laboratory at EDCrelnei.org
Elements of a Logic Model:Outcomes
Outcomes Checklist•Important•Reasonable•Realistic•Unintentional, possibly negative
27
![Page 28: Regional Educational Laboratory at EDC relnei.org Logic Models to Support Program Design, Implementation, and Evaluation Session I: Learning about Logic.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e055503460f94af1ac3/html5/thumbnails/28.jpg)
Regional Educational Laboratory at EDCrelnei.org
Elements of a Logic Model: Strategies and Activities
Strategies and activities: What you propose to do to address the problem
Activities, services, events, and products:•Are designed to address the problem
•Are, together, intended to lead to certain outcomes
28
![Page 29: Regional Educational Laboratory at EDC relnei.org Logic Models to Support Program Design, Implementation, and Evaluation Session I: Learning about Logic.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e055503460f94af1ac3/html5/thumbnails/29.jpg)
Regional Educational Laboratory at EDCrelnei.org
Elements of a Logic Model: Strategies and Activities
Strategies and activities: What you propose to do to address the problem
Example: Blended-Learning Program
29
Activities Sequence Strategy
Develop teacher training materials 1st Professional training
Deliver summer institute 2nd Professional training
Conduct technology audit 1st Infrastructure
And so forth…
![Page 30: Regional Educational Laboratory at EDC relnei.org Logic Models to Support Program Design, Implementation, and Evaluation Session I: Learning about Logic.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e055503460f94af1ac3/html5/thumbnails/30.jpg)
Regional Educational Laboratory at EDCrelnei.org
Elements of a Logic Model: Strategies and Activities
30
Any questions so far?
![Page 31: Regional Educational Laboratory at EDC relnei.org Logic Models to Support Program Design, Implementation, and Evaluation Session I: Learning about Logic.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e055503460f94af1ac3/html5/thumbnails/31.jpg)
Regional Educational Laboratory at EDCrelnei.org
Elements of a Logic Model: Resources
Resources (inputs): The material and intangible contributions that are or could reasonably be expected to be available to address the problem
Examples: •Money, materials, and equipment (material/tangible)
•People, time, and partnerships (intangible)
Resources are the inputs that enable the creation of the strategies and activities that are designed to respond to the stated problem.
31
![Page 32: Regional Educational Laboratory at EDC relnei.org Logic Models to Support Program Design, Implementation, and Evaluation Session I: Learning about Logic.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e055503460f94af1ac3/html5/thumbnails/32.jpg)
Regional Educational Laboratory at EDCrelnei.org
Elements of a Logic Model: Resources
Resources (inputs): The material and intangible contributions that are or could reasonably be expected to be available to address the problem
Case: College-Ready Program
32
• Community mentors
• Local university space for parent meetings
• Volunteer college admissions directors for application workshop
• Student volunteers for childcare at parent meetings
![Page 33: Regional Educational Laboratory at EDC relnei.org Logic Models to Support Program Design, Implementation, and Evaluation Session I: Learning about Logic.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e055503460f94af1ac3/html5/thumbnails/33.jpg)
Regional Educational Laboratory at EDCrelnei.org
Elements of a Logic Model: Resources
Intangible resources: What intangible resources are at your disposal?
Activity I.5: Intangible Resources
33
Brainstorm at least five nonmonetary resources that are available to you in a program you operate or manage.
![Page 34: Regional Educational Laboratory at EDC relnei.org Logic Models to Support Program Design, Implementation, and Evaluation Session I: Learning about Logic.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e055503460f94af1ac3/html5/thumbnails/34.jpg)
Regional Educational Laboratory at EDCrelnei.org
Elements of a Logic model: Assumptions
Assumptions: Beliefs about participants, staff, the program, and how change or improvement may be realized
Make explicit all implicit assumptions:
•Assumptions can be internal and external.
•Ask: What is known, and what is being assumed?
34
![Page 35: Regional Educational Laboratory at EDC relnei.org Logic Models to Support Program Design, Implementation, and Evaluation Session I: Learning about Logic.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e055503460f94af1ac3/html5/thumbnails/35.jpg)
Regional Educational Laboratory at EDCrelnei.org
Elements of a Logic model: Assumptions
Assumptions: Beliefs about participants, staff, the program, and how change or improvement may be realized
Case: Blended-Learning Program
35
Internal Assumptions External Assumptions
• The participating school leadership will continue to support the program.
• Three staff members will be sufficient to support the program in three schools.
• Access to a range of modalities will increase student engagement.
• Increased student engagement will increase academic achievement.
![Page 36: Regional Educational Laboratory at EDC relnei.org Logic Models to Support Program Design, Implementation, and Evaluation Session I: Learning about Logic.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e055503460f94af1ac3/html5/thumbnails/36.jpg)
Regional Educational Laboratory at EDCrelnei.org
Elements of a Logic model: Assumptions
Assumptions: Beliefs about participants, staff, the program, and how change or improvement may be realized
Activity I.6: Uncovering Internal and External Assumptions
What assumptions are you or your program making?
36
![Page 37: Regional Educational Laboratory at EDC relnei.org Logic Models to Support Program Design, Implementation, and Evaluation Session I: Learning about Logic.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e055503460f94af1ac3/html5/thumbnails/37.jpg)
Regional Educational Laboratory at EDCrelnei.org
The Logic in a Logic Model
The theory embedded in the model…
A series of if-then statements across the model
37
![Page 38: Regional Educational Laboratory at EDC relnei.org Logic Models to Support Program Design, Implementation, and Evaluation Session I: Learning about Logic.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e055503460f94af1ac3/html5/thumbnails/38.jpg)
Regional Educational Laboratory at EDCrelnei.org
The Logic in a Logic Model
The theory embedded in the model…
A series of if-then statements across the model
Case: Blended-Learning Program
If: Then: and If: Then:
38
District invests in blended learning in
three schools
District invests in blended learning in
three schools
Instruction will be personalized and
participating students will be more
engaged
Instruction will be personalized and
participating students will be more
engaged
Student achievement will increase as measured by standardized assessment
Student achievement will increase as measured by standardized assessment
![Page 39: Regional Educational Laboratory at EDC relnei.org Logic Models to Support Program Design, Implementation, and Evaluation Session I: Learning about Logic.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e055503460f94af1ac3/html5/thumbnails/39.jpg)
Regional Educational Laboratory at EDCrelnei.org
The Logic in a Logic Model
Activity I.7: If-Then Statements
Order the if-then statements in the example from the College-Ready Program case
39
![Page 40: Regional Educational Laboratory at EDC relnei.org Logic Models to Support Program Design, Implementation, and Evaluation Session I: Learning about Logic.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e055503460f94af1ac3/html5/thumbnails/40.jpg)
Regional Educational Laboratory at EDCrelnei.org
Next Steps
What we have accomplished so far?•Discussed the purpose of a logic model
•Presented the elements of a logic model
•Considered the logic embedded in a logic model
40
![Page 41: Regional Educational Laboratory at EDC relnei.org Logic Models to Support Program Design, Implementation, and Evaluation Session I: Learning about Logic.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e055503460f94af1ac3/html5/thumbnails/41.jpg)
Regional Educational Laboratory at EDCrelnei.org
Next Steps
Ask yourself the following: •Do I understand the elements of the logic model and how they differ?•Who should I consult in developing the model? What colleagues and stakeholders should be participants in developing the logic model? •Who will shepherd or see through the development of the logic model?•How do I know we have captured the theory of action? •How will we use the logic model? •How will we ensure we make it a living document?
41
![Page 42: Regional Educational Laboratory at EDC relnei.org Logic Models to Support Program Design, Implementation, and Evaluation Session I: Learning about Logic.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e055503460f94af1ac3/html5/thumbnails/42.jpg)
Regional Educational Laboratory at EDCrelnei.org
Next Steps
Your next steps…
Activity I.8:
What Are Your Next Steps with
Regard to Logic Models?
42
![Page 43: Regional Educational Laboratory at EDC relnei.org Logic Models to Support Program Design, Implementation, and Evaluation Session I: Learning about Logic.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e055503460f94af1ac3/html5/thumbnails/43.jpg)
Regional Educational Laboratory at EDCrelnei.org
Final Thoughts on Logic Models
Some final thoughts…
•Logic models are tools for program design, implementation, and evaluation.
•The process of developing a logic model is important: Engage stakeholders in developing a logic model.
•Logic models should be living documents and returned to frequently.
•Logic models are useful for evaluation but best when developed at the program design phase.
43
![Page 44: Regional Educational Laboratory at EDC relnei.org Logic Models to Support Program Design, Implementation, and Evaluation Session I: Learning about Logic.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e055503460f94af1ac3/html5/thumbnails/44.jpg)
Regional Educational Laboratory at EDCrelnei.org
Thank You!
For any questions about this workshop, contact:
44
![Page 45: Regional Educational Laboratory at EDC relnei.org Logic Models to Support Program Design, Implementation, and Evaluation Session I: Learning about Logic.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e055503460f94af1ac3/html5/thumbnails/45.jpg)
Regional Educational Laboratory at EDCrelnei.org
Logic Models to Support Program Design, Implementation, and Evaluation
Session II: From Logic Models to Program and Policy Evaluation
Hosted by the [insert alliance here]Moderator:Presenter:
DateTime
45
![Page 46: Regional Educational Laboratory at EDC relnei.org Logic Models to Support Program Design, Implementation, and Evaluation Session I: Learning about Logic.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e055503460f94af1ac3/html5/thumbnails/46.jpg)
Regional Educational Laboratory at EDCrelnei.org
Introductions
Workshop facilitator:
Participants:
Name and affiliation
46
![Page 47: Regional Educational Laboratory at EDC relnei.org Logic Models to Support Program Design, Implementation, and Evaluation Session I: Learning about Logic.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e055503460f94af1ac3/html5/thumbnails/47.jpg)
Regional Educational Laboratory at EDCrelnei.org
Agenda
47
Introduction and goals
Review of logic models
Introducing evaluations
Moving from logic models to evaluation questions
Generating indicators
Building an evaluation design
Putting it all together
![Page 48: Regional Educational Laboratory at EDC relnei.org Logic Models to Support Program Design, Implementation, and Evaluation Session I: Learning about Logic.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e055503460f94af1ac3/html5/thumbnails/48.jpg)
Regional Educational Laboratory at EDCrelnei.org
Session II – Goals
• Reintroduce logic models as an effective tool, specifically for evaluation.
• Practice using logic models to develop evaluation questions and indicators of success.
• Provide guidance on how to determine the appropriate evaluation for a specific program or policy.
48
![Page 49: Regional Educational Laboratory at EDC relnei.org Logic Models to Support Program Design, Implementation, and Evaluation Session I: Learning about Logic.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e055503460f94af1ac3/html5/thumbnails/49.jpg)
Regional Educational Laboratory at EDCrelnei.org
Review of Logic Models
A logic model is:•A graphic representation of theory of change
•A framework for planning, implementation, monitoring, and evaluation
A logic models is not:•A strategic plan or a fully developed plan for designing or managing a program or policy
•An evaluation design or evaluation method
49
![Page 50: Regional Educational Laboratory at EDC relnei.org Logic Models to Support Program Design, Implementation, and Evaluation Session I: Learning about Logic.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e055503460f94af1ac3/html5/thumbnails/50.jpg)
Regional Educational Laboratory at EDCrelnei.org
Review of Logic Models
50
Problem Statement
Resources
(inputs)
Strategies
andActivities
Outputs Short-Term
Outcomes
Long-Term
Outcomes
Impacts
Assumptions
![Page 51: Regional Educational Laboratory at EDC relnei.org Logic Models to Support Program Design, Implementation, and Evaluation Session I: Learning about Logic.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e055503460f94af1ac3/html5/thumbnails/51.jpg)
Regional Educational Laboratory at EDCrelnei.org
College Ready Logic Model Excerpt
51
Problem Statement: Low-income high school students in selected communities attend college at a lower rate than their middle-class peers, leading to more limited opportunities, higher rates of unemployment, and lower earnings.
Resources Strategies and activities
Outputs Short-term outcomes
Long-term outcomes
Impacts
Partnership with 3 public high schools
Establish local college mentorship program.
Recruited adequate number of mentors for student cohort.
Participating students apply to at least one college on time.
Participating students are accepted to and attend college, remaining enrolled into the third semester.
Low-incomestudents inparticipatingcommunitiesattend collegeat the same rate as middle-classpeers.
Assumptions: College attendance is desired goal for participating communities; high school leaders will remain consistent and support program; parents will show interest and participate in program.
![Page 52: Regional Educational Laboratory at EDC relnei.org Logic Models to Support Program Design, Implementation, and Evaluation Session I: Learning about Logic.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e055503460f94af1ac3/html5/thumbnails/52.jpg)
Regional Educational Laboratory at EDCrelnei.org
Review of Logic Models
Questions to ask about your logic model:•What elements of the logic model were hardest to develop?
•Is the problem statement the right “grain size”?
•Within the strategies and activities, did you identify overarching strategies?
•What assumptions did you uncover?
•What is the time frame for your outcomes?
•What are the impacts?
•What was your process for developing the model?
•What requires further explanation or discussion?
52
![Page 53: Regional Educational Laboratory at EDC relnei.org Logic Models to Support Program Design, Implementation, and Evaluation Session I: Learning about Logic.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e055503460f94af1ac3/html5/thumbnails/53.jpg)
Regional Educational Laboratory at EDCrelnei.org
Introducing Evaluation
Evaluation asks the questions:•Are we successful?
•Have we had an impact?
•What are the most influential aspects of the program?
53
![Page 54: Regional Educational Laboratory at EDC relnei.org Logic Models to Support Program Design, Implementation, and Evaluation Session I: Learning about Logic.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e055503460f94af1ac3/html5/thumbnails/54.jpg)
Regional Educational Laboratory at EDCrelnei.org
Introducing Evaluation
Consider:•Is the program or policy effective?
•Is the program or policy working as intended?
•What aspects of the program are working? What aspects are not working?
Timing:•Ask these questions at the onset of program design.
•Involve stakeholders in the evaluation design.
•Invest early in designing a good evaluation.
54
![Page 55: Regional Educational Laboratory at EDC relnei.org Logic Models to Support Program Design, Implementation, and Evaluation Session I: Learning about Logic.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e055503460f94af1ac3/html5/thumbnails/55.jpg)
Regional Educational Laboratory at EDCrelnei.org
Introducing Evaluation
Consider:•Is the program or policy effective?
•Is the program or policy working as intended?
•What aspects of the program are working? What aspects are not working?
Activity II.1: How Will You Know?
55
![Page 56: Regional Educational Laboratory at EDC relnei.org Logic Models to Support Program Design, Implementation, and Evaluation Session I: Learning about Logic.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e055503460f94af1ac3/html5/thumbnails/56.jpg)
Regional Educational Laboratory at EDCrelnei.org
Introducing Evaluation
Most evaluations are designed to improve or prove:
•Improve: Formative evaluations focus on strategies, activities, and outputs. They are also called process or implementation evaluations.
•Prove: Summative evaluations focus on outcomes. They are also called results or outcomes evaluations.
56
![Page 57: Regional Educational Laboratory at EDC relnei.org Logic Models to Support Program Design, Implementation, and Evaluation Session I: Learning about Logic.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e055503460f94af1ac3/html5/thumbnails/57.jpg)
Regional Educational Laboratory at EDCrelnei.org
Introducing Evaluation
Four types of evaluation:
•Formativeo Needs assessmento Process evaluation
• Summativeo Outcome evaluationo Impact evaluation
57
![Page 58: Regional Educational Laboratory at EDC relnei.org Logic Models to Support Program Design, Implementation, and Evaluation Session I: Learning about Logic.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e055503460f94af1ac3/html5/thumbnails/58.jpg)
Regional Educational Laboratory at EDCrelnei.org
Moving from Logic Model to Evaluation Questions
Formative questions:
• Asked while program is operating
• For program improvement or midcourse correction
Summative questions:
• Asked at completion or after the program• What was the result?• Was it effective?
58
![Page 59: Regional Educational Laboratory at EDC relnei.org Logic Models to Support Program Design, Implementation, and Evaluation Session I: Learning about Logic.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e055503460f94af1ac3/html5/thumbnails/59.jpg)
Regional Educational Laboratory at EDCrelnei.org
Moving from Logic Model to Evaluation Questions
Guidelines for good evaluation questions:
•Can the questions be answered given the program?
•Are the questions high priority?
•Are the questions practical and appropriate to the capacity you have to answer them?
•Are the questions clear and free of jargon?
59
![Page 60: Regional Educational Laboratory at EDC relnei.org Logic Models to Support Program Design, Implementation, and Evaluation Session I: Learning about Logic.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e055503460f94af1ac3/html5/thumbnails/60.jpg)
Regional Educational Laboratory at EDCrelnei.org
Moving from Logic Model to Evaluation Questions
Activity II.2: Formative and Summative Evaluation
What is a formative evaluation question you have about a program or policy?
or
What is a summative evaluation question you have about a program or policy?
60
![Page 61: Regional Educational Laboratory at EDC relnei.org Logic Models to Support Program Design, Implementation, and Evaluation Session I: Learning about Logic.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e055503460f94af1ac3/html5/thumbnails/61.jpg)
Regional Educational Laboratory at EDCrelnei.org
Moving from Logic Model to Evaluation Questions
When considering an evaluation, keep your audience in mind:
•Audience: Who wants to know?
(participants, funders, staff)
•Questions: What does the audience want to know?
(Is the policy helping? Did we reach the target population? How could the program be improved?)
•Use: How will results be used?
(continued participation, program changes, funding)
61
![Page 62: Regional Educational Laboratory at EDC relnei.org Logic Models to Support Program Design, Implementation, and Evaluation Session I: Learning about Logic.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e055503460f94af1ac3/html5/thumbnails/62.jpg)
Regional Educational Laboratory at EDCrelnei.org
Moving from Logic Model to Evaluation Questions
Audience Typical Questions Evaluation Use
Program staff Are we reaching our target population?
Program operations
Participants Is the program helping people like me?
Participation
Public officials Who does the program serve?
Support, commitment, scale-up, and duplication
Funders Is the program meeting its goals? Is the program worth the cost?
Ongoing funding, accountability
62
Source: W.K. Kellogg Foundation, 2006
![Page 63: Regional Educational Laboratory at EDC relnei.org Logic Models to Support Program Design, Implementation, and Evaluation Session I: Learning about Logic.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e055503460f94af1ac3/html5/thumbnails/63.jpg)
Regional Educational Laboratory at EDCrelnei.org
Moving from Logic Model to Evaluation Questions
Audience Typical Questions Evaluation Use
Program staff
Participants
Public officials
Funders
63
Activity II.3: Generating Questions for Different Audiences
![Page 64: Regional Educational Laboratory at EDC relnei.org Logic Models to Support Program Design, Implementation, and Evaluation Session I: Learning about Logic.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e055503460f94af1ac3/html5/thumbnails/64.jpg)
Regional Educational Laboratory at EDCrelnei.org
Moving from Logic Model to Evaluation Questions
64
Any questions so far?
![Page 65: Regional Educational Laboratory at EDC relnei.org Logic Models to Support Program Design, Implementation, and Evaluation Session I: Learning about Logic.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e055503460f94af1ac3/html5/thumbnails/65.jpg)
Regional Educational Laboratory at EDCrelnei.org
Generating Indicators
Activity II.4:
How Do We Know If a Child Has the flu?
65
![Page 66: Regional Educational Laboratory at EDC relnei.org Logic Models to Support Program Design, Implementation, and Evaluation Session I: Learning about Logic.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e055503460f94af1ac3/html5/thumbnails/66.jpg)
Regional Educational Laboratory at EDCrelnei.org
Generating Indicators
How will we know the program is successful?
Indicators of success…
66
![Page 67: Regional Educational Laboratory at EDC relnei.org Logic Models to Support Program Design, Implementation, and Evaluation Session I: Learning about Logic.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e055503460f94af1ac3/html5/thumbnails/67.jpg)
Regional Educational Laboratory at EDCrelnei.org
Generating Indicators
Indicators are:
•Specific, measureable targets
•Seen, heard, read, and felt
•Connected to strategies, activities, outputs, and outcomes
•Evidence representing phenomenon of interest
67
![Page 68: Regional Educational Laboratory at EDC relnei.org Logic Models to Support Program Design, Implementation, and Evaluation Session I: Learning about Logic.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e055503460f94af1ac3/html5/thumbnails/68.jpg)
Regional Educational Laboratory at EDCrelnei.org
.
68
For example, resources
(tangible and intangible)
For example, resources
(tangible and intangible)
InputsInputs
For example, strategies or
activities, participation
For example, strategies or
activities, participation
OutputsOutputs Outcomes or ImpactOutcomes or Impact
For example, short-term,
long-term, impact
For example, short-term,
long-term, impact
From the logic model
Generating Indicators: Using the Logic Model
![Page 69: Regional Educational Laboratory at EDC relnei.org Logic Models to Support Program Design, Implementation, and Evaluation Session I: Learning about Logic.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e055503460f94af1ac3/html5/thumbnails/69.jpg)
Regional Educational Laboratory at EDCrelnei.org
.
69
For example, resources
(tangible and intangible)
For example, resources
(tangible and intangible)
InputsInputs
For example, strategies or
activities, participation
For example, strategies or
activities, participation
Indicators
OutputsOutputs Outcomes or ImpactOutcomes or Impact
From the logic model
Amount of resources used
Amount of resources used
Generating Indicators: Using the Logic Model
For example, short-term,
long-term, impact
For example, short-term,
long-term, impact
![Page 70: Regional Educational Laboratory at EDC relnei.org Logic Models to Support Program Design, Implementation, and Evaluation Session I: Learning about Logic.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e055503460f94af1ac3/html5/thumbnails/70.jpg)
Regional Educational Laboratory at EDCrelnei.org
.
70
Number of workshops,number of
participants
Number of workshops,number of
participants
For example, resources
(tangible and intangible)
For example, resources
(tangible and intangible)
InputsInputs
For example, strategies or
activities, participation
For example, strategies or
activities, participation
Indicators
OutputsOutputs Outcomes or ImpactOutcomes or Impact
From the logic model
Amount of resources used
Amount of resources used
Generating Indicators: Using the Logic Model
For example, short-term,
long-term, impact
For example, short-term,
long-term, impact
![Page 71: Regional Educational Laboratory at EDC relnei.org Logic Models to Support Program Design, Implementation, and Evaluation Session I: Learning about Logic.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e055503460f94af1ac3/html5/thumbnails/71.jpg)
Regional Educational Laboratory at EDCrelnei.org
.
71
Number of workshops,number of
participants
Number of workshops,number of
participants
For example, resources
(tangible and intangible)
For example, resources
(tangible and intangible)
InputsInputs
For example, strategies or
activities, participation
For example, strategies or
activities, participation
Number & percent who learned
material, overall improvement
Number & percent who learned
material, overall improvement
Indicators
OutputsOutputs Outcomes or ImpactOutcomes or Impact
From the logic model
Amount of resources used
Amount of resources used
Generating Indicators: Using the Logic Model
For example, short-term,
long-term, impact
For example, short-term,
long-term, impact
![Page 72: Regional Educational Laboratory at EDC relnei.org Logic Models to Support Program Design, Implementation, and Evaluation Session I: Learning about Logic.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e055503460f94af1ac3/html5/thumbnails/72.jpg)
Regional Educational Laboratory at EDCrelnei.org
72
Number of classes delivered, number of
parents attended
Number of classes delivered, number of
parents attended
Deliverparent education
classes
Deliverparent education
classes
ActivityActivity OutcomeOutcome
Parents understand college application
process
Parents understand college application
process
Number of parents reporting increased
understanding
Number of parents reporting increased
understanding
Indicators
From the logic model
Generating Indicators: Using the Logic Model
![Page 73: Regional Educational Laboratory at EDC relnei.org Logic Models to Support Program Design, Implementation, and Evaluation Session I: Learning about Logic.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e055503460f94af1ac3/html5/thumbnails/73.jpg)
Regional Educational Laboratory at EDCrelnei.org
Ask these basic questions:
•What would achieving the goal reflected in the outcome look like?
•How would we know if we achieved it?
•If I were visiting the program, what would I see, hear, or read that would tell me that the program is doing what it intends?
73
Generating Indicators: Using the Logic Model
![Page 74: Regional Educational Laboratory at EDC relnei.org Logic Models to Support Program Design, Implementation, and Evaluation Session I: Learning about Logic.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e055503460f94af1ac3/html5/thumbnails/74.jpg)
Regional Educational Laboratory at EDCrelnei.org
74
Activity II.5: Process and Outcome Indicators
Look at the logic model and map a path:
Activity Output Outcome Indicator
Generating Indicators: Using the Logic Model
![Page 75: Regional Educational Laboratory at EDC relnei.org Logic Models to Support Program Design, Implementation, and Evaluation Session I: Learning about Logic.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e055503460f94af1ac3/html5/thumbnails/75.jpg)
Regional Educational Laboratory at EDCrelnei.org
Activity Deliver a set of parent workshops for college readiness
Output Six workshops developed and delivered, 100 parents recruited to participate
Outcome Parents increase their understanding of college application process
Indicators Process:
Outcome:
75
Example: College-Ready Program
Generating Indicators: Using the Logic Model
![Page 76: Regional Educational Laboratory at EDC relnei.org Logic Models to Support Program Design, Implementation, and Evaluation Session I: Learning about Logic.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e055503460f94af1ac3/html5/thumbnails/76.jpg)
Regional Educational Laboratory at EDCrelnei.org
Activity Deliver a set of parent workshops for college readiness
Output Six workshops developed and delivered, 100 parents recruited to participate
Outcome Parents increase their understanding of college application process
Indicators Process: 70 percent of parents attend at least five of the six workshops.
Outcome: 80 percent of students in program complete at least one college application by deadline.
76
Example: College-Ready Program
Generating Indicators: Using the Logic Model
![Page 77: Regional Educational Laboratory at EDC relnei.org Logic Models to Support Program Design, Implementation, and Evaluation Session I: Learning about Logic.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e055503460f94af1ac3/html5/thumbnails/77.jpg)
Regional Educational Laboratory at EDCrelnei.org
Generating Indicators: Identifying the Right Indicators
.
77
Number of workshops,number of
participants
Number of workshops,number of
participants
InputsInputs
Number & percent who learned
material, overall improvement
Number & percent who learned
material, overall improvement
Indicators
OutputsOutputs Outcomes or Impact
Outcomes or Impact
From the logic model
Amount of resources used
Amount of resources used
Evaluation questions
Were the inputs sufficient?
Were the inputs sufficient?
![Page 78: Regional Educational Laboratory at EDC relnei.org Logic Models to Support Program Design, Implementation, and Evaluation Session I: Learning about Logic.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e055503460f94af1ac3/html5/thumbnails/78.jpg)
Regional Educational Laboratory at EDCrelnei.org
Generating Indicators: Identifying the Right Indicators
.
78
Number of workshops,number of
participants
Number of workshops,number of
participants
InputsInputs
Number & percent who learned
material, overall improvement
Number & percent who learned
material, overall improvement
Indicators
OutputsOutputs Outcomes or Impact
Outcomes or Impact
From the logic model
Amount of resources used
Amount of resources used
Evaluation questions
Were the inputs sufficient?
Were the inputs sufficient?
Were workshops implemented as intended? Well
attended?
Were workshops implemented as intended? Well
attended?
![Page 79: Regional Educational Laboratory at EDC relnei.org Logic Models to Support Program Design, Implementation, and Evaluation Session I: Learning about Logic.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e055503460f94af1ac3/html5/thumbnails/79.jpg)
Regional Educational Laboratory at EDCrelnei.org
Generating Indicators: Identifying the Right Indicators
.
79
Number of workshops,number of
participants
Number of workshops,number of
participants
InputsInputs
Number & percent who learned
material, overall improvement
Number & percent who learned
material, overall improvement
Indicators
OutputsOutputs Outcomes or Impact
Outcomes or Impact
From the logic model
Amount of resources used
Amount of resources used
Evaluation questions
Were the inputs sufficient?
Were the inputs sufficient?
Were workshops implemented as intended? Well
attended?
Were workshops implemented as intended? Well
attended?
Did program change participant knowledge? Skill?
Did program change participant knowledge? Skill?
![Page 80: Regional Educational Laboratory at EDC relnei.org Logic Models to Support Program Design, Implementation, and Evaluation Session I: Learning about Logic.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e055503460f94af1ac3/html5/thumbnails/80.jpg)
Regional Educational Laboratory at EDCrelnei.org
Generating Indicators: Identifying the Right Indicators
• One indicator to measure drop-out reduction =
80
![Page 81: Regional Educational Laboratory at EDC relnei.org Logic Models to Support Program Design, Implementation, and Evaluation Session I: Learning about Logic.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e055503460f94af1ac3/html5/thumbnails/81.jpg)
Regional Educational Laboratory at EDCrelnei.org
Generating Indicators: Identifying the Right Indicators
• One indicator to measure drop-out reduction = o Graduation rate
81
![Page 82: Regional Educational Laboratory at EDC relnei.org Logic Models to Support Program Design, Implementation, and Evaluation Session I: Learning about Logic.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e055503460f94af1ac3/html5/thumbnails/82.jpg)
Regional Educational Laboratory at EDCrelnei.org
Generating Indicators: Identifying the Right Indicators
• One indicator to measure drop-out reduction = o Graduation rate
• Several indicators to measure parent involvement =
82
![Page 83: Regional Educational Laboratory at EDC relnei.org Logic Models to Support Program Design, Implementation, and Evaluation Session I: Learning about Logic.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e055503460f94af1ac3/html5/thumbnails/83.jpg)
Regional Educational Laboratory at EDCrelnei.org
Generating Indicators: Identifying the Right Indicators
• One indicator to measure drop-out reduction = o Graduation rate
• Several indicators to measure parent involvement =
o Attendance at school meetings
o Participation in parent–school organization
o Parent calls made to the school
o Attendance at school functions
83
![Page 84: Regional Educational Laboratory at EDC relnei.org Logic Models to Support Program Design, Implementation, and Evaluation Session I: Learning about Logic.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e055503460f94af1ac3/html5/thumbnails/84.jpg)
Regional Educational Laboratory at EDCrelnei.org
Generating Indicators: Quantitative and Qualitative
Quantitative: Outcomes focused and summative
Qualitative: Process focused and formative
84
![Page 85: Regional Educational Laboratory at EDC relnei.org Logic Models to Support Program Design, Implementation, and Evaluation Session I: Learning about Logic.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e055503460f94af1ac3/html5/thumbnails/85.jpg)
Regional Educational Laboratory at EDCrelnei.org
Generating Indicators: Quantitative and Qualitative
Indicators can be quantitative:
85
Evaluation Question Indicators
Did the program increase students’ interest in college?
Number of college applications completed
![Page 86: Regional Educational Laboratory at EDC relnei.org Logic Models to Support Program Design, Implementation, and Evaluation Session I: Learning about Logic.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e055503460f94af1ac3/html5/thumbnails/86.jpg)
Regional Educational Laboratory at EDCrelnei.org
Generating Indicators: Quantitative and Qualitative
Indicators can be quantitative or qualitative:
86
Evaluation Question Indicators
Did the program increase students’ interest in college?
Number of college applications completed (quantitative)
Guidance counselors report increased student interest (qualitative)
![Page 87: Regional Educational Laboratory at EDC relnei.org Logic Models to Support Program Design, Implementation, and Evaluation Session I: Learning about Logic.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e055503460f94af1ac3/html5/thumbnails/87.jpg)
Regional Educational Laboratory at EDCrelnei.org
Final Considerations about Indicators
Remember, indicators may:
•Match the outcomes of interest or questions asked
•Be singular for a given outcome or question
•Be quantitative or qualitative
•Vary based on the audience
87
![Page 88: Regional Educational Laboratory at EDC relnei.org Logic Models to Support Program Design, Implementation, and Evaluation Session I: Learning about Logic.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e055503460f94af1ac3/html5/thumbnails/88.jpg)
Regional Educational Laboratory at EDCrelnei.org
Building an Evaluation Design
Consider:
•Purpose of evaluation: Formative? Summative? Hybrid?
•Audience: Who is the audience for the evaluation? what do they want to know? How will the information be used?
•Capacity: Who will conduct the evaluation? What resources will be use? What is the time frame?
•Priority: What do you need to know?
88
![Page 89: Regional Educational Laboratory at EDC relnei.org Logic Models to Support Program Design, Implementation, and Evaluation Session I: Learning about Logic.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e055503460f94af1ac3/html5/thumbnails/89.jpg)
Regional Educational Laboratory at EDCrelnei.org
Data collection:
•What are pre-existing data sources (e.g., school attendance records, existing survey data)?
•What are existing instruments (e.g., existing surveys measuring same constructs)?
89
Building an Evaluation Design:Identifying Appropriate Data Sources
![Page 90: Regional Educational Laboratory at EDC relnei.org Logic Models to Support Program Design, Implementation, and Evaluation Session I: Learning about Logic.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e055503460f94af1ac3/html5/thumbnails/90.jpg)
Regional Educational Laboratory at EDCrelnei.org
Types of data:
•Administrative data
•Focus groups
•Interviews
•Observations
•Surveys
•Student test scores and grades
•Teacher assessments
•Other data sources
90
Building an Evaluation Design:Identifying Appropriate Data Sources
![Page 91: Regional Educational Laboratory at EDC relnei.org Logic Models to Support Program Design, Implementation, and Evaluation Session I: Learning about Logic.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e055503460f94af1ac3/html5/thumbnails/91.jpg)
Regional Educational Laboratory at EDCrelnei.org
Building an Evaluation Design:Identifying Appropriate Data Sources
Activity II.6: Consider Data Sources
91
Data Source Brainstorm
Consider your own program:
What relevant data sources do you already collect?
![Page 92: Regional Educational Laboratory at EDC relnei.org Logic Models to Support Program Design, Implementation, and Evaluation Session I: Learning about Logic.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e055503460f94af1ac3/html5/thumbnails/92.jpg)
Regional Educational Laboratory at EDCrelnei.org
Data Collection Framework
92
Building an Evaluation Design:Creating a Data Collection Framework
Strategyor
Activity
Outputor
Outcome
Formative Summative Indicator DataSourc
e
DataCollectionInstrument
WhenCollected
ByWhom
![Page 93: Regional Educational Laboratory at EDC relnei.org Logic Models to Support Program Design, Implementation, and Evaluation Session I: Learning about Logic.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e055503460f94af1ac3/html5/thumbnails/93.jpg)
Regional Educational Laboratory at EDCrelnei.org
Data Collection Framework Example: College-Ready Program
93
Building an Evaluation Design:Creating a Data Collection Framework
Strategyor
Activity
Outputor
Outcome
Formative Summative Indicator DataSource
DataCollectioninstrument
WhenCollected
ByWhom
Parent education strategy
High rate of parent attendance at workshops
✔ 70 percent of parents attend five out of six workshops
Admin-istrative data
Attendance log at workshops
At beginning of each session
Program director
Parent education strategy
Increased parent under-standing of college application process
✔ 85 percent of parents who attend more than four workshops report increased understand-ing
Parent feedback
Survey and interviews
Beginning of program, end of program
Program staff
![Page 94: Regional Educational Laboratory at EDC relnei.org Logic Models to Support Program Design, Implementation, and Evaluation Session I: Learning about Logic.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e055503460f94af1ac3/html5/thumbnails/94.jpg)
Regional Educational Laboratory at EDCrelnei.org
Building an Evaluation Design:Creating a Data Collection Framework Strategy or
ActivityOutput or Outcome
Indicator Data Sources
Data Collection Instrument
When Collected
By Whom
Short Term
Parent education strategy
Increased parent understanding of college application process
85 percent of parents who attend more than 4 workshops report increased understanding
Parent feedback
Survey and interviews
Beginning of program, end of program
Program staff
Long Term
Student education strategy
Increased student understanding of the college application process
80 percent of students who attend the workshops apply to college and get accepted to at least one college or university
Student feedback
Survey and interviews
End of program
Program staff
94
![Page 95: Regional Educational Laboratory at EDC relnei.org Logic Models to Support Program Design, Implementation, and Evaluation Session I: Learning about Logic.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e055503460f94af1ac3/html5/thumbnails/95.jpg)
Regional Educational Laboratory at EDCrelnei.org
Putting It All Together
If you have:
•Developed a logic model in collaboration with stakeholders
•Clarified who the audience is for the evaluation and how it will be used
•Identified and prioritized evaluation questions based on the logic model
•Selected indicators based on the outcomes of interest
•Identified data sources and a data collection plan
•Considered evaluation design, with awareness of resources, capacity, and timeline
Then…
95
![Page 96: Regional Educational Laboratory at EDC relnei.org Logic Models to Support Program Design, Implementation, and Evaluation Session I: Learning about Logic.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e055503460f94af1ac3/html5/thumbnails/96.jpg)
Regional Educational Laboratory at EDCrelnei.org
Putting It All Together
Create an evaluation prospectus:
•What are you going to evaluate?
•What is the purpose of the evaluation?
•How will the results of the evaluation be used?
•What specific questions will the evaluation answer?
•What data sources will be necessary to answer these questions?
•How will the data be analyzed (evaluation design)?
•What resources are needed to conduct this evaluation?
•What is the timeline for the evaluation?
•How will the results be shared or disseminated?
•Who will manage the evaluation?
96
![Page 97: Regional Educational Laboratory at EDC relnei.org Logic Models to Support Program Design, Implementation, and Evaluation Session I: Learning about Logic.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e055503460f94af1ac3/html5/thumbnails/97.jpg)
Regional Educational Laboratory at EDCrelnei.org
Putting It All Together
Timeline: Gantt chart
97
January February March April May June July
Develop survey
Select sample
Administer survey
Analyze survey
Compare data to indicators
Write up findings
![Page 98: Regional Educational Laboratory at EDC relnei.org Logic Models to Support Program Design, Implementation, and Evaluation Session I: Learning about Logic.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e055503460f94af1ac3/html5/thumbnails/98.jpg)
Regional Educational Laboratory at EDCrelnei.org
Review
• Logic models are a useful tool for program design, implementation, and evaluation.
• Planning for evaluation at the onset of program or policy development ensures an evaluation that is relevant and, potentially, more rigorous.
• Engaging stakeholders in the process of developing the logic model and evaluation encourages support and buy-in and increases authenticity.
98
![Page 99: Regional Educational Laboratory at EDC relnei.org Logic Models to Support Program Design, Implementation, and Evaluation Session I: Learning about Logic.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e055503460f94af1ac3/html5/thumbnails/99.jpg)
Regional Educational Laboratory at EDCrelnei.org
Next Steps
Your next steps…
What is one thing you’ve learned
or will take back with you to
your colleagues?
99
![Page 100: Regional Educational Laboratory at EDC relnei.org Logic Models to Support Program Design, Implementation, and Evaluation Session I: Learning about Logic.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e055503460f94af1ac3/html5/thumbnails/100.jpg)
Regional Educational Laboratory at EDCrelnei.org
Thank You!
• Contact Information:
For any questions: [Place name title and email here]
100