Transcript of E-101 Thursday Section 1 T HURSDAY, 1 ST NOV., 2012 11 – 12.30 P. M.
- Slide 1
- E-101 Thursday Section 1 T HURSDAY, 1 ST NOV., 2012 11 12.30 P.
M.
- Slide 2
- Roadmap for today Housekeeping Where are we? What are the
goals? Application Feedback sticky notes 2
- Slide 3
- Housekeeping 3 Paper II is coming up Paper III Dec. 20 th we
will start looking at this more closely once Paper II is back
- Slide 4
- 4
- Slide 5
- Where are we- Assessing the performance of a system Policy and
program evaluation Curriculum, Standards & Assessment 5
- Slide 6
- M&E - 6 One of the most impt. aspects of this course - make
sure you walk away with it Resources International Initiative for
Impact Evaluation J-PAL UNESCO IIEP UNICEF Innocenti Research
Center UNICEF Evaluation Database World Bank Evaluation Database
Other classes Haiyan Huas M & E class Tom Kahne - Program
Evaluation
- Slide 7
- Weiss Reading 7 What does she have to say? Summary? What are
the tensions that might arise in the evaluation process?
- Slide 8
- Monitoring & Evaluation 8 There is a difference between M
and E! Monitoring: The gathering of evidence to show what progress
has been made in the implementation of programs. Focuses on inputs
and outputs. Evaluation: Measuring changes in outcomes and
evaluating the impact of specific interventions on those
outcomes.
- Slide 9
- Monitoring & Evaluation 9
- Slide 10
- 10
- Slide 11
- 11
- Slide 12
- Reflections - 12 What are the tensions b/w an insider &
outsider role as an evaluator? If time is money, how does one
implement process? What are the ways in which impact might be
considered? How might cultural differences impact these? Creating
local capacity for communities to monitor their own programs &
hold states accountable for the delivery of basic services?
Dissemination to the beneficiaries?!
- Slide 13
- Implementation Issues? 13 Political Economy Policy Context
Methods
- Slide 14
- Political Economy 14 What is the policy purpose? Test
innovations & defend budget Allocate budget to certain programs
Pressure to demonstrate aid effectiveness and scale up Answer
electorate Reaching Targets
- Slide 15
- Policy Context 15 What questions need answering? Who are the
stakeholders who want answers? Do they also have the same amount of
clout? What policy questions need answers? How much of a change
would determine success? What does the government really need, and
what will it use?
- Slide 16
- Methods 16 Operationalizing the outcome Process/outcome
indicators Channels of impact theory of change Underlying
assumptions [log frame] Who has access to the program? Are they the
same as the intended beneficiaries? Data collection time vs. cost
Analysis Dissemination
- Slide 17
- Things to think about 17 What ? Type of information and data to
be consolidated How? Procedures and approaches for data collections
and analysis Why? Why are we doing this does it support the
program/policy When? Freq. of data collection, reporting Who?
Responsibilities and capacities of focal points and resource
persons
- Slide 18
- Sector Analysis: What is it? Sector Review: summary of the
state of the sector Sector Assessment: success of the sector in
meetings certain goals Sector Analysis: (most comprehensive)
includes review and assessment (above), and also describes sector
needs, constraints, and opportunities for improvement. (Kremmerer,
1994)
- Slide 19
- Sector Analysis: What is it? Tries to answer questions relating
to: External efficiency Internal efficiency Access & equity
Administration & supervision Costs & financing (Pigozzi
& Cieutat, 1998)
- Slide 20
- Sector Analysis: What is it? The contents of a sector analysis
may include: 1. Synthesis 2. Economic & Financial Analysis 3.
History, Structure & Management Capacity 4. Analysis by
Sub-Sectors: 5. Special Studies 6. A list of background documents
(Pigozzi & Cieutat,1988)
- Slide 21
- Sector Analysis: Steps I. Initiation II. Formation of the
National Steering Committee III. Determination of the scope of the
analysis IV. Formation of the subsector teams V. Formation of the
technical analysis team VI. Instrumentation and collection of data
VII. Preliminary analysis of data VIII. Discussion of preliminary
findings IX. Collection of any additional data X. Formulation of
draft recommendations XI. Discussion of draft recommendations XII.
Revision and prioritization of recommendations. (Kremmerer,
1994)
- Slide 22
- Opposition sectorsSupport sectorsOpposition sectors External
actors Sector positi on Anti-system Legal Opposition Ideological
support Core support Ideological support Legal Opposition Anti-
system The Government Social sectors Political parties Pressure
groups Political Mapping: An Example
- Slide 23
- Slide 24
- Stakeholder Map: An Example
- Slide 25
- Program Theory Program theoryrefers to the mechanisms that
mediate between the delivery (and receipt) of the program and the
emergence of the outcome of interest. The operative mechanism of
change isnt the program activities per se but the response that the
activities generate. - Carol Weiss, Evaluation, p. 57 You will be
reading a section of this book for next week.
- Slide 26
- Activity: Program Theory For each program, there may be
multiple pathways to the intended outcome. Source: Weiss, C.
(1997). Theory-based evaluation: Past, present, and future. New
Directions for Evaluation, (76), 41-55, p. 42.
- Slide 27
- From Program Theory to Implementation to Evaluation Logical
Frameworks can help you to organize: Design Implementation
Evaluation Developed by USAID in the 1960s Vary slightly, but
always include: 4 x 4 Narrative Summary, Indicators, Means of
Verification, and Assumptions
- Slide 28
- Log Frames 28 In terms of relevance, practice could be the
development of common understandings gained via an examination of
assumptions and creation of a common language for evaluation -
Garaway p.p. 718 Log frame as tool to do examine Relevance: what,
where, who Effectiveness: Process & implementation Efficiency:
Making the best use of resources Impact: Support the program
purpose
- Slide 29
- How do you develop a logic model? Linear process from beginning
to end Move around (iterative) Backwards (identify results then
determine the resources/inputs and activities that are required)
Can be described as If ___________ then ____________. If we have X
resources and do Y activities, then we will have Z Outputs, W
Outcomes and T Impact.
- Slide 30
- Putting It All Together Impact Outcome Output Impact Outcome
Output
- Slide 31
- How to annoy your family with what you learned at HGSE Your
Planned Work Your Intended Results Resources/ Inputs
ActivitiesOutputsOutcomes Impact Logic Model of a Family Trip
Holiday flight schedules Family schedules Holiday weather Your
exam/ paper due dates Create family schedule Get holiday flight
info Get tickets Arrange ground transport Tickets for all family
members Frequent flyer miles used Money saved Ground transport
Family members enjoy vacation Continued good family relations
- Slide 32
- Slide 33
- Logical Frameworks Project Structure Objectively Verifiable
Indicators Means of Verification Important Assumptions Goal:
Purpose: Outputs: Activities: (Inputs)
- Slide 34
- Logical Frameworks Project Structure Objectively Verifiable
Indicators Means of Verification Important Assumptions Goal Purpose
Outputs Activities (Inputs) IF AND THEN
- Slide 35
- Logical Frameworks Project Structure Objectively Verifiable
Indicators Means of Verification Important Assumptions Goal:
Purpose: To increase the number of primary school graduates.
Outputs: Activities: (Inputs)
- Slide 36
- Logical Frameworks Project Structure Objectively Verifiable
Indicators Means of Verification Important Assumptions Goal:
Develop human capacity in XX country. Purpose: To increase the
number of primary school graduates. Outputs: Activities:
(Inputs)
- Slide 37
- Logical Frameworks Project Structure Objectively Verifiable
Indicators Means of Verification Important Assumptions Goal:
Develop human capacity in Somalia. Purpose: To increase the number
of primary school graduates. Outputs: By abolishing school fees,
expect to see more children enrolling in primary Activities:
(Inputs)
- Slide 38
- Logical Frameworks Project Structure Objectively Verifiable
Indicators Means of Verification Important Assumptions Goal:
Develop human capacity in Somalia. Purpose: To increase the number
of primary school graduates. Outputs: By abolishing school fees,
expect to see more children enrolling in primary Activities:
Abolishing school fees.
- Slide 39
- Logical Frameworks Project Structure Objectively Verifiable
Indicators Means of Verification Important Assumptions Goal:
Develop human capacity in Somalia. Purpose: To increase the number
of primary school graduates. Outputs: By abolishing school fees,
expect to see more children enrolling in primary Activities:
Abolishing school fees. All primary schools no longer requires
fees.
- Slide 40
- Logical Frameworks Project Structure Objectively Verifiable
Indicators Means of Verification Important Assumptions Goal:
Develop human capacity in Somalia. Purpose: To increase the number
of primary school graduates. Outputs: By abolishing school fees,
expect to see more children enrolling in primary Activities:
Abolishing school fees. All primary schools no longer requires
fees. Poll families to ask what fees exist.
- Slide 41
- Logical Frameworks Project Structure Objectively Verifiable
Indicators Means of Verification Important Assumptions Goal:
Develop human capacity in Somalia. Purpose: To increase the number
of primary school graduates. Outputs: By abolishing school fees,
expect to see more children enrolling in primary Activities:
Abolishing school fees. All primary schools no longer requires
fees. Poll families to ask what fees exist. Primary can be funded
without fees.