What are linear statistical models?

E-101 Thursday Section1Thursday, 1st nov. , 201211 12.30 p.m.Unit 0: Shopping Class Slide #1Roadmap for todayHousekeepingWhere are we?What are the goals?ApplicationFeedback sticky notes

22Housekeeping3Paper II is coming up Paper III Dec. 20th we will start looking at this more closely once Paper II is back

34

4Where are we-5

M&E -6One of the most impt. aspects of this course - make sure you walk away with itResources International Initiative for Impact EvaluationJ-PALUNESCO IIEPUNICEF Innocenti Research CenterUNICEF Evaluation Database World Bank Evaluation DatabaseOther classes Haiyan Huas M & E classTom Kahne - Program Evaluation

6Weiss Reading7What does she have to say? Summary?What are the tensions that might arise in the evaluation process?

7Monitoring & Evaluation 8There is a difference between M and E!

Monitoring:The gathering of evidence to show what progress has been made in the implementation of programs. Focuses on inputs and outputs.

Evaluation:Measuring changes in outcomes and evaluating the impact of specific interventions on those outcomes. 8Monitoring & Evaluation 9

910

1011

11Reflections - 12What are the tensions b/w an insider & outsider role as an evaluator? If time is money, how does one implement process?What are the ways in which impact might be considered? How might cultural differences impact these?Creating local capacity for communities to monitor their own programs & hold states accountable for the delivery of basic services?Dissemination to the beneficiaries?!

12Implementation Issues?13Political EconomyPolicy ContextMethods

13Political Economy 14What is the policy purpose? Test innovations & defend budgetAllocate budget to certain programsPressure to demonstrate aid effectiveness and scale upAnswer electorateReaching Targets

14Policy Context15What questions need answering?Who are the stakeholders who want answers? Do they also have the same amount of clout?What policy questions need answers?How much of a change would determine success?What does the government really need, and what will it use?

15Methods16Operationalizing the outcomeProcess/outcome indicatorsChannels of impact theory of changeUnderlying assumptions [log frame] Who has access to the program? Are they the same as the intended beneficiaries?Data collection time vs. costAnalysis Dissemination

16Things to think about 17What ?Type of information and data to be consolidated

How?Procedures and approaches for data collections and analysis

Why?Why are we doing this does it support the program/policy

When? Freq. of data collection, reporting

Who? Responsibilities and capacities of focal points and resource persons17Sector Analysis: What is it?Sector Review: summary of the state of the sectorSector Assessment: success of the sector in meetings certain goalsSector Analysis: (most comprehensive) includes review and assessment (above), and also describes sector needs, constraints, and opportunities for improvement.

(Kremmerer, 1994)

Sector Analysis: What is it?Tries to answer questions relating to:External efficiencyInternal efficiencyAccess & equityAdministration & supervisionCosts & financing

(Pigozzi & Cieutat, 1998)Sector Analysis: What is it?The contents of a sector analysis may include:SynthesisEconomic & Financial AnalysisHistory, Structure & Management CapacityAnalysis by Sub-Sectors:Special StudiesA list of background documents

(Pigozzi & Cieutat,1988)Sector Analysis: StepsInitiationFormation of the National Steering CommitteeDetermination of the scope of the analysisFormation of the subsector teamsFormation of the technical analysis teamInstrumentation and collection of dataPreliminary analysis of dataDiscussion of preliminary findingsCollection of any additional dataFormulation of draft recommendationsDiscussion of draft recommendationsRevision and prioritization of recommendations.(Kremmerer, 1994)

What does this remind you of? Bardachs Eightfold path? Not so different, except that the scope is larger, and this framework places greater emphasis on involving stakeholders and identifying key personnel.21Opposition sectorsSupport sectorsOpposition sectorsExternal actorsSectorpositionAnti-systemLegal OppositionIdeological supportCore supportIdeological supportLegal OppositionAnti-systemThe GovernmentSocial sectorsPolitical partiesPressure groupsPolitical Mapping: An ExamplePolitical Mapping: An Example

Stakeholder Map: An Example

Program TheoryProgram theoryrefers to the mechanisms that mediate between the delivery (and receipt) of the program and the emergence of the outcome of interest. The operative mechanism of change isnt the program activities per se but the response that the activities generate.

- Carol Weiss, Evaluation, p. 57

You will be reading a section of this book for next week.Activity: Program TheoryFor each program, there may be multiple pathways to the intended outcome.

Source:Weiss, C. (1997). Theory-based evaluation: Past, present, and future. New Directions for Evaluation, (76), 41-55, p. 42.

From Program Theory to Implementation to EvaluationLogical Frameworks can help you to organize:DesignImplementationEvaluationDeveloped by USAID in the 1960sVary slightly, but always include: 4 x 4 Narrative Summary, Indicators, Means of Verification, and Assumptions

Log Frames28In terms of relevance, practice could be the development of common understandings gained via an examination of assumptions and creation of a common language for evaluation - Garaway p.p. 718

Log frame as tool to do examine Relevance: what, where, who Effectiveness: Process & implementationEfficiency: Making the best use of resourcesImpact: Support the program purpose

28How do you develop a logic model?Linear process from beginning to endMove around (iterative)Backwards (identify results then determine the resources/inputs and activities that are required)

Can be described as If ___________ then ____________.

If we have X resources and do Y activities, then we will have

Z Outputs, W Outcomes and T Impact.Putting It All Together

ImpactOutcomeOutcomeOutcomeOutputOutputOutputOutputOutputOutput?????????????ImpactOutcomeOutputOutputOutput????How to annoy your family with what you learned at HGSEYour Planned Work

Your Intended ResultsResources/ InputsActivitiesOutputsOutcomesImpactLogic Model of a Family TripHoliday flight schedulesFamily schedulesHoliday weatherYour exam/ paper due datesCreate family scheduleGet holiday flight infoGet ticketsArrange ground transportTickets for all family membersFrequent flyer miles usedMoney savedGround transportFamily members enjoy vacationContinued good family relations

CAS = Country Assistance StrategyD.O. = Development Objective32Logical FrameworksProject StructureObjectively Verifiable IndicatorsMeans of VerificationImportant AssumptionsGoal:Purpose:Outputs:Activities:(Inputs)This is just another form of a log frame. Essentially doing the same thing Unit 0: Shopping Class Slide #33Logical FrameworksProject StructureObjectively Verifiable IndicatorsMeans of VerificationImportant AssumptionsGoalPurposeOutputsActivities(Inputs)IFIFIFANDANDANDTHENTHENTHENLogical FrameworksProject StructureObjectively Verifiable IndicatorsMeans of VerificationImportant AssumptionsGoal:Purpose: To increase the number of primary school graduates.Outputs:Activities:(Inputs)Logical FrameworksProject StructureObjectively Verifiable IndicatorsMeans of VerificationImportant AssumptionsGoal: Develop human capacity in XX country.Purpose: To increase the number of primary school graduates.Outputs: Activities:(Inputs)Logical FrameworksProject StructureObjectively Verifiable IndicatorsMeans of VerificationImportant AssumptionsGoal: Develop human capacity in Somalia.Purpose: To increase the number of primary school graduates.Outputs: By abolishing school fees, expect to see more children enrolling in primaryActivities:(Inputs)Logical FrameworksProject StructureObjectively Verifiable IndicatorsMeans of VerificationImportant AssumptionsGoal: Develop human capacity in Somalia.Purpose: To increase the number of primary school graduates.Outputs: By abolishing school fees, expect to see more children enrolling in primaryActivities: Abolishing school fees.Logical FrameworksProject StructureObjectively Verifiable IndicatorsMeans of VerificationImportant AssumptionsGoal: Develop human capacity in Somalia.Purpose: To increase the number of primary school graduates.Outputs: By abolishing school fees, expect to see more children enrolling in primaryActivities: Abolishing school fees.All primary schools no longer requires fees.Logical FrameworksProject StructureObjectively Verifiable IndicatorsMeans of VerificationImportant AssumptionsGoal: Develop human capacity in Somalia.Purpose: To increase the number of primary school graduates.Outputs: By abolishing school fees, expect to see more children enrolling in primaryActivities: Abolishing school fees.All primary schools no longer requires fees.Poll families to ask what fees exist.Logical FrameworksProject StructureObjectively Verifiable IndicatorsMeans of VerificationImportant AssumptionsGoal: Develop human capacity in Somalia.Purpose: To increase the number of primary school graduates.Outputs: By abolishing school fees, expect to see more children enrolling in primaryActivities: Abolishing school fees.All primary schools no longer requires fees.Poll families to ask what fees exist.Primary can be funded without fees.