Merit and Needs-Based Scholarship Program (MNBSP) Evaluation ...
USING AN EVALUATION NEEDS ASSESSMENT TO UNPACK EVALUATION … · Evaluation Needs Assessment....
Transcript of USING AN EVALUATION NEEDS ASSESSMENT TO UNPACK EVALUATION … · Evaluation Needs Assessment....
![Page 1: USING AN EVALUATION NEEDS ASSESSMENT TO UNPACK EVALUATION … · Evaluation Needs Assessment. PROCESS Action plan Analyse, synthesise & report Collect data Plan & prepare Review &](https://reader033.fdocuments.in/reader033/viewer/2022053121/60a757bc3eed7c79616a0148/html5/thumbnails/1.jpg)
“CATCHING THE MEL BUG”
Kathryn Robb | Djirra
Mark Planigale | Lirata Ltd
Australian Evaluation Society Conference | 17 Sept 2019
USING AN EVALUATIONNEEDS ASSESSMENT TO
UNPACK EVALUATION CAPACITY
![Page 2: USING AN EVALUATION NEEDS ASSESSMENT TO UNPACK EVALUATION … · Evaluation Needs Assessment. PROCESS Action plan Analyse, synthesise & report Collect data Plan & prepare Review &](https://reader033.fdocuments.in/reader033/viewer/2022053121/60a757bc3eed7c79616a0148/html5/thumbnails/2.jpg)
![Page 3: USING AN EVALUATION NEEDS ASSESSMENT TO UNPACK EVALUATION … · Evaluation Needs Assessment. PROCESS Action plan Analyse, synthesise & report Collect data Plan & prepare Review &](https://reader033.fdocuments.in/reader033/viewer/2022053121/60a757bc3eed7c79616a0148/html5/thumbnails/3.jpg)
![Page 4: USING AN EVALUATION NEEDS ASSESSMENT TO UNPACK EVALUATION … · Evaluation Needs Assessment. PROCESS Action plan Analyse, synthesise & report Collect data Plan & prepare Review &](https://reader033.fdocuments.in/reader033/viewer/2022053121/60a757bc3eed7c79616a0148/html5/thumbnails/4.jpg)
![Page 5: USING AN EVALUATION NEEDS ASSESSMENT TO UNPACK EVALUATION … · Evaluation Needs Assessment. PROCESS Action plan Analyse, synthesise & report Collect data Plan & prepare Review &](https://reader033.fdocuments.in/reader033/viewer/2022053121/60a757bc3eed7c79616a0148/html5/thumbnails/5.jpg)
![Page 6: USING AN EVALUATION NEEDS ASSESSMENT TO UNPACK EVALUATION … · Evaluation Needs Assessment. PROCESS Action plan Analyse, synthesise & report Collect data Plan & prepare Review &](https://reader033.fdocuments.in/reader033/viewer/2022053121/60a757bc3eed7c79616a0148/html5/thumbnails/6.jpg)
Aboriginal
Family Violence
Legal Service
Djirra
Leadership &
MEL ManagerLirata Ltd
![Page 7: USING AN EVALUATION NEEDS ASSESSMENT TO UNPACK EVALUATION … · Evaluation Needs Assessment. PROCESS Action plan Analyse, synthesise & report Collect data Plan & prepare Review &](https://reader033.fdocuments.in/reader033/viewer/2022053121/60a757bc3eed7c79616a0148/html5/thumbnails/7.jpg)
EVALUATION CAPACITY BUILDING
“an intentional processthrough which we strengthen our ability
to design, implement and use effective MEL processesto help achieve our goals”
![Page 8: USING AN EVALUATION NEEDS ASSESSMENT TO UNPACK EVALUATION … · Evaluation Needs Assessment. PROCESS Action plan Analyse, synthesise & report Collect data Plan & prepare Review &](https://reader033.fdocuments.in/reader033/viewer/2022053121/60a757bc3eed7c79616a0148/html5/thumbnails/8.jpg)
EVALUATION CAPACITY BUILDING
• Strengths
• Weaknesses
• Opportunities
• Risks
skills
attitudesconfidence
knowledge
relationshipsleadership
resources
structures
processessystems
culture
CAPACITYEvaluation
Needs
Assessment
![Page 9: USING AN EVALUATION NEEDS ASSESSMENT TO UNPACK EVALUATION … · Evaluation Needs Assessment. PROCESS Action plan Analyse, synthesise & report Collect data Plan & prepare Review &](https://reader033.fdocuments.in/reader033/viewer/2022053121/60a757bc3eed7c79616a0148/html5/thumbnails/9.jpg)
PROCESS
Action
plan
Analyse,
synthesise
& report
Collect
data
Plan &
prepare
Review &
improve
![Page 10: USING AN EVALUATION NEEDS ASSESSMENT TO UNPACK EVALUATION … · Evaluation Needs Assessment. PROCESS Action plan Analyse, synthesise & report Collect data Plan & prepare Review &](https://reader033.fdocuments.in/reader033/viewer/2022053121/60a757bc3eed7c79616a0148/html5/thumbnails/10.jpg)
LENSES AND DOMAINS
Individual MEL capacity
Team & organisational MEL capacity
MEL life cycle
1. MEL knowledge, skills & experience
2. MEL attitudes & confidence
3. MEL focus
4. MEL systems
5. MEL leadership, roles & relationships
6. MEL resources
7. MEL culture, support & development
8. MEL design
9. Data collection
10. Analysis & reporting
11. Use & integration
![Page 11: USING AN EVALUATION NEEDS ASSESSMENT TO UNPACK EVALUATION … · Evaluation Needs Assessment. PROCESS Action plan Analyse, synthesise & report Collect data Plan & prepare Review &](https://reader033.fdocuments.in/reader033/viewer/2022053121/60a757bc3eed7c79616a0148/html5/thumbnails/11.jpg)
RATINGS
Beginning
Grappling
with
challenges
On the
wayEstablished
Strong &
sustainable
![Page 12: USING AN EVALUATION NEEDS ASSESSMENT TO UNPACK EVALUATION … · Evaluation Needs Assessment. PROCESS Action plan Analyse, synthesise & report Collect data Plan & prepare Review &](https://reader033.fdocuments.in/reader033/viewer/2022053121/60a757bc3eed7c79616a0148/html5/thumbnails/12.jpg)
COLLECT DATA
Interviews
Staff survey
Focus groups
Indicator mapping
Document review
Review of CLASS
data collection
![Page 13: USING AN EVALUATION NEEDS ASSESSMENT TO UNPACK EVALUATION … · Evaluation Needs Assessment. PROCESS Action plan Analyse, synthesise & report Collect data Plan & prepare Review &](https://reader033.fdocuments.in/reader033/viewer/2022053121/60a757bc3eed7c79616a0148/html5/thumbnails/13.jpg)
PARTICIPATION & OWNERSHIP
• Internally guided
• Externally facilitated
• Managers and workers
• Different teams
• Listen for ‘quiet voices’
• Cultural safety
• Awareness of power
![Page 14: USING AN EVALUATION NEEDS ASSESSMENT TO UNPACK EVALUATION … · Evaluation Needs Assessment. PROCESS Action plan Analyse, synthesise & report Collect data Plan & prepare Review &](https://reader033.fdocuments.in/reader033/viewer/2022053121/60a757bc3eed7c79616a0148/html5/thumbnails/14.jpg)
ANALYSE, SYNTHESISE
I would like to know
what contribution I
can have to M&E
and what is
expected of me
Don’t need to
waste time
and resources
on MEL
Have a meeting
‘yarn circle’ once a
week to discuss
success stories
and sharing
mistakes … as a
way we can
support each otherUnless we can
make this
efficient for
staff, it will fail
![Page 15: USING AN EVALUATION NEEDS ASSESSMENT TO UNPACK EVALUATION … · Evaluation Needs Assessment. PROCESS Action plan Analyse, synthesise & report Collect data Plan & prepare Review &](https://reader033.fdocuments.in/reader033/viewer/2022053121/60a757bc3eed7c79616a0148/html5/thumbnails/15.jpg)
![Page 16: USING AN EVALUATION NEEDS ASSESSMENT TO UNPACK EVALUATION … · Evaluation Needs Assessment. PROCESS Action plan Analyse, synthesise & report Collect data Plan & prepare Review &](https://reader033.fdocuments.in/reader033/viewer/2022053121/60a757bc3eed7c79616a0148/html5/thumbnails/16.jpg)
OUTCOMES – PLANNING ECB
• More strategic approach to evaluation capacity building activities
• Identify strengths and gaps
• Prioritise where to start
• Generate early ideas about MEL focus
![Page 17: USING AN EVALUATION NEEDS ASSESSMENT TO UNPACK EVALUATION … · Evaluation Needs Assessment. PROCESS Action plan Analyse, synthesise & report Collect data Plan & prepare Review &](https://reader033.fdocuments.in/reader033/viewer/2022053121/60a757bc3eed7c79616a0148/html5/thumbnails/17.jpg)
OUTCOMES – EVALUATING ECB
• More strategic approach to reviewing our evaluation capacity building journey
• A framework for understanding capacity
• A baseline
• A perspective from outside of the ECB process
![Page 18: USING AN EVALUATION NEEDS ASSESSMENT TO UNPACK EVALUATION … · Evaluation Needs Assessment. PROCESS Action plan Analyse, synthesise & report Collect data Plan & prepare Review &](https://reader033.fdocuments.in/reader033/viewer/2022053121/60a757bc3eed7c79616a0148/html5/thumbnails/18.jpg)
OUTCOMES – PARTNERSHIPS
• More strategic approach to partnering with stakeholders
• Understand the diversity of what people bring to the MEL journey
• “Flipping the script”
![Page 19: USING AN EVALUATION NEEDS ASSESSMENT TO UNPACK EVALUATION … · Evaluation Needs Assessment. PROCESS Action plan Analyse, synthesise & report Collect data Plan & prepare Review &](https://reader033.fdocuments.in/reader033/viewer/2022053121/60a757bc3eed7c79616a0148/html5/thumbnails/19.jpg)
MORE INFO
Available from Lirata (email Mark, or will be on website soon):
• Interview with Kathryn
• ENA framework
• ENA survey template to use and adapt
Mark Planigale [email protected] / +61 0429 136 596
Kathryn Robb [email protected] / +61 03 9244 3333
![Page 20: USING AN EVALUATION NEEDS ASSESSMENT TO UNPACK EVALUATION … · Evaluation Needs Assessment. PROCESS Action plan Analyse, synthesise & report Collect data Plan & prepare Review &](https://reader033.fdocuments.in/reader033/viewer/2022053121/60a757bc3eed7c79616a0148/html5/thumbnails/20.jpg)
QUESTIONS?
Catch me
if you can!