JPP Evaluation Debrief En
-
Upload
linh-duong -
Category
Documents
-
view
224 -
download
0
description
Transcript of JPP Evaluation Debrief En
Final External Evaluation of the Justice Partnership Programme
Debriefing Workshop Presentation
Prepared by Finn Hansen & Pierre Robert | 5 June 2015
JPP Evaluation Debriefing Seminar – Hanoi – June 2015 2
“The contents of this publication are the sole responsibility of the authors and can in no way be taken
to reflect the views of the European Union.”
JPP Evaluation Debriefing Seminar – Hanoi – June 2015 3
Seminar Objectives
Why this seminar?
• To present the preliminary findings, conclusions and recommendations of the JPP evaluation team.
• To discuss the findings and conclusions.
• To assess whether and how the JPP could have performed better.
• To discuss lessons learned for future cooperation
JPP Evaluation Debriefing Seminar – Hanoi – June 2015 4
Proposed Agenda
• 8:40: Presentation of the evaluation
• 10:00: Break
• 10:20: Discussion
• 11:20: Conclusions and next steps
• 11:30: Close
• Lunch
JPP Evaluation Debriefing Seminar – Hanoi – June 2015 5
The Programme
• JPP: 2010-2015, funding from Denmark, Sweden and EU – €18.7m (programme document figure)
Objectives:• Justice sector state institutions strengthened in their efforts to implement
judicial reforms
• Empowerment of a self-managing Vietnam Bar Federation (VBF) and lawyers supported
• Capacity of Non-Government Organisations (NGOs) to contribute to awareness of rights, access to justice and judicial reform enhanced.
JPP Evaluation Debriefing Seminar – Hanoi – June 2015 6
3 Programme Components
• Component 1: support to Ministry of Justice (MoJ), Supreme People’s Court (SPC) and Supreme People’s Procuracy (SPP) to improve procedures, strengthen institutions and develop staff capacity.
• Component 2: support to the VBF through the Canadian Bar Association to enhance organisational capacity and provide training to lawyers.
• Component 3: provide support to NGOs working on justice issues, essentially through the Justice Initiative Facility Fund (JIFF), including advice and training to grantee organisations.
JPP Evaluation Debriefing Seminar – Hanoi – June 2015 7
The evaluation
• Objective: to assess the JPP’s performance with regards to its key results areas (KRAs). To draw lessons for use in the implementation of the forthcoming EU JULE programme, due to start later in 2015.
• Methodology:– Study of documentation– 2 visits by evaluation team to Vietnam: April and May/June 2015– Debriefing– Evaluation draft and final report
JPP Evaluation Debriefing Seminar – Hanoi – June 2015 8
Evaluation criteria• Relevance:
– Correctly identified needs?– Appropriate response to those needs?
• Effectiveness:– Planned activities implemented?– Achievement of expected results?
• Efficiency:– Good value for money?– Management quality?
• Sustainability:– Established new working methods, spread new knowledge?– Will results continue after the programme ends?
• Impact:– Did the programme change attitudes?– Benefits for citizens?
JPP Evaluation Debriefing Seminar – Hanoi – June 2015 9
FINDINGS
JPP Evaluation Debriefing Seminar – Hanoi – June 2015 10
Component 3
• Supported NGO action on rights awareness, legal advice, research, dialogue.
• JIFF secretariat, grant committee established
• 78 grants in 47 provinces (10 grants for national activities)
• Innovative approach, new in the justice sector
• Examples:– Centre for Consultancy on Law and Policy in Heath on HIV/AIDS– Law Development Journal– ACDC: Legal advice for people with disabilities– Justice for « cancer villages »– CSAGA: mediation againt domestic violence– Hoa Binh Association and Legal Aid Centre: community consultation group
JPP Evaluation Debriefing Seminar – Hanoi – June 2015 11
Component 3
• Relevance:– Enhanced NGO contribution and participation in justice issues– KRAs appropriately selected– Contribution to dialogue on the performance of the justice system
• Effectiveness:– Increase in number of people involved in awareness raising activities– Increase in provision of legal advice and number of providers– More access to legal advice in remote areas– Advice to women, minority communities– Research and publications on judicial reform (including gender aspects)– Dialogue at central and local levels on judicial reform
JPP Evaluation Debriefing Seminar – Hanoi – June 2015 12
Component 3• Efficiency
– Good use of funds, efforts to keep project costs down– Good ratio of outputs to costs– Some synergies with other donors– 30% of budget (about €3m total) for JIFF management, large but needed
• Sustainability– Project level: short-term grants, little capacity to continue– Project management skills improved, credibility of grantees enhanced
• Impact– Some changes in attitude (e.g. less prejudices against HIV+, more
confidence in the law)– Some acceptance of NGOs as legitimate interlocutors for local dialogue
JPP Evaluation Debriefing Seminar – Hanoi – June 2015 13
Component 2
• VBF established 2009
• Additional to local-level Bar Associations
• Mandated by 2006 Law on Lawyers
• CBA conducted organisational assessment, training needs assessment
• Assessment of administrative systems, establishment of management processes
• Training for Board members, engagement with local BAs
JPP Evaluation Debriefing Seminar – Hanoi – June 2015 14
Component 2
• Relevance– Very clear need for organisational development– CBA was appropriate support: credible, good understanding of VBF– Weakness: overambitious goals in the time given
• Effectiveness– Development of operating rules: governance, administrative systems, staff
training– Support to regulatory framework: Code of Conduct and Ethics,
dissemination and training for lawyers (but: weak internal communications)– Support to VBF to speak on behalf of lawyers: contribution to legislation
(issuance of lawyer‘s card; Chapter VII of Criminal Procedure Code) – Professional training: support to professional development of lawyers,
development of training materials (but weak training capacity)
JPP Evaluation Debriefing Seminar – Hanoi – June 2015 15
Component 2
• Efficiency– Good management support by CBA– Improved management capacity of VBF staff– Need for further assistance, broader involvement of lawyers in VBF
management– Weak communications mechanisms, dificulties with membership fee
collection
• Sustainability– Important step towards institutionalisation, recognition of the profession at
national level as integral to the judicial sector.– No achievement of financial sustainability yet, need to diversify/reinforce
funding.
• Impact– Gradual change in perception of the role of VBF– Advocacy role on behalf of lawyers is still at early stage
JPP Evaluation Debriefing Seminar – Hanoi – June 2015 16
Component 1
• Worked with MoJ on procedures, institutions and capacity:– Legal aid, civil procedure, Law on Lawyers– Socialised bailiff system pilot– Judicial records– Capacity building (legal aid, lawyers training, support staff)
• Worked with SPC on legislation and management:– Civil Procedure Code– Administrative Procedure– Case file management (court dossier)– Publication of judgements– Training, seminars on international practices
• Worked with SPP on legislation and management:– Criminal Procedure Code– Organisation of the Procuracy– Capacity development, training, – Seminars on international practices
JPP Evaluation Debriefing Seminar – Hanoi – June 2015 17
Component 1
• Relevance– The KRAs are very relevant– But formulation of purpose was too broad (however, may be necessary for
a demand-led programme)– Good level of implication of institutions in programme design– Aligned with the priorities of the Judicial Reform Strategy
• Effectiveness– In all, 12 of 19 KRAs have been achieved, 6 are under completion, 1 is
pending (Criminal Procedure Code).– MoJ: Legal Aid strategy completed (to be approved), Law on Lawyers
implementation underway; capacity of provincial legal aid centres improved (from a low base)
– SPC: considerable JPP input in at least 3 KRAs (e.g. case file management (court dossier), publication of judgements); significant input in Criminal Procedure Code revision (also input from VBF supported by JPP); capacity development plan achieved.
JPP Evaluation Debriefing Seminar – Hanoi – June 2015 18
Component 1
• Effectiveness, continued– SPP: revision of CPC, including support for alignment of key provisions
with international standards (right to silence, to defence counsel, etc); Law on the organisation of the procuracy; capacity building of SPP‘s controlling function (e.g. control of places of detention).
– Key concern: training not always adapted to Vietnam‘s situation.
• Efficiency– Overall, critical mass reached in the of TA and international expertise
provided.– Committed, highly skilled and experienced TA team added value to
programme– But: delays in approvals of some activities, issue related to use of national
civil servants as experts.
JPP Evaluation Debriefing Seminar – Hanoi – June 2015 19
Component 1
• Sustainability– New legislation by definition continues beyond programme period.– Significant improvement to skills (senior staff refer to improvement in
quality of work in their department, e.g. on legal drafting).– New practices initiated/developed (e.g. publication of judgements, use of
precedents) with substantial technical input from JPP.– But: issue of financing for the future (e.g. legal aid, bailiff system)
• Impact– Enhanced awareness of international best practices in a range of fields of
relevance to Vietnam– Improved practices in legal drafting/consultation on new (and future?)
drafts.
JPP Evaluation Debriefing Seminar – Hanoi – June 2015 20
Conclusions
JPP Evaluation Debriefing Seminar – Hanoi – June 2015 21
Programme-level
• The programme was relevant:– Good understanding of the needs– Good design of the response of the 3 components
• Component 1 demand-led• Component 2 helped organisational development• Component 3 sound design, pro-active support to NGOs
• Programme was effective:– Good level of achievement of activities– Most anticipated results were met
• Programme was efficient:– Appropriate use of funds– Good programme management
• Satisfactory levels of sustainability and impact
JPP Evaluation Debriefing Seminar – Hanoi – June 2015 22
Key lessons learned
• Overly ambitious or unclear in objectives and some outcomes
• Original design failed to provide for linkages among the components
• Lack of appropriate programme-level monitoring and evaluation mechanism
• Overly complex programme management structure (PC, 5 PMUs, 3 TA teams)
• Comp. 3: some grants were too short term• Comp. 2: ended too early for VBF to be solidly established• Comp. 2: need for a VBF organisational development plan• Comp. 1: insufficient linkages fostered among 3 institutions• Comp. 1: training somewhat ad hoc, lacked predictability over 12-18 months
JPP Evaluation Debriefing Seminar – Hanoi – June 2015 23
Recommendations
• To EU, donors:– Continue to support the justice sector development. MoJ, SPP, SPC, VBF– Continue to support NGOs working on justice issues
• Project vs. “core“ funding• Thematic priorities?
– Ensure future programme management has stronger Monitoring and Evaluation (M&E) and synergies across components• The beneficiary institutions should be involved in the selection of indicators, in
agreement with donors
– Address need for more predictable, more widespread training (may require different planning processes within Component 1 institutions)
JPP Evaluation Debriefing Seminar – Hanoi – June 2015 24
Recommendations
• To the MoJ, SPC and SPP: – The three institutions should reduce the number of areas of work in which
they seek assistance from future cooperation, instead prioritising on-going support in a narrower range of fields.
– The three institutions should attempt to develop 12-months rolling training plans, to enhance the predictability of training sessions. They should consider whether to repeat some training sessions on an annual basis.
JPP Evaluation Debriefing Seminar – Hanoi – June 2015 25
Recommendations• To the VBF:
– The VBF should reinforce internal communications, to enhance lawyers’ awareness of its actions and interest in supporting them.
– The VBF should prepare a medium- to long-term organisational development action plan, which could form a basis for diversification of funding.
• To JIFF:– The duration of NGO grants should be longer, and monitoring should
include more focus on outcome at beneficiary level; – MoJ concerns regarding administration and liquidation should be attended
to, by providing the necessary support.– NGO grantees should be encouraged to establish thematic groups at
national level to share information and contribute to judicial reform.
JPP Evaluation Debriefing Seminar – Hanoi – June 2015 26
Discussion• Two questions:
1. Do you agree with the evaluators’ assessment?
2. How could the programme have performed better?
Thank you