MID-TERM EVALUATION OF THEMID-TERM EVALUATION OF THEMID-TERM EVALUATION OF THE“TRADE DEVELOPMENT CO-OPERATION AGREEMENT FACILITY (TDCA-F)”“TRADE DEVELOPMENT CO-OPERATION AGREEMENT FACILITY (TDCA-F)”“TRADE DEVELOPMENT CO-OPERATION AGREEMENT FACILITY (TDCA-F)”

Letter of Contract N°2013/330643 – Version 1Letter of Contract N°2013/330643 – Version 1Letter of Contract N°2013/330643 – Version 1

FINAL REPORTFINAL REPORTFINAL REPORT

Prepared by Flor E. HEALY & Julia du PISANIPrepared by Flor E. HEALY & Julia du PISANIPrepared by Flor E. HEALY & Julia du PISANIMarch 2014March 2014March 2014

EXECUTIVE SUMMARYEXECUTIVE SUMMARYEXECUTIVE SUMMARY

THE EUROPEAN UNION’S THE EUROPEAN UNION’S THE EUROPEAN UNION’S DEVELOPMENT CO-OPERATION INSTRUMENT (DCI)DEVELOPMENT CO-OPERATION INSTRUMENT (DCI)DEVELOPMENT CO-OPERATION INSTRUMENT (DCI)

SA-EU Strategic PartnershipThe Dialogue Facility

The project is fi nanced by the European Union

The project is implemented by IBF International Consulting in collaboration with BAa Consultors

This evaluation is supported and guided by the European Commission and presented by the IBF-led Consortium.The report does not necessarily refl ect the views and opinions of the European Commission

0

EXECUTIVE SUMMARY

1. BACKGROUND

1.1 Policy Background

1. South Africa and the EU signed a Trade, Development and Co-operation Agreement (TDCA) in 1999, which was provisional until it formally entered into force in May 2004 and was amended in 2009. The TDCA is the main legal basis for relations and co-operation between the EU and South Africa. The TDCA requires the South African and European Union partners to widen and deepen their co-operation. The Agreement has four pillars: (i) political dialogue; (ii) trade; (iii) economic and other issues; and (iv) development co-operation.

2. In addition in 2007, South Africa and the EU agreed to establish a Strategic Partnership and in May 2007, adopted a Joint Action Plan for achievement of this partnership. The Joint Action Plan has two strands: (i) enhanced high-level political dialogue at summit level; and (ii) stronger co-operation and structured policy dialogues in a number of economic, social and other areas. Political dialogue has been elevated to the highest level. The most recent EU-South Africa Summit took place in Pretoria in July 2013.

1

1.2 The TDCA Facility (or Dialogue Facility)

1. The overall objective of the TDCA Facility (‘Dialogue Facility’) is ‘to contribute to strengthening relations between the EU and South Africa’. The project purpose is ‘to facilitate the implementation of priority aspects of the Strategic Partnership and TDCA, and to raise awareness on the special relationship between the EU and South Africa’ (by providing seed money to mainly encourage new areas of co-operation). The Dialogue Facility has three key result areas, as follows:

i. Key Result Area 1: Improved sectoral policy dialogue and co-operation between the EU and South Africa;

ii. Key Result Area 2: Increased institutional capacity to help implement the areas of co-operation of the TDCA and the Strategic Partnership Action Plan;

iii. Key Result Area 3: Increased awareness and understanding of the Strategic Partnership and TDCA amongst the target group and indirect benefi ciaries.

2. In order to achieve the three result areas, the TDCA-F facilitates support through:

i. A Dialogue Support Facility: aimed at supporting dialogue initiatives (under Key Result Area 1) in the thematic areas that fall under the TDCA and the Joint Action Plan as well as other areas agreed by South Africa and the EU to be of mutual interest. Services provided by the facility include technical assistance, logistical support for conferences, workshops and events to cover venue, fl ights, etc.;

ii. A Rapid Response Mechanism: aimed at supporting niche initiatives to strengthen dialogue between South Africa and the EU under Key Result Area 1. This mechanism included budget for Short-term Expertise as well as Logistical Costs;

iii. A Grant Fund: a Call for Proposals was issued seeking to strengthen research in policy issues and to strengthen public-private partnership in this area;

iv. Additional Technical Support Activities: particularly in Key Result Areas 2 and 3 were implemented through a series of service contracts and framework contracts.

2

1.3 Mid-Term Evaluation

1. The Mid-Term Evaluation was focused on evaluating the Dialogue Facility only and did not concern itself with the broader political aspects of the TDCA (the Agreement) and the Strategic Partnership, other than to recognise the context within which the Dialogue Facility operates in support of the overall Agreement (i.e. the TDCA, the Strategic Partnership and the Joint Action Plan)1.

2. The main objectives of the evaluation were to provide the EU (EEAS and DEVCO2), the Government of South Africa and, where appropriate, the wider public with:

i. An overall independent assessment of the past performance of the intervention, under the Trade Development Co-Operation Agreement Facility (TDCA-F), paying particular attention to the results of the project against its objectives;

ii. Key lessons learned and recommendations in order to improve current and future actions.

3. The evaluation covered all the components of the TDCA-F implemented during the period February 2009 to September 2013, with a specifi c focus on relevance, eff ectiveness and effi ciency aspects. The lesson learning dimension of the evaluation is essential in determining whether a follow-up to the TDCA-Facility should be considered at the end of the current project and may inform the nature of such a follow-up programme.

1 A distinction is drawn between the Dialogue Facility (TDCA-F), which is a technical support facility to strengthen relations and policy engagement between South Africa and the EU at sectoral level, and the overall political dialogue under the TDCA, the Strategic Partnership.2 EC Directorate General: European External Action Service (EEAS) and EC Directorate General: Development Co-Operation (DEVCO).

3

EXECUTIVE SUMMARY

2. OVERALL ASSESSMENT

1. Overall, the Dialogue Facility (or TDCA-F) is a highly relevant and an innovative programme focused on strengthening relations between the EU and South Africa, through facilitating joint decision-making and implementation of the priority aspects of the Trade Development Co-Operation Agreement, the Strategic Partnership and the Joint Action Plan, in an evolving partnership context. Although the TDCA-F is a pilot programme and the process is a new approach in South Africa (for both the Government of South Africa and the EU Delegation to South Africa), signifi cant progress and achievement is evident in a short time- frame, despite some challenges at the early stages of the programme.

2. Recognising the need to learn from the pilot process, the programme can be adjusted to further improve overall operational performance and impact in the balance of the implementation period and in going forward beyond the current programme. The continuity and impetus of the dialogue process is critical and an integrated planning approach is required to addresses the balance of the programme, the ‘gap’ that may arise until a new programme commences, as well as the adequacy of available funding, including the need for increased co-funding. On-going technical and fi nancial supports to dialogues must continue until sustainability of the process is assured, and both the EU and South Africa must commit to ensuring its renewal and enhancement. A top-up to the existing Dialogue Facility is recommended to meet existing pipeline demands, as well as to target other aspects of dialogues not being addressed, and an increase in funding will be necessary to meet increasing demands for dialogue support. A follow-up programme is also recommended to maintain impetus and to develop further the dialogue process as a means of reinforcing the Strategic Partnership and to support implementation of the TDCA and Joint Action Plan.

3. In learning from the pilot process, the following issues will require focus and follow-up to further improve programme eff ectiveness and impact, notably:

i. The role and make-up of the Programme Steering Committee (PSC) as an ‘agenda setter’, with increased focus on strategic aspects of the programme, including joint decision- making and improved monitoring and evaluation (M&E);

4

ii. The role and make-up of the Progamme Management Unit (PMU), its cost-eff ectiveness and its current external location to a SA Department (ideally the Department of International Relations & Co-Operation);

iii. Diff erent modalities of funding and their impact on joint decision-making, fl exibility and M&E;

iv. Ensuring that the entry-point to the Dialogue Facility is through a national SA Department, where involvement of the Provinces, Civil Society Organisations or other dialogue partners may be envisaged;

v. Ensuring the project proposal submission process (or ‘fi rst come-fi rst served’) is used strategically and recognises the requirement to align with and address both Summit outcomes and Joint Action Plan priorities;

vi. Co-funding needs to be developed on two levels, notably (i) increased joint-funding for the overall fund in support to the dialogues; and (ii) increased co-funding within SA Departments, where possible;

vii. Ensuring capacity-building for dialogues is given more focus in support of new or emerging dialogues, where more emphasis on institutionalisation is required and innovative peer-to-peer approaches need to be adopted.

4. Following completion of the Mid-Term Evaluation of the TDCA-F (incorporating fi ndings derived from desk-research, fi eld-mission interviews and stakeholder meetings), and based on reference to the Financing Agreement/Addendum, Log-frame(s), and key project reports in assessing all aspects of project performance, and recognising the pilot nature of the programme, the Overall Assessment of the Dialogue Facility (TDCA-F) is: Highly Satisfactory.

5

EXECUTIVE SUMMARY

3. OVERVIEW OF THE EVALUATION

1. Relevance of the programme is evident in that the outcomes of ten of the projects/ grants supported by the Dialogue Facility were included in the over-arching dialogues at the Joint Co-Operation Council and mentioned in SA-EU Summit Joint Statements. The main value in the programme to-date has been the enabling of two-way access to international best-practice, in particular practices in EU Member States, while allowing SA to demonstrate to the EU its ability to develop and apply best-practice approaches in Science & Technology (DST) and in Taxation & Customs aspects (SARS). The programme has piloted a new approach to co-operation and partnership.

2. Eff ectiveness of the programme is demonstrated through engaging with the EU, in that twenty one bilateral dialogues3 resulted from the TDCA-F and outputs of dialogue projects supported led to policy enhancement in alignment of Customs systems with World Customs Organisation standards, Climate Change, National Health Insurance, Earth Geo- Satellite Systems, Broadband Roll-out and Green Growth.

There is potential for on-going projects to make policy contribution on Skills Development, Innovation, Taxation, Social Safety Net System and Regional Peace & Security. The programme contributed to strengthening relations in that it addressed constraints to dialogue. However, capacity-building was developed at the level of exposure to new concepts and processes, but more substantive capacity-building institutionalisation was less eff ective due to inadequate programme design and tight implementation time-lines. Instruments available under the TDCA-F (EU Framework Contracts, Short-term Experts and Incidental Budget Lines under the PMU Contract and a Grant modality) have been used in a complementary fashion to ensure that relevant services have been procured to support the dialogue projects. Awareness of the TDCA, the Strategic Partnership and the TDCA-F amongst the target group can be improved and will benefi t from further attention under future funding.

3 Environment: (1) Road to Rio; (2) Green Growth; Health: (3) National Health Insurance Conference; (4) Institute of Regulatory Science; Education: (5) Further Professionalising the Quality Management of Higher Education; (6) National Skills Planning; (7) New Infrastructure Skills for New Infrastructure Jobs; (8) Technical & Vocational Education & Training; Science & Technology: (9) National Innovation Policy; (10) Research Infrastructure Road Map (RIRM); (11) Set-up of EGSA (European Geostationary Navigation Overlay Services (EGNOS) in South Africa) Initiative; Energy: (12) Regional Energy Conference; Peace & Political: (13) The Kimberley Process; (14) Trilateral Peace & Security Project; SARS: (15) Authorised Economic Operators; (16) Rules of Origin; (17) Tariff s; (18) Increase Domestic Resources in 3 Countries; Social Development: (19) Eff ective Delivery of Social Grants; Communications: (20) Broadband, Digital TV; (21) E-Skills.

6

3. Effi ciency of the programme represents good management and value-for-money, though there are some minor operational aspects that will require adjustment in going forward, as a pilot process. The governance of the programme was good in respect of allocation of funding of €7m to-date to projects, but governance can be expanded beyond funding allocation to fully improve strategic alignment of projects with Summit themes and Joint Action Plan priorities, and increased PSC oversight during implementation of the projects. Furthermore, the composition of the PSC and its strategic role need to be considered in going forward. The programme has achieved a very high level of commitment of funds through a pragmatic approach of approving all compliant projects for which adequate funding has been available.

While this approach has worked well for the pilot programme, future programmes should consider balancing rapid allocation of funds with providing an opportunity for SA Departments with less capacity to respond to any requests for submission of proposals and to learn from the process. For the future, a mechanism should be introduced to monitor and evaluate the dialogues and their outcomes, and in so doing provide an opportunity for feedback and learning to relevant stakeholders.

4. Impact of the programme is not fully possible to assess at this stage of the programme, but early indications are that all projects have contributed important insights that have supported policy, strategy and operational improvements and institutional arrangements, in particular where dialogues are mature. For emerging dialogues, impacts are less evident. The longer-term potential impact of the TDCA-F support is particularly high for DST and SARS, which have long-standing and developed relationships with EU stakeholders. Longer-term benefi ts will possibly be reduced, especially for emerging dialogues, by limited systematic capacity-building support (e.g. twinning, mentoring, peer-to-peer and community of practice initiatives, which are currently not being applied) in respect of institutionalisation of learning from the dialogue projects supported under the Dialogue Facility.

5. Sustainability of the dialogue process (rather than the programme), indicates that dialogues with Departments that lead mature dialogues, and that have good capacity to engage in dialogue, will be sustainable. Dialogue projects funded for Departments engaging in emerging dialogues, with less capacity in conducting dialogues, are less likely to be sustainable unless further funding and capacity support is provided. Should a new programme not be funded, it is less likely that new dialogues will be established and a number of existing dialogues may be constrained and become ineff ective, as they may have reduced means of undertaking joint projects.

7

EXECUTIVE SUMMARY

6. In terms of EC Value-Added, EU Member States (MS) fund more than thirty projects in the sectors supported by the Dialogue Facility. The programme has the potential to add-value to the EU MS activities by raising the learning and experience under their projects to the policy dialogue level. Conversely, the EU MS projects have the potential to add-value to the Dialogue Facility by informing the dialogues based on the research undertaken and experience gained on these projects, which can be leveraged more in the future. As the Dialogue Facility is unique in its approach, it is considered to add-value to the activities of the EU Member States. EU Member States contributed directly to the TDCA-F projects through provision of experts, engaging with South Africa on study tours, and direct fi nancial contributions.

7. In terms of Coherence, the intent of the Dialogue Facility is to align with SA priorities which are of mutual interest to SA and the EU, as set-out in the TDCA and the Strategic Partnership Joint Action Plan. As many SA Departments are not active in dialogues currently, and do not have adequate resources or capacity to engage on these areas of mutual interest, the programme complements the national programming to demonstrate the strategic value of dialogue. Where Departments have adequate resources and capacity to engage in dialogue, the programme has allowed projects of mutual interest to be accelerated and for partnerships to be developed and matured. The TDCA-F is aligned with EU developmental policies and the Country Strategy Paper. The Dialogue Facility is complementary to these programmes in that they focus on Poverty Alleviation and Improved Governance, while the Dialogue Facility provides resources for bilateral engagement and projects that can provide insights to improve policy formulation and implementation, including accompanying institutional reforms required.

8. In terms of Cross-Cutting Issues, the cross-cutting (and/or thematic) issues off er interesting dialogue opportunities, particularly as they extend across sectors and Departments. To-date, there is little evidence of either the Gender or Natural Resources cross-cutting issues being addressed in any meaningful or direct manner through the Dialogue Facility. In terms of thematic issues, the Employment theme off ers signifi cant scope to stimulate new or emerging dialogues across SA Departments (including those Departments not yet engaged in the dialogue process), or to address many important and/ or sensitive issues that otherwise may not be taken-up in a specifi c sectoral dialogue.

4 KEY CONCLUSIONS

The following main Conclusions are made, notably:

4.1 Conclusions on Strategic Aspects

The following fi ve Strategic Conclusions are made:

8

1. Overall programme relevance is high and the programme is strategic in its focus and intent. In terms of its design, greater participatory (rather than consultative) processes could have enhanced joint decision-making on scoping the programme components and their funding, while addressing SA concerns. Capacity-building for dialogues is identifi ed in the design, but it is not adequately defi ned or scoped, with the result that its implementation is less eff ective and sustainable.

2. The dialogue process has proven to be strategic in its promotion of information-exchange, exposure to best-practice thinking and approaches, as well as strengthening of relationships between EU-SA, but it requires continued technical supports to ensure that it evolves to become a ‘Balanced Partnership’4 and that new dialogue areas are initiated.

3. In the context of a pilot process, governance of the programme was good, but it can be strengthened to ensure greater focus on strategic aspects of programme planning and implementation.

4. Capacity-Building for Dialogue5 is critical in ensuring that the overall dialogue process achieves impact and continues into the future, and greater focus is needed in ensuring that the dialogue process is institutionalised through more innovative capacity-building approaches6.

5. The continuity and impetus of the dialogue process is critical and an integrated planning approach is required to addresses the balance of the programme, the ‘gap’ that may arise until a new programme commences, as well as the adequacy of funding, including the need for increased co-funding.

4 A Balanced Partnership has two levels of interpretation and implementation: (i) in political terms, as the basis for engagements between EU and SA, where both parties respect each other as equals (though the intent towards equality is notional rather than actual (as the EU as a REC is a representation of 28 EU Member States); and (ii) in technical terms, where the EU is perceived to have had 50+ years to evolve and develop (becoming profi cient in developing competencies in negotiating, engaging in dialogues and strategic partnerships elsewhere in the world), where SA will need to evolve over time its ability to engage on ‘equal’ terms with the EU on technical aspects, for which it will need support for capacity-building for dialogue.5 Capacity-building for Dialogue is identifi ed as having 2 levels of focus: (i) the ‘exposure’ to the concept of Dialogues, as well as to EU modalities and procedures, in particular, the proposal submission process, requiring profi ciency in preparing, submitting and presenting a proposal for project funding, in support of a sectoral dialogue; and (ii) ‘institutionalisation’ of capacity for dialogues, which requires a more systematic and time-bound approach to ensure that skills developed in the process are not just vested in one individual, but transferred to others within the Department. Similarly, wider learning from experienced Departments can be transferred to other SA Departments through a process of peer-to-peer and ‘community of practice’ approaches (where learning is SA-led). As envisaged in the programme design, there should be more focus on twinning, mentoring, coaching and other facilitative techniques adopted in technical assistance supports (where learning is expert-led), in support of developing sustainable capacity for dialogues in all SA Departments.6 Capacity-building requires a long term vision by both partners, and more importantly, it involves a commitment to adopting a number of interventions that go beyond just training or transfer of technical skills to individuals, but a commitment to supporting organisational and institutional change through real partnerships and substantive experiential learning arrangements such as mentoring and twinning.

9

EXECUTIVE SUMMARY

4.2 Conclusions on Operational Aspects

The following two Operational Conclusions are made:

1. Alignment and prioritisation of dialogue projects is a critical aspect for SA involvement, recognising the importance of the National Development Plan (and other strategies) and the existence of the clusters.

2. While the outcomes of the Grants component have proven to be useful and will contribute to future dialogues, use of the Grant modality in the context of the Strategic Partnership served to create unforeseen challenges in the operation of the programme.

5 MAIN LESSONS LEARNED

The following main Lessons Learned are identifi ed, notably:

5.1 Strategic Lessons Learned

The following two strategic lessons Learned are identifi ed:

1. In the context of the evolving EU-SA Strategic Partnership, the use of development funding for political dialogues is potentially sensitive. Focus is needed on ensuring that joint decision-making is recognised as a core aspect of the Strategic Partnership approach, in upholding the aid eff ectiveness principles.

2. The dialogue process works best when it is allowed to evolve and develop naturally, often informally, to an extent where relationships are strengthened and the quality of the dialogue process is enhanced through inter-personal interactions rather than through formalised processes and approaches. However, the dialogue process is very reliant on the people (and personalities) aspect, which can have an adverse impact on the conduct of a dialogue when personnel changes occur. Greater focus should be paid to development of teams and human resources supports to dialogues.

5.2 Operational Lessons Learned

The following Operational Lesson Learned is identifi ed:

1. Now that dialogues have proven their worth as a strategic process, it is necessary to adopt a more selective and strategic approach to identifying and screening projects that should be supported under the dialogue facility.

10

6 MAIN RECOMMENDATIONS

The following main Recommendations are made, notably:

6.1 Overall Recommendation

1. The dialogue process requires on-going support for the foreseeable future through:

i. The balance of the current programme (with its operational phase ending on 26.02.2015);

ii. The ‘gap’ period envisaged (from 27.02.2015 until a new programme is implemented); and

iii. A successor programme(s), which should be enhanced and innovative in its support to prioritised sectoral and other key dialogues.

6.2 Balance of Current Programme – Requirement for a Top-Up Facility

The following three key recommendations are made in respect of the Balance of the Programme and the requirement for a top-up of the facility, notably:

1. A top-up to the existing Dialogue Facility is recommended to meet existing pipeline demands, as well as to target other aspects of dialogues not being addressed, and an increase in funding will be necessary to meet increasing demands for dialogue support.

2. The Programme Steering Committee needs to be reinvigorated and its composition and role needs to be reviewed to ensure that it continues to meet the needs of the programme as it moves to a critical stage of development.

3. A cross-cutting dialogue on ‘Gender’ should now be considered, as to-date, this important issue has not been addressed in a direct or meaningful manner in the current dialogue process.

11

EXECUTIVE SUMMARY

6.3 ‘Gap’ Period Issue

The following two recommendations are made in respect of the ‘Gap’ period, notably:

1. An integrated approach to planning for, and addressing, the requirements for (i) the balance of the programme, and (ii) the evident ‘gap’ in programme activities, should be adopted in order to ensure that suffi cient resources and eff orts are deployed in maintaining continuity and impetus in the dialogues process – A ‘Gap’ Strategy and Roadmap should be formulated by the PSC.

2. To ensure continuity and impetus in the dialogue process, there should be no break-off in programme activities and/or in supports available to the dialogue process.

6.4 Future Programme Requirement

The following four key recommendations are made in respect of the Requirement for a Future Programme, notably:

1. A new Dialogue Facility 2 programme is required to ensure continuity of supports to the evolving dialogue process and the programming process should commence immediately to ensure the ‘gap’ between the current and future programme is minimised.

2. Governance of any future programme can be enhanced through greater attention to the composition and role of both the PSC and PMU, ensuring that both are more representative of the needs of SA as a strategic partner, in a joint decision-making approach.

3. Increasing co-funding, beyond the current levels, is proposed, especially for those dialogues that are more matured in order to meet increasing demands for dialogue support and to enhance co-ownership and joint decision-making processes.

4. Capacity-Building for Dialogue needs to be assured in any new programme, where emphasis needs to be placed in developing more sustainable institutional capacity-building through a combination of external technical assistance (i.e. through twinning, mentoring, coaching) and application of peer-to-peer and ‘communities of practice’ approaches, through mature Departments, led by SA (under the guidance of the Department of International Relations & Co-Operation).

12

6.5 Recommendations on Operational Aspects (of the Programme)

The following three recommendations are made in respect of the Requirement for a Future Programme, notably:

1. In terms of Key Result Area 1 (i.e. Improved sectoral policy dialogue and co-operation between the EU and South Africa), continued use of the ‘fi rst come-fi rst served’ approach under the project proposal submission process should be reviewed, in particular as the programme is now well established and recognising that the demand for support for dialogues can be demonstrated (beyond the initial stages of the pilot process).

2. In terms of Key Result Area 2 (i.e. Increased institutional capacity to help implement the areas of co-operation of the TDCA and the Strategic Partnership Action Plan), more care and attention needs to be paid to the institutionalisation of the capacity-building component, especially in relation to twinning, peer-to-peer mentoring and coaching aspects, which to-date have not been undertaken in any meaningful manner.

3. In terms of Key Result Area 3 (i.e. Increased awareness and understanding of the Strategic Partnership and TDCA amongst the target group and indirect benefi ciaries), greater focus and attention should be paid to ensuring that awareness-raising and promotion of the Dialogue Facility and its benefi ts is made and that the direct target audience7 engages more meaningfully with the facility in order to derive benefi t from it in terms of awareness and capacity-building reinforcement.

6.6 Recommendation on Other Aspects (outside of the Programme)

The following recommendation is made in respect of future dialogues or programming outside of the TDCA-F, notably:

1. Lessons Learned and experiences from the Dialogue Facility can and should be taken to the regional level for relevance to the regional integration agenda, but with a national Department as the entry-point for any dialogue initiative.

7 Direct Target Audience in terms of all key SA Departments.

13

EXECUTIVE SUMMARY

ACKNOWLEDGEMENTSThe Evaluation Team wishes to acknowledge, with thanks,the support and guidance provided by the members of the Reference Group, and its Co-Chairs, in producing this Final Report.

The Evaluation Team is particularly thankful to all key stakeholders and individuals who met with them in the course of the mid-term evaluation exercise.

For more information on the Dialogue Facility and other relevant links, please visit: www.dailoguefacility.org