UNESCO Regional Office in Dakar

Final external evaluation of the extra budgetary

project on “Improvement of teacher qualification and

setting up of a system for the management of learning

outcomes in Guinea‐Bissau”

Final report

October 2015

Rui da Silva Consultant

rdasilva.email@gmail.com

1

Sumário executivo

O presente relatório visa apresentar os resultados da avaliação final externa do projeto extra orçamental “Melhoria da Qualificação de Professores e Implementação de Gestão de Resultados de Aprendizagem na Guiné-Bissau” promovido pelo Ministério da Educação Nacional da Guiné-Bissau e pelo escritório da UNESCO em Dakar, com o apoio financeiro do Governo Italiano. Este projeto decorreu de 2009 a 2015 em duas fases e tem como principais eixos de intervenção a formação inicial

e contínua de professores do ensino básico.

A avaliação seguiu uma abordagem quanti-qualitativa recorrendo como técnicas de recolha de dados a entrevistas presenciais e por telefone (13) e à análise documental (47). Devido à instabilidade política vivida durante o período de avaliação (Setembro de 2015) a missão à Guiné-Bissau foi cancelada, limitando desta forma a avaliação realizada.

Os resultados parecem mostrar que, de uma forma global, o projeto pode ser considerado como tendo

um valor acrescentado no panorama atual do sistema educativo da Guiné-Bissau, principalmente tendo em conta as mudanças ocorridas no setor da educação durante a implementação do mesmo. O projeto alcançou 5 dos 9 resultados esperado e parcialmente outros dois.

De uma forma geral, o projeto alcançou os seguintes resultados:

i) Produção de um balanço de âmbito nacional de competências académicas (língua portuguesa e matemática) de docentes em exercício na Guiné-Bissau que lecionam do 1.º ao 9.º ano de escolaridade (taxa de resposta na maioria das regiões superior a 90%);

ii) Produção de uma avaliação da formação inicial de professores do 3.º ciclo do ensino básico (7.º -9.º ano);

iii) Produção de uma avaliação das competências dos alunos do 3.º ano da Escola de Formação de Professores 17 de Fevereiro;

iv) Produção de um estudo sobre uma estratégia para a formação contínua de professores do 1.º e 2.º ciclos do ensino básico (1.º - 6.º ano);

v) Produção de uma proposta para o apoio à elaboração do novo currículo de formação inicial de professores do 1.º, 2º e 3.º ciclos do ensino básico (1.º - 9.º ano);

vi) Produção de 5 módulos e 5 guias para professores no âmbito da abordagem por competências, tendo por base a sua harmonização com os módulos produzidos pelo Banco Africano de Desenvolvimento;

vii) Realização de uma formação de formadores nos módulos produzidos para inspetores formadores das delegações regionais de educação;

viii) Formação em serviço de 298 professores do 1.º e 2.º ciclos do ensino básico (1.º - 6.º ano) na abordagem por competências em 5 disciplinas (língua portuguesa, matemática, expressões, psicopedagogia e ciências integradas. Após a realização de um pré-teste e de um pós-teste, verificou-se uma diferença significativa entre os resultados de todos os assuntos testados (F(6,267)=6,83, p<0,001) com a exceção da abordagem por competências. Contudo, analisados os valores médios, verificaram-se efeitos positivos em expressões, matemática e psicopedagogia;

ix) Desenvolvimento de capacidades que promoveu junto do Instituto Nacional de Desenvolvimento da Educação e das delegações regionais de educação;

x) Contribuiu para o desenvolvimento de capacidades do Instituto Nacional de Desenvolvimento da Educação e do Ministério da Educação Nacional;

xi) Apoio à realização dos exames nacionais de 2011; xii) Participação no esforço de coordenação e harmonização com parceiros/actores nacionais

do sector da educação.

No que concerne ao projeto, de uma forma global, parece que pode ser considerado como complementar às restantes intervenções do setor da educação e as suas atividades complementares

2

à dos restantes atores da educação (e.g. BAD, UNICEF, FEC). Contudo, o seu maior desafio durante a implementação (até ao momento desta avaliação) foi a validação por parte do Ministério da Educação Nacional do programa desenvolvido para a formação inicial de professores. Outro desafio será capitalizar os estudos produzidos de forma a contribuir para o desenvolvimento do sistema educativo.

Posto isto, e com base na análise de dados, podemos considerar que as principais recomendações são:

i) os materiais e estudos produzidos pelo projeto devem ser capitalizados, pelo contributo que podem fornecer ao sistema educativo no contexto de mudança atual no quadro das reformas que estão a ocorrer no setor;

ii) o escritório da UNESCO em Dakar deverá melhorar os mecanismos de monitorização do projeto de forma a ter acesso a mais dados mensuráveis e produzir evidências;

iii) o escritório da UNESCO em Dakar deverá incluir desde a fase de design/ planeamento uma abordagem sensível às questões de género;

iv) o escritório da UNESCO em Dakar poderá apoiar o Ministério da Educação Nacional a criar um mecanismo interno que apoie a harmonização e coordenação das intervenções no setor da educação;

v) o escritório da UNESCO em Dakar e o Ministério da Educação Nacional poderão desenvolver orientações na educação bilingue como estratégia para a promoção da qualidade da educação;

vi) o escritório da UNESCO em Dakar e o Ministério da Educação Nacional poderão promover a implementação de um projeto piloto de educação bilingue como estratégia para a promoção da qualidade da educação, em parceria com outros detentores de interesse do setor da educação;

vii) o escritório da UNESCO em Dakar poderá continuar a apoiar o Ministério da Educação Nacional (e.g. INDE e as delegações regionais) com base na abordagem de desenvolvimento de capacidades como estratégia para a promoção da qualidade de educação e para a redução das desigualdades regionais;

viii) o escritório da UNESCO em Dakar poderá colocar mais ênfase no nível macro das politicas educativas devido ao sistema educativo poder ser caracterizado a este nível por inconsistências e descontinuidades;

ix) o escritório da UNESCO em Dakar poderá apoiar o Ministério da Educação Nacional, em conjunto com os restantes parceiros do setor, no desenvolvimento de uma estratégia nacional de desenvolvimento profissional de professores (inicial, em serviço e contínua) tendo em consideração os desafios colocados ao sistema educativo, a legislação em vigor e os estudos produzidos;

x) o escritório da UNESCO em Dakar, poderá apoiar o Ministério da Educação Nacional na regulamentação da legislação existente no país e na coordenação das intervenções dos vários parceiros no setor. Estes aspetos estão relacionados com a necessidade de regulamentação da legislação existente depois da publicação da Lei de Bases do Sistema Educativo e do Estatuto da Carreira Docente, nomeadamente no que diz respeito aos exames nacionais, aos registos escolares, à educação bilingue, ao recrutamento de professores.

3

Executive Summary

This report aims to present the results of the final external evaluation of the extra budgetary project, "Improvement of teacher qualification and setting up of system for the management of learning outcomes in Guinea-Bissau" implemented by UNESCO Dakar in close collaboration with the Ministry of National Education of Guinea-Bissau. The project was implemented during the period of 2009 and 2015 and focused its intervention on pre‐service and in‐service teacher training for the basic

education.

The evaluation followed a quantitative-qualitative approach. The data collection strategy used in person/phone semi-structured interviews (13) and document analysis (47). Due to the political uncertainty during the time of the evaluation (September 2015) the field mission to Guinea-Bissau as cancelled, limiting further assessment.

The results show that in general the project can be considered as having an added value in the current situation of the education system of Guinea-Bissau, particularly in regard to the changes that occurred in the education system during the project implementation. The project achieved 5 of the 9 expected outputs and partially other 2. The project overall achieved:

i) Conducted a national academic skills assessment of in-service teacher (grades 1 to 9) on Portuguese language and Mathematics with a response rate in the majority of the regions higher than 90%;

ii) Conducted an assessment of the 1st and 2nd cycles (grades 1-6) basic education pre-service teacher training institution, 17 de Fevereiro, to evaluate the academic skills of students in their 3rd year of the pre-service teacher training.;

iii) Conducted the above equivalent assessment of students’ academic skills for 3rd cycle (grades 7-9) basic education pre-service teacher training institution, Tchico Té, ;

iv) Developed a study to further elaborate a strategy for in-service teacher training from the 1st and 2nd cycle of basic education (grade 1 to 6);

v) Developed a proposal for pre-service teacher training programs; vi) Developed 5 modules and 5 guides for in-service teachers within the competence based

approach harmonizing them with the modules produced by the African Development Bank;

vii) Training of inspectors/trainers from 11 regional education delegation from the Ministry of National Education on the produced modules;

viii) Training of 298 in-service teachers in competency based approach in 5 subjects (Portuguese, Expression, Mathematics, Psycho-pedagogy and Sciences). A pre and a post-training test was conducted, demonstrating a significant difference in scores of all subjects (F(6,267)=6,83, p<0,001), with the exception of competence-based approach related question. However, when looking to the average scores, a significant positive effect is observed only in Expression, Mathematics and Psycho-pedagogy;

ix) Contributed to INDE (National Institute for Educational Development) and Ministry of National Education regional delegations capacity development;

x) The project supported the realization of the national exams in 2011; and

xi) The project coordinated and harmonized the activities with other education stakeholders.

The project succeeded to coordinate the proposed activities with relevant education stakeholders and can be considered complementary to other interventions (e.g. UNICEF, FEC, AfDB). However the main challenge faced by the project (at the time of this evaluation) was the validation by the Ministry of National Education of the pre-service teacher training, certification, and degree system. Another challenge regard to the capitalization of the studies produced by the project in order to operate and/or develop the education system.

4

Taking this into consideration and the data analysis, we can consider that the main recommendations are:

i) the materials and the studies produced by the project should be capitalized by partners considering the contributions that it can provide to the education system in the context of current reforms;

ii) UNESCO Dakar should determine ways forward to improve the monitoring mechanisms in order to have more measurable data and generate evidence;

iii) UNESCO Dakar should include an explicit gender sensitive approach since the project design/planning stage.

iv) UNESCO Dakar could determine ways forward to support the MoE in the further

development of the national examination system as a tool for national planning,

monitoring and management for the quality of learning;

v) UNESCO and the MoE could determine ways forward for the launching within the MoE the establishment of a harmonization and coordination mechanism for education interventions;

vi) UNESCO and the MoE could develop a position paper/orientations on bilingual education developed as a strategy for quality education;

vii) UNESCO and the MoE could determine ways forward to support the implementation of the pilot bilingual education in partnership with other education stakeholders as a strategy for quality education/learning;

viii) UNESCO could determine ways forward to continue supporting Guinea-Bissau education system and MoE (e.g. INDE and regional delegations) based in the capacity development approach promoting a strategy for education quality and reducing the regional inequalities;

ix) UNESCO could determine ways forward to put more emphasis in the macro policy level because the education system can be characterized by inconsistencies and discontinuities at this level;

x) UNESCO Dakar could support the Ministry of National Education, in articulation with other education stakeholders, in the development of teacher professional development strategy (pre-service and in-service) based on the challenges posed to the education system and the adopted legislation; and

xi) UNESCO Dakar could assist the Ministry of National Education in the regulation of existing education legislation and in the coordination of all the education partners’ interventions. This stems from the fact that the education act and other legislation have been published but there is little reinforcement in their application as the Ministry of National Education must put in place appropriate mechanisms, and specific aspect from these laws that need further development regarding the teachers careers, national exams, bilingual education, school records, teacher recruitment, among others.

5

Table of Contents

Sumário executivo ................................................................................................................................... 1

Executive Summary ................................................................................................................................. 3

Table of Contents .................................................................................................................................... 5

Acronyms ................................................................................................................................................. 6

Disclaimer ................................................................................................................................................ 7

1. Introduction ......................................................................................................................................... 8

2. Overview ............................................................................................................................................ 10

2.1 Country and project context ....................................................................................................... 10

2.2 Theory of Change......................................................................................................................... 11

3. Findings and Conclusions ................................................................................................................... 14

3.1 Relevance..................................................................................................................................... 14

3.2 Effectiveness ................................................................................................................................ 17

3.3 Sustainability ............................................................................................................................... 19

3.4 Efficiency ...................................................................................................................................... 20

4. Lessons learned ............................................................................................................................. 22

5. Challenges ...................................................................................................................................... 23

6. Recommendations ............................................................................................................................. 23

7. Annexes ............................................................................................................................................. 25

7.1 Project expected results and the achievements ......................................................................... 25

7.2 People interviewed ...................................................................................................................... 29

7.3 Reviewed documents .................................................................................................................. 29

7.4 References ................................................................................................................................... 33

Figures index

Figure 1 Theory of change ..................................................................................................................... 13

Table index

Table 1 - Number of teacher trained ..................................................................................................... 17

Table 3 - Project expected results and the achievements phase I ........................................................ 25

Table 4 - Project expected results and the achievements phase II ....................................................... 27

Table 5 - Reviewed documents ............................................................................................................. 29

6

Acronyms

ADPP – Ajuda de Desenvolvimento de Povo para Povo

AfDB - African Development Bank

ECOWAS - Economic Community of West African States

EFA – Education for All

ESE-IPVC – School of Education - Polytechnic Institute of Viana do Castelo

FASPEBI - Fundação para o Apoio ao Desenvolvimento dos Povos do Arquipélago dos Bijagós

FEC - Foundation Faith and Cooperation

INDE - National Institute for Educational Development

MDG - Millennium Development Goals

MoE – Ministry of National Education

OECD - Organisation for Economic Co-operation and Development

TISSA - Teacher Training Initiative for sub-Saharan Africa

ToR – Terms of Reference

UNDP - United Nations Development Programme

UNESCO - United Nations Educational, Scientific and Cultural Organization

UNFPA - United Nations Population Fund

UNICEF - United Nations Children's Fund

WFP - World Food Programme

7

Disclaimer

The opinions expressed in this report are those of the Consultant, and do not reflect those of UNESCO. Responsibility for the opinions expressed rests solely with the author and does not imply any kind of endorsement from UNESCO.

All data gathered and presented are dully factual and truthful. The background of the project under evaluation was provided by UNESCO- Dakar staff, as well as the implementing team in Guinea-Bissau, however, opinions, conclusions, and recommendations expressed herein are those of the author based on the information (feedback) gathered.

Further claims are rightfully referenced, quoting the origin of the source.

8

1. Introduction

The project, “Improvement of teacher qualification and setting up of a system for the management of learning outcomes in Guinea-Bissau” aimed to develop an effective in-service and pre-service teacher training system to sustain a body of highly qualified teachers capable of delivering quality education. This report is the final external evaluation, which presents main results of the project evaluation and recommendations for UNESCO, the government and other partners who are working in the area of teacher training in the country.

In accordance with the terms of reference (ToR) this evaluation aimed to:

• assess relevance, effectiveness, sustainability, and efficiency of the project;

• measure the degree of implementation of the project and the quality of the process, products and achievements in relation to what was originally intended or subsequently formally amended;

• develop a better understanding of how and to what extent this project is contributing to facilitating teachers in achieving better qualification and the Ministry of Education in improving in-service and pre-service teacher training systems and policies; and

• formulate recommendations that will be useful for designing and implementing future similar interventions. It is also expected that the evaluation will finally be used for accountability and for learning purposes by key stakeholders.

In this way, the evaluation aims to gather evidence of the project’s key achievements and challenges to be used to inform decision-making processes of relevant stakeholders. It is also expected to enhance understanding of teacher’s issues in Guinea-Bissau allowing to orient future similar interventions.

The evaluation was conducted by a single independent consultant and followed a mixed method, quantitative and -qualitative approach. The data collection included in person/phone semi-structured interviews (13) and document analysis (47). The evaluation has the two following main limitations: i) the cancellation of the field mission to Guinea-Bissau due to the political situation during the time of the evaluation (August/September 2015). As a consequence, it was not possible to include a larger group of participants and reduced the data collection to interviews and document analysis; ii) the measurement of some of the outputs due to the absence of a baseline and/or other measurable data regarding the activities.

In accordance with the Terms of Reference (ToR), the evaluation had the following questions as a guide:

Relevance

1. Are the 3 components on pre-service and in-service training and pilot bilingual education consistent with the political orientations in Guinea-Bissau?

2. Is the project consistent with the C/5 and C/4 (including the relevant expected results, Strategic Objectives and UNESCO’s Global Priorities), the Millennium Development Goals and other international development goals?

3. Are the project interventions adapted to the local context, including the model used for the training of teachers on bilingual education?

4. To what extent are the components of this intervention aligned with complementary to/or overlapping with those of other education actors, implementing agencies or development partners?

5. To what extent is the pre-service teacher training programme proposals relevant to the pressing demand of increasing teacher’s body, taking into account the high number of unqualified in-service teachers in the country while considering their competence level?

9

Effectiveness

6. To what extent were the in-service and pre-service teacher trainings (grades 1 to 6) improved at policy level, at implementing level, and at the teachers’ level?

7. To what extent were the national capacities reinforced through the implementation of this project?

8. Does the model used for the training of teachers on bilingual education prove to be useful to improve learning and is it scalable?

9. What were the major factors influencing the achievement or non-achievement of the objectives of the project?

10. Did the project have effective monitoring mechanisms in place? 11. Has the project been cost-effective, i.e. could the outcomes have been achieved at lower cost

through adopting a different approach and/or using alternative delivery mechanisms? 12. To what extent has the project been gender sensitive within its interventions?

Sustainability

13. To what extent did UNESCO involve targeted beneficiaries (teachers, MoE, INDE, etc.) in the design, delivery, and monitoring of the project?

14. To what extent will national appropriation of the project results be ensured and more specifically by the Ministry of Education? In particular:

a. How the 5 developed training modules and guides for in-service teachers’ grades 1 to 6 will be used as reference at national level to deliver quality teaching in alignment with the curricular reform and the competence-based approach adopted by the Ministry?

b. How the pilot in-service teacher training on the 5 revised modules and guides can be extended nationally to reinforce the capacities of all the in-service teachers’ grade 1 to 6?

c. How the proposal of pre-service teacher training programmes will be used as a reference standard by the Ministry of Education?

15. To what extent will the benefits of national capacity reinforcement continue after UNESCO’s support ceased?

16. Are any potential donors/partners interested in maintaining/further building upon the project outputs of the components on pre- and in-service trainings and on bilingual education?

Efficiency

17. Have UNESCO’s organizational structure, managerial support and coordination mechanisms adequately performed in function of timely delivery?

18. Could more activities and outputs have been delivered with the same resources? 19. Has UNESCO been focusing on the areas where it is best equipped to intervene, while

aligning/coordinating with other developing partners?

This document is organized into seven main parts. Following these introductory notes, the document provides a brief overview, including the country and the project context, and the theory of change. Following this overview are the findings and conclusions, the lessons learned, and the challenges followed by the recommendations. The seventh and final part presents the annexes that it exposes a table with the Phase I and II project expected results and the achievements, a list of people interviewed, a list of reviewed documents and the references.

10

2. Overview

2.1 Country and project context

Guinea-Bissau is a west African small state with 1,6 million inhabitants, with an average annual rate of population increase estimated at 3% [1, 2]. The country has one of the lowest gross domestic products per capita in the world (USD1270), poor infrastructure, weak social indicators, and a poverty rate of 64,7% [2-4]. The country relies heavily on official development assistance, particularly for public services, and education is one of the sectors which depends heavily on bilateral and multilateral aid [2, 5]. Although the official language is Portuguese, only 5% of the population use it as mother tongue and around 10% purport to speak Portuguese. Most inhabitants speak one or more indigenous languages (including creole), of which there are 20 [5-7]. The Creole is the lingua franca and is spoken by about 15% as a mother tongue and 30%–40% as an second language [7].

Guinea-Bissau has experienced significant conflict and political and military instability both prior to and following independence (1973). It is a peripheral country in the world system where a myriad of international organizations intervene due to its post-colonial condition (e.g. Word Bank, UNICEF, UNESCO, African Development Bank, UNDP, WFP, EuropeAid, UNFPA). The World Bank and OECD have classified it as a fragile state and UN as a least development country and a small developing state [9, 10].

The fragility of Guinea-Bissau is linked to its political and military histories. An example of this instability is the April 2012 military coup, ending the second turn of the presidential elections, the tenth coup since independence in 1973 from Portuguese colonial rule [11] or the recent (August 2015) dismissed of the government by the current President installing a political crisis.

Conditions linked to state fragility such as poor governance, repression, corruption, low levels of social cohesion, inequality and exclusion can affect education provision [12, 13]. In Guinea-Bissau examples of such effects include: i) a net enrolment rate of 67% in primary education; ii) 13% of classrooms made of quirintim (bamboo mat) iii) lack of teacher and learning materials; iv) limited number of school days due to teacher strikes, triggered by delayed distribution of salaries; v) use of double or triple shifts, while having a significant teacher body with no pre-service training and in some cases no pedagogical training at all [1].

The project under analysis was named “Improvement of teacher qualification and setting up of system for the management of learning outcomes”. It aims to support the rebuilding of the Bissau-Guinean education system by focusing on improvement of in-service and pre-service teacher training (grades 1 to 6) and was implemented from January 2009 to October 2015 (see phase I and II project documents for more details for the historical development of the project). The expected impact of the Project was “to support the efforts of the Government to meet the national education for all (EFA) goals by supporting the implementation of its education sector plan through teacher policy development, implementation of the in-service training plan and setting up of a national system for exam”. More specially:

1. Improvement of in-service teachers training modules and teachers skills (grades 1 to 6); 2. Pilot introduction of bilingual education in Creole and Portuguese (grades 1 and 2); 3. Improvement of pre-service teacher training quality (grades 1 to 6); 4. Reinforcement of national capacities;

The project had four extensions. The first one aimed to continue some components and also to conduct additional activities with additional funds. The other three extensions occurred due the Guinea-Bissau political instability. The extensions caused by the political instability delayed the implementation of the project activities and consequently budget and program outputs adaptation.

11

2.2 Theory of Change

The theory of change is based on the fundamental logic that the key elements to improve the quality of education and achieve the national EFA goals is reinforcing national capacities, teacher training (in-service and pre-service) and setting up a national examination system. In contexts of fragility1 such as Guinea-Bissau, it is paramount to support the transition to a more stable educational system. By adopting a flexible planning the project tried to address this. The diagram (Figure 1) intends to represent the project theory of change2.

The inputs to the programme were of 1.6 million USD of project finance with a part-time coordination team in UNESCO office in Dakar, a full-time national project officer in Bissau and technical support by national experts. The theory of change asserts that a number of activities are necessary to deliver the outputs. These include the following activities:

• In-service teacher training o Conducted national assessment of teachers from primary and lower secondary

education (Portuguese and mathematics); o Development of a teacher in-service training plan and experimental training tools for

grades 1 to 4 (Module 1 – Basics of the teaching profession and Module 2 – Portuguese and mathematics);

o Harmonization of the teacher training modules for grades 1 to 6 and revision for alignment with the curricular reform in Guinea-Bissau on the adopted competence-based approach while considering the socio-cultural context in the country and gender issues. This concerns modules on 5 subjects: Mathematics, Portuguese, Expression, Integrated Science and Pedagogy/Didactic/Ethic and Responsibility;

o Elaboration of teacher guides on the 5 subjects mentioned above; o Training of trainers/school inspectors on competence based pedagogy within the 5

disciplines. Finalization of the modules and guides; o Reproduction of the modules and guides for future trainings (dissemination).

• Bilingual Education

o Bilingual education for grades 1 to 2; o Elaboration/revision of training materials for teachers and textbooks for the students; o Selection of schools for the implementation phase; o Training of in-service teachers from the selected schools; o Monitoring of the implementation of the pilot bilingual education in selected schools.

• Pre-service training

o Support primary education teachers training institutions reform; o Analysis of the current situation of pre-service training programs; o Organization of a national workshop to reflect on pre-service training programs; o Elaboration of proposals for national pre-service training programs (primary and lower

secondary levels).

• Capacities and resource management by the Ministry of Education improved

o Assessment of teachers qualification in primary and secondary education; o Hiring of national experts from the Ministry of Education; o Support the implementation of national exams; o Training to the use of national exams results for management purpose; o Supervision of the technical work on in-service and pre-service training by a technical

consultant;

1 Refer to the lack of capacity and/or willingness to provide basic services. 2 In the red boxes are the outputs and outcomes excluded during implementation of phase II. More information please see the project progress reports from UNESCO Dakar, March 24, 201 and from January 1, 2014 – April 15, 2015

12

o Technical advice provided throughout implementation process; o Purchase of IT equipment and consumables for INDE and Escola 17 de Fevereiro

(computers, photocopying machine, scanner, printer, paper, and toner).

• Project implemented with quality and in a timely and cost-effective manner

o Elaboration of monthly and semester report; o Elaboration of monthly work plan/timeline o Mid-term and final external evaluation; o Monitoring, coordination, and supervising missions; o Elaboration of lessons learnt given the completion of each project component, and the

respective recommendations; o Elaboration of project brochure.

At the next level of the theory of change the outputs are expected to lead directly to immediate outcomes:

• The Ministry of Education is able to design its strategy on the teaching profession development while aligning towards achieving the MDGs;

• Improvement of qualifications of primary and secondary teachers (in-service and pre-service training);

• First exam session realized and results made available for all candidates;

• Capacities and resource management by the Ministry of Education improved;

• Project implemented with quality and in a timely and cost-effective manner. At the next level (outcomes), the theory is that by increasing Ministry of National Education (MoE) capacity, setting up a national examination system and improve the in-service and pre-service teacher training, the project will support Government’s efforts to meet the national EFA goals as well as its political framework (2009-2020) for the development of the Education sector The key assumptions underlying the theory of change are:

• Funds are available on time;

• Project staff and consultants appointed on time;

• Appropriately qualified consultants;

• Teacher attended trainings;

• Teacher implement the teaching materials;

• Government commitment;

• MoE validate the materials produced;

• The project is able to positively leverage and coordinate with other agencies and projects within the sector;

• The enabling environment for education is improved though other projects/ agencies;

• Decision makers validate the use of bilingual education;

• Change will be detected and measurable evidencing progress on the achievements of outcomes;

• Equipment distributed and usable;

• The external environment induces to success.

13

Figure 1 Theory of change

14

3. Findings and Conclusions

This section is organized taking in consideration the 4 DAC-OECD standard evaluation criteria (relevance, effectiveness, sustainability and efficiency) taken in consideration during the evaluation. In the annexes there is a resume table of the project expected results and the achievements taking in consideration the project logical framework.

3.1 Relevance

1. Are the 3 components on pre-service and in-service training and pilot bilingual education consistent with the political orientations in Guinea-Bissau?

The in-service and pre-service teacher training seems consistent with Guinea-Bissau orientations namely with the Education Act (2011), the Education Sector Policy Paper (2009) and the Education Sector Plan (2009). The pilot bilingual education is not consistent with the Guinea-Bissau orientations because so far the country does not have a policy regarding the use of national languages. Nevertheless, this pilot experience was approved by INDE and recognized important due the contribution that can give to improve the education quality. The timing of the project implementation was ideal because the Guinea-Bissau education system since 2009 started important transformation with the approval of several laws and the start of the Global Partnership for Education project in the country.

2. Is the project consistent with the C/5 and C/4 (including the relevant expected results, Strategic Objectives and UNESCO’s Global Priorities), the Millennium Development Goals and other international development goals?

The project seems consistent with:

• the UNESCO’s Medium-Term Strategy for 2014-2021 (document 37 C/4) and the UNESCO's Programme and Budget for 2014-2017 (37 C/5): Global Priorities on Africa and gender and the strategic objectives/main lines of action 1 - Supporting Member States to develop education systems to foster high quality and inclusive lifelong learning for all, 3 - Advancing education for all (EFA) and shaping the future international education agenda. Teachers is equally one of the 7 thematic areas of expected results of the Major Programme I – Education’s Main Line of Action 1.

• the Millennium Development Goals (goal 2 - Achieve Universal Primary Education, and 3 - Promote Gender Equality and Empower Women), the Education for All goals (2 - Ensuring that by 2015 all children, particularly girls, children in difficult circumstances and those belonging to ethnic minorities, have access to, and complete, free and compulsory primary education of good quality, and 6 - Improving all aspects of the quality of education and ensuring excellence of all so that recognized and measurable learning outcomes are achieved by all, especially in literacy, numeracy and essential life skills).

• the UNESCO Convention against Discrimination in Education and with UNESCO’s guiding principles regarding mother tongue in order to contribute to quality of education in multilingual countries3.

3 See, for example UNESCO (2008) Mother Tongue Matters: Local Language as a Key to Effective Learning. Paris: UNESCO.

15

3. Are the project interventions adapted to the local context, including the model used for the training of teachers on bilingual education?

The project intervention seems adapted to the local context. The main factors contributing to this fact are that the executing institution for this project was INDE, the national coordinators are from Guinea-Bissau, the project tried to follow a local capacity development approach4 by strengthening existing capacities (e.g. INDE, inspectors/trainers), and by adopting a flexible planning suitable for the local context. However, INDE experienced some difficulties in executing the project especially linked to the implementation of the in-service teacher training in the regions. These difficulties are linked to education partner’s trainings articulation and logistical and administrative procedures associated to the organization of these type of trainings.

4. To what extent are the components of this intervention aligned with complementary to/or overlapping with those of other education actors, implementing agencies or development partners?

As to whether the project intervention was aligned i.e. complementary/or overlapping with those of other education actors it seems that the:

• project tried and succeeded to coordinate with education actors (specially with UNICEF, African Development Bank [AfDB] and Foundation Faith and Cooperation [FEC]) in order to be complementary and not overlap activities. The coordination/collaboration with UNICEF was frequent in both phases of the project in order to avoid duplication of activities; with AfDB the collaboration was in regard to the harmonization of the training modules and guides; and with FEC the in-service teacher training was coordinated in order to avoid duplication of trainers and trainees, while contributing with UNESCO Peace Education manual as an additional session of their training on school management and administration.;

• studies financed by the project5 were complementary to the intervention of other education actors (AfDB, World Bank, UNICEF, PLAN International, FEC, UNDP, UNFPA, ADPP, Portuguese Cooperation and Effective intervention);

• in-service teacher training was complementary because it seems that training contents of the

other partners are not the same and also the teachers are not the same. However, it seems that

the different educations stakeholders are following their own approach/strategy because of the

absence of a national strategy regarding the teachers professional development;

• project dialogue with the other education actors (e.g. UNICEF, AfDB and FEC) and the participation in phase II in the Local Education Group meetings by the project officer representing UNESCO, seems to contribute greatly to the coordination and harmonization efforts put in place, especially in phase II.

4 See, for example De Grauwe, A. (2009). Without capacity, there is no development. Paris: UNESCO-IIEP. And/or OECD. 2006. The challenge of capacity development: Working towards good practice. In DAC Guidelines and

Reference Series. Paris: OECD. 5 Benavente, A., Varly, P. (2010) Balanço de competências de docentes em exercício na Guiné Bissau. / Ramalho, G. (2010) Avaliação dos Alunos do 3ºAno da Escola de Formação de Professores 17 de Fevereiro. / Campos, B. (2010) Avaliação da Formação Inicial de Professores do Ensino Secundário da Guine-Bissau. Volume I. Avaliação Institucional e Pedagógica / Ramalho, G. (2010) Avaliação da Formação Inicial de Professores do Ensino Secundário da Guiné-Bissau - Volume II / Avaliação dos Alunos-Professores / Butlen, M., Martinez, C. (2011) Vers une politique et

une stratégie nationales pour la formation continue des professeurs de l’enseignement fondamental en Guinée-

Bissau. / Barreto, M. A. Saraiva, A. M. R., Ferreira, J. B. (2010) Apoio à elaboração do novo currículo de formação

inicial de professores do ensino básico (ensino primário e secundário geral) na Guiné-Bissau. Volume I e II. / Furtado, A. (2015) Planos curriculares e programas para formação Inicial de professores do ensino básico.

16

Regarding in-service teacher training both the project and FEC6 use the inspectors/trainers from the MoE

as trainers. Taking this into account both the project and FEC to some extent had the same target group.

However, this can result in a capacity development of the inspectors/trainers in different areas with

UNESCO’s training focusing in competency based approach and FEC in the curriculum themes and

educational management and administration. Nevertheless, FEC does not involve all the inspectors/

trainers from the country while the project involved 82.7% of total inspectors/trainers from the MoE.

5. To what extent is the pre-service teacher training programme proposals relevant to the pressing demand of increasing teacher’s body, taking into account the high number of unqualified in-service teachers in the country while considering their competence level

In regards to pre-service teacher training, it seems that there is a risk of neglect in regards to the revisions and program proposal produced under this project. This stems from the fact that Escola Superior de Educação do Instituto Politécnico de Viana do Castelo (ESE-IPVC)7 signed a protocol with MoE and collaborated in pre-service teacher training from 2010 to 2012 under a Portuguese Cooperation: PASEG. Nevertheless, it seems that the two interventions can be complementary and articulated eventually. However, because of the political instability, the MoE has not yet received the pre-service teacher training programs proposal from UNESCO at the time of this evaluation.

In order to establish a pre-service teacher training, certification, and degree system, two scenarios were proposed by the consultants in the reports/studies: scenario A consists in a pre-service teacher training for 3 years, leading to a bachelor’s degree; and scenario B consists in a pre-service teacher training for 2 years which give access to a professional diploma that allow the access to the teacher carrier.

Under this project a reflective workshop held in May 2015 in Bissau a third scenario emerged, scenario C, which integrated training that meets the needs of in-service teachers with more than 5 years of service, with professional experience and with training courses not certified. The scenarios A and B8 seem relevant to the pressing demand for increasing teacher body. Nevertheless, the scenario C must equally be taken into consideration. This stems from the fact that the pre-service teacher training institutions are not able to train the quantity of teachers needed per year in order to achieve the first 6 years of schooling for all by 2020 (+/- 450 teachers per year)9. There is a high number of in-service teachers in schools with no pre-service teacher training and without opportunities to be integrated in in-service teacher training schools/institutions10. These aspects are related to the holist diagnosis on teachers issues (TISSA) that initially constituted a project component. However this component was not carried out due to an internal decision11.

6 Foundation Faith and Cooperation (FEC) is a Portuguese non-governmental organization that supports the education sector in Guinea-Bissau since 2001. At this moment its implementing strategy is, among others, a national in-service teacher training Program - Programa Ensino de Qualidade em Português na Guiné Bissau. 7 From the information available, ESE-IPVC is supporting a 3 years bachelor degree on teacher training: program and training itself. 8 For more information see, for exemple Furtado, A. (2015) Planos curriculares e programas para formação inicial de professores do ensino básico. And Individual Consultant Report - Activity Report n° 05/2015. 9 For more information see UNESCO (2013) Guiné-Bissau - Relatório da Situação do Sistema Educativo. Margens de manobra para o desenvolvimento do sistema educativo numa perspetiva de universalização do Ensino Básico e de redução da pobreza. MENCCJD. Dacar, p.164. 10 For more information see Benavente, A.; Varly, P. (2010) Balanço de competências de docentes em exercício na

Guiné-Bissau. UNESCO/BREDA & MENCCJD, p.76. 11 For more information please see Project Progress Report from January 1, 2014 - April 15, 2015 and The Project Progress Report from March 24, 2015

17

3.2 Effectiveness

6. To what extent were the in-service and pre-service teacher trainings (grades 1 to 6) improved at policy level, at implementing level, and at the teachers’ level?

It is perceived that pre-service teacher training project activities did not have direct influence at the policy level, the implementing level, and the teachers’ level. However, this was not a project goal. Nevertheless this process seems to produce benefits that can be capitalized in the future. This stems from the fact that the products of the project activities can be a useful tool on pre-service teacher training certification framework, institutional memory, and proposal of a pre-service teacher training program. The in-service teacher training project activities seem to have had influence in the implementation level and the teacher’s level. Although it was not part of the project’s goals to have direct influence at the policy both pre-service and in-service teacher training left essential orientations, recommendations, and propositions which should be best utilized for policy reform in the teaching career as well as training.

From the data available in regard to the in-service teacher training 91 inspectors and inspectors/trainers from 11 regions benefitted from training in November 2014 during a 7 day-training on the competence-based approach and on the content of the 5 revised modules. In February 2015, 55 inspectors and inspectors/trainers among the 91 previously trained received training on the revised in-service teacher training modules. These inspectors/trainers training allowed the pilot test of the guides and modules. From 20 of July to 1 of August 2015, 298 teachers in 4 locations received in-service teacher training on the revised modules. More details in the table 1.

Furthermore, in October 2014 UNESCO collaborated with FEC on the training of 176 heads of classes and president of study commissions from grades 1 to 6 on (1) the basic education curricula cycles 1 and 2, (2) teaching and learning methodologies and (3) planning, as well as (4) UNESCO-ECOWAS manual on “Education for a Culture of Peace, Human Rights, Citizenship, Democracy and Regional Integration”.

In the in-service teacher training sessions held in July-August 2015, a randomized controlled trial was conducted by randomly distributing the participants between two groups (119 teachers control group and 155 teachers experimental group).

Table 1 - Number of teacher trained

Region Total number of

teacher trained

Number of female

teacher trained

% female teacher

trained

Setor Autónomo de Bissau 138 69 50%

Bula 93 27 29%

Gabú 46 14 30%

Buba 21 2 10%

Total 298 112 38%

Taking into consideration the data analysis report, it was observed a statistically significant increases in the total scores from the control group to the experimental group of all subjects (F(6,267)=6,83, p<0,001), with the exception of competence-based approach. However, when looking to the average scores, a significant positive effect is observed only in Expression, Mathematics and Psycho-pedagogy. In Integrated Science and Portuguese there is no significant effects and is observed a negative effect on competence-based approach. Nevertheless, we must take in consideration the duration of the training. Has stated in the study analysis report some improvements should be made to the questionnaire in order to improve future interventions data analysis.

18

7. To what extent were the national capacities reinforced through the implementation of this project?

National capacities were reinforced by the implementation of this project. This was made possible because the implementation agency was INDE, the modules were produced by national experts, the bilingual education team was from INDE, and the national officer was from Guinea-Bissau. It seems that the consultants that worked with the national experts were critical in this capacity development. The administrative support received from UNESCO Dakar office staff in order to effectively deliver the in-service teacher training contributed in addition to capacity development of INDE’s financial management.

8. Does the model used for the training of teachers on bilingual education prove to be useful to improve learning and is it scalable?

The effectiveness of the bilingual education was not assessed because the implementation of this activity in the selected schools did not start in November of 2014 but in 2015. The pilot phase took place in Bula, Mansoa, Safim and Setor Autónomo de Bissau (SAB) in the selected 5 plus 1 schools, with one experimental and one control class in each school. The 7 national experts were trained on bilingual education by the coordinator of this component, a manual was produced, 10 teachers in selected schools were trained by the experts and were provided with pedagogical materials, and the teachers were visited by the trainers (4 monitoring visits from May to June 2015).

9. What were the major factors influencing the achievement or non-achievement of the objectives of the project?

The Guinea-Bissau political instability, the limited communication means between Dakar and Bissau in particular when the new project officer was appointed (re-launch of phase II) and installed at INDE without electricity and internet connection, and the tardive budget allocation inherent to UNESCO’s administrative procedures seems to be the project main constraints. On the other hand, the commitment of the national experts, the mobilization and dialogue promoted by the national project coordinator (phase II) with the education partners and with the project team in Dakar, and the project team in Dakar were the major factors influencing positively the achievements of the project.

10. Did the project have effective monitoring mechanisms in place?

From the available data, an effective monitoring mechanism was put in place. The balance between distance and field presence seems adequate to the project implementation. Although at an early stage of phase II, it seems that the communication between Bissau and Dakar was an issue. This stems from the fact that the new project officer hired in phase II needed to adapt to the project demands and the absence of day to day contact difficult her adaptation.

11. Has the project been cost-effective, i.e. could the outcomes have been achieved at lower cost through adopting a different approach and/or using alternative delivery mechanisms?

It seems consensual among project stakeholders that were consulted that the project was cost-effective and the outcomes have been achieved at lower cost. The option for national consultants, the recycling of the project computer, and the reduction of the per diem initially predicted for the in-service teacher training are some examples that illustrate this aspect. In regard to the per diem, some of the national experts raised some concerns. According to them, the amount given to some teachers was not sufficient because of the longer distance that they had to travel to the training spot. However the data collected does not allow to have a full confirmation of the accuracy of this perception.

19

12. To what extent has the project been gender sensitive within its interventions?

In the project document forms there is no evidence that the gender issue are taken in consideration. However, during the project implementation this aspect was taken in consideration in the selection of the in-service teacher training participants and in the production of the in-service training modules and guides.

3.3 Sustainability

13. To what extent did UNESCO involve targeted beneficiaries (teachers, MoE, INDE, etc.) in the design, delivery, and monitoring of the project?

In regard to the involvement of the target beneficiaries in the project design, delivery and monitoring, there are several levels of participation. However it seems that the design (phase I) and the monitoring of the project only involved the project team. The revision of project document for phase II in order to contextualize and be more aligned with national priorities being put in place was completed after consulting the MoE and education partners. The MoE and INDE had a close involvement in the monitoring and in the delivery of the project, especially in phase II. The inspectors/ trainers were involved in the project delivery. In regard to the other target beneficiaries (teacher, inspectors/trainers) there is no evidence that they were involved in the project design, delivery and monitoring.

14. To what extent will national appropriation of the project results be ensured and more specifically by the Ministry of Education? In particular:

a. How the 5 developed training modules and guides for in-service teachers’ grades 1 to 6 will be used as reference at national level to deliver quality teaching in alignment with the curricular reform and the competence-based approach adopted by the Ministry? b. How the pilot in-service teacher training on the 5 revised modules and guides can be extended nationally to reinforce the capacities of all the in-service teachers’ grade 1 to 6? c. How the proposal of pre-service teacher training programmes will be used as a reference standard by the Ministry of Education?

It seems that the appropriation of the project results by the MoE is not ensured due to the recent (August 2015) dismissed of the government by the current President installing a political crisis. In this way the project team is waiting for the appointment of the new government to continue the appropriation of the actions completed under the project. However, the 5 training modules, the in-service teacher training and the pre-service programs have the potential to be appropriated by the MoE but several steps must be taken in order to achieve this. The 5 training modules are being revised (language, content, and competence based approach). These modules seem relevant with the current curriculum revision, however they should be revised when the curriculum reform is finalized in order to adapt them to finalized primary education programs.

15. To what extent will the benefits of national capacity reinforcement continue after UNESCO’s support ceased?

The benefits of the national capacity development will continue especially in INDE and in the regional delegations of the MoE. This stems from the fact that the national experts involved in the project are integrated in INDE and the inspectors/trainers will continue their functions in the regional delegations. However, due to the evaluation limitations we can’t assume that INDE and/ or the regional MoE delegations are now able of continue in a sustainable way the activities.

20

16. Are any potential donors/partners interested in maintaining/further building upon the project outputs of the components on pre- and in-service trainings and on bilingual education?

The project outputs have potential to be maintained and/or build up by other donors/partners. However, the project team should continue to establish the links with the relevant education stakeholders in order to achieve this. INDE institutional capacity development can be further supported by the Portuguese Cooperation that is in process of design a project that has this component. From the data available (interviews and documents analysis) it seems that the training modules could eventually be maintained and/or build up by UNICEF, African Development Bank, FEC, ADPP, among other education partners that have similar project activities to further develop the educative system. The pre-service teacher training has the potential to be maintained and/or build up by the ESE-IPVC. The in-service teacher training and the bilingual pilot education have the potential to be maintained and/or build up by UNICEF. Concerning bilingual education it seems that is recognized as important for quality education in regard the country multilingual context. However donors/partners are uncertain because of the absence of a political orientation in this regard by the MoE.

3.4 Efficiency

17. Have UNESCO’s organizational structure, managerial support and coordination mechanisms adequately performed in function of timely delivery?

In regard with the adequate performance of the UNESCO organizational structure, managerial support and coordination mechanisms, it seems that all performed well except some aspect concerning the timely budget transfer to the national experts, the in-service teacher training activities, the teacher qualification assessment and some consultants. It seems also that in the second phase of the project since the project team increased the project performance increased.

18. Could more activities and outputs have been delivered with the same resources?

From the available data it seems that this project have an overall budget allocation similar to other projects in the education sector in Guinea-Bissau (e.g. Portuguese Cooperation and FEC). In this regard the project followed the trend of the country

In relation to the costs associated to monetary compensations, during phase II they were provided according to the validated United Nations System Circular-GB. These compensations were mostly less than what was provided during phase I of the project. However, at no circumstances were the lower compensation (in relation to phase I) lacking justifiable authorization/decision. During phase I international experts were recruited to implement the project activities rather than national experts, who were privileged during phase II. Nevertheless, with regard to phase I, it is not possible to assess if there are national experts with the expertise/ skills necessary to conduct all the consultancies and diagnosis to support project activities and development. The recruitment of national experts during phase II might have slowed the implementation but by this way, national capacities were reinforced as well as promoted project ownership by the beneficiaries.

As a consequence, it seems that during phase I, if no time constraints, more activities and outputs could have been delivered with the same resources. Concerning phase II, it seems no more activities and outputs could have been delivered with the same resources. It also seems that human, financial, and material resources were utilized in the best way possible, in order to promote sustainability, ownership and capacity development of the project beneficiaries.

21

19. Has UNESCO been focusing on the areas where it is best equipped to intervene, while aligning/coordinating with other developing partners?

The project focused on areas that UNESCO is best equipped to intervene. The timing of the project implementation was ideal because the Guinea-Bissau education system since 2009 started important transformation with the approval of several laws and the start of the Global Partnership for Education project in the country. It seems also important to link in the future the project outputs with the qualification referential for basic education teachers in ECOWAS countries, currently being developed by UNESCO.

In conclusion, by taking in consideration all the aspects referred in this report we can considerer that the

project had an added value to the Guinea-Bissau education system. The project also reinforced the national

capacities and was adapted to the local context and consistent with the Guinea-Bissau and UNESCO

orientations. The only aspect that is not consistent with the Guinea-Bissau orientations is the pilot bilingual

education because so far the country does not have a policy/political position regarding the use of national

languages. However UNESCO worked closely with the MoE and INDE which finally granted support to the

pilot the bilingual education component during project implementation.

The project achieved 5 of the 9 expected outputs and partially other 2. Main project achievements are:

i) conducted a national assessment of in-service teacher from grades 1 to 9 academic skills

(Portuguese language and mathematics) with a response rate in the majority of the regions

higher than 90 %;

ii) conducted an assessment of the students’ academic skills from the 3rd year of the pre-service

teacher training institution from the 1st and 2nd cycles of basic education (grade 1 to 6), 17 de

Fevereiro;

iii) Conducted an assessment of the pre-service teacher training institution, Tchico Té, from the

3rd cycle of basic education (grades 7 to 9) and of the students’ academic skills from the 3rd

year of the pre-service teacher training institution Tchico Té;

iv) developed a study on a strategy for in-service teacher training from the 1st and 2nd cycle of

basic education (grade 1 to 6);

v) developed a proposal for pre-service teacher training programs;

vi) developed 5 modules and 5 guides for teachers within the competence based approach

harmonizing them with the modules produced by the African Development Bank;

vii) training of inspectors/ trainers from the Ministry of National Education regional delegation, on

the produced modules;

viii) training of 298 in-service teachers on competency based approach in 5 subjects (Portuguese,

Expression, Mathematics, Psycho-pedagogy and Sciences). A pre and a post-training test was

conducted, demonstrating a significant difference in scores of all subjects (F(6,267)=6,83,

p<0,001), with the exception of competence-based approach related question. However, when

looking to the average scores, a significant positive effect is observed only in Expression,

Mathematics and Psycho-pedagogy;

ix) contributed to INDE (National Institute for Educational Development) and Ministry of National

Education regional delegations capacity development;

x) the project supported the realization of the national exams in 2011; and

xi) the project coordinated and harmonized the activities with other education stakeholders.

22

The project succeeded to coordinate the proposed activities with relevant education stakeholders and can

be considered complementary to other interventions (e.g. UNICEF, FEC, AfDB).

The pre-service teacher training programs produced by the project had not yet been validated by the MoE

because of the recent political crisis which caused an emergency demolishment of the Government, and at

time of this evaluation no Minister of Education had yet been appointed. It was also proposed three

scenarios to establish a pre-service teacher training, certification, and degree system that should be taken

into consideration in order to respond to the challenges of the education system.

The project also contributed to the improvement of the in-service teacher training of 298 teachers (grades

1 to 6) from the Sector Autónomo de Bissau, Bula, Gabú and Buba regions. The teachers were trained in

themes regarding the modules produced with the project support and a pre-training and post-training test

was conducted to randomly assigning groups to assess training effects.

The studies produced by the project have the potential to improve the pre-service and the in-service

teacher training in terms of policy and in the improvement of the training system. However, they need to

be capitalized by the MoE and education partners/stakeholders in the near future in order to operate

and/or develop the education system and confirm their potential.

As to whether the project contributed to the MoE improving in-service and pre-service teacher training

systems and policies: this is under way. This stems from the fact that the project team is waiting for the

appointment of a new government, in order to continue mobilizing for an adequate appropriation of the

actions completed under the project.

The project outputs have potential to be maintained and/or build up by other donors/partners. However,

the project team should continue the appropriation and /or establish the links with the relevant

education stakeholders in order to achieve this.

4. Lessons learned

• Guinea-Bissau’s political instability requires flexible project planning;

• The capacity development approach (as opposed to unidirectional aid project without taking into account local capacities) seems more sustainable and produces more positive impact. This stems from the fact that it can improve the performance and skills of individuals, improve the performance of organizations and of the public administration; and improve the social, political and economic environment. However there is less measurable impact, due to the multidimensional process which depends on various means to achieve the desired results. This is divided into different levels, and is a long-term process that occurs through a series of changes, many of which relate to values, attitudes and norms and which are difficult, if not impossible to measure12;

• The fact that the national coordinators are from Guinea-Bissau seems to be an advantage;

• More harmonization and coordination are needed in the education sector taking further advantage of the Local Education Group;

12 For more information please see: De Grauwe, A. 2009. Without capacity, there is no development. Paris: UNESCO-IIEP. And/or Davies, Lynn. 2011. "Learning for state-building: capacity development, education and fragility." Comparative Education 47 (2):157-180. doi: 10.1080/03050068.2011.554085.

23

• All teacher training requires detailed planning and national coordination in order to not duplicate interventions and beneficiaries;

• According to the national experts and trainees more training days and more often training were

needed for the in-service teacher training. However the training period should be reviewed as it

was inappropriate given the raining season and harvest period.

5. Challenges

• The lack of validation by a Minister of National Education (the country experienced a unpredicted

and sudden disablement of the Government) of the pre-service teacher training program,

certification and degree system, delaying a progressive step towards the development of this

component for both the project and the country itself;

• Better capitalization of the studies produced by the project in order to operate and/or develop

the education system;

• Implementation of project activities by INDE due to/as a result of some difficulties in executing

the project especially linked to education partner’s trainings articulation, logistical and

administrative procedures associated to the organization of the in-service teacher training;

• Coordination and harmonization with ESE-IPVC in regard to the pre-service teacher training

activities;

• Implementation of the bilingual education activities, particularly in regard to the absence of a

national position paper/orientations on bilingual education;

• The Guinea-Bissau political instability;

• The communication between Dakar and Bissau in particular when the new project officer was

appointed (re-launch of phase II);

• Budget allocation inherent to UNESCO’s administrative procedures.

6. Recommendations

• UNESCO Dakar could determine ways forward to continue supporting Guinea-Bissau education system and MoE (e.g. INDE and regional delegations) based in the capacity development approach promoting a strategy for education quality and reducing the regional inequalities;

• UNESCO Dakar could determine ways forward to put more emphasis in the macro policy level because the education system can be characterized by inconsistencies and discontinuities at this level;

• UNESCO Dakar could determine ways forward to support the MoE in articulation with other education stakeholders, in the development of teacher professional development strategy (pre-service and in-service)13 based in the Lei de Bases do Sistema Educativo [Education Act], the

13 This strategy should include the pre-service and the in-serve teacher training. For more information see for example Day, C.; Sachs, J. (2010) International handbook on the continuing professional development of teachers. Berkshire: Open University Press. And UNESCO (2014) EFA Global Monitoring report 2013/4. Teaching and learning: Achieving quality for all. Paris: UNESCO.

24

Estatuto da Carreira Docente [Teacher Careers Statute], the national curriculum, the challenges posed by 9 years of compulsory schooling, and the findings from the studies carried out by this project;

• UNESCO Dakar and the MoE should determine ways forward to capitalize on the project materials and studies produced in order to use them to operate or develop the education system;

• UNESCO and the MoE could determine ways forward for the launching within the MoE the establishment of a harmonization and coordination mechanism for education interventions;

• UNESCO Dakar and the MoE could develop a position paper/orientations on bilingual education developed as a strategy for quality education;

• UNESCO Dakar and the MoE could determine ways forward to support the implementation of the pilot bilingual education in partnership with other education stakeholders as a strategy for quality education/learning. The results from bilingual education component supported efforts done in the past by FASPEBI (Fundação para o Apoio ao Desenvolvimento dos Povos do Arquipélago dos Bijagós) in Bubaque and Catió. It is recommended, however, that future interventions should continue in order to pilot this strategy in other regions of the country. In the future more schools/classes from Setor Autónomo de Bissau should be included (this would evidence more impact due to the concentration of the country’s population (25.1%);

• UNESCO Dakar could determine ways forward to support the MoE in the further development of the national examination system as a tool for national planning, monitoring and management for the quality of learning;

• UNESCO Dakar and the MoE could determine ways forward to support the development of training modules and guides for grades 7 to 9;

• UNESCO Dakar could determine ways forward to assist the MoE in order to further develop the regulation of existing education legislation concerning specific aspects about national exams, bilingual education, school records, teacher recruitment, among others;

• UNESCO Dakar could determine ways forward to finding non-monetary compensation strategies and/or include some of the project activities in the annual plan with MoE budget allocation, in order to promote the participation of target beneficiaries in activities when external funding is not available;

• UNESCO Dakar should determine ways forward to improve the monitoring mechanisms in order to have more measurable data and generate evidence;

• UNESCO Dakar should determine ways forward to improve the project indicators following, to the extent possible, the SMART approach (specific, measurable, achievable, relevant and time-bound);

• UNESCO Dakar should determine ways forward to continue the adoption of a flexible planning in order to be suitable to Guinea-Bissau context;

• UNESCO Dakar could determine best ways to improve internal administrative procedures specially inherent to budget allocation;

• UNESCO Dakar should include an explicit gender sensitive approach since the project design/planning stage.

25

7. Annexes

7.1 Project expected results and the achievements

Table 2 - Project expected results and the achievements phase I

Phase Expected results Indicators Progress/achievements Observations

Phase I

1 - Evidence based teachers’ qualification

Number of teacher in service assessed

6328

- 1 report produced in 2010 - Balanço de competências de docentes em exercício na Guiné Bissau

Results of the assessment of teacher’s’ qualification

- More than 90% of the in-service teachers were assessed;

- The assessment allowed the creation of 3 teachers profiles that can be taken in consideration in the definition of a national network of in-service teacher training;

- Is necessary that the in-service teacher training in the future be certified.

Phase I

2 - An in-service teachers’ training program for primary and secondary education. More teachers trained with better qualifications.

Number of teachers to be trained

n/a - Produced document: - Vers une politique et une stratégie nationales pour la formation continue des professeurs de l’enseignement fondamental en Guinée-Bissau. (2011) About 400 teachers were trained on module I (see evaluation report phase I)

Number and popularity of usage of the training modules and tools

n/a

Assessment of trained teacher qualification

n/a

Phase I

3 - New programs for teachers’ trainees in primary and secondary education. Building of a pre-

Content of new training programs

- New training programs were proposed.

- 2 reports produced in 2010 - Apoio à elaboração do novo currículo de formação inicial de professores do ensino básico

26

service training, defining the contents. More teachers trained with better qualifications.

(ensino primário e secundário geral) na Guiné-Bissau. Volume I and II

Selection of the students n/a

Phase I

4 - Diagnostic of the pre-service training system of Secondary School (including the students’ achievement). Development of curriculum, Management of resources

Results of the assessment of the existing institutes of

training - 3 studies produced

- Avaliação da Formação Inicial de Professores do Ensino Secundário da Guiné-Bissau - Volume I. Avaliação Institucional e Pedagógica (2010)

-Avaliação da Formação Inicial de Professores do Ensino Secundário da Guiné-Bissau - Volume II. Avaliação dos Alunos-Professores (2010)

- Avaliação dos Alunos-Professores - Avaliação dos Alunos do 3ºAno da Escola de Formação de Professores 17 de Fevereiro (2010)

Selection of the students 263 students assessed

Availability of a professional development

scheme n/a

Phase I

5 - Support to the implementation of national exams. Support to use these data as fact of a result based system for quality od learning management and monitoring

Nomination of a team in charge of the exams

The team was nominated

- The project supported the implementation of the national exams held in 2011 for grades 9 and 12. Because the National Exam Centre established by the MoE did not functioned appropriately and the political instability this result was achieved partially, beyond UNESCO control

Standardized exams produced

The exams for grade 9 and 12 were produced (2011)

Distribution and administration of exams

produced developed The exams were implemented (2011)

Data management procedures developed

n/a

Results diffusion path n/a

Phase I

6 - Development of local capacity and of management

Development of management practices in

training area and implementation of a

monitoring education quality process

n/a - Not available data to assess these aspects

Application of learning in MoE’s activities

n/a

27

Enables links and information diffusion between the different actors of the project

n/a

Table 3 - Project expected results and the achievements phase II

Phase Expected result Outputs Progress Observations

Phase II

1 - The Ministry of Education is able to design its strategy on the teaching profession development while aligning towards achieving the MDGs

A holistic diagnosis study on teacher issue is carried out

according to the methodological guide TTISSA

n/a

This result was not carried out due to an internal decision approved by

the Ministry of Education and by the donor

Phase II

2- Improvement of qualifications of primary and secondary teachers (in-service and pre-service training)

Training Modules for skills training of in service teacher for

Grade 1 to 6 improved

- The 5 training modules and guides are available and revision is under way concerning the competency based approach issues, linguistic accuracy and scientific content accuracy

In-service teachers skills Grade 1 to 6 improved

- 91 inspectors and inspectors/trainers (November 2014)

- 55 inspectors and inspectors/trainers (February 2015)

- In October 2014, 176 heads of classes and president of study commissions from grades 1 to 6 was also trained;

- In July/August 2015, 298 teachers received in-service teacher training on the revised modules

Reference Model for bilingual education in Creole and Portuguese developed

- Reference model for bilingual education produced

28

- 5 plus 1 schools were selected for the pilot phase;

- 10 teachers from the schools were trained;

- The teachers were visited 4 times by the trainers (from May to June 2015).

Pre-service teacher training programs in Guinea Bissau

improved

- The validation of the pre—service teacher training programs is still under way. The project team is waiting for the appointment of a new government to continue an adequate appropriation of this output

Phase II

3 - First exam session realized and results are available for all the candidates

Capacity of decentralized staff to manage this exam session

built

n/a

This result was not carried out due the National Exam Centre

established by the MoE is not functioning

Material support provided to the decentralized structures

and to the Secrétariat National des Examens nationaux (SNE)

n/a

Phase II

4 - Capacities and resource management by the Ministry of Education improved

Capacity of the Ministry of Education strengthened in

support to the implementation of the Education Sectoral Plan.

- The capacities of the INDE staff were reinforced in the development of training modules, in the organization of in-service teacher training, and on bilingual education.

- The regional inspectors and inspectors/trainers capacities were reinforced in the competency based approach.

Phase II

5 - Project implemented with quality and in a timely and cost-effective manner

The project is monitored and evaluated to ensure quality of

implementation and achievements

- The project was monitored combining progress reports, field visits and distance monitoring by project team in Dakar and permanent monitoring by national project officer.

- There are available monthly reports, annual progress reports and phase I evaluation reports

A brochure with the project main achievements, lesson learnt and recommendations are being prepared to be published by the end of the project.

29

7.2 People interviewed

• Mrs. Akemi Yonemura - UNESCO Office in Dakar project team (project supervisor)

• Mrs. Catherine Collin - UNESCO Office in Dakar project team (project consultant as project

manager)

• Mrs. Marise Évora Sagna - UNESCO Office in Dakar project (project consultant for technical

backstopping);

• Mrs. Marcelina Santos Ba - Project national coordinator (phase II);

• Mrs. Cristina Brugiolo - UNICEF Guinea-Bissau chief of education;

• Mr. Amadeu Badinca - National expert (mathematics);

• Mr. Domingos Mandau - National expert (sciences)

• Mr. Mateus Ialá - National expert (didactic/pedagogy)

• Mr. Fernando Demba Baldé - National expert (expressions)

• Mr. Alexandre Furtado - National consultant (technical assistant);

• Pe. Luigi Scantamburlo - Bilingual Education consultant;

• Mr. Fábio Sousa - Portuguese Cooperation Country Office Coordinator;

• Mr. Mamadu Saliu Djassi - MoE DGPASE director.

7.3 Reviewed documents

Table 4 - Reviewed documents

Year Subject Author Title

n/a Project document phase I n/a Phase 1 Project Document-Jan 2009-Dec 2010

n/a Project document phase II n/a Phase 2 Project Document-400 000USD

n/a Project document phase II revised

n/a Phase 2 Project Document_revised_180815

n/a Project document Marise Sagna

Le tableau comparatif des modules

élaborés, harmonisés et revus par

l’UNESCO – projet Amélioration de la

qualité de l’enseignement et par la

BAD – Education III

1994 PhD Thesis Carol Benson

Teaching beginning literacy in the ‘mother tongue’: A study of the experimental Kiriol/Portuguese primary project in Guinea-Bissau.

2009 Report Bártolo Campos; Alexandre Furtado.

Política Docente na Guiné-Bissau. Banco Mundial.

2009 MoE document MoE Despacho n.º CE/OM/2009

2009 Report Ana Benavente 1er Rapport De Mission - Du

20.11.09 au 12.12.09

30

2009 Report Pierre Varly

Rapport De Mission Du 21 Novembre

Au 12 Décembre 2009 Guinée-Bissau

- Evaluation des compétences des

enseignants – Unesco Breda

2010 Phase I mid-term evaluation report

Patrick Nkengne Nkengne

Rapport Evaluation Mi-Parcours

Projet UNESCO Guinee-Bissau

v4_version finale

2010 Progress report Jean-Mathieu Laroche

Progress reports for Italy_mars2010_final

2010 Report MENCCJD Three-year Plan for the Development of Education: 2011-2013

2010 Journal article Carol Benson

How multilingual African contexts are pushing educational research and practice in new directions. Language and Education, v. 24, n. 4, p. 323-336, 2010/06/07 2010. ISSN 0950-0782.

2010 Journal article Carol Benson Real and Potential Benefits of Bilingual Programmes in Developing Countries.

2010 Report Ana Benavente Pierre Varly

Balanço de competências de docentes em exercício na Guiné Bissau

2010 Report

Maria antónia Barreto Alzira Saraiva José Brites Ferreira

Apoio à elaboração do novo currículo de formação inicial de professores do ensino básico (ensino primário e secundário geral) na Guiné-Bissau. Volume I

2010 Report

Maria antónia Barreto Alzira Saraiva José Brites Ferreira

Apoio à elaboração do novo currículo de formação inicial de professores do ensino básico (ensino primário e secundário geral) na Guiné-Bissau. Volume II.

2010 Report Bártolo Campos Avaliação da Formação Inicial de Professores do Ensino Secundário da Guine-Bissau

2011 Report Max Butlen Christian Martinez

Vers une politique et une stratégie

nationales pour la formation

continue des professeurs de

l’enseignement fondamental en

Guinée-Bissau.

2010 Report Ana Benavente

Rapport de mission en Guinée

Bissau pour l’Unesco du 11 avril au

1er mai 2010 - Rapport sur la

formation des opérateurs de terrain

31

et la passation des tests et

questionnaires

2010 Report Pierre Varly

Rapport de mission pour l’UNESCO

en Guinée Bissau du 11 avril au 1er

mai 2010 - Appui aux opérations de

terrain/ Mise en place de la saisie

2010 Report Glória Ramalho Avaliação dos Alunos do 3ºAno da Escola de Formação de Professores 17 de Fevereiro

2010 Report Glória Ramalho

Avaliação da Formação Inicial de Professores do Ensino Secundário da Guiné-Bissau - Volume II. Avaliação dos Alunos-Professores

2010 Report Bártolo Campos

Avaliação da Formação Inicial de Professores do Ensino Secundário da Guiné-Bissau - Volume I. Avaliação Institucional e Pedagógica

2011 Phase I evaluation report François Robert, BISSAUPROJETQ-EVALF-101011

2011 Progress report Jean-Mathieu Laroche

Progress reports for Italy_mars2011

2012 Progress report Jean-Mathieu Laroche

Progress reports for Italy_mars2012

2013 Progress report Akemi Yonemura 534GBS1000 report 2013_approved by donors

2013 Report MENCCJD

Guiné-Bissau - Relatório da Situação

do Sistema Educativo. Margens de

manobra para o desenvolvimento do

sistema educativo numa perspetiva

de universalização do Ensino Básico

e de redução da pobreza.

2014 National Consultant report Alexandre Furtado

Actions développées par le

consultant entre 1 à 30 Novembre

2014

2014 Consultant report Maria José Nóvoa BREVE RELATÓRIO FINAL 1ªETAPA

2014 Progress report Akemi Yonemura, Guinea-Bissau Progress Reports for Italy Amendment FNL_2014_Oct_1

2015 Progress report Akemi Yonemura

Project Progress Report Guinea-Bissau Teacher Training Dakar Jan_2014_Apr_2015_2015 04 15_final

2015 5 modules and 5 guides National experts Didactic/pedagogy, Portuguese, Expression, Integrated Science, Mathematics

2015 Bilingual education materials Luigi Scantamburlo

Curso para os professores 2015

32

2015 Report on pre-service teacher training

Marise Sagna, UNESCO consultant review recommendations_MSagna_120815

2015 Curricular plans and revised programmes

National experts

Planos curriculares e programas para

formação inicial de professores do

1º-3º CICLOS 2

2015 INDE INDE Implementação da experiência piloto do ensino bilingue

2015 Individual Consultant Report Marise Sagna, Activity Report n° 05/2015 Period: 01-31 august 2015

2015 National experts report

Amadeu Badinca Domingos Gomes Domingos Mandau Mateus Ialá Fernando Demba Baldé

Seminário de formação de formadores em módulos e guias elaborados em pedagogia de integração – APC

2015 National experts report

Amadeu Badinca Domingos Gomes Domingos Mandau Mateus Ialá Fernando Demba Baldé

Relatório das atividades dos experts dos meses de maio/ junho de 2015

2015 Mission report Marise Sagna Mission Report - 02 - 06.02.2015

2015 Project document Xinxin Yu Data Analysis Report

2015 National Consultant report Alexandre Furtado

Principales actions développées pendant la période du 1 Janvier au 10 Février 2015

2015 National consultant report

Luigi Scantamburlo e Grupo de Trabalho

Relatório Final (Bilingual Education)

2015 National consultant report Alexandre Furtado

Planos curriculares e programas para formação Inicial de professores do ensino básico

2015 National consultant report

Luigi Scantamburlo e Grupo de Trabalho

Relatório Final - Projeto Piloto para o Ensino Bilingue Crioulo Guineense - Português

2015 Logistics Evaluation Report Marise Sagna

In-service Teacher Training on Competence Based Approach- Bissau, Bula, Buba, Gabu 20th July to 01st August, 2015

33

7.4 References

1. MENCCJD, Pedido Financeiro ao Fundo da Parceria Global para a Educação. 2011, Ministério da Educação Nacional, Ciência, Juventude e Desportos: Bissau.

2. Ministério da Economia Plano e Integração Regional, Segundo Documento de Estratégia

Nacional de Redução da Pobreza. 2011, Ministério da Economia, Plano e Integração Regional,: Bissau.

3. MENCCJD, Eléments de Diagnostic du Système Educatif Bissau-guinéen: Marges de

manœuvre pour le développement du système éducatif dans une perspective

d’universalisation de l’enseignement de base et de réduction de la pauvreté. 2009, República da Guiné-Bissau - Ministério da Educação Nacional: Bissau.

4. UNESCO Institute of Statistics. UIS statistics in brief. General profile - Guinea-Bissau. 2011 [cited 2013 december 30]; Available from: http://stats.uis.unesco.org/unesco/TableViewer/document.aspx?ReportId=124&IF_Language=eng&BR_Country=6240&BR_Region=40540.

5. Monteiro, J., A Educação na Guiné Bissau: Bases para uma estratégia sectorial renovada. 2005, República da Guiné-Bissau - Ministério da Educação Nacional: Bissau.

6. Observatório da Língua Portuguesa. Falantes de Português Língua Materna. 2009 [cited 2009 18 Dezembro]; Available from: http://www.observatoriolp.com.

7. Benson, C., How multilingual African contexts are pushing educational research and practice

in new directions. Language and Education, 2010. 24(4): p. 323-336. 8. O’Regan, D. and P. Thompson Advancing Stability and Reconciliation in Guinea-Bissau:

Lessons from Africa’s First Narco-State. ACSS Special Report No. 2, 2013. 58. 9. OCDE Resource Flows To Fragile And Conflict-Affected States - 2008 Annual Report. 2009.

108. 10. Banco Mundial, Global Monitoring Report 2007 - Millennium Development Goals:

Confronting the Challenges of Gender Equality and Fragile States. 2007, Banco Mundial: Washington. p. 249.

11. Sousa, J.S., Guiné-Bissau: A destruição de um país. Desafios e reflexões para uma nova

estratégia nacional. 2012, Coimbra: Author. 12. INEE, Minimum Standards for Education: Preparedness, Response, Recovery 2010, New York:

INEE. 13. Brannelly, L., S. Ndaruhutse, and C. Rigaud, Donors’ Engagement: supporting education in

fragile and conflict-affected states. 2009, Paris: UNESCO-IIEP,CfBT Education Trust.