1

Cleen environmental education endpoint evaluation

Draft Report

2

Preface

The submitted report on the final evaluation of the UNICEF project CLEEN, is the result of a

number of interviews, field visits to the project sites, discussions with beneficiaries and

stakeholders and reviews of project documents; therefore, what is presented here is the result of

information collected through a number of questionnaires as well as a cumulative opinion with

stakeholders.

The main terms of reference focuses on the evaluation of the completed projects in order to

provide UNICEF and other donors with specific information as to the impact of the project.

NACSS is pleased to report that the field evaluation team concludes, overall, that the

contribution of this project given to the communities of Korca, Lezha, Shkodra, and Tirana, is

deeply appreciated by the beneficiaries, teachers, parents and community members.

NACSS wishes to acknowledge with thanks the kind assistance of the UNICEF staff and the staff

of the NGOs , the interviewers and supervisor, and beneficiaries of the project sites in providing

3

much help and collaboration which the field evaluation team requested for the successful

completion of this assignment.

Geographical coverage: Tirana, Lezha, Shkodra, Korca

Period of evaluation: March 2009- July 2010

Preparation date of the report: December 2010

Composition of the evaluation team: NACSS Team

Composition of Field Work Team (Supervisors): Dhurata Bushati, Violeta Tici, Anila Alliu,

Albina Gjinaj

List of Acronyms

CtC Child to Child approach

COMBI Communication for Behavioral Impact

EE Environment Education

EO Education Office

EDEN Environmental NGO

CDE Center for Development of Education ( previous Institute of

Development of Education ICT )

4

Development of Education ICT )

FGD Focus Group Discussion

MoES Ministry of Education and Sciences

MEFWM Ministry of Environment, Forestry and Water Management

RED Regional Department of Education

ICT The Institute of Curricula and Training

PWG Project Working Group

UNICEF United Nation Children Fund

RED Regional Education Department

TABLE OF CONTENT

LIST OF ACRONYMS 3

1. BACKGROUND TO THE EVALUATION 5

1.1 INTRODUCTION

1.2 BACKGROUND TO THE PROJECT

1.3 WHO BENEFIT FROM THE PROJECT?

1.4 PROJECT RESULTS (IMPACT-OUTCOMES-OUTPUTS AND ACTIVITIES)

5

2. EVALUATION DESIGN AND METHODOLOGY 7

2.1 SCOPE OF THE PROJECT EVALUATION (CLEEN)

2.2 SAMPLE PLAN AND THE SOURCE OF INFORMATION

2.3 DATA COLLECTION TOOLS

2.4 EVALUATION FRAMEWORK

3. FINDINGS FROM FINAL EVALUATION. 15

4. CHALLENGES AND OPERATIONAL CONSTRAINS 300

5. CONCLUSIONS AND RECOMMENDATIONS FROM THE FINAL EVALUATION 311

5.1 GENERAL CONCLUSION 31

5.2 CONCLUSIONS FROM THE ASSESSMENT OF THE CONTROL GROUP – SCHOOLS WITH NO COMBI AND

CTC APPROACH 31

5.3 CONCLUSIONS FROM THE ASSESSMENT OF THE EXPERIMENTAL GROUP- SCHOOLS WITH

ENVIRONMENT EDUCATION PROJECT AND COMBI 31

5.4 RECOMMENDATIONS 33

6. ANNEXES 315

6

1. BACKGROUND TO THE EVALUATION

1.1 Introduction

Albania is facing a mountain of environmental challenges that are affecting seriously child

welfare and particularly the right of children to a highest standard of health. Official reports

show that in urban centres and particularly in big cities the quality of air is alarming; wastewater

is not treated and regularly leaks into clean and fresh water system; solid wastes are not properly

collected and processed and contribute to air and water pollution. This situation is affecting

children in a several ways. Periodic outbreaks of infection diseases are affecting children (but

not only) due to drinking water contamination while many beaches that children frequent during

summer time are considered high polluted. Children are not only victims of these environmental

risks but also they are getting use to consider a polluted environment as normal and having

passive attitudes/behaviours towards actions that protect and preserve environment.

1.2 Background to the project

The environmental challenges can be broadly categorized into three areas for action: the need to

clean up, to protect, and to preserve the environment and its natural resources. To address these

environmental challenges, various regulatory mechanisms, fiscal incentives, investments in

infrastructure and other measures are being developed. However, underlying many of these

challenges is a lack of awareness and understanding of environmental issues, and especially

among children at the primary level. Until now there have been few initiatives to raise children’s

awareness and understanding of environmental issues, and to get them to participate in cleaning

up, protecting and preserving the environment.

It is this project’s aim to raise children awareness and understanding of environmental issues,

fostering positive attitudes and encouraging child-led environmental actions. The primary grades

offer an excellent opportunity to raise environmental awareness and understanding among

children, and to promote action. Teachers and principals are eager to find topics to cover in these

hours, including associated materials. Children themselves are open to new ideas and action, and

can have a positive impact on their family and community. Further, given that children make up

roughly one third of the population, it is vital for sustainable development that children’s

environmental awareness, understanding and abilities to act all be improved.

In order to achieve the outcomes, the project has four main strategies:

(i) the development of competency-based curricular objectives and the definition of active and

child-centred standards for curriculum delivery;1

(ii) demand-led capacity building for teachers and key education personnel in the delivery of an

environmental studies curriculum using active learning teaching-learning methodologies;

(iii) the development and supply of essential teaching-learning materials; and

(iv) mobilizing communities to support child-led environmental actions.

1 Throughout this proposal the phrase ‘active learning’ is used. It is important to understand that ‘action’ here does

not refer mainly to experiences which make learning fun (though this is important too). Rather it refers primarily to

experience which encourages children to think, work together and link learning in school to its application to their

lives outside.

7

1.3 Who benefit from the project?

Children of grades 1-5 have improved environmental awareness, understanding and abilities.

This way they will be important actors in safeguarding their environment, but also act as

catalysts for behaviour change in their communities.

Teachers are trained to improve their teaching methodology by using active learning methods in

which children have an opportunity to focus on topics of interest, to apply knowledge and

develop skills, and to take action. They also will be able to identify learning objectives, including

not only knowledge but the development of life skills, and to monitor individual progress.

Schools have more opportunities to use active learning methods as a tool to achieve better results

for their students. Also schools will have a chance to be real partner and contributor in

community life.

Communities are encouraged to support children’s actions for the protection of their

environment. This will contribute also to increase their awareness on environmental issues and

take the appropriate measures to clean up, protect and preserve their neighborhoods.

1.4 Project Results (Impact-Outcomes-outputs and activities)

� children have mastered the environmental competencies;

� have positive attitudes towards the environment;

� have taken some action to improve the environment in the school, home and community

in a teaching-learning environment which promotes those actions

� community institutions are open to child environmental action.

In figures, this means that:

� at least 80 per cent of the children in the targeted classes ( a minimum of 75,000 students

yearly )will have mastered the environmental competencies of the curriculum; have

positive attitudes towards the environment; and have acted to improve the environment in

the school, home and community;

� 475 schools are able to define their environmental objectives;

� 2394 (1944 during the first year and 450 July-October 2010) teachers in primary schools

trained and supported to implement the environmental studies curriculum;

8

2. EVALUATION DESIGN AND METHODOLOGY

During the midterm evaluation process, NACSS has contributed to refine the UNICEF proposed

methodology which can be used also during endpoint. It has also contributed to build capacities

for managing and facilitating a joint evaluation process within REDs; it has strengthened skills

for joint analysis of key evaluation concerns through a participatory process.

The endpoint evaluation process went through these steps:

1. Define purpose and scope of evaluation

2. Define institutional set-up: NACSS used the same team to complete the field work

3. Develop evaluation framework: Evaluation framework includes evaluation tools,

structured according to standard evaluation criteria; it helps to further define focus of

evaluation and identify evaluation methods

4. Elaborate detailed evaluation work plan: The evaluation work plan is detailed based on

the limited time, NACSS have in its disposal

5. Data collection and analysis: Retrieve existing data from the stakeholders’ records

involved in the project ; In-depth interviews with beneficiaries (children, teachers,

community members), Field visits for schools and classes observation, taking pictures of

schools, classes as evidence

6. Ensure that stakeholders have sufficient opportunities to contribute to the findings

7. Data Analysis and Synthesis of Findings: Participatory methods support joint analysis

8. Steering committee meetings to review findings: The steering committee members

comments on the draft findings presented by the evaluators

9. Feedback to wider group of stakeholders: Stakeholders consulted during the evaluation

must receive feedback on the findings

10. Identify good practices and lessons learned: Identification of good practices and lessons

learned will turn the evaluation into a true learning exercise.

11. Recommendations for further steps : Outcomes from the evaluation will inform further

steps; recommendations should be presented in a clear and concise way to match

decision-makers interest

The project has embarked on a process that required a series of inputs such as designed activities,

resources, strategies and methods. The process outputs are continually monitored and evaluated

in order to compare them with the established plan and adjust the process when such need

presents itself. It is hard to evaluate what is the project impact if you don’t know where you were

to start with. Knowing where you started, and documenting it, give to the implementers and

donors chances to measure the expected impact of the project.

9

Control Group methodology was adopted as appropriate for measuring outputs and outcomes.

Data collection tools follow simple formats and trace the progress of the project providing

opportunity for modification as the need arises. In order to make comparisons the CLEEN

project has planned a baseline as well as an endpoint evaluation.

Data from diverse stakeholders are crosschecked and multiple data collections tools as described

below will be used: FGD, Questionnaires, Checklists, classroom and school observation,

COMBI records and Pictures/photos. (See Annex 1, Evaluation Tools)

2.1 Scope of the project Evaluation (CLEEN)

Develop an end point evaluation study for the Cleen project based on a given framework of

indicators which has been priorly developed. More specifically the baseline evaluation will be

answering the following questions:

• Does the EE lead to greater awareness towards environmental issues among the children?

• Do the changes that EE brings about in knowledge, attitude and behavior make a difference

in the daily life of children?

• Does the CtC approach in EE help them to feel more confident in knowing how to work on

environmental issues?

• Does the EE lead to more communication among children and between children and adults

regarding environmental issues?

• Does EE lead to more child-community action?

2.2 Sample plan and the source of information

The evaluating agency (NACSS), have selected the same sample of mid- term evaluation; 20

schools with and without project to submit to the base line evaluation. Some 15 schools

representing about 6 per cent of the total 251 that have been participating in the EE project and 5

schools without project constituted the sample to be studied. When selecting the schools, we

have observed the location and the size compatibility between experimental and control group of

schools (urban 11/rural 9). Detailed information about location and names of schools where the

project was implemented were provided by UNICEF and implementing agencies.

Table 1. Sample size

Schools No %

Experimental school COMBI&CLEEN 10 50

Experimental school CLEEN 5 25

Control Schools 5 25

10

Table 2. Selected Schools with and without project according to the regions

Experimental Schools Regions

COMBI & CLEEN CLEEN

Controlee schools

“Ali Demi” Primary school “Pal Engjelli” Primary school Baldushk -Primary school

“Emin Duraku” Primary school Luz i vogel -Primary school

Tirana

“Yrshek” Primary school

“Gjergj Kastrioti” Primary

school

“1 Maji” Primary school “Zejmen” Primary school Lezhe

“Gjerjgj Fishta” Primary

school

“Barbulloje” Primary school

“Ndre Mjeda” Primary school “Koplik” Primary school “Zogaj” Primary school Shkoder

“Gruemire” Primary school

“Sevasti Qiriazi” Primary

school

“Lumalas” Primary school “Bulgarec” Primary school

“Ismail Qemali” Primary

school

Korce

“Demokracia” Primary school

Table 3: Sampling framework and sample size.

Type of schools No of

Schools

No of

respondent-

Principals.

No of

respondent-

Teachers

No of

respondent-

Children

No of No of

respondent-

Parents.

Experimental

schools 1 (Schools

with COMBI +

CLEEN)

5

5 25

Individual

Q’airres - 600

FGD - 150

75

Experimental

schools 2

(Schools with

CLEEN)

10 10

50

Individual

Q’airres -1200

FGD -300

150

11

Control group

schools

(schools without

project)

5

5 25 Individual

Q’airres - 600

FGD - 150

75

Data are collected by a team of 36 interviewers and 4 supervisors (1 for each region) appointed

for this purpose. The interviewers are selected in collaboration with REDs . They are trained for

1 week by NACSS staff. (See Annex 2 “List Evaluated Schools and Interviewers”).

2.3 Data collection tools

Data were collected by using the tools below:

1. FGD,

2. KDF Questionnaires

3. Checklists,

4. classroom and school observation,

5. COMBI records and

6. Pictures/photos

In Annex 2 is the d account of use of tools, respondent’s number and schools involved:

2.4 Evaluation framework

The M&E Matrix for the CLEEN CtC Project that indicates how results can be assessed using a

results based framework,

1

2

Table 4. Results, Indicators and source of data

Results

Indicators

Baseline

data

Expected

data

Endpoint

data

Source of data

Assessment of Outputs

% trainers who score 80%

and above on a standardized

test after the TOT training.

100%

2

100%

100%

Trained M

aster trainers are competent to

deliver and complete the CtC training to

teachers

% of trained master trainers

who conduct quality teacher

training according to plan

80%

100%

100%

Pre and Post M

aster Trainer Q’airre

Observation checklist on quality of training.

Observations during the Training sessions

delivery

Master Trainer Annual Plan with targets

% trained teachers who

score 80% and above on a

standardized test after the

teacher training.

Over

80%

100%

100%

Teachers have received timely, quality

training in CtC (six steps and active

learning), know and know how to use EE

resources (modules, workbooks, kit)

% of all teachers complete

at least three issues (air,

water, pollution, nature,

earth) with 8 to 10 sessions

in each issue or 24 to 30

sessions of EE each year

53%

80%

95%

Pre and Post Teacher Q’airre .

Teacher’s log (plan on how to use CtC and

active learning for each of five modules)

Classroom Observation Tool

Annual Plan

Principals, Deputy Principals and

Inspectors understand CtC approach

(four steps, active learning, linking

learning with life), resources available

% Principals, Deputy

Principals and Inspectors

trained in CtC approach

100%

100%

100%

Attendance at CtC training

Records of meetings and available plans

(School Annual Plan, Directorate Annual

Plan)

2 Note from the international expert Raiha Idrees who conducted the TOT training” All of the trainers were all capable and can be used as trainers. Those selected opted to be part of the

training and dem

onstrated exceptional perform

ance”

1

3

% of teachers conducting

meetings at school level for

CtC sharing

20%

3

80%

100%

Number of directorate level

teacher meetings for CtC

sharing

20%

80%

80%

Parents discuss with

teachers in parent’s meeting

at least once per trimester

on EE.

0%

50%

60%

learning with life), resources available

(modules, workbooks, kit) + COMBI

and know how to support and assess EE

implementation

Number of community

meetings with schools on

EE, CtC and

COMBI9(endpoint

indicator)

20%

45%

60%

Plan)

Record of waste collection for COMBI

Focus groups with

teachers/parents/community

% classes that received all

materials at the beginning of

the CtC program

100%

100%

100%

Albania contextual EE Five Modules

(final versions), children’s workbooks

and EE environment kit are available to

all schools timely and used

% classes that have used all

three – at least three

modules + workbooks +

environmental kit each year

56%

80%

95%

Record of distribution

Annual Plan

Teacher’s log/checklist

Trained Teachers use active learning

methods in EE and in teaching other

subjects

% teachers using active

learning methods mentioned

in EE

67%

85%

95%

Teacher’s log/checklist

Annual Plan

Classroom observation Tool

3 The school meeting have been measured through focus groups due to lack of written records in the school

1

4

% of teachers use at least

one active learning method

for EE mentioned in the

module in all crosscutting

subjects of math, language,

science.

37%

Over 60%

Over 80%

Assessment of Outcomes

% children who know 75%

and above of EE core

messages

16%

75%

Over 95%

% children who have

completed the workbooks

and have got them

corrected

by teachers

50%

75%

100%

% children who report at

least two changes in their

home or community in each

of the three EE issues

38

%

75%

80%

Children have competencies (know,

know how to and do) clean up, preserve

and protect the environment in school, at

home and in the community

% of children from each

class who participate in

school level activities

70%

90%

Over 90%

CtC and control

KDF Q’airred Quiz (CtC and COMBI)

Only CtC

Children’s workbooks

FGD children

Children Tools:

•

Children’s communication map

•

Children’s Drawings or written (Before-

After Tool)

Number of parents who

report children have

communicated EE

43%

70%

75%

Number of parents who

report children have

communicated on COMBI

0%

70%

70%

Families know about EE issues from

their children and have changed some

practices

Number of parents who

report two changes in the

home in each of three EE

issues

30%

60%

Over 70%

CtC and Control

FGD with parents (CtC and COMBI)

With CtC

FGD with teachers (parents reporting change

in knowledge and behavior)

School-parent meeting records/minutes

Children Tools

Communication M

ap

1

5

%of parents who are

involved in recycling

0%

60%

50%

% schools that score 80%

and above on School

Observation Checklist

43%

70%

85%

Schools have cleaner, greener

environment

% of schools that have

completed the School

Action Plan

46%

80%

88%

School Observation Checklist

School Action Plan

Before and After Photo Essay

Community

% Schools that have

environment related projects

with communities

33%

60%

70%

Record/Photo of Activities

Inclusion of EE in country

curriculum

Yes

Yes

Yes

Recognition of the Cleen

training from authorities

Yes

yes

Yes

Policy level

Support to EE practice in

schools

Yes

Yes

Yes

Review of curriculum

Evidence of Directorate/key stakeholder visits

to observe EE sessions/schools

Evidence of expansion to other schools in

Directorates/country

16

3. FINDINGS FROM FINAL EVALUATION.

3.1 Findings related to output 1: Trained Master trainers are competent to deliver and

complete the CtC training to teachers.

Indicators:

o % trainers who score 80% and above on a

standardized test after the TOT training

o % of trained master trainers who conduct

quality teacher training according to plan.

Source:

o Pre and Post Master Trainer Q’airre

o Observation checklist on quality of training.

o Observations during the Training sessions

delivery

o Master Trainer Annual Plan with targets

Based on the training plan as well as ToT Q’airres, 100% of the trained master trainers

conducted quality teacher training. Observation of training reveals that teachers were

confident with: a) Child-to-Child approach to teaching and learning, b) the methodology and

materials of CLEEN project, and c) the roles and responsibilities of key stakeholders

involved in the project.

100% of teachers demonstrated their understanding of the lessons and activities outlined in

the training manuals through doing demonstration lessons and using the methodology and

material outlined in the CLEEN project. This achievement has been widely identified by

class observation.

3.2 Findings related to output 2: Teachers have received timely, quality training in CtC (six

steps and active learning), know and know how to use EE resources (modules, workbooks, kit).

Indicators:

o % trained teachers who score 80% and

o above on a standardized test after the

o teacher training

o % of all teachers complete at least three

issues (air, water, pollution, nature,

earth) with 8 to 10 sessions in each issue

or 24 to 30 sessions of EE each year.

Source

o Pre and Post Teacher Q’airre .

o Teacher’s log (plan on how to use CtC

and active learning for each of five

modules)

o Classroom Observation Tool

o Annual Plan

100 % of teachers trained scored over 80% after post training test and self-evaluation

form.

The interviews and findings from questionnaires to assess the effectiveness of the training

programme show that it has fulfilled its objective; 100 % of teachers are confident with a)

Child-to-Child approach to teaching and learning, b) the methodology and materials of

17

CLEEN project, and c) the roles and responsibilities of key stakeholders involved in the

project.

Also the teachers demonstrated their full understanding of the lessons and activities outlined

in the training manuals through doing demo lessons and exercised model lessons using the

methodology and material outlined in the CLEEN project.

The evaluation finds out that another important element of the project is the process of

“cluster training” taking place firstly in Tirana. The goal of this exercise is improving

teaching quality according to cluster philosophy, and creating a possibility to better manage

the environmental education in the elementary school in five regions. School clusters, by

building accountability systems at the school level will help to improve local monitoring.

Special training has been delivered for the teachers of the cluster activities. Debate sessions

among teachers have urged for a series of recommendation on how to improve environmental

education and have helped to share the knowledge with other teachers.

The interviewees admit that the training has been one of the most important elements of the

project and has influenced notably in the quality of teaching. During the first and second

phase of the project have been trained around 2400 teachers (1.944 in the first three years,

approximately 648 teachers every year, and 450 in the last period July-October 2010).

The training is highly valued from teachers and from the principals for its high quality. The

trained teachers display a marked improvement in teaching compared to the other teachers

based on the observation of Teacher’s log and Teacher’s check list and the reports from

INCA.

The teachers brought a lot of their practical experience and knowledge in their presentations

on the active learning methods such as posters, puppets, songs, poem reading and pictures

that were frequently used

3.3 Findings related to output 3: Principals, Deputy Principals and Inspectors understand CtC

approach (four step active learning, linking learning with real life) resources available

(modules, workbooks, kit) + Combi and know how to support and assess EE implementation.

Indicators:

o % Principals, Deputy Principals and

o Inspectors trained in CtC approach

o Number of teacher meetings at school

level

Source:

o Attendance at CtC training

o Records of meetings

o FGD with DOE and principals

In the course of project implementation are trained 330 Principals, Deputy Principals and

Inspectors, in CtC approach for environmental education. The total number includes some

300 school principals and 30 representatives from the Regional Education Directories.

18

Principals, the deputy principals and the RED inspectors demonstrated their understanding of

objectives, methodology, the Child- to -Child approach, the materials of CLEEN project,

roles and responsibilities of stakeholders involved in successful implementation of the

program.

Processed data from “Focus Group Discussion” of school principals on their support and

cooperation with teachers, parents, and community regarding EE show that: 32% of cases

rate “good”, 38% of cases rate “Satisfactory” and 30 % of cases rate “poor”.

Administrative support and school infrastructure are important factors in the development of

Environment Education. School principals say that they are proud of the Cleen

environmental project in their school, and the major part of them, close to 75% are quite clear

on the impact of Environment Education in other subjects and in improving students

achievements in the whole. They support Environment Education issues and monitor the

school action plan periodically.

They emphasize that external support for teachers such as parents organizations, community

etc), is an important part of Environment Education program development. Even during the

second phase of the project this issue remains a critical part of the project.

With regard to the teacher meetings at school level it results that these meetings take place

more at an informal level rather than in a structured plan. The endpoint evaluation noted that

during the second phase of implementation of the project DOEs and school headmasters

together with teachers should find a formal mechanism to disseminate their knowledge to

other schools at a regular basis aiming to the extension of the Environment Education to

other teachers and students. In Lezha, it is done successfully. The teachers that had

participated in the first phase of training and had exercised the modules were used as a

training pool in Lezha.

General findings for school principals on their support and cooperation with teachers,

parents, and community regarding Environment Education.

19

3.4 Findings related to output 4: Albania contextual Environment Education Five Modules

children’s workbooks and Environment Education environment kits are available to all schools

timely and used.

Indicators:

o % classes that received all materials at the

beginning of the CtC program.

o % classes that have used all three – at least

three modules + workbooks + environmental

kit each year.

Source:

o Record of distribution

o Annual Plan, Teacher’s log/checklist

100% of teachers have received the training and are provided currently with all materials of

the Cleen package namely; the teacher’s manual, the student’s workbooks and the activity

kits.

100% of classes have used at least three modules of the teacher’s manual, workbooks and

environmental kit, over the implementation of the Environment Education program.

Materials used for Environment Education classes like the “Teacher’s manual” with five

modules (Water, Air, Earth, Nature and Pollution), “Students workbooks” and the “ Activity

Kit-bag” with 24 items inside, have played a vital role for the development of a

comprehensive educational programme and the improved effectiveness of Environment

Education in all schools where the project is conducted.

All teachers state that the teacher’s book is designed in a user friendly manner to help

teachers during the Environment Education class and that this textbook has brought essential

change in how Environment Education issues are treated in schools.

Children’s tools through exercises, illustrations, and concrete examples of way of thinking

show an attempt not only to bring awareness for behaviour change, but prepare the child

toward a green lifestyle.

Feedback from control group interviews (focus groups with teachers and principals where

the project is not applied), testify that materials and tools play a very important role in

Environment Education and their lack affects the quality of Environment Education greatly.

3.5 Findings related to output 5: Trained Teachers use active learning methods in Environment

Education and in teaching other subjects.

20

Indicators:

o % teachers using active learning methods

mentioned in Environment Education.

o % of teachers use at least one active learning

method for Environment Education

mentioned in the module in all crosscutting

subjects of math, language, science.

Source:

o Annual Plan, Teacher’s log/checklist

o Classroom observation Tool

Findings from the evaluation show that almost over 80 % of teachers use active learning

methods mentioned in Environment Education. The most applied methods are: experiments,

poster preparation, discussions, drawings, brainstorming, field excursions and also, though

on lesser frequency, drama on Environment Education topics. Some of the active methods

used are: “Air management and keeping the environment clean”, “The Water Cycle”, ”What

is Earth and why is it important”, (Lezha District), “Open Lessons” (Tirana District) etc.

Frequences of methods used in classes

47

63

35

23

6

62

41

43

9

69

64

69

0 10 20 30 40 50 60 70 80

experiments

preparing posters

discusions

drawing

excursions

theatres

Baseline Final

By monitoring classes during Environment Education sessions at all levels we find that 80%

of cases the teaching rates as “good”. Teachers taught Environment Education by referring to

the academic plan supplementing it with material prepared beforehand, interactive methods

were adopted in a large part of the duration of the class, pupils got involved during class, and

the teaching session was comprehensive and entertaining for nearly all children.

The Environment Education project has a notable impact on other subjects

Focused discussion groups with teachers show that in 52 % of cases Environment Education

has had an influence on other subjects as well. The teachers book (Environment Education)

has helped teachers in the subjects of Knowledge, Social Education and Albanian language.

Teachers say they have used part of the book and extracts to illustrate and supplement the

above subject and they have also practiced active teaching methods. A further around 55% of

teachers underline that Environment Education has encouraged an increase in children’s

interest in these subjects and it will likely have a further impact on their grades in the same

subjects. Through the Teacher log-book (check list) and teacher observation and reflections,

21

we also found out that, 80% of teachers have accomplished all steps of the check list ( refer

to school observation check list ), above 80% have accomplished 4 of 7 steps (actions).

Immediate Impacts of Environment Education

The table below shows the student assessment scores (for both control and experimental

schools) gathered for 5 modules during second phase is much higher compared to scores

gathered during baseline.

The active involvement of students in the classes’ activities tells for a better

understanding and behaviors of them towards the environment issues.

Baseline Final Scores gathered

COMBI &

CLEEN

Schools

Control

schools

COMBI &

CLEEN

Schools

Control

schools

76-100 Excellent 125 18 308 84

51-75 Good 276 37 693 199

26-50 Satisfactory 1042 374 421 251

0-25 Poor 357 171 378 66

Total 1800 600 1800 600

Source: Focused discussion group (students)

These data are related to focus groups discussion developed with children vis- a -vis their

involvement, contribution and behavior about class, school, home and community, it results

In predominant majority of the cases, children are involved in keeping the classroom clean,

contribute to make the classroom look more attractive, prepare posters, drawings, take care

that their classroom is green and participate regularly in Environment Education activities in

class. On the other hand they score unevenly in Environment Education activities in their

community/neighborhood, that bring about the different scores shown in the above overall

result.

In relation to the above mentioned indicators (keeping the classroom clean, contribute to

make the classroom look more attractive, prepare posters, drawings, take care that their

classroom is green and participate regularly in Environment Education activities in class),

children not part of the project (children from control group), display poorer results that vary

from 3 % excellent, 6% good, 62% satisfactory and 3% poor during baseline. But, it is

clearly shown the impact of the program in schools without project.

Good results found out in Focus Group Discussion with teachers on: (i) planning, (ii)

implementation of the program in the class, (iii) improved teacher-child interactions and

22

links among daily activities, (iv) increased teachers’ ability to make classes more interesting

for the students, show for comprehensive skills transmitted from trainers to the

teachers/principals/inspectors in the Environment Education project schools.

Findings found out during baseline and final evaluation process, regarding tools that

teachers of experimental schools use in classes.

Excellent Good Satisfactory Poor

Baseline 11 53 34 2 100% Planning

Final 16 61 19 4 100%

Baseline 8 49 39 4 100% Implementation in the

classroom

Final 12 60 25 3 100%

Baseline 0 29 43 28 100% Teacher-child

interaction

Final 11 42 29 18 100%

Baseline 0 29 51 20 100% Daily activities

Final 6 51 33 10 100%

Source: Focused discussion group (teachers)

Teachers state that they are aware of the importance of Environment Education vis-a-vis the

increased awareness and knowledge of children about the environment, how to apply

knowledge in practice, with the purpose of preserving and protecting the natural resources in

the proper way. Environment Education encourages children to act protectively, help and

protect the environment, guides them toward good conduct as well as aiding them change

unpleasant behavior concerning the environment. Motivate daily life skills and enables their

participation not just in the classroom, but school, home and community.

Almost all participating teachers believe in the effectiveness of changing children behaviour.

75% of teachers state that children communicate at home with their parents and other

members of the family, as they bring basic material from home to help them in classroom

activities. In this way they show with factual material how they practice at home what they

learn at school. Teachers judge positively teacher-child interaction in the use of active,

entertaining and creative methods that involve all children. The implementing of real and

concrete projects (hands-on) and attitudes practiced regularly (as for example, reducing water

use, water saving, recycling, classroom projects, field excursions) create teaching

experiences that are memorized and learned which serve as useful life skills in the long run.

Percentage of teachers according the scores they gathered regarding teaching methods

Baseline Final

23

Scores gathered Baseline Final

COMBI &

CLEEN

Schools

Control

schools

COMBI &

CLEEN

Schools

Control

schools

45-60 Excellent 8 0 28 4

30-49 Good 34 8 62 33

15-29 Satisfactory 39 43 8 31

0-14 Poor 19 49 2 32

100% 100% 100% 100

Source: Focused discussion group (teachers)

3.7 Findings related to output 7: Families know about Environment Education issues from their

children and have changed some practices.

Indicators:

o Number of parents who report children

have communicated Environment

Education.

o Number of parents who report children

have communicated COMBI.

o Number of parents who report two

changes in the home in each of three

Environment Education issues.

o Number of parents who report changes in

plastic recycling due to COMBI.

Source:

o FGD with parents

o FGD with teachers (parents reporting

change in knowledge and behavior)

o School-parent meeting records/minutes

o Communication map

o Children tools

Children of the CLEEN project are becoming active agents of change. They are helping the

community to understand that their attitudes towards environment issues are crucial for the

community wellbeing. In this section we are presenting the contributions children’s are

making to protect the environment

Changes of children attitudes on environment are visible. Parents can distinguish them

clearly. They are involved indirectly in the Environment Education program activities.

The general findings show that there is a large difference between opinions of parents of

children who take part in the Environment Education program and those who do not.

Comparing with the baseline where almost 42 % of parents, children of whom participate in

Environment Education activities, report positive changes in their children relating to

knowledge of the Environment Education program, the final evaluation shows that around

70% of parents report changes such as involvement in their children’s’ activities, increases

communication at home on Environment Education issues, etc. Not all parents have helped

their kids in preparing projects and finding materials for the class of Environment Education

etc, older sisters and brothers constantly have been involved.

24

Control group findings show that very few children recognize Environment Education as a

separate activity, for which reason they are involved little in activities that relate to

environmental issues.

Scores gathered by parents

Baseline Final Scores gathered

COMBI &

CLEEN

Schools

Control

schools

COMBI &

CLEEN

Schools

Control

schools

Excellent 0 0 16 0

Good 58 0 44 9

Satisfactory 42 39 33 51

Poor 0 61 7 40

100% 100% 100% 100%

Baseline evaluation shows that parents of children not included in the Environment

Education program have responded ‘satisfactory’ in the 39 % of cases and ‘poor’ in the 61%

of cases, regarding their or their children’s’ involvement in Environment Education activities

taking place in the community. They state also, they do not participate in recycling activities,

and because of their lack of awareness they haven’t been able to influence their children on

these issues. While the final evaluation found out that figures are increased with regard of

parents’ involvement, but still their scores (for excellent and good) do not reach 60%, while

in the control groups, figures are very low.

The Environment Education program helps children to improve their communication

with their peers, teachers, parents and community members

The communication map is one of the instruments, which measures the frequency of

communication of children with different groups in relation to environment school/studies

issues and other issues specified by children themselves.

Studies

Who do children

talk to: High

communication Medium

communication Low

communication Total No of

Children

Baseline/final

evaluation

BL F BL F BL F BL F

Parents 261 306 157 111 32 33 450 450

Siblings 48 208 354 192 48 50 450 450

Teacher 311 401 83 33 56 16 450 450

Friends 236 358 167 50 47 42 450 450

25

Neighbor/community 13 69 21 101 416 280 450 450

Environment Related

Who do children

talk to: High

communication Medium

communication Low

communication Total No of

Children

Baseline/final

evaluation

BL F BL F BL F BL F

Parents 308 316 102 106 40 28 450 450

Siblings 37 92 54 214 359 144 450 450

Teacher 142 342 289 89 19 19 450 450

Friends 92 321 248 94 110 35 450 450

Neighbor/community 14 85 18 55 418 310 450 450

Tables above show that CLEEN project have had a visible impact in communication of

children with their teachers, parents and siblings with regard studies and environment.

In the case of control group schools results showed that children, on the same issues

(environment) communicate frequently only with their teachers, whilst with other groups

they had mostly little and on occasion average communication.

3.8 Findings related to output 8: Schools have cleaner, greener environment.

Indicators:

o % of schools that score 80% and above on

schools observation checklist.

o % schools that have completed the School

Action Plan.

Source:

o Observation Checklist

o School Action Plan

The Environment Education project has had an immediate impact on improvement of

school environment.

A green school has as attribute both health and learning. The quality of a green school is

assessed by how it treats common problems such as air pollution, hygiene and sanitary

conditions, hazardous places for children etc., which can affect not only children’s health but

also teaching effectiveness. For this reason, the Environment Education is expected to

encourage the design of schools that prevent and neutralize problems that harm the

environment.

26

To reach conclusions regarding the schools cleaner greener environment the results of the

survey are measured by “Environment Education schools assessment, A checklist for a

environmentally Green School” tool.

The above tool bases the analysis of measurement of a green school regarding 3 components:

A) Classroom - Observation of all Environment Education Classrooms,

B) School building and surroundings - Observation,

C) Environment Education Implementation Meeting with Principal and Environment

Education teachers- check records and other documents.

From the observation results we conclude that regarding component “A”, 55% of the cases

are graded as “excellent”, 45% of the cases are graded as “good”; regarding component “B”

33% of the cases are graded as “excellent”, good and Satisfactory, while, re Component C ,

46% Excellent, 38% Good and 15% satisfactory.

Evaluation of "a green school" through 3 evaluation criteria

95 6

8

55

0

5 2

0

5

10

15

20

Classes observed Schools and neighbourhood EE implementation

Excelent Good Satisfactory Poor

It is important to notice that 0% is found as “poor” for all 3 evaluation criteria.

Schools with project have an annual plan

The observations of the records in the schools shows that all the schools (15 schools with

project) have an Environment Education annual plan, 80% of them have an Environment

Education annual plan with 24-30 completed Environment Education classes, the others have

completed less than 24 classes. The above indicator is measured through “School level

monitoring tools” section A-standard 1.

From a more detailed survey of the schools it resulted that, 60% (compare to 20% found out

in BL) of the teacher from the surveyed schools with an annual Environment Education plan

have justifiably completed the standards 2 and 3 that means “ Teachers log for each sub-

topics” and the accompanying checklist, placed in a Teacher’s Diary, as well as ‘Self

assessment for teacher’ signed by the principal.

3.9 Findings related to output 9: Influence of the project in Community.

27

Indicators:

o % Schools that have environment related

projects with communities*.

Source:

o FDG community members

o Record photo Essay

The seed grant project under Cleen have started during second phase of the project

implementation. According records of INCA there are 82 projects supported with a financial

contribution between 60.000 – 80.000 ALL; The criteria defined to receive a site grant from

each school are:

1. Have adopted a whole school approach,

2. Show participation of children,

3. Show possible parents community involvement,

4. Be creative idea,

5. Have a balance between activities and purchasing , etc

General findings of community members (other than parents of children involved in the

Environment Education program).

The focus group discussion with community members on their information about the

Environment Education school activities and on how many Environment Education activities

have they been part of (e.g. recycling waste, how many activities have parents played active

role, helping children involved in community action, in how many areas has community

changed behaviour because of children and Environment Education), reveal that around 50%

percent of the cases respond to ‘good’, 21% satisfactory and the rest relates to “poor” results.

28

3.10 Findings related to output 10: Policy level

Indicators:

o Inclusion of Environment Education in

country curriculum

o Recognition of the training by authority

o Support to Environment Education

practice in schools

o Expanded to more schools

Source:

o Review of curriculum.

o Evidence of directorate/key stakeholder

visits to observe Environment Education

sessions/schools.

o Evidence of expansion to other schools in

directorates/country.

Interview with ministries and key

stakeholders, financial contribution by

Ministry/donors for Environment Education.

The Environment Education project is relevant with the key issues that effect

environmental education in Albania.

As stated in the National Environment Strategy, ‘there is still a widespread lack of

understanding of the relationship between the actions of individuals and their impact on the

environment.’ In response, the Strategy recommends that environmental awareness is

developed at all levels, and it calls for the improvement of environmental education materials

within the school system4 and for the development of a complete package of environmental

modules, including teacher training, sector modules, and local projects within the

community.5

Box. 7

Environmental Education as a Cross-Curricular Direction.

“The primary direction of environmental education is to help students achieve a

lifestyle in harmony and conformity with the environment in order to be able to prevent

environmental crises from further deepening. While protecting nature, students will

become sensible to local environmental problems … Students should develop a

positive and future-interested vision, which would bring lifestyles in harmony with the

environment and a positive attitude towards it. (Students) will analyse the

consequences of social and economic activities on the Albanian environment. Students

should actively participate in protecting and improving the environment near them,

thinking globally about it. They should try to prevent harm to the environment and

collaborate for the treatment of environmental problems.” (Institute of Pedagogical

Studies, 2004. New Curriculum Framework for Pre-University Education. ICS: Tirana.

pp. 28-29.)

By designing and implementing the Cleen project, UNICEF and the partners involved,

respond to both strategic documents by developing a project which targets the awareness,

4 MEFWM, 2006. op. cit. Recommendation S10.3.8 5 Ibid, recommendation S10.4.5

29

education information as well as having the children perform as agents of change for their

families and communities

UNICEF and the donor has been successful in engaging government partners6 at the

highest level

o UNICEF has work in partnership with government institutions (MoES, ICT, RED of

Korca, Tirana, Shkodra and Lezha), and NGO-s involved in project implementation.

This has facilitated the sharing of resources and expertise as well as ensured that the

programme is supported by a wide range of stakeholders. UNICEF has acted as a

catalyst for partnerships empowering partners in their development of environmental

education and training programmes.

Inclusion in the country curriculum

o The project has worked closely with the Institute of Curriculum and Training in order

to include the EE for primary school, Cleen under the curriculum framework of the

ICS. A group of specialists have been working in order to achieve the above.

o The project through an agreement with the Ministry of Education and Science has

made possible that at the beginning of the school year a circular of the Minister of

Education and Science urges schools to tackle EE as part of their set aside hours.

Accreditation of the training

o Once the training package has been prepared, tested and implemented, the CDE has

accredited the Cleen training program, which is a positive step to increase the interest

of the teachers who are part of the project as well as to increase the demand by

teachers who are out of the project. The accreditation process will potentially help the

sustainability of the project as more teachers will request the training in order to

improve their job performance and salary increase. Lately the ICT has chosen Cleen

as one out of four training recognized by ICT that schools/DoE can purchase through

their training budgets.

6 Ministry of Education and Science; Ministry of Environment, Forestry and Water Management; Institute for

National Curriculum and Standards; Educational Center for Training and Qualification; National Centre for

Evaluation and Assessment; Faculties of Education; Environmental Non-Governmental Organizations.

30

4. CHALLENGES AND OPERATIONAL CONSTRAINS

NACSS team encountered a number of operational constraints and challenges. These include:

COMBI approach. One of the main operational challenges identified by the implementation

organization and evaluation team in some of the school is the lack of reliable scheduling of

the plastic collection by recycle enterprise or others (Roma organizations) contracted for that

purpose. Our analyses show that economic and business considerations are the main cause

leading to lack of scheduling. The quantity of plastics collected by each school is not

adequate for the recycling enterprises. The observations of NACSS show that the level of the

commitment of the school directors in not at the same level as that of the teachers and that is

shown also in the engagement that the school headmasters have in the Combi approach . In

general the headmasters are not active in finding solutions to the waste collected but rather

passively waiting from a response from the project. In some cases the head masters were

unsatisfied that the recycle enterprise, or municipality have not respected the schedule of the

collection.

The involvement of community in the project actions was not at a very high level. So we

recommend that principals and teachers of the schools with project should place more

attention and commitment toward the involvement of communities in joint environment

activities.

During the “Class observation”, it was observed that the final results of the last questions of

the list of observation, in some cases were not answered, while it was scored with 1 or 0

points in some other cases. That was happened due to difficulties that students of 1 and 2

grades had to answer these questions.

One of the constrains identified during evaluation process was the indifference shown by the

parents but often more from the community members. During the site visits or FG Discussion

organized with parents and community members, the team of evaluators faced difficulties to

collect them and make them speak the project.

31

5. CONCLUSIONS AND RECOMMENDATIONS FROM THE FINAL EVALUATION

5.1 General conclusion

The NACSS team is pleased to see that this project has come to a successful conclusion;

The project design is comprehensive, appropriate, and responsive to national and local

contexts. It is successful in linking overall project objectives to national priorities.

The project implementation supports the national priorities and made possible the

involvement of multiple stakeholders; it is sufficiently flexible to support the initiation and

functioning of diverse strategies within a range of MASH action programs.

5.2 Conclusions from the assessment of the Control group – Schools with no COMBI

and CTC approach

� “The qualifications of teachers”- From the survey it is evident that in schools without

project the qualifications of teachers in environmental related issues is insufficient,

� Environment related curricula the environment protection and development concepts

are delivered in a cross-curricular manner and integrated into different subjects without

the sufficient amount of time and attention. Moreover, there is a lack of auxiliary

environmental literature and illustration means, such as books, leaflets, magazines or

videotapes.

� “The activities organised in schools”. There are no or very few environmental activities

organized by school or community. There are few such activities in city schools (without

project), and almost none in rural areas. The link between the school and the

family/community on environmental issues is non existent in the control group.

� “Green schools and classes”. Most teachers and principals of the control schools are

committed towards environment education and development of environment activities.

However, extremely low remuneration, poor working conditions, and inadequate

facilities have inevitably eroded motivation and satisfaction to work with environment

within school and in the neighbourhood.

5.3 Conclusions from the assessment of the Experimental group- Schools with

Environment Education project and COMBI

� “Development of the curriculum” that is focused on the development of the teacher’s

guide and student working materials, is bringing positive changes in environment

education for primary school. So, there has been a substantial change in the approach to

environmental issues that has supported children in developing an integrated series of

environmental competences.

� “The teacher’s guide and student’s book” The concepts presented in the text are in

accordance with students’ needs, and the text is enriched with practical activities to

facilitate learning. The concepts are dealt with in a logical correlation, in both texts.

� “Capacity building program” had a huge positive effect as it targeted a wide array of

actors consisting of teachers, principals and representatives of the regional education

departments. Capacity building program has influenced enhancement of knowledge on

environmental issues, skills in imparting environmental education, and ways of

32

implementing environmental education programs in primary schools. It played a crucial

role in a teaching-learning environment which promotes active learning.

� “Community institutions” were opened to child environmental action. Community

mobilisation” skills, gained from the students during the project implementation, assist

them in carrying out environmental action plans that go beyond school settings.

� “Environment Education achievements” The baseline evaluation have documented that

Environment Education project has achieved the following: has lead to a greater

awareness toward environment issues, (i) has improved critical thinking skills of the

students through provision of new knowledge and communication skills on

environmental issues; (ii) has motivated students to become more engaged in the

environment issues in their classroom, school and community through promotion of

positive attitudes and environmental friendly behaviour; (iii) the project has encouraged

civic engagement and environmental stewardship of the community members.

� The Environment Education project stimulates the relationships and interactions

between learning processes and social behaviour. It is combining aspects of knowledge

such as those on natural sciences (physic, chemistry, geography, biology), with aspects of

social behaviours, such as community mobilization, community work, public health etc

that reinforces the effect of the Environment Education project. It is hands-on, student-

cantered, inquiry-driven, and relevant to students’ everyday lives.

� The Environment Education project has been important for having positive impact on

student achievement in core subjects. When it is used as an extracurricular subject,

environmental education has a measurably positive impact not only on student

achievement in science but also in reading, math, and social studies.

� The apparent change was palpable. In the questionnaire of interest groups in the final

assessment clearly seen that two of these groups: a) Parents, b) community were less

interested and involved in realizing how best this project. Therefore we think that in the

future should be made possible a more accurate planning for the inclusion of these groups

in environmental issues. Sensitization should be higher especially in those areas where

school facilities are closer to this interest and then take a wider range. Teachers should

give their help in supporting the cause of environmental protection by giving children

more tasks that require the help of parents or community to be realized.

� In some schools COMBI worked better due to its integration with various activities such

as social mobilization, awareness campaigns, advocacy, and business involvement,

achieving in this way clearly identified objectives;

� In the best examples COMBI communication methods yielded because it was planned

well and it was responsive to community needs.

� Besides deficiencies and time constraints, regarding COMBI component, the desire for

participating of the students was great. They engaged at the plastic collection activities.

Children loved more activities where they not only take information developed by the

teacher by teaching but also when they take part to the mini-projects that they

themselves have done, outdoor excursions, factory visits conducted in the recycling of

waste (a project conducted with Combi), etc.

33

5.4 Recommendations

� Outdoor environment learning should be adopted more often as this educational practice

usually helps children to exchange their knowledge and improve their skills in problem

solving, choosing, constructing, and exploring.

� The school principals should take a leading role in the seed grant implementation and

recycling activities.

� The positive impact made evident by end point evaluation suggests to central and local

authorities to support the expanding of the Environment Education project in other

schools.

� The focus on capacity building needs to be continued, as this would lead toward

sustainable results. The REDs should assume the leading role in making the Environment

Education project experience transferable to other schools and other local institutions.

Until this new endeavor takes root, UNICEF or some other public agency needs continue

to facilitate and encourage capacity building through TOT, publication and dissemination

of materials- and utilization of the capacities and experience gathered during the

implementation of the project.

� Attention should focus also on involvement of private schools in the Environment

Education project. The experience gathered during Environment Education project should

be disseminated and made known to private education institutions.

� Raising awareness of environment issues is an ongoing commitment. The positive impact

of the project and the model needs to be continuously shared and documented. This

would help that all the actors involved can assist in sharing the model to the other areas

and schools who have not benefitted.

� It is recommendable that schools should have detailed and up-dated records on

environment action plans, parent involvement, meetings and involvements of community

members, etc

� Although it might be costly for MOES to take over the whole project costs, efforts should

be made at least to include the reprinting of the text books yearly;

� Cleen modules and child to child methodology could be very well feeding into the

curricula of the faculties preparing teachers

� The option of staring the environmental education earlier in life (preschool) should also

be considered as the younger the children are the faster they can learn.

� EE and COMBI, has the potential to be more widely used. UNICEF should explore to

expand the use of them, building on the success of this project and recommendations of

this evaluation.

� Past EE successes have the potential to be scaled up; results withdraw through this Final

evaluation ought to be well documented and disseminate and lessons learned widely

shared.

� COMBI communication methods yield results, if it is planned well and if it is responsive

to people’s needs. For this purpose, we recommend that the evaluation findings and

recommendations should be shared with communities through open events organized

with stakeholders (business, local government and REDs).We recommend that COMBI

34

should include the Interpersonal communication, at the community level, in homes and

particularly at service point and with Local government.

35

6. ANNEXES

6.1 Annex 1. Evaluation tools

36

3

7

6.2 Annex 2. Sample and work plan

38