Pirjo Kännö 2004

Evaluation of General Education in Finland and in Helsinki

Girona 1.12.2005

Pirjo Kännö

Pirjo Kännö 2004

Contents of the presentations 1. part (presentation)• Brief facts about Finland and Helsinki and about

education in Finland• Framework for evaluation of general education in

Finland • Helsinki: Local level example of the evaluation of

general education (system view) – BSC;EFQM – school self-evaluation, school audits

• Education application of the EFQM (example of processes)

• Challenges of evaluation of education 2. part (workshop)• Brief facts about Etu-Töölö Upper Secondary School• Self-evaluation plan (EFQM) for school year 2005-

2006 in Etu-Töölö Upper Secondary School

Pirjo Kännö 2004

Brief factsFinland• parliamentary republic (1917)• member of the EU (1995)• population ab. 5,3 million• 2 official languages: Finnish and

Swedish• Education (age):

pre-school, voluntary (6) comprehensive school (7-15)post-comprehensive (16-19)-general upper secondary- vocationalhigher educationadult education

• Main providers of education:municipalities

• Nokia, PISA

Helsinki• Capital of Finland (1812)• Population 550 000

(capital region 1,2 million)• Finnish speaking 88%,

Swedish speaking 6,5%• 190 schools in Helsinki

comprehensives, upper secondaries for young and adults and 15 vocational institutions (165 maintained by the city)

• Helsinki City Education Department employs 5900 people (4200 teachers)

• Services for 70 000 students

Pirjo Kännö 2004Organization of Helsinki City Education Section

GENERAL EDUCATION DIVISION

Head of Division

ADMINISTRATION AND DEVELOPMENT

CENTREResearch and

Development Manager

SWEDISH EDUCATION DIVISION

Head of Division

YOUTH AND ADULT EDUCATION DIVISION

Head of Division

· Finnish Comprehensive Schools (112)

· Swedish Comprehensive Schools (23)

· Upper Secondary Schools (4)

· Upper Secondary Schools for Adults (1)

· Vocational Institutions (3)· Finnish Upper Secondary

Schools (14)· Upper Secondary

Schools for Adults (3)· Young People’s

Workshops (4)· Apprenticeship Training

Bureau

· Strategic Services· Financial Services

· Personnel and Legal Services

· Acquisition and Real Estate Services

· IT Services· Media Centre

HEAD OF EDUCATION DEPARTMENT

EDUCATION DEPARTMENT

EDUCATION COMMITTEE

Finnish Division Swedish Division

11

.10

.20

05

Pirjo Kännö 2004

Framework for evaluation of general education in Finland

• since 1999 a statutory obligation:each provider of education (including schools) must - evaluate the education it offers and its effectiveness - participate in external evaluation (criteria and objects determined by the Ministry of Education)

• responsibility of the development of local (and school level) evaluation is on the provider of education (municipalities)

• National Council for Educational Evaluation (2003) supports local evaluation

• no inspection system in Finland• no regular testing system to all students in comprehensive level

(quota based testing done by National Board of Education in core subjects)

• matriculation examination: the first national test to all upper secondary students

Pirjo Kännö 2004

Evaluation of general education in Helsinki

• Framework and principles gradually developed within 10 – 15 years

• Evaluation strategy of general education 2000 -2004

• The evaluation guidelines of general education 2005-2007

• Development work going on constantly

Pirjo Kännö 2004

The Purpose of Evaluation

is

• to provide and analyse information in order

• to support decision making

• to improve preconditions of learning

• to promote learning

• to support development

Pirjo Kännö 2004

I STRATEGICEVALUATION

Mission, Values, Vision, Goals and Strategic Priorities

Objectives and Measures

Follow up/Evaluation

Environment analysis

II CONTINUOUS EVALUATION

TOOLS

BSC

EFQM Plan

Do Check

Act

The Framework of Evaluation

Pirjo Kännö 2004

VISIONLearning, able/capable, civilized/educated and

well-being Helsinki.High quality and internationally

respectededucation services

Economy/finance/resources

Staff, learningand well-being

Effectiveness/service capacity/

customers

High quality education services and learning

Social inclusion and participationSustainable development

Economic balance

2004General Education

Processes

Pirjo Kännö 2004

Leadership Key Performance Results

ProcessesPeople

Policy & Strategy

Partnerships & Resources

People results

Customer results

Society results

Enablers Results

Innovation and learning

The EFQM (European Foundation for Quality Management) Excellence Model

Pirjo Kännö 2004

Mission, vision, values

Evaluationof the present

state-approach

-deploymentAssessment and

review* strengths

*weaknesses(points for action)

Focusing on themost importantpoints for action

Action plan for thedevelopment

measures to be taken

Evaluation of theprogress

(Self-)evaluation ofthe success of the

development measures

Pirjo Kännö 2004

I Strategic Evaluation

DEPARTMENT LEVEL STRATEGY DOCUMENTS

STRATEGY PROCESS SCHOOL LEVEL STRATEGY DOCUMENTS

•School Legislation/Government Development Programme of Education/National Guidelines for Curriculum/ Evaluation Programme of The Education Evaluation Council•Common basic information provided by the city

Environment analysis Common basic information provided by the city

City Guidelines for Curriculum

Action and Financial Plan

City Guidelines for Curriculum

Separate strategies

Result Budget

Mission, values, vision

Goals and strategic priorities (BSC)

Functional and financial objectives, measures and targets (BSC)

Curriculum

Action Plan

Separate strategies

Budget

Annual Report Follow-up/Evaluation Annual Report

Staff

Finance

Customer

Processes

Mission Vision

Pirjo Kännö 2004

II Continuous Evaluation and Development

EDUCATION DATA DATA IN ADMINISTRATION SCHOOL LEVEL DATA

Evaluation Practices/reports- Self-evaluation Reports (EFQM)- School Audit Reports- Result Discussions Agreements--Group Compensation Reports

- Curriculum Evaluation Report-

-Management Information System

Evaluation Practices- Self-evaluation of Action (EFQM)

-Result Discussions- Incentive Group Compensation- Other Compensation

- Management Information System

Evaluation Practices- Self-evaluation of Action (EFQM)

- School Audits-Result Discussions-Incentive Group Compensation- Other Compensation- Evaluation of the Curriculum- Management Information Support

Customer Results

- Service Capacity Survey (parents/students)

Customer Results- Service Capacity Survey- Customer Jury- Documenting Spontaneous

Feedback

Customer Results

- Service Capacity Survey (parents/students)

People Results- Work Welfare Survey-Personnel report-Professional knowledge/skills survey

People Results- Work Welfare Survey Professional knowledge/skills survey

People Results- Work Welfare Survey -Professional knowledge/skills survey

Key Performance Results- Finance Follow-up Reports

Evaluation of Student Achievement- Evaluation of Learning-to-Learn skills-- School Welfare Survey-Environment report

Key Performance Results- Finance Follow-up Reports

- Realization of Action Plans

Key Performance Results- Finance Follow-Up Reports- Evaluation of Student Achievement- Evaluation of Learning-to-Learn Skills-School Welfare Survey-Environment survey

Pirjo Kännö 2004

Self-evaluation of Schools

Schools:• are expected to use the EFQM model as a tool • send a summary of self-evaluation results to the

education department once in four years (not in action yet)

Education Department:• supports schools by providing quality training• prepares a technical device to facilitate self-evaluationCity Administration:• supports quality development in all sectors:

- the Mayor’s Quality Award- training

Pirjo Kännö 2004

SCHOOL AUDITS 1- increasing networking and learning between schools

- increasing information flow between education department and schools

• HCED• HCED

SCHOOLSCHOOL

SCHOOLSCHOOL

Pirjo Kännö 2004

School audits 2

• continuous development, not inspection• a forum for professional network• learning through mutual reflection• sharing of good practices and experiences• to give feedback to other schools and to education

department• carried out by a trained pair or a group of peer

auditors (head teacher&teacher)• prepared audit plan and ”check lists” on agreed

targets • auditors write a report• a summary of the reports is written

Pirjo Kännö 2004

Schools• are expected to partricipate in audits regularly (target:

each school has been audited at least once by 2007)Administrationsupports schools by• planning and preparing the focus of school audits every

year • choosing the schools and auditors if needed• training the auditors• having a summary report made of the individual audit

reports• informing about the results

steers the work of schoolsmakes use of the results

School audits 3

Pirjo Kännö 2004

year Number of audited schools

Number of auditors

Targets of the school audits

1997 4 External auditors

Evaluation areas of the Finnish Quality Award Criteria (MB -model) - a pilot

1999 22 40 Strategic priorities of the general education division

2000 14 36 Action planning process in school, involving students and stakeholders, staff leadership and development, follow up of student achievement results

2001 22 72 Action planning process and staff and organisation leadership/management

2002 23 60 EFQM criteria: staff ; partnerships and processes 2003 18 45 EFQM criteria: processes/curriculum design

process as an example 2004 20 41 EFQM criteria: results/ school welfare survey

results as an example and pastoral care in schools

Targets of school audits in 1997-2004

Pirjo Kännö 2004

Incentive Group Compensation

• Strategy based tool for management • The Balanced Scorecard model in use• Managed and coordinated by the city administrationSchools:• 10-14 schools involved (2004-2005)• staff commit themselves to agreed development initiatives (based

on the strategic priorities/BSC)• staff are entitled to extra payment (max 5% of the salary costs)• realization of development initiatives evaluated against agreed

criteria• evaluation carried out by peer auditorsAdministration• All units of the education department are involved

Pirjo Kännö 2004

Plan for the evaluation of action and development in schools 1

The plan is required as part of the school

based curriculum:

• the same city level principles used in all schools

• separate school level part

Pirjo Kännö 2004

Plan for the evaluation of action and development in schools 2

City level principles:• School level evaluation is part of the local evaluation and part of the

every day work in school• The purpose of evaluation is to support learning, support the work of

the school staff and to provide and analyse information in order to support decision making and development

• Starting points for the annual planning of action and finance in schools are the curriculum, the strategic priorities and functional and financial targets of general education and the annual report of the previous school year. The school also makes use of the available external evaluation results in its planning.

• In their self evaluation schools are expected to pay regard to the common evaluation practices of the city and their results

• Schools are expected to take part in national and local research/surveys, make use of the provided information in developing their work

• Schools are allowed to use their own evaluation methods and choose their own targets of evaluation in addition to the common evaluation practices of the city

• The head teacher and teachers of the school are responsible for the evaluation at school level

Pirjo Kännö 2004

Plan for the evaluation of action and development in schools 3

Instruction for the school level part

The school decribes its evaluation processes: the annual planning and

evaluation of action and finance and continuous evaluation and

development using the following grouping:

1. Strategic evaluation (the annual action plan (school plan)/financial plan/annual report –process)- description of the preparing process: who is involved and how- how the results and conclusions of evaluation are made use of in the annual planning process and development

2. Continuous evaluation and developmentThe school describes in the curriculum how the common evaluation practices and its own practices are taken into consideration in self-evaluation and how and when the common evaluation practices are carried out

Pirjo Kännö 2004

Education application of the EFQM model

• Will shortly (2005) be published by the Council of Educational Evaluation in Finland as a recommended tool for self-evaluation

• Contents:1. EFQM-model as an evaluation method2. Why self-evaluation and quality management in education?3. The fundamental concepts of excellence and the good quality in a school (3.1-3.8)4. The structure of the EFQM-model5. The criteria and the sub-criteria (5.1-5.9)6. Alternative ways to do self-evaluation6.1. ”Quick-evaluation” – the questionnaire approach6.2. Evaluation based on the description document of the present state of school in all criteria6.3. The combination of 6.1. and 6.27. Writing out descriptions7.1. General discriptions7.2. Discription of the enablers and the results criteria8. How to do evaluation: RADAR –logic in practice

more info: www.koulutuksenarviointineuvosto.fi; anu.raisanen@eval.jyu.fi, harri.ronnholm@eval.jyu.fi

Pirjo Kännö 2004

CUSTOMERS

Provider of education SchoolStaff

Planning processes •Allocating resourcesSchool network and provision Common principles (e.g. curriculum guidelines, annual-/work planning)Preparation for decision making

Development and support processes• Evaluation• Development initiatives• Learning- and work environment• Administration and finance processes• Teaching material and information services• Information technology services

Supplier services• School transport services• Restaurant services• Cleaning services• Real estate services

CUSTOMER

WHO

HAS

RECEIVED

THE

SERVICE

Execution processes•Teaching and guidingStudent intakePastoral care/support Student assessmentMorning and afternoon care of pupilsInteraction relationsInvolving students

Planning processes•Curriculum planning•Annual-/work-/development planning • Planning for teaching- and guiding arrangements • Planning for pastoral care/other support services

• Functioning of the work community, groups and individualsFunctioning processes

•Cooperation, communication, interaction

Examples of key and support processes to customers in education

Pirjo Kännö 2004

Challenges of Evaluation

• Building an Evaluation Culture as Normal Part of the Everyday Work

• Making Systematic Use of the Gathered Information

• Finding Ways to Facilitate Evaluation

• Building of Working Management Information Systems

• Better resources needed

Pirjo Kännö 2004

Etu-Töölö Upper Secondary School

• Medium sized non-gradedupper secondary school in the centre of Helsinki

• Ab. 430 students aged 16-19• 33 teachers• Mission: To offer our students a

wide general education with goodcommunication skills as well as skills in studying and acquiring knowledgeand a solid foundation for further studies. We emphasize the active role of the student as a learner and as a builder of his/her own conciousness in the changing world.

more info: www.tyly.edu.hel.fi

Pirjo Kännö 2004

Action and development plan of schools

Contents:

1. Mission of school/approval of the plan (by School Board)

2. Conclusions from previous year’s action plan

3. Functioning of the school in 2005-2006:general arrangements, courses offered, advisory services, pastoral care, special needs services, teaching of students with different language and cultural backgrounds, religious occasions, activities outside of school, collaboration with other institutions, international contacts, school clubs, school/parent relations, involving students (student council)

4. Development planThree year planDevelopment targets in 2005-2006 (one of them:self-evaluation of school culture and action by using the EFQM model as a tool)In-service training plan of teachers

5. Calculation of the teaching resources for the school year

Pirjo Kännö 2004

EFQM self-evaluation plan 1/2

Aims1. to do EFQM self-evaluation using the questionnaire/”quick

evaluation” approach2. to prioritize improvement areas on the basis of the self-evaluation

results and to make concrete development plansTasks and schedules1. with the help of the principal (in this case) to make the ”Senior

Management Team” familiar with the EFQM model (education application) and the questionnaire approach in SMT meetings Oct 2005 – Mar 2006

2. to have an evaluation session of 3 hours with the whole staffof 25-30 teachers in Apr 2006 a. orientation to evaluation and deviding into 4-5 groups led by the members of the SMTb. each group self-evaluates 1-2 areas (criteria) documenting major strengths and improvement needs and concrete development suggestions (maybe simple sub-criteria scoring on 1-5 scale)

Pirjo Kännö 2004

EFQM self-evaluation plan 2/2

Tasks and schedules continues

3. SMT meeting in April: going through the results of the evaluation, prioritizing the development targets and scheduling them

4. Staff meeting in May: approving the SMT’s proposal for the development targets for the following two school years 2006-2007 and 2007-2008

Future plans• After 2008 (or 2007) new self-evaluation, this time using the

evaluation based on the description document of the present state of school in all criteria (enablers and results), maybe also more rigorous scoring

Pirjo Kännö 2004

Thank you for your attention!

Pirjo Kännö

Principal

Etu-Töölö Upper Secondary School

Arkadiankatu 26

00100 Helsinki

Finland

pirjo.kanno@edu.hel.fi

www.tyly.edu.hel.fi