The school self-assessment methodology and impact on the system

Kaidi HolmVET expert, Estonia

Torino 2009

STEP 1 – self-assessment• Estonian VET Quality Award since

2003 2003 - 9 schools, 2005 - 11 schools, 2006 – 9 schools, 2007 – 10 schools, 2008 – 8 schools

Based on EFQM model8 principles of organisational excellenceSelf-assessment combined with

external evaluationVoluntary

2

Estonian VET Excellence Model is based on EFQM Excellence Model and is:

a tool for Self-Assessmenta way to Benchmark with other schoolsa guide to identify areas for Improvementthe basis for a common Vocabulary and

a way of thinkinga Structure for the organization's

management system

EFQM Model is a practical tool that is mostly used by educational organisations in self-evaluation of the organisation, but it is also a useful framework for external audits as well as a structure for educational organisation's management system.

The EFQM Model forces to examine organisation's functions, operations and results as a whole. Excellent educational organisations identify key customers and customer groups for whom educational services are provided.

Organisation clarifies current and future needs of customers and customer groups and develops services accordingly.

Organisations follow the satisfaction of customers and try to forecast the changes.

In promoting continuous improvement, the EFQM Model relies on few fundamental concepts or approaches : 4

Source: www.efqm.com

The Fundamental Concepts of the EFQM Excellence Model (8 principles of TQM)

Result Orientation Excellence is achieving results that delight all the school´s relevant stakeholders (this includes students, employers, teachers and the other people employed, suppliers and society in general as well as those with financial interests in the organisation)

Customer FocusExcellence is creating sustainable customer value.School has to have clear focus on the needs of current and potential customers

Leadership and Constancy of PurposeExcellence is visionary and inspirational leadership, coupled with constancy of purpose. The behaviour of an organisation´s leaders creates a clarity and unity of purpose within the organisation and an environment in which the organisation and its people can excel.

Management by Processes and Facts

Excellence is managing the organization through a set of interdependent and interrelated systems, processes and facts. Organisations perform more effectively when all inter-related activities are understood and systematically managed and decisions concerning current operations and planned improvements are made using reliable information that includes stakeholder perceptions

People Development and InvolvementExcellence is maximizing the contribution of employees through their development and involvement.The full potential of an organisation´s people is best released through shared values and a culture of trust and empowerment, which encourages the involvement of everyone.

Continuous Learning, Innovation and ImprovementExcellence is challenging the status quo and effecting change by sharing of knowledge and utilizing learning to create innovation and improvement opportunities.

Partnership DevelopmentExcellence is developing and maintaining value-adding partnerships. An organisation works more effectively when it has mutually beneficial relationships build on trust, sharing of knowledge and integration with its Partners.

Corporate Social ResponsibilityExcellence is exceeding the minimum regulatory framework in which the organization operates and to strive to understand and respond to the expectations of their stakeholders in society. The long-term interest of the organisation and its people are best served by adopting an ethical approach and exceeding the expectations and regulations of the community.

Why EFQM model? Systematic and holistic approach to

management of the organisation Good ground for self-assessment and

comparative analysis with others Represents unified and worldwide accepted

understanding about excellent organisation Balance between different parts and aspects of

the organisation Quantifiable and comparable results Inclusive to other approaches and models Flexible and adjustable Oriented to improvement and learning

EFQM Customer results

12

Excellent organisations comprehensively measure and achieve outstanding results with respect to their customers.

Customer results of the EFQM Model include the monitoring and follow-up of customer perception measures (6a) and performance indicators (6b) both of which are used to analyse, assess, interpret and improve the performance of organisation and to forecast the changes in customer perceptions and needs. 13

Customer perception measuresPerception measures describe customers’

perceptions of the educational organisation and its operations. The starting point for customer satisfaction is that customers’ experiences should equal or exceed their needs and expectations.

Surveys and interviews are carried out among groups of students, representatives of working life, teaching staff and support staff in the organisation. The satisfaction of customers bying services produced by students, such as student-run restaurant or shop, may be measured too.

14

Examples of customer perception measures Student satisfaction with

◦teaching and learning, ◦studying arrangements ( incl. physical

facilities etc), ◦student assessment and self-assessment, ◦attitudes towards students, ◦practical training period and various support

services (cleaning services, canteen services, library etc)

◦Finnish National Board of Education◦http://www.edu.fi/english

15

Performance indicatorsCustomer-related performance indicators may

describe areas such as learning outcomes, number of completed qualifications, completion of qualifications within normative time, employment rate by field, the image of the organisation etc.

The number of students applying to the institution (with priority choice) can indicate its attractiveness to students, whereas low absence rates of students may indicate good motivation of students. Students’ employment after graduation and the number of those completing a qualification, in turn, indicate the overall performance of the organisation's core process.

16

Examples of performance indicatorsArea Name of indicator Description of

indicator   Motivation of students   

Drop-out rateDrop-outs as proportion to the total annual number of students.

Completion rateThe proportion of students having completed their studies in regular time.

AbsencesNumber of lessons missed per student and per group or grade.

Success of students in competitions

Participation and success of students in national and international competitions.

17

 Student-centered approach 

Resources spent for students with special educational needs

a) Annual number of students with special educational needs. b) Average amount of resources spent per student for special needs education.

Complaints of students

Number of complaints of students registered yearly.

Achievement of students 

Failure rate Number of failed exams as proportion to the total number of exams passed per student.

Pass rate to further studies

Share of students enrolled to further studies compared to the national average fy field of education. 18

Evaluation of resultsAccording to the EFQM Model educational

organisations evaluate customer results, people results, society results and key performance results.

Each result area includes the perception indicators (a) measuring the perceptions or satisfaction of customers or staff and the internal performance indicators (b) measuring internal performance of the organisation.

Results are reviewed from the following perspectives:

19

20

The scoring of results in the EFQM Model

Evaluation area Perception indicators

Internal performanceindicators

 Customer results 75 % 25 %

 People results 75 % 25 %

 Society results 25 % 75 %

 Key performance results 50 % 50 %

21

STEP 2 – quality of evidence All data, collected by schools

should be:

◦Relevant ◦Valid◦Reliable◦Representative◦Comparable

22

4 questionnaires to measure the satisfaction of our:

StudentsFormer studentsEmployers School staff

23

StudentsSchool climate and relations with other students

and scool staffStability and balanceSchool management MotivatorsCommunicationTeaching and learningAssessmentPhysical environmentTeachers Spare time (sport facilities etc.)Canteen, dormitory etc.Public image of schoolSatisfaction, recommendation 24

School staffSchool climate and relations with students

and colleaguesCommunicationPhysical environmentLeadership and managementProfessional developmentMotivatorsStability and balanceStudentsPublic image of schoolSatisfaction, recommendation

25

STEP 3 – quality of quality assuranceResearch on 2006/2007All Estonian VET schools involvedAwareness and implementation of

principles of organisational exellenceThe impact of self-assessment ,The impact of Estonian VET Quality

Award Further training needs43 questions

26

Satisfaction with school

27

stude

nts - s

atisfa

ction

with sc

hool

stude

nts - v

alue o

f edu

cation

manage

ment -

satisfa

ction w

ith sc

hool

Staff - s

atisfa

ction w

ith sc

hool

Emplo

yers

- satisf

action

with sc

hool

3.3

3.5

3.7

3.9

4.1

4.3

4.5

Q AwardNo

Staff -satisfaction with school

28

Headmasters Managers Teachers Other staff3.3

3.4

3.5

3.6

3.7

3.8

3.9

4

4.1

4.2

Q AwardNo

Thank you for your attention!

Kaidi Holmkaidi.holm@mail.ee