Evaluation of Schools for Quality I mprovement
36
Evaluation of Schools for Quality Improvement Vitor Alaiz UNIVERSIDADE CATÓLICA PORTUGUESA FACULDADE DE EDUCAÇÃO E PSICOLOGIA ool Development in Central Asia orino, 7-8 MAY 2009
description
Evaluation of Schools for Quality I mprovement. Vitor Alaiz UNIVERSIDADE CATÓLICA PORTUGUESA FACULDADE DE EDUCAÇÃO E PSICOLOGIA. School Development in Central Asia. Torino , 7-8 MAY 2009. Portuguese Educational system. Portuguese Educational System. - PowerPoint PPT Presentation
Transcript of Evaluation of Schools for Quality I mprovement
Evaluation of Schools for Quality Improvement
Vitor AlaizUNIVERSIDADE CATÓLICA PORTUGUESA
FACULDADE DE EDUCAÇÃO E PSICOLOGIA
School Development in Central Asia Torino, 7-8 MAY 2009
Portuguese Educational system
• 1950 - 1974
Dual system
Portuguese Educational System
• General education schools (Liceu)• VET schools
Dual system
VET
Professional
Courses
Vocational
Courses
Education and
Training Courses (CEF)
Apprenticeship
courses
State
General+Professional Second
ary school
s
Private
Professional courses Professional
schools
Pluralistic ways to quality assurance
Quality
Accreditation
Certification
Systemic
evaluation
Audit
Schools make …No
difference
All the difference
Difference
SchoolQuality
School effectiveness
School improvement
Quality assurance
SCHOOLQUALITY
EXTERNALEVALUATION
INTERNALEVALUATION
EFQMISO 9000
Contingencial Models
ExternalTests, surveys
Team visits
INDICATORSSYSTEMS
InternationalNational
Local
Programa AVESAvaliação de Escolas com Ensino Secundário10
Programa AVESAvaliação de Escolas com Ensino Secundário
AVES Program: what is it?
MODELExternal evaluation
that supports internal evaluation processes
Multidimensional evaluation: Qualitative + quantitative dimensions
Network of schools, considering the context of each one
CIPP modelCONTEXT
INPUTPROCESS
PRODUCT
Stufflebeam
Programa AVESAvaliação de Escolas com Ensino Secundário14
MultidimensionalityLEVELs DIMENSIONs
Input Initial academic performance (Math, Port, Hist, Nat Sc)Context Sociocultural context (1,2,3,4 levels)Processes School + classroom processes
Results
STUDENTS1. Final academic performance (Math, Port, Hist, Nat Sc)
2 – Values + atitudes3 – Learning strategies 4 – Cognitive skils 5 – Opinion about the school
PARENTSOpinion about the school
TEACHERSSchool climate
Programa AVESAvaliação de Escolas com Ensino Secundário15
Programa AVESAvaliação de Escolas com Ensino Secundário
TOOLS
ACADEMIC TESTS
each year > 200 000
80000 tests120000 questionaires
QUESTIONAIRES
Programa AVESAvaliação de Escolas com Ensino Secundário16
Questionaires
Tests
Data (statistical treatment)
Report 1 Port, Math, Hist, Nat Sc
Report 3Value Added
Report 2 Values, Attitudes,
Cogn Skils …
“Fluxogram” of Program AVES
Report 3School Climate
Programa AVESAvaliação de Escolas com Ensino Secundário17
Reports: academic performance
0
10
20
30
40
50
60
70
80
90
100Es
cala
0-10
0
Factores
Matemática 7º ano
Escola 32 42 33 47 37 41 48 35 38Contexto 1 44 32 53 40 44 49 39 41Total 36 26 41 35 37 43 32 34
Números/Cálculo
Proporcion. Estatística Geometria Nota Aquisição Compreen
sãoResolução
de Problema
Programa AVESAvaliação de Escolas com Ensino Secundário18
0,00%
5,00%
10,00%
15,00%
20,00%
25,00%
30,00%
35,00%
Esca
la 0-
100
CONTEXTO SOCIOCULTURAL
Distribuição dos alunos pelo Contexto
Escola 6 24,20% 19,20% 32,40% 24,20%Nível 4 Nível 3 Nível 2 Nível 1
Reports: graphic with the context
Programa AVESAvaliação de Escolas com Ensino Secundário19
Reports: OCV (values, attitudes and cognitive skils)
0102030405060708090
100
Esca
la 0-
100
Factores
OPINIÃO SOBRE A ESCOLA 7º ano
Escola 1 55,8 59,9 43,1 66,2 52,2 75,3 67,7Contexto 2 66,2 64,9 62,2 69,6 58,8 71,6 73,5Total 58,9 66,3 49,1 64,1 51,3 72,4 68,5
Ordem Professores Actividades Director de
TurmaClassificaç
ões Colegas Satisfação
Programa AVESAvaliação de Escolas com Ensino Secundário20
1.00 6.00 7.00 10.00 13.00
escola
10.00
20.00
30.00
40.00
50.00
60.00
70.00
80.00
39
41153207
154
320
71
154
41
inputoutput
3º Ciclo
Média de versus resultados à entrada
resultados à saída
1.00 6.00 7.00 10.00 13.00
escola
10.00
20.00
30.00
40.00
50.00
60.00
70.00
80.00
39
41153207
154
320
71
154
41
inputoutput
Report: value added
Programa AVESAvaliação de Escolas com Ensino Secundário21
Reports: school climate
Figura 1: VA do 3º Ciclo do Ensino Básico
Why school climate?“School’s climate, as a variable, is far more directly responsible for pupils’ success than their social, economic and racial background.“
(Brookover et al. (1979) cf. Lima, 2008: 113)
Why school climate?“Culture shapes school’s capacity for change and improvement.”
(Sarason, 1996; Stoll, 1999b; Stoll and Earl, 2003: 495)
“Grosin’s research (1995) in Sweeden identified positive links between school climate and student’s mathematic progress, reading comprehension and behaviour” (Hopkins, 2003: 518)
School climate Definition:
the way as organizational
culture is perceived by the
members of one organization (Gonzalez Galan, 2004).
Methodology Questionnaire
Items: 40Scale: Likert scale, 5 leves (totaly desagree, …)
Subjects: 2500 teachers
Schools: 27 general schools (HS)+ 11 VET schools
Data:
Statistical treatement : averages of each item in: each school + each school of each type + all schools
Statistical tests: significance of average difference
Questionnaire: matrix 1
Questionnaire: matrix 2
Items aggregated in 8
constructs
ResourcesLeadersh
ip Internal discipline (behaviour)
Social systemSatisfaction level
Job climatePerceived
effectiveness
Personal relationships
Questionnaire: items (example)
The school facilities encourage teaching and learning
The school has a well defined educational direction
The level of discipline at the school allows for the good functioning of lessons
Teachers feel that their work is appreciated by pupils
Teachers are unhappy with colleague interaction within the school
Teachers participate with professionalism in school activity
Teachers are confident with the results of their work
Teachers have autonomy in their teaching role
Disciplina + aulas P3: O nível de disciplina existente na escola permite o bom funcionamento das aulas.
Cumprim RI P16: As normas do regulamento interno são cumpridas por todos, sem excepção.
Cumprim RE P26: A direcção da escola procura que as normas e orientações do Ministério da Educação sejam sempre cumpridas.
3 - DISCIPLINA INTERNA
1,0
1,5
2,0
2,5
3,0
3,5
4,0
4,5
5,0
ESCOLA 62 4,3 3,5 4,3
TOTAL Escolas Profissionais 4,0 3,5 4,2
TOTAL EP+ER 3,9 3,3 4,3
p03 p16 p26
Legenda:
Report: example
Pfs aprec p alun P4: Os professores sentem que o seu trabalho é apreciado pelos alunos.Pfs motiv trab P17: Os professores estão motivados para o seu trabalho lectivo.Pfs reconh pais P27: Os professores vêem o seu trabalho reconhecido pelos pais e encarregados de educação.Pfs reconh Dir P31: Os professores vêem o seu trabalho reconhecido pela direcção da escola.Pfs aprec Dir P32: Os professores apreciam positivamente o trabalho da direcção da escola.Pfs mot trab nlect P37: Os professores estão motivados para o seu trabalho não lectivo.
4 - RELACIONAMENTO PESSOAL
1,01,5
2,02,53,0
3,54,0
4,55,0
ESCOLA 62 3,8 3,5 3,0 3,3 3,7 3,4
TOTAL Escolas Profissionais 3,9 3,9 3,7 3,9 3,9 3,5
TOTAL EP+ER 3,7 3,5 3,5 3,6 3,6 3,0
p04 p17 p27 p31 p32 p37
Legenda:
Report: example
Study:Comparison
General schools VET schools
School Climate
ConclusionSchool climate of a set of VET
schools may have been
better than the one
of a lot general schools
Colect information
CommunicateEvaluation
UTILITY
PlanEvaluation
Treatinformation
ACCURACY PROPRIETY
FEASIBILITY
Good Quality Evaluation
THE JOINT COMMITTEE ON STANDARDSFOR EDUCATIONAL EVALUATION
Thank you!Vitor Alaiz
UNIVERSIDADE CATÓLICA PORTUGUESAFACULDADE DE EDUCAÇÃO E PSICOLOGIA
School Development in Central Asia Torino, 7-8 MAY 2009
