P2V | WP6Valorisation of the Framework for the Evalution of ICT in Education

Inspectorate of Education, The Netherlands, Bert Jaap van Oel

P2V WP6

Scotland: HMIE. Sweden: Skolverket. Lithuania: National Agency for School

Evaluation. France: IGEN. Catalonia: Inspectorate of EducationThe Netherlands: Inspectorate of Education

Members of SICI www.sici-inspectorates.org

Partners

The road to P2VERNIST: “Let us look at the things we share” (Belgium, Scotland, Austria, NI, England) >>P2P: Peer learning leading to a shared framework based on existing materials. (France, Scotland, Sweden, Ireland, England) >>

P2V: valorising and extending use

P2V WP62007: Kick off meeting: meet and learn2007-2008: School visits in all countries:• 2 primary• 2 secondary

2008: Evaluation meeting Brussels

P2V WP6 deliverablesSchool reports with at least a public summaryEvalution report per country: methodologyEvaluation reportWebsite Public toolbox for SICI membersFramework available for self- and peer evaluation by schools

P2V WP6 Methodology

P2V WP6 methodologyTwo NL inspectors join local teamFour schools per countryOne weekLocal evaluator coordinates visitsLocal evaluator is leadingLocal evaluator writes report for schoolNL inspector writes report on visit

P2V Evaluation methodology

Preparatory conversation with the schoolSend out and analyse self-evaluation questionnaireStudy available materials, plans, vision, reportsSchool visit: • As separate activity: one day• When integrated: half a day extra for one

personWritten report

Methodology: School visit

Methodology: School visit

Meeting with leadership/ICT coordinator: vision, history, what are we going to see?Lesson observations, school tourInterviews: Learners; Teachers; AdministratorsFeedback session at the end of the day: preliminary conclusionsDraft written report: final conclusions reviewed by schoolFinal written report (public?)

Methodology: School visit

Gathering evidence from observations, interviews, learning materials and outcomesDocumenting evidence: take copies, pictures, notesTriangulation: different sources, different observersPaper policy versus work floor realitySchool’s own vision as starting point

The ICT evaluation toolbox

The ICT framework Conditions:

C1. Leadership, C2. Infrastructure and access, C3. Curriculum planning, C4. Quality assurance and improvement

Use: U1. Pupil use, U2. The teaching process, U3. Administrative use

Outcomes: O1. Impact on learning and standards

The ICT frameworkQuality Indicators: what is it?Evidence: what to look for?Sources: where to find it Scoring per QI: 0=not enough evidence1=weak2=insufficient, should improve3=sufficient, may be improved4=goodThe framework

MaterialsSelf-evaluation questionnaireInterview guidanceLesson observation formEvaluator guidance

Possibly: online questionnaires

Presentation of results

 

Leadership

Scoreprimary school

A

Scoresecondary school

B

Scoreprimary school

C

Scoresecondary

schoolD

C1.1There is a clear vision for the use of ICT 

0|1|2|3|4 0|1|2|3|4 0|1|2|3|40|1|2|3|4

C1.2 There is a strategy to realise the vision

0|1|2|3|40|1|2|3|

40|1|2|3|4

0|1|2|3|4

School visits

School visits Primary school A  

450 pupils, 1:34 computersinvolved in many projectssmall budget, hard to plan ICT developmententhusiastic principal, vice-principal is ICT coordinatorLessons in computer room and in classroomsICT is not in the national curriculum, ICT use by children in free timeParents question ICT useICT and ICT-skills integrated in subjects

School visits Primary school B 

250 pupils, 1:4Some interactive whiteboardsGood budgetenthusiastic principal, vice-principal is ICT coordinatorLessons in classrooms, individual use in hallways and classroomsICT and ICT-skills integrated in subjectsMore critical thinking about ICT use to be developed

Secondary school

953 pupils, 74 teachers, 1:9 computers, many resourcesclear ICT vision and strategy on ICT and teachers, less on pedagogyclear use of benchmarksgood lessons observed, strong pedagogy, active learnersresponsibilities are clear, ICT coordinatornot enough possibilities for enhancement of learningno accessibility for marks for teachers, pupils, parentsno Learning Management System yet

School visits

Impressions: peer learning

Impressions: peer learning

Who is in the lead?Fitting the methodology to local circumstances or fitting local circumstances to the methodology?Presence external evaluators supports objectivity (contextualised scoring vs more objective scoring)Discussion essential: preparation, briefing sessions, evaluationManaging expectations: evaluator guidance vs learning?

Impressions: using the toolbox

Framework is very usable in different contextsMethodical work supports acceptance of evaluationNeed for good preparation of visitsUsing the different materials requires getting used to: how do they fit together?The materials are much-needed to support decision making in one day

Impressions: the visit programme

Short! Quick! Hectic!Enough time for reflection?Enough time for evidence gathering?All elements should be in placeA full daySome arranged lessons/activities are ok

Inspectie van het Onderwijs

Kantoor UtrechtPark Voorn 4Postbus 27303500 GS UtrechtThe Netherlands

T (030) 669 06 00F (030) 662 20 91

www.onderwijsinspectie.nl

Achterblad

b.vanoel@owinsp.nl