Bruno Curvale President of the ENQA board Bologna expert Head of international affairs at AERES
description
Transcript of Bruno Curvale President of the ENQA board Bologna expert Head of international affairs at AERES
Bruno CurvalePresident of the ENQA board
Bologna expertHead of international affairs at AERES
29th of May 2009
Vilnius
ENQA experiences with external reviews of agencies
- Challenges -
Outline of the presentation• Introduction
• I - European Quality AssuranceMajor steps
• II - The ESG and the ENQA membership criteria• Similarities and differences
• III - ENQA experiences• ENQA coordinated reviews• The work of the board
• IV - The issue of consistency• Interpretation• Notions• Cultural dimension
• V - Improvements and remediation• Debriefing• Training
There are many challenges related to:
• The context
• The objectives of ENQA
• The link between the ESG and the ENQA membership criteria
• The complexity of the two stages evaluation process
• The nature of the judgements to be made by the review panels and the ENQA board
• The methodology itself
I - European Quality AssuranceMajor steps
Bologna process and quality assurance developments
1999Bologna
2001Prague
2003Berlin
2005Bergen
2007London
European cooperation in quality assurance
Primary responsibilit
y of HE institutions for quality
European Standards and Guidelines
European RegisterEQAR
Cooperation of QA agencies and HE institutions
E4 Group
1998 Reco. 98-561-CE 2006 Reco. 2006-143-CESource : Colin Tück, E4, B. Curvale
04/2008
2009LeuvenLouvain-la-Neuve
Evaluation of EQAR
II - The ESG and the ENQA membership criteria
ENQA objectivesSimilarities and differences between the ESG and
the ENQA criteria
ENQA objectives
• Promotion of agencies
• ENQA full membership as a recognised sign of professionalism
• A single external review of agencies for ENQA membership and EQAR listing
Differences and similarities between the ESG and the ENQA Criteria
ENQA criteria
▸ Principle based or rules based? Agencies shall follow the ESG and have to comply with the membership criteria
▸ Correspond to part 3 of ESG ▸ Integrate the part 2 of ESG (and by consequence the part 1 of the ESG)▸ Occasionally associate Standards (ENQA criteria 1)▸ Occasionally mix Standards and guidelines but keep the requirement
in the conditional (ENQA criteria 4)▸ Occasionally go further than the standard or the guidelines (ENQA
criterion 3, ENQA criterion 7)▸ Occasionally split Standards in different criteria (ENQA criteria
6i and 6ii)▸ Have additional requirements (ENQA criteria 8i, 8ii, 8iii)
But are fundamentally identical to the letter and the spirit of the ESG, unless it is the opposite
III - ENQA experienceENQA coordinated reviewsThe work of the board
The experience
• 7 ENQA coordinated reviews organised so far (ANECA, AQA, AQU, ACSUG, AGAE, ARACIS, EUA-IEP)
• 2 to be done (SKVC for sure and ECCE)
• 24 agencies (among 39 full members) are already externally reviewed (AAC, AQA, FHR, VLHORA, NEEA, EVA, ACQUIN, AQAS, ASIIN, FIBAA, GAC, ZEvA, HAC, HETAC, NQAI, NVAO, NOKUT, PKA, AGAE, ANECA, AQU, NAHE, OAQ, QAA)
• 8 candidates members are preparing to the review
The types and the coordination of the external review process
Type A (ENQA membership criteria / ESG)Type B (also other aspects)
- statutory functions (national missions, …)- ECA Code of Good Practice- special context (e.g. bi-national character, NVAO)- follow-up on previous review…
Co-ordination • national• ENQA (+ other non-national)
Sources: Tibor Szanto, Barcelona 17/03/09
The work of the panel
Is the agency…- fully compliant- substantially compliant- partially compliant- non-compliant
… with the criterion/ESG standard
Overall judgement not required (but can be made if the panel wishes to do so)
Sources: Tibor Szanto, Barcelona 17/03/09
The work of the board
To answer the question: Can it follows the analysis and thejudgements made by the panel and decide to grant the full ENQA membership.
What the board expects from the report:- ESG Parts II and III fully covered- Each criteria discussed separately- Structure (under each criteria):- evidence („description”, „facts and data”)- analysis („consideration”)- conclusion (judgement on compliance)- recommendation (if any)
Sources: Tibor Szanto, Barcelona 17/03/09
IV - The issue of consistencyConsistency of judgement
Consistency of judgement is a challenge
For the panels and for the ENQA board because of:
Situations have to interpreted
Notions used are difficult
Language issues
Cultural dimension
Implicit knowledge
V - Improvements and remediationDebriefingTraining
The experience is gathered
Well organised review processes are determinantInternal processes and external processes are equally important
ENQA is learning from what is done by the panelsDebriefing/brainstorming meeting for the training of experts for agency reviews, Barcelona, Spain, 17 March 2009
ENQA has started to train external reviewersTraining session/workshop for experts for agency reviews, 21-22 May 2009, Paris, France
The experience is shared
ENQA develops its documentation and makes it available to all:The Guidelines for national reviews of ENQA member agenciesThe Briefing pack for review panel members of ENQA coordinated reviewsThe Principles for ENQA coordinated reviews
are now merged in one single document:
The Guidelines for external reviews for the purpose of ENQA membership
ENQA will continue to develop the methodology of external reviews
The board has now a “book of precedents”, regularly updated by the ENQA secretariat
Thank you for your attention
…
4. Enable higher education institutions active within their territory to choose among quality assurance or accreditation agencies in the European register an agency which meets their needs and profile, provided that this is compatible with their national legislation or permitted by their national authorities
5. Allow higher education institutions to work towards a complementary assessment by another agency in the European Register, for example to enhance their international reputation.
…
Hereby recommend that member States:
The European parliament and the council of the European Union…
Extract of the Recommendation of 15 February 2006 on further cooperation in QA in HE
All agencies which comply substantially with the European Standards and Guidelines for Quality Assurance (ESG) can be admitted to the Register.
Substantial compliance with the European Standards and Guidelines is to be evidenced through an external review by independent experts. Such a review is coordinated either by a national authority or another organisation that is independent from the quality assurance agency under review.
Full ENQA membership, being also based on substantial compliance with the ESG, will normally constitute satisfactory evidence for inclusion in the Register.
EQAR Criteria
Extract from the EQAR website http://www.eqar.eu
Standard 3.3
Guidelines 3.3
Standard 3.1
ESG 3.1, 3.3 / ENQA Criterion 1: Activities
Extract from the Application form for membership of ENQA
“The Agency undertakes external quality assurance activities (at institutional or programme level) on a regular basis. These may involve evaluation, review, audit, assessment, accreditation or other similar activities and should be part of the core functions of the Agency. In undertaking its activities, the Agency should take into account the presence and effectiveness of the external quality assurance processes described in Part 2 of the European Standards and Guidelines (ESG”
Guidelines 3.5
Standard 3.5
ESG 3.5 / ENQA Criterion 4: Mission statement
Extract from the Application form for membership of ENQA
“The Agency should have clear and explicit goals and objectives for its work, contained in a publicly available statement. This statement should describe the goals and objectives of the Agency’s quality assurance processes, the division of labour with relevant stakeholders in higher education, especially the higher education institutions, and the cultural and historical context of its work. The statement should make clear that the external quality assurance process is a major activity of the Agency and that there exists a systematic approach to achieving its goals and objectives. There should also be documentation to demonstrate how the statement is translated into a clear policy and management plan.”
Standard 3.4
“The Agency should have adequate and proportional resources, both human and financial, to enable it to organise and run its external quality assurance process(es) in an effective and efficient manner, with appropriate provision for the development of its processes, procedures and staff.”
ESG 3.4 / ENQA Criterion 3: Resources
Extract from the Application form for membership of ENQA
Standard 3.8
Guidelines 3.8
“The Agency should have in place procedures for its own accountability. These procedures are required to include the following:
i. a published policy for the assurance of its own quality, made available on its website;
ii. documentation which demonstrates that:……
ii. a mandatory cyclical external review of its activities at least once every five years which includes a report on its conformity with the membership criteria of ENQA. Documentation could here either indicate that a review has taken place or is planned.”
ESG 3.8 / ENQA Criterion 7: Accountability procedures
Extract from the Application form for membership of ENQA
“are expected“ in the ESG guidelines
Review AND evaluation of the review
AgencyProcedures, “behaviour”
Q Standards
Review of the agency(ESG 3, 2 and 1)
+ specific ENQA criteria
Purposes
Evaluation of:- the review process- the review outcomes
National regulationsSpecific objectives
Inclusion in EQAR
ENQA full membership
External review
ENQA board
Register committee
1 2