Rafael Llavori de Micheo Head of the International Relations Unit ANECA Case study 4: Spain ECA...
-
Upload
ella-galloway -
Category
Documents
-
view
220 -
download
0
Transcript of Rafael Llavori de Micheo Head of the International Relations Unit ANECA Case study 4: Spain ECA...
Rafael Llavori de Micheo
Head of the International Relations Unit
ANECA
Case study 4: Spain
ECA Seminar
Changing Systems: achievements and new develop,ments in national accreditation systems
The Hague, 9th December 2009
Adapting the national system to the EHEA
The Spanish case study
The accreditation system
The institutional perspective: AUDIT
Open questions for the future
ContentsContents
1. Adapting the national Adapting the national system to the EHEAsystem to the EHEA
The Spanish HE system
UniversitiesUniversities
7575 universities universities
(50 public – 25 private)(50 public – 25 private)
Public universities – 1.326.734
Private Universities – 134.743Number of students enrolled 1.461.477
Adapting the HE system to the EHEA
Bologna process and quality assurance developments
1999
Bologna
2001
Prague
2003
Berlin
2005
Bergen
2007
London
European cooperation in quality assurance
Primary responsibility of HE institutions
for quality
European Standards and Guidelines
European RegisterEQAR
Cooperation of QA agencies and HE
institutions
E4 Group
1998 Reco. 98-561-CE 2006 Reco. 2006-143-CE
Source : Colin Tück, E4, B. Curvale
04/2008
2009
Leuven
Louvain-la-Neuve
Evaluation of EQAR
Adapting the HE system to the EHEA
Adapting the HE system to the EHEA
New Spanish Legal FrameworkNew Spanish Legal Framework
European HigherEuropean Higher
Education AreaEducation Area
•University Act (December 2001) and its reform (April 2007).
•Real Decreto 1044/2003, 1 August, European Degree Supplement
•Real Decreto 1125/2003, 5 September, ECTS
•Real Decreto 56/2007, 21 January, Postgraduate Official Programmes
•Real Decreto 1393/2007, 29 October, New Organisation of Programmes
•Real Decreto 1509/2008, 12 de September, Universities, Centres and Degrees Register
•University Act (December 2001) and its reform (April 2007).
•Real Decreto 1044/2003, 1 August, European Degree Supplement
•Real Decreto 1125/2003, 5 September, ECTS
•Real Decreto 56/2007, 21 January, Postgraduate Official Programmes
•Real Decreto 1393/2007, 29 October, New Organisation of Programmes
•Real Decreto 1509/2008, 12 de September, Universities, Centres and Degrees Register
•European Standrads and Guidelines for QA of HE•European Standrads and Guidelines for QA of HE
CHANGES FOR THE UNIVERSITY SYSTEMCHANGES FOR THE UNIVERSITY SYSTEM
Changes in the legal frameworkChanges in the legal framework
Adapting the HE system to the EHEA
2. The Spanish case studyThe Spanish case study
The Spanish HE system: the actors
Ministry of Education
Regional governments
Universities
• National legal framework: LOMLOU
• Programme accreditation
• Eval. and Accreditation of Teaching Staff
• Regional legal framework
• Funding public HEIs
• Follow-up and ex-post accreditation
• Evaluation of teaching staff (hired)
• Programme offer
• Selection of teaching staff
• Selection of students
The Spanish case study
University Council/Reg. Gov.
Universities
• Evaluation for accreditation (ANECA)
• Follow-up and ex-post accreditation (both)
• Teaching staff accreditation (ANECA) and Evaluation of Teaching staff (hired) (both)
• Institutional Evaluation practices DOCENTIA y AUDIT
• Internal QA systems
• Evaluation of Teaching activity
• Selection of teaching staff
ANECA/Regional Agencies
The Spanish case study
o The new legal framework assigns
ANECA and the regional agencies a
concrete role to be played
ANECAANECA Autonomous Communities AgenciesAutonomous Communities Agencies
o RD 1393/2007 establishes that ANECA: Prepares the procedures Elaborates the evaluation report for the ex
– ante accreditation Mutual recognition between ANECA and
Spanish ENQA full members / EQAR agencies
• ANECA and the A.C. agencies
• Do the follow-up procedure of the accredited programmes
• Elaborate a previous report for re-accreditation
Regions with regional agencies Regions without regional agencies
QA in Spain at the national level
BEFORE EHEA WITHIN EHEA
FOCUS Contents Competences (LO)
AUTONOMY Less Higher
WHO DESIGN THE PROGRAMMES?
The Ministry University
FRAMEWORK Catalogue Register (Flexible)
DEGREES OFFERED
Limited number Depending on the universities
QUALITY Important Key
EXTERNAL QUALITY
ASSURANCE
Voluntary Mandatory (Ex-ante, Follow-up
and Re-accreditation)
Adapting the HE system to the EHEA
3. The accreditation The accreditation procedureprocedure
Design
Programme design
UNIVERSITY
The programme accreditation processThe programme accreditation process
Accreditation
Fulfilment evaluation
Council Univ.
Follow-up
Implementation
UNIVERSItY
ANECA and Reg. Agencies
Ex – ante Evaluation
Accreditation
Council Univ.
ANECARegional Government
The accreditation procedure
The ex ante programme accreditation focuses on bachelor, master and doctorate level
Apply to all both existing and new programmes.
ScopeScope
The Council of Universities ministerial body, based upon the evaluation report of ANECA
ANECA has set up recognition agreements with regional agencies which fulfil the European standards (external reviewed by ENQA) to take part in the accreditation procedure.
Accrediting bodyAccrediting body
The accreditation procedure
The standards are established in the legal framework that defines the features of the official degrees (Real Decreto 1393/07).
It includes broad references to justify a new programme.
It establishes general descriptors for bachelor, master and doctorate based on the Dublin Descriptors and related to the national QF (MECES).
Academic standards for learning outcomes/specific competences of a particular programme still pending. They are evaluted according to external (international reference points).
Academic standardsAcademic standards
The accreditation procedure
Verification procedure of a programme/degree:Verification procedure of a programme/degree:
What is evaluated?
6.6. Personal académicoPersonal académico
7.7. Recursos materiales y serviciosRecursos materiales y servicios
8.8. Resultados previstosResultados previstos
9.9. Sistema de garantía de calidadSistema de garantía de calidad
10.10. Calendario de implantaciónCalendario de implantación
1.1. Descripción del títuloDescripción del título
2.2. JustificaciónJustificación
3.3. ObjetivosObjetivos
4.4. Acceso y admisión de estudiantesAcceso y admisión de estudiantes
5.5. Planificación de las enseñanzasPlanificación de las enseñanzas
The accreditation procedure
Evaluation committees by field of knowledge (5)
ANECA selects, appoints and trains
A chairperson.chairperson.
A different number of academic peersacademic peers according to the numer of proposals.
One of the experts belongs to the professional fieldprofessional field of the corresponding field of knowledge.
A studentstudent.
A secretarysecretary from ANECA
Who evaluates?
The accreditation procedure
Comisión de Rama de Conocimiento A
VOCAL 1
VOCAL 2
VOCAL 3
PRESIDENTE
SECRETARI O
Fase de alegaciones
I nforme de Evaluación Definitivo de cada Título
Comisión de Rama de Conocimiento A
Propuesta de Informe para cada Título
COMISIÓN DE EMISIÓN DE INFORMES
Estudiante
Valoración
VOCAL 4
VOCAL 5
VOCAL 6
VOCAL N
Comisión de Rama de Conocimiento B
PRESIDENTE
SECRETARI O
Estudiante
Valoración
Comisión de Rama de Conocimiento n
PRESIDENTE
SECRETARI O
Estudiante
Valoración
*Cada expediente será asignado a una Comisión de Rama de Conocimiento.
Comisión de Rama de Conocimiento nComisión de Rama de Conocimiento B
Propuesta de Informe para cada Título
Propuesta de Informe para cada Título
VOCAL 1
VOCAL 2
VOCAL 3
VOCAL 4
VOCAL 5
VOCAL 6
VOCAL N
VOCAL 1
VOCAL 2
VOCAL 3
VOCAL 4
VOCAL 5
VOCAL 6
VOCAL N
How it is evaluated?
The accreditation procedure
Evaluation committees for field of study
Committee for Delivering Reports
ANECA’s double contrast
The evaluation of the programmes are conducted by the Evalluation Committees for Field of StudyEvalluation Committees for Field of Study. ANECA will submit the evaluation reports to the Council of Universities through the
Committee for Delivering ReportsCommittee for Delivering Reports.
Consistency ad intra and
inter- committees
The accreditation procedure
How is it the
decision made?
ANECA delivers an evaluation ANECA delivers an evaluation reportreport either positive or negative. The report either positive or negative. The report is delivered following two steps:is delivered following two steps:
1. ANECA submits to the university a 1. ANECA submits to the university a provisional report including those provisional report including those aspects to be changed to obtain a aspects to be changed to obtain a positive decision proposals for positive decision proposals for enhancement. enhancement.
2. ANECA submits a reasoned report to 2. ANECA submits a reasoned report to the Council of Universities which is in the Council of Universities which is in charge of making the accreditation charge of making the accreditation decision.decision.
Once the programme has been Once the programme has been accredited and is included in the accredited and is included in the Register, ANECA publishes the report Register, ANECA publishes the report in its web-site.in its web-site.
The accreditation procedure
The procedure consists of the following stages:
1.1. Self-evaluationSelf-evaluation
2.2. External reviewExternal review
3.3. ReportReport
The accreditation procedure
ECA Principles for accreditation procedures
regarding joint programmes
The accreditation procedure
4. The institutional perspective: The institutional perspective: AUDITAUDIT
Recognition of Internal Quality Assurance
Systems in University Education Programme
The aim of this programme is to enhance and
strengthen the development and implementation of
Internal Quality Assurance SystemsInternal Quality Assurance Systems in all center
(schools and faculties) that offer university
education in Spain. Another objective is to create a
procedure for the recognition of such systems.
Voluntary participation of institutions (faculties/universities)
Open to all Spanish Universities
First call for proposals started in September 2007First call for proposals started in September 2007
Voluntary participation of institutions (faculties/universities)
Open to all Spanish Universities
First call for proposals started in September 2007First call for proposals started in September 2007
•Agency for the Galician University System Quality (ACSUG)
•Agency for the Catalonian University System Quality (AQU)
The institutional perspective: AUDIT
To draw up Criteria and Guidelines for IQAS.
To evaluate HEI’s IQAS design.
To write an Interim and a Final Reports with an advisory function.
AUDIT Evaluation Methodology
QAAs
To design IQAS, according to Criteria and Guidelines.
To improve their IQAS according to recommendations in the Interim and Final reports.
To implement their IQAS.
HEIs
Focuses on teaching and learning processes
MILESTONES MILESTONES of the AUDIT programme
1. To orientate in the designdesign of the Internal Quality Assurance Systems (IQAS) at University faculties
2. Certification of the designdesign of the IQAS
3. Certification of the Implemented SystemsImplemented Systems
MILESTONES MILESTONES of the AUDIT programme
1. To orientate in the designdesign of the Internal Quality Assurance Systems (IQAS) at University faculties
Compromise
Planning
Diagnosis
Definition and documentation
Design guidelines
•Design guides
•Guidelines
•Tools
Schools, Faculties/ UniversitiesAgencies
MILESTONES MILESTONES of the AUDIT programme
Participation in the accreditation
Improvement and implementation On the basis of the given
documentation if verifies the
fulfillment of the IAQS
requirements
Design verification report
2. Verification of the designdesign of the IAQS
Schools, Faculties/ UniversitiesAgencies
MILESTONES MILESTONES of the AUDIT programme
Implementation of IAQS Visit to the faculty /university
IAQS certification report
3. Certification of the implemented systemsimplemented systems
Schools, Faculties/ UniversitiesAgencies
Internal Quality Internal Quality Assurance SystemAssurance System
Stakeholders Examples of aspects considered in the development of quality assurance mechanisms
StudentsStudents Selection and admission of students, programme profile, organization and development of the studies, learning support systems, educational outcomes and response of the labour market...
University University (in its different levels, i.e. faculty, managers and support staff)
Selection and admission of students, programme profile, organization and development of the studies, learning support systems, faculty and support staff, resources, academic progress, educational outcomes and response of the labour market, information systems...
EmployersEmployers Educational offer, programme profile, information quality, response of the labour market…
Public Public administrationsadministrations
Educational offer, programme profile, information quality, response of the labour market, education quality and response of the labour market, costs…
Society in generalSociety in general Supply and demand of education, academic progress and results, response of the labour market…
FFocus on stakeholdersocus on stakeholders
Internal Quality Internal Quality Assurance SystemAssurance System
IInternal quality assurance guidelines
1. Quality policy and objectives
2. Design of educational offer
3. Development of the studies and other actions addressed to the students.
4. Faculty and support staff
5. Material resources and services
6. Educational outcomes
7. Public information
Internal Quality Internal Quality Assurance SystemAssurance System
1.4. How the HEI manages and improves its physical resources and services
The HEI/university must be provided with mechanisms so it can design, manage and improve its services and physical resources in order for student learning to develop appropriately.
1.5. How the HEI analyses and takes into consideration the outcomes.
The HEI/university must be provided with procedures to ensure that outcomes (learning, graduate employment and the satisfaction of the different interest groups) are measured, analysed and used[1] for decision-making and to enhance the quality of degree programmes.
1.6. How the HEI publishes information on degree programmes.
The HEI must be equipped with mechanisms to ensure that updated information on degrees and programmes is published periodically.
EEvaluation criteria
• Organisation and structure, with a well-defined scope and a meaning that is clear.
• All guidelines must be dealt with.
• Existence of a comprehensive process map (or similar) describing what the process is, who is involved, and how each process is carried out.
• Examples of procedures and indicators that are included in IQAS.
AUDIT Evaluation Methodology
TThe review panels
• The same kind of review panel has been developed by the three QAAs involved in the AUDIT initiative.
• Panel members:• Three academic staff members with experience in QA.• An QA expert, staff member of a HEI.• An QA professional. • A staff member of QAA.
Participation in AUDIT
ANECA : 71
ACSUG: 35
AQU: 20
Faculties and collegesQAAs Universities
43
2
8
• 53 Spanish Universities are participating in the programme.
• Majority of these universities are public.
• For most universities, 2 or 3 faculties or colleges.
• 10 universities have designed an IQAS for all their faculties or colleges.
• 52 Universities have completed IQAS design.
Source: Rodriguez et al. EQAF 2008
• High degree of involvement by the universities and a high demand from new universities to take part in the 2008 call.
• QA experts from different universities and regions of Spain set up workgroups to jointly learn about and draw up improved proposals for their universities.
• The AUDIT initiative is contributing to the updating of the QA culture in Spanish universities by encouraging the development of more systematic initiatives for QA.
• Universities have designed their IQAS following the criteria and guidelines laid down in AUDIT.
• Mutual recognition between agencies is an ongoing process.
• Project developed by ANECA to train experts to take part in the evaluations: 2009-2010
Conclusions and Lessons Learned
Source: Rodriguez et al. EQAF 2008
• Inadequate specification of the stakeholders and how they are involved in identifying the requirements of the IQAS;
• Imprecise aims for quality
• Incomplete definition of the bodies responsible for the management of quality in faculties and colleges
• Lack of specific accountability procedures
• Incomplete mechanisms for decision-making in the IQAS, especially in relation to decisions subsequent to self-evaluation processes.
Conclusions and Lessons Learned
Source: Rodriguez et al. EQAF 2008
The AUDIT initiative is very useful in HEIs, specially in HEIs without an IQAS or where the internal quality assurance procedures and mechanisms have still not been systematised.
A university’s mission, vision and values should serve as the basis for producing an IQAS, and consideration also needs to be given to strategic issues.
In an initial stage of the development of IQAS, the agencies need to play an advisory role as well as that of external review to encourage the growth of a true quality culture in the universities.
Conclusions and Lessons Learned
Lessons Learned
It is necessary to involve university QA experts, professionals and academics to create synergies between those who play key roles in the design, development and enhancement of IQAS.
It is necessary to develop standard procedures between the different peer reviews that are carried out to ensure that results between the various review panels and committees are coherent.
Cooperation by the QAAs in joint projects is beneficial to the management of peer review processes.
Source: Rodriguez et al. EQAF 2008
Is IQAS recognition really an added value for HEIs? and for society?
Can the IQAS guarantee the quality of degree programmes in an external review?
What differences are there between the IQAS design and the usual QA activities in HEIs? Are they similar or not?
What is the real involvement of HEIs policy and decision-makers in a QA system such as AUDIT?
How can a QAA undertake a follow-up process with the engagement of the university? What is the effective role of the QAA in this process?
Key Issues To Be Mastered
There are various issues that still need to be addressed:
Source: Rodriguez et al. EQAF 2008
5. Open questions for the futureOpen questions for the future
AUDIT programme / follow-up:The follow-up procedure of the accreditation procedure will be the key part of the process
The follow-up stems very much to the IQAS of HEIs AUDIT was crucial to strengthen Q culture within HEIs The role of the QA Units within HEIs is crucial to support
programme and institutional co-ordinators Information tools generated by the HEIs are at the centre of the
procedure It will result in non-intrusive and non-beurocratic procedures for
HEIs Student protection in terms of the fulfilment of the programme
proposal accredited must be assured
Open questions for the future1
Future trends in QA and accreditation in Spain
Open questions fofr the future 2
On-going approach to quality: Quality and transparency of the information Reforms in the training plan focused on :
– Student learning
– Flexible and student-needs tailored curricular paths
– Development towards learning outcomes
– Internal quality assurance systems focused in the ESG
– Highly qualified teaching staff
Future trends in QA and accreditation in Spain
Open questions for the future 3
The instituional approach will be therefore tested through the follow-up procedure. It must show adequacy and fitness for purpose.
The meta-evaluation of the procedure will be decisive in a future trethinking of the the whole accreditation procedure
Future trends in QA and accreditation in Spain
Challenges
Multidimensional Transparency tools: Qrossroads (Proyect TE@M ECA) Rankings Joint evaluations reducing procedures through standards shared
by the agencies avoiding duplication of work. How interacting with project led by associations of disciplines to
define learning outcomes at the cycle level evaluating those programmes by means of labels aside the national QA/accreditation frameworks.
Future trends in QA and accreditation in Spain
• The joint evaluation reducing the procedures by means of ddefining the standards shared by agencies in order to reduplicate proceses.
• How interact with projects led by associations of disciplines to define learning outcomes and giving quality labels apart from the national accreditation framework and agencies.
Challenges
Priorities for the next decade
Future trends in QA and accreditation in Spain
[email protected]@aneca.es
Thank you for your attention