Evaluating QA Agencies

INQAAHE BIENNIAL CONFERENCE, ABU DHABI 2009

Pre-conference workshop 30 March 2009 1.30 to 4.30pm

Dorte KristoffersenDeputy Executive DirectorHKCAAVQAPQN Vice-Presidentdkristoffersen@hkcaavq.edu.hk

Introduction to the workshop on Evaluating QA Agencies

Welcome

Introduction of participants

Overview of the workshop

Workshop outcomes

After the workshop the participants will be able to:

Formulate possible purposes and values of an external evaluation of a external QA Agency (EQAA)

Make decisions on the framework for an external evaluation

Make decisions on the organisation of the external evaluation

Understand the options for having the external evaluation recognised

Purpose and value of an external evaluation

Accountability locally and internationally ‘Taking your own medicine’ Effectiveness of performance Improvement Recognition Other?

Frameworks for evaluation of EQAAs

INQAAHE Guidelines for Good Practice The European Standards and GuidelinesThe Chiba PrinciplesOECD/Unesco Guidelines for Cross-border higher

educationNetwork membership criteriaEuropean Excellence modelOther?

Purpose and content of GGP

The main purpose of the Guidelines of Good Practice in Quality assurance (GGP) is to increase professional standards among INQAAHE member agencies

Section I. The EQAA: Accountability, Transparency, and Resources1. The Governance of the EQAA2. Resources3. Quality Assurance of the EQAA4. Reporting Public Information

Content of GGP con’t

Section II. Institutions of Higher Education and the EQAA: Relationship, Standards, and Internal Reviews

5. The Relationship Between the EQAA and Higher Education Institutions

6. The EQAA's Requirements for Institutional/Program Performance

7. The EQAA’s Requirements Institutional Self-Evaluation and Reporting to the EQAA

Content of GGP con’t

Section III. EQAA Review of Institutions: Evaluation, Decision, and Appeals

8. The EQAA's Evaluation of the Institution and/or Program

9. Decisions

10. Appeals

Purpose and content of ESG

The main purpose of the European Standards and Guidelines is to provide a source of assistance and guidance to EQAAs and HEI in developing their QA processes.

The ESG are divided into three parts:For internal QA within HEIFor external QA of HEFor external EQAAs

ESG for EQAAs

1. Use of external quality assurance procedures for higher education

2. Official Status

3. Activities

4. Resources

5. Mission Statement

6. Independence

7. External quality assurance criteria and processes used by the agencies

8. Accountability procedures

Discussion session #1

You constitute the Board of the Accreditation Agency of Olympia (AAS).

• Is it the right decision to initiate an evaluation?

• How would you formulate the Terms of Reference?

• Would the Guidelines of Good Practice or the ESG be a valuable framework for the process – please explain why and why not.

Evaluation approach

Independence, ie clear division of labour between EQAA and panel

Self evaluation report Peer Review and site visit Public report

Considerations for appropriate administrative arrangements

Administrative support, ie recruitment of panel members, organisation of travel, site visit program

The agency under review >< independent secretariat

Clear division of labour between the agency and the panel

Considerations forpanel composition

Size Profile Stakeholder representation Nationalities Language

Considerations forthe self evaluation process

Preparation of self evaluation report Existing documentation, ie handbooks,

policies, plans, regular surveys among client and stakeholders, and staff consultations

New evidence, ie analysis through the self evaluation process, tailor-made surveys, call for

Considerations forsite visit programme

Duration Balance between discussion/preparation,

study of evidence and interviews Interview groups , eg agency board and

staff, government, reviewers and other main stakeholders, ie educational institutions and students

Discussion session #1

You constitute the Board of the Accreditation Agency of Olympia (AAS).

• Is it the right decision to initiate an evaluation?

• How would you formulate the Terms of Reference?

• Would the Guidelines of Good Practice or the ESG be a valuable framework for the process – please explain why and why not.

Report requirements

Target group Public >< confidential Balance between description, analysis and

conclusion Balance between commendations,

identification of areas for improvement and recommendations to address areas for improvement

Discussion session #2

Taking the starting point in your own EQAA or in the situation in your home country if you are from a HEI, what issues would be relevant to take into consideration when organising the following stages of an external review:

• Organisational support• Self evaluation process, including involvement of all

relevant stakeholder• Panel formation • Site visit• Report writing and follow-up

Public recognition for external review

INQAAHE offers a service to its members to record a member agency that has been found to adhere comprehensively to the GGP

INQAAHE can organise the external review process against a fee

www.inqaahe.org

Public recognition for external review

The European Quality Assurance Register for Higher Education (EQAR) is a partnership of European organisations in higher education

EQAR is an information tool offering clear and reliable information on quality assurance provision by listing agencies who have proven compliance with the ESG through an external review (self evaluation, site visit by external experts (a total of 4 including at least one academic staff member, a student, one international member)

The European Register Committee’s (ERC) 11 members are identified by the EQAR Board in their capacity as independent experts in HE and QA

Application to be assessed by ERC Approval or non approval. Before announcement of non-approval

the application can make additional representation Approval will lead to inclusion on the Registerwww.eqar.eu

Thank you