AUN-QA Assessments, Challenges and Lesson LearntDr. TAN Kay Chuan

AUN-QA Chief Quality Officer and Lead Assessor

Director, Office of Quality ManagementNational University of Singapore

Outline

Background

Initiation Trial implementation & challenges Tools development

Implementation Actual quality assessments Challenges Special projects

Improvement

Lesson learnt and Challenges

Background

2000 2004 2006 2007 Present

Bangkok Accord

AUN-QAGuidelines

AUN-QAManual

AUN-QAAssessments

AUN-QARevised Manual

AUN-QAOperating Guidelines

Improvement

InitiationImplementation

Bangkok Accord

Appoint Chief Quality Officer (CQOs)

Formulate common AUN-QA policies, criteria and benchmarking procedures

Identify and encourage the implementation of QA good practices

Enhance mutual collaboration and information exchange

Invite and facilitate auditing, assessment and review by other member universities as well as by external bodies

Be responsible for the implementation of this Accord

Seek further and deeper engagement regarding QA in higher education with ASEAN Dialogue Partners

AUN-QA Guideline

Guidelines are benchmark standards towards quality and quality assurance

Strengthen the position of higher education in ASEAN

Promote the harmonisation of quality assurance in higher education within and outside ASEAN

Endorsed by AUN Board of Trustees as well as Presidents, Rectors and Vice-Chancellors of AUN member universities

AUN-QA Guideline

AUN-QA Criteria

Criteria 1: QA System

Criteria 2: Teaching and Learning Course Curriculum Academic Staff Student Assessment Learning Process Environmental Health and Safety Standards Learning Resources

Criteria 3: Research Funding and Facilities Research Output

Criteria 4: Services

Criteria 5: Ethics

Criteria 6: Human Resource Development (HRD)

AUN-QA Manual (2006)

Facilitate implementation of AUN-QA Guidelines

Improve clarity and interpretation of quality criteria and standards

Manual for Self-assessment and Auditing

Endorsed by AUN Board of Trustees as well as Presidents, Rectors and Vice-Chancellors of AUN member universities

AUN-QA Manual (2006)

AUN-QA Revised Manual (2011)

First revision of the AUN-QA Manual

Criteria and assessment process of AUN Actual Quality Assessment at Programme Level

Associated resources (templates and samples)

Effective August 2011

AUN-QA Standards – Institutional QA

Stakeholder Satisfaction

Quality Assurance and (Inter)national Benchmarking

ACHIEVEMENTS

EducationalActivities

Research

CommunityService

Policy Plan

Management

HumanResources

Funding

Mission

Goals

Aims

Source: AUN QA Network

AUN-QA Manual – IQA System

Internal Quality Assurance

Follow up

Student Progress

Pass RateDrop-out Rate

Feedback from the Labour Market

and Alumni

Research Performance

Student Evaluation

Course and CurriculumEvaluation

Research Evaluation

ServiceEvaluation

AssuranceStudent

Assessments

Assurance QualityStaff

QualityAssurance

Facilities

QualityAssurance

Student Support

SWOT Analysis

Inter-CollegialAudits

InformationSystem

QualityHandbook

MonitoringInstruments

EvaluationInstruments

Special QA Processes

Specific QAInstruments

AUN-QA Standards – Programme QA

Initiation

Trial Run

Burapha University, May 2007 To put into practice the “Manual for the Implementing of the

Guidelines” of AUN-QA models for IQA system, institutional level and programme level

Challenges Clarification of assessment criteria and self-assessment report

(SAR) Key documents and records in local language Lack of a structured assessment process Lack of assessment and reporting tools

Tool Development

Formalisation of assessment process (PDCA)

Tools DevelopmentDesktop Assessment Template

Standardisation of assessment and reporting tools

ExampleDesktop assessment

Tool DevelopmentAssessment Report Template

ExampleAssessment Report

Tool DevelopmentAssessment Feedback

Tool DevelopmentSAR Diagnosing

Requirement of Self-Assessment Report

Part 1: Introduction Content page Executive summary Organisation of the self-assessment Brief description of the university, faculty and department

Part 2: AUN-QA Criteria Requirements Write-up on how the university, faculty or department addresses the requirements of the

AUN-QA criteria (use checklist as a reference) Part 3: Strengths and Weaknesses Analysis

Summary of strengths Summary of Weaknesses Completed checklist Improvement plan

Part 4: Appendices Glossary Supporting documents and evidences

Implementation

Actual Assessment (2007 Onwards)

4 Years

7 Universities

13 Assessments

32 Programmes

Actual Assessment

Actual Assessment

Results Overview

Challenges in Actual Assessment

Shortage of qualified assessors

Different stages of QA development and maturity within ASEAN

Improving assessment efficiency and effectiveness

Recognition of AUN QA Assessment

Harmonisation of AUN QA framework within and outside ASEAN

Sharing the knowledge gained and best practices

Special Projects

Initiative for ASEAN Integration (IAI-CLMV) Project

ADB-CLM Project (Supported by Asia Development Bank)

ASEAN-QA Project (Collaboration with DAAD, Germany)

Training for Non-member Universities

ASEAN-QA: Joint QA Capacity Development in the ASEAN region

Intro IQA Theory

SAR Writing Finalizing SAR

Intro EQA Theory

Peer TrainingPreparing Site

Visits

Sens

itiz

atio

n: D

eleg

atio

n of

Vic

e-Pr

esid

ents

Site

Vis

its

EQA: Staff of QA Agencies

IQA: University QA Officers

Training for Non-member Universities

Sep 2011, 1st Workshop

Apr 2012, 2nd Workshop

Dec 2012, 3rd Workshop

More than 50 participants from the ASEAN Region

CLM Project

Enhance and strengthen the knowledge on QA system implementation and management particularly based on AUN-QA system,

Build up qualified University QA personnel team in CLM countries,

Establish QA system to leverage the quality of education management in CLM countries, and

Provide countries with options for harmonizing university QA system to ensure applicability across Southeast Asia.

Improve

Training of New Assessors

1st AUN New Assessors Training Workshop (2008) 7 new assessors

2nd AUN New Assessors Training Workshop (2009) 12 new assessors

Growing of qualified assessors from 6 to 25

Development path: junior to senior assessors

Revision of AUN-QA Manual for Programme Level

Original AUN-QA Model for Programme Level Revised AUN-QA Model for Programme Level

18 criteria, 72 statements 15 criteria, 67 statements

Guidelines for AUN Quality Assessment and Assessors

Assessment Certification

Guidelines for AUN Quality Assessment and Assessors

The establishment of an AUN-QA Council which wil govern and formulate policies for the operation of the AUN-QA Network

Define roles, responsibilities and requirements of the applying university, assessors, observers and staff of AUN Secretariat

Provide a common frame of reference and accountability to the AUN quality assessment process and stakeholders for the purpose of certification

Sharing the knowledge gained and best practices via proposed IT platform

38

Applicant

Assessor

AUN Secretariat

AUN Member Public User

• SAR Submissions

• Assessment Results

• Progress Reports

• Best Practices & Benchmarking

• List of AUN Certified Universities

• Links to other QA agencies

QA Agency

Lessons Learnt

Lessons Learnt

Trial implementation – validation of QA manual Formulate tools and assessment process before actual

implementation Start small to gain experience Training and development of assessors Review for continuous improvement Assist CLMV partners within ASEAN to catch-up Gain credibility before considering QA certification Reach out to QA organisations within and outside ASEAN Use of IT to hasten harmonisation and sharing best practices

Challenges

Globalisation of education

Rising demand for better quality university education

ASEAN Charter - promote greater mobility of students, faculty members, talent and labour

Harmonisation of AUN-QA and national/local quality assurance framework for higher

education

Quality Assurance in ASEAN

11 Countries: Different culture, language, political system, economy and HEI system

All have

IQA, but

EQA varies

Common commitment

to QA

Different approaches, but many similar criteria

Most have QA

agencies

Quality Assurance in ASEAN

Either No Quality Assurance Agency (QAA) or just being established

QAA without National Qualification Framework (NQF)

QAA + NQF

• Indonesia• Malaysia• Philippines• Thailand

• Brunei-Darussalam• Singapore • Cambodia

• Myanmar• Lao PDR• Vietnam• Timor Leste

SEAMEO RIHED: Quality Assurance Landscape in Southeast Asia

As of end 2011

Comparison of Assessment/Accreditation Criteria

AUN-QATimor Leste

Thailand Philippine Malaysia Indonesia Vietnam

1. Expected Learning Outcomes √ √ √ √ √ √

2. Programme Specification √   √ √ √ √ √

3. Programme Structure and Content √ √ √ √ √ √

4. Teaching and Learning Strategy   √ √ √ √ √

5. Student Assessment     √ √    

6. Academic Staff Quality √ √ √ √ √ √

7. Support Staff Quality √ √ √ √ √  

8. Student Quality √   √ √ √ √

9. Student Advice and Support √ √ √      

10. Facilities and Infrastructure √ √ √ √ √ √

11. Quality Assurance of Teaching and Learning Process

  √   √ √  

12. Staff Development Activities √ √ √   √  

13. Stakeholders Feedback √         √

14. Output √          

15. Stakeholders Satisfaction            

Harmonisation of AUN QA framework within and outside ASEAN

nus.edu.sg