1 1. Learning Outcomes At the end of the workshop, participants will learn to: Understand the AUN-QA...

1

AUN-QA Training Course for

Accomplishing Programme Assessment

1

Learning Outcomes

At the end of the workshop, participants will learn to:• Understand the AUN-QA criteria at institutional, IQA

system and programme levels;• Interpret the AUN-QA criteria for programme level;• Apply the PDCA approach to Self-assessment at

programme level;• Understand the requirements of Self-Assessment

Report (SAR);• Apply the technique for writing SAR; and• Appreciate the AUN-QA quality assessment

process

2

3

Quality in Higher Education

Excellence: The view of academia

Fitness for purpose: The view of external

assessors aimingfor improvement

Threshold: The view of accreditors

Value added: The view of students

Value for money: The view taxpayers and

governments

Client satisfaction: The view of students

and employers

Source: Green, D. What is Quality in Higher Education? Concepts, Policy and Practice 1994

M9QA in Higher Education

Qua

lity

4

Quality in Higher Education

Quality in higher education is a multi-dimensional concept, which should embrace all its functions, and activities: teaching and academic programmes, research and scholarship, staffing, students, buildings, facilities, equipment, services to the community and the academic environment. Internal self-evaluation and external review, conducted openly by independent specialists, if possible with international expertise, are vital for enhancing quality.

Source: The World Declaration on Higher Education for the Twenty First Century: Vision and Action (October 1998), Article 11, Quality Evaluation.

QA in Higher Education P6

5

What is Quality Assurance (QA)?

Quality assurance can be described as the systematic, structured and continuous attention to quality in terms of maintaining and improving quality.

Source: The Regional Report of Asia and the Pacific (UNESCO, 2003b)

QA in Higher Education

Quality assurance in higher education can be defined as systematic management and assessment procedures to monitor performance of higher education institutions.

Source: AUN-QA Manual for the Implementation of the Guidelines, P20

M20

6

QA Mechanisms

QA in Higher Education M20

QUALITY ASSURANCE SYSTEM

Internal Quality Assurance: • Monitoring• Evaluation• Improvement

Accreditation

External Quality Assurance: • Benchmarking• Audit• Assessment

7

IQA System

QA in Higher Education M22-25

Internal Quality Assurance

Follow up

Student Progress

Pass RateDrop-out Rate

Feedback from the Labour Market

and Alumni

Research Performance

Student Evaluation

Course and CurriculumEvaluation

Research Evaluation

ServiceEvaluation

AssuranceStudent

Assessments

Assurance QualityStaff

QualityAssurance Facilities

QualityAssurance

Student Support

SWOT Analysis

Inter-CollegialAudits

InformationSystem

QualityHandbook

MonitoringInstruments

EvaluationInstruments

Special QA Processes

Specific QAInstruments

8

External QA System


Stakeholder Satisfaction

Quality Assurance and (Inter)national Benchmarking

ACHIEVEMENTS

EducationalActivities

Research

CommunityService

Policy Plan

Management

HumanResources

Funding

Mission

Goals

Aims

9

Accreditation


Common Framework for Accreditation

Accreditation Framework

Quality Assessment

Model

Qualification Framework

Subject Benchmark Standards

Quality Concept

Basic Quality

Accreditation: Granting an institution or programme a quality mark that indicates certain standards have been met

10

Accreditation Agencies (within and outside ASEAN)


Country Accreditation Agency

USA Council for Higher Education Accreditation (CHEA) 1996

UK British Accreditation Council (BAC) 1984

Australia Australian Universities Quality Agency (AUQA) 2000

Cambodia Accreditation Committee of Cambodia (ACC) 2003

Indonesia National Accreditation Board for Higher Education (BAN-PT) 1994

Malaysia Malaysian Qualifications Authority (MQA) 2006

Thailand Office of the National Education Standards and Quality Assessment (ONESQA) 2000

The Philippines

Accrediting Agency of Chartered Colleges and Universities in the Philippines (AACCUP) 1989, Philippines Accrediting Association of Schools, Colleges and Universities (PAASCU) 1957

Vietnam General Department of Education Testing and Accreditation 2003Source: Adapted from Higher Education in the World 2007

11

Rationale for QA


• Quality graduates

• Labour market expectations

• Internationalisation of profession and globalisation

• Consumer protection

• From elite university to Institute of mass higher education

• Pressure to meet society’s needs

• Increasing importance of quality in higher education

• Student exchange and international cooperation

12

Evolution of AUN-QA

• 1995 - Establishment of ASEAN University Network (AUN)• 1998 – Establishment of AUN-QA Network• 1999 – Establishment of AUN-QA Network Task Force• 2000 – Establishment of the AUN-QA Centre for Chief Quality

Officers (CQOs)• 2001 – AUN-QA Common Policies, Criteria and Strategic Plan• 2004 – Endorsement of “AUN-QA Guidelines”• 2006 – Endorsement of the “Manual for the Implementation of

the Guidelines”• 2007 – Actual quality assessment at programme level• 2008 - Training of new AUN-QA assessors• 2011 - Guide to AUN Actual Quality Assessment at Programme

LevelGuidelines for AUN Quality Assessment & Assessors

• 2011 - DIES Training Course - ASEAN-QA

Evolution of AUN-QA M3-5

13

• Improve clarity and interpretation of quality criteria and standards

• Facilitate implementation of AUN-QA Guidelines

• Manual for Self-assessment and Auditing

M5-7

Manual for the Implementation of the AUN-QA Guidelines

Evolution of AUN-QA

14

Manual for the Implementation of the AUN-QA Guidelines

This manual aims to support the universities in the ASEAN region in:• Implementing the AUN-QA Guidelines

• Applying the AUN-QA Standards and Criteria

• Developing an adequate IQA System that fits the AUN-QA criteria and international developments

• Discovering their own quality by offering self-assessment instruments for IQA, the teaching/learning process and for some institutional aspects

• Applying for the AUN Quality label

M5-7Evolution of AUN-QA

Guide to AUN Actual Quality Assessment at Programme Level

Evolution of AUN-QA 15

16

• First revision of the AUN-QA Manual• Criteria and assessment process of

AUN Actual Quality Assessment at Programme Level

• Associated resources (templates and samples)

• Effective August 2011

Guide to AUN Actual Quality Assessment at Programme Level

Evolution of AUN-QA

17

AUN-QA Models

AUN-QA Models P7

Strategic(QA at

Institutional Level)

Systemic(Internal QA System)

Tactical(QA at Programme Level)

Stakeholders Satisfaction

Quality Assurance and (Inter)national benchmarking

ProgrammeSpecification

Programme Structure &

Content

StudentAssessment

Academic Staff Quality

Support Staff Quality

Student Quality

Facilities & Infrastructure

Quality Assurance of Teaching & Learning

StakeholdersFeedback

Pass Rates

Drop Out Rates

Employability

Expected

Learning

Outcomes

Achievements

Teaching & Learning Strategy

Student Advice & Support

StaffDevelopment

Activities

GraduationTime

Research

18

AUN-QA Models

QA at Programme Level (Revised)

P12AUN-QA Models

19

AUN-QA Models


Follow up

Student Progress



and Alumni


Student Evaluation


Research Evaluation

ServiceEvaluation

AssuranceStudent

Assessments



QualityAssurance

Student Support

SWOT Analysis


InformationSystem

QualityHandbook





Internal Quality Assurance (IQA) System

M30AUN-QA Models

20

QA at Institutional Level


Quality Assurance and (Inter)national Benchmarking

ACHIEVEMENTS

EducationalActivities

Research

CommunityService

Policy Plan

Management

HumanResources

Funding

Mission

Goals

Aims

AUN-QA Models

M71AUN-QA Models

21

1. The Mission Statement

QA at Institutional Level M73

Checklist:1. The Mission Statement1.1 The university has a clearly formulated mission statement1.2 The mission statement is publicly known1.3 The mission statement is in line with the academic and social context

Diagnostic questions:• What is the university's vision for the academic training it gives?• What is the university's vision for its research activities?• What is the university's vision for its role in the society?• Has the vision (= the long-term aims) been translated into a clearly formulatedmission statement (= targets to be met in the short-term)?• Has the mission statement been translated into achievable and operationalisedgoals and objectives?• What is the specific profile of this university compared with other universities inthe country in question?

22

2. The Policy Plan

M74

Checklist:2. The Policy Plan2.1 The university has a clear policy and strategic plan formulated in line with the mission statement.


Diagnostic questions:• Does the university have a clear policy in line with its mission and vision?• Has the policy been adequately translated into the strategic plan?• Who was involved in formulating the policy and strategic plan?• Are policy and strategic plan well known to all academic staff and students? Isthere general agreement on this?• Does the strategic plan reflect:

− The types of programme you are offering?− The choice of research fields?− The priorities set?− The main activities of the university?

23

3. The Management

M75

Checklist:3. The Management3.1 The university has a clear management structure in which the decision-making process, competencies and responsibilities have been clearly defined.


Diagnostic questions:• What kind of management structure does the university have: centralised and top down or decentralised and bottom up?• Have the role and functions of the central management, faculty management and the staff been clearly described?• Does the academic staff participate in the decision-making process on teaching and research?• Do students participate in the decision-making process in relation to theireducation?• Has the management structure of the university been endorsed by the academic community?• Is the internal organisation structure fit for purpose?• What management committees are in place? Are they working adequately?

24

4. Human Resources

M76-77

4. Human Resources4.1. The university develops and retains high-quality academics and support staff by clearly defining their responsibility, and by evaluating their performance on a regular basis.4.2. The university develops the body of knowledge possessed by its academics and support staff to keep paces with changes in each academic discipline.


Diagnostic questions:1. How does the university select and appoint its academic staff?2. Is an adequate staff appraisal system in place for use in evaluating performance and promotion?3. How is staff performance evaluated?4. What opportunities are given for staff/HR development and training?5. How does the university evaluate the efficiency of its staff/HR developmentactivities?6. How does the university stimulate the ethics of its students, academics and other staff?

25

4. Human Resources

M76-77

Checklist:4. Human Resources4.3. The university:a) establishes a system to consider the ability, potential and need toenhance the knowledge possessed by its academics and support staff in conducting activities that have a direct influence on the quality of teaching-learning. This should include the formulation of a concrete staff development plan;b) provides training to develop the potential of academics and Support staff in accordance with this plan;c) evaluates the efficiency of the training provided to ensure that its academics and support staff comprehend both the importance of and the relationship between the duties and activities that fall within their responsibility. This will affect the way the organisation attains its quality goals.d) Compile records of education, experience, training, and other essential qualifications required of lecturers (academic staff) and support staff.


26

4. Human Resources

M76-77

Checklist:4. Human Resources4.4 The university sets up a system of evaluation by committee to be conducted according to a set timetable at least twice a year prior to pay salary increments or promotions, or to the imposition of penalties.4.5. The university establishes an activity plan and evaluates activities to encourage students, academics and other staff to be conscientious in thought, speech, and behaviour, to be kind, compassionate and honest, to possess equanimity, to be circumspect, logically-minded and far-sighted, to be responsible and willing to make sacrifices for the good of society.4.6. The university enhances the professional ethics of its students, academics and other staff.


27

5. Funding

M77-78

Checklist:5. Funding5.1 The university has adequate funding to achieve the goals and aims.


Diagnostic questions:1. How is the university funded? (What percentage of the budget is public funding, student contributions, external funding?)2. The sources of the financial resources and the conditions attached to the funding are stated transparently and do not restrict the university's decision-making autonomy in teaching and research?3. Are the goals and aims realistic and achievable with the provided funding?

28

6. Educational Activities

M78

AUN-QA Criterion:• Based on AUN-QA criteria on teaching/learning (see self-assessment at

programme level)

Checklist:6. Educational Activities6.1 Use the outcomes of the self-assessment at programme level taking into

account the aspects from Appendix 5 – checklist on quality of a programme.

Diagnostic questions:Refer to diagnostic questions in self-assessment at programme level


29

7. Research

M79-80


Quality Assurance and (Inter)national benchmarking

ResearchPolicy

ResearchProgramme

Number ofPublications

Number ofDissertations

ResearchManagement

Organisation Research Code of Ethics

Funding Facilities External Funding

Goalsand

Aims

Research Output

Human Resource

ResearchTraining

Support PhDStudents

Awards StaffMembers


30

7. Research

M157

Checklist:7. Research7.1 The university establishes, implements and ensures uniform compliance

with university-wide research policies to maintain the integrity of the university, protect the safety and welfare of staff and experimental subjects and to ensure compliance with all other regulations governing the research process.

7.2 The university has designed policies and guidelines as guiding principles to conduct research and development activities.

7.3 The policies and guidelines set out the obligations on all researchers to be aware of good conduct in research and to comply with institutional and regulatory requirements.

7.4 The university supports scholarly research and creative activities that contribute to the mission of the university and ultimately provide intellectual, social and economic benefits to society.


31

7. Research

M157

Checklist:7. Research7.5 The university is committed to the highest professional standards of

scholarly research and research ethics.7.6 The researchers have familiarised themselves with the contents of

research policies and procedures. Misconduct in conducting or reporting research is considered a serious breach of academic responsibilities.

7.7 The university has a clear research policy, setting the direction of research and deciding on the research profile and research activities.

7.8 The university has a clear code of conduct for research including a code of ethics.


32

8. The Contribution to Society and the Community

M84-85

Checklist:8. The Contribution to Society and the Community8.1 The university has clear guidelines on consultancy and on the contribution to society and community service


Diagnostic questions:• What role does the university play in the local, national and international community• What are the key activities, which of these lie outside normal teaching or research? How do they relate to the mission?• What are the non-profit activities of the university?• Is there a clear policy on consultancy and the contribution to society and the community?• How is the income from consultancy regulated?

33

9. Achievements

M85-86

Benchmark Criterion:• A university has the means and opportunity to check whether the

achievements are in line with the expected outcomes.

Checklist:9. Achievements9.1 The university has the means and opportunities to check whether the achievements are in line with the expected outcomes.


Diagnostic questions:• Are the achieved outcomes in line with the formulated goals and aims?• How does the university check that it achieves what it wants to achieve?• If the achievement is not satisfactory, what action does the university take?

34

10. Stakeholder Satisfaction

M86

Checklist:10. Stakeholder Satisfaction10.1 The university has a structural method for obtaining feedback from stakeholders.


Diagnostic questions:• Is regular student evaluation carried out? How is it done? Is it adequate?• What is done with the results of student evaluations?• Does the university have an insight into the opinion and feedback of

graduateswhen they are employed?• Are the complaints or positive feedback received from alumni used to adapt

the programmes?• Are there any structured contacts with employers and the labour market forobtaining feedback?• How do the employers appreciate graduates?• Are there any specific complaints?• Are specific strengths appreciated by employers?• Does the university have any tools to obtain feedback from society?

35

11. Quality Assurance and (Inter)national Benchmarking

M87, M151

Benchmark Criterion:• A university has an efficient internal quality assurance system.

Checklist:11. Quality Assurance and (Inter)national Benchmarking11.1 The university uses the outcomes of the self-assessment of the IQA system (see Appendix 4).

Diagnostic questions:Refer to diagnostic questions in self-assessment of the IQA System.


36

QA at IQA System Level


Follow up

Student Progress



and Alumni


Student Evaluation


Research Evaluation

ServiceEvaluation

AssuranceStudent

Assessments



QualityAssurance

Student Support

SWOT Analysis


InformationSystem

QualityHandbook





Internal Quality Assurance (IQA) System

QA at IQA System Level M30

37

1. Quality Assurance: General Aspects

M31-32

AUN-QA Criterion:• An institution has a clear policy and associated procedures for the

assurance of the quality and standards of their programmes and awards. The university commits itself explicitly to the development of quality culture and quality awareness.

• To achieve this, the university develops and implements a strategy for the continuous enhancement of quality. The strategy, policy and procedures should have a formal status and be publicly available.

• They also include a role for students and other stakeholders.

Checklist:1. Policy1.1 The institution has a clear policy on IQA1.2 There is a clear formal strategy on IQA1.3 The role of all stakeholders is clearly described


38

1. Quality Assurance: General Aspects

M31-32

Diagnostic questions:• Does the university have a clearly formulated policy on IQA?• Does the policy statement explicitly contain:

- the relationship between teaching and research at the institution;- the institution's strategy on quality and standards;- how the quality assurance system is organised;- the responsibilities of departments, schools, faculties and otherorganisational units and individuals for assuring quality;- the involvement of students and stakeholders in quality assurance;- the ways in which the policy is implemented, monitored and revised

Examples:


39

2. The Monitoring System

M32-33

AUN-QA Criterion:An institution has a structured monitoring system to collect information on the quality of its activities. The monitoring system includes:- Student evaluation- A student progress system- Structural feedback from the labour market- Structural feedback from alumni- Number of publications- Number of grants

Checklist:2. Monitoring2.1 Student evaluation2.2 Student progress system2.3 Structured feedback from the labour market (employers)2.4 Structured feedback from the alumni


40

2. The Monitoring System

M32-33

Diagnostic questions:Does the university have an efficient monitoring system including:

- a system to track student progress- a system to register pass rates and dropout rates- structural feedback from the labour market (employers)- structural feedback from alumni- records concerning the number of publications registered by staff- records on the number of research grants

Examples:


41

3. Evaluation Instruments

M33-34

AUN-QA Criterion:An institution has formal mechanisms for the periodic review or evaluation of its:• core activities• programmes and degrees• research activities (if applicable)• contribution to society and the community

Checklist:3. Periodic Review of the Core Activities (Education, Research and the Contribution to Society and the Community)3.1 Period review of teaching and learning3.2 Period review of research3.3 Periodic review of the contribution to society and the community


42

3. Evaluation Instruments

M33-34

Diagnostic questions:Does the university carry out:

• student evaluation• course evaluation • curriculum evaluation • regular review of research outcomes• regular evaluation of the contribution to society and the community

Examples:


43

4. Special QA Processes: Student Assessment

M34-35

AUN-QA Criterion:• An institution has clear procedures to assure the assessment of

students.• Students are assessed on the basis of published criteria,

regulations and procedures that are applied consistently.• There are clear procedures to assure the quality of the

examinations.• There is an appeals procedure.

Checklist:4. Quality Assurance of the Student Assessment4.1 Criteria for assessment4.2 Assessment procedures4.3 Regulations to assure the quality of assessment4.4 Appeal procedures


44

4. Special QA Processes: Student Assessment

M34-35

Diagnostic questions:• Does the student assessment meet the principles of adult learning? Adults prefer to be assessed by criterion-referenced methods and by a combination of peer, self and teacher assessment.• Does the assessment method foster open, flexible, reflective and outcome based assessment?• Are the criteria made explicit?• Are the assessment strategies in line with clearly-defined learning outcomes?• Do the assessment arrangements correspond with all the aims and aspects of the curriculum as taught?• Is a range of assessment methods used in a planned manner to servediagnostic, formative, and summative purposes?• Is the scope and weighting of the assessment schemes known to allconcerned?• Are the standards applied in assessment schemes explicit and consistent across the curriculum?• Are procedures regularly applied to ensure that, as far as possible, assessment schemes are valid, reliable and fairly administered?• Do students have ready access to reasonable appeals procedures?• Is the reliability and validity of the assessment methods documented as required and regularly evaluated?• Are new assessment methods developed and tested?


45

5. Special QA Processes : Staff Quality

M35-36

AUN-QA Criterion: An institution has means to satisfy itself that its staff are qualified and competent to conduct the core activities of the institution: education, research and the contribution to society and the community: adequate staff appointment procedures, adequate staff appraisal system, staff development activities

Checklist: 5. Quality Assurance of Staff5.1 Staff appointment procedures5.2 Staff appraisal system5.3 Staff development activities


Diagnostic questions:• How is the staff recruitment system organised?• How is the promotion system organised? What criteria are important for promotion?• Are staff appraisals carried out? How are these done? What are the consequences?• Has a clear HR policy been put in place?• Are staff development activities carried out?

46

6. Special QA Processes: Quality of the Facilities

M36-37

AUN-QA Criterion: An institution has clear procedures to ensure that the quality of its facilities needed for student learning are adequate and appropriate for each programme offered:• adequate checks on the computer facilities• adequate checks on the library• adequate checks on the laboratories

Checklist: 6. Quality Assurance of Facilities6.1 Checking the computer facilities6.2 Checking the library facilities6.3 Checking the laboratories


Diagnostic questions:What procedures do you have to assure the quality of:

• the lecture halls, tutorial rooms, auditorium, etc?• the libraries? • the laboratories?• the learning resources • the research resources?

47

7. Special QA Processes: Quality of Student Support

M37

AUN-QA Criterion: An institution has clear procedures to assure the quality of the student support and student advice. In establishing a learning environment to support the achievement of quality student learning, teachers must do everything in their power to provide not only a physical and material environment that is supportive of learning and is appropriate to the activities involved, but also a social or psychological environment.

Checklist: 7. Quality Assurance of Student Support/Advice7.1 Information for students 7.2 Student advice/support7.3 student welfare 7.4 Student housing7.5 Sports facilities


Diagnostic questions: What procedures to you have to assure the quality of the student support activities:

• a tutoring system • student advice and/or counselling• student housing • sports facilities

48

8. Special QA Instruments: SWOT Analysis or Self-assessment

M38

AUN-QA Criterion:An institution regularly conducts, but at least once every 5 years, a self-assessment of its core activities and of the institution as a whole to learn about its strengths and weaknesses. This self-assessment will lead to a quality plan.

Checklist: 8. Self-assessment8.1 Self-assessment of the IQA system8.2 Self-assessment of teaching/learning8.3 Self-assessment of research8.4 Self-assessment of the contribution to society and the community8.5 Self-assessment of the university


Diagnostic questions:Does the university already have experience with the instrument of self-assessment?• Is there any connection with external assessment/accreditation?• Are the core activities being assessed?• If not yet done, are you planning to conduct self-assessment on a regular basis?• If you have (not yet) conducted self-assessment, how do you know what your quality is?

49

9. Special QA Instruments: Inter-collegial Audit

M38-39

AUN-QA Criterion:A self-assessment might be part of an External Quality Assessment (EQA) or accreditation process where the self-assessment report acts as input for the external review team. If the self-assessment is not connected to the EQA, the institution will be expected to organise an audit itself based on the self-assessment report.

Checklist: 9. Internal Audit9.1 Internal audit of teaching/learning9.2 Internal audit of research9.3 Internal audit of the contribution to society and the community9.4 Internal audit of the university


Diagnostic questions:Does the university have an audit system?• How often do you use it?• Do you have trained auditors? Where were they trained?• What is done with the outcomes of an audit? Give some examples.

50

10a. Special QA Instruments: Information Systems

M39-40

AUN-QA Criterion:

An institution should ensure that it collects, analyses and uses relevant information for the effective management of its core activities.

Checklist: 10a. Information Systems10a.1 Information management system in general10a.2 Information management system for teaching/learning10a.3 Information management system for research


Diagnostic questions:• What is done with the information collected by the monitoring

system?• Does the executive management use the information collected?• Are you using the instrument of benchmarking? How are you using it?

51

10b. Special QA Instruments: Public Information System

M40

AUN-QA Criterion:

An institution should regularly publish up-to-date, impartial and objective information, both quantitative and qualitative, about the programmes and awards that it offers.

Checklist: 10b. Public Information10b.1 Public information on the university10b.2 Public information on the educational programmes and awards/degrees offered10b.3 Public information on research activities


Diagnostic questions:• What procedures does the university have to assure the

quality of the information?• How do you ensure that the information is impartial?

52

11. Special QA Instruments: QA Handbook

M40-41

AUN-QA Criterion:An institution has a QA handbook that documents all regulations, processes and procedures concerning quality assurance. This handbook is public and known to all the people concerned.

Checklist: 11. QA Handbook11.1 Presence of a QA handbook11.2 Handbook is known to staff and students


Diagnostic questions:• Does the university already have a QA handbook?• What is the content of the QA handbook? Sum up the chapters.• What documents, processes and procedures are already available?• What documents, processes and practices need to be developed?• Who will collect and compile all the information?

1 1. Learning Outcomes At the end of the workshop, participants will learn to: Understand the AUN-QA...

Documents

Transcript of 1 1. Learning Outcomes At the end of the workshop, participants will learn to: Understand the AUN-QA...