Quality Assurance in internationalisation

Introduction

• UGent: - Decentralised institution

- Very active international actor

- Important role for individual academic staff member

- 530 Erasmus partners

• 2011: Growing need for central guidance

• Strategic Plan: “Create quality assessment system for international agreements”

- > Aim to make choices based on facts: structural use of data

- > Need for data

Data?

Data?

• Huge amounts of data is generated every minute, hour, day…

• Overload of data

• Majority of all data is unstructured

• Data sources are traditional and digital

• Data is generated within and without the organization

• Data scientist are sexy

– Data scientist will be the hottest job of the 21st century

– Data experts will be a scarce, valuable commodity

• But we need to be careful with our conclusions: “in 2009 in Belgium an average family counts 2,31 persons, but I still need to meet the first family with 2,31 persons!”

Data?

• 5 important questions need to be asked

– What kind of information are we looking for?

– Which sources do we have to our disposal?

– What data do we use?

– Quantity vs. Quality?

– How will we make our data attractive and easy to use?

Data?

• Quantitative analysis: % of graduates with international experience

– 2011-2012: 4555 graduates (at MA level), 716 of them had an international experience (at BA or MA level): 15,7%

– 2012-2013: 4746 graduates (at MA level), 855 of them had an international experience (at BA or MA level): 18%

Be careful: don’t use the combination of mobility's and graduates for a given academic year

Data?

• Quantitative analysis: Exchange numbers - outgoing

0 200 400 600 800 1000

88-89

…

07-08

08-09

09-10

10-11

11-12

12-13

Erasmus

Erasmus Belgica

Erasmus Placement

Institutional Bilateral Agreements

Other programmes

Data?

• Quantitative analysis: Exchange numbers - incoming

0 100 200 300 400 500 600 700 800 900 1000 1100

07-08

08-09

09-10

10-11

11-12

12-13

Erasmus Erasmus Belgica Erasmus Placement Erasmus Mundus Action 2 Individual Contacts IAESTE Institutional Bilateral Agreements Other programmes

Data?

• Quantitative analysis: Exchange Balance

0

100

200

300

400

500

600

Outgoing students Incoming students

Data?

• Quantitative analysis: International Students

0 20 40 60 80

100 120 140 160 180

Exchange Degree Phd

Country Average credits/day*

Average study succes** Reference group

Denmark 0,17 94% 12 Germany 0,15 78,18% 100 Finland 0,15 70,7% 34 France 0,17 74,11% 81 Italy 0,15 58,82% 174 Lithuania 0,16 87,85% 39 Norway 0,16 73,5% 6 Austria 0,16 65,83% 11 Poland 0,16 77,45% 117 Portugal 0,18 71,14% 50 Spain 0,16 78,91% 354

*30 credits/sem = +/- 0,2 credits/day

**UGent average = 80% Exchange average = 70%

Data?

Data?

• Qualitative analysis: Erasmus Student Survey

36.38%

55.25%

43.67%

73.08%

68.13%

54.87%

83.58%

64.87%

21.37%

32.84%

36.16%

41.33%

54.81%

61.12%

73.63%

81.46%

0.00% 20.00% 40.00% 60.00% 80.00% 100.00%

the reception/welcome

culture and society of host country

preparatory language courses

the host institution

the Learning Agreement and grade transfers

study programme and academic calander

administrative obligations

housing and accomodation

what did you wanted to know before departure? wat did you know before departure?

Data?

• Qualitative analysis: Erasmus Student Survey

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

new scientific and academic knowledge

new course content

new learning and teaching methods

During your Erasmus exchange you were confronted with

Excellent Good Sufficient Insufficient Unappliquable

Data?

• Qualitative analysis: Internal Quality Assessment Survey

19% 30% 27% 16% 8%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

MA

Item 8: You felt stimulated to undertake an international experience

strongly disagree disagree neutral agree strongly agree

8% 21% 21% 38% 11%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

MA

Item 15: During your studies you were confronted with international staff and students

strongly disagree disagree neutral agree stronly agree

Data?

• Qualitative analysis: Internal Quality Assessment Survey

6% 18% 39% 28% 9%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

MA

Item 21: International contacts and content are an added value for your study programme

strongly disagree disagree neutral agree stronly agree

Data?

• Qualitative analysis: Student Surveys: incoming students

How to use these data…

• How to use all this information for our quality assessment system for international agreements?

• What is quality in international cooperation?

• What makes a good partner good?

• On what data do we base our judgment?

Structural Partner Analysis

Quality Assessment Tool for International Cooperation (QuATIC) • Measuring quality of cooperation

• Based on limited indicators but maximum information

• With low work load for the staff involved

Structural Partner Analysis

Critical success factors • Grasping the complex concept of ‘quality’

• Grasping the diversity in internationalisation

• Grasping the decentralised character of UGent

• Minimal administrative workload for staff

• Based on objective criteria

Structural partner analysis

Quality of international cooperation Quality of the partners

involved

Impact of the cooperation Quality of the information exchange

What is quality in International Cooperation?

Structural partner analysis

Which indicators • Parameters for choice of indicators -> limited number!

- Relevance

- Availability

- Measurability

- Unique source of information

- No Matthew Effect?

• The data

• From data to a score

Structural partner analysis

Which indicators • Cluster 1: Quality of the partners

• C1I1: International Rankings ONLINE

• C1I2: Quality of incoming students OASIS

• C1I3: Support for outgoing students AEQ

• C1I4: Quality of the education AEQ

• Cluster 2: Quality of the information exchange • C2I1: Communication NEW

• C2I2: Availability of Information AEQ

• Cluster 3: Impact of the cooperation • C3I1: Involvement MoveOn

• C3I2: Mobility Rate MoveOn

• C3I3: Research Cooperation ReDeDa

• C3I4: Education Cooperation MoveOn

Structural partner analysis

Cluster 1: Quality of partner – International Rankings ONLINE

• Shanghai • THE • QS

- Quality of incoming students OASIS

• Study load per day • Study success

- Support for outgoing students AEQ

• Student satisfaction concerning academic support • Student satisfaction concerning administrative support

- Quality of the education AEQ

• Broadening of scientific or professional knowledge • Getting acquainted with new content • Getting acquainted with new pedagogical methods and tools

Structural partner analysis

Cluster 2: Quality of information exchange - Communication with the partner NEW

• Administrative (selection, nomination…) • Academic (offer, results…)

- Availability of information for students AEQ • Housing • Administrative obligations • Study Programme

Structural partner analysis

Cluster 3: Impact of the cooperation - Involvement MOVEON

• Number of faculties involved • Number of departments involved

- Mobility rate MOVEON • Activity level: annual mobility of students and staff? • Balance in student exchange

- Research cooperation ReDeDa • Joint PhDs • Joint Research Projects • Joint Publications (not available yet)

- Educational cooperation MOVEON • Joint programmes • Joint projects

Result: QuATIC

Quality Assessment Tool for International Cooperation • List of partners

• A score per partner per indicator

• Reports per partner with strengths and/or weaknesses in the cooperation

Result: QuATIC

Resultaat

Result: QuATIC

0

2

4

6

8

10 C1I1 - Ranking

C1I2 - Incoming Students

C1I3 - Student Support

C1I4 - Academic Value

C2I2 - Information Availability C3I1 - Involvement

C3I2 - Mobility Rate

C3I3 - Research Collaboration

C3C4 - Educational Collaboration

NL GRONING01

Result: QuATIC

Result: QuATIC

Next on the agenda

Towards an online peer assessment tool – External funding

– As many user institutions as possible

– Based on data sets that are available

– Delivering indicators that are relevant

– Peer quality assessment trough structural data exchange

Contact

Paul.Leys@UGent.be

Thijs.Verbeurgt@UGent.be