Evaluation of Key Project Deliverables in Part Fulfilment ... 1.227 Making Qualit… · (NCFHE) to...

Operational Programme II – Cohesion Policy 2007-2013

Empowering People for More Jobs and a Better Quality of Life

Training part-financed by the European Union

European Social Fund (ESF)

Co-financing rate: 85% EU Funds; 15% National Funds

Investing in your future

Evaluation of Key Project Deliverables in Part Fulfilment of ESF Project 1.227 ‘Making Quality Visible’

Quantitative Report 2015

2

Table of Contents

Background 3

Findings and Analysis 5

Profile of Respondents 5

Role of the Respondents 5

CPD Award 6

Seminar for Prospective Expert Evaluators of the External Quality Audits 8

Seminar for Prospective Student Evaluators of the External Quality Audits 10

Net-QAPE Meetings 11

Quality Audit Process 12

Concluding Conference 13

Overall Project 14

Appendix I: Tables 16

Profile of Respondents 16

Role of the Respondents 19

CPD Award 20

Seminar for Prospective Expert Evaluators of the External Quality Audits 45

Seminar for Prospective Student Evaluators of External Quality Audits 65

Net-QAPE Meetings 85

Quality Audit Process 98

3

Background

Grant Thornton has been engaged by the National Commission for Further and Higher Education

(NCFHE) to carry out an evaluation of the deliverables forming part of the ESF Project 1.227 “Making

Quality Visible”. For the evaluation exercise, Grant Thornton utilised both quantitative and qualitative

research tools. The aim of this report is to present the results of the data gathered through the

quantitative research exercise.

The data for the quantitative exercise was collected through Computer Assisted Web Interviewing

(CAWI), whereby interviewees were invited to complete the survey online within a one-month period.

It is to be noted that there were three categories of participants who participated in the ESF Project

1.227 “Making Quality Visible”, that is, administrators, tutors and students. Thus, a different

questionnaire was specifically designed for each category of participants. However, some participants

had multiple roles in the project, for example, some participants were administrators and tutors, and

others were administrators, tutors, and students. Thus, in order to reduce inputting mistakes and to

minimise missing information, it was decided that an additional two distinct questionnaire were

developed specifically targeting those who had multiple roles in the Project. The finalised five versions

of the questionnaires were forwarded to the NCFHE for approval.

Subsequent to the attainment of this endorsement, the finalised questionnaires were uploaded on Grant

Thornton’s data entry system and the link was sent to the individuals listed in the database provided by

NCFHE via email. Every week, Grant Thornton sent an email reminding the participants to to fill in

the survey. Furthermore, the links of the surveys were sent to the three educational institutions (the

University of Malta, the Institute of Tourism Studies and the Malta College for Arts, Science and

Technology) involved in the project, as originally agreed upon with the Project Manager of the

evaluation, to be forwarded to their employees who participated in the Project. Initially, due to a number

of misunderstandings, some issues were encountered with these institutions, and the links were not

forwarded to the end participants. The quantitative research was launched during the first week of

September 2015 and was originally targeted to be completed by the last week of September 2015.

However, upon the end of September, Grant Thornton was informed that MCAST intended to finally

forward the link of the questionnaires to the participants within MCAST. Thus, it was decided that the

collection period be extended by another two weeks in order to improve the response rate. At the end

of the collection period, in total, 122 participants completed the on-line questionnaires. It is to be noted

that Grant Thornton could not obtain the number of participants to whom MCAST had sent the link to

the surveys, thus, it is unable to estimate the response rate.

4

The following sections present an analysis of the data collection from the quantitative exercise.

5

Findings and Analysis

Profile of Respondents

The majority of the respondents were mainly male (54.1%) and mostly were aged 46+ (46.7%). Tutors

and administrators were also asked about their highest level of education attained, and the vast majority

completed tertiary education (75.4%). The administrators were also asked about their current position

within the educational institutions - with most of them currently holding a combined position as an

administrator / manager / director (37.7%).

Role of the Respondents

The respondents were asked about their role in the ESF Project 1.227 “Making Quality Visible”. Indeed,

participation in the CPD Award, Expert Evaluators Seminar, Student Evaluators Seminar and Net-

QAPE Meetings was dependent on the position of the respondents within the educational institution.

On the other hand, the EQA and the Concluding Conference were open to all the participants. Focusing

on the specific activities, the majority of the respondents participated in the Net-QAPE Meetings

(36.1%) while for the general activities, the majority of the respondents participated in the Final

Conference (40.2%).

6

CPD Award

As shown in the previous graph 19.7% of the respondents participated in the CPD Award. These

participants were asked whether they received a certificate, and 18.0% of the respondents stated that

they did get a certificate. The main reasons for not completing the course principally relate to the change

in circumstances of the institution (0.8%), the lack of time (0.8%), and due to travelling (0.8%).

Overall, respondents believed that the course was well organised (10.6%). Indeed, they believed that

detailed information was provided on the course (14.8%), the learning outcomes of the course were

clearly stated (16.4%), and the information on the course was provided well in advance (12.3%).

Respondents were also asked about the course content. In this regard, the respondents described the

course as being practical (12.3%) and that it reflected projected External Quality Audit requirements

(13.1%). Furthermore, the respondents stated that lecturers allowed for active participation and

interaction (15.6%).

Respondents were asked additional questions on the lecturers. The majority of the respondents were

satisfied with the lecturers’ performance. Indeed, the respondents believed that they were well prepared

for the delivery of the courses (13.1%) and that they had good knowledge of the subject (16.4%).

Furthermore, they also believed that the lecturers managed to use the technology available effectively

(12.3%) and that they provided useful material (12.3%). The respondents were also asked to identify the

main strengths of the lecturers. In this regard, the respondents mainly mentioned their knowledge of

the subject, as well as their experience in Quality Assurance which lecturers used as examples during the

lectures.

A number of questions related to the learning resources made available during the delivery of the course.

Once again, the respondents were satisfied with the learning resources. The respondents believed that

the resources were enough to meet their needs (12.3%). For the respondents, the lecture room was of

an acceptable size (16.4%) and the equipment used was useful for their learning (14.8%). The educational

material made available also helped respondents to understand the topics (13.1%). The respondents

found the educational material to be up to date (15.6%), easy to understand (14.0%), well structured

(14.0%), and relevant (16.4%).

Overall, the respondents found the teaching methods to be good. The teaching methods mainly relate

to the lectures (13.9%), presentations (15.6%), group activities / discussions (13.3%), and problem

handling (10.7%).

7

In the survey, a number of questions were also asked to gauge the respondents’ commitment to the

course. From the feedback provided it seems that the respondents were committed to the course.

Indeed, it appears that they attended almost all the lectures (18.0%) and that they were involved in the

group discussions (18.1%). Over and above the lectures, the respondents also carried out the

recommended hours of self-study (17.2%) and made use of the hours made available to them by the

lectures outside the lecture room (12.3%).

When questioned on the assessment methods, the majority of the respondents were satisfied with the

assessment. Indeed, the majority regarded the assessment as being appropriate for the course (12.3%).

Furthermore, the respondents believed that the assessment gave them the opportunity to show their

knowledge (14.0%) and that they were given enough time for the assessment (15.6%). Participants also

agreed that they were informed about the evaluation criteria of the assessment (16.4%) and that they

were given clear feedback on their performance during the assessment (11.5%).

Overall, respondents were either very satisfied (4.1%) or satisfied (10.7%) with the CPD Course.

Respondents were also prompted to provide recommendations on how the course could be improved.

In this regard, a number of relevant recommendations were put forward and the following list

summarises the main recommendations mentioned in the on-line questionnaire:

1. Replacing Professor Suzanne Gatt (main tutor) on the programme

2. The workload of the course should be reflective of the credit value of the programme. Apart

from the direct lecturing and seminar hours, the amount of time as preparation for the

assignment and expected deliverables should have also be taken into consideration.

3. Workshops need to be effectively managed. Participants left the workshops when these were

still underway and it did not give the opportunity to participants to interact with individuals

from other institutions.

4. The NCFHE should not implement and run an accredited course due to conflict of interest.

The goal posts were continuously changing throughout the course and the certificate was also

issued by NCFHE which reserved the right to withdraw it unless participants continue to submit

the material it requests.

5. Shorter lectures and more frequent

6. More to the point and concise materials

7. The assessment was unfair on participants who are not academics

8. Provision of more individual feedback to participants

9. More involvement of foreign speakers

10. Dissemination of reading material prior to the lectures

11. Dissemination of a training manual to participants

8

12. Venue of the training should be in a central location

13. Lectures need to be more interactive

14. A timetable for the sessions should be in place with no short notice changes

15. Organisation of visits to institutions to show the practices of each other rather than just talking

about them.

Seminar for Prospective Expert Evaluators of the External Quality Audits

Around 19.7% of the respondents also participated in the Seminar for Prospective Expert Evaluators of

the External Quality Audits. A number of questions were asked on the overall organisation of this

seminar. In this regard, the respondents believed that the seminar was well organised (18.8%).

Respondents agreed that detailed information was provided on the workload and input required (18.0%),

and that the information was provided well in advance (16.4%). Furthermore, the learning outcomes of

the seminar were also clearly stated (18.0%).

The majority of the respondents were satisfied with the content of the seminar. In this regard, the

majority of the respondents agreed that the seminar was practical (18.9%) and that it reflected projected

External Quality Audit requirements (18.8%). Respondents also thought that the lecturers allowed for

active participation and interaction during the seminar (18.0%).

The respondents were asked a number of questions on the lecturers who delivered the seminar. Overall,

respondents agreed that the lecturers were well prepared (18.0%) and had good knowledge of the subject

(18.1%). Furthermore, the lecturers effectively used the technology available for the delivery of the

seminar (16.4%). The lecturers also provided useful material (17.2%) and managed to make the subject

interesting (18.1%). When asked to identify the main strengths of the lecturers, the respondents pointed

out the experience of the lecturers.

The respondents agreed that the learning resources made available for the seminar were enough to meet

their needs (18.1%). They found the equipment in the lecture room to be useful for their learning

experience (16.4%). The lecture room was deemed to be of an acceptable size for the number of

participants attending the seminar (18.0%). When asked about the educational material made available

to them, they agreed that it helped them to better understand the topics being tackled (18.1%). They

also found the educational material up to date (18.1%), easy to understand (17.2%), well-structured

(18.9%). and relevant (18.8%).

9

When asked about the teaching methods, most of the respondents found the lectures (10.7%),

presentations (11.5%) and problem handling (9.8%) as being good. However, they mostly appreciated

the group activities and discussions as they described them as being very good (13.1%).

Respondents were asked a number of questions on additional considerations related to the seminar. The

respondents were very involved in the group discussions that developed during the seminar (19.7%).

This was further encouraged due to the conducive environment which made the respondents feel

respected (18.0%) and at ease in the sharing of opinions and in asking questions (17.2%). The seminar

met the expectations of the participants (18.1%) and they strongly agreed that they would recommend

the seminar (18.1%). Furthermore, the respondents also stated that the seminar helped them gain in-

depth knowledge on the subject (18.1%).

Overall, the respondents were either very satisfied (9.0%) or satisfied (9.0%) with the seminar.

Nonetheless, the respondents provided a number of recommendations on how the seminar could be

improved. In this regard they mentioned:

1. Improving the first session of the European evolution of quality in education - should have

focused more on the local scenario.

2. More practical work and simulations

3. Improving the seating arrangements

4. Venue to be centrally located

5. Longer session or organisation of a follow up session

6. Simulation sessions to be organised in rooms rather than a hall

7. Offering more varied food

8. Information needs to be provided before

9. More subjects need to be discussed

10. Smaller groups

10

Seminar for Prospective Student Evaluators of the External Quality Audits

Focusing on the seminar for prospective student evaluators of the External Quality Audits, around

13.1% of the respondents attended this seminar. Overall, the respondents were satisfied with the

organisation of the seminar. Indeed, they stated that the seminar was well organised (10.7%). The

respondents also agreed that detailed information was provided on the seminar workload and input

required (9.8%), and that the learning outcomes of the seminar were clearly stated from the beginning

(8.2%). Whereas most of the respondents agreed that information on the seminar was provided well in

advance (5.7%), there was also a high percentage of respondents (4.9%) who did not have any opinion

in this regard.

When questioned on the seminar content, the majority of the respondents strongly agreed (7.4%) with

the statement that the lecturers allowed for active participation and interaction during the delivery of the

seminar. Also, the respondents agreed that the seminar was practical (9.1%) and that it reflected the

projected External Quality Audit requirements (9.1%).

The respondents were also asked questions on the lecturers who delivered the seminar. In this regard,

the respondents strongly agreed that the lecturers had good knowledge of the subject (8.2%) and that

they made the subject interesting (5.7%). There was also general agreement that the lecturers were well

prepared and fully resourced (11.5%), they used technology effectively (8.2%), and provided useful

material (9.9%). When respondents were asked to identify the main strength of the lecturers, they

mentioned their experience and knowledge of the subject.

Respondents were also asked to express their opinion on the learning resources used during the seminar.

Although there was a high percentage of respondents who did not have opinion on whether the

resources made available were enough to meet their needs (4.9%), the majority agreed that these were

enough (6.5%). Respondents believed that the size of the room was acceptable for the number of

persons attending the seminar (12.3%) and that the equipment used was useful during the delivery

(10.7%). When asked about the educational material, most of the respondents (6.6%) agreed that these

were useful in helping them understand the topics. However, there was another 4.9% who felt neutral

on this point. Despite this, there was a general consensus that the educational material was up to date

(9.8%), easy to understand (9.1%), well structured (8.2%), and relevant (12.3%).

The respondents answering this section stated that they were satisfied with the teaching methods.

Indeed, they stated that the lectures (10.6%), presentations (11.5%), group activities/discussions

(10.7%), and problem handling (12.3%) were either very good or good.

11

Respondents were also asked their opinion about a number of other aspects related to the delivery of

the seminar. Respondents were involved in the discussions that developed during the seminar (13.1%)

and they stated that the seminar met their expectations (9.9%). The respondents felt respected (12.7%)

and they strongly agreed that they felt at ease in sharing their opinions and in asking questions (6.6%).

Respondents also agreed that the seminar helped them gain in-depth knowledge of the subject (9.9%).

Thus, respondents felt that they would recommend the seminar if it was to be provided again in the

future (10.7%).

Overall, the respondents were satisfied with the seminar (10.7%). Nevertheless, they put forward a

number of recommendations for future reference:

1. Avoid the use of acronyms

2. Provision of more information on the procedure of the External Audit

3. Have a better mix of students from the institutions to improve interaction

4. Provision of more days for the delivery of the seminar

5. Provision of practical workshops and case studies on quality audits and examples of bad audits

6. Provision of information on the seminar

7. Organisation of a follow up session

8. Provision of information on the work that was already done by the EQA.

Net-QAPE Meetings

This section applied to 27.0% of the respondents who stated that they had a role in the Net-QAPE

Meetings. In this regard, 24.6% stated that they regularly attended the meetings, with the majority

(14.8%) claiming that they attended most of the meetings.

12

Overall, there was positive feedback on the Net-QAPE Meetings. Indeed, the respondents found these

meetings to be useful (23.8%) and these met their expectations (18.8%). The respondents found these

meetings to be quite relevant, both for their personal development (21.3%) and for their institution

(23.0%). As a result, 23.8% of the respondents stated that they will put into practise the information

that was provided during the Net-QAPE meetings. Furthermore, the respondents were also in general

satisfied with the speakers who stated that they have effectively delivered the information (20.5%).

Respondents were asked a number of questions relating to different aspects of the Net-QAPE Meetings.

The majority of the respondents described the content and the organisation of the meetings either as

very good (7.4%) or good (14.8%). Respondents also found the interaction among the participants

during the meetings and the use of visual aids as being good at 20.5%. With regards to the logistics of

the meetings, the respondents found the location and date and time to be good with 17.1% and 18.9%

respectively. Respondents were also satisfied with the meeting facility (20.5%) and registration process

(24.6%). While in general, the respondents found the parking to be either very good (3.3%) or good

(13.1%), there were some who had issues and they assessed this aspect as being either fair (6.6%), poor

(2.5%), or very poor (0.8%).

Quality Audit Process

Around 36.1% of the respondents participated in the Quality Audit Process. The majority of the

respondents felt that they were sufficiently involved in the External Quality Audit (29.5%). Respondents

13

also stated that they felt respected by the evaluation penal throughout the External Quality Audit

(32.0%). When asked about whether they had enough time to prepare for the audit, there was an equal

percentage between those who agreed and those who felt neutral on this point at 13.1%. Overall, there

was general unanimity that the External Quality Audit will help their respective institution to strengthen

the External Quality Audit system (23.0%). Furthermore, the respondents stated that the instructions

and documents provided were sufficiently enlightening (14.8%) and that they were involved in the

internal quality assurance of the institution (29.6%).

Students were also asked about whether taking part in the External Quality Audit made them feel

empowered as students, with the majority of them stating that they did (4.1%). Similarly, the majority

of the students (4.1%) claimed that the External Quality Audit was useful.

When asked about whether the respondents found the Quality Audit Process as being a stressful

experience, there was an equal percentage of those who found it to be stressful and those who have a

neutral opinion on this issue (12.3%).

Concluding Conference

Around 40.2% of the respondents participated in the concluding conference. The respondents were

asked to provide feedback on the contents of the conference. The respondents claimed that the

conference was useful (32.8%) and that it met their expectations (31.9%). The information provided

during the conference was relevant to the personal development of 28.7% of the respondents and they

also claimed that it was relevant for their respective institution. Around 30.3% of the respondents stated

that they will put into practice the information provided during the conference and 32.0% of the

respondents stated that the speakers at the conference clearly delivered the information.

Respondents were also questioned about different aspects of the conference. The majority of the

respondents claimed that the content of the conference was either very good (9.8%) or good (23.0%).

When asked about the organisation of the conference, 33.6% stated that the organisation was good. The

interaction among participants at the conference was also good for the majority of the participants

(24.6%). The use of the visual aids was also deemed as good to the respondents (31.3%). Respondents

were asked questions on the logistics of the conference. The location of the conference was good to

31.1% of the respondents while the date and time of the conference was good to 30.0% of the

respondents. Around 32.8% and 37.7% of the respondents respectively rated the conference facility and

14

registration as either very good or good. The parking for the conference was rated as good by 30.4% of

the respondents.

When asked about whether they would be interested in attending similar future events, 23.0% of the

respondents stated that they were very interested and another 13.1% of the respondents stated that they

were interested.

Overall Project

The respondents were asked to rate the usefulness of the ESF Project 1.227 “Making Quality Visible”

and its deliverables for the improvement of Quality Assurance in Further and Higher Education in Malta.

In this regard, 34.4% of the respondents stated that they rate its usefulness as being very good and

another 32.0% rated it as being good.

At the end of the quantitative questionnaire, the respondents were asked to provide any additional

comments. The following is a list of the main comments received:

1. All the higher education institutions should have been evaluated by the same team.

2. Should have organised a seminar to teach students what to do when they were called for an

audit.

3. Poor sound system at the Dolmen Hotel.

4. Unaware of the next step following the seminar.

5. Agreement with the quality audit in further and higher education as adults should have value for

the courses they are following and for what they are paying.

6. EQA is a very useful tool and should be carried out more frequently. Quality Assurance

improves the services and facilities on offer and at the end improves the quality of higher

education while making students attractive to prospective employers. There should be a sequel

of these sessions at least once a year, to keep updated, sustain the created forum and share

challenges/achievements.

15

7. Difficulty in how the details apply to much smaller institutions since both the seminar and the

conference were mostly related to the experiences of the three major institutions. Should be

informed about upcoming external quality audits in which it is possible to participate in.

8. Personnel should be chosen carefully.

16

Appendix I: Tables

Profile of Respondents

Gender:

Frequency Percent

Male 66 54.1

Female 56 45.9

Total 122 100.0

Education:

Frequency Percent

Not applicable

24 19.7

Secondary 6 4.9

Tertiary 92 75.4

Total 122 100.0

17

Age:

Percent Frequency

18-20 .8 1

21-30 22.1 27

31-45 30.3 37

46+ 46.7 57

Total 100.0 122

18

Status:

Percent Frequency

Not applicable 46.7 57

Administrator/director/manager 37.7 46

Quality assurance officer 12.3 15

Lecturer .8 1

Clerk .8 1

Counsellor .8 1

Other .8 1

Total 100.0 122

19

Role of the Respondents

Role within the ESF Project 1.227

Percentage Frequency

CPD Award 19.7 24

Expert Evaluators Seminar

19.7 24

Student Evaluators Seminar

13.1 16

Net-QAPE Meetings 27.0 33

EQA 36.1 44

Conference 40.2 49

None 21.3

26

20

CPD Award

CPD Certificate

Percent Frequency

N/A 80.3 98

Yes 18.0 22

No 1.6 2

Total 100.0 122

21

Reasons for not completing the course:

Percent Frequency

Change in employment 0 0

Change in circumstances in my institution 0.8 1

Not enough time 0.8 1

Course was not useful 0 0

Other 0.8 1

Organisation of the course:

Detailed information was provided on the course

Percent Frequency

N/A 80.3 98

Strongly Agree

4.1 5

Agree 10.7 13

Neutral 2.5 3

Disagree 1.6 2

Strongly Disagree

.8 1

Total 100.0 122

22

The learning outcomes of the course were clearly stated

Percent Frequency

N/A 80.3 98

Strongly Agree

4.9 6

Agree 11.5 14

Neutral 1.6 2

Disagree .8 1

Strongly Disagree

.8 1

Total 100.0 122

23

Information on the course was provided well in advance

Percent Frequency

N/A 80.3 98

Strongly Agree

4.1 5

Agree 8.2 10

Neutral 4.1 5

Disagree 2.5 3

Strongly Disagree

.8 1

Total 100.0 122

The course was well organised

Percent Frequency

N/A 80.3 98

Strongly Agree

6.6 8

Agree 9.0 11

Neutral 3.3 4

Disagree .8 1

Total 100.0 122

24

Course Content

The lecturers allowed for active participation and interaction

Percent Frequency

N/A 80.3 98

Strongly Agree

6.6 8

Agree 9.0 11

Neutral 3.3 4

Disagree .8 1

Total 100.0 122

25

The course content reflected projected External Quality Audit requirements

Percent Frequency

N/A 80.3 98

Strongly Agree

4.1 5

Agree 9.0 11

Neutral 4.1 5

Disagree .8 1

Strongly Disagree

1.6 2

Total 100.0 122

The course was practical

Percent Frequency

N/A 80.3 98

Strongly Agree

2.5 3

Agree 9.8 12

Neutral 4.1 5

Disagree 2.5 3

Strongly Disagree

.8 1

Total 100.0 122

26

Lecturers

The lecturers were well prepared and fully resourced

Percent Frequency

N/A 80.3 98

Strongly Agree

5.7 7

Agree 7.4 9

Neutral 4.9 6

Strongly Disagree

1.6 2

Total 100.0 122

27

The lecturers had a good knowledge on the subject

Percent Frequency

N/A 80.3 98

Strongly Agree

4.9 6

Agree 11.5 14

Neutral 1.6 2

Strongly Disagree

1.6 2

Total 100.0 122

28

The lecturers used technology effectively to advance my learning

Percent Frequency

N/A 80.3 98

Strongly Agree

2.5 3

Agree 9.8 12

Neutral 5.7 7

Strongly Disagree

1.6 2

Total 100.0 122

The lecturers made the subject interesting

Percent Frequency

N/A 80.3 98

Strongly Agree

3.3 4

Agree 10.7 13

Neutral 4.1 5

Strongly Disagree

1.6 2

Total 100.0 122

29

The lecturers provided useful material

Percent Frequency

N/A 80.3 98

Strongly Agree

4.9 6

Agree 7.4 9

Neutral 4.9 6

Disagree .8 1

Strongly Disagree

1.6 2

Total 100.0 122

30

Learning Resources

The resources were enough to meet my needs

Percent Frequency

No answer

.8 1

N/A 80.3 98

Strongly Agree

3.3 4

Agree 9.0 11

Neutral 4.1 5

Disagree 2.5 3

Total 100.0 122

The number of participants in relation to the room size was acceptable

Percent Frequency

N/A 80.3 98

Strongly Agree

4.1 5

Agree 12.3 15

Neutral 2.5 3

Disagree .8 1

Total 100.0 122

31

The equipment of the learning area was useful

Percent Frequency

N/A 80.3 98

Strongly Agree

4.1 5

Agree 10.7 13

Neutral 4.1 5

Disagree .8 1

Total 100.0 122

32

The educational material helped me understand the topics

Percent Frequency

N/A 80.3 98

Strongly Agree

3.3 4

Agree 9.8 12

Neutral 3.3 4

Disagree 3.3 4

Total 100.0 122

The educational material was up to date

Percent Frequency

Valid N/A 80.3 98

Strongly Agree

4.1 5

Agree 11.5 14

Neutral 4.1 5

Total 100.0 122

33

The educational material was easy to understand

Percent Frequency

N/A 80.3 98

Strongly Agree

2.5 3

Agree 11.5 14

Neutral 4.9 6

Disagree .8 1

Total 100.0 122

34

The educational material was well structured

Percent Frequency

N/A 80.3 98

Strongly Agree

2.5 3

Agree 11.5 14

Neutral 4.1 5

Disagree .8 1

Strongly Disagree

.8 1

Total 100.0 122

The educational material was relevant

Percent Frequency

N/A 80.3 98

Strongly Agree

4.1 5

Agree 12.3 15

Neutral 3.3 4

Total 100.0 122

35

Teaching Methods:

Lectures

Percent Frequency

N/A 80.3 98

Very Good

5.7 7

Good 8.2 10

Fair 3.3 4

Poor 1.6 2

Very Poor

.8 1

Total 100.0 122

36

Presentations

Percent Frequency

N/A 80.3 98

Very Good

4.1 5

Good 11.5 14

Fair 2.5 3

Poor 1.6 2

Total 100.0 122

37

Group activities / discussions

Percent Frequency

N/A 80.3 98

Very Good

4.9 6

Good 7.4 9

Fair 4.9 6

Poor 2.5 3

Total 100.0 122

Problem handling

Percent Frequency

No answer

.8 1

N/A 80.3 98

Very Good

2.5 3

Good 8.2 10

Fair 3.3 4

Poor 4.1 5

Very Poor

.8 1

Total 100.0 122

38

Commitment

I attended almost all of the lectures

Percent Frequency

N/A 80.3 98

Strongly Agree

13.9 17

Agree 4.1 5

Neutral .8 1

Disagree .8 1

Total 100.0 122

39

I was involved in the group discussion during the lectures

Percent Frequency

N/A 80.3 98

Strongly Agree

7.4 9

Agree 10.7 13

Neutral .8 1

Disagree .8 1

Total 100.0 122

I carried out the recommended hours of self-study

Percent Frequency

N/A 80.3 98

Strongly Agree

9.8 12

Agree 7.4 9

Neutral 1.6 2

Strongly Disagree

.8 1

Total 100.0 122

40

I made use of the hours made available by the lecturers outside the classroom

Percent Frequency

N/A 80.3 98

Strongly Agree

6.6 8

Agree 5.7 7

Neutral 4.1 5

Disagree .8 1

Strongly Disagree

1.6 2

Don't Know

.8 1

Total 100.0 122

41

Assessment Methods

The assessment methods were appropriate for the course

Percent Frequency

N/A 80.3 98

Strongly Agree

4.1 5

Agree 8.2 10

Neutral 4.1 5

Disagree 1.6 2

Strongly Disagree

1.6 2

Total 100.0 122

42

The assessment methods gave the opportunity to participants to show their knowledge

Percent Frequency

N/A 80.3 98

Strongly Agree

3.3 4

Agree 10.7 13

Neutral 4.9 6

Strongly Disagree

.8 1

Total 100.0 122

43

Participants were informed on the evaluation criteria

Percent Frequency

N/A 80.3 98

Strongly Agree

4.9 6

Agree 11.5 14

Neutral 1.6 2

Disagree .8 1

Strongly Disagree

.8 1

Total 100.0 122

Participants were given enough time for the assessment

Percent Frequency

N/A 80.3 98

Strongly Agree

4.1 5

Agree 11.5 14

Neutral 1.6 2

Disagree .8 1

Strongly Disagree

1.6 2

Total 100.0 122

44

The lecturers provided clear feedback on the participants' performance

Percent Frequency

N/A 80.3 98

Strongly Agree

4.1 5

Agree 7.4 9

Neutral 4.9 6

Disagree 3.3 4

Total 100.0 122

45

Seminar for Prospective Expert Evaluators of the External Quality Audits

Detailed information was provided on the seminar workload and input required

Percent Frequency

N/A 80.3 98

Strongly Agree

4.1 5

Agree 13.9 17

Neutral .8 1

Disagree .8 1

Total 100.0 122

The learning outcomes of the seminar were clearly stated

Percent Frequency

N/A 80.3 98

Strongly Agree

4.1 5

Agree 13.9 17

Neutral 1.6 2

Total 100.0 122

46

Information on the seminar was provided well in advance

Percent Frequency

N/A 80.3 98

Strongly Agree

4.9 6

Agree 11.5 14

Neutral 2.5 3

Strongly Disagree

.8 1

Total 100.0 122

47

The seminar was well organised

Percent Frequency

N/A 80.3 98

Strongly Agree

9.0 11

Agree 9.8 12

Neutral .8 1

Total 100.0 122

Seminar Content


Percent Frequency

N/A 80.3 98

Strongly Agree

5.7 7

Agree 12.3 15

Neutral .8 1

Strongly Disagree

.8 1

Total 100.0 122

48

The seminar content reflected projected External Quality Audit requirements

Percent Frequency

N/A 80.3 98

Strongly Agree

5.7 7

Agree 13.1 16

Neutral .8 1

Total 100.0 122

49

The seminar was practical

Percent Frequency

N/A 80.3 98

Strongly Agree

4.1 5

Agree 14.8 18

Neutral .8 1

Total 100.0 122

Lecturers


Percent Frequency

N/A 80.3 98

Strongly Agree

5.7 7

Agree 12.3 15

Neutral 1.6 2

Total 100.0 122

50


Percent Frequency

N/A 80.3 98

Strongly Agree

6.6 8

Agree 11.5 14

Neutral 1.6 2

Total 100.0 122

51


Percent Frequency

N/A 80.3 98

Strongly Agree

4.1 5

Agree 12.3 15

Neutral 3.3 4

Total 100.0 122


Percent Frequency

N/A 80.3 98

Strongly Agree

7.4 9

Agree 10.7 13

Neutral .8 1

Disagree .8 1

Total 100.0 122

52


Percent Frequency

N/A 80.3 98

Strongly Agree

5.7 7

Agree 11.5 14

Neutral 2.5 3

Total 100.0 122

53

Learning resources


Percent Frequency

N/A 80.3 98

Strongly Agree

2.5 3

Agree 15.6 19

Neutral .8 1

Disagree .8 1

Total 100.0 122

The number of participants in relation to the room size was acceptable

Percent Frequency

N/A 80.3 98

Strongly Agree

2.5 3

Agree 11.5 14

Neutral 4.1 5

Disagree 1.6 2

Total 100.0 122

54


Percent Frequency

N/A 80.3 98

Strongly Agree

2.5 3

Agree 13.9 17

Neutral 3.3 4

Total 100.0 122

55


Percent Frequency

N/A 80.3 98

Strongly Agree

6.6 8

Agree 11.5 14

Neutral 1.6 2

Total 100.0 122


Percent Frequency

N/A 80.3 98

Strongly Agree

6.6 8

Agree 11.5 14

Neutral 1.6 2

Total 100.0 122

56


Percent Frequency

N/A 80.3 98

Strongly Agree

4.1 5

Agree 13.1 16

Neutral 2.5 3

Total 100.0 122

57


Percent Frequency

N/A 80.3 98

Strongly Agree

4.1 5

Agree 14.8 18

Neutral .8 1

Total 100.0 122


Percent Frequency

N/A 80.3 98

Strongly Agree

4.9 6

Agree 13.9 17

Neutral .8 1

Total 100.0 122

58

Teaching Methods

Lectures

Percent Frequency

N/A 80.3 98

Very good

6.6 8

Good 10.7 13

Fair 1.6 2

Poor .8 1

Total 100.0 122

59

Presentations

Percent Frequency

N/A 80.3 98

Very good

4.9 6

Good 11.5 14

Fair 3.3 4

Total 100.0 122


Percent Frequency

N/A 80.3 98

Very good

13.1 16

Good 5.7 7

Fair .8 1

Total 100.0 122

60

Problem handling

Percent Frequency

N/A 80.3 98

Very good

8.2 10

Good 9.8 12

Fair 1.6 2

Total 100.0 122

61

Additional considerations

I was involved in the group discussion during the seminar

Percent Frequency

N/A 80.3 98

Strongly Agree

10.7 13

Agree 9.0 11

Total 100.0 122

The seminar met my expectations

Percent Frequency

N/A 80.3 98

Strongly Agree

6.6 8

Agree 11.5 14

Neutral .8 1

Disagree .8 1

Total 100.0 122

62

I would definitely recommend this seminar if provided in the future

Percent Frequency

N/A 80.3 98

Strongly Agree

10.7 13

Agree 7.4 9

Neutral .8 1

Disagree .8 1

Total 100.0 122

63

I felt respected during the seminar

Percent Frequency

N/A 80.3 98

Strongly Agree

9.0 11

Agree 9.0 11

Disagree .8 1

Strongly Disagree

.8 1

Total 100.0 122

I felt at ease in sharing my opinions and ask questions in the classroom

Percent Frequency

N/A 80.3 98

Strongly Agree

9.0 11

Agree 8.2 10

Neutral 1.6 2

Disagree .8 1

Total 100.0 122

64

It helped me gain in-depth knowledge on the subject

Percent Frequency

N/A 80.3 98

Strongly Agree

6.6 8

Agree 11.5 14

Neutral 1.6 2

Total 100.0 122

65

Satisfaction

Percent Frequency

N/A 80.3 98

Very satisfied

9.0 11

Satisfied 9.0 11

Fair 1.6 2

Total 100.0 122

Seminar for Prospective Student Evaluators of External Quality Audits

Organisation of the Seminar

Detailed information was provided on the seminar workload and input required

Percent Frequency

N/A 86.9 106

Strongly Agree

1.6 2

Agree 8.2 10

Neutral 1.6 2

Disagree 1.6 2

Total 100.0 122

66

The learning outcomes of the seminar were clearly stated

Percent Frequency

N/A 86.9 106

Strongly Agree

.8 1

Agree 7.4 9

Neutral 4.9 6

Total 100.0 122

67

Information on the seminar was provided well in advance

Percent Frequency

N/A 86.9 106

Strongly Agree

.8 1

Agree 4.9 6

Neutral 4.9 6

Disagree 2.5 3

Total 100.0 122

The seminar was well organised

Percent Frequency

N/A 86.9 106

Strongly Agree

3.3 4

Agree 7.4 9

Neutral 1.6 2

Disagree .8 1

Total 100.0 122

68

Seminar Content


Percent Frequency

N/A 86.9 106

Strongly Agree

7.4 9

Agree 4.1 5

Neutral 1.6 2

Total 100.0 122

69

The seminar content reflected projected External Quality Audit requirements

Percent Frequency

N/A 86.9 106

Strongly Agree

2.5 3

Agree 6.6 8

Neutral 4.1 5

Total 100.0 122

The seminar was practical

Percent Frequency

N/A 86.9 106

Strongly Agree

2.5 3

Agree 6.6 8

Neutral 4.1 5

Total 100.0 122

70

Lecturers


Percent Frequency

N/A 86.9 106

Strongly Agree

4.1 5

Agree 7.4 9

Neutral 1.6 2

Total 100.0 122

71


Percent Frequency

N/A 86.9 106

Strongly Agree

8.2 10

Agree 4.9 6

Total 100.0 122


Percent Frequency

N/A 86.9 106

Strongly Agree

3.3 4

Agree 4.9 6

Neutral 4.9 6

Total 100.0 122

72


Percent Frequency

N/A 86.9 106

Strongly Agree

5.7 7

Agree 4.9 6

Neutral 2.5 3

Total 100.0 122

73


Percent Frequency

N/A 86.9 106

Strongly Agree

2.5 3

Agree 7.4 9

Neutral 2.5 3

Disagree .8 1

Total 100.0 122

Learning resources


Percent Frequency

N/A 86.9 106

Strongly Agree

.8 1

Agree 5.7 7

Neutral 4.9 6

Disagree .8 1

Strongly Disagree

.8 1

Total 100.0 122

74

The number of students in relation to the room size was acceptable

Percent Frequency

N/A 86.9 106

Strongly Agree

5.7 7

Agree 6.6 8

Neutral .8 1

Total 100.0 122

75


Percent Frequency

N/A 86.9 106

Strongly Agree

4.1 5

Agree 6.6 8

Neutral 1.6 2

Disagree .8 1

Total 100.0 122


Percent Frequency

N/A 86.9 106

Strongly Agree

2.5 3

Agree 4.1 5

Neutral 4.9 6

Disagree 1.6 2

Total 100.0 122

76


Percent Frequency

N/A 86.9 106

Strongly Agree

4.1 5

Agree 5.7 7

Neutral 2.5 3

Disagree .8 1

Total 100.0 122

77


Percent Frequency

N/A 86.9 106

Strongly Agree

2.5 3

Agree 6.6 8

Neutral 3.3 4

Strongly Disagree

.8 1

Total 100.0 122


Percent Frequency

N/A 86.9 106

Strongly Agree

2.5 3

Agree 5.7 7

Neutral 4.1 5

Strongly Disagree

.8 1

Total 100.0 122

78


Percent Frequency

N/A 86.9 106

Strongly Agree

3.3 4

Agree 9.0 11

Strongly Disagree

.8 1

Total 100.0 122

79

Teaching Methods

Lectures

Percent Frequency

N/A 86.9 106

Very good

4.9 6

Good 5.7 7

Fair 1.6 2

Poor .8 1

Total 100.0 122

Presentations

Percent Frequency

N/A 86.9 106

Very good

4.1 5

Good 7.4 9

Fair .8 1

Very poor

.8 1

Total 100.0 122

80


Percent Frequency

N/A 86.9 106

Very good

4.1 5

Good 6.6 8

Fair 1.6 2

Poor .8 1

Total 100.0 122

81

Problem handling

Percent Frequency

N/A 86.9 106

Good 10.7 13

Fair 1.6 2

Don't know

.8 1

Total 100.0 122

Additional considerations on the seminar

I was involved in the group discussions during the seminar

Percent Frequency

N/A 86.9 106

Strongly Agree

5.7 7

Agree 7.4 9

Total 100.0 122

82

The seminar met my expectations

Percent Frequency

N/A 86.9 106

Strongly Agree

2.5 3

Agree 7.4 9

Neutral 3.3 4

Total 100.0 122

83

I would definitely recommend this seminar if provided in the future

Percent Frequency

N/A 86.9 106

Strongly Agree

3.3 4

Agree 7.4 9

Neutral 1.6 2

Don't Know

.8 1

Total 100.0 122

I felt respected during the seminar

Percent Frequency

N/A 86.9 106

Strongly Agree

5.7 7

Agree 6.6 8

Neutral .8 1

Total 100.0 122

84

I felt at ease in sharing my opinions and ask questions in the classroom

Percent Frequency

N/A 86.9 106

Strongly Agree

6.6 8

Agree 5.7 7

Neutral .8 1

Total 100.0 122

85

It helped me gain in-depth knowledge on the subject

Percent Frequency

N/A 86.9 106

Strongly Agree

3.3 4

Agree 6.6 8

Neutral 2.5 3

Disagree .8 1

Total 100.0 122

Net-QAPE Meetings

Attendance

Percent Frequency

N/A 73.0 89

Yes 24.6 30

No 2.5 3

Total 100.0 122

86

Rate of attendance

Percent Frequency

N/A 73.0 89

All meetings

11.5 14

Most meetings

14.8 18

Few meetings

.8 1

Total 100.0 122

87

Content

The meetings were useful

Percent Frequency

N/A 73.0 89

Strongly Agree

8.2 10

Agree 15.6 19

Neutral 3.3 4

Total 100.0 122

88

The meetings met my expectations

Percent Frequency

N/A 73.0 89

Strongly Agree

4.9 6

Agree 13.9 17

Neutral 5.7 7

Disagree 2.5 3

Total 100.0 122

The information provided was relevant for my personal development

Percent Frequency

No answer

.8 1

N/A 73.0 89

Strongly Agree

3.3 4

Agree 18.0 22

Neutral 3.3 4

Disagree 1.6 2

Total 100.0 122

89

The information provided was relevant for my institution

Percent Frequency

N/A 73.0 89

Strongly Agree

3.3 4

Agree 19.7 24

Neutral 3.3 4

Disagree .8 1

Total 100.0 122

90

I will put into practise the information provided

Percent Frequency

N/A 73.0 89

Strongly Agree

6.6 8

Agree 17.2 21

Neutral 3.3 4

Total 100.0 122

The speakers delivered the information

Percent Frequency

N/A 73.0 89

Strongly Agree

6.6 8

Agree 13.9 17

Neutral 5.7 7

Don't Know

.8 1

Total 100.0 122

91

Aspects of the Net-QAPE Meetings

Content

Percent Frequency

N/A 73.0 89

Very good

7.4 9

Good 14.8 18

Fair 2.5 3

Poor 2.5 3

Total 100.0 122

92

Organisation

Percent Frequency

N/A 73.0 89

Very good

7.4 9

Good 14.8 18

Fair 2.5 3

Poor 1.6 2

Very poor

.8 1

Total 100.0 122

93

Interaction with participants

Percent Frequency

N/A 73.0 89

Very good

5.7 7

Good 14.8 18

Fair 3.3 4

Poor 3.3 4

Total 100.0 122

The use of visual aids

Percent Frequency

N/A 73.0 89

Very good

6.6 8

Good 13.9 17

Fair 4.1 5

Poor 1.6 2

Very poor

.8 1

Total 100.0 122

94

Location of the meetings

Percent Frequency

N/A 73.0 89

Very good

4.1 5

Good 13.1 16

Fair 8.2 10

Poor 1.6 2

Total 100.0 122

95

Date and time

Percent Frequency

N/A 73.0 89

Very good

2.5 3

Good 16.4 20

Fair 5.7 7

Poor 1.6 2

Very poor

.8 1

Total 100.0 122

Meeting facility

Percent Frequency

N/A 73.0 89

Very good

4.9 6

Good 15.6 19

Fair 6.6 8

Total 100.0 122

96

Registration

Percent Frequency

N/A 73.0 89

Very good

8.2 10

Good 16.4 20

Fair 1.6 2

Don't know

.8 1

Total 100.0 122

97

Parking

Percent Frequency

N/A 73.0 89

Very good

3.3 4

Good 13.1 16

Fair 6.6 8

Poor 2.5 3

Very poor

.8 1

Don't know

.8 1

Total 100.0 122

98

Quality Audit Process

Aspects

I felt sufficiently involved in the External Quality Audit

Percent Frequency

N/A 63.9 78

Strongly Agree

12.3 15

Agree 17.2 21

Neutral 5.7 7

Don't Know

.8 1

Total 100.0 122

99

I felt respected by the External Quality Audit evaluation panel

Percent Frequency

N/A 63.9 78

Strongly Agree

14.8 18

Agree 17.2 21

Neutral 2.5 3

Don't Know

1.6 2

Total 100.0 122

I had enough time to prepare for the External Quality Audit

Percent Frequency

No answer

.8 1

N/A 68.9 84

Strongly Agree

3.3 4

Agree 9.8 12

Neutral 13.1 16

Disagree 2.5 3

Strongly Disagree

.8 1

Don't Know

.8 1

Total 100.0 122

100

It helped my institution to strengthen the External Quality Audit system

Percent Frequency

N/A 68.9 84

Strongly Agree

7.4 9

Agree 15.6 19

Neutral 6.6 8

Don't Know

1.6 2

Total 100.0 122

101

The instructions and documents provided were sufficiently enlightening

Percent Frequency

N/A 79.5 97

Strongly Agree

2.5 3

Agree 12.3 15

Neutral 3.3 4

Disagree .8 1

Strongly Disagree

.8 1

Don't Know

.8 1

Total 100.0 122

I was involved in the internal quality assurance of the institution

Percent Frequency

N/A 63.9 78

Strongly Agree

10.7 13

Agree 18.9 23

Neutral 4.1 5

Don't Know

2.5 3

Total 100.0 122

102

I felt empowered as a student during the External Quality Audit

Percent Frequency

N/A 95.1 116

Strongly Agree

2.5 3

Agree 1.6 2

Neutral .8 1

Total 100.0 122

103

The External Quality Audit was useful

Percent Frequency

N/A 95.1 116

Strongly Agree

2.5 3

Agree 1.6 2

Disagree .8 1

Total 100.0 122

Stressful

Percent Frequency

N/A 63.9 78

Very stressful

4.1 5

Stressful 8.2 10

Neutral 12.3 15

Not stressful

7.4 9

Not stressful at all

3.3 4

Don't know

.8 1

Total 100.0 122

104

Concluding Conference

The conference was useful

Percent Frequency

N/A 59.8 73

Strongly Agree

11.5 14

Agree 21.3 26

Neutral 4.9 6

Disagree 2.5 3

Total 100.0 122

105

The conference met my expectations

Percent Frequency

No answer

.8 1

N/A 59.8 73

Strongly Agree

9.8 12

Agree 22.1 27

Neutral 5.7 7

Disagree .8 1

Strongly Disagree

.8 1

Total 100.0 122

The information provided was relevant for my personal development

Percent Frequency

N/A 59.8 73.0

Strongly Agree

8.2 10.0

Agree 20.5 25.0

Neutral 9.0 11.0

Disagree 1.6 2.0

No answer

0.8 1.0

Total 100.0 122.0

106

The information provided was very relevant for my institution

Percent Frequency

N/A 59.8 73

Strongly Agree

9.8 12

Agree 18.9 23

Neutral 11.5 14

Total 100.0 122

107

I will put into practice the information provided

Percent Frequency

No answer

.8 1

N/A 59.8 73

Strongly Agree

9.8 12

Agree 20.5 25

Neutral 9.0 11

Total 100.0 122

The speakers clearly delivered the information

Percent Frequency

No answer

.8 1

N/A 59.8 73

Strongly Agree

12.3 15

Agree 19.7 24

Neutral 6.6 8

Disagree .8 1

Total 100.0 122

108

Aspects

Content

Percent Frequency

N/A 59.8 73

Very good

9.8 12

Good 23.0 28

Fair 4.9 6

Poor 2.5 3

Total 100.0 122

109

Organisation

Percent Frequency

N/A 59.8 73

Very good

15.6 19

Good 18.0 22

Fair 4.1 5

Very poor

.8 1

No answer

1.6 2

Total 98.4 120

Interaction with participants

Percent Frequency

N/A 59.8 73

Very good

8.2 10

Good 16.4 20

Fair 13.9 17

Poor 1.6 2

Total 100.0 122

110

The use of visual aids

Percent Frequency

N/A 59.8 73

Very good

9.8 12

Good 21.3 26

Fair 8.2 10

Poor .8 1

Total 100.0 122

111

Location of the conference

Percent Frequency

N/A 59.8 73

Very good

9.8 12

Good 21.3 26

Fair 5.7 7

Poor 2.5 3

Very poor

.8 1

Total 100.0 122

Date and time

Frequency Percent

Valid N/A 73 59.8

Very good

13 10.7

Good 27 22.1

Fair 7 5.7

Poor 1 .8

No answer

1 .8

Total 122 100.0

112

Conference facility

Percent Frequency

N/A 59.8 73

Very good

11.5 14

Good 21.3 26

Fair 7.4 9

Total 100.0 122

113

Registration

Percent Frequency

N/A 59.8 73

Very good

15.6 19

Good 22.1 27

Fair 1.6 2

Don't know

.8 1

Total 100.0 122

Parking

Percent Frequency

N/A 59.8 73

Very good

11.5 14

Good 18.9 23

Fair 4.1 5

Poor 2.5 3

Very poor

.8 1

Don't know

1.6 2

No answer

.8 1

Total 100.0 122

114

Interest in similar future events

Percent Frequency

N/A 59.8 73

Very interested

23.0 28

Interested 13.1 16

Fair 3.3 4

Not interested

.8 1

Total 100.0 122

Evaluation of Key Project Deliverables in Part Fulfilment ... 1.227 Making Qualit… · (NCFHE) to...

Documents

Transcript of Evaluation of Key Project Deliverables in Part Fulfilment ... 1.227 Making Qualit… · (NCFHE) to...