Frequency of using iLearn with - Macquarie University Web viewThis is because they are considered to...

iLearn Evaluation ReportILEARN STAFF EXPERIENCE SURVEY 2014

Produced by Jayde Cahir (Chief Investigator), Helen Carter (Co-Investigator), Cathy Rytmeister (Statistical analysis and advice), Lucy Arthur (Faculty iLearn Support Coordination) and Rebecca Ritchie (Central iLearn Support Coordination).May 2015

1

TABLE OF CONTENTSTable of Contents........................................................................................................................................2List of Figures.............................................................................................................................................3List of Tables...............................................................................................................................................4iLearn Staff Experience Survey..................................................................................................................5Results and Discussion................................................................................................................................5Demographic Details...................................................................................................................................5Technologies used to access iLearn: Staff Survey Results.........................................................................5Staff Satisfaction with iLearn as a support for learning and teaching.......................................................6Staff Skills and Confidence Levels..............................................................................................................8Staff Confidence Levels...............................................................................................................................8Changing Practice.....................................................................................................................................11Staff Experience with iLearn....................................................................................................................13Statistics on the use of iLearn Functions: data from the iLearn system..................................................13Scaled responses on the perceived value of iLearn Functions: data from the surveys............................15iLearn Functions for organising students learning..................................................................................15iLearn Functions for engaging students with learning activities.............................................................19Staff Satisfaction with Services and Support...........................................................................................26Satisfaction levels with support services for iLearn and iTeach..............................................................26Staff Satisfaction with iTeach...................................................................................................................26Staff Satisfaction with iTeach Support.....................................................................................................26Staff Satisfaction with UNIT Guides.........................................................................................................27Staff Satisfaction with iTeach Training and Support................................................................................27Staff Satisfaction with OneHelp for iLearn support.................................................................................27Support services for iLearn provided by the Faculties and the LTC........................................................28Staff Satisfaction with Online Resources..................................................................................................28Staff Satisfaction with the iLearn Drop-in Clinics....................................................................................30Staff Satisfaction with iLearn training workshops...................................................................................30Staff Satisfaction with Faculty-based iLearn Support..............................................................................31iLearn Status Page....................................................................................................................................32Appendices................................................................................................................................................33Appendix 1: iLearn Staff Experience Survey Questions...........................................................................33Appendix 2: Survey Advertisements.........................................................................................................38Appendix 3: Improvements to the iLearn platform...................................................................................39Appendix 4: Statistical Analysis of Technologies Staff used to access iLearn.........................................40

Frequency of using iLearn with….........................................................................................................402.1 Using iLearn with….........................................................................................................................41Using iLearn with…...............................................................................................................................42

Appendix 5: Statistical Analysis of Staff Skills, Confidence Levels and Satisfaction...............................43Overall teaching experience with iLearn..............................................................................................43Overall teaching experience with iLearn..............................................................................................44Overall teaching experience with iLearn..............................................................................................45

Appendix 6: Statistical Analysis of Staff Experience with iLearn.............................................................47Organising students' learning...............................................................................................................47

2

Engaging students in learning activities...............................................................................................47The iLearn platform was effective in meeting my needs…...................................................................48with respect to organising students' learning......................................................................................48for engaging my students in learning activities....................................................................................48The iLearn platform was effective in meeting my needs…...................................................................49with respect to organising students' learning......................................................................................49for engaging my students in learning activities....................................................................................49

Appendix 7: Statistical Analysis of Staff Satisfaction with Services and Support....................................50Satisfaction with support and services.................................................................................................50Satisfaction with support and services.................................................................................................51Satisfaction with support and services.................................................................................................52

LIST OF FIGURESFigure 1: Satisfaction with iLearn as an online support for unit content..........................................6Figure 2: Satisfaction with iLearn as a support for organising my teaching....................................7Figure 3: Satisfaction with iLearn as a support for interaction and communication with students. 7Figure 4: Satisfaction with the way iLearn supports student engagement.......................................8Figure 5: Development of skills to effectively use iLearn..................................................................9Figure 6: Staff confidence when using iLearn...................................................................................9Figure 7: Staff confidence when using Echo360..............................................................................10Figure 8: Staff confidence when using iTeach.................................................................................10Figure 9: Staff confidence when using iTeach to create Unit Guides..............................................11Figure 10: Staff confidence when using iTeach to map student learning outcomes.......................11Figure 11: Change in teaching practice...........................................................................................12Figure 12: Change in future teaching practice................................................................................12Figure 13: Staff Experience of iLearn / Moodle...............................................................................13Figure 14: Undergraduate and Postgraduate Units used of iLearn Functions................................14Figure 15: iLearn Function Announcements....................................................................................16Figure 16: iLearn Function Online Assignment Submission............................................................16Figure 17: iLearn Function Turnitin................................................................................................17Figure 18: iLearn Function GradeMark...........................................................................................18Figure 19: iLearn Function Grades..................................................................................................18Figure 20: iLearn Function Calendar...............................................................................................19Figure 21: iLearn Function Discussion Forums...............................................................................20Figure 22: iLearn Function Quizzes.................................................................................................21Figure 23: iLearn Function Dialogue Module..................................................................................22Figure 24: iLearn Function Database..............................................................................................22Figure 25: iLearn Function Workshop Tool.....................................................................................23Figure 26: iLearn Function Blog......................................................................................................23Figure 27: iLearn Function Wiki......................................................................................................24Figure 28: iLearn Function Chat......................................................................................................25Figure 29: iLearn Function Twitter Feeds.......................................................................................25Figure 30: Staff satisfaction with the automatic enrolment of students facilitated by iTeach........26Figure 31: Staff satisfaction with OneHelp for resolution of issues relating to iTeach...................27Figure 32: Staff satisfaction with OneHelp for resolution of issues relating to iLearn functions.. .28Figure 33: Staff satisfaction with the ease of locating online "self-help" resources.......................29Figure 34: Staff satisfaction with the effectiveness of the online "self-help" resources..................29Figure 35: Staff Satisfaction with the iLearn Drop-in Clinics..........................................................30Figure 36: Staff Satisfaction with the iLearn Training Workshops.................................................31Figure 37: Staff Satisfaction with Faculty iLearn Support..............................................................31

LIST OF TABLESTable 1: The iLearn Staff Experience Survey Demographic Information for 2012 - 2014.............................5Table 2: Technologies used to access iLearn according to the 2014 staff survey..........................................6Table 3: Staff Satisfaction with iLearn as a support for learning and teaching.............................................6Table 4: Staff Confidence levels in using iLearn as a support for learning and teaching..............................8Table 5: Staff members changing practice by using iLearn as a support for learning and teaching..........11Table 6: The percentage of units that used the various iLearn functions....................................................14

3

Table 7: iLearn functions that facilitate organising students learning........................................................15Table 8: iLearn functions that facilitate engagement with learning activities.............................................20Table 9: Staff Satisfaction with support services for iLearn and iTeach......................................................26Table 10: Staff Satisfaction with support services provided by the Faculties and the LTC.........................28

ILEARN STAFF EXPERIENCE SURVEYThe 2014 iLearn Staff and Student Experience Surveys are part of a wider Learning and Teaching Centre (LTC) quality enhancement strategy for the University’s Learning Management System (LMS) and learning technologies more generally. In 2014, iLearn had moved into the second year of its operational phase thus the focus of the 2014 survey was to evaluate staff satisfaction with:

the functionality of iLearn from a teaching practice and technical perspective; the University’s services and support for iLearn; and to document trends in use to inform future needs in terms of functionality and support.

This focus aligns with the findings and recommendations of the iLearn Implementation 2012: Evaluation Report and the iLearn Evaluation Report 2013 in order to capture the needs and expectations of staff and to continue the ongoing quality enhancement cycle.

RESULTS AND DISCUSSION This evaluation study examined the second year of the operationalisation phrase of the iLearn platform. Data for this evaluation study was compiled through the iLearn Staff Experience Survey (Appendix 1). Invitations to complete the iLearn Staff Experience Survey were sent over a three-week period in Session 2, 2014, between the 8th and 29th October. The purpose of this survey was to:

monitor engagement with iLearn; evaluate satisfaction with services and support; and map changes in learning and teaching across the University.

The survey was sent only to members of staff using iLearn in Session 2, 2014. This approach to selecting the sample aligned with the creation of iLearn Staff Experience Survey in 2012 and 2013. As in previous years, the survey was delivered through University evaluation system, Teaching Evaluation for Development Service (TEDS), using EvaSys.

DEMOGRAPHIC DETAILSIn Session 2, 2014 invitations were sent to 1,713 iLearn users and a total of 210 responses were received, which represented 12.3% of the iLearn user population at that time. The response rate remains consistent with last year and provides adequate insight into the environment in which the technology is used.

iLearn Staff Experience Survey

Session 2, 2012

127 respondents

Session 2, 2013

208 respondents

Session 2, 2014

210 respondents

Faculty of Arts 28.3% 27.2% 22.9%Faculty of Science 26.0% 20.9% 29.5%Faculty of Human Science 24.4% 31.6% 21.0%Faculty of Business & Economics

First Experience of iLearn

Sessional StaffFixed Term or Continuing

21.3%

19.7%

23.2%72.8%

20.4%

13.6%

39.0%61.0%

26.7%

12.0%

41.7%58.3%

Table 1: The iLearn Staff Experience Survey Demographic Information for 2012 - 2014

TECHNOLOGIES USED TO ACCESS ILEARN: STAFF SURVEY RESULTSIn order to ascertain the technologies used to access iLearn, the survey respondents were asked to: “Please indicate the frequency with which you use the following devices to access iLearn”.

Technologies used to access iLearn 2014 At least once a day

A few times a

week

A few times

a mont

h

A few times a semeste

r

Never or

rarely

4

http://www.mq.edu.au/iLearn/docs/iLearn_EvaluationReport_2013.pdf

http://www.mq.edu.au/iLearn/documents.htm

http://www.mq.edu.au/iLearn/documents.htm

Desktop or laptop computer at other locations (n = 210)

51.9% 35.7% 5.2% 2.9% 4.3%

University provided computer on campus (n = 210)

42.9% 32.9% 5.2% 6.2% 12.9%

Smartphone using another network provider (n = 199)

14.1% 9.0% 6.5% 14.6% 55.8%

My own laptop using OneNet (n = 197) 13.2% 18.3% 12.2% 12.2% 44.2%Tablet at other locations (n = 200) 9.5% 9.0% 11.0% 9.0% 61.5%Smartphone at MQ using OneNet (n = 203) 7.9% 10.8% 9.9% 12.8% 58.6%Tablet at MQ using OneNet (n = 202) 3.5% 12.4% 11.9% 11.4% 60.9%Other device/network combination (n = 166) 1.2% 1.2% 0.6% 0.6% 96.4%

Table 2: Technologies used to access iLearn according to the 2014 staff survey

The table above relates to the use of various technologies to access iLearn, as it is important to monitor if the system is meeting the needs of new technological devices and if the network is able to support the system and devices in all circumstances. The staff responses to the last three surveys have shown a consistent approach to which technologies are relied upon to access iLearn with ‘Desktop or laptop computer at other locations’ and ‘University provided computer on campus’ being the most popular choices. It is possible that there might be some changes in 2015 with the use of the iLearn mobile enabled interface and the further proliferation of technologies like Smartphones and Tablets, however, at the moment, the use of these technologies ranked highly in the ‘never or rarely’ category. Further statistical analysis revealed that staff members in the Faculty of Arts indicate frequent use of 'Desktop computers at other locations' (p<0.01) in greater proportion than surveyed staff members in other faculties. Staff members that are employed in Full-time fixed or permanent positions indicate frequent use of ‘University provided computer on campus’ (p < 0.05) in greater proportion than the casual or sessional staff members surveyed. Staff members that are employed as casual or sessional staff indicate frequent use of 'Own laptop at MQ using OneNet' (p<0.01) to access iLearn in greater proportion than the surveyed staff members in Full-time fixed or permanent positions. These findings seem consistent with employment conditions of staff members (see Appendix 4).

STAFF SATISFACTION WITH ILEARN AS A SUPPORT FOR LEARNING AND TEACHINGThe table below provides a summary of the results from the 2014 survey.

2014 iLearn Staff Experience Survey Results Agreement

Neutral / Mixed

Feelings

Disagreement

N

I am satisfied with iLearn as an online support for my unit content.

72.4% 15.8% 11.8% 196I am satisfied with iLearn as a support for organising my teaching.

69.5% 17.4% 13.1% 190I am satisfied with the way iLearn supports interaction and communication with my students.

62.5% 18.8% 18.8% 197

I am satisfied with the way iLearn supports my students'. engagement with learning activities.

60.3% 24.2% 15.5% 194

Table 3: Staff Satisfaction with iLearn as a support for learning and teaching

The figures below offer an indication the surveyed staff members’ responses since 2012. The number of the surveyed staff members’ responses vary thus the number of responses (n=) received are indicated in each.

5

Disagreement

Neutral / Mixed Feelings

Agreement

13.6%

16.0%

70.4%

10.8%

21.1%

68.0%

11.8%

15.8%

72.4%

I am satisfied with iLearn as an online support for my unit content

2014, S2 (n = 196) 2013, S2 (n = 194) 2012, S2 (n = 125)

Figure 1: Satisfaction with iLearn as an online support for unit content

According to the 2014 staff survey responses 72.4% are in agreement that they are satisfied with iLearn as an online support for their unit content. This has increased since Session 2, 2013 (68%) and Session 2, 2012 (70.4%). The neutral / mixed feelings and the level of disagreement with the statement have also decreased since Session 2, 2012.

Disagreement


Agreement

10.6%

20.5%

68.8%

16.8%

22.4%

60.7%

13.1%

17.4%

69.5%

I am satisfied with iLearn as a support for organising my teaching

2014, S2 (n = 190) 2013, S2 (n = 196) 2012, S2 (n = 122)

Figure 2: Satisfaction with iLearn as a support for organising my teaching

Staff satisfaction with iLearn as a support for organising teaching, according to the survey responses in 2014, is 69.5%, this is higher than previous years. The neutral / mixed feelings category and the percentage of staff members that disagreed with this statement are also less than 2013 and 2012.

6

Disagreement


Agreement

21.9%

22.0%

56.1%

20.4%

19.8%

59.9%

18.8%

18.8%

62.5%

I am satisfied with the way iLearn supports in-teraction and communication with my students

2014, S2 (n = 197) 2013, S2 (n = 192) 2012, S2 (n = 123)

Figure 3: Satisfaction with iLearn as a support for interaction and communication with students

According to the 2014 staff responses 62.5% are in agreement that they are satisfied with the way iLearn supports interaction and communication with students, which is higher than previous years. The neutral / mixed feelings category and the percentage of staff members that disagreed with this statement are also less than 2013 and 2012. Furthermore, staff members employed as casual or sessional staff indicated that they are satisfied with the way iLearn supports their interaction and communication with students (p<0.05) to a greater extent than surveyed staff members in Full-time fixed or permanent positions (see Appendix 5).

Disagreement


Agreement

15.50%

22.00%

62.60%

15.60%

19.90%

64.50%

15.50%

24.20%

60.30%

I am satisfied with the way iLearn supports my students' engagement with learning activities

2014, S2 (n = 194) 2013, S2 (n = 186) 2012, S2 (n = 123)

Figure 4: Satisfaction with the way iLearn supports student engagement

Staff satisfaction with the way iLearn supports student engagement with learning activities, according to the survey responses in 2014, is 60.3%. The neutral / mixed feelings category has risen while the percentage of staff members that disagreed with this statement is almost identical to previous years.

7

STAFF SKILLS AND CONFIDENCE LEVELS Measuring staff skills and confidence levels is difficult when relying upon responses to scaled questions, however the following results offer an indication of the changes that have taken place between the initial introduction and implementation of iLearn to the second year of the operational phase.

STAFF CONFIDENCE LEVELSThe table below provides a summary of the results from the following six scaled questions in the 2014 survey:2014 iLearn Staff Experience Survey Results Agreeme

ntNeutral /

Mixed Feelings

Disagreement

N

I have developed the necessary skills to effectively use iLearn.

70.6% 21.6% 7.9% 204I feel confident when using iLearn in my teaching.

77.1% 19.4% 3.4% 206I feel confident when using Echo360 in my teaching.

62.6% 27.2% 10.1% 158

I feel confident when using iTeach to manage enrolments.

62.5% 21.5% 16% 144

I feel confident when using iTeach to create my Unit Guide.

61.3% 19.4% 19.3% 155

I feel confident when using iTeach to map learning outcomes, assessment tasks and graduate capabilities.

58.6% 19.7% 21.8% 152

Table 4: Staff Confidence levels in using iLearn as a support for learning and teaching

The figures below display the staff responses to the above statements in comparison to the survey results in 2012 and 2013.

Disagreement


Agreement

11.3%

21.0%

67.7%

13.9%

16.9%

69.2%

7.9%

21.6%

70.6%

I have developed the necessary skills to effect -ively use iLearn

2014, S2 (n = 204) 2013, S2 (n = 201) 2012, S2 (n = 124)

Figure 5: Development of skills to effectively use iLearn

According to the staff members surveyed 70.6% agree that they have developed the necessary skills to effectively use iLearn. This is almost identical to the 2013 results, however, there has been an increase in the ‘neutral / mixed feelings’ category and a decrease in the percentage of staff members that disagreed with this statement.

8

Disagreement


Agreement

7.3%

14.6%

78.1%

8.4%

15.8%

75.7%

3.4%

19.4%

77.1%

I feel confident when using iLearn in my teach-ing

201, S2 (n = 206) 2013, S2 (n = 202) 2012, S2 (n = 123)

Figure 6: Staff confidence when using iLearn

According to the staff members surveyed 77.1% agree that they feel confident when using iLearn in their teaching. This is slightly higher than Session 2, 2013 (75.7%) and slightly less than Session 2, 2012 (78.1%) nevertheless there was a decrease in the number of staff members surveyed that disagreed with this statement and consequently an increase in the ‘neutral / mixed feelings’ category.

Disagreement


Agreement

13.4%

26.7%

60.0%

12.0%

23.3%

64.6%

10.1%

27.2%

62.6%

I feel confident when using Echo360 in my teaching

2014, S2 (n = 158) 2013, S2 (n = 150) 2012, S2 (n = 90)

Figure 7: Staff confidence when using Echo360

A total of 62.6% of the staff members surveyed agree that they feel confident when using Echo360 in their teaching. This result shows a slight increase since Session 2, 2012 (60%) and a slight decrease from Session 2, 2013 (64.6%).

9

Disagreement


Agreement

24.5%

28.3%

47.2%

21.7%

22.4%

56.0%

16.0%

21.5%

62.5%

I feel confident when using iTeach to manage enrolments

2014, S2 (n = 144) 2013, S2 (n = 143) 2012, S2 (n = 106)

Figure 8: Staff confidence when using iTeach

A total of 62.5% of the staff members surveyed agree that they feel confident when using iTeach in their teaching. This is the highest result across the three surveys and accordingly there was a reduction in ‘neutral / mixed feelings’ category and a decrease in the ‘disagree’ and ‘strongly disagree’ categories. There were no statistical differences between Faculties or Staff Appointment, however, staff members with previous experience of iLearn are confident to a greater extent when using iTeach to manage enrolments than staff members using iTeach for the first time (p<0.01) (see Appendix 5).

Disagreement


Agreement

0.323

0.181

0.497

0.193

0.194

0.613

I feel confident when using iTeach to create my Unit Guide

2014, S2 (n = 155) 2013, S2 (n = 155) 2012

Figure 9: Staff confidence when using iTeach to create Unit Guides

The percentage of staff members that agree they feel confident when using iTeach (UNITS) to create their Unit guide has significantly increased since 2013. There has been a slight rise in the neutral/mixed feelings category but a significant drop in the levels of disagreement.

10

Disagreement


Agreement

0.352

0.179

0.468

0.218

0.197

0.586

I feel confident when using iTeach to map learning outcomes, assessment tasks and

graduate capabilities 2014, S2 (n = 152) 2013, S2 (n = 156) 2012

Figure 10: Staff confidence when using iTeach to map student learning outcomesFrom the 152 staff members that responded to this scaled question 58.6% agree with this statement, while there has been a slight rise in the neutral/mixed feelings category there has also been a significant drop in the levels of disagreement.

CHANGING PRACTICEStaff confidence levels contribute to maintaining an environment for change in practice. The table below provides a summary of the results from the following two scaled questions:

2014 iLearn Staff Experience Survey Results Agreement

Neutral / Mixed Feeling

s

Disagreement

N

As a result of using iLearn I have reconsidered the way I teach my unit.

51.9% 17.7% 30.4% 181

I would like to utilise the tools in iLearn to enhance my teaching in the future.

76.4% 12.3% 11.3% 195

Table 5: Staff members changing practice by using iLearn as a support for learning and teaching

The figures below display the staff responses to the above statements in comparison to the survey results in 2012 and 2013.

11

Disagreement


Agreement

29.8%

22.8%

47.4%

31.8%

18.8%

49.4%

30.4%

17.7%

51.9%

As a result of using iLearn I have reconsidered the way I teach my unit

2014, S2 (n = 181) 2013, S2 (n = 156) 2012, S2 (n = 114)

Figure 11: Change in teaching practice

The level of agreement to this survey question 51.9% continues to marginally rise and as a consequence the level of neutral / mixed feelings slowly decreasing over the past three years. Significantly almost a third of those surveyed continue to be in disagreement. Conversely, staff members employed as casual or sessional staff indicated that as a result of using iLearn they have reconsidered the way they teach their unit (p<0.05) to a greater extent than surveyed staff members in Full-time fixed or permanent positions. Whereas staff members, using iLearn for the first time are less likely to agree with this statement (p<0.05) (see Appendix 5).

Disagreement


Agreement

6.4%

5.6%

87.9%

11.9%

17.1%

71.0%

11.3%

12.3%

76.4%

I would like to utilise the tools in iLearn to enhance my teaching in the future

2014, S2 (n = 195) 2013, S2 (n = 193) 2012, S2 (n = 124)

Figure 12: Change in future teaching practice

Of the 195 staff members who responded to this survey question, 76.4% agree with this statement, which is higher than Session 2, 2013 but still lower than Session 2, 2012. The neutral / mixed feelings and the levels of disagreement have also decreased.

While the system data shows a slight increase in the use of alternative iLearn functions, it may be that there needs to be a stronger link between alternative iLearn functions and how it can benefit or

12

enhance online pedagogy. This category also includes the percentage of staff members surveyed that had no previous experience of iLearn before Session 2, 2014.

Surveyed Staff Members



0.197

0.136

0.12

I had no experience of iLearn/Moodle until this year

2014, S2 (n=208) 2013, S2 (n = 208) 2012, S2 (n = 127)

Figure 13: Staff Experience of iLearn / Moodle

In Session 1, 2012 a total of 74.4% of staff members surveyed had no previous experience of Moodle, however, in Session 2, 2014 only 12% of staff surveyed had no previous experience of iLearn / Moodle. This is a slight decrease from Session 2, 2012-2013 and a significant decrease from the iLearn implementation in Session 1, 2012. This result indicates that there needs to be a focus on the next phase of change, specifically, how the available technology can benefit or enhance online pedagogy.

STAFF EXPERIENCE WITH ILEARN This section focuses on staff engagement with iLearn by incorporating four forms of data sourced from: the iLearn system; the scaled responses; open ended responses; and statistical analysis.

STATISTICS ON THE USE OF ILEARN FUNCTIONS: DATA FROM THE ILEARN SYSTEMThe percentages listed in the table below were generated through the iLearn system. The purpose of generating this data is to compare the results between the use of the iLearn functions with the surveyed staff members responses to the ‘usefulness’ of each iLearn function. This comparison will not be possible for some of the iLearn functions, for example, there is no data available on Twitter Feeds, RSS Feeds, Announcements, Videos, Links to eReserve, Gradebook and Calendar. This is because they are considered to be ‘Moodle block instances’, thus it is problematic to extract this information from the iLearn platform. There are also multiple assignments used in each unit and the introduction of two classifications for assignments in 2013, however, the system showed in Session 2, 2014 there were: 317 units with one or more moodle assignments and 641units with one or more Turnitin assignments. This equates to 95.6% of iLearn units had either one or more moodle assignments or one or more Turnitin assignments in Session 2, 2014.

S2, 2012 S2, 2013 S2, 2014

Number of iLearn Units 1,160 1,034 1,002

iLearn Functions – The % of units that used…Discussion Forum 99.0% 99.1% 99.9%

Online Assignment Submission -------- -------- 95.6%

Labels 94.1% 90.3% 94.0%

13

Links to readings & external sites 83.5% 89.2% 92.5%

Echo360 61.3% 67.9% 75.0%

Turnitin (includes GradeMark) 31.0% 47.8% 61.6%

Dialogue Module 42.5% 36.9% 38.2%

Quiz 18.8% 20.6% 25.5%

Grouping/Groups 4.9% 6.2% 7.0%

Blog 2.4% 4.4% 5.5%

Chat 4.4% 6.7% 5.0%

Wiki 2.4% 4.3% 3.5%

Book Module 3.2% 3.7% 3.5%

Database 2.0% 2.4% 2.3%

Workshop Tool 1.2% 1.5% 1.4%

Lesson 0.6% 0.9% 1.0%

Table 6: The percentage of units that used the various iLearn functions

During the implementation of iLearn all units were required to have an online presence, however, some postgraduate courses do not have a corresponding iLearn unit thus only postgraduate coursework units were included in the survey’s research sample. The total numbers of units that these percentages are based, do not include the units in the system that were labelled as: ‘test’; ‘iLearn Admin’ or ‘Do not use’. It is also important to note that some internal and external units might be listed as separate courses however they only have one iLearn instance, hence the number of iLearn units listed above appear to be less than the courses offered by the University. The highest increase in use, according to the table above, was Turnitin, which also includes GradeMark. This could be attributed to this function having a well-defined purpose that relates to all units with a written assessment task. It could also be attributed to the fact that GradeMark was mandated in some Faculties, as this would have had an impact on the uptake. There was consistency in the use of: Discussion Forums; links to external sites; Echo360; Dialogue Module; Blogs; Book Module and Databases. There was a slight decrease in the use of Chat; Wiki and Workshop Tool. The 2014 iLearn Staff Experience Survey was the first year that we have included Grouping/Groups and Labels, however, Grouping and Groups has shown a slight increase in use over the past three years and the use of Labels has slightly increased since 2013.

14

Discussi

on Forum

Labels

Links

to read

ings & ex

ternal s

ites

Echo360

Turn-it-

in (inclu

des Grad

eMark

)

Dialogu

e Module

Quiz

Grouping/Groups

BlogChat

Wiki

Book Module

Database

Workshop To

ol

Lesson

0.00%

20.00%

40.00%

60.00%

80.00%

100.00%

120.00%

Learn functions in UG and PG Units

653 UG Units, S2 2014349 PG Units, S2 2014

Figure 14: Undergraduate and Postgraduate Units used of iLearn Functions

The above figure shows the percentage of undergraduate and postgraduate units that used the various iLearn functions. The percentage of use across all undergraduate and postgraduate units are very similar for all the iLearn functions except there is greater use of the Dialogue Module in postgraduate units and great use of Quizzes in undergraduate units.

SCALED RESPONSES ON THE PERCEIVED VALUE OF ILEARN FUNCTIONS: DATA FROM THE SURVEYSAll the iLearn functions are used for different purposes thus the staff experiences of learning and teaching using the range of iLearn functions, were organised in two categories:

iLearn Functions for organising teaching iLearn Functions for engaging with learning activities

At the beginning of each category there is a short summary interpreting the results, which is then summarised in a table. Further information is provided on each iLearn function that situates the results within any system data, open ended comments by staff members and statistical analysis by Faculty and Staff Appointment (see Appendix 6). The feedback in the open ended responses to: “Please tell us about any ongoing technical issues you have experienced during 2014 (for example, with iLearn, Echo360, iTeach or the network)” and “What would enable you to use iLearn more effectively in your teaching?” These questions had a total of 151 and 171 comments respectively will be explored in the summaries below.

ILEARN FUNCTIONS FOR ORGANISING STUDENTS LEARNINGAccording to the staff members surveyed in 2014 Announcements were the most effective in meeting staff members needs with regards to organising students’ learning (88.5%). This statement also had the highest response rate (n=191) from the 210 survey responses received and has continued to be ranked highly by staff members since the implementation of iLearn. Online Assignment Submission had the second highest level of agreement (85.2%) in this category followed by Turnitin (82.1%), Labels (72.2%), GradeMark (71.2%), Grades (70.1%), Activity logs (56.8%), Reports (49.5%) and Calendar (44.3%).

2014 iLearn Staff Experience Survey Results

Agreement

Neutral / Mixed

Feelings

Disagreement

N

Announcements 88.5% 6.3% 5.3% 191Online Assignment Submission

85.2% 12.1% 2.7% 182Turnitin 82.1% 14.3% 3.6% 168

15

Labels 72.2% 20.4% 7.2% 137Grademark 71.2% 14.7% 14.1% 163Grades 70.1% 17.5% 12.4% 177Activity Logs 56.8% 28% 15.2% 125Reports 49.5% 34.2% 16.2% 111Calendar 44.3% 32.2% 23.5% 115

Table 7: iLearn functions that facilitate organising students learning

The figures below display the staff responses to the above iLearn Functions in comparison to the survey results in 2012 and 2013.

ANNOUNCEMENTSThe results showed that 88.5% of staff members surveyed agree that Announcements were effective in meeting their needs with regard to organising students' learning (n=191). This has been a consistent finding since iLearn’s implementation in 2012.

Disagreement


Agreement

5.6%

5.6%

88.7%

5.7%

8.3%

86.0%

5.3%

6.3%

88.5%

Announcements2014, S2 (n = 191) 2013, S2 (n = 193) 2012, S2 (n = 124)

Figure 15: iLearn Function Announcements

There is no system data for Announcements as they are considered to be ‘Moodle block instances’. In response to “Please tell us about any ongoing technical issues you have experienced during 2014 (for example, with iLearn, Echo360, iTeach or the network)” There were two comments (n = 151) that suggested Announcements have vanished or that students consider these announcements to be spam from the University. There were five comments that related to Announcements in response to “What would enable you to use iLearn more effectively in your teaching?” (n = 171) that suggested improvements are required in how and where the message is sent as well as how it is categorised.

ONLINE ASSIGNMENT SUBMISSIONOnline Assignment Submission was also ranked highly by staff members who have completed the 2014 survey. A total of 85.2% agree Online Assignment Submission was effective in meeting their needs with regard to organising students' learning (n=182).

16

Disagreement


Agreement

19.7%

12.4%

68.0%

11.5%

15.6%

72.9%

2.7%

12.1%

85.2%

Online Assignment Submission2014, S2 (n = 182) 2012, S2 (n = 96) 2012, S1 (n = 153)

Figure 16: iLearn Function Online Assignment Submission

The system data shows that 95.6% of iLearn units had either one or more moodle assignments or one or more Turnitin assignments in Session 2, 2014. In response to “Please tell us about any ongoing technical issues you have experienced during 2014 (for example, with iLearn, Echo360, iTeach or the network)” There were five comments (n = 151) that revealed staff members have experienced issues with: the slowness of the system, delays in the email notifications and students uploading their assignments or reading feedback. There were three comments that related to Online Assignment Submission in response to “What would enable you to use iLearn more effectively in your teaching?” (n = 171) there were calls for improvements to track late submissions and process of assigning marks.

TURNITINTurnitin was ranked highly by staff members who have completed the 2014 survey; however, the open-ended responses revealed the difficulties experienced when using the system. Nevertheless, 82.1% agree Turnitin was effective in meeting their needs with regard to organising students' learning (n=168).

Disagreement


Agreement

0.088

0.165

0.747

0.036

0.143

0.821

Turn-it-in2014, S2 (n = 168) 2013, S2 (n = 158) 2012

Figure 17: iLearn Function Turnitin

According to the system data the highest increase in use over the last two years was Turnitin, which also includes GradeMark. This could be attributed to this function having a well-defined purpose that relates to all units with a written assessment task. The iLearn system data indicates that the

17

use of Turnitin has had an increase from 47.8% of units in Session 2, 2013 to 61.6% of units in Session 2, 2014. In response to “Please tell us about any ongoing technical issues you have experienced during 2014 (for example, with iLearn, Echo360, iTeach or the network)” there were 18 comments (n = 151) that revealed staff members have experience issues with: the site being unavailable, submission of marks, system slowness, non-compatibility with certain mobile devices, students having difficulties uploading their assignments and the need to access to a reliable manual. There were five comments that related to Turnitin in response to “What would enable you to use iLearn more effectively in your teaching?” (n = 171) one staff member said: “Marking on Turnitin is faster but the depth of the marking and feedback is not as good”. The other four comments referred to a faster and simpler system that is stable. Staff members in the Faculty of Arts and the Faculty of Business and Economics found Turnitin effective in meeting their needs with regards to organise students learning (p<0.05) to a greater extent than staff members surveyed in other faculties (see Appendix 6).

LABELSThe system data shows that 94.0% of iLearn units in Session 2, 2014 used Labels. The survey results revealed that 72.2% of staff members agree Labels were effective in meeting their needs with regard to organising students' learning (n=137). There is no survey data for this iLearn Function prior to 2014 thus no comparison graph can be shown. However, the iLearn system data indicates that the use of Lables has had an increase from 90.3% of units in Session 2, 2013 to 94.0% of units in Session 2, 2014. In response to “Please tell us about any ongoing technical issues you have experienced during 2014 (for example, with iLearn, Echo360, iTeach or the network)” there one comment (n=151) that related to labels. This staff member said: “The default iLearn style this year was overly complex and space wasting on the screen. I opted for an older style and simplified it further by removing icons from the labels in each week. It was tedious to have to recreate the labels for lectures, reading etc. it is a pity to waste screen space particularly with iPhone and iPad users whose screen is limited”. There was also one comment that related to Labels in response to “What would enable you to use iLearn more effectively in your teaching?” (n = 171). This staff member said: “More flexible design. The main collapsible "week" sections are fine, but I also need collapsible subsections within the weeks. Labels is an interim measure but not collapsible. Also an icon library with easily accessible icons including but certainly not limited to the standard iLearn ones”.

GRADEMARKGradeMark was also ranked highly by staff members who have completed the 2014 survey; however, the open-ended responses revealed some difficulties when using the system. Despite this a total of 71.2% agree GradeMark was effective in meeting their needs with regard to organising students' learning (n=163).

Disagreement


Agreement

0.18

0.222

0.597

0.141

0.147

0.712

GradeMark2014, S2 (n = 163) 2013, S2 (n = 144) 2012

Figure 18: iLearn Function GradeMark

According to the system data the highest increase in use over the last three years was Turnitin, which also includes GradeMark. Any ongoing issues related to the technology are also outlined in Turnitin. In response to “Please tell us about any ongoing technical issues you have experienced

18

during 2014 (for example, with iLearn, Echo360, iTeach or the network)” there were seven comments (n=151) that related to GradeMark being an unreliable system, the need for more flexibility with marking, being too time consuming to set-up and not saving grades. There were three comments that related to GradeMark in response to “What would enable you to use iLearn more effectively in your teaching?” (n = 171) the suggested improvements were capabilities for: allocating half marks, improved marking rubric and one staff member said “Grademark is great on Turnitin iPad app except that it is not possible to link comment to rubric in app version”. Staff members employed as casual or sessional staff found GradeMark to be effective in meeting their needs with regards to organising students learning (p<0.05) to a greater extent than surveyed staff members in Full-time fixed or permanent positions (see Appendix 6).

GRADESThe results showed that 70.1% of staff members surveyed agree that Grades (Gradebook) was effective in meeting their needs with regard to organising students' learning (n=177), which is a significant rise from last year.

Disagreement


Agreement

13.8%

20.0%

66.3%

17.1%

22.9%

60.0%

12.4%

17.5%

70.1%

Grades2014, S2 (n = 177) 2013, S2 (n = 140) 2012, S2 ( n = 80)

Figure 19: iLearn Function Grades

There is no system data for Grades as it is considered to be a ‘Moodle block instance’. In response to “Please tell us about any ongoing technical issues you have experienced during 2014 (for example, with iLearn, Echo360, iTeach or the network)” There were eight comments (n = 151) that indicated issues with the design, slowness, difficulty uploading grades and problems with hiding grades. There were nine comments that related to Grades in response to “What would enable you to use iLearn more effectively in your teaching?” (n = 171) the suggested improvements were: improved functionality including a red pen, calculation of grades and upload capability.

ACTIVITY LOGSThe results showed that 56.8% of staff members surveyed agree that Activity logs were effective in meeting their needs with regard to organising students' learning (n=125). There is no previous survey data for this iLearn Function thus no comparison graph can be shown. In response to “Please tell us about any ongoing technical issues you have experienced during 2014 (for example, with iLearn, Echo360, iTeach or the network)” there one comment (n=151) that related to Activity Logs commenting on the long loading time. There were four comments that related to Activity Logs in response to “What would enable you to use iLearn more effectively in your teaching?” (n = 171), which referred to a more streamlined process for monitoring students activity. One staff member said: “Activity logs are difficult to use, time consuming (having to check each individual students activity log is a huge time burden on teaching staff) and arduous as a mechanism for monitoring student activity”. Staff members in the Faculty of Business and Economics and Faculty of Human Sciences found Activity Logs effective in meeting their needs with regards to organise students learning (p<0.01) to a greater extent than staff members surveyed in other faculties (see Appendix 6).

REPORTSThe results showed that 49.5% staff members surveyed agree that Reports were effective in meeting their needs with regard to organising students' learning (n=111). There is no previous survey data for this iLearn

19

Function thus no comparison graph can be shown. This function requires further publicity because there are calls for reporting features by some staff members but it was unclear whether these comment were referring to Activity Logs and calling them Reports or if the comments were regarding the Reports that iLearn can generate.

CALENDARA total of 43.3% of staff members surveyed agree that the Calendar was effective in meeting their needs with regard to organising students' learning (n=115), which is a significant increase from last year.

Disagreement


Agreement

21.7%

35.0%

43.3%

27.3%

35.9%

36.7%

23.5%

32.2%

44.3%

Calendar2014, S2 (n = 115) 2013, S2 (n = 117) 2012, S2 (n = 60)

Figure 20: iLearn Function Calendar

There is no system data for Calendar as it is considered to be a ‘Moodle block instance’. There were no comments (n = 151) regarding the calendar in response to “Please tell us about any ongoing technical issues you have experienced during 2014 (for example, with iLearn, Echo360, iTeach or the network)” There was one comment in response to “What would enable you to use iLearn more effectively in your teaching?” (n = 171) that staff member said: “The calendar isn't particularly useful. Last time I checked I wasn't flexible enough to be an effective reminder about deadlines, upcoming events, etc”. Conversely, staff members in Faculty of Business and Economics found the calendar effective in meeting their needs with regards to organise students learning (p<0.05) to a greater extent than staff members surveyed in other faculties (see Appendix 6).

ILEARN FUNCTIONS FOR ENGAGING STUDENTS WITH LEARNING ACTIVITIESAccording to the staff members surveyed in 2014 Discussion Forms were the most effective in meeting staff members needs with regards to engaging students’ with learning activities (72.6%). This statement also had the highest response rate (n=183) from the 210 survey responses received and has continued to be ranked highly by staff members since the implementation of iLearn. Quizzes had the second highest level of agreement (70.3%) in this category followed by Groupings and Groups (62.1%), Dialogue Module (60.4%), Database (42.8%), Workshop Tool (38.2%), Blog (34.2%), Wiki (30.5%), Chat (27.3%) and Twitter Feeds (26%). The three iLearn functions with the highest Neutral / Mixed Feelings rankings were: Workshop Tool (38.2%) Twitter Feeds (34.8%) and Blogs (34.2%).

2013 iLearn Staff Experience Survey Results

Agreement

Neutral / Mixed

Feelings

Disagreement

N

Discussion Forums 72.6% 13.7% 13.7% 183Quizzes 70.3% 16.1% 13.6% 118Groupings and Groups 62.1% 19.4% 18.5% 103Dialogue Module 60.4% 19.4% 20.2% 139Database 42.8% 28.6% 28.5% 56Workshop Tool 38.2% 38.2% 23.7% 55Blog 34.2% 34.2% 31.5% 73Wiki 30.5% 30.5% 38.9% 59Chat 27.3% 33.3% 39.4% 66

20

Twitter Feeds 26% 34.8% 39.1% 46Table 8: iLearn functions that facilitate engagement with learning activities

DISCUSSION FORUMSThere was a slight increase from last year in the agreement levels as 72.6% of staff members surveyed agree that Discussion Forums were effective in meeting their needs for engaging their students in learning activities (n=183).

Disagreement


Agreement

12.0%

16.2%

71.8%

11.6%

20.0%

68.3%

13.7%

13.7%

72.6%

Discussion Forums2014, S2 (n = 183) 2013, S2 (n = 180) 2012, S2 (n = 117)

Figure 21: iLearn Function Discussion Forums

The iLearn system data indicates that the use of Discussion Forums has remained for the last three years (2012 – 99%; 2013 – 99.1% and 2014 – 99.9%).There was one comment (n = 151) regarding Discussion Forums in response to “Please tell us about any ongoing technical issues you have experienced during 2014 (for example, with iLearn, Echo360, iTeach or the network)”. This staff member said: “Students report receiving so much spam from iLearn discussion boards that they miss important announcements”. In response to “What would enable you to use iLearn more effectively in your teaching?” (n = 171) there were nine comments that referred to providing: an anonymous option, like and rate capabilities, group specific discussion forums. There were also comments requesting an end to the email notices and better organisation of the discussion threads.

QUIZZESThere was an increase from previous years as 70.3% of staff members agree Quizzes were effective in meeting their needs for engaging their students in learning activities (n=118).

Disagreement


Agreement

20.4%

18.4%

61.2%

22.1%

17.9%

60.0%

13.6%

16.1%

70.3%

Quizzes2014, S2 (n = 118) 2013, S2 (n = 95) 2012, S2 (n = 49)

Figure 22: iLearn Function Quizzes

21

The iLearn system data indicates that the use of Quizzes has increased from 20.6% of iLearn units in Session 2, 2013 to 25.5% of iLearn units in Session 2, 2014. There were four references (n = 151) to Quizzes in response to “Please tell us about any ongoing technical issues you have experienced during 2014 (for example, with iLearn, Echo360, iTeach or the network)” that revealed staff members have experience issues with the new theme not showing the answers to quiz questions. There was also a request to streamline the method of implementing an online quiz. In response to “What would enable you to use iLearn more effectively in your teaching?” (n = 171) there were seven comments that referred to flexibility in the type of quiz questions, for example, an open-ended response text box. There were also comments regarding the limitations of quizzes, experiencing problems with the inclusion of generic questions in quizzes, issues with using embedded quizzes as well as access to software for creating quizzes in Word. There were no statistical differences between Faculties or Staff Appointment, however, staff members using iLearn for the first time were less likely to agree that Quizzes were effective in meeting their needs for engaging their students in learning activities (p<0.05) than staff members with previous experience (see Appendix 6).

GROUPING AND GROUPSA total of 62.1% of staff members who have completed the 2014 survey, consider that grouping and groups were effective in meeting their needs for engaging their students in learning activities (n=103). There is no previous survey data for this iLearn Function. However, the iLearn system data indicates that the use of Grouping and Groups has had a slight increase from 6.2% of units in Session 2, 2013 to 7.0% of units in Session 2, 2014. There were three references (n = 151) to Grouping and Groups in response to “Please tell us about any ongoing technical issues you have experienced during 2014 (for example, with iLearn, Echo360, iTeach or the network)” that revealed staff members have experience issues with: resolving groupings and problems with enrolling students when they change tutorials or units. In response to “What would enable you to use iLearn more effectively in your teaching?” (n = 171) there were six comments that referred to providing different group scenarios each week, easier grouping by class, the need for auto organisation of students into groups and the procedure to create groups being too cumbersome. There were also requests for more workshops including more time allocated in workloads to attend these workshops as well as the need for the system to be able to produce a list of students that do not sign up for a group. Staff members in the Faculty Business and Economics, Faculty of Science, Faculty of Human Sciences found Groupings and Groups effective in meeting their needs with regards to engaging their students in learning activities (p<0.01) to a greater extent than staff members surveyed in the Faculty of Arts (see Appendix 6).

DIALOGUE MODULEThere was an increase from the 2013 survey results as 60.4% of staff members that the Dialogue Module (sending and receiving messages) was effective in meeting their needs for engaging their students in learning activities (n=139).

Disagreement


Agreement

27.3%

12.6%

60.0%

22.2%

26.2%

51.7%

20.2%

19.4%

60.4%

Dialogue Module2014, S2 (n = 139) 2013, S2 (n = 149) 2012, S2 (n = 95)

Figure 23: iLearn Function Dialogue Module

The iLearn system data indicates that the use of Dialogue Module has slightly increased from 36.9% of units in Session 2, 2013 to 38.2% of units in Session 2, 2014.There were two references (n = 151)

22

to the Dialogue Module in response to “Please tell us about any ongoing technical issues you have experienced during 2014 (for example, with iLearn, Echo360, iTeach or the network)” that requested one click access to dialogues and providing Carbon Copy (CC) and Blind Carbon Copy (BCC) options for messages. In response to “What would enable you to use iLearn more effectively in your teaching?” (n = 171) there were ten comments that focused on having a dialogue module that operates like email and has a group email function.

DATABASEThere was an increase in levels of agreement from previous surveys as a total of 42.8% agree Databases were effective in meeting their needs for engaging their students in learning activities (n=56).

Disagreement


Agreement

42.3%

23.1%

34.6%

35.9%

28.3%

35.9%

28.5%

28.6%

42.8%

Database2014, S2 (n = 56) 2013, S2 (n = 53) 2012, S2 (n = 26)

Figure 24: iLearn Function Database

The iLearn system data indicates that the use of the Database has marginally decreased from 2.4% of iLearn units in Session 2, 2013 to 2.3% of iLearn units in Session 2, 2014. There were no references to the Database in response to “Please tell us about any ongoing technical issues you have experienced during 2014 (for example, with iLearn, Echo360, iTeach or the network)”. However, in response to “What would enable you to use iLearn more effectively in your teaching?” (n = 171) there was one comment that called for further enhancements. This staff member said: “We use the database tool. However, we'd like to see a voting component added. So students can like a post and easily make comments similar to Facebook. Right now we have a work around using the choice activity to vote however, it's cumbersome”.

WORKSHOP TOOLThere has been a significant decrease in staff levels of agreement as, 38.2% of staff members agree that the Workshop Tool were effective in meeting their needs for engaging their students in learning activities (n=55).

23

Disagreement


Agreement

33.3%

19.0%

47.6%

31.5%

24.1%

44.4%

23.7%

38.2%

38.2%

Workshop Tool2014, S2 (n = 55) 2013, S2 (n = 54) 2012, S2 (n = 21)

Figure 25: iLearn Function Workshop Tool

The iLearn system data indicates that the use of the Workshop Tool has marginally decreased from 1.5% of iLearn units in Session 2, 2013 to 1.4% of iLearn units in Session 2, 2014. There were no references (n = 151) to Workshop Tool in response to “Please tell us about any ongoing technical issues you have experienced during 2014 (for example, with iLearn, Echo360, iTeach or the network)”. However, in response to “What would enable you to use iLearn more effectively in your teaching?” (n = 171) there were two comments that called for greater flexibility such as submission of late files and the workshop tool for voluntary participation. Staff members employed as casual or sessional staff found Workshops to be effective in meeting their needs with regards to engaging students in learning activities (p<0.05) to a greater extent than surveyed staff members in Full-time fixed or permanent positions (see Appendix 6).

BLOGA total of 34.2% of staff members consider the Blogs to be effective in meeting their needs for engaging their students in learning activities.

Disagreement


Agreement

35.5%

32.3%

32.3%

25.4%

38.1%

36.5%

31.5%

34.2%

34.2%

Blog2014, S2 (n = 73) 2013, S2 (n = 63) 2012, S2 (n = 31)

Figure 26: iLearn Function Blog

The iLearn system data indicates that the use of the Blog has marginally increased from 4.4% of iLearn units in Session 2, 2013 to 5.5% of iLearn units in Session 2, 2014. There were no references to blogs in response to “Please tell us about any ongoing technical issues you have experienced during 2013 (for example, with iLearn, Echo360, iTeach or the network)”. However, in response to “What would enable you to use iLearn more effectively in your teaching?” (n = 171) there were

24

three comments, two staff members who would like to use it and one staff member who used it in 14 tutorials and commented on how clunky it was. There were no statistical differences between Faculties or Staff Appointment, however, staff members using iLearn for the first time were less likely to agree that the Blog was effective in meeting their needs for engaging their students in learning activities (p<0.05) than staff members with previous experience (see Appendix 6).

WIKIA total of 30.5% agree that the Wiki was effective in meeting their needs for engaging their students in learning activities (n=59).

Disagreement


Argeement

35.7%

28.6%

35.7%

41.8%

32.7%

25.5%

38.9%

30.5%

30.5%

Wiki2014, S2 (n = 59) 2013, S2 (n = 55) 2012, S2 (n = 28)

Figure 27: iLearn Function Wiki

The iLearn system data indicates that the use of the Wiki has marginally decreased from 4.3% of iLearn units in Session 2, 2013 to 3.5% of iLearn units in Session 2, 2014. There was one comment referring to the Wiki in response to “Please tell us about any ongoing technical issues you have experienced during 2014 (for example, with iLearn, Echo360, iTeach or the network)”. This staff member said: “Group management in Wikis is badly designed and does not reflect user needs”. In response to “What would enable you to use iLearn more effectively in your teaching?” (n = 171) there were four comments that focused on improvements to the Wiki for ease of use and one comment that stated how it had been dumped for Google Docs to allow for synchronous editing. Staff members in the Faculty of Science found Wikis effective in meeting their needs with regards to engaging their students in learning activities (p<0.01) to a greater extent than staff members surveyed in other faculties (see Appendix 6).

CHATA total of 27.3% agree Chat was effective in meeting their needs for engaging their students in learning activities (n=66).

25

Disagreement


Agreement

41.4%

34.5%

24.1%

41.0%

31.1%

27.9%

39.4%

33.3%

27.3%

Chat2014, S2 (n = 66) 2013, S2 (n = 61) 2012, S2 (n = 29)

Figure 28: iLearn Function Chat

The iLearn system data indicates that the use of the Chat has marginally decreased from 6.7% of iLearn units in Session 2, 2013 to 5.0% of iLearn units in Session 2, 2014. There were no references to Chat in response to “Please tell us about any ongoing technical issues you have experienced during 2014 (for example, with iLearn, Echo360, iTeach or the network)”. However, in response to “What would enable you to use iLearn more effectively in your teaching?” (n = 171) there was one comment “We've used the chat a couple of times and it's difficult to see the connections between parties. Again can someone look outside the LMS for a Chat tool that makes having an online Chat with more than a dozen people easy to follow”.

TWITTER FEEDSA total of 26% agree Twitter feeds were effective in meeting their needs for engaging their students in learning activities (n=46), which is an increase from previous years.

Disagreement


Agreement

50.0%

28.6%

21.4%

41.5%

39.0%

19.5%

39.1%

34.8%

26.0%

Twitter Feeds2014, S2 (n = 46) 2013, S2 (n = 41) 2012, S2 (n = 14)

Figure 29: iLearn Function Twitter Feeds

There is no system data for Twitter Feeds as they are considered to be ‘Moodle block instances’. There were no references to Twitter Feeds in response to “Please tell us about any ongoing technical issues you have experienced during 2014 (for example, with iLearn, Echo360, iTeach or the network)”. However, in response to “What would enable you to use iLearn more effectively in your teaching?” (n = 171) One staff member said: “The twitter feed tool works well, but it would be great if it could filter re-tweets. When a tweet is retweeted it currently appears multiple times - and this is less than ideal”.

26

STAFF SATISFACTION WITH SERVICES AND SUPPORTIn order to ascertain levels of staff satisfaction with the services and support for the iLearn platform, the survey focused on two categories:

Support services for iLearn and iTeach; Support services for iLearn provided by the Faculties and the Learning and Teaching Centre

(LTC) including online resources.

SATISFACTION LEVELS WITH SUPPORT SERVICES FOR ILEARN AND ITEACHThe table below displays the surveyed staff members’ satisfaction with the support services for iLearn and iTeach. The number of the surveyed staff members’ responses vary thus the number of responses (n=) received are indicated in each.

2014 iLearn Staff Experience Survey Results Satisfied Neutral / Mixed

Feelings

Dissatisfied

N

Automatic enrolment of students facilitated by iTeach.

77.5% 17% 5.5% 147OneHelp for resolution of issues relating to iTeach functions.

62.6% 21.1% 16.3% 123Unit Guides (unitguides.mq.edu.au) for providing essential information to students.

53% 20.5% 26.6% 166

Training and support for using iTeach to create Unit Guides.

60% 24.6% 15.4% 130

OneHelp for resolution of issues relating to iLearn functions.

60.1% 22.1% 17.8% 163Table 9: Staff Satisfaction with support services for iLearn and iTeach

STAFF SATISFACTION WITH ITEACHA total of 77.5% of staff members surveyed are in agreement that they are satisfied with Automatic enrolment of students facilitated by iTeach (n=147).

Dissatisfied


Satisfied

13.5%

18.0%

68.4%

10.8%

15.5%

73.7%

5.5%

17.0%

77.5%

Automatic enrolment of students facilitated by iTeach

2014, S2 (n = 147) 2013, S2 (n = 148) 2012, S2 (n = 111)

Figure 30: Staff satisfaction with the automatic enrolment of students facilitated by iTeach

Since 2012, there has been an increase in the staff responses to the automatic enrolment of students facilitated by iTeach satisfied categories thus a significant drop in the ‘Netural / Mixed Feelings’ and a decrease in the dissatisfied categories. This could signify how the system has improved since its introduction. It could also be attributed to the time it takes to embed a significant cultural change.

STAFF SATISFACTION WITH ITEACH SUPPORTA total of 62.6% of staff members surveyed are in agreement that they are satisfied with OneHelp for resolution of issues relating to iTeach (n=123).

28

Dissatisfied


Satisfied

0.193

0.164

0.642

0.163

0.211

0.626

OneHelp for resolution of issues relating to iTeach

2014, S2 (n = 123) 2013, S2 (n = 140) 2012

Figure 31: Staff satisfaction with OneHelp for resolution of issues relating to iTeach

In response to “Please tell us about any ongoing technical issues you have experienced during 2014 (for example, with iLearn, Echo360, iTeach or the network)” there were three comments that related to iTeach (n = 151). These comments focused on: the system being cumbersome, problems giving access to guest lecturers or industry specialists and the systems structural problems. There were two comments in response to “Do you have any suggestions for improving the level and /or effectiveness of iLearn support and services?” (n = 82) one staff member said: “I find compiling unit guides on iTeach cumbersome and time consuming. I hate having to enter data in separate fields then combine it as I have 6 separate iLearn units for what is essentially the same unit guide. I would prefer to just upload a word document”. The other comment referred to the limited time for training in systems such as iTeach, when employed as a session staff member a few days before the session begins.

STAFF SATISFACTION WITH UNIT GUIDESA total of 53% of staff members surveyed are in agreement that they are satisfied with the Unit Guides (unitguides.mq.edu.au) for providing essential information to students (n=166). This was a new question thus there is no comparison data. In response to “Please tell us about any ongoing technical issues you have experienced during 2014 (for example, with iLearn, Echo360, iTeach or the network)” there were six comments that related to iTeach (n = 151). These comments focused on: the unnecessary detail, how it is a time consuming process, how the language is condescending, and there are rendering problems when published and printed. There were five comments in response to “Do you have any suggestions for improving the level and /or effectiveness of iLearn support and services?” (n = 82) that focused on: the difficulty using unit guides, navigation problems from iLearn to the unit guides, the document is too long and writing a unit and study guide for students is double the work. There were also nine comments in response to “Do you have any other comments you would like to add?” (n=87). These comments referred to: doubling up on information, the ugly interface, having an edit capacity after publication and connecting the assessments in the unit guide with grades. There were no other references to unit guides in the open-ended responses. Staff members employed as casual or sessional staff are satisfied that UNITS/Unit guides provide essential information to students (p<0.01) to a greater extent than surveyed staff members in Full-time fixed or permanent positions (see Appendix 7).

STAFF SATISFACTION WITH ITEACH TRAINING AND SUPPORTA total of 60% of staff members surveyed are in agreement that they are satisfied with Training and support for using iTeach to create Unit Guides (n=130). This was a new survey question to mirror the change in the system thus it is not possible to generate a comparison graph. However, in the 2013 survey staff members were asked to indicate their overall satisfaction with the training and support provided for UNITS (now iTeach) and 43.9% of the staff members surveyed indicated that they were satisfied thus there has been a significant increase in the last year.

STAFF SATISFACTION WITH ONEHELP FOR ILEARN SUPPORTA total of 60.1% of staff members surveyed are in agreement that they are satisfied with OneHelp for resolution of issues relating to iLearn functions (n=163).

29

Dissatisfied


Satisfied

15.5%

19.0%

65.4%

19.3%

14.3%

66.5%

17.8%

22.1%

60.1%

OneHelp for resolution of issues relating to iLearn functions

2014, S2 (n = 163) 2013, S2 (n = 161) 2012, S2 (n = 84)

Figure 32: Staff satisfaction with OneHelp for resolution of issues relating to iLearn functions

There were two comments in response to “Please tell us about any ongoing technical issues you have experienced during 2014 (for example, with iLearn, Echo360, iTeach or the network)” (n = 151) that related to OneHelp. These comments related to issues with Turnitin and students that were given the wrong advice through OneHelp. There were also seven comments in response to “Do you have any suggestions for improving the level and/or effectiveness of iLearn support and services?” (n = 82) that related to OneHelp. While staff members in general seem satisfied with OneHelp, there were comments that indicated they are would like the option of phone support. There were calls for an ‘urgent’ function within OneHelp and better student support.

SUPPORT SERVICES FOR ILEARN PROVIDED BY THE FACULTIES AND THE LTCThe table below displays the surveyed staff members’ satisfaction with the support services provided by the Faculties and the LTC. The number of the surveyed staff members’ responses vary thus the number of responses (n=) received are indicated in each.

2014 iLearn Staff Experience Survey Results Satisfied

Neutral / Mixed

Feelings

Dissatisfied

N

The ease of locating the iLearn online "self-help" resources.

52% 23.7% 24.3% 177The effectiveness of the iLearn online "self-help" resources.

45.6% 29.2% 25.2% 171The effectiveness of iLearn training workshops. 68.8% 22% 9.3% 141The support for using iLearn provided by the iLearn drop-in clinic.

75% 17.7% 7.2% 124

The level of Faculty-based support for using iLearn. 63% 19.7% 17.2% 157Table 10: Staff Satisfaction with support services provided by the Faculties and the LTC

STAFF SATISFACTION WITH ONLINE RESOURCESA total of 52% of staff members surveyed are in agreement that they are satisfied with ease of locating online "self-help" resources to assist me in using iLearn (n=177).

30

Dissatisfied


Satisfied

19.3%

14.3%

66.5%

0.243

0.237

0.52

Ease in locating online "self-help" resources2014, S2 (n = 177) 2013, S2 (n = 166) 2012

Figure 33: Staff satisfaction with the ease of locating online "self-help" resources

A total of 45.6% of staff members surveyed are in agreement that they are satisfied with the effectiveness of the online "self-help" resources in assisting my use of iLearn (n=171). There were two comments made in response to “Do you have any suggestions for improving the level and/or effectiveness of iLearn support and services?” (n=82). One staff member said they were satisfied and the other requested ease of access to Quick Guides.

Dissatisfied


Satisfied

0.246

0.344

0.412

0.252

0.292

0.456

The effectiveness of the online "self-help" resources

2014, S2 (n = 171) 2013, S2 (n = 163) 2012

Figure 34: Staff satisfaction with the effectiveness of the online "self-help" resources

The scaled responses indicate that has been a drop in the satisfaction levels with regards to the online resources ease of access; however there has been a slight increase in the satisfaction levels when staff members consider the effectiveness of the online resources. There were five comments in response to “Do you have any suggestions for improving the level and /or effectiveness of iLearn support and services?” (n = 82) two of these comments were very positive and the other two commented on how phone support is still needed when the self-help resources are not adequate. There was also one comment that suggested more resources for students were needed. There were also two comments in response to “Do you have any comments you would like to add?” (n = 80) both suggested improvements to tailoring the online resources.

31

STAFF SATISFACTION WITH THE ILEARN DROP-IN CLINICSA total of 75% of staff members surveyed are in agreement that they are satisfied with the support for using iLearn provided by the iLearn drop-in clinic (n=124).

Dissatisfied


Satisfied

0.037

0.164

0.799

0.072

0.177

0.75

Staff Satisfaction with the iLearn Drop-in Clinics2014, S2 (n = 124) 2013, S2 (n = 134) 2012

Figure 35: Staff Satisfaction with the iLearn Drop-in Clinics

In 2014, there were 159 visits and 137 visits from staff members to the iLearn Drop-in-Clinics in Session 1 and 2 respectively. In general, there has been very positive feedback. In response to “Do you have any suggestions for improving the level and/or effectiveness of iLearn support and services?” (n = 82) there were nine comments that focused on: the clinics need to begin earlier in the session, run all session and run at different times. One staff member also said that they were unaware of the drop-in clinics. There were also five comments in response to “Do you have any other comments you would like to add?” (n=87). One staff member said: “I can’t stress enough how valuable the drop in centre is for me pre-semester start up. The staff members are endlessly patient and helpful. Having the drop in centre reduces my stress re needing to get things ready to commence each semester. This semester I am convening 8 units so the work to set up each iLearn site is daunting but made much easier by the wonderful staff at the drop in centres. Please keep it going, same informal format and congratulate all involves with this practical and invaluable initiative. Thanks”.

STAFF SATISFACTION WITH ILEARN TRAINING WORKSHOPS A total of 68.8% of staff members surveyed are in agreement that they are satisfied with the effectiveness of iLearn training workshop (n=141).

32

Dissatisfied


Satisfied

0.092

0.222

0.686

0.093

0.22

0.688

Staff Satisfaction with the iLearn Training Workshops

2014, S2 (n = 141) 2013, S2 (n = 153) 2012

Figure 36: Staff Satisfaction with the iLearn Training Workshops

In response to “Do you have any suggestions for improving the level and/or effectiveness of iLearn support and services?” (n = 82) there were nine comments that related to iLearn Training Workshops. These comments referred to the high quality of the workshops, problems relating to casual staff appointments, specifically, not having the time or not having access to remuneration for time spent in training. There were also suggestions of linking the advertisement of training sessions to the academics first use of iLearn as well as offering training outside of peak times. In response to “Do you have any other comments you would like to add? (n=87)” there were four comments again reiterating not having the time or not having access to remuneration for time spent in training. In response to “What would enable you to use iLearn more effectively in your teaching?” (n = 171) there were six comments that simply listed ‘more training’ but no further details were provided.

STAFF SATISFACTION WITH FACULTY-BASED ILEARN SUPPORTA total of 63% of staff members surveyed are in agreement that they are satisfied with the level of Faculty-based support for using iLearn (n=157).

Dissatisfied


Satisfied

5.2%

7.0%

87.8%

13.9%

25.3%

60.7%

17.2%

19.7%

63.0%

Staff Satisfaction with Faculty iLearn Support2014, S2 (n = 157) 2013, S2 (n = 158) 2012, S2 (n = 115)

Figure 37: Staff Satisfaction with Faculty iLearn Support

In response to “Do you have any suggestions for improving the level and/or effectiveness of iLearn support and services?” (n = 82) there were eleven comments regarding Faculty Support. While

33

there were several compliments it was obvious that some staff members would like more ‘hands-on’ support and that they are very grateful when they do receive it. In response to “Do you have any other comments you would like to add?” (n=87) there were twelve comments that referred iLearn support and services. Again all but two of these comments were compliments about the excellent support staff members have received, for example, one staff member said: “I think the iLearn support team, at least the majority of individuals I have personally worked with, are fantastic. They are enthusiastic, professional, personable, and really want to help. They are a pleasure to work with”.

ILEARN STATUS PAGEA total of 13.3% of staff members surveyed have accessed the iLearn Status Page this year, 70.9% had not heard of it and 15.8% of staff were unsure (n=196). There is now a direct link to the status page from the iLearn page https://ilearn.mq.edu.au/login/index.php, thus these numbers should increase.

34

https://ilearn.mq.edu.au/login/index.php

APPENDICESAPPENDIX 1: ILEARN STAFF EXPERIENCE SURVEY QUESTIONSThe surveys consisted of a combination of multiple-choice, scaled and open-ended questions that focused on:

Demographics relating to the learning and teaching context. Technologies used to access iLearn. Experiences of learning and teaching using the iLearn functions, which focused on:

1. Organising teaching; or2. Engaging with learning activities.

Most valuable features and suggested improvements. Overall satisfaction with iLearn, iTeach and associated support services.

In 2014, additional questions were included in the iLearn Staff Experience Survey regarding the Online Unit Guide Facility.It is important to note that the iLearn Staff Experience Survey used a five point Likert scale (strongly agree, agree, neutral/mixed feelings, disagree and strongly disagree), which has been reduced to Agreement (strongly agree + agree), Neutral/mixed feelings and Disagreement (strongly disagree + disagree) in all the tables and graphs in this report. This same approach is used for the satisfaction questions, again the five point Likert scale (very satisfied, satisfied, neutral/mixed feelings, dissatisfied and very dissatisfied) has been reduced to Satisfied (very satisfied + satisfied), Neutral/mixed feelings and Dissatisfied (very dissatisfied + dissatisfied) this decision was made for readability purposes. In each of the tables and graphs the number of staff responses vary thus the number of responses (n=) received are indicated in each, in order to provide further clarity to the findings. Where possible, the iLearn Staff Experience Survey results are presented with the corresponding results from the 2012 and 2013 surveys. This was not always possible as each year the survey questions have slightly changed to correspond with the modifications within the system and the transition from the iLearn implementation to the operational phase.

35

APPENDIX 2: SURVEY ADVERTISEMENTS Various communication strategies were used to raise awareness of the iLearn surveys; for example, survey advertisements on screens across campus including the central courtyard, Campus Hub, Macquarie University Sport & Aquatic Centre, E4B, W6B and the Library. A link to the iLearn improvements document and the advertisements with further information regarding the surveys were published in LTC Faculty Reports, the LTC Blog Teche and the iLearn homepage (once the user had logged into the system). Individual emails were sent to all Departmental Administrators asking that they notify teaching staff about the surveys.

ADVERTISEMENTS FOR THE ILEARN HOMEPAGE

ADVERTISEMENT FOR CAMPUS SCREENS, POSTERS AND ATTACHMENTS TO EMAIL NOTIFICATIONS

40

http://teche.ltc.mq.edu.au/

APPENDIX 3: IMPROVEMENTS TO THE ILEARN PLATFORM

Since the iLearn implementation in 2012 the iLearn team has made continual improvements to the system. The improvements made in 2014 include:

the new iLearn theme; the introduction of a mobile friendly iLearn site; the iTeach-UNITS integration; the launch of iShare; the introduction of iLearn status page (http://status.ilearn.mq.edu.au/); the inclusion of a new question type in the IPA transcription; updates to Unit readings block and Category block; updates to Echo360 to enhance stability and maintain security; the streaming and screen capture capabilities for Echo360 were extended to an additional

20 venues across campus, and the new iTeach integration improved convenor access methods, resulting in fewer requests to grant academics access to their units within Echo360.

41

http://status.ilearn.mq.edu.au/

APPENDIX 4: STATISTICAL ANALYSIS OF TECHNOLOGIES STAFF USED TO ACCESS ILEARN

STATISTICAL ANALYSIS BY FACULTYNote: Faculty N = Maximum N: actual Faculty total for each row varies slightly due to missing values or NA responses. Furthermore, two respondents did not nominate their Faculty.Differences by Faculty significant at *α = 0.05, ** α = 0.01

Frequency of using iLearn with…

ArtsN = 48

FBEN = 56

FHSN = 44

ScienceN = 62

ALLN = 210

Uni-provided computer at MQ col % col % col % col % col %

Never or rarely 6 12 18 14 13

Moderate frequency 10 20 9 6 11

High frequency 83 68 73 79 76

Desktop or laptop computer at other location/s**Never or rarely 2 2 2 10 4



Own laptop at MQ using OneNet




Smartphone at MQ using OneNet




Smartphone using another network provider




Tablet at MQ using OneNet




Tablet using another network provider




Other device/network combination




42

STATISTICAL ANALYSIS BY STAFF APPOINTMENTComparison between views of continuing and fixed-term staff (CFT) and casual/sessional or adjunct staff (CAS).Note: Unless otherwise stated in tables, N = Maximum N; actual total varies due to missing values or NA responses. In the following table, percentages may not sum to 100 due to rounding error.Differences by Staff Appointment significant at *α = 0.05, ** α = 0.01

2.1 Using iLearn with…

Employment Status Employment Status

CFTN = 116

CASN = 83

ALLN = 210

CFTN = 116

CASN = 83

ALLN = 210

Uni-provided computer at MQ **

col % col % col %

Desktop or laptop computer at other location/s

col % col % col %

Never or rarely 4 27 13 Never or rarely 3 5 4

Moderate frequency 5 21 11 Moderate

frequency 9 7 8

High frequency 91 53 76 High frequency 89 88 88

Own laptop at MQ using OneNet * Smartphone at MQ using OneNet



frequency 20 26 23


Smartphone using another network provider Tablet at MQ using OneNet



frequency 25 18 23


Tablet using another network provider Other device/network combination


A few times a week 19 21 20 A few times a

week 1 1 1

At least once a day 21 16 18 At least once a

day 1 4 2

43

STATISTICAL ANALYSIS BY EXPERIENCEComparison between views of staff for whom this session was the first experience of iLearn at Macquarie (First time) and those who had previously used iLearn at Macquarie (Prev Exp).Note: Unless otherwise stated in tables, N = Maximum N; actual total varies due to missing values or NA responses. In the following table, percentages may not sum to 100 due to rounding error.Differences by Experience significant at *α = 0.05, ** α = 0.01

Using iLearn with…

Experience using iLearn Experience using iLearn

First time N

= 25

Prev ExpN = 183 ALL

N = 208

First time N

= 25

Prev ExpN = 183 ALL

N = 208

Uni-provided computer at MQ

col % col % col %

Desktop or laptop computer at other location/s

col % col % col %


Moderate frequency

1611 11 Moderate

frequency12

8 8


Own laptop at MQ using OneNet Smartphone at MQ using OneNet


Moderate frequency

1226 24 Moderate

frequency16

24 23


Smartphone using another network provider Tablet at MQ using OneNet


Moderate frequency

2022 21 Moderate

frequency24

23 23


Tablet using another network provider Other device/network combination


A few times a week

2819 20 A few times a

week0

1 1

At least once a day

421 18 At least once a

day0

3 2

44

APPENDIX 5: STATISTICAL ANALYSIS OF STAFF SKILLS, CONFIDENCE LEVELS AND SATISFACTION

STATISTICAL ANALYSIS BY FACULTYDifferences by Faculty significant at *α = 0.05, ** α = 0.01

Overall teaching experience with iLearn

ArtsN = 48

FBEN = 56

FHSN = 44

ScienceN = 62

ALLN = 210

Mean SDev Mean SDev Mean SDev Mean SDev Mean SDev

I have developed the necessary skills toeffectively use iLearn

3.9 0.9 3.9 0.9 3.9 1.0 3.8 0.9 3.9 0.9

I feel confident when using iLearn in my teaching.

4.2 0.8 4.1 0.7 4.1 0.9 3.9 0.9 4.1 0.8

I feel confident when using Echo360 in my teaching.

3.7 1.1 3.7 1.2 3.9 1.1 3.6 0.9 3.7 1.1


3.6 1.1 3.5 1.2 4.0 1.2 3.8 1.2 3.7 1.2

As a result of using iLearn I have reconsidered the way I teach my unit

3.5 1.4 3.5 1.1 3.2 1.3 3.0 1.3 3.3 1.3


4.0 1.1 4.3 0.9 4.0 1.3 4.0 1.1 4.1 1.1


3.8 1.2 3.9 1.0 4.0 1.0 3.8 1.0 3.8 1.0

I am satisfied with iLearn as a support for organising my teaching.

3.6 1.2 3.9 0.9 3.8 1.0 3.6 1.0 3.7 1.0

I am satisfied with the way iLearn supports interaction and communication with my students.

3.3 1.3 3.7 0.9 3.5 1.0 3.7 1.1 3.6 1.1

I am satisfied with the way iLearn supports

3.2 1.2 3.7 1.0 3.5 0.9 3.7 0.9 3.5 1.0

45

my students' engagement with learning activities.


3.6 1.1 3.7 1.2 3.6 1.1 3.6 1.2 3.6 1.1

I feel confident when using iTeach to map learning outcomes, assessment tasks and Graduate Capabilities

3.3 1.3 3.7 1.2 3.4 1.3 3.6 1.2 3.5 1.2

STATISTICAL ANALYSIS BY STAFF APPOINTMENTComparison between views of continuing and fixed-term staff (CFT) and casual/sessional or adjunct staff (CAS).Differences by Faculty significant at *α = 0.05, ** α = 0.01


CFTN = 116

CASN = 83

ALLN = 210

N Mean SDev N Mean SDev N Mean SDev

I have developed the necessary skills to effectively use iLearn

114 3.9 0.9 79 3.9 1.0204 3.9 0.9


115 4.0 0.8 80 4.1 0.9206 4.1 0.8


105 3.7 1.0 46 3.7 1.2158 3.7 1.1


106 3.8 1.2 31 3.6 1.2144 3.7 1.2

As a result of using iLearn I have reconsidered the way I teach my unit *

113 3.1 1.3 59 3.5 1.2181 3.3 1.3


114 3.9 1.2 71 4.3 1.0195 4.1 1.1


113 3.8 1.1 72 3.9 1.0196 3.8 1.0


110 3.6 1.1 69 3.9 0.9190 3.7 1.0

I am satisfied with the way iLearn supports interaction and communication with my students.*

113 3.4 1.2 74 3.8 0.9197 3.6 1.1

46

I am satisfied with the way iLearn supports my students' engagement with learning activities. **

111 3.4 1.1 73 3.8 0.9194 3.5 1.0

I feel confident when using iTeach to create my Unit Guide.

110 3.7 1.2 39 3.6 1.1155 3.6 1.1


108 3.6 1.3 38 3.5 1.1152 3.5 1.2



First timeN = 25

Prev ExpN = 183

ALLN = 210

iLearn tool N Mean SDev N Mean SDev N Mean SDev

I have developed the necessary skills to effectively use iLearn

24 3.5 0.9 178 3.9 0.9204 3.9 0.9


25 3.8 0.9 179 4.1 0.8206 4.1 0.8


12 3.5 1.2 144 3.8 1.1158 3.7 1.1

I feel confident when using iTeach to manage enrolments. **

7 2.3 1.1 136 3.8 1.1144 3.7 1.2

As a result of using iLearn I have reconsidered the way I teach my unit *

20 2.7 1.2 159 3.3 1.3181 3.3 1.3


23 3.7 1.4 170 4.1 1.1195 4.1 1.1


22 3.5 1.1 172 3.9 1.0196 3.8 1.0


21 3.7 1.0 167 3.7 1.0190 3.7 1.0

47

I am satisfied with the way iLearn supports interaction and communication with my students.

23 3.3 1.0 172 3.6 1.1197 3.6 1.1

I am satisfied with the way iLearn supports my students' engagement with learning activities.

24 3.2 1.1 168 3.6 1.0194 3.5 1.0


12 3.5 1.2 142 3.7 1.1155 3.6 1.1


11 3.3 0.8 140 3.6 1.3152 3.5 1.2

48

APPENDIX 6: STATISTICAL ANALYSIS OF STAFF EXPERIENCE WITH ILEARN

STATISTICAL ANALYSIS BY FACULTYNote: Unless otherwise stated in tables, N = Maximum N; actual total varies due to missing values or NA responses. In the following table, percentages may not sum to 100 due to rounding error.Differences by Experience significant at *α = 0.05, ** α = 0.01

Organising students' learning.

Arts FBE FHS Science ALL

iLearn tool N Mean SDev

N Mean

SDev

N Mean SDev

N Mean SDev

N Mean SDev

Announcements 47 4.2 1.1 50 4.4 0.7 40 4.3 1.0 54 4.3 0.9 191 4.3 0.9

Calendar * 31 3.0 1.2 27 3.8 1.1 22 2.9 1.1 35 3.3 1.1 115 3.3 1.2

Labels 37 3.8 1.0 32 4.0 1.0 32 4.0 1.1 36 4.0 1.0 137 3.9 1.0

Online Assignment Submission

48 4.4 0.8 44 4.5 0.6 38 4.2 1.0 52 4.2 1.0 182 4.3 0.8

Turnitin * 46 4.5 0.7 43 4.4 0.7 39 4.0 1.0 40 4.0 1.0 168 4.2 0.9

Grademark 44 4.0 1.1 45 4.0 1.1 35 3.6 1.3 39 3.7 1.2 163 3.9 1.2

Grades 42 3.9 1.0 51 4.1 1.1 39 3.7 1.2 45 3.7 1.2 177 3.9 1.1

Activity logs ** 33 3.3 1.2 33 3.8 1.1 26 4.1 0.9 33 3.2 0.9 125 3.6 1.1

Reports 28 3.1 1.2 33 3.7 1.1 22 3.9 0.9 28 3.2 0.9 111 3.5 1.1

Engaging students in learning activities

Arts FBE FHS Science ALL

iLearn tool N Mean SDev

N Mean Sdev

N Mean Sdev

N Mean Sdev

N Mean Sdev

Dialogue 35 3.3 1.2 37 3.5 1.2 33 3.6 1.3 34 3.8 1.2 139 3.5 1.2

Discussion Forums 46 3.7 1.2 46 3.9 1.1 41 4.0 1.1 50 3.9 1.1 183 3.9 1.1

Blog 22 2.6 1.2 19 3.1 1.3 15 3.1 1.2 17 3.1 1.4 73 3.0 1.3

Chat 18 2.3 1.2 16 3.1 1.3 15 3.0 1.3 17 2.7 1.4 66 2.7 1.3

Quizzes 32 3.5 1.2 28 4.1 1.1 24 3.8 1.1 34 3.9 1.1 118 3.8 1.2

Wiki * 15 2.1 1.0 19 2.7 1.2 11 2.9 1.3 14 3.4 1.1 59 2.8 1.2

Database 16 2.4 1.2 16 3.4 1.2 11 3.0 1.2 13 3.4 1.3 56 3.0 1.3

Workshops 18 2.7 1.1 17 3.1 1.2 8 3.3 0.9 12 3.4 1.2 55 3.1 1.2

Twitter feeds 16 2.1 1.1 13 2.8 1.1 8 2.5 1.2 9 3.3 1.5 46 2.6 1.3

Groupings and Groups **

29 2.9 1.4 30 3.8 1.2 20 3.7 1.2 24 3.9 1.1 103 3.5 1.3

49

STATISTICAL ANALYSIS BY STAFF APPOINTMENTNote: Unless otherwise stated in tables, N = Maximum N; actual total varies due to missing values or NA responses. In the following table, percentages may not sum to 100 due to rounding error.Differences by Experience significant at *α = 0.05, ** α = 0.01

The iLearn platform was effective in meeting my needs…

with respect to organising students' learning

CFTN = 116

CASN = 83

ALLN = 210


Announcements 114 4.3 1.0 67 4.3 0.9 191 4.3 0.9

Calendar 64 3.2 1.1 44 3.5 1.2 115 3.3 1.2

Labels 90 3.9 1.1 40 4.0 0.9 137 3.9 1.0

Online Assignment Submission 107 4.3 0.9 68 4.4 0.8 182 4.3 0.8

Turnitin 99 4.2 0.8 64 4.2 1.0 168 4.2 0.9

Grademark * 94 3.7 1.3 63 4.1 1.0 163 3.9 1.2

Grades 100 3.8 1.2 70 4.0 1.0 177 3.9 1.1

Activity logs 75 3.4 1.1 42 3.8 1.1 125 3.6 1.1

Reports 70 3.4 1.1 35 3.7 1.0 111 3.5 1.1

for engaging my students in learning activities.

CFTN = 116

CASN = 83

ALLN = 210


Dialogue 80 3.5 1.3 51 3.6 1.3 139 3.5 1.2

Discussion Forums 104 3.8 1.2 70 3.9 1.0 183 3.9 1.1

Blog 43 2.8 1.2 25 3.2 1.3 73 3.0 1.3

Chat 36 2.5 1.3 25 3.0 1.3 66 2.7 1.3

Quizzes 75 3.8 1.2 36 3.9 1.1 118 3.8 1.2

Wiki 35 2.6 1.2 21 2.9 1.3 59 2.8 1.2

Database 30 2.7 1.1 23 3.4 1.4 56 3.0 1.3

Workshops * 28 2.7 1.1 23 3.4 1.2 55 3.1 1.2

Twitter feeds 26 2.4 1.2 18 2.8 1.3 46 2.6 1.3

Groupings and Groups 65 3.4 1.4 32 3.8 1.1 103 3.5 1.3

50


The iLearn platform was effective in meeting my needs…

with respect to organising students' learning

First timeN = 25

Prev ExpN = 183

ALLN = 210


Announcements 21 4.1 1.0 168 4.3 0.9 191 4.3 0.9

Calendar 14 3.3 1.0 100 3.3 1.2 115 3.3 1.2

Labels 11 3.5 1.1 125 4.0 1.0 137 3.9 1.0

Online Assignment Submission 21 4.2 0.9 159 4.3 0.8 182 4.3 0.8

Turnitin 21 4.1 1.0 146 4.2 0.9 168 4.2 0.9

Grademark 20 3.9 1.2 142 3.9 1.2 163 3.9 1.2

Grades 17 3.9 1.2 158 3.9 1.1 177 3.9 1.1

Activity logs 14 3.5 1.0 109 3.6 1.1 125 3.6 1.1

Reports 8 3.3 1.2 101 3.5 1.1 111 3.5 1.1

for engaging my students in learning activities.

First timeN = 25

Prev ExpN = 183

ALLN = 210


Dialogue 14 3.3 1.6 125 3.6 1.2 139 3.5 1.2

Discussion Forums 17 3.4 1.3 164 3.9 1.1 183 3.9 1.1

Blog * 6 1.8 1.0 67 3.1 1.3 73 3.0 1.3

Chat 7 2.3 1.5 59 2.8 1.3 66 2.7 1.3

Quizzes * 11 3.1 1.4 106 3.9 1.1 118 3.8 1.2

Wiki 4 2.3 1.9 54 2.8 1.2 59 2.8 1.2

Database 7 3.0 1.7 49 3.0 1.2 56 3.0 1.3

Workshops 7 3.0 1.6 48 3.1 1.1 55 3.1 1.2

Twitter feeds 5 2.0 1.7 41 2.7 1.2 46 2.6 1.3

Groupings and Groups 10 3.3 1.5 93 3.6 1.3 103 3.5 1.3

51

APPENDIX 7: STATISTICAL ANALYSIS OF STAFF SATISFACTION WITH SERVICES AND SUPPORT

STATISTICAL ANALYSIS BY FACULTYDifferences by Faculty significant at *α = 0.05, ** α = 0.01

Satisfaction with support and services

ArtsN = 48

FBEN = 56

FHSN = 44

ScienceN = 62

ALLN = 210

Mean SDev

Mean SDev Mean SDev Mean SDev Mean SDev


4.0 1.0 4.1 1.0 4.4 1.0 4.3 0.7 4.2 0.9

OneHelp for resolution of issues relating to iTeach functions.

3.5 1.2 3.6 1.1 3.9 1.1 3.9 0.9 3.7 1.1


3.4 1.2 3.7 1.2 3.8 1.2 3.9 1.0 3.7 1.2

UNITS/Unit Guides for providing essential information to students

3.1 1.4 3.7 1.3 3.1 1.3 3.4 1.2 3.3 1.3

Training and support for using iTeach to create Unit Guides

3.6 1.0 3.5 1.2 3.7 1.1 3.6 1.1 3.6 1.1

Ease of locating online “self-help” resources to assist me in using iLearn.

3.3 1.0 3.4 1.0 3.4 1.2 3.2 1.1 3.3 1.1

The effectiveness of the online “self-help” resources in assisting my use of iLearn

3.1 1.1 3.3 1.0 3.3 1.2 3.3 1.1 3.2 1.1

The effectiveness of iLearn training workshops

3.7 1.0 3.8 1.1 4.1 0.9 3.7 1.2 3.8 1.0

The support for using iLearn provided by the iLearn drop-in clinic

4.1 1.0 3.9 1.0 4.2 1.1 4.2 1.1 4.1 1.0

The level of Faculty-based support for using iLearn

3.9 1.1 3.9 1.2 3.8 1.4 3.3 1.1 3.7 1.2

52

STATISTICAL ANALYSIS BY STAFF APPOINTMENTComparison between views of continuing and fixed-term staff (CFT) and casual/sessional or adjunct staff (CAS).Differences by Faculty significant at *α = 0.05, ** α = 0.01


CFTN = 116

CASN = 83

ALLN = 210



105 4.1 1.0 34 4.3 0.8 147 4.2 0.9


89 3.7 1.1 27 3.8 1.1 123 3.7 1.1


104 3.6 1.2 51 3.8 1.1 163 3.7 1.2

UNITS/Unit Guides for providing essential information to students. **

109 3.2 1.4 48 3.8 1.0 166 3.3 1.3


87 3.5 1.0 37 3.7 1.1 130 3.6 1.1


106 3.3 1.1 61 3.4 1.1 177 3.3 1.1


104 3.1 1.1 57 3.4 1.1 171 3.2 1.1


88 3.7 1.1 46 4.0 0.9 141 3.8 1.0


84 4.0 1.1 33 4.2 0.7 124 4.1 1.0


96 3.6 1.2 51 3.9 1.0 157 3.7 1.2

53



First timeN = 25

Prev ExpN = 183

ALLN = 208



9 3.8 0.7 136 4.2 1.0 147 4.2 0.9


10 3.8 1.0 113 3.7 1.1 123 3.7 1.1


16 3.6 1.2 145 3.7 1.2 163 3.7 1.2

UNITS/Unit Guides for providing essential information to students

13 3.7 1.3 151 3.3 1.3 166 3.3 1.3


9 3.4 1.1 121 3.6 1.1 130 3.6 1.1


18 3.2 1.1 157 3.3 1.1 177 3.3 1.1


18 3.1 1.1 151 3.2 1.1 171 3.2 1.1


15 3.9 1.2 126 3.8 1.0 141 3.8 1.0


7 3.4 1.1 117 4.1 1.0 124 4.1 1.0


15 3.5 1.3 142 3.7 1.2 157 3.7 1.2

54

Frequency of using iLearn with - Macquarie University Web viewThis is because they are considered to...

Documents

Transcript of Frequency of using iLearn with - Macquarie University Web viewThis is because they are considered to...