Evolving User Acceptance Testing for Moodle Environments

Amy Holland Canberra Institute of Technology

Amy.Holland@cit.edu.au

What is CIT’s eLearn?

• CIT upgraded from WebCT to Moodle 1.9 in June 2010. • CIT’s eLearn system consists of 3 environments, Moodle, EQUELLA

and at the time, Wimba live classroom. After Wimba was bought out, we moved to Adobe Connect. This is not integrated into the Moodle as with Wimba, but we have found that this does not hinder the teachers in one bit. As this system is not integrated and is also run by an external provider, user acceptance testing is not required by us.

CIT’s eLearn Upgrade Process

• Since this migration to Moodle, we have continually maintained and upgraded our system to keep up to date with the latest Moodle releases.

• In 2013, we took on the system maintenance and upgrades alone, no longer using our Moodle partner.

• This same year, CIT upgraded from Moodle 1.9 to Moodle 2.3, doing our utmost to keep a vanilla install of the system to keep all future upgrades simple. This meant no core changes, only Moodle plugins. This also now means fewer staff are needed to make any future changes and fix any issues that may occur.

• Out latest major upgrade was from Moodle 2.3 to Moodle 2.6 in January 2014.

http://pixabay.com/static/uploads/photo/2013/07/12/14/06/arrows-147751_640.png

UAT Documentation – Moodle 1.9

• Testing documents provided by Moodle Partner, Netspot • Completed prior to each upgrade, testing the basics of the system to

test functionality • Basic testing covered the roles testing, as we has specially created

roles for our help desk staff and teachers trainers. • Testing also covered the changes we were making to the system, and

confirmed that these were not causing any issues. • The test plans were repetitive as each task was to be completed

using each role, including our multiple helper roles, plus teachers and students.

Original Moodle installation for CIT

UAT Documentation – Moodle 2.3

• New testing documentation created in December 2012 to start testing as part of upgrade project plan

• Testing of 1.9 plugins bringing over to 2.3 • Testing in 3 environments

• Starting with eLearn Development – Stand-alone system, intranet access only. Testing was completed here while looking at and adjusting settings needing to achieve system needed for CIT purposes and policies

• Post development testing in eLearn Test, intranet only as well – Identical system to live. Testing here to confirm all associated programs, such as Student Management System and Equella worked as expected

• Post upgrade testing in eLearn Production – Live environment offline to staff until testing was completed by selected users

Moodle 1.9 – 2.3: April 2013 - Upgrade

UAT Documentation – Moodle 2.3

Moodle 1.9 – 2.3: April 2013 - Lessons Learnt

• Written test plans as used by the eLearn team do not work as well for teachers.

• Student testing plan was fine, providing the requires activities had been set up as part of the teacher testing.

• Upgrading and therefore testing over the Easter long weekend is a good time for us, not for the students.

• Better communication strategy needed for teachers and students. • Lindt bunnies were a great testing treat, brought in for us from our

boss over the long weekend!

UAT Documentation – Moodle 2.6

• New testing documentation created in October 2013 to start testing as part of upgrade project plan.

• Scenario testing created for teachers who wished to be involved in the testing process.

• New CIT theme to be created and tested. Theme creator for CIT created a mind-map to map out all pages of the theme so as not to miss anything.

• Testing out all our current plugins to ensure they were compatible with Moodle 2.6.

• Major upgrade outage time windows set and added to CIT staff calendar.

• Minor monthly updates set for last Friday of every month from 6am – 8am. Updates to be completed and tested in this time before system is made live.

Moodle 2.3 – 2.6: January 2014 - Upgrade

UAT Documentation – Moodle 2.3

Moodle 2.3 – 2.6: January 2014 - Lessons Learned

• Even though teachers and students signed up to testing, getting them to do it in a set time-frame was a struggle. In the end I think we had about 3 out of 25 come back before the Christmas break. Testing with teachers therefore needs to begin much earlier.

• Scenarios worked well for the teachers after finding out the year before that our test plans did not.

• The new communication strategy worked well, including videos of new features for teachers, sent via internal email, and a YouTube video for students, advertised on the CIT website - http://youtu.be/QEF1r4yP0-k

UAT Documentation – Moodle 2.6

• Change request process created to manage and test all changes to the system with the CIT ICT team.

• Plugin requests created for teachers to be able to request specific features of eLearn and for what purpose these are needed.

• All plugins installed and tested on local machine before being requested for eLearn Development environment

• Once installation, and successful testing completed in eLearn Development, change request goes through an approval process to be installed on eLearn Test and follow up testing completed on identical environment.

• Updates completed during last Friday morning of every month.

Continuous Updates to Moodle 2.6

UAT Documentation – Moodle 2.3

• Due to the nature of the community made plugins and therefore the hesitance of our ICT team to install them, we run a thorough testing process each time we add in a new plugin

• Each new plugin has it’s own request, test plan and once approved, guides created for it. Not only does this allow us to learn how to use each new plugin, it also allows us to then send these testing plans to those teachers who requested this and to have them see if the plugin does what they want it to do.

• Once installed successfully, the plugin test plan is then re-created into an on-going test plan to use when the individual plugin is updated. The test plan works towards a the eLearn how to guide which is also sent to all teachers informing them of our changes.

• Each plugin then has a mini test added to the full Moodle user acceptance testing documentation.

Plugin Testing

UAT Documentation – Moodle 2.3

• As we have our own theme, and all environments have their own issues, we have come across some parts of the testing plans that have been missed.

• When we come across any issues or oddities, these are noted and added to the test plan each time to check for the next update to make sure there have not been changes that have made something go awry.

• Due to the thoroughness of the User Acceptance Test plans, these issues usually only come up in the eLearn Test environment (due to it being most alike with our live system), and can be rectified, with plans updated, before the update is approved and installed in the live eLearn environment.

Issues