Assessment and Credit in Online Open Qualifications
-
Upload
john-gordon -
Category
Education
-
view
329 -
download
0
description
Transcript of Assessment and Credit in Online Open Qualifications
![Page 1: Assessment and Credit in Online Open Qualifications](https://reader034.fdocuments.in/reader034/viewer/2022051513/546a51b2af79597f788b4fc4/html5/thumbnails/1.jpg)
Assessment and Credit in Online Open Vocational Qualifications
John GordonOpus Learning
digital by default, digital by design30th January 2014
![Page 2: Assessment and Credit in Online Open Qualifications](https://reader034.fdocuments.in/reader034/viewer/2022051513/546a51b2af79597f788b4fc4/html5/thumbnails/2.jpg)
Opus Learning Ltd, Edinburgh
• We develop online open courses to SQA HND standards
– A white label service for students enrolled with other providers.
– Direct to students who are enrolled with Opus
• We provide the learning environment, and deliver the learning content and assessments.
![Page 3: Assessment and Credit in Online Open Qualifications](https://reader034.fdocuments.in/reader034/viewer/2022051513/546a51b2af79597f788b4fc4/html5/thumbnails/3.jpg)
In this morning’s talk, I hope to indicate how we have:– Benefited by using technology to support and
manage vocational learning;– Considered how to de-couple learning and
assessment– Managed downwards the risks and costs of using
technology.
![Page 4: Assessment and Credit in Online Open Qualifications](https://reader034.fdocuments.in/reader034/viewer/2022051513/546a51b2af79597f788b4fc4/html5/thumbnails/4.jpg)
Origin of Opus
• A spinout from CAPDM.com
• Developing and delivering Masters programmes for over 17 years
• www.capdm.com
• We deliver highly interactive courses, delivered via an online platform such as Moodle.
• With tools to manage and monitor learning with or without tutor support.
![Page 5: Assessment and Credit in Online Open Qualifications](https://reader034.fdocuments.in/reader034/viewer/2022051513/546a51b2af79597f788b4fc4/html5/thumbnails/5.jpg)
Structure of our programmes
• We build:– integrated frameworks of Units for SQA HNDs and
University undergrad programmes– within a framework of commercial discipline.
• Which are outward facing and globally oriented
![Page 6: Assessment and Credit in Online Open Qualifications](https://reader034.fdocuments.in/reader034/viewer/2022051513/546a51b2af79597f788b4fc4/html5/thumbnails/6.jpg)
The Main Issues of 2014• Learning and Assessment• MOOCs and SPOCs• Financing and selling• Badges and Certificates• Initial training and Life Long Learning• RPL and de-coupled accredited assessment
![Page 7: Assessment and Credit in Online Open Qualifications](https://reader034.fdocuments.in/reader034/viewer/2022051513/546a51b2af79597f788b4fc4/html5/thumbnails/7.jpg)
The problems in an online context
Online Learning does not change the problems of assessing learning
– Tracking learning– Authentication– Identifying problems in learning– Recognising need for intervention
MOOCs introduce new aspects– Learn without credit– Gain credit after learning– Building badges into credits– Peer support– Security issues
![Page 8: Assessment and Credit in Online Open Qualifications](https://reader034.fdocuments.in/reader034/viewer/2022051513/546a51b2af79597f788b4fc4/html5/thumbnails/8.jpg)
Interaction types
Student
Environment
Content Tutor
Student/Student
Content/ContentTutor/Tutor
Environment/Environment
Student/Tutor
Tutor/Environment
Student/Environment
Student/Content
Content/Environment
Tutor/Content
Based on diagram inAnderson 2003
![Page 9: Assessment and Credit in Online Open Qualifications](https://reader034.fdocuments.in/reader034/viewer/2022051513/546a51b2af79597f788b4fc4/html5/thumbnails/9.jpg)
Where do the major costs lie?
Student
Environment
ContentTutor
Community
Student/Student
Content/Content
Environment/Environment
Support and Assessment
Student and Institution Management
Student/Environment
Student/Content
Content/Environment
Review and Improvement
Based on diagram inAnderson 2003
Academic Activity
![Page 10: Assessment and Credit in Online Open Qualifications](https://reader034.fdocuments.in/reader034/viewer/2022051513/546a51b2af79597f788b4fc4/html5/thumbnails/10.jpg)
Interactions and MonitoringType of Interaction Tracking FunctionHuman to human Within Forums
Via a ticket system for formal student to tutor interactions
Student to Learning Platform Via built in tracking tools on Platform
Student to contentStudent and formative assessmentsStudents and personal reflections
Our in-built tracking and monitoring tool
![Page 11: Assessment and Credit in Online Open Qualifications](https://reader034.fdocuments.in/reader034/viewer/2022051513/546a51b2af79597f788b4fc4/html5/thumbnails/11.jpg)
Interacting with Content Objects
Learner interacting with content, leaving traces
Student interacts with content
Makes notes, responds to quiz, send messages, etc
Capture notesStore in MLE
Print off record of learning The Digital Workbook
![Page 12: Assessment and Credit in Online Open Qualifications](https://reader034.fdocuments.in/reader034/viewer/2022051513/546a51b2af79597f788b4fc4/html5/thumbnails/12.jpg)
The Digital Workbook is• A portfolio embedded in
the content• A dashboard for learning • Rich seam of data and
behaviours for mining• Ongoing formative
assessment, and potential summative assessment
• A pdf for the student
![Page 13: Assessment and Credit in Online Open Qualifications](https://reader034.fdocuments.in/reader034/viewer/2022051513/546a51b2af79597f788b4fc4/html5/thumbnails/13.jpg)
MOOC Value Proposition• Education Access
– On a massive, international scale. Currently, most students who are not enrolled college students.
– May change as colleges integrate MOOC content to traditional teaching• Experimentation
– New business model– New resources for learners– New learning technologies based on mining
• Brand Extension – MOOCs can extend the institution’s reach and reputation internationally. – Particularly among elite research institutions, MOOCs have become a way of
enhancing the institution’s brand and signalling innovation. – Successful professors can gain a global following, building their own reputation
http://net.educause.edu/ir/library/pdf/PUB4005.pdf
![Page 14: Assessment and Credit in Online Open Qualifications](https://reader034.fdocuments.in/reader034/viewer/2022051513/546a51b2af79597f788b4fc4/html5/thumbnails/14.jpg)
How does this affect Learning and Assessment?
![Page 15: Assessment and Credit in Online Open Qualifications](https://reader034.fdocuments.in/reader034/viewer/2022051513/546a51b2af79597f788b4fc4/html5/thumbnails/15.jpg)
Assessment In theSQA environment
All units are stand alone, they can be delivered in a very flexible manner, but there are very clear pointers to how assessment should be carried out.
It is no longer appropriate to base the assessment of competence on a final 3 hour written examination.
![Page 16: Assessment and Credit in Online Open Qualifications](https://reader034.fdocuments.in/reader034/viewer/2022051513/546a51b2af79597f788b4fc4/html5/thumbnails/16.jpg)
Assessment and Verification what happens
Candidate Requests
Assessment Instrument from VLE
Assessment sent to
Candidate – time limited
Candidate responds to Assessment
Assessor reviews
Assessment Attempt Makes
comments
Assessment pdf
Candidate Attempt
Satisfactoryyes
No
Assessor sends comments to
candidate
Assessor comments
Candidate Resubmits
Resubmit?
yes
Candidate attempt
NoSign assessment record detailing re-assessment
Sign assessment record detailing
Satisfactory
Candidate Assessment
Record
Send assessment record to
candidate – complete record in VLE/SQA Connect
Verifier consultation
Verifier consultation
![Page 17: Assessment and Credit in Online Open Qualifications](https://reader034.fdocuments.in/reader034/viewer/2022051513/546a51b2af79597f788b4fc4/html5/thumbnails/17.jpg)
Open Distance Learning
![Page 18: Assessment and Credit in Online Open Qualifications](https://reader034.fdocuments.in/reader034/viewer/2022051513/546a51b2af79597f788b4fc4/html5/thumbnails/18.jpg)
The Tensions in Technology Based Open and Distance Learning
• Who is actually at the end of the screen?• What is the level of Granularity for assessment?
– We want freedom for student to learn what they want when they want
– We want to integrate assessment across a group of units
• What is the cohort size?– We want individualised learning– We want efficiency in the use of tutor support
![Page 19: Assessment and Credit in Online Open Qualifications](https://reader034.fdocuments.in/reader034/viewer/2022051513/546a51b2af79597f788b4fc4/html5/thumbnails/19.jpg)
Who is at the end of the screen?
Opus Policy is:
Digital Authentication
+Physical Authentication
![Page 20: Assessment and Credit in Online Open Qualifications](https://reader034.fdocuments.in/reader034/viewer/2022051513/546a51b2af79597f788b4fc4/html5/thumbnails/20.jpg)
Managing Learning with Technology
• At Opus we track student behaviour. • We use 2 levels of tracking
– Moodle Based (the VLE) and – Content Based.
![Page 21: Assessment and Credit in Online Open Qualifications](https://reader034.fdocuments.in/reader034/viewer/2022051513/546a51b2af79597f788b4fc4/html5/thumbnails/21.jpg)
Tracking through the learning content
• Opus content is massively connected to the students workbook
• Every reflection, exercise, quiz, short response and so on are logged within the learning content and in the workbook
• The tutor can view, track and compare workbooks
![Page 22: Assessment and Credit in Online Open Qualifications](https://reader034.fdocuments.in/reader034/viewer/2022051513/546a51b2af79597f788b4fc4/html5/thumbnails/22.jpg)
DWB Examples – Source Page
![Page 23: Assessment and Credit in Online Open Qualifications](https://reader034.fdocuments.in/reader034/viewer/2022051513/546a51b2af79597f788b4fc4/html5/thumbnails/23.jpg)
![Page 24: Assessment and Credit in Online Open Qualifications](https://reader034.fdocuments.in/reader034/viewer/2022051513/546a51b2af79597f788b4fc4/html5/thumbnails/24.jpg)
DWB Dashboard –Managing groups of students
![Page 25: Assessment and Credit in Online Open Qualifications](https://reader034.fdocuments.in/reader034/viewer/2022051513/546a51b2af79597f788b4fc4/html5/thumbnails/25.jpg)
Some conclusions from tracking
• Tracking appears to:– Provide the potential of personalised learning
within mechanical presentation of study– Assist formative assessment in the content driving
tracking and the experience of learning progress– Provide feedback on Tutor effectiveness and the
effectiveness of the pedagogy underpinning the content
![Page 26: Assessment and Credit in Online Open Qualifications](https://reader034.fdocuments.in/reader034/viewer/2022051513/546a51b2af79597f788b4fc4/html5/thumbnails/26.jpg)
The Commercial Imperative
We wish to – drive down cost of assessment– Drive up the quality and efficiency of assessment
within the context of a well resourced learner.
We have to manage costs, and the use of human resource
![Page 27: Assessment and Credit in Online Open Qualifications](https://reader034.fdocuments.in/reader034/viewer/2022051513/546a51b2af79597f788b4fc4/html5/thumbnails/27.jpg)
Assessor/Tutor Activity Model in Distance Learning
1. Monitor VLE activity, identify any candidate with lack of progress and contact course leader for action.
2. Monitor Digital Workbook (if used) and identify any outliers in progress, contact course leader with details.
3. Respond to direct tickets from candidate, making use of FAQs for responses, if new response is required, copy response to course leader.
4. Responding to assessments; remediation advice
![Page 28: Assessment and Credit in Online Open Qualifications](https://reader034.fdocuments.in/reader034/viewer/2022051513/546a51b2af79597f788b4fc4/html5/thumbnails/28.jpg)
Assessor/Tutor Activity Model in MOOCs
1. Monitor VLE activity automatically. 2. Monitor Digital Workbook but ignore any outliers in
progress, contact course leader with details. 3. Send any queries to student peer group for support4. Responding to machine assessed assessments; only
automatic remediation advice
Take out the tutor!
![Page 29: Assessment and Credit in Online Open Qualifications](https://reader034.fdocuments.in/reader034/viewer/2022051513/546a51b2af79597f788b4fc4/html5/thumbnails/29.jpg)
Managing Resource
• We can time and cost elements in traditional online courses• Times and costs vary with cohort size in traditional courses• For the MOOC approach, times and costs must be
independent of cohort size.
![Page 30: Assessment and Credit in Online Open Qualifications](https://reader034.fdocuments.in/reader034/viewer/2022051513/546a51b2af79597f788b4fc4/html5/thumbnails/30.jpg)
Timing Model vs Cohort size
HND Business Timings Assumptions
Students per assessor
1 5 10 15 20
Average timing per unit for HND Business - minutes
93 85 77 70 64
![Page 31: Assessment and Credit in Online Open Qualifications](https://reader034.fdocuments.in/reader034/viewer/2022051513/546a51b2af79597f788b4fc4/html5/thumbnails/31.jpg)
Learning Centre Cost ModelSPOC
• Based on the previous models– Time per tutor/unit/student can be estimated– Payment to Tutor/assessor is known– Viability of the Distance Course can be calculated.
• We need to know these numbers and to watch them
![Page 32: Assessment and Credit in Online Open Qualifications](https://reader034.fdocuments.in/reader034/viewer/2022051513/546a51b2af79597f788b4fc4/html5/thumbnails/32.jpg)
Potential Cost Model
Possible costs/charges per studentawarding body charges = £250;tutoring = £1000; tech + online content licence = £300; agent fees = £250Centre overheads per student = £500
Total cost per student = £2300
![Page 33: Assessment and Credit in Online Open Qualifications](https://reader034.fdocuments.in/reader034/viewer/2022051513/546a51b2af79597f788b4fc4/html5/thumbnails/33.jpg)
Potential Centre Surplus
• In UK Charge to student £3500-£4500• In India Charge to student £1500-£2500
We cannot serve one of largest markets in the world, with this model.
We have to be able to manage costs.
![Page 34: Assessment and Credit in Online Open Qualifications](https://reader034.fdocuments.in/reader034/viewer/2022051513/546a51b2af79597f788b4fc4/html5/thumbnails/34.jpg)
MOOC Costs models
• Charge to student – Zero• Cost for development of module not well
reported– EdX charge $250k to produce a MOOC**– University of North Carolina quoted $150k for in-
house production*• Revenue shares between platform provider
and university
![Page 35: Assessment and Credit in Online Open Qualifications](https://reader034.fdocuments.in/reader034/viewer/2022051513/546a51b2af79597f788b4fc4/html5/thumbnails/35.jpg)
Learning from MOOCS
• Massive size of cohort• Highly programmed week by week• Clearly signposted weekly targets• Using the student body as the main tutor
toolset• Building a community of activists and lurkers• Managing and controlling costs• Generate revenue by conversions?
![Page 36: Assessment and Credit in Online Open Qualifications](https://reader034.fdocuments.in/reader034/viewer/2022051513/546a51b2af79597f788b4fc4/html5/thumbnails/36.jpg)
Learning from MOOCS
• Do not do assessments – Until students join full programme
• Keep the granularity of the course fairly small – not full degrees
• Set up an honour system – make the students police themselves and even support each others’ learning.
![Page 37: Assessment and Credit in Online Open Qualifications](https://reader034.fdocuments.in/reader034/viewer/2022051513/546a51b2af79597f788b4fc4/html5/thumbnails/37.jpg)
Applying all these lessons
• Manage the cohort size• Make sure investment reflects assessment and
tutor support provided – the added value of a non-free course
• Make assessments social, encourage group help. Track this activity!
![Page 38: Assessment and Credit in Online Open Qualifications](https://reader034.fdocuments.in/reader034/viewer/2022051513/546a51b2af79597f788b4fc4/html5/thumbnails/38.jpg)
Opus StrategySmall Private Open Courses
• Organise cohorts in groups of 10s, not groups of 10k’s or 100k’s
• Localise tutors and assessors• Build communities including assessors• Track activity and behaviour at keyboard level and at
cognitive level• Record activity – perhaps award a badge for partially
completed activity• Guide the learners towards completion, intervene as
necessary
![Page 39: Assessment and Credit in Online Open Qualifications](https://reader034.fdocuments.in/reader034/viewer/2022051513/546a51b2af79597f788b4fc4/html5/thumbnails/39.jpg)
Opus MOOC Experiment
• From the 1st April 2014, there will at least 2 Opus Units following SQA curriculum– With no registration at SQA– No access to Assessment Support Pack items– With formative assessment – peer assessed– With summative assessment with peer commenting– Open badges for course completion/attainment
• With the potential to register post mooc, and fast track to assessment.
![Page 40: Assessment and Credit in Online Open Qualifications](https://reader034.fdocuments.in/reader034/viewer/2022051513/546a51b2af79597f788b4fc4/html5/thumbnails/40.jpg)
The Opus Open Courses
• Management: Developing Self Management Skills based on course H1F1 34
• Project Management: An Introduction based on course F1NH 34
![Page 41: Assessment and Credit in Online Open Qualifications](https://reader034.fdocuments.in/reader034/viewer/2022051513/546a51b2af79597f788b4fc4/html5/thumbnails/41.jpg)
Developing Self Management SkillsSummative Assessment
• Students are encouraged to provide evidence in a format which suits their particular situation.
• It is particularly suited to an electronic portfolio method of assessment.
![Page 42: Assessment and Credit in Online Open Qualifications](https://reader034.fdocuments.in/reader034/viewer/2022051513/546a51b2af79597f788b4fc4/html5/thumbnails/42.jpg)
Project Management Summative Assessment
• Outcomes can be assessed by the development of project management documentation, for a workplace project for which the candidate has responsibility
![Page 43: Assessment and Credit in Online Open Qualifications](https://reader034.fdocuments.in/reader034/viewer/2022051513/546a51b2af79597f788b4fc4/html5/thumbnails/43.jpg)
Our to do list
• Design the courses• Design the peer assessment elements• Design the learning pathway (based on
current open online pedagogy)• Design the Badge structure• Work with SQA to provide pathway to
certification!!!!