TLC2016 - Turning Blackboard Learn into a Digital Examination Platform: lessons learned from 8 years...
-
Upload
blackboardemea -
Category
Education
-
view
401 -
download
2
Transcript of TLC2016 - Turning Blackboard Learn into a Digital Examination Platform: lessons learned from 8 years...
Turning Blackboard Learn into a Digital Examination Platform
Lessons learned through 8 years of practice
Geert De Schutter, Educational technologistWalter Roosels, Educational technologist
Exam in progress
Digital assessment/examination: Why?
• Strategic reasonsImage building
• Efficiency reasonsSaving time (large groups)
• Financial motivesStaff – student ratio
• Educational motivesObjectivity, quality of assessment
Digital assessment/examination: University of Antwerp
• Focus on efficiency and educational benefitsSaving time (large groups)
• Authenticity of assessmentUse various types of questions & multimedia
• Accurate, objective and quick correctionAutomating the repetitive character of correcting
Facts and figures (4): students and examinations
5
Total amount of exams per student
Total amount of students taking an exam
6
Facts and figures (1): Evolution since reference year 2009 - 2010
7
Facts and figures (3): 2014-2015
Capturing the vibe…
9
Question 1: What about infrastructure?
Availability of infrastructure
Campus CDE135 (4 rooms)
Campus CGB 90 (2 rooms)
Campus City275 (8 rooms)
Campus Zuid85 (3 rooms)
Campus PM77 (4 rooms)
Total # 2015-16662 pc’s
New (2016-17) campus CDE
200 (5 rooms)
New (2016-17)campus CGB
328 (8 rooms)
Total # 2016-171190 pc’s
Anti-cheat sheet
Availability vs suitability: preventing fraud and cheating
• Anti-cheat sheet PC
• Assign PC at random (numbered cards)
• Identity control (student card & identity card)
• Banning of watches and (smart)phones
• Use of Building Block examination course
• Consecutive sessions (without time interval)
Availability vs suitability: preventing fraud and cheating (2)
13
Availability vs suitability: preventing fraud and cheating (3)
• Block Internet access & access to file-shares
• FILTER per room (router configuration):– No internet access– Blackboard only
14
Question 2: What about software?
15
Does our software meet the expectations?
Blackboard?
16
Beneficial: student-friendly/authentic examination
• Questioning: use different types of questions– Not only MC questions
– Questions pertaining to knowledge and insight (<> easy correction)
• Enriched Content– Images in question text and answers
– Video/audio file
• Students– Essay: text editor, spelling checker, word count– Access to Test Results and Feedback
– Magnify text and images
– SPRINT (assistive software)
17
Beneficial (2): coping with security and anti-fraud measures
Test options:• Use random blocks of questions
• Show answers in Random Order• Randomise questions
• Use of different series (≠bgcolour) for questions
• Present all questions at once or one question at a time & prohibit backtracking
• Define Test availability exceptions
• Set Timer• Use password
• Restrict location=> IP-filter
But what about other courses? Content collection? …
Adaptive rule ?
18
Beneficial (3): quality assurance & reusability
• Quality control– Attempts statistics– Column statistics– Item Analysis– Download results=> run statistics
• Reusing questions– Electronic archive of pools of questions & tests
19
Beneficial (4): scoring and processing of results
• Grade questions
• Grade with usernames hidden
• Essay:– digital text is readable, better structure of text, word count– use of rubrics– print to HTML/WORD/PDF (program)?
• Use negative scores?
• Fill-in questions?– partial credit alternatives and recalculate score unsatisfactorily
20
Additional Software
• Building Blocks – BB Examination course– BB negative scores– BB Rename & report (feedback CC)
• In-house software– Application WordtoBB– Application MFIB (collect unique answers)– Program: Print Essay & word count– AlephQ (test software)
21
Building Block: Examination course
22
Why?
• Need of secure environment
• Avoid fraud via access to Blackboard– Courses– Organisations – Content Collection– Calendar– Social tools
23
What?
• Course tool in regular course
• Generates New course based on specific template
Link: Start ExamLink: End Exam
• Transfer of students & instructors regular course >> examination course(daily synchronisation)
24
Examination course “Start Exam”
How?
• Student clicks on the link “HIER”
• Alternative: Instructor changes mode for one/multiple/all students in Course tools, Examencursusregular student => exam student
What happens?• Courses and organisations unavailable
• Content Collection tab unavailable: (disable auha_contentcollection portal role)
• Grade Centre column “Examenlink” gets marked “1”
25
Before
“start exam”
After
No acces to Content Collection tab
Only exam course in course list
26
Examination course “End exam”
How?• Student clicks on link “End Exam”
• Alternative: Instructor changes mode for one/multiple/all students
What happens?• Courses and organisations available
• Content Collection tab available: (enable auha_contentcollection portal role)
• Grade Centre column “Examenlink” gets marked “0”
27
Examination course
Start exam
Access to test
Different tests
Instructions: what to do after submissionInstructions
Random block of MC, sorting, combining, jumbled sentence
Essay
Information
End exam
28
Question 3: What about support?
29
Administrative Support
Rooms:
• Availability, capacity, software, headphones, …
Communication towards students:• Sample exam, Practice test, • Use of Announcements • Course menu link “practical matters”:
– link to Education and Examination Regulation – guidelines & instructions concerning the procedure– video about the procedure– course information: competences, evaluation criteria– application of negative scores
Technical Support
• Exam course– Activate – Prepare tests & assignments– Include pictures & multimedia– Prepare settings/adaptive release
• Supervisors:– Procedure via VIDEO/PPS– Certified training – Checklist procedure/problems
• During exam– Start & finish
31
Lessons learned & advice
32
Lessons learned
• Good organization & preparation is key!
• Success of digital exams creates new demand– Demand for BB exams continues to grow– What about the capacity of computer rooms?
• 4 weeks of examinations: – Stressful - exhausting – intensive for BB-team
33
Downsides
• What to do with network problems/Blackboard problems?– Keep calm and call IT colleagues
• Loss of work:– File was not saved correctly when downloading/uploading– File could not be retrieved
• Procedure of Exam– Rather complicated– Difficult to explain to some teachers=> need for technical support!
• Settings for a test in different places : – Test options– Adaptive release– No password/IP-range for assignments
34
Key considerations
• Need for atypical questions– Chemistry, accountancy, languages,…
• Need for external correction procedures– Fill-in fields – Negative scores – Essay word count– Different tests for different types of questions
35
Conclusion
Goals achieved for students and teachers:
• Enriched questions with multimedia
• Improved readability and structure of essays
• Accurate, objective, quick processing and fast correction of answers
=>Saving a lot of time (large groups)
• Quality assurance
• Secure environment
36
Nevertheless:
In order to:• Integrate all security settings in one location• Avoid loss of work and network problems (DOS Attack)• Combine authentic examination and powerful correction
Take your assessments and examinations to the next level with Blackboard and AlephQ
So, don’t leave just yet, but stay for the next presentation!!!