Workshop: Automatic Assessment in

Mathematics Education

Matti Pauna, University of Helsinki, JEM

1

Supporting learning with web-based

practicing and assessment exercises

2

Outline

!! Introduction and background

!!Using web based assessment in learning

!!Mobile assessment

!!Authoring exercises and building assignments

!!Case studies and results

!!Hands on

3

Background

!! WebALT project 2005 – 2006

!! JEM (Joining Educational Mathematics) network 2006 – 2009

!! Coordinator: University of Helsinki, department of Mathematics and Statistics

!! Expertise on using TEL in the field of mathematics, focusing on the use of interactive exercises for practicing and assessment

!! WebALT: build a multilingual database of exercises using language independent encoding grammar (OpenMath & GF) and tools to author exercises

4

linalg1:determinant(linalg2:matrix( linalg2:matrixrow(a,b),

linalg2:matrixrow(c,d))) Linguistic hints for mood,tense and directive

5

JEM Thematic Network

" ! Joining Educational Mathematics 2006-2009

" ! Continues dissemination

" ! EU funded network

" ! Gathers technology developers, authors and users of e-content

c!

http://jem-thematic.net

7

Exercises

!! Currently implemented in MapleTA, powered by the Maple computer algebra system

!! Different types: multiple choice, open answer, fill in

blanks…

!! Most of the exercises allow answering by a mathematical formula whose correctness the system evaluates

!! Exercises are algorithmic and randomized

!! Problems include hints and complete solutions

!! Grading is done automatically, using Maple to evaluate student’s response against defined criteria

8

Exercises

!! To assess and practice basic mathematical skills

!! Illustrative graphs and

images

9

Question Banks !! Questions are divided into

banks according to the MathTax classification

!! Manual and ad hoc encoding, the system does not provide support for sufficient metadata

!! Repository supporting

LOM was developed for better management of eContent

10

Examples of use

11

Homework Weekly tests

Diagnostic e-tests

Remedial studies

Distance learning

Virtual online classes

!! Best suited for practicing basic skills which are the

tools required to build understanding and problem solving

!! High-schools, first year college level mathematics

Some benefits of the system

!! Easy to use, only web browser is needed

!! Allows individual practicing at own time, place, and pace in a “neutral” environment

!! Grading is done automatically by the computer which saves time of the instructors

!! Immediate feedback for students

!! Students gain understanding of their learning process and can concentrate on the weak points

and practice more

!! Diagnostic testing helps to design the teaching, for example, dividing students into groups according to their individual learning needs

12

Exercises on Mobile Devices

Smartphones

SMS quizzes

13

Quizzes in iPod or Smartphones

14

SMS Exercises

15

!! Questions, answers and feedback as text messages

!! Developed a Moodle module to cater quizzes via SMS gateway

!! For true time and place independent practicing

!! Just in time classroom quizzes

!! In many countries mobile phones are much more common than computers with internet access

Real time quizzes with phones as personal

response system (clickers)

16

Real time SMS quizzes activate students

17

!! Immediate feedback to correct mistakes right in the beginning

!! Raises the level of attention: students follow the

presentation more carefully knowing that they have to answer questions after it

!! Everybody gets involved

!! Eliminates the need for special devices, which often cause

configuration and other technical problems

Authoring interactive

exercises and quizzes

Exercise structure

Algorithms

Setting up quizzes

18

Question components

19

Question text

Question text with algorithmic parameters

Correct answer

All of these elements are instantiated from the algorithmic parameters

Authoring questions

!! Question bank editor

!! LaTeX macros

!! Editing the source (EDU) code

\begin{question}{MultipleChoice} \name{example 1} \qutext{Given $f(x)=(x+3)^2$, find $f(x+5)$} \choice*{$(x+8)^2$} \choice{$x^2+6x+14$} \choice{$x^2+10x+24$} \choice{none of these} \end{question}

20

Source format

21

22

Algorithm

$a = range(-5,5); condition:ne($a,0); $b = range(-3,3); condition:ne($b,0); condition:le(abs($b-($a)),5); $f = maple("($a)*x+($b)"); $p1=maple("[0,($b)]"); $p2=maple("[0,($b)-($a)]"); $p3=maple("[-1, ($b)-($a)]"); $y1=($b)-($a); $y2=if(lt($b,0), "($b)", $b); $deltay= -($a); $plot = plotmaple("plots:-display([plot($f, x = -5 .. 5, y = -5 .. 5, color = blue, thickness=2), plots:-coordplot(cartesian, [-5 .. 5, -5 .. 5], color = [gray, gray], linestyle = [solid, solid], grid = [11, 11])], axes = normal, tickmarks=[11,11], labels=[``,``], numpoints=300)");

23

Algorithm editor

24

Evaluating correctness of answer

Cannot analyze student’s response, partial grading not possible

25

Equations

!! Encoded in MathML

1.! LaTeX converted to MathML

2.! MathML source

3.! mathml function

$Deq = mathml(“$a*x^2+$b*x+$c”);

4.! Maple call

$Drad = maple(“printf(MathML[ExportPresentation]( sqrt(a+b)<>sqrt(a)+sqrt(b) )) ”);

5.! Equation editor

!! Displayed as images on the web page

26

Equation editor

27

Setting up assignments

28

!! Collect questions into quizzes

!! Set policies

Case studies

30

1. Exercises for Single Variable

Calculus

!! First year university course

!! About 100 exercises for limits, differentiation, integration, series, etc.

!! Maple T.A. exercises used as homework assignments and also a diagnostic test as self-assessment before the course

!! Bonus points were given to students from homeworks and they could take the assignment any number of times, the best attempt counted

!"#$%&'()&*+

,-./0$1/+2$%+

3%4*0*&1#+54/&*+

'.*(1&,-./+$2+

64"*-"-/+

2. High School exercises

!! Exercise sets for all 10 long term math courses in the Finnish curriculum developed since 2005

!! Algorithmic exercises for training basic methods and techniques

!! Students do not usually practice enough which is needed to learn mathematics and solve more demanding problems

74/&*+,-./0$1/+

2%$)+8&9.%.1'+

'$3&*/+&1+(&#(+

/*($$"+)4'(+

Essay questions

!! For expressing mathematical thinking verbally

!! For self evaluation

!! Implemented as essay questions or essay parts in

exercises and they cannot be graded automatically

3. Diagnostic Testing at Polytechnic

schools

!! To assess the level of mastering basic mathematics skills of new students, both for students themselves and instructors

!! Problem: Students have very varied background and teaching needs to be designed accordingly

!! A diagnostic test is organized in computer lab, first week of the semester, a brief introduction to the system was given

!! Algorithmic questions are different for each student, so cheating is harder

39

Results

:&/'%&5-0$1+;<=>+?+=@>+?+A=>++$2+3$&1'/BC+

Research: Kari Lehtonen

Diagnostic tests and practicing in technical engineering school. Results in the test before and after practicing.

Purple (four groups) n = 80, not using online assignments.

Blue group n = 40, using online assignments.

http://jem-thematic.net/en/node/872