ASAPAutomated System for the

Assessment of Programming

Project Motivation

• address the issues of teaching a technically demanding subject with discipline-specific requirements including– Increasing class sizes– Widening participation– Evolving content

ASAP objectives

• Publish appropriate standards for unit tests in educational sector

• Publish tools for authoring and using unit tests

• Looking at working closely with JPlag (Plagiarism Detection) and RoboProf (Question Generation) from project partners

Design

• Service based architecture – the JISC e-learning framework

• Standards – XML, WSDL, XML, SOAP

• Technologies and Tools – J2EE, Apache Axis, CVS

• User Agents – Blackboard, WebCT

Automated Assessment Service

public

test

docs

student

work

student

database

private

tests

docs

1. Edit

unit

test

repo -

sitory

3. Access 4. Submit6. Feedback

Manage

7. Record

5. Evaluate

2. Produce

user agent services

application services

common services

8. Present

• Instructor creates (or edits) test documents in the unit test repository (1,2)

• Student accesses document and attempts assignment (formative or summative)

• Solution is submitted (4) and evaluated using the assessment service [private unit tests] (5)

• The result communicated to the student (6) and the database of student marks (7)

• Instructors viewdetails of assessments (8)

Use Case

Associated Services• Partners providing complimentary services

• JPlag – Provides Plagiarism testing on submitted code

• Roboprof (Dublin)– Provides Objective Assessment

User Agents

• Generic Web Portal– Initial interface to ASAP– Evolving from a test environment

• Blackboard v6 (Kingston / de Montfort)

• Web-CT v2 (City University)

TAPAS

• First stage of project• Prototype system• Currently being tested

for Java Programming Assessment at Kingston University

• 1st Gen. Self-serve• 2nd Gen. Asses. Server• 3rd Gen - WSDL• 4th Gen - eLF?

TAPAS• Precursor to ASAP• Internally funded• Assessment Server

– Provides unit testing to assess Java program code

• Blackboard Building Block– VLE ‘User Agent’– Communicates with assessment server– Instructor: uploads testing code and

assignment– Learner: Downloads assignment– Uploads source code– Assessment server compiles and tests– Gives feedback and uploads results to

Blackboard gradebook

Blackboard Interface

Automated Java Marker Server

Feedback from service