The LearningOnline Network with CAPAA Cross-Institutional Open-Source

Learning Content Management and Assessment System

Gerd Kortemeyer

Michigan State University

PERG Seminar

University of Maryland

Some of the History

1992 CAPA

•Started by Ed Kashy et al. in Cyclotron

•Individualized assessment system for science and math

•Immediate feedback, multiple tries - mastery based

•Used paper copies of assignments and terminal input

•X-Windows problem editing

•Got Web student interface in 95

1997 LectureOnline

•Started by Wolfgang Bauer, Walt Benenson, Gary Westfall, and Gerd Kortemeyer in Cyclotron

•Learning content management and individualized assessment system for science and math

•Sharing of content between courses

•Completely web-based interface

1999 LON-CAPA

•The LearningOnline Network with CAPA

•Collaboration of CAPA and LectureOnline groups

•“The best of both worlds”

•Sharing of content between courses and institutions

•Reusability of content on different levels of granularity

•Distributed and Scalable

LON-CAPA Today•Attracted NSF Information Technology Research grant “Investigation of a Model for Online Resource Creation and Sharing in Educational Settings” (more later)

•Has over 20 partner institutions (more later)

•Useful and usable for “casual” (non-”power”) faculty user outside science and math

•LON-CAPA is a full-featured course management system

•Software platform has GNU General Public License and is free of licensing fees

•2003 ComputerWorld 21st Century Achievement Award

•Version 1.0 was rolled out this Fall

Today LON-CAPA is used …

•With material written by faculty teaching course or “re-used” from other faculty + helproom •For homework in addition to “traditional” lecture and textbook + helproom•Hybrid with “traditional” lecture, homework and textbook online + helproom•Completely “virtual” courses•AP courses•Prelab quizzes•In-class exercises•Class sizes: 10-3,000

Today LON-CAPA is used … Helproom

•Helproom at MSU started out:

–Very few computers,

no seating in front of computers

–Round tables

–Big sheets of paper

–Whiteboard all around the walls

–TAs present

–Basement “cave”

–Absolutely crowded

•The all-new helproom since last year:

–Lots of desktop computers in cubicles

–#@$%@!$&#

Student Numbers at MSU

Subject Areas MSU Fall 2003

System-wide Use

•24,000 students/semester system-wide

•Middle Schools: 300•High Schools: 500•Community Colleges: 50•Four-year Colleges: 300•Universities: >22,000.

Cross-Institutional

Distributed Server Network

•LON-CAPA is built as a geographically distributed network of

constantly

connected servers

Logical Domains

•The network is logically divided into domains such as “MSU”, “FSU” or “Publisher X”

•Domains limit the flow of user information

•Domains can limit access to content resources

•Domains limit the extent of user privileges

Authentication

•Any user in the network can log into any machine in the network

•Example: Michigan State University user logs into server at North Dakota State University

Authorization

•Any LON-CAPA user can have roles for any domain.•Example: Michigan State University faculty is Course Coordinator for a course at North Dakota State University

Virtual Filesystem•The shared content repository appears as one virtual filesystem

Resource Assembly

•Pages can be constructed from fragments and other pages

•Sequences can be constructed from fragments, pages, and other

sequences

•Courses point to (top-level) sequences

•Maps at every level are simply other content resources

Resource Assembly

•Graphical resource assembly tool (RAT) to construct maps

•Working on

branching

based on

conditions using

performance data,

course data,

and preferences

•Individualizedcurriculum

Shared Resource Pool

Shared resource pool currently holds approximately

•6,000 original homework and exam problems

•5,000 images

•150 movies

•180 java applets

•3,000 content pages. LON-CAPA Pool available in NSF

National Science Digital Library

Individualized Assessment

“Individualized Assessment”

•Individualized problems: different numbers, different graphs, different options, …•“Classical” online homework elements: multiple choice, option response, mix-and-match, string, etc •“STEM” elements: numerical, multicomponent numerical, physical units, symbolic math, individualized simulations•Free combination of the above elements•Adaptive immediate feedback•Multiple attempts - mastery based

“Individualized Assessment”

•Example: Individualized graph, numerical answer

“Individualized Assessment”

•Example: Individualized labels, options

Conceptual:Functions Graphed

+RandomLabels

“Individualized Assessment”

•Example: Multicomponent numerical with individualized animation and physical units

Individualized Assessment

Text for Applet

Applet:Initial and

Final ActionFrames

Student evaluations of CAPA

Evaluations of CAPA by students in Physics for

Scientists and Engineers

CAPA as Tool for Learning/Understanding

0

20

40

60

80

100

120

A B C D E

Number of Students

A - Strongly in favor of recitationsB - Moderately in favor of recitationsC - IndifferentD - Moderately in favor of CAPAE - Strongly in favor of CAPA

Fall -Traditional… Spring - Technology

Phy184Spring 95N=156Birge, Cowen

Question: If I had a choice between the CAPA scheme as used in PHY184 and the recitation scheme used in PHY183, I would be:

0 20 40 60 80

CAPA/Help room preferred

Methods equivalent

Recitation preferred

Number of Students

Did the different methods affect your learning/understanding of physics?

Fall – TechnologySpring - Traditional

PHY184Spring ‘96N=104Berz, Tomanek

Time-on-task with and without CAPA

Educational Effectiveness in Physics

Technology Impact

Prof. Danielewicz (Physics)Prof. Danielewicz (Physics)

0.0 1.0 1.5 2.0 2.5 3.0 3.5 4.0

Grade

0

5

10

15

201996-1998N = 1435

0

5

10

15

201999N = 472

0

5

10

15

20

251992-1994N = 1537%

2000N = 449

0

5

10

15

20

Distribution of grades in Introductory Physics for Scientists and Engineers

1992-1994 w/o CAPA1996+ with CAPA

Gender Differences in Physics

• Grade distributions for male and female students for a two-semester course (PHY231 and PHY232).

• PHY231 was taught the traditional way, and in PHY232 CAPA was used.

0

5

10

15

20

25

30

35

0.0 1.0 1.5 2.0 2.5 3.0 3.5 4.0

0

5

10

15

20

25

30

35

0.0 1.0 1.5 2.0 2.5 3.0 3.5 4.0

%

% Phy231

Phy232

Male

Female

Phy231

Phy232

Phy231

Phy232

Grade

Item Analysis

<conceptgroup concept=”Fruit”> <foil>Apple</foil> <foil>Orange</foil></conceptgroup><conceptgroup concept=”Metals”> <foil>Copper</foil> <foil>Gold</foil></conceptgroup>

Doing Item Analysis onRandomized Questions

1. Two masses have the same acceleration if the string does not stretch.2. Weight of the two masses is greater than the tension of the string attached to the ceiling3. The top tension equals the bottom tensions. (massless pulley)4. Tension holding the two masses are equal if mass of pulley = 05. Sub-system accelerates upwards or downwards accordingly6. Center of mass accelerates downward

First 18.6 % of responses

Predicting Success, Learners at Risk

Covariates included ACT, College GPA, Initial Physics knowledge (Force Concepts Inventory), and number of absences

Success Prediction Power

Best Predictor Total Number of Correct Answers

Good Predictor Total Number of Tries

Predictor •Gets Problems Correct on the First Try

•Time spent

Minor Predictor Participation in online Discussions

“Cheating”

Influence of Websites

0.1290.147-0.348Final

Exam

0.1600.166-0.352Midterm Exams

0.0690.098-0.302Quizzes

-0.1090.1180.041Homework

Monitored

(looking)

Monitored

(posting)

3rd Party

The NSF ITR Grant Project

The Project

•“Investigation of a Model for Online Resource Creation and Sharing in Education Settings”•Uses LON-CAPA as model system•Five year project, $2.1M, funded by the National Science Foundation Information Technology Research Program

The Project: Research Goals

•Open source code development concept for Instructional Management System•Pooling online educational resources between departments, colleges, universities and schools•Business models for “trading” such resources on small level of granularity•Business models for services and support•Metrics for resource quality and effectiveness•Curriculum adaptivity to learner

types and remediation needs

“Economy of Scale”

•Content: System more attractive the more content is online•Code base: GNU GPL, system improvements from larger development community•Support: mechanisms for peer-to-peer support•Training: mechanisms for peer-to-peer training

On the web ...

www.lon-capa.org

help.lon-capa.org

bugs.lon-capa.org

mail.lon-capa.org

install.lon-capa.org