Jason ColeConsultant

As presented at theSakai Summer Conference

12 June 2007 | Amsterdam, Netherlands

The public face of eLearning

The public viewof higher education

Georg

eto

wn

U

niv

ers

ity

The public perception

Tuition and Required Fees Public Universities

0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

1999-2000 2000-2001 2001-2002 2002-2003 2003-2004

Pe

rce

nt

cha

ng

e

Tuition public universities

Consumer price index

Digest of Education Statistics 2004, NCES

Georg

eto

wn

U

niv

ers

ity

The press

Federal Study Finds No Edge for Students Using Technology-Based Reading and Math Products

Georg

eto

wn

U

niv

ers

ity

The U.S. Congress

… the breach of trust between schools and students. There is an important relationship there that some schools, though certainly not all, have been far too cavalier with.

Senator Robert P. Casey, 6 June 2007

Georg

eto

wn

U

niv

ers

ity

The Spellings Commission

• And some [students] never complete their degrees at all, at least in part because most colleges and universities don’t accept responsibility for making sure that those they admit actually succeed.

• Many students who do earn degrees have not actually mastered the reading, writing, and thinking skills we expect of college graduates.

U.S. Department of Education, 18 September 2006

Georg

eto

wn

U

niv

ers

ity

But the Commission wrote

• “We recommend that America’s colleges and universities embrace a culture of continuous innovation and quality improvement. We urge these institutions to develop new pedagogies, curricula and technologies to improve learning, particularly in the areas of science and mathematics.”

And eLearning

Georg

eto

wn

U

niv

ers

ity

U.S. Department of Education study of

educational software• “Congress posed questions about the

effectiveness of educational technology and how effectiveness is related to conditions and practices. ... On average, after one year, products did not increase or decrease test scores by amounts that were statistically different from zero.”

• As reported in the press: education technology doesn’t work.

Georg

eto

wn

U

niv

ers

ity

Training: A comment

• The most important training [for eLearning users] is how to use the technology to achieve lesson objectives, not how to use the software.

• Training requires assistance during the early use of the software.

Debra Sprague, Graduate School of Education, George Mason University, responding to questions about the study at the Blackboard Forum, National Press Club,

Washington, DC USA 11 May 2007

Georg

eto

wn

U

niv

ers

ity

Presidents on e-Learning

• “Based on his work with the University of South Australia and his conversations with presidents and financial officers, [Bill Becker] said there is a general belief that eLearning increases the cost of education. He said the cost of the distance learning courses at the University of South Australia exceed those offered in the classroom because of the amount of time that faculty spend responding to students.”

“Access and Persistence Symposium,” September 8, 2005, Washington, DC

Publishers have taken the lead on reporting

effectiveness of eLearning

Georg

eto

wn

U

niv

ers

itySuccess in math courses

Georg

eto

wn

U

niv

ers

ity

Use of supplementary material

Georg

eto

wn

U

niv

ers

ity

Faculty believe

And now

Georg

eto

wn

U

niv

ers

ity

Observations

• Evidence suggests eLearning appears to be most effective for first and second year students.

• Pedagogy is important to effectiveness of eLearning; “best practices” must be available to faculty.

• Methods of instruction must be adapted to student preparation, motivation, and constraints.

Georg

eto

wn

U

niv

ers

ity

Evidence being used

• Improvements when eLearning is used:

• Retention

• Satisfactory completion

• Performance in sequent courses

• Graduation rates

• Scope of mastery of the subject

• Student engagement

Georg

eto

wn

U

niv

ers

ity

More work

• “What works” - Colleges and universities should produce and document evidence of educational effectiveness of eLearning.

• Pedagogy - The key to effectiveness is guidance for faculty on the effective use of eLearning resources, not the learning system software

Georg

eto

wn

U

niv

ers

ity

“Further study”

• Why are faculty turning to publisher’s online services instead of local eLearning services?

• Which eLearning services are most effective for which courses?

Thank You

Jason ColeCognition and Instruction

Associates

Supplementary material

Georg

eto

wn

U

niv

ers

ity

Evaluation of Education Technology: High School Algebra

“Effectiveness of Reading and Mathematics Software Products: Findings from the First

Student Cohort

Report to Congress,” Institute of Education Sciences, U.S. Department of Education, March

2007.

Georg

eto

wn

U

niv

ers

ity

Effectiveness of Reading andMathematics Software

Products• U.S. Congressionally mandated report

by the Institute of Education Sciences, U.S. Department of Education.

• Issued March 2007

• First and fourth grade reading, sixth grade mathematics and high school algebra.

• Context: The Department again did not seek funding for educational technology.

Georg

eto

wn

U

niv

ers

ity

Summary findings

• “Congress posed questions about the effectiveness of educational technology and how effectiveness is related to conditions and practices. ... On average, after one year, products did not increase or decrease test scores by amounts that were statistically different from zero.”

• As reported in the press: education technology doesn’t work.

Georg

eto

wn

U

niv

ers

ity

Study findings

• “Nearly all teachers received training and believed the training prepared them to use the products.”

• “Technical difficulties using products mostly were minor.”

• “When products were being used, students were more likely to engage in individual practice and teachers were more likely to facilitate student learning rather than lecture.”

Georg

eto

wn

U

niv

ers

ity

Warning

These data are based on teaching high school algebra and would not be representative of other subjects, levels of instruction, or students with different characteristics.

Georg

eto

wn

U

niv

ers

ity

Training

• Teachers received about 12 hours of training, including practice using the software.

• At the end 81% were “confident they were prepared to use the product”

• By the time of the first classroom observation, only 66% considered themselves prepared to use the software.

Georg

eto

wn

U

niv

ers

ity

Training: A comment

• The most important training is how to use the technology to achieve lesson objectives, not how to use the software.

• Training requires assistance during the early use of the software.

Debra Sprague, Graduate School of Education, George Mason University, responding to questions about the study at the Blackboard Forum, National

Press Club, Washington, DC USA 11 May 2007

Georg

eto

wn

U

niv

ers

ity

Use of the software

A B C AllTutorials Few Many ManyPractice Many Many Many

Indivudalization Automatic P,A T,P,A T,P,A Teacher P,A T,P,A T,P Student T,P T,P,A T,P

Days per school year 40 9 20 22Minutes per day used 41 28 38 34Hours of annual use 28 5 13 15

Software Products

T - Tutorial, P - Practice, A - Assessment

Georg

eto

wn

U

niv

ers

ity

Test Results

Using Technology Traditional Difference

Overall score 37 38 -0.86Concepts 36 37 -1.70Processes 35 36 -1.09Skills 41 41 0.39

ETS End-of-Course Algebra Examination

Georg

eto

wn

U

niv

ers

ity

Cost of software

• Software provides tutorial, practice, and assessment opportunities.

• Average licensing fees about $15 per student for the school year; a range of $7 to $30.

• [Teachers reported] students used the software and average of 118 minutes per week for 23 weeks or 46 hours (of 180 hours).

Georg

eto

wn

U

niv

ers

ity

Impact on classroom activities

Using Technology Traditional

Teacher RoleLeader 12% 50%Facilitator 63% 28%Monitor/observer 16% 14%Working on other tasks 6% 7%Other 4% 1%

Instructional ActivityIndividual practice 85% 32%Lecture 8% 36%Question and answer 3% 15%Review of student work 1% 13%Other 3% 5%

Student on task 74% 65%