Quality Assessment Studies in the area of E-Learning

Symeon Retalis (retal@unipi.gr) University of Piraeus

Department of Technology Education and Digital Systems

Computer Supported Learning Engineering Lab [http://cosy.ted.unipi.gr]

Why do we care about the quality in e-learning? E-learning, which

was once a poor and often unwelcome stepchild within the academic community,

is becoming increasingly more visible as a part of all educational levels.

Is distance education effective or not?

In his painstaking annotated bibliography The No Significant Difference Phenomenon (1999), Russell lists 355 sources dating back to 1928 that state that there is no evidence of any kind that

categorically proves that technology has any impact on learning – either positive or negative

[http://www.nosignificantdifference.org].

The quality assessment needs to see the true educational value of what’s on offer.

Key shortcomings of original research in quality assessment Much of the research does not control for extraneous

variables and therefore cannot show cause and effect. The validity and reliability of the instruments used to

measure student outcomes and attitudes are questionable.

Many studies do not adequately control for the feelings and attitudes of the students and faculty—what the educational research refers to as “reactive effects.”

It is not taken into consideration how the different learning styles of students relate to the use of particular technologies

An example: Findings in NCSL [Pinelle & Gutwin, 2000] “A Review of Groupware Evaluations” The main findings are that

almost one-third (1/3)of the groupware systems were not evaluated in any formal way,

that only about one-quarter (1/4) of the articles included evaluations in a real-world setting,

and that a wide variety of evaluation techniques are in use. From the studies included in this survey, 41% of the articles that included evaluations were of actual real world software

implementations, but only 25% considered the software’s organizational and work impact.

Their main conclusions from the review are that more attention must be paid to

evaluating groupware systems and that there is room for additional evaluation

techniques that are simple and low in cost.

TELL: Towards Effective network supported coLLaborative learning activities

[http://www.softlab.ece.ntua.gr/research/research_projects/tell/]

The E-LEN projecthttp://www.tisip.no/E-LEN

Validation of quality

approaches in specific areas

Need for experimentation in

real classroom environments

The necessity of e-learning centers

supporting the integration of NT in education

The need for new methods & tools for

instructional design

Design patterns

√ No “novel effect” anymore!

√ We need to examine what works and what does not in a

complex e-learning environment with relation to the high order

learning outcomes

There is no silver bullet

√ Designing effective e-learning environments will always be

hard!

√ there is no silver bullet

Quality Assessment Studies in the area of E-Learning

Symeon Retalis (retal@unipi.gr) University of Piraeus

Department of Technology Education and Digital Systems

Computer Supported Learning Engineering Lab [http://cosy.ted.unipi.gr]