The Institutions Report

Brigitte Denis, Pascal Detroz and Lidia Graves

Introduction

This chapter will focus on the analysis of data collected by the Institution Questionnaire. The survey was sent to the institutions supporting distance education courses, independently of whether they were a university, college or professional training centres.

The questionnaire had four major sections: the institutional aspects, the tutors and students in distance education, the media in distance education, and the assessment.

The Institutional Aspects - The major objective was to know the total number of courses being offered by each institution. In addition we asked how many of the courses were available in a distance mode.

The Tutors and Students in Distance Education - Here we wished to ascertain whether or not participating tutors received any type of professional training in distance education (including curriculum development, pedagogy and the use of technology). Secondly we investigated the nature education experience. For example, we wished to know if learners had opportunities to work together. If this was an aspect of course delivery, was it provided face-to-face or through telematics, and what was the educational purpose of such co-operative working?

The Media in Distance Education - The principal objectives were to find out the types of media which were being used in each institution to support courses.Where and by whom were course materials developed? Were learners required to be on site when using such materials or could they be used at a distance?

The Assessment - This section of the questionnaire focused on the key aspect of assessment and evaluation of courses with particular reference to both the methods of asssessment and the purpose of evalaution. Questions such as ÒWho is assessed?Ó, ÒWho are the evaluators?Ó, ÒWhat kind of assessment is provided?Ó, ÒWith the help of which evaluation tools?Ó were central to this part of the study.

The original questionnaire, the French version, was checked for readibility and understandability. Afterwards, the questionnaire was translated into English, Finish,

76

German, Portuguese, Spanish, Italian and Swedish. It is important to mention that in the process of translating the English version, the word Course encountered differences in its definition in the different European countries. For example, in Country X the word would mean a course among others to be trained in a specific discipline, whereas in another country it would mean the individual subject areas to make up a degree. We endeavoured to standardise key meanings within the different versions of the questionnaire to make the results comparable across European countries.

The questionnaire was sent to the institutions across the EU who were providing some level of distance education. The English, French and Spanish versions of the questionnaire were also available on the Web.

Institutional aspects

This section provides information about the structure of the institutions through the following topics :∑ Which kind of institutions provide distance education -in terms of size, structure and

typology.∑ The number of students and tutors involved in these institutions? What are the

characteristics of these institutions?∑ The nature of the curriculum provided in distance education institutions - including the

number of courses and subject areas covered.

Graphic 1 Number of courses given in a "distance learning" way versus the number of

courses given in the whole institution. N=36

0

50

100

150

200

250

300

350

77

1000

B=Belgium, UK=United Kingdom, Fi=Finland, Fr=France, Ge=Germany, It=Italy, P=Portugal, SP=Spain, Sw=Sweden.

Size and structure of the institutions or of the distance education departments

The size of the distance education (DE) institutions or DE departments within institutions varies greatly from one country to another and also within the countries. Some DE centres provide a small number of courses but are based in large institutions, for example the French Òcentres de tŽlŽ-enseignementÓ that are included in the universities.

From Graphic 1we can observe that the maximum number of distance education courses (DEC) which are given in any one of the institutions of our survey is 1000 (France 7, which is a national organisation of several DE university) and the minimum number is 1 (cf. the cases of Italy 1, Portugal 2 and 5, Belgium 4) (see Graphic 1).

From these results, we can distinguish two different situations. On the one hand, some institutions (N=13) have a large number of distance education courses (greater than 60) but do not provide any presential courses (the exceptions are Finland 2 and Portugal 7). On the other hand, there are institutions which provide a rather small number of distance education courses (less than 30) and also provide more traditional presential education courses (9 out of 21 cases).

From our sample, we can conclude that when an institution has a structure where more than sixty courses are given at a distance, this mode of delivery will be the sole mode and there will be no traditional face-to-face courses. Such institutitonsas are specialist distance education providers. Conversely, when an institution from our sample provides less than thirty courses in a distance way, that is in almost half of the cases, it also provide traditional face-to-face education.

Distance education or presential education?

The level of involvment in ODE (Open and Distance Education) and in traditional education modes by institutions is presented in Graphic 1. For instance, if the number of courses in the whole institution is the same as the number of distance education courses, all the courses of the institution are then distance courses and the institution is entirely involved in ODE.

Taking into account the thirty-six institutions of our sample, we note that only thirty-three

78

can be analysed: two institutions (UK 3 and UK 4) did not respond to two questions (number of courses and number of distance education courses) and one other did not respond to the question related to the number of distance education courses (Belgium 2). Of the thirty-three institutions analysed, two thirds (21/33) do not provide any presential course. We suggest that:∑ The institutions oriented towards distance education only are younger institutions than

those that provide presential education as well.∑ This kind of institutions which provide both presential education and distance education

courses tend to be involved in education of students post 18 years. Many, but not all, of these institutions have begun to provide distance education courses as a result of developments in new information and communications technology (NICT).

∑ The institutions and departments which do not provide presential courses were created only to provide ODE.

We now examine the first two hypotheses in more depth. Let us examine the Belgian case. The institution which provides both presential and distance education (Belgium 4) is a department of a university created 178 years ago. While Belgium 1 is a young commercial institution. Belgium 3, the most important Belgian distance centre was created in 1959 and is more recent than the two Belgian high schools of our sample. This centre is devoted only to ODE.

All the Italian (except for Italy 6), Spanish, Swedish and UK institutions of our sample provide distance education courses only. We view these data with some circumspection as we are aware that data from two traditional UK institutions were lost from the sample. The case of the French institutions is also of interest to us. All the institutions of our sample are universities which provide both presential and distance education courses. But, in each of these universities there is a special department called Òcentre de tŽlŽ-enseignementÓ which providse only distance education courses. These centres are part of the university but have their own structure. The status of these centres is somewhat anomalous. The people in charge of these centres found it difficult to say if they are part of the university (in this case, there would be more courses than distance courses), or an autonomous institution in their own right (in this case the number of courses are equal to the number of distance courses). At one level, they are integrated to the university but at another they operate as independant units and share the characteristics of many newer institutions. Unfortunately, we cannot say from our data whether the rise in ODE is directly linked to the evolution of NICT. We know it is the case of some Belgian universities, but even when we have this kind of information, ití’s difficult to see if the reasons are an added value of NICT for distance education or simply a Ôband-wagonÕ effect.

79

Typology of the institutions

Our next question sort to establish, for each curriculum area, whether it is taught in a distance way, in a presential one or in both distance and presential ways.

The combination of these modalities provides a typology of the institutions. We have established seven categories of incstitutions : A: Institutions which provide only distance education courses (DE)B: Institutions which provide distance education courses and also presential courses but

not for the same topic (DE or PE). C: Institutions which provide distance education courses and also presential-and-distance

courses (DE and (DE + PE)).D: Institutions which provide presential courses and presential-and-distance courses (PL

and (DE + PL)).E: Institutions which provide only “ ìpresential-and-distanceî ” courses (DE + PL).F: Institutions which provide distance courses, presential courses and presential-and-

distance courses (DE + PE + (DE + PL)).G: Institutions which provide only presential education (PL, but they are not considered in

our sample).

Graphic 2 shows that :

Type A : DE for all courses.

Almost half of the institutions in our sample (16/36) are distance education institutions that do not provide any presential courses. We recognise that our sample is biased towards this category - institutions heavily involved in DE were more likely to respond to our survey because they were enthusiastic users of this mode of education. Nevertheless let us consider two assumptions:

This case could refer to specific distance education institutions which have as a prime objective of providing distance education structures for many courses. Do such institutions offer a wide or normal range of courses? Are they generalists or specialists? We assume that these institutions should provide many courses related to a wide range of domains.

It could refer also to ODE institutions that are specialists, offering few courses and topics.

80

Such institutions tend to be vocational in nature providing training for groups such as secretaries or technicians in various medical fields.

Six institutions out of 15 provide many courses related to a wide range of domains, which is typical of the first assumption (see Table 1). For instance, for Italy 5 there are 184 courses within 6 different domains.

Graphic 2 Typology of the institutions

N=36

16

5

3

5

34

0

2

4

6

8

10

12

14

16

A B C D E F

Table 1:

N=15 Few courses (<21)

Many courses (>21)

Few domains (<4)

6 courses 1 course

Many domains (>4)

2 courses 6 courses

81

It could also refer to institutions whose the target is not to implement a global distance education structure, but to teach one specific topic. It should be the case if the number of courses is low and these courses are focused in a restricted number of domains. This is the case for 6 institutions out of 15 (see Table 1). For instance, for Italy 1, we have one course and one subject.

So, only 3 institutions out of 15 do not fit to our model, which demonstrate its potential. Out of these three institutions, the two which provide a limited number of courses but manage many domains come from Spain. The institution which offers many courses but in a restricted number of domains is based in France.

Type B : DE or PL, depending on the courses.

Five institutions out of the 36 provide distance and presential education courses but never both for the same course. We suggest that some courses are directly implemented in a distance way while others have been transformed from a presential to a distance way possibly to reach a bigger target public.

Type C : DE and (DE + PL), depending on the courses.

A small number of institutions (3/36) provide courses either at a distance or both at a distance and face-to-face. These institutions have no separate face-to-face courses. It seems to be the case that for distance education institutions it easier to teach a small part of some courses in a presential way, and this is the explanation for the dual-modality. For example, one such institution has part of its course implemented in a presential mode in order to allow learners who have no access to the Web to obtain the same information as those who have personal access to the web.

Type D : PL and (DE + PL), depending on the courses.

Five institutions out of the 36 focus on presential courses among these courses some may have an ODE element. These do not provide courses supported only by distance education methods. The reasons for using ODE in such courses are open to conjecture. Such institutions may be testing advantages related to distance education or there maybe an economic imperative driving the development of ODE elements to these courses.

82

Type E: (DE + PL) for all courses.

Three institutions out of 36 combine presential and distance modalities for all their courses. Is it the case that some DE institutions find it easier to provide a small part of some courses in a presential way, or is it that some presential institutions find it appropriate to teach a small part of some courses in a distance way.

The first hypothesis is probably more valid than the second as the question of the relevance of ODE is not automatically asked in presential education. The costs and benefits of presential education do tend to be seriously considered when designing distance education courses, however.

Type F: DE or PL or (DE + PL), depending on the courses.

Four institutions out of 36 provide distance courses, presential courses and distance and presential courses, with some courses being provided in a presential or distance way while others are offered through both delivery modes.

Tutors and learners in distance education

Number of tutors by institution

Graphic 3 shows the number of full time equivalent tutors (professors) working in each institution of our sample. We note a huge difference between institutions on this point. The number of tutors varies from only two tutors to over six thousand. It would seem sensible to formulate the hypotheses that the number of tutors increases with the number of:

a. courses provided by the institution b. learners enrolled in the institution

83

Graphic 3Number of teacher (full time equivalent) in the distance learning institutions

8 20 2

350

75

5000

609

70 80

10 13

100

500

3565

25

6000

4 236 40 50

950

3 0 19 2 3

119100

27

1929

150

0

100

200

300

400

500

600

700

When we test these two hypotheses, we find that the number of tutors is not linked to the size of the institution in terms of number of courses on offer. Effectively, when we evaluate the ratio number of courses to number of tutors, we observe that the results range from 1 full time equivalent managing 34 courses to 482 full-time equivalent managing one type of course. Paradoxically, the correlation between these two variables is high (+0.79). However the correlation is highly influenced by a outlier data in this case institution France7 which has 6000 courses and 1000 tutors. When we delete this outlier point, the correlation falls to (-0.19) which is not significant. We can conclude, therefore, that there is no general relationship between the number of courses and the number of tutors within institutions.

When we calculate the correlation between the number of tutors and the number of learners the results are very diffrent. The correlation between these two variables is positive (+0.64) and is significant at the one in twenty level. Not surprisingly, the number of students is a key factor determining the number of tutors employed within an institution..

It is possible that the number of students and the number of courses are under the influence of other variables. We are aware that in distance education the role of the tutor can be very varied. Of course, one of the roles is to implement the content of the courses, but other roles are to ensure that the documents reach the students, to manage them, to correct the studentsÕ work . The overall number of tutors per course may also be influenced by the degree of help available to carry out tasks such as basic administration or indeed more pedagogic roles such

84

5000 60001929

B=Belgium, UK=United Kingdom, Fi=Finland, Fr=France, Ge=Germany, It=Italy, P=Portugal, SP=Spain, Sw=Sweden.

as computerised assessment of student . If tutors benefit from such facilities, they do not need to see or correct all of the studentsÕ work, so their workload is reduced or time is made available for other activities. NICT and administrative tasks may come together if tutors use the Web to implement course modifications without contacting learners on an individual basis. Throgh the use of NICT the link between the number of teachers and the number of students in any one institution could be weakened.

To test this hypothesis, we re-analysed the data for those institutions which used telematics, and calculated the correlation between the number of tutors and the number of learners. The conclusion is that this criteria stay significant under that condition - a positive correlation of +0.98 was achieved which is statistically significant, but with 2 outlier institutions France7 (1,000 tutors and 400,000 learners) and UK3 (5,000 tutors and 250,000 learners). Once these points are deleted the correlation falls to +0.58 but remains signifiant at the 0.05 level. We believe this shows that when an institution uses the Web, even if it is for all courses, only a small part of the course is WEB based. Thus there is only a minimal impact on the tutorsÕ workloads an that this reduction is largely related to the management of students. In the same way, when tutors use computerised assessment techniques they are still actively involved in organising the assessment tasks and also in providing feed-back to students.

We further conjectured that the number of tutors was dependant on some strategic point of view at the country level. Correlations between number of students and number of tutors in each country shows that this criteria is not relevant for Italy and Portugal, but is relevant for the French institutions. In France, the number of tutors ais related to the number of students. However, this fact is probably due to the nature of our French sample which contains only university institutions, all members of a big institution which is called C.N.E.D (Centre National dÕEnseignement Ô Distance). It is possible that this correlation is due to a global politics of the C.N.E.D. concerning the number of tutors involved in distance education. Also the number of tutors enrolled in this institution is proportional to the number of students enrolled in the courses : the funds provided to these centres would be calculated referring to the enrolment of students and be revised periodically as it often happens in different universities.

In conclusion we are not able to definitively state what is determining the number of DE tutors in our sample although student numbers is important. Nevertheless, we could go on and examine two further hypotheses.

What could be the impact of the age of the institutions? When an institution is young new courses need to be produced and at this stage there will need to be significant staff resourcing

85

to undertake the task of course development. In some institutions, administrative work could be performed by non-academic staff, that is not by tutors, and these staff will not have been captured in the staff of our sample. Unfortunately, these two hypotheses cannot be evaluated with the data collected in our questionnaire.

TutorsÕ training and expertise

Here we ask what specific pedagogic training have tutors undergone (see Graphic 4). We have made the assumption that each tutor is an expert of his or her own subject domain and have elected to focus on pedagogic training alone. We asked such questions as whether or not tutors had expertise in distance education or indeed in communications or in the use of new technologies in teaching?

The data show that in more than a half part of the institutions (19/36), the majority of tutors have specific training in pedagogy and instructional design. Yete, a quarter of the institutions (9/36) have no members of their staff with such training. Only in nine institution have the majority of academic staff completed specialist training in assessment and evaluation. Indeed, two thirds (22/36) of institutions have no members of the staff with such training!

Graphic 4 Comparison of types of training available in the working teams

N=36

10 9 8 9

5 4 5

22

74

9

16

85

11 12

13

1517

7

30

26

0

5

10

15

20

25

30

allmembers

of thestaff

mostmembers

of thestaff

somemembers

of thestaff

nonemembers

of thestaff

training in pedagogic andinstructional design

specif ic training inassessment andevaluation training in communicationand group dynamics

training in NTIC

computer skills

Other

86

In a third of the institutions (11/36) the majority of their teaching staff have relevant training in communication and group dynamics, and a further quarter (9/36) have some trained tutors. Just under half of the int=stitutions (16/36) have no specialists in this topic.

Specific training in NITC has been provided to the majority of staff in about a third of the institutions (13/36). Again a further twenty-five percent of institutions (9/36) Hhave some level of expertoise while a third (12/36) have no specialists in this area. Specific computer training is not widespread - nineteen out of thirty-six had at least some members of the institution staff specialized in this topic. Only one of the institutions recorded that all tutors had computer skills training, while three of the institutions stated that the majority of staff were so trained.

Other specific training mentioned (10/36) was largely related to expertise in a subject domain knowledge.

We predicted that those institutions which use computers and telematics would have a greater proportion of staff who were ICT or computer literate than the those institutions who did not use NICT for course delivery. If we inspect Graphics 4 and 5 we find that of the eighteen institutions which use computers and telematics two thirds of these institutions (11/18) consider that all, most or some of the tutors have computer training and only seven out of eighteen of these institutions state that no tutor has received this type of training.

Among the two types of institutions those using telematics and those not using telematics, the division is not completely equal concerning the computer skills of the staff:

∑ all the members are considered to have received computer training in one out of eighteen for those which use NICT and for none out of eighteen for others;

∑ most of the members are considered to have received computer training in two out of eighteen for those which use NTIC and one out of eighteen for others;

∑ some members have skills in computers and telematics usage in eight out of eighteen institutions which use NICT versus seven out of eighteen for others;

∑ no tutors have been trained in this topic in seven out of eighteen for those which use NICT and ten out of eighteen for those which do not.

87

Graphic 5Teacher's training in institution which use telematics

N=18

5 5 533 3 3

9

42

57

6

2

64

12

87

21 0

15

02468

10121416

allmembers

of thestaff

mostmembers

of thestaff

somemembers

of thestaff

nonemembers

of thestaff

training in pedagogic andinstructional designspecific training in assessmentand evaluationtraining in communication andgroup dynamicstraining in NTIC

computer skills

Other

Similar comments can be made about skills in NITC. In our global sample, a third of the institutions (12/36) state that no tutors have NITC skills whereas only four out of eighteen institutions which use computers and/or telematics, record such a position.

In institutions which have at least part of thieir courses computerised or based on telematic communication, the tutors have significantly more training in these two domains than in traditional DE institutions.

Additional data shows that nineteen out of thirty-six institutions do not have a core of skills which the majority of tutors share. There is a bi-modal distribution here. The first category contains the institutions (17 out of 36) who require all tutors to develop a body of core skills, such as skills in assessment annd evaluation. The second group of institutions do have specific pedagogic skills requirements.

We also observed that nine out of thirty-six institutions have more than one type of skills shared by all the tutors.

Graphic 6 provides information about the sum of competencies available in each institution. The minimum level of competencies is twenty-four on the graphic This means that none of the tutors have skills described above in one of the 6 skills (6 * response 4 at question 6). The maximum is 6 (6 * response 1 at question 4) which means that all the tutors have training in the 6 domains.

88

Graphic 6Sum of teachers' competencies

0

5

10

15

20

25

The data show that seven institutions have the minimum level of competencies (score=24). In these institutions, we suggest that priority is given to domain expertise. The countries which seem to be disadvantaged on this field are Sweden (average of 24), UK (average of 21.25) and France (average of 20.37). The countries which have the largest amount of specialised trainers are Belgium (16.75), Portugal (16.1) and Italy (16.8), but there are some individual differences that influence the average score of these countries.

Graphic 7 shows that only 14 out of 36 institutions record that they have an expert in evaluation and assessment. In some cases evaluation matybe provided by an external body, so that there is no specific need for such expertise.

89

Graphic 7Number of institutions which have at least an expert in...

N=36

27

1420

2419

10

05

1015202530

training inpedagogic andinstructional

desing

specifictraining in

assessmentand evaluation

training incommunication

and groupdynamic

training inNTIC

computer skills Other

If an institution attaches importance to a specific skill (for instance NITC or assessment), all members or most members of the teaching staff will be trained in these skills or will have mastered skills through day to day usages. Otherwise, if an institution attaches no importance to a specific skill, the presence of this skill in the teaching team depends on the previous tutorí’s training, or on a strategic choice the individual tutor has made once they have been enrolled in the institution. It could be that skills in assessment are less attractive (and, unfortunately, possibly less valorized) than computer sciences.

Curriculum in the distance education institutions

a) Disciplines

The distribution of disciplines taught in the thirty-six DE institutions is shown in the table below.

The most active ODE disciplines are business, language, mathematics and statistics, the natural and physical sciences, computer sciences, and literature. Topics focused on medicine, engineering, and education are less represented in the ODE institutions. Medicine courses are provided in just thirteen out of thirty-six institutions. These data contrast sharply with the data from the tutor questionnaire.

90

Disciplines N Proportion

Medicine 13 .28

Arts 15 .41

Engineering 15 .41

Education 18 .5

Humanities 20 .55

Social sciences 23 .65

Other 20 .55

Literature 22 .61

Computers sciences 23 .65

Natural & physical sciences 23 .65

Maths and Statistics 24 .66.Language 25 .69

Business 27 .75

b) Learning modalities linked to disciplines

Our sample contains only institutions which are providing at least some courses through distance education. These institutions must have developed or are in the process of developing structures which can be used to support distance education, even if those structures are only in an embryonic stage of development..

We now investigate the relationship between disciplines and mode of delivery, whether by distance or presential methods (Graphic 8). We argue that if the percentage of presential courses related to a topic is almost equal to the percentage of distance courses, this mean that :∑ this topic is considered by the institution to be as easy to teach or to learn in a presential

way (pedagogical aspect) as in a distance one;∑ this topic requires some structures which are difficult to implement at a distance

(infrastructure aspect),∑ the institution is a traditional teaching institution which has now decided to develop some

91

courses at a distance.

We have no discipline with a higher incidence of presential courses than of distance education courses. The two disciplines with a similar frequencies in both categories are those of Arts and Medicine each with six presential as against seven distance courses. Medicine is the single discipline which is taught either in a presential or in a distance way but never in a presential and in a distance way. In all the cases where medicine is taught at distance, at least seven others disciplines are taught at distance in those institutions. Arts courses at a distance are also confined institutions which provide at least seven other disciplines though the ODE mode. So, Medicine and Arts are provided in a distance way only in institutions which have a developed infrastructure supporting distance education.

Graphic 8 Presential, distance and both presential and distance learning

N=36

7

5

6

3

6

6

6

8

6

5

7

7

4

11

11

13

7

9

7

7

12

18

8

11

15

11

4

7

6

8

5

2

0

4

3

2

5

1

5

0 2 4 6 8 10 12 14 16 18

litterature

social sciences

languageeducation

humanitiesarts

medicine

math and statbusiness

engineeringcomputer science

natural and physical science

other

both

distance

presential

Organisation of the courses

What aids the role of face-to-face teaching in ODE courses? Do the learners meet other learners or tutors during a distance education course and if so, what are the goals of these meetings?

92

a) Meetings

We observe that about half of the institutions (16/36) say that the students meet together as a whole group in all courses, twelve out of thirty-six say that they meet together for part of the courses and eight say they never meet together (Graphic 9). Learners meet the tutor in a group in twenty-two out of thirty-six for all courses, eleven for part of the courses and three out of thirty-six never meet together. While learners meet the tutor on and individual basis in nine out of thirty-six for all courses, fifteen out of thirty-six for parts of the courses. In twelve out of thirty-six institutions students never meet the tutor on a one-to-one basis.

In two-thirds of the institutions (25 out of 36), learners meet someone in all courses and that generally (24 out of 25) at least the tutor is involved in these meetings. The number of institutions that organise meetings rises to thirty-five out of thirty-six if we also consider the meetings organised just for part of the courses.

The only institution that states that it never organises meeting is Germany 1. In conclusion, it would appear that live meetings remain an integral part of distance education courses.

Graphic 9 Number of institutions in which the learners meet together or meet

the teacher N=36

16

22

9

12 11

15

8

3

12

0

5

10

15

20

25

Learnersmeet as a

w holegroup

Learnersmeet the

teacher in agroup

Learnersmeet theteacher

individually

for all courses

for part of the courses

never

b) Goals of Meetings

93

What are the aims of these meetings? Graphic 10 shows that proportionally the most mentioned goals of such meetings are : ∑ sharing of knowledge, ∑ giving the learners social support, ∑ practical work, and re-focusing on the topics. ∑ For individual tutor and tutee meetings, completing practical work is not seen as an

important goal.

The goals articulated here would be difficult to achieve at a distance although there is increasing evidence of the use of computer conferencing for social support of students.

Let us look more closely at the stated goals.

Sharing knowledge interactively with the whole group in real time is difficult but not impossible to do at a distance. The FORUM and the NESTOR software (developed in Belgium 4) and of the LEARN-NETT project (Belgium 2) which permit the sharing of information and support co-operative working. FIRSTCLASS is widely used in the UK an is the standard program for the Open University. Such tools may eventually reduce the need for face-to-face meetings. At the moment interactions between learners, when they exist at distance are mostly done by e-mail.

Practical work seems almost impossible to support at a distance. Video-conferencing techniques may aid practical work in a distance way, in the future

Re-focusing on the topics and providing social support are more easily accommodated at a distance. But we cannot deny the impact of interacting directly with the whole group. For example, by solving socio-cognitive conflicts between pairs, one could re-focus on the topics, go deeper, or by sharing feelings on the course with pairs, one could profit of social support.

Assessment does appears difficult at a distance. For formative and individual assessment it is possible to construct such assessments but it is time consuming. It more normal for tutors to send materials to students, await the reply, and then correct work and provide feed-back. Automation of this process is possible but involves the construction of test item banks, the development of adequate feedback mechanisms and of management software. In Belgium 4, Web quizzes are available to students to self-assess their knowledge and to familiarise them with the evaluation techniques such multiple choice questions that they will encounter during

94

the examinations (cf. Boxus et al, 1990). This mode of evaluation demands specific competencies. For assessment which leads to a certification, it is almost impossible. As far as formal assessment is concerned, we face the additional problems of security and plagiarism.

In conclusion, we believe that giving part of the course in a presential way helps the tutors. On the one hand, they gain a lot of time and decrease the number of distance interactions with the learners, and on the second hand, it decreases their need for new materials.

Graphic 10 When there is a meeting, what are the aims of the meeting ?

4641

30

19

35

24

3238

148

53

11

19

0

24

32

8

32

43

30

41 41

30

118

3

05

101520253035404550

Learnersmeet as a

w holegroup

Learnersmeet the

teacher in agroup

Learnersmeet theteacher

individually

share of knolw edge

assessment

practical w orks

field w ork

class

lecture

refocusing on the topics

social support

other (netw orking)

Media in distance education

Media supporting the DL process

The media used to support the DL process can be classified into two groups :∑ Communication support.∑ Learning support.

The first group includes mail, fax, e-mail and video-conferencing. The second includes written materials, audio and video tapes, TV and radio programmes, CD-ROM, computer software and the Internet.

a) Communication supports

The classical technologies do permit interaction between learners and tutors but new

95

%

technologies increase the speed and the facility of these interactions.

Are the institutions using these new technologies ? Graphic 11 shows that there no institution uses always fax-based communication, but there are seven institutions out of thirty-six which always use e-mail. What is the relationship between fax and e-mail. The data show that a small number of institutions (4/11) often use fax but never use e-mail. They can be described as the ;classical interaction categoryÕ. A second group always use the e-mail facilities and often use the fax (4/7). It seems that the institutions of that category prefer e-mail but provide fax facilities for learners who do not have a computer and modem facilities.

Video-conferencing is a special case because it is a very young technology and is only now starting to be used. Whether institutions use it often (4/36) or rarely (14/36), the use is experimental and evaluative rather than embedded in the culture of the course.

Graphic 11 Number of institutions which use follow ing media always, often, seldom

and never N=36

0 2 4 6 8 10 12 14 16 18

Fax

E-mail

Video-conférence

Internet

never

seldom

often

alw ays

b) Learning support Again a dichotomy is apparent, with the classic technologies such as video and audio tapes, TV and radio programmes supporting one group of courses and a reliance on traditional written material such as manuals, syllabus and notes for a second group. The traditional written materials may have some level of interactivity but are single media presentations while the classic technologies allow multimedia but do not enhance interactivity.

The new technologies permit both interactivity and multimedia presentation of material. Do

96

the ODE institutions really used these enhanced facilities, however? ?

Classical Technologies versus New Technologies. Graph 11 shows that few institutions use the new technologies for all courses. Some technologies are rarely if ever used by a large number of the institutions. Thus the majority of institutions rarely if ever use radio (27/36), TV (23/36), computer software (21/36), and CD-ROM (17/36) to support or deliver their courses. Written materials are provided by the majority (27/36) of the institutions. The most interesting finding is that the number of institutions which make extensive use of classical technologies (radio etc.) is no greater than the number of those which use new technologies. Paper-based course predominate at all levels.

What are the relationships between classical technologies and the new ones? We predicted that since new technologies allow a better interactivity and provide a multimedia approach, there will be a transition from the traditional technologies to the new ones. Thus the data should show that:∑ a small number of institutions preserve an approach based only on technical traditional

media, ∑ a great number of institutions use both (technical traditional and new media) in order to

facilitate the transition from old to new technologies,∑ a small number of institutions use only new media because transition has already been

made.

Our predictions match the data. Effectively, among twenty-nine institutions which use new hypermedia technologies, twenty-five also use traditional media such as audio and video tapes, radio and TV programmes. Only four out of twenty-nine institutions which use new media do not use classical ones. At the same time, out of thirty institutions which use classical media only six do not use new ones. Institutions who were traditionally involved in a technical approach to course delivery appear well disposed to new media. We cannot generalises these results as the sample is not large.

Traditional written material versus new technologies. We predicted that new technologies will replace the traditional written material for ODE courses. But the large number of institutions (27 out of 36) using written material for all courses compared to the small number of institutions using always new media shows that this is not so. There are institutions and courses that are making this change. Of the nine institutions for which written material is not the dominant medium , seven are active users of new media. Among these seven institutions, three are always use CD-ROM (so all the institutions which use always CD-ROM belong to this category). So it could indicate that when the importance of the written material decreases, that of the new media increases.

What are the problems associated with the use of new technologies for teaching and learning?

We have argued that the new media offer advantages compared to the traditional ones. At the

97

same time, the use of multimedia still stays at a low level. So the question is what could be the difficulties linked to the use of new technologies compared to traditional ones? We will consider this question under three aspects : the institutionÕs point of view, the tutorÕs point of view and the learnerÕs point of view.

a) The institution point of view

A first difficulty could be linked to the production of the learning material. Most of tutors know how to record audio or video tapes but few have the technical skills to produce CD-ROM. Are the tutors able to do such a task? Has the institution a specific department to design this kind of new tools ? In terms of Ôjust in timeÕ teaching, it could be interesting to answer these questions. Effectively, if the tutors are not able to produce or to change such materials, they have then to ask an external institution to do the work and it could be too time consuming and costly to be considered worthwhile.

Graph 12% of media "produced" inside the institution which use it

81

69

74

47

57

66

51

72

63

0 10 20 30 40 50 60 70 80 90

Written material

Audio tapes

video tapes

tv programmes

radio programmes

cd-rom

computer software

Video-conférence

Internet

Graphic 12 shows that quantity of learning materials supported by new technologies produced outside of the institution ranges from thirty-four to forty-nine per cent. This is also true for TV and radio materials. However, video and audio tapes are different. They are more likely to be produced in-house. Only twenty-six percent of video and twenty-one per cent of TV programmes are produced outside the institutions.

While video is linear and therefore matches traditional teaching, CD ROM material does not have to be linear. This presents a very different way of thinking about information for many tutors and is therefore an additional problem . This may be a partial explanation for the need for external expertise were new media are concerned.

Our comparison between traditional and new media shows that:∑ The quantity of media produced inside the institution is varies whether it is for traditional

multimedia and new multimedia.

98

∑ Written material is easier to produce within the institutions than multi-media.

b) The tutorÕs usability point of view

A second difficulty in implementing new media revolves round the problem of modifying the content. Can the content be modified, and is this easily achieved by the tutor alone or will he or she require help form within or outside of the institution ? This is critically important. If tutors can not develop or modify material easily then they are likely to become frustrated and reject the use of this medium.

Graphic 13 shows that learning content supported by audio and video tapes is easier to modify than those supported by CD-ROM, computer software, TV or radio programmes. It could be due to the more difficult technologies that these media require or to the properties of CD-ROM (which cannot be overwritten). The Internet, on the other hand, is seen as easier to change than all of the traditional technologies except for written materials. It seems to show that the second hypothesis is confirmed. Another interesting topic is the fact that e-mail is easier to modify than the fax. The reason could be that in an e-mail, the production tools are the same as the sending tools.

Graph 13Number of institutions thinking that the following media (they use)

are easy, difficult, or impossible to change

1655

123

27

118

12

1212

143

38

72

43

7

49

79

38

612

68

7

0 5 10 15 20 25 30 35

Written material (N=32)

Audio tapes (N=26)video tapes (N=26)

tv programmes (N=13)

radio programmes (N=8)cd-rom (N=19)

computer software (N=15)

Fax (N=21)E-mail (N=21)

Video-conférence (N=19)

Internet (N=26)

no change

diff icult change

easy change

c) The learnerÕs usability point of view

With the traditional distance learning systems, students only need a pen, sometimes a video

99

or an audio recorder and some stamps or a fax machine. Now the new technologies demand a multimedia computer, a modem, maybe a video-conferencing system. Not everybody has yet such type of a system. Does that fact has an impact on the presence of the learners in the institution ? In other words, are the students more present in the institutions which use new media ?

Graph 14 shows that this is the case. With traditional media, only one institution commented that the presence of the learner is not always necessary and this independently of the media being used. On the one hand, eight out of nine institutions using radio programmes and ten of thirteen using TV programmes say that the learners are never present in the institution when using them. These programmes are available remotely. For new media, the survey shows that thirteen out of nineteen of the institutions which use CD-ROM say that the learners are never present when using this media. A similar pattern emerges for other new media.

There are a few cases were the presence of the learners is required and this is generally linked to video-conferencing. This is a predictable result given the specialist nature of the equipment, the overall expense and the need for an expert to support what is still a novel technology.

In conclusion, we would argue that the most important factor influencing the use of NICT is access to the equipment for the learner. In addition institution must recognise that they will need to provide time and money re-skill tutors if their goal is to replace written material by new multimedia and interactive technology..

Graphic 14 Number of institution which require the presence of the learners

when using following media

100000

100

50

223

00

22

22

33

63

73

142

33

17

2321

1710

813

101616

1016

0 5 10 15 20 25 30 35

Written material (N=32)

video tapes (N=27)

radio programmes (N=9)

computer software (N=15)

E-mail (N=21)

Internet (N=26)

learners alw ays present learners present most of the time Learners seldom present Learners never present

Assessment

100

In this section, we will focus on the assessment aspects. We will try to respond to questions such as “Who is assessed ?”, “Who are the evaluators ?”, “What kind of assessment is undertaken?” and “What evaluation tools are available to tutors?”

Who or what is assessed?

Assessment can be at many different levels and for a variety of purposes. One can assess learners, tutors, the didactic material and, of course, the costs and benefits of the situation. What kinds of assessment are conducted in our target institutions?

a) Assessment of the learners. Graphic 15 shows that thirty out of thirty-six institutions assess learners during the year. Twelve out of thirty conduct such assessments more than once a year.

b) Assessment of the tutors. Two thirds of the institutions (23/36) assess the tutors during the year. It seems that strategy differs from one country for another. In all the countries (except France) almost all the institution evaluate teachers. But in France there is only one institution out of seven which make such an evaluation

c) Assessment of the didactic material. We observed that twenty-seven out of thirty-six institutions assess at least a part of their didactic material. We would expect that an institution evaluates the use of new didactic material or modes of delivery such as video-conferencing, because there is a need for validation of the new products. This assumption is not supported (see table below). The number of institutions which assess the adequacy of the medium used is no larger for those using new media than for those using more traditional course delivery.

Graphique 15 Number of institutions which assess students, tutors, learning

materials and cost benefit N=3630

2327

11

6

139

25

0

5

10

15

20

25

30

assessmentof student

assessmentof tutors

assessmentof didacticmaterial

cost-benefitanalysis

Yes

No

101

Assessment of the didactic material

MediaN of

institutions using the medium

N of evaluatio

ns

Proportion

Written material 32 17 .53

Audio tapes 26 10 .38

Video tapes 26 11 .42

Tv programmes 13 5 .38

Radio programmes

9 4 .44

CD ROM 19 9 .47

Computer software

15 6 .4

Fax 21 5 .23

E-mail 21 11 .52

Video-confŽrence

19 6 .31

Internet 26 11 .42

d) Cost-benefit analysis

102

We notice that, only eleven institutions out of thirty-six are conducting a cost-benefit analysis. It seems we are facing a cultural difference: four institutions out of eleven that perform cost-benefit analyses are Italian and three are Spanish. This means that all of the Spanish institutions of our sample performs cost-benefit analysis.

In addition two of the thirty-six institutions do not provide any evaluation and they appear to focus on the single goal of transmission of learning materials. While eight institutions, four out of six Italian institutions and the 3 Spanish institution, assess all the courses.

Who conducts the assessment?

We predicted two general scenarios. On the one hand, assessment could be made inside the institution while on the other hand, it could be made by an external body such as a client company, a national authority or an external expert institution.

Graphic 16 shows that in most cases assessment is conducted by the tutor (28/34). This result should be viewed in conjunction with the finding that twenty-two institutions out of thirty-four do not have a recognised expert in assessment.

Graphic 16 Number of institutions in which at least part of the courses are assessed by...

N=3428

42 1

4

9

13

00

5

10

15

20

25

30

individual tutor a central officein the institution

The clientcompany

a nationalauthority

an externalexpert/institutio

n

computerizedevaluation

student self-correction

other

We also observe that nine out of thirty-four institutions use computerised assessment. While in thirteen institutions out of thirty-four, students are involved in self-assessment. It is important to note that such assessment offers formative feed-back . This topic will be

103

analysed further.

In summary most of the assessment takes place within the institution with limited recourse to outside expertise.

What are the aims of the assessment ?

Assessment procedures may meet many different goals. They may be:∑ formative assessment to give the learners feedback on their progress; ∑ summative assessment as in a final examination;∑ self-assessment to develop the learnerÕs meta-cognitive process; ∑ informal or formal assessment.

Graphic 17 shows that three institutions in four (27/34) conduct formative assessment . We offer two reasons for this. Firstly, tutors lacking daily interaction with the learners, may feel the need to compensate for the day-to-day informal feed-back possible in presential learning mode. Tutors are forced to provide feed-back to the learners to maintain motivation and to provide direction in the learning process.

Graphic 17 When there is an assessment, it could aims at...

for x numbers of institutionsN=34

27

22

1821

0

5

10

15

20

25

30

assessmentaims at

formativeevaluation

assessmentaims at

summative

assessmentaims at self-assessment

assessmentaims atformal

qualif ication

Secondly, this may be due to an over-eagerness of management. We note that of the twenty-

104

seven institutions conducting formative evaluation, three do not provide feedback to the students! We can suppose that this inconsistency is due to a perceived institutional need to monitor courses rather than to support learners.

Summative assessment is conducted in two thirds of the institutions (22/34). The link between these two assessment modes is strong, twenty-one institutions conducting both modes of assessment and evaluation. Only one institution conducts summative assessment without formative assessment.

Half of the institutions have a goal of learner self-assessment (18/34). We also find that more than the half of the institutions (21/34) provide assessment directed to a formal qualification. It is important to note that in order to provide a formal qualification, an institution needs to be recognised at a national level.

How many institutions evaluate the knowledge acquired through different media ?

Here we investigated the relationship between mode of delivery and the focus of assessment. For instance, if an institution provides written documents and videotapes for a course but assesses only the information transmitted by the written material, we can assume that learning materials transmitted by the videotapes are only additional documentation.

MediaNumber of

institutions using the medium

Number of evaluations

Proportion

Written material 32 27 .84

CD ROM 19 9 .47

Audio tapes 26 12 .46

TV programmes 13 6 .46

Video tapesCD ROM

26 109

.38

.46E-mail 21 8 .38

InternetE-mailInternet

26 989

.34

.33

.36Computer software

15 5 .33

105

Radio programmesFax

9 33

.33

.15

Video-conference

19 4 .21

FaxE-mail 21

38

.14

.33

The table shows the relation between institutions which make an evaluation on the knowledge acquired using a specific type of media versus the number of institutions which use that media.

The table shows that knowledge transmitted by written material is the most often assessed (.84). Where TV, audio tapes or CD-ROM are is used, transmitted information is also assessed in fifty per cent of cases. This declines for the case of computer software, for the Internet and for e-mail where less than a third of the material is assessed. Tutors may be avoiding disadvantaging learners who do not have access to technology or they may see such materials as peripheral to the main body of knowledge being transmitted.

Are the institutions giving feed-back to the learners and, if yes, what type of feed-back?

Out of the thirty-four institutions which assess learners, twenty-five institutions provide feed-back most of the time to learners (Graphic 18). However, eight institutions never provide feed-back. In these cases, the learners build their knowledge without having external confirmation of their progress. Such feed-back to the learners, may be general or specific.

106

Graphic 18 Number of institutions which give feed-backs to the students and

frequency of these feed-backs N=37

21

43

8

0

5

10

15

20

25

alw ays most of the time sometimes never

Graphic 19 On which topics are the feed-backs given ?

10,0

7

9,0

77,0 7

0,0

1,0

2,03,0

4,0

5,0

6,0

7,08,0

9,0

10,0

Feed-backs dealw ith a globalassessment

Feed-backs dealsw ith a specif ic

focused assessment

Score on an exam

appraisal

diagnostic test

Graphic 19 shows that the ten institutions out of sixteen who provide general feed-back, do so on the basis of examination scores, while nine provide such feed-back based on an appraisal and seven based on diagnostic test. Of course providing feed-backs on the basis of scores on for example examinations does not preclude feed-back based on other criteria.

The institutions for which feed-back tends to be specific in nature provide feed-back on the basis of score at the examinations (7/14), diagnostic tests (7/14) or on the basis of appraisals

107

(7/14). Feed-back dealing with specific focused assessment is based less on the appraisal (7/14) than feedback dealing with general assessment (9/16). The latter feed-back is also more likely to be based on examination scores (10/16) than feed back dealing with specific focused assessment (7/14).

What types of assessment are used in ODE institutions?

In the questionnaire we have considered different types of assessment and sometimes different types of assessors (the tutor, the student or peers, the computer). We have also considered the fact that the type of assessment procedure could differ if the aims of assessment were formative or summative.

The following table highlights the relationship between type of assessment employed and the assessment procedure.

ALL COURSES

PARTS OF THE

COURSES

NEVER TOTAL

ADurin

gA

BEndB

CDuring

C

DEndD

EDuring

E

FendF

G(A+C)

G

H(B+D)

H

(G+H)(G+H)

Multiple choice question marked by tutor

6 6 12 7 14 19 18 13 31

Multiple choice questions marked by student or peers

1 2 10 2 21 28 11 4 15

Multiple choice questions marked by computer

7 6 7 3 18 23 14 9 23

Comprehension tasks marked by tutors

7 6 11 8 14 16 18 14 32

Comprehension tasks marked by student or peers

3 1 4 2 25 29 7 3 10

Problem solving test marked by tutors

5 6 7 5 20 19 12 13 25

108

Problem solving test marked by student, peers

2 1 6 1 24 30 8 2 10

Course essays 7 4 8 7 17 21 15 11 27Practical test 3 2 8 7 21 23 11 9 20Project file or diary 2 2 6 7 24 23 8 9 17Oral presentation or examination

4 6 4 5 24 21 8 11 19

Long essay or dissertation

3 3 4 9 25 21 7 11 18

Written examination 6 12 7 6 19 14 13 18 31TOTAL 150 127 277

Thirty-two out of thirty-six institutions answered this question for both formative and summative evaluation giving 64 responses in total. The data show that the three preferred tasks were:∑ multiple choice questions marked by tutor (31 out of 64 possible response.)∑ -written examination (31 out of 64) and ∑ comprehension task marked by tutors (32 out of 64).

The four procedures the least used were ∑ comprehension tasks marked by student or peers (10 out of 64), ∑ problem solving tests marked by student, peers (10 out of 64), ∑ multiple choice questions marked by student or peers (15 out of 64).∑ and the project file or diary (17 out of 64).

These limited choices are reinforced by the fact that the majority of institutions which use these evaluation procedures often use them for only part of a course (38 out of 52). Note the low number of procedures marked by students. Where studentsÕ self-correction occurs it is for formative assessment only (25 cases out of 35). Summative assessment is the prerogative of tutors and, in a few cases, computers.

Other results shows that some evaluation procedure fit better with the formative assessment than the summative one. This is the case for MCQs, comprehension tasks and for problem solving tests. Whereas oral presentations, examination, long essay or dissertation are associated with summative assessment.

The frequent choice of the written examination and of oral examination for summative assessment should be reinforced by the fact in twelve cases out of eighteen for written examinations and in six cases out of eleven for oral examination. The institutions which say that they use this kind of assessment also say that this is for all of their courses. It is different for long essay or dissertation for which only three institutions out of twelve which use this

109

type of assessment for summative evaluation, say that this is for all courses.

Conclusions and perspectives

The sample of ODE institutions analysed in this chapter is not very large and cannot be considered as representative of ODE practices in Europe but the data do act as indicators. We observed that ODE institutions vary in size and structures. The number of courses, tutors, and learners differ from one institution to another and there appears to be no simple pattern to explain those variations. For instance, the number of tutors working in a DL institution does not depend on the number of learners enrolled or on the number of DL courses provided .

As the typology described above shows, ODE providers do not always provide only distance courses. Many disciplines are taught in ODE institutions studied here. Some of them seem to fit more comfortably with ODE than others

The expertise of the tutors in other fields than their disciplines varies. They may have been trained in pedagogy and instructional design, but are generally not experts in evaluation. Some of them are specialists in communication and group dynamics, and have computer skills. The fact that institutions use or not computers and/or telematics influence a bit the tutors type of training

Meetings are generally organised to support DL courses. The learners meet together and meet the teacher often in a group session and sometimes individually. The meeting goals are mostly focused on the share of knowledge, the social support of the learners and practical work.

Learning and communication support are varied: mail, fax, e-mail, video-conference, written material, audio and video tapes, CD-ROM, the Internet, are all used. The use of NICT is patchy and will remain so until:∑ the majority of learners have access to relevant equipment∑ ODE centres have the structure to support the use of such technologies. They should

invest not only in equipment but also in the tutorsÕ training or in new staff qualified in NICT.

Assessment of learners is the norm and in many cases the didactic materials and the tutors are also evaluated. In a few cases cost-benefit analyses have been undertaken but these one seem to be confined to Italy and Spain in our sample. Generally, the tutors are assess the learners although computerised assessment occurs in few institutions. The assessment procedure is still based on traditional procedures including oral and written examinations, and multiple choice questions. Procedures and tools of self assessment are used sparingly.

110

Formative evaluation is widely practised and ODE tutors provide feedback to the learners, but not frequently and not in all cases.

The development of effective usable tools to help evaluation in ODE is necessary. This theme will be taken up in Chapter 10 of this text.

References

Boxus, L. (1990). Principes communs pour évaluer les résultats cognitifs d’une formation. Eurotechnet. Bruxelles

Alves,ÊC.C. (1994). Interactive systems in education: learning from past mistakes. Eurotechnet. Liège. 1994

Leclercq, D. & Gilles, J.-L. (1994). GUESS, un logiciel pour entraîner à l'auto-estimation de sa compétence cognitive. Actes du colloque QCM et questionnaires fermés, Paris: ESIEE.

Bosmans, J., Boxus, L., Denis, B et al (1996). Manuel d’utilisation du logiciel wincheck. STE-ULG. Liège

Jans, V., Poumay, M. & Baldewyns, L. (1996), How to animate a group of students in an electronic classroom ?, Placing the Student at the Centre. Current Implementations of Student-Centered Education, Maastricht University, 20th Anniversary Conference, Maastricht, Nov. 1996, 76-84.

Zeiliger, R., Reggers, T., Baldewijns, L. & Jans, V. (1997). Facilitating Web Navigagtion : Integrated Tools for Active and Cooperative Learners. ICCE’97 Kuching. Malaysia

111