Mediating the message: television programme design and students' understanding

Instructional Science 20:3-23 (1991) 3 © Kluwer Academic Publishers, Dordrecht - Printed in the Nethedands

Mediating the message: television programme design and students' understanding

DI ANA L A U R I L L A R D Institute of Educational Technology, The Open University, Walton Hall, Milton Keynes, MK7 6AA, U.K.

Abstract . The research reported represents a non-standard approach to basic research on educational television, in that it uses qualitative methods to describe learning and to illuminate which programme features are important. It was based on observation, interview and test data from five groups of 20-30 Open University students, each watching a social science television programme as part of their course. A qualitative analysis of students' summaries of the programmes showed that only half achieved the intended learning outcomes. A further analysis of the students' responses to key extracts showed that this is more likely to be because of programme structure than because of presentational quality. The research design and results are related to similar studies ~n students learning from text, and contrasted with the methodology adopted in other studies of the effects of educational television.

1. Introduction

There are now many thousands of hours of educational television, both broadcast and video, generated for use in schools, colleges, universities and in training. By contrast, there are relatively few studies relating to how to teach through this medium. Producers of educational television and video have little help from

research in deciding how best to present their material to make it maximally effective for their learners. They have to rely on their professional competence in making good television. But educational programmes have a more explicit objective than general programmes have. They are usually aiming to convey a particular message to the audience, and are meant to integrate with an existing curriculum by enhancing certain special areas of it. Producers in this field do not

have the luxury available to the artist of saying 'make of it what you will'. Teachers and trainers using educational television in their courses expect students to make of it what they themselves would. So it is important that educational television producers enhance their skills with an understanding of how students learn from television. The research study reported here attempts to increase that understanding.

One of the main strands of research in educational television has concentrated on comparative studies, such as whether television is as effective as other teaching methods. The consensus view - that it is successful "... when the program content is well organized, meets high pedagogical standards, and is pre- sented in a learning context" (Ide, 1974, p. 336) - is not a surprising result. These

4

studies may be useful to the teacher who wants to know whether television could possibly be effective, but do not help the producer who wants to know how to do it. In his survey of this research area, Allen concludes:

"That students learn from televised teaching cannot be doubted, but the condi- tions under which such learning takes place and the specific characteristics of televised presentations that bring this about are yet to be determined, and most research ignored such questions." (Allen, 1971)

Later studies focused on general features of the medium as they related to the processes involved in learning, such as the role of the 'zoom' in supplanting men- tal processes, or the role of production techniques such as incongruity in gaining attention (Salomon, 1979), or the relation between programme features and recall of specific items of content (e.g., Boeckmann, Nessmann and Petermandl, 1988). These are interesting general principles that do explore how the characteristics of television can bring about learning.

These general production principles do not relate to the nature of the content being learned, however. Especially in an educational context, a producer must focus on how the message is communicated and whether it is understood, not simply recalled. Television is a medium that gives students a specific and peculiar kind of access to knowledge. What form does that knowledge lake when it is accessed in this particular way? How do the forms of access, namely the way a producer presents the material, relate to the content of learning? We need an understanding of the interaction between the student and the particular subject matter as mediated by the programme.

A methodology that comes closer to this kind of aim is practised in the related field of research on learning from textual media. Several studies have demon- strated the phenomenon that, for students reading the same educational text, there are major qualitative differences in learning outcomes in terms of their match with the structure of the original text. (Matron and Saljo, 1976; Marton and Wenestam, 1978; Svensson, 1977; and in the context of the subjects of this study, Beaty, 1987). Similar qualitative differences in learning outcome can also be found in learning tasks such as problem solving (Laurillard, 1984) and lectures (Hodgson, 1984). It seemed likely that such a robust phenomenon would also occur in learning from television, and a pilot study confirmed that there are, indeed major qualitative differences in learning outcome for students watching the same TV programmes.

With this background, the current study set out to (a) establish appropriate descriptions for these qualitative differences and (b) investigate how they are constituted. The stages in the research procedure are described below.

5

2. The research strategy

2.1 Background - selection of materials and subjects

The subjects of the study were 126 Open University undergraduates, registered for the foundation-level social science course D102. Like most of the university's courses, this course is largely text-based, self-paced and studied off-campus (for a general description of this form of teaching, see Kaye, 1973). Face-to-face tuition is provided at residential schools and in evening tutorials at study centres.

D102 students typically watch a 25-minute TV programme each week as part of a 10-12 hour weekly study schedule. The programmes are designed as an inte- gral part of the course, and related to the other course materials via a 'TV Study Guide' for each programme. Students watch them, therefore, with the expectation of learning something useful for their understanding of the course.

It is very important in a study of this kind, that sets out to describe the characteristics of learning through a particular medium, that the materials used should be very high quality. Otherwise the findings will tell us more about the incompe- tence of the designer than the generalisable aspects of the medium. Accordingly, this study is based on a comparison of five highly-rated television programmes from the D102 course. Earlier questionnaire-based evaluation studies had shown that (a) these television programmes were the second highest rated among all the Open University foundation courses for viewing rate, helpfulness and relevance (Grundin, 1983), and Co) for 87% of the students they added something to the course by either reinforcing or clarifying the ideas in the text (Laurillard, 1983).

2.2 Data collection

The logistics of this kind of research are extremely complex in an Open University context, because it is important not to disrupt students' normal pattern of learning. For example, the programme they were asked to watch had to be the

one they would have watched anyway that week. In order to obtain adequate numbers of student responses to a particular television programme it was necessary to make use of tutorial time, and this was negotiated with tutors and students on the c o m e . Students are required to attend a 2 hour evening tutorial at a local study centre. Each week responses were collected flom 20-30 students for each of the five programmes studied.

Each session lasted approximately one hour, and was structured as follows:

Introduction - explanation of the study, outline of session

Preparation - students given time to check text-based study guide introduc- ing the TV programme

6

TV programme - students watched programme uninterrupted

Consolidation - students given time to complete notes

Questionnaire - students asked to answer Questions 1-5 (see Appendix 1)

students asked to summarise programme (Question 6)

Extracts - S t u d e n t s watched each extract (1-2 minutes) then answered 2-part Questions 7-10

Factual quest ions- students wrote answers to factual Questions on each extract (Question 11)

Final summary - students asked to check summary and add anything they wish (Question 12).

Piloting had shown that students sometimes wanted to review their summary after going through the extracts again, so Question 12 was added to allow for this. However, in the main study this option was hardly ever taken up.

Only data from those students who were seeing the programme for the first time was used in the study.

2.3 Analysis of programme design

It is difficult to define a systematic procedure for analysing the content of a television progr.arnme. The aim was to define the intended message and content of each programme so that this could be compared with the learning outcomes achieved by the students. Thus there had to be some correspondence between the forms of the two types of description. The cognitive psychology research literature on the structure of texts provides little help, as these are generally very short, and have very simple structures, bearing little relation to the complexity of a half-hour television programme. However the pilot study had confirmed that the educational research findings on how students learn from study texts were broadly applicable to learning from television, so this was used as a starting point.

According to these studies, we can expect that a meaningful communication is likely to have a hierarchical structure. However, the logical relations between parts of that structure are not identical. Statements which are components of an argument bear a different relation to the argument than statements exemplifying it. This difference is important in TV programmes because so much of the material is illustration and exemplification. Figure 1 shows how a hierarchical structure needs to be elaborated to take account of this. The analysis therefore used repeated viewings of a programme to establish (a) the overall message(s), (-b) the internal structure of the message, e.g. relational or predicative, (c) how the message is established via component arguments, (d) how the evidence is established for each of the statements identified.

I

~Examp~A °f A~ I

EARB ~ ARB Example of ARB Main relational point

A B

Component point Component point

Figure 1. General form of logical structure of teaching message

2.4 Validation of programme message structure

The logical structure underlying each programme was translated into a verbal summary description, and given to the producer and academic responsible (here- after referred to as the 'designers') for validation. This stage was essential, as the

intended message of a TV programme is often even less explicit than it is for a piece of text. The amendments made were incorporated into the final form used

for comparison with students' responses.

Each programme was characterised as having a three-level structure: main

point; component points; examples. The verbal summary description was an

agreed description of content according to this structure. Summaries of programme

content for all five programmes are given in Appendix 2. These 'definitive' verbal descriptions of the programmes' content made it possible to compare stu-

dents' articulations of a programme with its intended message. They do not

describe what could be learned from the programmes; the intended message defines the minimum the designers hoped would be learned.

2.5 Analysis of students' summaries

For each programme, the students' summaries (answers to Question 6) were sorted into groups according to their structural characteristics, as compared with

the structure defined for the programme. Almost no students took advantage of the opportunity given in question 12 to change their summary. If a student included a statement similar to the Level 1 description (see Appendix 2 for exam- pies), this was categorised as a Level 1 response, even if other level statements were included. The level of each student's summary was recorded. The same

categories were used to analyse the match between the content of each programme extract and a student's interpretation of iL as given by their answers to Questions 7 to 10.

In addition, students' interpretations of the programme extracts were analysed qualitatively to generate further categories of description of how students interpret TV.

2.6 Validation of summaries

The complete set of student summaries and extract interpretations for each programme was given to a second judge for validation, together with the programme description, and the qualitative categories for interpretations of extracts. Each judge was asked to categorise each summary as Level 1, 2 or 3, according to the descriptions in Appendix 2, and each extract in terms of the qualitative categories (see section 3.3).

The categorisations for all five programmes achieved better than 75% agreement without discussion, i.e. more than 75% of students were assigned to identical categories by the two judges, and the final categorisations used were those agreed by both judges, with discussion.

3. The research rmdings

The study provided several types of data: a formal description of the programme content and structure, students' summaries of programme content, their interpretations of key sequences, and their factual recall of key sequences. By relating the interpretation to the intended meaning for both sequences and whole programmes, it is possible to build a picture of how students perceive and interpret educational television.

In addition, by comparing the differential relations between the intended programme content, and the students' own interpretations, over the five programmes, we can also see how televisual design features make the underlying structure of the content more or less accessible to accurate interpretation.

3.1 Students' understanding of the main point of the programme

The students' summaries were analysed for each programme by sorting them into groups of summaries of similar quality with respect to the pre-defined levels of description.

Previous studies have shown that the outcome space for learning from text and television typically includes a high-level response that correctly describes the main point, a low-level response that either misrepresents the point, or betrays

9

some structural mistake, such as focusing on an illustrative example as the main point, and one or two medium-level forms of response that only partially repre-

sent the main content. The different forms of misrepresentation tend to be related to the structure of the content. For example, using a text with a principle-example structure, Marton and Wenestam (1979) showed that a high proportion of students misrepresented the content as being about the example itself. A previous study, using television programmes with a complex, relational main point, showed that many students failed to mention this in their summary, but did cover the

constituent arguments correctly (Laurillard, 1982). Structural differences of this kind suggest that although most teaching presentations have an underlying logical structure, this is not necessarily preserved in the students' interpretations. Investigation of how this misperception of structure occurs is the focus of the following sections; what concerns us here is establishing its existence, and describing it. For a message complex enough to have a hierarchical logical struc-

ture, the complete outcome space of possible interpretations of that message includes all logical subsets of that structure. If we represent such a message (which might be a lecture, a book chapter, a TV programme etc.) in generalised form as in Figure 1, then we can expect students' summaries to consist of statements of the form

ARB e.g., Pluralism and Marxism provide alternative explanations of aspects of British society

A e.g., Pluralism can account for the decision to enter the EEC by concentrating on the management of consensus

B e.g., Marxism can account for the decision to enter the EEC by looking at economic and political structures and the exercise of power within them

E A e.g., entry was decided by a referendum

E B e.g., the move for entry was supported by capitalists needing an improved economy

or some combination of these. This is the simplest form that such a structure could take. Statements A and B themselves could in turn have this hierarchical form, and further complexity can be introduced by adding main or component points.

For some texts or programmes the underlying structure can be extremely complex, involving several levels of argument. Illustrative examples may exist for all levels of argument, but typically are only present for a subset of the statements. For a text or programme with a hierarchical structure such as Figure 1, we could expect three main 'levels' of learning outcome: statements about the main point, ARB, would be Level 1, statements about constituent arguments, A and/or B, would be Level 2, statements about examples of these, EARB, E A, or E B, would be Level 3.

10

In the present study, the proportions of students categorised on each level vary for different programmes. Table 1 shows the proportions of students who are categorised as producing summaries at Level 1, the level of the main argument, for each of the five programmes studied (see Appendix 2 for detailed descriptions). Notice that for these students none was at Level 3, as one would expect from adult learners experienced in learning from TV. However, some of the percentages of students achieving Level 1 are quite low, being around 50%. This is a normal figure for studies of learning outcomes from reading text, although figures vary a great deal between texts and between groups of students. The present study looked at programmes from one course, and used similar student groups, but there is nevertheless considerable variation in the scores for each programme, from at least 80% of students' summaries at Level 1 for TV17 and TV19, to around only 50% for the other three programmes. Thus for some programmes, only half the students are achieving the intended learning outcome.

Table 1. Comparison of proportion of student summaries at different structural levels for five TV programmes

Television Structural levels % of student suture- programme within programmes aries at each level

TV 14: Pictures Level 1: Main point 46 of Politics Level 2: Component points of Level 1 54

Level 3: Examples illustrating Level 2 0

TV15: Fairy-tale Level 1: Main point 38 Democracy? Level 2: Component points of Level 1 62


TV16: The Real Level 1: Main point 52 World Level 2: Component points of Level I 48


TV17: Force Level I : Main point 88 and Violence Level 2: Component points of Level 1 12


TV19: Social Level 1: Main point 80 Integradon Level 2: Component points of Level 1 20


11

3.2 Students' interpretations of key sequences

To analyse how these differences in outcome are constituted, we begin by looking at students' performance on interpreting the programme extracts. These were short 1-2 minute sequences, chosen because they contained information, either visual or verbal, necessary for an understanding of the main point. The content of each sequence was analysed by inspection, and defined in terms of the content categories generated for the whole programme. For each sequence, the researcher

has to decide whether it covers the main point explicitly, a constituent point, an iUustration, or some combination of these. Students' descriptions of the sequence are analysed in terms of these categories.

For the 20 extracts (4 for each of 5 programmes), the amount of misinterpretation varies widely, from 0% of the students misinterpreting one extract, to a high of 40% on another, with an average of 22%. Misinterpretation is here defined as a response that contains none of the defined content categories. In some cases the students may describe the sequence in a quite imaginative, and compatible way, but if they do not articulate the same points as those contained in the sequence, these are counted as 'misinterpretations' because they will not contribute to the students' understanding of the programme's intention, notwithstanding the fact that they might be very valuable points to learn.

As an example of the kind of misinterpretations that can occur, consider the first extract selected from TV19 on 'Social Integration'. The exlract lasts about 1 minute; the visual content throughout consists of extracts from Sesame Street showing children playing in fields; the audio begins with 10 seconds of music from Sesame Street, and the remainder is the presenter's voice over, describing Sesame Street, its purpose and its contrast with the UK programme Playschool. Students were played the sequence and then asked to write down (a) what were you thinking about at that moment? and (b) what is the point being made by this sequence? The qualitative differences in interpretation are revealed in answers to (b), while answers to (a) are taken as the student's comment on the sequence. For this sequence, most students gave correct interpretations categorised as component points of the main point of the programme. These responses were of the following type:

"By contrast with Playschool in Britain, Sesame Street was street-based working-class with an emphasis on urban life, with children being active in it. Teaching by different method."

"It is different from our TV programmes. It's working class and urban, children active, not observers".

These answers reflect the important points being made by this sequence and mention both the conlrast with UK programmes, and the nature of the difference.

12

The following responses were counted as "misinterpretations'.

"To illustrate the middle-class approach to children's TV and its apparent failure in the USA."

"Urban children in New York lack rural knowledge, but provide their own entertainment."

"That this programme would only apply to a small minority in this country."

None of the above points appeared in this sequence, nor indeed anywhere else in the programme. They are called 'misinterpretations' rather than 'alternative interpretations' because they conflict with the designers' intentions: they would not accept that the sequence was making any of those points, and would not agree with them.

This sequence contributed to the overall argument of the programme that children are socialised by the television they watch. The students who responded with these unexpected interpretations of this sequence could still pick up that point from other sequences, but in fact tended not to. The important point is that students can exhibit a surprisingly large number of local 'misinterpretations' of key parts of the programme. A more detailed analysis of the categories of misinterpretation is given in the next section.

3.3 Forms of misinterpretation

The 110 interpretations of extracts (i.e., 22%) that corresponded to none of the previously defined content categories were analysed qualitatively. This analysis, as validated by second judges, yielded five general categories of misinterpretation (see Table 2).

Table 2. Occ~urrence of forms of misinterpretation of ' IV extracts

Forms of % occurrence in misinterpretation descriptions of extracts

Over-selection 40

Over-generalisation 7

Personal interpretation 40

Critique 7

Inadequate response 6

13

Of the forms of misinterpretation exhibited by students across all five programmes, by far the majority fall into two main categories: 'over-selection'

(40%), i.e., attention to only part of the sequence, and 'personal interpretation' (40%), i.e., contributing an interpretation that contrasts with that of the presenter.

The main problem seems to be that students sometimes concentrate too much on the example being shown as part of the extract, at the expense of understanding the principle it is meant to relate to. This happens in two distinct ways: selection, which suggests a mistaken perception of the structure of the extract, and personal interpretation, which suggests that the student is bringing their own ideas to the interpretation, rather than focusing on those offered by the programme. However, there is no evidence in this data to suggest that students are consistent in the way they misinterpret an extract. Over-selection is not a mistake any student makes more than once.

Similarly, only five of the 126 students exhibit any consistency in using a personal interpretation of the extracts. An illustration of this can be found in one student's responses to extracts from TV17. In both cases, he uses the sequence to pursue a question of interest to himself. The first extract made the point that 'states not accepted by their people as legitimate need to use force', illustrating this with film of E1 Salvador. The student responded:

"I was thinking about why people are behaving so violently. Because they

think that the government is not legitimate? But why do people pursue illegiti-

mate power?"

And after the second extract which argued that violence changes over time within any one country, the same student responded:

"I was thinking about the whole affair as part of the struggle for the 'will to power'."

In both cases the response was triggered by what the student saw, but was very personal, making no attempt to recapture the points being put across by the programme. This is not to say that the student does not learn something valuable from the experience. He is actively reflecting on what he sees, making sense of it in terms of other experiences, and possibly thereby developing his understanding. The question being explored here is whether 'misinterpretation' of this kind

affects the student's perception of the intended message of the programme. A second illustration comes from another student's responses to extracts from

TV14. The second extract shows how a government uses elections to cope with a crisis, with reference to the 1974 miners' strike, showing film of both the miners' leader's speech and an interview with Mr Heath. This student used his own political judgement to evaluate the example offered, and came to an alternative conclusion:

14

"that two options (i.e., the miners" and Mr Heath's) can exist in opposition to each other and yet both be almost correct."

Again, in response to the fourth extract, which illustrated the conflict between capital and labour with another Heath interview, the student made a personal political judgement to conclude that:

"Mr Heath could quite possibly have believed he was right to try to enforce his decision on the miners."

In both cases, the student focused on the content of the example, and used his own ideas to generate an idiosyncratic conclusion, with, again, no attempt to reproduce what the programme said about the example.

Although a large number of students occasionally exhibit personal interpretations of programme extracts, it does not seem to be a stable student characteristic. Only five students out of the 126 exhibit the kind of consistency illustrated above, and this suggests that it plays a rather insignificant role in the problem of misinterpretation of a programme as a whole.

3.4 Students' integration of key sequences into the overall message structure

We can also examine how far students integrate key sequences into the underlying structure of the programme. It is possible that students may correctly interpret

the key sequences that contribute to the main point, but fail to integrate them properly in order to reproduce the main point as part of their overall summary. To examine this possibility, we can consider the students who correctly interpret those extract sequences that contain the main point explicitly, and determine what proportion of them include this point in their overall summary.

Considering the four extracts that contain the programme's main point explicitly, we can see considerable variation in the integration of these into the students' summaries. Table 3 shows that the percentage of students who turn a correct sequence interpretation into a correct perception of the message varied from 33% to 100%.

The figures show that only in the case of TV17 extract 2 could we account for failure to perceive the overall message by suggesting that those students are inca- pable of recognising the point at all. For the other cases, the problem appears to be that most students can discern the point, but do not recognise its status. The most striking case is TV17 extract 4, where the point is made so clearly, that all the students interpret that sequence correctly, and yet only two thirds appear to recognise its status as one of the main messages of the programme. It seems therefore, that some students fail to discern the underlying structure - they may perceive the elements correctly, but do not necessarily integrate them into the appropriate structure. This suggests that integration is a skill that is logically distinct from correct interpretation of the parts of a programme.

15

Table 3. Comparisons between interpretation of extract and summary

Extracts that explicitly % of students who interpret % of these who give contain main point extract in terms of main point main point in summary

TVI4 Extract 1 54 33

TV17 Extract 2 31 100



3.5 Perception of underlying structure

We have established that some students fail to perceive the underlying message structure of a television programme, in the sense that they perceive the main point correctly when it is made, but fail to accord it the appropriate status. It is possible that this may be because some students assume a linear rather than a hierarchical structure, make no effort to discern the structure that is present, and tend to ignore the cues that point to it. We should also consider however, the extent to which different programmes display their structure, and how easy it is to discern it.

One important feature is the amount of time the programme devoms explicitly

to the main point. A structural analysis of the content of each programme is repro- duced in Figure 3, which shows how over the course of each programme the focus moves between the various levels of the discourse. It can be seen from this that the amount of screen time devoted explicitly to the Level 1 point is usually small (7%, 12%, 9%, 15%) but relatively high (30%) for one programme, TV19.

The programmes summarised in Figure 3 all embody, to varying degrees, the design principle that 'in order to understand a statement, we must show an instantiation of it'. The implicit assumption is that the student who can readily understand a particular instance of racial discrimination, say, will be able to recall easily the details of that instance, will be able to extract from it the essential prop- erties, or relations, and so begin to build an idea of the general concept. As the concept becomes more familiar with use, the student will be able to cease referral back to the particular instance. This kind of assumption no doubt underlies all our attempts to clarify communicated ideas by means of examples. The approach adopted in the progrmnmes under discussion, is to assist the students' abstraction processes through exemplification, a process akin to 'supplantation' (Salomon, 1979).

16

Level 1 : Main point

Level 2: Component points

Level 3 : Examples of Level 2



Level 3: Examples of Level 2



Level 3: Examples el Level 2 ~ .

Level 1: m

L M a i n poinls


Level 3: Examples of Level 1



Level 3: Examples el Level 2

m

mm

m

mm m m m m m m m m m

mmmmmm m m m m m m

m

m m

m m

TV14: Pictures of Politics

Explicit screen lime

m

m

m

m 7%

72%

21%

12%

m m

m mm 20%

TV15: Fairy-tale Democracy?

TV16: The Real World

• m

m

m

62%

9%

76%

15%

15%

37%

m m 48%

TV17: Force and Violence

m

~ m mmm m m m Im•

m m m m m m m m ===

TV19: Social Integration

30%

50%

20%

Figure 2. Structural analysis of content for five TV programmes

17

A close examination of how the approach is put into practice, however, reveals some important differences between the programmes. The evidence/example level in Figure 3 is labelled in each case by the level of the point it is illustrating.

If we now consider what proportion of time is spent in exemplification of the main point, Figure 3 shows that in most cases there is no time at all spent in the exemplification of the main point itself. It is established simply by an appeal to logic, by the simple s t a t e m e n t of the relation between established components. Instead, examples are used to illustrate component points, or Level 2 argument. Table 4 shows the total screen time devoted to either an explicit statement of the main point at Level 1 or an instantiation of it at Level 3 across the five programmes, and compares it with the total time spent on Level 2, including examples of Level 2 points.

There is an important difference between TV17 and TV19 in the way they present their main point. One of the main points made by TV17 is that 'the state is violent in several ways'. This is illustrated by means of some examples of state violence - riot police, civil war, atomic bombs - each illustrated with powerful and shocking images. Each example here itself embodies the main point being made. Together with the Level 1 statements made before and after the examples, which guide the students towards extracting the appropriate features, the programme provides a good example of both the exemplification and the abstraction processes that are part of concept development.

TV19 had to get the point across in a different way. The point it had to make is that 'children's television programmes have a socialising effect', and it operation- alised this claim by translating it into the statement that "the programmes' social characteristics vary for different age groups and cultures". A message of this type cannot possibly be instantiated by a single example; it requires a series of different examples which together establish the variation. And that is how the programme is structured. It sets out a number of descriptions of the social characteristics of children's programmes, e.g. 'family-oriented', 'working-class', 'didactic' etc., and gives examples of the particular type produced for different age groups and different cultures. The examples are therefore all exemplifications of component points, rather than the main point. However, it returns repeatedly to a statement of the main point, to the extent that the total screen time devoted to it is considerably higher than for the first three programmes (see Table 4).

This more explicit focus on the main point of the programme assists students' access to the slructure whether it is done by continual restatement of the main point (as in TV19) or by exemplification of the main point (as in TV17): both these programmes produced a much higher proportion of high level student summaries (80% and 88% respectively) compared with those for the other three programmes (46%, 38% and 52%), as shown in Table 1.

18

Table 4. Comparison of screen time spent on Level l and on Level 2, across programmes

TV14 TVI5 TV16 TVI7 TV19

7 12 9 15 30 % explicit screen urne on Levd 1

% total time on Level 1, 7 12 9 including examples

63 30

% total lime on Level 2, 93 88 91 37 70 including examples

4. Discussion

All the programmes analysed here are didactic in the sense that they have an intended learning outcome, and make this explicit at some point in the commentary. Producers and academics were able to agree on what the intended learning outcomes were, and on how the structure of each programme supported them. It is therefore important that students should understand the intended message. The study showed that in some cases only half the students did so. This may seem low, but it is a lot better than some of the scores achieved for student understanding of academic texts (Marton et al, 1984). It is therefore important to understand

why designers' intentions are sometimes thwarted, and how they are sometimes fulfilled.

Firstly, there seems to be no stable characteristic exhibited by students in this study that suggests we can explain it as an individual's inability to 'read' television programmes. However there is evidence that some students find it difficult to discern the message structure of a programme in the sense that although they understand the content sequence by sequence, they do not always organise these into an appropriate structural whole. The programme is seen as a collection of sequences of equal status, reminiscent of those students who do something similar with text.

Secondly, there is evidence that designers can assist students in recognising the intended message, by giving the main point a higher profile. This is done in two different ways in the programmes considered here. In one, the main point is repeatedly exemplified directly. In the other it cannot be, because it is comparative in form and therefore depends upon bringing together two or more contrasting examples. However in this programme, the main point is repeatedly emphasised in the commentary. By contrast, the other programmes use exemplification for points subordinate to the main point, but do not compensate for this by

19

repeatedly making explicit reference to it. Thus the programmes that give adequate exposure to the intended message by giving screen time to the main point, through either description or direct exemplification, elicit a higher proportion of student summaries that match the intended message.

In general, we can conclude that there is a relationship between the logical structure of the main point of a programme and the presentational form the programme should take. Illustrative examples are powerful enough that they carry the message most students take away. Unless that message corresponds to the main point, the latter is in danger of being lost on those students who tend to 'horizontalise'.

The richness of the personal interpretations found in this study is evidence that students learn a lot from these programmes. That is not disputed. The point is that if designers have an explicit didactic intention, and wish their message to be understood, then they must assist students in this. It is the designers' task to preserve the synergy between image and argument. If the visuals match the argument, then there is a natural synergy between them. If the argument is essentially abstract, and the visuals can only match its components, then the 'image- argument synergy' can be preserved, it seems, by not allowing the concrete component visuals to take over, and by emphasising the abstract nature of the argument, in the case described here, via to-camera presentation and studio discussion.

The study has achieved its aim of establishing qualitative differences in learning outcomes and provides a description of these that relates explicitly to the underlying message structure of the programme. In contrast to the text studies referred to, no students were found to confuse the main message with the illustrative examples (see Table 1), but for some of the programmes, a significant proportion confused it with component points of secondary importance. The further aim of investigating how these differences were constituted was achieved by comparing the designs of the programmes which were differentially successful, and by showing that the mediation of the underlying message depends upon pre- serving the image-argument synergy.

5. Methodological discussion

In her useful survey of research on instructional television, Cambre defines three main types of study: basic research, which seeks to determine the effects of media design on the intended audience, formative evaluation, which is carried out for the development team to establish effectiveness, and impact studies, to determine effectiveness once the material is in use, i.e., akin to summative evaluation (Cambre, 1987). The study described in this paper falls into the ftrst category, but represents a non-standard approach to basic research on educational television.

20

The more standard approach is to examine the relationship between design features of TV programmes and how well students recall the related content (e.g., Boeckrnann et al., 1988; Golden, 1990; Davies; 1989). That methodology requires the pre-specification of critical features, tests of recall of content, and an investigation of the correlations between the two. These studies are valuable for telling us how to make a point appear salient, but do not tell us how to make it understood. Recall and understanding are not necessarily related. Indeed, Kelley found that student participants in TV literacy courses improved their inferences and judgments about TV news and documentary programmes even though they showed no improvement on recall tests (Kelley, Gunter and Buckle, 1987). As the present study shows, students do a lot with what they see. The information is elaborately processed and the precise source of the information that generated the inference may be lost while the meaning remains. It is possible that recall of specifics is relatively unimportant. Much more important is the way understanding occurs.

Another approach, more common in the US than the UK, is to relate student performance on a test of recall or understanding to student variables rather than to characteristics of the programme. The present study did not follow this approach either: there was no attempt, for example, to explain personal interpretations in terms of prior knowledge. Such findings would be unusable in any case - we would hardly prevent students from watching if they had inadequate prior knowledge. It is far more valuable to look at the different ways the student population as a whole interprets the programmes, and to illuminate the relationships between these and the programme characteristics - something we can do something about.

The study described here is an example, therefore, of an alternative methodological approach to the study of educational television: using qualitative methods to describe the phenomena of learning through television, to identify the ways in which programmes are understood, and to use these together with the slructural relations between components and the whole programme to illuminate which programme features are important.

References

Allen, W. H. (1971). Instructional media research: past, present and future. AV Communication Review. 19, 10.

Beaty, E. M. (1987). Understanding eoncepts in social science: towards an effective evaluation strategy, Instructional Science, 15, 341-359.

Boeckrnarm, K., Nessmann, K. and Petermandl, M. (1988). Effects of formal features in educalional video programmes on recall Journal of Educational Television, 14(2), 107-120.

Cambre, M. A. (1987). A reappraisal of instructional television. ERIC Clearinghouse on Information Resources, New York, ED 296 720.

Davies, M. M. (1989). Why can people jump higher on the moon? A study of what children learn from Corners, a children's TV programme../'ourna/ofEducational Television, 15(1), 25-36.

21

Golden, A. (1990)_ The effects of quality and clarity on the recall of photographic illustrations. Brit&h Journal of Educational Technology, 21(1), 21-30.

Grundin, H. (1983). Audio-visual media in the Open University. Results of a survey of 93 oourses, IET papers on broadcasting No 224.

Hodgson, V. (1984). Learning through lectures. In F. Marton, D. Hounsell and N. Entwistle ('Eds.), The exper&nce of learning. Edinburgh: Scottish Academic Press.

Ide, T. 1L (1974). The potentials and limitations of television as an educational medium. In D. Olson (lEd.), Media and symbols: the forms of expression. Chicago: University of Chicago Press.

Kaye, A. R. (1973). The design and evaluation of science courses at the Open University. Instructional Science, 2, 119-191.

Kelley, P., Gunter, B. and Buckle, L. (1987). 'Reading' television in the classroom: more results from the television literacy project. Journal of Educational Telev&ion, 13(1), 7-19.

Laurillard, D. (1982). D102 Audio-visual media evaluation: first year report. IET papers on broadcasting No 212. IET, Open University, Milton Keynes.

Laurillard, D. (1983). D102 Evaluation report IET, Open University, Milton Keynes. Laurillard, D. (1984). Learning through problem-solving. In F. Marton, D. Hounsell and N. Entwistle

(Eds.), The experience of learning. Edinburgh: Scottish Academic Press. Martun, F. and Saljo, R. (1976). On qualitative differences in learning, I: Outcome and process. British

Journal of Educational Psychology, 46, 4-11. Marton, F. and Wenestaa'n, C-G. (1979). Qualitative differences in the understanding and retention of

the main point in some texts based on the principle-example structure. M. M. Graneberg, P. E. Morris and R. N. Sykes ('Eds.), Pract&al aspects of memory. London: Academic Press.

Marton, F., Hounsell, D. and Entwlstle, N. (Eds.), The e~erience of learning. Edinburgh: Scottish Academic Press.

Salomon, G. (1979). Interaction of media, cognition and learning. San Francisco: Jossey-Bass. Svensson, L. (1977). On qualitative differences in learning: study skill and learning. British Journal of

Educational Psychology, 47, 233-243.

2 2

Appendix 1: Q u e s t i o n n a i r e

The questionnaire was worded as follows:

1. Name

2. Was D102 your first choice for a Foundation Course? Yes/No

3, How many OU courses have you completed? None/one/more than one.

4. Do you have any experience of social science: in working context? Yes/No

Academically? Yes/No

5. How many times have you seen this TV programme: None/once/twice?

6. Please summarise the programme as if you were describing it to a friend on the course who had missed it:

(a) What were you thinking about at that moment?

Co) What is the point being made by this sequence?

8,9, 10 as for 7

11. (Factual questions about each extract, different for each programme).

12. Now look at your answer to question 6 again. If you want to put anything different, you can write it here:

7.

Appendix 2: Analysis of programme content

Each of the five programmes has been summarised m terms of a series of statements designed to capture the main ideas and their interrelationships. Each statement is categorised according to its supposed level in the overall hierarchical structure (see Figure 1). Each summary has been checked with the academic and the producer associated with that programme.

Summary of TV] 4 : Pict~,es of Politics

Level 1 A and B

Pluralism and Marxism both provide explanations of how society like ours holds together while changing.

Level 2 A 1, A2... Pluralism explains the maintenance of equilibrium via the responsiveness of the state to demands, and the mechanism of elections; the state is open, arbitrates betwee~ factions, provides a choice of government, provides an election ff them is a crisis.

Levels 3 EAn An example was Heath calling an election in response m the miners ' strike in 1974.

Levels 2 B 1 . B 2, ... Marxism explains that conflict is contained, in the interests of capital, via the processes of legitimation, and using force ff necessary, the system of production creates classes, classes have conflicting interests, the state stops class COILfliCL

The state is separate frtma capital and labour, favours capital, makes concessions to labour.

The process of legitimation represents capital as being the status quo, and as being in the interests of society.

Elections provide only a limited choice.

In crisis, the state may use force against labour.

Level 3 EBn

An example was Heath's plans for emergency measures during the miners ' strike.

23

Summary of TVl 5 : Fairy-Tale Democracy

Level 1 A and B

Pluralism and Marxism provide aliemative explanations of aspects of British society, such as the economy, Common Market entry.

Level 2 A 1,A2 .....

Pluralism can account for the decision to enter the Common Market by consensus.

Level 3 EAn

For example, the decision on entry was made by referendum.

Level 2 B 1,B2 ..... Marxism can also account for the decision to enter the Common Market, it looks at economic and political structures and the exercise of power within them, and considers the legitimation of the capitalist structure.

Level 3 EBn For example the move for entry was supported by capitalists needing an improved economy.

Summary of TV16 :The Real World

Level 1 A Pluralists describe policy/decision making in terms of interactions between representatives of three groups.

Level 2 A 1, A2,- ' Elected Representatives, Appointed Officials, Interested Groups - each of whom contribute to policy/decision-making in different ways.

Levds EAn An example of how some individuals think these interactions do and should operate can be found in the decision to implement nuclear power in this country.

Summary of TV17: Force and Violence

Level 1 A The state is violent in several ways:

l eve l 2 A 1 ,A2"'"

It uses violence against other states, it may use violence against its own people, violence varies from country to country, the state legitimises some forms of violence, the state has the right to use force.

Level 3 E A Examples of state violence

Level 1 B Violence can have beneficial effects on society.

Examples are: China: a lack of violence in 17th C. led to later revolution

England: violence in the Civil War led to present day stability.

The French Revolution: violence led to later stability.

The US Civil War: violence led to later stability.

Summary of TV19 : Social Integration

Level 1 A Children's TV programmes reflect the culture/societies in which they are made.

Level 2A 1, A2"- British pre-school programmes tend to be family-oriented, home-based middle-class, with children passive onlookers. They depict a secure world without tension.

School-age programmes on British TV are more actively oriented; the approach is problem-centred, caring, charity-oriented, non-controversial.

A few teenage programmes are adult oriented, informal, enteaaining, conflict-oriented, realistic. It is largely in this area that tension and conflict are allowed to surface.

Mediating the message: television programme design and students' understanding

Documents

Transcript of Mediating the message: television programme design and students' understanding