Learning with audio-visible and invisible authors

Lessons learnt from a Political science course at USI – University of Lugano, Switzerland

Terry Inglese

UCSB, 14th November 2006

My background

• TV program designer and archivist at the Swiss public TV

• Indipendent documentary filmmaker

• PhD student • Multimedia

instructional designer

Combination of

academic and professional

skills

The memory institutions and the digital revolution

• Archives - from storing objects to the management of the life cycle of digital and digitized products;

• Libraries - from the reading room to the digital information service centre;

• Museums - from collections to narrative connections and new experiences.

• from a static into a dynamic change

Database and Narrative (Manovich – 2001)

• “The emergence of new media coincides … with accessing and reusing existing media objects as with creating new ones.”

• (Manovich, 2001, pp. 35, 36)

TV archive and new media

• New media tend to repackage old media through reproduction, manipulation, convergence.

• Benjamin (1977) and Baudrillard (1996) – the loss of aura that the original work of art would lose

if inserted into the age of mechanical reproduction.

• In our case studies, besides an apparent feeling of loss, there is a sense of gain that new media are providing, a gain that is not material, but cognitive and affective.

The auratic value of new media

• New media are not just invented to meet the needs that already exist, but also new ones.

• Kress and Van Leeuwen, “what is lost may return, and what is gained many yet turn out to be loss. The new technologies’ emphasis on multimodality, three dimensionality and interactivity can be seen as a return of many of the things that were lost in the transition from orality to literacy, as a second orality, in other words.” (Kress, Van Leeuwen, 2001, p. 92).

• New media possess an auratic value, as the one of the medieval manuscripts, because these media become manuscript media, stored, transmitted, encoded and re-encoded in a digital form.

The conceptual orality

• From this perspective, archives provide a return to a conceptual orality, that is to say, “a return to the medieval framework wherein words or documents gained meaning only as they were closely related to their context and to actions arising from that context. In that oral tradition, meaning lies not in the records themselves, but (in) the transactions and customs to which they borrow witness as ‘evidences.’ (...).”

• (Taylor, in Cook, Dodds, 2003, p. 23).

Taylor

• “a catalogue remains a catalogue, but pattern recognition is the chef-d’oeuvre of human intelligence”.

The added value: • Discovering patterns of knowledge • Extracting more from less

(Taylor, in Cook, Dodds, 2003 )

My work

Goal: • to recognize the pedagogical as well as

instructional benefits of studying with TV multimedia archives as learning tools, taking into account the students’ perspective.

• The genre: TV interviews with Social Science scholars usually read and studied only through texts.

RTSI – Swiss public TV and radio

• Radio – RSI - 30ies

• TV – RTSI - 60ies

• because of the lack of the higher education institution like a university, the RTSI produced highly cultural programs (TV and radio) enhancing education and culture (Inglese, 1998)

• Genre of TV program – the interview

Availability and Affordances concepts

availability concept: archival content, produced in the past with analogue technologies, is easily ‘transformed’ into digital support. This technological shift increases the potential availability of ‘multimedia contents’;

affordance concept: students learn more deeply when they feel a personal relationship with the author to be studied, if the author (hypothesis) is offered – beside the text - through a multimedia & multimodal format such as a TV interview.

(Inglese, Mayer, Rigotti (2007, in press))

Media for learning

• Media can be defined by:

1. the technology

2. the symbol systems

3. the processing capabilities

media attributes and instructional methods

• it is not possible to separate the media attributes from the instructional methods

• “What we have learned from all the media comparison research is that it’s not the medium, but rather the instructional methods that cause learning.” (Mayer, Clark, 2003, p. 21)

• Learning depends on the quality of instructional messages and instructional methods rather than the medium per se.

Multimedia learning - Mayer • The cognitive theory of multimedia learning. (2001, 2003, 2005, etc.)

• Multimedia:• presentation of material using both words and pictures. • Multimedia learning is “a sense-making activity in which the learner

seeks to build a coherent mental representation from the presented material.”

• “the learner’s job is to make sense of the presented material; thus, the learner is an active sense maker who experiences a multimedia presentation and tries to organize and integrate the presented material into a coherent mental representation.” (Mayer, 2001, p. 13-15)

• The cognitive theory of multimedia learning is consistent with empirical research results based on how people learn and the understanding of the main cognitive processes in learning.

Multimedia learning

Social agency theory - Mayer

• assumptions that social cues in multimedia instructional messages prime a social response in learners

• the learner can interpret a multimedia instructional message as – an instance of information delivery or/and – a social communication event. Influencing the type of schemas activated in the learnerthe type of cognitive processing, and the quality of the learning outcomes

– Multimedia principle (Mayer, 2001)– Personalization principle (Mayer, 2003) – I-you vs. 3rd person– Sound cues – voice and the concept of persona (Mayer, 2005,

2006)

‘the media equation paradigm’.

• people apply the same dynamics from human-human interactions to human-computer interactions.

• Reeves and Nass (1996) define this phenomenon ‘the media equation paradigm’. This paradigm states that the media equal real life.

• All people automatically and unconsciously respond socially to media, and therefore also to computer and television, … especially nowadays with the new and old media convergence.

• Media features: closeness, size, visual and audio fidelity, motion, memory, and voice.

Research questions

• Would students learn (read and write) better if teachers could use texts and audiovisual materials, such as TV archived interviews with scholars that are normally studied through texts only? – retention and motivation (did students retain and enjoy more with

an audio-visible author vs. an invisible text-based author?) • Grade• Number of written words• Quote

• Does reading text of scholars offered through multimedia instructional messages change? If yes, how and why?

• the social presence of the author

• Kamin, Intrator and Kim (2000) have shown despite the appeal of using multimedia documents to enhance reading, there are still several cognitive and affective processes in reading and processing texts that need to be investigated.

The case study - Definitions

• Audio-visible author– Multimedia and multimodal interview format– I-you relationship– & Text

• Invisible author – Only text– No I-you relationship– Detached, formal and academic language

(strong text)

1. Case study

The final written exam texts

• 105 students took the written exam • 73 native Italian speakers and 32 non-native ones. • The non-native Italian students could write it in their own

languages: German, French, Spanish or English.

• 7 questions:– 2 questions related to the audio-visible authors and – 1 related to the invisible author.

• 10-to-1 grading scale, with 10 as the highest grade and 1 as the lowest.

• Karl Popper – audio-visible author + text

• Paul Feyerabend – audio-visible + text

• Andrea Semprini – invisible author – only text (a text that is invisible)

grades

7

7.53

6.81

7

7.67

6.67

6

6.2

6.4

6.6

6.8

7

7.2

7.4

7.6

7.8

Feyerabend - vis Popper - vis Semprini - invis

mean grades Ita

mean grades non Ita

words

246

166

133

194

146 143

0

50

100

150

200

250

300

Feyerabend - vis Popper - vis Semprini - invis

mean words Ita

mean words non Ita

quotes

1.6

2.41

0.26

1.5

2.03

0.31

0

0.5

1

1.5

2

2.5

3

Feyerabend - vis Popper - vis Semprini - invis

means quotes Ita

means quotes non Ita

2. Case study

• 108 university freshmen

• 60 native Italian speakers and 48 non-native ones (15 different countries and speaking a total of 13 different languages).

• the average age in the class was from 20 to 25 years old

• Claude Lévi-Strauss – audio-visible author + text

• Paul Feyerabend – audio-visible author + text

• Andrea Semprini – invisible author

grades

7.98 7.997.17

7.787.04

5.8

0

1

2

3

4

5

6

7

8

9

Feyerabend - vis Lévi Strauss -vis

Semprini - invis

mean grades Ita

mean grades non Ita

words

150

173161155 158

112

020406080

100120140160180200

Feyerabend -vis

Lévi Strauss -vis

Semprini - invis

mean words Ita

mean words non Ita

quotes

3.28

2.57

0.61

32.75

0.33

0

0.5

1

1.5

2

2.5

3

3.5

Feyerabend -vis

Lévi Strauss -vis

Semprini - invis

means quotes Ita

means quotes non Ita

Qualitative data

• thinking and feeling aloud methodology to collect the ongoing cognitive and affective responses and processes of students while reading the texts aloud.

• After reading, three questions: • 1. which is the most comprehensible author and why? • 2. which is the most interesting author and why? • 3. which is the most closer author and why?

• 1. to imagine how the invisible author looks like• 2. Bus scenario• 3. Taxi scenario

Measuring the author‘s presence in their texts

Levi Strauss (162 Words) Feyerabend (258 Words) Semprini (197 Words)I 0% 0.40% 0%you 0% 0% 0%we 1.90% 1.90% 1%questions 0% 0.80% 0%examples 0% 0.80% 0%

Total 1.90% 3.90% 1%

Taxi

author chosen for the taxi ride

0

2

4

6

8

10

12

Feyerabend Levi Strauss Semprini anybody 28 I

tali

an a

nd

no

n I

tali

an s

tud

ents

16 Italian

12 non Italian

comprehensibility, interest and emotion

0

5

10

15

20

25

30

morecomprehensible

more interesting more emotional 28 I

tali

an a

nd

no

n I

tali

an s

tud

ents

Feyerabend

LeviStrauss

Semprini

In deep-interviews with students

1. Authors

2. Texts

3. Reading act

4. Context

Students‘ metaphors

• The invisible author: realistic, precise, scolastic, serious, authoritative, rigorous, clear, scientific, vague, hostile, sterile, as an anchor, it does not touch

• Reading the invisible author’s texts: heavy, dense, rigid, tiring, hard, twisted, muddled, cold, abstract, complex, distant

Students‘ metaphors

• The audio-visible authors: sense of movement, participation, feeling of nearness and familiarity, involvement, relationship, access, contact, possibility of identification with the author, clearity, fresh air, sympathy, an encountering that leaves a trace, convincing effect, influencing effect, openness, sense of freedom, an embrace, a testimony, “having more than a weapon (to study)” (italian expression), a breathing, an encountering that touches

• Reading the audio-visible authors’s texts: light, fluid, a flowing and pleasant experience, linear, untied, warm, concrete, easy, near

3.rd case study

M. Foucault – audio-visible author – (difficult to read and study)

B. Constant – invisible (only text) but visible (I-you) author– (easy to read and study)

grades

7.62 7.65

7.18

6.8

6.2

6.4

6.6

6.8

7

7.2

7.4

7.6

7.8

Foucault Constant

mean grades Ita

mean grades non Ita

words

201.5

152.5

177

124

0

50

100

150

200

250

Foucault Constant

mean words Ita

mean words non Ita

quotes

2.49

1.36

2.42

1.05

0

0.5

1

1.5

2

2.5

3

Foucault Constant

means quotes Ita

means quotes non Ita

Additional audio-visible authors?

would you have preferred to have other authors explained in this course in an audio-visible

format?

0

10

20

30

40

50

60

1

stu

den

ts

YES

INDIFFEERENT

NO

Some conclusions

• 2 language and multimedia – (Inglese, Mayer, Rigotti, 2007, in press)

• Literacy and digital media • Importance of author‘s awareness and the social

presence of the author in expository texts• Designing instructional messages more speech-

like • Reconsidering Orality in combination with

Writing • Importance of using archives for educational

purposes, for example TV and radio ones

Some interpretations

• For Brandt (1990), reading is considered as an involvement act, where readers try to reach across texts to other human beings, having to be more consciously aware of what is taking place on the other side of the communication, as is necessary when the discourse is oral.

• Readers read not to separate from others, but to reach out to them. The motive for reading is to find other minds.

Some interpretations

• Literacy failures are not failures of separation but rather failures of involvement. They arise not from overdependence on context but from the lack of access to a context for making sense of print.

• Instead of viewing the oral as antagonist to the literature, it is necessary to understand better how the oral sustains the literate.

• (Brandt, 1990, p. 7)

Looking for UCSB collaboration

• accessing US grant money and US scholars that would like to use these audio-visible authors too, for instructional purposes and for research!

• developing together the Social Science audio-visible authors‘ database in collaboration with the RTSI Swiss public TV and USI Lugano.