2003.02.03 - SLIDE 1IS246 - SPRING 2003

Lecture 04: Formalist Media Theory

IS246Multimedia Information

(FILM 240, Section 4)

Prof. Marc DavisUC Berkeley SIMS

Monday and Wednesday 2:00 pm – 3:30 pmSpring 2003

http://www.sims.berkeley.edu/academics/courses/is246/s03/

2003.02.03 - SLIDE 2IS246 - SPRING 2003

Today’s Agenda

• Review of Last Time

– Semiotics

• Formalist Media Theory

– Film Form

– Narrative Form

– Narration

– Introduction to Editing

• Discussion

• Action Items for Next Time

2003.02.03 - SLIDE 3IS246 - SPRING 2003

Today’s Agenda

• Review of Last Time

– Semiotics

• Formalist Media Theory

– Film Form

– Narrative Form

– Narration

– Introduction to Editing

• Discussion

• Action Items for Next Time

2003.02.03 - SLIDE 4IS246 - SPRING 2003

• Sign, Signified, Signifier – The linguistic sign is the unity of the signifier

(a sound-image) and the signified (a concept)

Linguistic Sign

Concept

Sound-Image

2003.02.03 - SLIDE 5IS246 - SPRING 2003

Linguistic Signs and Language

• The sign is arbitrary

• A multiplicity of signs is necessary to form any language

• The system of signs in language is over-complex

• Language exhibits a collective inertia toward innovation

2003.02.03 - SLIDE 6IS246 - SPRING 2003

Linguistic Values

• Values are composed of– A dissimilar thing that can be exchanged for

the thing of which the value is to be determined

– Similar things that can be compared with the thing of which the value is to be determined

Signified

Signifier

Signified

Signifier

Signified

Signifier

2003.02.03 - SLIDE 7IS246 - SPRING 2003

Differences

• “Everything that has been said up to this point boils down to this: in language there are only differences. Even more important: a difference generally implies positive terms between which the difference is set up; but in language there are only differences without positive terms.” (p. 120).

• “In reality the idea evokes not a form but a whole latent system that makes possible the oppositions necessary for the formation of the sign. By itself the sign would have no signification.” (p. 130).

2003.02.03 - SLIDE 8IS246 - SPRING 2003

Syntagmatic and Associative Relations

2003.02.03 - SLIDE 9IS246 - SPRING 2003

Today’s Agenda

• Review of Last Time

– Semiotics

• Formalist Media Theory

– Film Form

– Narrative Form

– Narration

– Introduction to Editing

• Discussion

• Action Items for Next Time

2003.02.03 - SLIDE 10IS246 - SPRING 2003

Why Study Formalist Film Theory?

• To provide a theoretical foundation for understanding the forms and functions of time-based media

• Unlike “Film Studies” per se, we use that understanding not to interpret films, but to analyze and design multimedia information systems– Video capture– Video analysis– Video retrieval– Video assembly– Video reuse– Video summarization (e.g., meeting recording)– User interfaces to audio-visual content and that use

audio-visual content

2003.02.03 - SLIDE 11IS246 - SPRING 2003

Today’s Agenda

• Review of Last Time

– Semiotics

• Formalist Media Theory

– Film Form

– Narrative Form

– Narration

– Introduction to Editing

• Discussion

• Action Items for Next Time

2003.02.03 - SLIDE 12IS246 - SPRING 2003

A

2003.02.03 - SLIDE 13IS246 - SPRING 2003

AB

2003.02.03 - SLIDE 14IS246 - SPRING 2003

ABA

2003.02.03 - SLIDE 15IS246 - SPRING 2003

ABAC

2003.02.03 - SLIDE 16IS246 - SPRING 2003

ABACA

2003.02.03 - SLIDE 17IS246 - SPRING 2003

Expectations

• Suspense– Delay in fulfilling an established

expectation

• Surprise– Result of an expectation that is revealed

to be incorrect

• Curiosity– Construct hypotheses about prior events

2003.02.03 - SLIDE 18IS246 - SPRING 2003

Perceiving Artistic Form

• Form– “The overall system of relations that we can

perceive among the elements in the whole film”

• In perceiving form, the spectator draws on– Cues within the work– Prior experiences

• Derived from everyday life• From other artworks

– Conventions and norms

2003.02.03 - SLIDE 19IS246 - SPRING 2003

Principles of Film Form

• Function– What is this element doing there?– How does it cue us to respond?– Motivation (justification for the presence of an

element)• Similarity and repetition

– Motif (any significant repeated element in a film)– Parallelism (cues to compare two or more distinct

elements by highlighting some similarity)• Difference and variation• Development

– Progression moving from beginning to middle to end• Unity/Disunity

2003.02.03 - SLIDE 20IS246 - SPRING 2003

Viewer’s Activity

• “The constant interplay between similarity and difference, repetition and variation, leads the viewer to an active developing awareness of the film’s formal system.” (p. 56)

2003.02.03 - SLIDE 21IS246 - SPRING 2003

Today’s Agenda

• Review of Last Time

– Semiotics

• Formalist Media Theory

– Film Form

– Narrative Form

– Narration

– Introduction to Editing

• Discussion

• Action Items for Next Time

2003.02.03 - SLIDE 22IS246 - SPRING 2003

Narrative Form

• Narrative– A chain of events in cause-effect relationship

occurring in time and space

• Story and Plot– Story

• Set of all events in a narrative, both the ones explicitly represented and those the viewer infers

– Plot• Everything visibly and audibly present in the film• All the story events that are directly depicted

2003.02.03 - SLIDE 23IS246 - SPRING 2003

Story and Plot

Story

Plot

Presumed and inferredevents

Explicitly presentedevents

Added nondiegeticmaterial

2003.02.03 - SLIDE 24IS246 - SPRING 2003

Teeth Brushing Example

• Brushing Teeth– Protagonist stands in front of bathroom mirror– Protagonist opens medicine cabinet to

remove toothbrush and toothpaste tube– Protagonist squeezes out toothpaste on

toothbrush– Protagonist brushes teeth– Protagonist drinks water from glass– Protagonist spits out water and toothpaste

residue

2003.02.03 - SLIDE 25IS246 - SPRING 2003

Time

• Temporal order– Flashback– Flashforward

• Temporal duration– Story duration– Plot duration– Screen duration

• Temporal frequency– Repetition of events

2003.02.03 - SLIDE 26IS246 - SPRING 2003

Temporal Duration

• Story Duration– Example: Brushing teeth in story world (5 minutes)

• Plot Duration– Example: Brushing teeth in plot world (1 minute: 6

steps of ~10 seconds each)

• Screen Duration– Example: Brushing teeth (12 seconds: 3 shots of ~4

seconds each)

2003.02.03 - SLIDE 27IS246 - SPRING 2003

Space

• Story space

• Plot space

• Screen space and offscreen space

2003.02.03 - SLIDE 28IS246 - SPRING 2003

Today’s Agenda

• Review of Last Time

– Semiotics

• Formalist Media Theory

– Film Form

– Narrative Form

– Narration

– Introduction to Editing

• Discussion

• Action Items for Next Time

2003.02.03 - SLIDE 29IS246 - SPRING 2003

Narration

• Plot’s way of distributing story information in order to achieve specific effects

• Moment-by-moment process that guides us in building the story out of the plot

• Involves range and depth of story information

2003.02.03 - SLIDE 30IS246 - SPRING 2003

Range of Story Information

• Spectrum of knowledge of the story world that viewers and characters have– Unrestricted (omniscient) narration– Restricted narration

• Creates “hierarchy of knowledge” among viewer and characters

• “Who knows what when?”

2003.02.03 - SLIDE 31IS246 - SPRING 2003

Depth of Story Information

• How “deeply” the plot plunges into a character’s psychological states

• Continuum between objectivity and subjectivity• Subjectivity

– Perceptual subjectivity (hear and see what character perceives)• Point-of-view shot• Sound perspective

– Mental subjectivity (hear and see what character thinks)• Internal voices• Internal images

• “How deeply do I know the character’s perceptions, feelings, and thoughts?”

• Range and depth of knowledge are independent variables

2003.02.03 - SLIDE 32IS246 - SPRING 2003

“Classical Hollywood Cinema”

• Action primarily arises from individual characters as causal agents

• The process of achieving goals desired by one or more characters drives the narrative’s development

• The protagonists’ goals come into conflict with other characters’ goals (antagonists) to create conflict

2003.02.03 - SLIDE 33IS246 - SPRING 2003

“Classical Hollywood Cinema”

• The cause-effect chain drives narrative events• Plot time tends to depend on the story’s cause-

effect chain– “Dead time” is rarely shown– Appointments bring characters together at a specific

time and usually place– Deadlines makes plot duration dependent on the

cause-effect chain

• Narration tends to be “objective” and unrestricted

• Narrative usually has strong closure at the end (cause-effect chain ends with final effect)

2003.02.03 - SLIDE 34IS246 - SPRING 2003

Today’s Agenda

• Review of Last Time

– Semiotics

• Formalist Media Theory

– Film Form

– Narrative Form

– Narration

– Introduction to Editing

• Discussion

• Action Items for Next Time

2003.02.03 - SLIDE 35IS246 - SPRING 2003

Kuleshov and Isenhour

• Kuelshov: role of montage in cinema– Kuleshov-Pudovkin experiment and the

“Kuleshov Effect”

• Isenhour: context and order in film editing– The meaning of a shot will vary with its

context– By changing the order of shots, the meaning

is changed

2003.02.03 - SLIDE 36IS246 - SPRING 2003

Today’s Agenda

• Review of Last Time

– Semiotics

• Formalist Media Theory

– Film Form

– Narrative Form

– Narration

– Introduction to Editing

• Discussion

• Action Items for Next Time

2003.02.03 - SLIDE 37IS246 - SPRING 2003

Discussion Questions

• How can the “story/plot” distinction be leveraged in designing video summarization systems?

• How can the mode of “Classical Hollywood Cinema” be applied to non-fictional uses of video?

• What implications does the “Kuleshov Effect” have for designing metadata for multimedia and multimedia databases?

2003.02.03 - SLIDE 38IS246 - SPRING 2003

Today’s Agenda

• Review of Last Time

– Semiotics

• Formal Media Theory

– Film Form

– Narrative Form

– Narration

– Introduction to Editing

• Discussion

• Action Items for Next Time

2003.02.03 - SLIDE 39IS246 - SPRING 2003

Readings for Next Time

• Wednesday 02/05– Textbook

• David Bordwell and Kristin Thompson: Film Art: An Introduction. Pages: 155-326

• Priority of focus– Ch. 8 The Relation of Shot to Shot: Editing– Ch. 7 The Shot: Cinematographic Properties– Ch. 6 The Shot: Mise-en-scene– Ch. 9 Sound in the Cinema