i

University of Amsterdam FNWI

Artificial Intelligence

Bachelorthesis

24-07-2012

Extracting semantic properties to capture the

dynamic structure within the motifs of the Thematic

Model

Student: Rosemary Novita Moerbeek, 5942756

Supervisor: Frank Nack

1

2

Index

1 Abstract 3

2 Introduction 3

2.1 Narratives, Themes and Motifs 3

2.2 Structure and Culture 5

3 Related Work 6

4 Method 7

4.1 Function 7

4.2 Weight 8

4.2.1 Derivation 8

4.3 Combination 10

4.3.1 Derivation 10

4.4 Position 11

4.4.1 Derivation 11

4.5 Order 12

4.5.1 Derivation 12

4.6 Commotion 13

4.6.1 Derivation 13

4.7 Prototyping 17

5 Evaluation 17

5.1 First Narrative 17

5.2 Second Narrative 20

6 Conclusion 23

7 References 24

3

1 Abstract This thesis aims to tackle the problem of capturing semantic structures within narratives,

using a distinguishable approach bridging the semantic gap. Using the components of

Hargood’s Thematic Model [1], we try to capture the patterns of features denoting motifs

using five semantic properties (weight, combination, position, order, commotion) which can

be applied to a pattern matching algorithm to automatically generate narrative structures.

2 Introduction

Stories are part of our everyday live. We are naturally communicating and receiving

experiences from direct and indirect audiences and sources through a variety of media.

However, it appears difficult to establish an objective, effective and machine readable model

of narratives. This thesis presents a method that can improve a recently found machine

readable model of narratives with the purpose of partly eliminating subjectivity from the

original model. To understand the original model as well as the improved version, the next

sections will enlighten some background motivations.

2.1 Narratives, Themes and Motifs

The terms stories and narratives are sometimes used interchangeable, however, they are not

exactly the same. Structuralist theories define a narrative as composed of any series of human

experiences [2]. Furthermore, narratives can be deconstructed into a story and a discourse [3]

where the story represents a chronology of all the information to be communicated and the

discourse defines what parts of the story are told and how those arts are presented.

Narratives are a way of presenting information in its broadest sense, whether to inform,

persuade or entertain. They are used to make sense of information and experiences to

ourselves and to others and they apply to every sort of medium.

Narrative systems are systems that inhabit functionalities that disseminate, create or analyze

narratives. Since narratives play a great role in our everyday life, narrative systems that help

us process these narratives more efficiently would be very valuable. Let’s consider a few

future possibilities.

1) Information could be presented more engaging to the user by adjusting the structure –

not the content- to his or her personal preferences.

2) A semantic structure could be assigned to the unstructured results of a search engine

query, helping the user find his way through the data overload we encounter every

day.

3) Computer games could adapt and regenerate their plot based on the actions of the

player.

The idea of a narrative system is not new. Systems exist that can analyze, represent or

generate narratives. However, the main limitation of these systems is that their generated

4

narratives do not have the cohesion of a human-made narrative [4]. A plot is said to be

coherent when a human interpreter can understand the way in which the events of a narrative

have meaning and relevance to the outcome of the story [5].

A large amount of research is done on the literal content of the narrative such as the plot [5] or

the used language [6]. However, little attention has been given to the notion of themes in

narratives despite its contribution to the richness and cohesiveness of narratives. Themes

allow the author to transmit a certain context without having to explicitly mention the

components of that context. The reader already knows the components and only needs the

theme as a trigger to link them to the narrative. Most narrative systems have no proper

concept of what a theme is, how it can be represented or interpreted.

Hargood [1] recently tried to cover this section of narratology by setting up the Thematic

Model which is a representation model for themes using semiotic term expansion. The choice

for semiotic term expansion suits themesbest due to the implicit nature of a theme. Increasing

the understanding of such a theme might lead to narrative systems that create more coherent

and rich narratives.

The general assumption of the model is that a theme, being a conceptual entity, is build up

from a collection of other themes or motifs which, in context of each other, connote the

theme. Motifs are directly denoted by features, which are the visible computable elements

(either by authored tags, metadata or

computer vision detected annotations).

The use of the terms connotation and

denotation may seem subtle, but turns

out to be essential for the model. Note

that the idea of connotation is that signs

have meaning beyond their literal

expression and therefor the denotative

sign becomes a signified [7]. For

example, one may connote the concept

of a celebration from the sign of a

wedding cake, rather than just denoting

the concept of a cake. The ability of

making such a connotation arises from

our cultural and social background and

from our experiences, which makes a

connotation a subjective notion.

Fig. 1: The Thematic Model[1]

The Thematic Model is not capable of representing stronger and weaker relations between

features and motifs and motifs and themes, although this evidently applies in realistic

situations. Furthermore, the model cannot represent certain combinations of signs denoting

5

the signified. Serious problems could arise from this limitation. For example, one might want

to represent the motif diner as denoted by the combined features spoon, fork and knife. With

this representation, it is not enough to detect just a knife in the natom. An additional fork and

spoon has to be detected as well to denote the motif diner.

Another limitation arises when the model is used to represent motifs and themes over a series

of natoms which together form a narrative. It is not possible to represent the dynamics of this

narrative. One might want to represent a motif over a series of natoms as denoted by a certain

pattern of features rather than just the presence of these features. The order, quantity or

position of features is not expressible, even though these contain valuable information which

could lead to a more accurate denotation.

In this thesis, a method will be presented that dynamically captures the semantic pattern of a

narrative by assigning five semantic properties to the features to which a pattern recognition

algorithm can be applied. Section 4 shows how this function is derived. The general idea is

that, with these properties, we can produce a narrative from a new dataset simulating the

pattern of a known narrative or known genre, while updating and adjusting the function if

needed for the next narrative generation.

Let’s take a closer look at how this method works. Assume we start with an approved, linear

structured narrative, consisting of a number of natoms. The natoms we’ll use are photographs,

but the method works uniformly for other sorts of media. For all features that we can detect in

the natoms, a separate function is created. After analyzing the narrative, all created functions

together represent the semantic pattern of that narrative and since it was assumed to be an

approved narrative, this pattern is usable for narrative generation. However, we will not use

the abstract properties of the pattern as input for a supervised learning algorithm. Instead, we

will introduce five semantic properties, all derived from the semantic pattern mentioned

above. This choice is based on the fact that for a learning system to succeed, the

representation must be crafted with knowledge about the application domain.

2.2 Structure and Culture

The composition of narratives is subject to cultural differences in (at least) two ways:

intentional structure differences and attentional state differences [8]. Intentional structure

refers to the relations between actions and consequences within the narrative. The extent to

which these relations are comprehensible determines the perceived cohesion of the story.

However, there are great cultural differences in what people find normal reactions, depending

on their personal experiences and knowledge from their environment. For example, someone

from the western culture may find it perfectly understandable that a homeless dog gets

adopted by a caring family, while this situation would not be so obvious for someone from a

culture that do not keep animals as pets.

The attentional state is an abstraction of the focus of attention as the narrative unfolds,

modeled by a set of focus spaces. Since people from different cultures are used to different

attentional states in stories, they often have problems sympathizing with subjects of the story

6

when this known pattern is changed. It is no coincidence that all western fairytales start with

an introduction of their main characters (setup) before a dramatic problem arises (conflict)

which has to be defeated (resolution) to live happily ever after.

The method presented in this thesis has no difficulties with the above mentioned problems

because the functions adapt to the user’s habits and preferences.

3 Related work

Much work is done on modeling a narrative structure. One of the first works on structuralist

concepts is done by Propp [9], who identified 8 character archetypes and a sequence of 31

functions characterizing folk tales. The deconstructing of narratives in their component

elements expresses in his work was one of the first contributions to automatic story

generation.

For fully automatic narrative generation to succeed, the task of automatic photo annotation

must first be accomplished. This is important for the training phase as well as the actual

generation. The paper of Tuffield presents the Photocopain system [10], which is a semi-

automatic photo annotation system which combines the metadata of the photograph with a

source of information from various information extracting tools. Combining such a photo

annotator which the method presented in this thesis, would highly expand the pool of usable

natoms which in turn can be used to refine the prototype patterns.

Using the modeled structure of narratives, Screenplay System’s software program Dramatica

actually generates narratives, guided by concrete directions of the ‘human writer’. The writer

makes the choices, and Dramatica matches it with a known narrative structure from its

enormous database. Choices have to be made until just one structure matches the directions.

However, this kind of knowledge based system becomes very fragile when faced with a

dynamic problem domain or any task it was not specifically designed to do [11].

The work of Brooks [12] presents a computational narrative model which divides the task into

three functional areas: “defining an abstract narrative structure, defining a collection and

organization of story pieces with some representation of their meaning and a navigational

strategy or reasoning through that collection of story pieces”. It defines an elegant framework

including all parties involved in automatic narrating: the artist, the agent, the audience and the

story itself. Furthermore, it identifies three environments mapping to the three functional

areas mentioned earlier. These environments articulate important boundaries that are

implicitly used in this thesis. The method presented in this thesis distances itself from the

environment referred as Structural Environment by Brooks. This is the environment in which

the structure of the narrative is described in simple abstract terms. In this thesis, we reside in

the other two environments of Brooks: the Representational Environment, in which

knowledge of the various story elements is captured in the form of relationships between story

elements and the Presentational Environment, in which software agents sequence the different

story elements according to an agent’s individual stylistic preferences.

7

4 Method

The following sections will present a method that dynamically captures the semantic pattern

of a narrative by assigning five semantic properties to the features. The general idea is that,

with these properties, we can produce a narrative from a new dataset simulating the pattern of

a known narrative or known genre, while updating and adjusting the function if needed for the

next narrative generation.

The method is based on the machine learning assumption that a machine is able to predict

target labels with a sufficient set of features. The quality of feature selection is therefore of

great influence on the results of the predictor.

For the sake of consistency, all components of the method are listed below together with their

description as used in this particular context [1].

Narrative: a structured, purposeful, communication of an experience to an audience.

Natom: is a `narrative atom'. A singular irreducible component of storytelling, where further

reduction is either impossible or would cause the natom to no longer make sense. E.g. a

photograph, a sentence or short paragraph, a short video clip.

Feature: is identifiable evidence of the presence of a particular piece of content within a

narrative or piece of narrative. This might be a piece of metadata, a tag, or automatically

extracted keywords or descriptions.

Motif: is an atomic component of a theme. Motifs are generalizations or classifications of

elements with the narrative that connote a theme, they are directly denoted by features within

the narrative. E.g. Snow, Rose, Grave, Champagne.

Theme: is a high level concept that is part of the subtext of a narrative. A theme is not directly

present within the narrative but is connoted through the presence of motifs. E.g. Winter, Love,

Death, Celebration.

Property: is a meta-feature; a feature of a feature.

4.1 Function

To be able to use the data extracted from the narrative (a series of natoms), a function is fitted

onto the data points of every single tagged feature. A semantic pattern is expresses by the

combinations of all function together. However, to apply a simple pattern recognizing

algorithm on the separate functions is not likely to produce usable results. The features must

not be seen as isolated units since the relations between them contain the information we are

seeking. But even a pattern recognizer that takes several functions as input will not uncover a

workable semantic pattern. This problem arises from the concept of a semantic gap which is

the abstraction of the difference between two descriptions of the same object by different

representations. Two different representations can never represent the exact same semantic

meaning. Therefore, if we want to extract semantics that are meaningful to the human mind,

we need to bridge the semantic gap going from a computational representation of the narrative

8

to a human understandable representation. This can only be done by using properties that

already capture some semantics as an input for a machine learning algorithm. Five of these

semantic properties are listed below. Note that they represent features in a machine learning

context, but to avoid confusion with the higher level features of the thematic model, we will

call them properties.

4.2 Weight

An important property of a feature is how strong it relates to a motif. That is, with what level

of certainty does the feature connote a motif? One can easily imagine that the feature snow

fall connotes the motif snow with a very high certainty, while the feature bitter sky has a much

lower certainty of connoting the same motif (feature examples used from Hargood [1]). This

may seem trivial, but assigning weight factors to features makes it possible to assign a level of

confidence to the connotation of the whole motif (usually connoted by more than one feature).

Consequently, it becomes possible to keep track on various alternatives and assign definite

connotations after the whole narrative is analyzed. Combined with a simple path finding

algorithm, this metadata could avoid making certain connotation mistakes in analyzing tasks,

which otherwise would have resulted in noise. In creation tasks however, this property is even

more important. It gives the user the opportunity to define a certain threshold with which he

or she can set the level of obviousness of the created narrative. A high threshold produces a

more cliché story, while a lower threshold causes the production of a story that is somewhat

more difficult, but perhaps more interesting to read.

4.2.1 Derivation

Calculating weight factors is done for every motif separately. The input is defined by the

features denoting M. If these features are already known, the derivation works optimal. If they

are not exactly known on forehand, one can just insert a sequence of features of which the

denoting features are members (but not known). In the latter case, a very low weight will be

given to the features not denoting the motif, and a much higher weight will be given to the

sequence of features denoting the motif. A threshold can be applied to discover this sequence.

9

Calculating weight factors for the features of motif M

Input:

Sequence of features denoting motif M: [f1, f2, …, fq]

w(f1), .. w(fq) 1/q % set initial weights

alpha range [0.1, 0.001] % learning rate

n 1 % natoms counting

for j 1 to q % for all features

xj integral(fj) % count appearances

yj avg(xj) % get average appearances

if(yj<xj) % if fj appears more than average

zj xj-yj % get difference

w(fj) w(fj) + alpha*zj % adjust weight (increase)

for k 1 to q % for all other features

if(k /= j) % not the same as fj

w(fk) w(fk)–(alpha*zj)/(q-1) % adjust weight (decrease)

end for % end for

else % if fj appears less than average

zj yj-xj % get difference

w(fj) w(fj) – alpha*zj % adjust weight (decrease)

for(h 1 tot q) % for all other features

if(k /= j) % not the same as fj

w(fh) w(fh)+(alpha*zj)/(q-1) % adjust weight (increase)

end for % end for

avg(xj) (xj/n)+avg(xj)

n = n + 1

end for

10

4.3 Combinations

Some features cannot connote a motif by themselves; they need another feature, or any

number of features, to be present in the same natom to connote a motif. This includes also the

possibility that a natom needs to appear twice or more in the same natom. It is easy to find

examples of this phenomenon in real life. It takes more than just a spoon to denote a diner and

one swallow does not make a summer.

Therefore, the combination property is introduced. It captures the dependences between all

features, but needs only be assigned to features which have any dependency relations. For

every dependency relation, a weight is given between 0 and 1.

4.3.1 Derivation

We now have a vector for every feature which specifies for every feature (including the same

feature) how often they appear in the same natom on a continuous scale from 0 to 1. It is

optional to set thresholds to define when a combination is mandatory (for example, v[q] >

0.8) or just common (0.3 < v[q] > 0.8).

Input:

Sequence of all features: [f1, f2, …, fq]

sequence of all natoms: [n1, n2, .. nm]

for j 1 to q

vj null vector with length q

for k 1 to m

if fj is in nm

For s 1 to q

if fs is in nm % if the two features combine in this

natom

v[s] v[s] + count(fs) % put number of appearances in vector

else

v[s] v[s] % features do not combine

End for

End for

normalize vj so that all entries sum up to 1

End for

11

4.4 Position

The position in which features appear is of great importance to the semantics of the story. An

object appearing in the beginning of a narrative is more likely to present a main character than

something that gets introduced just before the narrative ends. This assumption at least holds

for classical structured stories [12] and stories following the Freytag’s pyramid structure [13],

where the beginning of the story is used to define the setting for the rest of the story. After

this introduction (also called exposition or setup), a problem arises (conflict or rising action).

The most important factors are introduced in these two first acts. The position property is

introduced to capture this semantic information.

Its contribution to creation tasks is more evident than to analyzing tasks. However, even in

analyzing tasks this property does reduce the probability of making an incorrect motif

classification. In creation tasks, this property is very important. It ensures that features which

are important to the narrative are indeed introduced in the first or second act. This property is

thus responsible for a human understandable arrangement of the created narrative.

4.4.1 Derivation

The durations of the separate acts are not known. This means no sharp line can be drawn

between ‘important objects introduced in the beginning of the narrative’ and ‘less important

objects introduced just before the end of the narrative’.

Therefore, we will use a function with a smooth transition from 1 to 0:

Fig 2: Position function

The position property is calculated by applying the position function to the first occurrence of

every feature. It is important to only consider the first occurrence, since main characters and

objects are likely to appear throughout the whole narrative, which would obviously reduce

their position property and that is just what we do not want here.

An example:

A certain object is introduced in the third natom of a sequence of 30 natoms.

12

Another object is introduced in the twentieth natom of the same sequence:

4.5 Order

What does it mean for a feature to be preceded by another feature? Sometimes nothing of

course, but sometimes a lot. The order property captures the structure within the motif. It can

be the case that a certain feature is only (partly) connoting a motif, if it is preceded by another

(series of) feature(s). This property is most similar to the combination property, but there is

one essential difference: the order property takes time into account. One could argue that the

combination property is only a special case of the order property: namely, the case that the

time between the appearances of two (or more) features, is negligible. The reason to express

them nonetheless as separated properties is that they have very different semantic meaning.

With the order property, we can express both the notion of causality and as well as

conditionality.

4.5.1 Derivation

Input:

Sequence of all features: [f1, f2, …, fq]

sequence of all natoms: [n1, n2, .. nm]

k window size

for j 1 to q

vj null vector with length q

zj null vector with length q

for k 1 to m

if fj is in nm

For s 1 to q

if fs appears in the last k natoms

v[s] v[s] + count(fs) % put number of appearances in vector

else

v[s] v[s] % features do not combine

End for

End for

normalize vj so that all entries sum up to 1

End for

13

4.6 Commotion

Some features cause more commotion than others. This property is introduced to distinguish

between background objects and foreground objects. While a background object (e.g. a tree,

chair or sky) can be important to a motif, it usually is not important to the semantics of the

narrative. A background object sometimes appears often in a sequence of natoms, but is only

part of the context of the narrative and does not add meaning to the story being told.

Misjudging such a background object is not difficult: a tree can appear frequently in a

narrative which plays in the woods, acquiring high weighting properties, but has nothing to do

with the meaning of the narrative.

The commotion property assigns a high value (ranging from 0 to 1) to features that cause

some commotion in the narrative; that is, if it causes a significant change in the semantic

pattern.

4.6.1 Derivation

To be able to detect a sudden change in the semantic pattern, a similarity measure is needed.

Consider the following data fragment:

Feature Natom1 Natom2 Natom3 Natom4 Natom5

1 1 1 1 0 0

2 0 0 3 0 0

3 2 1 1 4 5

4 1 0 0 6 3

5 0 1 0 2 3

Figure 3: Semantic pattern

A sudden change in the pattern can be noticed at natom4, while the first three natoms are

much alike. In semantic terms, natom4 shows a reaction to an earlier action. The calculation of

the commotion property can be divided in two separated tasks: detecting a significant change

in the semantic pattern and finding features that could be responsible for this change to

happen. Below, we will discuss each task apart.

14

4.6.1.1 Detecting change

In default, a window of one natom is used to compare with the next window. It is optional to

increase both windows, for example to increase robustness of the method. It is convenient to

apply bigger windows for films (a few seconds) or long texts (one or two paragraphs), but for

narratives with relatively short sequences of natoms, we will use one natom as window size.

A cosine similarity function is applied to express similarity between natoms.

function detect_commotion % detects significant changes between natoms

% in vector form

input:

sequence of natoms [n1, n2, .. nm]

q = number of features

t = threshold for similarity [-1 to 1]

for i 1 to m

get feature vector vi % entries of vector from semantic pattern

get feature vector vi+1 % next natom

if sim > t % no significant change

proceed with next natom

else

get_commotion_features % execute second task

15

4.6.1.2 Detecting responsible feature(s)

function get_commotion_features

input:

natomi showing significant change

sequence F of available features [f1,f2, .. fq]

sequence C of initial commotion property [c1, c2,.. cq]

n number of natoms to check % default: 4

t threshold for required slope

set all property values to 0

for m 1 to q

for j 1 to n % for every natom to check

v vector with length n

v[j] f(i-j) % number of appearances in preceding natom

end for

% Next, we calculate the slope of the data points (least square)

if slope > t % check to filter out unrelated features

cm cm + 1 % increase commotion property of fm

end for

normalize C

end

end for

16

4. 7 Prototyping

A valuable application of the properties as mentioned above is the specification of prototypes

or genres in terms of the properties. Having a prototype of a certain story type (e.g. drama,

documentary, detective), captured with the five introduced properties, will definitely help

generating similar structured stories, without losing the ability to create new structures or

contents as is the case with knowledge based narrative generators. If sufficient labeled data

can be analyzed and recorded, it becomes possible to create a narrative of a genre by matching

the prototype pattern with new data. An elegant collaboration of narratology and machine

learning has more potential in accomplishing this task than both academic fields would have

alone.

5 Evaluation

The operating of the method presented in this thesis will be showed using short sequences of

natoms that nonetheless have narrative structure and meaning. Analyzing larger sequences

works similarly and may result in even more accurate findings, but to avoid losing grip on the

semantics, smaller sequences are preferred for evaluation matters. Consider the following

narratives, each consisting of four natoms:

(1a) (1b) (1c) (1d)

(2a) (2b) (2c) (2d)

17

5.1 First narrative

The first sequence portrays some common daily rituals from sunrise until evening. One starts

the day with a hot cup of coffee, does some cleaning activity, spends some time behind a pc

and ends the day with cooking. Captured by the thematic model would look like the following

scheme.

Fig 4.: Data applied to model

After the annotation, listed in the above table, the properties can be extracted, following the

directions as specified in chapter 4.

Feature N1a N1b N1c

N1d

Hand

Cup

Coffee

Sponge

Glove

Foam

Plate

Keyboard

Computer

Knife

Cuttingboard

Onion

1

1

1

0

0

0

0

0

0

0

0

0

1

0

0

1

1

1

1

0

0

0

0

0

1

0

0

0

0

0

0

1

1

0

0

0

1

0

0

0

0

0

0

0

0

1

1

1

18

5.1.1 Weight property

The following table lists the adjusted weights of each feature after analyzing the whole

narrative with learning rate 0,1

Fig 5.: assigning weighted values to features

The most interesting part of the result is the hand feature. This feature appears in every single

motif, suggesting its appearance is independent from each of the four motifs. The results are

showing the same trend: the weight of the hand feature is decreasing, while the weights of all

other denoting features are increasing. Furthermore, when a lot of different features are

denoting the same motif, they get a lower initial weight assigned. In this evaluation, only one

example per motif is analyzed, but after analyzing a larger number of motif examples, this

drawback is neutralized.

Feature MotifCoffee Motifcleaning MotifPC MotifCooking

Hand

Cup

Coffee

Sponge

Glove

Foam

Plate

Keyboard

Computer

Knife

Cuttingboard

Onion

0.333

0.408

0.408

0

0

0

0

0

0

0

0

0

0.0125

0

0

0.275

0.275

0.275

0.275

0

0

0

0

0

0.333

0

0

0

0

0

0

0.408

0.408

0

0

0

0.0625

0

0

0

0

0

0

0

0

0.325

0.325

0.325

19

5.1.2 Combinations property

A representative selection of the combination properties are expressed in the scheme below.

Note that it is a relational property and that the relations are not necessarily symmetric.

Fig 6.: Schematic drawing of combinational relations

5.1.3 Position property

The position property is meaningless for this particular sequence of natoms, because it thrives

on the assumption that the narrative has the classic dramatic construction (setup, conflict,

resolution). This assumption does not apply on this story, so we will not consider the position

property here.

20

5.1.4 Order property

The order property is expresses in a relation between features. When two features often

precede each other, their relation gets a higher weight. In this short example, all relations still

have the same weight, because they appear only once. The duration between their appearances

is recorded as well. The scheme below represents the order relations between its features.

Fig 7.: Schematic drawing of ordering relations

5.1.5 Commotion property

The commotion property is, for other reasons than the position property, also not appropriate

to apply on this set of natoms. The commotion property needs smoother transitions between

the natoms to be meaningful. A close look at the semantic pattern reveals that all four

columns diverge exactly the same from each other. This means that after each natom, the

same portion of commotion arises.

21

5.2 Second narrative

The second narrative portrays a typical fishing day: getting onto a boat, doing some angling,

catching a fish and eating the caught fish.

Fig 8.: semantic pattern of the second narrative

Fig 9.: Second narrative modeled

5.2.1 Weight property

Weight properties are meaningful for almost every sequence of natoms. Deriving the weight

properties for the second narrative gives the following results.

Fig 10.: Assigning weights to the features of second narrative

Feature N2a N2b N2c N2d

Boat

Water

Fishing rod

Fish

Baked fish

Plate

1

1

0

0

0

0

1

1

3

0

0

0

0

1

1

1

0

0

0

0

0

0

1

1

Feature Motifboating Motifangling Motifcatch Motifbakedfish

Boat

Water

Fishing rod

Fish

Baked fish

Plate

0.550

0.525

0

0

0

0

0

0.525

0.700

0

0

0

0

0.358

0.333

0.408

0

0

0

0

0

0

0.575

0.575

22

5.2.2 Combination property

The non-symmetric property of the relations expresses a natural understanding: if there is

water, it is not very often the case that there is also a fish, but if we see a fish, it is probably

swimming in the water. The combination relations are captured in the following scheme.

Fig 11.: Schematic drawing of the combinatorial relations

5.2.3 Commotion property

Looking at figure 8, one can easily see that the last column is very different from the others.

The method for finding commotion properties detects such a sudden change in the pattern

using a similarity measure to compare frames. The next step is to seek responsible features for

this sudden change. This is done by calculating the slope of the fitted function onto the

semantic pattern of every feature that precedes the last natom. With this approach, the features

boat and water drop off and the method finds that the feature fish or fishing rod was

responsible for the commotion, which is true.

23

6 Conclusion

Automatic storytelling is an ambitious but tempting task. With possible applications in every

media related domain, its relevance is beyond doubt. The method presented in this thesis aims

to contribute towards this ultimate goal, distinguishing itself from related work by linking the

essences of narratology and machine learning.

The property evaluating appears promising as the results correspond to our own knowledge

and beliefs about narratives. However, it is yet too early for a definite conclusion. Due to the

subjectivity of the correctness of a generated narrative, conclusions cannot be drawn without

considering the subjects.

In this thesis, we approach the Semantic Gap by putting semantics in the properties that can

be used as inputs for a machine learning algorithm. The property evaluation suggests that this

is a fruitful approach since it follows human intuition without either using static knowledge

based rules or a large amount of abstract properties.

A disadvantage of the method which became known in the evaluation is the fact that not all

properties are equally meaningful for all narratives. This is the case with abstract properties as

well, but because of our small set of properties, this disadvantage is more serious here. It

forces developers to select training data with great care, which can be time consuming.

As this method is situated between the academic fields of narratology and machine learning, it

operates like an interface between the two. Integration from both sides will take it to the next

level.

24

7 References

[1] Hargood (2011), “Semiotic Term Expansion as the Basis for Thematic Models in

Narrative Systems”, University of Southampton, PhD Thesis

[2] M. McQuillan. The Narrative Reader. Routledge, London, 2000

[3] R. Barthes and L. Duisit. An introduction to the structural analysis of narrative. New

Literary History, 1966.

[4] T. Murray, T. Shen, J. Piemonte, C. Condit, and J. Thibedeau. Adaptivity for conceptual

and narrative flow in hyperbooks: The metalinks system. Adaptive Hypermedia and Adaptive

Web-Based Systems, 2000.

[5] M. Riedl and R. Young. Character-Focused Narrative Generation for Execution. Virtual

Storytelling, Vol. 2897Springer Berlin / Heidelberg ,2003.

[6] M. Cavazza and F. Charles. Dialogue generation in character-based interactive

storytelling. In AAAI First Annual Articial Intelligence and Interactive Digital Entertainment

Conference, California, USA, ,2005.

[7] R. Barthes. Mythologies. Editions du Seuil, 1957.

[8] B. Grosz , and C. L. Sidner in Comput. Linguist., Vol. 12 Cambridge, MA, USA: MIT

Press, p. 175—204, 1986.

[9] Propp. Morphology of the folktale. Austin, University of Texas Press, 1928

[10] M.Tuffield, S. Harris, D.P. Dupplaw, A. Chakravarthy, C. Brewster, N. Gibbins, K.

O'Hara, F. Ciravegna, D. Sleeman, N.R. Shadbolt, and Y. Wilks. Image Annotation with

Photocopain, in 'Proceedings of the First International Workshop on Semantic Web

Annotations for Multimedia (SWAMM)’, 2006.

[11] Screenplay-Systems, Dramatica. Screenplay Systems: Burbank, California, 1994.

[12] K. M. Brooks, Do Story Agents Use Rocking Chairs? The Theory and Implementation of

One Model for Computational Narrative., in Philippe Aigrain; Wendy Hall; Thomas D. C.

Little & V. Michael Bove Jr., ed., 'ACM Multimedia' , ACM Press, , pp. 317-328 , 1996.

[13] Freytag, G. (1965 (Nachdruck; EA 1863)), Die Technik des Dramas , Darmstadt .

25