The ER Model: from Plot generation to Visual...

53
Rafael Pérez y Pérez Universidad Autónoma Metropolitana, Cuajimalpa The ER Model: from Plot generation to Visual Composition

Transcript of The ER Model: from Plot generation to Visual...

Page 1: The ER Model: from Plot generation to Visual Compositionprosecco.computationalcreativity.net/sites/default/files/S04P03-Rafa… · Actions 1 and previous stories 1. Building the knowledge

Rafael Pérez y Pérez

Universidad Autónoma Metropolitana, Cuajimalpa

The ER Model: from Plot generation to Visual Composition

Page 2: The ER Model: from Plot generation to Visual Compositionprosecco.computationalcreativity.net/sites/default/files/S04P03-Rafa… · Actions 1 and previous stories 1. Building the knowledge

Rafael

Rafael

Nora

Eduardo

Mari Vicente Santiago

Page 3: The ER Model: from Plot generation to Visual Compositionprosecco.computationalcreativity.net/sites/default/files/S04P03-Rafa… · Actions 1 and previous stories 1. Building the knowledge

The Engagement-Reflection

Model

Page 4: The ER Model: from Plot generation to Visual Compositionprosecco.computationalcreativity.net/sites/default/files/S04P03-Rafa… · Actions 1 and previous stories 1. Building the knowledge

Material to be evaluated

Engagement

Reflection

Guidelines

The process of developing new stories consists of a cycle between the Engaged and Reflective States

Page 5: The ER Model: from Plot generation to Visual Compositionprosecco.computationalcreativity.net/sites/default/files/S04P03-Rafa… · Actions 1 and previous stories 1. Building the knowledge

It is inspired on Sharples (1999) cognitive account of writing:

Sharples, M. (1999). How we write? Writing as creative design. London: Routledge.

Page 6: The ER Model: from Plot generation to Visual Compositionprosecco.computationalcreativity.net/sites/default/files/S04P03-Rafa… · Actions 1 and previous stories 1. Building the knowledge

The Engagement-Reflection

Computer Model

Page 7: The ER Model: from Plot generation to Visual Compositionprosecco.computationalcreativity.net/sites/default/files/S04P03-Rafa… · Actions 1 and previous stories 1. Building the knowledge
Page 8: The ER Model: from Plot generation to Visual Compositionprosecco.computationalcreativity.net/sites/default/files/S04P03-Rafa… · Actions 1 and previous stories 1. Building the knowledge

K-B creation

E-R Cycle

input

Initial action

Output

Knowledge Base

Page 9: The ER Model: from Plot generation to Visual Compositionprosecco.computationalcreativity.net/sites/default/files/S04P03-Rafa… · Actions 1 and previous stories 1. Building the knowledge

K-B creation

E-R Cycle

input

Initial action

Output

The user

- Parameters - Set of actions - Examples of “Previous Stories”

Knowledge Base

Page 10: The ER Model: from Plot generation to Visual Compositionprosecco.computationalcreativity.net/sites/default/files/S04P03-Rafa… · Actions 1 and previous stories 1. Building the knowledge

Automatic Plot Evaluation

I am also developing computer models for automatic plot evaluation

Page 11: The ER Model: from Plot generation to Visual Compositionprosecco.computationalcreativity.net/sites/default/files/S04P03-Rafa… · Actions 1 and previous stories 1. Building the knowledge
Page 12: The ER Model: from Plot generation to Visual Compositionprosecco.computationalcreativity.net/sites/default/files/S04P03-Rafa… · Actions 1 and previous stories 1. Building the knowledge

Collaborative Storytelling

Interdisciplinary Group on Computational Creativity

Page 13: The ER Model: from Plot generation to Visual Compositionprosecco.computationalcreativity.net/sites/default/files/S04P03-Rafa… · Actions 1 and previous stories 1. Building the knowledge
Page 14: The ER Model: from Plot generation to Visual Compositionprosecco.computationalcreativity.net/sites/default/files/S04P03-Rafa… · Actions 1 and previous stories 1. Building the knowledge
Page 15: The ER Model: from Plot generation to Visual Compositionprosecco.computationalcreativity.net/sites/default/files/S04P03-Rafa… · Actions 1 and previous stories 1. Building the knowledge

Actions 1 and previous stories 1.

Building the knowledge base.

Knowledge Base 1

E

R

Agent 1

Actions 2 and previous stories 2.

Building the knowledge base.

Knowledge Base 2

E

R

Agent 2

Improvisación

Previous Stories 1

Previous Stories 2

Page 16: The ER Model: from Plot generation to Visual Compositionprosecco.computationalcreativity.net/sites/default/files/S04P03-Rafa… · Actions 1 and previous stories 1. Building the knowledge

Actions 1 and previous stories 1.

Building the knowledge base.

Knowledge Base 1

E

R

Agent 1

Actions 2 and previous stories 2.

Building the knowledge base.

Knowledge Base 2

E

R

Agent 2

Improvisación

Previous Stories 1

Previous Stories 2

We are interested in explicitly representing social hierarchy

and social norms

Page 17: The ER Model: from Plot generation to Visual Compositionprosecco.computationalcreativity.net/sites/default/files/S04P03-Rafa… · Actions 1 and previous stories 1. Building the knowledge

Actions 1 and previous stories 1.

Building the knowledge base.

Knowledge Base 1

E

R

Agent 1

Actions 2 and previous stories 2.

Building the knowledge base.

Knowledge Base 2

E

R

Agent 2

Improvisación

Previous Stories 1

Previous Stories 2

We have developed the hypothesis that, as the difference between the two sets of social norms grows the originality

of the collective narratives developed by the system increases

Page 18: The ER Model: from Plot generation to Visual Compositionprosecco.computationalcreativity.net/sites/default/files/S04P03-Rafa… · Actions 1 and previous stories 1. Building the knowledge

Actions 1 and previous stories 1.

Building the knowledge base.

Knowledge Base 1

E

R

Agent 1

Actions 2 and previous stories 2.

Building the knowledge base.

Knowledge Base 2

E

R

Agent 2

Improvisación

Previous Stories 1

Previous Stories 2

However, if such a difference keeps on growing the system

will reach a point where agents cannot communicate anymore and therefore they will be incapable of

developing collective narratives

Page 19: The ER Model: from Plot generation to Visual Compositionprosecco.computationalcreativity.net/sites/default/files/S04P03-Rafa… · Actions 1 and previous stories 1. Building the knowledge

Actions 1 and previous stories 1.

Building the knowledge base.

Knowledge Base 1

E

R

Agent 1

Actions 2 and previous stories 2.

Building the knowledge base.

Knowledge Base 2

E

R

Agent 2

Improvisación

Previous Stories 1

Previous Stories 2

We are interested in identifying and studying the characteristics of this breaking point

Page 20: The ER Model: from Plot generation to Visual Compositionprosecco.computationalcreativity.net/sites/default/files/S04P03-Rafa… · Actions 1 and previous stories 1. Building the knowledge

Exploring different ways of presenting the plots generated by the

system

Page 21: The ER Model: from Plot generation to Visual Compositionprosecco.computationalcreativity.net/sites/default/files/S04P03-Rafa… · Actions 1 and previous stories 1. Building the knowledge

MEXICA presents its outputs in three possible ways:

As a sequence of actions

As a more elaborated (predefined) text

As a visual narrative

(How text influence image

How image influence text)

Page 22: The ER Model: from Plot generation to Visual Compositionprosecco.computationalcreativity.net/sites/default/files/S04P03-Rafa… · Actions 1 and previous stories 1. Building the knowledge

Visual Daydreamer

Page 23: The ER Model: from Plot generation to Visual Compositionprosecco.computationalcreativity.net/sites/default/files/S04P03-Rafa… · Actions 1 and previous stories 1. Building the knowledge
Page 24: The ER Model: from Plot generation to Visual Compositionprosecco.computationalcreativity.net/sites/default/files/S04P03-Rafa… · Actions 1 and previous stories 1. Building the knowledge
Page 25: The ER Model: from Plot generation to Visual Compositionprosecco.computationalcreativity.net/sites/default/files/S04P03-Rafa… · Actions 1 and previous stories 1. Building the knowledge
Page 26: The ER Model: from Plot generation to Visual Compositionprosecco.computationalcreativity.net/sites/default/files/S04P03-Rafa… · Actions 1 and previous stories 1. Building the knowledge

In the visual narrative , each action is represented by a scene.

Page 27: The ER Model: from Plot generation to Visual Compositionprosecco.computationalcreativity.net/sites/default/files/S04P03-Rafa… · Actions 1 and previous stories 1. Building the knowledge

This is an example combining illustrations and elaborated

predefined texts

Page 28: The ER Model: from Plot generation to Visual Compositionprosecco.computationalcreativity.net/sites/default/files/S04P03-Rafa… · Actions 1 and previous stories 1. Building the knowledge

Princess's frame of mind was very volatile and without thinking about it princess charged against jaguar knight

Page 29: The ER Model: from Plot generation to Visual Compositionprosecco.computationalcreativity.net/sites/default/files/S04P03-Rafa… · Actions 1 and previous stories 1. Building the knowledge

Princess threw some dust in jaguar knight's face. Then, using a dagger princess perforated jaguar knight's chess

Page 30: The ER Model: from Plot generation to Visual Compositionprosecco.computationalcreativity.net/sites/default/files/S04P03-Rafa… · Actions 1 and previous stories 1. Building the knowledge

Princess got that depress that committed suicide

Page 31: The ER Model: from Plot generation to Visual Compositionprosecco.computationalcreativity.net/sites/default/files/S04P03-Rafa… · Actions 1 and previous stories 1. Building the knowledge

The End

Page 32: The ER Model: from Plot generation to Visual Compositionprosecco.computationalcreativity.net/sites/default/files/S04P03-Rafa… · Actions 1 and previous stories 1. Building the knowledge
Page 33: The ER Model: from Plot generation to Visual Compositionprosecco.computationalcreativity.net/sites/default/files/S04P03-Rafa… · Actions 1 and previous stories 1. Building the knowledge

Trope Tank Lab

Working at the

Nick Montfort Fox Harrell

Page 34: The ER Model: from Plot generation to Visual Compositionprosecco.computationalcreativity.net/sites/default/files/S04P03-Rafa… · Actions 1 and previous stories 1. Building the knowledge

We are trying to design a more complete narrative

generator.

SLANT

Page 35: The ER Model: from Plot generation to Visual Compositionprosecco.computationalcreativity.net/sites/default/files/S04P03-Rafa… · Actions 1 and previous stories 1. Building the knowledge

The general idea is, based on our previous work, to produce a new interesting model

Page 36: The ER Model: from Plot generation to Visual Compositionprosecco.computationalcreativity.net/sites/default/files/S04P03-Rafa… · Actions 1 and previous stories 1. Building the knowledge

Exploring the use of the ER Model in Design

Page 37: The ER Model: from Plot generation to Visual Compositionprosecco.computationalcreativity.net/sites/default/files/S04P03-Rafa… · Actions 1 and previous stories 1. Building the knowledge

The ERI Designer

Page 38: The ER Model: from Plot generation to Visual Compositionprosecco.computationalcreativity.net/sites/default/files/S04P03-Rafa… · Actions 1 and previous stories 1. Building the knowledge
Page 39: The ER Model: from Plot generation to Visual Compositionprosecco.computationalcreativity.net/sites/default/files/S04P03-Rafa… · Actions 1 and previous stories 1. Building the knowledge
Page 40: The ER Model: from Plot generation to Visual Compositionprosecco.computationalcreativity.net/sites/default/files/S04P03-Rafa… · Actions 1 and previous stories 1. Building the knowledge

A Computer Model for the Automatic Generation of Visual Compositions

Page 41: The ER Model: from Plot generation to Visual Compositionprosecco.computationalcreativity.net/sites/default/files/S04P03-Rafa… · Actions 1 and previous stories 1. Building the knowledge
Page 42: The ER Model: from Plot generation to Visual Compositionprosecco.computationalcreativity.net/sites/default/files/S04P03-Rafa… · Actions 1 and previous stories 1. Building the knowledge
Page 43: The ER Model: from Plot generation to Visual Compositionprosecco.computationalcreativity.net/sites/default/files/S04P03-Rafa… · Actions 1 and previous stories 1. Building the knowledge
Page 44: The ER Model: from Plot generation to Visual Compositionprosecco.computationalcreativity.net/sites/default/files/S04P03-Rafa… · Actions 1 and previous stories 1. Building the knowledge

Current PhD research projects

Page 45: The ER Model: from Plot generation to Visual Compositionprosecco.computationalcreativity.net/sites/default/files/S04P03-Rafa… · Actions 1 and previous stories 1. Building the knowledge

A virtual developmental agent

Wendy Aguilar

Page 46: The ER Model: from Plot generation to Visual Compositionprosecco.computationalcreativity.net/sites/default/files/S04P03-Rafa… · Actions 1 and previous stories 1. Building the knowledge

• We are working on a computer model that:

Objectives

We are interested in the origins of the creative behavior.

• Initially is provided with basic knowledge structures and skills and, through interaction with its environment, it develops automatically novel behaviors

Page 47: The ER Model: from Plot generation to Visual Compositionprosecco.computationalcreativity.net/sites/default/files/S04P03-Rafa… · Actions 1 and previous stories 1. Building the knowledge

The virtual world

The agent

Page 48: The ER Model: from Plot generation to Visual Compositionprosecco.computationalcreativity.net/sites/default/files/S04P03-Rafa… · Actions 1 and previous stories 1. Building the knowledge

The computer program implementing the agent’s architecture

Page 49: The ER Model: from Plot generation to Visual Compositionprosecco.computationalcreativity.net/sites/default/files/S04P03-Rafa… · Actions 1 and previous stories 1. Building the knowledge

A Computer Model for Social Norms Representation

Carlos Iván Guerrero Román

Page 50: The ER Model: from Plot generation to Visual Compositionprosecco.computationalcreativity.net/sites/default/files/S04P03-Rafa… · Actions 1 and previous stories 1. Building the knowledge

Representation of social norms

Page 51: The ER Model: from Plot generation to Visual Compositionprosecco.computationalcreativity.net/sites/default/files/S04P03-Rafa… · Actions 1 and previous stories 1. Building the knowledge

Development of a computational model for the generalization of the Knowledge Inspired in

the theory R-R (Representational -Redescription)

Edgar Morales Palafox

Page 52: The ER Model: from Plot generation to Visual Compositionprosecco.computationalcreativity.net/sites/default/files/S04P03-Rafa… · Actions 1 and previous stories 1. Building the knowledge
Page 53: The ER Model: from Plot generation to Visual Compositionprosecco.computationalcreativity.net/sites/default/files/S04P03-Rafa… · Actions 1 and previous stories 1. Building the knowledge

www.rafaelperezyperez.com

[email protected]

Thanks!