JoschaBach Implementing Emotion and Motivation in AI ...

Joscha Bach

ImplementingEmotionandMotivationinAIArchitectures

TutorialAAAI2018

FromComputationtoEmotion

Howcanacomputationalsystemexperienceemotion?

Whatisemotion?Whatisexperience?Relationshipbetweenexperiencerandexperience?

AAAI 2018 Emotion and Motivation Tutorial 2

Emotionalstatesascognitiveconfigurations


Emotionsascognitiveconfigurations


Overview

• AGIperspectiveonminds• Basicarchitecturalcomponents• Modelingmotivation(MicroPsi model)• Modelsofemotion• EmotioninthePsitheory• Modelingpersonality• Emotion,selfandprosociality


BasicperspectiveongeneralAI

• Mindasmachine

• Machine=computationalsystem• computation=regularstatechange

• universalcomputation:setofcomputablefunctionsthatcancomputeallcomputablefunctions(whengivenunboundedresources)



• Accesstouniverseviadiscernibledifferences® Information

• Meaningofinformation:relationshiptochangesinotherinformation

• Intelligence:abilitytomodel• Modelingisfunctionapproximation• Purposeofmodelingisregulation,tomaximizerewards



• ClassicalAI:directmodelingofcognitivefunctionality® “firstorderAI”

• DeepLearning:systemsthatmodelfunctionalitythemselves;compositionalfunctionapproximation® “secondorderAI”

• MetaLearning:systemsthatlearnhowtobuildlearningsystems® “thirdorderAI”


BasicperspectiveongeneralAI:openquestions

• Arehumansmetalearningsystems?• Isevolutiona(slowandineffective)searchformetalearners?

• Isthereaclassofuniversalfunctionapproximatorsthatcanapproximateanyfunctionthatcanbeapproximatedbyacomputer(whengivenunboundedresources),anddoesitcontainitselfandus?


AIperspectiveonthemind

• Constructedarchitectures(Minsky,Simon,Newell)— classicalcognitivearchitectures

• Generatedarchitectures(Schmidhuber,Hutter,…):generalrecursivefunctionapproximation+rewardsystem

• Hybridperspective:mostlygenerated,butwithcomplexprerequisitesandbiases


LayersofCognition

Reflective

Deliberative

Reactive


ColumnsofCognition

Perception CognitiveProcessing Action


CognitiveGrid

ReflexivePerception Meta-

ManagementManagement

Action

DeliberativePerception

Planning,Reasoning

DeliberativeAction

ReactivePerception Reflexes Reflexive

Action


ConceptualAnalysis:HCogAff(Sloman2001)

Emotion and Motivation TutorialAAAI 2018 15

Artificial GeneralIntelligence

Methodsshouldfocusoncomponentsandperformancesnecessaryforintelligence:

• Whole,testable architectures• UniversalRepresentations:

Dynamicmodel of environment,possible worlds,and agent• (Semi-)UniversalProblemSolving:

Learning,Planning,Reasoning,Analogies,ActionControl,Reflection ...

• UniversalMotivation:Polythematic,adaptivegoal identification

• Emotionand affect


• Universalmentalrepresentations(compositional +distributedà neurosymbolic)

Componentsfor Cognitive AI



• (Semi-)Generalproblem solving:Operations over theserepresentations

(neural learning,categorization,planning,reflection,consolidation,...)


Op3 Op4Op2Op1 Opn...


• Perceptual grounding



Perception


• Perceptual grounding and action


Perception Action


• Modelof current situation,and protocol of past situations



Perception Action

Situationmodel

Protocol memory


• Modelof self




Perception Action

Situationmodel

Protocol memory

Self model

• Abstractions of objects,episodes and types




Perception Action

Self model

Declarative memory

Procedural memory Frame

• Anticipation of future developments




Perception Action

Self model

Declarative memory

Procedural memory Frame Expect.

Plans

• Actionselection and executive control



Perception Action

Longterm memory Worldmodel

Mentalstage

• Actionselection and executive control



Perception Action


Mentalstage

Action selection

• Universalmotivation:autonomous identification of goals



Perception Action


Mentalstage

Action selection

Motivational system:Motive Selection

and Decision making

• Emotionalmodulation and affect



Perception Action

Memory

Action selection


and Decision making

Urges/drives

Resolution

Arousal

Selectionthreshold

Securing rate


CognitiveArchitectures

SoarACT-R

“ClassicalCognitiveArchitectures” tendtofocusoncognitionasanisolatedproblemsolvingcapability.


AAAI 2018 Emotion and Motivation Tutorial

Perception Action

Memory

Action selection


and Decision making

Urges/drives

Resolution

Arousal

Selectionthreshold

Securing rate

31

MicroPsi architecture


PSItheoryPrinciples of Synthetic Intelligence

(Dörner1999;Bach2003,2009)


MicroPsiArchitecture–simplified

Short TermMemory/

Local PerceptualSpace

Long Term Memory(LTM)

Body Parameters

MicroPSI Agent

UrgeSensor PerceptSensorSensors/

Modulators

MemoryMaintenance

Behaviour Script Space /Execution Space

Motivation Execution

Internal Behaviours

Perception

Meta-Management

ExternalBehaviors

Action


AgentFunctionality

• Episodiclearning• Goal-directedbehavior,motivationalsystem• Emotionalmodulation• Hypothesisbasedperception• Simpleplanning• Executionofhierarchicalplans

AAAI 2018 35

Implementation:MicroPsi (Bach03,05,04,06)

Node Net Editor

Net Simulator/Agent

Execution

World Editor

Monitoring ConsoleApplication

World Simulator

3D DisplayServer

3D DisplayClient

Eclipse Environment

Emotion and Motivation Tutorial

AAAI 2018 36

Node Net Editor

Net Simulator/Agent

Execution

World Editor


World Simulator

3D DisplayServer

3D DisplayClient

Eclipse Environment

Implementation:MicroPsi(Bach03,05,04,06)

Low-levelperception


AAAI 2018 37

Node Net Editor

Net Simulator/Agent

Execution

World Editor


World Simulator

3D DisplayServer

3D DisplayClient

Eclipse Environment


Low-levelperception

ControlandsimulationEmotion and Motivation Tutorial

AAAI 2018 38

Node Net Editor

Net Simulator/Agent

Execution

World Editor


World Simulator

3D DisplayServer

3D DisplayClient

Eclipse Environment


Low-levelperception

ControlandsimulationMulti-agentinteractionEmotion and Motivation Tutorial


Node Net Editor

Net Simulator/Agent

Execution

World Editor


World Simulator

3D DisplayServer

3D DisplayClient

Eclipse Environment


Low-levelperception

ControlandsimulationMulti-agentinteraction

Robotcontrol


Differentsimulationenvironments

AAAI 2018 41

Implementation:MicroPsi 2(Bach,Welland,Vuine,Herger12,14,…)


Cognitive Artificial Intelligence


• UniversalRepresentations:Dynamicmodel of environment,possible worlds,and agent

• (Semi-)UniversalProblemSolving:Learning,Planning,Reasoning,Analogies,ActionControl,Reflection ...


• Emotionand affect• Whole,testable architectures


ModelingMotivation



ModelingMotivationinaCognitiveArchitecture

• GeneralintelligenceneedsGeneralMotivation• Motivationalsystemstructurescognition• Motivationaldynamics:physiological,socialandcognitivedrives

• Intentionselectionandactioncontrol• Motivationvs.affect

Motivationvs.emotion

• Motivation:– reflectsneeds– givesrisetogoalsanddirectedbehavior– doesnothavetobeassociatedwithemotions

• Emotion:– modulatesperception,cognition,action– receivesvalencefrommotivation– receivesobjectsfromcognition– leadstoaffectiveexpression


MotivationalSystem

• Allgoals attempt to satisfy a(hard-wired)demandà flexiblegoals,but(evolutionary)suitable behavior


Intention

Motive selection

Urges/drives

Food

Wat

er

Inte

grity

Affi

liatio

n

Legi

timac

y

Com

pete

nce

Cer

tain

ty

Aes

thet

ics

MotivationalSystem

• Drivescorrespond to set of demands of the agent


Physiological

Food

Wat

er

Inte

grity

Affi

liatio

n

Inte

rnal

Le

gitim

acy

Gen

eral

and

Spec

ific

Com

pete

nce

Unc

erta

inty

Red

uctio

n

Evol

utio

nary

and

Abs

tract

Aes

thet

ics

Social Cognitive

PhysiologicalDrives

• if autonomous regulation of body processes failsà actively managephysiology (seek food,water,healing,shelter,rest,warmth,...)à escape perilious situationsà implicitly seek physical survival


Social Drives

• Affiliation:structuresocialinteractionbeyondrationalutility

• increasedby‘legitimacysignals’,decreasedby‘antilegitimacysignals’ (andadaptivelyovertime);allowsfornon-materialrewardandpunishment

• external legitimacy:groupacceptance• internal legitimacy:“honor”,conformancetointernalizedsocialnorms


Cognitive Drives

• Competence– epistemic (problem specific)– general (ability to satisfy demands)– effect oriented

• Uncertainty reduction– novelty seeking

• Aesthetics– evolutionary preferences (stimulus oriented)– abstract (representation oriented)


MotivationalSystem

Allpossible goals correspond to (at leastone)demand


Physiological

Food

Wat

er

Inte

grity

Affi

liatio

n

Inte

rnal

Le

gitim

acy

Gen

eral

and

Spec

ific

Com

pete

nce

Unc

erta

inty

Red

uctio

n

Evol

utio

nary

and

Abs

tract

Aes

thet

ics

Social Cognitive

FromNeedstoBehavior

NeedsUrge

Signals

Memory

Perception

Action

Priming + Modulation

Learning

Decision Making



Rewardsignals

Pleasureanddistress:– Change ofademandisreflectedinpleasureor distresssignal

– Strengthisproportional toamountofchange

– Pleasureanddistresssignalsdeliverreinforcementvaluesforbehavioralproceduresandepisodicsequencesanddefineappetitive andaversive goals.

• drive =demand +urge indicator

MotivationalSystem


Wat

ertarget

currentlevel urge indicator

swater

• motive =urge +goal situation

MotivationalLearning


Wat

ertarget


swater

goal

• motive =urge +goal situation



Wat

ertarget


swater

goal situation

aversive situation

+

–

Goals

• Goal:situationoractionthataffordstosatisfyaneed

• Aversivegoal:situationoractionthatfrustrateaneed

• Allbehaviorisdirectedonsatisfyinganappetitivegoaloravoidinganaversivegoal

• Needsarepredefined,goalsarelearned


Physiologicalneeds

• Thirst• Hunger• Rest• Warmth• Libido• …

à Survivalasemergentproperty


Socialneeds

• Affiliation(Attentionfromothers,externallegitimacy)

• Internallegitimacy• Nurturing(caringforothers)• Affection• Dominance


Cognitiveneeds

• Competence:– Skillacquisition(epistemiccompetence)– Coping/controlability(generalcompetence)– Effectgeneration

• Uncertaintyreduction:– Exploration

• Aesthetics:– Stimulusoriented– Structureoriented(abstractaesthetics)


Needsandurges


• association by learning:



urgeindicator

si goalsituation

aversivesituation

+

–

w1

w2

changeindicator

Δsi

V+

V-

A+

A-

valence associator

demand

• retrogradient reinforcement



urgeindicator

siw1

changeindicator

Δsi

V+ A+

valence associator

demand

protocol chaingoal

+

Motivator:



urge

siw1

Δsi

V+ A+

valence associator

autonomousregulation

goal

situations leading up to goal =plan

+

Intention:



urge

si

autonomousregulation

goal 1

goal 2

goal 3

goal n

...

Motivationallearning


Motiveselection


Needparameters

• Strength:relativeimportance• Decay:rateofreplenishment• Gain:effectofsatisfaction• Loss:effectoffrustration

• differentconfigurationofneedparameters=differentpersonalitytraits


Anticipatedneeds

• Actualrewarddeterminesreinforcement(striatum,basalganglia)

• Anticipatedrewarddeterminesaction(prefrontaldopamine)

• Valenced reactionsarenotonlycausedbypresentrewards,butalsobyimagined/anticipatedrewards(expectations,memories)


Implementationofmotivation

𝑁𝑒𝑒𝑑 ≔

𝑡𝑦𝑝𝑒 ∈ 𝑝ℎ𝑦𝑠𝑖𝑜𝑙𝑜𝑔𝑖𝑐𝑎𝑙, 𝑠𝑜𝑐𝑖𝑎𝑙, 𝑐𝑜𝑔𝑛𝑖𝑡𝑖𝑣𝑒 ,𝑣𝑎𝑙𝑢𝑒𝑡 ∈ 0, 1 ,𝑣𝑎𝑙𝑢𝑒𝑡0 ∈ 0, 1 ,𝑢𝑟𝑔𝑒𝑡 ∈ 0, 1 ,

𝑢𝑟𝑔𝑒𝑛𝑐𝑦𝑡 ∈ 0, 1 ,𝑝𝑙𝑒𝑎𝑠𝑢𝑟𝑒𝑡 ∈ 0, 1 ,𝑝𝑎𝑖𝑛𝑡 ∈ 0, 1 ,𝑤𝑒𝑖𝑔ℎ𝑡 ∈ ℝ+,

𝑑𝑒𝑐𝑎𝑦𝑡𝑖𝑚𝑒 ∈ −1, 1 ,𝑔𝑎𝑖𝑛 ∈ 0, 1 ,𝑙𝑜𝑠𝑠 ∈ 0, 1 ,

𝑠𝑎𝑡𝑖𝑠𝑓𝑎𝑐𝑡𝑖𝑜𝑛𝑖𝑚𝑔 ∈ 0, 1 ,𝑓𝑟𝑢𝑠𝑡𝑟𝑎𝑡𝑖𝑜𝑛𝑖𝑚𝑔 ∈ 0, 1 ,

𝑝𝑙𝑒𝑎𝑠𝑢𝑟𝑒𝑠𝑒𝑛𝑠𝑖𝑡𝑖𝑣𝑖𝑡𝑦 ∈ ℝ+,𝑝𝑎𝑖𝑛𝑠𝑒𝑛𝑠𝑖𝑡𝑖𝑣𝑖𝑡𝑦 ∈ ℝ+,

𝑝𝑙𝑒𝑎𝑠𝑢𝑟𝑒𝑑𝑒𝑐𝑎𝑦𝑡𝑖𝑚𝑒 ∈ ℝ+,𝑝𝑎𝑖𝑛𝑑𝑒𝑐𝑎𝑦𝑡𝑖𝑚𝑒 ∈ ℝ+,

𝑝𝑙𝑒𝑎𝑠𝑢𝑟𝑒𝑠𝑒𝑛𝑠𝑖𝑡𝑖𝑣𝑖𝑡𝑦𝑖𝑚𝑔 ∈ 0, 1 ,𝑝𝑎𝑖𝑛𝑠𝑒𝑛𝑠𝑖𝑡𝑖𝑣𝑖𝑡𝑦𝑖𝑚𝑔 ∈ 0, 1


Urgestrengthandurgency

• 𝑢𝑟𝑔𝑒𝑡 = 𝑤𝑒𝑖𝑔ℎ𝑡 1 − 𝑣𝑎𝑙𝑢𝑒𝑡−1 012

• 𝑢𝑟𝑔𝑒𝑛𝑐𝑦𝑡 = 𝑤𝑒𝑖𝑔ℎ𝑡 𝑘−𝑟𝑒𝑚𝑎𝑖𝑛𝑖𝑛𝑔𝑡𝑖𝑚𝑒𝑡𝑘 0

12


Valueanddecayofaneed

• 𝑣𝑎𝑙𝑢𝑒𝑡 =

decay 𝑣𝑎𝑙𝑢𝑒𝑡−1+𝑔𝑎𝑖𝑛×𝛿𝑡𝑐𝑜𝑛𝑠𝑢𝑚𝑒

+𝑙𝑜𝑠𝑠×𝛿𝑡𝑎𝑣𝑒𝑟𝑠𝑖𝑣𝑒

+𝑔𝑎𝑖𝑛×𝑠𝑎𝑡𝑖𝑠𝑓𝑎𝑐𝑡𝑖𝑜𝑛𝑖𝑚𝑔𝛿𝑡𝑖𝑚𝑔𝑐𝑜𝑛𝑠𝑢𝑚𝑒

+𝑙𝑜𝑠𝑠×𝑓𝑟𝑢𝑠𝑡𝑟𝑎𝑡𝑖𝑜𝑛𝑖𝑚𝑔𝛿𝑡𝑖𝑚𝑔𝑎𝑣𝑒𝑟𝑠𝑖𝑣𝑒

0

1

• decay 𝑣𝑡−1 ≔ 𝜎 𝜎−1 𝑣𝑡−1 + duration 𝑡,𝑡−1𝑑𝑒𝑐𝑎𝑦𝑡𝑖𝑚𝑒


Pleasureandpainassociatedwithaneed

• 𝑝𝑙𝑒𝑎𝑠𝑢𝑟𝑒𝑡 =decay 𝑝𝑙𝑒𝑎𝑠𝑢𝑟𝑒𝑡−1, 𝑝𝑙𝑒𝑎𝑠𝑢𝑟𝑒𝑑𝑒𝑐𝑎𝑦𝑡𝑖𝑚𝑒+𝑔𝑎𝑖𝑛×𝑝𝑙𝑒𝑎𝑠𝑢𝑟𝑒𝑠𝑒𝑛𝑠𝑖𝑡𝑖𝑣𝑖𝑡𝑦×𝛿𝑡𝑐𝑜𝑛𝑠𝑢𝑚𝑒

+𝑔𝑎𝑖𝑛×𝑝𝑙𝑒𝑎𝑠𝑢𝑟𝑒𝑠𝑒𝑛𝑠.𝑖𝑚𝑔×𝛿𝑡𝑖𝑚𝑔𝑐𝑜𝑛𝑠𝑢𝑚𝑒

0

1

• 𝑝𝑎𝑖𝑛𝑡 =

decay 𝑝𝑎𝑖𝑛𝑡−1, 𝑝𝑎𝑖𝑛𝑑𝑒𝑐𝑎𝑦𝑡𝑖𝑚𝑒+𝑙𝑜𝑠𝑠×𝑝𝑎𝑖𝑛𝑠𝑒𝑛𝑠𝑖𝑡𝑖𝑣𝑖𝑡𝑦×𝛿𝑡𝑎𝑣𝑒𝑟𝑠𝑖𝑣𝑒

+𝑙𝑜𝑠𝑠×𝑝𝑎𝑖𝑛𝑠𝑒𝑛𝑠𝑖𝑡𝑖𝑣𝑖𝑡𝑦𝑖𝑚𝑔×𝛿𝑡𝑖𝑚𝑔𝑎𝑣𝑒𝑟𝑠𝑖𝑣𝑒

+painfromdepletion(value𝑡−1) 0

1

• painfromdepletion 𝑣𝑎𝑙𝑢𝑒 ≔ 1 − 𝑣𝑎𝑙𝑢𝑒𝜃 0

12


Eventsandconsumptions

• 𝐸𝑣𝑒𝑛𝑡 ≔

𝑐𝑜𝑛𝑠𝑢𝑚𝑝𝑡𝑖𝑜𝑛 ∈ 𝐶𝑜𝑛𝑠𝑢𝑚𝑝𝑡𝑖𝑜𝑛𝑠,𝑒𝑥𝑝𝑒𝑐𝑡𝑒𝑑𝑟𝑒𝑤𝑎𝑟𝑑 ∈ −1, 1 ,

𝑐𝑒𝑟𝑡𝑎𝑖𝑛𝑡𝑦 ∈ 0, 1 ,𝑠𝑘𝑖𝑙𝑙 ∈ 0, 1 ,

𝑟𝑒𝑚𝑎𝑖𝑛𝑖𝑛𝑔𝑡𝑖𝑚𝑒𝑡 ∈ ℝ+

• 𝐶𝑜𝑛𝑠𝑢𝑚𝑝𝑡𝑖𝑜𝑛 ≔

𝑛𝑒𝑒𝑑 ∈ 𝑁𝑒𝑒𝑑𝑠,𝑟𝑒𝑤𝑎𝑟𝑑𝑡 ∈ ℝ,

𝑡𝑜𝑡𝑎𝑙𝑟𝑒𝑤𝑎𝑟𝑑 ∈ ℝ,𝑟𝑒𝑤𝑎𝑟𝑑𝑑𝑢𝑟𝑎𝑡𝑖𝑜𝑛 ∈ ℝ+,

𝑚𝑎𝑥𝑟𝑒𝑤𝑎𝑟𝑑 ∈ ℝ,𝑑𝑖𝑠𝑐𝑜𝑢𝑛𝑡 ∈ 0, 1


Rewardsignalandrewardsummation

• 𝑠𝑖𝑔𝑛𝑎𝑙 𝑡 : = 𝑡𝑒−12𝑡2

• 𝑡1 = 𝑡 − 𝑡𝑜𝑛𝑠𝑒𝑡 ]

𝑟𝑒𝑤𝑎𝑟𝑑𝑑𝑢𝑟𝑎𝑡𝑖𝑜𝑛duration(t, t − 1)

• 𝑡1 = 𝑡 − 𝑡𝑜𝑛𝑠𝑒𝑡 ]

𝑟𝑒𝑤𝑎𝑟𝑑𝑑𝑢𝑟𝑎𝑡𝑖𝑜𝑛duration(t, t − 1)

• 𝑟𝑒𝑤𝑎𝑟𝑑𝑡 =]×𝑡𝑜𝑡𝑎𝑙𝑟𝑒𝑤𝑎𝑟𝑑𝑟𝑒𝑤𝑎𝑟𝑑𝑑𝑢𝑟𝑎𝑡𝑖𝑜𝑛 ∫ 𝑡𝑒−

12𝑡2𝑡2

𝑡1 −𝑚𝑎𝑥𝑟𝑒𝑤𝑎𝑟𝑑

𝑚𝑎𝑥𝑟𝑒𝑤𝑎𝑟𝑑

= ]×𝑡𝑜𝑡𝑎𝑙𝑟𝑒𝑤𝑎𝑟𝑑𝑟𝑒𝑤𝑎𝑟𝑑𝑑𝑢𝑟𝑎𝑡𝑖𝑜𝑛

𝑒−12𝑡12− 𝑒−

12𝑡22

−𝑚𝑎𝑥𝑟𝑒𝑤𝑎𝑟𝑑

𝑚𝑎𝑥𝑟𝑒𝑤𝑎𝑟𝑑


Actualizedrewardschangevaluesofneeds

• 𝛿𝑡𝑐𝑜𝑛𝑠𝑢𝑚𝑒 = 𝑟𝑒𝑤𝑎𝑟𝑑𝑡𝑐𝑜𝑛𝑠𝑢𝑚𝑝𝑡𝑖𝑜𝑛

0

∞

• 𝛿𝑡𝑎𝑣𝑒𝑟𝑠𝑖𝑣𝑒 = 𝑟𝑒𝑤𝑎𝑟𝑑𝑡𝑐𝑜𝑛𝑠𝑢𝑚𝑝𝑡𝑖𝑜𝑛

−∞

0


Dealingwithanticipatedrewards

• 𝛿𝑡𝑖𝑚𝑔𝑐𝑜𝑛𝑠𝑢𝑚𝑒 = 𝑐𝑒𝑟𝑡𝑎𝑖𝑛𝑡𝑦×𝑠𝑘𝑖𝑙𝑙× 𝑟𝑒𝑤𝑎𝑟𝑑𝑡

𝑐𝑜𝑛𝑠𝑢𝑚𝑝𝑡𝑖𝑜𝑛

1+𝑑×𝑟𝑒𝑚𝑎𝑖𝑛.𝑡𝑖𝑚𝑒𝑡 0

∞

• 𝛿𝑡𝑖𝑚𝑔𝑎𝑣𝑒𝑟𝑠𝑖𝑣𝑒 = 𝑐𝑒𝑟𝑡.×(1 − 𝑠𝑘𝑖𝑙𝑙)× 𝑟𝑒𝑤𝑎𝑟𝑑𝑡

𝑐𝑜𝑛𝑠𝑢𝑚𝑝𝑡𝑖𝑜𝑛

1+𝑑×𝑟𝑒𝑚𝑎𝑖𝑛.𝑡𝑖𝑚𝑒𝑡 −∞

0

(d isdiscount;hyperbolicdiscounting)


Cognitive Artificial Intelligence


• UniversalRepresentations:Dynamicmodel of environment,possible worlds,and agent

• (Semi-)UniversalProblemSolving:Learning,Planning,Reasoning,Analogies,ActionControl,Reflection ...


• Emotionand affect• Whole,testable architectures


ModelingEmotion


Emotionalexpression

• PaulEkman:FacialActionCoding


Emotionalexpression



Affectivecomputing

Recognize,process,simulate,expresshumanaffects


Appraisalmodels

• MagdaArnold,RichardLazarus:– Emotionascognitiveappraisalsofrelations,motivation,cognition

• KlausScherer:– Stimulusevaluationchecks– innate(sensorymotor)® learned(schemas)® deliberate– relevance® implication® copingpotential® normativesignific.



Conceptualanalysis:Ortony,Clore,Collins1988:


AffectivedimensionsinthePSItheory(Dörner1999)

Valence(appentence/aversion)

Arousal(unspecificsympathicussyndrome)

Selectionthreshold(motivedominance)


Compare:Affectivedimensions(Wundt1910)


AffectivedimensionsinthePSItheory

Valence(appentence/aversion)

Arousal(unspecificsympathicussyndrome)

Selectionthreshold(motivedominance)

Resolutionlevel(focus)

Securingrate

Goaldirectedness


ThePsitheoryaboutemotion

• Affectisseenasaconfiguration ofacognitivesystem

• Modulatorsofcognition:– arousal,selectionthreshold,securingthreshold,resolutionlevel

– estimateofcompetenceandcertainty– pleasure/distresssignals® valence

• Affectivestateisemergentpropertyofmodulation

• Directedaffects(higher-levelemotions)emergebyassociationofdemandwithappetive oraversiveobjects/situations


Purposeofemotionalmodulation

• Controlwidth,depthandbiasofoperationsonmentalrepresentationsoftheagent®modifyperception,memory,planningandactionselection

• Reducecomplexityofcognitiveprocesses


ModulationinPSI/MicroPsi


Primarymodulators

Attentional modulators


Modulatordynamics


Modulatorparameters

• Baseline• Range• Volatility• Duration

• Differentmodulatorparameterconfigurations=differenttemperaments


Emotions as directed affect +Modulation

Examples:

Fear:anticipation of aversive events (à neg.valence)+arousalAnxiety:uncertainty (à neg.valence)+low competence +arousal,highsecuring behavior (frequent background checks)



Examples:

Anger:Perceived obstacle (usually agent)manifestly prevented reachingof anactive,motivationally relevantgoal (àneg.valence),sanctioningbehavior tendency (à goal relevance is re-directed to sanctioning ofobstacle),arousal,low resolution level,highaction readyness,highselection threshold

Sadness:Manifestprevention from all conceived ways of reachingactive,relevantgoal,without relevantobstacle (à neg.valence),support-seeking behavior (by increased demand for affiliation),low arousal,inhibition of active goalà decreased action readyness



Examples:

• Pride:highcompetence (à low securing rate),highinternal legitimacy,likely coincidence with highexternallegitimacy

• Joy:higharousal +highperceived reward signal fromsatisfying ademand

• Bliss:low arousal +highperceived reward signal fromsatisfying ademand (since physiological demands ofteninvolve higharousal,mostly related to cognitive demands,suchas aesthetics)


Implementationofaffectivemodulation

• 𝑀𝑜𝑑𝑢𝑙𝑎𝑡𝑜𝑟 ≔

𝑚𝑖𝑛 ∈ ℝ𝑚𝑎𝑥 ∈ ℝ,

𝑙𝑒𝑣𝑒𝑙𝑡 ∈ [𝑚𝑖𝑛,𝑚𝑎𝑥],𝑏𝑎𝑠𝑒𝑙𝑖𝑛𝑒 ∈ [𝑚𝑖𝑛,𝑚𝑎𝑥],

𝑣𝑜𝑙𝑎𝑡𝑖𝑙𝑖𝑡𝑦 ∈ ℝ+,𝑑𝑒𝑐𝑎𝑦𝑡𝑖𝑚𝑒 ∈ ℝ+

• 𝑖𝑛𝑡𝑒𝑟𝑣𝑎𝑙 = d𝑚𝑎𝑥 − 𝑏𝑎𝑠𝑒𝑙𝑖𝑛𝑒, if𝑙𝑒𝑣𝑒𝑙𝑡−1 > 𝑏𝑎𝑠𝑒𝑙𝑖𝑛𝑒𝑏𝑎𝑠𝑒𝑙𝑖𝑛𝑒 − 𝑚𝑖𝑛, else

• 𝛿𝑡 = 𝑡𝑎𝑟𝑔𝑒𝑡×𝑖𝑛𝑡𝑒𝑟𝑣𝑎𝑙 + 𝑏𝑎𝑠𝑒𝑙𝑖𝑛𝑒 − 𝑙𝑒𝑣𝑒𝑙𝑡−1 𝑣𝑜𝑙𝑎𝑡𝑖𝑙𝑖𝑡𝑦


Implementationofvalence

• marginalsum 𝑉, 𝑙𝑖𝑚𝑖𝑡 : = ∑ 𝑆𝑛|𝑉|𝑛=0 l 𝑆𝑛 ≔

𝑙𝑖𝑚𝑖𝑡−𝑆𝑛−1𝑙𝑖𝑚𝑖𝑡

𝑣𝑛• 𝑙𝑖𝑚𝑖𝑡 = max 𝜔|𝜔 ∈ 𝑤𝑒𝑖𝑔ℎ𝑡𝑛𝑒𝑒𝑑𝑠

• 𝑐𝑜𝑚𝑏𝑖𝑛𝑒𝑑𝑝𝑎𝑖𝑛 = marginalsum weightneed×𝑝𝑎𝑖𝑛𝑛𝑒𝑒𝑑

• 𝑐𝑜𝑚𝑏𝑖𝑛𝑒𝑑𝑝𝑙𝑒𝑎𝑠𝑢𝑟𝑒 = marginalsum weightneed×𝑝𝑙𝑒𝑎𝑠𝑢𝑟𝑒𝑛𝑒𝑒𝑑

• 𝑡𝑎𝑟𝑔𝑒𝑡𝑣𝑎𝑙𝑒𝑛𝑐𝑒 = 𝑐𝑜𝑚𝑏𝑖𝑛𝑒𝑑𝑝𝑙𝑒𝑎𝑠𝑢𝑟𝑒−𝑐𝑜𝑚𝑏𝑖𝑛𝑒𝑑𝑝𝑎𝑖𝑛𝑙𝑖𝑚𝑖𝑡


Implementationofarousal

• 𝑐𝑜𝑚𝑏𝑖𝑛𝑒𝑑𝑢𝑟𝑔𝑒 = marginalsum weightneed×𝑢𝑟𝑔𝑒𝑛𝑒𝑒𝑑𝑙𝑖𝑚𝑖𝑡

• 𝑐𝑜𝑚𝑏𝑖𝑛𝑒𝑑𝑢𝑟𝑔𝑒𝑛𝑐𝑦 = marginalsum weightneed×𝑢𝑟𝑔𝑒𝑛𝑐𝑦𝑛𝑒𝑒𝑑𝑙𝑖𝑚𝑖𝑡

• 𝑡𝑎𝑟𝑔𝑒𝑡𝑎𝑟𝑜𝑢𝑠𝑎𝑙 = 𝑐𝑜𝑚𝑏𝑖𝑛𝑒𝑑𝑢𝑟𝑔𝑒 + 𝑐𝑜𝑚𝑏𝑖𝑛𝑒𝑑𝑢𝑟𝑔𝑒𝑛𝑐𝑦 − 1


Implementationofaggression/regression

• 𝑒𝑝𝑖𝑠𝑡𝑒𝑚𝑖𝑐𝑐𝑜𝑚𝑝𝑒𝑡𝑒𝑛𝑐𝑒 = 𝑠𝑘𝑖𝑙𝑙𝑐𝑢𝑟𝑟𝑒𝑛𝑡𝑔𝑜𝑎𝑙𝑒𝑣𝑒𝑛𝑡

• 𝑔𝑒𝑛𝑒𝑟𝑎𝑙𝑐𝑜𝑚𝑝𝑒𝑡𝑒𝑛𝑐𝑒 = valuecompetence×𝑒𝑝𝑖𝑠𝑡𝑒𝑚𝑖𝑐𝑐𝑜𝑚𝑝.�

• 𝑡𝑎𝑟𝑔𝑒𝑡𝑎𝑔𝑔𝑟𝑒𝑠𝑠𝑖𝑜𝑛 = 𝑔𝑒𝑛𝑒𝑟𝑎𝑙𝑐𝑜𝑚𝑝.+𝑒𝑝𝑖𝑠𝑡𝑒𝑚𝑖𝑐𝑐𝑜𝑚𝑝.−1


Emotionviewer


Personalitymodeling

• Motivationparameters:Personalityproperties• Modulationparameters:Temperament


IndividualVariations by Parameterizing

Possible grounding of personality properties (FFM):• Openness:appreciation of art and new ideas,curiousity

• Conscientiousness:rulefollowing vs.chaotic• Extraversion:tendency to seek stimulation byenvironment and others

• Agreeableness:tendency for cooperativeness andcompassion

• Neuroticism:emotionalstability,effect of failureto self-confidence


NeedsandBigFive


Example:Mappingto FFM(BigFive)

Demanddynamics:


Physiological

Food Water Integrity Affiliation Internal Legitimacy

Gen./Epis.Competence

UncertaintyReduction Aesthetics

Social Cognitive


Demanddynamics:


Physiological

Affiliation Internal Legitimacy


UncertaintyReduction Aesthetics

Social Cognitive

Food Water Integrity


Demanddynamics:




UncertaintyReduction

Aesthetics


Valence:Pleasure/Pain signals





V+

V-

Aesthetics






V+

Aesthetics

V-

Neuroticism:strongerexperienceofnegativeemotions,loweremotionalstability(strongnegativereward/strongerlossofcompetence,certainty;strongerdecay->morefrequentneedforreplenishment,possiblylowergain),pronenesstoanxietyduetolossofcertainty


Extraversion:surgency,activity insocial relations,expressivity(stronggain for affiliation and competence,highdecay of affiliation)





V+

V-

Aesthetics


Openness:desire for novelty,intellectual independence,non-conservatism,appreciation for art and new ideas(stronggain for uncertainty reduction,highepistemic competence foruncertainty reduction,probably strongrewards for alldemands)





V-

AestheticsV+


Agreeableness:(strongpositiveand negativereward for affiliation,lowergains for competence)





V+

V-

Aesthetics


Conscientousness,Rigidity:(highloss incompetence,highselection threshold,possibly low effect of demands onarousal and resolutionlevel)





V+

V-

Aesthetics


• Why notone free variableperFFMdimension?


FFMdoes nottell the complete story

Shyness !=Introversion(highloss of affiliation,low competence)





V+

V-

Aesthetics

Howcanweevaluateamodelofmotivation?

• Games!

Emotion and Motivation TutorialAAAI 2018 118

SpaceInvaders(1978,TomohiroNishikado)


WorldofWarcraft (RobPardo,JeffKaplanetal.2004)


Playerpersonalitytypes

RichardBartle(1996):“Hearts,Clubs,Diamonds,Spades:PlayersWhosuitMUDs”


Motivationandpersonality

• Personalitypropertiescanbemodeledasmotivationalvariability

Affiliation

CompetenceUncertaintyreduction


Needsandplayertypes(withS.Tekovsky)


EmotionandtheSelf


Neocortexasamodelingsystem

• corticalmodelingisapredictivemodelofsensorypatterns,asadynamicsimulationofworldandagent



• Primarymodel:environmentandagent• Secondarymodel:interactionofagentwithenvironment

• Tertiarymodel:functioningofsecondarymodelingàSelf

• Consciousnessasamodelofattention


Integrationofemotionandmotivation

• Controlandselfregulationdependsonlearnedfunctionalrepresentationsincorticalstructure

• Attentionalbiases

• Attentionismechanismfordirectedlearning• Experientialaccessviaattentionalprotocol

• Structureofselfdeterminesexperienceofemotion


Socialemotions

• Objectofemotionrequiresrepresentationandmotivationalrelevance

• Socialurges:affiliation,romanticaffect,libido,dominancearetransactional

• Love:non-transactionalemotion

• Loverequiressharedpurpose,vialegitimacy


Acknowledgements

WorkonMicroPsi2iscollaborativeeffort:- RonnieVuine,DominikWelland,PriskaHerger,JonasKemperarecontributorstothecurrentversion

- Architecture/conceptshavebeeninspiredbyDietrichDörner,AaronSloman,MarvinMinsky,StanFranklinandmanyothers

- SupportfromHumboldtUniversityofBerlin,UniversityofOsnabrück(InstituteforCognitiveScience),BerlinSchoolofMindandBrain,HarvardProgramofEvolutionaryDynamics,MITMediaLab


Thank you!

Interesting questions:

• Is recursive function approximation plusmotivationalsystem sufficient for generalintelligence?

• Could we functionally recreate human-likemindswith our model?

• How does amotivated/emotionalsystem evolvewhen it can modify itself?


JoschaBach Implementing Emotion and Motivation in AI ...

Documents

Transcript of JoschaBach Implementing Emotion and Motivation in AI ...