Motivation and Cognition: From Regulatory Fit to Reinforcement Learning

Motivation and Cognition: Motivation and Cognition: From Regulatory Fit to From Regulatory Fit to

Reinforcement LearningReinforcement Learning

Darrell A. WorthyDarrell A. Worthy

University of Texas, AustinUniversity of Texas, Austin

Motivation and CognitionMotivation and Cognition Why study motivation?Why study motivation?

Need to understand Need to understand how goals and rewards how goals and rewards influence cognition and influence cognition and behavior.behavior.

More complicated than More complicated than anecdotal notionsanecdotal notions

Approach vs. avoidance Approach vs. avoidance distinctiondistinction

Global incentive vs. local Global incentive vs. local goal pursuit mechanism goal pursuit mechanism

Leads to regulatory fit or Leads to regulatory fit or mismatchmismatch

Regulatory fit affects Regulatory fit affects cognition and behaviorcognition and behavior

Overview of TalkOverview of Talk Regulatory Fit FrameworkRegulatory Fit Framework Regulatory fit affects cognitionRegulatory fit affects cognition Tests of the Regulatory Fit HypothesisTests of the Regulatory Fit Hypothesis

Extend framework to examine effects of Extend framework to examine effects of social pressure.social pressure.

Regulatory Fit and Decision-makingRegulatory Fit and Decision-making

Future DirectionsFuture Directions

Regulatory Fit Regulatory Fit FrameworkFramework

FitFit MismatchMismatch

MismatchMismatch FitFit

Promotion Focus

Prevention Focus

Gain

sL

oss

es

Local

Goal

Pu

rsu

it

Mech

an

ism

Global Incentive

Global incentive Global incentive focus interacts focus interacts with local with local reward structure reward structure

Produces a Fit or Produces a Fit or a Mismatch (e.g. a Mismatch (e.g. Higgins, 2000).Higgins, 2000).

Almost all Almost all cognitive cognitive research involves research involves promotion focus promotion focus with gains with gains reward reward structure.structure.

Manipulation of Regulatory Manipulation of Regulatory FocusFocus

(Global Task Goal)(Global Task Goal)Promotion Promotion Focus Focus (Approach)(Approach)

Achieve Global Task Achieve Global Task Performance Criterion Performance Criterion Raffle ticket for $50Raffle ticket for $50

Prevention Prevention Focus Focus (Avoidance)(Avoidance)

Achieve Global Task Achieve Global Task Performance Criterion Performance Criterion Keep $50 raffle ticket given Keep $50 raffle ticket given initiallyinitially

Manipulation of Goal-Pursuit Manipulation of Goal-Pursuit MechanismMechanism

(Local Trial-by-trial Task Goal)(Local Trial-by-trial Task Goal)

GainsGainsCorrect Response = 3 pointsCorrect Response = 3 points

Incorrect Response = 1 pointIncorrect Response = 1 point

LossesLossesCorrect Response = -1 pointCorrect Response = -1 point

Incorrect Response = -3 pointIncorrect Response = -3 point

Effects of Regulatory Fit Effects of Regulatory Fit Previous researchPrevious research

Regulatory Fit leads to:Regulatory Fit leads to: Increased sense of ‘feeling right’ (Higgins, Increased sense of ‘feeling right’ (Higgins,

2000)2000) Increased motivational strength (Spiegel et Increased motivational strength (Spiegel et

al., 2004)al., 2004) Increased “cognitive flexibility” (Shah et al., Increased “cognitive flexibility” (Shah et al.,

1998)1998)

Flexibility can be defined within tasksFlexibility can be defined within tasks Category-learning -willingness to test Category-learning -willingness to test

various strategiesvarious strategies Decision-making -willingness to explore Decision-making -willingness to explore

the environmentthe environment

Perceptual Perceptual ClassificationClassification

Excellent for testing the effects of Excellent for testing the effects of regulatory fitregulatory fit

Stimuli with small number of dimensionsStimuli with small number of dimensions Lines that vary in length, orientation and Lines that vary in length, orientation and

positionposition ‘‘Gabor’ patches that vary in frequency and Gabor’ patches that vary in frequency and

orientationorientation Experimenter control of category structureExperimenter control of category structure Extensive set of tools for modeling Extensive set of tools for modeling

performance of individual participantsperformance of individual participants Can assess the strategies participants use in Can assess the strategies participants use in

the taskthe task

Explicit, Explicit, Hypothesis-testingHypothesis-testing system system mediates learning of mediates learning of “rule-based” (RB) category structures.“rule-based” (RB) category structures.

-Frontally mediated-Frontally mediated

-Verbalizable rules -Verbalizable rules

Implicit, Implicit, Procedural Procedural learning systemlearning system mediates learning mediates learning of “information-integration” (II) category structures.of “information-integration” (II) category structures.

-Striatally mediated-Striatally mediated

- Verbalizable rules - Verbalizable rules hurt hurt performanceperformance(Maddox and Ashby, 2004; Ashby et al., 1998)(Maddox and Ashby, 2004; Ashby et al., 1998)

Multiple systems mediate different classification tasks

Categorization TasksCategorization Tasks

Rule-Based Information-Integration

Bar Width

Ori

enta

tion

A B

Bar Width

Ori

enta

tion

A

B

Learned Explicitly Learned Implicitly

Increased cognitive flexibility will Increased cognitive flexibility will increase rule useincrease rule use Enhance performance on rule-based tasksEnhance performance on rule-based tasks Will harm performance on information-integration taskWill harm performance on information-integration task

Rule-use disrupts the procedural systemRule-use disrupts the procedural system

Recent tests of this hypothesis (Markman et al., 2005; Maddox Recent tests of this hypothesis (Markman et al., 2005; Maddox et al., 2006; Grimm et al., 2008)et al., 2006; Grimm et al., 2008) Manipulated regulatory focus and reward structure between subjectsManipulated regulatory focus and reward structure between subjects Used rule-based and information-integration tasksUsed rule-based and information-integration tasks

Influence of Regulatory Fit

Regulatory Fit and Regulatory Fit and ClassificationClassification

Rule-based performance was better in a fit Information-integration performance was better in a mismatch

Fit increases rule-useHelps on rule-based, hurts on information-integration

Choking & Choking & Excelling Under Excelling Under

PressurePressure

Worthy, Markman, & Maddox, 2009a, 2009b; Worthy, Markman, & Maddox, 2009a, 2009b; Worthy, Markman, & Maddox, 2008; Markman, Worthy, Markman, & Maddox, 2008; Markman,

Maddox & Worthy 2006Maddox & Worthy 2006

Choking Under PressureChoking Under Pressure

Anecdotal phenomenon (e.g. sports, Anecdotal phenomenon (e.g. sports, test-taking, etc.)test-taking, etc.)

People perform worse than normal People perform worse than normal when under pressurewhen under pressure

Some also seem to excel under Some also seem to excel under pressurepressure

Might pressure be similar to a Might pressure be similar to a prevention focus?prevention focus?

Motivation and Pressure Motivation and Pressure Working Memory Distraction Hypothesis of chokingWorking Memory Distraction Hypothesis of choking

Pressure reduces WM capacityPressure reduces WM capacity Should see main effectsShould see main effects Pressure decreases rule-usePressure decreases rule-use

Alternative: Pressure affects cognition through its Alternative: Pressure affects cognition through its effects on the motivational stateeffects on the motivational state

Working Hypothesis:Working Hypothesis: Pressure induces an “avoidance” or “prevention” Pressure induces an “avoidance” or “prevention”

motivational statemotivational state Interacts with goal pursuit mechanism to influence Interacts with goal pursuit mechanism to influence

regulatory fitregulatory fit

Pressure and Category-Pressure and Category-LearningLearning

Low pressure – “do Low pressure – “do your best”your best”

High pressure:High pressure:-Paired with a ‘partner’-Paired with a ‘partner’-If both of you reach -If both of you reach criterion, both get $6criterion, both get $6-If one of you fails -If one of you fails neither get $6 bonusneither get $6 bonus-Partner has already -Partner has already reached criterionreached criterion-Trying to -Trying to preventprevent the the negative end-state of negative end-state of letting their partner letting their partner downdown

Run gains and lossesRun gains and losses

FitFit MismatchMismatch

MismatchMismatch FitFit

Promotion Focus Prevention Focus

Gai

ns

Los

ses

Loc

al G

oal P

urs

uit

Mec

han

ism

Global Incentive

Low Pressure High Pressure

WM Distraction vs. WM Distraction vs. Regulatory FitRegulatory Fit

Pressure decreases Pressure decreases WMWM Poor rule-based Poor rule-based

performanceperformance Better information-Better information-

integrationintegrationPredictions Based on Regulatory Fit

Hypothesis

Gain Loss Gain Loss


Pro

po

rtio

n C

orr

ec

tLowPressure

High Pressure

Predictions Based on WM Distraction Hypothesis

Gain Loss Gain Loss


Pro

po

rtio

n C

orr

ec

t

LowPressure

High Pressure

Pressure induces a Pressure induces a prevention focusprevention focus Will interact with Will interact with

the reward the reward structurestructure

MethodMethod 2 (Pressure-level) X 2 (Reward Structure) X 2 (Task Type) between-subjects design Performed 8 80-trial blocksRule-Based Information-Integration

Worthy, et al., 2009, Worthy, et al., 2009, Psychonomic Bulletin & ReviewPsychonomic Bulletin & Review

ResultsResults

Accuracy across all blocks

0.500.550.600.650.700.750.800.85

Gain Loss Gain Loss


Pro

po

rtio

n C

orr

ec

t

LowPressure

High Pressure

Predictions Based on WM Distraction Hypothesis

Gain Loss Gain Loss


Pro

port

ion

Cor

rect

LowPressure

High Pressure

Predictions Based on Regulatory Fit Hypothesis

Gain Loss Gain Loss


Pro

port

ion

Cor

rect

LowPressure

High Pressure

Worthy, et al., 2009, Worthy, et al., 2009, Psychonomic Bulletin and Psychonomic Bulletin and ReviewReview

Decision Bound ModelingDecision Bound Modeling

Used to infer strategy use.Used to infer strategy use. Decision bound models assume stimuli are Decision bound models assume stimuli are

classified based on which side of the decision classified based on which side of the decision bound they fall onbound they fall on

Several models are fit to the dataSeveral models are fit to the data Best-fitting model gives information about Best-fitting model gives information about

which strategy each participant probably used which strategy each participant probably used to classify the stimulito classify the stimuli

Decision Bound ModelingDecision Bound ModelingBest Fit by Frequency Model

-50

0

50

100

150

200

250

300

350

200 225 250 275 300 325 350 375

Spatial Frequency

Sp

ati

al

Ori

en

tati

on

Best fit by Orientation Model

-50

0

50

100

150

200

250

300

350

200 225 250 275 300 325 350 375 400

Spatial Frequency

Sp

ati

al O

rien

tati

on

Best Fit by Optimal General Linear Classifier Model

-100

-50

0

50

100

150

200

250

300

100 150 200 250 300 350 400 450

Spatial Frequency

Spa

tial O

rient

atio

n

Best Fit By Random Response Model

-100

-50

0

50

100

150

200

250

300

100 150 200 250 300 350 400 450

Spatial Frequency

Sp

atia

l Ori

enta

tio

n

Model Fitting ProcedureModel Fitting Procedure

Fit each participant’s data on a block-by-block Fit each participant’s data on a block-by-block basisbasis

Used AIC to determine best fitting model for Used AIC to determine best fitting model for that blockthat blockPenalizes for free parametersPenalizes for free parameters

Examined the proportion of data sets best fit Examined the proportion of data sets best fit by each model over all blocks of the task.by each model over all blocks of the task.

Model-Based AnalysisModel-Based AnalysisBest Fit by Frequency Model

-50

0

50

100

150

200

250

300

350

200 225 250 275 300 325 350 375

Spatial Frequency

Sp

ati

al

Ori

en

tati

on

Best Fit by Optimal General Linear Classifier Model

-100

-50

0

50

100

150

200

250

300

100 150 200 250 300 350 400 450

Spatial Frequency

Spa

tial O

rien

tatio

n

Best strategy for Best strategy for rule-based taskrule-based task

Best strategy for Best strategy for information-information-integration taskintegration task

Proportion Fit by Best Proportion Fit by Best ModelModel

Proportion Fit by Best Model

00.10.20.30.40.50.60.70.80.9

Gain Loss Gain Loss


Pro

po

rtio

n O

pti

ma

l

LowPressure

High Pressure

Increase in accuracy likely due to improved strategy use.

Worthy, et al., 2009, Psychonomic Bulletin & ReviewWorthy, et al., 2009, Psychonomic Bulletin & Review

SummarySummary

Pressure does appear to operate like a prevention focus during classification learning.

Not main effect where WM is decreased

Gains mismatches with pressure-induced prevention focus

Pressure hurts rule-based performancePressure helps information-integration performance.

Losses fits with pressure-induced prevention focus

Pressure helps rule-based performance.Pressure hurts information-integration performance.

Pressure and ExpertsPressure and Experts Examined effects Examined effects

of pressure after of pressure after extensive training.extensive training. RB or II taskRB or II task 5 640-trial sessions5 640-trial sessions

Difference Between Session 4 and Session 5 accuracy

-0.025-0.020-0.015-0.010-0.0050.0000.0050.0100.0150.0200.0250.030


Category Structure

Diff

eren

ce in

Acc

urac

y

Control

Pressure

Worthy et al., 2009, Attention, Perception and Psychophysics

Supports a different account for effects of Supports a different account for effects of pressure on expertspressure on experts

Real World ChokingReal World Choking

Examined clutch free-throw performance Examined clutch free-throw performance among NBA athletesamong NBA athletes Considered point-differential between shooter’s Considered point-differential between shooter’s

team.team. Compared percentage to career percentageCompared percentage to career percentage

Expected and Observed Proportions of Free Throws Made for Each Point

Differential

0.640.66

0.680.700.72

0.740.76

0.780.80

-5 -4 -3 -2 -1 0 1 2 3 4 5

Point Differential

Pro

po

rtio

n M

ad

e

Expected

Observed

Worthy et al., 2009, International Journal of Creativity and Problem Solving

Regulatory Fit Regulatory Fit and Decision-and Decision-

MakingMaking

Worthy, Maddox, & Markman, 2007Worthy, Maddox, & Markman, 2007

Decision-making from Decision-making from experienceexperience

Basic Design‘Gambling’ task Participants choose from two or more decks of cardsMust either maximize gains or minimize losses

Gains Losses

ModelingModeling

Task is amenable to reinforcement Task is amenable to reinforcement learning modelinglearning modeling

Can estimate parameters that Can estimate parameters that describe performancedescribe performance

Expected Value (EV)Expected Value (EV)

EV – How many points one expects to gain or lose EV – How many points one expects to gain or lose from selecting a given deckfrom selecting a given deck

Used to determine which option to chooseUsed to determine which option to choose Example Example

EVEVred deckred deck= 7 points= 7 points

EVEVblue deckblue deck= 3 points= 3 points

Exploration/Exploitation Exploration/Exploitation DilemmaDilemma

Exploit Exploit the option with the highest EVthe option with the highest EVoror

Explore Explore other options with lower EVsother options with lower EVs Must balance the need to exploit with Must balance the need to exploit with

the need for new informationthe need for new information Exploration may be more frontally Exploration may be more frontally

mediated (e.g. Daw et al., 2006).mediated (e.g. Daw et al., 2006). Working hypothesis: Regulatory fit Working hypothesis: Regulatory fit

will increase exploration will increase exploration

Task DesignTask Design

Can design tasks to favor more Can design tasks to favor more exploratory or exploitative strategies.exploratory or exploitative strategies.

Experiment 1 – Exploration-optimalExperiment 1 – Exploration-optimal Experiment 2 – Exploitation-optimal Experiment 2 – Exploitation-optimal

(Gains only)(Gains only) Use behavioral and model-based Use behavioral and model-based

analyses to test the regulatory fit analyses to test the regulatory fit hypothesishypothesis

Worthy et al., 2007Worthy et al., 2007

Experiment 1Experiment 1

Designed a task where exploring the deck with Designed a task where exploring the deck with lower EV led to better-long-term performance.lower EV led to better-long-term performance.

Had to be willing to explore the Advantageous deckHad to be willing to explore the Advantageous deck Fit should increase exploration; performance Fit should increase exploration; performance

Points Based on Number of Cards Drawn from Each Deck

0

2

4

6

8

10

1 6 11 16 21 26 31 36 41 46 51 56 61 66 71 76

Cards Drawn From Deck

Poi

nts

Ear

ned

Advantageous

Disadvantageous

MethodsMethods

Used raffle-ticket manipulation to Used raffle-ticket manipulation to manipulate regulatory focusmanipulate regulatory focus

Promotion Promotion Focus Focus (Approach)(Approach)

Achieve Global Achieve Global Performance Criterion Performance Criterion Raffle ticket for $50Raffle ticket for $50

Prevention Prevention Focus Focus (Avoidance)(Avoidance)

Achieve Global Achieve Global Performance Criterion Performance Criterion Keep $50 raffle ticket Keep $50 raffle ticket given initiallygiven initially

MethodsMethods

Achieved global criterion by either Achieved global criterion by either maximize gains or minimizing lossesmaximize gains or minimizing losses

GainsGainsGained between 1 and 10 Gained between 1 and 10

points on each draw; points on each draw; maximized gainsmaximized gains

LossesLossesLost between -10 and -1 Lost between -10 and -1

points on each draw; points on each draw; minimized lossesminimized losses

Behavioral resultsBehavioral resultsAverage Distance from Criterion

-40

-35

-30

-25

-20

-15

-10

-5

0

Gains Losses

Po

ints

Bel

ow

Cri

teri

on

Promotion Prevention

Participants in a regulatory fit came Participants in a regulatory fit came significantly closer to the performance significantly closer to the performance criterion than participants in a mismatchcriterion than participants in a mismatch

Modeling Choice Modeling Choice BehaviorBehavior

EVs of each option are updated via an EVs of each option are updated via an exponential recency-weighted algorithmexponential recency-weighted algorithm

][ 11 kkkk EVrEVEV

Current EVNew EV RewardRecency Parameter

Current EV

•If reward is greater than the current EV the EV increases

•If reward is less than the current EV the EV decreases

Action SelectionAction SelectionAction selection is probabilistically determined via choice rules (e.g. Luce, 1959)

Softmax Rule

n

b

bEV

aEV

tat

t

e

eP

1

))((

))((

,

Probability of choosing option “A”

EV for option “A”

Exploitation parameter

Sum of EVs for all options

• Higher values indicate greater exploitation • Lower values indicate greater exploration • Can directly parameterize degree of exploratory vs. exploitative behavior

Model-based resultsModel-based results Fit reinforcement-learning model to estimate the degree Fit reinforcement-learning model to estimate the degree

of exploratory vs. exploitative behavior.of exploratory vs. exploitative behavior.

Exploitation parameter

0.0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

Gains Losses

Ex

plo

ita

tio

n


Participants in a regulatory fit had significantly lower Participants in a regulatory fit had significantly lower estimated exploitation-parameter values.estimated exploitation-parameter values.

Experiment 2Experiment 2 Designed a task where exploitation of the deck with Designed a task where exploitation of the deck with

the best expected value led to the best performance.the best expected value led to the best performance.

Reward Values Given for Each Deck Based on Trial Number

0

2

4

6

8

10

Trial

Po

ints

Aw

ard

ed

Deck A

Deck B

If fit increases exploration then If fit increases exploration then participants in a fit should do participants in a fit should do worse.worse.

ResultsResults

Only ran participants with a gains reward Only ran participants with a gains reward structurestructure

Participants in a regulatory fit were Participants in a regulatory fit were further from the performance criterionfurther from the performance criterion

Average Distance from Criterion

-120

-100

-80

-60

-40

-20

0

Po

int

Belo

w C

rite

rio

n


Model-Based ResultsModel-Based Results

Participants in a fit were less exploitative Participants in a fit were less exploitative than those in a mismatchthan those in a mismatch

Exploitation Parameter

0.0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1.0

Exp

loit

atio

n


SummarySummary

Regulatory fit influenced the decision-making behavior

Fit – greater explorationMismatch greater exploitation

Social pressure induces a prevention focusInfluences regulatory fitDifferential performance on category-learning tasks

Three-way interaction Regulatory focus – Promotion vs. preventionReward Structure – Maximize gains vs. minimize lossesTask Demands – Rule-based vs. information-integration; exploration-optimal vs. exploitation-optimal

Expected Reward Expected Reward ComparisonComparison

Extended decision-Extended decision-making paradigm to making paradigm to ratio vs. difference ratio vs. difference comparisonscomparisons Are EVs compared via Are EVs compared via

ratio or differences?ratio or differences? Manipulated whether Manipulated whether

difference or ratio difference or ratio preserved.preserved.

Changing the ratio Changing the ratio between EVs affected between EVs affected performanceperformance

Worthy et al., 2008, Memory and Cognition

Total Adjusted Points Earned

450

460

470

480

490

500

510

520

530

540

Control DifferencePreserving

RatioPreserving

Po

ints

Research ApproachResearch Approach Categorization and Decision-making Categorization and Decision-making

taskstasks Behavioral analysisBehavioral analysis

Mathematical modelingMathematical modeling Decision-bound modelingDecision-bound modeling Reinforcement-learning modelingReinforcement-learning modeling

Ground theories in neuroscienceGround theories in neuroscience Leads to novel predictionsLeads to novel predictions

Current & Future Current & Future DirectionsDirections

‘Why’ does regulatory fit influence behavior and cognition Working memory hypothesis

Fit increases WM memory capacityNot yet directly testedTest using regulatory focus and social pressure manipulation in WM tasks.Test by adding WM span as an additional factor on categorization and decision-making tasks.

Regulatory fit and short-term vs. long-term decision-making

Does fit reduce future discounting?People in a fit may focus more on long-term outcomes


Individual DifferencesAre some less susceptible to situational factors than others?Why do some people tend to choke, while others excel?

Aging and decision-makingOlder adults appear to be more exploratory than younger adultMay value long-term over short-term outcomesPositivity biasNeural differences

Gender and decision-makingMen appear to be more exploitative than women


Social vs. Monetary rewardsGive incrementally happier or angrier faces as feedback in decision-making tasks.Can use same modeling approachCompare to monetary rewardsNeural mechanisms


Category learningFeedback timing

Very important for procedural learning system

Retention and generalizationDesirable difficulties

Naturalistic stimuli (x-rays – tumor detection)Interactions between multiple systems – competition vs. cooperation

Thanks!Thanks!

AcknowledgementsAcknowledgements

Todd Maddox, Art Markman, Bo Zhu, Todd Maddox, Art Markman, Bo Zhu, MaddoxLab research assistants.MaddoxLab research assistants.

Supported by NIMH grant MH077708 Supported by NIMH grant MH077708 to WTM and ABM, and a supplement to WTM and ABM, and a supplement

to DAW.to DAW.

ReferencesReferencesDaw, N.D., O’Doherty, J.P., Dayan, P., Seymour, B., & Dolan, R. (2006). Cortical Substrates for Daw, N.D., O’Doherty, J.P., Dayan, P., Seymour, B., & Dolan, R. (2006). Cortical Substrates for exploratory decisions in exploratory decisions in humans. humans. Nature,Nature, 441 (15), 876-879. 441 (15), 876-879.Grimm, L. R., Markman, A. B., Maddox, W. T., Baldwin, G. C. (2008) Differential Effects of Grimm, L. R., Markman, A. B., Maddox, W. T., Baldwin, G. C. (2008) Differential Effects of Regulatory Fit on Category Regulatory Fit on Category Learning. Learning. Journal of Experimental Social Psychology. 44Journal of Experimental Social Psychology. 44, 920-, 920-927.927.Higgins, E. T. (1997). Beyond pleasure and pain. Higgins, E. T. (1997). Beyond pleasure and pain. American Psychologist, 52American Psychologist, 52, 1280-1300., 1280-1300.

Higgins, E. T. (2000). Making a good decision: Value from fit. Higgins, E. T. (2000). Making a good decision: Value from fit. American Psychologist, American Psychologist, 5555, 1217-1230., 1217-1230.Maddox, W.T., & Ashby, F.G. (1993). Comparing decision bound and exemplar models of Maddox, W.T., & Ashby, F.G. (1993). Comparing decision bound and exemplar models of categorization. categorization. Perception and Perception and Psychophysics, 53Psychophysics, 53, 49-70., 49-70.Maddox, W. T., & Ashby, F. G. (2004). Dissociating explicit and procedural-learning based Maddox, W. T., & Ashby, F. G. (2004). Dissociating explicit and procedural-learning based systems of perceptual category systems of perceptual category learning. learning. Behavioural ProcessesBehavioural Processes, , 6666, 309-332., 309-332.Maddox, W. T., Markman, A. B., & Baldwin, G. C. (2006). Using classification to understand the Maddox, W. T., Markman, A. B., & Baldwin, G. C. (2006). Using classification to understand the motivation-learning interface. motivation-learning interface. Psychology of Learning and Motivation, 47Psychology of Learning and Motivation, 47, 213-250., 213-250.Markman, A.B., Maddox, W.T., Worthy, D.A. (2006) Choking and excelling under pressure. Markman, A.B., Maddox, W.T., Worthy, D.A. (2006) Choking and excelling under pressure. Psychological Science. 17Psychological Science. 17, 944-, 944- 948.948.Shah, J., Higgins, E. T., & Friedman, R. S. (1998). Performance incentives and means: How Shah, J., Higgins, E. T., & Friedman, R. S. (1998). Performance incentives and means: How regulatory focus influences goal regulatory focus influences goal attainment. attainment. Journal of Personality and Social Journal of Personality and Social Psychology, 74Psychology, 74, 285 - 293., 285 - 293.Spiegel, S., Grant-Pillow, H., & Higgins, E. T. (2004). How regulatory fit enhances motivational Spiegel, S., Grant-Pillow, H., & Higgins, E. T. (2004). How regulatory fit enhances motivational strength during goal pursuit. strength during goal pursuit. European Journal of Social Psychology, 34European Journal of Social Psychology, 34, 39-54., 39-54.Worthy, D.A., Maddox, W.T., & Markman, A.B. (2007). Regulatory Fit Effects in a Choice Task. Worthy, D.A., Maddox, W.T., & Markman, A.B. (2007). Regulatory Fit Effects in a Choice Task. Psychonomic Bulleting and Psychonomic Bulleting and ReviewReview,, 14 14, 1125-1132. , 1125-1132. Worthy, D.A., Maddox, W.T., & Markman, A.B. (2008). Ratio and Difference Comparisons of Worthy, D.A., Maddox, W.T., & Markman, A.B. (2008). Ratio and Difference Comparisons of Expected Reward in Decision Expected Reward in Decision Making Tasks. Making Tasks. Memory and Cognition, 36, Memory and Cognition, 36, 1460-1469.1460-1469.Worthy, D.A., Markman, A.B., & Maddox, W.T. (2009A). What is pressure? Evidence for social Worthy, D.A., Markman, A.B., & Maddox, W.T. (2009A). What is pressure? Evidence for social pressure as a type of regulatory pressure as a type of regulatory focus. focus. Psychonomic Bulletin and Review, 16, Psychonomic Bulletin and Review, 16, 344-344-349349..Worthy, D.A., Markman, A.B., & Maddox, W.T. (2009B). Choking and excelling at the free Worthy, D.A., Markman, A.B., & Maddox, W.T. (2009B). Choking and excelling at the free throw linethrow line. The International . The International Journal of Creativity & Problem Solving, 19, 53-58.Journal of Creativity & Problem Solving, 19, 53-58. Worthy, D.A., Markman, A.B., & Maddox, W.T. (2009C). Choking and Excelling Under Pressure Worthy, D.A., Markman, A.B., & Maddox, W.T. (2009C). Choking and Excelling Under Pressure in Experienced Classifiers. in Experienced Classifiers. Attention, Perception and Psychophysics, 71, Attention, Perception and Psychophysics, 71, 924-935.924-935.

Aging and Decision-Aging and Decision-MakingMaking

Estimated Exploitation Parameter Values

0

0.2

0.4

0.6

0.8

1

Gain Loss

Exp

loita

tion

Old

Young

Older adults use a more exploratory than younger Older adults use a more exploratory than younger adults.adults.

Task favored an exploitative strategyTask favored an exploitative strategyWorthy et al., in preparationWorthy et al., in preparation


-Task favored an exploratory strategy-Task favored an exploratory strategy

Exploitation Parameter Values Exploratory Task

0

0.2

0.4

0.6

0.8

1

Gain Loss

Expl

oita

tion

Older

Younger

Worthy et al., in preparationWorthy et al., in preparation


-Looked at “Directed Exploration” – not just more -Looked at “Directed Exploration” – not just more randomrandom

Reward Given Based on Previous Long-term Increasing Options

0

20

40

60

80

100

1 2 3 4 5 6 7 8 9 10

Number of Long-Term Increasing Options in Last 10 Trials

Rew

ard

Long-term decreasing

Long-term increasing



-Older adults explore the decision space-Older adults explore the decision space

-Do not focus only on short-term rewards-Do not focus only on short-term rewards

Advantageous Choices Over first 100 trials

0123456789

10

1 2 3 4 5 6 7 8 9 10

Block (10-trials)

Num

ber o

f Adv

anta

geou

s C

hoic

es Older

Younger


Gender and Decision-Gender and Decision-MakingMaking

-Males tend to be more exploitative than females-Males tend to be more exploitative than females

Exploitation Parameter Values Based on Gender

0

0.2

0.4

0.6

0.8

1

1.2

Gains Losses

Exp

loita

tionnio

n

Males

Females

Feedback Delay and Feedback Delay and Category LearningCategory Learning

--Feedback timing important for II learning onlyFeedback timing important for II learning only-500ms appears to be the best time for -500ms appears to be the best time for procedural system to receive feedbackprocedural system to receive feedback

Accuracy with Different Feedback Delay Intervals

0.550.600.650.700.750.80

RB II

Pro

portio

n Cor

rect

0ms

500ms

1000ms


Feedback Delay and Feedback Delay and Category LearningCategory Learning

--Separated visual and motor Separated visual and motor responseresponse feedback feedback componentscomponents-Important for system to receive visual and -Important for system to receive visual and motor information that a response has been motor information that a response has been mademade Worthy et al., in preparationWorthy et al., in preparation

Accuracy with Different Offset and Feedback Delay Intervals

0.55

0.65

0.75

250ms-250ms 250ms-500ms 0ms-500ms

Pro

portio

n Cor

rect

Desirable Difficulties in II Desirable Difficulties in II learninglearning

--Discontinuous categories are more difficult to Discontinuous categories are more difficult to learn but may lead to better transfer learn but may lead to better transfer performance.performance.

Discontinuous

Length

Ori

en

tati

on

Continuous

Length

Ori

en

tati

on

Maddox et al., in preparationMaddox et al., in preparation

Desirable Difficulties in II Desirable Difficulties in II learninglearning

--Continuous categories are learned easier, but Continuous categories are learned easier, but transfer performance is worse.transfer performance is worse.

Maddox et al., in preparationMaddox et al., in preparation

Accuracy

0.30

0.40

0.50

0.60

0.70

0.80

1 2 3 4 5 6 7 8 9 10 1 2 3 4 T

Pro

po

rtio

n C

orr

ect

DiscontinuousContinuous

Motivation and Cognition: From Regulatory Fit to Reinforcement Learning

Documents

Transcript of Motivation and Cognition: From Regulatory Fit to Reinforcement Learning