Thecostofcognitivecontrolasasolutiontothestability-flexibilitydilemma

SebastianMusslick1,SeongJunJang1,MichaelShvartsman1,AmitaiShenhav2&JonathanD.Cohen11PrincetonNeuroscienceInstitute,PrincetonUniversity2DepartmentofCognitive,Linguistic,andPsychologicalSciences,BrownInstituteforBrainScience,BrownUniversity

Introduction

Model

ComputationalAnalysesLowerConstraintsonControl(HigherGain)...

OptimalConstraintsonControlasaFunctionofDemandforFlexibility

ModelPredictionsandExperimentResults

Conclusions References&Acknowledgements

»improveoveralltaskperformance »improvecognitivestability »impaircognitiveflexibility

TaskEnvironment

lowswitchrate(25%switches)

highswitchrate(75%switches)

g

ExperimentGroups FindOptimalGainthatMaximizesAccuracyAcrossExperiment

ModelPredictionsBasedonOptimalGain

HumanBehavior

Model

n=17

n=17

lowswitchrate(25%switches)

highswitchrate(75%switches)

»Itisoptimaltoconstraincontrolwhendemandforflexibilityishigh

1 2 3Gain

0.6

0.7

0.8

0.9

1

RT

Ove

rall

Perf

orm

ance

(s)

1 2 3Gain

0

5

10

15

20

Erro

r Rat

eO

vera

ll Pe

rfor

man

ce (%

)

1 2 3Gain

0.6

0.8

1

1.2

RT

Inco

ngru

ency

Cos

t (s)

1 2 3Gain

0

10

20

30

40

Erro

r Rat

eIn

cong

ruen

cy C

ost (

%)

1 2 3Gain

0

0.1

0.2

0.3

0.4

RT

Switc

h C

ost (

s)

1 2 3Gain

0

10

20

30

Erro

r Rat

eSw

itch

Cos

t (%

)

0 1 2 3 4 5Gain

80

90

100

Ove

rall

Accu

racy

(%)

75 % Switch Rate25 % Switch Rate

HumanParticipants

25% 75%

0

0.1

0.2

RT

Switc

h C

osts

(s)

25% 75%-0.05

0

0.05

0.1

RT

Inco

ngru

ency

Cos

ts (s

)

25% 75%

0.6

0.7

0.8

0.9

Mea

n R

T (s

)

t(16)=-2.43,p=0.0138 n.s. t(16)=2.11,p=0.0254

25% 75%

4

5

6

RT

Inco

ngru

ency

Cos

ts (s

)

25% 75%2

2.2

2.4

2.6

2.8

RT

Switc

h C

osts

(s)

25% 75%

3

3.5

4

Mea

n R

T (s

)

T.Goschke,AttentionandperformanceXVIII,18,331-335(2000)S.Musslicketal.,CogSci,806-811(2018)A.Shenhav,M.M.Bovtinick,J.D.Cohen,Neuron79,217-40(2013)A.Shenhavetal.,AnnuRevNeurosci40,99-124(2017)R.M.Shiffrin,W.Schneider,Psychol.Rev.84,127-190(1977)ThispublicationwasmadepossiblethroughthesupportofagrantfromtheJohnTempletonFoundation.TheopinionsexpressedinthispublicationarethoseoftheauthorsanddonotnecessarilyreflecttheviewsoftheJohnTempletonFoundation.

BetweenSubjectsDesign

§  Constraintsoncognitivecontrolcanimprovecognitiveflexibilityattheexpenseofcognitivestability

§  Higherconstraintsoncognitivecontrolareoptimalinenvironmentswithhigherdemandforflexibility

§  Costsassociatedwithcognitivecontrolmayreflectanapproximatesolutiontothestability-flexibilitydilemmainneuralsystems

Isthemajorityofthedotsmovingupordown?

Isthemajorityofthedots

blueorred?

K Lup downblue red

CuedTaskSwitchingParadigm

…

taskcue executionpreparation ITI

motionminiblockcolorminiblock

…

g

ControlModule

different task when the environment changes. This is evi-dent empirically in the form of costs to performance whenswitching from one task to another, which are magnifiedwith increases in the allocation of control to the initial task(Goschke, 2000). The dilemma is consistent with the task-set inertia hypothesis, according to which the task-set of apreviously performed task persists and interferes with initialperformance of a subsequent task following a switch (Allport,Styles, & Hsieh, 1994).

We use a recurrent neural network model of task perfor-mance and control to explore how different choices of aglobal control parameter (gain modulation) that determinesits maximal intensity, influence the stability and flexibilityof performance in a task switching environment. Critically,we determine the optimal value of this parameter as a func-tion of the demand for flexibility in the task environment,and show how these results can explain differences in hu-man task switch costs as a function of task switch probability.Finally, we conclude with a discussion about how computa-tional dilemmas such as the stability-flexibility tradeoff mayhelp provide a normative account of previously unexplained –and what may otherwise appear to be irrational – constraintson cognitive control. The code for all simulations used in thiswork is available at github.com/musslick/CogSci-2018b.

Recurrent Neural Network Model

To explore the effect of constraints on the intensity of con-trol, we simulate control configurations as activity states ofprocessing units in a neural network (control module) that un-fold over the course of trials. Within each trial, the process-ing units of the network engage an evidence accumulationprocess that integrates information about the stimulus, and isused to generate a response (decision module). In this sectionwe describe the processing dynamics for both the control anddecision modules, as well as the environments in which themodel is tasked to perform.

Control Module

We simulate the intensities of two different control signals asactivities of two processing units indexed by i, i 2 {1,2} in arecurrent neural network. The activity of each processing unitunit i at a given trial T represents the intensity of the controlsignal for one of two tasks and is determined by its net input

netTi = wi,iactT�1

i +wi, jactT�1j + Ii (1)

that is a linear combination of the unit’s own activity at theprevious trial actT�1

i multiplied by the self-recurrent weightwi,i, the activity of the other unit j 2 1,2, j 6= i at the previ-ous trial actT�1

j multiplied by an inhibitory weight wi, j, aswell as an external input Ii provided to the unit (see Figure 1).The self-recurrent and mutually-inhibitory weights induce at-tractors within the control module, such that it can maintainits activity over time in one state or the other, but not both.The latter implements a capacity constraint on control with

regard to the number of control-dependent tasks the networkcan support. These weights also determine the activity of theunits in each attractor state, that is also regulated by a gainparameter that we use to implement the intensity constraint,as discussed further below. The external input acts as a gat-ing signal to the corresponding control unit (Braver & Cohen,1999) and is set to 1 if the task represented by the control unitneeds to be performed and set to 0 otherwise. The net inputof each unit is averaged over time

netTi = t · (netT

i )+(1� t)netiT�1 (2)

where netiT�1 corresponds to the time averaged net inputat the previous trial and t 2R : 0 t 1 is the rate constant.A higher t leads to a faster change in activation for both units.Finally, the activity of each unit is a sigmoid function of itstime averaged net input

actTi =

11+ e�g·netT

i(3)

where g is a gain parameter that regulates the slope ofthe activation function. The sigmoid activation function con-strains the activity of both units to lie between 0 and 1. Thegain of the activation function effectively regulates the dis-tance between the two control states, with lower gain leadingto a lower activation of the currently relevant control unit. Inthis model, we use g to implement a constraint on the inten-sity of control, and explore its effect on the network’s balancebetween stability and flexibility.

!"," !",$ !$,$%&'" %&'$

($("ControlModule

DecisionModule

StimulusDimensions

)" )$

timestepswithintrial*+,-' = )" + )$ +)"%&'" + )$%&'$

Response2

Response1!$,"

Figure 1: Recurrent neural network model used in simula-tions. Each of the two processing units in the control module(blue) receive an external input signal I1, I2 that indicates thecurrently relevant task. The dynamics of the network unfoldover the course of trials and are determined by recurrent con-nectivity w1,1,w2,2 for each unit, as well as mutual inhibitionw1,2,w2,1 between units. The activity of each control unit bi-ases the processing of a corresponding stimulus dimension ona given trial. On each trial, the decision module accumulatesevidence for both stimulus dimensions towards one of tworesponses until a threshold is reached.

Decision Module

On each trial the decision module integrates informationalong two stimulus dimensions S1 and S2 of a single stim-ulus to determine a response. Each dimension (e.g., color or

different task when the environment changes. This is evi-dent empirically in the form of costs to performance whenswitching from one task to another, which are magnifiedwith increases in the allocation of control to the initial task(Goschke, 2000). The dilemma is consistent with the task-set inertia hypothesis, according to which the task-set of apreviously performed task persists and interferes with initialperformance of a subsequent task following a switch (Allport,Styles, & Hsieh, 1994).

We use a recurrent neural network model of task perfor-mance and control to explore how different choices of aglobal control parameter (gain modulation) that determinesits maximal intensity, influence the stability and flexibilityof performance in a task switching environment. Critically,we determine the optimal value of this parameter as a func-tion of the demand for flexibility in the task environment,and show how these results can explain differences in hu-man task switch costs as a function of task switch probability.Finally, we conclude with a discussion about how computa-tional dilemmas such as the stability-flexibility tradeoff mayhelp provide a normative account of previously unexplained –and what may otherwise appear to be irrational – constraintson cognitive control. The code for all simulations used in thiswork is available at github.com/musslick/CogSci-2018b.

Recurrent Neural Network Model

To explore the effect of constraints on the intensity of con-trol, we simulate control configurations as activity states ofprocessing units in a neural network (control module) that un-fold over the course of trials. Within each trial, the process-ing units of the network engage an evidence accumulationprocess that integrates information about the stimulus, and isused to generate a response (decision module). In this sectionwe describe the processing dynamics for both the control anddecision modules, as well as the environments in which themodel is tasked to perform.

Control Module

We simulate the intensities of two different control signals asactivities of two processing units indexed by i, i 2 {1,2} in arecurrent neural network. The activity of each processing unitunit i at a given trial T represents the intensity of the controlsignal for one of two tasks and is determined by its net input

netTi = wi,iactT�1

i +wi, jactT�1j + Ii (1)

that is a linear combination of the unit’s own activity at theprevious trial actT�1

i multiplied by the self-recurrent weightwi,i, the activity of the other unit j 2 1,2, j 6= i at the previ-ous trial actT�1

j multiplied by an inhibitory weight wi, j, aswell as an external input Ii provided to the unit (see Figure 1).The self-recurrent and mutually-inhibitory weights induce at-tractors within the control module, such that it can maintainits activity over time in one state or the other, but not both.The latter implements a capacity constraint on control with

regard to the number of control-dependent tasks the networkcan support. These weights also determine the activity of theunits in each attractor state, that is also regulated by a gainparameter that we use to implement the intensity constraint,as discussed further below. The external input acts as a gat-ing signal to the corresponding control unit (Braver & Cohen,1999) and is set to 1 if the task represented by the control unitneeds to be performed and set to 0 otherwise. The net inputof each unit is averaged over time

netTi = t · (netT

i )+(1� t)netiT�1 (2)

where netiT�1 corresponds to the time averaged net inputat the previous trial and t 2R : 0 t 1 is the rate constant.A higher t leads to a faster change in activation for both units.Finally, the activity of each unit is a sigmoid function of itstime averaged net input

actTi =

11+ e�g·netT

i(3)

where g is a gain parameter that regulates the slope ofthe activation function. The sigmoid activation function con-strains the activity of both units to lie between 0 and 1. Thegain of the activation function effectively regulates the dis-tance between the two control states, with lower gain leadingto a lower activation of the currently relevant control unit. Inthis model, we use g to implement a constraint on the inten-sity of control, and explore its effect on the network’s balancebetween stability and flexibility.

!"," !",$ !$,$%&'" %&'$

($("ControlModule

DecisionModule

StimulusDimensions

)" )$

timestepswithintrial*+,-' = )" + )$ +)"%&'" + )$%&'$

Response2

Response1!$,"

Figure 1: Recurrent neural network model used in simula-tions. Each of the two processing units in the control module(blue) receive an external input signal I1, I2 that indicates thecurrently relevant task. The dynamics of the network unfoldover the course of trials and are determined by recurrent con-nectivity w1,1,w2,2 for each unit, as well as mutual inhibitionw1,2,w2,1 between units. The activity of each control unit bi-ases the processing of a corresponding stimulus dimension ona given trial. On each trial, the decision module accumulatesevidence for both stimulus dimensions towards one of tworesponses until a threshold is reached.

Decision Module

On each trial the decision module integrates informationalong two stimulus dimensions S1 and S2 of a single stim-ulus to determine a response. Each dimension (e.g., color or

gainregulatescontrolattractorlandscape(constraintoncontrolsignalintensity)

DecisionModule

shape) can take one of two values (e.g., red or green; roundor square), each of which is associated with one of two re-sponses (e.g. pressing left or right button). Each of the twotasks requires mapping the current value of one of the twostimulus dimensions to its corresponding response, while ig-noring the other dimension. Since both tasks involve the samepair of responses, stimuli can be congruent (stimulus valuesin both dimensions associated with the same response) or in-congruent (associated with different responses). We simu-late the response integration process using a drift diffusionmodel (DDM, Ratcliff, 1978), in which the drift is determinedby the combined stimulus information from each dimension,weighted by input received from the control module (as de-scribed below), and evidence is accumulated over time untilone of two response thresholds is reached. The drift rate isdecomposed into an automatic and controlled component

dri f t = (S1 +S2)| {z }automatic

+actT1 S1 +actT

2 S2| {z }controlled

(4)

where the automatic component reflects automatic process-ing of each stimulus dimension that is unaffected by control.The absolute magnitude of S1,S2 depends on the strength ofthe association of each stimulus with a given response andits sign depends on the response (e.g. S1 < 0 if the associatedresponse is to press the left button, S1 > 0 if the associated re-sponse is to press the right button). Thus, for congruent trialsS1 and S2 have the same sign, and the opposite sign for incon-gruent (conflict) trials. For the simulations described below,the strength of the associations was equal along the two stim-ulus dimensions. The controlled component of the drift rateis the sum of the two stimulus values, each weighted by theactivation of the corresponding control unit. Thus, each unitin the control module biases processing towards one of thestimulus dimensions, similar to other computational modelsof cognitive control (e.g. Cohen et al., 1990; Mante, Sus-sillo, Shenoy, & Newsome, 2013; Musslick et al., 2015). Asa result, progressively greater activation of a control unit im-proves performance – speeds responses and improves accu-racy – for the corresponding task. Mean reaction times (RTs)and error rates for a given parameterization of drift rate attrial T are derived from an analytical solution to the DDM(Bogacz, Brown, Moehlis, Holmes, & Cohen, 2006).

Task Environment and Processing Dynamics

We used the model to simulate performance while switchingbetween 100 mini-blocks of the two tasks (Meiran, 1996).Each mini-block consisted of six trials of the same task. Atask cue presented before and throughout each mini-block in-structed the model to adjust control signals to the currentlyrelevant task i, by setting Ii to 1 for the task-relevant controlunit and I j 6=i to 0 for the other. On each trial, a stimulus waspresented comprised of a value along each dimension S1 andS2, the decision module integrated the input, and generateda response that was deemed correct if it corresponded to the

one associated with the stimulus value along the dimensionindicated by the task cue.

Effects of Network Gain

on Stability and Flexibility

The intensity of the control signals is functionally constrainedby the gain parameter of the activation function in Equation 3:lowering gain lowers the activity of a control unit for a givenpositive net input, and thus constrains the maximum signalintensity of the task-relevant control unit. Here, we exam-ine how manipulations of gain influence the model’s perfor-mance, and in particular measures of stability and flexibilityin the task switching design described above.

To do so we varied g from 0.1 to 3 in steps of 0.1. The con-trol module of each model was parameterized with balancedrecurrent and inhibitory weights, wi,i = 1,wi, j =�1 and a rateconstant of t = 0.9. The decision module (DDM) was pa-rameterized1 with a threshold of z = 0.0475, a non-decisiontime of T0 = 0.2 and a noise of c = 0.04. We simulated per-formance of each model on 10 different randomly permutedtask switching sequences of the type described above. Eachsequence was generated with a 50% task switch rate (i.e. onehalf of the mini-blocks required to switch to another task withrespect to the previous mini-block whereas the other half ofthe mini-blocks required to repeat the previous task). Tasktransitions were counterbalanced with each task and responsecongruency conditions.

For each simulation, we assessed the mean difference inreaction times (RTs) and error rates between incongruentand congruent trials as a measure of cognitive stability. In-congruent trials typically lead to slower reaction times andhigher error rates than congruent trials due to response con-flict, referred to as the incongruency effect. The stability-flexibility dilemma suggests that increased control intensity(implemented here by higher values of g) should augmentsensitivity to the task-relevant stimulus dimensions and di-minish it to the task irrelevant dimension, thereby reducingthe incongruency effect (Goschke, 2000). We also measuredmean task performance as an index of task stability. Finally,we assessed the flexibility in terms of performance costs as-sociated with a task switch relative to a task repetition fromone mini-block to another. Specifically, we computed the per-formance difference between the first trial of switch vs. rep-etition mini-blocks. We predicted that increasing g would in-crease task switch costs, as this would increase the distancebetween the attractors in the control module and thus makeit harder to switch between them. We evaluated the effect ofgain on each of these measures by regressing each against thegain of the network across all task sequences and networks.

The results in Figure 2a indicate that all tested models

1The parameter values were chosen to yield reasonable modelperformance, i.e. an average task accuracy higher than chance.While each parameter has a quantitative effect on the results de-scribed below, the qualitative (direction) of the effects remainsrobust across a wide range of parameter values (for details, seegithub.com/musslick/CogSci-2018b).

w2,1

nety2

act y2

nety2

act y2

act1 act2w2,2w1,2w1,1 0 1

0

1lowgain

act1 0 10

1highgain

act1

different task when the environment changes. This is evi-dent empirically in the form of costs to performance whenswitching from one task to another, which are magnifiedwith increases in the allocation of control to the initial task(Goschke, 2000). The dilemma is consistent with the task-set inertia hypothesis, according to which the task-set of apreviously performed task persists and interferes with initialperformance of a subsequent task following a switch (Allport,Styles, & Hsieh, 1994).

We use a recurrent neural network model of task perfor-mance and control to explore how different choices of aglobal control parameter (gain modulation) that determinesits maximal intensity, influence the stability and flexibilityof performance in a task switching environment. Critically,we determine the optimal value of this parameter as a func-tion of the demand for flexibility in the task environment,and show how these results can explain differences in hu-man task switch costs as a function of task switch probability.Finally, we conclude with a discussion about how computa-tional dilemmas such as the stability-flexibility tradeoff mayhelp provide a normative account of previously unexplained –and what may otherwise appear to be irrational – constraintson cognitive control. The code for all simulations used in thiswork is available at github.com/musslick/CogSci-2018b.

Recurrent Neural Network Model

To explore the effect of constraints on the intensity of con-trol, we simulate control configurations as activity states ofprocessing units in a neural network (control module) that un-fold over the course of trials. Within each trial, the process-ing units of the network engage an evidence accumulationprocess that integrates information about the stimulus, and isused to generate a response (decision module). In this sectionwe describe the processing dynamics for both the control anddecision modules, as well as the environments in which themodel is tasked to perform.

Control Module

We simulate the intensities of two different control signals asactivities of two processing units indexed by i, i 2 {1,2} in arecurrent neural network. The activity of each processing unitunit i at a given trial T represents the intensity of the controlsignal for one of two tasks and is determined by its net input

netTi = wi,iactT�1

i +wi, jactT�1j + Ii (1)

that is a linear combination of the unit’s own activity at theprevious trial actT�1

i multiplied by the self-recurrent weightwi,i, the activity of the other unit j 2 1,2, j 6= i at the previ-ous trial actT�1

j multiplied by an inhibitory weight wi, j, aswell as an external input Ii provided to the unit (see Figure 1).The self-recurrent and mutually-inhibitory weights induce at-tractors within the control module, such that it can maintainits activity over time in one state or the other, but not both.The latter implements a capacity constraint on control with

regard to the number of control-dependent tasks the networkcan support. These weights also determine the activity of theunits in each attractor state, that is also regulated by a gainparameter that we use to implement the intensity constraint,as discussed further below. The external input acts as a gat-ing signal to the corresponding control unit (Braver & Cohen,1999) and is set to 1 if the task represented by the control unitneeds to be performed and set to 0 otherwise. The net inputof each unit is averaged over time

netTi = t · (netT

i )+(1� t)netiT�1 (2)

where netiT�1 corresponds to the time averaged net inputat the previous trial and t 2R : 0 t 1 is the rate constant.A higher t leads to a faster change in activation for both units.Finally, the activity of each unit is a sigmoid function of itstime averaged net input

actTi =

11+ e�g·netT

i(3)

where g is a gain parameter that regulates the slope ofthe activation function. The sigmoid activation function con-strains the activity of both units to lie between 0 and 1. Thegain of the activation function effectively regulates the dis-tance between the two control states, with lower gain leadingto a lower activation of the currently relevant control unit. Inthis model, we use g to implement a constraint on the inten-sity of control, and explore its effect on the network’s balancebetween stability and flexibility.

!"," !",$ !$,$%&'" %&'$

($("ControlModule

DecisionModule

StimulusDimensions

)" )$

timestepswithintrial*+,-' = )" + )$ +)"%&'" + )$%&'$

Response2

Response1!$,"

Figure 1: Recurrent neural network model used in simula-tions. Each of the two processing units in the control module(blue) receive an external input signal I1, I2 that indicates thecurrently relevant task. The dynamics of the network unfoldover the course of trials and are determined by recurrent con-nectivity w1,1,w2,2 for each unit, as well as mutual inhibitionw1,2,w2,1 between units. The activity of each control unit bi-ases the processing of a corresponding stimulus dimension ona given trial. On each trial, the decision module accumulatesevidence for both stimulus dimensions towards one of tworesponses until a threshold is reached.

Decision Module

On each trial the decision module integrates informationalong two stimulus dimensions S1 and S2 of a single stim-ulus to determine a response. Each dimension (e.g., color or

activityofeachtaskunitdeterminesamountofcognitivecontrolappliedtothetask(controlsignalintensity)

dynamicsofcontrolmoduleunfoldacrosstrials

activitytrajectoryduringtaskswitch

act2 act2

S1

accumulationofevidencetowardsresponseovertimestepstwithintrialT

timestepst

response1

S1 colorcoherencemotioncoherence

MotionTask ColorTask

CognitiveControl:Abilitytooverridehabitualresponsesinordertosuccessfullyguidebehaviorintheserviceofcurrenttaskgoals

»Cognitivecontrolisfundamentallylimited(Shiffrin&Schneider,1977)

»Costsattachedtoincreasesincontrolsignalintensity(Shenhav,Botvinick&Cohen,2013;Shenhavetal.,2017)

morecognitivecontrol

Task1 Task2

lesscognitivecontrol

HypothesisConstraintsoncognitivecontrolpromotecognitive

flexibilityatexpenseofcognitivestability

»Whyconstraintheamountofcognitivecontrolallocatedtoataskwithacost?

Stability-FlexibilityDilemma(Goschke,2000)

K

response2L

(Musslicketal.,2018)

»Costsofcontrolapproximateoptimalconstraints

0.2 0.4 0.6 0.8Task Switch Probability

0.008

0.01

0.012

Opt

imal

Con

trol

Cos

t c

Cost(signal) = ec*signal0.2 0.4 0.6 0.8Task Switch Probability

0

0.5

1

Opt

imal

Con

trol

Sign

al In

tens

ity

Relevant TaskIrrelevant Task

0.2 0.4 0.6 0.8Task Switch Probability

1.5

2

2.5

Opt

imal

Gai

n

EVC = P(correct | signal)−Cost(signal)

selectcontrolsignalintensitythatmaximizes

where

≈

Ii

DriftDiffusionModel