Dynamic Decision Making in Complex Task Environments:

Principles and Neural Mechanisms

Progress and Future DirectionsNovember 17, 2009

A High-Stakes, Time-Critical Decision

• A diffuse form is coming toward you rapidly: What should you do?

– You could shoot at it, but it may be your friend

– You can hold your fire, but it might shoot you!

– You could wait to decide, but that might be risky too

• How can we optimize our choices, and the timing of our choices, in time-critical, uncertain situations?

FY07 MURI BAA06-028 Topic 15

Building Bridges between Neuroscience, Cognition, and Human Decision Making

Objective: The general goal is to form a complete and thorough understanding of basic human decision processes … by building a lattice of theoretical models with bridges that span across fields …. The main effort of this work is intended to be in the direction of new integrative theoretical developments … using mathematical and/or computation modeling … accompanied and supported by rigorous empirical model tests and empirical model comparisons …. .

From BAA 06-028, Topic 15

Our MURI Grant

• Builds on past neurophysiological, behavioral and theoretical investigations of the dynamics of decision making in humans and non-human primates toward the development of an integrated theory.

• Extends the empirical effort by employing fMRI, EEG, and MEG convergently to understand the distributed brain systems involved in decision making.

• Bridges to investigations concerned with decision making processes in real-life situations (e.g. those faced by air-traffic controllers and pilots).

Who We Are

• PIs:– McClelland, Newsome (Stanford)– Holmes, Cohen (Princeton)– Urban (Carnegie Mellon)– Johnston (NASA Ames)

• And many other professional scientists, post-docs, and graduate students in applied mathematics, neuroscience, cognitive science, and engineering

A Classical Model of Decision Making:The Drift Diffusion Model of Choice Between Two Alternative Decisions

• At each time step a small sample of noisy information is obtained; each sample adds to a cumulative relative evidence variable.

• Mean of the noisy samples is + for one alternative, – for the other, with standard deviation .

• When a bound is reached, the corresponding choice is made.

• Alternatively, in ‘time controlled’ or ‘interrogation’ tasks, respond when signal is given, based on value of the relative evidence variable.

The DDM is an optimal model, and it is consistent with neurophysiology

• It achieves the fastest possible decision on average for a given level of accuracy

• It can be tuned to optimize performance under different kinds of task conditions– Different prior probabilities– Different costs and payoffs– Variation in the time between trials…

• The activity of neurons in a brain area associated with decision making seems to reflect the DD process

Neural Basis of Decision Making in Monkeys (Shadlen & Newsome;

Roitman & Shadlen, 2002)

RT task paradigm of R&T.

Motion coherence anddirection is varied fromtrial to trial.

Neural Basis of Decision Making in Monkeys: Results

Data are averaged over many different neurons that areassociated with intended eye movements to the locationof target.

Two Problems with the DDM

• Accuracy should gradually improve toward ceiling levels as more time is allowed, even for very hard discriminations, but this is not what is observed in human data.

• The model predicts correct and incorrect RT’s will have the same distribution, but incorrect RT’s are generally slower than correct RT’s.

Hard -> Easy

RT

Errors

CorrectResponses

Pro

b. C

orre

ct

Hard

Easy

Usher and McClelland (2001)Leaky Competing Accumulator

Model• Inspired by known neural

mechanisms

• Addresses the process of decidingbetween two alternatives basedon external input (1 + 2 = 1) with leakage, mutual inhibition, and noise:

dx1/dt = 1-k(x1)–f(x2)+1

dx2/dt = 2-k(x2)–f(x1)+2

f(x) = [x]+

Wong & Wang (2006)

~Usher & McClelland (2001)

One-dimensional reduced version of the LCAM (U&M, 2001; Bogacz et al, 2006)

• Neglect the non-linearity in the LCAM:

dx1/dt = 1-k(x1)–(x2)+1

dx2/dt = 2-k(x2)–(x1)+2

Then subtract and let x = x1-x2,

I = 1 - 2 ; 12

To obtain:dx/dt = I -k(x)+(x)+

• This allows precise mathematical analysis, while approximating the outcomes found in the full non-linear version of the LCA…

• And it reduces to the DDM if k- = 0

• So can see a direct link between neural mechanisms and optimal decision making under uncertainty

Roles of k and

x1 – x2 represents the difference in activation of the twoaccumulators for the same value of 1 – 2. Time proceedsfrom stimulus onset. Distribution of values of x1-x2 is shownat three different time points for three combinations of and

BifurcationProducesA Decision-Like Outcome

Time-accuracy curves for different |k-|

|k-= 0|k-= .2|k- = .4

Can we Distinguish Alternative Explanations?

• LCAM fits time-accuracy data well, but there are other possible reasons for bounded accuracy– Trial-to-trial variation in the direction of drift– Bounded integration prior to the termination of the stimulus

• Distinguishing these alternatives as best we can is one of the goals of our project

Specific Aims

• Aim 1: Extend the theory of the dynamics of decision making to address integration of stimulus and payoff information in real-time decision making task situations, integrating behavioral, neuroscientific, and theoretical investigations.

• Aims 2 and 3: Extend this multi-pronged approach to more complex tasks and task environments

– Continuous time and space– Uncertain timing of stimulus onset– Real-life situations including distraction, multi-

tasking, and payoff uncertainty

Some of the Questions we Will be Addressing in Our Work

• To what extent can humans and other participants achieve optimality in decision making, in a range of different decision contexts?  What can we learn from deviations from optimality?

• Within the space of dynamic decision making models, can we find evidence that distinguishes among alternatives consistent with existing data?

• Can we identify the brain mechanisms that underlie the decision process and determine how these mechanisms achieve optimal performance?

• To what extent are the dynamics of decision making fixed characteristics of the decision mechanism, and to what extent are the tunable to task demands?  If so, how is this tuning achieved?

The Schedule

0930 Introduction and scientific background   Jay McClelland

1000 Integrating Payoff and stimulus information           

Bill Newsome

1030 Modeling the dynamics of choice behavior         Phil Holmes

1100 Break

1115 Integration of payoff and stimulus

information in humans      Jay McClelland

1145 Using fMRI to identify decision structures in humans           Jon Cohen

1215 Discussion

1230 Lunch

1330 EEG and MEG studies of decision dynamics             Patrick Simen

1400 Effects of stimulus perturbations and switches on decision dynamics             Juan Gao

1430 Developing continuous measures of decision state         Joel Lachter & James Johnston

1430 Break

1515 Multi-single unit recording studies in primates          Bill Newsome

1545 Other future directions and general discussion      Jay McClelland