Post on 12-Jan-2016
description
Hidden Process Models:Decoding Overlapping Cognitive
States with Unknown Timing
Rebecca A. HutchinsonTom M. Mitchell
Carnegie Mellon University
NIPS Workshops: New Directions on Decoding Mental States from fMRI Data
December 8, 2006
2
Overview
• Open questions we address:– Treating fMRI as the time series that it is.– Allowing the testing of hypotheses.
• Open questions we do NOT address:– Interpretability of time series or spatial representation
of activity.
• This talk– Motivation– HPMs (in 1 slide!)– Preliminary results
3
Motivation
• Goal: connect fMRI to cognitive modeling.
• Cognitive Model:– Set of cognitive processes hypothesized to occur
during a given fMRI experiment.
• Cognitive Process:– Spatial-temporal hemodynamic response function.– Timing distribution relative to experiment landmarks
(like stimulus presentations and behavioral data).
4
Study: Pictures and Sentences
• Task: Decide whether sentence describes picture correctly, indicate with button press.
• 13 normal subjects, 40 trials per subject.• Sentences and pictures describe 3 symbols: *,
+, and $, using ‘above’, ‘below’, ‘not above’, ‘not below’.
• Images are acquired every 0.5 seconds.
Read Sentence
View Picture Read Sentence
View PictureFixation
Press Button
4 sec. 8 sec.t=0
Rest
5
One Cognitive Model
Read Sentence
View Picture Read Sentence
View PictureFixation
Press Button
4 sec. 8 sec.t=0
Rest
• ViewPicture – begins when picture stimulus is presented
• ReadSentence– begins when sentence stimulus is presented
• Decide– begins within 4 seconds of 2nd stimulus
ViewPicture or ReadSentence ViewPicture or ReadSentence
Decide
6
Hidden Markov Models (HMMs)
Hidden Process Models (HPMs)
States (1 time point) Processes (time window)
Latent state sequence Latent process instances
No external input Use experiment design and behavioral data
State transition matrix Process-specific timing distributions
State-specific emission distributions
Process-specific response signatures
1 hidden Markov chain governs observed data
Process instances can overlap in space and time
Forward-backward training algorithm
EM training algorithm
7
ViewPicture in Visual Cortex
8
ReadSentence in Visual Cortex
9
ViewPicture
10
ReadSentence
11
Decide
0 0.5 1 1.5 2 2.5 3 3.5
0 0 0 0 0.025 0.05 0.075 0.85
Seconds following the second stimulus
Multinomial probabilities on these time points
12
Comparing ModelsHPM Avg. Test Set LL
PS -1.0784 * 10^6
PSD -1.0759 * 10^6
PS+S-D -1.0742 * 10^6
PSD+D- -1.0742 * 10^6
PSDB -1.0741 * 10^6
PSDyDn -1.0737 * 10^6
PSDyDnDc** -1.0717 * 10^6
PSDyDnDcB -1.0711 * 10^6
5-fold cross-validation, 1 subject
P = ViewPicture
S = ReadSentence
S+ = ReadAffirmativeSentence
S- = ReadNegatedSentence
D = Decide
D+ = DecideAfterAffirmative
D- = DecideAfterNegated
Dy = DecideYes
Dn = DecideNo
Dc = DecideConfusion
B = Button
** - This HPM can also classify Dy vs. Dn with 92.0% accuracy. GNBC gets 53.9%. (using the window from the second stimulus to the end of the trial)
13
Conclusions
• Simultaneous estimation of spatial-temporal signature (HRF) and temporal onset of cognitive processes.
• Framework for principled comparison of different cognitive models in terms of real data.