![Structural Deep Clustering Network · 2020-02-13 · embedding clustering [4] adds a reconstruction loss to the objective of DEC as a constraint to help autoencoder learn a better](https://static.fdocuments.in/doc/165x107/5ea64aac5ea36010523a237d/structural-deep-clustering-network-2020-02-13-embedding-clustering-4-adds-a.jpg)
Model-based clustering using generative embedding
1
4 Results on synthetic fMRI data Our analysis discovered the correct number of clusters (two) when the groups were well separated or there was a sufficiently high signal-to-noise ratio. -2 0 2 -2 0 2 B 21 B 12 ground truth -2 0 2 -2 0 2 B 21 B 12 estimates 1 2 3 4 5 0 20 40 log model evidence 1 2 3 4 5 0 0.5 1 balanced purity -2 0 2 -2 0 2 B 21 B 12 ground truth -2 0 2 -2 0 2 B 21 B 12 estimates 1 2 3 4 5 0 20 40 60 80 log model evidence 1 2 3 4 5 0 0.5 1 balanced purity -2 0 2 -2 0 2 B 21 B 12 ground truth -2 0 2 -2 0 2 B 21 B 12 estimates 1 2 3 4 5 0 10 20 log model evidence 1 2 3 4 5 0 0.5 1 balanced purity -2 0 2 -2 0 2 B 21 B 12 ground truth -2 0 2 -2 0 2 B 21 B 12 estimates 1 2 3 4 5 0 20 40 log model evidence 1 2 3 4 5 0 0.5 1 balanced purity ground truth signal-to-noise ratio (SNR) group separation high low low high 3 Model-based clustering We introduce generative embedding for model-based clustering using a combination of dynamic causal models (DCM) and variational Gaussian Mixture Models (GMM) clustering [5]. 2 Datasets 5 Results on empirical fMRI data Using a linear support vector machine (SVM), we were able to predict a subject‘s diagnostic status with an accuracy of 78% (left). We then adopted an unsupervised exploratory approach: generative embedding inferred that the data comprised two subgroups. These subgroups showed a 71% correspondence with schizophrenic patiens and healthy controls (right). To assess the empirical validity of our approach, we analysed fMRI data from schizophrenia patients and healthy controls engaged in a working-memory task [4]. Using a DCM of prefrontal and parietal activity, we asked whether we could discover the diagnostic category ‘schizophrenia’ from patterns of connectivity. 1 Introduction • An important problem in psychiatry is the lack of diagnostic classifications that are based on pathophysiological mechanisms rather than symptoms. • It is conceivable that one could solve this problem by constructing generative models of brain function that enable inference on the computational and neuronal processes that underlie an observed collection of symptoms. • We recently showed that generative embedding based on such models can yield highly accurate predictions of a symptom-based diagnostic state from fMRI data [1,2]. • In this study, we are beginning to address the open question of whether generative embedding might allow us to discover clinically relevant conditions when such conditions are not known a priori. Model-based clustering using generative embedding Kay H Brodersen 1,2,3 ∙ Zhihao Lin 2 ∙ Lorenz Deserno 4,5 ∙ Ajita Gupta 2 ∙ Will D Penny 6 ∙ Alexander P Leff 6 Morteza H Chehreghani 2 ∙ Alberto-Giovanni Busetto 2,7 ∙ Florian Schlagenhauf 4,5 ∙ Joachim M Buhmann 2 ∙ Klaas E Stephan 1,3,6 1 Translational Neuromodeling Unit (TNU), University of Zurich & ETH Zurich, Switzerland 2 Department of Computer Science, ETH Zurich, Switzerland 3 Laboratory for Social and Neural Systems Research, Department of Economics, University of Zurich, Switzerland 4 Department of Psychiatry and Psychotherapy, Charité-Universitätsmedizin Berlin, Germany 5 Max Planck Institute for Cognitive and Brain Sciences, Leipzig, Germany 6 Wellcome Trust Centre for Neuroimaging, University College London, United Kingdom 7 Competence Center for Systems Physiology and Metabolic Diseases, Zurich, Switzerland 6 Conclusions • Clustering using generative embedding may enable us to decompose groups of patients with similar symptoms into pathophysiologically distinct subtypes. • In contrast to conventional activation-based, correlation-based, or symptom- based clustering schemes, our approach exploits discriminative information encoded in ‘hidden’ physiological quantities such as synaptic connection strengths. • Critically, generative embedding enables a mechanistic interpretation of the discovered structures. References 1. Brodersen, K.H., Haiss, F., Ong, C.S., Jung, F., Tittgemeyer, M., Buhmann, J.M., Weber, B., Stephan, K.E. (2011). Model-based feature construction for multivariate decoding. NeuroImage, 56, 601-615. 2. Brodersen, K.H., Schofield, T.M., Leff, A.P., Ong, C.S., Lomakina, E.I., Buhmann, J.M., Stephan, K.E. (2011). Generative embedding for model-based classification of fMRI data. PLoS Comput Biol, 7(6): e1002079. 3. Friston, K.J., Harrison, L., & Penny, W., 2003. Dynamic causal modelling. NeuroImage, 19(4), 1273-1302. 4. Deserno, L., Sterzer, P., Wüstenberg, T., Heinz, A., & Schlagenhauf, F. (2012). Reduced prefrontal-parietal effective connectivity and working memory deficits in schizophrenia. Journal of Neuroscience, 31(1), 12-20. 5. Penny, W.. (in preparation). Variational Bayes for d-dimensional Gaussian mixture models. 6. Stephan, K. E., Friston, K. J., & Frith, C. D. (2009). Dysconnection in schizophrenia: From abnormal synaptic plasticity to failures of self-monitoring. Schizophrenia Bulletin, 35(3), 509-527. Synthetic fMRI data (n = 80) Empirical fMRI data (n = 83) subgroup 1 To investigate the theoretical properties of our approach, we generated fMRI data for two synthetic subject groups using a simple four-region DCM [3], as shown above. For each subject, the true DCM parameters were drawn from a Gaussian with a group-specific mean. The two subgroups differed in terms of modulatory effects on their intrinsic connectivity ( ). We then generated fMRI data from these DCMs, estimated the model parameters, and submitted these estimates to clustering. 0 0.5 1 balanced accuracy 1 2 3 4 5 6 7 8 0 50 100 log model evidence 1 2 3 4 5 6 7 8 0 0.2 0.4 0.6 0.8 1 balanced purity Model-based solutions can be interpreted in terms of the underlying generative model. In the model underlying cluster 1, which contained almost exclusively healthy controls, working memory had a significantly stronger modulatory effect than in cluster 2, which was mostly composed of patients. Supervised setting: support vector classification conventional classification generative embedding 78% 71% New unsupervised setting: GMM clustering number of clusters number of clusters best model PC dLPFC VC WM PC dLPFC VC WM cluster 2 cluster 1 Clustering solution + SZ patient healthy control PC dLPFC VC WM stimulus step 3 — embedding step 1 — extraction measurements from an individual subject subject-specific generative model representation in model-based feature space A → B A → C B → B B → C A C B step 4 — clustering A C B jointly discriminative connection strengths? step 6 — interpretation emerging groups of similar subjects? 1 0 agreement with known group labels? balanced purity step 5 — validation step 2 — modelling time series in regions of interest 2 1 3 4 modulatory input modulatory input stimulus input subgroup 2 2 1 3 4 modulatory input modulatory input stimulus input MDS axis 1 MDS axis 2
-
Upload
ajita-gupta -
Category
Documents
-
view
34 -
download
0
description
Poster presentation at the Human Brain Mapping conference in Beijing, China, 2012.
Transcript of Model-based clustering using generative embedding
4 Results on synthetic fMRI data
Our analysis discovered the correct number of clusters (two) when the groups were well separated or there was a sufficiently high signal-to-noise ratio.
-2 0 2-2
0
2
B21
B1
2
ground truth
-2 0 2-2
0
2
B21
B1
2
estimates
1 2 3 4 50
10
20
log m
odel evid
ence
1 2 3 4 50
0.5
1
bala
nced p
urity
-2 0 2-2
0
2
B21
B1
2
ground truth
-2 0 2-2
0
2
B21
B1
2estimates
1 2 3 4 50
20
40
log m
odel evid
ence
1 2 3 4 50
0.5
1
bala
nced p
urity
-2 0 2-2
0
2
B21
B1
2
ground truth
-2 0 2-2
0
2
B21
B1
2
estimates
1 2 3 4 50
20
40
log m
odel evid
ence
1 2 3 4 50
0.5
1
bala
nced p
urity
-2 0 2-2
0
2
B21
B1
2
ground truth
-2 0 2-2
0
2
B21
B1
2
estimates
1 2 3 4 50
20
40
60
80lo
g m
odel evid
ence
1 2 3 4 50
0.5
1
bala
nced p
urity
-2 0 2-2
0
2
B21
B1
2
ground truth
-2 0 2-2
0
2
B21
B1
2
estimates
1 2 3 4 50
10
20
log m
odel evid
ence
1 2 3 4 50
0.5
1
bala
nced p
urity
-2 0 2-2
0
2
B21
B1
2
ground truth
-2 0 2-2
0
2
B21
B1
2
estimates
1 2 3 4 50
20
40
log m
odel evid
ence
1 2 3 4 50
0.5
1
bala
nced p
urity
-2 0 2-2
0
2
B21
B1
2
ground truth
-2 0 2-2
0
2
B21
B1
2
estimates
1 2 3 4 50
20
40
log m
odel evid
ence
1 2 3 4 50
0.5
1
bala
nced p
urity
-2 0 2-2
0
2
B21
B1
2
ground truth
-2 0 2-2
0
2
B21
B1
2
estimates
1 2 3 4 50
20
40
60
80
log m
odel evid
ence
1 2 3 4 50
0.5
1
bala
nced p
urity
-2 0 2-2
0
2
B21
B1
2
ground truth
-2 0 2-2
0
2
B21
B1
2
estimates
1 2 3 4 50
10
20lo
g m
odel evid
ence
1 2 3 4 50
0.5
1
bala
nced p
urity
-2 0 2-2
0
2
B21
B1
2
ground truth
-2 0 2-2
0
2
B21
B1
2
estimates
1 2 3 4 50
20
40
log m
odel evid
ence
1 2 3 4 50
0.5
1
bala
nced p
urity
-2 0 2-2
0
2
B21
B1
2
ground truth
-2 0 2-2
0
2
B21
B1
2
estimates
1 2 3 4 50
20
40
log m
odel evid
ence
1 2 3 4 50
0.5
1
bala
nced p
urity
-2 0 2-2
0
2
B21
B1
2
ground truth
-2 0 2-2
0
2
B21
B1
2
estimates
1 2 3 4 50
20
40
60
80lo
g m
odel evid
ence
1 2 3 4 50
0.5
1
bala
nced p
urity
-2 0 2-2
0
2
B21
B1
2
ground truth
-2 0 2-2
0
2
B21
B1
2
estimates
1 2 3 4 50
10
20
log m
odel evid
ence
1 2 3 4 50
0.5
1
bala
nced p
urity
-2 0 2-2
0
2
B21
B1
2
ground truth
-2 0 2-2
0
2
B21
B1
2
estimates
1 2 3 4 50
20
40
log m
odel evid
ence
1 2 3 4 50
0.5
1
bala
nced p
urity
-2 0 2-2
0
2
B21
B1
2
ground truth
-2 0 2-2
0
2
B21
B1
2
estimates
1 2 3 4 50
20
40
log m
odel evid
ence
1 2 3 4 50
0.5
1
bala
nced p
urity
-2 0 2-2
0
2
B21
B1
2
ground truth
-2 0 2-2
0
2
B21
B1
2
estimates
1 2 3 4 50
20
40
60
80
log m
odel evid
ence
1 2 3 4 50
0.5
1
bala
nced p
urity
sig
na
l-to
-no
ise
ra
tio
(S
NR
)
group separation
hig
h
low
low high
3 Model-based clustering
We introduce generative embedding for model-based clustering using a combination of dynamic causal models (DCM) and variational Gaussian Mixture Models (GMM) clustering [5].
2 Datasets
5 Results on empirical fMRI data
Using a linear support vector machine (SVM), we were able to predict a subject‘s diagnostic status with an accuracy of 78% (left). We then adopted an unsupervised exploratory approach: generative embedding inferred that the data comprised two subgroups. These subgroups showed a 71% correspondence with schizophrenic patiens and healthy controls (right).
To assess the empirical validity of our approach, we analysed fMRI data from schizophrenia patients and healthy controls engaged in a working-memory task [4]. Using a DCM of prefrontal and parietal activity, we asked whether we could discover the diagnostic category ‘schizophrenia’ from patterns of connectivity.
1 Introduction
• An important problem in psychiatry is the lack of diagnostic classifications that are based on pathophysiological mechanisms rather than symptoms.
• It is conceivable that one could solve this problem by constructing generative models of brain function that enable inference on the computational and neuronal processes that underlie an observed collection of symptoms.
• We recently showed that generative embedding based on such models can yield highly accurate predictions of a symptom-based diagnostic state from fMRI data [1,2].
• In this study, we are beginning to address the open question of whether generative embedding might allow us to discover clinically relevant conditions when such conditions are not known a priori.
Model-based clustering using generative embedding Kay H Brodersen1,2,3 ∙ Zhihao Lin2 ∙ Lorenz Deserno4,5 ∙ Ajita Gupta2 ∙ Will D Penny6 ∙ Alexander P Leff6
Morteza H Chehreghani2 ∙ Alberto-Giovanni Busetto2,7 ∙ Florian Schlagenhauf4,5 ∙ Joachim M Buhmann2 ∙ Klaas E Stephan1,3,6
1 Translational Neuromodeling Unit (TNU), University of Zurich & ETH Zurich, Switzerland 2 Department of Computer Science, ETH Zurich, Switzerland 3 Laboratory for Social and Neural Systems Research, Department of Economics, University of Zurich, Switzerland 4 Department of Psychiatry and Psychotherapy, Charité-Universitätsmedizin Berlin, Germany 5 Max Planck Institute for Cognitive and Brain Sciences, Leipzig, Germany 6 Wellcome Trust Centre for Neuroimaging, University College London, United Kingdom 7 Competence Center for Systems Physiology and Metabolic Diseases, Zurich, Switzerland
6 Conclusions
• Clustering using generative embedding may enable us to decompose groups of patients with similar symptoms into pathophysiologically distinct subtypes.
• In contrast to conventional activation-based, correlation-based, or symptom-based clustering schemes, our approach exploits discriminative information encoded in ‘hidden’ physiological quantities such as synaptic connection strengths.
• Critically, generative embedding enables a mechanistic interpretation of the discovered structures.
References
1. Brodersen, K.H., Haiss, F., Ong, C.S., Jung, F., Tittgemeyer, M., Buhmann, J.M., Weber, B., Stephan, K.E. (2011). Model-based feature construction for multivariate decoding. NeuroImage, 56, 601-615.
2. Brodersen, K.H., Schofield, T.M., Leff, A.P., Ong, C.S., Lomakina, E.I., Buhmann, J.M., Stephan, K.E. (2011). Generative embedding for model-based classification of fMRI data. PLoS Comput Biol, 7(6): e1002079.
3. Friston, K.J., Harrison, L., & Penny, W., 2003. Dynamic causal modelling. NeuroImage, 19(4), 1273-1302.
4. Deserno, L., Sterzer, P., Wüstenberg, T., Heinz, A., & Schlagenhauf, F. (2012). Reduced prefrontal-parietal effective connectivity and working memory deficits in schizophrenia. Journal of Neuroscience, 31(1), 12-20.
5. Penny, W.. (in preparation). Variational Bayes for d-dimensional Gaussian mixture models.
6. Stephan, K. E., Friston, K. J., & Frith, C. D. (2009). Dysconnection in schizophrenia: From abnormal synaptic plasticity to failures of self-monitoring. Schizophrenia Bulletin, 35(3), 509-527.
Synthetic fMRI data (n = 80) Empirical fMRI data (n = 83)
subgroup 1
To investigate the theoretical properties of our approach, we generated fMRI data for two synthetic subject groups using a simple four-region DCM [3], as shown above. For each subject, the true DCM parameters were drawn from a Gaussian with a group-specific mean. The two subgroups differed in terms of modulatory effects on their intrinsic connectivity (𝜇𝐵 ). We then generated fMRI data from these DCMs, estimated the model parameters, and submitted these estimates to clustering.
0
0.5
1
bala
nced a
ccura
cy
1 2 3 4 5 6 7 80
50
100
log m
odel evid
ence
1 2 3 4 5 6 7 80
0.2
0.4
0.6
0.8
1
bala
nced p
urity
0
0.5
1
bala
nced a
ccura
cy
1 2 3 4 5 6 7 80
50
100
log m
odel evid
ence
1 2 3 4 5 6 7 80
0.2
0.4
0.6
0.8
1
bala
nced p
urity
0
0.5
1
bala
nced a
ccura
cy
1 2 3 4 5 6 7 80
50
100
log m
odel evid
ence
1 2 3 4 5 6 7 80
0.2
0.4
0.6
0.8
1
bala
nced p
urity
Model-based solutions can be interpreted in terms of the underlying generative model. In the model underlying cluster 1, which contained almost exclusively healthy controls, working memory had a significantly stronger modulatory effect than in cluster 2, which was mostly composed of patients.
Supervised setting: support vector classification
co
nve
nti
on
al
cla
ss
ific
ati
on
ge
ne
rati
ve
e
mb
ed
din
g
78%
71%
New unsupervised setting: GMM clustering
number of clusters number of clusters
best model
PC dLPFC
VC
WM PC dLPFC
VC
WM
cluster 2 cluster 1 Clustering solution
+ SZ patient
healthy control
PC dLPFC
VC
WM
stimulus
step 3 — embedding
step 1 — extraction
measurements from an individual subject
subject-specific generative model
representation in model-based feature space
A → B
A → C
B → B
B → C
A
C B
step 4 — clustering
A
C B
jointly discriminative connection strengths?
step 6 — interpretation
emerging groups of similar subjects?
1
0
agreement with known group labels?
ba
lan
ced
pu
rity
step 5 — validation
step 2 — modelling
time series in regions of interest
2 1
3 4
modulatory input
modulatory input
stimulus input
subgroup 2 2 1
3 4
modulatory input
modulatory input
stimulus input
MDS axis 1
MD
S a
xis
2
![Epileptic Seizure Forecasting With Generative Adversarial ...clustering,Gaussianmixturemodels,HiddenMarkovModels and autoencoders [13], [14]. Most of these unsupervised learning techniques](https://static.fdocuments.in/doc/165x107/60b177a4482be642596be326/epileptic-seizure-forecasting-with-generative-adversarial-clusteringgaussianmixturemodelshiddenmarkovmodels.jpg)
