Brain Maps like Mine
-
Upload
vanessa-sochat -
Category
Science
-
view
358 -
download
0
Transcript of Brain Maps like Mine
![Page 1: Brain Maps like Mine](https://reader036.fdocuments.in/reader036/viewer/2022062903/58a389c71a28ab9c7e8b5b43/html5/thumbnails/1.jpg)
Brain Maps Like Minesemantic and computational image comparison
methods for meta-analysis and reproducibility of brain statistical maps
Vanessa SochatResearch In Progress
October 20, 2015
![Page 2: Brain Maps like Mine](https://reader036.fdocuments.in/reader036/viewer/2022062903/58a389c71a28ab9c7e8b5b43/html5/thumbnails/2.jpg)
Outline
BackgroundWhy do we want to compare images?
![Page 3: Brain Maps like Mine](https://reader036.fdocuments.in/reader036/viewer/2022062903/58a389c71a28ab9c7e8b5b43/html5/thumbnails/3.jpg)
Outline
BackgroundWhy do we want to compare images?
Computational Image ComparisonImpact of Image Thresholding on Similarity Metrics
![Page 4: Brain Maps like Mine](https://reader036.fdocuments.in/reader036/viewer/2022062903/58a389c71a28ab9c7e8b5b43/html5/thumbnails/4.jpg)
Outline
BackgroundWhy do we want to compare images?
Computational Image ComparisonImpact of Image Thresholding on Similarity Metrics
Semantic Image ComparisonOntological and Graph Based Methods
![Page 5: Brain Maps like Mine](https://reader036.fdocuments.in/reader036/viewer/2022062903/58a389c71a28ab9c7e8b5b43/html5/thumbnails/5.jpg)
Meta-analysis to synthesize understanding of human cognition and reproducibility of brain statistical maps.
Why Compare Images?
![Page 6: Brain Maps like Mine](https://reader036.fdocuments.in/reader036/viewer/2022062903/58a389c71a28ab9c7e8b5b43/html5/thumbnails/6.jpg)
Are we good at reproducibility?
![Page 7: Brain Maps like Mine](https://reader036.fdocuments.in/reader036/viewer/2022062903/58a389c71a28ab9c7e8b5b43/html5/thumbnails/7.jpg)
Why Compare Images?
1. For ReproducibilityWe do not not what a replication looks like.
![Page 8: Brain Maps like Mine](https://reader036.fdocuments.in/reader036/viewer/2022062903/58a389c71a28ab9c7e8b5b43/html5/thumbnails/8.jpg)
Why Compare Images?
1. For ReproducibilityWe do not not what a replication looks like.
2. For Meta AnalysisWhat does all the research say about “anxiety?”
![Page 9: Brain Maps like Mine](https://reader036.fdocuments.in/reader036/viewer/2022062903/58a389c71a28ab9c7e8b5b43/html5/thumbnails/9.jpg)
Outline
BackgroundWhy do we want to compare images?
Computational Image ComparisonImpact of Image Thresholding on Similarity Metrics
Semantic Image ComparisonOntological and Graph Based Methods
![Page 10: Brain Maps like Mine](https://reader036.fdocuments.in/reader036/viewer/2022062903/58a389c71a28ab9c7e8b5b43/html5/thumbnails/10.jpg)
Results
What if there is data missing?Should I tranform the images first?What am I trying to optimize?
How similar are these results?
![Page 11: Brain Maps like Mine](https://reader036.fdocuments.in/reader036/viewer/2022062903/58a389c71a28ab9c7e8b5b43/html5/thumbnails/11.jpg)
Goal: assess influence of different degrees of image thresholding on the outcome of pairwise image comparison
![Page 12: Brain Maps like Mine](https://reader036.fdocuments.in/reader036/viewer/2022062903/58a389c71a28ab9c7e8b5b43/html5/thumbnails/12.jpg)
Goal: assess influence of different degrees of image thresholding on the outcome of pairwise image comparison
VARIABLESthresholdsmetricsoptimization
![Page 13: Brain Maps like Mine](https://reader036.fdocuments.in/reader036/viewer/2022062903/58a389c71a28ab9c7e8b5b43/html5/thumbnails/13.jpg)
Methods Results
1. Define thresholds
Z = +/- [0,13]
![Page 14: Brain Maps like Mine](https://reader036.fdocuments.in/reader036/viewer/2022062903/58a389c71a28ab9c7e8b5b43/html5/thumbnails/14.jpg)
Methods Results
GIVEN
SINGLE VALUE IMPUTATION
COMPLETE CASE ANALYSIS
2. Define comparison method
![Page 15: Brain Maps like Mine](https://reader036.fdocuments.in/reader036/viewer/2022062903/58a389c71a28ab9c7e8b5b43/html5/thumbnails/15.jpg)
Methods Results
3. Define our similarity metrics
Pearson’s R
where
"Correlation coefficient" by Kiatdd - Own work. Licensed under CC BY-SA 3.0 via Wikimedia Commons
Spearman’s Rank Correlation Coefficient
![Page 16: Brain Maps like Mine](https://reader036.fdocuments.in/reader036/viewer/2022062903/58a389c71a28ab9c7e8b5b43/html5/thumbnails/16.jpg)
Methods Results
4. What are we optimizing?
![Page 17: Brain Maps like Mine](https://reader036.fdocuments.in/reader036/viewer/2022062903/58a389c71a28ab9c7e8b5b43/html5/thumbnails/17.jpg)
Methods Results
Data
465 single subjects7 tasks47 contrast images
working memoryanimalsgamblinglanguagerelationalemotionsocialmotor
“cat” vs. “dog”
![Page 18: Brain Maps like Mine](https://reader036.fdocuments.in/reader036/viewer/2022062903/58a389c71a28ab9c7e8b5b43/html5/thumbnails/18.jpg)
Methods Results
Subsampling Procedure
For each of 500 subsamples: Subset data to unrelated groups A and B For each unthresholded map, A i in A Apply each threshold in Z = +/- 0:13, and Z = + 0:13 to all of B Calculate similarity for each of B to A i with CCA or SVI Assign correct classification if contrast A i most similar to equivalent contrast in B
![Page 19: Brain Maps like Mine](https://reader036.fdocuments.in/reader036/viewer/2022062903/58a389c71a28ab9c7e8b5b43/html5/thumbnails/19.jpg)
Methods Results
Results
Highest classification accuracy for our dataset: Pearson Complete Case Analysis +/- 1.0
0.984
![Page 20: Brain Maps like Mine](https://reader036.fdocuments.in/reader036/viewer/2022062903/58a389c71a28ab9c7e8b5b43/html5/thumbnails/20.jpg)
Results
...
![Page 21: Brain Maps like Mine](https://reader036.fdocuments.in/reader036/viewer/2022062903/58a389c71a28ab9c7e8b5b43/html5/thumbnails/21.jpg)
Methods Results
https://github.com/vsoch/image-comparison-thresholding
![Page 22: Brain Maps like Mine](https://reader036.fdocuments.in/reader036/viewer/2022062903/58a389c71a28ab9c7e8b5b43/html5/thumbnails/22.jpg)
Results
![Page 23: Brain Maps like Mine](https://reader036.fdocuments.in/reader036/viewer/2022062903/58a389c71a28ab9c7e8b5b43/html5/thumbnails/23.jpg)
Methods Results
Conclusions
1. More data is not always better minimum degree of thresholding improves accuracy random field theory may be too much
2. Question the choice of metric, threshold, etc. complete case analysis, pearson, worked for us...
3. “What is the quantitative language that we should use to compare two images?”
![Page 24: Brain Maps like Mine](https://reader036.fdocuments.in/reader036/viewer/2022062903/58a389c71a28ab9c7e8b5b43/html5/thumbnails/24.jpg)
Outline
BackgroundWhy do we want to compare images?
Computational Image ComparisonImpact of Image Thresholding on Similarity Metrics
Semantic Image ComparisonOntological and Graph Based Methods
![Page 25: Brain Maps like Mine](https://reader036.fdocuments.in/reader036/viewer/2022062903/58a389c71a28ab9c7e8b5b43/html5/thumbnails/25.jpg)
cat dog
Semantic Image Comparison
![Page 26: Brain Maps like Mine](https://reader036.fdocuments.in/reader036/viewer/2022062903/58a389c71a28ab9c7e8b5b43/html5/thumbnails/26.jpg)
Is semantic comparison of images useful to classify cognitive states?
![Page 27: Brain Maps like Mine](https://reader036.fdocuments.in/reader036/viewer/2022062903/58a389c71a28ab9c7e8b5b43/html5/thumbnails/27.jpg)
The Approach:- “graph” based similarity- “probabilistic” based similarity- compare the two to spatial similarity
Semantic Similarity: OverviewIs semantic comparison of images useful to classify cognitive states?
Goals:- completely automated- assess predictive power of semantic similarity- relate to computational (spatial) similarity
![Page 28: Brain Maps like Mine](https://reader036.fdocuments.in/reader036/viewer/2022062903/58a389c71a28ab9c7e8b5b43/html5/thumbnails/28.jpg)
cat
dog
Semantic Similarity: OverviewIs semantic comparison of images useful to classify cognitive states?
![Page 29: Brain Maps like Mine](https://reader036.fdocuments.in/reader036/viewer/2022062903/58a389c71a28ab9c7e8b5b43/html5/thumbnails/29.jpg)
cat
dog
Semantic Similarity: OverviewIs semantic comparison of images useful to classify cognitive states?
![Page 30: Brain Maps like Mine](https://reader036.fdocuments.in/reader036/viewer/2022062903/58a389c71a28ab9c7e8b5b43/html5/thumbnails/30.jpg)
ONTOLOGY
Tools
Semantic Similarity: MethodIs semantic comparison of images useful to classify cognitive states?
Goals:- completely automated
- Cognitive Atlas, NeuroVault, Pybraincompare
catdog
graph similarity( , )
IMAGE DATA METHODS
![Page 31: Brain Maps like Mine](https://reader036.fdocuments.in/reader036/viewer/2022062903/58a389c71a28ab9c7e8b5b43/html5/thumbnails/31.jpg)
Data:- 93 brain maps tagged in NeuroVault with contrast → concept- programatically retrieve data, run methods, and output result
and visualization.
General Workflow- publish interesting results- tag with a contrast, and associated cognitive concepts- assess semantic similarity
- graph based- probabilistic
Semantic Similarity: MethodIs semantic comparison of images useful to classify cognitive states?
![Page 32: Brain Maps like Mine](https://reader036.fdocuments.in/reader036/viewer/2022062903/58a389c71a28ab9c7e8b5b43/html5/thumbnails/32.jpg)
Semantic SimilarityGraph Based Methods
![Page 33: Brain Maps like Mine](https://reader036.fdocuments.in/reader036/viewer/2022062903/58a389c71a28ab9c7e8b5b43/html5/thumbnails/33.jpg)
cat dog
feline canine
mammal
graph similarity( , )
![Page 34: Brain Maps like Mine](https://reader036.fdocuments.in/reader036/viewer/2022062903/58a389c71a28ab9c7e8b5b43/html5/thumbnails/34.jpg)
Graph Similarity: Method
visual feline recognition
visual canine recognition
Wang’s Method
- aggregates semantic contributions of ancestor terms
1. We start with associated concepts.
visual canine recognition
animal recognitionis a kind of
is part ofis a kind of recognition
canine fear response
![Page 35: Brain Maps like Mine](https://reader036.fdocuments.in/reader036/viewer/2022062903/58a389c71a28ab9c7e8b5b43/html5/thumbnails/35.jpg)
Wang’s Method
- aggregates semantic contributions of ancestor terms
2. We then take weights at intersection
visual canine recognition
animal recognitionis a kind of
is part ofis a kind of recognition
canine fear response
visual feline recognition
animal recognitionis a kind of
is part ofis a kind of recognition
feline fear response
S( , ) = sum(intersected weights) sum(all weights)
Graph Similarity: Method
![Page 36: Brain Maps like Mine](https://reader036.fdocuments.in/reader036/viewer/2022062903/58a389c71a28ab9c7e8b5b43/html5/thumbnails/36.jpg)
Semantic
Graph Similarity: First Round
![Page 37: Brain Maps like Mine](https://reader036.fdocuments.in/reader036/viewer/2022062903/58a389c71a28ab9c7e8b5b43/html5/thumbnails/37.jpg)
Semantic SimilarityProbabilistic Methods
![Page 38: Brain Maps like Mine](https://reader036.fdocuments.in/reader036/viewer/2022062903/58a389c71a28ab9c7e8b5b43/html5/thumbnails/38.jpg)
Reverse InferenceP(cognitive process | activation)
database of images a new result
![Page 39: Brain Maps like Mine](https://reader036.fdocuments.in/reader036/viewer/2022062903/58a389c71a28ab9c7e8b5b43/html5/thumbnails/39.jpg)
Reverse Inferencefor image classification and concept validation
a new result
P( | )
P( | )
P(cognitive process | a spatial map)
P(node mental process|activation) = P(activation|mental process) * P(mental process)
P(activation|mental process) * P(mental process) + P(A|~mental process) * P(~mental process)
![Page 40: Brain Maps like Mine](https://reader036.fdocuments.in/reader036/viewer/2022062903/58a389c71a28ab9c7e8b5b43/html5/thumbnails/40.jpg)
P( | )
What does a high score say?about the cognitive concept?
P( | )contributes evidence for
![Page 41: Brain Maps like Mine](https://reader036.fdocuments.in/reader036/viewer/2022062903/58a389c71a28ab9c7e8b5b43/html5/thumbnails/41.jpg)
Data:93 brain maps tagged in NeuroVault with contrast → conceptprogramatically retrieve data, run methods, and output result and visualization.
For each of 93 brainmaps, as query image: For each of 140 concept nodes, node, in Cognitive Atlas: calculate P(node|query image) Assign correct classification if P(node|query image) > 0.5
Probabilistic Similarity Method
![Page 42: Brain Maps like Mine](https://reader036.fdocuments.in/reader036/viewer/2022062903/58a389c71a28ab9c7e8b5b43/html5/thumbnails/42.jpg)
Probabilistic Similarity Preliminary Results
![Page 43: Brain Maps like Mine](https://reader036.fdocuments.in/reader036/viewer/2022062903/58a389c71a28ab9c7e8b5b43/html5/thumbnails/43.jpg)
Probabilistic Similarity Area Under the Curve Across Concepts
![Page 44: Brain Maps like Mine](https://reader036.fdocuments.in/reader036/viewer/2022062903/58a389c71a28ab9c7e8b5b43/html5/thumbnails/44.jpg)
Summary
Image comparison is essential formeta-analysis and reproducibility
![Page 45: Brain Maps like Mine](https://reader036.fdocuments.in/reader036/viewer/2022062903/58a389c71a28ab9c7e8b5b43/html5/thumbnails/45.jpg)
Summary
Image comparison is essential formeta-analysis and reproducibility
A small amount of image thresholdingaids to find images of similar contrast
![Page 46: Brain Maps like Mine](https://reader036.fdocuments.in/reader036/viewer/2022062903/58a389c71a28ab9c7e8b5b43/html5/thumbnails/46.jpg)
Summary
Image comparison is essential formeta-analysis and reproducibility
A small amount of image thresholdingaids to find images of similar contrast
Semantic Image Comparisonis a promising strategy to assess reproducibility
![Page 47: Brain Maps like Mine](https://reader036.fdocuments.in/reader036/viewer/2022062903/58a389c71a28ab9c7e8b5b43/html5/thumbnails/47.jpg)
Summary
Image comparison is essential formeta-analysis and reproducibility
A small amount of image thresholdingaids to find images of similar contrast
Semantic Image Comparisonis a promising strategy to assess reproducibility
![Page 48: Brain Maps like Mine](https://reader036.fdocuments.in/reader036/viewer/2022062903/58a389c71a28ab9c7e8b5b43/html5/thumbnails/48.jpg)
Acknowledgements
INCF/ NidashSatra GhoshNolan NicholsJessica TurnerTom NicholsJB PolineDavid Keator
CollaboratorsTal Yarkoni
Nipy
FundingMicrosoft ResearchSGF and NSF
PoldracklabRuss PoldrackChris GorgolewskiCraig MoodieSanmi KoyejoPatrick BissettJoke DurnezIan EisenbergMac ShineJoe Wexler
BMIDaniel RubinRuss AltmanMark MusenRebecca SawyerMary JeanneNancySteven BagleyJohn DiMario
![Page 50: Brain Maps like Mine](https://reader036.fdocuments.in/reader036/viewer/2022062903/58a389c71a28ab9c7e8b5b43/html5/thumbnails/50.jpg)
Coordinate-Based Approaches
column 1: raw data: big black dots showing the local maxima that are reported, dotted line is “true” simulated signal, black thick line is that signal with added noise.column 2: shows the results of ALE: the result is more of a curve because the “ALE statistic” reflects a probability value that at least one peak is within r mm of each voxel, so the highest values of course correspond to actual peaks.column 3: kernel density analysis (KDA) gives us a value at each voxel that represent the number of peaks within r mm of that voxel. If we divide by voxel resolution we can turn that into a “density”column 4: is MULTI kernel density analysis, which is the same as KDA, but the procedure is done for each study. The resulting “contrast indicator maps” are either 1 (yes, there is a peak within r mm) or 0 (nope).
ALE
KDA
MKDA
![Page 51: Brain Maps like Mine](https://reader036.fdocuments.in/reader036/viewer/2022062903/58a389c71a28ab9c7e8b5b43/html5/thumbnails/51.jpg)
Pairwise Image
Comparison
![Page 52: Brain Maps like Mine](https://reader036.fdocuments.in/reader036/viewer/2022062903/58a389c71a28ab9c7e8b5b43/html5/thumbnails/52.jpg)
Animals paradigm
vsoch.github.io/experiment
![Page 53: Brain Maps like Mine](https://reader036.fdocuments.in/reader036/viewer/2022062903/58a389c71a28ab9c7e8b5b43/html5/thumbnails/53.jpg)
visual canine recognition
CONCEPTS
visual feline recognition
visual feline recognition
RELATIONSHIPS
is a kind of
animal recognition
visual canine recognition
is a kind of
GRAPH SIMILARITY
0.8
![Page 54: Brain Maps like Mine](https://reader036.fdocuments.in/reader036/viewer/2022062903/58a389c71a28ab9c7e8b5b43/html5/thumbnails/54.jpg)
Pearson Correlation(rho)