NIDM-Results. A standard for describing and sharing neuroimaging results: application to image-based...
-
Upload
camille-maumet -
Category
Science
-
view
164 -
download
2
Transcript of NIDM-Results. A standard for describing and sharing neuroimaging results: application to image-based...
![Page 1: NIDM-Results. A standard for describing and sharing neuroimaging results: application to image-based meta-analysis](https://reader034.fdocuments.in/reader034/viewer/2022052420/587e8ed61a28ab672b8b538d/html5/thumbnails/1.jpg)
A standard for describing and sharing neuroimaging resultsApplication to image-based meta-analysisSéminaire NeuroSpin - October 3rd, 2016
Camille MaumetNeuroimaging Statistics, University of Warwick, UK
![Page 2: NIDM-Results. A standard for describing and sharing neuroimaging results: application to image-based meta-analysis](https://reader034.fdocuments.in/reader034/viewer/2022052420/587e8ed61a28ab672b8b538d/html5/thumbnails/2.jpg)
Agenda
• Background– Reporting neuroimaging results– Meta-analysis
• The NeuroImaging Data Model• Example of image-based meta-analysis
![Page 3: NIDM-Results. A standard for describing and sharing neuroimaging results: application to image-based meta-analysis](https://reader034.fdocuments.in/reader034/viewer/2022052420/587e8ed61a28ab672b8b538d/html5/thumbnails/3.jpg)
Background
![Page 4: NIDM-Results. A standard for describing and sharing neuroimaging results: application to image-based meta-analysis](https://reader034.fdocuments.in/reader034/viewer/2022052420/587e8ed61a28ab672b8b538d/html5/thumbnails/4.jpg)
Reporting of neuroimaging results
4
Analysis
Pre-processing
Publication
Raw data
Pre-processed data
Results
Publication
![Page 5: NIDM-Results. A standard for describing and sharing neuroimaging results: application to image-based meta-analysis](https://reader034.fdocuments.in/reader034/viewer/2022052420/587e8ed61a28ab672b8b538d/html5/thumbnails/5.jpg)
Reporting of neuroimaging results
5
Publication
Peak locationsFigure
(selected slices)Thresholded
statistics
Analysis
Pre-processing
Publication
Raw data
Pre-processed data
Results
Publication
![Page 6: NIDM-Results. A standard for describing and sharing neuroimaging results: application to image-based meta-analysis](https://reader034.fdocuments.in/reader034/viewer/2022052420/587e8ed61a28ab672b8b538d/html5/thumbnails/6.jpg)
Reporting of neuroimaging results
6
Publication
Peak locationsFigure
(selected slices)Thresholded
statistics
Analysis
Pre-processing
Publication
Raw data
Pre-processed data
Results
Publication
❌ Data loss❌ Ambiguous/incomplete reporting❌ Metadata is not searchable
– Reporting guidelines - COBIDAS, Poldrack, other
– Guidelines: SAMPL: need the stats value
• Metadata that is not machine readable
– Brainmap / Neurosynth• Reproducibility
– If we want to understand better if effects reproduce we need careful description of the original analysis to be able to define what is causing the variation
– OHBM Replication award– BIDS
![Page 7: NIDM-Results. A standard for describing and sharing neuroimaging results: application to image-based meta-analysis](https://reader034.fdocuments.in/reader034/viewer/2022052420/587e8ed61a28ab672b8b538d/html5/thumbnails/7.jpg)
Reproducibility & provenance
ProvenanceW3C PROV
A reproducibility crisis?
Mauvaise pratiques (fishing)
Description incomplèteUnderpowered
studies
Publication bias
7
http://www.acmedsci.ac.uk/policy/policy-projects/reproducibility-and-reliability-of-biomedical-research/ https://www.w3.org/TR/prov-dm/
![Page 8: NIDM-Results. A standard for describing and sharing neuroimaging results: application to image-based meta-analysis](https://reader034.fdocuments.in/reader034/viewer/2022052420/587e8ed61a28ab672b8b538d/html5/thumbnails/8.jpg)
Data sharing
8
Analysis
Pre-processing
Publication
Raw data
Pre-processed data
Results
Publication
– Reporting guidelines - COBIDAS, Poldrack, other
– Guidelines: SAMPL: need the stats value
• Metadata that is not machine readable
– Brainmap / Neurosynth• Reproducibility
– If we want to understand better if effects reproduce we need careful description of the original analysis to be able to define what is causing the variation
– OHBM Replication award– BIDS
![Page 9: NIDM-Results. A standard for describing and sharing neuroimaging results: application to image-based meta-analysis](https://reader034.fdocuments.in/reader034/viewer/2022052420/587e8ed61a28ab672b8b538d/html5/thumbnails/9.jpg)
Publicly available
data
Data sharing
9
Analysis
Pre-processing
Publication
Raw data
Pre-processed data
Results
Publication
– Reporting guidelines - COBIDAS, Poldrack, other
– Guidelines: SAMPL: need the stats value
• Metadata that is not machine readable
– Brainmap / Neurosynth• Reproducibility
– If we want to understand better if effects reproduce we need careful description of the original analysis to be able to define what is causing the variation
– OHBM Replication award– BIDS
![Page 10: NIDM-Results. A standard for describing and sharing neuroimaging results: application to image-based meta-analysis](https://reader034.fdocuments.in/reader034/viewer/2022052420/587e8ed61a28ab672b8b538d/html5/thumbnails/10.jpg)
Publicly available
data
Data sharing
10
Analysis
Pre-processing
Publication
Raw data
Pre-processed data
Results
Publication
– Reporting guidelines - COBIDAS, Poldrack, other
– Guidelines: SAMPL: need the stats value
• Metadata that is not machine readable
– Brainmap / Neurosynth• Reproducibility
– If we want to understand better if effects reproduce we need careful description of the original analysis to be able to define what is causing the variation
– OHBM Replication award– BIDS
< 5% of data shared in public repositoriesB
UT
![Page 11: NIDM-Results. A standard for describing and sharing neuroimaging results: application to image-based meta-analysis](https://reader034.fdocuments.in/reader034/viewer/2022052420/587e8ed61a28ab672b8b538d/html5/thumbnails/11.jpg)
Data sharing vs results reporting
• Impediments– Ethical considerations– Privacy issues– Psychological barriers– Time consuming process
Results
• Results sharing is best practice
=> reporting rather than data sharing
11
![Page 12: NIDM-Results. A standard for describing and sharing neuroimaging results: application to image-based meta-analysis](https://reader034.fdocuments.in/reader034/viewer/2022052420/587e8ed61a28ab672b8b538d/html5/thumbnails/12.jpg)
Application to meta-analyses
![Page 13: NIDM-Results. A standard for describing and sharing neuroimaging results: application to image-based meta-analysis](https://reader034.fdocuments.in/reader034/viewer/2022052420/587e8ed61a28ab672b8b538d/html5/thumbnails/13.jpg)
Meta-analysis
Analysis
Pre-processing
Raw data
Pre-processed data
Results
Study 1
Analysis
Pre-processing
Raw data
Pre-processed data
Results
Study 2
Analysis
Pre-processing
Raw data
Pre-processed data
Results
Study n...
Meta-analysisResults
13
![Page 14: NIDM-Results. A standard for describing and sharing neuroimaging results: application to image-based meta-analysis](https://reader034.fdocuments.in/reader034/viewer/2022052420/587e8ed61a28ab672b8b538d/html5/thumbnails/14.jpg)
Neuroimaging meta-analysis
• Rich fMRI literature– > 30,000 articles (“fMRI” pubmed)
• Increase statistical power• Synthesize information across studies
14
![Page 15: NIDM-Results. A standard for describing and sharing neuroimaging results: application to image-based meta-analysis](https://reader034.fdocuments.in/reader034/viewer/2022052420/587e8ed61a28ab672b8b538d/html5/thumbnails/15.jpg)
Coordinate- or Image-Based?
15
Acquisition Analysis
Experiment Raw data Results
Acquisition Analysis
Experiment Raw data Results
…
Publication
Publication
Paper
Paper
![Page 16: NIDM-Results. A standard for describing and sharing neuroimaging results: application to image-based meta-analysis](https://reader034.fdocuments.in/reader034/viewer/2022052420/587e8ed61a28ab672b8b538d/html5/thumbnails/16.jpg)
Coordinate- or Image-Based?
16
Acquisition Analysis
Experiment Raw data Results
Acquisition Analysis
Experiment Raw data Results
…
Publication
Publication
Paper
Paper
Coordinate-based meta-analysis
Coordinate-based meta-analysis
![Page 17: NIDM-Results. A standard for describing and sharing neuroimaging results: application to image-based meta-analysis](https://reader034.fdocuments.in/reader034/viewer/2022052420/587e8ed61a28ab672b8b538d/html5/thumbnails/17.jpg)
Image-based meta-analysis
Shared results
Full results reporting
Coordinate- or Image-Based?
17
Acquisition Analysis
Experiment Raw data Results
Acquisition Analysis
Experiment Raw data Results
…
Publication
Publication
Paper
Paper
Coordinate-based meta-analysis
Coordinate-based meta-analysis Image-based meta-analysis
![Page 18: NIDM-Results. A standard for describing and sharing neuroimaging results: application to image-based meta-analysis](https://reader034.fdocuments.in/reader034/viewer/2022052420/587e8ed61a28ab672b8b538d/html5/thumbnails/18.jpg)
Image-based meta-analysis• Gold standard: Third-level Mixed-Effects GLM• Requirements
– study-level Contrast estimates and Standard error maps.
– Same units
Contrast and std. err. maps
18
![Page 19: NIDM-Results. A standard for describing and sharing neuroimaging results: application to image-based meta-analysis](https://reader034.fdocuments.in/reader034/viewer/2022052420/587e8ed61a28ab672b8b538d/html5/thumbnails/19.jpg)
Units depend on• Data scaling
• Scaling of the explanatory variables
• Scaling of the contrast
Units of contrast estimatesPre-processed
data
Data scalingScaled
pre-proc. data
Model parameter estimation Parameter
estimates
Contrast estimation Contrast and
std. err. maps
19
Y = β +
[ 1 1 1 1 ] vs. [ ¼ ¼ ¼ ¼ ]
![Page 20: NIDM-Results. A standard for describing and sharing neuroimaging results: application to image-based meta-analysis](https://reader034.fdocuments.in/reader034/viewer/2022052420/587e8ed61a28ab672b8b538d/html5/thumbnails/20.jpg)
Image-Based Meta-AnalysisOther approaches
20
Meta-Analysis Method Inputs Neuroimaging Implementation
‘Gold Standard’ MFX Con’s + SE’s FSL’s FEATSPM spm_mfxAFNI 3dMEMA
RFX GLMStouffer’s RFX
Con’sZ’s
FSL, SPM, AFNI, etc…
FFX GLMFisher’sStouffer’sStouffer’s Weighted
Con’s +SE’sZ’sZ’sZ’s + N’s
IBMA toolbox
OHBM 2016 "Neuroimaging Meta-Analysis" Educational Course, “Practical Intensity Based Meta-Analysis”: www.warwick.ac.uk/tenichols/presentations/ohbm2016
![Page 21: NIDM-Results. A standard for describing and sharing neuroimaging results: application to image-based meta-analysis](https://reader034.fdocuments.in/reader034/viewer/2022052420/587e8ed61a28ab672b8b538d/html5/thumbnails/21.jpg)
Required for Image-based meta-analysis• Images
– Contrast and standard error maps– Statistic maps
• Contrast vectors• Number of subjects per groups• Neuroimaging software used (data scaling)
21
![Page 22: NIDM-Results. A standard for describing and sharing neuroimaging results: application to image-based meta-analysis](https://reader034.fdocuments.in/reader034/viewer/2022052420/587e8ed61a28ab672b8b538d/html5/thumbnails/22.jpg)
The Neuroimaging Data Model
![Page 23: NIDM-Results. A standard for describing and sharing neuroimaging results: application to image-based meta-analysis](https://reader034.fdocuments.in/reader034/viewer/2022052420/587e8ed61a28ab672b8b538d/html5/thumbnails/23.jpg)
INCF Neuroimaging Task Force
• International collaboration– 13 labs, >12 tools
– Weekly teleconferences, focused workshops, GitHub
– Open
• Resources– http://nidm.nidash.org/ – http://github.com/incf-nidash/
23
![Page 24: NIDM-Results. A standard for describing and sharing neuroimaging results: application to image-based meta-analysis](https://reader034.fdocuments.in/reader034/viewer/2022052420/587e8ed61a28ab672b8b538d/html5/thumbnails/24.jpg)
NIDM: a set of specifications to describe neuroimaging data
24
![Page 25: NIDM-Results. A standard for describing and sharing neuroimaging results: application to image-based meta-analysis](https://reader034.fdocuments.in/reader034/viewer/2022052420/587e8ed61a28ab672b8b538d/html5/thumbnails/25.jpg)
NIDM: a set of specifications to describe neuroimaging data
25
![Page 26: NIDM-Results. A standard for describing and sharing neuroimaging results: application to image-based meta-analysis](https://reader034.fdocuments.in/reader034/viewer/2022052420/587e8ed61a28ab672b8b538d/html5/thumbnails/26.jpg)
NIDM-Results
• Metadata selected according to– Meta-analysis use-case– Best practices– Neuroimaging software
• Automatically retrieved
26
![Page 27: NIDM-Results. A standard for describing and sharing neuroimaging results: application to image-based meta-analysis](https://reader034.fdocuments.in/reader034/viewer/2022052420/587e8ed61a28ab672b8b538d/html5/thumbnails/27.jpg)
Model
• Semantic web
• PROV for provenance• STATO (statistical ontology) • NeuroLex/RRID (neuroscience lexicon)• Dublin Core, NEPOMUK file ontology, cryptographic
hash functions
27
![Page 28: NIDM-Results. A standard for describing and sharing neuroimaging results: application to image-based meta-analysis](https://reader034.fdocuments.in/reader034/viewer/2022052420/587e8ed61a28ab672b8b538d/html5/thumbnails/28.jpg)
NIDM-Results pack: Compressed file containing a NIDM-Results serialization and some or all of the referenced image data files.
NIDM-Results
Standar error map
Statistic mapE.g. Z-map
Contrast map Design matrix(png and csv)
NIDM-Resultsgraph
.nidm
.zip
28
![Page 30: NIDM-Results. A standard for describing and sharing neuroimaging results: application to image-based meta-analysis](https://reader034.fdocuments.in/reader034/viewer/2022052420/587e8ed61a28ab672b8b538d/html5/thumbnails/30.jpg)
30
Specification
http://nidm.nidash.org/specs/nidm-results.html
Definition
Attributes
Examples
![Page 31: NIDM-Results. A standard for describing and sharing neuroimaging results: application to image-based meta-analysis](https://reader034.fdocuments.in/reader034/viewer/2022052420/587e8ed61a28ab672b8b538d/html5/thumbnails/31.jpg)
Harmonisation across software
• Model of the error– Prob. distribution:– Variance:
– Dependence:
31
heterogeneous
Independent noise
homogeneous
Compound Symmetry
Serially correlated
Arbitrarily correlated
Gaussian Non-Parametric …
global
local
regularized
![Page 32: NIDM-Results. A standard for describing and sharing neuroimaging results: application to image-based meta-analysis](https://reader034.fdocuments.in/reader034/viewer/2022052420/587e8ed61a28ab672b8b538d/html5/thumbnails/32.jpg)
Error models: SPM, FSL and AFNI
32
Group levelSubject level
![Page 33: NIDM-Results. A standard for describing and sharing neuroimaging results: application to image-based meta-analysis](https://reader034.fdocuments.in/reader034/viewer/2022052420/587e8ed61a28ab672b8b538d/html5/thumbnails/33.jpg)
Error models: non-parametric
33
Group levelSign-flipping
Heterogeneous local
Independent noise
NonParametric Symmetric
Group levelLabel permutation
Homogeneous local
Exchangeable noise local
NonParametric
![Page 34: NIDM-Results. A standard for describing and sharing neuroimaging results: application to image-based meta-analysis](https://reader034.fdocuments.in/reader034/viewer/2022052420/587e8ed61a28ab672b8b538d/html5/thumbnails/34.jpg)
Example meta-analysis
![Page 35: NIDM-Results. A standard for describing and sharing neuroimaging results: application to image-based meta-analysis](https://reader034.fdocuments.in/reader034/viewer/2022052420/587e8ed61a28ab672b8b538d/html5/thumbnails/35.jpg)
SPM export
35www.fil.ion.ucl.ac.uk/spm/
• Available as part of SPM
![Page 36: NIDM-Results. A standard for describing and sharing neuroimaging results: application to image-based meta-analysis](https://reader034.fdocuments.in/reader034/viewer/2022052420/587e8ed61a28ab672b8b538d/html5/thumbnails/36.jpg)
FSL export
$nidmfsl ds107.gfeat -g Control 49
• Installable using pip• To be included in the next FSL release
36
https://github.com/incf-nidash/nidmresults-fslhttp://fsl.fmrib.ox.ac.uk
![Page 40: NIDM-Results. A standard for describing and sharing neuroimaging results: application to image-based meta-analysis](https://reader034.fdocuments.in/reader034/viewer/2022052420/587e8ed61a28ab672b8b538d/html5/thumbnails/40.jpg)
Example of meta-analysis
Coordinate-based meta-analysis Image-based meta-analysis
Scripts available at http://github.com/incf-nidash/nidmresults-paper40
![Page 41: NIDM-Results. A standard for describing and sharing neuroimaging results: application to image-based meta-analysis](https://reader034.fdocuments.in/reader034/viewer/2022052420/587e8ed61a28ab672b8b538d/html5/thumbnails/41.jpg)
Conclusions
![Page 42: NIDM-Results. A standard for describing and sharing neuroimaging results: application to image-based meta-analysis](https://reader034.fdocuments.in/reader034/viewer/2022052420/587e8ed61a28ab672b8b538d/html5/thumbnails/42.jpg)
Conclusion
NIDM-Results: a model for full reporting of mass-univariate results
Future work:
• Model extensions (e.g. permutation)• NIDM-Experiment (BIDS)• NIDM-Workflows• Ecosystem of tools
42
![Page 43: NIDM-Results. A standard for describing and sharing neuroimaging results: application to image-based meta-analysis](https://reader034.fdocuments.in/reader034/viewer/2022052420/587e8ed61a28ab672b8b538d/html5/thumbnails/43.jpg)
Acknowledgements
43
Thank you! To all the INCF NIDASH task force members.NIDM working group
Tibor Auer, Alexander Bowring, Gully Burns, Samir Das, Fariba Fana, Guillaume Flandin, Satra Ghosh, Tristan Glatard, Chris Gorgolewski, Karl Helmer, David Keator, Nolan Nichols, Tom Nichols, Jean-Baptiste Poline, Vanessa Sochat, Jason Steffener, Jessica Turner.
INCF NIDASH - Other members
This work is supported by the
Gang Chen, Richard Reynolds, Ziad Saad, Robert Cox (AFNI), Mark Jenkinson, Matthew Webster, Paul McCarthy, Eugene Duff, Steve Smith (FSL), Guillaume Flandin (SPM).
Neuroimaging software teams
Meta-analysis datasets Tracey group at FMRIB.
David Kennedy, Cameron Craddock, Yaroslav Halchenko, Michael Hanke, Christian Haselgrove, Arno Klein, Daniel Marcus, Russell Poldrack, Rich Stoner.
![Page 44: NIDM-Results. A standard for describing and sharing neuroimaging results: application to image-based meta-analysis](https://reader034.fdocuments.in/reader034/viewer/2022052420/587e8ed61a28ab672b8b538d/html5/thumbnails/44.jpg)
Q & A• Github: https://github.com/incf-nidash• Website: http://nidm.nidash.org • Manuscript: http://dx.doi.org/10.1101/041798
NIDM