Machine Learning for Medical Image Analysis
Transcript of Machine Learning for Medical Image Analysis
![Page 1: Machine Learning for Medical Image Analysis](https://reader030.fdocuments.in/reader030/viewer/2022040511/624a0b4c53ccf375d0381c5d/html5/thumbnails/1.jpg)
A. Criminisi
Machine Learning for Medical Image Analysis
![Page 2: Machine Learning for Medical Image Analysis](https://reader030.fdocuments.in/reader030/viewer/2022040511/624a0b4c53ccf375d0381c5d/html5/thumbnails/2.jpg)
Overview
• Introduction to machine learning
• Decision forests
• Applications in medical image analysis• Anatomy localization in CT Scans
• Spine Detection in CT Scans
• Brain Tumour Segmentation in MR Scans
![Page 3: Machine Learning for Medical Image Analysis](https://reader030.fdocuments.in/reader030/viewer/2022040511/624a0b4c53ccf375d0381c5d/html5/thumbnails/3.jpg)
Machine Learning
Lots of labelled data A predictor (e.g. a classifier)
Training algorithm
Previously unseen data Predicted labelPredictor
Training phase
Test phase
![Page 4: Machine Learning for Medical Image Analysis](https://reader030.fdocuments.in/reader030/viewer/2022040511/624a0b4c53ccf375d0381c5d/html5/thumbnails/4.jpg)
Training
algorithm
structure & parametersmeasurements (features)
&
associated ‘class’ labels
(colors used to show class labels)
Training data set Learned model
Training phase (usually offline)
Supervised Machine Learning (classification)
![Page 5: Machine Learning for Medical Image Analysis](https://reader030.fdocuments.in/reader030/viewer/2022040511/624a0b4c53ccf375d0381c5d/html5/thumbnails/5.jpg)
Supervised Machine Learning (classification)
structure + parameters predicted class label
Input test data point Learned model Output
measurements (features) only
Test phase (run time, online)
![Page 6: Machine Learning for Medical Image Analysis](https://reader030.fdocuments.in/reader030/viewer/2022040511/624a0b4c53ccf375d0381c5d/html5/thumbnails/6.jpg)
Data representation, feature vectors and data points
Features in 2D space Features in 3D space
Data point =
Feature vector
![Page 7: Machine Learning for Medical Image Analysis](https://reader030.fdocuments.in/reader030/viewer/2022040511/624a0b4c53ccf375d0381c5d/html5/thumbnails/7.jpg)
Data representation, feature vectors and data points
Features in 2D space
![Page 8: Machine Learning for Medical Image Analysis](https://reader030.fdocuments.in/reader030/viewer/2022040511/624a0b4c53ccf375d0381c5d/html5/thumbnails/8.jpg)
Application: Kinect body part recognition
Input test depth image Body part segmentation
image measurements
made relative to pixel
classifier per-pixel prediction
of class label
e.g. depth, color, neighbors
Task: assigning body part labels to each pixel in Kinect depth images
![Page 9: Machine Learning for Medical Image Analysis](https://reader030.fdocuments.in/reader030/viewer/2022040511/624a0b4c53ccf375d0381c5d/html5/thumbnails/9.jpg)
Overview
• Introduction to machine learning
• Decision forests
• Applications in medical image analysis• Anatomy localization in CT Scans
• Spine Detection in CT Scans
• Brain Tumour Segmentation in MR Scans
![Page 10: Machine Learning for Medical Image Analysis](https://reader030.fdocuments.in/reader030/viewer/2022040511/624a0b4c53ccf375d0381c5d/html5/thumbnails/10.jpg)
Decision trees
terminal (leaf) node
internal
(split) node
root node0
1 2
3 4 5 6
7 8 9 10 11 12 13 14
A general (binary) tree structure
Is top
part blue?
Is bottom
part green?Is bottom
part blue?
A decision tree
![Page 11: Machine Learning for Medical Image Analysis](https://reader030.fdocuments.in/reader030/viewer/2022040511/624a0b4c53ccf375d0381c5d/html5/thumbnails/11.jpg)
Decision forests
Forest prediction is an aggregate of the predictions across all trees (e.g. average probability)
![Page 12: Machine Learning for Medical Image Analysis](https://reader030.fdocuments.in/reader030/viewer/2022040511/624a0b4c53ccf375d0381c5d/html5/thumbnails/12.jpg)
Decision forests: key concepts
• Forest is an “ensemble” (collection) of trees
• The output of a forest aggregates the outputs of multiple trees• e.g. average
• Number of trees will depend on application• with lots of data you can get away with fewer, deeper trees (e.g. Kinect)
• less data probably requires more trees
![Page 13: Machine Learning for Medical Image Analysis](https://reader030.fdocuments.in/reader030/viewer/2022040511/624a0b4c53ccf375d0381c5d/html5/thumbnails/13.jpg)
Decision trees: test time predictiontest input data
prediction
![Page 14: Machine Learning for Medical Image Analysis](https://reader030.fdocuments.in/reader030/viewer/2022040511/624a0b4c53ccf375d0381c5d/html5/thumbnails/14.jpg)
D=
5D
=1
3
Parameters: T=400 predictor model = prob.
Weak learner: axis aligned Weak learner: oriented line Weak learner: conic section
Effect of tree depth and randomness
![Page 15: Machine Learning for Medical Image Analysis](https://reader030.fdocuments.in/reader030/viewer/2022040511/624a0b4c53ccf375d0381c5d/html5/thumbnails/15.jpg)
Overview
• Introduction to machine learning
• Decision forests
• Applications in medical image analysis• Anatomy localization in CT Scans
• Spine Detection in CT Scans
• Brain Tumour Segmentation in MR Scans
![Page 16: Machine Learning for Medical Image Analysis](https://reader030.fdocuments.in/reader030/viewer/2022040511/624a0b4c53ccf375d0381c5d/html5/thumbnails/16.jpg)
Anatomy Localization in 3D Computed Tomography Scans
- Direct mapping of voxels to organ bounding boxes.
- No search, no sliding window.
- No atlas registration.
Input CT scan Output anatomy localization
Key idea: each voxel votes (probabilistically) for the position of each organ’s bounding box.
![Page 17: Machine Learning for Medical Image Analysis](https://reader030.fdocuments.in/reader030/viewer/2022040511/624a0b4c53ccf375d0381c5d/html5/thumbnails/17.jpg)
sp
lee
nli
ve
r
ga
ll b
lad
de
rle
ft k
idn
ey
High variability in appearance, shape, location, resolution, noise, pathologies …
Organ labelling: why is it hard?
![Page 18: Machine Learning for Medical Image Analysis](https://reader030.fdocuments.in/reader030/viewer/2022040511/624a0b4c53ccf375d0381c5d/html5/thumbnails/18.jpg)
Different image cropping, noise, contrast/no-contrast, resolution, scanners, body shapes/sizes, patient position…
Organ labelling: the ground-truth database
![Page 19: Machine Learning for Medical Image Analysis](https://reader030.fdocuments.in/reader030/viewer/2022040511/624a0b4c53ccf375d0381c5d/html5/thumbnails/19.jpg)
Node split function
Node optimization
Node training
Input data point
Output
Error in model fit
Feature response
• Each voxel in the volume votes for the position of the 6 box sides
• We wish to learn a set of discriminative points (landmarks, clusters)
which can predict the kidney position with high confidence.
(voxel position in volume)
(bound. box continuous pos.)
(mean over displaced 3D boxes)
(weighted uncertainty for all organs)
(relative displacement)
(Gaussian repres. of distribs)
Regressing an n-D piece-wise constant model
Multiple organs
Organ labelling: regression forest
![Page 20: Machine Learning for Medical Image Analysis](https://reader030.fdocuments.in/reader030/viewer/2022040511/624a0b4c53ccf375d0381c5d/html5/thumbnails/20.jpg)
Organ labelling: context-rich visual featuresPossible visual features Computing the feature response
…
Capturing spatial context
![Page 21: Machine Learning for Medical Image Analysis](https://reader030.fdocuments.in/reader030/viewer/2022040511/624a0b4c53ccf375d0381c5d/html5/thumbnails/21.jpg)
Organ labelling: automatic landmark discovery
Input CT scan and detected landmark regions
Here the system is trained to
detect left and right kidneys.
The system learns to use bottom
of lung and top of pelvis to localize
kidneys with highest confidence.
Discovery of landmark regions
![Page 22: Machine Learning for Medical Image Analysis](https://reader030.fdocuments.in/reader030/viewer/2022040511/624a0b4c53ccf375d0381c5d/html5/thumbnails/22.jpg)
Overview
• Introduction to machine learning
• Decision forests
• Applications in medical image analysis• Anatomy localization in CT Scans
• Spine Detection in CT Scans
• Brain Tumour Segmentation in MR Scans
![Page 23: Machine Learning for Medical Image Analysis](https://reader030.fdocuments.in/reader030/viewer/2022040511/624a0b4c53ccf375d0381c5d/html5/thumbnails/23.jpg)
Vertebrae Detection and Classification
![Page 24: Machine Learning for Medical Image Analysis](https://reader030.fdocuments.in/reader030/viewer/2022040511/624a0b4c53ccf375d0381c5d/html5/thumbnails/24.jpg)
Name of this
vertebra?
Where? Which?
![Page 25: Machine Learning for Medical Image Analysis](https://reader030.fdocuments.in/reader030/viewer/2022040511/624a0b4c53ccf375d0381c5d/html5/thumbnails/25.jpg)
Challenges
![Page 26: Machine Learning for Medical Image Analysis](https://reader030.fdocuments.in/reader030/viewer/2022040511/624a0b4c53ccf375d0381c5d/html5/thumbnails/26.jpg)
• Repetitive nature of structures
• Variability of normal anatomy
• Presence of pathologies
• Varying image acquisition(FOV, noise level, resolution, …)
Challenges
![Page 27: Machine Learning for Medical Image Analysis](https://reader030.fdocuments.in/reader030/viewer/2022040511/624a0b4c53ccf375d0381c5d/html5/thumbnails/27.jpg)
Patient-specific coordinate system
• Guided visualization/navigationin diagnostic tools
• Longitudinal assessmentafter surgical Intervention
• Shape/population analysis fordisease modelling
Impact on Clinical Routine!
Impact on Clinical Research!
Clinical motivation
![Page 28: Machine Learning for Medical Image Analysis](https://reader030.fdocuments.in/reader030/viewer/2022040511/624a0b4c53ccf375d0381c5d/html5/thumbnails/28.jpg)
Some results
![Page 29: Machine Learning for Medical Image Analysis](https://reader030.fdocuments.in/reader030/viewer/2022040511/624a0b4c53ccf375d0381c5d/html5/thumbnails/29.jpg)
Some results
![Page 30: Machine Learning for Medical Image Analysis](https://reader030.fdocuments.in/reader030/viewer/2022040511/624a0b4c53ccf375d0381c5d/html5/thumbnails/30.jpg)
Some results
![Page 31: Machine Learning for Medical Image Analysis](https://reader030.fdocuments.in/reader030/viewer/2022040511/624a0b4c53ccf375d0381c5d/html5/thumbnails/31.jpg)
Overview
• Introduction to machine learning
• Decision forests
• Applications in medical image analysis• Anatomy localization in CT Scans
• Spine Detection in CT Scans
• Brain Tumour Segmentation in MR Scans
![Page 32: Machine Learning for Medical Image Analysis](https://reader030.fdocuments.in/reader030/viewer/2022040511/624a0b4c53ccf375d0381c5d/html5/thumbnails/32.jpg)
Segmentation of
tumorous tissues:
---- Active cells
---- Necrotic core
---- Edema
---- Background
3D MRI input data
T1-gad T1
T2
DTI-p
FLAIR
DTI-q
Automatic Segmentation of Brain Tumour
![Page 33: Machine Learning for Medical Image Analysis](https://reader030.fdocuments.in/reader030/viewer/2022040511/624a0b4c53ccf375d0381c5d/html5/thumbnails/33.jpg)
Tumour
Tissue
Classification
Training a Pixel-Wise Forest Classifier
![Page 34: Machine Learning for Medical Image Analysis](https://reader030.fdocuments.in/reader030/viewer/2022040511/624a0b4c53ccf375d0381c5d/html5/thumbnails/34.jpg)
New Patient,
previously unseen
Tumour
Tissue
Classification
Testing the Pixel-Wise Forest Classifier
![Page 35: Machine Learning for Medical Image Analysis](https://reader030.fdocuments.in/reader030/viewer/2022040511/624a0b4c53ccf375d0381c5d/html5/thumbnails/35.jpg)
1st Step: Obtain Expert Segmentation
Building the Training Database of Patients’ Images
![Page 36: Machine Learning for Medical Image Analysis](https://reader030.fdocuments.in/reader030/viewer/2022040511/624a0b4c53ccf375d0381c5d/html5/thumbnails/36.jpg)
Building the Training Database of Patients’ Images1st Step: Obtain Expert Segmentation
![Page 37: Machine Learning for Medical Image Analysis](https://reader030.fdocuments.in/reader030/viewer/2022040511/624a0b4c53ccf375d0381c5d/html5/thumbnails/37.jpg)
1st Step: Obtain Expert Segmentation
Building the Training Database of Patients’ Images
![Page 38: Machine Learning for Medical Image Analysis](https://reader030.fdocuments.in/reader030/viewer/2022040511/624a0b4c53ccf375d0381c5d/html5/thumbnails/38.jpg)
Glioblastoma Segmentation
![Page 39: Machine Learning for Medical Image Analysis](https://reader030.fdocuments.in/reader030/viewer/2022040511/624a0b4c53ccf375d0381c5d/html5/thumbnails/39.jpg)
Glioblastoma Segmentation
![Page 40: Machine Learning for Medical Image Analysis](https://reader030.fdocuments.in/reader030/viewer/2022040511/624a0b4c53ccf375d0381c5d/html5/thumbnails/40.jpg)
Machine learning can have a huge impact on medicine!