TBA #23 GE Corporate R&D Niskayuna, NY [email protected].
-
Upload
jordan-osborne -
Category
Documents
-
view
220 -
download
1
Transcript of TBA #23 GE Corporate R&D Niskayuna, NY [email protected].
![Page 2: TBA #23 GE Corporate R&D Niskayuna, NY lorensen@crd.ge.com.](https://reader034.fdocuments.in/reader034/viewer/2022051401/56649e145503460f94afe53a/html5/thumbnails/2.jpg)
Unification of Vision, Geometry and Graphics
Through Toolkits
Bill Lorensen
GE Corporate R&D
Niskayuna, NY
![Page 3: TBA #23 GE Corporate R&D Niskayuna, NY lorensen@crd.ge.com.](https://reader034.fdocuments.in/reader034/viewer/2022051401/56649e145503460f94afe53a/html5/thumbnails/3.jpg)
What is a Toolkit?
Mathematics+
Algorithms+
Software
Edelsbrunner, 2001
![Page 4: TBA #23 GE Corporate R&D Niskayuna, NY lorensen@crd.ge.com.](https://reader034.fdocuments.in/reader034/viewer/2022051401/56649e145503460f94afe53a/html5/thumbnails/4.jpg)
Dual Interests
![Page 5: TBA #23 GE Corporate R&D Niskayuna, NY lorensen@crd.ge.com.](https://reader034.fdocuments.in/reader034/viewer/2022051401/56649e145503460f94afe53a/html5/thumbnails/5.jpg)
Marching Cubes 1984
![Page 6: TBA #23 GE Corporate R&D Niskayuna, NY lorensen@crd.ge.com.](https://reader034.fdocuments.in/reader034/viewer/2022051401/56649e145503460f94afe53a/html5/thumbnails/6.jpg)
Baseball Visualization 1989
![Page 7: TBA #23 GE Corporate R&D Niskayuna, NY lorensen@crd.ge.com.](https://reader034.fdocuments.in/reader034/viewer/2022051401/56649e145503460f94afe53a/html5/thumbnails/7.jpg)
Stream Polygons - 1991
![Page 8: TBA #23 GE Corporate R&D Niskayuna, NY lorensen@crd.ge.com.](https://reader034.fdocuments.in/reader034/viewer/2022051401/56649e145503460f94afe53a/html5/thumbnails/8.jpg)
Triangle Decimation - 1992
![Page 9: TBA #23 GE Corporate R&D Niskayuna, NY lorensen@crd.ge.com.](https://reader034.fdocuments.in/reader034/viewer/2022051401/56649e145503460f94afe53a/html5/thumbnails/9.jpg)
IEEE CG&A 1992
![Page 10: TBA #23 GE Corporate R&D Niskayuna, NY lorensen@crd.ge.com.](https://reader034.fdocuments.in/reader034/viewer/2022051401/56649e145503460f94afe53a/html5/thumbnails/10.jpg)
Swept Surfaces 1993
Removal Removal PathPath
Swept Swept SurfaceSurface
![Page 11: TBA #23 GE Corporate R&D Niskayuna, NY lorensen@crd.ge.com.](https://reader034.fdocuments.in/reader034/viewer/2022051401/56649e145503460f94afe53a/html5/thumbnails/11.jpg)
Virtual Endoscopy 1994
![Page 12: TBA #23 GE Corporate R&D Niskayuna, NY lorensen@crd.ge.com.](https://reader034.fdocuments.in/reader034/viewer/2022051401/56649e145503460f94afe53a/html5/thumbnails/12.jpg)
Creating Models from Segmented Medical Data
![Page 13: TBA #23 GE Corporate R&D Niskayuna, NY lorensen@crd.ge.com.](https://reader034.fdocuments.in/reader034/viewer/2022051401/56649e145503460f94afe53a/html5/thumbnails/13.jpg)
Surface and Volume Rendering
![Page 14: TBA #23 GE Corporate R&D Niskayuna, NY lorensen@crd.ge.com.](https://reader034.fdocuments.in/reader034/viewer/2022051401/56649e145503460f94afe53a/html5/thumbnails/14.jpg)
Hypothesis
Many real world problems cannot be solved by a single discipline
![Page 15: TBA #23 GE Corporate R&D Niskayuna, NY lorensen@crd.ge.com.](https://reader034.fdocuments.in/reader034/viewer/2022051401/56649e145503460f94afe53a/html5/thumbnails/15.jpg)
Core Technologies for 3D Medical Image Analysis
• Registration– Intra-modality (MRI to MRI, CT to CT)– Inter-modality (MRI to PET)– Model to Modality (Atlas to MRI)– Metadata to Modality (Clinical data,
biochip to MRI/CT)
• Filters– Edge preserving– Noise reduction– Non uniform intensity correction
• Segmentation– Edge detection– Region growing– Multi-channel
• Pattern Recognition– Tissue classification
• Visualization– Surface / volume rendering
– Fusion
• Quantification– Area, volume, shape
• Change detection– Longitudinal tracking
– Signal variation
• Information Analysis/Visualization
![Page 16: TBA #23 GE Corporate R&D Niskayuna, NY lorensen@crd.ge.com.](https://reader034.fdocuments.in/reader034/viewer/2022051401/56649e145503460f94afe53a/html5/thumbnails/16.jpg)
Discipline-specific Toolkits
• Use “best of breed” algorithms implemented by domain experts– Point matching– Voronoi diagram computation– Registration– Pose estimation– Isosurface extraction– Mathematical morphology– Skeletonization– Subdivision surfaces– Similarity measures– Surface simplification– Geometric compression
![Page 17: TBA #23 GE Corporate R&D Niskayuna, NY lorensen@crd.ge.com.](https://reader034.fdocuments.in/reader034/viewer/2022051401/56649e145503460f94afe53a/html5/thumbnails/17.jpg)
Discipline-specific Toolkits
• Examples– vtk, The Visualization Toolkit– Open Inventor, Graphics– Insight, Segmentation and Registration– CGAL, Computational Geometry– vxl, Image Understanding– Khoros, Image Processing
![Page 18: TBA #23 GE Corporate R&D Niskayuna, NY lorensen@crd.ge.com.](https://reader034.fdocuments.in/reader034/viewer/2022051401/56649e145503460f94afe53a/html5/thumbnails/18.jpg)
vtk, The Visualization Toolkit
• Open source toolkit for scientific visualization, computer graphics, and image processing
• C++ Class Library• 250,000 Lines of Code
– (~120,000 executable)• 20+ developers• 8 years of development• 1000 user mailing list
public.kitware.com/VTK
![Page 19: TBA #23 GE Corporate R&D Niskayuna, NY lorensen@crd.ge.com.](https://reader034.fdocuments.in/reader034/viewer/2022051401/56649e145503460f94afe53a/html5/thumbnails/19.jpg)
Insight Segmentation and Registration Toolkit
![Page 20: TBA #23 GE Corporate R&D Niskayuna, NY lorensen@crd.ge.com.](https://reader034.fdocuments.in/reader034/viewer/2022051401/56649e145503460f94afe53a/html5/thumbnails/20.jpg)
What is it?
• A common Application Programmers Interface (API).– A framework for software development– A toolkit for registration and segmentation– An Open Source resource for future research
• A validation model for segmentation and registration.– A framework for validation development– Assistance for algorithm designers– A seed repository for validated segmentations
![Page 21: TBA #23 GE Corporate R&D Niskayuna, NY lorensen@crd.ge.com.](https://reader034.fdocuments.in/reader034/viewer/2022051401/56649e145503460f94afe53a/html5/thumbnails/21.jpg)
Who’s sponsoring it?
The NationalScienceFoundation
The NationalInstitute for Dental and Craniofacial Research
The National Institute of Neurological Disorders and Stroke
$7.5 million, 3 year contract
![Page 22: TBA #23 GE Corporate R&D Niskayuna, NY lorensen@crd.ge.com.](https://reader034.fdocuments.in/reader034/viewer/2022051401/56649e145503460f94afe53a/html5/thumbnails/22.jpg)
Who’s creating it?
![Page 23: TBA #23 GE Corporate R&D Niskayuna, NY lorensen@crd.ge.com.](https://reader034.fdocuments.in/reader034/viewer/2022051401/56649e145503460f94afe53a/html5/thumbnails/23.jpg)
Contractor Roles
• GE CRD/Brigham and Womens– Architecture, algorithms, testing, validation
• Kitware– Architecture, user community support
• Insightful (formerly MathSoft)/UPenn– Statistical segmentation, mutual information registration, deformable
registration, level sets– Beta test management
• Utah– Level sets, low level image processing
• UNC/Pitt– Image processing, registration, high-dimensional segmentation
• UPenn/Columbia– Deformable surfaces, fuzzy connectedness, hybrid methods
![Page 24: TBA #23 GE Corporate R&D Niskayuna, NY lorensen@crd.ge.com.](https://reader034.fdocuments.in/reader034/viewer/2022051401/56649e145503460f94afe53a/html5/thumbnails/24.jpg)
Toolkit Requirements
• Shall handle large datasets– Visible Human data on a 512MB PC
• Shall run on multiple platforms– Sun, SGI, Linux, Windows
• Shall provide multiple language api’s• Shall support parallel processing• Shall have no visualization system
dependencies• Shall support multi-dimensional images• Shall support n-component data
![Page 25: TBA #23 GE Corporate R&D Niskayuna, NY lorensen@crd.ge.com.](https://reader034.fdocuments.in/reader034/viewer/2022051401/56649e145503460f94afe53a/html5/thumbnails/25.jpg)
Insight - Schedule
• Alpha Release, April 4, 2001.– Source code snapshot– Some non-consortium participation
• Limited Public Alpha Version, Aug 8, 2001.• Public Beta Release, December 15, 2001.• Software Developer’s Consortium Meeting
– Nov. 8-9, 2001, NLM, Bethesda.
www.itk.org
![Page 26: TBA #23 GE Corporate R&D Niskayuna, NY lorensen@crd.ge.com.](https://reader034.fdocuments.in/reader034/viewer/2022051401/56649e145503460f94afe53a/html5/thumbnails/26.jpg)
Testing Design
• Distributed testing– Developers and users must be able to easily
contribute testing results– Pulled together in a central dashboard
• Separate data from presentation• Cross-platform solution• Strive to have the same code tested in all
locations
![Page 27: TBA #23 GE Corporate R&D Niskayuna, NY lorensen@crd.ge.com.](https://reader034.fdocuments.in/reader034/viewer/2022051401/56649e145503460f94afe53a/html5/thumbnails/27.jpg)
![Page 28: TBA #23 GE Corporate R&D Niskayuna, NY lorensen@crd.ge.com.](https://reader034.fdocuments.in/reader034/viewer/2022051401/56649e145503460f94afe53a/html5/thumbnails/28.jpg)
![Page 29: TBA #23 GE Corporate R&D Niskayuna, NY lorensen@crd.ge.com.](https://reader034.fdocuments.in/reader034/viewer/2022051401/56649e145503460f94afe53a/html5/thumbnails/29.jpg)
Using vtk and Insight
Registration of Volumetric Medical Data
![Page 30: TBA #23 GE Corporate R&D Niskayuna, NY lorensen@crd.ge.com.](https://reader034.fdocuments.in/reader034/viewer/2022051401/56649e145503460f94afe53a/html5/thumbnails/30.jpg)
Mutual Information
• Computes “mutual information” between two datasets, a reference and target– MI(X,Y) = H(X) + H(Y) – H(X,Y)
• Small parameter set• Developed by Sandy Wells (BWH) and Paul Viola
(MIT) in 1995• Defacto standard for automatic, intensity based
registration
![Page 31: TBA #23 GE Corporate R&D Niskayuna, NY lorensen@crd.ge.com.](https://reader034.fdocuments.in/reader034/viewer/2022051401/56649e145503460f94afe53a/html5/thumbnails/31.jpg)
Insight Mutual Information Registration
• There is no MI open source implementation• The Insight Registration and Segmentation
Toolkit has an implementation• GE and Brigham as Insight contractors have
early access to the code• Code was developed at MathSoft (now called
Insightful)• GE was able to “guide” development with
input from Sandy Wells
![Page 32: TBA #23 GE Corporate R&D Niskayuna, NY lorensen@crd.ge.com.](https://reader034.fdocuments.in/reader034/viewer/2022051401/56649e145503460f94afe53a/html5/thumbnails/32.jpg)
Longitudinal MRI Study
• Register multiple volumetric MRI datasets of a patient taken over an extended time
• Create a batch processing facility to process dozens of datasets
• Resample the datasets
![Page 33: TBA #23 GE Corporate R&D Niskayuna, NY lorensen@crd.ge.com.](https://reader034.fdocuments.in/reader034/viewer/2022051401/56649e145503460f94afe53a/html5/thumbnails/33.jpg)
Approach
• Validate the algorithm• Pick a set of parameters that can be used
across all the studies• For each pair of datasets
– Perform registration– Output a transform
• View the resampled source dataset in context with the target dataset
![Page 34: TBA #23 GE Corporate R&D Niskayuna, NY lorensen@crd.ge.com.](https://reader034.fdocuments.in/reader034/viewer/2022051401/56649e145503460f94afe53a/html5/thumbnails/34.jpg)
Division of Labor
vtk
itk
vtk
Read data
Normalize data
Export data
Import Data
Register
Report transform
Read data
Reslice
Display
MRIRegistration.cxx
MultiCompare.tcl
![Page 35: TBA #23 GE Corporate R&D Niskayuna, NY lorensen@crd.ge.com.](https://reader034.fdocuments.in/reader034/viewer/2022051401/56649e145503460f94afe53a/html5/thumbnails/35.jpg)
The Pipeline
ImageReader ImageCast ImageShiftScale
ImageStatistics
ImageShrink3D
ImageExportImportImage
ImageToImageRigidMutualInformationGradientDescentRegistration
vnl_quaternion Matrix4x4
![Page 36: TBA #23 GE Corporate R&D Niskayuna, NY lorensen@crd.ge.com.](https://reader034.fdocuments.in/reader034/viewer/2022051401/56649e145503460f94afe53a/html5/thumbnails/36.jpg)
Oregon Data
• 25 Registrations• 13 Subjects• Qualitative comparison• One set of parameters for all studies
![Page 37: TBA #23 GE Corporate R&D Niskayuna, NY lorensen@crd.ge.com.](https://reader034.fdocuments.in/reader034/viewer/2022051401/56649e145503460f94afe53a/html5/thumbnails/37.jpg)
Longitudinal MRI No Registration
Checkerboard
SourceOriginalimage
Difference
TargetOriginal image
![Page 38: TBA #23 GE Corporate R&D Niskayuna, NY lorensen@crd.ge.com.](https://reader034.fdocuments.in/reader034/viewer/2022051401/56649e145503460f94afe53a/html5/thumbnails/38.jpg)
Longitudinal MRI Registration
Checkerboard
SourceOriginalimage
Difference
TargetOriginal image
![Page 39: TBA #23 GE Corporate R&D Niskayuna, NY lorensen@crd.ge.com.](https://reader034.fdocuments.in/reader034/viewer/2022051401/56649e145503460f94afe53a/html5/thumbnails/39.jpg)
Multi Field MRI Data
• Register 1.5T and 3T to 4T data• Resampled 1.5T and 3T to correspond to the
4T sampling• Volume rendering of the 3 datasets from the
same view
![Page 40: TBA #23 GE Corporate R&D Niskayuna, NY lorensen@crd.ge.com.](https://reader034.fdocuments.in/reader034/viewer/2022051401/56649e145503460f94afe53a/html5/thumbnails/40.jpg)
1.5T vs 4T MRI No Registration
Checkerboard
SourceOriginalImage
Difference
TargetOriginal Image
![Page 41: TBA #23 GE Corporate R&D Niskayuna, NY lorensen@crd.ge.com.](https://reader034.fdocuments.in/reader034/viewer/2022051401/56649e145503460f94afe53a/html5/thumbnails/41.jpg)
1.5T vs 4T MRI Registration
Checkerboard
SourceOriginalImage
Difference
TargetOriginal Image
![Page 42: TBA #23 GE Corporate R&D Niskayuna, NY lorensen@crd.ge.com.](https://reader034.fdocuments.in/reader034/viewer/2022051401/56649e145503460f94afe53a/html5/thumbnails/42.jpg)
3D Visualization of the same subjectScanned with different MR field Strengths
4T
3T 1.5T
All Registered To 4T
![Page 43: TBA #23 GE Corporate R&D Niskayuna, NY lorensen@crd.ge.com.](https://reader034.fdocuments.in/reader034/viewer/2022051401/56649e145503460f94afe53a/html5/thumbnails/43.jpg)
CT Lung Longitudinal Study
• Register two CT exams of the same patient taken at two different times
• Side-by-side synchronized view for visual comparison
![Page 44: TBA #23 GE Corporate R&D Niskayuna, NY lorensen@crd.ge.com.](https://reader034.fdocuments.in/reader034/viewer/2022051401/56649e145503460f94afe53a/html5/thumbnails/44.jpg)
Lung CT No Registration
Checkerboard
SourceOriginalImage
Difference
TargetOriginal Image
![Page 45: TBA #23 GE Corporate R&D Niskayuna, NY lorensen@crd.ge.com.](https://reader034.fdocuments.in/reader034/viewer/2022051401/56649e145503460f94afe53a/html5/thumbnails/45.jpg)
Lung CT Registration
Checkerboard
SourceOriginalImage
Difference
TargetOriginal Image
![Page 46: TBA #23 GE Corporate R&D Niskayuna, NY lorensen@crd.ge.com.](https://reader034.fdocuments.in/reader034/viewer/2022051401/56649e145503460f94afe53a/html5/thumbnails/46.jpg)
microPet/Volume CT
![Page 47: TBA #23 GE Corporate R&D Niskayuna, NY lorensen@crd.ge.com.](https://reader034.fdocuments.in/reader034/viewer/2022051401/56649e145503460f94afe53a/html5/thumbnails/47.jpg)
Back to the Software
![Page 48: TBA #23 GE Corporate R&D Niskayuna, NY lorensen@crd.ge.com.](https://reader034.fdocuments.in/reader034/viewer/2022051401/56649e145503460f94afe53a/html5/thumbnails/48.jpg)
Why Now?
• Internet enables distributed software development
• There are some successful Open Source projects
• A basic set of algorithms (and sometimes mathematics) exist
• Light weight software engineering processes exist– Low investment to support software development– Minimally invasive
![Page 49: TBA #23 GE Corporate R&D Niskayuna, NY lorensen@crd.ge.com.](https://reader034.fdocuments.in/reader034/viewer/2022051401/56649e145503460f94afe53a/html5/thumbnails/49.jpg)
Software Trends
Lightweight Software Engineering Processes
![Page 50: TBA #23 GE Corporate R&D Niskayuna, NY lorensen@crd.ge.com.](https://reader034.fdocuments.in/reader034/viewer/2022051401/56649e145503460f94afe53a/html5/thumbnails/50.jpg)
IEEE Computer October, 1999
![Page 51: TBA #23 GE Corporate R&D Niskayuna, NY lorensen@crd.ge.com.](https://reader034.fdocuments.in/reader034/viewer/2022051401/56649e145503460f94afe53a/html5/thumbnails/51.jpg)
Extreme Programming
![Page 52: TBA #23 GE Corporate R&D Niskayuna, NY lorensen@crd.ge.com.](https://reader034.fdocuments.in/reader034/viewer/2022051401/56649e145503460f94afe53a/html5/thumbnails/52.jpg)
Extreme Testing
![Page 53: TBA #23 GE Corporate R&D Niskayuna, NY lorensen@crd.ge.com.](https://reader034.fdocuments.in/reader034/viewer/2022051401/56649e145503460f94afe53a/html5/thumbnails/53.jpg)
Continuous Testing
![Page 54: TBA #23 GE Corporate R&D Niskayuna, NY lorensen@crd.ge.com.](https://reader034.fdocuments.in/reader034/viewer/2022051401/56649e145503460f94afe53a/html5/thumbnails/54.jpg)
![Page 55: TBA #23 GE Corporate R&D Niskayuna, NY lorensen@crd.ge.com.](https://reader034.fdocuments.in/reader034/viewer/2022051401/56649e145503460f94afe53a/html5/thumbnails/55.jpg)
Insight Project Management
• Robust code repository (cvs)• Active mailing list (mailman)• Automated documentation (doxygen)• Stable, cross platform build environment (cmake)• Weekly t-cons• Stable nightly build and test (300 builds)• Continuous build• Stable nightly dashboard (dart)• Quarterly face-to-face developer meetings• Semi-annual project meetings
![Page 56: TBA #23 GE Corporate R&D Niskayuna, NY lorensen@crd.ge.com.](https://reader034.fdocuments.in/reader034/viewer/2022051401/56649e145503460f94afe53a/html5/thumbnails/56.jpg)
Recipe for Success
• Vision
• Openness
• Community
• Strong core team
• Core Architecture
• Funding
![Page 57: TBA #23 GE Corporate R&D Niskayuna, NY lorensen@crd.ge.com.](https://reader034.fdocuments.in/reader034/viewer/2022051401/56649e145503460f94afe53a/html5/thumbnails/57.jpg)
Unification of Vision, Geometry and Graphics
Through Toolkits
Bill Lorensen
GE Corporate R&D
Niskayuna, NY