3-D Visualization of 3-D Visualization of Functional Brain Map DataFunctional Brain Map Data
A.V. Poliakov; E.B. Moore; J.F. Brinkley, A.V. Poliakov; E.B. Moore; J.F. Brinkley, Structural Informatics GroupStructural Informatics Group
Department of Biological StructureDepartment of Biological Structure
University of WashingtonUniversity of Washington
UW Human Brain ProjectUW Human Brain ProjectStructural Informatics Group, Dr. BrinkleyStructural Informatics Group, Dr. Brinkley
CollaboratorsCollaboratorsNeurosurgery, Dr. OjemannNeurosurgery, Dr. OjemannPsychiatry, Dr. CorinaPsychiatry, Dr. CorinaRadiology, Dr. MaravillaRadiology, Dr. Maravilla
Medical ProcedureMedical ProcedureNeurosurgery for intractable epilepsy Neurosurgery for intractable epilepsy
Objective Objective Remove epileptic foci while sparing the patient’s language functionRemove epileptic foci while sparing the patient’s language function
MethodMethodLanguage mapping using electrical stimulationLanguage mapping using electrical stimulation
Information SystemInformation System Help researches organize and analyze the dataHelp researches organize and analyze the dataUnique dataUnique data
Study languageStudy languageEvaluate non-invasive imaging techniques for surgical planningEvaluate non-invasive imaging techniques for surgical planning
Data acquired during surgery Data acquired during surgery Electrical stimulation map Electrical stimulation map PhonogramsPhonograms
Data acquired prior to surgeryData acquired prior to surgery MR scans ofMR scans of cortical anatomy, veins, arteries cortical anatomy, veins, arteries Functional MRIFunctional MRI
Information System for Language Information System for Language Mapping Mapping Primary Sources of DataPrimary Sources of Data
Data acquired during surgery Data acquired during surgery Electrical stimulation map Electrical stimulation map PhonogramsPhonograms Single unit recordingSingle unit recording
Data acquired prior to surgeryData acquired prior to surgery MR scans ofMR scans of cortical anatomy, veins, arteries cortical anatomy, veins, arteries Functional MRIFunctional MRI EEG in grid patientsEEG in grid patients
Information System for Language Information System for Language Mapping Mapping Primary Sources of DataPrimary Sources of Data
Brain MapperBrain Mapper
• Alignment • Masking • Reconstructing 3D models• Mapping CSM Sites
Information System for Language Information System for Language Mapping Mapping
Information System for Language Information System for Language Mapping Mapping
In developmentIn development
fMRI DB and Automated Data fMRI DB and Automated Data Analysis Analysis
Single Unit DBSingle Unit DB
Data ManagementData ManagementTalairach Daemon QueryTalairach Daemon Query
Query the Brain Mapping Repository using XQueryQuery the Brain Mapping Repository using XQuery
Information System for Language Information System for Language Mapping Mapping
SPM
Web InterfaceWeb Interface
Web InterfaceWeb Interface
Web InterfaceWeb Interface
Web InterfaceWeb Interface
Web InterfaceWeb Interface
Web InterfaceWeb Interface
Web InterfaceWeb Interface
Web InterfaceWeb Interface
Web InterfaceWeb Interface
3D Visualization3D VisualizationSystem ArchitectureSystem Architecture
– Client-serverClient-server– Server-based rendering and data storage Server-based rendering and data storage – Web-based client (e.g. Java or forms-based)Web-based client (e.g. Java or forms-based)
Web Interface
Server
Client #1CGI script
Client #2Java Applet
Client #3. . .
Internet
Graphics Server
snapshot snapshot
Stimulation Sites
3-DModels
3-D Image Volumes
Client
Data
Experiment Management System
Relational Database
2-D Images
Brain Mapper
3D Visualization3D VisualizationRequestsRequestsNew UsersNew Users
Different data modalitiesDifferent data modalities
Various software packages Various software packages
SpectroscopySpectroscopyERP Source SignalERP Source SignalfMRIfMRI
Transcranial Magnetic StimulationTranscranial Magnetic Stimulation
Collaborators Collaborators New features New features
Remote collaborationRemote collaboration
3D Visualization3D Visualization
Limitations of current systemLimitations of current systemRelatively slow software renderingRelatively slow software rendering
Non-portable architecture Non-portable architecture SGI Irix and Linux onlySGI Irix and Linux only
Large code base Large code base Needs to be maintained and developedNeeds to be maintained and developed
Reliance on outdated software toolsReliance on outdated software tools
3D Visualization3D VisualizationOther possible approachesOther possible approaches
OpenGL, DirectX (Windows)OpenGL, DirectX (Windows)VTK, Java3DVTK, Java3DHigher level toolkitsHigher level toolkits
Java3DJava3DCross PlatformCross PlatformSupports hardware acceleration Supports hardware acceleration Off-screen rendering Off-screen rendering JavaJava
Pilot studiesPilot studies Performance Performance Off-screen renderingOff-screen rendering Client-serverClient-server
Proof of conceptProof of concept
BrainJ3D – Next GenerationBrainJ3D – Next Generation
Live DemoLive Demo
BrainJ3DBrainJ3D
BrainJ3D BrainJ3D Features Seen in DemoFeatures Seen in Demo
• CutawayCutaway
• MappingMapping
• fMRI OverlaysfMRI Overlays
• WorkspaceWorkspace
BrainJ3DBrainJ3DOther FeaturesOther Features
• Supports common formats and data types Supports common formats and data types (Analyze, Minc, data created with SPM, FSL, etc.)(Analyze, Minc, data created with SPM, FSL, etc.)– Leverages functionality of other programsLeverages functionality of other programs– Increases user baseIncreases user base– Use Java utilitiesUse Java utilities
• 3D model loaders3D model loaders• NetCDF libraryNetCDF library• 2D image API2D image API
• Configuration and internal files in XMLConfiguration and internal files in XML– More portableMore portable– Easy to customizeEasy to customize
BrainJ3DBrainJ3DOther FeaturesOther Features
• Client/ServerClient/Server– Client/Server mode looks and behaves Client/Server mode looks and behaves
exactly like standalone modeexactly like standalone mode– Reduced performanceReduced performance
•Sending the image is a bottleneckSending the image is a bottleneck
– Takes advantage of built in Java securityTakes advantage of built in Java security– Thin client only requires installation of JavaThin client only requires installation of Java
• Java3D and accelerated graphics card not Java3D and accelerated graphics card not requiredrequired
BrainJ3DBrainJ3DArchitectureArchitecture
BrainJ3DBrainJ3D
Java and Java3D provide viable development Java and Java3D provide viable development environment for scientific visualization environment for scientific visualization applicationsapplications
+ Cross Platform ( Linux, Windows and Solaris)+ Cross Platform ( Linux, Windows and Solaris)+ Supports hardware acceleration and scene optimization+ Supports hardware acceleration and scene optimization+ Supports hardware accelerated off-screen rendering+ Supports hardware accelerated off-screen rendering+ Mature software package that provides needed + Mature software package that provides needed
functionalityfunctionality+ Java+ Java
Modern object-oriented languageModern object-oriented languageNumerous libraries and utilitiesNumerous libraries and utilitiesWeb-based and remote technologies are well supportedWeb-based and remote technologies are well supported
BrainJ3DBrainJ3D
Future DevelopmentFuture DevelopmentDifferent way to access programDifferent way to access program
Java Web StartJava Web Start
HTML GUIHTML GUI
Advanced Visualization TechniquesAdvanced Visualization TechniquesTime series (movie) mode for VolumesTime series (movie) mode for Volumes
Show functional activation on Surface modelShow functional activation on Surface model
Other FeaturesOther FeaturesAnalysis and visualization features requested by usersAnalysis and visualization features requested by users
Current and Future WorkCurrent and Future WorkIntegration with Information System for Language Integration with Information System for Language
MappingMapping
Improving InteroperabilityImproving InteroperabilitySupport for other data modalities Support for other data modalities
Spectroscopy, ERP source signal, Transcranial Magnetic StimulationSpectroscopy, ERP source signal, Transcranial Magnetic Stimulation
Importing data from other Brain Imaging software packages Importing data from other Brain Imaging software packages FreeSurfer and SureFitFreeSurfer and SureFit
* Interoperability is a fast moving target, often beyond our control* Interoperability is a fast moving target, often beyond our control
Refining Client/Server modeRefining Client/Server modeSecuritySecurity
PerformancePerformance
UW Human Brain ProjectUW Human Brain Project
Structural Informatics GroupStructural Informatics Group
http://sig.biostr.washington.edu/http://sig.biostr.washington.edu/
AcknowledgementsAcknowledgements
Human Brain Project grant DC02310Human Brain Project grant DC02310National Institute of Deafness and Other Communication National Institute of Deafness and Other Communication
Disorders and National Institute for Mental HealthDisorders and National Institute for Mental Health
Top Related