Neuroterrain + MBL Atlas Normalization · Neuroterrain + MBL Atlas Normalization ... assignment of...
Transcript of Neuroterrain + MBL Atlas Normalization · Neuroterrain + MBL Atlas Normalization ... assignment of...
2004 All Hands Meeting
Neuroterrain + MBL Atlas Normalization
Image Processing Pipeline & Interactive tools to support atlas normalization
2005
Pipeline For Atlas NormalizationDatabase Driven• Use PostgreSQL (open-source RDBMS)
Useful native types for spatial processing (2D objects & methods) + straight-forward C API for adding new types (3D & atlas specific classes)]Complex native types used directly in query predicates to filter data
• Highly granular data model for tracking all processing detailUse of Materialized Views when large scale de-normalization required
• Thorough collection of all algorithmic I/O requirementsAlgorithm ontology - I/O data types, run-time HW/SW requirements, etc.Assemble pipeline definitions from atomic algorithm objects
• Job Controlassignment of available processing “jobs” across a distributed collection of heterogeneous runtime environments driven & monitored via the RDBMS
• Audit trail of algorithm/pipeline runsUsed for many purposes - including round-trip QC & adjustment of algorithm efficacy.
Pipeline For Atlas NormalizationJ2EE architecture• Open-source tools• Enterprise Java Beans [EJB] (JBoss)
server-based collection of re-usuable logicUsed by ALL other systems - Servlets, WS, Applications & Applets
• Servlets + Portlets - for Web UIs (Tomcat)• Web Services - for networked remote procedure calls [RPC] (Axis)• Aspect-Oriented Paradigm
servers provide functionality accessible to programmer via minimal API (e.g., do not need to build centralized authentication de novo)application-specific customization via XML config files - no direct coding required.Examples:
• JNDI - directory services for access to RDBMS (no JDBC) & other resources• Centralized authentication & authorization• Fail-over & multi-serving scaling
• GRID connection - processing & storage (Globus)
Pipeline For Atlas Normalization
J2SE - Applications & Applets for complex end-user tools• Web start when practical (client-side platform problems)• Use web services both for database and image repository access.• Provides ontology based querying to select MBL data
Again - via WS-connection to ontology repository• Heavy use of JAI/Java 2D/Java 3D for UI image processing
Couldn’t use ImageJ• imaging implementation not sufficient performance for some of the GUI-based
image processing we need to do• ImageJ = Java imaging circa 1997 when images were simply static JPEGs or
GIFs to serve up in applets• Absolutely no image processing framework in Java at that time - Wayne R.
used his extensive experience developing NIH Image to perform Herculean task of creating a full-featured image manipulation Java App at a time when Java didn’t support such functionality. Even GUI elements such as “progress bars” & line drawing he needed to implement from scratch. This legacy code still lies at the core of ImageJ.
Pipeline For Atlas Normalization
Two major applications for MBL Brain Image Processing• Neuroterrain-MBL Normalization Pipeline QC Tool• Neuroterrain-MBL Integrated Brain Atlas Viewer
Java atlas viewer (NetOStat) derived from original Mac app. MacOStat
Brain Atlas Server• Gustafson C, Tretiak O, Bertrand L, Nissanov J. (2004) Design and
implementation of software for assembly and browsing of 3D brainatlases.Comput Methods Programs Biomed. 2004 Apr;74(1):53-61.
• Uses unique 3D data manipulation paradigm (“Macrovoxel”) to enable real-time atlas slicing via the net.
• Manages multiple atlas datasets & associated brain region VOIs.• Mac-based (being ported to Unix)
Neuroterrain-MBL Normalization Pipeline QC Tool
Thumbnail View• 3 way split frame
file system view to find directory with image stack.• Familiar drill up & down for rapid navigation
Thumbnail view of image stack• Scroll to rapidly view entire stack - thumb size adjustable• Easy click-n-drag to re-order sections - auto-shift entire groups of
images to account for wide-spread stack misalignment• Contextual menu - delete an image, add a blank to substitute for lost
sectionsFull view of individual image
• Grid & scale to assist image review• Simple contrast adjustments to enhance detail when necessary• Contextual menu - re-orient by flipping relative to x or y axis, quick
switch to full slide image view
Neuroterrain-MBL Normalization Pipeline QC Tool
Full Slide View - shows section in context of original slide• Section Navigation
“movie” controls - run stack as “movie” - very fast way to do final confirmation on QC (“play”, “pause”, “stop”, “FF”,”rewind”, “step+”, “step-”)Slider to quickly jump to specific imageOriginal Slide images
• Pair of original slide images for specific brain presented as horizontal mosaic• Using controls listed above automatically shifts outline control over appropriate
image in slide• Outline is “active” navigation tool - move outline to new section updates full
view• Section Image basic info
Lists status info (“deleted”, “flipped”) - active controlsLists original geometric location of section on original slide - geometric relation of section to slide must be maintained to support investigator using atlas to select brain regions for high-rez cell stack imaging.
• Image Manipulation“knife” tool to separate concatenated sections“sewing machine” tool to concatenate sections split by the auto-segmenter.
Pipeline QC Tool - Thumbnail View
Pipeline QC Tool - Slide View
2004 All Hands MeetingMBL ImageRepository
P4/Lnx ClusterG4/Mac Cluster
Nrtrn J2EEApp Svr
Slide SegmentationCoarse Alignment
Auto-segment ROIsWarp
Pathfinder
Coarse AlignmentWarp
Inet, Inet2 &Grid
Router
Switch
MBL Brain Image Stack IP Pipeline
Neuroterrain-MBL Integrated Brain Atlas Viewer
Atlas Viewer• Select from multiple atlas data sets• Select & draw brain region ROI
assign color/ROItoggle ROIs on & off
• Use wire-frame “virtual knife” navigator for rapid navigationKnife position info - x,y,z rotation - cutting plane translationSelect cutting plane (coronal, horizontal or sagittal)Select whether mouse movement translates knife along cutting plane or rotates along x,y, or z.
• Atlas position info - x,y,z location
Neuroterrain-MBL Integrated Brain Atlas Viewer
MBL Section Image Viewer• Stack navigation - slider, step buttons or “jump to”
Coupled to atlas viewer - selecting new brain section adjusts atlas virtual knife to equivalent registered location.
• Use web-service based connection to MBL image repositorypose an ontology-based query against MBL RDBMS to define list of brains to view (e.g., “retrieve all brains for female BXD5 mice in age range 20 -40 days of strain X”)Image stacks delivered over the net via WS (SOAP + attachments) - either as individual images or a compressed archive - archive(s) are decompressed in the background, while investigator views another stack.
• Select individual MBL brain for viewing from result set via menu• Multi-mode viewing - brains for which multiple “views” of stack exist
(e.g., “raw”, “linear affine adjusted”, “non-linear warped”, etc.) - can select which set to view
Neuroterrain-MBL Integrated Atlas Viewer
Neuroterrain-MBL Integrated Atlas Viewer
Neuroterrain-MBL Integrated Atlas Viewer