OpenStack in Action 4! Susheel Varma - VPH-Share: Patient-Centred Multi-scale Cloud-enabled...
-
Upload
enovance -
Category
Technology
-
view
219 -
download
1
description
Transcript of OpenStack in Action 4! Susheel Varma - VPH-Share: Patient-Centred Multi-scale Cloud-enabled...
Reproducibility a principal driver of the
scientific method
Patient Centred Multi-Scale Cloud-Enabled Computational Workflows
Dr Susheel Varma University of Sheffield
3 #OIA4 Paris, France 04-‐Dec-‐13
in silico Medicine
“…is the direct use of computer simulation in the diagnosis, treatment, or prevention of a disease.”
Predict disease Personalise treatment
4 #SummerSchool 20-‐Jun-‐13 Cloud Platform
(Public / Private)
Patient Data Workflow Inputs
Workflow Outputs
Semantic Services
Patient Centred Computational Workflows
Patie
nt A
vata
r
App
licat
ion
Info
stru
ctur
e
HPC Infrastructure (DEISA / PRACE)
Pers
onal
ised
M
odel
Knowledge Discovery Data Inference
Compute Services
Storage Services
Knowledge Management
Data Services: Patient/Population
euHeart @neurIST
VPH OP ViroLab
Project No: 269978 VPH-Share
Partners: CYFRONET, PL Sheffield Teaching Hospitals, UK ATOS Origin, ES Kings College London, UK Empirica, DE CiNECA, IT Nine Health CIC, UK INRIA, FR IOR, IT Open Univ., UK Philips Elec., NL TU Eindhoven, NL Univ. Auckland, NZ Uv Amsterdam, NL UCL, UK Univ. Vienna, AT AATRM, ES FCRB, ES
Co-ordinator: University of Sheffield, UK
Select Workflow Retrieve ExisGng Data
Transform or Infer Data
Run Workflow
Return Results
VPH-Share A0 Master Plan
5 #OIA4 Paris, France 04-‐Dec-‐13
ATM
Atmosphere Cloud Platform
Atomic Service Deployment Wizard
MAFEventBus
Authentication Services Workflow
Execution Service
Workflow Registry
Atomic Service Registry
Atomic Service Manager Data Browser Atomic Service
Generic Invoker
Master Interface
Cloud Facade
Visualisation Tools Workflow Composer
VPH-
Share
Client
Generic Workflow Documen
t
Atomic Service
Description
Cloud Clients
libcloud
provider
libcloud
provider
Monitoring Controller
High PerformanceExecution Engine (AHE)
Extension Points
SPRUCE
HARC
Steering
AHE Services API
AHE Runtime
AHE Engine
App Regis
try
JBPM
Workflow &
Main Logic
AHE Database Hibernate ORM
App State Objec
t
Storage Module
Connector Module
External Data
Storage
External HPC
Platform
Security Module
Allocation Management Service
AMS Manag
er (Java) OSGi
bundles
Apache
Karaf
Scheduler / Optimizer
… Algo n Algo 1
REST API & HTML Service
(Ruby) Sinatra & Passenger
Domain Model (Ruby)
Atmosphere Internal Registry
MongoDB
Virtual Machine Template Registry
Data Buckets (C-DISC, CSV,
…)
Databases (SQLServer, …)
External Structured
Data Providers
Data Publishing
Suite (GUI)
Schema Crawler
SPARQL Discovery
Browser Search
RDB2RDF Service
…
LOD Databases
Silk, LinQuer Service
…
LD Databases
Multi-Ontology/Archetype
Search Services
Taverna Server
Service Registry
Load Balancer
Proxy Controller
Data Reliability & Integrity Services
PSLoader
External Cloud Data
Storage
Semantic Services
VoID Document Database
Database 1 Query Services
(SPARQL & SQL)
Database Services
Integration Points
Database 2 Query Services
(SPARQL & SQL)
Database n Query Services
(SPARQL & SQL)
Individual Relational Databases
VoID Services
……
……
VoID Document
Atmosphere Cloud Platform
Monitoring System
Atomic Service Instance Contents
Raw Operating System (Linux)
LOBCDER Federated Storage Access
Root Volume
VPH-Share Tool / App
Web Service Wrapper Soaplab2, CXF, soap4r Remote
Access Service
Web Service Security Agent
Monitoring Agent (Munin)
Hypervisor
Driver
Manager
Compute Worker
Network Worker
Object Storage (Swift)
Dashboard
Queue Scheduler
Proxy
Account
Container
Object
ASIProxy
Private Compute & Storage Cloud (OpenStack Example)
Data Volume
Data Resource Catalog
LOBCDER Data Federation Middleware
Data Stores
Connection Module
…
Request Manager
Access & Control Frontend
Virtual Resource System
… Driver 1 Driver n Cloud Storage Driver
Data Infrastructure Services
Images
NOVA API
6
Multi-Scale Scientific Workflows
#OIA4 Paris, France 04-‐Dec-‐13
Multi-Scale Challenges
• State Space Explosion
• Inverse Parameter Identification
• Parameter Sensitivity
• Incomplete Inputs
• Strong Spatio-Temporal Coupling
• Chaos and Unstablility
• Uncertainty Cascade
10 #OIA4 Paris, France 04-‐Dec-‐13
11 #OIA4 Paris, France 04-‐Dec-‐13
VPH-Share Flagship Workflows
euHeart 16 Partners € 19.05 million Jun 08 – Nov 12
SoMware VisualizaGon
MulG-‐scale Models Imaging MulGmodal AcquisiGon
RegistraGon Anatomical
PaGent Therapy
Geometry
FuncGonal
Microstructure
Vasculature Coronary Flow Perfusion
Mechanics AcGvaGon
EC Cell Ventricular Flow
Ontologies
CellML
FieldML
X-‐Ray/MR MoGon
Electrical/MR ExcitaGon GIMIAS
CMGUI
Modelling tools and technologies
PaGent
ComputaGonal OpenCMISS SOFA OPENFEM LIFEV Numerical basis reducGon, POD FEM, FD, ALE Parallel CompuGng PETsc, MUMPS Data assimilaGon unscented Kalmann filtering variaGonal approaches
Current PracGce Conges(ve Heart Failure
13 #OIA4 Paris, France 04-‐Dec-‐13
euHeart Simulation Workflow
14 #OIA4 Paris, France 04-‐Dec-‐13
euHeart Simulation Workflow
15 #OIA4 Paris, France 04-‐Dec-‐13
@neurIST Simulation Workflow
15
Medical image
Input: DICOM Output: 3D image DescripGon: Converts a DICOM image to VTK image
Segmenta(on
Input: Image,ROI Output: surface mesh DescripGon: vessels and aneurysm extracGon
Mesh Edi(ng
Input: surface mesh Output: surface mesh DescripGon: clipping vessels, cleaning surface (cell removal, closing holes, smoothing…)
Skeletoniza(on
Input: surface mesh. Output: skeleton. DescripGon: necessary to set the boundary condiGons
Aneurysm isola(on
Input: surface mesh Output: surface mesh DescripGon: aneurysm isolaGon
Morphology Descriptors
Input: surface mesh Output: xml, vtk DescripGon: surface, depth… and ZMI calculaGon
Volumetric Mesh
Input: surface mesh Output: volumetric mesh DescripGon: creates a volumetric mesh of the selected geometry
Flow Simula(on
Input: volumetric mesh, ccl Output: wall shear stress map DescripGon: solves flow equaGons
Flow Simula(on post-‐processing
Input: wall shear stress Output: .csv file DescripGon: computes hemodynamic descriptors
CFD preprocessor
Input: xml, surface mesh Output: surface mesh, ccl DescripGon: Defines hemodynamic model
Input: surface, 1D model Output: xml, vtk DescripGon: boundary condiGons for CFD
Selec(ng Boundary Condi(ons
Neck Selec(on
Input: surface mesh Output: surface mesh DescripGon: aneurysm neck surface and dome selecGon
GIMIAS
@neuFuse
ANSYS (ICEM)
ANSYS (CFX)
Manual interacGon Common opera(ons
Morphological analysis
Hemodynamic analysis
16 #OIA4 Paris, France 04-‐Dec-‐13
• Gathered: scattered across different repositories/catalogues
• All necessary elements available and accessible maybe open
• Documented sufficiently well – Explicitly Transparent: How, Why, What,
Where, Who, When – Comprehensive: Just Enough – Comprehensible: Independent
understanding • Skills and resources to repeat
– Crowd sourced? Supercomputer?
PublicaGon
Models, Techniques, Algorithms
Data
Provenance AgribuGon Credit
Context InvesGgaGon
Study Assay
17 #OIA4 Paris, France 04-‐Dec-‐13
PublicaGon
Models, Techniques, Algorithms
Data
Provenance AgribuGon Credit
Context InvesGgaGon
Study Assay
Accessible Capable Reusable
INSTRUMENTs Samples,Specimens Strains
74% / 26%
31% / 8%
19
Embodying a Patient Avatar
#OIA4 Paris, France 04-‐Dec-‐13
20 #OIA4 Paris, France 04-‐Dec-‐13
• A large virtual catalogue of every item of data, information & knowledge of • Patient; or • Collection of patients
(Avatars)
• It also needs to be shared securely, to be able • To be Searched, Browsed &
Analysed • For Healthcare, Research &
Education
Vital Signs
Medical Devices
Results
Patient History
Procedures
Medical Images
Histology
Genomics
Proteomics
Patient Avatar
Patient Avatar – Goals • Integrating fragmented data and knowledge into a single
cohesive unit of data
• Creating a centralised repository with reliable population and patient data around which VPH tools and applications could be built
• Providing diagnostic or prognostic decision and treatment planning support using predictive models built around a patient-specific avatar
• Providing a comprehensive overview of a patient with missing data based on averages from population phenotype to explore and test ideas virtually
21 #OIA4 Paris, France 04-‐Dec-‐13
Current Access to Clinical Data
22 #OIA4 Paris, France 04-‐Dec-‐13
Owner or Facilitator Description Number (records) Accessibility
Sheffield Teaching Hospitals
ACS data Clinical data, outcomes ~750 Philips
Others on request
Nine Health HES extract Data catalogues ~ 5 millon On demand
FRCB Clinical data Ethics approved for entire record On demand
AATRM Images ~ 6 million On demand
@neurIST Image data CRIM data
Derived data
~100 (600) ~1400 ~300
On demand
euHeart 3D Models 216 cardiac ~90 other Registered users
Virolab Rule database Publication corpus
4 Rule DBs ~1.2 GB Registered users
VPH-OP Extensive clinical baseline data Images
281 Mixed On demand
23
Knowledge Management, Discovery & Semantic Services
#OIA4 Paris, France 04-‐Dec-‐13
Health Language Terminology
24 #OIA4 Paris, France 04-‐Dec-‐13
Health Language Terminology Terminology Sets • SNOMED CT / CA extensions • ICD-9P&CM • ICD-10 • ICD-10-CM / PCS • CPT-4 • HL7 • HCPCS • APC, DRG, MS-DRG • LOINC • ICPC1&2 • DSM IV • MeSH • Pharmacy (FDB, Multum) – NDC • RxNorm • Nursing (NIC, NOC, NANDA) • LCD/NCD/NCCI • Consumer Friendly Terminology (CFT)
• CDT • Multiple Languages • Local Codes • Nomenclature • ICD-10 (GM/AM/CA) • ICD-O • UK Admin Extension • UK Gap Extension • HRG • OPCS-4 • CCI • Read 2 • Read 4-byte • SNOMED Facets • Clinical Specialty Subsets
Mappings • SNOMED CT to ICD-9-CM • SNOMED CT to ICD-10 • SNOMED CT to OPCS-4 • ICD-9-CM to SNOMED CT • SNOMED CT to CPT • CPT to SNOMED CT • ICD-9-CM to ICD-10-CM/
PCS • ICD-10-CM/PCS to ICD-9-
CM • SNOMED to MeSH • DSM IV to SNOMED • ICD-9-CM Procedures to
SNOMED • HL7 to CHI • Language to Language
(e.g., English to Spanish)
Web of Semantically Linked Data
25 #OIA4 Paris, France 04-‐Dec-‐13
26 #OIA4 Paris, France 04-‐Dec-‐13
Tabular Data
Non-Tabular Data
Clinical Information Systems
Data Publishing Suite
Semantic Services
Computational Workflows and Services 1
2
3 4 8 9 10 6 7
Medications
Vital Signs
Lab Reports
Demographic
Vital Signs
Images
Demographic
Risk Factors
Genomic Data
Parameter Estimation
Uncertainty Propagation
Patient Avatar
RDF Graphs
Reference Data
Pseudo Identifier (UID)Demographics
HeightWeight
Vital SignsHeart RateBlood Pressure
Systolic Diastolic
Aneurysm Related Health EventRisk FactorsAneurysm Imaging StudyMedications
Pseudo Identifier (UID)Demographics
HeightWeight
Vital SignsHeart RateBlood Pressure
Systolic Diastolic
Pulmonary FunctionRisk FactorsCardiac Imaging StudyMedications
Pseudo Identifier (UID)Demographics
HeightWeight
Vital SignsHeart RateBlood Pressure
Systolic Diastolic
Orthopaedic Health EventGynaecological InformationBone Phenotype Imaging StudyMedications Physiological
Envelope
27 #OIA4 Paris, France 04-‐Dec-‐13
Patient Avatar & Workflow Archetypes
Baseline HistoryCardiac Health EventLab TestsMedicationsCardiac Imaging StudyImaging Details
(c) euHeart Workflow Archetype
Sexual HealthHIV Subtype InformationSpecimens
(d) ViroLab Workflow Archetype
Aneurysm Related Health EventSupporting Risk FactorsSpecimens (Blood, Tissue)MedicationsAneurysm Imaging StudyImaging Details
(b) @neurIST Workflow Archetype
Hand Grip StrengthOrthopeadic Health EventMedicationsSpecimens (Blood, Tissue)Bone Phenotype Imaging Study
(e) VPHOP Workflow ArchetypePatient Pseudoidentifier (PID)DemographicsPersonal & Social HistoryFitness & LifestyleEmployment DetailsVital SignsSubstance Use
(a) Common Shared Archetype
Baseline HistoryCardiac Health EventLab TestsMedicationsCardiac Imaging StudyImaging Details
(c) euHeart Workflow Archetype
Sexual HealthHIV Subtype InformationSpecimens
(d) ViroLab Workflow Archetype
Aneurysm Related Health EventSupporting Risk FactorsSpecimens (Blood, Tissue)MedicationsAneurysm Imaging StudyImaging Details
(b) @neurIST Workflow Archetype
Hand Grip StrengthOrthopeadic Health EventMedicationsSpecimens (Blood, Tissue)Bone Phenotype Imaging Study
(e) VPHOP Workflow ArchetypePatient Pseudoidentifier (PID)DemographicsPersonal & Social HistoryFitness & LifestyleEmployment DetailsVital SignsSubstance Use
(a) Common Shared Archetype
Baseline HistoryCardiac Health EventLab TestsMedicationsCardiac Imaging StudyImaging Details
(c) euHeart Workflow Archetype
Sexual HealthHIV Subtype InformationSpecimens
(d) ViroLab Workflow Archetype
Aneurysm Related Health EventSupporting Risk FactorsSpecimens (Blood, Tissue)MedicationsAneurysm Imaging StudyImaging Details
(b) @neurIST Workflow Archetype
Hand Grip StrengthOrthopeadic Health EventMedicationsSpecimens (Blood, Tissue)Bone Phenotype Imaging Study
(e) VPHOP Workflow ArchetypePatient Pseudoidentifier (PID)DemographicsPersonal & Social HistoryFitness & LifestyleEmployment DetailsVital SignsSubstance Use
(a) Common Shared Archetype
Baseline HistoryCardiac Health EventLab TestsMedicationsCardiac Imaging StudyImaging Details
(c) euHeart Workflow Archetype
Sexual HealthHIV Subtype InformationSpecimens
(d) ViroLab Workflow Archetype
Aneurysm Related Health EventSupporting Risk FactorsSpecimens (Blood, Tissue)MedicationsAneurysm Imaging StudyImaging Details
(b) @neurIST Workflow Archetype
Hand Grip StrengthOrthopeadic Health EventMedicationsSpecimens (Blood, Tissue)Bone Phenotype Imaging Study
(e) VPHOP Workflow ArchetypePatient Pseudoidentifier (PID)DemographicsPersonal & Social HistoryFitness & LifestyleEmployment DetailsVital SignsSubstance Use
(a) Common Shared Archetype
Baseline HistoryCardiac Health EventLab TestsMedicationsCardiac Imaging StudyImaging Details
(c) euHeart Workflow Archetype
Sexual HealthHIV Subtype InformationSpecimens
(d) ViroLab Workflow Archetype
Aneurysm Related Health EventSupporting Risk FactorsSpecimens (Blood, Tissue)MedicationsAneurysm Imaging StudyImaging Details
(b) @neurIST Workflow Archetype
Hand Grip StrengthOrthopeadic Health EventMedicationsSpecimens (Blood, Tissue)Bone Phenotype Imaging Study
(e) VPHOP Workflow ArchetypePatient Pseudoidentifier (PID)DemographicsPersonal & Social HistoryFitness & LifestyleEmployment DetailsVital SignsSubstance Use
(a) Common Shared Archetype
Knowledge Management
28 #OIA4 Paris, France 04-‐Dec-‐13
29
Cloud-Enabled Computational Workflows
#OIA4 Paris, France 04-‐Dec-‐13
Cloud-Enabled Computation Workflows • portal.vph-share.eu – Provides access to Clinical Data and Scientific applications
and workflows
• Taverna Workbench – Provides Desktop tool for composition of Scientific Workflows with VPH-Share applications and data
• OnlineHPC.com – Allows the creation of Scientific Workflows composed of External and VPH-Share Applications and VPH-Share data
• Meta-Workflow Manager – Executes multiple Scientific Workflow Engines on the VPH-Share Platform
• Workflow Cloud Plugin – Allows execution of Scientific Web-Services and Application on multiple cloud platforms
• Command-Line Wrapper – Allows developers to wrap command-line applications into a (REST/SOAP) webservice via wsme and GIMIAS
• NoMachine RDP – Provides Remote Desktop services for applications that require user interaction
30 #OIA4 Paris, France 04-‐Dec-‐13
Cloud-Enabled Computation Workflows
31 #OIA4 Paris, France 04-‐Dec-‐13
32 #OIA4 Paris, France 04-‐Dec-‐13
Clinical Researcher
Workflow Manager API
VPH-‐Share plugin
Taverna Server
VPH-‐Share Workflow
Cloud Façade
Web-‐based Remote Desktop
AS without interacGon
AS with interacGon
CLIENT-‐SIDE SERVER-‐SIDE
AS AS
AS
AS AS AS
External ApplicaGon
STORAGE
VPH-‐Share plugin
Taverna Workbench
Web services
GIMIAS CLPs
…
VPH-‐Share plugin
Taverna On-‐line
Web services
…
33
VPH-Share HPC & Cloud Platform
#OIA4 Paris, France 04-‐Dec-‐13
Scientific Cloud Platform
34 #OIA4 Paris, France 04-‐Dec-‐13
ATM
Atmosphere Cloud Platform
Atomic Service Deployment Wizard
MAFEventBus
Authentication Services Workflow
Execution Service
Workflow Registry
Atomic Service Registry
Atomic Service Manager Data Browser Atomic Service
Generic Invoker
Master Interface
Cloud Facade
Visualisation Tools Workflow Composer
VPH-
Share
Client
Generic Workflow Documen
t
Atomic Service
Description
Cloud Clients
libcloud
provider
libcloud
provider
Monitoring Controller
High PerformanceExecution Engine (AHE)
Extension Points
SPRUCE
HARC
Steering
AHE Services API
AHE Runtime
AHE Engine
App Regis
try
JBPM
Workflow &
Main Logic
AHE Database Hibernate ORM
App State Objec
t
Storage Module
Connector Module
External Data
Storage
External HPC
Platform
Security Module
Allocation Management Service
AMS Manag
er (Java) OSGi
bundles
Apache
Karaf
Scheduler / Optimizer
… Algo n Algo 1
REST API & HTML Service
(Ruby) Sinatra & Passenger
Domain Model (Ruby)
Atmosphere Internal Registry
MongoDB
Virtual Machine Template Registry
Data Buckets (C-DISC, CSV,
…)
Databases (SQLServer, …)
External Structured
Data Providers
Data Publishing
Suite (GUI)
Schema Crawler
SPARQL Discovery
Browser Search
RDB2RDF Service
…
LOD Databases
Silk, LinQuer Service
…
LD Databases
Multi-Ontology/Archetype
Search Services
Taverna Server
Service Registry
Load Balancer
Proxy Controller
Data Reliability & Integrity Services
PSLoader
External Cloud Data
Storage
Semantic Services
VoID Document Database
Database 1 Query Services
(SPARQL & SQL)
Database Services
Integration Points
Database 2 Query Services
(SPARQL & SQL)
Database n Query Services
(SPARQL & SQL)
Individual Relational Databases
VoID Services
……
……
VoID Document
Atmosphere Cloud Platform
Monitoring System
Atomic Service Instance Contents
Raw Operating System (Linux)
LOBCDER Federated Storage Access
Root Volume
VPH-Share Tool / App
Web Service Wrapper Soaplab2, CXF, soap4r Remote
Access Service
Web Service Security Agent
Monitoring Agent (Munin)
Hypervisor
Driver
Manager
Compute Worker
Network Worker
Object Storage (Swift)
Dashboard
Queue Scheduler
Proxy
Account
Container
Object
ASIProxy
Private Compute & Storage Cloud (OpenStack Example)
Data Volume
Data Resource Catalog
LOBCDER Data Federation Middleware
Data Stores
Connection Module
…
Request Manager
Access & Control Frontend
Virtual Resource System
… Driver 1 Driver n Cloud Storage Driver
Data Infrastructure Services
Images
NOVA API
Platform Architecture
35 #OIA4 Paris, France 04-‐Dec-‐13
Physical resources
Atomic Service Instances Deployed by AMS on available resources as required by WF mgmt or generic AS
invoker
Raw OS (Linux variant)
LOB Federated storage access
Web Service cmd. wrapper
Generic VNC server
VPH-Share Tool / App.
DRI Service
Atmosphere persistence layer (internal registry)
VM templates
AS images
Available cloud
infrastructure
Managed datasets
101101 011010 111011
101101 011010 111011
101101 011010 111011
AM Service
LOB federated storage access
Cloud stack clients
HPC resource client/backend
Data and Compute Cloud Planorm
VPH-‐Share Master UI
AS mgmt. interface
Generic AS invoker
ComputaGon UI extensions
Data mgmt. interface
Generic data retrieval
Data mgmt. UI extensions
Remote access to Atomic Svc. UIs
Custom AS client
Workflow descripGon and execuGon
Developer ScienGst
Admin
Security mgmt. interface
Security framework
Web Service security agent
Modules available in first prototype
Cloud Platform Architecture
36 #OIA4 Paris, France 04-‐Dec-‐13
VPH-‐Share Master Int.
Admin Developer ScienGst
Development Mode
VPH-‐Share Core Services Host
ComputaGonal Cloud Site
Worker Node
Worker Node
Worker Node
Worker Node
Worker Node
Worker Node
Worker Node
Worker Node
Head Node
Image store (Glance)
Cloud Facade (secure
RESTful API )
Atmosphere Management Service (AMS)
Cloud stack plugins (JClouds)
Atmosphere Internal
Registry (AIR)
Cloud Manager
Generic Invoker
Workflow management
External applicaGon
Cloud Facade client
• The platform provides a set of APIs for the VPH-Share Master Interface and other applications, enabling Atomic Services to be developed.
Customized applicaGons may directly interface the Cloud Facade via its RESTful APIs
Accessible HPC Execution Platform
37 #OIA4 Paris, France 04-‐Dec-‐13
� Provides virtualized access to high performance execuGon environments � Seamlessly provides access to high performance compuGng to workflows that
require more computaGonal power than clouds can provide � Deploys and extends the ApplicaGon HosGng Environment – provides a set of web
services to start and control applicaGons on HPC resources
GridFTP AHE Web Services
(RESTlets)
Grid resources running Local Resource Manager (PBS, SGE, Loadleveler etc.)
ApplicaGon HosGng Environment Auxiliary component of the cloud planorm, responsible for managing access to tradiGonal (grid-‐based) high performance compuGng environments. Provides a Web Service interface for clients.
Invoke the Web Service API of AHE to delegate computaGon to the grid
ApplicaGon -‐-‐ or -‐-‐
Workflow environment
-‐-‐ or -‐-‐
End user
Present security token (obtained from authenGcaGon service)
Tomcat container WebDAV
User access layer
QCG CompuGng
Job Submission Service (OGSA BES / Globus
GRAM) RealityGrid SWS
Resource client layer
Delegate credenGals, instanGate compuGng tasks, poll for execuGon status and retrieve results on behalf of the client
Unstructured Data Storage
38 #OIA4 Paris, France 04-‐Dec-‐13
LOBCDER host (149.156.10.143)
LOBCDER service backend
Resource catalogue
WebDAV servlet
Resource factory
Storage driver
Storage driver (SWIFT)
SWIFT storage backend
Core component host (vph.cyfronet.pl) Data Manager
Portlet (VPH-‐Share
Master Interface component)
Atomic Service Instance (10.100.x.x) Service payload
(VPH-‐Share applicaGon component)
External host Generic WebDAV client
GUI-‐based access
Mounted on local FS (e.g. via davfs2)
• VPH-‐Share federated data storage module (LOBCDER) enables data sharing in the context of VPH-‐Share applicaGons
• The module is capable of interfacing various types of storage resources and supports SWIFT cloud storage (support for Amazon S3 is under development)
• LOBCDER exposes a WebDAV interface and can be accessed by any DAV-‐compliant client. It can also be mounted as a component of the local client filesystem using any DAV-‐to-‐FS driver (such as davfs2).
EncrypGon keys
REST-‐interface
Master Interface component Ticket validaGon service
Auth service
Amazon S3 Storage backend
Platform by Numbers • 4 Data centers
– CYFRONET, Krakow – UoS, Sheffield – STH, Sheffield – UNV, Vienna
• 80+ Cloud Hosts • 100+ VMs baseline, 331 VMs Peak • 50TB+ Data Storage • 75+ Scientific Applications • 25+ Scientific Workflows • €70k Public Cloud Burst Funds
39 #OIA4 Paris, France 04-‐Dec-‐13
Elephant in the Room
• Security & Privacy • Legislation & Ethics • Training
• Long Tail of Physicians and Care-takers
• P4 Medicine Journey • Predictive • Preventative • Personalised • Participatory
40 #OIA4 Paris, France 04-‐Dec-‐13
41 #OIA4 Paris, France 04-‐Dec-‐13
<Thank You!>
42
Dr Susheel Varma <[email protected]> VPH-Share - Scientific Workflows Coordinator Department of Cardiovascular Science The Medical School, The University of Sheffield Beech Hill Road, Sheffield S10 2RX UK T: +44 (0)114 271 2863
#OIA4 04-‐Dec-‐13
• @neurIST- Integrated Biomedical Informatics for the Management of Cerebral Aneurysms (http://www.aneurist.org)
• IST project funded within the EU FP6 • Duration: 2006-2010 • Budget: 17M€ • Participants: 28 institutions
– Public and private, – Industry, hospitals, academia, – 12 European countries
• External collaborators: from USA, New Zealand, Japan)
• @neurIST main objective: @neurIST will transform the management of cerebral aneurysms by providing new insights, personalized risk assessment, and methods for the design of improved medical devices and treatment protocols.
@neurIST Simulation Workflow
@neurIST Framework
VPH-‐Share
DICOM
Input: DICOM Output: 3D image DescripGon: Converts a DICOM image to VTK image
Volume Rendering
Input: 3D image Output: 3D image DescripGon: aneurysm and vessels VisualisaGon
Bounding Box
Input: 3D image Output: ROI DescripGon: volume selecGon
GAR Segmenta(on
Input: Image,ROI Output: surface mesh DescripGon: vessels and aneurysm extracGon
Mesh Edi(ng
Input: surface mesh Output: surface mesh DescripGon: clipping vessels, cleaning surface (cell removal, closing holes, smoothing…)
Skeletoniza(on
Input: surface mesh. Output: skeleton. DescripGon: necessary to set the boundary condiGons
Aneurysm isola(on
Input: surface mesh Output: surface mesh DescripGon: aneurysm isolaGon
Morphology Descriptors
Input: surface mesh Output: xml, vtk DescripGon: surface, depth… and ZMI calculaGon
Volumetric Mesh
Input: surface mesh Output: volumetric mesh DescripGon: creates a volumetric mesh of the selected geometry
Flow Simula(on
Input: volumetric mesh, ccl Output: wall shear stress map DescripGon: solves flow equaGons
Flow Simula(on post-‐processing
Input: wall shear stress Output: .csv file DescripGon: computes hemodynamic descriptors
CFD preprocessor
Input: xml, surface mesh Output: surface mesh, ccl DescripGon: Defines hemodynamic model
Input: surface, 1D model Output: xml, vtk DescripGon: boundary condiGons for CFD
Selec(ng Boundary Condi(ons
Neck Selec(on
Input: surface mesh Output: surface mesh DescripGon: aneurysm neck surface and dome selecGon
GIMIAS
@neuFuse
ANSYS (ICEM)
ANSYS (CFX)
Manual interacGon Common opera(ons
Morphological analysis
Hemodynamic analysis
Two Workflows from @neurIST
• Morphological, hemodynamic and structural analyses have been linked to aneurysm genesis, growth and rupture.
• Evidence indicating differences in morphology and flow between ruptured and unruptured aneurysms have been shown for reduced patient cohorts.
• Structural wall mechanics has been used to justify the growth and remodelling happening at the aneurysm level.
Confidence in physical measures
+
images
+ BC, material
+ BC, material
Morphological analysis
Direct diagnos6c power
+
Morphological descriptors
Structural descriptors
Hemodynamic descriptors
Haemodynamic analysis
Structural analysis
PracGcally, morphological characterizaGons might currently have the highest predic(ve capabili(es with respect to the other analyses.
Morphological Workflow
The @neurIST morphological workflow specification in Taverna:
Implementation in VPH-Share
• An automatic segmentation method based on Geodesic Active Regions (GAR) and an image standardization technique is used
• The method: – eliminates most of the dependency on the operator, and on
the specific imaging protocols and equipment used. – is able to segment (extract the surface mesh) a region of
interest with a size of 2563 voxels in 17+4 min (avg+std dev) on a PC (Intel quad-core, 2.4 GHz, 4GB memory).
Hernandez, M. et al. 2007 Non-‐parametric geodesic acGve regions: method and evaluaGon for cerebral aneurysms segmentaGon in 3DRA and CTA. Med. Image Anal. 11, 224–241. Bogunovic, H. et al. 2011 Automated segmentaGon of cerebral vasculature with aneurysms in 3DRA and TOF MRA using geodesic acGve regions: an evaluaGon study. Med. Phys. 38, 210–222.
A surface mesh represenGng the vascular geometry is required to perform the @neurIST morphological and hemodynamic analyses
Medical image from imaging equipment
Surface mesh a<er segmenta6on
GAR Segmentation
• The surface mesh obtained after the GAR segmentation needs to be manually manipulated by an operator to either remove or correct:
– some of the artifacts not belonging to the cerebral vasculature – those parts of the geometry not relevant for the subsequent
analyses (morphological or hemodynamic).
Surface mesh a<er segmenta6on
A surface mesh represenGng the vascular geometry is required to perform the @neurIST morphological and hemodynamic analyses
Kissing vessels Remove cells Close holes
O P
Surface mesh a<er correc6on and aneurysm isola6on
Mesh Editing
• User is asked to manually delineate the neck • Unfortunately, automatic methods are not an option
because there are: – unacceptable differences in a large number of cases among
methods and manual selection of experts – too complex vascular topologies where there is not even an
agreement among experts about where the aneurysm neck is
Several morphological measurements are based on the aneurysm sac, to idenGfy it, the aneurysm neck is required
Surface mesh a<er correc6on and aneurysm isola6on
Manual delinea6on
Manual aneurysm neck selec6on
vs Automat ic
Too large differences in performance and lack of consensus
Too complex vascular topologies
Neck DelineaGon
• Among the wide variety of existing morphological descriptors, @neurIST chose to compute:
– Basic size indices describing the aneurysm sac: aspect ratio, non-sphericity index, aneurysm volume and surface area.
– Complex indices describing the sac and a portion of the surrounding vasculature: Zernike moment invariants (volume and surface-based).
Morphological descriptors of various complexity are automaGcally extracted and stored for their subsequent analysis
Ujiie, H. et al. 1999 Effects of size and shape (aspect raGo). Neurosurgery 45, 119–130. / Ma, B., Harbaugh, R. E. & Raghavan, M. L. 2004 Three-‐dimensional geometrical characterizaGon of cerebral aneurysms. Ann. Biomed. Eng. 32, 264–273. / Raghavan, M. L., Ma, B. & Harbaugh, R. E. 2005 QuanGfied aneurysm shape and rupture risk. J. Neurosurg. 102, 355–362. Pozo, J. M. et al. 2011 Efficient 3D Geometric and Zernike moments computaGon from unstructured surface meshes. IEEE Trans. Pagern Anal. Machine Intell. 33, 471–484.
Manual aneurysm neck selec6on
Complex indices (Zernike moment invariants)
Basic size indices describing aneurysm sac
depth
neck
@neurIST morphological descriptors
Morphological Descriptors
Medical image from imaging equipment
@neurIST morphological descriptors
Complex indices (Zernike moment invariants)
Basic size indices describing aneurysm sac
depth
neck
Morphological Analysis Workflow
@neurIST: Morphology Results