Ocean Observatories Initiative OOI Cyberinfrastructure Architecture Overview Michael Meisinger...
-
Upload
kristopher-arnold -
Category
Documents
-
view
217 -
download
0
Transcript of Ocean Observatories Initiative OOI Cyberinfrastructure Architecture Overview Michael Meisinger...
Ocean Observatories Initiative
OOI Cyberinfrastructure
Architecture OverviewMichael Meisinger
September 29, 2009
Overview
• Integrated Observatory Network
• Deployment Strategy: Capability Container
• Integration Strategy: Reliable Messaging
• Service Architecture and Subsystems
• Scope of Release 1
2
Integrated Observatory Functions
• Interactive Ocean Observing• Interactive Ocean Modeling & Data Assimilation• Automated Data Product Generation• Discipline-Driven Semantic Organization of Data• Interactive Instrument Network• Integrated Observatory Management• User-Driven Integration of Resource
ObservatoryActivity Model
Open System
Integrated Observatory Users• Scientists
– Principal Investigator– Project Scientist – Support Scientist– Graduate Student
• Engineers– Project Engineer – Instrument Provider – Instrument Technician – Application Developer– Mission safety and security
• Data Professionals– Data Analyst– Data Modeler– Archivist/Curator
• Operations– Program Manager– Observatory Manager– Logistic Coordinator– Mission Planner– Scheduler
• Educators– Developer of Educational
Materials– Teacher– Librarian
• General Public– Outreach material
providers– Science Public– Everyone else
[Persona Whitepaper]
Integrated Observatory Extent
SensorPlatforms
PhysicalInterface
MarineOperations
Shore SideOperations
End UserOperations
MobileCore
FixedCore
ResearchPlatforms
AdministratorCyber OperatorMarine Operator
User
Marine
Cyber
Operational Authority
ObservatoryOperations
Science Teams & Educators
ObservatoryManagement
MarineManagement
User EnvironmentManagement
Marine Net Observatory Net
Capability Container
Infrastructure
Data Sets
Observation Plans
Processes
Instruments
RecoveryPolicies
Execution
Messaging
Management
Agreements
Models
Provision
Routing
Monitoring
Education & Participation
Community Collaboration
OrchestrationGovernance
State Mediation
Activities Resources
Knowledge
Interfaces
Repository
Catalog
Resources Interactions Communications Processing
ScientificInvestigation
CyberInfrastructure Capability Container
CI Interfaces and Subsystems
Interactive Observatory
Operating System with Core
Applications
Deployment Scenario
8
Execution Environments• Strategy
– One software development and one operational environment across all execution environments: Capability Container
– Specific profile and adapter per execution env.• Marine Observatory Infrastructure
– Mooring DCL/CPM embedded CPUs– Mobile asset (AUV, Glider, Profiler) embedded
• Terrestrial Infrastructure– Local clusters– National infrastructure (TeraGrid, OSG)– Commercial Cloud Infrastructure
9
Network Architecture
11
Network Deployment
Integration Infrastructure• The Common Operating Infrastructure (COI)
– Distributed, secure, reliable, service-oriented, message-based– Integration and communication infrastructure
• The Common Execution Infrastructure (CEI) – virtualization and provisioning of computational and storage resources
Common Operating Infrastructure
Data Management
(Science)
Sensing & Acquisition
Data Management (Information Distribution)
Analysis & Synthesis
Identity Management
State Management
GovernanceFramework
Resource Management
Planning & Prosecution
Exchange
Service Framework
Presentation Framework
Common Execution
Infrastructure
Scope of Release 1• Common Operating Infrastructure
– Messaging Service in Federated Facility– Identity and Policy Management– Distributed Service Application Framework
• Common Execution Infrastructure– Management of Executable Processes– Elastic Computing– Resource Management
• Data Management– Common Data and Metadata Model– Dynamic Data Distribution and Data Catalog– Persistent Archive Framework
• Sensing and Acquisition– Instrument Direct Access– Data Acquisition– Instrument Management and Agent Framework
13
Release 1: Data
Distribution Network
Data Collection Workflow
Deployment Pattern
15
OOI CI Kick-Off MeetingSept 9-11, 2009
Subsystem Dependencies
16
OOI CI Kick-Off MeetingSept 9-11, 2009
Application Services
17
OOI CI Kick-Off MeetingSept 9-11, 2009
Infrastructure Services
18
OOI CI Kick-Off MeetingSept 9-11, 2009
Development Dependencies
19
Technology Readiness LevelTRL Definition Description
1 Basic principles observed and reported.
Lowest level of technology readiness. Scientific research begins to be translated into applied research and development. Examples might include paper studies of a technology's basic properties.
2 Technology concept and/or application formulated.
Invention begins. Once basic principles are observed, practical applications can be invented. Applications are speculative and there may be no proof or detailed analysis to support the assumptions. Examples are limited to analytic studies.
3 Analytical and experimental critical function and/or characteristic proof of concept.
Active research and development is initiated. This includes analytical studies and laboratory studies to physically validate analytical predictions of separate elements of the technology. Examples include components that are not yet integrated or representative.
4 Component and/or breadboard validation in laboratory environment.
Basic technological components are integrated to establish that they will work together. This is relatively "low fidelity" compared to the eventual system. Examples include integration of "ad hoc" hardware in the laboratory.
5 Component and/or breadboard validation in relevant environment.
Fidelity of breadboard technology increases significantly. The basic technological components are integrated with reasonably realistic supporting elements so it can be tested in a simulated environment. Examples include "high fidelity" laboratory integration of components.
6 System/subsystem model or prototype demonstration in a relevant environment.
Representative model or prototype system, which is well beyond that of TRL 5, is tested in a relevant environment. Represents a major step up in a technology's demonstrated readiness. Examples include testing a prototype in a high-fidelity laboratory environment or in simulated operational environment.
7 System prototype demonstration in an operational environment.
Prototype near, or at, planned operational system. Represents a major step up from TRL 6, requiring demonstration of an actual system prototype in an operational environment such as an aircraft, vehicle, or space. Examples include testing the prototype in a test bed aircraft.
8 Actual system completed and qualified through test and demonstration.
Technology has been proven to work in its final form and under expected conditions. In almost all cases, this TRL represents the end of true system development. Examples include developmental test and evaluation of the system in its intended weapon system to determine if it meets design specifications.
9 Actual system proven through successful mission operations.
Actual application of the technology in its final form and under mission conditions, such as those encountered in operational test and evaluation. Examples include using the system under operational mission conditions.
Technology CatalogSubsystem Functional Component Technology TRLAnalysis & Synthesis Analysis Application/Language Matlab 9Analysis & Synthesis Workflow Execution Engine Kepler 6Analysis & Synthesis Meta-Workflow Resource Mapper Pegasus 9Analysis & Synthesis Visualization Toolkit VTK 9Analysis & Synthesis Graphics Engine OpenSceneGraph 9Analysis & Synthesis Visualization Application IDV 9Analysis & Synthesis Spatial Display Application OSSIM Planet 7Analysis & Synthesis Spatial Display Application GoogleEarth & Maps 9Data Management Dataset Catalog THREDDS 7Data Management Semantic Framework VSTO Semantic Framework 8Data Management Semantic Query ESG Facetted Search 8Data Management Data Integration with Applications NetCDF lib 8Data Management Data Integration with Applications Matlab lib for OpenDAP access 8Data Management Dataset Management & Distribution OpenDAP Hyrax Server 7Data Management Dataset Preservation iRODS 7Data Management General Purpose Database MySQL cluster 9Planning & Prosecution Resource Planner ASPEN 9Planning & Prosecution Controller CASPER 9Planning & Prosecution Autonomous Comm Middleware MOOS 8Planning & Prosecution Autonomous Helm MOOS-IvP Helm 8Sensing & Acquisition Virtual Resource Framework IaaS 6Sensing & Acquisition Instrument Drivers Antelope ORB 9Sensing & Acquisition Instrument Drivers SIAM 9Sensing & Acquisition Instrument Drivers PUCK 8Sensing & Acquisition Resource Monitoring Application Intermapper 9CEI Grid Scheduler Condor 9CEI Cloud Computing Environment Amazon EC2 9CEI Cloud Computing Environment Nimbus 8CEI VM Packaging & Distribution Elastic Server (CohesiveFT) 8CEI Virtualization Layer Software Individual virtual-ization vendors, e.g., Amazon AMI)9COI Message Broker System RabbitMQ 8COI IOC Container Spring 9COI Enterprise Service Bus (ESB) MULE 8COI Identity Management & Federation COmanage 7COI Identity Management & Federation Shibboleth 9COI Identity Management & Federation GridShib 7COI IdM, Governance Globus Toolkit 9COI Identity Management & Federation MyProxy 9COI Identity Management & Federation ProtectNetwork 9COI Policy Management & Governance Sun XACML Implementation
Technology Catalog
Subsystem Functional Component Technology TRLAnalysis & Synthesis Analysis Application/Language Matlab 9Analysis & Synthesis Workflow Execution Engine Kepler 6Analysis & Synthesis Meta-Workflow Resource Mapper Pegasus 9Analysis & Synthesis Visualization Toolkit VTK 9Analysis & Synthesis Graphics Engine OpenSceneGraph 9Analysis & Synthesis Visualization Application IDV 9Analysis & Synthesis Spatial Display Application OSSIM Planet 7Analysis & Synthesis Spatial Display Application GoogleEarth & Maps 9Data Management Dataset Catalog THREDDS 7Data Management Semantic Framework VSTO Semantic Framework 8Data Management Semantic Query ESG Facetted Search 8Data Management Data Integration with Applications NetCDF lib 8Data Management Data Integration with Applications Matlab lib for OpenDAP access 8Data Management Dataset Management & Distribution OpenDAP Hyrax Server 7Data Management Dataset Preservation iRODS 7Data Management General Purpose Database MySQL cluster 9Planning & Prosecution Resource Planner ASPEN 9Planning & Prosecution Controller CASPER 9Planning & Prosecution Autonomous Comm Middleware MOOS 8Planning & Prosecution Autonomous Helm MOOS-IvP Helm 8Sensing & Acquisition Virtual Resource Framework IaaS 6Sensing & Acquisition Instrument Drivers Antelope ORB 9Sensing & Acquisition Instrument Drivers SIAM 9Sensing & Acquisition Instrument Drivers PUCK 8Sensing & Acquisition Resource Monitoring Application Intermapper 9CEI Grid Scheduler Condor 9CEI Cloud Computing Environment Amazon EC2 9CEI Cloud Computing Environment Nimbus 8CEI VM Packaging & Distribution Elastic Server (CohesiveFT) 8CEI Virtualization Layer Software Individual virtual-ization vendors, e.g., Amazon AMI)9COI Message Broker System RabbitMQ 8COI IOC Container Spring 9COI Enterprise Service Bus (ESB) MULE 8COI Identity Management & Federation COmanage 7COI Identity Management & Federation Shibboleth 9COI Identity Management & Federation GridShib 7COI IdM, Governance Globus Toolkit 9COI Identity Management & Federation MyProxy 9COI Identity Management & Federation ProtectNetwork 9COI Policy Management & Governance Sun XACML Implementation
23
Thanks !
Subsystem
• Marine Observatory Facility – Observatory Management– Instrument Management & Instrument Activation Services– Marine Resource Scheduling for Power, Bandwidth, etc.– Instrument Resource Repository
• Marine Infrastructure Interface– Instrument and Platform Agents– Instrument Direct Access
• Data Acquisition and Processing– Raw Data Acquisition, Buffering, Metadata association and Persistence– Data Processing, QA/QC, Calibration & Validation– Data Product Generation, Publication & Persistence– Data Process Repository – Data Product Catalog & Repository
Sensing & AcquisitionSensing & Acquisition
Data Management
Planning & Prosecution
Analysis & Synthesis
Common Execution Infrastructure
Common Operating Infrastructure
Capability Container
Subsystem
• Data and Information Access– Search & Navigation– External observatory access (IOOS, Neptune Canada, …)
• Transformation and Mediation– Attribution & Association– Aggregation– Syntactical Transformation– Ontology-based mediation between vocabularies
• Dynamic Data/Information Distribution– Persistent Archive– Information Catalog & Repository
Sensing & Acquisition
Data Management
Planning & Prosecution
Analysis & Synthesis
Common Execution Infrastructure
Common Operating Infrastructure
Capability Container
Data Management
Subsystem
• Laboratory & Classroom Facility– Interactive Analysis and Visualization– Workflow Management– Event Detection
• Numerical Model Integration– Data Assimilation– Model Development, Testing & Validation– Model Parameterization & Execution– Model Catalog & Repository
Sensing & Acquisition
Data Management
Planning & Prosecution
Analysis & Synthesis
Common Execution Infrastructure
Common Operating Infrastructure
Capability Container
Analysis & Synthesis
Subsystem
• Interactive Observatory Facility– Observation Planning– Mission Coordination and Asset Safety
Protection– Mission Catalog & Repository– Mission Simulator Framework
• Event Response Services
• Adaptive Observations
• Portable Control Software
Sensing & Acquisition
Data Management
Planning & Prosecution
Analysis & Synthesis
Common Execution Infrastructure
Common Operating Infrastructure
Capability Container
Planning & Prosecution
Subsystem
• Computing Management– Resource Provisioning and Management– Execution Engine Catalog & Repository– Process Management– Process Catalog & Repository
• Elastic Computing
• Integration with National Computing Infrastructure
Sensing & Acquisition
Data Management
Planning & Prosecution
Analysis & Synthesis
Common Execution Infrastructure
Common Operating Infrastructure
Capability Container
Common Execution Infrastructure
Subsystem
• Operating and Integration Framework– Communication Services– Service Framework– Presentation Framework
• Resource Lifecycle Management
• Security and Policy Management– Identity Management & Authentication – Policy Authoring, Management & Governance
• Facility Services
Sensing & Acquisition
Data Management
Planning & Prosecution
Analysis & Synthesis
Common Execution Infrastructure
Common Operating Infrastructure
Capability Container
Common Operating Infrastructure
Integration StrategyThe CI’s Common Operating Infrastructure mitigates technology risk, maintainability and scalability issues through an integration strategy for application and resource integration:
1. Service-oriented, message-based integration infrastructure • Reduces inter-dependencies between technologies
2. Virtualization of computing and storage• Enables scalability meeting demand
3. Flexible and dynamic packaging and deployment• Enables CI deployment where needed
4. Multi-facility concept based on agreements• Enables new facilities to join the OOI network