Post on 04-Jan-2016
Grid of Grids Information Management
Kick off Meeting
AnabasJune 30 2006
ACKNOWLEDGMENTS
We are grateful to:
• Bill McQuay, • Raymund Garcia•The Air Force Research Laboratory - AFRL IFSD• Indiana University CGL (Subcontractor Partner)• Ball Aerospace (Subcontractor Partner)
General Goal• Build Net Centric Core Enterprise Services
in fashion compatible with GGF/OGF and Industry
• Add key additional services including those for sensors and GIS
• Support Systems of Systems by federating Grids of Grids supporting a heterogeneous software production model allowing DoD greater sustainability and choice of vendors
• Build tool to allow easy construction of Grids of Grids
Gerstner and Commercialization of GoG Architecture• Reference [1]: Who Says Elephants Can't Dance, Louis Gerstner, Jr., 2002 is practically
a primer on IBM's vision and strategy to re-gain its influence to setting IT industry agenda.
• "The industry's disaggregation into thousands of niche players would make IT integration services a huge growth segment of the industry." [1] – Grid of Grids builds the information management infrastructure for efficient
integration of components.• "Over the next decade, customers would increasingly value companies that could
provide solutions - solutions that integrated technology from various suppliers and more importantly, integrated technology into the processes of an enterprise." and "The two companies that have enjoyed the highest market valuation in the IT industry in over much of the last decade have been component manufacturers -- Intel and Microsoft. There is no doubt that a strategy built around providing fundamental building blocks of the computing infrastructure has proven to be extremely successful in this industry." [1] – Grid of Grids has huge commercial potential by offering both the integration
infrastructure for component manufacturers and providing Network-Centric Enterprise Services as essential building blocks .
• "The dazzlingly wired-world would also change the course of computing in profound ways. For one thing, it was virtually certain that world would be built on open industry standards. There would be no other way to fulfill the promise of ubiquitous connections among all the businesses users, devices, and systems that would participate in a truly networked world. If that standards-based world came to pass, it would represent a major shift in the prevailing competitive landscape." [1] – Grid of Grids information management with its foundation on GGF/OGF open
standards is a key to integration.
Technical Advisory Board• Build a group of some 6-10 knowledgeable people
to guide project • Mainly from Industry including Boeing, Lockheed
Martin, Ball Aerospace, Aerospace Corporation, Mitre, DISA, and the core technology companies (IBM, Microsoft)
• Use to guide both technology and commercialization strategy
• Meet F2F, email and telecons.• Brings significant external expertise to Grid of
Grids team, and visibility of Grid of Grids to other world-renowned institutions.
Database
SS
SS
SS
SS
SS
SS
SS
SS
SS
SS
FS
FS
FS
FS
FS
FS
FS
FS FS
FS
FS
FS
FS
FS
FS
FS
FS FS
FS
FS
PortalFS
OS
OS
OS
OS
OS
OS
OS
OS
OS
OS
OS
OS
MD
MD
MD
MD
MD
MD
MD
MD
MD
MetaDataFilter Service
Sensor Service
OtherService
AnotherGrid
Raw Data Data Information Knowledge Wisdom
Decisions
SS
SS
AnotherService
AnotherService
SSAnother
Grid SS
AnotherGrid
SS
SS
SS
SS
SS
SS
SS
SS
FS
SOAP Messages
Portal
The Grid and Web Service Institutional Hierarchy
OGSA GS-*and some WS-*GGF/W3C/….XGSP (Collab)
WS-* fromOASIS/W3C/Industry
Apache Axis.NET etc.
Must set standards to get interoperability
2: System Services and Features(WS-* from OASIS/W3C/Industry)
Handlers like WS-RM, Security, UDDI Registry
3: Generally Useful Services and Features(OGSA and other GGF, W3C) Such as
“Collaborate”, “Access a Database” or “Submit a Job”
4: Application or Community of Interest (CoI)Specific Services such as “Map Services”, “Run
BLAST” or “Simulate a Missile”
1: Container and Run Time (Hosting) Environment (Apache Axis, .NET etc.)
XBMLXTCE VOTABLECMLCellML
The Ten areas covered by the 60 core WS-* Specifications
WS-* Specification Area Examples
1: Core Service Model XML, WSDL, SOAP
2: Service Internet WS-Addressing, WS-MessageDelivery; Reliable Messaging WSRM; Efficient Messaging MOTM
3: Notification WS-Notification, WS-Eventing (Publish-Subscribe)
4: Workflow and Transactions BPEL, WS-Choreography, WS-Coordination
5: Security WS-Security, WS-Trust, WS-Federation, SAML, WS-SecureConversation
6: Service Discovery UDDI, WS-Discovery
7: System Metadata and State WSRF, WS-MetadataExchange, WS-Context
8: Management WSDM, WS-Management, WS-Transfer
9: Policy and Agreements WS-Policy, WS-Agreement
10: Portals and User Interfaces WSRP (Remote Portlets)
Activities in Global Grid Forum Working Groups
GGF Area GS-* and OGSA Standards Activities
1: Architecture High Level Resource/Service Naming (level 2 of slide 6),Integrated Grid Architecture
2: Applications Software Interfaces to Grid, Grid Remote Procedure Call, Checkpointing and Recovery, Interoperability to Job Submittal services, Information Retrieval,
3: Compute Job Submission, Basic Execution Services, Service Level Agreements for Resource use and reservation, Distributed Scheduling
4: Data Database and File Grid access, Grid FTP, Storage Management, Data replication, Binary data specification and interface, High-level publish/subscribe, Transaction management
5: Infrastructure Network measurements, Role of IPv6 and high performance networking, Data transport
6: Management Resource/Service configuration, deployment and lifetime, Usage records and access, Grid economy model
7: Security Authorization, P2P and Firewall Issues, Trusted Computing
Net-Centric Core Enterprise Services Core Enterprise Services Service Functionality
NCES1: Enterprise Services Management (ESM)
including life-cycle management
NCES2: Information Assurance (IA)/Security
Supports confidentiality, integrity and availability. Implies reliability and autonomic features
NCES3: Messaging Synchronous or asynchronous cases
NCES4: Discovery Searching data and services
NCES5: Mediation Includes translation, aggregation, integration, correlation, fusion, brokering publication, and other transformations for services and data. Possibly agents
NCES6: Collaboration Provision and control of sharing with emphasis on synchronous real-time services
NCES7: User Assistance Includes automated and manual methods of optimizing the user GiG experience (user agent)
NCES8: Storage Retention, organization and disposition of all forms of data
NCES9: Application Provisioning, operations and maintenance of applications.
Produce the Needed Core Services
• We can classify services in many ways and following 2 charts are one way; slightly changed from proposal as NCOW and our work changed a little.
• Green is “in hand”; our team leaders• Orange is “in hand” with outside but available
solutions• Red has problems – Security does not have
industry consensus while current Scheduling work does not address DoD real-time service and network requirements
The Core Features/Service Areas IService or Feature WS-* GS-* NCES
(DoD)Comments
A: Broad Principles
FS1: Use SOA: Service Oriented Arch.
WS1 Core Service Architecture, Build Grids on Web Services. Industry best practice
FS2: Grid of Grids Distinctive Strategy for legacy subsystems and modular architecture
B: Core Services
FS3: Service Internet, Messaging
WS2 NCES3 Streams/Sensors. Team
FS4: Notification WS3 NCES3 JMS, MQSeries.
FS5 Workflow WS4 NCES5 Grid Programming
FS6 : Security WS5 GS7 NCES2 Grid-Shib, Permis Liberty Alliance ...
FS7: Discovery WS6 NCES4 UDDI
FS8: System Metadata & State
WS7 Globus MDSSemantic Grid, WS-Context
FS9: Management WS8 GS6 NCES1 CIM
FS10: Policy WS9 ECS
The Core Feature/Service Areas IIService or Feature WS-* GS-* NCES Comments
B: Core Services (Continued)
FS11: Portals and User assistance
WS10 NCES7 Portlets JSR168, NCES Capability Interfaces
FS12: Computing GS3
FS13: Data and Storage GS4 NCES8 NCOW Data StrategyFederation at data/information layer major research area; CGL leading role
FS14: Information GS4 JBI for DoD, WFS for OGC
FS15: Applications and User Services
GS2 NCES9 Standalone ServicesProxies for jobs
FS16: Resources and Infrastructure
GS5 Ad-hoc networks
FS17: Collaboration and Virtual Organizations
GS7 NCES6 XGSP, Shared Web Service ports
FS18: Scheduling and matching of Services and Resources
GS3 Current work only addresses scheduling “batch jobs”. Need networks and services
Additional Services• Sensors have low level support listed as FS3; higher
level integration using SensorML and Filters well understood. Some work in phase I
• GIS Grid services pioneered by team and already shown in phase I
• Mediation (Interoperability) Services needed to link Grids (defined as a collection of ≥ 1 Services)– Need to generalize existing solutions for Sensor Grids
and for MQSeries-SOAP Mediation– View NaradaBrokering as a SOAP Intermediary
Grid of Grids Building Tool• This will provide a graphical interface to build grids
from existing libraries of Services and Grids• Meta-data (provenance) needs to be specified• Grids (services) need to be linked• This will be built by extending an existing workflow
engine which is aimed at a more tightly couple version of the builder problem– We will consider HPSearch (CGL), Taverna (Open
source from UK OMII), BPEL with user interface (OMII or IU LEAD project)
– Semantic Grid provenance “add-ons”– We have experience with all these approaches
Applications• Indiana University will apply to Earthquake Science
and Emergency Response• Ball Aerospace will apply to Matrix• Anabas will work with Technical Advisory Board to
identify commercialization plan and possible prototype applications in first 12 months
• Anabas will implement appropriately in last half of project
Out of Scope for Phase II• Many areas are still evolving significantly
– Mediation/Interoperation– Security– Scheduling of non-compute Resources– Data/Information Federation– Semantic Grid and management
• We will not test scalability on large number of services, sensors and component Grids
• Integrating legacy systems not addressed• Grid of Grids building tool is “new idea” – can expect will benefit from
further work
Research Tasks
The R&D Effort is divided into five major tasks:
• Task 1: Implementation of Collaboration Grid Middleware
• Task 2: Enhanced NCOW Core Enterprise Services (NCES) with Enterprise Control Services and Metadata Services
• Task 3: Design and implementation of Grid of Grids mediation algorithms and NCOW services
• Task 4: Design and Implementation of Net-Centric Collaboration Grid Builder Tool
• Task 5: Technology Demonstration
Task Organization
1. Requirements2. Analysis3. Design4. Implement5. Test & Measure6. Repeat
Task Assignments
Task Anabas Ball CGLRequirementsCollaboration Grid Middleware (NCGGCM) (P) (S) (S)NCOW, GiG (P) (S)CES+ (11 service layers) (P) (S) (S)GWSA (P) (S) (S)Grid of Grids (GoG) (P) (S) (S)Mediation Algorithms (P) (S) (S)K2 service (S) (P)NCOW services (e.g., JBI publish, etc.) (S) (P)Net-Centric Collaboration Grid Builder Tool (P) (S) (S)CES+ (P) (S) (S)Wizards for each service (P) (S) (S)Collaboration and Messaging (P) (S) (S)GIS, Sensor and additional services (P) (S) (S)GIS and sensor (P) (S) (S)K2 service (S) (P)NCOW services (e.g., JBI publish, etc.) (S) (P)C2IEDM service (S) (P)Demonstration/Commercial Products (C) (C) (S)Crisis Management for Earthquake (C) (S) (C)Customer provided suggestions for DoD product (C) (C)
Task Assignments
Task Anabas Ball CGLResearch and Concept AnalysisCollaboration Grid Middleware (NCGGCM) (P) (S) (S)Grid of Grids (GoG) (P) (S) (S)Net-Centric Collaboration Grid Builder Tool (P) (S) (S)GIS, Sensor and additional services (P) (S) (S)Demonstration/Commercial Products (P) (S) (S)
Task Assignments
Task Anabas Ball CGLDesignCollaboration Grid Middleware (NCGGCM) (P) (S) (S)Grid of Grids (GoG) (P) (S) (S)Mediation Algorithms (P) (S) (S)K2 service (S) (P)NCOW services (e.g., JBI publish, etc.) (S) (P)Net-Centric Collaboration Grid Builder Tool (P) (S) (S)GIS, Sensor and additional services (P) (S) (S)GIS and sensor (P) (S) (S)K2 service (S) (P)NCOW services (e.g., JBI publish, etc.) (S) (P)C2IEDM Services (S) (P)Demonstration/Commercial Products (P) (S)
Task Assignments
Task Anabas Ball CGLImplementCollaboration Grid Middleware (NCGGCM) (P) (S) (S)Grid of Grids (GoG) (P) (S) (S)Mediation Algorithms (P) (S) (S)K2 service (S) (P)NCOW services (e.g., JBI publish, etc.) (S) (P)Net-Centric Collaboration Grid Builder Tool (P) (S) (S)GIS, Sensor and additional services (P) (S) (S)GIS and sensor (P) (S) (S)K2 service (S) (P)NCOW services (e.g., JBI publish, etc.) (S) (P)C2IEDM service (S) (P)Demonstration/Commercial Products (P) (S)
Task Assignments
Task Anabas Ball CGLTesting & EvaluationCollaboration Grid Middleware (NCGGCM) (P) (S) (S)Grid of Grids (GoG) (P) (S) (S)Mediation Algorithms (P) (S) (S)K2 service (S) (P)NCOW services (e.g., JBI publish, etc.) (S) (P)Net-Centric Collaboration Grid Builder Tool (P) (S) (S)GIS, Sensor and additional services (P) (S) (S)GIS and sensor (P) (S) (S)K2 service (S) (P)NCOW services (e.g., JBI publish, etc.) (S) (P)C2IEDM service (S) (P)Demonstration/Commercial Products (P) (S)
Task Assignments
Task Anabas Ball CGLRefine and RefactorCollaboration Grid Middleware (P) (S) (S)Grid of Grids (GoG) (P) (S) (S)Net-Centric Collaboration Grid Builder Tool (P) (S) (S)GIS, Sensor and additional services (P) (S) (S)Demonstration/Commercial Products (P) (S)
Documentation
Monthly Report
Final Report
Reviews &
Demonstrations
Refine & Refactor
Testing & Evaluation
Implement
Design
Research & Analysis
Requirements
242322212019181716151413121110987654321Task and Event
Month from Contract Start
Milestones & Timelines