OGSA-DAI Open Grid Services Architecture – Data Access and Integration
-
Upload
debra-cotton -
Category
Documents
-
view
50 -
download
1
description
Transcript of OGSA-DAI Open Grid Services Architecture – Data Access and Integration
Description and Aims
OGSA–DAIProvide a uniform access framework for heterogeneous data resources on the GridData resources:
Relational databases XML collections Can widen scope, e.g. files, any data source/sink
Middleware which:Reduces development cost of data centric Grid applicationsFacilitates Grid data centric application developmentFacilitates data integrationIncreased collaboration
Status
Functional Scope defined goalsHad a 66 point functional scope for Phase IIAll MUSTs/SHOULDs achieved except: Statement Metadata (partially complete)
– We do not list all SQL operations a GDS supports
MAYs completed: Transforming application data (compression, XSLT) Caching of data Block transfer of data Scripting (though we no longer call it that) Persistent components
Workplan
All workpackages and deliverables completedThe project lasted 92 (78 + 14) weeks.Over the 92 weeks (4/2/02 to 31/10/03):
138 SM of effort at EPCC were planned140.2 SM were expended1% overrun
Workpackage (WP) Status
1. Programme Management 1.1 End of Phase 2 review D1.1 End of Phase 2 Report
FINISHED OracleComplete
2. Architecture 2.1 High Level Design D2.1 High level system architecture Due M3, 31/12/2002, delivered 25/11/2002 2.2 Architectural definition D2.2 Architecture framework definition Due M3, 31/12/2002, delivered 7/11/2002
FINISHEDEPCC/IBMCompleteEPCC/IBM/NeSC/UoMComplete
3. Development 3.1 Detailed Component design D3.1.1 Release design documentation (Release 1) D3.1.2 Release design documentation (Release 2) D3.1.3 Release design documentation (Release 3)3.2 Component implementation D3.2.1 Release implementation code (Release 1) D3.2.2 Release implementation code (Release 2) D3.2.3 Release implementation code (Release 3)3.3 Write documentation D3.3.1 Release system and user documentation (Release 1) D3.3.2 Release system and user documentation (Release 2) D3.3.3 Release system and user documentation (Release 3)3.4 Develop test strategy and test suites D3.4 Release test strategy and test suites3.5 Perform tests D3.5.1 Documented test results (Release 1) D3.5.2 Documented test results (Release 2) D3.5.3 Documented test results (Release 3)
FINISHEDEPCC/IBMCompleteCompleteCompleteEPCC/IBMCompleteCompleteCompleteEPCC/IBMCompleteCompleteCompleteEPCC/IBMCompleteEPCC/IBMCompleteCompleteComplete
4. Distributed Query Processing4.1 Revise DQP model D4.1 Revised DQP model4.2 Develop prototype D4.2 Distributed query prototype
FINISHEDUoM/UoNCompleteUoM/UoNComplete
5. Release Management Complete (Phase 2)
Releases
Releases added functionality in staged deliveriesKept on target
Max slippage was 2 weeks due to GGF
Made available through project website and GTR
http://www.ogsadai.org.ukhttp://gtr.globus.org
Early adopters had early access to release candidates1199 downloads at 31st Oct 2003
50% from UK
Release Release Date
Release 3.0.2 26/09/03
Release 3 31/07/03
Release 2 interim
11/06/03
Release 2 15/04/03
Release 1 interim
28/02/03
Release 1 15/01/03
Technical Achievements
Grid Data ServicePerform documents allow for powerful “scripting”
Composition of requests (encapsulation of activities)
Activity Framework easily extended by developersVariety of delivery/upload mechanisms
SOAP/HTTP, GridFTP, GDT
Can achieve complex composition patterns e.g. distributed queries using temporary tables
Grid Data Service FactorySimple to configureSupported databases:
MySQL, DB2, Oracle, XIndice
Other “working” databases: SQL Server, Postgres, Access (via JDBC/ODBC)
DAI Service Group RegistryFramework for service discovery
Dissemination (1)
Selected Presentations (EPCC during Phase II only)10th Anniversary of Poznan Supercomputing Centre, October 24, 2003Designing and Building Grid Services Workshop, Chicago, October 8, 2003Glasgow Kelvin Hub opening, September 17, 2003All Hands presentations and demonstrations, Nottingham, September 2-4, 2003DAIS F2F, NeSC, August 21 – 22, 2003ASTAR Visit, NeSC, July 14, 2003Virtual Observatory as a Data Grid, NeSC, June 30 – July 2, 2003Geoffrey Fox visit, EPCC, April 4, 2003NeSC Review, NeSC, March 28, 2003OGSA-DAI / Informatics meeting, NeSC, March 27, 2003OGSA Experiences Panel, GGF7 Tokyo, March 4-7, 2003NeSC Open Day, NeSC, January 17, 2003
Dissemination (2)
PostersNeSC Review, NeSC, September 30, 2003UK e-Science All Hands, Nottingham, September 2-4, 2003 GlobusWorld January 13 – 17, 2003
Publications through GGFDAIS – File Access, September 19, 2003DAIS – Grid Data Service Specification, September 19, 2003DAIS – Relational Specialisation, September 19, 2003DAIS – XML Specialisation, September 19, 2003DFDL – Basic Structures Ontology, August 5, 2003DFDL – Primitive Type Ontology, August 5, 2003DFDL – Structural Description, August 5, 2003DFDL – XML Representation, August 5, 2003DFDL – Primer, June 4, 2003
Other notable publications“Grid Security for Dummies”, available from OGSA-DAI website, October 29, 2003
Training
Course and tutorials by EPCC staff were run at:eScience Summer School, NeSC, September 29 – October 3rd, 2003International Summer School on Grid Computing, Naples, July 13 – 25, 2003OGSA-DAI Training Course, NeSC, April 22, 2003OGSA-DAI Tutorial, GGF7 Tokyo, March 4, 2003 Creating Grid Services using GT3 and Java course, NeSC, February 24, 2003OGSA-DAI Training Course, NeSC, February 11, 2003OGSA-DAI Training Course, NeSC, January 8, 2003
“Show and tell” method of increasing exposure
Support
Support for OGSA-DAI through Grid Support Centre from Release 2
Very useful to encourage user take-up
Query deskRegular stream of queries
Active user list“Power Users” submitted answers to other users questionsDiscussed innovative ways of extending OGSA-DAI
Exploitation
Projects started at both EPCC/NeSC and IBM using OGSA-DAI:
eDiaMoNDFirstDIGINWABRIDGESEdSkyQueryG
Many more projects using OGSA-DAIPresentations, Visits and Training
have been vital to uptake of OGSA-DAI
Future plans
Work continues under the DAIT projectResearch and develop OGSA-DAI softwareImprove performance and scalabilityLiase with technology adopters
Make sure OGSA-DAI works for them
Liase with Globus Globus Alliance OGSA-DAI also distributed through Globus Toolkit
Continue standardisation process through DAIS
We’ve done Data Access… now it’s time for Data Integration!
Project Participants
EPCCAli Anjomshoaa, Mario Antonioletti, Rob Baxter, Neil Chue Hong, Ally Hume, Mike Jackson, Amy Krause, Jeremy Nowell, Charaka Palansuriya, Tom Sugden, Martin Westhead
IBM UKBrian Collins, Simon Laws, Andrew Borley, James Magowan, Neil Hardman, George Hicken, Manfred Oevers, Alan Knox
IBM USSusan Malaika, Inderpal Narang
NeSCMalcolm Atkinson
Oracle UKDave Pearson
University of ManchesterNorman Paton, Nedim Alpdemir
University of NewcastlePaul Watson, Arijit Mukherjee