Interoperability Grids, Clouds and Collaboratories Ruth Pordes
-
Upload
sybill-brooks -
Category
Documents
-
view
31 -
download
0
description
Transcript of Interoperability Grids, Clouds and Collaboratories Ruth Pordes
![Page 1: Interoperability Grids, Clouds and Collaboratories Ruth Pordes](https://reader030.fdocuments.in/reader030/viewer/2022032805/56813347550346895d9a41c4/html5/thumbnails/1.jpg)
Interoperability@Chep2009
Interoperability Interoperability
Grids, Clouds and Collaboratories Grids, Clouds and Collaboratories
Ruth Pordes
Executive Director Open Science Grid, Fermilab
![Page 2: Interoperability Grids, Clouds and Collaboratories Ruth Pordes](https://reader030.fdocuments.in/reader030/viewer/2022032805/56813347550346895d9a41c4/html5/thumbnails/2.jpg)
Interoperability@Chep2009
(Inter-)National
Community Campus
Our (e.g. OSG’s) Past – the 3 types of grids
![Page 3: Interoperability Grids, Clouds and Collaboratories Ruth Pordes](https://reader030.fdocuments.in/reader030/viewer/2022032805/56813347550346895d9a41c4/html5/thumbnails/3.jpg)
Interoperability@Chep2009
Clouds
National & International
Communities & ad-hoc VOs Campuses
& Regions
Networks
And now
HPC Digital Libraries
the software is inevitably different & changing, with different external semantics.
![Page 4: Interoperability Grids, Clouds and Collaboratories Ruth Pordes](https://reader030.fdocuments.in/reader030/viewer/2022032805/56813347550346895d9a41c4/html5/thumbnails/4.jpg)
Interoperability@Chep2009
National Science Foundation Landscape
4
![Page 5: Interoperability Grids, Clouds and Collaboratories Ruth Pordes](https://reader030.fdocuments.in/reader030/viewer/2022032805/56813347550346895d9a41c4/html5/thumbnails/5.jpg)
Interoperability@Chep2009
We must integrate & interface many more types of resource services
- Cross-campus infrastructures & community grids.- Multi-organization grids & multi-site uniform clusters.- Unique leadership class, peta-scale, exascale & the
ubiquitous laptops- Huge shared data repositories &
large local data silos.- Shared services & encapsulated
virtual environments.- Commercial & community clouds.
And these systems also overlap.
5
![Page 6: Interoperability Grids, Clouds and Collaboratories Ruth Pordes](https://reader030.fdocuments.in/reader030/viewer/2022032805/56813347550346895d9a41c4/html5/thumbnails/6.jpg)
Interoperability@Chep2009
People work across them all
Clouds
National & International
Communities & ad-hoc VOs Campuses
& Regions
Networks
HPCDigital
Libraries
![Page 7: Interoperability Grids, Clouds and Collaboratories Ruth Pordes](https://reader030.fdocuments.in/reader030/viewer/2022032805/56813347550346895d9a41c4/html5/thumbnails/7.jpg)
Interoperability@Chep2009 7
Bridges (aka gateways, adaptors, routers) connect the diverse resources layered on the network fabric.
![Page 8: Interoperability Grids, Clouds and Collaboratories Ruth Pordes](https://reader030.fdocuments.in/reader030/viewer/2022032805/56813347550346895d9a41c4/html5/thumbnails/8.jpg)
Interoperability@Chep2009 8
With Capability
![Page 9: Interoperability Grids, Clouds and Collaboratories Ruth Pordes](https://reader030.fdocuments.in/reader030/viewer/2022032805/56813347550346895d9a41c4/html5/thumbnails/9.jpg)
Interoperability@Chep2009
Why include Collaboratories as well as Virtual Organizations ?
“center without wallscenter without walls, in which the nation’s researchers can perform their research without regard to physical location, interacting with colleagues, accessing instrumentation, sharing data and computational resources, [and] accessing information in digital libraries” (Wulf, 1989).
“a system which combines the interests of the scientific community at combines the interests of the scientific community at large with those of the computer science and engineering community large with those of the computer science and engineering community to create integrated, tool-oriented computing and communication systems to support scientific collaboration” (Bly, 1998, p. 31).
experimental and empirical research environment in which scientists work and communicate with each other to design systems, participate in collaborative science, and conduct experiments to evaluate and improve systems.and conduct experiments to evaluate and improve systems.
“.a new networked organizational form that also includes social processes; collaboration techniques; formal and informal communication; and agreement on norms, principles, values, and rules” agreement on norms, principles, values, and rules” (Cogburn, 2003, p. 86).
9
![Page 10: Interoperability Grids, Clouds and Collaboratories Ruth Pordes](https://reader030.fdocuments.in/reader030/viewer/2022032805/56813347550346895d9a41c4/html5/thumbnails/10.jpg)
Interoperability@Chep2009
Goals remain - Supporting People to
Do their own scholarship Do their own scholarship (research, science, education...)- Have local access to their local system using local methods.- Access unique computing capabilities remote from their site.
and
Collaborate with others - Collaborate with others - both ad-hoc and sustainedboth ad-hoc and sustained- Geographically and/or organizationally distributed.- Use data from same or different intrument(s).- Pool resources.
- Share and reuse their resources, software & services.- Combine their knowledge.
10
![Page 11: Interoperability Grids, Clouds and Collaboratories Ruth Pordes](https://reader030.fdocuments.in/reader030/viewer/2022032805/56813347550346895d9a41c4/html5/thumbnails/11.jpg)
Interoperability@Chep2009
11
Collaboratory
Workflow
![Page 12: Interoperability Grids, Clouds and Collaboratories Ruth Pordes](https://reader030.fdocuments.in/reader030/viewer/2022032805/56813347550346895d9a41c4/html5/thumbnails/12.jpg)
Interoperability@Chep2009 12
bridges integrate the piecesand workflows manage the use
![Page 13: Interoperability Grids, Clouds and Collaboratories Ruth Pordes](https://reader030.fdocuments.in/reader030/viewer/2022032805/56813347550346895d9a41c4/html5/thumbnails/13.jpg)
Interoperability@Chep2009
of course communities also work together
Clouds
National & International
Communities & ad-hoc VOs Campuses
& Regions
Networks
HPCDigital
Libraries
![Page 14: Interoperability Grids, Clouds and Collaboratories Ruth Pordes](https://reader030.fdocuments.in/reader030/viewer/2022032805/56813347550346895d9a41c4/html5/thumbnails/14.jpg)
Interoperability@Chep200914
Our Original Vision
![Page 15: Interoperability Grids, Clouds and Collaboratories Ruth Pordes](https://reader030.fdocuments.in/reader030/viewer/2022032805/56813347550346895d9a41c4/html5/thumbnails/15.jpg)
Interoperability@Chep2009
e.g. CDF and D0 combining results.
e.g.BioEnergy Centers collaborating to share data.
![Page 16: Interoperability Grids, Clouds and Collaboratories Ruth Pordes](https://reader030.fdocuments.in/reader030/viewer/2022032805/56813347550346895d9a41c4/html5/thumbnails/16.jpg)
Interoperability@Chep2009
e.g.
Climate applications using both weather data and geophysical history.
Material scientists using data across X-ray, neutron and other probes
![Page 17: Interoperability Grids, Clouds and Collaboratories Ruth Pordes](https://reader030.fdocuments.in/reader030/viewer/2022032805/56813347550346895d9a41c4/html5/thumbnails/17.jpg)
Interoperability@Chep2009
Bridge capabilities
- Transform, interpret & translate.- Act as borders.
17
![Page 18: Interoperability Grids, Clouds and Collaboratories Ruth Pordes](https://reader030.fdocuments.in/reader030/viewer/2022032805/56813347550346895d9a41c4/html5/thumbnails/18.jpg)
Interoperability@Chep2009
Bridge capabilities
- Transform, interpret & translate.- Act as borders.
- Have multiple end points.- Control the flow and queue the traffic.
18
![Page 19: Interoperability Grids, Clouds and Collaboratories Ruth Pordes](https://reader030.fdocuments.in/reader030/viewer/2022032805/56813347550346895d9a41c4/html5/thumbnails/19.jpg)
Interoperability@Chep2009
Bridge capabilities
- Transform, interpret & translate.- Act as borders.- Have multiple ends.- Control the flow and queue the traffic.
- Filter access and identify problem requests.- Include information kiosks that provide maps,
dictionaries etc.
19
![Page 20: Interoperability Grids, Clouds and Collaboratories Ruth Pordes](https://reader030.fdocuments.in/reader030/viewer/2022032805/56813347550346895d9a41c4/html5/thumbnails/20.jpg)
Interoperability@Chep2009
Bridge capabilities
- Transform, interpret & translate.- Act as borders.- Have multiple ends.- Control the flow and queue the traffic.- Filter access and identify problem requests.- Can have information kiosks that provide maps,
dictionaries.
- Allow each side to evolve independently.- Apply and check for policy.
20
![Page 21: Interoperability Grids, Clouds and Collaboratories Ruth Pordes](https://reader030.fdocuments.in/reader030/viewer/2022032805/56813347550346895d9a41c4/html5/thumbnails/21.jpg)
Interoperability@Chep2009
Bridge capabilities
21
be opened and closed
![Page 22: Interoperability Grids, Clouds and Collaboratories Ruth Pordes](https://reader030.fdocuments.in/reader030/viewer/2022032805/56813347550346895d9a41c4/html5/thumbnails/22.jpg)
Interoperability@Chep2009
Bridge capabilities
22
be opened and closed
offer further discussion of boundary and interaction betweenthe network and middleware layers?
![Page 23: Interoperability Grids, Clouds and Collaboratories Ruth Pordes](https://reader030.fdocuments.in/reader030/viewer/2022032805/56813347550346895d9a41c4/html5/thumbnails/23.jpg)
Interoperability@Chep2009
Some examples: jobs, information, management
- Advanced Resource Connector (ARC) Compute Element gateway Multiplexing submissions from a variety of remote execution
clients into a common queue and ARC submission framework.
Centralized data management across all ARC sites.
23
![Page 24: Interoperability Grids, Clouds and Collaboratories Ruth Pordes](https://reader030.fdocuments.in/reader030/viewer/2022032805/56813347550346895d9a41c4/html5/thumbnails/24.jpg)
Interoperability@Chep2009
Information Service Adaptors
- Apply policy filters as well as translate- Information can be injected at different stages in the flow.
24
![Page 25: Interoperability Grids, Clouds and Collaboratories Ruth Pordes](https://reader030.fdocuments.in/reader030/viewer/2022032805/56813347550346895d9a41c4/html5/thumbnails/25.jpg)
Interoperability@Chep2009
Condor job router
– uploading jobs between different Security infrastructures .
- executing locally or remote as needed.
25
![Page 26: Interoperability Grids, Clouds and Collaboratories Ruth Pordes](https://reader030.fdocuments.in/reader030/viewer/2022032805/56813347550346895d9a41c4/html5/thumbnails/26.jpg)
Interoperability@Chep2009
FermiGrid OSG-TeraGrid gateway Maintains state in “osg space” while dispatching work and
data to TeraGrid. Will map users in VOs to TeraGrid allocation policies. Collects MDS4 into common pool with LDAP information.
End-point Identity (Shibboleth, Bridge-CAs) -X509 Allow application of varying security policy
Virtual Machine/Cloud administration services
Experiment submit-agents
Portals and Hubs
26
![Page 27: Interoperability Grids, Clouds and Collaboratories Ruth Pordes](https://reader030.fdocuments.in/reader030/viewer/2022032805/56813347550346895d9a41c4/html5/thumbnails/27.jpg)
Interoperability@Chep2009
Human – Software Bridges?
27
![Page 28: Interoperability Grids, Clouds and Collaboratories Ruth Pordes](https://reader030.fdocuments.in/reader030/viewer/2022032805/56813347550346895d9a41c4/html5/thumbnails/28.jpg)
Interoperability@Chep2009
Bridges are part of the interoperability solution
Integration to adapt to the most cost-effective computing, storage, software and services that are out there.
Agility to follow several cycles of technology buzz-words simultaneously, mix and match.
A natural place to handle inter-domain differences beyond APIs.
28
![Page 29: Interoperability Grids, Clouds and Collaboratories Ruth Pordes](https://reader030.fdocuments.in/reader030/viewer/2022032805/56813347550346895d9a41c4/html5/thumbnails/29.jpg)
Interoperability@Chep2009 29
it’s not just that bridges are topical..