CC-NIE ADAMANT

5
CC-NIE ADAMANT • Transforming Computational Science with ADAMANT (Adaptive Data-aware Multi-domain Application Network Topologies) – RENCI/UNC Chapel Hill (Ilia Baldine [email protected]) – Duke University (Jeff Chase [email protected]) – USC/ISI (Ewa Deelman [email protected]) • An integration award (not a campus infrastructure) • Enable computational workflow-driven science on multi-domain IaaS – Integrate ORCA resource provisioning with Pegasus WMS – Enable provisioning of compute/storage/network resources by workflows in response to their needs through application-specific topology embedding – Enable pre-planned movements of data over engineered dynamic connections between domains in support of computational workflow tasks

description

CC-NIE ADAMANT. Transforming Computational Science with ADAMANT (Adaptive Data-aware Multi-domain Application Network Topologies) RENCI/UNC Chapel Hill (Ilia Baldine [email protected] ) Duke University (Jeff Chase [email protected] ) USC/ISI ( Ewa Deelman [email protected] ) - PowerPoint PPT Presentation

Transcript of CC-NIE ADAMANT

Page 1: CC-NIE  ADAMANT

CC-NIE ADAMANT• Transforming Computational Science with ADAMANT (Adaptive Data-

aware Multi-domain Application Network Topologies)– RENCI/UNC Chapel Hill (Ilia Baldine [email protected])– Duke University (Jeff Chase [email protected])– USC/ISI (Ewa Deelman [email protected])

• An integration award (not a campus infrastructure)• Enable computational workflow-driven science on multi-domain IaaS

– Integrate ORCA resource provisioning with Pegasus WMS– Enable provisioning of compute/storage/network resources by workflows in

response to their needs through application-specific topology embedding– Enable pre-planned movements of data over engineered dynamic

connections between domains in support of computational workflow tasks

Page 2: CC-NIE  ADAMANT

Proposed work 1/3• Application-specific IaaS topology embedding

– Select appropriate pools of computational resources based on workflow needs• Include GENI racks, supercomputers, OSG sites as potential sources of computational

resources– Create dynamic on-ramps to move raw data sets into the slice for computation

and move processed data sets for permanent storage• Treat Science DMZ, DYNES, GENI rack, OSG site locations among convenient on-ramp

endpoints– Working with OSG sites in Wisconsin, RENCI

• Use available mechanisms to move data (GridFTP, FTD, iRods)– Integrate resource provisioning logic into Pegasus WMS– Use several driving applications to demonstrate:

• Astronomy• Earth science (CyberShake)• High-throughput sequencing• They have different demands for computations and data movements and are

representative of larger classes of applications

Page 3: CC-NIE  ADAMANT

Proposed work 2/3• Automatic address space management and authorization

– Each slice has a dedicated Layer 2 dataplane with its own address space • Typically private, can be public

– Moving data via on-ramps requires automatically managing address spaces between slice and on-ramp touchpoints

– Use OpenFlow and custom controllers as a mechanism to direct and properly authorize traffic moving the data sets between an ephemeral slice and e.g. persistent campus infrastructure• When possible rely on campus OpenFlow deployments as an enabling

capability

Page 4: CC-NIE  ADAMANT

Proposed work 3/3

• Bridge Layer 2 air gaps– Whenever a dynamic L2 provider does not extend all

the way to campus (e.g. via a regional), mechanisms must be devised to get around this to support ADAMANT needs

• Enable workflow ensemble management – Computational science more widely relies on ensembles

of workflows for e.g. performing parameter sweeps– Need tools to evaluate resource needs of a workflow

ensemble in order to provision those

Page 5: CC-NIE  ADAMANT

SC11 demo of Solar Fuels application