CC-NIE ADAMANT
description
Transcript of CC-NIE ADAMANT
CC-NIE ADAMANT• Transforming Computational Science with ADAMANT (Adaptive Data-
aware Multi-domain Application Network Topologies)– RENCI/UNC Chapel Hill (Ilia Baldine [email protected])– Duke University (Jeff Chase [email protected])– USC/ISI (Ewa Deelman [email protected])
• An integration award (not a campus infrastructure)• Enable computational workflow-driven science on multi-domain IaaS
– Integrate ORCA resource provisioning with Pegasus WMS– Enable provisioning of compute/storage/network resources by workflows in
response to their needs through application-specific topology embedding– Enable pre-planned movements of data over engineered dynamic
connections between domains in support of computational workflow tasks
Proposed work 1/3• Application-specific IaaS topology embedding
– Select appropriate pools of computational resources based on workflow needs• Include GENI racks, supercomputers, OSG sites as potential sources of computational
resources– Create dynamic on-ramps to move raw data sets into the slice for computation
and move processed data sets for permanent storage• Treat Science DMZ, DYNES, GENI rack, OSG site locations among convenient on-ramp
endpoints– Working with OSG sites in Wisconsin, RENCI
• Use available mechanisms to move data (GridFTP, FTD, iRods)– Integrate resource provisioning logic into Pegasus WMS– Use several driving applications to demonstrate:
• Astronomy• Earth science (CyberShake)• High-throughput sequencing• They have different demands for computations and data movements and are
representative of larger classes of applications
Proposed work 2/3• Automatic address space management and authorization
– Each slice has a dedicated Layer 2 dataplane with its own address space • Typically private, can be public
– Moving data via on-ramps requires automatically managing address spaces between slice and on-ramp touchpoints
– Use OpenFlow and custom controllers as a mechanism to direct and properly authorize traffic moving the data sets between an ephemeral slice and e.g. persistent campus infrastructure• When possible rely on campus OpenFlow deployments as an enabling
capability
Proposed work 3/3
• Bridge Layer 2 air gaps– Whenever a dynamic L2 provider does not extend all
the way to campus (e.g. via a regional), mechanisms must be devised to get around this to support ADAMANT needs
• Enable workflow ensemble management – Computational science more widely relies on ensembles
of workflows for e.g. performing parameter sweeps– Need tools to evaluate resource needs of a workflow
ensemble in order to provision those
SC11 demo of Solar Fuels application