D u k e S y s t e m s

Virtualizing, Sharing, Interconnecting

Part 2: servers and pipes

Jeff ChaseDept. of Computer Science

Duke University

NSF CIO Meeting, Boston, July 12, 2012

A CIO’s view?

GENI

research computing

Everything

everything else

Drawing is not to scale

A broader view of GENI

• GENI is:– a constituency demanding attention;

– a process that can help to meet the needs of other constituencies;

– a bundle of technologies coming to campus.

• Some are already there.

• GENI engages key technologies that can help give campus users what they want.– virtualizing, sharing, and interconnecting

– infrastructure as a distributed service

Sponsored by the National Science Foundation 4July 10, 2012

GENI Portal Home Page

Constructing “slices”

• I like to use TinkerToys as a metaphor for creating a slice in GENI.

• The parts are virtual infrastructure resources: compute, networking, storage, etc.

• Parts come in many types, shapes, sizes.

• Parts interconnect in various ways. • We combine them to create useful

built-to-order assemblies.• Some parts are programmable.• Where do the parts come from?

ExoGENI Racks

• Packaged infrastructure pod– servers, network, storage

– off-the-shelf cloud software (‘exo’)

– GENI-enabled ‘special sauce’

• Cookie-cutter deployment – funded @14 campuses

– linked for sharing, “plug and play”

– multiple use

Open Resource Control Architecture

ExoGENI software structure

Executive summary

Virtualization Layer

Physical: metal and glass

Orchestration Service

GENI API ... API

Cloud Service

Stuff we build(automation)

Standard stuff you need anyway

Competing Racks “brands”

• InstaGENI– Emulab/PlanetLab

– lightweight VMs (vservers)

– bare-metal provisioning

– HP is an engaged sponsor

• ExoGENI– ORCA

– off-the-shelf cloud software

– hypervisor VMs (KVM) or bare-metal

– IBM is an engaged vendor

EC2 The canonical public cloud

Virtual Appliance

Image

Infrastructure as a Service (IaaS)“Consumers of IaaS have access to virtual computers, network-accessible storage, network infrastructure components, and other fundamental computing resources…and are billed according to the amount or duration of the resources consumed.”

Client Server(s)

Cloud > server-based computing

• Client/server model (1980s - )

• Now called Software-as-a-Service (SaaS)

Cloud Provider(s)

Host

GuestClient Service

Host/guest model

• Service is hosted by a third party.– flexible programming model

– cloud APIs for service to allocate/link resources

– on-demand: pay as you grow

OS

VMM

Physical

Platform

Client Service

IaaS: infrastructure services

Deployment of private clouds is growing rapidly w/ open IaaS cloud software.

Hosting performance and isolation is determined by virtualization layer

Virtual machines: VMware, KVM, etc.

A cloud solution for your campus?

OS

VMM (optional)

Physical

Platform

Client Service

PaaS cloud services define the high-level programming models, e.g., for clusters or specific application classes.

PaaS: platform services

Hadoop, grids,batch job services, etc. can also be viewed as PaaS category.

Note: can deploy them over IaaS.

OpenStack, the Cloud Operating SystemManagement Layer That Adds Automation & Control

[Anthony Young @ Rackspace]

Managing images

• “Let a thousand flowers bloom.”

• Curated image collections are needed!

• University IT can help.

• “Virtual appliance marketplace”

Connectivity: the missing link

Cloud Providers

Virtual Compute and Storage Infrastructure

Transport Network Providers

Cloud APIs (Amazon EC2 ..)Cloud APIs (Amazon EC2 ..) Dynamic circuit APIs (NLR Sherpa, DOE OSCARS, I2 ION, OGF NSI …)

Virtual Network Infrastructure

Linking clouds with L2 circuits

c1 c2CA B

A B

logical pipe (path)

cross-domain link

circuit providers

cloud sites

Campus clouds can serve as on-ramps to

national fabrics.

Circuit stitching

node

Produce/consume VLAN tag

Extends OpenStack and Euca to configure

virtual NICs and attach VLANs.

Connect adjacent circuits at network exchanges

(e.g., StarLight).

The last-hop provider to the

cloud site is your campus or RON.

SC11 Demo: Solar Fuels Workflow

ExoGENI: recap• ExoGENI is a network of standard OpenStack cloud sites

deployed (deploying) at campuses.– Initial sites centrally managed from RENCI, other providers may join and

advertise portions of their resources.

• Layered orchestration software (ORCA) manages multi-cloud slices and integrates with GENI.– Proxies GENI APIs, checks identity/authorization.

• Circuit backplane for L2 network connectivity.– By agreement with circuit providers....

• Configurable/flexible L3 connectivity.– “Easy button” to configure IP network within slice.

– Host campuses may offer L3 connectivity to slices.

Cloud-Based Credential Store

IdP

Issue user credentials

PA

Create project

SA

Registeruser

Issue project x

credentials

Create slice in x

Issueslice s

credentials

Create sliver in s

1

3

52 4

Delegate

Credentials: who has access?

IdP

Issue user credentials Users have

roles e.g., student, faculty.

Registeruser

user registered

GENI uses Shibboleth IdPs

IdP.geniUserTIdP.studentT

IdP.enrolled(CS-114)T

IdP.geniUserDIdP.facultyD

• An IdP asserts facts about users.

• User attributes may include inCommon attributes harvested through indirect delegation to Shibboleth IdPs.

• These attributes may have parameters with simple values (strings or numbers).

D

T

Please work with inCommon and release IdP attributes to GENI!

Trust management: generalizing PKI• An entity A delegates trust to another by endorsing its

public key for possession of an attribute or role.

• The delegation is a fact written as a logic statement and issued in a credential signed by A.

• Other entities reason from these facts according to their own policy rules, which are declared in logic.

• Policy rules may also be signed and transmitted.

trustsA B

A.trusts B CertificateTerm of validity

Issuer’s name (or key)

Signature

Payload: statement

Summary• GENI is about incorporating new foundational

infrastructure into campuses.– general-purpose multi-use

– best-of-breed off-the-shelf

• GENI is automation for this infrastructure.– ease of use, power, flexibility, safety

– Many others are working on these problems.

– GENI is the focal point in the academic space.

• Rolling deployment, ramping up....

• Best practices @ central IT will help!