Post on 23-Feb-2016
description
Rick McGeerChief Scientist, US Ignite
March 17, 2014
The GENI Experiment Engine
A Fast, Simple On-Ramp for GENI
3
“Hello, World” Across GENI in FIVE MINUTES
4
5
Objectives• “Hello, World” Across GENI in < 5 Minutes
– Allocate GEE “Slicelet”– Upload/Deploy Program (one-click)– Execute– Tear Down
• Non-Objective: be all things to all people– Making things fast & easy means giving up some options– Easy to do something simple, go to “full GENI” for complex stuff
• Use as a starter for a “full GENI” slice
6
GENI Experiment Engine Architecture
7
2 Principles, 3 Services• Design Principles
– Don’t develop new GENI facilities: expose existing facilities– Pre-allocate to bring stuff up quickly
• Services– GEE Compute Engine: PaaS “Slicelet” + Deployment service– GEE Storage Engine: Global filesystem across the GENI
Infrastructure– GEE Message Service: Distributed Messaging System across
the GENI Infrastructure
8
GEE Compute Engine• GEE Slicelet
– Network of PlanetLab slivers– Private network between slivers (full port space)– Python VM installed and running in slivers
• Pre-allocated– Access through use-once key pair– Fixed topology, IP addresses, etc.
• Automated Deployment (optional)– Fabric-as-a-Service through GEE Portal
9
GEE Network• Two interfaces/sliver
– Private network (full access)– Public internet (outbound only)
• Problem: how to expose the private network– IP addresses are specific to pre-allocated Slicelet– Programmer doesn’t know what addresses she’ll get per sliver– But she does know where slivers are (all Slicelets have slivers in the same places)
• Symbolic addresses (Python constants) for Slicelet nodes on private network– Resolve to IP addresses in deployed sliver– E.g. GEENetwork.NORTHWESTERN binds to 10.24.128.7 – Resolution in simple binding in downloaded internal file
10
GEE Network Implementation
• Tunneling using GRE• Use of VLANs
– Enough now, more coming– Performance/broadcast isolation within a
VLAN (TBD: experience)• Overlay on Nick Bastin’s VTS…
– Very attractive…
11
GEE Filesystem• Overall concept: POSIX Filesystem over the
GENI substrate• Will use standard Filesystem calls• Volumes to indicate file location• Implementation
– Swift repositories as block store– Syndicate as meta data repository
12
GEE Message System• Still in design phase• Global message queues• Current design: RabbitMQ-as-a-Service
– Multiple clients– RESTful API
• Still TBD: centralized vs. distributed• Plan on initial implementation GEC 20
13
How Do I Use This Stuff?• GEE Compute Engine (today! Well, almost…)
– Login at site, ask for slicelet (4 clicks)– Unpack downloaded zip file
• (Easy) add experirment task to downloaded fabfile.py and run $ fab <task>• (Harder) ssh into slivers, yum, scp, run, etc…
• GEE Filesystem (GEC 20+)– Download Python library– Open, read, write, etc….
• GEE Message System (GEC 20, for sure)– Download Python library– Send, receive,….
14
Current Status• GEE Compute Engine
– Slicelets allocated and demonstrated here(!)– Production Use by GEC 20
• GEE Network– Design alternatives being evaluated
• GEE Filesystem Engine– Swift Adapter for Syndicate written– Demonstratable for GEC 20
• GEE Message Engine– Preliminary version under test– Demonstratable for GEC 20
15
GEE Team• Compute Engine:
– Andy Bavier, PW• Networking
– Joe Mambretti and Jim Chen, NW/StarLight• Filesystem
– Stephen Tredger, U Vic• Message System
– Rick McGeer, Ignite• Overall Architecture
– Rick, Andy, Joe, Jack Brassil (HP), Glenn Ricart (US Ignite), Marshall Brinn• Thanks to our collaborators and friends
– Jude Nelson, Nick Bastin, Rob Ricci, Niky Riga, Mark Berman
Thanks!