From Next Gen Cloud Research To Future Campus …...2016/10/04 · CloudLab updated: 9/28/16...
Transcript of From Next Gen Cloud Research To Future Campus …...2016/10/04 · CloudLab updated: 9/28/16...
![Page 1: From Next Gen Cloud Research To Future Campus …...2016/10/04 · CloudLab updated: 9/28/16 Kuang-Ching (KC) Wang Associate Professor, Networking CTO Director, Center of Excellence](https://reader035.fdocuments.in/reader035/viewer/2022063020/5fe19859f0d78405b922a1b1/html5/thumbnails/1.jpg)
updated: 9/28/16 CloudLab
Kuang-Ching (KC) Wang
Associate Professor, Networking CTO
Director, Center of Excellence for Next Gen Computing & Creativity
From Next Gen Cloud Research
To Future Campus Computing Infrastructure
![Page 2: From Next Gen Cloud Research To Future Campus …...2016/10/04 · CloudLab updated: 9/28/16 Kuang-Ching (KC) Wang Associate Professor, Networking CTO Director, Center of Excellence](https://reader035.fdocuments.in/reader035/viewer/2022063020/5fe19859f0d78405b922a1b1/html5/thumbnails/2.jpg)
updated: 9/28/16 CloudLab
The CloudLab Team
Robert Ricci (PI)Eric EideSteve CorbatóKobus Van der Merwe
Aditya Akella (co-PI)Remzi Arpaci-DusseauMiron Livny
KC Wang (co-PI)Jim BoKumJim Pepin
Chip EllioK (co-PI)Larry Landweber
Mike Zink (co-PI)David Irwin
Glenn Ricart (co-PI)
2
WINLAB ORBIT
CloudLab Now Speaks Wireless!!
Suman Banerjee
![Page 3: From Next Gen Cloud Research To Future Campus …...2016/10/04 · CloudLab updated: 9/28/16 Kuang-Ching (KC) Wang Associate Professor, Networking CTO Director, Center of Excellence](https://reader035.fdocuments.in/reader035/viewer/2022063020/5fe19859f0d78405b922a1b1/html5/thumbnails/3.jpg)
updated: 9/28/16 CloudLab 3
Clouds are great but have “well known” constraints
The line between can’s and cannot’s …?
Clouds are around and gaining importance by day
Lots of opportunities for research
![Page 4: From Next Gen Cloud Research To Future Campus …...2016/10/04 · CloudLab updated: 9/28/16 Kuang-Ching (KC) Wang Associate Professor, Networking CTO Director, Center of Excellence](https://reader035.fdocuments.in/reader035/viewer/2022063020/5fe19859f0d78405b922a1b1/html5/thumbnails/4.jpg)
updated: 9/28/16 CloudLab 4
CloudLab enables research on the future of cloud computing architectures and the new applications they enable
![Page 5: From Next Gen Cloud Research To Future Campus …...2016/10/04 · CloudLab updated: 9/28/16 Kuang-Ching (KC) Wang Associate Professor, Networking CTO Director, Center of Excellence](https://reader035.fdocuments.in/reader035/viewer/2022063020/5fe19859f0d78405b922a1b1/html5/thumbnails/5.jpg)
updated: 9/28/16 CloudLab
• A “meta-cloud” for building clouds• Build your own cloud on our hardware resources• Agnostic to specific cloud software
• Run existing cloud software stacks (like OpenStack, Hadoop, etc.)• … or new ones built from the ground up
• Control and visibility all the way to the bare metal• “Sliceable” for multiple, isolated experiments at once
The CloudLab Vision
With CloudLab, it’s as easy to get an entire cloud as it is to get a VM in a cloud
5
![Page 6: From Next Gen Cloud Research To Future Campus …...2016/10/04 · CloudLab updated: 9/28/16 Kuang-Ching (KC) Wang Associate Professor, Networking CTO Director, Center of Excellence](https://reader035.fdocuments.in/reader035/viewer/2022063020/5fe19859f0d78405b922a1b1/html5/thumbnails/6.jpg)
updated: 9/28/16 CloudLab 6
![Page 7: From Next Gen Cloud Research To Future Campus …...2016/10/04 · CloudLab updated: 9/28/16 Kuang-Ching (KC) Wang Associate Professor, Networking CTO Director, Center of Excellence](https://reader035.fdocuments.in/reader035/viewer/2022063020/5fe19859f0d78405b922a1b1/html5/thumbnails/7.jpg)
updated: 9/28/16 CloudLab
DIBBs,Wireless,
OtherInstruments
Focus Today – What CloudLab Means to … ?
7
CloudResearchers
Industry
OtherClouds
Synergy Value Ecosystem Evolution
DomainResearchers
UniversityIT
![Page 8: From Next Gen Cloud Research To Future Campus …...2016/10/04 · CloudLab updated: 9/28/16 Kuang-Ching (KC) Wang Associate Professor, Networking CTO Director, Center of Excellence](https://reader035.fdocuments.in/reader035/viewer/2022063020/5fe19859f0d78405b922a1b1/html5/thumbnails/8.jpg)
updated: 9/28/16 CloudLab
What Is CloudLab?
Utah Wisconsin Clemson Federated Facilities
Slice B
Stock OpenStack
CC-NIE, Internet2 AL2S, Regionals
Slice A
Geo-Distributed Storage Research
Slice D
Allocation and Scheduling Research for Cyber-Physical Systems
Slice C
Virtualization and Isolation Research
8
A place to build your own cloud
![Page 9: From Next Gen Cloud Research To Future Campus …...2016/10/04 · CloudLab updated: 9/28/16 Kuang-Ching (KC) Wang Associate Professor, Networking CTO Director, Center of Excellence](https://reader035.fdocuments.in/reader035/viewer/2022063020/5fe19859f0d78405b922a1b1/html5/thumbnails/9.jpg)
updated: 9/28/16 CloudLab
• Built on Emulab and GENI (“ProtoGENI”)• In active development at Utah since 1999• Several thousand users (incl. GENI users)• Provisions, then gets out of the way
• “Run-time” services are optional
• Controllable through a web interface and GENI APIs
• Scientific instrument for repeatable research• Physical isolation for most resources• Profiles capture everything needed for experiments
• Software, data, and hardware details• Can be shared and published (eg. in papers)
Technology Foundations
9
![Page 10: From Next Gen Cloud Research To Future Campus …...2016/10/04 · CloudLab updated: 9/28/16 Kuang-Ching (KC) Wang Associate Professor, Networking CTO Director, Center of Excellence](https://reader035.fdocuments.in/reader035/viewer/2022063020/5fe19859f0d78405b922a1b1/html5/thumbnails/10.jpg)
updated: 9/28/16 CloudLab
Many Sites, One Facility
10
= GENI racks
![Page 11: From Next Gen Cloud Research To Future Campus …...2016/10/04 · CloudLab updated: 9/28/16 Kuang-Ching (KC) Wang Associate Professor, Networking CTO Director, Center of Excellence](https://reader035.fdocuments.in/reader035/viewer/2022063020/5fe19859f0d78405b922a1b1/html5/thumbnails/11.jpg)
updated: 9/28/16 CloudLab
CloudLab’s Hardware One facility, one account, three locations (+ more!)
Wisconsin Clemson Utah
• About 5,000 cores each (15,000 total)• 8-16 cores per node• Baseline: 4GB RAM / core• Latest virtualization hardware
• TOR / Core switching design• 10 Gb to nodes, SDN• 100 Gb to Internet2 AL2S• Partnerships with multiple vendors
• Storage and net.• Per node:
• 128 GB RAM• 2x1TB Disk• 400 GB SSD
• Clos topology• Cisco and HP
• High-memory • 16 GB RAM / core• 16 cores / node• Bulk block store• Net. up to 40Gb• High capacity• Dell
• Power-efficient• ARM64 / x86• Power monitors• Flash on ARMs• Disk on x86• Very dense• HP
11
![Page 12: From Next Gen Cloud Research To Future Campus …...2016/10/04 · CloudLab updated: 9/28/16 Kuang-Ching (KC) Wang Associate Professor, Networking CTO Director, Center of Excellence](https://reader035.fdocuments.in/reader035/viewer/2022063020/5fe19859f0d78405b922a1b1/html5/thumbnails/12.jpg)
updated: 9/28/16 CloudLab
• At Rutgers, CloudLab will integrate with ORBIT• At Utah, CloudLab will integrate with PhantomNet, a
mobile network testbed• At Wisconsin, CloudLab will integrate with outdoor
wireless from LTE to SDRs• Once in place, CloudLab can support research such as:
• Repeatable, combined mobile/cloud experimentation• Cloud-RAN• mobile cloud• cloud offloading• 5G• …
And Wireless
12
![Page 13: From Next Gen Cloud Research To Future Campus …...2016/10/04 · CloudLab updated: 9/28/16 Kuang-Ching (KC) Wang Associate Professor, Networking CTO Director, Center of Excellence](https://reader035.fdocuments.in/reader035/viewer/2022063020/5fe19859f0d78405b922a1b1/html5/thumbnails/13.jpg)
updated: 9/28/16 CloudLab
Profiles: Packaged environments
13
Your Software,
Data, Compute, & Network
![Page 14: From Next Gen Cloud Research To Future Campus …...2016/10/04 · CloudLab updated: 9/28/16 Kuang-Ching (KC) Wang Associate Professor, Networking CTO Director, Center of Excellence](https://reader035.fdocuments.in/reader035/viewer/2022063020/5fe19859f0d78405b922a1b1/html5/thumbnails/14.jpg)
updated: 9/28/16 CloudLab
Creating Profile via GUI
14
GENI RSpec
![Page 15: From Next Gen Cloud Research To Future Campus …...2016/10/04 · CloudLab updated: 9/28/16 Kuang-Ching (KC) Wang Associate Professor, Networking CTO Director, Center of Excellence](https://reader035.fdocuments.in/reader035/viewer/2022063020/5fe19859f0d78405b922a1b1/html5/thumbnails/15.jpg)
updated: 9/28/16 CloudLab
Creating Profile via Code
15
#!/usr/bin/env python """An example of constructing a profile with a single Xen VM. Instructions: Wait for the profile instance to start, and then log in to the VM via the ssh port specified below. (Note that in this case, you will need to access the VM through a high port on the physical host, since we have not requested a public IP address for the VM itself.) """ # Import the Portal object. import geni.portal as portal # Import the ProtoGENI library. import geni.rspec.pg as pg # Create the Portal context. pc = portal.Context() # Create a Request object to start building the RSpec. rspec = pg.Request() # Create a XenVM and add it to the RSpec. node = pg.XenVM("node") rspec.addResource(node) # Print the RSpec to the enclosing page. pc.printRequestRSpec(rspec)
![Page 16: From Next Gen Cloud Research To Future Campus …...2016/10/04 · CloudLab updated: 9/28/16 Kuang-Ching (KC) Wang Associate Professor, Networking CTO Director, Center of Excellence](https://reader035.fdocuments.in/reader035/viewer/2022063020/5fe19859f0d78405b922a1b1/html5/thumbnails/16.jpg)
updated: 9/28/16 CloudLab
• CloudLab can be used with a GENI account, and vice-versa• GENI Racks: ~ 50 small clusters around the country• Programmable wide-area network
• Openflow at dozens of sites• Connected in one layer 2 domain
• Large clusters (100s of nodes) at several sites• Wireless and mobile
• WiMax at 8 institutions• LTE / EPC testbed (“PhantomNet”) at Utah
• International partners• Europe (FIRE), Brazil, Japan
Federated with GENI
16
![Page 17: From Next Gen Cloud Research To Future Campus …...2016/10/04 · CloudLab updated: 9/28/16 Kuang-Ching (KC) Wang Associate Professor, Networking CTO Director, Center of Excellence](https://reader035.fdocuments.in/reader035/viewer/2022063020/5fe19859f0d78405b922a1b1/html5/thumbnails/17.jpg)
updated: 9/28/16 CloudLab
CloudLab Users So Far
17
May 2016: 300 projects 1,250 users 21,000 experiments
![Page 18: From Next Gen Cloud Research To Future Campus …...2016/10/04 · CloudLab updated: 9/28/16 Kuang-Ching (KC) Wang Associate Professor, Networking CTO Director, Center of Excellence](https://reader035.fdocuments.in/reader035/viewer/2022063020/5fe19859f0d78405b922a1b1/html5/thumbnails/18.jpg)
updated: 9/28/16 CloudLab
• Data center networking• Intra- and inter-data center
• Genomics, near-real-time GPU-based remote surgery, …
• Network function virtualization• Virtual machines, containers
• Database
• Parallel computing
• Storage
Reported Research Use of CloudLab
18
• Edge cloud
• Cybersecurity
• Resource allocation
• Web retrieval
• Cyber physical systems
• Robotics
• Smart grid
• …
![Page 19: From Next Gen Cloud Research To Future Campus …...2016/10/04 · CloudLab updated: 9/28/16 Kuang-Ching (KC) Wang Associate Professor, Networking CTO Director, Center of Excellence](https://reader035.fdocuments.in/reader035/viewer/2022063020/5fe19859f0d78405b922a1b1/html5/thumbnails/19.jpg)
updated: 9/28/16 CloudLab
• Enables cloud architecture & application research by• providing the latest, bleeding edge hardware feature• providing deep programmability on network, compute, storage• plugging in researcher provided specialized compute devices (e.g.,
FPGA/GPUs, wireless systems acquired from other grants)• All these come with beKer isolation (slices), repeatability (profiles)
• Enables novel computing methods research for domain sciences• Exploring benefits of novel compute methods with CloudLab
features, to inform creation of next gen production infrastructure.• Focus is on new architecture at modest scale, not compute cycles
CloudLab To Researchers
19
![Page 20: From Next Gen Cloud Research To Future Campus …...2016/10/04 · CloudLab updated: 9/28/16 Kuang-Ching (KC) Wang Associate Professor, Networking CTO Director, Center of Excellence](https://reader035.fdocuments.in/reader035/viewer/2022063020/5fe19859f0d78405b922a1b1/html5/thumbnails/20.jpg)
updated: 9/28/16 CloudLab
• (Level 1) For CloudLab sites, IT is heroic when it comes to hosting CloudLab, being• Innovative, flexible, and agile to changes• A partner in both design and operation
• (Level 2) Bootstraps IT innovations• Seeing CloudLab as a sandbox, for exploring “cloud strategies”,
“new features” (in collaboration with researchers*)• Seeing CloudLab as a model for campus computing infrastructure
• (Level 3) Works with ACI-REF to reach out to researchers
CloudLab To IT
20
![Page 21: From Next Gen Cloud Research To Future Campus …...2016/10/04 · CloudLab updated: 9/28/16 Kuang-Ching (KC) Wang Associate Professor, Networking CTO Director, Center of Excellence](https://reader035.fdocuments.in/reader035/viewer/2022063020/5fe19859f0d78405b922a1b1/html5/thumbnails/21.jpg)
updated: 9/28/16 CloudLab
• For vendors, is a place to introduce novel capabilities by partnering with CloudLab• HP, Dell, Cisco were the core partners in phase I• Startups offering/developing new solutions for CloudLab• SBIR partnership
• For industry research labs, is a place for innovative research• E.g., Dell Research partnered with Clemson researchers to research
scalable OpenStack architecture for Smart Grid
• Is a bridge between vendors and broad research communities• Communicating new vendor features that have the potential for novel
research use• Communicate researcher needs to vendors to drive new features
CloudLab To Industry
21
![Page 22: From Next Gen Cloud Research To Future Campus …...2016/10/04 · CloudLab updated: 9/28/16 Kuang-Ching (KC) Wang Associate Professor, Networking CTO Director, Center of Excellence](https://reader035.fdocuments.in/reader035/viewer/2022063020/5fe19859f0d78405b922a1b1/html5/thumbnails/22.jpg)
updated: 9/28/16 CloudLab
Other clouds can include national and institutional research clouds, commercial public clouds, ... • Is created for a DIFFERENT, complementary purpose
• To enable research on new cloud architecture and applications• NSF CloudLab is a TESTBED for research• While other clouds serve production computing needs
• Can shed light on future production cloud design choices
• Can be used to build sandboxes for pre-production dev
CloudLab To Other Clouds
22
![Page 23: From Next Gen Cloud Research To Future Campus …...2016/10/04 · CloudLab updated: 9/28/16 Kuang-Ching (KC) Wang Associate Professor, Networking CTO Director, Center of Excellence](https://reader035.fdocuments.in/reader035/viewer/2022063020/5fe19859f0d78405b922a1b1/html5/thumbnails/23.jpg)
updated: 9/28/16 CloudLab
• Across domain sciences, increasing focus on novel integration of data and compute, as DIBBs (Data Infrastructure Building Blocks)• CloudLab has worked with NSF DIBBS project to explore
innovative approaches for new science workflows
• Future Internet foresees increasing integration between edge infrastructure and computing backend• Smart grid, wireless, …• CloudLab is a perfect enabler for prototyping novel approaches for
feasibility studies and architectural experiments
CloudLab To Other DIBBS & instruments
23
![Page 24: From Next Gen Cloud Research To Future Campus …...2016/10/04 · CloudLab updated: 9/28/16 Kuang-Ching (KC) Wang Associate Professor, Networking CTO Director, Center of Excellence](https://reader035.fdocuments.in/reader035/viewer/2022063020/5fe19859f0d78405b922a1b1/html5/thumbnails/24.jpg)
updated: 9/28/16 CloudLab
• CloudLab mission is to enable researchers’ research
• CloudLab is NOT about speed and feed, but it IS about newest and coolest technology components from Industry
• CloudLab is designed and run by a partnership among researchers, IT, and industry. All parties are learning and contributing knowledge and ideas.
To CloudLab, the Ecosystem is the Core
24
![Page 25: From Next Gen Cloud Research To Future Campus …...2016/10/04 · CloudLab updated: 9/28/16 Kuang-Ching (KC) Wang Associate Professor, Networking CTO Director, Center of Excellence](https://reader035.fdocuments.in/reader035/viewer/2022063020/5fe19859f0d78405b922a1b1/html5/thumbnails/25.jpg)
updated: 9/28/16 CloudLab
To Go From Here … 25
OtherDIBBs,
Instruments
CloudResearchers
Industry
OtherClouds
Synergy Value Ecosystem Evolution
DomainResearchers
IT
![Page 26: From Next Gen Cloud Research To Future Campus …...2016/10/04 · CloudLab updated: 9/28/16 Kuang-Ching (KC) Wang Associate Professor, Networking CTO Director, Center of Excellence](https://reader035.fdocuments.in/reader035/viewer/2022063020/5fe19859f0d78405b922a1b1/html5/thumbnails/26.jpg)
updated: 9/28/16 CloudLab
Learn more, sign up, share your research:
www.CloudLab.us
This material is based upon work supported by the National Science Foundation under Grant No. 1419199. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.
26