A sampling of interactive and collaborative learning environments
Autonomic Virtual Networks and Applications in Cloud and Collaborative Computing Environments
description
Transcript of Autonomic Virtual Networks and Applications in Cloud and Collaborative Computing Environments
![Page 1: Autonomic Virtual Networks and Applications in Cloud and Collaborative Computing Environments](https://reader031.fdocuments.in/reader031/viewer/2022013012/5681692c550346895de06e8c/html5/thumbnails/1.jpg)
Center for Autonomic ComputingIntel Portland, April 30, 2010
Autonomic Virtual Networks and Applications in Cloud and
Collaborative Computing Environments
Renato FigueiredoAssociate Professor
Center for Autonomic ComputingACIS Lab
University of Florida
![Page 2: Autonomic Virtual Networks and Applications in Cloud and Collaborative Computing Environments](https://reader031.fdocuments.in/reader031/viewer/2022013012/5681692c550346895de06e8c/html5/thumbnails/2.jpg)
2
Outlook Architecting autonomic virtual networks
Isolation, security, encapsulation, dynamic configuration, migration
Self-configuration, self-healing, self-optimization
Applications in cloud and collaborative environments Virtual Private Clusters Social VPNs
Archer: a collaborative environment for computer architecture simulation
Ongoing/future work
![Page 3: Autonomic Virtual Networks and Applications in Cloud and Collaborative Computing Environments](https://reader031.fdocuments.in/reader031/viewer/2022013012/5681692c550346895de06e8c/html5/thumbnails/3.jpg)
33
BackgroundCollaboration, entertainment: streaming, data sharing, games
Resource aggregation:Cross-institution sharing,opportunistic computing,on-demand provisioning
PublicInternet
NAT
NATSelf-configuring
End-to-endVirtual Private Network
![Page 4: Autonomic Virtual Networks and Applications in Cloud and Collaborative Computing Environments](https://reader031.fdocuments.in/reader031/viewer/2022013012/5681692c550346895de06e8c/html5/thumbnails/4.jpg)
4
Self-organizing virtual networks
Focus: Software overlays that provide virtual network
infrastructure over existing Internet infrastructure Why virtual?
Support unmodified TCP/IP applications and existing Internet physical infrastructure
Hide heterogeneity of physical network (firewalls, NATs), avoid IPv4 address space constraints
Why self-organizing? Autonomous behavior: low management cost
compared to typical VPNs Decentralized architecture for scalability and fault
tolerance
![Page 5: Autonomic Virtual Networks and Applications in Cloud and Collaborative Computing Environments](https://reader031.fdocuments.in/reader031/viewer/2022013012/5681692c550346895de06e8c/html5/thumbnails/5.jpg)
5
Virtual networking Isolation: dealt with similarly to VMs
Multiple, isolated virtual networks time-share physical network
Key technique: tunneling (VPNs) Related work
Grid computing VNET (P. Dinda at Northwestern U.) Violin (D. Xu at Purdue U.) ViNe (J. Fortes at U. Florida) PVC (F. Cappello at INRIA)
“P2P” VPNs Hamachi, tinc, Gbridge
![Page 6: Autonomic Virtual Networks and Applications in Cloud and Collaborative Computing Environments](https://reader031.fdocuments.in/reader031/viewer/2022013012/5681692c550346895de06e8c/html5/thumbnails/6.jpg)
6
The IP-over-P2P (IPOP) Approach
Isolation Virtual address space decoupled from Internet
address space Self-managing
Self-organizing, self-healing topology Decentralized – structured peer-to-peer (P2P)
No global state, no central points of failure Self-optimizing IP overlay routing
On-demand direct/relay connections Self-configuring decentralized NAT traversal
![Page 7: Autonomic Virtual Networks and Applications in Cloud and Collaborative Computing Environments](https://reader031.fdocuments.in/reader031/viewer/2022013012/5681692c550346895de06e8c/html5/thumbnails/7.jpg)
7
Use case scenarios Sharing resources/services in a virtual end host
VM provides isolation Virtual appliances provide software encapsulation
Distributed virtual appliance clusters Homogeneous software environment on top of
heterogeneous infrastructure Homogeneous virtual network on top of wide-area,
NATed environments Cross-institution collaboration; cloud-bursting
![Page 8: Autonomic Virtual Networks and Applications in Cloud and Collaborative Computing Environments](https://reader031.fdocuments.in/reader031/viewer/2022013012/5681692c550346895de06e8c/html5/thumbnails/8.jpg)
8
Example: virtual clusters
Physical machines
Switched
network
NOWs, COWs “WOWs”• Wide-area
• Virtual machines
(VMs)
• Self-organizing
overlay IP tunnels,
P2P routingInstallation
image
Virtual machinesVM image
• Local-area
• Physical machines
• Self-organizing switching
(e.g. Ethernet spanning
tree)
![Page 9: Autonomic Virtual Networks and Applications in Cloud and Collaborative Computing Environments](https://reader031.fdocuments.in/reader031/viewer/2022013012/5681692c550346895de06e8c/html5/thumbnails/9.jpg)
9
Use case scenarios There are various successful overlays enabling
peer-to-peer communication among users VoIP sessions over skype File transfers over bittorrent iChat (video, chat, desktop sharing)
Application (and/or platform) specific Users: richer set of applications over a generic
IP network for communication and collaboration But they don’t have public IPs, and don’t want to
directly connect to all users – hence NATs And they don’t want to or know how to configure and
discover network services manually
![Page 10: Autonomic Virtual Networks and Applications in Cloud and Collaborative Computing Environments](https://reader031.fdocuments.in/reader031/viewer/2022013012/5681692c550346895de06e8c/html5/thumbnails/10.jpg)
10
Example: Social VPNs
Alice
CarolBob
SocialNetworkWeb interface
Social network(e.g. Facebook)
Overlay network(IPOP)
carol.facebook.ipop10.10.0.2
node0.alice.facebook.ipop10.10.0.3
SocialNetwork API
Social network Information system
Alice’s public keysBob’s public keysCarol’s public key
Bob: browses Alice’s SMB share Alice’s services:Samba shareRDP serverVoIP, ChatAdvertise to Bob, Carol
![Page 11: Autonomic Virtual Networks and Applications in Cloud and Collaborative Computing Environments](https://reader031.fdocuments.in/reader031/viewer/2022013012/5681692c550346895de06e8c/html5/thumbnails/11.jpg)
11
IP-over-P2P Tunneling As in many other VPNs, use virtual network
device to capture/inject IP (e.g. tap/tun) Tunnel IP over UDP or TCP
Unlike traditional VPNs, tunnels are not established by an administrator Rather, IPOP implements self-organizing techniques
to discover, establish and maintain overlay links Each IPOP peer is capable of picking packets,
injecting packets, and routing
![Page 12: Autonomic Virtual Networks and Applications in Cloud and Collaborative Computing Environments](https://reader031.fdocuments.in/reader031/viewer/2022013012/5681692c550346895de06e8c/html5/thumbnails/12.jpg)
12
Virtual network architecture
Application
VNIC
VirtualRouter
VirtualRouter
VNIC
Application
Wide-areaOverlay network
Isolated, private virtualaddress space
10.10.1.2
10.10.1.1
Unmodified applicationsConnect(10.10.1.2,80)
Capture/tunnel, scalable,resilient, self-configuringrouting and object store
![Page 13: Autonomic Virtual Networks and Applications in Cloud and Collaborative Computing Environments](https://reader031.fdocuments.in/reader031/viewer/2022013012/5681692c550346895de06e8c/html5/thumbnails/13.jpg)
13
Bi-directional structured overlay (Brunet library) Constant number of edges (K) per node O((1/k)log2(n)) overlay hops Self-organizing topology
Nearedge
Overlayrouter
Overlay architecture
Overlayrouter
Shortcut(far) edge
OrderedID space
![Page 14: Autonomic Virtual Networks and Applications in Cloud and Collaborative Computing Environments](https://reader031.fdocuments.in/reader031/viewer/2022013012/5681692c550346895de06e8c/html5/thumbnails/14.jpg)
14
Abstract bi-directional communication channels Edges can use various transports:
UDP; TCP; DTLS; Tunnel UDP/DTLS:
NAT traversal
“Tunnel” edge
Overlayrouter
Overlay Edges
Overlayrouter
UDPedgeTCP edge
![Page 15: Autonomic Virtual Networks and Applications in Cloud and Collaborative Computing Environments](https://reader031.fdocuments.in/reader031/viewer/2022013012/5681692c550346895de06e8c/html5/thumbnails/15.jpg)
15
Reflection: learn NAT-mapped endpointsFrom public overlay peers
Peers exchange “connect to me” through overlaySet up hole punching
Self-configuring
2. Exchange learnedEndpoint with peer
NAT traversal
1. Reflection:udp://IP:port
3. Simultaneousopen: NAT traversal
![Page 16: Autonomic Virtual Networks and Applications in Cloud and Collaborative Computing Environments](https://reader031.fdocuments.in/reader031/viewer/2022013012/5681692c550346895de06e8c/html5/thumbnails/16.jpg)
16
Greedy routing relies on consistent bi-directional ring topology
Faults in structure due to routing outages, symmetric NATs
Tunnel near edges
Self-healing structure
Peers exchangeneighbor set
Unavailable physical path
Tunneledge
![Page 17: Autonomic Virtual Networks and Applications in Cloud and Collaborative Computing Environments](https://reader031.fdocuments.in/reader031/viewer/2022013012/5681692c550346895de06e8c/html5/thumbnails/17.jpg)
17
Create direct edges based on traffic inspection O(log2(N)) -> O(1)
Direct connection when NAT traversal possible Relay through a peer – “far” tunnel edge
2. Exchange learnedEndpoint with peer
Self-optimization
1. Reflection:udp://IP:port
3. Simultaneousopen: NAT traversal
![Page 18: Autonomic Virtual Networks and Applications in Cloud and Collaborative Computing Environments](https://reader031.fdocuments.in/reader031/viewer/2022013012/5681692c550346895de06e8c/html5/thumbnails/18.jpg)
18
Bootstrapping
New P2P
node
Forms a “leaf” connection with a well-known nodeSelected at random from list of “bootstrap” nodes
Sends “Connect to me” CTM request addressed to itselfReceived by nearest neighbors
Forwarder
CTM request
Received by left and
right neighbors
![Page 19: Autonomic Virtual Networks and Applications in Cloud and Collaborative Computing Environments](https://reader031.fdocuments.in/reader031/viewer/2022013012/5681692c550346895de06e8c/html5/thumbnails/19.jpg)
19
Autonomous IP allocation One P2P overlay supports multiple IPOP namespaces
IP routing within a namespace Each IPOP namespace: a unique string
Distributed Hash Table (DHT) stores mapping Key=namespace Value=DHCP configuration (IP range, lease, ...)
IPOP node configured with a namespace Query namespace for DHCP configuration Guess an IP address at random within range Attempt to store in DHT
Key=namespace+IP Value=IPOPid (160-bit)
IP->P2P Address resolution: Given namespace+IP, lookup IPOPid
![Page 20: Autonomic Virtual Networks and Applications in Cloud and Collaborative Computing Environments](https://reader031.fdocuments.in/reader031/viewer/2022013012/5681692c550346895de06e8c/html5/thumbnails/20.jpg)
20
Avoiding overlay overheads
VNIC
VirtualRouter
VirtualRouter
VNIC
Application
Wide-areaOverlay network Local
Interface
LAN Router
NIC
Application
NIC
Application
![Page 21: Autonomic Virtual Networks and Applications in Cloud and Collaborative Computing Environments](https://reader031.fdocuments.in/reader031/viewer/2022013012/5681692c550346895de06e8c/html5/thumbnails/21.jpg)
21
VN Interfaces● Each machine has local
VN Interface
● ARP, DHCP captured locally● Router responds as
gateway● DHCP: DHT put/get
Virtual Network Device
NIC
APP
VPN Client Software
VPN Overlay
Virtual Network Device
NIC
APP
VPN Client Software
Virtual LAN
![Page 22: Autonomic Virtual Networks and Applications in Cloud and Collaborative Computing Environments](https://reader031.fdocuments.in/reader031/viewer/2022013012/5681692c550346895de06e8c/html5/thumbnails/22.jpg)
22
Supporting VN Routers● Single VN (Router) for
entire cluster● Avoid need for VN
software stack on end host
● Avoid VN overhead on LAN communication
IP=10.1.1.2Eth=A:B:C:D:E:0
IP=10.1.1.3Eth=A:B:C:D:E:1
IP=10.1.1.4Eth=A:B:C:D:E:2
TAP Device
VPN Software NIC0
NIC1
Virtual Router
Internet
![Page 23: Autonomic Virtual Networks and Applications in Cloud and Collaborative Computing Environments](https://reader031.fdocuments.in/reader031/viewer/2022013012/5681692c550346895de06e8c/html5/thumbnails/23.jpg)
23
VN Hybrid
● VN instance for each member in a cluster
● VN hosts in the same LAN bypass VN software stack
IP0=128.227.56.41/24IP1=10.250.5.5/16
IP0=128.227.56.33/24IP0=128.227.56.21/24IP1=10.250.255.1/16
VPN Software
TAP Device
ETH0
VETH0_0
VETH0_110.250.1.25/16
BRIDGE128.227.56.40/24
Internet
![Page 24: Autonomic Virtual Networks and Applications in Cloud and Collaborative Computing Environments](https://reader031.fdocuments.in/reader031/viewer/2022013012/5681692c550346895de06e8c/html5/thumbnails/24.jpg)
24
Autonomic features Self-configuration [IPDPS’06, HPDC’06, PCgrid’07]
Routing tables using structured P2P links NAT traversal, DHCP over DHT
Self-optimization [HPDC’06] Direct shortcut connections created/trimmed based upon IP
traffic inspection for fast end-to-end IP tunnels Proximity neighbor selection based on network coordinate
estimates for improved structured routing Self-healing [HPDC’08]
“Tunnel” edges created to maintain overlay structure to deal with routing outages and NATs/firewalls that are not traversable
VLAN routers, overlay bypass within VLAN [VTDC09, SC09]
![Page 25: Autonomic Virtual Networks and Applications in Cloud and Collaborative Computing Environments](https://reader031.fdocuments.in/reader031/viewer/2022013012/5681692c550346895de06e8c/html5/thumbnails/25.jpg)
25
Overlay security architecture
Abstract senders encapsulate security logic Supports both edge (point-to-point) and IPOP (end-
to-end) authentication and encryption Public key infrastructure
Keys/certificates Symmetric key exchange
DTLS (Datagram TLS) library or native IPOP stack UDP-based; amenable to NAT traversal
IPsec tunneling also supported
![Page 26: Autonomic Virtual Networks and Applications in Cloud and Collaborative Computing Environments](https://reader031.fdocuments.in/reader031/viewer/2022013012/5681692c550346895de06e8c/html5/thumbnails/26.jpg)
26
Performance IPOP implementation
C# user-level router Tap virtual network device
Latency (ms) Bwidth (Mb/s) Mem (KB)
Host 0.27 941 n/a
C 0.34 738 9988
C# 0.37 716 21500
IPOP 0.52 284 38312
IPOP sec 0.75 55 50976
![Page 27: Autonomic Virtual Networks and Applications in Cloud and Collaborative Computing Environments](https://reader031.fdocuments.in/reader031/viewer/2022013012/5681692c550346895de06e8c/html5/thumbnails/27.jpg)
27
Security management Overlay point-to-point and/or end-to-end
security need to be configured PKI management can be complex and error-prone
Certificate signing/distribution, revocation Approach: leverage Web 2.0, social networking
infrastructures for security management SocialVPN: enable point-to-point VPN connectivity
among socially-networked peers GroupVPN: enable sharing of resources with all-to-all
VPN connectivity within a group of users