CloudNets: Combining Clouds with Virtual Networking
Stefan Schmid December, 2013
CloudNets: Combining Clouds with Virtual Networking
Stefan Schmid December, 2013
3 Telekom Innovation Laboratories Stefan Schmid
Vision: Virtual Networking Cloud Resources.
Cloud computing is a big success!
But what is the point of clouds if they cannot be accessed?
4 Telekom Innovation Laboratories Stefan Schmid
Next Natural Step for Virtualization!
Success of Node Virtualization
a.k.a. end-host virtualization VMWare revamped server business OpenStack VM = flexible allocation, migration.. «Elastic computing»
Trend of Link Virtualization
MPLS, VPN networks, VLANs Software Defined Networks
(SDN), OpenFlow, ... «The VMWare of the net» «Elastic networking»
Unified, fully virtualized networks: CloudNets „Combine networking with heterougeneous cloud resources (e.g., storage, CPU, ...)!“
5 Telekom Innovation Laboratories Stefan Schmid
The Vision: Sharing Resources.
6 Telekom Innovation Laboratories Stefan Schmid
For Tomorrow’s Internet…
Today: Internet is one solution for everything!
CloudNets: Flexibly specifiable virtual networks, executing different protocol stacks, cohabiting the same substrate.
Innovation!
Innovation!
Innovation?
ERCIM News 2012
Vision: facilitate innovation in network core
Make network core programmable (e.g., own intrusion detection system) Service-tailored networks (for social networks, bulk data transfer, life streaming, etc.) Co-existing virtual networks with QoS guarantees No dependencies on IPv4, BGP, ...
7 Telekom Innovation Laboratories Stefan Schmid
Requests with Flexible Specification. Optimization and Migration?
Datacenters „VPN++“
Migration / Service Deployment Spillover/Out-Sourcing Goal: Move with the sun, with the commuters, (QoS) allow for maintenance, avoid roaming costs…: e.g., SAP/game/translator server, small CDN server…
Goal: Fully specified CloudNet mapping constraints (e.g., end-points for a telco), but with QoS guarantees (e.g., bandwidth) along links
Berlin
Tel Aviv Palo Alto
1Mbit/s 1Mbit/s
1Mbit/s
„November 22,
1pm-2pm!“
Berlin
„any European cloud provider (e.g. due to legal issues?)“
Berlin (corporate access network)
< 50ms
< 50ms
„50 TB storage, 10 Tflops computation!“
any
any any < 10ms
> 100 MB/s
„Guaranteed resources, job deadlines met, no overhead!“
“Network may delay execution: costly for per hour priced VM!”
< 10ms
> 100 MB/s < 10ms > 100 MB/s
Elastic computing
See, e.g., Octopus system (SIGCOMM 2011)
8 Telekom Innovation Laboratories Stefan Schmid
Vision: Not only in data centers, but WAN ISP network with resources at Points-of-Presence («nano datacenters»): For service deployment!
9 Telekom Innovation Laboratories Stefan Schmid
Opens New Business Roles.
Roles
Service Provider (SP): uses CloudNets to offer its services (streaming, OSN, CDN, ...): e.g., value-added application CloudNet, or transport CloudNet
Virtual Network Operator (VNO): Installs and operates CloudNet over topology provided by VNP, offers tailored connectivity service, triggers cross-provider migration (by setting requirements), ...
Virtual Network Provider (VNP): „Broker“/reseller that assembles virtual resources from different PIPs to provide virtual topology (no need to know PIP, can be recursive, ...)
Physical Infrastructure Provider (PIP): Owns and manages physical infrastructure
Focus of our work architecture (unlike, e.g., single authority in GINI testbed), on multitude of players, providers, ...! (Of course, cross-layer infos if all same company...)
Roles
QoS from PIP up to VNO or service provider: accounting via complete set of contracts! (unlike „sending party pays“)
SIGCOMM VISA 2009
10 Telekom Innovation Laboratories Stefan Schmid
SIGCOMM VISA 2009 Federated CloudNet Architecture. New Business Roles.
Roles in CloudNet Arch.
Service Provider (SP)
(offers services over the top)
Virtual Network Operator (VNO)
(operates CloudNet, Layer 3+, triggers migration)
Physical Infrastructure Provider (PIP)
(resource and bit pipe provider)
Virtual Network Provider (VNP)
(resource broker, compiles resources)
As in Internet today:
Netflix, Google, World of Warcraft…
As in Internet today: Telekom, AT&T, …
+ resource control interface (bootstrapping etc.)
11 Telekom Innovation Laboratories Stefan Schmid
SIGCOMM VISA 2009 Federated CloudNet Architecture. New Business Roles.
Roles in CloudNet Arch.
Service Provider (SP)
(offers services over the top)
Virtual Network Operator (VNO)
(operates CloudNet, Layer 3+, triggers migration)
Physical Infrastructure Provider (PIP)
(resource and bit pipe provider)
Virtual Network Provider (VNP)
(resource broker, compiles resources)
As in Internet today:
Netflix, Google, World of Warcraft…
As in Internet today: Telekom, AT&T, …
+ resource control interface (bootstrapping etc.)
knows application
knows network (uses resources at
PoPs!)
12 Telekom Innovation Laboratories Stefan Schmid
SIGCOMM VISA 2009 Federated CloudNet Architecture. New Business Roles.
Provide layer 2: assembles CloudNets, resource and management interfaces, provides indirection layer, across PIPs!
View: PIP graph
Resource broker… (recursive)
A virtualized infrastructure opens new roles for the allocation of resources and the operation of networks!
Roles in CloudNet Arch.
Service Provider (SP)
(offers services over the top)
Virtual Network Operator (VNO)
(operates CloudNet, Layer 3+, triggers migration)
Physical Infrastructure Provider (PIP)
(resource and bit pipe provider)
Virtual Network Provider (VNP)
(resource broker, compiles resources)
13 Telekom Innovation Laboratories Stefan Schmid
Provide layer 2: assembles CloudNets, resource and management interfaces, provides indirection layer, across PIPs!
View: PIP graph
Resource broker… (recursive)
SIGCOMM VISA 2009 Federated CloudNet Architecture. New Business Roles.
A virtualized infrastructure opens new roles for the allocation of resources and the operation of networks!
Roles in CloudNet Arch.
Service Provider (SP)
(offers services over the top)
Virtual Network Operator (VNO)
(operates CloudNet, Layer 3+, triggers migration)
Physical Infrastructure Provider (PIP)
(resource and bit pipe provider)
Virtual Network Provider (VNP)
(resource broker, compiles resources)
PIP graph view: cross-provider QoS
contracts
14 Telekom Innovation Laboratories Stefan Schmid
SIGCOMM VISA 2009 Federated CloudNet Architecture. New Business Roles.
Build upon layer 2 (op on virt IDs): clean slate! (OSN, …)
Routing, addressing, multi-path/redundancy… (view inside CloudNet!)
Trigger migration (use provisioning interface of SP or live)
A virtualized infrastructure opens new roles for the allocation of resources and the operation of networks!
Roles in CloudNet Arch.
Service Provider (SP)
(offers services over the top)
Virtual Network Operator (VNO)
(operates CloudNet, Layer 3+, triggers migration)
Physical Infrastructure Provider (PIP)
(resource and bit pipe provider)
Virtual Network Provider (VNP)
(resource broker, compiles resources)
15 Telekom Innovation Laboratories Stefan Schmid
Build upon layer 2 (op on virt IDs): clean slate! (OSN, …)
Routing, addressing, multi-path/redundancy… (view inside CloudNet!)
Trigger migration (use provisioning interface of SP or live)
SIGCOMM VISA 2009 Federated CloudNet Architecture. New Business Roles.
A virtualized infrastructure opens new roles for the allocation of resources and the operation of networks!
Roles in CloudNet Arch.
Service Provider (SP)
(offers services over the top)
Virtual Network Operator (VNO)
(operates CloudNet, Layer 3+, triggers migration)
Physical Infrastructure Provider (PIP)
(resource and bit pipe provider)
Virtual Network Provider (VNP)
(resource broker, compiles resources)
Innovation!
IP hour glass: innovation only on low and high layers
16 Telekom Innovation Laboratories Stefan Schmid
SIGCOMM VISA 2009 Federated CloudNet Architecture. New Business Roles.
Co
ntra
ct b
as
ed
:
Co
mm
un
ica
te re
qu
irem
en
ts
/sp
ecs d
ow
n th
e h
iera
rch
y
API
API
API
AP
Is: e
.g., p
rovis
ion
ing
inte
rfa
ce
s (
mig
ratio
n)
Roles in CloudNet Arch.
Service Provider (SP)
(offers services over the top)
Virtual Network Operator (VNO)
(operates CloudNet, Layer 3+, triggers migration)
Physical Infrastructure Provider (PIP)
(resource and bit pipe provider)
Virtual Network Provider (VNP)
(resource broker, compiles resources)
17 Telekom Innovation Laboratories Stefan Schmid
Roles: Zoom In.
Service Provider (SP): Netflix, Akamai, startup, .. Specifies resources abstractly (latency, push content to customer...) If services migratable: offer provisioning interface (API that VNO can call)
Virtual Network Operator (VNO): Implements layer 3 on top of VNP layer 2: addressing, routing, ... Decides how to realize SP specification: use redundant physical paths => additional
specs To fulfill specs, calls API of SP to migrate (if application migratable), or makes live
migration itself (not to violate specs) Clean slate architecture possible!
Virtual Network Provider (VNP): Builds layer 2 „Broker“/reseller that assembles virtual resources from different PIPs to provide
virtual topology (no need to know PIP, can be recursive, ...)
Physical Infrastructure Provider (PIP): Bit-pipe provider, owns and manages physical infrastructure
18 Telekom Innovation Laboratories Stefan Schmid
Use Cases (1): Migrate Resources. Resource allocation and migration where needed (energy saving otherwise!).
CDN in CloudNet or attached to VNet?
19 Telekom Innovation Laboratories Stefan Schmid
Use Cases (2).
Physical infrastructure
(e.g., accessed by mobile clients)
Specification:
1. close to mobile clients
2. >100 kbit/s bandwidth for synchronization
Provider 1
Provider 2
CloudNet requests
CloudNet 2: Mobile service w/ QoS CloudNet 1: Computation
Specification:
1. > 1 GFLOPS per node
2. Monday 3pm-5pm
3. multi provider ok
Connecting Providers (Geographic Footprint).
20 Telekom Innovation Laboratories Stefan Schmid
Research Overview.
Flexible Specification How to store and communicate CloudNets?
- Specify without losing flexibility
- Communicate non-topological requirements: consistency across multiple roles?
- Allow for aggregation and abstraction
ICCCN 2012
21 Telekom Innovation Laboratories Stefan Schmid
Research Overview.
Prototype
Plugin architecture
- Own cloud operating system
- Currently: VLAN based, but OpenFlow plugin started
- Provisioning interfaces, negotiation interfaces
- PIP and VNP role implemented
- Seamless migration, e.g., of streaming service
- Wide-area: OpenVPN tunnels
- Wide-area testbed: Munich (NTT Docomo) and Berlin
J. Information Technology 2013
22 Telekom Innovation Laboratories Stefan Schmid
Research Overview.
Migration Embedding
Two steps:
- Quick heuristic (spec => greedy)
- Optimizing “long-lived” or “heavy hitter” CloudNets only (mixed integer program)
E.g., move VM by reconfiguring VLANs
23 Telekom Innovation Laboratories Stefan Schmid
Architecture / Anatomy of Prototype.
Plugin based
VLink technology (VLAN, OpenFlow, MPLS, ...)
Embedding algorithm (two stage!)
Cloud operating system
Prototype: proof-of-concept (flexibility&generality rather than speed)
VLink plugin: VLANs (VPN tunnel for WAN)
Embedding: MIP
Cloud OS: own implementation
Plan: other plugins, OpenStack, ...
24 Telekom Innovation Laboratories Stefan Schmid
Two sites: TU Berlin and NTT Docomo Eurolabs Munich
Testbed.
25 Telekom Innovation Laboratories Stefan Schmid
The Prototype (1).
State of the Art
Prototype based on KVM+Xen (nodes) and VLANs (links)
Layer 2 comform: no need for end-to-end / routing (routing inside CloudNet only)
Different VNets may have same
internal virtual node addresses VLAN ensures isolation
PIP and VNP implemented
Journal IT 2013
26 Telekom Innovation Laboratories Stefan Schmid
The Prototype (2).
VLAN1
VLAN2
VLAN2
VLAN1 VLAN2
Migration
IP 1
IP 2
IP 3
IP 1
IP 2
IP 3
Open vSwitch supports VLAN bridging
Demultiplexing eth1 plus VLAN-tag to port in kernel
To VM looks like Ethernet (no VLAN)
Each virtual link is a VLAN (broadcast domain)
Migration: reconfigure VLANs, not addresses of virtual nodes!
Transparent for users...
27 Telekom Innovation Laboratories Stefan Schmid
The Prototype (3).
Life of a CloudNet request:
Topology broken down for PIPs
Transit link (tunnel bridge and OpenVPN) One VPN tunnel for control plane One VPN tunnel for data plane
To VM looks like Ethernet (no VLAN)
28 Telekom Innovation Laboratories Stefan Schmid
Need for a Language. Communicate CloudNets, substrate resources and embeddings to business partners or customers:
Store embedding state internally:
ICCCN 2012
SP PIP
29 Telekom Innovation Laboratories Stefan Schmid
Exploiting Flexibilities: Resource Description Language.
Network Elements = nodes or links! Connected via Network Interfaces
Support for omission
Support for multicast links
Support for white and black lists...
30 Telekom Innovation Laboratories Stefan Schmid
FleRD Requirements.
Support all kinds of node (storage, computation, ...) and link
(latency, bw, full-duplex/asymmetric, ...) resources (heterogeneity)
Extensible, allow for syntactic changes over time, no need for
global agreement on semantic values Facilitate resource leasing and allow PIPs to open abstract
views on their substrate Allow for vagueness and omission: customers are unlikely to
specifiy each CloudNet detail (e.g., KVM or Xen is fine, outsource to any European cloud provider): this opens ways for optimization (exploiting flexibilities)!
Allow for aggregation of resources (business secret?)
Non-topological requirements (e.g., wordsize compatibility)
aggregate
resources
31 Telekom Innovation Laboratories Stefan Schmid
Exploiting Flexibilities: Resource Description Language.
Use Case: Web Service
32 Telekom Innovation Laboratories Stefan Schmid
Exploiting Flexibilities: Resource Description Language.
Web Service / Overlay 0:
Mapping Layer: an virtual element per substrate element (n:m mapping)
Overlay 1: with splitters, two virtual links...
33 Telekom Innovation Laboratories Stefan Schmid
Exploiting Flexibilities: Resource Description Language.
Underlay 1: all-provider view (splitter once collocated and once seperate)
Underlay 0: provider 1 view (NFS at SHost3)
34 Telekom Innovation Laboratories Stefan Schmid
http://www.youtube.com/watch?v=llJce0F1zHQ
Demo.
YouTube Migration Demo
35 Telekom Innovation Laboratories Stefan Schmid
(Theoretical) Research Overview.
Access Control
and Embedding
Service Migration Security Issues
36 Telekom Innovation Laboratories Stefan Schmid
Offline Embedding.
Access Control
and Embedding
Service Migration Security Issues
37 Telekom Innovation Laboratories Stefan Schmid
How to Embed CloudNets Efficiently?
Computationally hard...
Our 2-stage approach:
Stage 1: Map quickly and heuristically
(dedicated resources)
Stage 2: Migrate long-lived CloudNets
to «better» locations (min max load, max
free resources, ...)
Typically: heavy-tailed durations, so old
CloudNets will stay longer!
38 Telekom Innovation Laboratories Stefan Schmid
Flexible Embeddings.
Logo
T-Labs History
General Mathematical Program (MIP)
Advantages: 1. Generic (backbone vs datacenter) and allows for migration 2. Allows for different objective functions 3. Optimal embedding: for backgound optimization of heavy-tailed (i.e., long-lived) or «heavy hitter» CloudNets, quick placement e.g., by clustering But: slow...
UCC 2012
39 Telekom Innovation Laboratories Stefan Schmid
The Solution.
Logo
T-Labs History
General Mathematical Program (MIP)
Advantages: 1. Generic (backbone vs datacenter) and allows for migration 2. Allows for different objective functions 3. Optimal embedding: for backgound optimization of heavy-tailed (i.e., long-lived) CloudNets, quick placement e.g., by clustering But: slow...
Advantages of MIP:
- Very general
- Supports easy replacement of objective functions
- Can use standard, optimized software tools such as CPLEX, Gorubi, etc.
UCC 2012
40 Telekom Innovation Laboratories Stefan Schmid
Generality of the MIP.
Objective functions: - minimize maximum load (= load balance) - maximize free resources (= compress as much as possible), ... Migration support: - costs for migration: per element, may depend on destination, etc. - answer questions such as «what is cost/benefit if I migrate now?» Embedding: - embedding full-duplex on full-duplex links - full-duplex on half-duplex links - or even multiple endpoint links (e.g., wireless) supported!
41 Telekom Innovation Laboratories Stefan Schmid
On the Use of Migration.
Migration: Useful to increase the number of embeddable CloudNets, especially in under-provisioned scenarios
42 Telekom Innovation Laboratories Stefan Schmid
Performance of the MIP: Setup.
Substrate: Rocketfuel ISP topologies (with 25 nodes) CloudNets: Out-sourcing scenario, CloudNets with up to ten nodes, subset of nodes fixed (access points) and subset flexible (cloud resources) Solver: CPLEX on 8-core Xeon (2.5GHz)
fixed access network
flexible nodes
43 Telekom Innovation Laboratories Stefan Schmid
Performance of the MIP.
- Runtime below 1 minute per CloudNet, slightly increasing under load
- Impact of CloudNet size relatively small
44 Telekom Innovation Laboratories Stefan Schmid
Performance of the MIP.
- Enabling option to migrate can increase execution time significantly (log scale!)
- Also number of flexible CloudNet components is important
45 Telekom Innovation Laboratories Stefan Schmid
Use of Flexibility.
How much link resources are needed to embed a CloudNet with specificity s%?
PoS
UCC 2012
Up to 60%, even a little bit more if no migrations are possible!
Skewed (Zipf) distributions worst when not matching.
46 Telekom Innovation Laboratories Stefan Schmid
CloudNet Embedding.
Goal: Where to realize CloudNet such that spec is met? Objective, e.g.: minimize allocation resources, minimize max load, save energy, … Currently focus on optimizing existing CloudNets (heavy-tailed lifetime assumption): but we are also working on quick embeddings (clustering, iterative, ...)
Mapping and Allocation
Logo
T-Labs History
Goal: Decide online which VNet requests to accept, such that profit is maximized
Online Access Control
47 Telekom Innovation Laboratories Stefan Schmid
Competitive Access Control: Model (1)
Physical Infrastructure
CPU, location, ...
link capacity
Berlin
bandwidth
CloudNet (VPN-like)
48 Telekom Innovation Laboratories Stefan Schmid
CloudNet
100 $
Specification of CloudNet request: - terminal locations to be connected - benefit if CloudNet accepted (all-or-nothing, no preemption) - desired bandwidth and allowed traffic patterns - a routing model - duration (from when until when?)
If CloudNets with these specifications arrive over time, which ones to accept online?
Competitive Access Control: Model (2)
Objective: maximize sum of benefits of accepted CloudNets
49 Telekom Innovation Laboratories Stefan Schmid
Competitive Access Control: Model (3)
Which ones to accept?
50 Telekom Innovation Laboratories Stefan Schmid
CloudNet Specifications (1): Traffic Model.
51 Telekom Innovation Laboratories Stefan Schmid
CloudNet Specifications (2): Routing Model.
52 Telekom Innovation Laboratories Stefan Schmid
CloudNet Specifications (2): Routing Model.
Relay nodes may add to embedding costs! (resources depend, e.g., on packet rate)
53 Telekom Innovation Laboratories Stefan Schmid
Competitive Embeddings.
Online algorithms make decisions at time t without any knowledge of inputs / requests at times t’>t.
Online Algorithm
Competitive analysis framework:
An r-competitive online algorithm ALG gives a worst-case performance guarantee: the performance is at most a factor r worse than an optimal offline algorithm OPT!
Competitive Analysis
Competitive ratio r, r = Cost(ALG) / cost(OPT) The price of not knowing the future!
Competitive Ratio
No need for complex predictions but still good!
54 Telekom Innovation Laboratories Stefan Schmid
Buchbinder&Naor: Primal-Dual Approach.
Algorithm design and analysis follows online primal-dual approach recently invented by Buchbinder&Naor! (Application to general VNet embeddings, traffic&routing models, router loads, duration, approx oracles, ...)
1. Formulate dynamic primal and dual LP
2. Derive GIPO algorithm which always produces feasible primal solutions and where Primal >= 2*Dual
55 Telekom Innovation Laboratories Stefan Schmid
Result.
Theorem
The presented online algorithm GIPO is log-competitive
in the amount of resources in the physical network!
If capacities can be exceeded by a log factor, it is even
constant competitive.
However, competitive ratio also depends on max benefit!
ICDCN 2012
56 Telekom Innovation Laboratories Stefan Schmid
Algorithm and Proof Sketch (1).
Embedding oracle: GIPO invokes an oracle procedure to
determine cost of CloudNet embedding!
57 Telekom Innovation Laboratories Stefan Schmid
Algorithm and Proof Sketch (1).
If resource cost lower than benefit: accept!
58 Telekom Innovation Laboratories Stefan Schmid
Algorithm and Proof Sketch (1).
update allocations for
accepted CloudNet...
59 Telekom Innovation Laboratories Stefan Schmid
Algorithm and Proof Sketch (1).
otherwise reject
(no change in substrate)
60 Telekom Innovation Laboratories Stefan Schmid
Algorithm and Proof Sketch (1).
Algorithm efficient... except for oracle (static, optimal embedding)!
What if we only use a suboptimal embedding here?
otherwise reject
(no change in substrate)
61 Telekom Innovation Laboratories Stefan Schmid
Algorithm and Proof Sketch (2).
Problem: computation of optimal embeddings NP-hard!
Thus: use approximate embeddings! (E.g., Steiner tree)
GIPO: Embedding approx.:
<insert your favorite
approx algo>
Competitive ratio Approx ratio r
Lemma
The approximation does not reduce the overall competitive
ratio by much: we get *r ratio!
62 Telekom Innovation Laboratories Stefan Schmid
Proof Sketch (1): Simplified LP.
maximize
benefit!
realization of i-th
request (will be integer,
accept fully or not at all)
... while ensuring
capacity and
no more than demand!
63 Telekom Innovation Laboratories Stefan Schmid
Proof Sketch (2): Simplified LP. essentially, exponential load...
64 Telekom Innovation Laboratories Stefan Schmid
Proof Sketch (3): Simplified LP.
oracle
(triangle only)
update primal
variables if accepted
65 Telekom Innovation Laboratories Stefan Schmid
Proof Sketch (4): Simplified LP.
after each request,
primal variables
constitute feasible
solutions...
66 Telekom Innovation Laboratories Stefan Schmid
On the Benefit of Collocation. Fuerst et al.:
CLOUDNETS 2012
Greedy vs SecondNet vs ViNE: Greedy collocation algorithm beats them all...!
Google cluster: many small networks, over 90% allow for collocation
67 Telekom Innovation Laboratories Stefan Schmid
Mixed Integer Programs
68 Telekom Innovation Laboratories Stefan Schmid
Problem 1: Classic VNet Embedding (VNEP).
- Map virtual nodes to substrate nodes - Collocation possible
- But not splitting of virtual nodes
- Map virtual links - One
- Linear combination
- Hose
- Mixed Integer Model - VINO
- Open Problems
- Everything
Rost et al.:
Tech Report
69 Telekom Innovation Laboratories Stefan Schmid
Problem 2: Embedding with Time Flexibilities (TVNEP).
- VNets come with time flexibilities
- Example: delay-tolerant computations, bulk data transfers, etc.
- Where to embed and when to schedule VNets?
Rost et al.:
IPDPS 2014
70 Telekom Innovation Laboratories Stefan Schmid
Problem 2: Embedding with Time Flexibilities (TVNEP).
- Continuous time (less binary variables): state model (explicit states) and delta model (only differences) - Delta model yields bad relaxations
- State model has more variables, but still better
- Compact variant: - State reduction: feasibility check only at start of
request sufficient (when finishes less resources)
- Minimize smear-out: Distribute start and end to as few event points as possible
- «Merge» multiple endpoints with start points
- Compute temporal dependencies graph cuts
71 Telekom Innovation Laboratories Stefan Schmid
Problem 3: VirtuCast / In-Network Processing (CVSAP).
- Network Function Virtualization
- Can aggregate / split streams in network
- E.g., streaming or wide-area monitoring
- Universal nodes need to be activated:
Joint optimization of processing and
communication? - Note: DAG may not be optimal!
Rost et al.:
OPODIS 2013
N unicasts Steiner
receiver
sender
Communication costs! Processing costs!
Generalization?
72 Telekom Innovation Laboratories Stefan Schmid
Problem 3: VirtuCast / In-Network Processing (CVSAP).
- Multi-Commodity flow bad: for 200 Steiner nodes and 6800 edges, 1.3 mio binary variables!
- Single-Commodity approach: solvable!
- Idea: single commodity and then path decomposition
- Open question: can we even relax path variables and optimally round it afterwards?
73 Telekom Innovation Laboratories Stefan Schmid
Migration.
Access Control
and Embedding
Service Migration Security Issues
74 Telekom Innovation Laboratories Stefan Schmid
Online Service Migration.
Logo
T-Labs History
Goal: E.g., QoS (=“move with the sun”, or with commuters); or for maintenance or to turn off resources (energy conservation).
Online Migration and Allocation
75 Telekom Innovation Laboratories Stefan Schmid
The Virtual Service Migration Problem.
Given a virtual network with guaranteed bandwidth: where to migrate service?
Simple model: one service, constant migration cost (interruption), access along graph.
on service!
Bienkowski et al.:
SIGCOMM VISA 2010
const bw
76 Telekom Innovation Laboratories Stefan Schmid
The Virtual Service Migration Problem.
Given a virtual network with guaranteed bandwidth: where to migrate service?
Simple model: one service, constant migration cost (interruption), access along graph.
on service!
const bw
COUNT(v)
COUNT(v)
COUNT(v)
COUNT(v)
COUNT(v)
COUNT(v) Idea: - Learn from the past: migrate to center of gravity of best location in the past
- Amortize: migrate only when access cost at current node is as high as migration cost!
77 Telekom Innovation Laboratories Stefan Schmid
Center-of-Gravity Algo: Example.
77
Before phase 1:
active
inactive
on service!
78 Telekom Innovation Laboratories Stefan Schmid
Center-of-Gravity Algo: Example.
78
Before phase 2:
active
inactive
on service!
79 Telekom Innovation Laboratories Stefan Schmid
Center-of-Gravity Algo: Example.
79
End of epoch:
active
inactive
on service!
Of course, not converging if demand is dynamic!
(Simplified example.)
80 Telekom Innovation Laboratories Stefan Schmid
Center-of-Gravity Algo: Result.
Competitive analysis? Assume constant bandwidths!
r = ALG / OPT ?
Lower bound cost of OPT:
In an epoch, each node has
at least access cost m, or
there was a migration of cost m.
Upper bound cost of ALG:
We can show that each phase
has cost at most 2m (access
plus migration), and there are
at most log(n) many phases
per epoch!
Theorem
ALG is log(n) competitive! A special uniform metrical task system (graph metric for access)!
81 Telekom Innovation Laboratories Stefan Schmid
Optimality?
Theorem «Center of Gravity» algorithm is
log(n) competitive!
Also a much simpler randomized
algorithm achieves this!
Schneider et al.:
INFOCOM 2013
log(n)/loglog(n) lower bound follows from online function tracking reduction!
on service!
F(x)
82 Telekom Innovation Laboratories Stefan Schmid
Optimality?
Theorem «Center of Gravity» algorithm is
log(n) competitive!
Also a much simpler randomized
algorithm achieves this!
Schneider et al.:
INFOCOM 2013
log(n)/loglog(n) lower bound follows from online function tracking reduction!
on service!
F(x)
There is an asymptotically optimal called FOLLOWER!
83 Telekom Innovation Laboratories Stefan Schmid
The Online Algorithm FOLLOWER.
Simplified Follower
1. 1. Fi are requests handled while service at fi
2. 2. to compute fi+1 (new pos), Follower only takes into account requests during fi: Fi
3. 3. migrate to center of gravity of Fi, as soon as migration costs there are amortized (and «reset counters» immediately)!
Concepts: - Learn from the past: migrate to center of gravity of best
location in the past
- Amortize: migrate only when access cost at current
node is as high as migration cost!
84 Telekom Innovation Laboratories Stefan Schmid
The Online Algorithm FOLLOWER.
Concepts: - Learn from the past: migrate to center of gravity of best
location in the past
- Amortize: migrate only when access cost at current
node is as high as migration cost!
Simplified Follower
1. 1. Fi are requests handled while service at fi
2. 2. to compute fi+1 (new pos), Follower only takes into account requests during fi: Fi
3. 3. migrate to center of gravity of Fi, as soon as migration costs there are amortized (and «reset counters» immediately)!
85 Telekom Innovation Laboratories Stefan Schmid
Intuition.
85
on service!
86 Telekom Innovation Laboratories Stefan Schmid
Intuition.
86
on service! Fi
= fi
87 Telekom Innovation Laboratories Stefan Schmid
Intuition.
87
on service!
Fi
= fi+1
88 Telekom Innovation Laboratories Stefan Schmid
Modeling Access and Migration Costs.
Access Costs 1. Latency along shortest path in graph.
2. (Graph distances, and in particular: metric!)
Migration Costs 1. Generalized models:
- E.g., depends on bandwidth along path (duration of service interruption)
- E.g., depends on distance travelled
(latency)
- Discount: e.g., VNP (number of
migrations, distance travelled, ...)
89 Telekom Innovation Laboratories Stefan Schmid
Modeling Access and Migration Costs.
Access Costs 1. Latency along shortest path in graph.
2. (Graph distances, and in particular: metric!)
Migration Costs 1. Generalized models:
- E.g., depends on bandwidth along path (duration of service interruption)
- E.g., depends on distance travelled
(latency)
- Discount: e.g., VNP (number of
migrations, distance travelled, ...)
90 Telekom Innovation Laboratories Stefan Schmid
Competitive Ratio of FOLLOWER.
Competitive analysis? FOLLOWER / OPT?
Theorem If no discounts are given,
Follower is log(n)/loglog(n) competitive!
Theorem If migration costs depend on travelled
distance (page migration), competitive
ratio is O(1), even with discounts.
Simple model with migration
costs = bandwidth, and
homogeneous
Page migration model with
migration costs = distance,
but discounts
91 Telekom Innovation Laboratories Stefan Schmid
Related Work.
- Metrical Task Systems:
- Classical online problem where server at certain location («state») serves requests at certain costs; state transitions also come at certain costs («migration»)
- Depending on migration cost function more general (we have graph access costs) and less general (we allow for migration discounts)
- E.g., uniform space metrical task system: migration costs constant, but access costs more general than graph distances! Lower bound of log(n) vs log(n)/loglog(n) upper bound in our case.
- Online Page Migration
- Classical online problem from the 80ies; we generalize cost function to distance discounts, while keeping O(1)-competitive
Our work lies between!
92 Telekom Innovation Laboratories Stefan Schmid
Simulation.
Dynamics due to mobility: requests cycle through a 24h pattern: in the morning, requests distributed widely (people in suburbs), then focus in city centers; in the evening, reverse.
Commuter Scenario
Dynamics due to time zone effects: request originate in China first, then more requests come from European countries, and finally from the U.S.
Time Zone Scenario
Algorithm which uses optimal static server placements for a given request seq.
Static Algorithm
Predictable scenarios, but we do not exploit that. Reality less predictable!
93 Telekom Innovation Laboratories Stefan Schmid
Results.
Competitive ratio generally relatively low. Increases for more correlated requests and more dynamics.
94 Telekom Innovation Laboratories Stefan Schmid
Related Work.
- Metrical Task Systems:
- Classical online problem where server at certain location («state») serves requests at certain costs; state transitions also come at certain costs («migration»)
- Depending on migration cost function more general (we have graph access costs) and less general (we allow for migration discounts)
- E.g., uniform space metrical task system: migration costs constant, but access costs more general than graph distances! Lower bound of log(n) vs log(n)/loglog(n) upper bound in our case.
- Online Page Migration
- Classical online problem from the 80ies; we generalize cost function to distance discounts, while keeping O(1)-competitive
Our work lies between!
95 Telekom Innovation Laboratories Stefan Schmid
Extension: Inter-Provider Migration.
Migration across provider boundary costs transit/roaming costs (# transit
providers), detailed topology not known, etc.
PIP 1
PIP 2
PIP 4
PIP 3 Theorem Competitive ALGs still exist!
PIP 3
Adding page migration aspects...
96 Telekom Innovation Laboratories Stefan Schmid
Extension: Multiple Servers.
Multiple servers allocated and migrated dynamically depending
on demand and load, servers have running costs, etc.
PIP 3 Theorem Competitive ALGs still exist!
on service!
on service!
USENIX Hot-ICE 2011
97 Telekom Innovation Laboratories Stefan Schmid
Extension: Economical Aspects.
How to price resources?
Pay-as-you-go vs Pay-as-you-come?
on service!
Hu et al.: ICDCN 2013
98 Telekom Innovation Laboratories Stefan Schmid
Migration of Entire CloudNets. INFOCOM Demo 2013
2 pm in Japan
2 pm in Europe
Latency-resource tradeoffs: move-with-the-sun vs move-with-the-moon?
99 Telekom Innovation Laboratories Stefan Schmid
Security of Embedding.
Access Control
and Embedding
Service Migration Security Issues
100 Telekom Innovation Laboratories Stefan Schmid
Security Issues.
Are CloudNet embeddings a threat for ISPs?
Do embeddings leak information about infrastructure?
?
101 Telekom Innovation Laboratories Stefan Schmid
Request Complexity.
Are CloudNet embeddings a threat for ISPs?
How many embeddings needed to fully reveal topology?
Request Complexity
?
?
?
Yes
Yes
No
102 Telekom Innovation Laboratories Stefan Schmid
Embedding Model.
unit capacity
unit capacity
arbitrary node demand arbitrary link
demand
103 Telekom Innovation Laboratories Stefan Schmid
Embedding Model.
unit capacity
unit capacity
arbitrary node demand arbitrary link
demand
relay cost ε>0 (e.g., packet rate)
104 Telekom Innovation Laboratories Stefan Schmid
Embedding Model.
unit capacity
unit capacity
arbitrary node demand arbitrary link
demand
relay cost ε>0 (e.g., packet rate) We will ask for unit capacities on
nodes and links! Essentially a graph immersion problem: disjoint paths for virtual links...
105 Telekom Innovation Laboratories Stefan Schmid
Some Properties Simple...
«Is the network 2-connected?»
?
No
106 Telekom Innovation Laboratories Stefan Schmid
Example: Tree.
How to discover a tree? Graph growing: 1. Test whether triangle fits? (loop-free) 2. Try to add neighbors to node as long as possible, then continue with other node
v
107 Telekom Innovation Laboratories Stefan Schmid
Example: Tree.
How to discover a tree? Graph growing: 1. Test whether triangle fits? (loop-free) 2. Try to add neighbors to node as long as possible, then continue with other node
108 Telekom Innovation Laboratories Stefan Schmid
Example: Tree.
How to discover a tree? Graph growing: 1. Test whether triangle fits? (loop-free) 2. Try to add neighbors to node as long as possible, then continue with other node
109 Telekom Innovation Laboratories Stefan Schmid
Example: Tree.
How to discover a tree? Graph growing: 1. Test whether triangle fits? (loop-free) 2. Try to add neighbors to node as long as possible, then continue with other node
110 Telekom Innovation Laboratories Stefan Schmid
Example: Tree.
How to discover a tree? Graph growing: 1. Test whether triangle fits? (loop-free) 2. Try to add neighbors to node as long as possible, then continue with other node
111 Telekom Innovation Laboratories Stefan Schmid
Example: Tree.
How to discover a tree? Graph growing: 1. Test whether triangle fits? (loop-free) 2. Try to add neighbors to node as long as possible, then continue with other node
Virtual links may be embedded over multiple physical links!
112 Telekom Innovation Laboratories Stefan Schmid
Tree Solution: Graph Growing.
TREE ALGORITHM: line strategy 1. Binary search on longest path («anchor»):
...
2. Last and first node explored, explore «branches» at pending nodes
Analysis: Amortized analysis on links: Per discovered physical link at most one query, plus at most one per physical node (no incident links).
113 Telekom Innovation Laboratories Stefan Schmid
Greedy Graph Growing on General Graphs? (1)
Finding path...
114 Telekom Innovation Laboratories Stefan Schmid
Greedy Graph Growing on General Graphs? (1)
Finding neighbors...
115 Telekom Innovation Laboratories Stefan Schmid
Greedy Graph Growing on General Graphs? (1)
Finding more neighbors...
116 Telekom Innovation Laboratories Stefan Schmid
Greedy Graph Growing on General Graphs? (1)
How to close the gap? Adding connections between existing CloudNet nodes is expensive: try all pairs!
117 Telekom Innovation Laboratories Stefan Schmid
Greedy Graph Growing on General Graphs? (1)
How to close the gap? Adding connections between existing CloudNet nodes is expensive: try all pairs!
118 Telekom Innovation Laboratories Stefan Schmid
Greedy Graph Growing on General Graphs? (2)
Simple solution: First try to find the «knitting»! - The «two-or-more» connected components - Later «expand nodes» and «expand edges»
119 Telekom Innovation Laboratories Stefan Schmid
Greedy Graph Growing on General Graphs? (2)
Simple solution: First try to find the «knitting»! - The «two-or-more» connected components - Later «expand nodes» and «expand edges»
120 Telekom Innovation Laboratories Stefan Schmid
Greedy Graph Growing on General Graphs? (3)
Idea: Ask graph «motif» only if it’s guaranteed that it cannot be embedded over a more highly connected subgraph! (And connectivity has to be added later.) Careful: What goes first also depends on entire motif sequences! - A cannot be embedded into B and B cannot be embedded into A - But A can be embedded into BB!
1 2 3 4 5
6 7
8 2
1 3 4 5
6 7
8
Relay cost: 4 ε
A B BB
121 Telekom Innovation Laboratories Stefan Schmid
Remark.
Minor vs embedding:
Even with unit link capacity, for small epsilon, graph A may be embeddable ( ) into graph B although A is not a minor of B!
Planar graph (and hence K5-minor free): But K5 can be embedded here!
Graph A is a minor of B if A can be obtained from B by (1) deleting nodes, (2) deleting edges, or (3) contracting two nodes along edges.
Graph Minor
122 Telekom Innovation Laboratories Stefan Schmid
Dictionary Attack: Expansion Framework.
Examples Tree motifs: Cactus motifs:
Basic “knittings” of the graph.
Motif Define an order on motif sequences: Constraints on which sequence to ask first in order not to overlook a part of the topology. (E.g., by embedding links across multiple hops.)
Dictionary
Poset
Framework
Explore branches according
to dictionary order,
exploiting poset property.
Poset = partially ordered set (1) Reflexive: G G (2) Transitive: G G’ and G’ G’’, then G G’’ (3) Antisymmetric: G G’ and G’ G implies G=G’ (isomorphic)
123 Telekom Innovation Laboratories Stefan Schmid
query order
Dictionary Attack: Expansion Framework.
solves, e.g.,
Dictionary dag (for chain C, cycle Y, diamond D, ...) with attachment points:
poset poset
Complexity: Depends on dictionary depth and number of attachment points
124 Telekom Innovation Laboratories Stefan Schmid
Overview of Results.
PIP 3 Tree Can be explored in O(n)
requests. This is optimal!
PIP 3 General Graph Can be explored in O(n2)
requests. This is optimal!
Cactus Graph Can be explored in O(n)
requests. This is optimal!
Lower bound: via number of possible trees and binary information.
Idea: Make spanning tree and then try all edges. (Edges directly does not work!)
Via «graph motifs»! A general framework exploiting poset relation.
125 Telekom Innovation Laboratories Stefan Schmid
Overview of Results.
PIP 3 Tree Can be explored in O(n)
requests. This is optimal!
PIP 3 General Graph Can be explored in O(n2)
requests. This is optimal!
Cactus Graph Can be explored in O(n)
requests. This is optimal!
Lower bound: via number of possible trees and binary information.
Idea: Make spanning tree and then try all edges. (Edges directly does not work!)
Via «graph motifs»! A general framework exploiting poset relation.
126 Telekom Innovation Laboratories Stefan Schmid
Dictionary Attacks: Expand Framework.
Examples Tree motifs: Cactus motifs:
Basic “knittings” of the graph.
Motif Define an order on motif sequences: Constraints on which sequence to ask first in order not to overlook a part of the topology. (E.g., by embedding links across multiple hops.)
Dictionary
Partially ordered set: embedding relation fulfills reflexivity, anti-symmetry, transitivity.
Poset
Framework
Explore branches according
to dictionary order,
exploiting poset property.
127 Telekom Innovation Laboratories Stefan Schmid
Application: BigFoot EU Project.
http://www.bigfootproject.eu
“BigData is the answer: what was the question?” How to interact with fast growing data volumes? Which technology to obtain insights?
BigFoot in three points:
Automatic and self-tuned deployments through virtualization Cross-layer optimization Data interaction made easy
Applications: Billing & revenue assurance Customer segmentation for service personalization Pattern analysis for infrastructure provisioning
Attack attribution Multi-feature classification
128 Telekom Innovation Laboratories Stefan Schmid
Application: BigFoot EU Project.
http://www.bigfootproject.eu
“BigData is the answer: what was the question?” How to interact with fast growing data volumes? Which technology to obtain insights?
BigFoot in three points:
Automatic and self-tuned deployments through virtualization Cross-layer optimization Data interaction made easy
Applications: Billing & revenue assurance Customer segmentation for service personalization Pattern analysis for infrastructure provisioning
Attack attribution Multi-feature classification
129 Telekom Innovation Laboratories Stefan Schmid
Big Data: OSN Analysis (Google+). Schiöberg et al.:
ACM WebSci 2012
Now 100M+ users... ... from all over the world!
Research: # of «followers» (easy) Which time zones follow
which time zones? (easy) K-cores, «stars», ... (easy) Diameter?? Evolution of centralities??
130 Telekom Innovation Laboratories Stefan Schmid
Big Data: OSN Analysis (Google+). Schiöberg et al.:
ACM WebSci 2012
Now 100M+ users... ... from all over the world!
Research: # of «followers» (easy) Which time zones follow
which time zones? (easy) K-cores, «stars», ... (easy) Diameter?? Evolution of centralities??
131 Telekom Innovation Laboratories Stefan Schmid
Big Data: OSN Analysis (Google+). Schiöberg et al.:
ACM WebSci 2012
Now 100M+ users... ... from all over the world!
Research: # of «followers» (easy) Which time zones follow
which time zones? (easy) K-cores, «stars», ... (easy) Diameter?? Evolution of centralities??
132 Telekom Innovation Laboratories Stefan Schmid
BigFoot Architecture.
WP3Batch Analytics and Interactive Query Engines
WP2
WP4
WP5
Applications
Virtualization Layer
Distributed Data Stores
HadoopDistributedFilesystem
InfrastructureVirtualization
High-level queryto
low-level program translation
Optimization Library
Service-orientedQuery Engine
DistributedDatastore
Data Partitioning and Placement
Data Store Support
ServiceDeployment
ServiceOptimization
Data miningAlgorithms
Statistical
Patterns
Analysis
Interactive
Queries
Virtualization Layer
1 2 3
HadoopMapReduce
Compiler
Big
Fo
ot
So
ftw
are
Sta
ck
Testbed: Hadoop (maybe Stratosphere) OpenStack resource management
(+ own cloud operating system) OpenFlow (link virtualization)
133 Telekom Innovation Laboratories Stefan Schmid
Conclusion.
- CloudNets:
- Elastic computing and networking
- Federated architecture
- Competitive analysis: a framework to design and prove performance of online algorithms
- Good when:
- No reliable prediction models exist
- No data available
- Worst case guarantees matter
- Examples: online embedding and service migration
- Fully incorporated in prototype
134 Telekom Innovation Laboratories Stefan Schmid
The Project Website.
135 Telekom Innovation Laboratories Stefan Schmid
Collaborators and Publications. People
T-Labs / TU Berlin: Anja Feldmann, Carlo Fürst, Johannes Grassler, Arne Ludwig, Matthias Rost, Gregor Schaffrath, Stefan Schmid
Uni Wroclaw: Marcin Bienkowski Uni Tel Aviv: Guy Even, Moti Medina NTT DoCoMo Eurolabs: Group around Wolfgang Kellerer LAAS: Gilles Tredan ABB: Yvonne Anne Pignolet IBM Research: Johannes Schneider Arizona State Uni: Xinhui Hu, Andrea Richa
Publications
Prototype: J. Information Technology 2013, ICCCN 2012, ERCIM News 2012, SIGCOMM VISA 2009
Migration: ToN 2013, INFOCOM 2013, ICDCN 2013 + Elsevier TCS Journal, Hot-ICE 2011, IPTComm 2011, SIGCOMM VISA 2010
Embedding: IPDPS 2014, OPODIS 2013, CLOUDNETS 2013 (Best Paper), INFOCOM 2013 (Mini-Conference), CLOUDNETS 2012, 2 x ACM UCC 2012, DISC 2012, ICDCN 2012 (Best Paper Distributed Computing Track)
136 Telekom Innovation Laboratories Stefan Schmid
Dr. Stefan Schmid
Telekom Innovation Laboratories Ernst-Reuter-Platz 7, D-10587 Berlin E-mail: [email protected]
Project website: http://www.net.t-labs.tu-
berlin.de/~stefan/virtu.shtml
Contact.
Top Related