Cellular-Internet Convergence:

Evolving the Internet Architecture to

Support Mobility Services as the Norm

Johannesburg Summit

May 20-21, 2013

D. Raychaudhuri

WINLAB, Rutgers University

ray@winlab.rutgers.edu

Introduction

WINLAB

Introduction: Mobility as the key driver for

the future Internet

Historic shift from PC’s to mobile computing and embedded devices… Cisco VNI report predicts smartphone traffic alone will grow by ~10x by

2017, accounting for 7.5 Exabytes/mo; M2M services also taking off …

Mobile devices as a whole will dominate Internet usage wired devices

will contribute only 39% of traffic by 2016

Motivates efficient integration of cellular nets with IP in short term, and re-

evaluation of Internet architecture to support general mobility requirements

in long-term

WINLAB

Introduction: Industry Approach Towards Flat

IP Architecture for Mobile Networks

Cellular network standards (3GPP, LTE) have steadily migrated towards tighter integration with IP From centralized controllers and gateways to “flat” architectures

Separation of control and data planes all IP in LTE

Still requires some gateways, e.g. MME and SGW

Figure from:

Cisco White Paper:

“Evolution of the Mobile Internet”

2010

WINLAB

Introduction: Cellular and Internet Technology

Evolution Trend

Opportunity for greater convergence of Internet and cellular/mobile network standards

~1975… ~1990 ~2000 ~2010

~2020

Basic

IP Addressing

& Routing

BGP/CIDR

Inter-Domain

Routing

Mobile IP

VOIP/SIP

Future

Internet

Protocol

GSM

Mobile

Network

Core Cellular

Network

Technologies

Core

Internet

Technologies

Service-Level

Internet

Technologies

??

HIP

LISP

Etc.

CDN

IPv6 Mobility

Ext

Open

DNS Web

Services Cloud

Services

SDN/

Virtual Net

ICN, FIA

3GPP 3GPP2

(IP-based)

Flat

IP-based

LTE

“5G”

Arch

??

SDR, Open BTS

MMS Service-Level

Cellular

Technologies

Android

Services

Mobile

Cloud

Services

….

WINLAB

Operator Network

Running future IP

Protocols

Introduction: Next-Steps Towards Cellular-

Internet Convergence Truly flat “future IP” architecture under consideration

No gateways – mobility functionality distributed between routers/BSs/APs running

the same control and data protocols; standard mobility & service control API’s

Multiple radio access technologies simply plug-in to universal mobile Internet

Generalized view of mobility service requirements, not limited to simple device

mobility – e.g. multicast, content addressability, context delivery, …

Edge Networks with

Mobility Services

RAN A

RAN B

RAN A RAN

Net B

RAN C

Enhanced Packet Core

(Operator Network)

MME

SGW

Standard

IP Router

Future IP control plane

w/ mobility support

To/From Global

Internet

futureIP

Intra-Domain

Router

futureIP

Inter-Domain

Router

WINLAB

Mobility Requirements:

Supporting Device Migration as Basic Service

End-point mobility as a basic service of the future Internet

Any network connected object or device should be reachable on an efficiently

routed path as it migrates from one network to another

Requirements similar to mobile IP in the Internet today and dynamic

handoff/roaming in cellular networks

Mobility service should be scalable (billions of devices) and fast ~50-100 ms

INTERNET

Wireless

Access Net #3

Wireless

Access

Network

#2

BS2 User/Device

Mobility

WINLAB

Mobility Requirements:

Handling BW Variation & Disconnection Wireless medium has inherent fluctuations in bit-rate (as much

as 10:1 in 3G/4G access), heterogeneity and disconnection Poses a fundamental protocol design challenge

New requirements include in-network storage/delay tolerant delivery, dynamic rerouting (late binding), etc.

Transport layer implications end-to-end TCP vs. hop-by-hop

INTERNET

Wireless

Access Net #3

Wireless

Access

Network #2

BS-1

AP-2

Mobile devices with varying BW due to SNR variation,

Shared media access and heterogeneous technologies

Time Disconnection

interval

Bit

Rate

(Mbps)

Dis-

connect

AP-2

BS-1

WINLAB

Mobility Requirements:

Multicast as a Basic Network Service Many mobility services (content, context) involve multicast

The wireless medium is inherently multicast, making it possible to reach multiple end-user devices with a single transmission

Fine-grain packet level multicast desirable at network routers

INTERNET

Session level Multicast Overlay (e.g. PIM-SIM)

Wireless

Access Net #11

INTERNET

Access

Network

(Eithernet)

Radio

Broadcast

Medium

Packet-level Multicast at Routers/AP’s/BSs

RP

Wireless

Access

Net #32

Pkt Mcast at Routers

WINLAB

Mobility Requirements:

Multi-Homing as a Standard Service Multiple/heterogeneous radio access technologies (e.g.

3G/4G and WiFi) increasingly the norm Implies the need for separating “identity” from “locators” (network addresses)

Requires routing framework that supports packet level multicasting where needed for efficient delivery to multiple networks

Support for alternative routing policies – “best path”, “all paths”, etc.

INTERNET

Wireless

Access Net #3

Wireless

Access

Network #2

LTE BS

WiFi

AP

Multihomed devices may utilize two or more interfaces to improve communications

quality/cost, with policies such as “deliver on best interface” or “deliver only on WiFi”

Mobile device

With dual-radio NICs

WINLAB

Mobility Requirements :

Multi-Network Access, Multipath Wired Internet devices typically have a single Ethernet interface

associated with a static network/AS

In contrast, mobile devices typically have ~2-3 radios and can see ~5-10 distinct networks/AS’s at any given location

Basic property - multiple paths to a single destination leads to fundamentally different routing, both intra and inter domain!

INTERNET

Single “virtual link” in wired Internet Wireless

Access Net #1

Wireless

Access Network

Wireless

Access Net #3

Wireless

Edge

Network

INTERNET

Access

Network

(Eithernet)

BS-1

BS-2

BS-3

AP1

Mobile device with multi-path reachability

Multi

Radio

NIC’s

Ethernet

NiC

Multiple

Potential

Paths

WINLAB

Mobility Requirements:

Content Retrieval & Delivery Capabilities

Content Owner’s

Server

In-network cache

Get (“content_ID”) Send(“content_ID”, “user_ID”))

In-network

cache

Alternative paths

for retrieval

or delivery

Delivery of content to/from mobile devices a key service requirement in future networks

This requirement currently served by overlay CDN’s

In-network support for content addressability and caching is desirable service primitives such as get(content-ID, ..)

WINLAB

Context-aware delivery often associated with mobile services Examples of context are group membership, location, network state, …

Requires framework for defining and addressing context (e.g. “taxis in New

Brunswick”)

Anycast and multicast services for message delivery to dynamic group

Mobile

Device

trajectory

Context = geo-coordinates & first_responder

Send (context, data)

Context-based

Multicast delivery

Context

GUID

Global Name

Resolution service

ba123

341x

Context

Naming

Service

NA1:P7, NA1:P9, NA2,P21, ..

Mobility Requirements:

Supporting Context-Aware/M2M Services

WINLAB

Access

Network

)

Mobility Requirements:

Ad Hoc & Network Mobility Wireless devices can form ad hoc networks with or without

connectivity to the core Internet

These ad hoc networks may also be mobile and may be capable of peering along the edge

Requires rethinking of interdomain routing, trust model, etc. Ad Hoc Network Formation, Intermittent Connection to Wired Internet & Network Mobility

INTERNET

Access

Network

)

WINLAB

Mobility Requirements:

Spectrum Coordination as a Network Service

As more and more data is carried by unlicensed wireless networks,

spectrum coordination should be offered as a network service

Management plane offers global visibility for cooperative setting of

radio resource parameters across independent access networks

WiFi AP locations in a 0.4x0.5 sq.mile area in Manhattan, NY

Network Management Plane

Interface for Radio Parameter Map

(e.g. Frequency, Power, Rate, ..)

Inter-network spectrum

coordination procedures

MobilityFirst Protocol

Design

WINLAB

Designing a Mobility-Centric Internet

Emerging mobile applications, M2M devices, V2V networks etc. involve

more than simple device mobility support from the network

Some examples of services required are:

Multi-homing, multi-path

Efficient multicast

Ad-hoc modes, DTN delivery

Content mobility, caching

Service mobility

Context services, geo-location

Enhanced authentication & trust

In-network cloud services

….

The “MobilityFirst” future Internet architecture (FIA) project is aimed at

a clean-slate redesign of the IP stack & service/control API’s to meet

these and other anticipated requirements

Clients,

Servers

Embedded

devices

Mobility Service &

Control API’s

(universal future IP standard)

Network

Services

Network

Transport

Universal future IP

protocols

WINLAB

MobilityFirst Design: Architecture Features

Routers with Integrated

Storage & Computing Heterogeneous

Wireless Access

End-Point mobility

with multi-homing In-network

content cache

Network Mobility &

Disconnected Mode

Hop-by-hop

file transport Edge-aware

Inter-domain

routing

Named devices, content,

and context

11001101011100100…0011

Public Key Based

Global Identifier (GUID)

Storage-aware

Intra-domain

routing

Service API with

unicast, multi-homing,

mcast, anycast, content

query, etc.

Strong authentication, privacy

Ad-hoc p2p

mode

Human-readable

name

Connectionless Packet Switched Network

with hybrid name/address routing

MobilityFirst Protocol Design Goals: - 10B+ mobile/wireless devices

- Mobility as a basic service

- BW variation & disconnection tolerance

- Ad-hoc edge networks & network mobility

- Multihoming, multipath, multicast

- Content & context-aware services

- Strong security/trust and privacy model

WINLAB

MobilityFirst Design: Technology Solution

Global Name

Resolution Service

(GNRS)

Hybrid GUID/NA

Global Routing (Edge-aware, mobile,

Late binding, etc.)

Storage-Aware

& DTN Routing

(GSTAR)

in Edge Networks

Optional

Compute Layer

Plug-Ins (cache, privacy, etc.)

Hop-by-Hop

Transport

(w/bypass option)

Name-Based

Services (mobility, mcast,

content, context,

M2M)

Name Certification

Service (NCS)

Meta-level

Network Services

Core Transport

Services

Pure connectionless packet switching with in-network storage

Flexible name-based network service layer

WINLAB

MF Design: Protocol Stack

IP

Hop-by-Hop Block Transfer

Link Layer 1

(802.11)

Link Layer 2

(LTE)

Link Layer 3

(Ethernet)

Link Layer 4

(SONET)

Link Layer 5

(etc.)

GSTAR Routing MF Inter-Domain

E2E TP1 E2E TP2 E2E TP3 E2E TP4

App 1 App 2 App 3 App 4

GUID Service Layer Narrow Waist GNRS

MF Routing

Control Protocol

NCS Name

Certification

& Assignment

Service

Global Name

Resolution

Service

Data Plane Control Plane

Socket API

Switching

Option

Optional Compute

Layer

Plug-In A

WINLAB

MF Design: Name-Address Separation

GUIDs Separation of names (ID) from

network addresses (NA)

Globally unique name (GUID)

for network attached objects User name, device ID, content, context,

AS name, and so on

Multiple domain-specific naming

services

Global Name Resolution Service

for GUID NA mappings

Hybrid GUID/NA approach Both name/address headers in PDU

“Fast path” when NA is available

GUID resolution, late binding option

Globally Unique Flat Identifier (GUID)

John’s _laptop_1

Sue’s_mobile_2

Server_1234

Sensor@XYZ

Media File_ABC

Host

Naming

Service

Network

Sensor

Naming

Service

Content

Naming

Service

Global Name Resolution Service

Network address

Net1.local_ID

Net2.local_ID

Context

Naming

Service

Taxis in NB

WINLAB

MF Design: Service Abstractions (1)

MobilityFirst offers a named-object service API that supports

mobility, disconnection, multi-homing, multicast in a natural way

Replaces the point-to-point virtual link abstraction of IP …

Example: Sending to a mobile device with multiple interfaces IP Abstraction: Virtual Link

Dest

Device

Network

Interface IP Addr=X

Name=

MAC

Send(IP=X, data)

Static

MAC=X

binding

Dynamic

GUID – NA

bindings

MF Abstraction: Multi-homed Network Object

NA=X1 NA=X2 NA=X3

GUID=Y Network

Attached

Object

Send(GUID=Y, data, options)

..options for multi-homing & late binding

e.g., Y may be a mobile device with 3 interfaces (WiFi & 2 cellular)

WINLAB

MF Design: Service Abstractions (2)

Use of MF Service API for content retrieval and dynamic group

multicast (..membership may be specified by context)

MF Abstraction: Get Replicated Content Object

NA=X1 NA=X2 NA=X3

Content Object

With GUID=A

Get (Content_GUID=A, options)

..option for shortest path

e.g., A is a replicated content object at multiple network locations e.g., Z may be a context group of M2M devices or a cloud service

GUID=A GUID=A

MF Abstraction: Send to Group Object with

Multicast reachability

NA=X2

GUID1 GUID2 GUID3 Group

GUID = Z

Send (GUID=Z, data, options)

..option for mcast delivery

Broadcast Medium

WINLAB

Building Mobile Networks with MF

MobilityFirst enables both tightly and loosely coupled

architectures for mobile networks Current Mobile Networks

Wi-Fi ISP 1

ISP 2

ISP 3

GSM/CDMA/LTE

Internet

Global Name Resolution

ServiceMobility Support

Through GNRS

Wireless Cooperation through Geographic Routing

AAA Server

Mobility Management Entity (MME)

GSM/CDMA/LTE

Internet

Control Path Elements

Data Path Elements

Gateway Node

Make-before-break Handover Controlled QoS inside network

• Planned Deployment

• Licensed Spectrum

• Fine-grained Managed QoS

• Centralized Mobility Support

• Homogeneous Topology

• Network-wide Authentication

Loosely Coupled Network-of-Networks

• Ad-hoc Deployment

• Unlicensed Spectrum

• Coarse-grained Managed

• In-network Mobility Support

• Heterogeneous topology

• Authentication at APs

WINLAB

MF Protocol Example: Mobility Service via

Name Resolution at Device End-Points

MobilityFirst Network

(Data Plane)

GNRS

Register “John Smith22’s devices” with NCS

GUID lookup

from directory

GUID assigned

GUID = 11011..011

Represents network

object with 2 devices

Send (GUID = 11011..011, SID=01, data)

Send (GUID = 11011..011, SID=01, NA99, NA32, data)

GUID <-> NA lookup

NA99

NA32

GNRS update

(after link-layer association)

DATA

SID

NAs

Packet sent out by host

GNRS query

GUID

Service API capabilities:

- send (GUID, options, data)

Options = anycast, mcast, time, ..

- get (content_GUID, options)

Options = nearest, all, ..

Name Certification

Services (NCS)

WINLAB

10 100 1,00020 500

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Round Trip Query Latency in milliseconds (log scale)

Cum

ulat

ive

Dis

tribu

tion

Func

tion

(CD

F)

K = 1

K = 2

K = 3

K = 4

K = 5

K = 5,

95th

Percentileat 91 ms

K = 1,

95th

Percentileat 202 ms

MF Protocol Design: Realizing the GNRS

Fast GNRS implementation based on DHT between routers GNRS entries (GUID <-> NA) stored at Router Addr = hash(GUID)

Results in distributed in-network directory with fast access (~100 ms)

Internet Scale Simulation Results

Using DIMES database

WINLAB

GNRS Scalability Results

We did a trace driven simulation to assess scalability

Question: How much load on GNRS if everyone driving in a

given area uses MF for mobility management:

0 10 20 30 40 50 60 70 80 900

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Number of Updates/second

Cu

mu

lative

Dis

trib

utio

n F

un

ctio

n (

CD

F)

Cell Radius = 250 m

Cell Radius = 500 m

Cell Radius = 750 m

Traces from Rutgers Intelligent Transportation Systems Lab:

Captures peak time traffic of >16K vehicles in a ~8 sq.km urban

area of Jersey CIty, NJ

GNRS load not too high even for very frequent

handovers and high density of vehicles

WINLAB

MF Protocol Design: Storage-Aware

Routing (GSTAR) Storage aware (CNF, generalized DTN) routing exploits in-network

storage to deal with varying link quality and disconnection

Routing algorithm adapts seamlessly adapts from switching (good

path) to store-and-forward (poor link BW/short disconnection) to

DTN (longer disconnections)

Storage has benefits for wired networks as well..

Storage

Router

Low BW

cellular link

Mobile

Device

trajectory

High BW

WiFi link

Temporary

Storage at

Router

Initial Routing Path

Re-routed path

For delivery

Sample CNF routing result

PDU

WINLAB

MF Protocol Design: Hybrid GUID/NA

Storage Router in MobilityFirst

GUID-Address Mapping – virtual DHT table

NA Forwarding Table – stored physically at router

GUID NA

11001..11 NA99,32

Dest NA Port #, Next Hop

NA99 Port 5, NA11

GUID –based forwarding

(slow path)

Network Address Based Forwarding

(fast path)

Router

Storage

Store when:

- Poor short-term path quality

- Delivery failure, no NA entry

- GNRS query failure

- etc.

NA32 Port 7, NA51

DATA

SID GUID=

11001…11 NA99,NA32

NA62 Port 5, NA11

To NA11

To NA51

Look up GUID-NA table when:

- no NAs in pkt header

- encapsulated GUID

- delivery failure or expired NA entry

Look up NA-next hop table when:

- pkt header includes NAs

- valid NA to next hop entry

DATA

DATA

Hybrid name-address based routing in MobilityFirst requires a new

router design with in-network storage and two lookup tables:

“Virtual DHT” table for GUID-to-NA lookup as needed

Conventional NA-to-port # forwarding table for “fast path”

Also, enhanced routing algorithm for store/forward decisions

WINLAB

GNRS + Storage Routing Performance

Result

Detailed NS3 Simulations to

compare MF with TCP/IP

Hotspot AP Deployment:

Includes gaps and overlaps

Cars move according to realistic

traces & request browsing type

traffic (req. size: 10KB to 5MB)

0 10 20 30 40 50 60 700

0.2

0.4

0.6

0.8

1

File Transfer Time (sec)

CD

F

Empirical CDF of file transfer time

MF: d = 200

TCP/IP: d = 200

MF: Avg. d = 400

TCP/IP: d = 400

d: Average distance between APs

0 50 100 150 2000

20

40

60

80

100

Time (sec)

To

tal D

ata

Re

ce

ive

d (

MB

its)

Single Car: Aggregate Throughput vs.Time

TCP/IP-30miles/hr

TCP/IP-50miles/hr

TCP/IP-70miles/hr

MF-30miles/hr

MF-50miles/hr

MF-70miles/hr

WINLAB

MF Protocol Example: Handling Disconnection

Data Plane

Send data file to “John Smith22’s

laptop”, SID= 11 (unicast, mobile

delivery)

NA99

NA75

Delivery failure at NA99 due to device mobility

Router stores & periodically checks GNRS binding

Deliver to new network NA75 when GNRS updates

GUID NA75

DATA

GUID NA99 rebind to NA75

DATA

DATA

GUID SID

DATA

SID GUID

NA99

Device

mobility

Disconnection

interval

Store-and-forward mobility service example

WINLAB

LTE/WiFi HetNet Results: MF vs. TCP

MF provides several benefits in a heterogeneous wireless

environment: Seamless mobility across network domains via dynamic GUID-NA bindings

Routers automatically store packets in transit during periods of disconnection

Simultaneous use of multiple networks is also possible

0 20 40 60 80 100 1200

100

200

300

400

500

600

700

800

900

1000

Time (sec)

Ag

gre

ga

te T

hro

ug

hp

ut (M

Byte

s)

Aggregate Throughput with Time

MobilityFirst

TCP/IP

Throughput boost

due to

transmission of

stored packets

TCP takes more time to

re-start session (DHCP

+ Application reset)

WINLAB

MF Protocol Example: Dual Homing Service

Data Plane

Send data file to “John Smith22’s

laptop”, SID= 129 (multihoming –

all interfaces)

NA99

NA32

Router bifurcates PDU to NA99 & NA32

(no GUID resolution needed)

GUID NetAddr= NA32

DATA

GUID NetAddr= NA99

DATA

DATA

GUID SID

DATA

SID GUID=

11001…11 NA99,NA32

DATA

Multihoming service example

WINLAB

MF Multipath Performance Result

Multipath service with data striping between LTE and WiFi

Using backpressure propagation and path quality info

GNRS Server

DataGUIDYChunk

NA1

Receiver: GUIDY

Sender: GUIDX

Query:(GUIDY)

Response:(NA1,NA2, Policy: Stripe)

DataGUIDY

Chunk

NA1

NA2

Chunk

Chunk Chunk

ChunkChunk

Chunk

DataGUIDY NA2

Backpressure

Path QualityNet Addr4NA1

36NA2

SplittingLogic

Back-pressure

0 10 20 30 40 50 60 70 80 90 1000

200

400

600

800

1000

Time (sec)

Ag

gre

ga

te T

hro

ug

hp

ut (M

b)

MobilityFirst Multihoming

Oracle Application

Using only LTE

Using only Wi-Fi

MobilityFirst Protocol

Prototyping & Validation

WINLAB

MF Host Protocol Stack

36

Network API

E2E Transport

GUID Services

Routing

‘Hop’ Link Transport

Interface Manager

WiFi WiMAX

App-1 App-2

Security

‘Socket’ API open send send_to recv recv_from close

Network Layer

User policies

Linux PC/laptop with WiMAX & WiFi

Android device with WiMAX & WiFi

Device: HTC Evo 4G, Android v2.3 (rooted), NDK (C++ dev)

Integrate

Early Dev.

Context API

App-3

Context Services

Sensors

WINLAB

MF Click Software Router

37

Inter-Domain (EIR)

Multicast

Lightweight, scalable multicast

• GNRS for maintenance of

multicast memberships

• Heuristic approaches to

reduce network load, limit

duplicated buffering, and

improve aggregate delivery

delays

• Click prototype, with SID for

multicast flows

• Evaluating hail a cab

application as a example

multipoint delivery scenario

WINLAB

OpenFlow/SDN Implementation of MF

38

MF Protocol Stack

Protocol stack embedded within controller

Label switching, NA or GUID-based routing (incl. GNRS lookup)

Controllers interact with other controllers and network support services such as GNRS

Flow rule is set up for the remaining packets in the chunk based on Hop ID (which is inserted as a VLAN tag in all packets) E.g., SRC MAC = 04:5e:3f:76:84:4a, VLAN = 101 => OUT PORT = 16

WINLAB

MF Router Prototype on FLARE SDN

Platform from U Tokyo (Nakao)

Objectives Multi-site deployment of MobilityFirst

routing and name resolution services

Impact of large RTTs on MobilityFirst

network protocols

High performance evaluation of

MobilityFirst delivery services on

FLARE - 1Gbps, 10Gbps

Augmented Click router elements

compiled down to FLARE native

Evaluation of FLARE platform for

design and evaluation of next-

generation network protocols

Demo at GEC-16, March 2013

WINLAB 40

GENI 4G Access Deployment at Rutgers

for Validation of 4G/WiFi Access Scenario

Rt.1 campus deployment since 2009

“Open Base Station” with external SDN/VM

controller enabling experimentation with

new network protocols

Integrated with GENI control framework

Omni-directional antenna

(elev. < 6ft above roof!)

Outdoor Unit (ODU)

RF Module ( sector)

Base Module

Open

WiMAX in Phase 1

LTE in Phase 2

WINLAB

MF Multi-Site GENI Deployment –

Demo at GEC-16, March 2013

Salt Lake, UT

Cambridge,

MA

N. Brunswick,

NJ

Ann Arbor, MI Madison, WI

Tokyo, Japan

Lincoln, NE

Los Angeles,

CA Clemson,

SC

Long-term (non-

GENI)

MobilityFirst Access

Net

Short-term

Wide Area ProtoGENI

Palo Alto, CA

ProtoGENI

MobilityFirst

Routing and Name

Resolution

Service Sites

I2

NL

R

Atlanta, GA

WINLAB

Resources

Project website: http://mobilityfirst.winlab.rutgers.edu

GENI website: www.geni.net

ORBIT website: www.orbit-lab.org