1

Overview and Status Update of

Hui Zhang

Carnegie Mellon

2

Slides Used In First Retreat

Vision: 100 Mbps to 100 million households

1Gbps to 1 million business

Approach: clean slate, design from first principles

Holistic and inter-disciplinary design Consider the network as a whole

– Access, metro, core regions

– Service and physical transport layers

Consider technology trends for scaling, cost, future-safeness

Archiect with explicit considerations of economic, dependability, security, mult-service

Design with explict goals of enabling tractable analysis and modeling

3

What is the 100x100 Project?

Elevator Speech

What is 100x100 project up to?

What is the “meat”?

– What are the key technical challenges?

– What are the key insights and expected contributions?

Is 100x100 project just about speed?

Why 100 Mbps?

– It is too slow, it is too fast

Why cannot IP do 100x100?

4

Clean Slate Design

Why is Clean State Design necessary?

Have you established that incremental evolution is impossible?

Is Clean State Design useful?

Any hope of introducing a “revolutionary” solution?

Mainstream views

incremental fixing of Internet is sufficient

radical new solutions have little chance of being deployed

6

Context

IP is a great success because its architects set the right goal: global best-effort reachability Global addressing scheme Internetworking architecture Simple best-effort service

Success is a double-edged sword the world demands more from IP and the Internet Convergence telecommunication infrastructure that

provides 7x24x365 service Dependability, scale, security, economic sustainability

7

Convergence Vision

Long-haul DWDM backbone(OC48/192)

POP -

VoIP + Data + Video over IP

8

Have We Already Achieved Convergence?

Long HaulMetro CoreSubscriber/

Enterprise

LAN

HAN

Metro Hub Office

End Office/ Collocation

INTERCITYG()

RouterVoice

Switch

ACCESS INTEROFFICEG(SONET)

TransportServices

Metro Access

Router

Service Node/ASPVoice

Switch

Wireless

Server

Server

RF

Cable

Copper

Fiber

ATM

OXC

Voice Switch

Backbone Router

ISP

9

UUNet 10/03/02 Outage

as seen from AT&T

IP Robustness?

State of the Art is lacking in robustness on every time scale: days, hours, minutes, seconds

10

Molasses

Data Plane Distributed routers forwarding

packets

Based on FIB or labels

OSPFBGP

OSPFBGP

OSPFBGP

Packet filters

Link metrics

Control Plane• Multiple routing processes on each

router• Each router with different

configuration program• Huge number of control knobs:

metrics, ACLs, policy

FIB

FIB

FIB

Configs

Management Plane• Figure out what is happening in

network• Decide how to change it

Shell scripts Tomography

DatabasesPlanning tools

OSPFSNMP rancid modems

11

A Study of Operational Production Networks(Joint Between ATT and CMU)

Obtained anonymized configuration files for 31 active networks (>8,000 configuration files)

6 Tier-1 and Tier-2 Internet backbone networks

25 enterprise networks

Sizes between 10 and 1,200 routers

12

Configuration State for One Network

13

Example Router Configuration File

14

Packet Filters FIBs

Data Plane ControlPlane

ManagementPlane

Log

ic t

o C

ombi

ne

OS

PF

and

BG

P R

IBs

Hardwired State

Dynamic State

Configuration State

State Dependency

Legends

Complex Interaction of States

15

Molasses

Data Plane

Distributed routers forwarding packets

Based on FIB or labels

OSPFBGP

OSPFBGP

OSPFBGP

Packet filters

Link metrics

Control Plane• Multiple routing processes

on each router• Each router with different

configuration program• Huge number of control

knobs: metrics, ACLs, policy

Management Plane• Figure out what is

happening in network• Decide how to change it

Shell scripts Tomography

DatabasesPlanning tools

State everywhere!

• Dynamic state in FIBs

• Configured state in settings, policies, packet filters

• Programmed state in magic constants, timers

• Many dependencies between bits of state

State updated in uncoordinated, decentralized way!

16

Systems of Systems

Systems are designed as components to be used in larger systems in different contexts, for different purposes, interacting with different components

Example: OSPF and BGP are complex systems in its own right, they are components in a routing system of a network, interacting with each other and packet filters, interacting with management tools …

Complex configuration to enable flexibility

The glue has tremendous impact on network performance

State of art: multiple interactive distributed programs written in assembly language

Lack of intellectual framework to understand global behavior

17

Development of the Elevator Pitch

Best-effort service model is a critical reason for Internet’s success

Success is a double-edged sword

Expectation: with true two-way broadband connectivity to every household (100x100), IP be convergence telecommunication infrastructure that provides 7x24x365 service

Far from being dependable, scaleable, secure, economic sustainable

Even when providing best-effort service, it is already extremely complex

18

What is the Solution?

Simplify,

Simplify,

simplify!

But based on what?

19

Observations that Lead to Simplifications

Tremendous focus on protocols/protocol architecture, should also focus on networks and network architecture

Interplay between network architecture and protocol architecture

Architecture agnostic to technology trend Should leverage technology trends that enable simple network architecture

Box/Switch/Router centric distributed control has fundamental limitations, should also explore

Control architecture that can implement a wide range of network-wide

policies

20

Protocol Architecture vs. Network Architecture

Protocols: TCP, IP, BGP, OSPF

They work in arbitrary network, but do not work as well in any network

Dependable network requires

– Structured network, and

– protocols that can take advantage of network architecture

21

Structured Access/Metro Networks

22

Structured Backbone Networks

23

Structured Network Interconnection

Backbone Network

Access Routers

CPE Routers

24

Observation One

Structured network + protocols taking advantage of network structures achieve

Dependable, simple, and understandable network

Dependable, simple, and understandable protocols

25

Observations that Lead to Simplifications

Tremendous focus on protocols/protocol architecture, should also focus on networks and network architecture

Interplay between network architecture and protocol architecture

Architecture agnostic to technology trend Should leverage technology trends that enable simple network architecture

Box/Switch/Router centric distributed control has fundamental limitations, should also explore

Control architecture that can implement a wide range of network-wide

policies

26

Key Technology Trends

Trend 1: Deep fiber deployment key to achieving ubiquitous, high capacity connectivity to home Wireless: important complementary technology

Copper & cable can also be used for last 1000 feet

Fiber enables not only scalable high speed, but also longer distance

Trend 2: low cost, low energy, high capacity, auto-configured, environmentally hardened access packet switches

Trend 3: ultra high capacity (perabit) backbone switches

27

Implications for Network Architecture

Deep fiber + access switch large scale packet access networks

In contrast, traditional access network size limited by copper transmission distance

Large scale access network + ultra high speed backbone switch architecture with

regional node that terminates access network and,

backbone network with a smaller number of richly connected switches

28

An Example Structured 100x100 Network

29

Observations that Lead to Simplifications

Tremendous focus on protocols/protocol architecture, should also focus on networks and network architecture

Interplay between network architecture and protocol architecture

Architecture agnostic to technology trend Should leverage technology trends that enable simple network architecture

Box/Switch/Router centric distributed control has fundamental limitations, should also explore

Control architecture that can implement a wide range of network-wide

policies

30

Robust, Simple Control and Management

Network is about coordination of switches Distributed state management Multiple goals

– Reachability

– Policy control

– Security

– Resiliency

– Traffic Engineering, load balancing

– VPN

Multiple layers (switching + optical) Diverse switching primitives (OXC, label switches, IP switches, vLAN

switches)

Status quo of control and management: extreme complex, non-linear and fragile

31

Good Abstractions Reduce Complexity

All decision making logic lifted out of control plane

Eliminates duplicate logic in management plane

Dissemination plane provides a control channel to/from data plane

ManagementPlane

Control Plane

Data Plane

DecisionPlane

Dissemination

Data Plane

Configs

FIBs FIBs

32

Development of the Elevator Pitch

Internet technology is far from being adequate to support 100x100 vision

Goal: dependable, scaleable, secure, and economic sustainable telecommunication infrastructure

It is already extremely complex

Solution: simplify, simply, simply!

Simplicity should also mean that the design is understandable

Three key ideas that may lead to simplification

Take advantage of structured network

Leverage and develop technologies that enable structured networks

Design simple and powerful network-wide control abstractions/mechanisms

33

Why 100x100? Why not 1000x100?

Key is to focus us on an end goal of a network

Principles derived can be general

34

Why Clean State Design?

A powerful research methodology that helps to crystallize the issues Many good examples in systems research that take one idea to extreme:

RISC, SmallTalk, NFS, IP

A mind set that may result in different research, e.g. Incremental approach to security

– How to detect and stop Blaster, Code Red?

Clean state design approach– What would be the fundamental capability of a strategic adversary?

– What are the fundamental limitations/possibilities of any network-based or host-based security mechanism?

– What should be the minimal & necessary set of layer 3 security mechanism?

A concrete and complete different design point highlights possibilities Understanding the target first helps to plan the trajectory of evolution

35

Big Bets Research

We make big bets to get big breakthroughs

Visionary ideas carrying intellectual risk

Can’t predict outcomes in advance

The Christopher Columbus Effect

Randy Bryant: Dean of SCS, CMU“Strategic Vision for CS in CMU”

36

Can We Make a Difference?

Monopoly positions in all technology areas

Microsoft in OS

Cisco in router

Intel in processor

Oracle in database

People are usually

too optimistic in prediction of two years out, but

too pessimistic in prediction of five or ten years out

37

Can We Change IP and the Internet?

What is IP anyway? Service interface: services seen by VoIP, Web

Data plane (IPv4, IPv6 packet formats)

Control plane (OSPF, ISIS, BGP, LDP)

Management plane

38

Learning from Ethernet Evolution Experience

Current Implementations:

Everything Changed Except Name and Framing

Ethernet

Conc..

Router

Server

WAN

HUB

Switch

•Switched solution

•Little use for collision domains

•80% of traffic leaves the LAN

•Servers, routers 10 x station speed

•10/100/1000 Mbps, 10gig coming: Copper, Fiber

WAN

LAN

Ethernet or 802.3

•Bus-based Local Area Network

•Collision Domain, CSMA/CD

•Bridges and Repeaters for distance/capacity extension

•1-10Mbps: coax, twisted pair (10BaseT)

B/R

Early Implementations

39

How To Evolve?

One possible path to future Fix packet format: IP or IPv6 or Ethernet or MPLS frame

formats Evolve switch-switch protocols (NNI)

– control/management plane, where the intelligence is

40

Telephone Network

ATM Net

X.25 Net

41

ATM Net

Telephone

Network

X.25 Net IP

42

ATM Net

Telephone Network

X.25 Net IP

43

Telep

ATM

X.25

IP

44

Telep ATM

X.25 IP