Rfc 4594 Configuration Guidelines for Diffserv Service Classes
QBone Service Specification and TE Early DiffServ Deployment Experiences and Reduced Expectations
description
Transcript of QBone Service Specification and TE Early DiffServ Deployment Experiences and Reduced Expectations
Ben Teitelbaum <[email protected]>January 25th, 2001TEQUILA Workshop on Internet Design for SLS Delivery
QBone Service Specification and TEEarly DiffServ Deployment Experiences and Reduced Expectations
Service Specification and TE for the QBone—Amsterdam (January 25th, 2001) 2
“The Holy Grail of computer networking is to design a network that has the flexibility and low cost of the Internet, yet offers the end-to-end quality of service guarantees of the telephone network.”
- S. Keshav
Service Specification and TE for the QBone—Amsterdam (January 25th, 2001) 3
The Paradox of Internet QoS
Internet’s success largely due to its lack of QoS!
Best-Efforts: scalability on steroids• Routers: complexity of forwarding very low • L2: minimal requirements• Providers: “magic”-free provisioning and low user expectations• Developers: E2E transport and simplicity of Berkeley socket
interface
QoS complicates everything (costs), but opens many opportunities (benefits)
• Developers: new apps exploit the ubiquity of the Internet• Providers: may make more efficient use of the network
Big Question: How can we build upon DiffServ without compromising scalability?
Service Specification and TE for the QBone—Amsterdam (January 25th, 2001) 4
architecture
deployment
How We Got Here (short version)
• Began chanting: “enable advanced applications,…”
• Assessed requirements “What do you want?”“What can you give us?”
• Recommended DiffServ• Selected “Premium” service to meet demands of loss/jitter sensitive apps
• Charted QBone initiative • Specified QBone architecture• Now proceeding to implement it and tweak the architecture
Service Specification and TE for the QBone—Amsterdam (January 25th, 2001) 5
QBone Architecture
A Service: QBone Premium Service• Built on Expedited Forwarding (EF) (RFC 2598)• Assurance: near-zero loss & low, bounded jitter for marked traffic conforming to a specified peak rate
–a.k.a. “virtual leased line”, “virtual wire”
Reservation Setup Protocol• Now: long-lived, manual setup• Proposed: SIBBS protocol between QBone domains; RSVP end-to-end between hosts
QBone Measurement Architecture• Uniform collection of QoS metrics• Uniform dissemination interface
Service Specification and TE for the QBone—Amsterdam (January 25th, 2001) 6
QBone Measurement Architecture1/2
Collection metrics, EF and BE...• Active metrics (paths)
– One-way delay-variation
– One-way loss
– Traceroutes
– e.g. IPPM Surveyors
• Passive metrics (interfaces)– Load
– EF reservation load
– Discards (suggested)
– Link bandwidths (suggested)
– e.g. OCxMon, RTFM, MIBs
Boundary Router
Intra-Domain Premium Path
Inter-Domain Premium Path
ActiveMeasurements
MIB-basedstatistics
PassiveMeasurements
PassiveMeasurements
PM node PM node
AM node
QBone Domain2
QBone Domain1
QBone Domain3
Service Specification and TE for the QBone—Amsterdam (January 25th, 2001) 7
QBone Measurement Architecture2/2
Dissemination • Standard URL query syntax:
• whois server to learn canonical names for QBone domains, routers, sniffers, etc
label ::= <alphanum> { <alphanum> }router ::= label"-ROUTER”probe ::= label"-PROBE”sniffer ::= label "-SNIFFER”host ::= router | probe | snifferpath ::= host "/" hostPHB ::= "BE" | "EF”metric ::= "LOSS" | "ONEWAY" | "PING" | "IPDV" | "LOAD" | "TRACEROUTE" | "COMMITMENT" | "RESERVATION"year ::= digit digit digit digitmonth ::= "01" | "02" | ... | "12”day ::= "01" | "02" | ... | "31”YYYYMMDD ::= year month dayaggregation ::= <unsigned_integer>prefix ::= “http” | “ftp” | <other>query ::= prefix ”://" path "/" PHB "/" metric "/” YYYYMMDD "/" metric "/" aggregation
Service Specification and TE for the QBone—Amsterdam (January 25th, 2001) 8
Simple Interdomain Bandwidth Broker Signaling (SIBBS)
Basic Idea• Simple domain-to-domain reservation request/response protocol to
signal changes in the aggregate reservations of globally well-known services
Design Goals• Simple & extensible• Bootstrap wide-area experimentation with QoS-needy apps, while
providing hooks for clouds to experiment with various QoS TE approaches
• Integrate with end-to-end signaling capabilities of hosts
Status• Protocol draft nearing completion
(see: http://qbone.internet2.edu/bb/)• New protocol for now; looking at mapping to COPS
Service Specification and TE for the QBone—Amsterdam (January 25th, 2001) 9
SIBBS: Basic Reservation Setup
BB
BB
BB
RTR ES RTR RTR RTR ES
RAR RAR
RAA RAA
Web interface RSVPDIAMETERCOPSRYO
CLISNMPCOPS
Service Specification and TE for the QBone—Amsterdam (January 25th, 2001) 10
Virtual Peerings & Core Tunnels
BB BB BB SIBBS SIBBS
Src
RTR
RTR
Dest
“Virtual” SIBBS peering BB BB
“Core Tunnel”
Service Specification and TE for the QBone—Amsterdam (January 25th, 2001) 11
SIBBS Approach to SLSs
Basic approach:• Define services and reservations, but punt on SLSs
Services • Globally specified, standardized, and named• Include many parameters from draft-tequila-sls (scope,
traffic envelope, performance guarantees, etc)
Reservations• Identify a service ID• Specify service parameters
SLSs• Each SLS along a reservation path should be regarded as a black
box that answers admissions questions and extends reservations across the peerings
Service Specification and TE for the QBone—Amsterdam (January 25th, 2001) 12
Simple Interdomain Bandwidth Broker Signalling (SIBBS)
BB BB BB
H H
DS Domain1 DS Domain3DS Domain2
S LS1-2 S LS2-3
R RRR
Control
Data
Service Specification and TE for the QBone—Amsterdam (January 25th, 2001) 13
QBone E2E Picture
GigaPoPA
CampusA
CampusB
GigaPoPB
CampusC
CampusD
Backbone
X Kbps of QPS from hither to………..yon
Key
Service Specification and TE for the QBone—Amsterdam (January 25th, 2001) 14
Abilene TopologyJanuary, 2001
•47 connectors •183 participants
•34 connections to 20 peer networks
Service Specification and TE for the QBone—Amsterdam (January 25th, 2001) 15
Miami
NYC (60 Hudson)
LA
AbileneSingaren
SEATTLE
NYC (Telehouse
25 Broadway)
STARTAP
ArgentinaBrazilChileColumbiaBrazil
Developing International Peerings
Courtesy: Linda Winkler, STAR TAP
CERNSINET
AbileneESnetvBNS
JAnetSURFnetNORDUnet
AbileneESnetCA*net3AARnetCERNET
CUDIHARNETSINET
AbilenevBNSESnetDRENNRENNISN
CA*net3SURFnetMIRnetNORDUnetREUNA
IUCCAPANCERNSINETSingarenTanet
GEMNETRenater
DFNINFNDANTECA*net3
Service Specification and TE for the QBone—Amsterdam (January 25th, 2001) 16
Abilene Load Snapshot
Service Specification and TE for the QBone—Amsterdam (January 25th, 2001) 17
APS Participation
Goal:• Make APS a reference implementation of the QBone architecture
Current Participants• MAGPI (U. Penn)• iCAIR• PSC (Penn State)• OARNet (Ohio State)• NASA EOS
Others in the wings• TF-NGN (through DANTE)• MIRnet• Various other international
• ANL• UIUC• DOE Science GRID (peering transit network)
Service Specification and TE for the QBone—Amsterdam (January 25th, 2001) 18
Initial Engineering Plan (obsolete)
Measurement
Edge Policing
Manual Setup
EF Core Forwarding
Automated Setup
EF Edge Forwarding
Shaping
Sweetwater Midland Odessa Pecos
(GTS)
(Surveyor + SNMP + HTTP)
(“Firehose” CAR)
(Whiteboard + CLI)
(MDRR)
(MDRR)
(BB)
Service Specification and TE for the QBone—Amsterdam (January 25th, 2001) 19
APS Measurements
Status: • Collecting EF/BE loads and CAR conform/exceed stats• Not currently monitoring IPDV, but Abilene Surveyor nodes now OC-3 connected and operational
• Ohio-ITEC hosting APS measurements and QBone-wide whois server
Near Future: • IPDV along edge-to-edge QBone paths• Abilene Surveyor timing improvements
–Better NTP–New CDMA timing sources (can't get GPS in Qwest POPs)
• Collection of AS-level traffic matrices
Service Specification and TE for the QBone—Amsterdam (January 25th, 2001) 20
Committed Access Rate (CAR)
Classifies traffic based on certain matching criteria (interface, DSCP, or ACL) and meters it to a leaky bucket traffic profile
Depending on metering result, different actions applied (drop, transmit, set DSCP,…)
Syntax:rate-limit {input | output} [access-group [rate-limit] acl-index] bps burst-normal burst-max conform-action action exceed-action action
Service Specification and TE for the QBone—Amsterdam (January 25th, 2001) 21
CAR Experience
For the most part, CAR is exactly what the DiffServ doctor ordered
However, there are some limitations…• PPS performance cost• Quirky constraints on token bucket depths• Not easy to do "virtual trunk" style classification
Problem: we want...
"virtual trunk""firehose"
But what we get is…
Service Specification and TE for the QBone—Amsterdam (January 25th, 2001) 22
Abilene Architecture Limitation: “Porous” Edge Problem
DoQoS problem with current architecture
MDRR (EF forwarding) on interior interfaces easily subverted by unpoliced connectors
Service Specification and TE for the QBone—Amsterdam (January 25th, 2001) 23
How to “Crisp” the Edge?
Problem: EF requires that all connectors be policed
Solution:• Short term: Stochastically detect illegal EF traffic with NetFlow and/or OCxMon passive monitoring; gigaPoP would be asked to police
• Longer term: Wait for E3 edge cards, deploy them aggressively, and police everywhere
• Road not taken:Re-write all non-participant traffic with DSCP 000000 (using PIRC hack); need to pass some DSCP values (reason why coming up…)
Service Specification and TE for the QBone—Amsterdam (January 25th, 2001) 24
Looking Forward: Resource Accounting and Admissions
Problem: how to account for link EF capacity and commitment in routed network?
Potential solution: DiffServ-Aware MPLS-TE• Basic idea:
–Edge-to-edge MPLS tunnels–LSP setup constrained by available EF bandwidths–OSPF augmented to carry QoS link state attributes–See draft-lefaucheur-diff-te-reqts
• “Solves” DiffServ admissions problem• Does not solve general DiffServ provisioning problem• Status: completed extensive lab test of Cisco’s implementation of this idea; considering field trial
Service Specification and TE for the QBone—Amsterdam (January 25th, 2001) 25
Current Engineering Plan
Measurement
Edge Policing
Manual Setup
EF Core Forwarding
Automated Setup
EF Edge Forwarding
Shaping
Sweetwater Midland Odessa Pecos
(GTS)
(Surveyor + SNMP + HTTP + whois + traffic matrices)
(CAR + QPPB)
(Whiteboard + CLI)
(MDRR)
(MDRR)
(SIBBS? + DSTE?)
Service Specification and TE for the QBone—Amsterdam (January 25th, 2001) 26
QPS Status (Good News)
The Good News:• With enough elbow grease, E2E services can be built• DiffServ functionality in most modern routers• Many hosts now support QoS signaling (RSVP)• Numerous testbed trials• Partial implementations in ESnet and Abilene will form nucleus of QBone
Service Specification and TE for the QBone—Amsterdam (January 25th, 2001) 27
SC2000 Interdomain QoS DemoNovember 6-9, 2000
Premium service over two wide-area paths• LBNL-ESnet-Abilene-SCinet-Internet2 booth• Stanford-CalREN2-Abilene-SCinet-Internet2 booth
Congestion induced at multiple points
CD-quality interactive audio application shown with/without QoS
SC2000 Network Challenge Winner: Award for "Most Captivating and Best Tuned" Demo
Service Specification and TE for the QBone—Amsterdam (January 25th, 2001) 28
Service Specification and TE for the QBone—Amsterdam (January 25th, 2001) 29
QPS Status (Bad News)
The Bad News:• Router vendor hype/reality mismatch• Deploying QPS “requires upgrading the world”• Low demand for QPS (app/net chicken/egg dynamic)• Elbow grease in short supply in most campus IT shops
Service Specification and TE for the QBone—Amsterdam (January 25th, 2001) 30
Bottom Line
Progress on QPS will continue to be slow
Internet2 will continue with a QPS work program in 2001…
• Deploy incrementally where it counts• Revise/complete architecture (joint work with TF-NGN)• Work to build QoS-sensitive application user communities who
understand their needs
.However, QPS will no longer be the only Internet2 QoS activity
• Need to reap lower hanging fruit from DS (I2SS)• Much stronger push on measurements/monitoring• Big Internet2 E2E performance initiative ramping up
Service Specification and TE for the QBone—Amsterdam (January 25th, 2001) 31
E2E Performance and Measurements
Internet2 E2E Performance Initiative• Typical BE TCP throughputs often much less than one would expect• Common problems
– Broken TCP stacks
– Ethernet auto-negotiation failures
• Evolving attributes of initiative– Performance Emergency Response Teams (PERTs)
– Sharper tools for measurement, monitoring, and analysis
Measurement arsenal• WEB100 (Mathis et al.)• Open Internet2 measurement architecture (à la QBone)• Open source and protocols for one-way delay measurement• Application-level performance fault-isolation (reflector infrastructure)
Service Specification and TE for the QBone—Amsterdam (January 25th, 2001) 32
Internet2 Scavenger Service
Basic idea• A lower priority class of best-effort• Voluntary marking hints to network that degraded service is OK (think of a "nice" for the network)
Intended uses• Non-time-critical traffic (e.g. server-to-server NNTP, anonymous FTP, network backups)
• Bulk data transfers using TCP• Non-mission applications (e.g. Napster, games, etc)• New kinds of distributed applications that attempt to use idle network capacity
Service Specification and TE for the QBone—Amsterdam (January 25th, 2001) 33
I2SS Service Specification
Rigorously defining the E2E I2SS is difficult!• Hoping to define relative to best effort
I2SS traffic indicated by DSCP 001000• Modification of class selector PHB• Note that the I2SS codepoint has global significance
I2SS domain requirements• Traffic leaving must be marked I2SS, if it entered so
Router requirements (all SHOULDs)• Forward I2SS independently giving it a lower probability of timely
forwarding OR forward in same manner as BE• Offer I2SS a very small minimum departure rate (unlike BH PDB)• Offer I2SS all un-used bandwidth
Service Specification and TE for the QBone—Amsterdam (January 25th, 2001) 34
For more information...
Internet2 Home:• http://www.internet2.edu/
Internet2 QoS Working Group Home:• http://www.internet2.edu/wg/qos/
QBone Home:• http://qbone.internet2.edu/
Abilene Premium Service Home:• http://www.internet2.edu/abilene/qos/
Service Specification and TE for the QBone—Amsterdam (January 25th, 2001) 35