Course Overview and Introduction Nick Feamster CS 6250: Computer Networking Fall 2011.
Networking Nick Feamster Georgia Tech. 2 Goal of This Tutorial Teach engineers the basics of...
-
Upload
caleb-roach -
Category
Documents
-
view
219 -
download
5
Transcript of Networking Nick Feamster Georgia Tech. 2 Goal of This Tutorial Teach engineers the basics of...
Networking
Nick FeamsterGeorgia Tech
2
Goal of This Tutorial
bull Teach engineers the basics of networking and ISP operations
bull Networks todayndash Business modelsndash Operations (NOC operators)
bull Common problemsbull Measurement Monitoring and Security
3
Todayrsquos Networks
bull Service provider business models
bull Network operations center
bull Network operators and engineers
4
Business Models
bull Increasingly commoditized (see Geoff Hustonrsquos talk at NANOG)
bull Status quo Establish transit costs bill at 95th percentile of usage
bull Future differential pricing preference for certain groups of users applications
5
Billing for Internet Usage
bull 95th Percentile billingndash Customer network pays for ldquocommitted information
raterdquo (CIR)ndash Throughput measured every 5 minutes (typically with
SNMP flow statistics also can be used for billing)ndash Customer billed based on 95th percentile
6
Net Neutrality
7
Network Operations
bull Operators run the day-to-day operations of the networkndash Adjusting to shifts in traffic failures etcndash Responding to security threatsndash Provisioning new customers
8
Point-of-Presence (PoP)
bull A ldquoclusterrdquo of routers in a single physical location
bull Inter-PoP linksndash Long distances
ndash High bandwidth
bull Intra-PoP linksndash Cables between racks or floors
ndash Aggregated bandwidth
PoP
9
Example Abilene Network Topology
10
Another Example Backbone
11
GeorgiaTech
Internet Routing Overview
bull Intradomain (ie ldquointra-ASrdquo) routingbull Interdomain routing
Comcast
Abilene
ATampT Cogent
Autonomous Systems (ASes)
12
Internet Routing Protocol BGP
Route Advertisement
Autonomous Systems (ASes)
Session
Traffic
Destination Next-hop AS Path1302070016
1302070016
19258989
6625025244
105782637
174hellip 2637
13
Two Flavors of BGP
bull External BGP (eBGP) exchanging routes between ASes
bull Internal BGP (iBGP) disseminating routes to external destinations among the routers within an AS
eBGPiBGP
Question Whatrsquos the difference between IGP and iBGP
14
IPv4 Addresses Networks of Networks
bull 32-bit number in ldquodotted-quadrdquo notation
ndash wwwccgatechedu --- 130207736
10000010 11001111 00000111 00100100
Network (16 bits) Host (16 bits)
130 207 7 36
bull Problem 232 addresses is a lot of table entries
bull Solution Routing based on network and host
ndash 1302070016 is a 16-bit prefix with 216 IP addresses
Topological Addressing
15
Pre-1994 Classful Addressing
Network ID Host ID
8 16
Class A
32
0
Class B 10
Class C 110
Multicast AddressesClass D 1110
Reserved for experimentsClass E 1111
24
8 blocks (eg MIT has 180008)
16 blocks (eg Georgia Tech has 1302070016)
24 blocks (eg ATampT Labs has 19220225024)
Simple Forwarding Address range specifies network ID length
16
Classless Interdomain Routing (CIDR)
IP Address 65142480 ldquoMaskrdquo 2552552520
01000001 00001110 11111000 00000000
11111111 11111111 11111100 00000000
Use two 32-bit numbers to represent a network Network number = IP address + Mask
Example BellSouth Prefix 6514248022
Address no longer specifies network ID rangeNew forwarding trick Longest Prefix Match
17
Benefits of CIDR
bull Efficiency Can allocate blocks of prefixes on a finer granularity
bull Hierarchy Prefixes can be aggregated into supernets (Not always done Typically not in fact)
Customer 1
Customer 2
ATampT Internet
1220249024
1220231024120008
18
Growth of IP Prefixes
19
1994-1998 Linear Growth
bull About 10000 new entries per yearbull In theory less instability at the edges (why)
Source Geoff Huston
20
Around 2000 Fast Growth Resumes
Claim remaining 8s will be exhausted within the next 5-10 years
T Hain ldquoA Pragmatic Report on IPv4 Address Space Consumptionrdquo Cisco IPJ September 2005
21
Fast growth resumes
Rapid growth in routing tables
Dot-Bomb Hiccup
Significant contributor Multihoming
Source Geoff Huston
22
The Address Allocation Process
bull Allocation policies of RIRs affect pressure on IPv4 address space
IANA
AfriNIC APNIC ARIN LACNIC RIPE
httpwwwianaorgassignmentsipv4-address-space
Georgia Tech
23
Common Problems
bull Diagnosis and troubleshooting (hence measurement)bull Traffic engineeringbull Securitybull Design and capacity planning
24
What can go wrong
Two-thirds of the problems are caused by configuration of the routing protocol
Some downtime is very hard to protect againsthellip
25
Measurement and Monitoring
26
Passive vs Active Measurement
bull Passive Measurement Collection of packets flow statistics of traffic that is already flowing on the networkndash Packet tracesndash Flow statisticsndash Application-level logs
bull Active Measurement Inject ldquoprobingrdquo traffic to measure various characteristicsndash Traceroutendash Pingndash Application-level probes (eg Web downloads)
27
Billing for Internet Usage
bull 95th Percentile billingndash Customer network pays for ldquocommitted information
raterdquo (CIR)ndash Throughput measured every 5 minutes (typically with
SNMP flow statistics also can be used for billing)ndash Customer billed based on 95th percentile
28
Passive Traffic Data Measurement
bull SNMP bytepacket counts everywhere
bull Packet monitoring selected locations
bull Flow monitoring typically at edges (if possible)ndash Direct computation of the traffic matrixndash Input to denial-of-service attack detection
bull Deep Packet Inspection also at edge where possible
29
Simple Network Management Protocol
bull Management Information Base (MIB)ndash Information storendash Unique variables named by
OIDsndash Accessed with SNMP
bull Specific MIBs for bytepacket counts (per link)
Manager Agent
SNMP
DB
ManagedObjects
30
SNMP (Passive)
bull Advantage ubiquitousndash Supported on all networking equipmentndash Multiple products for polling and analyzing data
bull Disadvantages ndash Coarse granularityndash Cannot express complex queries on the datandash Unreliable delivery of the data using UDP
bull Utilityndash Link utilization (billing)ndash Traffic matrix inference
31
Packet-level Monitoring
bull Passive monitoring to collect full packet contents (or at least headers)
bull Advantages lots of detailed informationndash Precise timing informationndash Information in packet headers
bull Disadvantages overheadndash Hard to keep up with high-speed linksndash Often requires a separate monitoring device
32
Full Packet Capture (Passive)
Example Georgia Tech OC3Mon
bull Rack-mounted PCbull Optical splitterbull Data Acquisition and
Generation (DAG) card
Source endacecom
33
What is a flow
bull Source IP addressbull Destination IP addressbull Source portbull Destination portbull Layer 3 protocol typebull TOS byte (DSCP)bull Input logical interface (ifIndex)
34
Cisco NetFlow
bull Basic output ldquoFlow recordrdquondash Most common version is v5
bull Current version (9) is being standardized in the IETF (template-based)ndash More flexible record formatndash Much easier to add new flow record types
Core Network
Collection and Aggregation
Collector (PC)Approximately 1500 bytes
20-50 flow recordsSent more frequently if traffic increases
35
Flow Record Contents
bull Source and Destination IP address and portbull Packet and byte countsbull Start and end timesbull ToS TCP flags
Basic information about the flowhellip
hellipplus information related to routing
bull Next-hop IP addressbull Source and destination ASbull Source and destination prefix
36
flow 1 flow 2 flow 3 flow 4
Aggregating Packets into Flows
bull Criteria 1 Set of packets that ldquobelong togetherrdquondash Sourcedestination IP addresses and port numbersndash Same protocol ToS bits hellip ndash Same inputoutput interfaces at a router (if known)
bull Criteria 2 Packets that are ldquocloserdquo together in timendash Maximum inter-packet spacing (eg 15 sec 30 sec)ndash Example flows 2 and 4 are different flows due to time
37
Reducing Measurement Overhead
bull Filtering on interfacendash destination prefix for a customerndash port number for an application (eg 80 for Web)
bull Sampling before insertion into flow cachendash Random deterministic or hash-based samplingndash 1-out-of-n or stratified based on packetflow sizendash Two types packet-level and flow-level
bull Aggregation after cache evictionndash packetsflows with same next-hop ASndash packetsflows destined to a particular service
38
Packet Sampling for Flow Monitoring
bull Packet sampling before flow creation (Sampled Netflow)ndash 1-out-of-m sampling of individual packets (eg m=100)ndash Create of flow records over the sampled packets
bull Reducing overheadndash Avoid per-packet overhead on (m-1)m packetsndash Avoid creating records for a large number of small flows
bull Increasing overhead (in some cases)ndash May split some long transfers into multiple flow records ndash hellip due to larger time gaps between successive packets
time
not sampled
two flowstimeout
39
Sampling Flow-Level Sampling
bull Sampling of flow records evicted from flow cachendash When evicting flows from table or when analyzing flows
bull Stratified sampling to put weight on ldquoheavyrdquo flowsndash Select all long flows and sample the short flows
bull Reduces the number of flow records ndash Still measures the vast majority of the traffic
Flow 1 40 bytesFlow 2 15580 bytesFlow 3 8196 bytesFlow 4 5350789 bytesFlow 5 532 bytesFlow 6 7432 bytes
sample with 100 probability
sample with 01 probability
sample with 10 probability
40
Two Main Approaches
bull Packet-level Monitoringndash Keep packet-level statisticsndash Examine (and potentially log) variety of packet-level
statistics Essentially anything in the packetndash Timing
bull Flow-level Monitoringndash Monitor packet-by-packet (though sometimes
sampled)ndash Keep aggregate statistics on a flow
41
Packet Capture on High-Speed Links
Example Georgia Tech ldquoOC3Monrdquo
bull Rack-mounted PCbull Optical splitterbull Data Acquisition and
Generation (DAG) card
Source endacecom
42
Characteristics of Packet Capture
bull Allows inspection on every packet on 10G links
bull Disadvantagesndash Costlyndash Requires splitting optical fibersndash Must be able to filterstore data
43
Data Measurement Repositories
bull AbileneInternet 2 Observatoryndash Configuration examplesndash SNMP datandash ISIS BGP routing data NetFlow traffic data
bull RouteViewsndash BGP updatesndash BGP table snapshots
44
Multihoming and Traffic Engineering
45
What is Multihoming
bull The use of redundant network links for the purposes of external connectivity
bull Can be achieved at many layers of the protocol stack and many places in the networkndash Multiple network interfaces in a PC
ndash An ISP with multiple upstream interfaces
bull Can refer to having multiple connections tondash The same ISP
ndash Multiple ISPs
46
Why Multihome
bull Redundancybull Availabilitybull Performancebull Cost
Interdomain traffic engineering the process by which a multihomed network configures its
network to achieve these goals
47
Redundancy
bull Maintain connectivity in the face ofndash Physical connectivity problems (fiber cut device
failures etc)ndash Failures in upstream ISP
48
Performance
bull Use multiple network links at once to achieve higher throughput than just over a single link
bull Allows incoming traffic to be load-balanced
70 of traffic30 of traffic
49
Multihoming in IP Networks Today
bull Stub AS no transit service for other ASesndash No need to use BGP
bull Multi-homed stub AS has connectivity to multiple immediate upstream ISPsndash Need BGPndash No need for a public AS numberndash No need for IP prefix allocation
bull Multi-homed transit AS connectivity to multiple ASes and transit servicendash Need BGP public AS number IP prefix allocation
50
BGP or no
bull Advantages of static routingndash Cheapersmaller routers (less true nowadays)ndash Simpler to configure
bull Advantages of BGPndash More control of your destiny (have providers stop
announcing you)ndash Fastermore intelligent selection of where to send
outbound packetsndash Better debugging of net problems (you can see the
Internet topology now)
51
Same Provider or Multiple
bull If your provider is reliable and fast and affordably and offers good tech-support you may want to multi-home initially to them via some backup path (slow is better than dead)
bull Eventually yoursquoll want to multi-home to different providers to avoid failure modes due to one providerrsquos architecture decisions
52
Multihomed Stub One Link
bull Downstream ISPrsquos routers configure default (ldquostaticrdquo) routes pointing to border router
bull Upstream ISP advertises reachability
Upstream ISP
Multiple links between same pair of routers
Default routes to ldquoborderrdquo
ldquoStubrdquoISP
53
Multihomed Stub Multiple Links
bull Use BGP to share loadbull Use private AS number (why is this OK)bull As before upstream ISP advertises prefix
Upstream ISP
Multiple links to different upstream routers
ldquoStubrdquoISP
Internal routing for ldquohot potatordquo
BGP for load balance at edge
54
Multihomed Stub Multiple ISPs
bull Many possibilitiesndash Load sharingndash Primary-backupndash Selective use of different ISPs
bull Requires BGP public AS number etc
ldquoStubrdquoISP
Upstream
ISP 1
Upstream
ISP 2
55
Multihomed Transit Network
bull BGP everywherebull Incoming and outcoming trafficbull Challenge balancing load on intradomain and egress
links given an offered traffic load
TransitISP
ISP 1
ISP 2
ISP 3
56
Interdomain Traffic Engineering
bull The process by which a network operator configures the network to achievendash Traffic load balancendash Redundancy (primarybackup) etc
bull Two tasksndash Outbound traffic controlndash Inbound traffic control
bull Key Problems Predictability and Scalability
57
Outbound Traffic Control
bull Easier to control than inbound trafficndash Destination-based routing sender determines where
the packets go
bull Control over next-hop AS onlyndash Cannot control selection of the entire path
Provider 1 Provider 2
Control with local preference
58
Outbound Traffic Load Balancingbull Control routes to provider per-prefix
ndash Assign local preference across destination prefixesndash Change the local preference assignments over time
bull Useful inputs to load balancingndash End-to-end path performance datandash Outbound traffic statistics per destination prefix
bull Challenge Getting from traffic volumes to groups of prefixes that should be assigned to each link
Premise of ldquointelligent route controlrdquo preoducts
59
Traffic Engineering Goals
bull Predictabilityndash Ensure the BGP decision process is deterministicndash Assume that BGP updates are (relatively) stable
bull Limit overhead introduced by routing changesndash Minimize frequency of changes to routing policiesndash Limit number of prefixes affected by changes
bull Limit impact on how traffic enters the networkndash Avoid new routes that might change neighborrsquos mindndash Select route with same attributes or at least path length
60
Managing Scalebull Destination prefixes
ndash More than 90000 destination prefixes
bull Donrsquot want to have per-prefix routing policies
ndash Small fraction of prefixes contribute most of the traffic
bull Focus on the small number of heavy hitters
ndash Define routing policies for selected prefixes
bull Routing choicesndash About 27000 unique ldquorouting choicesrdquo
bull Help in reducing the scale of the problem
ndash Small fraction of ldquorouting choicesrdquo contribute most traffic
bull Focus on the very small number of ldquorouting choicesrdquo
ndash Define routing policies on common attributes
61
Achieving Predictability
bull Route prediction with static analysisndash Helpful to know effects before deploymentndash Static analysis can help
TopologyBGP policy
configuration
eBGP routes
Offered traffic
BGP routingmodel
Flow of traffic through the network
62
Challenges to Predictabilitybull For transit ISPs effects on incoming traffic
ndash Lack of coordination strikes again
63
ldquoHot Potatordquo routing
Inter-AS Negotiation
bull Coordination aids predictabilityndash Negotiate where to sendndash Inbound and outboundndash Mutual benefits
bull How to implementndash What info to exchangendash Protecting privacyndash How to prioritize choicesndash How to prevent cheating
Destination 2
Destination 1
multiplepeeringpoints
Provider A
Provider B
64
Outbound Multihoming Goals
bull Redundancyndash Dynamic routing will failover to backup link
bull Performancendash Select provider with best performance per prefix
ndash Requires active probing
bull Costndash Select provider per prefix over time to minimize the
total financial cost
65
Inbound Traffic Control
bull More difficult no control over neighborsrsquo decisions
bull Three common techniques (previously discussed)ndash AS path prependingndash Communities and local preferencendash Prefix splitting
How does todayrsquos paper (MONET) control inbound traffic
66
How many links are enough
K upstream ISPs
Not much benefit beyond 4 ISPs
Akella et al ldquoPerformance Benefits of Multihomingrdquo SIGCOMM 2003
67
Problems with Multihoming in IPv4
bull Routing table growthndash Provider-based addressingndash Advertising prefix out multiple ISPs ndash canrsquot aggregate
bull Poor control over inbound trafficndash Existing mechanisms do not allow hosts to control
inbound traffic
68
GeorgiaTech
Internet Routing Overview
bull Intradomain (ie ldquointra-ASrdquo) routingbull Interdomain routing
Comcast
Abilene
ATampT Cogent
Autonomous Systems (ASes)
69
Configuration Problems ldquoAS 7007rdquoldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
UUNet
Florida InternetBarn
Sprint
70
Diagnosis and Troubleshootingldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
ldquoMicrosofts websites were offline for up to 23 hoursbecause of a [router] misconfigurationhellipit took nearly a day to determine what was wrong and undo the changesrdquo -- wiredcom January 25 2001
ldquoWorldCom Inchellipsuffered a widespread outage on its Internet backbone that affected roughly 20 percent of its US customer base The network problemshellipaffected millions of computer users worldwide A spokeswoman attributed the outage to a route table issue -- cnncom October 3 2002
A number of Covad customers went out from 5pm today due to supposedly a DDOS (distributed denial of service attack) on a key Level3 data center which later was described as a route leak (misconfiguration)ldquo
-- dslreportscom February 23 2004
71
Operator Mailing List (NANOG)Date Mon 18 Oct 2004 091515 -0700Subject Level 3 US east coast issues
Level 3 experiencing widespread unspecified routing issues on the US east coast Master ticket 1086844 Anyone have more specific information
Date Mon 18 Oct 2004 122034 -0400 (EDT)Subject Re Level 3 US east coast issues
Level 3 is currently experiencing a backbone outage causing routing instability and packet loss We are working to restore and will be sending hourly updateshellip
72
Operator Mailing List
0102030405060708090
Filtering RouteLeaks
RouteHijacks
RouteInstability
RoutingLoops
Blackholes
Occurr
ences o
ver
10 Y
ears
1995-1997 1998-2001 2001-2004
Note Only includes problems openly discussed on this list
Compare 83 power outages 1 fire
73
Routing Configuration
Ranking route selection
Dissemination internal route advertisement
Filtering route advertisement
Customer
Competitor
Primary
Backup
74
Internet Business Model (Simplified)
bull CustomerProvider One AS pays another for reachability to some set of destinations
bull ldquoSettlement-freerdquo Peering Bartering Two ASes exchange routes with one another
Provider
Peer
Customer
Preferences implemented with local preference manipulation
Destination
Pay to use
Get paid to use
Free to use
75
Peering Contracts Consistent Export
bull Rules of settlement-free peeringndash Advertise routes at all peering pointsndash Advertised routes must have equal ldquoAS path lengthrdquo
Sprint
ATampT
Enables ldquohot potatordquo routing
ldquoequally goodrdquoroutes
76
Consistent Export
bull Malicedeceptionbull iBGP signaling partitionbull Inconsistent export policy
neighbor 10123route-map PEER permit 10 set prepend 123
neighbor 10456route-map PEER permit 10 set prepend 123 123
Possible Causes
10123 456 1
10456 456 2
Neighbor AS Export
1 1 123
Export Clause Prepend
2 1 123 123
Two different Export Policies
77
Inconsistent Export in Practice
Feamster et al ldquoBorderGuard Detecting Cold Potatoes from Peersrdquo ACM IMC October 2004
78
Blackholes
Date Thu 18 Jul 2002 060510 -0400 (EDT)From Chad Oleary ltcolpoboxcomgtSubject Re problems with 701To ltnanogmeritedugt
Were starting to see the same issues with UUNet again Anyone elseseeing this Trying to reach Qwest
traceroute to 631461901 (631461901) 30 hops max 38 byte packets 1 esc-lp2-gwe-solutionscorpcom (631182201) 1167 ms 1163 ms 1142 ms 2 500Serial2-10GW1TPA2ALTERNET (1571301499) 1097 ms 1059 ms 1044 ms 3 161at-1-0-0XL4ATL1ALTERNET (1526381190) 13839 ms 14108 ms 16638 ms 4 0so-3-1-0XL2ATL5ALTERNET (152630238) 14370 ms 14587 ms 14553 ms 5 POS7-0BR2ATL5ALTERNET (1526382193) 13928 ms 14099 ms 14053 ms 6 7 hellip
79
Security
80
Security ldquoBogonrdquo Routes
Feamster et al ldquoAn Empirical Study of lsquoBogonrsquo Route Advertisementsrdquo ACM CCR January 2005
81
Spam Phishing etc
bull Unsolicited commercial emailbull As of about August 2008 estimates indicate that
about 95 of all email is spambull Common spam filtering techniques
ndash Content-based filtersndash DNS Blacklist (DNSBL) lookups Significant fraction of
todayrsquos DNS traffic
Can IP addresses from which spam is received be spoofed
82
Spam and Routing
83
Worms and Botnets
84
What is a Worm
bull Code that replicates and propagates across the networkndash Often carries a ldquopayloadrdquo
bull Usually spread via exploiting flaws in open servicesndash ldquoVirusesrdquo require user action to spread
bull First worm Robert Morris November 1988ndash 6-10 of all Internet hosts infected ()
bull Many more since but none on that scale until July 2001
85
Example Worm Code Red
bull Initial version July 13 2001
bull Exploited known ISAPI vulnerability in Microsoft IIS Web servers
bull 1st through 20th of each month spread20th through end of each month attack
bull Payload Web site defacementbull Scanning Random IP addressesbull Bug failure to seed random number generator
86
Code Red Revisions
bull Released July 19 2001
bull Payload flooding attack on wwwwhitehousegovndash Attack was mounted at the IP address of the Web site
bull Bug died after 20th of each month
bull Random number generator for IP scanning fixed
87
Code Red Host Infection Rate
Exponential infection rate
Measured using backscatter technique
88
Designing Fast-Spreading Worms
bull Hit-list scanningndash Time to infect first 10k hosts dominates infection timendash Solution Reconnaissance (stealthy scans etc)
bull Permutation scanningndash Observation Most scanning is redundantndash Idea Shared permutation of address space Start scanning
from own IP address Re-randomize when another infected machine is found
bull Internet-scale hit listsndash Flash worm complete infection within 30 seconds
89
Botnets
bull Bots Autonomous programs performing tasksbull Plenty of ldquobenignrdquo bots
ndash eg weatherbug
bull Botnets group of bots ndash Typically carries malicious connotationndash Large numbers of infected machinesndash Machines ldquoenlistedrdquo with infection vectors like worms
(last lecture)
bull Available for simultaneous control by a masterbull Size up to 350000 nodes (from todayrsquos paper)
90
ldquoRallyingrdquo the Botnet
bull Easy to combine worm backdoor functionalitybull Problem how to learn about successfully
infected machines
bull Optionsndash Emailndash Hard-coded email address
91
Botnet Control
bull Botnet master typically runs some IRC server on a well-known port (eg 6667)
bull Infected machine contacts botnet with pre-programmed DNS name (eg big-botde)
bull Dynamic DNS allows controller to move about freely
Infected Machine
DynamicDNS
BotnetController
(IRC server)
92
Some Defenses
93
Idea 1 Ingress Filtering
bull RFC 2827 Routers install filters to drop packets from networks that are not downstream
bull Feasible at edgesbull Difficult to configure closer to network ldquocorerdquo
20469207024 Internet
Drop all packets with source address other than 20469207024
94
Idea 2 uRPF Checks
bull Unicast Reverse Path Forwardingndash Cisco ldquoip verify unicast reverse-pathrdquo
bull Requires symmetric routing
Accept packet from interface only if forwarding table entry for source IP address matches ingress interface
100183
A10018
10015
101203
100181 24
10011 24
Strict Mode uRPF
Enabled
ldquoArdquo Routing TableDestination Next Hop1001024 Int 110018024 Int 2
100183 from wrong interface
95
Problems with uRPF
bull Asymmetric routing
96
S-BGP
bull Address-based PKI validate signaturesndash Authentication of
bull ownership for IP address blocks bull AS number bull an ASs identity and bull a BGP routers identity
ndash Use existing infrastructure (Internet registries etc)ndash Routing origination is digitally signedndash BGP updates are digitally signed
1048708 bull Route attestations A new optional BGP transitive path attribute
ndash carries digital signatures covering the routing information in updates
97
Practical Problems with S-BGP
bull Requires Public-Key Infrastructure
bull Lots of digital signatures to calculate and verifyndash Message overheadndash CPU overhead
bull Calculation expense is greatest when topology is changingndash Caching can help
bull Route aggregation is problematic (maybe thatrsquos OK)
bull Secure route withdrawals when link or node fails
bull Address ownership data out of date
bull Deployment
- Networking
- Goal of This Tutorial
- Todayrsquos Networks
- Business Models
- Billing for Internet Usage
- Net Neutrality
- Network Operations
- Point-of-Presence (PoP)
- Example Abilene Network Topology
- Another Example Backbone
- Internet Routing Overview
- Internet Routing Protocol BGP
- Two Flavors of BGP
- IPv4 Addresses Networks of Networks
- Pre-1994 Classful Addressing
- Classless Interdomain Routing (CIDR)
- Benefits of CIDR
- Growth of IP Prefixes
- 1994-1998 Linear Growth
- Around 2000 Fast Growth Resumes
- Fast growth resumes
- The Address Allocation Process
- Common Problems
- What can go wrong
- Measurement and Monitoring
- Passive vs Active Measurement
- Slide 27
- Passive Traffic Data Measurement
- Simple Network Management Protocol
- SNMP (Passive)
- Packet-level Monitoring
- Full Packet Capture (Passive)
- What is a flow
- Cisco NetFlow
- Flow Record Contents
- Aggregating Packets into Flows
- Reducing Measurement Overhead
- Packet Sampling for Flow Monitoring
- Sampling Flow-Level Sampling
- Two Main Approaches
- Packet Capture on High-Speed Links
- Characteristics of Packet Capture
- Data Measurement Repositories
- Multihoming and Traffic Engineering
- What is Multihoming
- Why Multihome
- Redundancy
- Performance
- Multihoming in IP Networks Today
- BGP or no
- Same Provider or Multiple
- Multihomed Stub One Link
- Multihomed Stub Multiple Links
- Multihomed Stub Multiple ISPs
- Multihomed Transit Network
- Interdomain Traffic Engineering
- Outbound Traffic Control
- Outbound Traffic Load Balancing
- Traffic Engineering Goals
- Managing Scale
- Achieving Predictability
- Challenges to Predictability
- Inter-AS Negotiation
- Outbound Multihoming Goals
- Inbound Traffic Control
- How many links are enough
- Problems with Multihoming in IPv4
- Slide 68
- Slide 69
- Diagnosis and Troubleshooting
- Operator Mailing List (NANOG)
- Operator Mailing List
- Routing Configuration
- Internet Business Model (Simplified)
- Peering Contracts Consistent Export
- Consistent Export
- Inconsistent Export in Practice
- Blackholes
- Security
- Security ldquoBogonrdquo Routes
- Spam Phishing etc
- Spam and Routing
- Worms and Botnets
- What is a Worm
- Example Worm Code Red
- Code Red Revisions
- Code Red Host Infection Rate
- Designing Fast-Spreading Worms
- Botnets
- ldquoRallyingrdquo the Botnet
- Botnet Control
- Some Defenses
- Idea 1 Ingress Filtering
- Idea 2 uRPF Checks
- Problems with uRPF
- S-BGP
- Practical Problems with S-BGP
-
2
Goal of This Tutorial
bull Teach engineers the basics of networking and ISP operations
bull Networks todayndash Business modelsndash Operations (NOC operators)
bull Common problemsbull Measurement Monitoring and Security
3
Todayrsquos Networks
bull Service provider business models
bull Network operations center
bull Network operators and engineers
4
Business Models
bull Increasingly commoditized (see Geoff Hustonrsquos talk at NANOG)
bull Status quo Establish transit costs bill at 95th percentile of usage
bull Future differential pricing preference for certain groups of users applications
5
Billing for Internet Usage
bull 95th Percentile billingndash Customer network pays for ldquocommitted information
raterdquo (CIR)ndash Throughput measured every 5 minutes (typically with
SNMP flow statistics also can be used for billing)ndash Customer billed based on 95th percentile
6
Net Neutrality
7
Network Operations
bull Operators run the day-to-day operations of the networkndash Adjusting to shifts in traffic failures etcndash Responding to security threatsndash Provisioning new customers
8
Point-of-Presence (PoP)
bull A ldquoclusterrdquo of routers in a single physical location
bull Inter-PoP linksndash Long distances
ndash High bandwidth
bull Intra-PoP linksndash Cables between racks or floors
ndash Aggregated bandwidth
PoP
9
Example Abilene Network Topology
10
Another Example Backbone
11
GeorgiaTech
Internet Routing Overview
bull Intradomain (ie ldquointra-ASrdquo) routingbull Interdomain routing
Comcast
Abilene
ATampT Cogent
Autonomous Systems (ASes)
12
Internet Routing Protocol BGP
Route Advertisement
Autonomous Systems (ASes)
Session
Traffic
Destination Next-hop AS Path1302070016
1302070016
19258989
6625025244
105782637
174hellip 2637
13
Two Flavors of BGP
bull External BGP (eBGP) exchanging routes between ASes
bull Internal BGP (iBGP) disseminating routes to external destinations among the routers within an AS
eBGPiBGP
Question Whatrsquos the difference between IGP and iBGP
14
IPv4 Addresses Networks of Networks
bull 32-bit number in ldquodotted-quadrdquo notation
ndash wwwccgatechedu --- 130207736
10000010 11001111 00000111 00100100
Network (16 bits) Host (16 bits)
130 207 7 36
bull Problem 232 addresses is a lot of table entries
bull Solution Routing based on network and host
ndash 1302070016 is a 16-bit prefix with 216 IP addresses
Topological Addressing
15
Pre-1994 Classful Addressing
Network ID Host ID
8 16
Class A
32
0
Class B 10
Class C 110
Multicast AddressesClass D 1110
Reserved for experimentsClass E 1111
24
8 blocks (eg MIT has 180008)
16 blocks (eg Georgia Tech has 1302070016)
24 blocks (eg ATampT Labs has 19220225024)
Simple Forwarding Address range specifies network ID length
16
Classless Interdomain Routing (CIDR)
IP Address 65142480 ldquoMaskrdquo 2552552520
01000001 00001110 11111000 00000000
11111111 11111111 11111100 00000000
Use two 32-bit numbers to represent a network Network number = IP address + Mask
Example BellSouth Prefix 6514248022
Address no longer specifies network ID rangeNew forwarding trick Longest Prefix Match
17
Benefits of CIDR
bull Efficiency Can allocate blocks of prefixes on a finer granularity
bull Hierarchy Prefixes can be aggregated into supernets (Not always done Typically not in fact)
Customer 1
Customer 2
ATampT Internet
1220249024
1220231024120008
18
Growth of IP Prefixes
19
1994-1998 Linear Growth
bull About 10000 new entries per yearbull In theory less instability at the edges (why)
Source Geoff Huston
20
Around 2000 Fast Growth Resumes
Claim remaining 8s will be exhausted within the next 5-10 years
T Hain ldquoA Pragmatic Report on IPv4 Address Space Consumptionrdquo Cisco IPJ September 2005
21
Fast growth resumes
Rapid growth in routing tables
Dot-Bomb Hiccup
Significant contributor Multihoming
Source Geoff Huston
22
The Address Allocation Process
bull Allocation policies of RIRs affect pressure on IPv4 address space
IANA
AfriNIC APNIC ARIN LACNIC RIPE
httpwwwianaorgassignmentsipv4-address-space
Georgia Tech
23
Common Problems
bull Diagnosis and troubleshooting (hence measurement)bull Traffic engineeringbull Securitybull Design and capacity planning
24
What can go wrong
Two-thirds of the problems are caused by configuration of the routing protocol
Some downtime is very hard to protect againsthellip
25
Measurement and Monitoring
26
Passive vs Active Measurement
bull Passive Measurement Collection of packets flow statistics of traffic that is already flowing on the networkndash Packet tracesndash Flow statisticsndash Application-level logs
bull Active Measurement Inject ldquoprobingrdquo traffic to measure various characteristicsndash Traceroutendash Pingndash Application-level probes (eg Web downloads)
27
Billing for Internet Usage
bull 95th Percentile billingndash Customer network pays for ldquocommitted information
raterdquo (CIR)ndash Throughput measured every 5 minutes (typically with
SNMP flow statistics also can be used for billing)ndash Customer billed based on 95th percentile
28
Passive Traffic Data Measurement
bull SNMP bytepacket counts everywhere
bull Packet monitoring selected locations
bull Flow monitoring typically at edges (if possible)ndash Direct computation of the traffic matrixndash Input to denial-of-service attack detection
bull Deep Packet Inspection also at edge where possible
29
Simple Network Management Protocol
bull Management Information Base (MIB)ndash Information storendash Unique variables named by
OIDsndash Accessed with SNMP
bull Specific MIBs for bytepacket counts (per link)
Manager Agent
SNMP
DB
ManagedObjects
30
SNMP (Passive)
bull Advantage ubiquitousndash Supported on all networking equipmentndash Multiple products for polling and analyzing data
bull Disadvantages ndash Coarse granularityndash Cannot express complex queries on the datandash Unreliable delivery of the data using UDP
bull Utilityndash Link utilization (billing)ndash Traffic matrix inference
31
Packet-level Monitoring
bull Passive monitoring to collect full packet contents (or at least headers)
bull Advantages lots of detailed informationndash Precise timing informationndash Information in packet headers
bull Disadvantages overheadndash Hard to keep up with high-speed linksndash Often requires a separate monitoring device
32
Full Packet Capture (Passive)
Example Georgia Tech OC3Mon
bull Rack-mounted PCbull Optical splitterbull Data Acquisition and
Generation (DAG) card
Source endacecom
33
What is a flow
bull Source IP addressbull Destination IP addressbull Source portbull Destination portbull Layer 3 protocol typebull TOS byte (DSCP)bull Input logical interface (ifIndex)
34
Cisco NetFlow
bull Basic output ldquoFlow recordrdquondash Most common version is v5
bull Current version (9) is being standardized in the IETF (template-based)ndash More flexible record formatndash Much easier to add new flow record types
Core Network
Collection and Aggregation
Collector (PC)Approximately 1500 bytes
20-50 flow recordsSent more frequently if traffic increases
35
Flow Record Contents
bull Source and Destination IP address and portbull Packet and byte countsbull Start and end timesbull ToS TCP flags
Basic information about the flowhellip
hellipplus information related to routing
bull Next-hop IP addressbull Source and destination ASbull Source and destination prefix
36
flow 1 flow 2 flow 3 flow 4
Aggregating Packets into Flows
bull Criteria 1 Set of packets that ldquobelong togetherrdquondash Sourcedestination IP addresses and port numbersndash Same protocol ToS bits hellip ndash Same inputoutput interfaces at a router (if known)
bull Criteria 2 Packets that are ldquocloserdquo together in timendash Maximum inter-packet spacing (eg 15 sec 30 sec)ndash Example flows 2 and 4 are different flows due to time
37
Reducing Measurement Overhead
bull Filtering on interfacendash destination prefix for a customerndash port number for an application (eg 80 for Web)
bull Sampling before insertion into flow cachendash Random deterministic or hash-based samplingndash 1-out-of-n or stratified based on packetflow sizendash Two types packet-level and flow-level
bull Aggregation after cache evictionndash packetsflows with same next-hop ASndash packetsflows destined to a particular service
38
Packet Sampling for Flow Monitoring
bull Packet sampling before flow creation (Sampled Netflow)ndash 1-out-of-m sampling of individual packets (eg m=100)ndash Create of flow records over the sampled packets
bull Reducing overheadndash Avoid per-packet overhead on (m-1)m packetsndash Avoid creating records for a large number of small flows
bull Increasing overhead (in some cases)ndash May split some long transfers into multiple flow records ndash hellip due to larger time gaps between successive packets
time
not sampled
two flowstimeout
39
Sampling Flow-Level Sampling
bull Sampling of flow records evicted from flow cachendash When evicting flows from table or when analyzing flows
bull Stratified sampling to put weight on ldquoheavyrdquo flowsndash Select all long flows and sample the short flows
bull Reduces the number of flow records ndash Still measures the vast majority of the traffic
Flow 1 40 bytesFlow 2 15580 bytesFlow 3 8196 bytesFlow 4 5350789 bytesFlow 5 532 bytesFlow 6 7432 bytes
sample with 100 probability
sample with 01 probability
sample with 10 probability
40
Two Main Approaches
bull Packet-level Monitoringndash Keep packet-level statisticsndash Examine (and potentially log) variety of packet-level
statistics Essentially anything in the packetndash Timing
bull Flow-level Monitoringndash Monitor packet-by-packet (though sometimes
sampled)ndash Keep aggregate statistics on a flow
41
Packet Capture on High-Speed Links
Example Georgia Tech ldquoOC3Monrdquo
bull Rack-mounted PCbull Optical splitterbull Data Acquisition and
Generation (DAG) card
Source endacecom
42
Characteristics of Packet Capture
bull Allows inspection on every packet on 10G links
bull Disadvantagesndash Costlyndash Requires splitting optical fibersndash Must be able to filterstore data
43
Data Measurement Repositories
bull AbileneInternet 2 Observatoryndash Configuration examplesndash SNMP datandash ISIS BGP routing data NetFlow traffic data
bull RouteViewsndash BGP updatesndash BGP table snapshots
44
Multihoming and Traffic Engineering
45
What is Multihoming
bull The use of redundant network links for the purposes of external connectivity
bull Can be achieved at many layers of the protocol stack and many places in the networkndash Multiple network interfaces in a PC
ndash An ISP with multiple upstream interfaces
bull Can refer to having multiple connections tondash The same ISP
ndash Multiple ISPs
46
Why Multihome
bull Redundancybull Availabilitybull Performancebull Cost
Interdomain traffic engineering the process by which a multihomed network configures its
network to achieve these goals
47
Redundancy
bull Maintain connectivity in the face ofndash Physical connectivity problems (fiber cut device
failures etc)ndash Failures in upstream ISP
48
Performance
bull Use multiple network links at once to achieve higher throughput than just over a single link
bull Allows incoming traffic to be load-balanced
70 of traffic30 of traffic
49
Multihoming in IP Networks Today
bull Stub AS no transit service for other ASesndash No need to use BGP
bull Multi-homed stub AS has connectivity to multiple immediate upstream ISPsndash Need BGPndash No need for a public AS numberndash No need for IP prefix allocation
bull Multi-homed transit AS connectivity to multiple ASes and transit servicendash Need BGP public AS number IP prefix allocation
50
BGP or no
bull Advantages of static routingndash Cheapersmaller routers (less true nowadays)ndash Simpler to configure
bull Advantages of BGPndash More control of your destiny (have providers stop
announcing you)ndash Fastermore intelligent selection of where to send
outbound packetsndash Better debugging of net problems (you can see the
Internet topology now)
51
Same Provider or Multiple
bull If your provider is reliable and fast and affordably and offers good tech-support you may want to multi-home initially to them via some backup path (slow is better than dead)
bull Eventually yoursquoll want to multi-home to different providers to avoid failure modes due to one providerrsquos architecture decisions
52
Multihomed Stub One Link
bull Downstream ISPrsquos routers configure default (ldquostaticrdquo) routes pointing to border router
bull Upstream ISP advertises reachability
Upstream ISP
Multiple links between same pair of routers
Default routes to ldquoborderrdquo
ldquoStubrdquoISP
53
Multihomed Stub Multiple Links
bull Use BGP to share loadbull Use private AS number (why is this OK)bull As before upstream ISP advertises prefix
Upstream ISP
Multiple links to different upstream routers
ldquoStubrdquoISP
Internal routing for ldquohot potatordquo
BGP for load balance at edge
54
Multihomed Stub Multiple ISPs
bull Many possibilitiesndash Load sharingndash Primary-backupndash Selective use of different ISPs
bull Requires BGP public AS number etc
ldquoStubrdquoISP
Upstream
ISP 1
Upstream
ISP 2
55
Multihomed Transit Network
bull BGP everywherebull Incoming and outcoming trafficbull Challenge balancing load on intradomain and egress
links given an offered traffic load
TransitISP
ISP 1
ISP 2
ISP 3
56
Interdomain Traffic Engineering
bull The process by which a network operator configures the network to achievendash Traffic load balancendash Redundancy (primarybackup) etc
bull Two tasksndash Outbound traffic controlndash Inbound traffic control
bull Key Problems Predictability and Scalability
57
Outbound Traffic Control
bull Easier to control than inbound trafficndash Destination-based routing sender determines where
the packets go
bull Control over next-hop AS onlyndash Cannot control selection of the entire path
Provider 1 Provider 2
Control with local preference
58
Outbound Traffic Load Balancingbull Control routes to provider per-prefix
ndash Assign local preference across destination prefixesndash Change the local preference assignments over time
bull Useful inputs to load balancingndash End-to-end path performance datandash Outbound traffic statistics per destination prefix
bull Challenge Getting from traffic volumes to groups of prefixes that should be assigned to each link
Premise of ldquointelligent route controlrdquo preoducts
59
Traffic Engineering Goals
bull Predictabilityndash Ensure the BGP decision process is deterministicndash Assume that BGP updates are (relatively) stable
bull Limit overhead introduced by routing changesndash Minimize frequency of changes to routing policiesndash Limit number of prefixes affected by changes
bull Limit impact on how traffic enters the networkndash Avoid new routes that might change neighborrsquos mindndash Select route with same attributes or at least path length
60
Managing Scalebull Destination prefixes
ndash More than 90000 destination prefixes
bull Donrsquot want to have per-prefix routing policies
ndash Small fraction of prefixes contribute most of the traffic
bull Focus on the small number of heavy hitters
ndash Define routing policies for selected prefixes
bull Routing choicesndash About 27000 unique ldquorouting choicesrdquo
bull Help in reducing the scale of the problem
ndash Small fraction of ldquorouting choicesrdquo contribute most traffic
bull Focus on the very small number of ldquorouting choicesrdquo
ndash Define routing policies on common attributes
61
Achieving Predictability
bull Route prediction with static analysisndash Helpful to know effects before deploymentndash Static analysis can help
TopologyBGP policy
configuration
eBGP routes
Offered traffic
BGP routingmodel
Flow of traffic through the network
62
Challenges to Predictabilitybull For transit ISPs effects on incoming traffic
ndash Lack of coordination strikes again
63
ldquoHot Potatordquo routing
Inter-AS Negotiation
bull Coordination aids predictabilityndash Negotiate where to sendndash Inbound and outboundndash Mutual benefits
bull How to implementndash What info to exchangendash Protecting privacyndash How to prioritize choicesndash How to prevent cheating
Destination 2
Destination 1
multiplepeeringpoints
Provider A
Provider B
64
Outbound Multihoming Goals
bull Redundancyndash Dynamic routing will failover to backup link
bull Performancendash Select provider with best performance per prefix
ndash Requires active probing
bull Costndash Select provider per prefix over time to minimize the
total financial cost
65
Inbound Traffic Control
bull More difficult no control over neighborsrsquo decisions
bull Three common techniques (previously discussed)ndash AS path prependingndash Communities and local preferencendash Prefix splitting
How does todayrsquos paper (MONET) control inbound traffic
66
How many links are enough
K upstream ISPs
Not much benefit beyond 4 ISPs
Akella et al ldquoPerformance Benefits of Multihomingrdquo SIGCOMM 2003
67
Problems with Multihoming in IPv4
bull Routing table growthndash Provider-based addressingndash Advertising prefix out multiple ISPs ndash canrsquot aggregate
bull Poor control over inbound trafficndash Existing mechanisms do not allow hosts to control
inbound traffic
68
GeorgiaTech
Internet Routing Overview
bull Intradomain (ie ldquointra-ASrdquo) routingbull Interdomain routing
Comcast
Abilene
ATampT Cogent
Autonomous Systems (ASes)
69
Configuration Problems ldquoAS 7007rdquoldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
UUNet
Florida InternetBarn
Sprint
70
Diagnosis and Troubleshootingldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
ldquoMicrosofts websites were offline for up to 23 hoursbecause of a [router] misconfigurationhellipit took nearly a day to determine what was wrong and undo the changesrdquo -- wiredcom January 25 2001
ldquoWorldCom Inchellipsuffered a widespread outage on its Internet backbone that affected roughly 20 percent of its US customer base The network problemshellipaffected millions of computer users worldwide A spokeswoman attributed the outage to a route table issue -- cnncom October 3 2002
A number of Covad customers went out from 5pm today due to supposedly a DDOS (distributed denial of service attack) on a key Level3 data center which later was described as a route leak (misconfiguration)ldquo
-- dslreportscom February 23 2004
71
Operator Mailing List (NANOG)Date Mon 18 Oct 2004 091515 -0700Subject Level 3 US east coast issues
Level 3 experiencing widespread unspecified routing issues on the US east coast Master ticket 1086844 Anyone have more specific information
Date Mon 18 Oct 2004 122034 -0400 (EDT)Subject Re Level 3 US east coast issues
Level 3 is currently experiencing a backbone outage causing routing instability and packet loss We are working to restore and will be sending hourly updateshellip
72
Operator Mailing List
0102030405060708090
Filtering RouteLeaks
RouteHijacks
RouteInstability
RoutingLoops
Blackholes
Occurr
ences o
ver
10 Y
ears
1995-1997 1998-2001 2001-2004
Note Only includes problems openly discussed on this list
Compare 83 power outages 1 fire
73
Routing Configuration
Ranking route selection
Dissemination internal route advertisement
Filtering route advertisement
Customer
Competitor
Primary
Backup
74
Internet Business Model (Simplified)
bull CustomerProvider One AS pays another for reachability to some set of destinations
bull ldquoSettlement-freerdquo Peering Bartering Two ASes exchange routes with one another
Provider
Peer
Customer
Preferences implemented with local preference manipulation
Destination
Pay to use
Get paid to use
Free to use
75
Peering Contracts Consistent Export
bull Rules of settlement-free peeringndash Advertise routes at all peering pointsndash Advertised routes must have equal ldquoAS path lengthrdquo
Sprint
ATampT
Enables ldquohot potatordquo routing
ldquoequally goodrdquoroutes
76
Consistent Export
bull Malicedeceptionbull iBGP signaling partitionbull Inconsistent export policy
neighbor 10123route-map PEER permit 10 set prepend 123
neighbor 10456route-map PEER permit 10 set prepend 123 123
Possible Causes
10123 456 1
10456 456 2
Neighbor AS Export
1 1 123
Export Clause Prepend
2 1 123 123
Two different Export Policies
77
Inconsistent Export in Practice
Feamster et al ldquoBorderGuard Detecting Cold Potatoes from Peersrdquo ACM IMC October 2004
78
Blackholes
Date Thu 18 Jul 2002 060510 -0400 (EDT)From Chad Oleary ltcolpoboxcomgtSubject Re problems with 701To ltnanogmeritedugt
Were starting to see the same issues with UUNet again Anyone elseseeing this Trying to reach Qwest
traceroute to 631461901 (631461901) 30 hops max 38 byte packets 1 esc-lp2-gwe-solutionscorpcom (631182201) 1167 ms 1163 ms 1142 ms 2 500Serial2-10GW1TPA2ALTERNET (1571301499) 1097 ms 1059 ms 1044 ms 3 161at-1-0-0XL4ATL1ALTERNET (1526381190) 13839 ms 14108 ms 16638 ms 4 0so-3-1-0XL2ATL5ALTERNET (152630238) 14370 ms 14587 ms 14553 ms 5 POS7-0BR2ATL5ALTERNET (1526382193) 13928 ms 14099 ms 14053 ms 6 7 hellip
79
Security
80
Security ldquoBogonrdquo Routes
Feamster et al ldquoAn Empirical Study of lsquoBogonrsquo Route Advertisementsrdquo ACM CCR January 2005
81
Spam Phishing etc
bull Unsolicited commercial emailbull As of about August 2008 estimates indicate that
about 95 of all email is spambull Common spam filtering techniques
ndash Content-based filtersndash DNS Blacklist (DNSBL) lookups Significant fraction of
todayrsquos DNS traffic
Can IP addresses from which spam is received be spoofed
82
Spam and Routing
83
Worms and Botnets
84
What is a Worm
bull Code that replicates and propagates across the networkndash Often carries a ldquopayloadrdquo
bull Usually spread via exploiting flaws in open servicesndash ldquoVirusesrdquo require user action to spread
bull First worm Robert Morris November 1988ndash 6-10 of all Internet hosts infected ()
bull Many more since but none on that scale until July 2001
85
Example Worm Code Red
bull Initial version July 13 2001
bull Exploited known ISAPI vulnerability in Microsoft IIS Web servers
bull 1st through 20th of each month spread20th through end of each month attack
bull Payload Web site defacementbull Scanning Random IP addressesbull Bug failure to seed random number generator
86
Code Red Revisions
bull Released July 19 2001
bull Payload flooding attack on wwwwhitehousegovndash Attack was mounted at the IP address of the Web site
bull Bug died after 20th of each month
bull Random number generator for IP scanning fixed
87
Code Red Host Infection Rate
Exponential infection rate
Measured using backscatter technique
88
Designing Fast-Spreading Worms
bull Hit-list scanningndash Time to infect first 10k hosts dominates infection timendash Solution Reconnaissance (stealthy scans etc)
bull Permutation scanningndash Observation Most scanning is redundantndash Idea Shared permutation of address space Start scanning
from own IP address Re-randomize when another infected machine is found
bull Internet-scale hit listsndash Flash worm complete infection within 30 seconds
89
Botnets
bull Bots Autonomous programs performing tasksbull Plenty of ldquobenignrdquo bots
ndash eg weatherbug
bull Botnets group of bots ndash Typically carries malicious connotationndash Large numbers of infected machinesndash Machines ldquoenlistedrdquo with infection vectors like worms
(last lecture)
bull Available for simultaneous control by a masterbull Size up to 350000 nodes (from todayrsquos paper)
90
ldquoRallyingrdquo the Botnet
bull Easy to combine worm backdoor functionalitybull Problem how to learn about successfully
infected machines
bull Optionsndash Emailndash Hard-coded email address
91
Botnet Control
bull Botnet master typically runs some IRC server on a well-known port (eg 6667)
bull Infected machine contacts botnet with pre-programmed DNS name (eg big-botde)
bull Dynamic DNS allows controller to move about freely
Infected Machine
DynamicDNS
BotnetController
(IRC server)
92
Some Defenses
93
Idea 1 Ingress Filtering
bull RFC 2827 Routers install filters to drop packets from networks that are not downstream
bull Feasible at edgesbull Difficult to configure closer to network ldquocorerdquo
20469207024 Internet
Drop all packets with source address other than 20469207024
94
Idea 2 uRPF Checks
bull Unicast Reverse Path Forwardingndash Cisco ldquoip verify unicast reverse-pathrdquo
bull Requires symmetric routing
Accept packet from interface only if forwarding table entry for source IP address matches ingress interface
100183
A10018
10015
101203
100181 24
10011 24
Strict Mode uRPF
Enabled
ldquoArdquo Routing TableDestination Next Hop1001024 Int 110018024 Int 2
100183 from wrong interface
95
Problems with uRPF
bull Asymmetric routing
96
S-BGP
bull Address-based PKI validate signaturesndash Authentication of
bull ownership for IP address blocks bull AS number bull an ASs identity and bull a BGP routers identity
ndash Use existing infrastructure (Internet registries etc)ndash Routing origination is digitally signedndash BGP updates are digitally signed
1048708 bull Route attestations A new optional BGP transitive path attribute
ndash carries digital signatures covering the routing information in updates
97
Practical Problems with S-BGP
bull Requires Public-Key Infrastructure
bull Lots of digital signatures to calculate and verifyndash Message overheadndash CPU overhead
bull Calculation expense is greatest when topology is changingndash Caching can help
bull Route aggregation is problematic (maybe thatrsquos OK)
bull Secure route withdrawals when link or node fails
bull Address ownership data out of date
bull Deployment
- Networking
- Goal of This Tutorial
- Todayrsquos Networks
- Business Models
- Billing for Internet Usage
- Net Neutrality
- Network Operations
- Point-of-Presence (PoP)
- Example Abilene Network Topology
- Another Example Backbone
- Internet Routing Overview
- Internet Routing Protocol BGP
- Two Flavors of BGP
- IPv4 Addresses Networks of Networks
- Pre-1994 Classful Addressing
- Classless Interdomain Routing (CIDR)
- Benefits of CIDR
- Growth of IP Prefixes
- 1994-1998 Linear Growth
- Around 2000 Fast Growth Resumes
- Fast growth resumes
- The Address Allocation Process
- Common Problems
- What can go wrong
- Measurement and Monitoring
- Passive vs Active Measurement
- Slide 27
- Passive Traffic Data Measurement
- Simple Network Management Protocol
- SNMP (Passive)
- Packet-level Monitoring
- Full Packet Capture (Passive)
- What is a flow
- Cisco NetFlow
- Flow Record Contents
- Aggregating Packets into Flows
- Reducing Measurement Overhead
- Packet Sampling for Flow Monitoring
- Sampling Flow-Level Sampling
- Two Main Approaches
- Packet Capture on High-Speed Links
- Characteristics of Packet Capture
- Data Measurement Repositories
- Multihoming and Traffic Engineering
- What is Multihoming
- Why Multihome
- Redundancy
- Performance
- Multihoming in IP Networks Today
- BGP or no
- Same Provider or Multiple
- Multihomed Stub One Link
- Multihomed Stub Multiple Links
- Multihomed Stub Multiple ISPs
- Multihomed Transit Network
- Interdomain Traffic Engineering
- Outbound Traffic Control
- Outbound Traffic Load Balancing
- Traffic Engineering Goals
- Managing Scale
- Achieving Predictability
- Challenges to Predictability
- Inter-AS Negotiation
- Outbound Multihoming Goals
- Inbound Traffic Control
- How many links are enough
- Problems with Multihoming in IPv4
- Slide 68
- Slide 69
- Diagnosis and Troubleshooting
- Operator Mailing List (NANOG)
- Operator Mailing List
- Routing Configuration
- Internet Business Model (Simplified)
- Peering Contracts Consistent Export
- Consistent Export
- Inconsistent Export in Practice
- Blackholes
- Security
- Security ldquoBogonrdquo Routes
- Spam Phishing etc
- Spam and Routing
- Worms and Botnets
- What is a Worm
- Example Worm Code Red
- Code Red Revisions
- Code Red Host Infection Rate
- Designing Fast-Spreading Worms
- Botnets
- ldquoRallyingrdquo the Botnet
- Botnet Control
- Some Defenses
- Idea 1 Ingress Filtering
- Idea 2 uRPF Checks
- Problems with uRPF
- S-BGP
- Practical Problems with S-BGP
-
3
Todayrsquos Networks
bull Service provider business models
bull Network operations center
bull Network operators and engineers
4
Business Models
bull Increasingly commoditized (see Geoff Hustonrsquos talk at NANOG)
bull Status quo Establish transit costs bill at 95th percentile of usage
bull Future differential pricing preference for certain groups of users applications
5
Billing for Internet Usage
bull 95th Percentile billingndash Customer network pays for ldquocommitted information
raterdquo (CIR)ndash Throughput measured every 5 minutes (typically with
SNMP flow statistics also can be used for billing)ndash Customer billed based on 95th percentile
6
Net Neutrality
7
Network Operations
bull Operators run the day-to-day operations of the networkndash Adjusting to shifts in traffic failures etcndash Responding to security threatsndash Provisioning new customers
8
Point-of-Presence (PoP)
bull A ldquoclusterrdquo of routers in a single physical location
bull Inter-PoP linksndash Long distances
ndash High bandwidth
bull Intra-PoP linksndash Cables between racks or floors
ndash Aggregated bandwidth
PoP
9
Example Abilene Network Topology
10
Another Example Backbone
11
GeorgiaTech
Internet Routing Overview
bull Intradomain (ie ldquointra-ASrdquo) routingbull Interdomain routing
Comcast
Abilene
ATampT Cogent
Autonomous Systems (ASes)
12
Internet Routing Protocol BGP
Route Advertisement
Autonomous Systems (ASes)
Session
Traffic
Destination Next-hop AS Path1302070016
1302070016
19258989
6625025244
105782637
174hellip 2637
13
Two Flavors of BGP
bull External BGP (eBGP) exchanging routes between ASes
bull Internal BGP (iBGP) disseminating routes to external destinations among the routers within an AS
eBGPiBGP
Question Whatrsquos the difference between IGP and iBGP
14
IPv4 Addresses Networks of Networks
bull 32-bit number in ldquodotted-quadrdquo notation
ndash wwwccgatechedu --- 130207736
10000010 11001111 00000111 00100100
Network (16 bits) Host (16 bits)
130 207 7 36
bull Problem 232 addresses is a lot of table entries
bull Solution Routing based on network and host
ndash 1302070016 is a 16-bit prefix with 216 IP addresses
Topological Addressing
15
Pre-1994 Classful Addressing
Network ID Host ID
8 16
Class A
32
0
Class B 10
Class C 110
Multicast AddressesClass D 1110
Reserved for experimentsClass E 1111
24
8 blocks (eg MIT has 180008)
16 blocks (eg Georgia Tech has 1302070016)
24 blocks (eg ATampT Labs has 19220225024)
Simple Forwarding Address range specifies network ID length
16
Classless Interdomain Routing (CIDR)
IP Address 65142480 ldquoMaskrdquo 2552552520
01000001 00001110 11111000 00000000
11111111 11111111 11111100 00000000
Use two 32-bit numbers to represent a network Network number = IP address + Mask
Example BellSouth Prefix 6514248022
Address no longer specifies network ID rangeNew forwarding trick Longest Prefix Match
17
Benefits of CIDR
bull Efficiency Can allocate blocks of prefixes on a finer granularity
bull Hierarchy Prefixes can be aggregated into supernets (Not always done Typically not in fact)
Customer 1
Customer 2
ATampT Internet
1220249024
1220231024120008
18
Growth of IP Prefixes
19
1994-1998 Linear Growth
bull About 10000 new entries per yearbull In theory less instability at the edges (why)
Source Geoff Huston
20
Around 2000 Fast Growth Resumes
Claim remaining 8s will be exhausted within the next 5-10 years
T Hain ldquoA Pragmatic Report on IPv4 Address Space Consumptionrdquo Cisco IPJ September 2005
21
Fast growth resumes
Rapid growth in routing tables
Dot-Bomb Hiccup
Significant contributor Multihoming
Source Geoff Huston
22
The Address Allocation Process
bull Allocation policies of RIRs affect pressure on IPv4 address space
IANA
AfriNIC APNIC ARIN LACNIC RIPE
httpwwwianaorgassignmentsipv4-address-space
Georgia Tech
23
Common Problems
bull Diagnosis and troubleshooting (hence measurement)bull Traffic engineeringbull Securitybull Design and capacity planning
24
What can go wrong
Two-thirds of the problems are caused by configuration of the routing protocol
Some downtime is very hard to protect againsthellip
25
Measurement and Monitoring
26
Passive vs Active Measurement
bull Passive Measurement Collection of packets flow statistics of traffic that is already flowing on the networkndash Packet tracesndash Flow statisticsndash Application-level logs
bull Active Measurement Inject ldquoprobingrdquo traffic to measure various characteristicsndash Traceroutendash Pingndash Application-level probes (eg Web downloads)
27
Billing for Internet Usage
bull 95th Percentile billingndash Customer network pays for ldquocommitted information
raterdquo (CIR)ndash Throughput measured every 5 minutes (typically with
SNMP flow statistics also can be used for billing)ndash Customer billed based on 95th percentile
28
Passive Traffic Data Measurement
bull SNMP bytepacket counts everywhere
bull Packet monitoring selected locations
bull Flow monitoring typically at edges (if possible)ndash Direct computation of the traffic matrixndash Input to denial-of-service attack detection
bull Deep Packet Inspection also at edge where possible
29
Simple Network Management Protocol
bull Management Information Base (MIB)ndash Information storendash Unique variables named by
OIDsndash Accessed with SNMP
bull Specific MIBs for bytepacket counts (per link)
Manager Agent
SNMP
DB
ManagedObjects
30
SNMP (Passive)
bull Advantage ubiquitousndash Supported on all networking equipmentndash Multiple products for polling and analyzing data
bull Disadvantages ndash Coarse granularityndash Cannot express complex queries on the datandash Unreliable delivery of the data using UDP
bull Utilityndash Link utilization (billing)ndash Traffic matrix inference
31
Packet-level Monitoring
bull Passive monitoring to collect full packet contents (or at least headers)
bull Advantages lots of detailed informationndash Precise timing informationndash Information in packet headers
bull Disadvantages overheadndash Hard to keep up with high-speed linksndash Often requires a separate monitoring device
32
Full Packet Capture (Passive)
Example Georgia Tech OC3Mon
bull Rack-mounted PCbull Optical splitterbull Data Acquisition and
Generation (DAG) card
Source endacecom
33
What is a flow
bull Source IP addressbull Destination IP addressbull Source portbull Destination portbull Layer 3 protocol typebull TOS byte (DSCP)bull Input logical interface (ifIndex)
34
Cisco NetFlow
bull Basic output ldquoFlow recordrdquondash Most common version is v5
bull Current version (9) is being standardized in the IETF (template-based)ndash More flexible record formatndash Much easier to add new flow record types
Core Network
Collection and Aggregation
Collector (PC)Approximately 1500 bytes
20-50 flow recordsSent more frequently if traffic increases
35
Flow Record Contents
bull Source and Destination IP address and portbull Packet and byte countsbull Start and end timesbull ToS TCP flags
Basic information about the flowhellip
hellipplus information related to routing
bull Next-hop IP addressbull Source and destination ASbull Source and destination prefix
36
flow 1 flow 2 flow 3 flow 4
Aggregating Packets into Flows
bull Criteria 1 Set of packets that ldquobelong togetherrdquondash Sourcedestination IP addresses and port numbersndash Same protocol ToS bits hellip ndash Same inputoutput interfaces at a router (if known)
bull Criteria 2 Packets that are ldquocloserdquo together in timendash Maximum inter-packet spacing (eg 15 sec 30 sec)ndash Example flows 2 and 4 are different flows due to time
37
Reducing Measurement Overhead
bull Filtering on interfacendash destination prefix for a customerndash port number for an application (eg 80 for Web)
bull Sampling before insertion into flow cachendash Random deterministic or hash-based samplingndash 1-out-of-n or stratified based on packetflow sizendash Two types packet-level and flow-level
bull Aggregation after cache evictionndash packetsflows with same next-hop ASndash packetsflows destined to a particular service
38
Packet Sampling for Flow Monitoring
bull Packet sampling before flow creation (Sampled Netflow)ndash 1-out-of-m sampling of individual packets (eg m=100)ndash Create of flow records over the sampled packets
bull Reducing overheadndash Avoid per-packet overhead on (m-1)m packetsndash Avoid creating records for a large number of small flows
bull Increasing overhead (in some cases)ndash May split some long transfers into multiple flow records ndash hellip due to larger time gaps between successive packets
time
not sampled
two flowstimeout
39
Sampling Flow-Level Sampling
bull Sampling of flow records evicted from flow cachendash When evicting flows from table or when analyzing flows
bull Stratified sampling to put weight on ldquoheavyrdquo flowsndash Select all long flows and sample the short flows
bull Reduces the number of flow records ndash Still measures the vast majority of the traffic
Flow 1 40 bytesFlow 2 15580 bytesFlow 3 8196 bytesFlow 4 5350789 bytesFlow 5 532 bytesFlow 6 7432 bytes
sample with 100 probability
sample with 01 probability
sample with 10 probability
40
Two Main Approaches
bull Packet-level Monitoringndash Keep packet-level statisticsndash Examine (and potentially log) variety of packet-level
statistics Essentially anything in the packetndash Timing
bull Flow-level Monitoringndash Monitor packet-by-packet (though sometimes
sampled)ndash Keep aggregate statistics on a flow
41
Packet Capture on High-Speed Links
Example Georgia Tech ldquoOC3Monrdquo
bull Rack-mounted PCbull Optical splitterbull Data Acquisition and
Generation (DAG) card
Source endacecom
42
Characteristics of Packet Capture
bull Allows inspection on every packet on 10G links
bull Disadvantagesndash Costlyndash Requires splitting optical fibersndash Must be able to filterstore data
43
Data Measurement Repositories
bull AbileneInternet 2 Observatoryndash Configuration examplesndash SNMP datandash ISIS BGP routing data NetFlow traffic data
bull RouteViewsndash BGP updatesndash BGP table snapshots
44
Multihoming and Traffic Engineering
45
What is Multihoming
bull The use of redundant network links for the purposes of external connectivity
bull Can be achieved at many layers of the protocol stack and many places in the networkndash Multiple network interfaces in a PC
ndash An ISP with multiple upstream interfaces
bull Can refer to having multiple connections tondash The same ISP
ndash Multiple ISPs
46
Why Multihome
bull Redundancybull Availabilitybull Performancebull Cost
Interdomain traffic engineering the process by which a multihomed network configures its
network to achieve these goals
47
Redundancy
bull Maintain connectivity in the face ofndash Physical connectivity problems (fiber cut device
failures etc)ndash Failures in upstream ISP
48
Performance
bull Use multiple network links at once to achieve higher throughput than just over a single link
bull Allows incoming traffic to be load-balanced
70 of traffic30 of traffic
49
Multihoming in IP Networks Today
bull Stub AS no transit service for other ASesndash No need to use BGP
bull Multi-homed stub AS has connectivity to multiple immediate upstream ISPsndash Need BGPndash No need for a public AS numberndash No need for IP prefix allocation
bull Multi-homed transit AS connectivity to multiple ASes and transit servicendash Need BGP public AS number IP prefix allocation
50
BGP or no
bull Advantages of static routingndash Cheapersmaller routers (less true nowadays)ndash Simpler to configure
bull Advantages of BGPndash More control of your destiny (have providers stop
announcing you)ndash Fastermore intelligent selection of where to send
outbound packetsndash Better debugging of net problems (you can see the
Internet topology now)
51
Same Provider or Multiple
bull If your provider is reliable and fast and affordably and offers good tech-support you may want to multi-home initially to them via some backup path (slow is better than dead)
bull Eventually yoursquoll want to multi-home to different providers to avoid failure modes due to one providerrsquos architecture decisions
52
Multihomed Stub One Link
bull Downstream ISPrsquos routers configure default (ldquostaticrdquo) routes pointing to border router
bull Upstream ISP advertises reachability
Upstream ISP
Multiple links between same pair of routers
Default routes to ldquoborderrdquo
ldquoStubrdquoISP
53
Multihomed Stub Multiple Links
bull Use BGP to share loadbull Use private AS number (why is this OK)bull As before upstream ISP advertises prefix
Upstream ISP
Multiple links to different upstream routers
ldquoStubrdquoISP
Internal routing for ldquohot potatordquo
BGP for load balance at edge
54
Multihomed Stub Multiple ISPs
bull Many possibilitiesndash Load sharingndash Primary-backupndash Selective use of different ISPs
bull Requires BGP public AS number etc
ldquoStubrdquoISP
Upstream
ISP 1
Upstream
ISP 2
55
Multihomed Transit Network
bull BGP everywherebull Incoming and outcoming trafficbull Challenge balancing load on intradomain and egress
links given an offered traffic load
TransitISP
ISP 1
ISP 2
ISP 3
56
Interdomain Traffic Engineering
bull The process by which a network operator configures the network to achievendash Traffic load balancendash Redundancy (primarybackup) etc
bull Two tasksndash Outbound traffic controlndash Inbound traffic control
bull Key Problems Predictability and Scalability
57
Outbound Traffic Control
bull Easier to control than inbound trafficndash Destination-based routing sender determines where
the packets go
bull Control over next-hop AS onlyndash Cannot control selection of the entire path
Provider 1 Provider 2
Control with local preference
58
Outbound Traffic Load Balancingbull Control routes to provider per-prefix
ndash Assign local preference across destination prefixesndash Change the local preference assignments over time
bull Useful inputs to load balancingndash End-to-end path performance datandash Outbound traffic statistics per destination prefix
bull Challenge Getting from traffic volumes to groups of prefixes that should be assigned to each link
Premise of ldquointelligent route controlrdquo preoducts
59
Traffic Engineering Goals
bull Predictabilityndash Ensure the BGP decision process is deterministicndash Assume that BGP updates are (relatively) stable
bull Limit overhead introduced by routing changesndash Minimize frequency of changes to routing policiesndash Limit number of prefixes affected by changes
bull Limit impact on how traffic enters the networkndash Avoid new routes that might change neighborrsquos mindndash Select route with same attributes or at least path length
60
Managing Scalebull Destination prefixes
ndash More than 90000 destination prefixes
bull Donrsquot want to have per-prefix routing policies
ndash Small fraction of prefixes contribute most of the traffic
bull Focus on the small number of heavy hitters
ndash Define routing policies for selected prefixes
bull Routing choicesndash About 27000 unique ldquorouting choicesrdquo
bull Help in reducing the scale of the problem
ndash Small fraction of ldquorouting choicesrdquo contribute most traffic
bull Focus on the very small number of ldquorouting choicesrdquo
ndash Define routing policies on common attributes
61
Achieving Predictability
bull Route prediction with static analysisndash Helpful to know effects before deploymentndash Static analysis can help
TopologyBGP policy
configuration
eBGP routes
Offered traffic
BGP routingmodel
Flow of traffic through the network
62
Challenges to Predictabilitybull For transit ISPs effects on incoming traffic
ndash Lack of coordination strikes again
63
ldquoHot Potatordquo routing
Inter-AS Negotiation
bull Coordination aids predictabilityndash Negotiate where to sendndash Inbound and outboundndash Mutual benefits
bull How to implementndash What info to exchangendash Protecting privacyndash How to prioritize choicesndash How to prevent cheating
Destination 2
Destination 1
multiplepeeringpoints
Provider A
Provider B
64
Outbound Multihoming Goals
bull Redundancyndash Dynamic routing will failover to backup link
bull Performancendash Select provider with best performance per prefix
ndash Requires active probing
bull Costndash Select provider per prefix over time to minimize the
total financial cost
65
Inbound Traffic Control
bull More difficult no control over neighborsrsquo decisions
bull Three common techniques (previously discussed)ndash AS path prependingndash Communities and local preferencendash Prefix splitting
How does todayrsquos paper (MONET) control inbound traffic
66
How many links are enough
K upstream ISPs
Not much benefit beyond 4 ISPs
Akella et al ldquoPerformance Benefits of Multihomingrdquo SIGCOMM 2003
67
Problems with Multihoming in IPv4
bull Routing table growthndash Provider-based addressingndash Advertising prefix out multiple ISPs ndash canrsquot aggregate
bull Poor control over inbound trafficndash Existing mechanisms do not allow hosts to control
inbound traffic
68
GeorgiaTech
Internet Routing Overview
bull Intradomain (ie ldquointra-ASrdquo) routingbull Interdomain routing
Comcast
Abilene
ATampT Cogent
Autonomous Systems (ASes)
69
Configuration Problems ldquoAS 7007rdquoldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
UUNet
Florida InternetBarn
Sprint
70
Diagnosis and Troubleshootingldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
ldquoMicrosofts websites were offline for up to 23 hoursbecause of a [router] misconfigurationhellipit took nearly a day to determine what was wrong and undo the changesrdquo -- wiredcom January 25 2001
ldquoWorldCom Inchellipsuffered a widespread outage on its Internet backbone that affected roughly 20 percent of its US customer base The network problemshellipaffected millions of computer users worldwide A spokeswoman attributed the outage to a route table issue -- cnncom October 3 2002
A number of Covad customers went out from 5pm today due to supposedly a DDOS (distributed denial of service attack) on a key Level3 data center which later was described as a route leak (misconfiguration)ldquo
-- dslreportscom February 23 2004
71
Operator Mailing List (NANOG)Date Mon 18 Oct 2004 091515 -0700Subject Level 3 US east coast issues
Level 3 experiencing widespread unspecified routing issues on the US east coast Master ticket 1086844 Anyone have more specific information
Date Mon 18 Oct 2004 122034 -0400 (EDT)Subject Re Level 3 US east coast issues
Level 3 is currently experiencing a backbone outage causing routing instability and packet loss We are working to restore and will be sending hourly updateshellip
72
Operator Mailing List
0102030405060708090
Filtering RouteLeaks
RouteHijacks
RouteInstability
RoutingLoops
Blackholes
Occurr
ences o
ver
10 Y
ears
1995-1997 1998-2001 2001-2004
Note Only includes problems openly discussed on this list
Compare 83 power outages 1 fire
73
Routing Configuration
Ranking route selection
Dissemination internal route advertisement
Filtering route advertisement
Customer
Competitor
Primary
Backup
74
Internet Business Model (Simplified)
bull CustomerProvider One AS pays another for reachability to some set of destinations
bull ldquoSettlement-freerdquo Peering Bartering Two ASes exchange routes with one another
Provider
Peer
Customer
Preferences implemented with local preference manipulation
Destination
Pay to use
Get paid to use
Free to use
75
Peering Contracts Consistent Export
bull Rules of settlement-free peeringndash Advertise routes at all peering pointsndash Advertised routes must have equal ldquoAS path lengthrdquo
Sprint
ATampT
Enables ldquohot potatordquo routing
ldquoequally goodrdquoroutes
76
Consistent Export
bull Malicedeceptionbull iBGP signaling partitionbull Inconsistent export policy
neighbor 10123route-map PEER permit 10 set prepend 123
neighbor 10456route-map PEER permit 10 set prepend 123 123
Possible Causes
10123 456 1
10456 456 2
Neighbor AS Export
1 1 123
Export Clause Prepend
2 1 123 123
Two different Export Policies
77
Inconsistent Export in Practice
Feamster et al ldquoBorderGuard Detecting Cold Potatoes from Peersrdquo ACM IMC October 2004
78
Blackholes
Date Thu 18 Jul 2002 060510 -0400 (EDT)From Chad Oleary ltcolpoboxcomgtSubject Re problems with 701To ltnanogmeritedugt
Were starting to see the same issues with UUNet again Anyone elseseeing this Trying to reach Qwest
traceroute to 631461901 (631461901) 30 hops max 38 byte packets 1 esc-lp2-gwe-solutionscorpcom (631182201) 1167 ms 1163 ms 1142 ms 2 500Serial2-10GW1TPA2ALTERNET (1571301499) 1097 ms 1059 ms 1044 ms 3 161at-1-0-0XL4ATL1ALTERNET (1526381190) 13839 ms 14108 ms 16638 ms 4 0so-3-1-0XL2ATL5ALTERNET (152630238) 14370 ms 14587 ms 14553 ms 5 POS7-0BR2ATL5ALTERNET (1526382193) 13928 ms 14099 ms 14053 ms 6 7 hellip
79
Security
80
Security ldquoBogonrdquo Routes
Feamster et al ldquoAn Empirical Study of lsquoBogonrsquo Route Advertisementsrdquo ACM CCR January 2005
81
Spam Phishing etc
bull Unsolicited commercial emailbull As of about August 2008 estimates indicate that
about 95 of all email is spambull Common spam filtering techniques
ndash Content-based filtersndash DNS Blacklist (DNSBL) lookups Significant fraction of
todayrsquos DNS traffic
Can IP addresses from which spam is received be spoofed
82
Spam and Routing
83
Worms and Botnets
84
What is a Worm
bull Code that replicates and propagates across the networkndash Often carries a ldquopayloadrdquo
bull Usually spread via exploiting flaws in open servicesndash ldquoVirusesrdquo require user action to spread
bull First worm Robert Morris November 1988ndash 6-10 of all Internet hosts infected ()
bull Many more since but none on that scale until July 2001
85
Example Worm Code Red
bull Initial version July 13 2001
bull Exploited known ISAPI vulnerability in Microsoft IIS Web servers
bull 1st through 20th of each month spread20th through end of each month attack
bull Payload Web site defacementbull Scanning Random IP addressesbull Bug failure to seed random number generator
86
Code Red Revisions
bull Released July 19 2001
bull Payload flooding attack on wwwwhitehousegovndash Attack was mounted at the IP address of the Web site
bull Bug died after 20th of each month
bull Random number generator for IP scanning fixed
87
Code Red Host Infection Rate
Exponential infection rate
Measured using backscatter technique
88
Designing Fast-Spreading Worms
bull Hit-list scanningndash Time to infect first 10k hosts dominates infection timendash Solution Reconnaissance (stealthy scans etc)
bull Permutation scanningndash Observation Most scanning is redundantndash Idea Shared permutation of address space Start scanning
from own IP address Re-randomize when another infected machine is found
bull Internet-scale hit listsndash Flash worm complete infection within 30 seconds
89
Botnets
bull Bots Autonomous programs performing tasksbull Plenty of ldquobenignrdquo bots
ndash eg weatherbug
bull Botnets group of bots ndash Typically carries malicious connotationndash Large numbers of infected machinesndash Machines ldquoenlistedrdquo with infection vectors like worms
(last lecture)
bull Available for simultaneous control by a masterbull Size up to 350000 nodes (from todayrsquos paper)
90
ldquoRallyingrdquo the Botnet
bull Easy to combine worm backdoor functionalitybull Problem how to learn about successfully
infected machines
bull Optionsndash Emailndash Hard-coded email address
91
Botnet Control
bull Botnet master typically runs some IRC server on a well-known port (eg 6667)
bull Infected machine contacts botnet with pre-programmed DNS name (eg big-botde)
bull Dynamic DNS allows controller to move about freely
Infected Machine
DynamicDNS
BotnetController
(IRC server)
92
Some Defenses
93
Idea 1 Ingress Filtering
bull RFC 2827 Routers install filters to drop packets from networks that are not downstream
bull Feasible at edgesbull Difficult to configure closer to network ldquocorerdquo
20469207024 Internet
Drop all packets with source address other than 20469207024
94
Idea 2 uRPF Checks
bull Unicast Reverse Path Forwardingndash Cisco ldquoip verify unicast reverse-pathrdquo
bull Requires symmetric routing
Accept packet from interface only if forwarding table entry for source IP address matches ingress interface
100183
A10018
10015
101203
100181 24
10011 24
Strict Mode uRPF
Enabled
ldquoArdquo Routing TableDestination Next Hop1001024 Int 110018024 Int 2
100183 from wrong interface
95
Problems with uRPF
bull Asymmetric routing
96
S-BGP
bull Address-based PKI validate signaturesndash Authentication of
bull ownership for IP address blocks bull AS number bull an ASs identity and bull a BGP routers identity
ndash Use existing infrastructure (Internet registries etc)ndash Routing origination is digitally signedndash BGP updates are digitally signed
1048708 bull Route attestations A new optional BGP transitive path attribute
ndash carries digital signatures covering the routing information in updates
97
Practical Problems with S-BGP
bull Requires Public-Key Infrastructure
bull Lots of digital signatures to calculate and verifyndash Message overheadndash CPU overhead
bull Calculation expense is greatest when topology is changingndash Caching can help
bull Route aggregation is problematic (maybe thatrsquos OK)
bull Secure route withdrawals when link or node fails
bull Address ownership data out of date
bull Deployment
- Networking
- Goal of This Tutorial
- Todayrsquos Networks
- Business Models
- Billing for Internet Usage
- Net Neutrality
- Network Operations
- Point-of-Presence (PoP)
- Example Abilene Network Topology
- Another Example Backbone
- Internet Routing Overview
- Internet Routing Protocol BGP
- Two Flavors of BGP
- IPv4 Addresses Networks of Networks
- Pre-1994 Classful Addressing
- Classless Interdomain Routing (CIDR)
- Benefits of CIDR
- Growth of IP Prefixes
- 1994-1998 Linear Growth
- Around 2000 Fast Growth Resumes
- Fast growth resumes
- The Address Allocation Process
- Common Problems
- What can go wrong
- Measurement and Monitoring
- Passive vs Active Measurement
- Slide 27
- Passive Traffic Data Measurement
- Simple Network Management Protocol
- SNMP (Passive)
- Packet-level Monitoring
- Full Packet Capture (Passive)
- What is a flow
- Cisco NetFlow
- Flow Record Contents
- Aggregating Packets into Flows
- Reducing Measurement Overhead
- Packet Sampling for Flow Monitoring
- Sampling Flow-Level Sampling
- Two Main Approaches
- Packet Capture on High-Speed Links
- Characteristics of Packet Capture
- Data Measurement Repositories
- Multihoming and Traffic Engineering
- What is Multihoming
- Why Multihome
- Redundancy
- Performance
- Multihoming in IP Networks Today
- BGP or no
- Same Provider or Multiple
- Multihomed Stub One Link
- Multihomed Stub Multiple Links
- Multihomed Stub Multiple ISPs
- Multihomed Transit Network
- Interdomain Traffic Engineering
- Outbound Traffic Control
- Outbound Traffic Load Balancing
- Traffic Engineering Goals
- Managing Scale
- Achieving Predictability
- Challenges to Predictability
- Inter-AS Negotiation
- Outbound Multihoming Goals
- Inbound Traffic Control
- How many links are enough
- Problems with Multihoming in IPv4
- Slide 68
- Slide 69
- Diagnosis and Troubleshooting
- Operator Mailing List (NANOG)
- Operator Mailing List
- Routing Configuration
- Internet Business Model (Simplified)
- Peering Contracts Consistent Export
- Consistent Export
- Inconsistent Export in Practice
- Blackholes
- Security
- Security ldquoBogonrdquo Routes
- Spam Phishing etc
- Spam and Routing
- Worms and Botnets
- What is a Worm
- Example Worm Code Red
- Code Red Revisions
- Code Red Host Infection Rate
- Designing Fast-Spreading Worms
- Botnets
- ldquoRallyingrdquo the Botnet
- Botnet Control
- Some Defenses
- Idea 1 Ingress Filtering
- Idea 2 uRPF Checks
- Problems with uRPF
- S-BGP
- Practical Problems with S-BGP
-
4
Business Models
bull Increasingly commoditized (see Geoff Hustonrsquos talk at NANOG)
bull Status quo Establish transit costs bill at 95th percentile of usage
bull Future differential pricing preference for certain groups of users applications
5
Billing for Internet Usage
bull 95th Percentile billingndash Customer network pays for ldquocommitted information
raterdquo (CIR)ndash Throughput measured every 5 minutes (typically with
SNMP flow statistics also can be used for billing)ndash Customer billed based on 95th percentile
6
Net Neutrality
7
Network Operations
bull Operators run the day-to-day operations of the networkndash Adjusting to shifts in traffic failures etcndash Responding to security threatsndash Provisioning new customers
8
Point-of-Presence (PoP)
bull A ldquoclusterrdquo of routers in a single physical location
bull Inter-PoP linksndash Long distances
ndash High bandwidth
bull Intra-PoP linksndash Cables between racks or floors
ndash Aggregated bandwidth
PoP
9
Example Abilene Network Topology
10
Another Example Backbone
11
GeorgiaTech
Internet Routing Overview
bull Intradomain (ie ldquointra-ASrdquo) routingbull Interdomain routing
Comcast
Abilene
ATampT Cogent
Autonomous Systems (ASes)
12
Internet Routing Protocol BGP
Route Advertisement
Autonomous Systems (ASes)
Session
Traffic
Destination Next-hop AS Path1302070016
1302070016
19258989
6625025244
105782637
174hellip 2637
13
Two Flavors of BGP
bull External BGP (eBGP) exchanging routes between ASes
bull Internal BGP (iBGP) disseminating routes to external destinations among the routers within an AS
eBGPiBGP
Question Whatrsquos the difference between IGP and iBGP
14
IPv4 Addresses Networks of Networks
bull 32-bit number in ldquodotted-quadrdquo notation
ndash wwwccgatechedu --- 130207736
10000010 11001111 00000111 00100100
Network (16 bits) Host (16 bits)
130 207 7 36
bull Problem 232 addresses is a lot of table entries
bull Solution Routing based on network and host
ndash 1302070016 is a 16-bit prefix with 216 IP addresses
Topological Addressing
15
Pre-1994 Classful Addressing
Network ID Host ID
8 16
Class A
32
0
Class B 10
Class C 110
Multicast AddressesClass D 1110
Reserved for experimentsClass E 1111
24
8 blocks (eg MIT has 180008)
16 blocks (eg Georgia Tech has 1302070016)
24 blocks (eg ATampT Labs has 19220225024)
Simple Forwarding Address range specifies network ID length
16
Classless Interdomain Routing (CIDR)
IP Address 65142480 ldquoMaskrdquo 2552552520
01000001 00001110 11111000 00000000
11111111 11111111 11111100 00000000
Use two 32-bit numbers to represent a network Network number = IP address + Mask
Example BellSouth Prefix 6514248022
Address no longer specifies network ID rangeNew forwarding trick Longest Prefix Match
17
Benefits of CIDR
bull Efficiency Can allocate blocks of prefixes on a finer granularity
bull Hierarchy Prefixes can be aggregated into supernets (Not always done Typically not in fact)
Customer 1
Customer 2
ATampT Internet
1220249024
1220231024120008
18
Growth of IP Prefixes
19
1994-1998 Linear Growth
bull About 10000 new entries per yearbull In theory less instability at the edges (why)
Source Geoff Huston
20
Around 2000 Fast Growth Resumes
Claim remaining 8s will be exhausted within the next 5-10 years
T Hain ldquoA Pragmatic Report on IPv4 Address Space Consumptionrdquo Cisco IPJ September 2005
21
Fast growth resumes
Rapid growth in routing tables
Dot-Bomb Hiccup
Significant contributor Multihoming
Source Geoff Huston
22
The Address Allocation Process
bull Allocation policies of RIRs affect pressure on IPv4 address space
IANA
AfriNIC APNIC ARIN LACNIC RIPE
httpwwwianaorgassignmentsipv4-address-space
Georgia Tech
23
Common Problems
bull Diagnosis and troubleshooting (hence measurement)bull Traffic engineeringbull Securitybull Design and capacity planning
24
What can go wrong
Two-thirds of the problems are caused by configuration of the routing protocol
Some downtime is very hard to protect againsthellip
25
Measurement and Monitoring
26
Passive vs Active Measurement
bull Passive Measurement Collection of packets flow statistics of traffic that is already flowing on the networkndash Packet tracesndash Flow statisticsndash Application-level logs
bull Active Measurement Inject ldquoprobingrdquo traffic to measure various characteristicsndash Traceroutendash Pingndash Application-level probes (eg Web downloads)
27
Billing for Internet Usage
bull 95th Percentile billingndash Customer network pays for ldquocommitted information
raterdquo (CIR)ndash Throughput measured every 5 minutes (typically with
SNMP flow statistics also can be used for billing)ndash Customer billed based on 95th percentile
28
Passive Traffic Data Measurement
bull SNMP bytepacket counts everywhere
bull Packet monitoring selected locations
bull Flow monitoring typically at edges (if possible)ndash Direct computation of the traffic matrixndash Input to denial-of-service attack detection
bull Deep Packet Inspection also at edge where possible
29
Simple Network Management Protocol
bull Management Information Base (MIB)ndash Information storendash Unique variables named by
OIDsndash Accessed with SNMP
bull Specific MIBs for bytepacket counts (per link)
Manager Agent
SNMP
DB
ManagedObjects
30
SNMP (Passive)
bull Advantage ubiquitousndash Supported on all networking equipmentndash Multiple products for polling and analyzing data
bull Disadvantages ndash Coarse granularityndash Cannot express complex queries on the datandash Unreliable delivery of the data using UDP
bull Utilityndash Link utilization (billing)ndash Traffic matrix inference
31
Packet-level Monitoring
bull Passive monitoring to collect full packet contents (or at least headers)
bull Advantages lots of detailed informationndash Precise timing informationndash Information in packet headers
bull Disadvantages overheadndash Hard to keep up with high-speed linksndash Often requires a separate monitoring device
32
Full Packet Capture (Passive)
Example Georgia Tech OC3Mon
bull Rack-mounted PCbull Optical splitterbull Data Acquisition and
Generation (DAG) card
Source endacecom
33
What is a flow
bull Source IP addressbull Destination IP addressbull Source portbull Destination portbull Layer 3 protocol typebull TOS byte (DSCP)bull Input logical interface (ifIndex)
34
Cisco NetFlow
bull Basic output ldquoFlow recordrdquondash Most common version is v5
bull Current version (9) is being standardized in the IETF (template-based)ndash More flexible record formatndash Much easier to add new flow record types
Core Network
Collection and Aggregation
Collector (PC)Approximately 1500 bytes
20-50 flow recordsSent more frequently if traffic increases
35
Flow Record Contents
bull Source and Destination IP address and portbull Packet and byte countsbull Start and end timesbull ToS TCP flags
Basic information about the flowhellip
hellipplus information related to routing
bull Next-hop IP addressbull Source and destination ASbull Source and destination prefix
36
flow 1 flow 2 flow 3 flow 4
Aggregating Packets into Flows
bull Criteria 1 Set of packets that ldquobelong togetherrdquondash Sourcedestination IP addresses and port numbersndash Same protocol ToS bits hellip ndash Same inputoutput interfaces at a router (if known)
bull Criteria 2 Packets that are ldquocloserdquo together in timendash Maximum inter-packet spacing (eg 15 sec 30 sec)ndash Example flows 2 and 4 are different flows due to time
37
Reducing Measurement Overhead
bull Filtering on interfacendash destination prefix for a customerndash port number for an application (eg 80 for Web)
bull Sampling before insertion into flow cachendash Random deterministic or hash-based samplingndash 1-out-of-n or stratified based on packetflow sizendash Two types packet-level and flow-level
bull Aggregation after cache evictionndash packetsflows with same next-hop ASndash packetsflows destined to a particular service
38
Packet Sampling for Flow Monitoring
bull Packet sampling before flow creation (Sampled Netflow)ndash 1-out-of-m sampling of individual packets (eg m=100)ndash Create of flow records over the sampled packets
bull Reducing overheadndash Avoid per-packet overhead on (m-1)m packetsndash Avoid creating records for a large number of small flows
bull Increasing overhead (in some cases)ndash May split some long transfers into multiple flow records ndash hellip due to larger time gaps between successive packets
time
not sampled
two flowstimeout
39
Sampling Flow-Level Sampling
bull Sampling of flow records evicted from flow cachendash When evicting flows from table or when analyzing flows
bull Stratified sampling to put weight on ldquoheavyrdquo flowsndash Select all long flows and sample the short flows
bull Reduces the number of flow records ndash Still measures the vast majority of the traffic
Flow 1 40 bytesFlow 2 15580 bytesFlow 3 8196 bytesFlow 4 5350789 bytesFlow 5 532 bytesFlow 6 7432 bytes
sample with 100 probability
sample with 01 probability
sample with 10 probability
40
Two Main Approaches
bull Packet-level Monitoringndash Keep packet-level statisticsndash Examine (and potentially log) variety of packet-level
statistics Essentially anything in the packetndash Timing
bull Flow-level Monitoringndash Monitor packet-by-packet (though sometimes
sampled)ndash Keep aggregate statistics on a flow
41
Packet Capture on High-Speed Links
Example Georgia Tech ldquoOC3Monrdquo
bull Rack-mounted PCbull Optical splitterbull Data Acquisition and
Generation (DAG) card
Source endacecom
42
Characteristics of Packet Capture
bull Allows inspection on every packet on 10G links
bull Disadvantagesndash Costlyndash Requires splitting optical fibersndash Must be able to filterstore data
43
Data Measurement Repositories
bull AbileneInternet 2 Observatoryndash Configuration examplesndash SNMP datandash ISIS BGP routing data NetFlow traffic data
bull RouteViewsndash BGP updatesndash BGP table snapshots
44
Multihoming and Traffic Engineering
45
What is Multihoming
bull The use of redundant network links for the purposes of external connectivity
bull Can be achieved at many layers of the protocol stack and many places in the networkndash Multiple network interfaces in a PC
ndash An ISP with multiple upstream interfaces
bull Can refer to having multiple connections tondash The same ISP
ndash Multiple ISPs
46
Why Multihome
bull Redundancybull Availabilitybull Performancebull Cost
Interdomain traffic engineering the process by which a multihomed network configures its
network to achieve these goals
47
Redundancy
bull Maintain connectivity in the face ofndash Physical connectivity problems (fiber cut device
failures etc)ndash Failures in upstream ISP
48
Performance
bull Use multiple network links at once to achieve higher throughput than just over a single link
bull Allows incoming traffic to be load-balanced
70 of traffic30 of traffic
49
Multihoming in IP Networks Today
bull Stub AS no transit service for other ASesndash No need to use BGP
bull Multi-homed stub AS has connectivity to multiple immediate upstream ISPsndash Need BGPndash No need for a public AS numberndash No need for IP prefix allocation
bull Multi-homed transit AS connectivity to multiple ASes and transit servicendash Need BGP public AS number IP prefix allocation
50
BGP or no
bull Advantages of static routingndash Cheapersmaller routers (less true nowadays)ndash Simpler to configure
bull Advantages of BGPndash More control of your destiny (have providers stop
announcing you)ndash Fastermore intelligent selection of where to send
outbound packetsndash Better debugging of net problems (you can see the
Internet topology now)
51
Same Provider or Multiple
bull If your provider is reliable and fast and affordably and offers good tech-support you may want to multi-home initially to them via some backup path (slow is better than dead)
bull Eventually yoursquoll want to multi-home to different providers to avoid failure modes due to one providerrsquos architecture decisions
52
Multihomed Stub One Link
bull Downstream ISPrsquos routers configure default (ldquostaticrdquo) routes pointing to border router
bull Upstream ISP advertises reachability
Upstream ISP
Multiple links between same pair of routers
Default routes to ldquoborderrdquo
ldquoStubrdquoISP
53
Multihomed Stub Multiple Links
bull Use BGP to share loadbull Use private AS number (why is this OK)bull As before upstream ISP advertises prefix
Upstream ISP
Multiple links to different upstream routers
ldquoStubrdquoISP
Internal routing for ldquohot potatordquo
BGP for load balance at edge
54
Multihomed Stub Multiple ISPs
bull Many possibilitiesndash Load sharingndash Primary-backupndash Selective use of different ISPs
bull Requires BGP public AS number etc
ldquoStubrdquoISP
Upstream
ISP 1
Upstream
ISP 2
55
Multihomed Transit Network
bull BGP everywherebull Incoming and outcoming trafficbull Challenge balancing load on intradomain and egress
links given an offered traffic load
TransitISP
ISP 1
ISP 2
ISP 3
56
Interdomain Traffic Engineering
bull The process by which a network operator configures the network to achievendash Traffic load balancendash Redundancy (primarybackup) etc
bull Two tasksndash Outbound traffic controlndash Inbound traffic control
bull Key Problems Predictability and Scalability
57
Outbound Traffic Control
bull Easier to control than inbound trafficndash Destination-based routing sender determines where
the packets go
bull Control over next-hop AS onlyndash Cannot control selection of the entire path
Provider 1 Provider 2
Control with local preference
58
Outbound Traffic Load Balancingbull Control routes to provider per-prefix
ndash Assign local preference across destination prefixesndash Change the local preference assignments over time
bull Useful inputs to load balancingndash End-to-end path performance datandash Outbound traffic statistics per destination prefix
bull Challenge Getting from traffic volumes to groups of prefixes that should be assigned to each link
Premise of ldquointelligent route controlrdquo preoducts
59
Traffic Engineering Goals
bull Predictabilityndash Ensure the BGP decision process is deterministicndash Assume that BGP updates are (relatively) stable
bull Limit overhead introduced by routing changesndash Minimize frequency of changes to routing policiesndash Limit number of prefixes affected by changes
bull Limit impact on how traffic enters the networkndash Avoid new routes that might change neighborrsquos mindndash Select route with same attributes or at least path length
60
Managing Scalebull Destination prefixes
ndash More than 90000 destination prefixes
bull Donrsquot want to have per-prefix routing policies
ndash Small fraction of prefixes contribute most of the traffic
bull Focus on the small number of heavy hitters
ndash Define routing policies for selected prefixes
bull Routing choicesndash About 27000 unique ldquorouting choicesrdquo
bull Help in reducing the scale of the problem
ndash Small fraction of ldquorouting choicesrdquo contribute most traffic
bull Focus on the very small number of ldquorouting choicesrdquo
ndash Define routing policies on common attributes
61
Achieving Predictability
bull Route prediction with static analysisndash Helpful to know effects before deploymentndash Static analysis can help
TopologyBGP policy
configuration
eBGP routes
Offered traffic
BGP routingmodel
Flow of traffic through the network
62
Challenges to Predictabilitybull For transit ISPs effects on incoming traffic
ndash Lack of coordination strikes again
63
ldquoHot Potatordquo routing
Inter-AS Negotiation
bull Coordination aids predictabilityndash Negotiate where to sendndash Inbound and outboundndash Mutual benefits
bull How to implementndash What info to exchangendash Protecting privacyndash How to prioritize choicesndash How to prevent cheating
Destination 2
Destination 1
multiplepeeringpoints
Provider A
Provider B
64
Outbound Multihoming Goals
bull Redundancyndash Dynamic routing will failover to backup link
bull Performancendash Select provider with best performance per prefix
ndash Requires active probing
bull Costndash Select provider per prefix over time to minimize the
total financial cost
65
Inbound Traffic Control
bull More difficult no control over neighborsrsquo decisions
bull Three common techniques (previously discussed)ndash AS path prependingndash Communities and local preferencendash Prefix splitting
How does todayrsquos paper (MONET) control inbound traffic
66
How many links are enough
K upstream ISPs
Not much benefit beyond 4 ISPs
Akella et al ldquoPerformance Benefits of Multihomingrdquo SIGCOMM 2003
67
Problems with Multihoming in IPv4
bull Routing table growthndash Provider-based addressingndash Advertising prefix out multiple ISPs ndash canrsquot aggregate
bull Poor control over inbound trafficndash Existing mechanisms do not allow hosts to control
inbound traffic
68
GeorgiaTech
Internet Routing Overview
bull Intradomain (ie ldquointra-ASrdquo) routingbull Interdomain routing
Comcast
Abilene
ATampT Cogent
Autonomous Systems (ASes)
69
Configuration Problems ldquoAS 7007rdquoldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
UUNet
Florida InternetBarn
Sprint
70
Diagnosis and Troubleshootingldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
ldquoMicrosofts websites were offline for up to 23 hoursbecause of a [router] misconfigurationhellipit took nearly a day to determine what was wrong and undo the changesrdquo -- wiredcom January 25 2001
ldquoWorldCom Inchellipsuffered a widespread outage on its Internet backbone that affected roughly 20 percent of its US customer base The network problemshellipaffected millions of computer users worldwide A spokeswoman attributed the outage to a route table issue -- cnncom October 3 2002
A number of Covad customers went out from 5pm today due to supposedly a DDOS (distributed denial of service attack) on a key Level3 data center which later was described as a route leak (misconfiguration)ldquo
-- dslreportscom February 23 2004
71
Operator Mailing List (NANOG)Date Mon 18 Oct 2004 091515 -0700Subject Level 3 US east coast issues
Level 3 experiencing widespread unspecified routing issues on the US east coast Master ticket 1086844 Anyone have more specific information
Date Mon 18 Oct 2004 122034 -0400 (EDT)Subject Re Level 3 US east coast issues
Level 3 is currently experiencing a backbone outage causing routing instability and packet loss We are working to restore and will be sending hourly updateshellip
72
Operator Mailing List
0102030405060708090
Filtering RouteLeaks
RouteHijacks
RouteInstability
RoutingLoops
Blackholes
Occurr
ences o
ver
10 Y
ears
1995-1997 1998-2001 2001-2004
Note Only includes problems openly discussed on this list
Compare 83 power outages 1 fire
73
Routing Configuration
Ranking route selection
Dissemination internal route advertisement
Filtering route advertisement
Customer
Competitor
Primary
Backup
74
Internet Business Model (Simplified)
bull CustomerProvider One AS pays another for reachability to some set of destinations
bull ldquoSettlement-freerdquo Peering Bartering Two ASes exchange routes with one another
Provider
Peer
Customer
Preferences implemented with local preference manipulation
Destination
Pay to use
Get paid to use
Free to use
75
Peering Contracts Consistent Export
bull Rules of settlement-free peeringndash Advertise routes at all peering pointsndash Advertised routes must have equal ldquoAS path lengthrdquo
Sprint
ATampT
Enables ldquohot potatordquo routing
ldquoequally goodrdquoroutes
76
Consistent Export
bull Malicedeceptionbull iBGP signaling partitionbull Inconsistent export policy
neighbor 10123route-map PEER permit 10 set prepend 123
neighbor 10456route-map PEER permit 10 set prepend 123 123
Possible Causes
10123 456 1
10456 456 2
Neighbor AS Export
1 1 123
Export Clause Prepend
2 1 123 123
Two different Export Policies
77
Inconsistent Export in Practice
Feamster et al ldquoBorderGuard Detecting Cold Potatoes from Peersrdquo ACM IMC October 2004
78
Blackholes
Date Thu 18 Jul 2002 060510 -0400 (EDT)From Chad Oleary ltcolpoboxcomgtSubject Re problems with 701To ltnanogmeritedugt
Were starting to see the same issues with UUNet again Anyone elseseeing this Trying to reach Qwest
traceroute to 631461901 (631461901) 30 hops max 38 byte packets 1 esc-lp2-gwe-solutionscorpcom (631182201) 1167 ms 1163 ms 1142 ms 2 500Serial2-10GW1TPA2ALTERNET (1571301499) 1097 ms 1059 ms 1044 ms 3 161at-1-0-0XL4ATL1ALTERNET (1526381190) 13839 ms 14108 ms 16638 ms 4 0so-3-1-0XL2ATL5ALTERNET (152630238) 14370 ms 14587 ms 14553 ms 5 POS7-0BR2ATL5ALTERNET (1526382193) 13928 ms 14099 ms 14053 ms 6 7 hellip
79
Security
80
Security ldquoBogonrdquo Routes
Feamster et al ldquoAn Empirical Study of lsquoBogonrsquo Route Advertisementsrdquo ACM CCR January 2005
81
Spam Phishing etc
bull Unsolicited commercial emailbull As of about August 2008 estimates indicate that
about 95 of all email is spambull Common spam filtering techniques
ndash Content-based filtersndash DNS Blacklist (DNSBL) lookups Significant fraction of
todayrsquos DNS traffic
Can IP addresses from which spam is received be spoofed
82
Spam and Routing
83
Worms and Botnets
84
What is a Worm
bull Code that replicates and propagates across the networkndash Often carries a ldquopayloadrdquo
bull Usually spread via exploiting flaws in open servicesndash ldquoVirusesrdquo require user action to spread
bull First worm Robert Morris November 1988ndash 6-10 of all Internet hosts infected ()
bull Many more since but none on that scale until July 2001
85
Example Worm Code Red
bull Initial version July 13 2001
bull Exploited known ISAPI vulnerability in Microsoft IIS Web servers
bull 1st through 20th of each month spread20th through end of each month attack
bull Payload Web site defacementbull Scanning Random IP addressesbull Bug failure to seed random number generator
86
Code Red Revisions
bull Released July 19 2001
bull Payload flooding attack on wwwwhitehousegovndash Attack was mounted at the IP address of the Web site
bull Bug died after 20th of each month
bull Random number generator for IP scanning fixed
87
Code Red Host Infection Rate
Exponential infection rate
Measured using backscatter technique
88
Designing Fast-Spreading Worms
bull Hit-list scanningndash Time to infect first 10k hosts dominates infection timendash Solution Reconnaissance (stealthy scans etc)
bull Permutation scanningndash Observation Most scanning is redundantndash Idea Shared permutation of address space Start scanning
from own IP address Re-randomize when another infected machine is found
bull Internet-scale hit listsndash Flash worm complete infection within 30 seconds
89
Botnets
bull Bots Autonomous programs performing tasksbull Plenty of ldquobenignrdquo bots
ndash eg weatherbug
bull Botnets group of bots ndash Typically carries malicious connotationndash Large numbers of infected machinesndash Machines ldquoenlistedrdquo with infection vectors like worms
(last lecture)
bull Available for simultaneous control by a masterbull Size up to 350000 nodes (from todayrsquos paper)
90
ldquoRallyingrdquo the Botnet
bull Easy to combine worm backdoor functionalitybull Problem how to learn about successfully
infected machines
bull Optionsndash Emailndash Hard-coded email address
91
Botnet Control
bull Botnet master typically runs some IRC server on a well-known port (eg 6667)
bull Infected machine contacts botnet with pre-programmed DNS name (eg big-botde)
bull Dynamic DNS allows controller to move about freely
Infected Machine
DynamicDNS
BotnetController
(IRC server)
92
Some Defenses
93
Idea 1 Ingress Filtering
bull RFC 2827 Routers install filters to drop packets from networks that are not downstream
bull Feasible at edgesbull Difficult to configure closer to network ldquocorerdquo
20469207024 Internet
Drop all packets with source address other than 20469207024
94
Idea 2 uRPF Checks
bull Unicast Reverse Path Forwardingndash Cisco ldquoip verify unicast reverse-pathrdquo
bull Requires symmetric routing
Accept packet from interface only if forwarding table entry for source IP address matches ingress interface
100183
A10018
10015
101203
100181 24
10011 24
Strict Mode uRPF
Enabled
ldquoArdquo Routing TableDestination Next Hop1001024 Int 110018024 Int 2
100183 from wrong interface
95
Problems with uRPF
bull Asymmetric routing
96
S-BGP
bull Address-based PKI validate signaturesndash Authentication of
bull ownership for IP address blocks bull AS number bull an ASs identity and bull a BGP routers identity
ndash Use existing infrastructure (Internet registries etc)ndash Routing origination is digitally signedndash BGP updates are digitally signed
1048708 bull Route attestations A new optional BGP transitive path attribute
ndash carries digital signatures covering the routing information in updates
97
Practical Problems with S-BGP
bull Requires Public-Key Infrastructure
bull Lots of digital signatures to calculate and verifyndash Message overheadndash CPU overhead
bull Calculation expense is greatest when topology is changingndash Caching can help
bull Route aggregation is problematic (maybe thatrsquos OK)
bull Secure route withdrawals when link or node fails
bull Address ownership data out of date
bull Deployment
- Networking
- Goal of This Tutorial
- Todayrsquos Networks
- Business Models
- Billing for Internet Usage
- Net Neutrality
- Network Operations
- Point-of-Presence (PoP)
- Example Abilene Network Topology
- Another Example Backbone
- Internet Routing Overview
- Internet Routing Protocol BGP
- Two Flavors of BGP
- IPv4 Addresses Networks of Networks
- Pre-1994 Classful Addressing
- Classless Interdomain Routing (CIDR)
- Benefits of CIDR
- Growth of IP Prefixes
- 1994-1998 Linear Growth
- Around 2000 Fast Growth Resumes
- Fast growth resumes
- The Address Allocation Process
- Common Problems
- What can go wrong
- Measurement and Monitoring
- Passive vs Active Measurement
- Slide 27
- Passive Traffic Data Measurement
- Simple Network Management Protocol
- SNMP (Passive)
- Packet-level Monitoring
- Full Packet Capture (Passive)
- What is a flow
- Cisco NetFlow
- Flow Record Contents
- Aggregating Packets into Flows
- Reducing Measurement Overhead
- Packet Sampling for Flow Monitoring
- Sampling Flow-Level Sampling
- Two Main Approaches
- Packet Capture on High-Speed Links
- Characteristics of Packet Capture
- Data Measurement Repositories
- Multihoming and Traffic Engineering
- What is Multihoming
- Why Multihome
- Redundancy
- Performance
- Multihoming in IP Networks Today
- BGP or no
- Same Provider or Multiple
- Multihomed Stub One Link
- Multihomed Stub Multiple Links
- Multihomed Stub Multiple ISPs
- Multihomed Transit Network
- Interdomain Traffic Engineering
- Outbound Traffic Control
- Outbound Traffic Load Balancing
- Traffic Engineering Goals
- Managing Scale
- Achieving Predictability
- Challenges to Predictability
- Inter-AS Negotiation
- Outbound Multihoming Goals
- Inbound Traffic Control
- How many links are enough
- Problems with Multihoming in IPv4
- Slide 68
- Slide 69
- Diagnosis and Troubleshooting
- Operator Mailing List (NANOG)
- Operator Mailing List
- Routing Configuration
- Internet Business Model (Simplified)
- Peering Contracts Consistent Export
- Consistent Export
- Inconsistent Export in Practice
- Blackholes
- Security
- Security ldquoBogonrdquo Routes
- Spam Phishing etc
- Spam and Routing
- Worms and Botnets
- What is a Worm
- Example Worm Code Red
- Code Red Revisions
- Code Red Host Infection Rate
- Designing Fast-Spreading Worms
- Botnets
- ldquoRallyingrdquo the Botnet
- Botnet Control
- Some Defenses
- Idea 1 Ingress Filtering
- Idea 2 uRPF Checks
- Problems with uRPF
- S-BGP
- Practical Problems with S-BGP
-
5
Billing for Internet Usage
bull 95th Percentile billingndash Customer network pays for ldquocommitted information
raterdquo (CIR)ndash Throughput measured every 5 minutes (typically with
SNMP flow statistics also can be used for billing)ndash Customer billed based on 95th percentile
6
Net Neutrality
7
Network Operations
bull Operators run the day-to-day operations of the networkndash Adjusting to shifts in traffic failures etcndash Responding to security threatsndash Provisioning new customers
8
Point-of-Presence (PoP)
bull A ldquoclusterrdquo of routers in a single physical location
bull Inter-PoP linksndash Long distances
ndash High bandwidth
bull Intra-PoP linksndash Cables between racks or floors
ndash Aggregated bandwidth
PoP
9
Example Abilene Network Topology
10
Another Example Backbone
11
GeorgiaTech
Internet Routing Overview
bull Intradomain (ie ldquointra-ASrdquo) routingbull Interdomain routing
Comcast
Abilene
ATampT Cogent
Autonomous Systems (ASes)
12
Internet Routing Protocol BGP
Route Advertisement
Autonomous Systems (ASes)
Session
Traffic
Destination Next-hop AS Path1302070016
1302070016
19258989
6625025244
105782637
174hellip 2637
13
Two Flavors of BGP
bull External BGP (eBGP) exchanging routes between ASes
bull Internal BGP (iBGP) disseminating routes to external destinations among the routers within an AS
eBGPiBGP
Question Whatrsquos the difference between IGP and iBGP
14
IPv4 Addresses Networks of Networks
bull 32-bit number in ldquodotted-quadrdquo notation
ndash wwwccgatechedu --- 130207736
10000010 11001111 00000111 00100100
Network (16 bits) Host (16 bits)
130 207 7 36
bull Problem 232 addresses is a lot of table entries
bull Solution Routing based on network and host
ndash 1302070016 is a 16-bit prefix with 216 IP addresses
Topological Addressing
15
Pre-1994 Classful Addressing
Network ID Host ID
8 16
Class A
32
0
Class B 10
Class C 110
Multicast AddressesClass D 1110
Reserved for experimentsClass E 1111
24
8 blocks (eg MIT has 180008)
16 blocks (eg Georgia Tech has 1302070016)
24 blocks (eg ATampT Labs has 19220225024)
Simple Forwarding Address range specifies network ID length
16
Classless Interdomain Routing (CIDR)
IP Address 65142480 ldquoMaskrdquo 2552552520
01000001 00001110 11111000 00000000
11111111 11111111 11111100 00000000
Use two 32-bit numbers to represent a network Network number = IP address + Mask
Example BellSouth Prefix 6514248022
Address no longer specifies network ID rangeNew forwarding trick Longest Prefix Match
17
Benefits of CIDR
bull Efficiency Can allocate blocks of prefixes on a finer granularity
bull Hierarchy Prefixes can be aggregated into supernets (Not always done Typically not in fact)
Customer 1
Customer 2
ATampT Internet
1220249024
1220231024120008
18
Growth of IP Prefixes
19
1994-1998 Linear Growth
bull About 10000 new entries per yearbull In theory less instability at the edges (why)
Source Geoff Huston
20
Around 2000 Fast Growth Resumes
Claim remaining 8s will be exhausted within the next 5-10 years
T Hain ldquoA Pragmatic Report on IPv4 Address Space Consumptionrdquo Cisco IPJ September 2005
21
Fast growth resumes
Rapid growth in routing tables
Dot-Bomb Hiccup
Significant contributor Multihoming
Source Geoff Huston
22
The Address Allocation Process
bull Allocation policies of RIRs affect pressure on IPv4 address space
IANA
AfriNIC APNIC ARIN LACNIC RIPE
httpwwwianaorgassignmentsipv4-address-space
Georgia Tech
23
Common Problems
bull Diagnosis and troubleshooting (hence measurement)bull Traffic engineeringbull Securitybull Design and capacity planning
24
What can go wrong
Two-thirds of the problems are caused by configuration of the routing protocol
Some downtime is very hard to protect againsthellip
25
Measurement and Monitoring
26
Passive vs Active Measurement
bull Passive Measurement Collection of packets flow statistics of traffic that is already flowing on the networkndash Packet tracesndash Flow statisticsndash Application-level logs
bull Active Measurement Inject ldquoprobingrdquo traffic to measure various characteristicsndash Traceroutendash Pingndash Application-level probes (eg Web downloads)
27
Billing for Internet Usage
bull 95th Percentile billingndash Customer network pays for ldquocommitted information
raterdquo (CIR)ndash Throughput measured every 5 minutes (typically with
SNMP flow statistics also can be used for billing)ndash Customer billed based on 95th percentile
28
Passive Traffic Data Measurement
bull SNMP bytepacket counts everywhere
bull Packet monitoring selected locations
bull Flow monitoring typically at edges (if possible)ndash Direct computation of the traffic matrixndash Input to denial-of-service attack detection
bull Deep Packet Inspection also at edge where possible
29
Simple Network Management Protocol
bull Management Information Base (MIB)ndash Information storendash Unique variables named by
OIDsndash Accessed with SNMP
bull Specific MIBs for bytepacket counts (per link)
Manager Agent
SNMP
DB
ManagedObjects
30
SNMP (Passive)
bull Advantage ubiquitousndash Supported on all networking equipmentndash Multiple products for polling and analyzing data
bull Disadvantages ndash Coarse granularityndash Cannot express complex queries on the datandash Unreliable delivery of the data using UDP
bull Utilityndash Link utilization (billing)ndash Traffic matrix inference
31
Packet-level Monitoring
bull Passive monitoring to collect full packet contents (or at least headers)
bull Advantages lots of detailed informationndash Precise timing informationndash Information in packet headers
bull Disadvantages overheadndash Hard to keep up with high-speed linksndash Often requires a separate monitoring device
32
Full Packet Capture (Passive)
Example Georgia Tech OC3Mon
bull Rack-mounted PCbull Optical splitterbull Data Acquisition and
Generation (DAG) card
Source endacecom
33
What is a flow
bull Source IP addressbull Destination IP addressbull Source portbull Destination portbull Layer 3 protocol typebull TOS byte (DSCP)bull Input logical interface (ifIndex)
34
Cisco NetFlow
bull Basic output ldquoFlow recordrdquondash Most common version is v5
bull Current version (9) is being standardized in the IETF (template-based)ndash More flexible record formatndash Much easier to add new flow record types
Core Network
Collection and Aggregation
Collector (PC)Approximately 1500 bytes
20-50 flow recordsSent more frequently if traffic increases
35
Flow Record Contents
bull Source and Destination IP address and portbull Packet and byte countsbull Start and end timesbull ToS TCP flags
Basic information about the flowhellip
hellipplus information related to routing
bull Next-hop IP addressbull Source and destination ASbull Source and destination prefix
36
flow 1 flow 2 flow 3 flow 4
Aggregating Packets into Flows
bull Criteria 1 Set of packets that ldquobelong togetherrdquondash Sourcedestination IP addresses and port numbersndash Same protocol ToS bits hellip ndash Same inputoutput interfaces at a router (if known)
bull Criteria 2 Packets that are ldquocloserdquo together in timendash Maximum inter-packet spacing (eg 15 sec 30 sec)ndash Example flows 2 and 4 are different flows due to time
37
Reducing Measurement Overhead
bull Filtering on interfacendash destination prefix for a customerndash port number for an application (eg 80 for Web)
bull Sampling before insertion into flow cachendash Random deterministic or hash-based samplingndash 1-out-of-n or stratified based on packetflow sizendash Two types packet-level and flow-level
bull Aggregation after cache evictionndash packetsflows with same next-hop ASndash packetsflows destined to a particular service
38
Packet Sampling for Flow Monitoring
bull Packet sampling before flow creation (Sampled Netflow)ndash 1-out-of-m sampling of individual packets (eg m=100)ndash Create of flow records over the sampled packets
bull Reducing overheadndash Avoid per-packet overhead on (m-1)m packetsndash Avoid creating records for a large number of small flows
bull Increasing overhead (in some cases)ndash May split some long transfers into multiple flow records ndash hellip due to larger time gaps between successive packets
time
not sampled
two flowstimeout
39
Sampling Flow-Level Sampling
bull Sampling of flow records evicted from flow cachendash When evicting flows from table or when analyzing flows
bull Stratified sampling to put weight on ldquoheavyrdquo flowsndash Select all long flows and sample the short flows
bull Reduces the number of flow records ndash Still measures the vast majority of the traffic
Flow 1 40 bytesFlow 2 15580 bytesFlow 3 8196 bytesFlow 4 5350789 bytesFlow 5 532 bytesFlow 6 7432 bytes
sample with 100 probability
sample with 01 probability
sample with 10 probability
40
Two Main Approaches
bull Packet-level Monitoringndash Keep packet-level statisticsndash Examine (and potentially log) variety of packet-level
statistics Essentially anything in the packetndash Timing
bull Flow-level Monitoringndash Monitor packet-by-packet (though sometimes
sampled)ndash Keep aggregate statistics on a flow
41
Packet Capture on High-Speed Links
Example Georgia Tech ldquoOC3Monrdquo
bull Rack-mounted PCbull Optical splitterbull Data Acquisition and
Generation (DAG) card
Source endacecom
42
Characteristics of Packet Capture
bull Allows inspection on every packet on 10G links
bull Disadvantagesndash Costlyndash Requires splitting optical fibersndash Must be able to filterstore data
43
Data Measurement Repositories
bull AbileneInternet 2 Observatoryndash Configuration examplesndash SNMP datandash ISIS BGP routing data NetFlow traffic data
bull RouteViewsndash BGP updatesndash BGP table snapshots
44
Multihoming and Traffic Engineering
45
What is Multihoming
bull The use of redundant network links for the purposes of external connectivity
bull Can be achieved at many layers of the protocol stack and many places in the networkndash Multiple network interfaces in a PC
ndash An ISP with multiple upstream interfaces
bull Can refer to having multiple connections tondash The same ISP
ndash Multiple ISPs
46
Why Multihome
bull Redundancybull Availabilitybull Performancebull Cost
Interdomain traffic engineering the process by which a multihomed network configures its
network to achieve these goals
47
Redundancy
bull Maintain connectivity in the face ofndash Physical connectivity problems (fiber cut device
failures etc)ndash Failures in upstream ISP
48
Performance
bull Use multiple network links at once to achieve higher throughput than just over a single link
bull Allows incoming traffic to be load-balanced
70 of traffic30 of traffic
49
Multihoming in IP Networks Today
bull Stub AS no transit service for other ASesndash No need to use BGP
bull Multi-homed stub AS has connectivity to multiple immediate upstream ISPsndash Need BGPndash No need for a public AS numberndash No need for IP prefix allocation
bull Multi-homed transit AS connectivity to multiple ASes and transit servicendash Need BGP public AS number IP prefix allocation
50
BGP or no
bull Advantages of static routingndash Cheapersmaller routers (less true nowadays)ndash Simpler to configure
bull Advantages of BGPndash More control of your destiny (have providers stop
announcing you)ndash Fastermore intelligent selection of where to send
outbound packetsndash Better debugging of net problems (you can see the
Internet topology now)
51
Same Provider or Multiple
bull If your provider is reliable and fast and affordably and offers good tech-support you may want to multi-home initially to them via some backup path (slow is better than dead)
bull Eventually yoursquoll want to multi-home to different providers to avoid failure modes due to one providerrsquos architecture decisions
52
Multihomed Stub One Link
bull Downstream ISPrsquos routers configure default (ldquostaticrdquo) routes pointing to border router
bull Upstream ISP advertises reachability
Upstream ISP
Multiple links between same pair of routers
Default routes to ldquoborderrdquo
ldquoStubrdquoISP
53
Multihomed Stub Multiple Links
bull Use BGP to share loadbull Use private AS number (why is this OK)bull As before upstream ISP advertises prefix
Upstream ISP
Multiple links to different upstream routers
ldquoStubrdquoISP
Internal routing for ldquohot potatordquo
BGP for load balance at edge
54
Multihomed Stub Multiple ISPs
bull Many possibilitiesndash Load sharingndash Primary-backupndash Selective use of different ISPs
bull Requires BGP public AS number etc
ldquoStubrdquoISP
Upstream
ISP 1
Upstream
ISP 2
55
Multihomed Transit Network
bull BGP everywherebull Incoming and outcoming trafficbull Challenge balancing load on intradomain and egress
links given an offered traffic load
TransitISP
ISP 1
ISP 2
ISP 3
56
Interdomain Traffic Engineering
bull The process by which a network operator configures the network to achievendash Traffic load balancendash Redundancy (primarybackup) etc
bull Two tasksndash Outbound traffic controlndash Inbound traffic control
bull Key Problems Predictability and Scalability
57
Outbound Traffic Control
bull Easier to control than inbound trafficndash Destination-based routing sender determines where
the packets go
bull Control over next-hop AS onlyndash Cannot control selection of the entire path
Provider 1 Provider 2
Control with local preference
58
Outbound Traffic Load Balancingbull Control routes to provider per-prefix
ndash Assign local preference across destination prefixesndash Change the local preference assignments over time
bull Useful inputs to load balancingndash End-to-end path performance datandash Outbound traffic statistics per destination prefix
bull Challenge Getting from traffic volumes to groups of prefixes that should be assigned to each link
Premise of ldquointelligent route controlrdquo preoducts
59
Traffic Engineering Goals
bull Predictabilityndash Ensure the BGP decision process is deterministicndash Assume that BGP updates are (relatively) stable
bull Limit overhead introduced by routing changesndash Minimize frequency of changes to routing policiesndash Limit number of prefixes affected by changes
bull Limit impact on how traffic enters the networkndash Avoid new routes that might change neighborrsquos mindndash Select route with same attributes or at least path length
60
Managing Scalebull Destination prefixes
ndash More than 90000 destination prefixes
bull Donrsquot want to have per-prefix routing policies
ndash Small fraction of prefixes contribute most of the traffic
bull Focus on the small number of heavy hitters
ndash Define routing policies for selected prefixes
bull Routing choicesndash About 27000 unique ldquorouting choicesrdquo
bull Help in reducing the scale of the problem
ndash Small fraction of ldquorouting choicesrdquo contribute most traffic
bull Focus on the very small number of ldquorouting choicesrdquo
ndash Define routing policies on common attributes
61
Achieving Predictability
bull Route prediction with static analysisndash Helpful to know effects before deploymentndash Static analysis can help
TopologyBGP policy
configuration
eBGP routes
Offered traffic
BGP routingmodel
Flow of traffic through the network
62
Challenges to Predictabilitybull For transit ISPs effects on incoming traffic
ndash Lack of coordination strikes again
63
ldquoHot Potatordquo routing
Inter-AS Negotiation
bull Coordination aids predictabilityndash Negotiate where to sendndash Inbound and outboundndash Mutual benefits
bull How to implementndash What info to exchangendash Protecting privacyndash How to prioritize choicesndash How to prevent cheating
Destination 2
Destination 1
multiplepeeringpoints
Provider A
Provider B
64
Outbound Multihoming Goals
bull Redundancyndash Dynamic routing will failover to backup link
bull Performancendash Select provider with best performance per prefix
ndash Requires active probing
bull Costndash Select provider per prefix over time to minimize the
total financial cost
65
Inbound Traffic Control
bull More difficult no control over neighborsrsquo decisions
bull Three common techniques (previously discussed)ndash AS path prependingndash Communities and local preferencendash Prefix splitting
How does todayrsquos paper (MONET) control inbound traffic
66
How many links are enough
K upstream ISPs
Not much benefit beyond 4 ISPs
Akella et al ldquoPerformance Benefits of Multihomingrdquo SIGCOMM 2003
67
Problems with Multihoming in IPv4
bull Routing table growthndash Provider-based addressingndash Advertising prefix out multiple ISPs ndash canrsquot aggregate
bull Poor control over inbound trafficndash Existing mechanisms do not allow hosts to control
inbound traffic
68
GeorgiaTech
Internet Routing Overview
bull Intradomain (ie ldquointra-ASrdquo) routingbull Interdomain routing
Comcast
Abilene
ATampT Cogent
Autonomous Systems (ASes)
69
Configuration Problems ldquoAS 7007rdquoldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
UUNet
Florida InternetBarn
Sprint
70
Diagnosis and Troubleshootingldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
ldquoMicrosofts websites were offline for up to 23 hoursbecause of a [router] misconfigurationhellipit took nearly a day to determine what was wrong and undo the changesrdquo -- wiredcom January 25 2001
ldquoWorldCom Inchellipsuffered a widespread outage on its Internet backbone that affected roughly 20 percent of its US customer base The network problemshellipaffected millions of computer users worldwide A spokeswoman attributed the outage to a route table issue -- cnncom October 3 2002
A number of Covad customers went out from 5pm today due to supposedly a DDOS (distributed denial of service attack) on a key Level3 data center which later was described as a route leak (misconfiguration)ldquo
-- dslreportscom February 23 2004
71
Operator Mailing List (NANOG)Date Mon 18 Oct 2004 091515 -0700Subject Level 3 US east coast issues
Level 3 experiencing widespread unspecified routing issues on the US east coast Master ticket 1086844 Anyone have more specific information
Date Mon 18 Oct 2004 122034 -0400 (EDT)Subject Re Level 3 US east coast issues
Level 3 is currently experiencing a backbone outage causing routing instability and packet loss We are working to restore and will be sending hourly updateshellip
72
Operator Mailing List
0102030405060708090
Filtering RouteLeaks
RouteHijacks
RouteInstability
RoutingLoops
Blackholes
Occurr
ences o
ver
10 Y
ears
1995-1997 1998-2001 2001-2004
Note Only includes problems openly discussed on this list
Compare 83 power outages 1 fire
73
Routing Configuration
Ranking route selection
Dissemination internal route advertisement
Filtering route advertisement
Customer
Competitor
Primary
Backup
74
Internet Business Model (Simplified)
bull CustomerProvider One AS pays another for reachability to some set of destinations
bull ldquoSettlement-freerdquo Peering Bartering Two ASes exchange routes with one another
Provider
Peer
Customer
Preferences implemented with local preference manipulation
Destination
Pay to use
Get paid to use
Free to use
75
Peering Contracts Consistent Export
bull Rules of settlement-free peeringndash Advertise routes at all peering pointsndash Advertised routes must have equal ldquoAS path lengthrdquo
Sprint
ATampT
Enables ldquohot potatordquo routing
ldquoequally goodrdquoroutes
76
Consistent Export
bull Malicedeceptionbull iBGP signaling partitionbull Inconsistent export policy
neighbor 10123route-map PEER permit 10 set prepend 123
neighbor 10456route-map PEER permit 10 set prepend 123 123
Possible Causes
10123 456 1
10456 456 2
Neighbor AS Export
1 1 123
Export Clause Prepend
2 1 123 123
Two different Export Policies
77
Inconsistent Export in Practice
Feamster et al ldquoBorderGuard Detecting Cold Potatoes from Peersrdquo ACM IMC October 2004
78
Blackholes
Date Thu 18 Jul 2002 060510 -0400 (EDT)From Chad Oleary ltcolpoboxcomgtSubject Re problems with 701To ltnanogmeritedugt
Were starting to see the same issues with UUNet again Anyone elseseeing this Trying to reach Qwest
traceroute to 631461901 (631461901) 30 hops max 38 byte packets 1 esc-lp2-gwe-solutionscorpcom (631182201) 1167 ms 1163 ms 1142 ms 2 500Serial2-10GW1TPA2ALTERNET (1571301499) 1097 ms 1059 ms 1044 ms 3 161at-1-0-0XL4ATL1ALTERNET (1526381190) 13839 ms 14108 ms 16638 ms 4 0so-3-1-0XL2ATL5ALTERNET (152630238) 14370 ms 14587 ms 14553 ms 5 POS7-0BR2ATL5ALTERNET (1526382193) 13928 ms 14099 ms 14053 ms 6 7 hellip
79
Security
80
Security ldquoBogonrdquo Routes
Feamster et al ldquoAn Empirical Study of lsquoBogonrsquo Route Advertisementsrdquo ACM CCR January 2005
81
Spam Phishing etc
bull Unsolicited commercial emailbull As of about August 2008 estimates indicate that
about 95 of all email is spambull Common spam filtering techniques
ndash Content-based filtersndash DNS Blacklist (DNSBL) lookups Significant fraction of
todayrsquos DNS traffic
Can IP addresses from which spam is received be spoofed
82
Spam and Routing
83
Worms and Botnets
84
What is a Worm
bull Code that replicates and propagates across the networkndash Often carries a ldquopayloadrdquo
bull Usually spread via exploiting flaws in open servicesndash ldquoVirusesrdquo require user action to spread
bull First worm Robert Morris November 1988ndash 6-10 of all Internet hosts infected ()
bull Many more since but none on that scale until July 2001
85
Example Worm Code Red
bull Initial version July 13 2001
bull Exploited known ISAPI vulnerability in Microsoft IIS Web servers
bull 1st through 20th of each month spread20th through end of each month attack
bull Payload Web site defacementbull Scanning Random IP addressesbull Bug failure to seed random number generator
86
Code Red Revisions
bull Released July 19 2001
bull Payload flooding attack on wwwwhitehousegovndash Attack was mounted at the IP address of the Web site
bull Bug died after 20th of each month
bull Random number generator for IP scanning fixed
87
Code Red Host Infection Rate
Exponential infection rate
Measured using backscatter technique
88
Designing Fast-Spreading Worms
bull Hit-list scanningndash Time to infect first 10k hosts dominates infection timendash Solution Reconnaissance (stealthy scans etc)
bull Permutation scanningndash Observation Most scanning is redundantndash Idea Shared permutation of address space Start scanning
from own IP address Re-randomize when another infected machine is found
bull Internet-scale hit listsndash Flash worm complete infection within 30 seconds
89
Botnets
bull Bots Autonomous programs performing tasksbull Plenty of ldquobenignrdquo bots
ndash eg weatherbug
bull Botnets group of bots ndash Typically carries malicious connotationndash Large numbers of infected machinesndash Machines ldquoenlistedrdquo with infection vectors like worms
(last lecture)
bull Available for simultaneous control by a masterbull Size up to 350000 nodes (from todayrsquos paper)
90
ldquoRallyingrdquo the Botnet
bull Easy to combine worm backdoor functionalitybull Problem how to learn about successfully
infected machines
bull Optionsndash Emailndash Hard-coded email address
91
Botnet Control
bull Botnet master typically runs some IRC server on a well-known port (eg 6667)
bull Infected machine contacts botnet with pre-programmed DNS name (eg big-botde)
bull Dynamic DNS allows controller to move about freely
Infected Machine
DynamicDNS
BotnetController
(IRC server)
92
Some Defenses
93
Idea 1 Ingress Filtering
bull RFC 2827 Routers install filters to drop packets from networks that are not downstream
bull Feasible at edgesbull Difficult to configure closer to network ldquocorerdquo
20469207024 Internet
Drop all packets with source address other than 20469207024
94
Idea 2 uRPF Checks
bull Unicast Reverse Path Forwardingndash Cisco ldquoip verify unicast reverse-pathrdquo
bull Requires symmetric routing
Accept packet from interface only if forwarding table entry for source IP address matches ingress interface
100183
A10018
10015
101203
100181 24
10011 24
Strict Mode uRPF
Enabled
ldquoArdquo Routing TableDestination Next Hop1001024 Int 110018024 Int 2
100183 from wrong interface
95
Problems with uRPF
bull Asymmetric routing
96
S-BGP
bull Address-based PKI validate signaturesndash Authentication of
bull ownership for IP address blocks bull AS number bull an ASs identity and bull a BGP routers identity
ndash Use existing infrastructure (Internet registries etc)ndash Routing origination is digitally signedndash BGP updates are digitally signed
1048708 bull Route attestations A new optional BGP transitive path attribute
ndash carries digital signatures covering the routing information in updates
97
Practical Problems with S-BGP
bull Requires Public-Key Infrastructure
bull Lots of digital signatures to calculate and verifyndash Message overheadndash CPU overhead
bull Calculation expense is greatest when topology is changingndash Caching can help
bull Route aggregation is problematic (maybe thatrsquos OK)
bull Secure route withdrawals when link or node fails
bull Address ownership data out of date
bull Deployment
- Networking
- Goal of This Tutorial
- Todayrsquos Networks
- Business Models
- Billing for Internet Usage
- Net Neutrality
- Network Operations
- Point-of-Presence (PoP)
- Example Abilene Network Topology
- Another Example Backbone
- Internet Routing Overview
- Internet Routing Protocol BGP
- Two Flavors of BGP
- IPv4 Addresses Networks of Networks
- Pre-1994 Classful Addressing
- Classless Interdomain Routing (CIDR)
- Benefits of CIDR
- Growth of IP Prefixes
- 1994-1998 Linear Growth
- Around 2000 Fast Growth Resumes
- Fast growth resumes
- The Address Allocation Process
- Common Problems
- What can go wrong
- Measurement and Monitoring
- Passive vs Active Measurement
- Slide 27
- Passive Traffic Data Measurement
- Simple Network Management Protocol
- SNMP (Passive)
- Packet-level Monitoring
- Full Packet Capture (Passive)
- What is a flow
- Cisco NetFlow
- Flow Record Contents
- Aggregating Packets into Flows
- Reducing Measurement Overhead
- Packet Sampling for Flow Monitoring
- Sampling Flow-Level Sampling
- Two Main Approaches
- Packet Capture on High-Speed Links
- Characteristics of Packet Capture
- Data Measurement Repositories
- Multihoming and Traffic Engineering
- What is Multihoming
- Why Multihome
- Redundancy
- Performance
- Multihoming in IP Networks Today
- BGP or no
- Same Provider or Multiple
- Multihomed Stub One Link
- Multihomed Stub Multiple Links
- Multihomed Stub Multiple ISPs
- Multihomed Transit Network
- Interdomain Traffic Engineering
- Outbound Traffic Control
- Outbound Traffic Load Balancing
- Traffic Engineering Goals
- Managing Scale
- Achieving Predictability
- Challenges to Predictability
- Inter-AS Negotiation
- Outbound Multihoming Goals
- Inbound Traffic Control
- How many links are enough
- Problems with Multihoming in IPv4
- Slide 68
- Slide 69
- Diagnosis and Troubleshooting
- Operator Mailing List (NANOG)
- Operator Mailing List
- Routing Configuration
- Internet Business Model (Simplified)
- Peering Contracts Consistent Export
- Consistent Export
- Inconsistent Export in Practice
- Blackholes
- Security
- Security ldquoBogonrdquo Routes
- Spam Phishing etc
- Spam and Routing
- Worms and Botnets
- What is a Worm
- Example Worm Code Red
- Code Red Revisions
- Code Red Host Infection Rate
- Designing Fast-Spreading Worms
- Botnets
- ldquoRallyingrdquo the Botnet
- Botnet Control
- Some Defenses
- Idea 1 Ingress Filtering
- Idea 2 uRPF Checks
- Problems with uRPF
- S-BGP
- Practical Problems with S-BGP
-
6
Net Neutrality
7
Network Operations
bull Operators run the day-to-day operations of the networkndash Adjusting to shifts in traffic failures etcndash Responding to security threatsndash Provisioning new customers
8
Point-of-Presence (PoP)
bull A ldquoclusterrdquo of routers in a single physical location
bull Inter-PoP linksndash Long distances
ndash High bandwidth
bull Intra-PoP linksndash Cables between racks or floors
ndash Aggregated bandwidth
PoP
9
Example Abilene Network Topology
10
Another Example Backbone
11
GeorgiaTech
Internet Routing Overview
bull Intradomain (ie ldquointra-ASrdquo) routingbull Interdomain routing
Comcast
Abilene
ATampT Cogent
Autonomous Systems (ASes)
12
Internet Routing Protocol BGP
Route Advertisement
Autonomous Systems (ASes)
Session
Traffic
Destination Next-hop AS Path1302070016
1302070016
19258989
6625025244
105782637
174hellip 2637
13
Two Flavors of BGP
bull External BGP (eBGP) exchanging routes between ASes
bull Internal BGP (iBGP) disseminating routes to external destinations among the routers within an AS
eBGPiBGP
Question Whatrsquos the difference between IGP and iBGP
14
IPv4 Addresses Networks of Networks
bull 32-bit number in ldquodotted-quadrdquo notation
ndash wwwccgatechedu --- 130207736
10000010 11001111 00000111 00100100
Network (16 bits) Host (16 bits)
130 207 7 36
bull Problem 232 addresses is a lot of table entries
bull Solution Routing based on network and host
ndash 1302070016 is a 16-bit prefix with 216 IP addresses
Topological Addressing
15
Pre-1994 Classful Addressing
Network ID Host ID
8 16
Class A
32
0
Class B 10
Class C 110
Multicast AddressesClass D 1110
Reserved for experimentsClass E 1111
24
8 blocks (eg MIT has 180008)
16 blocks (eg Georgia Tech has 1302070016)
24 blocks (eg ATampT Labs has 19220225024)
Simple Forwarding Address range specifies network ID length
16
Classless Interdomain Routing (CIDR)
IP Address 65142480 ldquoMaskrdquo 2552552520
01000001 00001110 11111000 00000000
11111111 11111111 11111100 00000000
Use two 32-bit numbers to represent a network Network number = IP address + Mask
Example BellSouth Prefix 6514248022
Address no longer specifies network ID rangeNew forwarding trick Longest Prefix Match
17
Benefits of CIDR
bull Efficiency Can allocate blocks of prefixes on a finer granularity
bull Hierarchy Prefixes can be aggregated into supernets (Not always done Typically not in fact)
Customer 1
Customer 2
ATampT Internet
1220249024
1220231024120008
18
Growth of IP Prefixes
19
1994-1998 Linear Growth
bull About 10000 new entries per yearbull In theory less instability at the edges (why)
Source Geoff Huston
20
Around 2000 Fast Growth Resumes
Claim remaining 8s will be exhausted within the next 5-10 years
T Hain ldquoA Pragmatic Report on IPv4 Address Space Consumptionrdquo Cisco IPJ September 2005
21
Fast growth resumes
Rapid growth in routing tables
Dot-Bomb Hiccup
Significant contributor Multihoming
Source Geoff Huston
22
The Address Allocation Process
bull Allocation policies of RIRs affect pressure on IPv4 address space
IANA
AfriNIC APNIC ARIN LACNIC RIPE
httpwwwianaorgassignmentsipv4-address-space
Georgia Tech
23
Common Problems
bull Diagnosis and troubleshooting (hence measurement)bull Traffic engineeringbull Securitybull Design and capacity planning
24
What can go wrong
Two-thirds of the problems are caused by configuration of the routing protocol
Some downtime is very hard to protect againsthellip
25
Measurement and Monitoring
26
Passive vs Active Measurement
bull Passive Measurement Collection of packets flow statistics of traffic that is already flowing on the networkndash Packet tracesndash Flow statisticsndash Application-level logs
bull Active Measurement Inject ldquoprobingrdquo traffic to measure various characteristicsndash Traceroutendash Pingndash Application-level probes (eg Web downloads)
27
Billing for Internet Usage
bull 95th Percentile billingndash Customer network pays for ldquocommitted information
raterdquo (CIR)ndash Throughput measured every 5 minutes (typically with
SNMP flow statistics also can be used for billing)ndash Customer billed based on 95th percentile
28
Passive Traffic Data Measurement
bull SNMP bytepacket counts everywhere
bull Packet monitoring selected locations
bull Flow monitoring typically at edges (if possible)ndash Direct computation of the traffic matrixndash Input to denial-of-service attack detection
bull Deep Packet Inspection also at edge where possible
29
Simple Network Management Protocol
bull Management Information Base (MIB)ndash Information storendash Unique variables named by
OIDsndash Accessed with SNMP
bull Specific MIBs for bytepacket counts (per link)
Manager Agent
SNMP
DB
ManagedObjects
30
SNMP (Passive)
bull Advantage ubiquitousndash Supported on all networking equipmentndash Multiple products for polling and analyzing data
bull Disadvantages ndash Coarse granularityndash Cannot express complex queries on the datandash Unreliable delivery of the data using UDP
bull Utilityndash Link utilization (billing)ndash Traffic matrix inference
31
Packet-level Monitoring
bull Passive monitoring to collect full packet contents (or at least headers)
bull Advantages lots of detailed informationndash Precise timing informationndash Information in packet headers
bull Disadvantages overheadndash Hard to keep up with high-speed linksndash Often requires a separate monitoring device
32
Full Packet Capture (Passive)
Example Georgia Tech OC3Mon
bull Rack-mounted PCbull Optical splitterbull Data Acquisition and
Generation (DAG) card
Source endacecom
33
What is a flow
bull Source IP addressbull Destination IP addressbull Source portbull Destination portbull Layer 3 protocol typebull TOS byte (DSCP)bull Input logical interface (ifIndex)
34
Cisco NetFlow
bull Basic output ldquoFlow recordrdquondash Most common version is v5
bull Current version (9) is being standardized in the IETF (template-based)ndash More flexible record formatndash Much easier to add new flow record types
Core Network
Collection and Aggregation
Collector (PC)Approximately 1500 bytes
20-50 flow recordsSent more frequently if traffic increases
35
Flow Record Contents
bull Source and Destination IP address and portbull Packet and byte countsbull Start and end timesbull ToS TCP flags
Basic information about the flowhellip
hellipplus information related to routing
bull Next-hop IP addressbull Source and destination ASbull Source and destination prefix
36
flow 1 flow 2 flow 3 flow 4
Aggregating Packets into Flows
bull Criteria 1 Set of packets that ldquobelong togetherrdquondash Sourcedestination IP addresses and port numbersndash Same protocol ToS bits hellip ndash Same inputoutput interfaces at a router (if known)
bull Criteria 2 Packets that are ldquocloserdquo together in timendash Maximum inter-packet spacing (eg 15 sec 30 sec)ndash Example flows 2 and 4 are different flows due to time
37
Reducing Measurement Overhead
bull Filtering on interfacendash destination prefix for a customerndash port number for an application (eg 80 for Web)
bull Sampling before insertion into flow cachendash Random deterministic or hash-based samplingndash 1-out-of-n or stratified based on packetflow sizendash Two types packet-level and flow-level
bull Aggregation after cache evictionndash packetsflows with same next-hop ASndash packetsflows destined to a particular service
38
Packet Sampling for Flow Monitoring
bull Packet sampling before flow creation (Sampled Netflow)ndash 1-out-of-m sampling of individual packets (eg m=100)ndash Create of flow records over the sampled packets
bull Reducing overheadndash Avoid per-packet overhead on (m-1)m packetsndash Avoid creating records for a large number of small flows
bull Increasing overhead (in some cases)ndash May split some long transfers into multiple flow records ndash hellip due to larger time gaps between successive packets
time
not sampled
two flowstimeout
39
Sampling Flow-Level Sampling
bull Sampling of flow records evicted from flow cachendash When evicting flows from table or when analyzing flows
bull Stratified sampling to put weight on ldquoheavyrdquo flowsndash Select all long flows and sample the short flows
bull Reduces the number of flow records ndash Still measures the vast majority of the traffic
Flow 1 40 bytesFlow 2 15580 bytesFlow 3 8196 bytesFlow 4 5350789 bytesFlow 5 532 bytesFlow 6 7432 bytes
sample with 100 probability
sample with 01 probability
sample with 10 probability
40
Two Main Approaches
bull Packet-level Monitoringndash Keep packet-level statisticsndash Examine (and potentially log) variety of packet-level
statistics Essentially anything in the packetndash Timing
bull Flow-level Monitoringndash Monitor packet-by-packet (though sometimes
sampled)ndash Keep aggregate statistics on a flow
41
Packet Capture on High-Speed Links
Example Georgia Tech ldquoOC3Monrdquo
bull Rack-mounted PCbull Optical splitterbull Data Acquisition and
Generation (DAG) card
Source endacecom
42
Characteristics of Packet Capture
bull Allows inspection on every packet on 10G links
bull Disadvantagesndash Costlyndash Requires splitting optical fibersndash Must be able to filterstore data
43
Data Measurement Repositories
bull AbileneInternet 2 Observatoryndash Configuration examplesndash SNMP datandash ISIS BGP routing data NetFlow traffic data
bull RouteViewsndash BGP updatesndash BGP table snapshots
44
Multihoming and Traffic Engineering
45
What is Multihoming
bull The use of redundant network links for the purposes of external connectivity
bull Can be achieved at many layers of the protocol stack and many places in the networkndash Multiple network interfaces in a PC
ndash An ISP with multiple upstream interfaces
bull Can refer to having multiple connections tondash The same ISP
ndash Multiple ISPs
46
Why Multihome
bull Redundancybull Availabilitybull Performancebull Cost
Interdomain traffic engineering the process by which a multihomed network configures its
network to achieve these goals
47
Redundancy
bull Maintain connectivity in the face ofndash Physical connectivity problems (fiber cut device
failures etc)ndash Failures in upstream ISP
48
Performance
bull Use multiple network links at once to achieve higher throughput than just over a single link
bull Allows incoming traffic to be load-balanced
70 of traffic30 of traffic
49
Multihoming in IP Networks Today
bull Stub AS no transit service for other ASesndash No need to use BGP
bull Multi-homed stub AS has connectivity to multiple immediate upstream ISPsndash Need BGPndash No need for a public AS numberndash No need for IP prefix allocation
bull Multi-homed transit AS connectivity to multiple ASes and transit servicendash Need BGP public AS number IP prefix allocation
50
BGP or no
bull Advantages of static routingndash Cheapersmaller routers (less true nowadays)ndash Simpler to configure
bull Advantages of BGPndash More control of your destiny (have providers stop
announcing you)ndash Fastermore intelligent selection of where to send
outbound packetsndash Better debugging of net problems (you can see the
Internet topology now)
51
Same Provider or Multiple
bull If your provider is reliable and fast and affordably and offers good tech-support you may want to multi-home initially to them via some backup path (slow is better than dead)
bull Eventually yoursquoll want to multi-home to different providers to avoid failure modes due to one providerrsquos architecture decisions
52
Multihomed Stub One Link
bull Downstream ISPrsquos routers configure default (ldquostaticrdquo) routes pointing to border router
bull Upstream ISP advertises reachability
Upstream ISP
Multiple links between same pair of routers
Default routes to ldquoborderrdquo
ldquoStubrdquoISP
53
Multihomed Stub Multiple Links
bull Use BGP to share loadbull Use private AS number (why is this OK)bull As before upstream ISP advertises prefix
Upstream ISP
Multiple links to different upstream routers
ldquoStubrdquoISP
Internal routing for ldquohot potatordquo
BGP for load balance at edge
54
Multihomed Stub Multiple ISPs
bull Many possibilitiesndash Load sharingndash Primary-backupndash Selective use of different ISPs
bull Requires BGP public AS number etc
ldquoStubrdquoISP
Upstream
ISP 1
Upstream
ISP 2
55
Multihomed Transit Network
bull BGP everywherebull Incoming and outcoming trafficbull Challenge balancing load on intradomain and egress
links given an offered traffic load
TransitISP
ISP 1
ISP 2
ISP 3
56
Interdomain Traffic Engineering
bull The process by which a network operator configures the network to achievendash Traffic load balancendash Redundancy (primarybackup) etc
bull Two tasksndash Outbound traffic controlndash Inbound traffic control
bull Key Problems Predictability and Scalability
57
Outbound Traffic Control
bull Easier to control than inbound trafficndash Destination-based routing sender determines where
the packets go
bull Control over next-hop AS onlyndash Cannot control selection of the entire path
Provider 1 Provider 2
Control with local preference
58
Outbound Traffic Load Balancingbull Control routes to provider per-prefix
ndash Assign local preference across destination prefixesndash Change the local preference assignments over time
bull Useful inputs to load balancingndash End-to-end path performance datandash Outbound traffic statistics per destination prefix
bull Challenge Getting from traffic volumes to groups of prefixes that should be assigned to each link
Premise of ldquointelligent route controlrdquo preoducts
59
Traffic Engineering Goals
bull Predictabilityndash Ensure the BGP decision process is deterministicndash Assume that BGP updates are (relatively) stable
bull Limit overhead introduced by routing changesndash Minimize frequency of changes to routing policiesndash Limit number of prefixes affected by changes
bull Limit impact on how traffic enters the networkndash Avoid new routes that might change neighborrsquos mindndash Select route with same attributes or at least path length
60
Managing Scalebull Destination prefixes
ndash More than 90000 destination prefixes
bull Donrsquot want to have per-prefix routing policies
ndash Small fraction of prefixes contribute most of the traffic
bull Focus on the small number of heavy hitters
ndash Define routing policies for selected prefixes
bull Routing choicesndash About 27000 unique ldquorouting choicesrdquo
bull Help in reducing the scale of the problem
ndash Small fraction of ldquorouting choicesrdquo contribute most traffic
bull Focus on the very small number of ldquorouting choicesrdquo
ndash Define routing policies on common attributes
61
Achieving Predictability
bull Route prediction with static analysisndash Helpful to know effects before deploymentndash Static analysis can help
TopologyBGP policy
configuration
eBGP routes
Offered traffic
BGP routingmodel
Flow of traffic through the network
62
Challenges to Predictabilitybull For transit ISPs effects on incoming traffic
ndash Lack of coordination strikes again
63
ldquoHot Potatordquo routing
Inter-AS Negotiation
bull Coordination aids predictabilityndash Negotiate where to sendndash Inbound and outboundndash Mutual benefits
bull How to implementndash What info to exchangendash Protecting privacyndash How to prioritize choicesndash How to prevent cheating
Destination 2
Destination 1
multiplepeeringpoints
Provider A
Provider B
64
Outbound Multihoming Goals
bull Redundancyndash Dynamic routing will failover to backup link
bull Performancendash Select provider with best performance per prefix
ndash Requires active probing
bull Costndash Select provider per prefix over time to minimize the
total financial cost
65
Inbound Traffic Control
bull More difficult no control over neighborsrsquo decisions
bull Three common techniques (previously discussed)ndash AS path prependingndash Communities and local preferencendash Prefix splitting
How does todayrsquos paper (MONET) control inbound traffic
66
How many links are enough
K upstream ISPs
Not much benefit beyond 4 ISPs
Akella et al ldquoPerformance Benefits of Multihomingrdquo SIGCOMM 2003
67
Problems with Multihoming in IPv4
bull Routing table growthndash Provider-based addressingndash Advertising prefix out multiple ISPs ndash canrsquot aggregate
bull Poor control over inbound trafficndash Existing mechanisms do not allow hosts to control
inbound traffic
68
GeorgiaTech
Internet Routing Overview
bull Intradomain (ie ldquointra-ASrdquo) routingbull Interdomain routing
Comcast
Abilene
ATampT Cogent
Autonomous Systems (ASes)
69
Configuration Problems ldquoAS 7007rdquoldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
UUNet
Florida InternetBarn
Sprint
70
Diagnosis and Troubleshootingldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
ldquoMicrosofts websites were offline for up to 23 hoursbecause of a [router] misconfigurationhellipit took nearly a day to determine what was wrong and undo the changesrdquo -- wiredcom January 25 2001
ldquoWorldCom Inchellipsuffered a widespread outage on its Internet backbone that affected roughly 20 percent of its US customer base The network problemshellipaffected millions of computer users worldwide A spokeswoman attributed the outage to a route table issue -- cnncom October 3 2002
A number of Covad customers went out from 5pm today due to supposedly a DDOS (distributed denial of service attack) on a key Level3 data center which later was described as a route leak (misconfiguration)ldquo
-- dslreportscom February 23 2004
71
Operator Mailing List (NANOG)Date Mon 18 Oct 2004 091515 -0700Subject Level 3 US east coast issues
Level 3 experiencing widespread unspecified routing issues on the US east coast Master ticket 1086844 Anyone have more specific information
Date Mon 18 Oct 2004 122034 -0400 (EDT)Subject Re Level 3 US east coast issues
Level 3 is currently experiencing a backbone outage causing routing instability and packet loss We are working to restore and will be sending hourly updateshellip
72
Operator Mailing List
0102030405060708090
Filtering RouteLeaks
RouteHijacks
RouteInstability
RoutingLoops
Blackholes
Occurr
ences o
ver
10 Y
ears
1995-1997 1998-2001 2001-2004
Note Only includes problems openly discussed on this list
Compare 83 power outages 1 fire
73
Routing Configuration
Ranking route selection
Dissemination internal route advertisement
Filtering route advertisement
Customer
Competitor
Primary
Backup
74
Internet Business Model (Simplified)
bull CustomerProvider One AS pays another for reachability to some set of destinations
bull ldquoSettlement-freerdquo Peering Bartering Two ASes exchange routes with one another
Provider
Peer
Customer
Preferences implemented with local preference manipulation
Destination
Pay to use
Get paid to use
Free to use
75
Peering Contracts Consistent Export
bull Rules of settlement-free peeringndash Advertise routes at all peering pointsndash Advertised routes must have equal ldquoAS path lengthrdquo
Sprint
ATampT
Enables ldquohot potatordquo routing
ldquoequally goodrdquoroutes
76
Consistent Export
bull Malicedeceptionbull iBGP signaling partitionbull Inconsistent export policy
neighbor 10123route-map PEER permit 10 set prepend 123
neighbor 10456route-map PEER permit 10 set prepend 123 123
Possible Causes
10123 456 1
10456 456 2
Neighbor AS Export
1 1 123
Export Clause Prepend
2 1 123 123
Two different Export Policies
77
Inconsistent Export in Practice
Feamster et al ldquoBorderGuard Detecting Cold Potatoes from Peersrdquo ACM IMC October 2004
78
Blackholes
Date Thu 18 Jul 2002 060510 -0400 (EDT)From Chad Oleary ltcolpoboxcomgtSubject Re problems with 701To ltnanogmeritedugt
Were starting to see the same issues with UUNet again Anyone elseseeing this Trying to reach Qwest
traceroute to 631461901 (631461901) 30 hops max 38 byte packets 1 esc-lp2-gwe-solutionscorpcom (631182201) 1167 ms 1163 ms 1142 ms 2 500Serial2-10GW1TPA2ALTERNET (1571301499) 1097 ms 1059 ms 1044 ms 3 161at-1-0-0XL4ATL1ALTERNET (1526381190) 13839 ms 14108 ms 16638 ms 4 0so-3-1-0XL2ATL5ALTERNET (152630238) 14370 ms 14587 ms 14553 ms 5 POS7-0BR2ATL5ALTERNET (1526382193) 13928 ms 14099 ms 14053 ms 6 7 hellip
79
Security
80
Security ldquoBogonrdquo Routes
Feamster et al ldquoAn Empirical Study of lsquoBogonrsquo Route Advertisementsrdquo ACM CCR January 2005
81
Spam Phishing etc
bull Unsolicited commercial emailbull As of about August 2008 estimates indicate that
about 95 of all email is spambull Common spam filtering techniques
ndash Content-based filtersndash DNS Blacklist (DNSBL) lookups Significant fraction of
todayrsquos DNS traffic
Can IP addresses from which spam is received be spoofed
82
Spam and Routing
83
Worms and Botnets
84
What is a Worm
bull Code that replicates and propagates across the networkndash Often carries a ldquopayloadrdquo
bull Usually spread via exploiting flaws in open servicesndash ldquoVirusesrdquo require user action to spread
bull First worm Robert Morris November 1988ndash 6-10 of all Internet hosts infected ()
bull Many more since but none on that scale until July 2001
85
Example Worm Code Red
bull Initial version July 13 2001
bull Exploited known ISAPI vulnerability in Microsoft IIS Web servers
bull 1st through 20th of each month spread20th through end of each month attack
bull Payload Web site defacementbull Scanning Random IP addressesbull Bug failure to seed random number generator
86
Code Red Revisions
bull Released July 19 2001
bull Payload flooding attack on wwwwhitehousegovndash Attack was mounted at the IP address of the Web site
bull Bug died after 20th of each month
bull Random number generator for IP scanning fixed
87
Code Red Host Infection Rate
Exponential infection rate
Measured using backscatter technique
88
Designing Fast-Spreading Worms
bull Hit-list scanningndash Time to infect first 10k hosts dominates infection timendash Solution Reconnaissance (stealthy scans etc)
bull Permutation scanningndash Observation Most scanning is redundantndash Idea Shared permutation of address space Start scanning
from own IP address Re-randomize when another infected machine is found
bull Internet-scale hit listsndash Flash worm complete infection within 30 seconds
89
Botnets
bull Bots Autonomous programs performing tasksbull Plenty of ldquobenignrdquo bots
ndash eg weatherbug
bull Botnets group of bots ndash Typically carries malicious connotationndash Large numbers of infected machinesndash Machines ldquoenlistedrdquo with infection vectors like worms
(last lecture)
bull Available for simultaneous control by a masterbull Size up to 350000 nodes (from todayrsquos paper)
90
ldquoRallyingrdquo the Botnet
bull Easy to combine worm backdoor functionalitybull Problem how to learn about successfully
infected machines
bull Optionsndash Emailndash Hard-coded email address
91
Botnet Control
bull Botnet master typically runs some IRC server on a well-known port (eg 6667)
bull Infected machine contacts botnet with pre-programmed DNS name (eg big-botde)
bull Dynamic DNS allows controller to move about freely
Infected Machine
DynamicDNS
BotnetController
(IRC server)
92
Some Defenses
93
Idea 1 Ingress Filtering
bull RFC 2827 Routers install filters to drop packets from networks that are not downstream
bull Feasible at edgesbull Difficult to configure closer to network ldquocorerdquo
20469207024 Internet
Drop all packets with source address other than 20469207024
94
Idea 2 uRPF Checks
bull Unicast Reverse Path Forwardingndash Cisco ldquoip verify unicast reverse-pathrdquo
bull Requires symmetric routing
Accept packet from interface only if forwarding table entry for source IP address matches ingress interface
100183
A10018
10015
101203
100181 24
10011 24
Strict Mode uRPF
Enabled
ldquoArdquo Routing TableDestination Next Hop1001024 Int 110018024 Int 2
100183 from wrong interface
95
Problems with uRPF
bull Asymmetric routing
96
S-BGP
bull Address-based PKI validate signaturesndash Authentication of
bull ownership for IP address blocks bull AS number bull an ASs identity and bull a BGP routers identity
ndash Use existing infrastructure (Internet registries etc)ndash Routing origination is digitally signedndash BGP updates are digitally signed
1048708 bull Route attestations A new optional BGP transitive path attribute
ndash carries digital signatures covering the routing information in updates
97
Practical Problems with S-BGP
bull Requires Public-Key Infrastructure
bull Lots of digital signatures to calculate and verifyndash Message overheadndash CPU overhead
bull Calculation expense is greatest when topology is changingndash Caching can help
bull Route aggregation is problematic (maybe thatrsquos OK)
bull Secure route withdrawals when link or node fails
bull Address ownership data out of date
bull Deployment
- Networking
- Goal of This Tutorial
- Todayrsquos Networks
- Business Models
- Billing for Internet Usage
- Net Neutrality
- Network Operations
- Point-of-Presence (PoP)
- Example Abilene Network Topology
- Another Example Backbone
- Internet Routing Overview
- Internet Routing Protocol BGP
- Two Flavors of BGP
- IPv4 Addresses Networks of Networks
- Pre-1994 Classful Addressing
- Classless Interdomain Routing (CIDR)
- Benefits of CIDR
- Growth of IP Prefixes
- 1994-1998 Linear Growth
- Around 2000 Fast Growth Resumes
- Fast growth resumes
- The Address Allocation Process
- Common Problems
- What can go wrong
- Measurement and Monitoring
- Passive vs Active Measurement
- Slide 27
- Passive Traffic Data Measurement
- Simple Network Management Protocol
- SNMP (Passive)
- Packet-level Monitoring
- Full Packet Capture (Passive)
- What is a flow
- Cisco NetFlow
- Flow Record Contents
- Aggregating Packets into Flows
- Reducing Measurement Overhead
- Packet Sampling for Flow Monitoring
- Sampling Flow-Level Sampling
- Two Main Approaches
- Packet Capture on High-Speed Links
- Characteristics of Packet Capture
- Data Measurement Repositories
- Multihoming and Traffic Engineering
- What is Multihoming
- Why Multihome
- Redundancy
- Performance
- Multihoming in IP Networks Today
- BGP or no
- Same Provider or Multiple
- Multihomed Stub One Link
- Multihomed Stub Multiple Links
- Multihomed Stub Multiple ISPs
- Multihomed Transit Network
- Interdomain Traffic Engineering
- Outbound Traffic Control
- Outbound Traffic Load Balancing
- Traffic Engineering Goals
- Managing Scale
- Achieving Predictability
- Challenges to Predictability
- Inter-AS Negotiation
- Outbound Multihoming Goals
- Inbound Traffic Control
- How many links are enough
- Problems with Multihoming in IPv4
- Slide 68
- Slide 69
- Diagnosis and Troubleshooting
- Operator Mailing List (NANOG)
- Operator Mailing List
- Routing Configuration
- Internet Business Model (Simplified)
- Peering Contracts Consistent Export
- Consistent Export
- Inconsistent Export in Practice
- Blackholes
- Security
- Security ldquoBogonrdquo Routes
- Spam Phishing etc
- Spam and Routing
- Worms and Botnets
- What is a Worm
- Example Worm Code Red
- Code Red Revisions
- Code Red Host Infection Rate
- Designing Fast-Spreading Worms
- Botnets
- ldquoRallyingrdquo the Botnet
- Botnet Control
- Some Defenses
- Idea 1 Ingress Filtering
- Idea 2 uRPF Checks
- Problems with uRPF
- S-BGP
- Practical Problems with S-BGP
-
7
Network Operations
bull Operators run the day-to-day operations of the networkndash Adjusting to shifts in traffic failures etcndash Responding to security threatsndash Provisioning new customers
8
Point-of-Presence (PoP)
bull A ldquoclusterrdquo of routers in a single physical location
bull Inter-PoP linksndash Long distances
ndash High bandwidth
bull Intra-PoP linksndash Cables between racks or floors
ndash Aggregated bandwidth
PoP
9
Example Abilene Network Topology
10
Another Example Backbone
11
GeorgiaTech
Internet Routing Overview
bull Intradomain (ie ldquointra-ASrdquo) routingbull Interdomain routing
Comcast
Abilene
ATampT Cogent
Autonomous Systems (ASes)
12
Internet Routing Protocol BGP
Route Advertisement
Autonomous Systems (ASes)
Session
Traffic
Destination Next-hop AS Path1302070016
1302070016
19258989
6625025244
105782637
174hellip 2637
13
Two Flavors of BGP
bull External BGP (eBGP) exchanging routes between ASes
bull Internal BGP (iBGP) disseminating routes to external destinations among the routers within an AS
eBGPiBGP
Question Whatrsquos the difference between IGP and iBGP
14
IPv4 Addresses Networks of Networks
bull 32-bit number in ldquodotted-quadrdquo notation
ndash wwwccgatechedu --- 130207736
10000010 11001111 00000111 00100100
Network (16 bits) Host (16 bits)
130 207 7 36
bull Problem 232 addresses is a lot of table entries
bull Solution Routing based on network and host
ndash 1302070016 is a 16-bit prefix with 216 IP addresses
Topological Addressing
15
Pre-1994 Classful Addressing
Network ID Host ID
8 16
Class A
32
0
Class B 10
Class C 110
Multicast AddressesClass D 1110
Reserved for experimentsClass E 1111
24
8 blocks (eg MIT has 180008)
16 blocks (eg Georgia Tech has 1302070016)
24 blocks (eg ATampT Labs has 19220225024)
Simple Forwarding Address range specifies network ID length
16
Classless Interdomain Routing (CIDR)
IP Address 65142480 ldquoMaskrdquo 2552552520
01000001 00001110 11111000 00000000
11111111 11111111 11111100 00000000
Use two 32-bit numbers to represent a network Network number = IP address + Mask
Example BellSouth Prefix 6514248022
Address no longer specifies network ID rangeNew forwarding trick Longest Prefix Match
17
Benefits of CIDR
bull Efficiency Can allocate blocks of prefixes on a finer granularity
bull Hierarchy Prefixes can be aggregated into supernets (Not always done Typically not in fact)
Customer 1
Customer 2
ATampT Internet
1220249024
1220231024120008
18
Growth of IP Prefixes
19
1994-1998 Linear Growth
bull About 10000 new entries per yearbull In theory less instability at the edges (why)
Source Geoff Huston
20
Around 2000 Fast Growth Resumes
Claim remaining 8s will be exhausted within the next 5-10 years
T Hain ldquoA Pragmatic Report on IPv4 Address Space Consumptionrdquo Cisco IPJ September 2005
21
Fast growth resumes
Rapid growth in routing tables
Dot-Bomb Hiccup
Significant contributor Multihoming
Source Geoff Huston
22
The Address Allocation Process
bull Allocation policies of RIRs affect pressure on IPv4 address space
IANA
AfriNIC APNIC ARIN LACNIC RIPE
httpwwwianaorgassignmentsipv4-address-space
Georgia Tech
23
Common Problems
bull Diagnosis and troubleshooting (hence measurement)bull Traffic engineeringbull Securitybull Design and capacity planning
24
What can go wrong
Two-thirds of the problems are caused by configuration of the routing protocol
Some downtime is very hard to protect againsthellip
25
Measurement and Monitoring
26
Passive vs Active Measurement
bull Passive Measurement Collection of packets flow statistics of traffic that is already flowing on the networkndash Packet tracesndash Flow statisticsndash Application-level logs
bull Active Measurement Inject ldquoprobingrdquo traffic to measure various characteristicsndash Traceroutendash Pingndash Application-level probes (eg Web downloads)
27
Billing for Internet Usage
bull 95th Percentile billingndash Customer network pays for ldquocommitted information
raterdquo (CIR)ndash Throughput measured every 5 minutes (typically with
SNMP flow statistics also can be used for billing)ndash Customer billed based on 95th percentile
28
Passive Traffic Data Measurement
bull SNMP bytepacket counts everywhere
bull Packet monitoring selected locations
bull Flow monitoring typically at edges (if possible)ndash Direct computation of the traffic matrixndash Input to denial-of-service attack detection
bull Deep Packet Inspection also at edge where possible
29
Simple Network Management Protocol
bull Management Information Base (MIB)ndash Information storendash Unique variables named by
OIDsndash Accessed with SNMP
bull Specific MIBs for bytepacket counts (per link)
Manager Agent
SNMP
DB
ManagedObjects
30
SNMP (Passive)
bull Advantage ubiquitousndash Supported on all networking equipmentndash Multiple products for polling and analyzing data
bull Disadvantages ndash Coarse granularityndash Cannot express complex queries on the datandash Unreliable delivery of the data using UDP
bull Utilityndash Link utilization (billing)ndash Traffic matrix inference
31
Packet-level Monitoring
bull Passive monitoring to collect full packet contents (or at least headers)
bull Advantages lots of detailed informationndash Precise timing informationndash Information in packet headers
bull Disadvantages overheadndash Hard to keep up with high-speed linksndash Often requires a separate monitoring device
32
Full Packet Capture (Passive)
Example Georgia Tech OC3Mon
bull Rack-mounted PCbull Optical splitterbull Data Acquisition and
Generation (DAG) card
Source endacecom
33
What is a flow
bull Source IP addressbull Destination IP addressbull Source portbull Destination portbull Layer 3 protocol typebull TOS byte (DSCP)bull Input logical interface (ifIndex)
34
Cisco NetFlow
bull Basic output ldquoFlow recordrdquondash Most common version is v5
bull Current version (9) is being standardized in the IETF (template-based)ndash More flexible record formatndash Much easier to add new flow record types
Core Network
Collection and Aggregation
Collector (PC)Approximately 1500 bytes
20-50 flow recordsSent more frequently if traffic increases
35
Flow Record Contents
bull Source and Destination IP address and portbull Packet and byte countsbull Start and end timesbull ToS TCP flags
Basic information about the flowhellip
hellipplus information related to routing
bull Next-hop IP addressbull Source and destination ASbull Source and destination prefix
36
flow 1 flow 2 flow 3 flow 4
Aggregating Packets into Flows
bull Criteria 1 Set of packets that ldquobelong togetherrdquondash Sourcedestination IP addresses and port numbersndash Same protocol ToS bits hellip ndash Same inputoutput interfaces at a router (if known)
bull Criteria 2 Packets that are ldquocloserdquo together in timendash Maximum inter-packet spacing (eg 15 sec 30 sec)ndash Example flows 2 and 4 are different flows due to time
37
Reducing Measurement Overhead
bull Filtering on interfacendash destination prefix for a customerndash port number for an application (eg 80 for Web)
bull Sampling before insertion into flow cachendash Random deterministic or hash-based samplingndash 1-out-of-n or stratified based on packetflow sizendash Two types packet-level and flow-level
bull Aggregation after cache evictionndash packetsflows with same next-hop ASndash packetsflows destined to a particular service
38
Packet Sampling for Flow Monitoring
bull Packet sampling before flow creation (Sampled Netflow)ndash 1-out-of-m sampling of individual packets (eg m=100)ndash Create of flow records over the sampled packets
bull Reducing overheadndash Avoid per-packet overhead on (m-1)m packetsndash Avoid creating records for a large number of small flows
bull Increasing overhead (in some cases)ndash May split some long transfers into multiple flow records ndash hellip due to larger time gaps between successive packets
time
not sampled
two flowstimeout
39
Sampling Flow-Level Sampling
bull Sampling of flow records evicted from flow cachendash When evicting flows from table or when analyzing flows
bull Stratified sampling to put weight on ldquoheavyrdquo flowsndash Select all long flows and sample the short flows
bull Reduces the number of flow records ndash Still measures the vast majority of the traffic
Flow 1 40 bytesFlow 2 15580 bytesFlow 3 8196 bytesFlow 4 5350789 bytesFlow 5 532 bytesFlow 6 7432 bytes
sample with 100 probability
sample with 01 probability
sample with 10 probability
40
Two Main Approaches
bull Packet-level Monitoringndash Keep packet-level statisticsndash Examine (and potentially log) variety of packet-level
statistics Essentially anything in the packetndash Timing
bull Flow-level Monitoringndash Monitor packet-by-packet (though sometimes
sampled)ndash Keep aggregate statistics on a flow
41
Packet Capture on High-Speed Links
Example Georgia Tech ldquoOC3Monrdquo
bull Rack-mounted PCbull Optical splitterbull Data Acquisition and
Generation (DAG) card
Source endacecom
42
Characteristics of Packet Capture
bull Allows inspection on every packet on 10G links
bull Disadvantagesndash Costlyndash Requires splitting optical fibersndash Must be able to filterstore data
43
Data Measurement Repositories
bull AbileneInternet 2 Observatoryndash Configuration examplesndash SNMP datandash ISIS BGP routing data NetFlow traffic data
bull RouteViewsndash BGP updatesndash BGP table snapshots
44
Multihoming and Traffic Engineering
45
What is Multihoming
bull The use of redundant network links for the purposes of external connectivity
bull Can be achieved at many layers of the protocol stack and many places in the networkndash Multiple network interfaces in a PC
ndash An ISP with multiple upstream interfaces
bull Can refer to having multiple connections tondash The same ISP
ndash Multiple ISPs
46
Why Multihome
bull Redundancybull Availabilitybull Performancebull Cost
Interdomain traffic engineering the process by which a multihomed network configures its
network to achieve these goals
47
Redundancy
bull Maintain connectivity in the face ofndash Physical connectivity problems (fiber cut device
failures etc)ndash Failures in upstream ISP
48
Performance
bull Use multiple network links at once to achieve higher throughput than just over a single link
bull Allows incoming traffic to be load-balanced
70 of traffic30 of traffic
49
Multihoming in IP Networks Today
bull Stub AS no transit service for other ASesndash No need to use BGP
bull Multi-homed stub AS has connectivity to multiple immediate upstream ISPsndash Need BGPndash No need for a public AS numberndash No need for IP prefix allocation
bull Multi-homed transit AS connectivity to multiple ASes and transit servicendash Need BGP public AS number IP prefix allocation
50
BGP or no
bull Advantages of static routingndash Cheapersmaller routers (less true nowadays)ndash Simpler to configure
bull Advantages of BGPndash More control of your destiny (have providers stop
announcing you)ndash Fastermore intelligent selection of where to send
outbound packetsndash Better debugging of net problems (you can see the
Internet topology now)
51
Same Provider or Multiple
bull If your provider is reliable and fast and affordably and offers good tech-support you may want to multi-home initially to them via some backup path (slow is better than dead)
bull Eventually yoursquoll want to multi-home to different providers to avoid failure modes due to one providerrsquos architecture decisions
52
Multihomed Stub One Link
bull Downstream ISPrsquos routers configure default (ldquostaticrdquo) routes pointing to border router
bull Upstream ISP advertises reachability
Upstream ISP
Multiple links between same pair of routers
Default routes to ldquoborderrdquo
ldquoStubrdquoISP
53
Multihomed Stub Multiple Links
bull Use BGP to share loadbull Use private AS number (why is this OK)bull As before upstream ISP advertises prefix
Upstream ISP
Multiple links to different upstream routers
ldquoStubrdquoISP
Internal routing for ldquohot potatordquo
BGP for load balance at edge
54
Multihomed Stub Multiple ISPs
bull Many possibilitiesndash Load sharingndash Primary-backupndash Selective use of different ISPs
bull Requires BGP public AS number etc
ldquoStubrdquoISP
Upstream
ISP 1
Upstream
ISP 2
55
Multihomed Transit Network
bull BGP everywherebull Incoming and outcoming trafficbull Challenge balancing load on intradomain and egress
links given an offered traffic load
TransitISP
ISP 1
ISP 2
ISP 3
56
Interdomain Traffic Engineering
bull The process by which a network operator configures the network to achievendash Traffic load balancendash Redundancy (primarybackup) etc
bull Two tasksndash Outbound traffic controlndash Inbound traffic control
bull Key Problems Predictability and Scalability
57
Outbound Traffic Control
bull Easier to control than inbound trafficndash Destination-based routing sender determines where
the packets go
bull Control over next-hop AS onlyndash Cannot control selection of the entire path
Provider 1 Provider 2
Control with local preference
58
Outbound Traffic Load Balancingbull Control routes to provider per-prefix
ndash Assign local preference across destination prefixesndash Change the local preference assignments over time
bull Useful inputs to load balancingndash End-to-end path performance datandash Outbound traffic statistics per destination prefix
bull Challenge Getting from traffic volumes to groups of prefixes that should be assigned to each link
Premise of ldquointelligent route controlrdquo preoducts
59
Traffic Engineering Goals
bull Predictabilityndash Ensure the BGP decision process is deterministicndash Assume that BGP updates are (relatively) stable
bull Limit overhead introduced by routing changesndash Minimize frequency of changes to routing policiesndash Limit number of prefixes affected by changes
bull Limit impact on how traffic enters the networkndash Avoid new routes that might change neighborrsquos mindndash Select route with same attributes or at least path length
60
Managing Scalebull Destination prefixes
ndash More than 90000 destination prefixes
bull Donrsquot want to have per-prefix routing policies
ndash Small fraction of prefixes contribute most of the traffic
bull Focus on the small number of heavy hitters
ndash Define routing policies for selected prefixes
bull Routing choicesndash About 27000 unique ldquorouting choicesrdquo
bull Help in reducing the scale of the problem
ndash Small fraction of ldquorouting choicesrdquo contribute most traffic
bull Focus on the very small number of ldquorouting choicesrdquo
ndash Define routing policies on common attributes
61
Achieving Predictability
bull Route prediction with static analysisndash Helpful to know effects before deploymentndash Static analysis can help
TopologyBGP policy
configuration
eBGP routes
Offered traffic
BGP routingmodel
Flow of traffic through the network
62
Challenges to Predictabilitybull For transit ISPs effects on incoming traffic
ndash Lack of coordination strikes again
63
ldquoHot Potatordquo routing
Inter-AS Negotiation
bull Coordination aids predictabilityndash Negotiate where to sendndash Inbound and outboundndash Mutual benefits
bull How to implementndash What info to exchangendash Protecting privacyndash How to prioritize choicesndash How to prevent cheating
Destination 2
Destination 1
multiplepeeringpoints
Provider A
Provider B
64
Outbound Multihoming Goals
bull Redundancyndash Dynamic routing will failover to backup link
bull Performancendash Select provider with best performance per prefix
ndash Requires active probing
bull Costndash Select provider per prefix over time to minimize the
total financial cost
65
Inbound Traffic Control
bull More difficult no control over neighborsrsquo decisions
bull Three common techniques (previously discussed)ndash AS path prependingndash Communities and local preferencendash Prefix splitting
How does todayrsquos paper (MONET) control inbound traffic
66
How many links are enough
K upstream ISPs
Not much benefit beyond 4 ISPs
Akella et al ldquoPerformance Benefits of Multihomingrdquo SIGCOMM 2003
67
Problems with Multihoming in IPv4
bull Routing table growthndash Provider-based addressingndash Advertising prefix out multiple ISPs ndash canrsquot aggregate
bull Poor control over inbound trafficndash Existing mechanisms do not allow hosts to control
inbound traffic
68
GeorgiaTech
Internet Routing Overview
bull Intradomain (ie ldquointra-ASrdquo) routingbull Interdomain routing
Comcast
Abilene
ATampT Cogent
Autonomous Systems (ASes)
69
Configuration Problems ldquoAS 7007rdquoldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
UUNet
Florida InternetBarn
Sprint
70
Diagnosis and Troubleshootingldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
ldquoMicrosofts websites were offline for up to 23 hoursbecause of a [router] misconfigurationhellipit took nearly a day to determine what was wrong and undo the changesrdquo -- wiredcom January 25 2001
ldquoWorldCom Inchellipsuffered a widespread outage on its Internet backbone that affected roughly 20 percent of its US customer base The network problemshellipaffected millions of computer users worldwide A spokeswoman attributed the outage to a route table issue -- cnncom October 3 2002
A number of Covad customers went out from 5pm today due to supposedly a DDOS (distributed denial of service attack) on a key Level3 data center which later was described as a route leak (misconfiguration)ldquo
-- dslreportscom February 23 2004
71
Operator Mailing List (NANOG)Date Mon 18 Oct 2004 091515 -0700Subject Level 3 US east coast issues
Level 3 experiencing widespread unspecified routing issues on the US east coast Master ticket 1086844 Anyone have more specific information
Date Mon 18 Oct 2004 122034 -0400 (EDT)Subject Re Level 3 US east coast issues
Level 3 is currently experiencing a backbone outage causing routing instability and packet loss We are working to restore and will be sending hourly updateshellip
72
Operator Mailing List
0102030405060708090
Filtering RouteLeaks
RouteHijacks
RouteInstability
RoutingLoops
Blackholes
Occurr
ences o
ver
10 Y
ears
1995-1997 1998-2001 2001-2004
Note Only includes problems openly discussed on this list
Compare 83 power outages 1 fire
73
Routing Configuration
Ranking route selection
Dissemination internal route advertisement
Filtering route advertisement
Customer
Competitor
Primary
Backup
74
Internet Business Model (Simplified)
bull CustomerProvider One AS pays another for reachability to some set of destinations
bull ldquoSettlement-freerdquo Peering Bartering Two ASes exchange routes with one another
Provider
Peer
Customer
Preferences implemented with local preference manipulation
Destination
Pay to use
Get paid to use
Free to use
75
Peering Contracts Consistent Export
bull Rules of settlement-free peeringndash Advertise routes at all peering pointsndash Advertised routes must have equal ldquoAS path lengthrdquo
Sprint
ATampT
Enables ldquohot potatordquo routing
ldquoequally goodrdquoroutes
76
Consistent Export
bull Malicedeceptionbull iBGP signaling partitionbull Inconsistent export policy
neighbor 10123route-map PEER permit 10 set prepend 123
neighbor 10456route-map PEER permit 10 set prepend 123 123
Possible Causes
10123 456 1
10456 456 2
Neighbor AS Export
1 1 123
Export Clause Prepend
2 1 123 123
Two different Export Policies
77
Inconsistent Export in Practice
Feamster et al ldquoBorderGuard Detecting Cold Potatoes from Peersrdquo ACM IMC October 2004
78
Blackholes
Date Thu 18 Jul 2002 060510 -0400 (EDT)From Chad Oleary ltcolpoboxcomgtSubject Re problems with 701To ltnanogmeritedugt
Were starting to see the same issues with UUNet again Anyone elseseeing this Trying to reach Qwest
traceroute to 631461901 (631461901) 30 hops max 38 byte packets 1 esc-lp2-gwe-solutionscorpcom (631182201) 1167 ms 1163 ms 1142 ms 2 500Serial2-10GW1TPA2ALTERNET (1571301499) 1097 ms 1059 ms 1044 ms 3 161at-1-0-0XL4ATL1ALTERNET (1526381190) 13839 ms 14108 ms 16638 ms 4 0so-3-1-0XL2ATL5ALTERNET (152630238) 14370 ms 14587 ms 14553 ms 5 POS7-0BR2ATL5ALTERNET (1526382193) 13928 ms 14099 ms 14053 ms 6 7 hellip
79
Security
80
Security ldquoBogonrdquo Routes
Feamster et al ldquoAn Empirical Study of lsquoBogonrsquo Route Advertisementsrdquo ACM CCR January 2005
81
Spam Phishing etc
bull Unsolicited commercial emailbull As of about August 2008 estimates indicate that
about 95 of all email is spambull Common spam filtering techniques
ndash Content-based filtersndash DNS Blacklist (DNSBL) lookups Significant fraction of
todayrsquos DNS traffic
Can IP addresses from which spam is received be spoofed
82
Spam and Routing
83
Worms and Botnets
84
What is a Worm
bull Code that replicates and propagates across the networkndash Often carries a ldquopayloadrdquo
bull Usually spread via exploiting flaws in open servicesndash ldquoVirusesrdquo require user action to spread
bull First worm Robert Morris November 1988ndash 6-10 of all Internet hosts infected ()
bull Many more since but none on that scale until July 2001
85
Example Worm Code Red
bull Initial version July 13 2001
bull Exploited known ISAPI vulnerability in Microsoft IIS Web servers
bull 1st through 20th of each month spread20th through end of each month attack
bull Payload Web site defacementbull Scanning Random IP addressesbull Bug failure to seed random number generator
86
Code Red Revisions
bull Released July 19 2001
bull Payload flooding attack on wwwwhitehousegovndash Attack was mounted at the IP address of the Web site
bull Bug died after 20th of each month
bull Random number generator for IP scanning fixed
87
Code Red Host Infection Rate
Exponential infection rate
Measured using backscatter technique
88
Designing Fast-Spreading Worms
bull Hit-list scanningndash Time to infect first 10k hosts dominates infection timendash Solution Reconnaissance (stealthy scans etc)
bull Permutation scanningndash Observation Most scanning is redundantndash Idea Shared permutation of address space Start scanning
from own IP address Re-randomize when another infected machine is found
bull Internet-scale hit listsndash Flash worm complete infection within 30 seconds
89
Botnets
bull Bots Autonomous programs performing tasksbull Plenty of ldquobenignrdquo bots
ndash eg weatherbug
bull Botnets group of bots ndash Typically carries malicious connotationndash Large numbers of infected machinesndash Machines ldquoenlistedrdquo with infection vectors like worms
(last lecture)
bull Available for simultaneous control by a masterbull Size up to 350000 nodes (from todayrsquos paper)
90
ldquoRallyingrdquo the Botnet
bull Easy to combine worm backdoor functionalitybull Problem how to learn about successfully
infected machines
bull Optionsndash Emailndash Hard-coded email address
91
Botnet Control
bull Botnet master typically runs some IRC server on a well-known port (eg 6667)
bull Infected machine contacts botnet with pre-programmed DNS name (eg big-botde)
bull Dynamic DNS allows controller to move about freely
Infected Machine
DynamicDNS
BotnetController
(IRC server)
92
Some Defenses
93
Idea 1 Ingress Filtering
bull RFC 2827 Routers install filters to drop packets from networks that are not downstream
bull Feasible at edgesbull Difficult to configure closer to network ldquocorerdquo
20469207024 Internet
Drop all packets with source address other than 20469207024
94
Idea 2 uRPF Checks
bull Unicast Reverse Path Forwardingndash Cisco ldquoip verify unicast reverse-pathrdquo
bull Requires symmetric routing
Accept packet from interface only if forwarding table entry for source IP address matches ingress interface
100183
A10018
10015
101203
100181 24
10011 24
Strict Mode uRPF
Enabled
ldquoArdquo Routing TableDestination Next Hop1001024 Int 110018024 Int 2
100183 from wrong interface
95
Problems with uRPF
bull Asymmetric routing
96
S-BGP
bull Address-based PKI validate signaturesndash Authentication of
bull ownership for IP address blocks bull AS number bull an ASs identity and bull a BGP routers identity
ndash Use existing infrastructure (Internet registries etc)ndash Routing origination is digitally signedndash BGP updates are digitally signed
1048708 bull Route attestations A new optional BGP transitive path attribute
ndash carries digital signatures covering the routing information in updates
97
Practical Problems with S-BGP
bull Requires Public-Key Infrastructure
bull Lots of digital signatures to calculate and verifyndash Message overheadndash CPU overhead
bull Calculation expense is greatest when topology is changingndash Caching can help
bull Route aggregation is problematic (maybe thatrsquos OK)
bull Secure route withdrawals when link or node fails
bull Address ownership data out of date
bull Deployment
- Networking
- Goal of This Tutorial
- Todayrsquos Networks
- Business Models
- Billing for Internet Usage
- Net Neutrality
- Network Operations
- Point-of-Presence (PoP)
- Example Abilene Network Topology
- Another Example Backbone
- Internet Routing Overview
- Internet Routing Protocol BGP
- Two Flavors of BGP
- IPv4 Addresses Networks of Networks
- Pre-1994 Classful Addressing
- Classless Interdomain Routing (CIDR)
- Benefits of CIDR
- Growth of IP Prefixes
- 1994-1998 Linear Growth
- Around 2000 Fast Growth Resumes
- Fast growth resumes
- The Address Allocation Process
- Common Problems
- What can go wrong
- Measurement and Monitoring
- Passive vs Active Measurement
- Slide 27
- Passive Traffic Data Measurement
- Simple Network Management Protocol
- SNMP (Passive)
- Packet-level Monitoring
- Full Packet Capture (Passive)
- What is a flow
- Cisco NetFlow
- Flow Record Contents
- Aggregating Packets into Flows
- Reducing Measurement Overhead
- Packet Sampling for Flow Monitoring
- Sampling Flow-Level Sampling
- Two Main Approaches
- Packet Capture on High-Speed Links
- Characteristics of Packet Capture
- Data Measurement Repositories
- Multihoming and Traffic Engineering
- What is Multihoming
- Why Multihome
- Redundancy
- Performance
- Multihoming in IP Networks Today
- BGP or no
- Same Provider or Multiple
- Multihomed Stub One Link
- Multihomed Stub Multiple Links
- Multihomed Stub Multiple ISPs
- Multihomed Transit Network
- Interdomain Traffic Engineering
- Outbound Traffic Control
- Outbound Traffic Load Balancing
- Traffic Engineering Goals
- Managing Scale
- Achieving Predictability
- Challenges to Predictability
- Inter-AS Negotiation
- Outbound Multihoming Goals
- Inbound Traffic Control
- How many links are enough
- Problems with Multihoming in IPv4
- Slide 68
- Slide 69
- Diagnosis and Troubleshooting
- Operator Mailing List (NANOG)
- Operator Mailing List
- Routing Configuration
- Internet Business Model (Simplified)
- Peering Contracts Consistent Export
- Consistent Export
- Inconsistent Export in Practice
- Blackholes
- Security
- Security ldquoBogonrdquo Routes
- Spam Phishing etc
- Spam and Routing
- Worms and Botnets
- What is a Worm
- Example Worm Code Red
- Code Red Revisions
- Code Red Host Infection Rate
- Designing Fast-Spreading Worms
- Botnets
- ldquoRallyingrdquo the Botnet
- Botnet Control
- Some Defenses
- Idea 1 Ingress Filtering
- Idea 2 uRPF Checks
- Problems with uRPF
- S-BGP
- Practical Problems with S-BGP
-
8
Point-of-Presence (PoP)
bull A ldquoclusterrdquo of routers in a single physical location
bull Inter-PoP linksndash Long distances
ndash High bandwidth
bull Intra-PoP linksndash Cables between racks or floors
ndash Aggregated bandwidth
PoP
9
Example Abilene Network Topology
10
Another Example Backbone
11
GeorgiaTech
Internet Routing Overview
bull Intradomain (ie ldquointra-ASrdquo) routingbull Interdomain routing
Comcast
Abilene
ATampT Cogent
Autonomous Systems (ASes)
12
Internet Routing Protocol BGP
Route Advertisement
Autonomous Systems (ASes)
Session
Traffic
Destination Next-hop AS Path1302070016
1302070016
19258989
6625025244
105782637
174hellip 2637
13
Two Flavors of BGP
bull External BGP (eBGP) exchanging routes between ASes
bull Internal BGP (iBGP) disseminating routes to external destinations among the routers within an AS
eBGPiBGP
Question Whatrsquos the difference between IGP and iBGP
14
IPv4 Addresses Networks of Networks
bull 32-bit number in ldquodotted-quadrdquo notation
ndash wwwccgatechedu --- 130207736
10000010 11001111 00000111 00100100
Network (16 bits) Host (16 bits)
130 207 7 36
bull Problem 232 addresses is a lot of table entries
bull Solution Routing based on network and host
ndash 1302070016 is a 16-bit prefix with 216 IP addresses
Topological Addressing
15
Pre-1994 Classful Addressing
Network ID Host ID
8 16
Class A
32
0
Class B 10
Class C 110
Multicast AddressesClass D 1110
Reserved for experimentsClass E 1111
24
8 blocks (eg MIT has 180008)
16 blocks (eg Georgia Tech has 1302070016)
24 blocks (eg ATampT Labs has 19220225024)
Simple Forwarding Address range specifies network ID length
16
Classless Interdomain Routing (CIDR)
IP Address 65142480 ldquoMaskrdquo 2552552520
01000001 00001110 11111000 00000000
11111111 11111111 11111100 00000000
Use two 32-bit numbers to represent a network Network number = IP address + Mask
Example BellSouth Prefix 6514248022
Address no longer specifies network ID rangeNew forwarding trick Longest Prefix Match
17
Benefits of CIDR
bull Efficiency Can allocate blocks of prefixes on a finer granularity
bull Hierarchy Prefixes can be aggregated into supernets (Not always done Typically not in fact)
Customer 1
Customer 2
ATampT Internet
1220249024
1220231024120008
18
Growth of IP Prefixes
19
1994-1998 Linear Growth
bull About 10000 new entries per yearbull In theory less instability at the edges (why)
Source Geoff Huston
20
Around 2000 Fast Growth Resumes
Claim remaining 8s will be exhausted within the next 5-10 years
T Hain ldquoA Pragmatic Report on IPv4 Address Space Consumptionrdquo Cisco IPJ September 2005
21
Fast growth resumes
Rapid growth in routing tables
Dot-Bomb Hiccup
Significant contributor Multihoming
Source Geoff Huston
22
The Address Allocation Process
bull Allocation policies of RIRs affect pressure on IPv4 address space
IANA
AfriNIC APNIC ARIN LACNIC RIPE
httpwwwianaorgassignmentsipv4-address-space
Georgia Tech
23
Common Problems
bull Diagnosis and troubleshooting (hence measurement)bull Traffic engineeringbull Securitybull Design and capacity planning
24
What can go wrong
Two-thirds of the problems are caused by configuration of the routing protocol
Some downtime is very hard to protect againsthellip
25
Measurement and Monitoring
26
Passive vs Active Measurement
bull Passive Measurement Collection of packets flow statistics of traffic that is already flowing on the networkndash Packet tracesndash Flow statisticsndash Application-level logs
bull Active Measurement Inject ldquoprobingrdquo traffic to measure various characteristicsndash Traceroutendash Pingndash Application-level probes (eg Web downloads)
27
Billing for Internet Usage
bull 95th Percentile billingndash Customer network pays for ldquocommitted information
raterdquo (CIR)ndash Throughput measured every 5 minutes (typically with
SNMP flow statistics also can be used for billing)ndash Customer billed based on 95th percentile
28
Passive Traffic Data Measurement
bull SNMP bytepacket counts everywhere
bull Packet monitoring selected locations
bull Flow monitoring typically at edges (if possible)ndash Direct computation of the traffic matrixndash Input to denial-of-service attack detection
bull Deep Packet Inspection also at edge where possible
29
Simple Network Management Protocol
bull Management Information Base (MIB)ndash Information storendash Unique variables named by
OIDsndash Accessed with SNMP
bull Specific MIBs for bytepacket counts (per link)
Manager Agent
SNMP
DB
ManagedObjects
30
SNMP (Passive)
bull Advantage ubiquitousndash Supported on all networking equipmentndash Multiple products for polling and analyzing data
bull Disadvantages ndash Coarse granularityndash Cannot express complex queries on the datandash Unreliable delivery of the data using UDP
bull Utilityndash Link utilization (billing)ndash Traffic matrix inference
31
Packet-level Monitoring
bull Passive monitoring to collect full packet contents (or at least headers)
bull Advantages lots of detailed informationndash Precise timing informationndash Information in packet headers
bull Disadvantages overheadndash Hard to keep up with high-speed linksndash Often requires a separate monitoring device
32
Full Packet Capture (Passive)
Example Georgia Tech OC3Mon
bull Rack-mounted PCbull Optical splitterbull Data Acquisition and
Generation (DAG) card
Source endacecom
33
What is a flow
bull Source IP addressbull Destination IP addressbull Source portbull Destination portbull Layer 3 protocol typebull TOS byte (DSCP)bull Input logical interface (ifIndex)
34
Cisco NetFlow
bull Basic output ldquoFlow recordrdquondash Most common version is v5
bull Current version (9) is being standardized in the IETF (template-based)ndash More flexible record formatndash Much easier to add new flow record types
Core Network
Collection and Aggregation
Collector (PC)Approximately 1500 bytes
20-50 flow recordsSent more frequently if traffic increases
35
Flow Record Contents
bull Source and Destination IP address and portbull Packet and byte countsbull Start and end timesbull ToS TCP flags
Basic information about the flowhellip
hellipplus information related to routing
bull Next-hop IP addressbull Source and destination ASbull Source and destination prefix
36
flow 1 flow 2 flow 3 flow 4
Aggregating Packets into Flows
bull Criteria 1 Set of packets that ldquobelong togetherrdquondash Sourcedestination IP addresses and port numbersndash Same protocol ToS bits hellip ndash Same inputoutput interfaces at a router (if known)
bull Criteria 2 Packets that are ldquocloserdquo together in timendash Maximum inter-packet spacing (eg 15 sec 30 sec)ndash Example flows 2 and 4 are different flows due to time
37
Reducing Measurement Overhead
bull Filtering on interfacendash destination prefix for a customerndash port number for an application (eg 80 for Web)
bull Sampling before insertion into flow cachendash Random deterministic or hash-based samplingndash 1-out-of-n or stratified based on packetflow sizendash Two types packet-level and flow-level
bull Aggregation after cache evictionndash packetsflows with same next-hop ASndash packetsflows destined to a particular service
38
Packet Sampling for Flow Monitoring
bull Packet sampling before flow creation (Sampled Netflow)ndash 1-out-of-m sampling of individual packets (eg m=100)ndash Create of flow records over the sampled packets
bull Reducing overheadndash Avoid per-packet overhead on (m-1)m packetsndash Avoid creating records for a large number of small flows
bull Increasing overhead (in some cases)ndash May split some long transfers into multiple flow records ndash hellip due to larger time gaps between successive packets
time
not sampled
two flowstimeout
39
Sampling Flow-Level Sampling
bull Sampling of flow records evicted from flow cachendash When evicting flows from table or when analyzing flows
bull Stratified sampling to put weight on ldquoheavyrdquo flowsndash Select all long flows and sample the short flows
bull Reduces the number of flow records ndash Still measures the vast majority of the traffic
Flow 1 40 bytesFlow 2 15580 bytesFlow 3 8196 bytesFlow 4 5350789 bytesFlow 5 532 bytesFlow 6 7432 bytes
sample with 100 probability
sample with 01 probability
sample with 10 probability
40
Two Main Approaches
bull Packet-level Monitoringndash Keep packet-level statisticsndash Examine (and potentially log) variety of packet-level
statistics Essentially anything in the packetndash Timing
bull Flow-level Monitoringndash Monitor packet-by-packet (though sometimes
sampled)ndash Keep aggregate statistics on a flow
41
Packet Capture on High-Speed Links
Example Georgia Tech ldquoOC3Monrdquo
bull Rack-mounted PCbull Optical splitterbull Data Acquisition and
Generation (DAG) card
Source endacecom
42
Characteristics of Packet Capture
bull Allows inspection on every packet on 10G links
bull Disadvantagesndash Costlyndash Requires splitting optical fibersndash Must be able to filterstore data
43
Data Measurement Repositories
bull AbileneInternet 2 Observatoryndash Configuration examplesndash SNMP datandash ISIS BGP routing data NetFlow traffic data
bull RouteViewsndash BGP updatesndash BGP table snapshots
44
Multihoming and Traffic Engineering
45
What is Multihoming
bull The use of redundant network links for the purposes of external connectivity
bull Can be achieved at many layers of the protocol stack and many places in the networkndash Multiple network interfaces in a PC
ndash An ISP with multiple upstream interfaces
bull Can refer to having multiple connections tondash The same ISP
ndash Multiple ISPs
46
Why Multihome
bull Redundancybull Availabilitybull Performancebull Cost
Interdomain traffic engineering the process by which a multihomed network configures its
network to achieve these goals
47
Redundancy
bull Maintain connectivity in the face ofndash Physical connectivity problems (fiber cut device
failures etc)ndash Failures in upstream ISP
48
Performance
bull Use multiple network links at once to achieve higher throughput than just over a single link
bull Allows incoming traffic to be load-balanced
70 of traffic30 of traffic
49
Multihoming in IP Networks Today
bull Stub AS no transit service for other ASesndash No need to use BGP
bull Multi-homed stub AS has connectivity to multiple immediate upstream ISPsndash Need BGPndash No need for a public AS numberndash No need for IP prefix allocation
bull Multi-homed transit AS connectivity to multiple ASes and transit servicendash Need BGP public AS number IP prefix allocation
50
BGP or no
bull Advantages of static routingndash Cheapersmaller routers (less true nowadays)ndash Simpler to configure
bull Advantages of BGPndash More control of your destiny (have providers stop
announcing you)ndash Fastermore intelligent selection of where to send
outbound packetsndash Better debugging of net problems (you can see the
Internet topology now)
51
Same Provider or Multiple
bull If your provider is reliable and fast and affordably and offers good tech-support you may want to multi-home initially to them via some backup path (slow is better than dead)
bull Eventually yoursquoll want to multi-home to different providers to avoid failure modes due to one providerrsquos architecture decisions
52
Multihomed Stub One Link
bull Downstream ISPrsquos routers configure default (ldquostaticrdquo) routes pointing to border router
bull Upstream ISP advertises reachability
Upstream ISP
Multiple links between same pair of routers
Default routes to ldquoborderrdquo
ldquoStubrdquoISP
53
Multihomed Stub Multiple Links
bull Use BGP to share loadbull Use private AS number (why is this OK)bull As before upstream ISP advertises prefix
Upstream ISP
Multiple links to different upstream routers
ldquoStubrdquoISP
Internal routing for ldquohot potatordquo
BGP for load balance at edge
54
Multihomed Stub Multiple ISPs
bull Many possibilitiesndash Load sharingndash Primary-backupndash Selective use of different ISPs
bull Requires BGP public AS number etc
ldquoStubrdquoISP
Upstream
ISP 1
Upstream
ISP 2
55
Multihomed Transit Network
bull BGP everywherebull Incoming and outcoming trafficbull Challenge balancing load on intradomain and egress
links given an offered traffic load
TransitISP
ISP 1
ISP 2
ISP 3
56
Interdomain Traffic Engineering
bull The process by which a network operator configures the network to achievendash Traffic load balancendash Redundancy (primarybackup) etc
bull Two tasksndash Outbound traffic controlndash Inbound traffic control
bull Key Problems Predictability and Scalability
57
Outbound Traffic Control
bull Easier to control than inbound trafficndash Destination-based routing sender determines where
the packets go
bull Control over next-hop AS onlyndash Cannot control selection of the entire path
Provider 1 Provider 2
Control with local preference
58
Outbound Traffic Load Balancingbull Control routes to provider per-prefix
ndash Assign local preference across destination prefixesndash Change the local preference assignments over time
bull Useful inputs to load balancingndash End-to-end path performance datandash Outbound traffic statistics per destination prefix
bull Challenge Getting from traffic volumes to groups of prefixes that should be assigned to each link
Premise of ldquointelligent route controlrdquo preoducts
59
Traffic Engineering Goals
bull Predictabilityndash Ensure the BGP decision process is deterministicndash Assume that BGP updates are (relatively) stable
bull Limit overhead introduced by routing changesndash Minimize frequency of changes to routing policiesndash Limit number of prefixes affected by changes
bull Limit impact on how traffic enters the networkndash Avoid new routes that might change neighborrsquos mindndash Select route with same attributes or at least path length
60
Managing Scalebull Destination prefixes
ndash More than 90000 destination prefixes
bull Donrsquot want to have per-prefix routing policies
ndash Small fraction of prefixes contribute most of the traffic
bull Focus on the small number of heavy hitters
ndash Define routing policies for selected prefixes
bull Routing choicesndash About 27000 unique ldquorouting choicesrdquo
bull Help in reducing the scale of the problem
ndash Small fraction of ldquorouting choicesrdquo contribute most traffic
bull Focus on the very small number of ldquorouting choicesrdquo
ndash Define routing policies on common attributes
61
Achieving Predictability
bull Route prediction with static analysisndash Helpful to know effects before deploymentndash Static analysis can help
TopologyBGP policy
configuration
eBGP routes
Offered traffic
BGP routingmodel
Flow of traffic through the network
62
Challenges to Predictabilitybull For transit ISPs effects on incoming traffic
ndash Lack of coordination strikes again
63
ldquoHot Potatordquo routing
Inter-AS Negotiation
bull Coordination aids predictabilityndash Negotiate where to sendndash Inbound and outboundndash Mutual benefits
bull How to implementndash What info to exchangendash Protecting privacyndash How to prioritize choicesndash How to prevent cheating
Destination 2
Destination 1
multiplepeeringpoints
Provider A
Provider B
64
Outbound Multihoming Goals
bull Redundancyndash Dynamic routing will failover to backup link
bull Performancendash Select provider with best performance per prefix
ndash Requires active probing
bull Costndash Select provider per prefix over time to minimize the
total financial cost
65
Inbound Traffic Control
bull More difficult no control over neighborsrsquo decisions
bull Three common techniques (previously discussed)ndash AS path prependingndash Communities and local preferencendash Prefix splitting
How does todayrsquos paper (MONET) control inbound traffic
66
How many links are enough
K upstream ISPs
Not much benefit beyond 4 ISPs
Akella et al ldquoPerformance Benefits of Multihomingrdquo SIGCOMM 2003
67
Problems with Multihoming in IPv4
bull Routing table growthndash Provider-based addressingndash Advertising prefix out multiple ISPs ndash canrsquot aggregate
bull Poor control over inbound trafficndash Existing mechanisms do not allow hosts to control
inbound traffic
68
GeorgiaTech
Internet Routing Overview
bull Intradomain (ie ldquointra-ASrdquo) routingbull Interdomain routing
Comcast
Abilene
ATampT Cogent
Autonomous Systems (ASes)
69
Configuration Problems ldquoAS 7007rdquoldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
UUNet
Florida InternetBarn
Sprint
70
Diagnosis and Troubleshootingldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
ldquoMicrosofts websites were offline for up to 23 hoursbecause of a [router] misconfigurationhellipit took nearly a day to determine what was wrong and undo the changesrdquo -- wiredcom January 25 2001
ldquoWorldCom Inchellipsuffered a widespread outage on its Internet backbone that affected roughly 20 percent of its US customer base The network problemshellipaffected millions of computer users worldwide A spokeswoman attributed the outage to a route table issue -- cnncom October 3 2002
A number of Covad customers went out from 5pm today due to supposedly a DDOS (distributed denial of service attack) on a key Level3 data center which later was described as a route leak (misconfiguration)ldquo
-- dslreportscom February 23 2004
71
Operator Mailing List (NANOG)Date Mon 18 Oct 2004 091515 -0700Subject Level 3 US east coast issues
Level 3 experiencing widespread unspecified routing issues on the US east coast Master ticket 1086844 Anyone have more specific information
Date Mon 18 Oct 2004 122034 -0400 (EDT)Subject Re Level 3 US east coast issues
Level 3 is currently experiencing a backbone outage causing routing instability and packet loss We are working to restore and will be sending hourly updateshellip
72
Operator Mailing List
0102030405060708090
Filtering RouteLeaks
RouteHijacks
RouteInstability
RoutingLoops
Blackholes
Occurr
ences o
ver
10 Y
ears
1995-1997 1998-2001 2001-2004
Note Only includes problems openly discussed on this list
Compare 83 power outages 1 fire
73
Routing Configuration
Ranking route selection
Dissemination internal route advertisement
Filtering route advertisement
Customer
Competitor
Primary
Backup
74
Internet Business Model (Simplified)
bull CustomerProvider One AS pays another for reachability to some set of destinations
bull ldquoSettlement-freerdquo Peering Bartering Two ASes exchange routes with one another
Provider
Peer
Customer
Preferences implemented with local preference manipulation
Destination
Pay to use
Get paid to use
Free to use
75
Peering Contracts Consistent Export
bull Rules of settlement-free peeringndash Advertise routes at all peering pointsndash Advertised routes must have equal ldquoAS path lengthrdquo
Sprint
ATampT
Enables ldquohot potatordquo routing
ldquoequally goodrdquoroutes
76
Consistent Export
bull Malicedeceptionbull iBGP signaling partitionbull Inconsistent export policy
neighbor 10123route-map PEER permit 10 set prepend 123
neighbor 10456route-map PEER permit 10 set prepend 123 123
Possible Causes
10123 456 1
10456 456 2
Neighbor AS Export
1 1 123
Export Clause Prepend
2 1 123 123
Two different Export Policies
77
Inconsistent Export in Practice
Feamster et al ldquoBorderGuard Detecting Cold Potatoes from Peersrdquo ACM IMC October 2004
78
Blackholes
Date Thu 18 Jul 2002 060510 -0400 (EDT)From Chad Oleary ltcolpoboxcomgtSubject Re problems with 701To ltnanogmeritedugt
Were starting to see the same issues with UUNet again Anyone elseseeing this Trying to reach Qwest
traceroute to 631461901 (631461901) 30 hops max 38 byte packets 1 esc-lp2-gwe-solutionscorpcom (631182201) 1167 ms 1163 ms 1142 ms 2 500Serial2-10GW1TPA2ALTERNET (1571301499) 1097 ms 1059 ms 1044 ms 3 161at-1-0-0XL4ATL1ALTERNET (1526381190) 13839 ms 14108 ms 16638 ms 4 0so-3-1-0XL2ATL5ALTERNET (152630238) 14370 ms 14587 ms 14553 ms 5 POS7-0BR2ATL5ALTERNET (1526382193) 13928 ms 14099 ms 14053 ms 6 7 hellip
79
Security
80
Security ldquoBogonrdquo Routes
Feamster et al ldquoAn Empirical Study of lsquoBogonrsquo Route Advertisementsrdquo ACM CCR January 2005
81
Spam Phishing etc
bull Unsolicited commercial emailbull As of about August 2008 estimates indicate that
about 95 of all email is spambull Common spam filtering techniques
ndash Content-based filtersndash DNS Blacklist (DNSBL) lookups Significant fraction of
todayrsquos DNS traffic
Can IP addresses from which spam is received be spoofed
82
Spam and Routing
83
Worms and Botnets
84
What is a Worm
bull Code that replicates and propagates across the networkndash Often carries a ldquopayloadrdquo
bull Usually spread via exploiting flaws in open servicesndash ldquoVirusesrdquo require user action to spread
bull First worm Robert Morris November 1988ndash 6-10 of all Internet hosts infected ()
bull Many more since but none on that scale until July 2001
85
Example Worm Code Red
bull Initial version July 13 2001
bull Exploited known ISAPI vulnerability in Microsoft IIS Web servers
bull 1st through 20th of each month spread20th through end of each month attack
bull Payload Web site defacementbull Scanning Random IP addressesbull Bug failure to seed random number generator
86
Code Red Revisions
bull Released July 19 2001
bull Payload flooding attack on wwwwhitehousegovndash Attack was mounted at the IP address of the Web site
bull Bug died after 20th of each month
bull Random number generator for IP scanning fixed
87
Code Red Host Infection Rate
Exponential infection rate
Measured using backscatter technique
88
Designing Fast-Spreading Worms
bull Hit-list scanningndash Time to infect first 10k hosts dominates infection timendash Solution Reconnaissance (stealthy scans etc)
bull Permutation scanningndash Observation Most scanning is redundantndash Idea Shared permutation of address space Start scanning
from own IP address Re-randomize when another infected machine is found
bull Internet-scale hit listsndash Flash worm complete infection within 30 seconds
89
Botnets
bull Bots Autonomous programs performing tasksbull Plenty of ldquobenignrdquo bots
ndash eg weatherbug
bull Botnets group of bots ndash Typically carries malicious connotationndash Large numbers of infected machinesndash Machines ldquoenlistedrdquo with infection vectors like worms
(last lecture)
bull Available for simultaneous control by a masterbull Size up to 350000 nodes (from todayrsquos paper)
90
ldquoRallyingrdquo the Botnet
bull Easy to combine worm backdoor functionalitybull Problem how to learn about successfully
infected machines
bull Optionsndash Emailndash Hard-coded email address
91
Botnet Control
bull Botnet master typically runs some IRC server on a well-known port (eg 6667)
bull Infected machine contacts botnet with pre-programmed DNS name (eg big-botde)
bull Dynamic DNS allows controller to move about freely
Infected Machine
DynamicDNS
BotnetController
(IRC server)
92
Some Defenses
93
Idea 1 Ingress Filtering
bull RFC 2827 Routers install filters to drop packets from networks that are not downstream
bull Feasible at edgesbull Difficult to configure closer to network ldquocorerdquo
20469207024 Internet
Drop all packets with source address other than 20469207024
94
Idea 2 uRPF Checks
bull Unicast Reverse Path Forwardingndash Cisco ldquoip verify unicast reverse-pathrdquo
bull Requires symmetric routing
Accept packet from interface only if forwarding table entry for source IP address matches ingress interface
100183
A10018
10015
101203
100181 24
10011 24
Strict Mode uRPF
Enabled
ldquoArdquo Routing TableDestination Next Hop1001024 Int 110018024 Int 2
100183 from wrong interface
95
Problems with uRPF
bull Asymmetric routing
96
S-BGP
bull Address-based PKI validate signaturesndash Authentication of
bull ownership for IP address blocks bull AS number bull an ASs identity and bull a BGP routers identity
ndash Use existing infrastructure (Internet registries etc)ndash Routing origination is digitally signedndash BGP updates are digitally signed
1048708 bull Route attestations A new optional BGP transitive path attribute
ndash carries digital signatures covering the routing information in updates
97
Practical Problems with S-BGP
bull Requires Public-Key Infrastructure
bull Lots of digital signatures to calculate and verifyndash Message overheadndash CPU overhead
bull Calculation expense is greatest when topology is changingndash Caching can help
bull Route aggregation is problematic (maybe thatrsquos OK)
bull Secure route withdrawals when link or node fails
bull Address ownership data out of date
bull Deployment
- Networking
- Goal of This Tutorial
- Todayrsquos Networks
- Business Models
- Billing for Internet Usage
- Net Neutrality
- Network Operations
- Point-of-Presence (PoP)
- Example Abilene Network Topology
- Another Example Backbone
- Internet Routing Overview
- Internet Routing Protocol BGP
- Two Flavors of BGP
- IPv4 Addresses Networks of Networks
- Pre-1994 Classful Addressing
- Classless Interdomain Routing (CIDR)
- Benefits of CIDR
- Growth of IP Prefixes
- 1994-1998 Linear Growth
- Around 2000 Fast Growth Resumes
- Fast growth resumes
- The Address Allocation Process
- Common Problems
- What can go wrong
- Measurement and Monitoring
- Passive vs Active Measurement
- Slide 27
- Passive Traffic Data Measurement
- Simple Network Management Protocol
- SNMP (Passive)
- Packet-level Monitoring
- Full Packet Capture (Passive)
- What is a flow
- Cisco NetFlow
- Flow Record Contents
- Aggregating Packets into Flows
- Reducing Measurement Overhead
- Packet Sampling for Flow Monitoring
- Sampling Flow-Level Sampling
- Two Main Approaches
- Packet Capture on High-Speed Links
- Characteristics of Packet Capture
- Data Measurement Repositories
- Multihoming and Traffic Engineering
- What is Multihoming
- Why Multihome
- Redundancy
- Performance
- Multihoming in IP Networks Today
- BGP or no
- Same Provider or Multiple
- Multihomed Stub One Link
- Multihomed Stub Multiple Links
- Multihomed Stub Multiple ISPs
- Multihomed Transit Network
- Interdomain Traffic Engineering
- Outbound Traffic Control
- Outbound Traffic Load Balancing
- Traffic Engineering Goals
- Managing Scale
- Achieving Predictability
- Challenges to Predictability
- Inter-AS Negotiation
- Outbound Multihoming Goals
- Inbound Traffic Control
- How many links are enough
- Problems with Multihoming in IPv4
- Slide 68
- Slide 69
- Diagnosis and Troubleshooting
- Operator Mailing List (NANOG)
- Operator Mailing List
- Routing Configuration
- Internet Business Model (Simplified)
- Peering Contracts Consistent Export
- Consistent Export
- Inconsistent Export in Practice
- Blackholes
- Security
- Security ldquoBogonrdquo Routes
- Spam Phishing etc
- Spam and Routing
- Worms and Botnets
- What is a Worm
- Example Worm Code Red
- Code Red Revisions
- Code Red Host Infection Rate
- Designing Fast-Spreading Worms
- Botnets
- ldquoRallyingrdquo the Botnet
- Botnet Control
- Some Defenses
- Idea 1 Ingress Filtering
- Idea 2 uRPF Checks
- Problems with uRPF
- S-BGP
- Practical Problems with S-BGP
-
9
Example Abilene Network Topology
10
Another Example Backbone
11
GeorgiaTech
Internet Routing Overview
bull Intradomain (ie ldquointra-ASrdquo) routingbull Interdomain routing
Comcast
Abilene
ATampT Cogent
Autonomous Systems (ASes)
12
Internet Routing Protocol BGP
Route Advertisement
Autonomous Systems (ASes)
Session
Traffic
Destination Next-hop AS Path1302070016
1302070016
19258989
6625025244
105782637
174hellip 2637
13
Two Flavors of BGP
bull External BGP (eBGP) exchanging routes between ASes
bull Internal BGP (iBGP) disseminating routes to external destinations among the routers within an AS
eBGPiBGP
Question Whatrsquos the difference between IGP and iBGP
14
IPv4 Addresses Networks of Networks
bull 32-bit number in ldquodotted-quadrdquo notation
ndash wwwccgatechedu --- 130207736
10000010 11001111 00000111 00100100
Network (16 bits) Host (16 bits)
130 207 7 36
bull Problem 232 addresses is a lot of table entries
bull Solution Routing based on network and host
ndash 1302070016 is a 16-bit prefix with 216 IP addresses
Topological Addressing
15
Pre-1994 Classful Addressing
Network ID Host ID
8 16
Class A
32
0
Class B 10
Class C 110
Multicast AddressesClass D 1110
Reserved for experimentsClass E 1111
24
8 blocks (eg MIT has 180008)
16 blocks (eg Georgia Tech has 1302070016)
24 blocks (eg ATampT Labs has 19220225024)
Simple Forwarding Address range specifies network ID length
16
Classless Interdomain Routing (CIDR)
IP Address 65142480 ldquoMaskrdquo 2552552520
01000001 00001110 11111000 00000000
11111111 11111111 11111100 00000000
Use two 32-bit numbers to represent a network Network number = IP address + Mask
Example BellSouth Prefix 6514248022
Address no longer specifies network ID rangeNew forwarding trick Longest Prefix Match
17
Benefits of CIDR
bull Efficiency Can allocate blocks of prefixes on a finer granularity
bull Hierarchy Prefixes can be aggregated into supernets (Not always done Typically not in fact)
Customer 1
Customer 2
ATampT Internet
1220249024
1220231024120008
18
Growth of IP Prefixes
19
1994-1998 Linear Growth
bull About 10000 new entries per yearbull In theory less instability at the edges (why)
Source Geoff Huston
20
Around 2000 Fast Growth Resumes
Claim remaining 8s will be exhausted within the next 5-10 years
T Hain ldquoA Pragmatic Report on IPv4 Address Space Consumptionrdquo Cisco IPJ September 2005
21
Fast growth resumes
Rapid growth in routing tables
Dot-Bomb Hiccup
Significant contributor Multihoming
Source Geoff Huston
22
The Address Allocation Process
bull Allocation policies of RIRs affect pressure on IPv4 address space
IANA
AfriNIC APNIC ARIN LACNIC RIPE
httpwwwianaorgassignmentsipv4-address-space
Georgia Tech
23
Common Problems
bull Diagnosis and troubleshooting (hence measurement)bull Traffic engineeringbull Securitybull Design and capacity planning
24
What can go wrong
Two-thirds of the problems are caused by configuration of the routing protocol
Some downtime is very hard to protect againsthellip
25
Measurement and Monitoring
26
Passive vs Active Measurement
bull Passive Measurement Collection of packets flow statistics of traffic that is already flowing on the networkndash Packet tracesndash Flow statisticsndash Application-level logs
bull Active Measurement Inject ldquoprobingrdquo traffic to measure various characteristicsndash Traceroutendash Pingndash Application-level probes (eg Web downloads)
27
Billing for Internet Usage
bull 95th Percentile billingndash Customer network pays for ldquocommitted information
raterdquo (CIR)ndash Throughput measured every 5 minutes (typically with
SNMP flow statistics also can be used for billing)ndash Customer billed based on 95th percentile
28
Passive Traffic Data Measurement
bull SNMP bytepacket counts everywhere
bull Packet monitoring selected locations
bull Flow monitoring typically at edges (if possible)ndash Direct computation of the traffic matrixndash Input to denial-of-service attack detection
bull Deep Packet Inspection also at edge where possible
29
Simple Network Management Protocol
bull Management Information Base (MIB)ndash Information storendash Unique variables named by
OIDsndash Accessed with SNMP
bull Specific MIBs for bytepacket counts (per link)
Manager Agent
SNMP
DB
ManagedObjects
30
SNMP (Passive)
bull Advantage ubiquitousndash Supported on all networking equipmentndash Multiple products for polling and analyzing data
bull Disadvantages ndash Coarse granularityndash Cannot express complex queries on the datandash Unreliable delivery of the data using UDP
bull Utilityndash Link utilization (billing)ndash Traffic matrix inference
31
Packet-level Monitoring
bull Passive monitoring to collect full packet contents (or at least headers)
bull Advantages lots of detailed informationndash Precise timing informationndash Information in packet headers
bull Disadvantages overheadndash Hard to keep up with high-speed linksndash Often requires a separate monitoring device
32
Full Packet Capture (Passive)
Example Georgia Tech OC3Mon
bull Rack-mounted PCbull Optical splitterbull Data Acquisition and
Generation (DAG) card
Source endacecom
33
What is a flow
bull Source IP addressbull Destination IP addressbull Source portbull Destination portbull Layer 3 protocol typebull TOS byte (DSCP)bull Input logical interface (ifIndex)
34
Cisco NetFlow
bull Basic output ldquoFlow recordrdquondash Most common version is v5
bull Current version (9) is being standardized in the IETF (template-based)ndash More flexible record formatndash Much easier to add new flow record types
Core Network
Collection and Aggregation
Collector (PC)Approximately 1500 bytes
20-50 flow recordsSent more frequently if traffic increases
35
Flow Record Contents
bull Source and Destination IP address and portbull Packet and byte countsbull Start and end timesbull ToS TCP flags
Basic information about the flowhellip
hellipplus information related to routing
bull Next-hop IP addressbull Source and destination ASbull Source and destination prefix
36
flow 1 flow 2 flow 3 flow 4
Aggregating Packets into Flows
bull Criteria 1 Set of packets that ldquobelong togetherrdquondash Sourcedestination IP addresses and port numbersndash Same protocol ToS bits hellip ndash Same inputoutput interfaces at a router (if known)
bull Criteria 2 Packets that are ldquocloserdquo together in timendash Maximum inter-packet spacing (eg 15 sec 30 sec)ndash Example flows 2 and 4 are different flows due to time
37
Reducing Measurement Overhead
bull Filtering on interfacendash destination prefix for a customerndash port number for an application (eg 80 for Web)
bull Sampling before insertion into flow cachendash Random deterministic or hash-based samplingndash 1-out-of-n or stratified based on packetflow sizendash Two types packet-level and flow-level
bull Aggregation after cache evictionndash packetsflows with same next-hop ASndash packetsflows destined to a particular service
38
Packet Sampling for Flow Monitoring
bull Packet sampling before flow creation (Sampled Netflow)ndash 1-out-of-m sampling of individual packets (eg m=100)ndash Create of flow records over the sampled packets
bull Reducing overheadndash Avoid per-packet overhead on (m-1)m packetsndash Avoid creating records for a large number of small flows
bull Increasing overhead (in some cases)ndash May split some long transfers into multiple flow records ndash hellip due to larger time gaps between successive packets
time
not sampled
two flowstimeout
39
Sampling Flow-Level Sampling
bull Sampling of flow records evicted from flow cachendash When evicting flows from table or when analyzing flows
bull Stratified sampling to put weight on ldquoheavyrdquo flowsndash Select all long flows and sample the short flows
bull Reduces the number of flow records ndash Still measures the vast majority of the traffic
Flow 1 40 bytesFlow 2 15580 bytesFlow 3 8196 bytesFlow 4 5350789 bytesFlow 5 532 bytesFlow 6 7432 bytes
sample with 100 probability
sample with 01 probability
sample with 10 probability
40
Two Main Approaches
bull Packet-level Monitoringndash Keep packet-level statisticsndash Examine (and potentially log) variety of packet-level
statistics Essentially anything in the packetndash Timing
bull Flow-level Monitoringndash Monitor packet-by-packet (though sometimes
sampled)ndash Keep aggregate statistics on a flow
41
Packet Capture on High-Speed Links
Example Georgia Tech ldquoOC3Monrdquo
bull Rack-mounted PCbull Optical splitterbull Data Acquisition and
Generation (DAG) card
Source endacecom
42
Characteristics of Packet Capture
bull Allows inspection on every packet on 10G links
bull Disadvantagesndash Costlyndash Requires splitting optical fibersndash Must be able to filterstore data
43
Data Measurement Repositories
bull AbileneInternet 2 Observatoryndash Configuration examplesndash SNMP datandash ISIS BGP routing data NetFlow traffic data
bull RouteViewsndash BGP updatesndash BGP table snapshots
44
Multihoming and Traffic Engineering
45
What is Multihoming
bull The use of redundant network links for the purposes of external connectivity
bull Can be achieved at many layers of the protocol stack and many places in the networkndash Multiple network interfaces in a PC
ndash An ISP with multiple upstream interfaces
bull Can refer to having multiple connections tondash The same ISP
ndash Multiple ISPs
46
Why Multihome
bull Redundancybull Availabilitybull Performancebull Cost
Interdomain traffic engineering the process by which a multihomed network configures its
network to achieve these goals
47
Redundancy
bull Maintain connectivity in the face ofndash Physical connectivity problems (fiber cut device
failures etc)ndash Failures in upstream ISP
48
Performance
bull Use multiple network links at once to achieve higher throughput than just over a single link
bull Allows incoming traffic to be load-balanced
70 of traffic30 of traffic
49
Multihoming in IP Networks Today
bull Stub AS no transit service for other ASesndash No need to use BGP
bull Multi-homed stub AS has connectivity to multiple immediate upstream ISPsndash Need BGPndash No need for a public AS numberndash No need for IP prefix allocation
bull Multi-homed transit AS connectivity to multiple ASes and transit servicendash Need BGP public AS number IP prefix allocation
50
BGP or no
bull Advantages of static routingndash Cheapersmaller routers (less true nowadays)ndash Simpler to configure
bull Advantages of BGPndash More control of your destiny (have providers stop
announcing you)ndash Fastermore intelligent selection of where to send
outbound packetsndash Better debugging of net problems (you can see the
Internet topology now)
51
Same Provider or Multiple
bull If your provider is reliable and fast and affordably and offers good tech-support you may want to multi-home initially to them via some backup path (slow is better than dead)
bull Eventually yoursquoll want to multi-home to different providers to avoid failure modes due to one providerrsquos architecture decisions
52
Multihomed Stub One Link
bull Downstream ISPrsquos routers configure default (ldquostaticrdquo) routes pointing to border router
bull Upstream ISP advertises reachability
Upstream ISP
Multiple links between same pair of routers
Default routes to ldquoborderrdquo
ldquoStubrdquoISP
53
Multihomed Stub Multiple Links
bull Use BGP to share loadbull Use private AS number (why is this OK)bull As before upstream ISP advertises prefix
Upstream ISP
Multiple links to different upstream routers
ldquoStubrdquoISP
Internal routing for ldquohot potatordquo
BGP for load balance at edge
54
Multihomed Stub Multiple ISPs
bull Many possibilitiesndash Load sharingndash Primary-backupndash Selective use of different ISPs
bull Requires BGP public AS number etc
ldquoStubrdquoISP
Upstream
ISP 1
Upstream
ISP 2
55
Multihomed Transit Network
bull BGP everywherebull Incoming and outcoming trafficbull Challenge balancing load on intradomain and egress
links given an offered traffic load
TransitISP
ISP 1
ISP 2
ISP 3
56
Interdomain Traffic Engineering
bull The process by which a network operator configures the network to achievendash Traffic load balancendash Redundancy (primarybackup) etc
bull Two tasksndash Outbound traffic controlndash Inbound traffic control
bull Key Problems Predictability and Scalability
57
Outbound Traffic Control
bull Easier to control than inbound trafficndash Destination-based routing sender determines where
the packets go
bull Control over next-hop AS onlyndash Cannot control selection of the entire path
Provider 1 Provider 2
Control with local preference
58
Outbound Traffic Load Balancingbull Control routes to provider per-prefix
ndash Assign local preference across destination prefixesndash Change the local preference assignments over time
bull Useful inputs to load balancingndash End-to-end path performance datandash Outbound traffic statistics per destination prefix
bull Challenge Getting from traffic volumes to groups of prefixes that should be assigned to each link
Premise of ldquointelligent route controlrdquo preoducts
59
Traffic Engineering Goals
bull Predictabilityndash Ensure the BGP decision process is deterministicndash Assume that BGP updates are (relatively) stable
bull Limit overhead introduced by routing changesndash Minimize frequency of changes to routing policiesndash Limit number of prefixes affected by changes
bull Limit impact on how traffic enters the networkndash Avoid new routes that might change neighborrsquos mindndash Select route with same attributes or at least path length
60
Managing Scalebull Destination prefixes
ndash More than 90000 destination prefixes
bull Donrsquot want to have per-prefix routing policies
ndash Small fraction of prefixes contribute most of the traffic
bull Focus on the small number of heavy hitters
ndash Define routing policies for selected prefixes
bull Routing choicesndash About 27000 unique ldquorouting choicesrdquo
bull Help in reducing the scale of the problem
ndash Small fraction of ldquorouting choicesrdquo contribute most traffic
bull Focus on the very small number of ldquorouting choicesrdquo
ndash Define routing policies on common attributes
61
Achieving Predictability
bull Route prediction with static analysisndash Helpful to know effects before deploymentndash Static analysis can help
TopologyBGP policy
configuration
eBGP routes
Offered traffic
BGP routingmodel
Flow of traffic through the network
62
Challenges to Predictabilitybull For transit ISPs effects on incoming traffic
ndash Lack of coordination strikes again
63
ldquoHot Potatordquo routing
Inter-AS Negotiation
bull Coordination aids predictabilityndash Negotiate where to sendndash Inbound and outboundndash Mutual benefits
bull How to implementndash What info to exchangendash Protecting privacyndash How to prioritize choicesndash How to prevent cheating
Destination 2
Destination 1
multiplepeeringpoints
Provider A
Provider B
64
Outbound Multihoming Goals
bull Redundancyndash Dynamic routing will failover to backup link
bull Performancendash Select provider with best performance per prefix
ndash Requires active probing
bull Costndash Select provider per prefix over time to minimize the
total financial cost
65
Inbound Traffic Control
bull More difficult no control over neighborsrsquo decisions
bull Three common techniques (previously discussed)ndash AS path prependingndash Communities and local preferencendash Prefix splitting
How does todayrsquos paper (MONET) control inbound traffic
66
How many links are enough
K upstream ISPs
Not much benefit beyond 4 ISPs
Akella et al ldquoPerformance Benefits of Multihomingrdquo SIGCOMM 2003
67
Problems with Multihoming in IPv4
bull Routing table growthndash Provider-based addressingndash Advertising prefix out multiple ISPs ndash canrsquot aggregate
bull Poor control over inbound trafficndash Existing mechanisms do not allow hosts to control
inbound traffic
68
GeorgiaTech
Internet Routing Overview
bull Intradomain (ie ldquointra-ASrdquo) routingbull Interdomain routing
Comcast
Abilene
ATampT Cogent
Autonomous Systems (ASes)
69
Configuration Problems ldquoAS 7007rdquoldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
UUNet
Florida InternetBarn
Sprint
70
Diagnosis and Troubleshootingldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
ldquoMicrosofts websites were offline for up to 23 hoursbecause of a [router] misconfigurationhellipit took nearly a day to determine what was wrong and undo the changesrdquo -- wiredcom January 25 2001
ldquoWorldCom Inchellipsuffered a widespread outage on its Internet backbone that affected roughly 20 percent of its US customer base The network problemshellipaffected millions of computer users worldwide A spokeswoman attributed the outage to a route table issue -- cnncom October 3 2002
A number of Covad customers went out from 5pm today due to supposedly a DDOS (distributed denial of service attack) on a key Level3 data center which later was described as a route leak (misconfiguration)ldquo
-- dslreportscom February 23 2004
71
Operator Mailing List (NANOG)Date Mon 18 Oct 2004 091515 -0700Subject Level 3 US east coast issues
Level 3 experiencing widespread unspecified routing issues on the US east coast Master ticket 1086844 Anyone have more specific information
Date Mon 18 Oct 2004 122034 -0400 (EDT)Subject Re Level 3 US east coast issues
Level 3 is currently experiencing a backbone outage causing routing instability and packet loss We are working to restore and will be sending hourly updateshellip
72
Operator Mailing List
0102030405060708090
Filtering RouteLeaks
RouteHijacks
RouteInstability
RoutingLoops
Blackholes
Occurr
ences o
ver
10 Y
ears
1995-1997 1998-2001 2001-2004
Note Only includes problems openly discussed on this list
Compare 83 power outages 1 fire
73
Routing Configuration
Ranking route selection
Dissemination internal route advertisement
Filtering route advertisement
Customer
Competitor
Primary
Backup
74
Internet Business Model (Simplified)
bull CustomerProvider One AS pays another for reachability to some set of destinations
bull ldquoSettlement-freerdquo Peering Bartering Two ASes exchange routes with one another
Provider
Peer
Customer
Preferences implemented with local preference manipulation
Destination
Pay to use
Get paid to use
Free to use
75
Peering Contracts Consistent Export
bull Rules of settlement-free peeringndash Advertise routes at all peering pointsndash Advertised routes must have equal ldquoAS path lengthrdquo
Sprint
ATampT
Enables ldquohot potatordquo routing
ldquoequally goodrdquoroutes
76
Consistent Export
bull Malicedeceptionbull iBGP signaling partitionbull Inconsistent export policy
neighbor 10123route-map PEER permit 10 set prepend 123
neighbor 10456route-map PEER permit 10 set prepend 123 123
Possible Causes
10123 456 1
10456 456 2
Neighbor AS Export
1 1 123
Export Clause Prepend
2 1 123 123
Two different Export Policies
77
Inconsistent Export in Practice
Feamster et al ldquoBorderGuard Detecting Cold Potatoes from Peersrdquo ACM IMC October 2004
78
Blackholes
Date Thu 18 Jul 2002 060510 -0400 (EDT)From Chad Oleary ltcolpoboxcomgtSubject Re problems with 701To ltnanogmeritedugt
Were starting to see the same issues with UUNet again Anyone elseseeing this Trying to reach Qwest
traceroute to 631461901 (631461901) 30 hops max 38 byte packets 1 esc-lp2-gwe-solutionscorpcom (631182201) 1167 ms 1163 ms 1142 ms 2 500Serial2-10GW1TPA2ALTERNET (1571301499) 1097 ms 1059 ms 1044 ms 3 161at-1-0-0XL4ATL1ALTERNET (1526381190) 13839 ms 14108 ms 16638 ms 4 0so-3-1-0XL2ATL5ALTERNET (152630238) 14370 ms 14587 ms 14553 ms 5 POS7-0BR2ATL5ALTERNET (1526382193) 13928 ms 14099 ms 14053 ms 6 7 hellip
79
Security
80
Security ldquoBogonrdquo Routes
Feamster et al ldquoAn Empirical Study of lsquoBogonrsquo Route Advertisementsrdquo ACM CCR January 2005
81
Spam Phishing etc
bull Unsolicited commercial emailbull As of about August 2008 estimates indicate that
about 95 of all email is spambull Common spam filtering techniques
ndash Content-based filtersndash DNS Blacklist (DNSBL) lookups Significant fraction of
todayrsquos DNS traffic
Can IP addresses from which spam is received be spoofed
82
Spam and Routing
83
Worms and Botnets
84
What is a Worm
bull Code that replicates and propagates across the networkndash Often carries a ldquopayloadrdquo
bull Usually spread via exploiting flaws in open servicesndash ldquoVirusesrdquo require user action to spread
bull First worm Robert Morris November 1988ndash 6-10 of all Internet hosts infected ()
bull Many more since but none on that scale until July 2001
85
Example Worm Code Red
bull Initial version July 13 2001
bull Exploited known ISAPI vulnerability in Microsoft IIS Web servers
bull 1st through 20th of each month spread20th through end of each month attack
bull Payload Web site defacementbull Scanning Random IP addressesbull Bug failure to seed random number generator
86
Code Red Revisions
bull Released July 19 2001
bull Payload flooding attack on wwwwhitehousegovndash Attack was mounted at the IP address of the Web site
bull Bug died after 20th of each month
bull Random number generator for IP scanning fixed
87
Code Red Host Infection Rate
Exponential infection rate
Measured using backscatter technique
88
Designing Fast-Spreading Worms
bull Hit-list scanningndash Time to infect first 10k hosts dominates infection timendash Solution Reconnaissance (stealthy scans etc)
bull Permutation scanningndash Observation Most scanning is redundantndash Idea Shared permutation of address space Start scanning
from own IP address Re-randomize when another infected machine is found
bull Internet-scale hit listsndash Flash worm complete infection within 30 seconds
89
Botnets
bull Bots Autonomous programs performing tasksbull Plenty of ldquobenignrdquo bots
ndash eg weatherbug
bull Botnets group of bots ndash Typically carries malicious connotationndash Large numbers of infected machinesndash Machines ldquoenlistedrdquo with infection vectors like worms
(last lecture)
bull Available for simultaneous control by a masterbull Size up to 350000 nodes (from todayrsquos paper)
90
ldquoRallyingrdquo the Botnet
bull Easy to combine worm backdoor functionalitybull Problem how to learn about successfully
infected machines
bull Optionsndash Emailndash Hard-coded email address
91
Botnet Control
bull Botnet master typically runs some IRC server on a well-known port (eg 6667)
bull Infected machine contacts botnet with pre-programmed DNS name (eg big-botde)
bull Dynamic DNS allows controller to move about freely
Infected Machine
DynamicDNS
BotnetController
(IRC server)
92
Some Defenses
93
Idea 1 Ingress Filtering
bull RFC 2827 Routers install filters to drop packets from networks that are not downstream
bull Feasible at edgesbull Difficult to configure closer to network ldquocorerdquo
20469207024 Internet
Drop all packets with source address other than 20469207024
94
Idea 2 uRPF Checks
bull Unicast Reverse Path Forwardingndash Cisco ldquoip verify unicast reverse-pathrdquo
bull Requires symmetric routing
Accept packet from interface only if forwarding table entry for source IP address matches ingress interface
100183
A10018
10015
101203
100181 24
10011 24
Strict Mode uRPF
Enabled
ldquoArdquo Routing TableDestination Next Hop1001024 Int 110018024 Int 2
100183 from wrong interface
95
Problems with uRPF
bull Asymmetric routing
96
S-BGP
bull Address-based PKI validate signaturesndash Authentication of
bull ownership for IP address blocks bull AS number bull an ASs identity and bull a BGP routers identity
ndash Use existing infrastructure (Internet registries etc)ndash Routing origination is digitally signedndash BGP updates are digitally signed
1048708 bull Route attestations A new optional BGP transitive path attribute
ndash carries digital signatures covering the routing information in updates
97
Practical Problems with S-BGP
bull Requires Public-Key Infrastructure
bull Lots of digital signatures to calculate and verifyndash Message overheadndash CPU overhead
bull Calculation expense is greatest when topology is changingndash Caching can help
bull Route aggregation is problematic (maybe thatrsquos OK)
bull Secure route withdrawals when link or node fails
bull Address ownership data out of date
bull Deployment
- Networking
- Goal of This Tutorial
- Todayrsquos Networks
- Business Models
- Billing for Internet Usage
- Net Neutrality
- Network Operations
- Point-of-Presence (PoP)
- Example Abilene Network Topology
- Another Example Backbone
- Internet Routing Overview
- Internet Routing Protocol BGP
- Two Flavors of BGP
- IPv4 Addresses Networks of Networks
- Pre-1994 Classful Addressing
- Classless Interdomain Routing (CIDR)
- Benefits of CIDR
- Growth of IP Prefixes
- 1994-1998 Linear Growth
- Around 2000 Fast Growth Resumes
- Fast growth resumes
- The Address Allocation Process
- Common Problems
- What can go wrong
- Measurement and Monitoring
- Passive vs Active Measurement
- Slide 27
- Passive Traffic Data Measurement
- Simple Network Management Protocol
- SNMP (Passive)
- Packet-level Monitoring
- Full Packet Capture (Passive)
- What is a flow
- Cisco NetFlow
- Flow Record Contents
- Aggregating Packets into Flows
- Reducing Measurement Overhead
- Packet Sampling for Flow Monitoring
- Sampling Flow-Level Sampling
- Two Main Approaches
- Packet Capture on High-Speed Links
- Characteristics of Packet Capture
- Data Measurement Repositories
- Multihoming and Traffic Engineering
- What is Multihoming
- Why Multihome
- Redundancy
- Performance
- Multihoming in IP Networks Today
- BGP or no
- Same Provider or Multiple
- Multihomed Stub One Link
- Multihomed Stub Multiple Links
- Multihomed Stub Multiple ISPs
- Multihomed Transit Network
- Interdomain Traffic Engineering
- Outbound Traffic Control
- Outbound Traffic Load Balancing
- Traffic Engineering Goals
- Managing Scale
- Achieving Predictability
- Challenges to Predictability
- Inter-AS Negotiation
- Outbound Multihoming Goals
- Inbound Traffic Control
- How many links are enough
- Problems with Multihoming in IPv4
- Slide 68
- Slide 69
- Diagnosis and Troubleshooting
- Operator Mailing List (NANOG)
- Operator Mailing List
- Routing Configuration
- Internet Business Model (Simplified)
- Peering Contracts Consistent Export
- Consistent Export
- Inconsistent Export in Practice
- Blackholes
- Security
- Security ldquoBogonrdquo Routes
- Spam Phishing etc
- Spam and Routing
- Worms and Botnets
- What is a Worm
- Example Worm Code Red
- Code Red Revisions
- Code Red Host Infection Rate
- Designing Fast-Spreading Worms
- Botnets
- ldquoRallyingrdquo the Botnet
- Botnet Control
- Some Defenses
- Idea 1 Ingress Filtering
- Idea 2 uRPF Checks
- Problems with uRPF
- S-BGP
- Practical Problems with S-BGP
-
10
Another Example Backbone
11
GeorgiaTech
Internet Routing Overview
bull Intradomain (ie ldquointra-ASrdquo) routingbull Interdomain routing
Comcast
Abilene
ATampT Cogent
Autonomous Systems (ASes)
12
Internet Routing Protocol BGP
Route Advertisement
Autonomous Systems (ASes)
Session
Traffic
Destination Next-hop AS Path1302070016
1302070016
19258989
6625025244
105782637
174hellip 2637
13
Two Flavors of BGP
bull External BGP (eBGP) exchanging routes between ASes
bull Internal BGP (iBGP) disseminating routes to external destinations among the routers within an AS
eBGPiBGP
Question Whatrsquos the difference between IGP and iBGP
14
IPv4 Addresses Networks of Networks
bull 32-bit number in ldquodotted-quadrdquo notation
ndash wwwccgatechedu --- 130207736
10000010 11001111 00000111 00100100
Network (16 bits) Host (16 bits)
130 207 7 36
bull Problem 232 addresses is a lot of table entries
bull Solution Routing based on network and host
ndash 1302070016 is a 16-bit prefix with 216 IP addresses
Topological Addressing
15
Pre-1994 Classful Addressing
Network ID Host ID
8 16
Class A
32
0
Class B 10
Class C 110
Multicast AddressesClass D 1110
Reserved for experimentsClass E 1111
24
8 blocks (eg MIT has 180008)
16 blocks (eg Georgia Tech has 1302070016)
24 blocks (eg ATampT Labs has 19220225024)
Simple Forwarding Address range specifies network ID length
16
Classless Interdomain Routing (CIDR)
IP Address 65142480 ldquoMaskrdquo 2552552520
01000001 00001110 11111000 00000000
11111111 11111111 11111100 00000000
Use two 32-bit numbers to represent a network Network number = IP address + Mask
Example BellSouth Prefix 6514248022
Address no longer specifies network ID rangeNew forwarding trick Longest Prefix Match
17
Benefits of CIDR
bull Efficiency Can allocate blocks of prefixes on a finer granularity
bull Hierarchy Prefixes can be aggregated into supernets (Not always done Typically not in fact)
Customer 1
Customer 2
ATampT Internet
1220249024
1220231024120008
18
Growth of IP Prefixes
19
1994-1998 Linear Growth
bull About 10000 new entries per yearbull In theory less instability at the edges (why)
Source Geoff Huston
20
Around 2000 Fast Growth Resumes
Claim remaining 8s will be exhausted within the next 5-10 years
T Hain ldquoA Pragmatic Report on IPv4 Address Space Consumptionrdquo Cisco IPJ September 2005
21
Fast growth resumes
Rapid growth in routing tables
Dot-Bomb Hiccup
Significant contributor Multihoming
Source Geoff Huston
22
The Address Allocation Process
bull Allocation policies of RIRs affect pressure on IPv4 address space
IANA
AfriNIC APNIC ARIN LACNIC RIPE
httpwwwianaorgassignmentsipv4-address-space
Georgia Tech
23
Common Problems
bull Diagnosis and troubleshooting (hence measurement)bull Traffic engineeringbull Securitybull Design and capacity planning
24
What can go wrong
Two-thirds of the problems are caused by configuration of the routing protocol
Some downtime is very hard to protect againsthellip
25
Measurement and Monitoring
26
Passive vs Active Measurement
bull Passive Measurement Collection of packets flow statistics of traffic that is already flowing on the networkndash Packet tracesndash Flow statisticsndash Application-level logs
bull Active Measurement Inject ldquoprobingrdquo traffic to measure various characteristicsndash Traceroutendash Pingndash Application-level probes (eg Web downloads)
27
Billing for Internet Usage
bull 95th Percentile billingndash Customer network pays for ldquocommitted information
raterdquo (CIR)ndash Throughput measured every 5 minutes (typically with
SNMP flow statistics also can be used for billing)ndash Customer billed based on 95th percentile
28
Passive Traffic Data Measurement
bull SNMP bytepacket counts everywhere
bull Packet monitoring selected locations
bull Flow monitoring typically at edges (if possible)ndash Direct computation of the traffic matrixndash Input to denial-of-service attack detection
bull Deep Packet Inspection also at edge where possible
29
Simple Network Management Protocol
bull Management Information Base (MIB)ndash Information storendash Unique variables named by
OIDsndash Accessed with SNMP
bull Specific MIBs for bytepacket counts (per link)
Manager Agent
SNMP
DB
ManagedObjects
30
SNMP (Passive)
bull Advantage ubiquitousndash Supported on all networking equipmentndash Multiple products for polling and analyzing data
bull Disadvantages ndash Coarse granularityndash Cannot express complex queries on the datandash Unreliable delivery of the data using UDP
bull Utilityndash Link utilization (billing)ndash Traffic matrix inference
31
Packet-level Monitoring
bull Passive monitoring to collect full packet contents (or at least headers)
bull Advantages lots of detailed informationndash Precise timing informationndash Information in packet headers
bull Disadvantages overheadndash Hard to keep up with high-speed linksndash Often requires a separate monitoring device
32
Full Packet Capture (Passive)
Example Georgia Tech OC3Mon
bull Rack-mounted PCbull Optical splitterbull Data Acquisition and
Generation (DAG) card
Source endacecom
33
What is a flow
bull Source IP addressbull Destination IP addressbull Source portbull Destination portbull Layer 3 protocol typebull TOS byte (DSCP)bull Input logical interface (ifIndex)
34
Cisco NetFlow
bull Basic output ldquoFlow recordrdquondash Most common version is v5
bull Current version (9) is being standardized in the IETF (template-based)ndash More flexible record formatndash Much easier to add new flow record types
Core Network
Collection and Aggregation
Collector (PC)Approximately 1500 bytes
20-50 flow recordsSent more frequently if traffic increases
35
Flow Record Contents
bull Source and Destination IP address and portbull Packet and byte countsbull Start and end timesbull ToS TCP flags
Basic information about the flowhellip
hellipplus information related to routing
bull Next-hop IP addressbull Source and destination ASbull Source and destination prefix
36
flow 1 flow 2 flow 3 flow 4
Aggregating Packets into Flows
bull Criteria 1 Set of packets that ldquobelong togetherrdquondash Sourcedestination IP addresses and port numbersndash Same protocol ToS bits hellip ndash Same inputoutput interfaces at a router (if known)
bull Criteria 2 Packets that are ldquocloserdquo together in timendash Maximum inter-packet spacing (eg 15 sec 30 sec)ndash Example flows 2 and 4 are different flows due to time
37
Reducing Measurement Overhead
bull Filtering on interfacendash destination prefix for a customerndash port number for an application (eg 80 for Web)
bull Sampling before insertion into flow cachendash Random deterministic or hash-based samplingndash 1-out-of-n or stratified based on packetflow sizendash Two types packet-level and flow-level
bull Aggregation after cache evictionndash packetsflows with same next-hop ASndash packetsflows destined to a particular service
38
Packet Sampling for Flow Monitoring
bull Packet sampling before flow creation (Sampled Netflow)ndash 1-out-of-m sampling of individual packets (eg m=100)ndash Create of flow records over the sampled packets
bull Reducing overheadndash Avoid per-packet overhead on (m-1)m packetsndash Avoid creating records for a large number of small flows
bull Increasing overhead (in some cases)ndash May split some long transfers into multiple flow records ndash hellip due to larger time gaps between successive packets
time
not sampled
two flowstimeout
39
Sampling Flow-Level Sampling
bull Sampling of flow records evicted from flow cachendash When evicting flows from table or when analyzing flows
bull Stratified sampling to put weight on ldquoheavyrdquo flowsndash Select all long flows and sample the short flows
bull Reduces the number of flow records ndash Still measures the vast majority of the traffic
Flow 1 40 bytesFlow 2 15580 bytesFlow 3 8196 bytesFlow 4 5350789 bytesFlow 5 532 bytesFlow 6 7432 bytes
sample with 100 probability
sample with 01 probability
sample with 10 probability
40
Two Main Approaches
bull Packet-level Monitoringndash Keep packet-level statisticsndash Examine (and potentially log) variety of packet-level
statistics Essentially anything in the packetndash Timing
bull Flow-level Monitoringndash Monitor packet-by-packet (though sometimes
sampled)ndash Keep aggregate statistics on a flow
41
Packet Capture on High-Speed Links
Example Georgia Tech ldquoOC3Monrdquo
bull Rack-mounted PCbull Optical splitterbull Data Acquisition and
Generation (DAG) card
Source endacecom
42
Characteristics of Packet Capture
bull Allows inspection on every packet on 10G links
bull Disadvantagesndash Costlyndash Requires splitting optical fibersndash Must be able to filterstore data
43
Data Measurement Repositories
bull AbileneInternet 2 Observatoryndash Configuration examplesndash SNMP datandash ISIS BGP routing data NetFlow traffic data
bull RouteViewsndash BGP updatesndash BGP table snapshots
44
Multihoming and Traffic Engineering
45
What is Multihoming
bull The use of redundant network links for the purposes of external connectivity
bull Can be achieved at many layers of the protocol stack and many places in the networkndash Multiple network interfaces in a PC
ndash An ISP with multiple upstream interfaces
bull Can refer to having multiple connections tondash The same ISP
ndash Multiple ISPs
46
Why Multihome
bull Redundancybull Availabilitybull Performancebull Cost
Interdomain traffic engineering the process by which a multihomed network configures its
network to achieve these goals
47
Redundancy
bull Maintain connectivity in the face ofndash Physical connectivity problems (fiber cut device
failures etc)ndash Failures in upstream ISP
48
Performance
bull Use multiple network links at once to achieve higher throughput than just over a single link
bull Allows incoming traffic to be load-balanced
70 of traffic30 of traffic
49
Multihoming in IP Networks Today
bull Stub AS no transit service for other ASesndash No need to use BGP
bull Multi-homed stub AS has connectivity to multiple immediate upstream ISPsndash Need BGPndash No need for a public AS numberndash No need for IP prefix allocation
bull Multi-homed transit AS connectivity to multiple ASes and transit servicendash Need BGP public AS number IP prefix allocation
50
BGP or no
bull Advantages of static routingndash Cheapersmaller routers (less true nowadays)ndash Simpler to configure
bull Advantages of BGPndash More control of your destiny (have providers stop
announcing you)ndash Fastermore intelligent selection of where to send
outbound packetsndash Better debugging of net problems (you can see the
Internet topology now)
51
Same Provider or Multiple
bull If your provider is reliable and fast and affordably and offers good tech-support you may want to multi-home initially to them via some backup path (slow is better than dead)
bull Eventually yoursquoll want to multi-home to different providers to avoid failure modes due to one providerrsquos architecture decisions
52
Multihomed Stub One Link
bull Downstream ISPrsquos routers configure default (ldquostaticrdquo) routes pointing to border router
bull Upstream ISP advertises reachability
Upstream ISP
Multiple links between same pair of routers
Default routes to ldquoborderrdquo
ldquoStubrdquoISP
53
Multihomed Stub Multiple Links
bull Use BGP to share loadbull Use private AS number (why is this OK)bull As before upstream ISP advertises prefix
Upstream ISP
Multiple links to different upstream routers
ldquoStubrdquoISP
Internal routing for ldquohot potatordquo
BGP for load balance at edge
54
Multihomed Stub Multiple ISPs
bull Many possibilitiesndash Load sharingndash Primary-backupndash Selective use of different ISPs
bull Requires BGP public AS number etc
ldquoStubrdquoISP
Upstream
ISP 1
Upstream
ISP 2
55
Multihomed Transit Network
bull BGP everywherebull Incoming and outcoming trafficbull Challenge balancing load on intradomain and egress
links given an offered traffic load
TransitISP
ISP 1
ISP 2
ISP 3
56
Interdomain Traffic Engineering
bull The process by which a network operator configures the network to achievendash Traffic load balancendash Redundancy (primarybackup) etc
bull Two tasksndash Outbound traffic controlndash Inbound traffic control
bull Key Problems Predictability and Scalability
57
Outbound Traffic Control
bull Easier to control than inbound trafficndash Destination-based routing sender determines where
the packets go
bull Control over next-hop AS onlyndash Cannot control selection of the entire path
Provider 1 Provider 2
Control with local preference
58
Outbound Traffic Load Balancingbull Control routes to provider per-prefix
ndash Assign local preference across destination prefixesndash Change the local preference assignments over time
bull Useful inputs to load balancingndash End-to-end path performance datandash Outbound traffic statistics per destination prefix
bull Challenge Getting from traffic volumes to groups of prefixes that should be assigned to each link
Premise of ldquointelligent route controlrdquo preoducts
59
Traffic Engineering Goals
bull Predictabilityndash Ensure the BGP decision process is deterministicndash Assume that BGP updates are (relatively) stable
bull Limit overhead introduced by routing changesndash Minimize frequency of changes to routing policiesndash Limit number of prefixes affected by changes
bull Limit impact on how traffic enters the networkndash Avoid new routes that might change neighborrsquos mindndash Select route with same attributes or at least path length
60
Managing Scalebull Destination prefixes
ndash More than 90000 destination prefixes
bull Donrsquot want to have per-prefix routing policies
ndash Small fraction of prefixes contribute most of the traffic
bull Focus on the small number of heavy hitters
ndash Define routing policies for selected prefixes
bull Routing choicesndash About 27000 unique ldquorouting choicesrdquo
bull Help in reducing the scale of the problem
ndash Small fraction of ldquorouting choicesrdquo contribute most traffic
bull Focus on the very small number of ldquorouting choicesrdquo
ndash Define routing policies on common attributes
61
Achieving Predictability
bull Route prediction with static analysisndash Helpful to know effects before deploymentndash Static analysis can help
TopologyBGP policy
configuration
eBGP routes
Offered traffic
BGP routingmodel
Flow of traffic through the network
62
Challenges to Predictabilitybull For transit ISPs effects on incoming traffic
ndash Lack of coordination strikes again
63
ldquoHot Potatordquo routing
Inter-AS Negotiation
bull Coordination aids predictabilityndash Negotiate where to sendndash Inbound and outboundndash Mutual benefits
bull How to implementndash What info to exchangendash Protecting privacyndash How to prioritize choicesndash How to prevent cheating
Destination 2
Destination 1
multiplepeeringpoints
Provider A
Provider B
64
Outbound Multihoming Goals
bull Redundancyndash Dynamic routing will failover to backup link
bull Performancendash Select provider with best performance per prefix
ndash Requires active probing
bull Costndash Select provider per prefix over time to minimize the
total financial cost
65
Inbound Traffic Control
bull More difficult no control over neighborsrsquo decisions
bull Three common techniques (previously discussed)ndash AS path prependingndash Communities and local preferencendash Prefix splitting
How does todayrsquos paper (MONET) control inbound traffic
66
How many links are enough
K upstream ISPs
Not much benefit beyond 4 ISPs
Akella et al ldquoPerformance Benefits of Multihomingrdquo SIGCOMM 2003
67
Problems with Multihoming in IPv4
bull Routing table growthndash Provider-based addressingndash Advertising prefix out multiple ISPs ndash canrsquot aggregate
bull Poor control over inbound trafficndash Existing mechanisms do not allow hosts to control
inbound traffic
68
GeorgiaTech
Internet Routing Overview
bull Intradomain (ie ldquointra-ASrdquo) routingbull Interdomain routing
Comcast
Abilene
ATampT Cogent
Autonomous Systems (ASes)
69
Configuration Problems ldquoAS 7007rdquoldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
UUNet
Florida InternetBarn
Sprint
70
Diagnosis and Troubleshootingldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
ldquoMicrosofts websites were offline for up to 23 hoursbecause of a [router] misconfigurationhellipit took nearly a day to determine what was wrong and undo the changesrdquo -- wiredcom January 25 2001
ldquoWorldCom Inchellipsuffered a widespread outage on its Internet backbone that affected roughly 20 percent of its US customer base The network problemshellipaffected millions of computer users worldwide A spokeswoman attributed the outage to a route table issue -- cnncom October 3 2002
A number of Covad customers went out from 5pm today due to supposedly a DDOS (distributed denial of service attack) on a key Level3 data center which later was described as a route leak (misconfiguration)ldquo
-- dslreportscom February 23 2004
71
Operator Mailing List (NANOG)Date Mon 18 Oct 2004 091515 -0700Subject Level 3 US east coast issues
Level 3 experiencing widespread unspecified routing issues on the US east coast Master ticket 1086844 Anyone have more specific information
Date Mon 18 Oct 2004 122034 -0400 (EDT)Subject Re Level 3 US east coast issues
Level 3 is currently experiencing a backbone outage causing routing instability and packet loss We are working to restore and will be sending hourly updateshellip
72
Operator Mailing List
0102030405060708090
Filtering RouteLeaks
RouteHijacks
RouteInstability
RoutingLoops
Blackholes
Occurr
ences o
ver
10 Y
ears
1995-1997 1998-2001 2001-2004
Note Only includes problems openly discussed on this list
Compare 83 power outages 1 fire
73
Routing Configuration
Ranking route selection
Dissemination internal route advertisement
Filtering route advertisement
Customer
Competitor
Primary
Backup
74
Internet Business Model (Simplified)
bull CustomerProvider One AS pays another for reachability to some set of destinations
bull ldquoSettlement-freerdquo Peering Bartering Two ASes exchange routes with one another
Provider
Peer
Customer
Preferences implemented with local preference manipulation
Destination
Pay to use
Get paid to use
Free to use
75
Peering Contracts Consistent Export
bull Rules of settlement-free peeringndash Advertise routes at all peering pointsndash Advertised routes must have equal ldquoAS path lengthrdquo
Sprint
ATampT
Enables ldquohot potatordquo routing
ldquoequally goodrdquoroutes
76
Consistent Export
bull Malicedeceptionbull iBGP signaling partitionbull Inconsistent export policy
neighbor 10123route-map PEER permit 10 set prepend 123
neighbor 10456route-map PEER permit 10 set prepend 123 123
Possible Causes
10123 456 1
10456 456 2
Neighbor AS Export
1 1 123
Export Clause Prepend
2 1 123 123
Two different Export Policies
77
Inconsistent Export in Practice
Feamster et al ldquoBorderGuard Detecting Cold Potatoes from Peersrdquo ACM IMC October 2004
78
Blackholes
Date Thu 18 Jul 2002 060510 -0400 (EDT)From Chad Oleary ltcolpoboxcomgtSubject Re problems with 701To ltnanogmeritedugt
Were starting to see the same issues with UUNet again Anyone elseseeing this Trying to reach Qwest
traceroute to 631461901 (631461901) 30 hops max 38 byte packets 1 esc-lp2-gwe-solutionscorpcom (631182201) 1167 ms 1163 ms 1142 ms 2 500Serial2-10GW1TPA2ALTERNET (1571301499) 1097 ms 1059 ms 1044 ms 3 161at-1-0-0XL4ATL1ALTERNET (1526381190) 13839 ms 14108 ms 16638 ms 4 0so-3-1-0XL2ATL5ALTERNET (152630238) 14370 ms 14587 ms 14553 ms 5 POS7-0BR2ATL5ALTERNET (1526382193) 13928 ms 14099 ms 14053 ms 6 7 hellip
79
Security
80
Security ldquoBogonrdquo Routes
Feamster et al ldquoAn Empirical Study of lsquoBogonrsquo Route Advertisementsrdquo ACM CCR January 2005
81
Spam Phishing etc
bull Unsolicited commercial emailbull As of about August 2008 estimates indicate that
about 95 of all email is spambull Common spam filtering techniques
ndash Content-based filtersndash DNS Blacklist (DNSBL) lookups Significant fraction of
todayrsquos DNS traffic
Can IP addresses from which spam is received be spoofed
82
Spam and Routing
83
Worms and Botnets
84
What is a Worm
bull Code that replicates and propagates across the networkndash Often carries a ldquopayloadrdquo
bull Usually spread via exploiting flaws in open servicesndash ldquoVirusesrdquo require user action to spread
bull First worm Robert Morris November 1988ndash 6-10 of all Internet hosts infected ()
bull Many more since but none on that scale until July 2001
85
Example Worm Code Red
bull Initial version July 13 2001
bull Exploited known ISAPI vulnerability in Microsoft IIS Web servers
bull 1st through 20th of each month spread20th through end of each month attack
bull Payload Web site defacementbull Scanning Random IP addressesbull Bug failure to seed random number generator
86
Code Red Revisions
bull Released July 19 2001
bull Payload flooding attack on wwwwhitehousegovndash Attack was mounted at the IP address of the Web site
bull Bug died after 20th of each month
bull Random number generator for IP scanning fixed
87
Code Red Host Infection Rate
Exponential infection rate
Measured using backscatter technique
88
Designing Fast-Spreading Worms
bull Hit-list scanningndash Time to infect first 10k hosts dominates infection timendash Solution Reconnaissance (stealthy scans etc)
bull Permutation scanningndash Observation Most scanning is redundantndash Idea Shared permutation of address space Start scanning
from own IP address Re-randomize when another infected machine is found
bull Internet-scale hit listsndash Flash worm complete infection within 30 seconds
89
Botnets
bull Bots Autonomous programs performing tasksbull Plenty of ldquobenignrdquo bots
ndash eg weatherbug
bull Botnets group of bots ndash Typically carries malicious connotationndash Large numbers of infected machinesndash Machines ldquoenlistedrdquo with infection vectors like worms
(last lecture)
bull Available for simultaneous control by a masterbull Size up to 350000 nodes (from todayrsquos paper)
90
ldquoRallyingrdquo the Botnet
bull Easy to combine worm backdoor functionalitybull Problem how to learn about successfully
infected machines
bull Optionsndash Emailndash Hard-coded email address
91
Botnet Control
bull Botnet master typically runs some IRC server on a well-known port (eg 6667)
bull Infected machine contacts botnet with pre-programmed DNS name (eg big-botde)
bull Dynamic DNS allows controller to move about freely
Infected Machine
DynamicDNS
BotnetController
(IRC server)
92
Some Defenses
93
Idea 1 Ingress Filtering
bull RFC 2827 Routers install filters to drop packets from networks that are not downstream
bull Feasible at edgesbull Difficult to configure closer to network ldquocorerdquo
20469207024 Internet
Drop all packets with source address other than 20469207024
94
Idea 2 uRPF Checks
bull Unicast Reverse Path Forwardingndash Cisco ldquoip verify unicast reverse-pathrdquo
bull Requires symmetric routing
Accept packet from interface only if forwarding table entry for source IP address matches ingress interface
100183
A10018
10015
101203
100181 24
10011 24
Strict Mode uRPF
Enabled
ldquoArdquo Routing TableDestination Next Hop1001024 Int 110018024 Int 2
100183 from wrong interface
95
Problems with uRPF
bull Asymmetric routing
96
S-BGP
bull Address-based PKI validate signaturesndash Authentication of
bull ownership for IP address blocks bull AS number bull an ASs identity and bull a BGP routers identity
ndash Use existing infrastructure (Internet registries etc)ndash Routing origination is digitally signedndash BGP updates are digitally signed
1048708 bull Route attestations A new optional BGP transitive path attribute
ndash carries digital signatures covering the routing information in updates
97
Practical Problems with S-BGP
bull Requires Public-Key Infrastructure
bull Lots of digital signatures to calculate and verifyndash Message overheadndash CPU overhead
bull Calculation expense is greatest when topology is changingndash Caching can help
bull Route aggregation is problematic (maybe thatrsquos OK)
bull Secure route withdrawals when link or node fails
bull Address ownership data out of date
bull Deployment
- Networking
- Goal of This Tutorial
- Todayrsquos Networks
- Business Models
- Billing for Internet Usage
- Net Neutrality
- Network Operations
- Point-of-Presence (PoP)
- Example Abilene Network Topology
- Another Example Backbone
- Internet Routing Overview
- Internet Routing Protocol BGP
- Two Flavors of BGP
- IPv4 Addresses Networks of Networks
- Pre-1994 Classful Addressing
- Classless Interdomain Routing (CIDR)
- Benefits of CIDR
- Growth of IP Prefixes
- 1994-1998 Linear Growth
- Around 2000 Fast Growth Resumes
- Fast growth resumes
- The Address Allocation Process
- Common Problems
- What can go wrong
- Measurement and Monitoring
- Passive vs Active Measurement
- Slide 27
- Passive Traffic Data Measurement
- Simple Network Management Protocol
- SNMP (Passive)
- Packet-level Monitoring
- Full Packet Capture (Passive)
- What is a flow
- Cisco NetFlow
- Flow Record Contents
- Aggregating Packets into Flows
- Reducing Measurement Overhead
- Packet Sampling for Flow Monitoring
- Sampling Flow-Level Sampling
- Two Main Approaches
- Packet Capture on High-Speed Links
- Characteristics of Packet Capture
- Data Measurement Repositories
- Multihoming and Traffic Engineering
- What is Multihoming
- Why Multihome
- Redundancy
- Performance
- Multihoming in IP Networks Today
- BGP or no
- Same Provider or Multiple
- Multihomed Stub One Link
- Multihomed Stub Multiple Links
- Multihomed Stub Multiple ISPs
- Multihomed Transit Network
- Interdomain Traffic Engineering
- Outbound Traffic Control
- Outbound Traffic Load Balancing
- Traffic Engineering Goals
- Managing Scale
- Achieving Predictability
- Challenges to Predictability
- Inter-AS Negotiation
- Outbound Multihoming Goals
- Inbound Traffic Control
- How many links are enough
- Problems with Multihoming in IPv4
- Slide 68
- Slide 69
- Diagnosis and Troubleshooting
- Operator Mailing List (NANOG)
- Operator Mailing List
- Routing Configuration
- Internet Business Model (Simplified)
- Peering Contracts Consistent Export
- Consistent Export
- Inconsistent Export in Practice
- Blackholes
- Security
- Security ldquoBogonrdquo Routes
- Spam Phishing etc
- Spam and Routing
- Worms and Botnets
- What is a Worm
- Example Worm Code Red
- Code Red Revisions
- Code Red Host Infection Rate
- Designing Fast-Spreading Worms
- Botnets
- ldquoRallyingrdquo the Botnet
- Botnet Control
- Some Defenses
- Idea 1 Ingress Filtering
- Idea 2 uRPF Checks
- Problems with uRPF
- S-BGP
- Practical Problems with S-BGP
-
11
GeorgiaTech
Internet Routing Overview
bull Intradomain (ie ldquointra-ASrdquo) routingbull Interdomain routing
Comcast
Abilene
ATampT Cogent
Autonomous Systems (ASes)
12
Internet Routing Protocol BGP
Route Advertisement
Autonomous Systems (ASes)
Session
Traffic
Destination Next-hop AS Path1302070016
1302070016
19258989
6625025244
105782637
174hellip 2637
13
Two Flavors of BGP
bull External BGP (eBGP) exchanging routes between ASes
bull Internal BGP (iBGP) disseminating routes to external destinations among the routers within an AS
eBGPiBGP
Question Whatrsquos the difference between IGP and iBGP
14
IPv4 Addresses Networks of Networks
bull 32-bit number in ldquodotted-quadrdquo notation
ndash wwwccgatechedu --- 130207736
10000010 11001111 00000111 00100100
Network (16 bits) Host (16 bits)
130 207 7 36
bull Problem 232 addresses is a lot of table entries
bull Solution Routing based on network and host
ndash 1302070016 is a 16-bit prefix with 216 IP addresses
Topological Addressing
15
Pre-1994 Classful Addressing
Network ID Host ID
8 16
Class A
32
0
Class B 10
Class C 110
Multicast AddressesClass D 1110
Reserved for experimentsClass E 1111
24
8 blocks (eg MIT has 180008)
16 blocks (eg Georgia Tech has 1302070016)
24 blocks (eg ATampT Labs has 19220225024)
Simple Forwarding Address range specifies network ID length
16
Classless Interdomain Routing (CIDR)
IP Address 65142480 ldquoMaskrdquo 2552552520
01000001 00001110 11111000 00000000
11111111 11111111 11111100 00000000
Use two 32-bit numbers to represent a network Network number = IP address + Mask
Example BellSouth Prefix 6514248022
Address no longer specifies network ID rangeNew forwarding trick Longest Prefix Match
17
Benefits of CIDR
bull Efficiency Can allocate blocks of prefixes on a finer granularity
bull Hierarchy Prefixes can be aggregated into supernets (Not always done Typically not in fact)
Customer 1
Customer 2
ATampT Internet
1220249024
1220231024120008
18
Growth of IP Prefixes
19
1994-1998 Linear Growth
bull About 10000 new entries per yearbull In theory less instability at the edges (why)
Source Geoff Huston
20
Around 2000 Fast Growth Resumes
Claim remaining 8s will be exhausted within the next 5-10 years
T Hain ldquoA Pragmatic Report on IPv4 Address Space Consumptionrdquo Cisco IPJ September 2005
21
Fast growth resumes
Rapid growth in routing tables
Dot-Bomb Hiccup
Significant contributor Multihoming
Source Geoff Huston
22
The Address Allocation Process
bull Allocation policies of RIRs affect pressure on IPv4 address space
IANA
AfriNIC APNIC ARIN LACNIC RIPE
httpwwwianaorgassignmentsipv4-address-space
Georgia Tech
23
Common Problems
bull Diagnosis and troubleshooting (hence measurement)bull Traffic engineeringbull Securitybull Design and capacity planning
24
What can go wrong
Two-thirds of the problems are caused by configuration of the routing protocol
Some downtime is very hard to protect againsthellip
25
Measurement and Monitoring
26
Passive vs Active Measurement
bull Passive Measurement Collection of packets flow statistics of traffic that is already flowing on the networkndash Packet tracesndash Flow statisticsndash Application-level logs
bull Active Measurement Inject ldquoprobingrdquo traffic to measure various characteristicsndash Traceroutendash Pingndash Application-level probes (eg Web downloads)
27
Billing for Internet Usage
bull 95th Percentile billingndash Customer network pays for ldquocommitted information
raterdquo (CIR)ndash Throughput measured every 5 minutes (typically with
SNMP flow statistics also can be used for billing)ndash Customer billed based on 95th percentile
28
Passive Traffic Data Measurement
bull SNMP bytepacket counts everywhere
bull Packet monitoring selected locations
bull Flow monitoring typically at edges (if possible)ndash Direct computation of the traffic matrixndash Input to denial-of-service attack detection
bull Deep Packet Inspection also at edge where possible
29
Simple Network Management Protocol
bull Management Information Base (MIB)ndash Information storendash Unique variables named by
OIDsndash Accessed with SNMP
bull Specific MIBs for bytepacket counts (per link)
Manager Agent
SNMP
DB
ManagedObjects
30
SNMP (Passive)
bull Advantage ubiquitousndash Supported on all networking equipmentndash Multiple products for polling and analyzing data
bull Disadvantages ndash Coarse granularityndash Cannot express complex queries on the datandash Unreliable delivery of the data using UDP
bull Utilityndash Link utilization (billing)ndash Traffic matrix inference
31
Packet-level Monitoring
bull Passive monitoring to collect full packet contents (or at least headers)
bull Advantages lots of detailed informationndash Precise timing informationndash Information in packet headers
bull Disadvantages overheadndash Hard to keep up with high-speed linksndash Often requires a separate monitoring device
32
Full Packet Capture (Passive)
Example Georgia Tech OC3Mon
bull Rack-mounted PCbull Optical splitterbull Data Acquisition and
Generation (DAG) card
Source endacecom
33
What is a flow
bull Source IP addressbull Destination IP addressbull Source portbull Destination portbull Layer 3 protocol typebull TOS byte (DSCP)bull Input logical interface (ifIndex)
34
Cisco NetFlow
bull Basic output ldquoFlow recordrdquondash Most common version is v5
bull Current version (9) is being standardized in the IETF (template-based)ndash More flexible record formatndash Much easier to add new flow record types
Core Network
Collection and Aggregation
Collector (PC)Approximately 1500 bytes
20-50 flow recordsSent more frequently if traffic increases
35
Flow Record Contents
bull Source and Destination IP address and portbull Packet and byte countsbull Start and end timesbull ToS TCP flags
Basic information about the flowhellip
hellipplus information related to routing
bull Next-hop IP addressbull Source and destination ASbull Source and destination prefix
36
flow 1 flow 2 flow 3 flow 4
Aggregating Packets into Flows
bull Criteria 1 Set of packets that ldquobelong togetherrdquondash Sourcedestination IP addresses and port numbersndash Same protocol ToS bits hellip ndash Same inputoutput interfaces at a router (if known)
bull Criteria 2 Packets that are ldquocloserdquo together in timendash Maximum inter-packet spacing (eg 15 sec 30 sec)ndash Example flows 2 and 4 are different flows due to time
37
Reducing Measurement Overhead
bull Filtering on interfacendash destination prefix for a customerndash port number for an application (eg 80 for Web)
bull Sampling before insertion into flow cachendash Random deterministic or hash-based samplingndash 1-out-of-n or stratified based on packetflow sizendash Two types packet-level and flow-level
bull Aggregation after cache evictionndash packetsflows with same next-hop ASndash packetsflows destined to a particular service
38
Packet Sampling for Flow Monitoring
bull Packet sampling before flow creation (Sampled Netflow)ndash 1-out-of-m sampling of individual packets (eg m=100)ndash Create of flow records over the sampled packets
bull Reducing overheadndash Avoid per-packet overhead on (m-1)m packetsndash Avoid creating records for a large number of small flows
bull Increasing overhead (in some cases)ndash May split some long transfers into multiple flow records ndash hellip due to larger time gaps between successive packets
time
not sampled
two flowstimeout
39
Sampling Flow-Level Sampling
bull Sampling of flow records evicted from flow cachendash When evicting flows from table or when analyzing flows
bull Stratified sampling to put weight on ldquoheavyrdquo flowsndash Select all long flows and sample the short flows
bull Reduces the number of flow records ndash Still measures the vast majority of the traffic
Flow 1 40 bytesFlow 2 15580 bytesFlow 3 8196 bytesFlow 4 5350789 bytesFlow 5 532 bytesFlow 6 7432 bytes
sample with 100 probability
sample with 01 probability
sample with 10 probability
40
Two Main Approaches
bull Packet-level Monitoringndash Keep packet-level statisticsndash Examine (and potentially log) variety of packet-level
statistics Essentially anything in the packetndash Timing
bull Flow-level Monitoringndash Monitor packet-by-packet (though sometimes
sampled)ndash Keep aggregate statistics on a flow
41
Packet Capture on High-Speed Links
Example Georgia Tech ldquoOC3Monrdquo
bull Rack-mounted PCbull Optical splitterbull Data Acquisition and
Generation (DAG) card
Source endacecom
42
Characteristics of Packet Capture
bull Allows inspection on every packet on 10G links
bull Disadvantagesndash Costlyndash Requires splitting optical fibersndash Must be able to filterstore data
43
Data Measurement Repositories
bull AbileneInternet 2 Observatoryndash Configuration examplesndash SNMP datandash ISIS BGP routing data NetFlow traffic data
bull RouteViewsndash BGP updatesndash BGP table snapshots
44
Multihoming and Traffic Engineering
45
What is Multihoming
bull The use of redundant network links for the purposes of external connectivity
bull Can be achieved at many layers of the protocol stack and many places in the networkndash Multiple network interfaces in a PC
ndash An ISP with multiple upstream interfaces
bull Can refer to having multiple connections tondash The same ISP
ndash Multiple ISPs
46
Why Multihome
bull Redundancybull Availabilitybull Performancebull Cost
Interdomain traffic engineering the process by which a multihomed network configures its
network to achieve these goals
47
Redundancy
bull Maintain connectivity in the face ofndash Physical connectivity problems (fiber cut device
failures etc)ndash Failures in upstream ISP
48
Performance
bull Use multiple network links at once to achieve higher throughput than just over a single link
bull Allows incoming traffic to be load-balanced
70 of traffic30 of traffic
49
Multihoming in IP Networks Today
bull Stub AS no transit service for other ASesndash No need to use BGP
bull Multi-homed stub AS has connectivity to multiple immediate upstream ISPsndash Need BGPndash No need for a public AS numberndash No need for IP prefix allocation
bull Multi-homed transit AS connectivity to multiple ASes and transit servicendash Need BGP public AS number IP prefix allocation
50
BGP or no
bull Advantages of static routingndash Cheapersmaller routers (less true nowadays)ndash Simpler to configure
bull Advantages of BGPndash More control of your destiny (have providers stop
announcing you)ndash Fastermore intelligent selection of where to send
outbound packetsndash Better debugging of net problems (you can see the
Internet topology now)
51
Same Provider or Multiple
bull If your provider is reliable and fast and affordably and offers good tech-support you may want to multi-home initially to them via some backup path (slow is better than dead)
bull Eventually yoursquoll want to multi-home to different providers to avoid failure modes due to one providerrsquos architecture decisions
52
Multihomed Stub One Link
bull Downstream ISPrsquos routers configure default (ldquostaticrdquo) routes pointing to border router
bull Upstream ISP advertises reachability
Upstream ISP
Multiple links between same pair of routers
Default routes to ldquoborderrdquo
ldquoStubrdquoISP
53
Multihomed Stub Multiple Links
bull Use BGP to share loadbull Use private AS number (why is this OK)bull As before upstream ISP advertises prefix
Upstream ISP
Multiple links to different upstream routers
ldquoStubrdquoISP
Internal routing for ldquohot potatordquo
BGP for load balance at edge
54
Multihomed Stub Multiple ISPs
bull Many possibilitiesndash Load sharingndash Primary-backupndash Selective use of different ISPs
bull Requires BGP public AS number etc
ldquoStubrdquoISP
Upstream
ISP 1
Upstream
ISP 2
55
Multihomed Transit Network
bull BGP everywherebull Incoming and outcoming trafficbull Challenge balancing load on intradomain and egress
links given an offered traffic load
TransitISP
ISP 1
ISP 2
ISP 3
56
Interdomain Traffic Engineering
bull The process by which a network operator configures the network to achievendash Traffic load balancendash Redundancy (primarybackup) etc
bull Two tasksndash Outbound traffic controlndash Inbound traffic control
bull Key Problems Predictability and Scalability
57
Outbound Traffic Control
bull Easier to control than inbound trafficndash Destination-based routing sender determines where
the packets go
bull Control over next-hop AS onlyndash Cannot control selection of the entire path
Provider 1 Provider 2
Control with local preference
58
Outbound Traffic Load Balancingbull Control routes to provider per-prefix
ndash Assign local preference across destination prefixesndash Change the local preference assignments over time
bull Useful inputs to load balancingndash End-to-end path performance datandash Outbound traffic statistics per destination prefix
bull Challenge Getting from traffic volumes to groups of prefixes that should be assigned to each link
Premise of ldquointelligent route controlrdquo preoducts
59
Traffic Engineering Goals
bull Predictabilityndash Ensure the BGP decision process is deterministicndash Assume that BGP updates are (relatively) stable
bull Limit overhead introduced by routing changesndash Minimize frequency of changes to routing policiesndash Limit number of prefixes affected by changes
bull Limit impact on how traffic enters the networkndash Avoid new routes that might change neighborrsquos mindndash Select route with same attributes or at least path length
60
Managing Scalebull Destination prefixes
ndash More than 90000 destination prefixes
bull Donrsquot want to have per-prefix routing policies
ndash Small fraction of prefixes contribute most of the traffic
bull Focus on the small number of heavy hitters
ndash Define routing policies for selected prefixes
bull Routing choicesndash About 27000 unique ldquorouting choicesrdquo
bull Help in reducing the scale of the problem
ndash Small fraction of ldquorouting choicesrdquo contribute most traffic
bull Focus on the very small number of ldquorouting choicesrdquo
ndash Define routing policies on common attributes
61
Achieving Predictability
bull Route prediction with static analysisndash Helpful to know effects before deploymentndash Static analysis can help
TopologyBGP policy
configuration
eBGP routes
Offered traffic
BGP routingmodel
Flow of traffic through the network
62
Challenges to Predictabilitybull For transit ISPs effects on incoming traffic
ndash Lack of coordination strikes again
63
ldquoHot Potatordquo routing
Inter-AS Negotiation
bull Coordination aids predictabilityndash Negotiate where to sendndash Inbound and outboundndash Mutual benefits
bull How to implementndash What info to exchangendash Protecting privacyndash How to prioritize choicesndash How to prevent cheating
Destination 2
Destination 1
multiplepeeringpoints
Provider A
Provider B
64
Outbound Multihoming Goals
bull Redundancyndash Dynamic routing will failover to backup link
bull Performancendash Select provider with best performance per prefix
ndash Requires active probing
bull Costndash Select provider per prefix over time to minimize the
total financial cost
65
Inbound Traffic Control
bull More difficult no control over neighborsrsquo decisions
bull Three common techniques (previously discussed)ndash AS path prependingndash Communities and local preferencendash Prefix splitting
How does todayrsquos paper (MONET) control inbound traffic
66
How many links are enough
K upstream ISPs
Not much benefit beyond 4 ISPs
Akella et al ldquoPerformance Benefits of Multihomingrdquo SIGCOMM 2003
67
Problems with Multihoming in IPv4
bull Routing table growthndash Provider-based addressingndash Advertising prefix out multiple ISPs ndash canrsquot aggregate
bull Poor control over inbound trafficndash Existing mechanisms do not allow hosts to control
inbound traffic
68
GeorgiaTech
Internet Routing Overview
bull Intradomain (ie ldquointra-ASrdquo) routingbull Interdomain routing
Comcast
Abilene
ATampT Cogent
Autonomous Systems (ASes)
69
Configuration Problems ldquoAS 7007rdquoldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
UUNet
Florida InternetBarn
Sprint
70
Diagnosis and Troubleshootingldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
ldquoMicrosofts websites were offline for up to 23 hoursbecause of a [router] misconfigurationhellipit took nearly a day to determine what was wrong and undo the changesrdquo -- wiredcom January 25 2001
ldquoWorldCom Inchellipsuffered a widespread outage on its Internet backbone that affected roughly 20 percent of its US customer base The network problemshellipaffected millions of computer users worldwide A spokeswoman attributed the outage to a route table issue -- cnncom October 3 2002
A number of Covad customers went out from 5pm today due to supposedly a DDOS (distributed denial of service attack) on a key Level3 data center which later was described as a route leak (misconfiguration)ldquo
-- dslreportscom February 23 2004
71
Operator Mailing List (NANOG)Date Mon 18 Oct 2004 091515 -0700Subject Level 3 US east coast issues
Level 3 experiencing widespread unspecified routing issues on the US east coast Master ticket 1086844 Anyone have more specific information
Date Mon 18 Oct 2004 122034 -0400 (EDT)Subject Re Level 3 US east coast issues
Level 3 is currently experiencing a backbone outage causing routing instability and packet loss We are working to restore and will be sending hourly updateshellip
72
Operator Mailing List
0102030405060708090
Filtering RouteLeaks
RouteHijacks
RouteInstability
RoutingLoops
Blackholes
Occurr
ences o
ver
10 Y
ears
1995-1997 1998-2001 2001-2004
Note Only includes problems openly discussed on this list
Compare 83 power outages 1 fire
73
Routing Configuration
Ranking route selection
Dissemination internal route advertisement
Filtering route advertisement
Customer
Competitor
Primary
Backup
74
Internet Business Model (Simplified)
bull CustomerProvider One AS pays another for reachability to some set of destinations
bull ldquoSettlement-freerdquo Peering Bartering Two ASes exchange routes with one another
Provider
Peer
Customer
Preferences implemented with local preference manipulation
Destination
Pay to use
Get paid to use
Free to use
75
Peering Contracts Consistent Export
bull Rules of settlement-free peeringndash Advertise routes at all peering pointsndash Advertised routes must have equal ldquoAS path lengthrdquo
Sprint
ATampT
Enables ldquohot potatordquo routing
ldquoequally goodrdquoroutes
76
Consistent Export
bull Malicedeceptionbull iBGP signaling partitionbull Inconsistent export policy
neighbor 10123route-map PEER permit 10 set prepend 123
neighbor 10456route-map PEER permit 10 set prepend 123 123
Possible Causes
10123 456 1
10456 456 2
Neighbor AS Export
1 1 123
Export Clause Prepend
2 1 123 123
Two different Export Policies
77
Inconsistent Export in Practice
Feamster et al ldquoBorderGuard Detecting Cold Potatoes from Peersrdquo ACM IMC October 2004
78
Blackholes
Date Thu 18 Jul 2002 060510 -0400 (EDT)From Chad Oleary ltcolpoboxcomgtSubject Re problems with 701To ltnanogmeritedugt
Were starting to see the same issues with UUNet again Anyone elseseeing this Trying to reach Qwest
traceroute to 631461901 (631461901) 30 hops max 38 byte packets 1 esc-lp2-gwe-solutionscorpcom (631182201) 1167 ms 1163 ms 1142 ms 2 500Serial2-10GW1TPA2ALTERNET (1571301499) 1097 ms 1059 ms 1044 ms 3 161at-1-0-0XL4ATL1ALTERNET (1526381190) 13839 ms 14108 ms 16638 ms 4 0so-3-1-0XL2ATL5ALTERNET (152630238) 14370 ms 14587 ms 14553 ms 5 POS7-0BR2ATL5ALTERNET (1526382193) 13928 ms 14099 ms 14053 ms 6 7 hellip
79
Security
80
Security ldquoBogonrdquo Routes
Feamster et al ldquoAn Empirical Study of lsquoBogonrsquo Route Advertisementsrdquo ACM CCR January 2005
81
Spam Phishing etc
bull Unsolicited commercial emailbull As of about August 2008 estimates indicate that
about 95 of all email is spambull Common spam filtering techniques
ndash Content-based filtersndash DNS Blacklist (DNSBL) lookups Significant fraction of
todayrsquos DNS traffic
Can IP addresses from which spam is received be spoofed
82
Spam and Routing
83
Worms and Botnets
84
What is a Worm
bull Code that replicates and propagates across the networkndash Often carries a ldquopayloadrdquo
bull Usually spread via exploiting flaws in open servicesndash ldquoVirusesrdquo require user action to spread
bull First worm Robert Morris November 1988ndash 6-10 of all Internet hosts infected ()
bull Many more since but none on that scale until July 2001
85
Example Worm Code Red
bull Initial version July 13 2001
bull Exploited known ISAPI vulnerability in Microsoft IIS Web servers
bull 1st through 20th of each month spread20th through end of each month attack
bull Payload Web site defacementbull Scanning Random IP addressesbull Bug failure to seed random number generator
86
Code Red Revisions
bull Released July 19 2001
bull Payload flooding attack on wwwwhitehousegovndash Attack was mounted at the IP address of the Web site
bull Bug died after 20th of each month
bull Random number generator for IP scanning fixed
87
Code Red Host Infection Rate
Exponential infection rate
Measured using backscatter technique
88
Designing Fast-Spreading Worms
bull Hit-list scanningndash Time to infect first 10k hosts dominates infection timendash Solution Reconnaissance (stealthy scans etc)
bull Permutation scanningndash Observation Most scanning is redundantndash Idea Shared permutation of address space Start scanning
from own IP address Re-randomize when another infected machine is found
bull Internet-scale hit listsndash Flash worm complete infection within 30 seconds
89
Botnets
bull Bots Autonomous programs performing tasksbull Plenty of ldquobenignrdquo bots
ndash eg weatherbug
bull Botnets group of bots ndash Typically carries malicious connotationndash Large numbers of infected machinesndash Machines ldquoenlistedrdquo with infection vectors like worms
(last lecture)
bull Available for simultaneous control by a masterbull Size up to 350000 nodes (from todayrsquos paper)
90
ldquoRallyingrdquo the Botnet
bull Easy to combine worm backdoor functionalitybull Problem how to learn about successfully
infected machines
bull Optionsndash Emailndash Hard-coded email address
91
Botnet Control
bull Botnet master typically runs some IRC server on a well-known port (eg 6667)
bull Infected machine contacts botnet with pre-programmed DNS name (eg big-botde)
bull Dynamic DNS allows controller to move about freely
Infected Machine
DynamicDNS
BotnetController
(IRC server)
92
Some Defenses
93
Idea 1 Ingress Filtering
bull RFC 2827 Routers install filters to drop packets from networks that are not downstream
bull Feasible at edgesbull Difficult to configure closer to network ldquocorerdquo
20469207024 Internet
Drop all packets with source address other than 20469207024
94
Idea 2 uRPF Checks
bull Unicast Reverse Path Forwardingndash Cisco ldquoip verify unicast reverse-pathrdquo
bull Requires symmetric routing
Accept packet from interface only if forwarding table entry for source IP address matches ingress interface
100183
A10018
10015
101203
100181 24
10011 24
Strict Mode uRPF
Enabled
ldquoArdquo Routing TableDestination Next Hop1001024 Int 110018024 Int 2
100183 from wrong interface
95
Problems with uRPF
bull Asymmetric routing
96
S-BGP
bull Address-based PKI validate signaturesndash Authentication of
bull ownership for IP address blocks bull AS number bull an ASs identity and bull a BGP routers identity
ndash Use existing infrastructure (Internet registries etc)ndash Routing origination is digitally signedndash BGP updates are digitally signed
1048708 bull Route attestations A new optional BGP transitive path attribute
ndash carries digital signatures covering the routing information in updates
97
Practical Problems with S-BGP
bull Requires Public-Key Infrastructure
bull Lots of digital signatures to calculate and verifyndash Message overheadndash CPU overhead
bull Calculation expense is greatest when topology is changingndash Caching can help
bull Route aggregation is problematic (maybe thatrsquos OK)
bull Secure route withdrawals when link or node fails
bull Address ownership data out of date
bull Deployment
- Networking
- Goal of This Tutorial
- Todayrsquos Networks
- Business Models
- Billing for Internet Usage
- Net Neutrality
- Network Operations
- Point-of-Presence (PoP)
- Example Abilene Network Topology
- Another Example Backbone
- Internet Routing Overview
- Internet Routing Protocol BGP
- Two Flavors of BGP
- IPv4 Addresses Networks of Networks
- Pre-1994 Classful Addressing
- Classless Interdomain Routing (CIDR)
- Benefits of CIDR
- Growth of IP Prefixes
- 1994-1998 Linear Growth
- Around 2000 Fast Growth Resumes
- Fast growth resumes
- The Address Allocation Process
- Common Problems
- What can go wrong
- Measurement and Monitoring
- Passive vs Active Measurement
- Slide 27
- Passive Traffic Data Measurement
- Simple Network Management Protocol
- SNMP (Passive)
- Packet-level Monitoring
- Full Packet Capture (Passive)
- What is a flow
- Cisco NetFlow
- Flow Record Contents
- Aggregating Packets into Flows
- Reducing Measurement Overhead
- Packet Sampling for Flow Monitoring
- Sampling Flow-Level Sampling
- Two Main Approaches
- Packet Capture on High-Speed Links
- Characteristics of Packet Capture
- Data Measurement Repositories
- Multihoming and Traffic Engineering
- What is Multihoming
- Why Multihome
- Redundancy
- Performance
- Multihoming in IP Networks Today
- BGP or no
- Same Provider or Multiple
- Multihomed Stub One Link
- Multihomed Stub Multiple Links
- Multihomed Stub Multiple ISPs
- Multihomed Transit Network
- Interdomain Traffic Engineering
- Outbound Traffic Control
- Outbound Traffic Load Balancing
- Traffic Engineering Goals
- Managing Scale
- Achieving Predictability
- Challenges to Predictability
- Inter-AS Negotiation
- Outbound Multihoming Goals
- Inbound Traffic Control
- How many links are enough
- Problems with Multihoming in IPv4
- Slide 68
- Slide 69
- Diagnosis and Troubleshooting
- Operator Mailing List (NANOG)
- Operator Mailing List
- Routing Configuration
- Internet Business Model (Simplified)
- Peering Contracts Consistent Export
- Consistent Export
- Inconsistent Export in Practice
- Blackholes
- Security
- Security ldquoBogonrdquo Routes
- Spam Phishing etc
- Spam and Routing
- Worms and Botnets
- What is a Worm
- Example Worm Code Red
- Code Red Revisions
- Code Red Host Infection Rate
- Designing Fast-Spreading Worms
- Botnets
- ldquoRallyingrdquo the Botnet
- Botnet Control
- Some Defenses
- Idea 1 Ingress Filtering
- Idea 2 uRPF Checks
- Problems with uRPF
- S-BGP
- Practical Problems with S-BGP
-
12
Internet Routing Protocol BGP
Route Advertisement
Autonomous Systems (ASes)
Session
Traffic
Destination Next-hop AS Path1302070016
1302070016
19258989
6625025244
105782637
174hellip 2637
13
Two Flavors of BGP
bull External BGP (eBGP) exchanging routes between ASes
bull Internal BGP (iBGP) disseminating routes to external destinations among the routers within an AS
eBGPiBGP
Question Whatrsquos the difference between IGP and iBGP
14
IPv4 Addresses Networks of Networks
bull 32-bit number in ldquodotted-quadrdquo notation
ndash wwwccgatechedu --- 130207736
10000010 11001111 00000111 00100100
Network (16 bits) Host (16 bits)
130 207 7 36
bull Problem 232 addresses is a lot of table entries
bull Solution Routing based on network and host
ndash 1302070016 is a 16-bit prefix with 216 IP addresses
Topological Addressing
15
Pre-1994 Classful Addressing
Network ID Host ID
8 16
Class A
32
0
Class B 10
Class C 110
Multicast AddressesClass D 1110
Reserved for experimentsClass E 1111
24
8 blocks (eg MIT has 180008)
16 blocks (eg Georgia Tech has 1302070016)
24 blocks (eg ATampT Labs has 19220225024)
Simple Forwarding Address range specifies network ID length
16
Classless Interdomain Routing (CIDR)
IP Address 65142480 ldquoMaskrdquo 2552552520
01000001 00001110 11111000 00000000
11111111 11111111 11111100 00000000
Use two 32-bit numbers to represent a network Network number = IP address + Mask
Example BellSouth Prefix 6514248022
Address no longer specifies network ID rangeNew forwarding trick Longest Prefix Match
17
Benefits of CIDR
bull Efficiency Can allocate blocks of prefixes on a finer granularity
bull Hierarchy Prefixes can be aggregated into supernets (Not always done Typically not in fact)
Customer 1
Customer 2
ATampT Internet
1220249024
1220231024120008
18
Growth of IP Prefixes
19
1994-1998 Linear Growth
bull About 10000 new entries per yearbull In theory less instability at the edges (why)
Source Geoff Huston
20
Around 2000 Fast Growth Resumes
Claim remaining 8s will be exhausted within the next 5-10 years
T Hain ldquoA Pragmatic Report on IPv4 Address Space Consumptionrdquo Cisco IPJ September 2005
21
Fast growth resumes
Rapid growth in routing tables
Dot-Bomb Hiccup
Significant contributor Multihoming
Source Geoff Huston
22
The Address Allocation Process
bull Allocation policies of RIRs affect pressure on IPv4 address space
IANA
AfriNIC APNIC ARIN LACNIC RIPE
httpwwwianaorgassignmentsipv4-address-space
Georgia Tech
23
Common Problems
bull Diagnosis and troubleshooting (hence measurement)bull Traffic engineeringbull Securitybull Design and capacity planning
24
What can go wrong
Two-thirds of the problems are caused by configuration of the routing protocol
Some downtime is very hard to protect againsthellip
25
Measurement and Monitoring
26
Passive vs Active Measurement
bull Passive Measurement Collection of packets flow statistics of traffic that is already flowing on the networkndash Packet tracesndash Flow statisticsndash Application-level logs
bull Active Measurement Inject ldquoprobingrdquo traffic to measure various characteristicsndash Traceroutendash Pingndash Application-level probes (eg Web downloads)
27
Billing for Internet Usage
bull 95th Percentile billingndash Customer network pays for ldquocommitted information
raterdquo (CIR)ndash Throughput measured every 5 minutes (typically with
SNMP flow statistics also can be used for billing)ndash Customer billed based on 95th percentile
28
Passive Traffic Data Measurement
bull SNMP bytepacket counts everywhere
bull Packet monitoring selected locations
bull Flow monitoring typically at edges (if possible)ndash Direct computation of the traffic matrixndash Input to denial-of-service attack detection
bull Deep Packet Inspection also at edge where possible
29
Simple Network Management Protocol
bull Management Information Base (MIB)ndash Information storendash Unique variables named by
OIDsndash Accessed with SNMP
bull Specific MIBs for bytepacket counts (per link)
Manager Agent
SNMP
DB
ManagedObjects
30
SNMP (Passive)
bull Advantage ubiquitousndash Supported on all networking equipmentndash Multiple products for polling and analyzing data
bull Disadvantages ndash Coarse granularityndash Cannot express complex queries on the datandash Unreliable delivery of the data using UDP
bull Utilityndash Link utilization (billing)ndash Traffic matrix inference
31
Packet-level Monitoring
bull Passive monitoring to collect full packet contents (or at least headers)
bull Advantages lots of detailed informationndash Precise timing informationndash Information in packet headers
bull Disadvantages overheadndash Hard to keep up with high-speed linksndash Often requires a separate monitoring device
32
Full Packet Capture (Passive)
Example Georgia Tech OC3Mon
bull Rack-mounted PCbull Optical splitterbull Data Acquisition and
Generation (DAG) card
Source endacecom
33
What is a flow
bull Source IP addressbull Destination IP addressbull Source portbull Destination portbull Layer 3 protocol typebull TOS byte (DSCP)bull Input logical interface (ifIndex)
34
Cisco NetFlow
bull Basic output ldquoFlow recordrdquondash Most common version is v5
bull Current version (9) is being standardized in the IETF (template-based)ndash More flexible record formatndash Much easier to add new flow record types
Core Network
Collection and Aggregation
Collector (PC)Approximately 1500 bytes
20-50 flow recordsSent more frequently if traffic increases
35
Flow Record Contents
bull Source and Destination IP address and portbull Packet and byte countsbull Start and end timesbull ToS TCP flags
Basic information about the flowhellip
hellipplus information related to routing
bull Next-hop IP addressbull Source and destination ASbull Source and destination prefix
36
flow 1 flow 2 flow 3 flow 4
Aggregating Packets into Flows
bull Criteria 1 Set of packets that ldquobelong togetherrdquondash Sourcedestination IP addresses and port numbersndash Same protocol ToS bits hellip ndash Same inputoutput interfaces at a router (if known)
bull Criteria 2 Packets that are ldquocloserdquo together in timendash Maximum inter-packet spacing (eg 15 sec 30 sec)ndash Example flows 2 and 4 are different flows due to time
37
Reducing Measurement Overhead
bull Filtering on interfacendash destination prefix for a customerndash port number for an application (eg 80 for Web)
bull Sampling before insertion into flow cachendash Random deterministic or hash-based samplingndash 1-out-of-n or stratified based on packetflow sizendash Two types packet-level and flow-level
bull Aggregation after cache evictionndash packetsflows with same next-hop ASndash packetsflows destined to a particular service
38
Packet Sampling for Flow Monitoring
bull Packet sampling before flow creation (Sampled Netflow)ndash 1-out-of-m sampling of individual packets (eg m=100)ndash Create of flow records over the sampled packets
bull Reducing overheadndash Avoid per-packet overhead on (m-1)m packetsndash Avoid creating records for a large number of small flows
bull Increasing overhead (in some cases)ndash May split some long transfers into multiple flow records ndash hellip due to larger time gaps between successive packets
time
not sampled
two flowstimeout
39
Sampling Flow-Level Sampling
bull Sampling of flow records evicted from flow cachendash When evicting flows from table or when analyzing flows
bull Stratified sampling to put weight on ldquoheavyrdquo flowsndash Select all long flows and sample the short flows
bull Reduces the number of flow records ndash Still measures the vast majority of the traffic
Flow 1 40 bytesFlow 2 15580 bytesFlow 3 8196 bytesFlow 4 5350789 bytesFlow 5 532 bytesFlow 6 7432 bytes
sample with 100 probability
sample with 01 probability
sample with 10 probability
40
Two Main Approaches
bull Packet-level Monitoringndash Keep packet-level statisticsndash Examine (and potentially log) variety of packet-level
statistics Essentially anything in the packetndash Timing
bull Flow-level Monitoringndash Monitor packet-by-packet (though sometimes
sampled)ndash Keep aggregate statistics on a flow
41
Packet Capture on High-Speed Links
Example Georgia Tech ldquoOC3Monrdquo
bull Rack-mounted PCbull Optical splitterbull Data Acquisition and
Generation (DAG) card
Source endacecom
42
Characteristics of Packet Capture
bull Allows inspection on every packet on 10G links
bull Disadvantagesndash Costlyndash Requires splitting optical fibersndash Must be able to filterstore data
43
Data Measurement Repositories
bull AbileneInternet 2 Observatoryndash Configuration examplesndash SNMP datandash ISIS BGP routing data NetFlow traffic data
bull RouteViewsndash BGP updatesndash BGP table snapshots
44
Multihoming and Traffic Engineering
45
What is Multihoming
bull The use of redundant network links for the purposes of external connectivity
bull Can be achieved at many layers of the protocol stack and many places in the networkndash Multiple network interfaces in a PC
ndash An ISP with multiple upstream interfaces
bull Can refer to having multiple connections tondash The same ISP
ndash Multiple ISPs
46
Why Multihome
bull Redundancybull Availabilitybull Performancebull Cost
Interdomain traffic engineering the process by which a multihomed network configures its
network to achieve these goals
47
Redundancy
bull Maintain connectivity in the face ofndash Physical connectivity problems (fiber cut device
failures etc)ndash Failures in upstream ISP
48
Performance
bull Use multiple network links at once to achieve higher throughput than just over a single link
bull Allows incoming traffic to be load-balanced
70 of traffic30 of traffic
49
Multihoming in IP Networks Today
bull Stub AS no transit service for other ASesndash No need to use BGP
bull Multi-homed stub AS has connectivity to multiple immediate upstream ISPsndash Need BGPndash No need for a public AS numberndash No need for IP prefix allocation
bull Multi-homed transit AS connectivity to multiple ASes and transit servicendash Need BGP public AS number IP prefix allocation
50
BGP or no
bull Advantages of static routingndash Cheapersmaller routers (less true nowadays)ndash Simpler to configure
bull Advantages of BGPndash More control of your destiny (have providers stop
announcing you)ndash Fastermore intelligent selection of where to send
outbound packetsndash Better debugging of net problems (you can see the
Internet topology now)
51
Same Provider or Multiple
bull If your provider is reliable and fast and affordably and offers good tech-support you may want to multi-home initially to them via some backup path (slow is better than dead)
bull Eventually yoursquoll want to multi-home to different providers to avoid failure modes due to one providerrsquos architecture decisions
52
Multihomed Stub One Link
bull Downstream ISPrsquos routers configure default (ldquostaticrdquo) routes pointing to border router
bull Upstream ISP advertises reachability
Upstream ISP
Multiple links between same pair of routers
Default routes to ldquoborderrdquo
ldquoStubrdquoISP
53
Multihomed Stub Multiple Links
bull Use BGP to share loadbull Use private AS number (why is this OK)bull As before upstream ISP advertises prefix
Upstream ISP
Multiple links to different upstream routers
ldquoStubrdquoISP
Internal routing for ldquohot potatordquo
BGP for load balance at edge
54
Multihomed Stub Multiple ISPs
bull Many possibilitiesndash Load sharingndash Primary-backupndash Selective use of different ISPs
bull Requires BGP public AS number etc
ldquoStubrdquoISP
Upstream
ISP 1
Upstream
ISP 2
55
Multihomed Transit Network
bull BGP everywherebull Incoming and outcoming trafficbull Challenge balancing load on intradomain and egress
links given an offered traffic load
TransitISP
ISP 1
ISP 2
ISP 3
56
Interdomain Traffic Engineering
bull The process by which a network operator configures the network to achievendash Traffic load balancendash Redundancy (primarybackup) etc
bull Two tasksndash Outbound traffic controlndash Inbound traffic control
bull Key Problems Predictability and Scalability
57
Outbound Traffic Control
bull Easier to control than inbound trafficndash Destination-based routing sender determines where
the packets go
bull Control over next-hop AS onlyndash Cannot control selection of the entire path
Provider 1 Provider 2
Control with local preference
58
Outbound Traffic Load Balancingbull Control routes to provider per-prefix
ndash Assign local preference across destination prefixesndash Change the local preference assignments over time
bull Useful inputs to load balancingndash End-to-end path performance datandash Outbound traffic statistics per destination prefix
bull Challenge Getting from traffic volumes to groups of prefixes that should be assigned to each link
Premise of ldquointelligent route controlrdquo preoducts
59
Traffic Engineering Goals
bull Predictabilityndash Ensure the BGP decision process is deterministicndash Assume that BGP updates are (relatively) stable
bull Limit overhead introduced by routing changesndash Minimize frequency of changes to routing policiesndash Limit number of prefixes affected by changes
bull Limit impact on how traffic enters the networkndash Avoid new routes that might change neighborrsquos mindndash Select route with same attributes or at least path length
60
Managing Scalebull Destination prefixes
ndash More than 90000 destination prefixes
bull Donrsquot want to have per-prefix routing policies
ndash Small fraction of prefixes contribute most of the traffic
bull Focus on the small number of heavy hitters
ndash Define routing policies for selected prefixes
bull Routing choicesndash About 27000 unique ldquorouting choicesrdquo
bull Help in reducing the scale of the problem
ndash Small fraction of ldquorouting choicesrdquo contribute most traffic
bull Focus on the very small number of ldquorouting choicesrdquo
ndash Define routing policies on common attributes
61
Achieving Predictability
bull Route prediction with static analysisndash Helpful to know effects before deploymentndash Static analysis can help
TopologyBGP policy
configuration
eBGP routes
Offered traffic
BGP routingmodel
Flow of traffic through the network
62
Challenges to Predictabilitybull For transit ISPs effects on incoming traffic
ndash Lack of coordination strikes again
63
ldquoHot Potatordquo routing
Inter-AS Negotiation
bull Coordination aids predictabilityndash Negotiate where to sendndash Inbound and outboundndash Mutual benefits
bull How to implementndash What info to exchangendash Protecting privacyndash How to prioritize choicesndash How to prevent cheating
Destination 2
Destination 1
multiplepeeringpoints
Provider A
Provider B
64
Outbound Multihoming Goals
bull Redundancyndash Dynamic routing will failover to backup link
bull Performancendash Select provider with best performance per prefix
ndash Requires active probing
bull Costndash Select provider per prefix over time to minimize the
total financial cost
65
Inbound Traffic Control
bull More difficult no control over neighborsrsquo decisions
bull Three common techniques (previously discussed)ndash AS path prependingndash Communities and local preferencendash Prefix splitting
How does todayrsquos paper (MONET) control inbound traffic
66
How many links are enough
K upstream ISPs
Not much benefit beyond 4 ISPs
Akella et al ldquoPerformance Benefits of Multihomingrdquo SIGCOMM 2003
67
Problems with Multihoming in IPv4
bull Routing table growthndash Provider-based addressingndash Advertising prefix out multiple ISPs ndash canrsquot aggregate
bull Poor control over inbound trafficndash Existing mechanisms do not allow hosts to control
inbound traffic
68
GeorgiaTech
Internet Routing Overview
bull Intradomain (ie ldquointra-ASrdquo) routingbull Interdomain routing
Comcast
Abilene
ATampT Cogent
Autonomous Systems (ASes)
69
Configuration Problems ldquoAS 7007rdquoldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
UUNet
Florida InternetBarn
Sprint
70
Diagnosis and Troubleshootingldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
ldquoMicrosofts websites were offline for up to 23 hoursbecause of a [router] misconfigurationhellipit took nearly a day to determine what was wrong and undo the changesrdquo -- wiredcom January 25 2001
ldquoWorldCom Inchellipsuffered a widespread outage on its Internet backbone that affected roughly 20 percent of its US customer base The network problemshellipaffected millions of computer users worldwide A spokeswoman attributed the outage to a route table issue -- cnncom October 3 2002
A number of Covad customers went out from 5pm today due to supposedly a DDOS (distributed denial of service attack) on a key Level3 data center which later was described as a route leak (misconfiguration)ldquo
-- dslreportscom February 23 2004
71
Operator Mailing List (NANOG)Date Mon 18 Oct 2004 091515 -0700Subject Level 3 US east coast issues
Level 3 experiencing widespread unspecified routing issues on the US east coast Master ticket 1086844 Anyone have more specific information
Date Mon 18 Oct 2004 122034 -0400 (EDT)Subject Re Level 3 US east coast issues
Level 3 is currently experiencing a backbone outage causing routing instability and packet loss We are working to restore and will be sending hourly updateshellip
72
Operator Mailing List
0102030405060708090
Filtering RouteLeaks
RouteHijacks
RouteInstability
RoutingLoops
Blackholes
Occurr
ences o
ver
10 Y
ears
1995-1997 1998-2001 2001-2004
Note Only includes problems openly discussed on this list
Compare 83 power outages 1 fire
73
Routing Configuration
Ranking route selection
Dissemination internal route advertisement
Filtering route advertisement
Customer
Competitor
Primary
Backup
74
Internet Business Model (Simplified)
bull CustomerProvider One AS pays another for reachability to some set of destinations
bull ldquoSettlement-freerdquo Peering Bartering Two ASes exchange routes with one another
Provider
Peer
Customer
Preferences implemented with local preference manipulation
Destination
Pay to use
Get paid to use
Free to use
75
Peering Contracts Consistent Export
bull Rules of settlement-free peeringndash Advertise routes at all peering pointsndash Advertised routes must have equal ldquoAS path lengthrdquo
Sprint
ATampT
Enables ldquohot potatordquo routing
ldquoequally goodrdquoroutes
76
Consistent Export
bull Malicedeceptionbull iBGP signaling partitionbull Inconsistent export policy
neighbor 10123route-map PEER permit 10 set prepend 123
neighbor 10456route-map PEER permit 10 set prepend 123 123
Possible Causes
10123 456 1
10456 456 2
Neighbor AS Export
1 1 123
Export Clause Prepend
2 1 123 123
Two different Export Policies
77
Inconsistent Export in Practice
Feamster et al ldquoBorderGuard Detecting Cold Potatoes from Peersrdquo ACM IMC October 2004
78
Blackholes
Date Thu 18 Jul 2002 060510 -0400 (EDT)From Chad Oleary ltcolpoboxcomgtSubject Re problems with 701To ltnanogmeritedugt
Were starting to see the same issues with UUNet again Anyone elseseeing this Trying to reach Qwest
traceroute to 631461901 (631461901) 30 hops max 38 byte packets 1 esc-lp2-gwe-solutionscorpcom (631182201) 1167 ms 1163 ms 1142 ms 2 500Serial2-10GW1TPA2ALTERNET (1571301499) 1097 ms 1059 ms 1044 ms 3 161at-1-0-0XL4ATL1ALTERNET (1526381190) 13839 ms 14108 ms 16638 ms 4 0so-3-1-0XL2ATL5ALTERNET (152630238) 14370 ms 14587 ms 14553 ms 5 POS7-0BR2ATL5ALTERNET (1526382193) 13928 ms 14099 ms 14053 ms 6 7 hellip
79
Security
80
Security ldquoBogonrdquo Routes
Feamster et al ldquoAn Empirical Study of lsquoBogonrsquo Route Advertisementsrdquo ACM CCR January 2005
81
Spam Phishing etc
bull Unsolicited commercial emailbull As of about August 2008 estimates indicate that
about 95 of all email is spambull Common spam filtering techniques
ndash Content-based filtersndash DNS Blacklist (DNSBL) lookups Significant fraction of
todayrsquos DNS traffic
Can IP addresses from which spam is received be spoofed
82
Spam and Routing
83
Worms and Botnets
84
What is a Worm
bull Code that replicates and propagates across the networkndash Often carries a ldquopayloadrdquo
bull Usually spread via exploiting flaws in open servicesndash ldquoVirusesrdquo require user action to spread
bull First worm Robert Morris November 1988ndash 6-10 of all Internet hosts infected ()
bull Many more since but none on that scale until July 2001
85
Example Worm Code Red
bull Initial version July 13 2001
bull Exploited known ISAPI vulnerability in Microsoft IIS Web servers
bull 1st through 20th of each month spread20th through end of each month attack
bull Payload Web site defacementbull Scanning Random IP addressesbull Bug failure to seed random number generator
86
Code Red Revisions
bull Released July 19 2001
bull Payload flooding attack on wwwwhitehousegovndash Attack was mounted at the IP address of the Web site
bull Bug died after 20th of each month
bull Random number generator for IP scanning fixed
87
Code Red Host Infection Rate
Exponential infection rate
Measured using backscatter technique
88
Designing Fast-Spreading Worms
bull Hit-list scanningndash Time to infect first 10k hosts dominates infection timendash Solution Reconnaissance (stealthy scans etc)
bull Permutation scanningndash Observation Most scanning is redundantndash Idea Shared permutation of address space Start scanning
from own IP address Re-randomize when another infected machine is found
bull Internet-scale hit listsndash Flash worm complete infection within 30 seconds
89
Botnets
bull Bots Autonomous programs performing tasksbull Plenty of ldquobenignrdquo bots
ndash eg weatherbug
bull Botnets group of bots ndash Typically carries malicious connotationndash Large numbers of infected machinesndash Machines ldquoenlistedrdquo with infection vectors like worms
(last lecture)
bull Available for simultaneous control by a masterbull Size up to 350000 nodes (from todayrsquos paper)
90
ldquoRallyingrdquo the Botnet
bull Easy to combine worm backdoor functionalitybull Problem how to learn about successfully
infected machines
bull Optionsndash Emailndash Hard-coded email address
91
Botnet Control
bull Botnet master typically runs some IRC server on a well-known port (eg 6667)
bull Infected machine contacts botnet with pre-programmed DNS name (eg big-botde)
bull Dynamic DNS allows controller to move about freely
Infected Machine
DynamicDNS
BotnetController
(IRC server)
92
Some Defenses
93
Idea 1 Ingress Filtering
bull RFC 2827 Routers install filters to drop packets from networks that are not downstream
bull Feasible at edgesbull Difficult to configure closer to network ldquocorerdquo
20469207024 Internet
Drop all packets with source address other than 20469207024
94
Idea 2 uRPF Checks
bull Unicast Reverse Path Forwardingndash Cisco ldquoip verify unicast reverse-pathrdquo
bull Requires symmetric routing
Accept packet from interface only if forwarding table entry for source IP address matches ingress interface
100183
A10018
10015
101203
100181 24
10011 24
Strict Mode uRPF
Enabled
ldquoArdquo Routing TableDestination Next Hop1001024 Int 110018024 Int 2
100183 from wrong interface
95
Problems with uRPF
bull Asymmetric routing
96
S-BGP
bull Address-based PKI validate signaturesndash Authentication of
bull ownership for IP address blocks bull AS number bull an ASs identity and bull a BGP routers identity
ndash Use existing infrastructure (Internet registries etc)ndash Routing origination is digitally signedndash BGP updates are digitally signed
1048708 bull Route attestations A new optional BGP transitive path attribute
ndash carries digital signatures covering the routing information in updates
97
Practical Problems with S-BGP
bull Requires Public-Key Infrastructure
bull Lots of digital signatures to calculate and verifyndash Message overheadndash CPU overhead
bull Calculation expense is greatest when topology is changingndash Caching can help
bull Route aggregation is problematic (maybe thatrsquos OK)
bull Secure route withdrawals when link or node fails
bull Address ownership data out of date
bull Deployment
- Networking
- Goal of This Tutorial
- Todayrsquos Networks
- Business Models
- Billing for Internet Usage
- Net Neutrality
- Network Operations
- Point-of-Presence (PoP)
- Example Abilene Network Topology
- Another Example Backbone
- Internet Routing Overview
- Internet Routing Protocol BGP
- Two Flavors of BGP
- IPv4 Addresses Networks of Networks
- Pre-1994 Classful Addressing
- Classless Interdomain Routing (CIDR)
- Benefits of CIDR
- Growth of IP Prefixes
- 1994-1998 Linear Growth
- Around 2000 Fast Growth Resumes
- Fast growth resumes
- The Address Allocation Process
- Common Problems
- What can go wrong
- Measurement and Monitoring
- Passive vs Active Measurement
- Slide 27
- Passive Traffic Data Measurement
- Simple Network Management Protocol
- SNMP (Passive)
- Packet-level Monitoring
- Full Packet Capture (Passive)
- What is a flow
- Cisco NetFlow
- Flow Record Contents
- Aggregating Packets into Flows
- Reducing Measurement Overhead
- Packet Sampling for Flow Monitoring
- Sampling Flow-Level Sampling
- Two Main Approaches
- Packet Capture on High-Speed Links
- Characteristics of Packet Capture
- Data Measurement Repositories
- Multihoming and Traffic Engineering
- What is Multihoming
- Why Multihome
- Redundancy
- Performance
- Multihoming in IP Networks Today
- BGP or no
- Same Provider or Multiple
- Multihomed Stub One Link
- Multihomed Stub Multiple Links
- Multihomed Stub Multiple ISPs
- Multihomed Transit Network
- Interdomain Traffic Engineering
- Outbound Traffic Control
- Outbound Traffic Load Balancing
- Traffic Engineering Goals
- Managing Scale
- Achieving Predictability
- Challenges to Predictability
- Inter-AS Negotiation
- Outbound Multihoming Goals
- Inbound Traffic Control
- How many links are enough
- Problems with Multihoming in IPv4
- Slide 68
- Slide 69
- Diagnosis and Troubleshooting
- Operator Mailing List (NANOG)
- Operator Mailing List
- Routing Configuration
- Internet Business Model (Simplified)
- Peering Contracts Consistent Export
- Consistent Export
- Inconsistent Export in Practice
- Blackholes
- Security
- Security ldquoBogonrdquo Routes
- Spam Phishing etc
- Spam and Routing
- Worms and Botnets
- What is a Worm
- Example Worm Code Red
- Code Red Revisions
- Code Red Host Infection Rate
- Designing Fast-Spreading Worms
- Botnets
- ldquoRallyingrdquo the Botnet
- Botnet Control
- Some Defenses
- Idea 1 Ingress Filtering
- Idea 2 uRPF Checks
- Problems with uRPF
- S-BGP
- Practical Problems with S-BGP
-
13
Two Flavors of BGP
bull External BGP (eBGP) exchanging routes between ASes
bull Internal BGP (iBGP) disseminating routes to external destinations among the routers within an AS
eBGPiBGP
Question Whatrsquos the difference between IGP and iBGP
14
IPv4 Addresses Networks of Networks
bull 32-bit number in ldquodotted-quadrdquo notation
ndash wwwccgatechedu --- 130207736
10000010 11001111 00000111 00100100
Network (16 bits) Host (16 bits)
130 207 7 36
bull Problem 232 addresses is a lot of table entries
bull Solution Routing based on network and host
ndash 1302070016 is a 16-bit prefix with 216 IP addresses
Topological Addressing
15
Pre-1994 Classful Addressing
Network ID Host ID
8 16
Class A
32
0
Class B 10
Class C 110
Multicast AddressesClass D 1110
Reserved for experimentsClass E 1111
24
8 blocks (eg MIT has 180008)
16 blocks (eg Georgia Tech has 1302070016)
24 blocks (eg ATampT Labs has 19220225024)
Simple Forwarding Address range specifies network ID length
16
Classless Interdomain Routing (CIDR)
IP Address 65142480 ldquoMaskrdquo 2552552520
01000001 00001110 11111000 00000000
11111111 11111111 11111100 00000000
Use two 32-bit numbers to represent a network Network number = IP address + Mask
Example BellSouth Prefix 6514248022
Address no longer specifies network ID rangeNew forwarding trick Longest Prefix Match
17
Benefits of CIDR
bull Efficiency Can allocate blocks of prefixes on a finer granularity
bull Hierarchy Prefixes can be aggregated into supernets (Not always done Typically not in fact)
Customer 1
Customer 2
ATampT Internet
1220249024
1220231024120008
18
Growth of IP Prefixes
19
1994-1998 Linear Growth
bull About 10000 new entries per yearbull In theory less instability at the edges (why)
Source Geoff Huston
20
Around 2000 Fast Growth Resumes
Claim remaining 8s will be exhausted within the next 5-10 years
T Hain ldquoA Pragmatic Report on IPv4 Address Space Consumptionrdquo Cisco IPJ September 2005
21
Fast growth resumes
Rapid growth in routing tables
Dot-Bomb Hiccup
Significant contributor Multihoming
Source Geoff Huston
22
The Address Allocation Process
bull Allocation policies of RIRs affect pressure on IPv4 address space
IANA
AfriNIC APNIC ARIN LACNIC RIPE
httpwwwianaorgassignmentsipv4-address-space
Georgia Tech
23
Common Problems
bull Diagnosis and troubleshooting (hence measurement)bull Traffic engineeringbull Securitybull Design and capacity planning
24
What can go wrong
Two-thirds of the problems are caused by configuration of the routing protocol
Some downtime is very hard to protect againsthellip
25
Measurement and Monitoring
26
Passive vs Active Measurement
bull Passive Measurement Collection of packets flow statistics of traffic that is already flowing on the networkndash Packet tracesndash Flow statisticsndash Application-level logs
bull Active Measurement Inject ldquoprobingrdquo traffic to measure various characteristicsndash Traceroutendash Pingndash Application-level probes (eg Web downloads)
27
Billing for Internet Usage
bull 95th Percentile billingndash Customer network pays for ldquocommitted information
raterdquo (CIR)ndash Throughput measured every 5 minutes (typically with
SNMP flow statistics also can be used for billing)ndash Customer billed based on 95th percentile
28
Passive Traffic Data Measurement
bull SNMP bytepacket counts everywhere
bull Packet monitoring selected locations
bull Flow monitoring typically at edges (if possible)ndash Direct computation of the traffic matrixndash Input to denial-of-service attack detection
bull Deep Packet Inspection also at edge where possible
29
Simple Network Management Protocol
bull Management Information Base (MIB)ndash Information storendash Unique variables named by
OIDsndash Accessed with SNMP
bull Specific MIBs for bytepacket counts (per link)
Manager Agent
SNMP
DB
ManagedObjects
30
SNMP (Passive)
bull Advantage ubiquitousndash Supported on all networking equipmentndash Multiple products for polling and analyzing data
bull Disadvantages ndash Coarse granularityndash Cannot express complex queries on the datandash Unreliable delivery of the data using UDP
bull Utilityndash Link utilization (billing)ndash Traffic matrix inference
31
Packet-level Monitoring
bull Passive monitoring to collect full packet contents (or at least headers)
bull Advantages lots of detailed informationndash Precise timing informationndash Information in packet headers
bull Disadvantages overheadndash Hard to keep up with high-speed linksndash Often requires a separate monitoring device
32
Full Packet Capture (Passive)
Example Georgia Tech OC3Mon
bull Rack-mounted PCbull Optical splitterbull Data Acquisition and
Generation (DAG) card
Source endacecom
33
What is a flow
bull Source IP addressbull Destination IP addressbull Source portbull Destination portbull Layer 3 protocol typebull TOS byte (DSCP)bull Input logical interface (ifIndex)
34
Cisco NetFlow
bull Basic output ldquoFlow recordrdquondash Most common version is v5
bull Current version (9) is being standardized in the IETF (template-based)ndash More flexible record formatndash Much easier to add new flow record types
Core Network
Collection and Aggregation
Collector (PC)Approximately 1500 bytes
20-50 flow recordsSent more frequently if traffic increases
35
Flow Record Contents
bull Source and Destination IP address and portbull Packet and byte countsbull Start and end timesbull ToS TCP flags
Basic information about the flowhellip
hellipplus information related to routing
bull Next-hop IP addressbull Source and destination ASbull Source and destination prefix
36
flow 1 flow 2 flow 3 flow 4
Aggregating Packets into Flows
bull Criteria 1 Set of packets that ldquobelong togetherrdquondash Sourcedestination IP addresses and port numbersndash Same protocol ToS bits hellip ndash Same inputoutput interfaces at a router (if known)
bull Criteria 2 Packets that are ldquocloserdquo together in timendash Maximum inter-packet spacing (eg 15 sec 30 sec)ndash Example flows 2 and 4 are different flows due to time
37
Reducing Measurement Overhead
bull Filtering on interfacendash destination prefix for a customerndash port number for an application (eg 80 for Web)
bull Sampling before insertion into flow cachendash Random deterministic or hash-based samplingndash 1-out-of-n or stratified based on packetflow sizendash Two types packet-level and flow-level
bull Aggregation after cache evictionndash packetsflows with same next-hop ASndash packetsflows destined to a particular service
38
Packet Sampling for Flow Monitoring
bull Packet sampling before flow creation (Sampled Netflow)ndash 1-out-of-m sampling of individual packets (eg m=100)ndash Create of flow records over the sampled packets
bull Reducing overheadndash Avoid per-packet overhead on (m-1)m packetsndash Avoid creating records for a large number of small flows
bull Increasing overhead (in some cases)ndash May split some long transfers into multiple flow records ndash hellip due to larger time gaps between successive packets
time
not sampled
two flowstimeout
39
Sampling Flow-Level Sampling
bull Sampling of flow records evicted from flow cachendash When evicting flows from table or when analyzing flows
bull Stratified sampling to put weight on ldquoheavyrdquo flowsndash Select all long flows and sample the short flows
bull Reduces the number of flow records ndash Still measures the vast majority of the traffic
Flow 1 40 bytesFlow 2 15580 bytesFlow 3 8196 bytesFlow 4 5350789 bytesFlow 5 532 bytesFlow 6 7432 bytes
sample with 100 probability
sample with 01 probability
sample with 10 probability
40
Two Main Approaches
bull Packet-level Monitoringndash Keep packet-level statisticsndash Examine (and potentially log) variety of packet-level
statistics Essentially anything in the packetndash Timing
bull Flow-level Monitoringndash Monitor packet-by-packet (though sometimes
sampled)ndash Keep aggregate statistics on a flow
41
Packet Capture on High-Speed Links
Example Georgia Tech ldquoOC3Monrdquo
bull Rack-mounted PCbull Optical splitterbull Data Acquisition and
Generation (DAG) card
Source endacecom
42
Characteristics of Packet Capture
bull Allows inspection on every packet on 10G links
bull Disadvantagesndash Costlyndash Requires splitting optical fibersndash Must be able to filterstore data
43
Data Measurement Repositories
bull AbileneInternet 2 Observatoryndash Configuration examplesndash SNMP datandash ISIS BGP routing data NetFlow traffic data
bull RouteViewsndash BGP updatesndash BGP table snapshots
44
Multihoming and Traffic Engineering
45
What is Multihoming
bull The use of redundant network links for the purposes of external connectivity
bull Can be achieved at many layers of the protocol stack and many places in the networkndash Multiple network interfaces in a PC
ndash An ISP with multiple upstream interfaces
bull Can refer to having multiple connections tondash The same ISP
ndash Multiple ISPs
46
Why Multihome
bull Redundancybull Availabilitybull Performancebull Cost
Interdomain traffic engineering the process by which a multihomed network configures its
network to achieve these goals
47
Redundancy
bull Maintain connectivity in the face ofndash Physical connectivity problems (fiber cut device
failures etc)ndash Failures in upstream ISP
48
Performance
bull Use multiple network links at once to achieve higher throughput than just over a single link
bull Allows incoming traffic to be load-balanced
70 of traffic30 of traffic
49
Multihoming in IP Networks Today
bull Stub AS no transit service for other ASesndash No need to use BGP
bull Multi-homed stub AS has connectivity to multiple immediate upstream ISPsndash Need BGPndash No need for a public AS numberndash No need for IP prefix allocation
bull Multi-homed transit AS connectivity to multiple ASes and transit servicendash Need BGP public AS number IP prefix allocation
50
BGP or no
bull Advantages of static routingndash Cheapersmaller routers (less true nowadays)ndash Simpler to configure
bull Advantages of BGPndash More control of your destiny (have providers stop
announcing you)ndash Fastermore intelligent selection of where to send
outbound packetsndash Better debugging of net problems (you can see the
Internet topology now)
51
Same Provider or Multiple
bull If your provider is reliable and fast and affordably and offers good tech-support you may want to multi-home initially to them via some backup path (slow is better than dead)
bull Eventually yoursquoll want to multi-home to different providers to avoid failure modes due to one providerrsquos architecture decisions
52
Multihomed Stub One Link
bull Downstream ISPrsquos routers configure default (ldquostaticrdquo) routes pointing to border router
bull Upstream ISP advertises reachability
Upstream ISP
Multiple links between same pair of routers
Default routes to ldquoborderrdquo
ldquoStubrdquoISP
53
Multihomed Stub Multiple Links
bull Use BGP to share loadbull Use private AS number (why is this OK)bull As before upstream ISP advertises prefix
Upstream ISP
Multiple links to different upstream routers
ldquoStubrdquoISP
Internal routing for ldquohot potatordquo
BGP for load balance at edge
54
Multihomed Stub Multiple ISPs
bull Many possibilitiesndash Load sharingndash Primary-backupndash Selective use of different ISPs
bull Requires BGP public AS number etc
ldquoStubrdquoISP
Upstream
ISP 1
Upstream
ISP 2
55
Multihomed Transit Network
bull BGP everywherebull Incoming and outcoming trafficbull Challenge balancing load on intradomain and egress
links given an offered traffic load
TransitISP
ISP 1
ISP 2
ISP 3
56
Interdomain Traffic Engineering
bull The process by which a network operator configures the network to achievendash Traffic load balancendash Redundancy (primarybackup) etc
bull Two tasksndash Outbound traffic controlndash Inbound traffic control
bull Key Problems Predictability and Scalability
57
Outbound Traffic Control
bull Easier to control than inbound trafficndash Destination-based routing sender determines where
the packets go
bull Control over next-hop AS onlyndash Cannot control selection of the entire path
Provider 1 Provider 2
Control with local preference
58
Outbound Traffic Load Balancingbull Control routes to provider per-prefix
ndash Assign local preference across destination prefixesndash Change the local preference assignments over time
bull Useful inputs to load balancingndash End-to-end path performance datandash Outbound traffic statistics per destination prefix
bull Challenge Getting from traffic volumes to groups of prefixes that should be assigned to each link
Premise of ldquointelligent route controlrdquo preoducts
59
Traffic Engineering Goals
bull Predictabilityndash Ensure the BGP decision process is deterministicndash Assume that BGP updates are (relatively) stable
bull Limit overhead introduced by routing changesndash Minimize frequency of changes to routing policiesndash Limit number of prefixes affected by changes
bull Limit impact on how traffic enters the networkndash Avoid new routes that might change neighborrsquos mindndash Select route with same attributes or at least path length
60
Managing Scalebull Destination prefixes
ndash More than 90000 destination prefixes
bull Donrsquot want to have per-prefix routing policies
ndash Small fraction of prefixes contribute most of the traffic
bull Focus on the small number of heavy hitters
ndash Define routing policies for selected prefixes
bull Routing choicesndash About 27000 unique ldquorouting choicesrdquo
bull Help in reducing the scale of the problem
ndash Small fraction of ldquorouting choicesrdquo contribute most traffic
bull Focus on the very small number of ldquorouting choicesrdquo
ndash Define routing policies on common attributes
61
Achieving Predictability
bull Route prediction with static analysisndash Helpful to know effects before deploymentndash Static analysis can help
TopologyBGP policy
configuration
eBGP routes
Offered traffic
BGP routingmodel
Flow of traffic through the network
62
Challenges to Predictabilitybull For transit ISPs effects on incoming traffic
ndash Lack of coordination strikes again
63
ldquoHot Potatordquo routing
Inter-AS Negotiation
bull Coordination aids predictabilityndash Negotiate where to sendndash Inbound and outboundndash Mutual benefits
bull How to implementndash What info to exchangendash Protecting privacyndash How to prioritize choicesndash How to prevent cheating
Destination 2
Destination 1
multiplepeeringpoints
Provider A
Provider B
64
Outbound Multihoming Goals
bull Redundancyndash Dynamic routing will failover to backup link
bull Performancendash Select provider with best performance per prefix
ndash Requires active probing
bull Costndash Select provider per prefix over time to minimize the
total financial cost
65
Inbound Traffic Control
bull More difficult no control over neighborsrsquo decisions
bull Three common techniques (previously discussed)ndash AS path prependingndash Communities and local preferencendash Prefix splitting
How does todayrsquos paper (MONET) control inbound traffic
66
How many links are enough
K upstream ISPs
Not much benefit beyond 4 ISPs
Akella et al ldquoPerformance Benefits of Multihomingrdquo SIGCOMM 2003
67
Problems with Multihoming in IPv4
bull Routing table growthndash Provider-based addressingndash Advertising prefix out multiple ISPs ndash canrsquot aggregate
bull Poor control over inbound trafficndash Existing mechanisms do not allow hosts to control
inbound traffic
68
GeorgiaTech
Internet Routing Overview
bull Intradomain (ie ldquointra-ASrdquo) routingbull Interdomain routing
Comcast
Abilene
ATampT Cogent
Autonomous Systems (ASes)
69
Configuration Problems ldquoAS 7007rdquoldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
UUNet
Florida InternetBarn
Sprint
70
Diagnosis and Troubleshootingldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
ldquoMicrosofts websites were offline for up to 23 hoursbecause of a [router] misconfigurationhellipit took nearly a day to determine what was wrong and undo the changesrdquo -- wiredcom January 25 2001
ldquoWorldCom Inchellipsuffered a widespread outage on its Internet backbone that affected roughly 20 percent of its US customer base The network problemshellipaffected millions of computer users worldwide A spokeswoman attributed the outage to a route table issue -- cnncom October 3 2002
A number of Covad customers went out from 5pm today due to supposedly a DDOS (distributed denial of service attack) on a key Level3 data center which later was described as a route leak (misconfiguration)ldquo
-- dslreportscom February 23 2004
71
Operator Mailing List (NANOG)Date Mon 18 Oct 2004 091515 -0700Subject Level 3 US east coast issues
Level 3 experiencing widespread unspecified routing issues on the US east coast Master ticket 1086844 Anyone have more specific information
Date Mon 18 Oct 2004 122034 -0400 (EDT)Subject Re Level 3 US east coast issues
Level 3 is currently experiencing a backbone outage causing routing instability and packet loss We are working to restore and will be sending hourly updateshellip
72
Operator Mailing List
0102030405060708090
Filtering RouteLeaks
RouteHijacks
RouteInstability
RoutingLoops
Blackholes
Occurr
ences o
ver
10 Y
ears
1995-1997 1998-2001 2001-2004
Note Only includes problems openly discussed on this list
Compare 83 power outages 1 fire
73
Routing Configuration
Ranking route selection
Dissemination internal route advertisement
Filtering route advertisement
Customer
Competitor
Primary
Backup
74
Internet Business Model (Simplified)
bull CustomerProvider One AS pays another for reachability to some set of destinations
bull ldquoSettlement-freerdquo Peering Bartering Two ASes exchange routes with one another
Provider
Peer
Customer
Preferences implemented with local preference manipulation
Destination
Pay to use
Get paid to use
Free to use
75
Peering Contracts Consistent Export
bull Rules of settlement-free peeringndash Advertise routes at all peering pointsndash Advertised routes must have equal ldquoAS path lengthrdquo
Sprint
ATampT
Enables ldquohot potatordquo routing
ldquoequally goodrdquoroutes
76
Consistent Export
bull Malicedeceptionbull iBGP signaling partitionbull Inconsistent export policy
neighbor 10123route-map PEER permit 10 set prepend 123
neighbor 10456route-map PEER permit 10 set prepend 123 123
Possible Causes
10123 456 1
10456 456 2
Neighbor AS Export
1 1 123
Export Clause Prepend
2 1 123 123
Two different Export Policies
77
Inconsistent Export in Practice
Feamster et al ldquoBorderGuard Detecting Cold Potatoes from Peersrdquo ACM IMC October 2004
78
Blackholes
Date Thu 18 Jul 2002 060510 -0400 (EDT)From Chad Oleary ltcolpoboxcomgtSubject Re problems with 701To ltnanogmeritedugt
Were starting to see the same issues with UUNet again Anyone elseseeing this Trying to reach Qwest
traceroute to 631461901 (631461901) 30 hops max 38 byte packets 1 esc-lp2-gwe-solutionscorpcom (631182201) 1167 ms 1163 ms 1142 ms 2 500Serial2-10GW1TPA2ALTERNET (1571301499) 1097 ms 1059 ms 1044 ms 3 161at-1-0-0XL4ATL1ALTERNET (1526381190) 13839 ms 14108 ms 16638 ms 4 0so-3-1-0XL2ATL5ALTERNET (152630238) 14370 ms 14587 ms 14553 ms 5 POS7-0BR2ATL5ALTERNET (1526382193) 13928 ms 14099 ms 14053 ms 6 7 hellip
79
Security
80
Security ldquoBogonrdquo Routes
Feamster et al ldquoAn Empirical Study of lsquoBogonrsquo Route Advertisementsrdquo ACM CCR January 2005
81
Spam Phishing etc
bull Unsolicited commercial emailbull As of about August 2008 estimates indicate that
about 95 of all email is spambull Common spam filtering techniques
ndash Content-based filtersndash DNS Blacklist (DNSBL) lookups Significant fraction of
todayrsquos DNS traffic
Can IP addresses from which spam is received be spoofed
82
Spam and Routing
83
Worms and Botnets
84
What is a Worm
bull Code that replicates and propagates across the networkndash Often carries a ldquopayloadrdquo
bull Usually spread via exploiting flaws in open servicesndash ldquoVirusesrdquo require user action to spread
bull First worm Robert Morris November 1988ndash 6-10 of all Internet hosts infected ()
bull Many more since but none on that scale until July 2001
85
Example Worm Code Red
bull Initial version July 13 2001
bull Exploited known ISAPI vulnerability in Microsoft IIS Web servers
bull 1st through 20th of each month spread20th through end of each month attack
bull Payload Web site defacementbull Scanning Random IP addressesbull Bug failure to seed random number generator
86
Code Red Revisions
bull Released July 19 2001
bull Payload flooding attack on wwwwhitehousegovndash Attack was mounted at the IP address of the Web site
bull Bug died after 20th of each month
bull Random number generator for IP scanning fixed
87
Code Red Host Infection Rate
Exponential infection rate
Measured using backscatter technique
88
Designing Fast-Spreading Worms
bull Hit-list scanningndash Time to infect first 10k hosts dominates infection timendash Solution Reconnaissance (stealthy scans etc)
bull Permutation scanningndash Observation Most scanning is redundantndash Idea Shared permutation of address space Start scanning
from own IP address Re-randomize when another infected machine is found
bull Internet-scale hit listsndash Flash worm complete infection within 30 seconds
89
Botnets
bull Bots Autonomous programs performing tasksbull Plenty of ldquobenignrdquo bots
ndash eg weatherbug
bull Botnets group of bots ndash Typically carries malicious connotationndash Large numbers of infected machinesndash Machines ldquoenlistedrdquo with infection vectors like worms
(last lecture)
bull Available for simultaneous control by a masterbull Size up to 350000 nodes (from todayrsquos paper)
90
ldquoRallyingrdquo the Botnet
bull Easy to combine worm backdoor functionalitybull Problem how to learn about successfully
infected machines
bull Optionsndash Emailndash Hard-coded email address
91
Botnet Control
bull Botnet master typically runs some IRC server on a well-known port (eg 6667)
bull Infected machine contacts botnet with pre-programmed DNS name (eg big-botde)
bull Dynamic DNS allows controller to move about freely
Infected Machine
DynamicDNS
BotnetController
(IRC server)
92
Some Defenses
93
Idea 1 Ingress Filtering
bull RFC 2827 Routers install filters to drop packets from networks that are not downstream
bull Feasible at edgesbull Difficult to configure closer to network ldquocorerdquo
20469207024 Internet
Drop all packets with source address other than 20469207024
94
Idea 2 uRPF Checks
bull Unicast Reverse Path Forwardingndash Cisco ldquoip verify unicast reverse-pathrdquo
bull Requires symmetric routing
Accept packet from interface only if forwarding table entry for source IP address matches ingress interface
100183
A10018
10015
101203
100181 24
10011 24
Strict Mode uRPF
Enabled
ldquoArdquo Routing TableDestination Next Hop1001024 Int 110018024 Int 2
100183 from wrong interface
95
Problems with uRPF
bull Asymmetric routing
96
S-BGP
bull Address-based PKI validate signaturesndash Authentication of
bull ownership for IP address blocks bull AS number bull an ASs identity and bull a BGP routers identity
ndash Use existing infrastructure (Internet registries etc)ndash Routing origination is digitally signedndash BGP updates are digitally signed
1048708 bull Route attestations A new optional BGP transitive path attribute
ndash carries digital signatures covering the routing information in updates
97
Practical Problems with S-BGP
bull Requires Public-Key Infrastructure
bull Lots of digital signatures to calculate and verifyndash Message overheadndash CPU overhead
bull Calculation expense is greatest when topology is changingndash Caching can help
bull Route aggregation is problematic (maybe thatrsquos OK)
bull Secure route withdrawals when link or node fails
bull Address ownership data out of date
bull Deployment
- Networking
- Goal of This Tutorial
- Todayrsquos Networks
- Business Models
- Billing for Internet Usage
- Net Neutrality
- Network Operations
- Point-of-Presence (PoP)
- Example Abilene Network Topology
- Another Example Backbone
- Internet Routing Overview
- Internet Routing Protocol BGP
- Two Flavors of BGP
- IPv4 Addresses Networks of Networks
- Pre-1994 Classful Addressing
- Classless Interdomain Routing (CIDR)
- Benefits of CIDR
- Growth of IP Prefixes
- 1994-1998 Linear Growth
- Around 2000 Fast Growth Resumes
- Fast growth resumes
- The Address Allocation Process
- Common Problems
- What can go wrong
- Measurement and Monitoring
- Passive vs Active Measurement
- Slide 27
- Passive Traffic Data Measurement
- Simple Network Management Protocol
- SNMP (Passive)
- Packet-level Monitoring
- Full Packet Capture (Passive)
- What is a flow
- Cisco NetFlow
- Flow Record Contents
- Aggregating Packets into Flows
- Reducing Measurement Overhead
- Packet Sampling for Flow Monitoring
- Sampling Flow-Level Sampling
- Two Main Approaches
- Packet Capture on High-Speed Links
- Characteristics of Packet Capture
- Data Measurement Repositories
- Multihoming and Traffic Engineering
- What is Multihoming
- Why Multihome
- Redundancy
- Performance
- Multihoming in IP Networks Today
- BGP or no
- Same Provider or Multiple
- Multihomed Stub One Link
- Multihomed Stub Multiple Links
- Multihomed Stub Multiple ISPs
- Multihomed Transit Network
- Interdomain Traffic Engineering
- Outbound Traffic Control
- Outbound Traffic Load Balancing
- Traffic Engineering Goals
- Managing Scale
- Achieving Predictability
- Challenges to Predictability
- Inter-AS Negotiation
- Outbound Multihoming Goals
- Inbound Traffic Control
- How many links are enough
- Problems with Multihoming in IPv4
- Slide 68
- Slide 69
- Diagnosis and Troubleshooting
- Operator Mailing List (NANOG)
- Operator Mailing List
- Routing Configuration
- Internet Business Model (Simplified)
- Peering Contracts Consistent Export
- Consistent Export
- Inconsistent Export in Practice
- Blackholes
- Security
- Security ldquoBogonrdquo Routes
- Spam Phishing etc
- Spam and Routing
- Worms and Botnets
- What is a Worm
- Example Worm Code Red
- Code Red Revisions
- Code Red Host Infection Rate
- Designing Fast-Spreading Worms
- Botnets
- ldquoRallyingrdquo the Botnet
- Botnet Control
- Some Defenses
- Idea 1 Ingress Filtering
- Idea 2 uRPF Checks
- Problems with uRPF
- S-BGP
- Practical Problems with S-BGP
-
14
IPv4 Addresses Networks of Networks
bull 32-bit number in ldquodotted-quadrdquo notation
ndash wwwccgatechedu --- 130207736
10000010 11001111 00000111 00100100
Network (16 bits) Host (16 bits)
130 207 7 36
bull Problem 232 addresses is a lot of table entries
bull Solution Routing based on network and host
ndash 1302070016 is a 16-bit prefix with 216 IP addresses
Topological Addressing
15
Pre-1994 Classful Addressing
Network ID Host ID
8 16
Class A
32
0
Class B 10
Class C 110
Multicast AddressesClass D 1110
Reserved for experimentsClass E 1111
24
8 blocks (eg MIT has 180008)
16 blocks (eg Georgia Tech has 1302070016)
24 blocks (eg ATampT Labs has 19220225024)
Simple Forwarding Address range specifies network ID length
16
Classless Interdomain Routing (CIDR)
IP Address 65142480 ldquoMaskrdquo 2552552520
01000001 00001110 11111000 00000000
11111111 11111111 11111100 00000000
Use two 32-bit numbers to represent a network Network number = IP address + Mask
Example BellSouth Prefix 6514248022
Address no longer specifies network ID rangeNew forwarding trick Longest Prefix Match
17
Benefits of CIDR
bull Efficiency Can allocate blocks of prefixes on a finer granularity
bull Hierarchy Prefixes can be aggregated into supernets (Not always done Typically not in fact)
Customer 1
Customer 2
ATampT Internet
1220249024
1220231024120008
18
Growth of IP Prefixes
19
1994-1998 Linear Growth
bull About 10000 new entries per yearbull In theory less instability at the edges (why)
Source Geoff Huston
20
Around 2000 Fast Growth Resumes
Claim remaining 8s will be exhausted within the next 5-10 years
T Hain ldquoA Pragmatic Report on IPv4 Address Space Consumptionrdquo Cisco IPJ September 2005
21
Fast growth resumes
Rapid growth in routing tables
Dot-Bomb Hiccup
Significant contributor Multihoming
Source Geoff Huston
22
The Address Allocation Process
bull Allocation policies of RIRs affect pressure on IPv4 address space
IANA
AfriNIC APNIC ARIN LACNIC RIPE
httpwwwianaorgassignmentsipv4-address-space
Georgia Tech
23
Common Problems
bull Diagnosis and troubleshooting (hence measurement)bull Traffic engineeringbull Securitybull Design and capacity planning
24
What can go wrong
Two-thirds of the problems are caused by configuration of the routing protocol
Some downtime is very hard to protect againsthellip
25
Measurement and Monitoring
26
Passive vs Active Measurement
bull Passive Measurement Collection of packets flow statistics of traffic that is already flowing on the networkndash Packet tracesndash Flow statisticsndash Application-level logs
bull Active Measurement Inject ldquoprobingrdquo traffic to measure various characteristicsndash Traceroutendash Pingndash Application-level probes (eg Web downloads)
27
Billing for Internet Usage
bull 95th Percentile billingndash Customer network pays for ldquocommitted information
raterdquo (CIR)ndash Throughput measured every 5 minutes (typically with
SNMP flow statistics also can be used for billing)ndash Customer billed based on 95th percentile
28
Passive Traffic Data Measurement
bull SNMP bytepacket counts everywhere
bull Packet monitoring selected locations
bull Flow monitoring typically at edges (if possible)ndash Direct computation of the traffic matrixndash Input to denial-of-service attack detection
bull Deep Packet Inspection also at edge where possible
29
Simple Network Management Protocol
bull Management Information Base (MIB)ndash Information storendash Unique variables named by
OIDsndash Accessed with SNMP
bull Specific MIBs for bytepacket counts (per link)
Manager Agent
SNMP
DB
ManagedObjects
30
SNMP (Passive)
bull Advantage ubiquitousndash Supported on all networking equipmentndash Multiple products for polling and analyzing data
bull Disadvantages ndash Coarse granularityndash Cannot express complex queries on the datandash Unreliable delivery of the data using UDP
bull Utilityndash Link utilization (billing)ndash Traffic matrix inference
31
Packet-level Monitoring
bull Passive monitoring to collect full packet contents (or at least headers)
bull Advantages lots of detailed informationndash Precise timing informationndash Information in packet headers
bull Disadvantages overheadndash Hard to keep up with high-speed linksndash Often requires a separate monitoring device
32
Full Packet Capture (Passive)
Example Georgia Tech OC3Mon
bull Rack-mounted PCbull Optical splitterbull Data Acquisition and
Generation (DAG) card
Source endacecom
33
What is a flow
bull Source IP addressbull Destination IP addressbull Source portbull Destination portbull Layer 3 protocol typebull TOS byte (DSCP)bull Input logical interface (ifIndex)
34
Cisco NetFlow
bull Basic output ldquoFlow recordrdquondash Most common version is v5
bull Current version (9) is being standardized in the IETF (template-based)ndash More flexible record formatndash Much easier to add new flow record types
Core Network
Collection and Aggregation
Collector (PC)Approximately 1500 bytes
20-50 flow recordsSent more frequently if traffic increases
35
Flow Record Contents
bull Source and Destination IP address and portbull Packet and byte countsbull Start and end timesbull ToS TCP flags
Basic information about the flowhellip
hellipplus information related to routing
bull Next-hop IP addressbull Source and destination ASbull Source and destination prefix
36
flow 1 flow 2 flow 3 flow 4
Aggregating Packets into Flows
bull Criteria 1 Set of packets that ldquobelong togetherrdquondash Sourcedestination IP addresses and port numbersndash Same protocol ToS bits hellip ndash Same inputoutput interfaces at a router (if known)
bull Criteria 2 Packets that are ldquocloserdquo together in timendash Maximum inter-packet spacing (eg 15 sec 30 sec)ndash Example flows 2 and 4 are different flows due to time
37
Reducing Measurement Overhead
bull Filtering on interfacendash destination prefix for a customerndash port number for an application (eg 80 for Web)
bull Sampling before insertion into flow cachendash Random deterministic or hash-based samplingndash 1-out-of-n or stratified based on packetflow sizendash Two types packet-level and flow-level
bull Aggregation after cache evictionndash packetsflows with same next-hop ASndash packetsflows destined to a particular service
38
Packet Sampling for Flow Monitoring
bull Packet sampling before flow creation (Sampled Netflow)ndash 1-out-of-m sampling of individual packets (eg m=100)ndash Create of flow records over the sampled packets
bull Reducing overheadndash Avoid per-packet overhead on (m-1)m packetsndash Avoid creating records for a large number of small flows
bull Increasing overhead (in some cases)ndash May split some long transfers into multiple flow records ndash hellip due to larger time gaps between successive packets
time
not sampled
two flowstimeout
39
Sampling Flow-Level Sampling
bull Sampling of flow records evicted from flow cachendash When evicting flows from table or when analyzing flows
bull Stratified sampling to put weight on ldquoheavyrdquo flowsndash Select all long flows and sample the short flows
bull Reduces the number of flow records ndash Still measures the vast majority of the traffic
Flow 1 40 bytesFlow 2 15580 bytesFlow 3 8196 bytesFlow 4 5350789 bytesFlow 5 532 bytesFlow 6 7432 bytes
sample with 100 probability
sample with 01 probability
sample with 10 probability
40
Two Main Approaches
bull Packet-level Monitoringndash Keep packet-level statisticsndash Examine (and potentially log) variety of packet-level
statistics Essentially anything in the packetndash Timing
bull Flow-level Monitoringndash Monitor packet-by-packet (though sometimes
sampled)ndash Keep aggregate statistics on a flow
41
Packet Capture on High-Speed Links
Example Georgia Tech ldquoOC3Monrdquo
bull Rack-mounted PCbull Optical splitterbull Data Acquisition and
Generation (DAG) card
Source endacecom
42
Characteristics of Packet Capture
bull Allows inspection on every packet on 10G links
bull Disadvantagesndash Costlyndash Requires splitting optical fibersndash Must be able to filterstore data
43
Data Measurement Repositories
bull AbileneInternet 2 Observatoryndash Configuration examplesndash SNMP datandash ISIS BGP routing data NetFlow traffic data
bull RouteViewsndash BGP updatesndash BGP table snapshots
44
Multihoming and Traffic Engineering
45
What is Multihoming
bull The use of redundant network links for the purposes of external connectivity
bull Can be achieved at many layers of the protocol stack and many places in the networkndash Multiple network interfaces in a PC
ndash An ISP with multiple upstream interfaces
bull Can refer to having multiple connections tondash The same ISP
ndash Multiple ISPs
46
Why Multihome
bull Redundancybull Availabilitybull Performancebull Cost
Interdomain traffic engineering the process by which a multihomed network configures its
network to achieve these goals
47
Redundancy
bull Maintain connectivity in the face ofndash Physical connectivity problems (fiber cut device
failures etc)ndash Failures in upstream ISP
48
Performance
bull Use multiple network links at once to achieve higher throughput than just over a single link
bull Allows incoming traffic to be load-balanced
70 of traffic30 of traffic
49
Multihoming in IP Networks Today
bull Stub AS no transit service for other ASesndash No need to use BGP
bull Multi-homed stub AS has connectivity to multiple immediate upstream ISPsndash Need BGPndash No need for a public AS numberndash No need for IP prefix allocation
bull Multi-homed transit AS connectivity to multiple ASes and transit servicendash Need BGP public AS number IP prefix allocation
50
BGP or no
bull Advantages of static routingndash Cheapersmaller routers (less true nowadays)ndash Simpler to configure
bull Advantages of BGPndash More control of your destiny (have providers stop
announcing you)ndash Fastermore intelligent selection of where to send
outbound packetsndash Better debugging of net problems (you can see the
Internet topology now)
51
Same Provider or Multiple
bull If your provider is reliable and fast and affordably and offers good tech-support you may want to multi-home initially to them via some backup path (slow is better than dead)
bull Eventually yoursquoll want to multi-home to different providers to avoid failure modes due to one providerrsquos architecture decisions
52
Multihomed Stub One Link
bull Downstream ISPrsquos routers configure default (ldquostaticrdquo) routes pointing to border router
bull Upstream ISP advertises reachability
Upstream ISP
Multiple links between same pair of routers
Default routes to ldquoborderrdquo
ldquoStubrdquoISP
53
Multihomed Stub Multiple Links
bull Use BGP to share loadbull Use private AS number (why is this OK)bull As before upstream ISP advertises prefix
Upstream ISP
Multiple links to different upstream routers
ldquoStubrdquoISP
Internal routing for ldquohot potatordquo
BGP for load balance at edge
54
Multihomed Stub Multiple ISPs
bull Many possibilitiesndash Load sharingndash Primary-backupndash Selective use of different ISPs
bull Requires BGP public AS number etc
ldquoStubrdquoISP
Upstream
ISP 1
Upstream
ISP 2
55
Multihomed Transit Network
bull BGP everywherebull Incoming and outcoming trafficbull Challenge balancing load on intradomain and egress
links given an offered traffic load
TransitISP
ISP 1
ISP 2
ISP 3
56
Interdomain Traffic Engineering
bull The process by which a network operator configures the network to achievendash Traffic load balancendash Redundancy (primarybackup) etc
bull Two tasksndash Outbound traffic controlndash Inbound traffic control
bull Key Problems Predictability and Scalability
57
Outbound Traffic Control
bull Easier to control than inbound trafficndash Destination-based routing sender determines where
the packets go
bull Control over next-hop AS onlyndash Cannot control selection of the entire path
Provider 1 Provider 2
Control with local preference
58
Outbound Traffic Load Balancingbull Control routes to provider per-prefix
ndash Assign local preference across destination prefixesndash Change the local preference assignments over time
bull Useful inputs to load balancingndash End-to-end path performance datandash Outbound traffic statistics per destination prefix
bull Challenge Getting from traffic volumes to groups of prefixes that should be assigned to each link
Premise of ldquointelligent route controlrdquo preoducts
59
Traffic Engineering Goals
bull Predictabilityndash Ensure the BGP decision process is deterministicndash Assume that BGP updates are (relatively) stable
bull Limit overhead introduced by routing changesndash Minimize frequency of changes to routing policiesndash Limit number of prefixes affected by changes
bull Limit impact on how traffic enters the networkndash Avoid new routes that might change neighborrsquos mindndash Select route with same attributes or at least path length
60
Managing Scalebull Destination prefixes
ndash More than 90000 destination prefixes
bull Donrsquot want to have per-prefix routing policies
ndash Small fraction of prefixes contribute most of the traffic
bull Focus on the small number of heavy hitters
ndash Define routing policies for selected prefixes
bull Routing choicesndash About 27000 unique ldquorouting choicesrdquo
bull Help in reducing the scale of the problem
ndash Small fraction of ldquorouting choicesrdquo contribute most traffic
bull Focus on the very small number of ldquorouting choicesrdquo
ndash Define routing policies on common attributes
61
Achieving Predictability
bull Route prediction with static analysisndash Helpful to know effects before deploymentndash Static analysis can help
TopologyBGP policy
configuration
eBGP routes
Offered traffic
BGP routingmodel
Flow of traffic through the network
62
Challenges to Predictabilitybull For transit ISPs effects on incoming traffic
ndash Lack of coordination strikes again
63
ldquoHot Potatordquo routing
Inter-AS Negotiation
bull Coordination aids predictabilityndash Negotiate where to sendndash Inbound and outboundndash Mutual benefits
bull How to implementndash What info to exchangendash Protecting privacyndash How to prioritize choicesndash How to prevent cheating
Destination 2
Destination 1
multiplepeeringpoints
Provider A
Provider B
64
Outbound Multihoming Goals
bull Redundancyndash Dynamic routing will failover to backup link
bull Performancendash Select provider with best performance per prefix
ndash Requires active probing
bull Costndash Select provider per prefix over time to minimize the
total financial cost
65
Inbound Traffic Control
bull More difficult no control over neighborsrsquo decisions
bull Three common techniques (previously discussed)ndash AS path prependingndash Communities and local preferencendash Prefix splitting
How does todayrsquos paper (MONET) control inbound traffic
66
How many links are enough
K upstream ISPs
Not much benefit beyond 4 ISPs
Akella et al ldquoPerformance Benefits of Multihomingrdquo SIGCOMM 2003
67
Problems with Multihoming in IPv4
bull Routing table growthndash Provider-based addressingndash Advertising prefix out multiple ISPs ndash canrsquot aggregate
bull Poor control over inbound trafficndash Existing mechanisms do not allow hosts to control
inbound traffic
68
GeorgiaTech
Internet Routing Overview
bull Intradomain (ie ldquointra-ASrdquo) routingbull Interdomain routing
Comcast
Abilene
ATampT Cogent
Autonomous Systems (ASes)
69
Configuration Problems ldquoAS 7007rdquoldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
UUNet
Florida InternetBarn
Sprint
70
Diagnosis and Troubleshootingldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
ldquoMicrosofts websites were offline for up to 23 hoursbecause of a [router] misconfigurationhellipit took nearly a day to determine what was wrong and undo the changesrdquo -- wiredcom January 25 2001
ldquoWorldCom Inchellipsuffered a widespread outage on its Internet backbone that affected roughly 20 percent of its US customer base The network problemshellipaffected millions of computer users worldwide A spokeswoman attributed the outage to a route table issue -- cnncom October 3 2002
A number of Covad customers went out from 5pm today due to supposedly a DDOS (distributed denial of service attack) on a key Level3 data center which later was described as a route leak (misconfiguration)ldquo
-- dslreportscom February 23 2004
71
Operator Mailing List (NANOG)Date Mon 18 Oct 2004 091515 -0700Subject Level 3 US east coast issues
Level 3 experiencing widespread unspecified routing issues on the US east coast Master ticket 1086844 Anyone have more specific information
Date Mon 18 Oct 2004 122034 -0400 (EDT)Subject Re Level 3 US east coast issues
Level 3 is currently experiencing a backbone outage causing routing instability and packet loss We are working to restore and will be sending hourly updateshellip
72
Operator Mailing List
0102030405060708090
Filtering RouteLeaks
RouteHijacks
RouteInstability
RoutingLoops
Blackholes
Occurr
ences o
ver
10 Y
ears
1995-1997 1998-2001 2001-2004
Note Only includes problems openly discussed on this list
Compare 83 power outages 1 fire
73
Routing Configuration
Ranking route selection
Dissemination internal route advertisement
Filtering route advertisement
Customer
Competitor
Primary
Backup
74
Internet Business Model (Simplified)
bull CustomerProvider One AS pays another for reachability to some set of destinations
bull ldquoSettlement-freerdquo Peering Bartering Two ASes exchange routes with one another
Provider
Peer
Customer
Preferences implemented with local preference manipulation
Destination
Pay to use
Get paid to use
Free to use
75
Peering Contracts Consistent Export
bull Rules of settlement-free peeringndash Advertise routes at all peering pointsndash Advertised routes must have equal ldquoAS path lengthrdquo
Sprint
ATampT
Enables ldquohot potatordquo routing
ldquoequally goodrdquoroutes
76
Consistent Export
bull Malicedeceptionbull iBGP signaling partitionbull Inconsistent export policy
neighbor 10123route-map PEER permit 10 set prepend 123
neighbor 10456route-map PEER permit 10 set prepend 123 123
Possible Causes
10123 456 1
10456 456 2
Neighbor AS Export
1 1 123
Export Clause Prepend
2 1 123 123
Two different Export Policies
77
Inconsistent Export in Practice
Feamster et al ldquoBorderGuard Detecting Cold Potatoes from Peersrdquo ACM IMC October 2004
78
Blackholes
Date Thu 18 Jul 2002 060510 -0400 (EDT)From Chad Oleary ltcolpoboxcomgtSubject Re problems with 701To ltnanogmeritedugt
Were starting to see the same issues with UUNet again Anyone elseseeing this Trying to reach Qwest
traceroute to 631461901 (631461901) 30 hops max 38 byte packets 1 esc-lp2-gwe-solutionscorpcom (631182201) 1167 ms 1163 ms 1142 ms 2 500Serial2-10GW1TPA2ALTERNET (1571301499) 1097 ms 1059 ms 1044 ms 3 161at-1-0-0XL4ATL1ALTERNET (1526381190) 13839 ms 14108 ms 16638 ms 4 0so-3-1-0XL2ATL5ALTERNET (152630238) 14370 ms 14587 ms 14553 ms 5 POS7-0BR2ATL5ALTERNET (1526382193) 13928 ms 14099 ms 14053 ms 6 7 hellip
79
Security
80
Security ldquoBogonrdquo Routes
Feamster et al ldquoAn Empirical Study of lsquoBogonrsquo Route Advertisementsrdquo ACM CCR January 2005
81
Spam Phishing etc
bull Unsolicited commercial emailbull As of about August 2008 estimates indicate that
about 95 of all email is spambull Common spam filtering techniques
ndash Content-based filtersndash DNS Blacklist (DNSBL) lookups Significant fraction of
todayrsquos DNS traffic
Can IP addresses from which spam is received be spoofed
82
Spam and Routing
83
Worms and Botnets
84
What is a Worm
bull Code that replicates and propagates across the networkndash Often carries a ldquopayloadrdquo
bull Usually spread via exploiting flaws in open servicesndash ldquoVirusesrdquo require user action to spread
bull First worm Robert Morris November 1988ndash 6-10 of all Internet hosts infected ()
bull Many more since but none on that scale until July 2001
85
Example Worm Code Red
bull Initial version July 13 2001
bull Exploited known ISAPI vulnerability in Microsoft IIS Web servers
bull 1st through 20th of each month spread20th through end of each month attack
bull Payload Web site defacementbull Scanning Random IP addressesbull Bug failure to seed random number generator
86
Code Red Revisions
bull Released July 19 2001
bull Payload flooding attack on wwwwhitehousegovndash Attack was mounted at the IP address of the Web site
bull Bug died after 20th of each month
bull Random number generator for IP scanning fixed
87
Code Red Host Infection Rate
Exponential infection rate
Measured using backscatter technique
88
Designing Fast-Spreading Worms
bull Hit-list scanningndash Time to infect first 10k hosts dominates infection timendash Solution Reconnaissance (stealthy scans etc)
bull Permutation scanningndash Observation Most scanning is redundantndash Idea Shared permutation of address space Start scanning
from own IP address Re-randomize when another infected machine is found
bull Internet-scale hit listsndash Flash worm complete infection within 30 seconds
89
Botnets
bull Bots Autonomous programs performing tasksbull Plenty of ldquobenignrdquo bots
ndash eg weatherbug
bull Botnets group of bots ndash Typically carries malicious connotationndash Large numbers of infected machinesndash Machines ldquoenlistedrdquo with infection vectors like worms
(last lecture)
bull Available for simultaneous control by a masterbull Size up to 350000 nodes (from todayrsquos paper)
90
ldquoRallyingrdquo the Botnet
bull Easy to combine worm backdoor functionalitybull Problem how to learn about successfully
infected machines
bull Optionsndash Emailndash Hard-coded email address
91
Botnet Control
bull Botnet master typically runs some IRC server on a well-known port (eg 6667)
bull Infected machine contacts botnet with pre-programmed DNS name (eg big-botde)
bull Dynamic DNS allows controller to move about freely
Infected Machine
DynamicDNS
BotnetController
(IRC server)
92
Some Defenses
93
Idea 1 Ingress Filtering
bull RFC 2827 Routers install filters to drop packets from networks that are not downstream
bull Feasible at edgesbull Difficult to configure closer to network ldquocorerdquo
20469207024 Internet
Drop all packets with source address other than 20469207024
94
Idea 2 uRPF Checks
bull Unicast Reverse Path Forwardingndash Cisco ldquoip verify unicast reverse-pathrdquo
bull Requires symmetric routing
Accept packet from interface only if forwarding table entry for source IP address matches ingress interface
100183
A10018
10015
101203
100181 24
10011 24
Strict Mode uRPF
Enabled
ldquoArdquo Routing TableDestination Next Hop1001024 Int 110018024 Int 2
100183 from wrong interface
95
Problems with uRPF
bull Asymmetric routing
96
S-BGP
bull Address-based PKI validate signaturesndash Authentication of
bull ownership for IP address blocks bull AS number bull an ASs identity and bull a BGP routers identity
ndash Use existing infrastructure (Internet registries etc)ndash Routing origination is digitally signedndash BGP updates are digitally signed
1048708 bull Route attestations A new optional BGP transitive path attribute
ndash carries digital signatures covering the routing information in updates
97
Practical Problems with S-BGP
bull Requires Public-Key Infrastructure
bull Lots of digital signatures to calculate and verifyndash Message overheadndash CPU overhead
bull Calculation expense is greatest when topology is changingndash Caching can help
bull Route aggregation is problematic (maybe thatrsquos OK)
bull Secure route withdrawals when link or node fails
bull Address ownership data out of date
bull Deployment
- Networking
- Goal of This Tutorial
- Todayrsquos Networks
- Business Models
- Billing for Internet Usage
- Net Neutrality
- Network Operations
- Point-of-Presence (PoP)
- Example Abilene Network Topology
- Another Example Backbone
- Internet Routing Overview
- Internet Routing Protocol BGP
- Two Flavors of BGP
- IPv4 Addresses Networks of Networks
- Pre-1994 Classful Addressing
- Classless Interdomain Routing (CIDR)
- Benefits of CIDR
- Growth of IP Prefixes
- 1994-1998 Linear Growth
- Around 2000 Fast Growth Resumes
- Fast growth resumes
- The Address Allocation Process
- Common Problems
- What can go wrong
- Measurement and Monitoring
- Passive vs Active Measurement
- Slide 27
- Passive Traffic Data Measurement
- Simple Network Management Protocol
- SNMP (Passive)
- Packet-level Monitoring
- Full Packet Capture (Passive)
- What is a flow
- Cisco NetFlow
- Flow Record Contents
- Aggregating Packets into Flows
- Reducing Measurement Overhead
- Packet Sampling for Flow Monitoring
- Sampling Flow-Level Sampling
- Two Main Approaches
- Packet Capture on High-Speed Links
- Characteristics of Packet Capture
- Data Measurement Repositories
- Multihoming and Traffic Engineering
- What is Multihoming
- Why Multihome
- Redundancy
- Performance
- Multihoming in IP Networks Today
- BGP or no
- Same Provider or Multiple
- Multihomed Stub One Link
- Multihomed Stub Multiple Links
- Multihomed Stub Multiple ISPs
- Multihomed Transit Network
- Interdomain Traffic Engineering
- Outbound Traffic Control
- Outbound Traffic Load Balancing
- Traffic Engineering Goals
- Managing Scale
- Achieving Predictability
- Challenges to Predictability
- Inter-AS Negotiation
- Outbound Multihoming Goals
- Inbound Traffic Control
- How many links are enough
- Problems with Multihoming in IPv4
- Slide 68
- Slide 69
- Diagnosis and Troubleshooting
- Operator Mailing List (NANOG)
- Operator Mailing List
- Routing Configuration
- Internet Business Model (Simplified)
- Peering Contracts Consistent Export
- Consistent Export
- Inconsistent Export in Practice
- Blackholes
- Security
- Security ldquoBogonrdquo Routes
- Spam Phishing etc
- Spam and Routing
- Worms and Botnets
- What is a Worm
- Example Worm Code Red
- Code Red Revisions
- Code Red Host Infection Rate
- Designing Fast-Spreading Worms
- Botnets
- ldquoRallyingrdquo the Botnet
- Botnet Control
- Some Defenses
- Idea 1 Ingress Filtering
- Idea 2 uRPF Checks
- Problems with uRPF
- S-BGP
- Practical Problems with S-BGP
-
15
Pre-1994 Classful Addressing
Network ID Host ID
8 16
Class A
32
0
Class B 10
Class C 110
Multicast AddressesClass D 1110
Reserved for experimentsClass E 1111
24
8 blocks (eg MIT has 180008)
16 blocks (eg Georgia Tech has 1302070016)
24 blocks (eg ATampT Labs has 19220225024)
Simple Forwarding Address range specifies network ID length
16
Classless Interdomain Routing (CIDR)
IP Address 65142480 ldquoMaskrdquo 2552552520
01000001 00001110 11111000 00000000
11111111 11111111 11111100 00000000
Use two 32-bit numbers to represent a network Network number = IP address + Mask
Example BellSouth Prefix 6514248022
Address no longer specifies network ID rangeNew forwarding trick Longest Prefix Match
17
Benefits of CIDR
bull Efficiency Can allocate blocks of prefixes on a finer granularity
bull Hierarchy Prefixes can be aggregated into supernets (Not always done Typically not in fact)
Customer 1
Customer 2
ATampT Internet
1220249024
1220231024120008
18
Growth of IP Prefixes
19
1994-1998 Linear Growth
bull About 10000 new entries per yearbull In theory less instability at the edges (why)
Source Geoff Huston
20
Around 2000 Fast Growth Resumes
Claim remaining 8s will be exhausted within the next 5-10 years
T Hain ldquoA Pragmatic Report on IPv4 Address Space Consumptionrdquo Cisco IPJ September 2005
21
Fast growth resumes
Rapid growth in routing tables
Dot-Bomb Hiccup
Significant contributor Multihoming
Source Geoff Huston
22
The Address Allocation Process
bull Allocation policies of RIRs affect pressure on IPv4 address space
IANA
AfriNIC APNIC ARIN LACNIC RIPE
httpwwwianaorgassignmentsipv4-address-space
Georgia Tech
23
Common Problems
bull Diagnosis and troubleshooting (hence measurement)bull Traffic engineeringbull Securitybull Design and capacity planning
24
What can go wrong
Two-thirds of the problems are caused by configuration of the routing protocol
Some downtime is very hard to protect againsthellip
25
Measurement and Monitoring
26
Passive vs Active Measurement
bull Passive Measurement Collection of packets flow statistics of traffic that is already flowing on the networkndash Packet tracesndash Flow statisticsndash Application-level logs
bull Active Measurement Inject ldquoprobingrdquo traffic to measure various characteristicsndash Traceroutendash Pingndash Application-level probes (eg Web downloads)
27
Billing for Internet Usage
bull 95th Percentile billingndash Customer network pays for ldquocommitted information
raterdquo (CIR)ndash Throughput measured every 5 minutes (typically with
SNMP flow statistics also can be used for billing)ndash Customer billed based on 95th percentile
28
Passive Traffic Data Measurement
bull SNMP bytepacket counts everywhere
bull Packet monitoring selected locations
bull Flow monitoring typically at edges (if possible)ndash Direct computation of the traffic matrixndash Input to denial-of-service attack detection
bull Deep Packet Inspection also at edge where possible
29
Simple Network Management Protocol
bull Management Information Base (MIB)ndash Information storendash Unique variables named by
OIDsndash Accessed with SNMP
bull Specific MIBs for bytepacket counts (per link)
Manager Agent
SNMP
DB
ManagedObjects
30
SNMP (Passive)
bull Advantage ubiquitousndash Supported on all networking equipmentndash Multiple products for polling and analyzing data
bull Disadvantages ndash Coarse granularityndash Cannot express complex queries on the datandash Unreliable delivery of the data using UDP
bull Utilityndash Link utilization (billing)ndash Traffic matrix inference
31
Packet-level Monitoring
bull Passive monitoring to collect full packet contents (or at least headers)
bull Advantages lots of detailed informationndash Precise timing informationndash Information in packet headers
bull Disadvantages overheadndash Hard to keep up with high-speed linksndash Often requires a separate monitoring device
32
Full Packet Capture (Passive)
Example Georgia Tech OC3Mon
bull Rack-mounted PCbull Optical splitterbull Data Acquisition and
Generation (DAG) card
Source endacecom
33
What is a flow
bull Source IP addressbull Destination IP addressbull Source portbull Destination portbull Layer 3 protocol typebull TOS byte (DSCP)bull Input logical interface (ifIndex)
34
Cisco NetFlow
bull Basic output ldquoFlow recordrdquondash Most common version is v5
bull Current version (9) is being standardized in the IETF (template-based)ndash More flexible record formatndash Much easier to add new flow record types
Core Network
Collection and Aggregation
Collector (PC)Approximately 1500 bytes
20-50 flow recordsSent more frequently if traffic increases
35
Flow Record Contents
bull Source and Destination IP address and portbull Packet and byte countsbull Start and end timesbull ToS TCP flags
Basic information about the flowhellip
hellipplus information related to routing
bull Next-hop IP addressbull Source and destination ASbull Source and destination prefix
36
flow 1 flow 2 flow 3 flow 4
Aggregating Packets into Flows
bull Criteria 1 Set of packets that ldquobelong togetherrdquondash Sourcedestination IP addresses and port numbersndash Same protocol ToS bits hellip ndash Same inputoutput interfaces at a router (if known)
bull Criteria 2 Packets that are ldquocloserdquo together in timendash Maximum inter-packet spacing (eg 15 sec 30 sec)ndash Example flows 2 and 4 are different flows due to time
37
Reducing Measurement Overhead
bull Filtering on interfacendash destination prefix for a customerndash port number for an application (eg 80 for Web)
bull Sampling before insertion into flow cachendash Random deterministic or hash-based samplingndash 1-out-of-n or stratified based on packetflow sizendash Two types packet-level and flow-level
bull Aggregation after cache evictionndash packetsflows with same next-hop ASndash packetsflows destined to a particular service
38
Packet Sampling for Flow Monitoring
bull Packet sampling before flow creation (Sampled Netflow)ndash 1-out-of-m sampling of individual packets (eg m=100)ndash Create of flow records over the sampled packets
bull Reducing overheadndash Avoid per-packet overhead on (m-1)m packetsndash Avoid creating records for a large number of small flows
bull Increasing overhead (in some cases)ndash May split some long transfers into multiple flow records ndash hellip due to larger time gaps between successive packets
time
not sampled
two flowstimeout
39
Sampling Flow-Level Sampling
bull Sampling of flow records evicted from flow cachendash When evicting flows from table or when analyzing flows
bull Stratified sampling to put weight on ldquoheavyrdquo flowsndash Select all long flows and sample the short flows
bull Reduces the number of flow records ndash Still measures the vast majority of the traffic
Flow 1 40 bytesFlow 2 15580 bytesFlow 3 8196 bytesFlow 4 5350789 bytesFlow 5 532 bytesFlow 6 7432 bytes
sample with 100 probability
sample with 01 probability
sample with 10 probability
40
Two Main Approaches
bull Packet-level Monitoringndash Keep packet-level statisticsndash Examine (and potentially log) variety of packet-level
statistics Essentially anything in the packetndash Timing
bull Flow-level Monitoringndash Monitor packet-by-packet (though sometimes
sampled)ndash Keep aggregate statistics on a flow
41
Packet Capture on High-Speed Links
Example Georgia Tech ldquoOC3Monrdquo
bull Rack-mounted PCbull Optical splitterbull Data Acquisition and
Generation (DAG) card
Source endacecom
42
Characteristics of Packet Capture
bull Allows inspection on every packet on 10G links
bull Disadvantagesndash Costlyndash Requires splitting optical fibersndash Must be able to filterstore data
43
Data Measurement Repositories
bull AbileneInternet 2 Observatoryndash Configuration examplesndash SNMP datandash ISIS BGP routing data NetFlow traffic data
bull RouteViewsndash BGP updatesndash BGP table snapshots
44
Multihoming and Traffic Engineering
45
What is Multihoming
bull The use of redundant network links for the purposes of external connectivity
bull Can be achieved at many layers of the protocol stack and many places in the networkndash Multiple network interfaces in a PC
ndash An ISP with multiple upstream interfaces
bull Can refer to having multiple connections tondash The same ISP
ndash Multiple ISPs
46
Why Multihome
bull Redundancybull Availabilitybull Performancebull Cost
Interdomain traffic engineering the process by which a multihomed network configures its
network to achieve these goals
47
Redundancy
bull Maintain connectivity in the face ofndash Physical connectivity problems (fiber cut device
failures etc)ndash Failures in upstream ISP
48
Performance
bull Use multiple network links at once to achieve higher throughput than just over a single link
bull Allows incoming traffic to be load-balanced
70 of traffic30 of traffic
49
Multihoming in IP Networks Today
bull Stub AS no transit service for other ASesndash No need to use BGP
bull Multi-homed stub AS has connectivity to multiple immediate upstream ISPsndash Need BGPndash No need for a public AS numberndash No need for IP prefix allocation
bull Multi-homed transit AS connectivity to multiple ASes and transit servicendash Need BGP public AS number IP prefix allocation
50
BGP or no
bull Advantages of static routingndash Cheapersmaller routers (less true nowadays)ndash Simpler to configure
bull Advantages of BGPndash More control of your destiny (have providers stop
announcing you)ndash Fastermore intelligent selection of where to send
outbound packetsndash Better debugging of net problems (you can see the
Internet topology now)
51
Same Provider or Multiple
bull If your provider is reliable and fast and affordably and offers good tech-support you may want to multi-home initially to them via some backup path (slow is better than dead)
bull Eventually yoursquoll want to multi-home to different providers to avoid failure modes due to one providerrsquos architecture decisions
52
Multihomed Stub One Link
bull Downstream ISPrsquos routers configure default (ldquostaticrdquo) routes pointing to border router
bull Upstream ISP advertises reachability
Upstream ISP
Multiple links between same pair of routers
Default routes to ldquoborderrdquo
ldquoStubrdquoISP
53
Multihomed Stub Multiple Links
bull Use BGP to share loadbull Use private AS number (why is this OK)bull As before upstream ISP advertises prefix
Upstream ISP
Multiple links to different upstream routers
ldquoStubrdquoISP
Internal routing for ldquohot potatordquo
BGP for load balance at edge
54
Multihomed Stub Multiple ISPs
bull Many possibilitiesndash Load sharingndash Primary-backupndash Selective use of different ISPs
bull Requires BGP public AS number etc
ldquoStubrdquoISP
Upstream
ISP 1
Upstream
ISP 2
55
Multihomed Transit Network
bull BGP everywherebull Incoming and outcoming trafficbull Challenge balancing load on intradomain and egress
links given an offered traffic load
TransitISP
ISP 1
ISP 2
ISP 3
56
Interdomain Traffic Engineering
bull The process by which a network operator configures the network to achievendash Traffic load balancendash Redundancy (primarybackup) etc
bull Two tasksndash Outbound traffic controlndash Inbound traffic control
bull Key Problems Predictability and Scalability
57
Outbound Traffic Control
bull Easier to control than inbound trafficndash Destination-based routing sender determines where
the packets go
bull Control over next-hop AS onlyndash Cannot control selection of the entire path
Provider 1 Provider 2
Control with local preference
58
Outbound Traffic Load Balancingbull Control routes to provider per-prefix
ndash Assign local preference across destination prefixesndash Change the local preference assignments over time
bull Useful inputs to load balancingndash End-to-end path performance datandash Outbound traffic statistics per destination prefix
bull Challenge Getting from traffic volumes to groups of prefixes that should be assigned to each link
Premise of ldquointelligent route controlrdquo preoducts
59
Traffic Engineering Goals
bull Predictabilityndash Ensure the BGP decision process is deterministicndash Assume that BGP updates are (relatively) stable
bull Limit overhead introduced by routing changesndash Minimize frequency of changes to routing policiesndash Limit number of prefixes affected by changes
bull Limit impact on how traffic enters the networkndash Avoid new routes that might change neighborrsquos mindndash Select route with same attributes or at least path length
60
Managing Scalebull Destination prefixes
ndash More than 90000 destination prefixes
bull Donrsquot want to have per-prefix routing policies
ndash Small fraction of prefixes contribute most of the traffic
bull Focus on the small number of heavy hitters
ndash Define routing policies for selected prefixes
bull Routing choicesndash About 27000 unique ldquorouting choicesrdquo
bull Help in reducing the scale of the problem
ndash Small fraction of ldquorouting choicesrdquo contribute most traffic
bull Focus on the very small number of ldquorouting choicesrdquo
ndash Define routing policies on common attributes
61
Achieving Predictability
bull Route prediction with static analysisndash Helpful to know effects before deploymentndash Static analysis can help
TopologyBGP policy
configuration
eBGP routes
Offered traffic
BGP routingmodel
Flow of traffic through the network
62
Challenges to Predictabilitybull For transit ISPs effects on incoming traffic
ndash Lack of coordination strikes again
63
ldquoHot Potatordquo routing
Inter-AS Negotiation
bull Coordination aids predictabilityndash Negotiate where to sendndash Inbound and outboundndash Mutual benefits
bull How to implementndash What info to exchangendash Protecting privacyndash How to prioritize choicesndash How to prevent cheating
Destination 2
Destination 1
multiplepeeringpoints
Provider A
Provider B
64
Outbound Multihoming Goals
bull Redundancyndash Dynamic routing will failover to backup link
bull Performancendash Select provider with best performance per prefix
ndash Requires active probing
bull Costndash Select provider per prefix over time to minimize the
total financial cost
65
Inbound Traffic Control
bull More difficult no control over neighborsrsquo decisions
bull Three common techniques (previously discussed)ndash AS path prependingndash Communities and local preferencendash Prefix splitting
How does todayrsquos paper (MONET) control inbound traffic
66
How many links are enough
K upstream ISPs
Not much benefit beyond 4 ISPs
Akella et al ldquoPerformance Benefits of Multihomingrdquo SIGCOMM 2003
67
Problems with Multihoming in IPv4
bull Routing table growthndash Provider-based addressingndash Advertising prefix out multiple ISPs ndash canrsquot aggregate
bull Poor control over inbound trafficndash Existing mechanisms do not allow hosts to control
inbound traffic
68
GeorgiaTech
Internet Routing Overview
bull Intradomain (ie ldquointra-ASrdquo) routingbull Interdomain routing
Comcast
Abilene
ATampT Cogent
Autonomous Systems (ASes)
69
Configuration Problems ldquoAS 7007rdquoldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
UUNet
Florida InternetBarn
Sprint
70
Diagnosis and Troubleshootingldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
ldquoMicrosofts websites were offline for up to 23 hoursbecause of a [router] misconfigurationhellipit took nearly a day to determine what was wrong and undo the changesrdquo -- wiredcom January 25 2001
ldquoWorldCom Inchellipsuffered a widespread outage on its Internet backbone that affected roughly 20 percent of its US customer base The network problemshellipaffected millions of computer users worldwide A spokeswoman attributed the outage to a route table issue -- cnncom October 3 2002
A number of Covad customers went out from 5pm today due to supposedly a DDOS (distributed denial of service attack) on a key Level3 data center which later was described as a route leak (misconfiguration)ldquo
-- dslreportscom February 23 2004
71
Operator Mailing List (NANOG)Date Mon 18 Oct 2004 091515 -0700Subject Level 3 US east coast issues
Level 3 experiencing widespread unspecified routing issues on the US east coast Master ticket 1086844 Anyone have more specific information
Date Mon 18 Oct 2004 122034 -0400 (EDT)Subject Re Level 3 US east coast issues
Level 3 is currently experiencing a backbone outage causing routing instability and packet loss We are working to restore and will be sending hourly updateshellip
72
Operator Mailing List
0102030405060708090
Filtering RouteLeaks
RouteHijacks
RouteInstability
RoutingLoops
Blackholes
Occurr
ences o
ver
10 Y
ears
1995-1997 1998-2001 2001-2004
Note Only includes problems openly discussed on this list
Compare 83 power outages 1 fire
73
Routing Configuration
Ranking route selection
Dissemination internal route advertisement
Filtering route advertisement
Customer
Competitor
Primary
Backup
74
Internet Business Model (Simplified)
bull CustomerProvider One AS pays another for reachability to some set of destinations
bull ldquoSettlement-freerdquo Peering Bartering Two ASes exchange routes with one another
Provider
Peer
Customer
Preferences implemented with local preference manipulation
Destination
Pay to use
Get paid to use
Free to use
75
Peering Contracts Consistent Export
bull Rules of settlement-free peeringndash Advertise routes at all peering pointsndash Advertised routes must have equal ldquoAS path lengthrdquo
Sprint
ATampT
Enables ldquohot potatordquo routing
ldquoequally goodrdquoroutes
76
Consistent Export
bull Malicedeceptionbull iBGP signaling partitionbull Inconsistent export policy
neighbor 10123route-map PEER permit 10 set prepend 123
neighbor 10456route-map PEER permit 10 set prepend 123 123
Possible Causes
10123 456 1
10456 456 2
Neighbor AS Export
1 1 123
Export Clause Prepend
2 1 123 123
Two different Export Policies
77
Inconsistent Export in Practice
Feamster et al ldquoBorderGuard Detecting Cold Potatoes from Peersrdquo ACM IMC October 2004
78
Blackholes
Date Thu 18 Jul 2002 060510 -0400 (EDT)From Chad Oleary ltcolpoboxcomgtSubject Re problems with 701To ltnanogmeritedugt
Were starting to see the same issues with UUNet again Anyone elseseeing this Trying to reach Qwest
traceroute to 631461901 (631461901) 30 hops max 38 byte packets 1 esc-lp2-gwe-solutionscorpcom (631182201) 1167 ms 1163 ms 1142 ms 2 500Serial2-10GW1TPA2ALTERNET (1571301499) 1097 ms 1059 ms 1044 ms 3 161at-1-0-0XL4ATL1ALTERNET (1526381190) 13839 ms 14108 ms 16638 ms 4 0so-3-1-0XL2ATL5ALTERNET (152630238) 14370 ms 14587 ms 14553 ms 5 POS7-0BR2ATL5ALTERNET (1526382193) 13928 ms 14099 ms 14053 ms 6 7 hellip
79
Security
80
Security ldquoBogonrdquo Routes
Feamster et al ldquoAn Empirical Study of lsquoBogonrsquo Route Advertisementsrdquo ACM CCR January 2005
81
Spam Phishing etc
bull Unsolicited commercial emailbull As of about August 2008 estimates indicate that
about 95 of all email is spambull Common spam filtering techniques
ndash Content-based filtersndash DNS Blacklist (DNSBL) lookups Significant fraction of
todayrsquos DNS traffic
Can IP addresses from which spam is received be spoofed
82
Spam and Routing
83
Worms and Botnets
84
What is a Worm
bull Code that replicates and propagates across the networkndash Often carries a ldquopayloadrdquo
bull Usually spread via exploiting flaws in open servicesndash ldquoVirusesrdquo require user action to spread
bull First worm Robert Morris November 1988ndash 6-10 of all Internet hosts infected ()
bull Many more since but none on that scale until July 2001
85
Example Worm Code Red
bull Initial version July 13 2001
bull Exploited known ISAPI vulnerability in Microsoft IIS Web servers
bull 1st through 20th of each month spread20th through end of each month attack
bull Payload Web site defacementbull Scanning Random IP addressesbull Bug failure to seed random number generator
86
Code Red Revisions
bull Released July 19 2001
bull Payload flooding attack on wwwwhitehousegovndash Attack was mounted at the IP address of the Web site
bull Bug died after 20th of each month
bull Random number generator for IP scanning fixed
87
Code Red Host Infection Rate
Exponential infection rate
Measured using backscatter technique
88
Designing Fast-Spreading Worms
bull Hit-list scanningndash Time to infect first 10k hosts dominates infection timendash Solution Reconnaissance (stealthy scans etc)
bull Permutation scanningndash Observation Most scanning is redundantndash Idea Shared permutation of address space Start scanning
from own IP address Re-randomize when another infected machine is found
bull Internet-scale hit listsndash Flash worm complete infection within 30 seconds
89
Botnets
bull Bots Autonomous programs performing tasksbull Plenty of ldquobenignrdquo bots
ndash eg weatherbug
bull Botnets group of bots ndash Typically carries malicious connotationndash Large numbers of infected machinesndash Machines ldquoenlistedrdquo with infection vectors like worms
(last lecture)
bull Available for simultaneous control by a masterbull Size up to 350000 nodes (from todayrsquos paper)
90
ldquoRallyingrdquo the Botnet
bull Easy to combine worm backdoor functionalitybull Problem how to learn about successfully
infected machines
bull Optionsndash Emailndash Hard-coded email address
91
Botnet Control
bull Botnet master typically runs some IRC server on a well-known port (eg 6667)
bull Infected machine contacts botnet with pre-programmed DNS name (eg big-botde)
bull Dynamic DNS allows controller to move about freely
Infected Machine
DynamicDNS
BotnetController
(IRC server)
92
Some Defenses
93
Idea 1 Ingress Filtering
bull RFC 2827 Routers install filters to drop packets from networks that are not downstream
bull Feasible at edgesbull Difficult to configure closer to network ldquocorerdquo
20469207024 Internet
Drop all packets with source address other than 20469207024
94
Idea 2 uRPF Checks
bull Unicast Reverse Path Forwardingndash Cisco ldquoip verify unicast reverse-pathrdquo
bull Requires symmetric routing
Accept packet from interface only if forwarding table entry for source IP address matches ingress interface
100183
A10018
10015
101203
100181 24
10011 24
Strict Mode uRPF
Enabled
ldquoArdquo Routing TableDestination Next Hop1001024 Int 110018024 Int 2
100183 from wrong interface
95
Problems with uRPF
bull Asymmetric routing
96
S-BGP
bull Address-based PKI validate signaturesndash Authentication of
bull ownership for IP address blocks bull AS number bull an ASs identity and bull a BGP routers identity
ndash Use existing infrastructure (Internet registries etc)ndash Routing origination is digitally signedndash BGP updates are digitally signed
1048708 bull Route attestations A new optional BGP transitive path attribute
ndash carries digital signatures covering the routing information in updates
97
Practical Problems with S-BGP
bull Requires Public-Key Infrastructure
bull Lots of digital signatures to calculate and verifyndash Message overheadndash CPU overhead
bull Calculation expense is greatest when topology is changingndash Caching can help
bull Route aggregation is problematic (maybe thatrsquos OK)
bull Secure route withdrawals when link or node fails
bull Address ownership data out of date
bull Deployment
- Networking
- Goal of This Tutorial
- Todayrsquos Networks
- Business Models
- Billing for Internet Usage
- Net Neutrality
- Network Operations
- Point-of-Presence (PoP)
- Example Abilene Network Topology
- Another Example Backbone
- Internet Routing Overview
- Internet Routing Protocol BGP
- Two Flavors of BGP
- IPv4 Addresses Networks of Networks
- Pre-1994 Classful Addressing
- Classless Interdomain Routing (CIDR)
- Benefits of CIDR
- Growth of IP Prefixes
- 1994-1998 Linear Growth
- Around 2000 Fast Growth Resumes
- Fast growth resumes
- The Address Allocation Process
- Common Problems
- What can go wrong
- Measurement and Monitoring
- Passive vs Active Measurement
- Slide 27
- Passive Traffic Data Measurement
- Simple Network Management Protocol
- SNMP (Passive)
- Packet-level Monitoring
- Full Packet Capture (Passive)
- What is a flow
- Cisco NetFlow
- Flow Record Contents
- Aggregating Packets into Flows
- Reducing Measurement Overhead
- Packet Sampling for Flow Monitoring
- Sampling Flow-Level Sampling
- Two Main Approaches
- Packet Capture on High-Speed Links
- Characteristics of Packet Capture
- Data Measurement Repositories
- Multihoming and Traffic Engineering
- What is Multihoming
- Why Multihome
- Redundancy
- Performance
- Multihoming in IP Networks Today
- BGP or no
- Same Provider or Multiple
- Multihomed Stub One Link
- Multihomed Stub Multiple Links
- Multihomed Stub Multiple ISPs
- Multihomed Transit Network
- Interdomain Traffic Engineering
- Outbound Traffic Control
- Outbound Traffic Load Balancing
- Traffic Engineering Goals
- Managing Scale
- Achieving Predictability
- Challenges to Predictability
- Inter-AS Negotiation
- Outbound Multihoming Goals
- Inbound Traffic Control
- How many links are enough
- Problems with Multihoming in IPv4
- Slide 68
- Slide 69
- Diagnosis and Troubleshooting
- Operator Mailing List (NANOG)
- Operator Mailing List
- Routing Configuration
- Internet Business Model (Simplified)
- Peering Contracts Consistent Export
- Consistent Export
- Inconsistent Export in Practice
- Blackholes
- Security
- Security ldquoBogonrdquo Routes
- Spam Phishing etc
- Spam and Routing
- Worms and Botnets
- What is a Worm
- Example Worm Code Red
- Code Red Revisions
- Code Red Host Infection Rate
- Designing Fast-Spreading Worms
- Botnets
- ldquoRallyingrdquo the Botnet
- Botnet Control
- Some Defenses
- Idea 1 Ingress Filtering
- Idea 2 uRPF Checks
- Problems with uRPF
- S-BGP
- Practical Problems with S-BGP
-
16
Classless Interdomain Routing (CIDR)
IP Address 65142480 ldquoMaskrdquo 2552552520
01000001 00001110 11111000 00000000
11111111 11111111 11111100 00000000
Use two 32-bit numbers to represent a network Network number = IP address + Mask
Example BellSouth Prefix 6514248022
Address no longer specifies network ID rangeNew forwarding trick Longest Prefix Match
17
Benefits of CIDR
bull Efficiency Can allocate blocks of prefixes on a finer granularity
bull Hierarchy Prefixes can be aggregated into supernets (Not always done Typically not in fact)
Customer 1
Customer 2
ATampT Internet
1220249024
1220231024120008
18
Growth of IP Prefixes
19
1994-1998 Linear Growth
bull About 10000 new entries per yearbull In theory less instability at the edges (why)
Source Geoff Huston
20
Around 2000 Fast Growth Resumes
Claim remaining 8s will be exhausted within the next 5-10 years
T Hain ldquoA Pragmatic Report on IPv4 Address Space Consumptionrdquo Cisco IPJ September 2005
21
Fast growth resumes
Rapid growth in routing tables
Dot-Bomb Hiccup
Significant contributor Multihoming
Source Geoff Huston
22
The Address Allocation Process
bull Allocation policies of RIRs affect pressure on IPv4 address space
IANA
AfriNIC APNIC ARIN LACNIC RIPE
httpwwwianaorgassignmentsipv4-address-space
Georgia Tech
23
Common Problems
bull Diagnosis and troubleshooting (hence measurement)bull Traffic engineeringbull Securitybull Design and capacity planning
24
What can go wrong
Two-thirds of the problems are caused by configuration of the routing protocol
Some downtime is very hard to protect againsthellip
25
Measurement and Monitoring
26
Passive vs Active Measurement
bull Passive Measurement Collection of packets flow statistics of traffic that is already flowing on the networkndash Packet tracesndash Flow statisticsndash Application-level logs
bull Active Measurement Inject ldquoprobingrdquo traffic to measure various characteristicsndash Traceroutendash Pingndash Application-level probes (eg Web downloads)
27
Billing for Internet Usage
bull 95th Percentile billingndash Customer network pays for ldquocommitted information
raterdquo (CIR)ndash Throughput measured every 5 minutes (typically with
SNMP flow statistics also can be used for billing)ndash Customer billed based on 95th percentile
28
Passive Traffic Data Measurement
bull SNMP bytepacket counts everywhere
bull Packet monitoring selected locations
bull Flow monitoring typically at edges (if possible)ndash Direct computation of the traffic matrixndash Input to denial-of-service attack detection
bull Deep Packet Inspection also at edge where possible
29
Simple Network Management Protocol
bull Management Information Base (MIB)ndash Information storendash Unique variables named by
OIDsndash Accessed with SNMP
bull Specific MIBs for bytepacket counts (per link)
Manager Agent
SNMP
DB
ManagedObjects
30
SNMP (Passive)
bull Advantage ubiquitousndash Supported on all networking equipmentndash Multiple products for polling and analyzing data
bull Disadvantages ndash Coarse granularityndash Cannot express complex queries on the datandash Unreliable delivery of the data using UDP
bull Utilityndash Link utilization (billing)ndash Traffic matrix inference
31
Packet-level Monitoring
bull Passive monitoring to collect full packet contents (or at least headers)
bull Advantages lots of detailed informationndash Precise timing informationndash Information in packet headers
bull Disadvantages overheadndash Hard to keep up with high-speed linksndash Often requires a separate monitoring device
32
Full Packet Capture (Passive)
Example Georgia Tech OC3Mon
bull Rack-mounted PCbull Optical splitterbull Data Acquisition and
Generation (DAG) card
Source endacecom
33
What is a flow
bull Source IP addressbull Destination IP addressbull Source portbull Destination portbull Layer 3 protocol typebull TOS byte (DSCP)bull Input logical interface (ifIndex)
34
Cisco NetFlow
bull Basic output ldquoFlow recordrdquondash Most common version is v5
bull Current version (9) is being standardized in the IETF (template-based)ndash More flexible record formatndash Much easier to add new flow record types
Core Network
Collection and Aggregation
Collector (PC)Approximately 1500 bytes
20-50 flow recordsSent more frequently if traffic increases
35
Flow Record Contents
bull Source and Destination IP address and portbull Packet and byte countsbull Start and end timesbull ToS TCP flags
Basic information about the flowhellip
hellipplus information related to routing
bull Next-hop IP addressbull Source and destination ASbull Source and destination prefix
36
flow 1 flow 2 flow 3 flow 4
Aggregating Packets into Flows
bull Criteria 1 Set of packets that ldquobelong togetherrdquondash Sourcedestination IP addresses and port numbersndash Same protocol ToS bits hellip ndash Same inputoutput interfaces at a router (if known)
bull Criteria 2 Packets that are ldquocloserdquo together in timendash Maximum inter-packet spacing (eg 15 sec 30 sec)ndash Example flows 2 and 4 are different flows due to time
37
Reducing Measurement Overhead
bull Filtering on interfacendash destination prefix for a customerndash port number for an application (eg 80 for Web)
bull Sampling before insertion into flow cachendash Random deterministic or hash-based samplingndash 1-out-of-n or stratified based on packetflow sizendash Two types packet-level and flow-level
bull Aggregation after cache evictionndash packetsflows with same next-hop ASndash packetsflows destined to a particular service
38
Packet Sampling for Flow Monitoring
bull Packet sampling before flow creation (Sampled Netflow)ndash 1-out-of-m sampling of individual packets (eg m=100)ndash Create of flow records over the sampled packets
bull Reducing overheadndash Avoid per-packet overhead on (m-1)m packetsndash Avoid creating records for a large number of small flows
bull Increasing overhead (in some cases)ndash May split some long transfers into multiple flow records ndash hellip due to larger time gaps between successive packets
time
not sampled
two flowstimeout
39
Sampling Flow-Level Sampling
bull Sampling of flow records evicted from flow cachendash When evicting flows from table or when analyzing flows
bull Stratified sampling to put weight on ldquoheavyrdquo flowsndash Select all long flows and sample the short flows
bull Reduces the number of flow records ndash Still measures the vast majority of the traffic
Flow 1 40 bytesFlow 2 15580 bytesFlow 3 8196 bytesFlow 4 5350789 bytesFlow 5 532 bytesFlow 6 7432 bytes
sample with 100 probability
sample with 01 probability
sample with 10 probability
40
Two Main Approaches
bull Packet-level Monitoringndash Keep packet-level statisticsndash Examine (and potentially log) variety of packet-level
statistics Essentially anything in the packetndash Timing
bull Flow-level Monitoringndash Monitor packet-by-packet (though sometimes
sampled)ndash Keep aggregate statistics on a flow
41
Packet Capture on High-Speed Links
Example Georgia Tech ldquoOC3Monrdquo
bull Rack-mounted PCbull Optical splitterbull Data Acquisition and
Generation (DAG) card
Source endacecom
42
Characteristics of Packet Capture
bull Allows inspection on every packet on 10G links
bull Disadvantagesndash Costlyndash Requires splitting optical fibersndash Must be able to filterstore data
43
Data Measurement Repositories
bull AbileneInternet 2 Observatoryndash Configuration examplesndash SNMP datandash ISIS BGP routing data NetFlow traffic data
bull RouteViewsndash BGP updatesndash BGP table snapshots
44
Multihoming and Traffic Engineering
45
What is Multihoming
bull The use of redundant network links for the purposes of external connectivity
bull Can be achieved at many layers of the protocol stack and many places in the networkndash Multiple network interfaces in a PC
ndash An ISP with multiple upstream interfaces
bull Can refer to having multiple connections tondash The same ISP
ndash Multiple ISPs
46
Why Multihome
bull Redundancybull Availabilitybull Performancebull Cost
Interdomain traffic engineering the process by which a multihomed network configures its
network to achieve these goals
47
Redundancy
bull Maintain connectivity in the face ofndash Physical connectivity problems (fiber cut device
failures etc)ndash Failures in upstream ISP
48
Performance
bull Use multiple network links at once to achieve higher throughput than just over a single link
bull Allows incoming traffic to be load-balanced
70 of traffic30 of traffic
49
Multihoming in IP Networks Today
bull Stub AS no transit service for other ASesndash No need to use BGP
bull Multi-homed stub AS has connectivity to multiple immediate upstream ISPsndash Need BGPndash No need for a public AS numberndash No need for IP prefix allocation
bull Multi-homed transit AS connectivity to multiple ASes and transit servicendash Need BGP public AS number IP prefix allocation
50
BGP or no
bull Advantages of static routingndash Cheapersmaller routers (less true nowadays)ndash Simpler to configure
bull Advantages of BGPndash More control of your destiny (have providers stop
announcing you)ndash Fastermore intelligent selection of where to send
outbound packetsndash Better debugging of net problems (you can see the
Internet topology now)
51
Same Provider or Multiple
bull If your provider is reliable and fast and affordably and offers good tech-support you may want to multi-home initially to them via some backup path (slow is better than dead)
bull Eventually yoursquoll want to multi-home to different providers to avoid failure modes due to one providerrsquos architecture decisions
52
Multihomed Stub One Link
bull Downstream ISPrsquos routers configure default (ldquostaticrdquo) routes pointing to border router
bull Upstream ISP advertises reachability
Upstream ISP
Multiple links between same pair of routers
Default routes to ldquoborderrdquo
ldquoStubrdquoISP
53
Multihomed Stub Multiple Links
bull Use BGP to share loadbull Use private AS number (why is this OK)bull As before upstream ISP advertises prefix
Upstream ISP
Multiple links to different upstream routers
ldquoStubrdquoISP
Internal routing for ldquohot potatordquo
BGP for load balance at edge
54
Multihomed Stub Multiple ISPs
bull Many possibilitiesndash Load sharingndash Primary-backupndash Selective use of different ISPs
bull Requires BGP public AS number etc
ldquoStubrdquoISP
Upstream
ISP 1
Upstream
ISP 2
55
Multihomed Transit Network
bull BGP everywherebull Incoming and outcoming trafficbull Challenge balancing load on intradomain and egress
links given an offered traffic load
TransitISP
ISP 1
ISP 2
ISP 3
56
Interdomain Traffic Engineering
bull The process by which a network operator configures the network to achievendash Traffic load balancendash Redundancy (primarybackup) etc
bull Two tasksndash Outbound traffic controlndash Inbound traffic control
bull Key Problems Predictability and Scalability
57
Outbound Traffic Control
bull Easier to control than inbound trafficndash Destination-based routing sender determines where
the packets go
bull Control over next-hop AS onlyndash Cannot control selection of the entire path
Provider 1 Provider 2
Control with local preference
58
Outbound Traffic Load Balancingbull Control routes to provider per-prefix
ndash Assign local preference across destination prefixesndash Change the local preference assignments over time
bull Useful inputs to load balancingndash End-to-end path performance datandash Outbound traffic statistics per destination prefix
bull Challenge Getting from traffic volumes to groups of prefixes that should be assigned to each link
Premise of ldquointelligent route controlrdquo preoducts
59
Traffic Engineering Goals
bull Predictabilityndash Ensure the BGP decision process is deterministicndash Assume that BGP updates are (relatively) stable
bull Limit overhead introduced by routing changesndash Minimize frequency of changes to routing policiesndash Limit number of prefixes affected by changes
bull Limit impact on how traffic enters the networkndash Avoid new routes that might change neighborrsquos mindndash Select route with same attributes or at least path length
60
Managing Scalebull Destination prefixes
ndash More than 90000 destination prefixes
bull Donrsquot want to have per-prefix routing policies
ndash Small fraction of prefixes contribute most of the traffic
bull Focus on the small number of heavy hitters
ndash Define routing policies for selected prefixes
bull Routing choicesndash About 27000 unique ldquorouting choicesrdquo
bull Help in reducing the scale of the problem
ndash Small fraction of ldquorouting choicesrdquo contribute most traffic
bull Focus on the very small number of ldquorouting choicesrdquo
ndash Define routing policies on common attributes
61
Achieving Predictability
bull Route prediction with static analysisndash Helpful to know effects before deploymentndash Static analysis can help
TopologyBGP policy
configuration
eBGP routes
Offered traffic
BGP routingmodel
Flow of traffic through the network
62
Challenges to Predictabilitybull For transit ISPs effects on incoming traffic
ndash Lack of coordination strikes again
63
ldquoHot Potatordquo routing
Inter-AS Negotiation
bull Coordination aids predictabilityndash Negotiate where to sendndash Inbound and outboundndash Mutual benefits
bull How to implementndash What info to exchangendash Protecting privacyndash How to prioritize choicesndash How to prevent cheating
Destination 2
Destination 1
multiplepeeringpoints
Provider A
Provider B
64
Outbound Multihoming Goals
bull Redundancyndash Dynamic routing will failover to backup link
bull Performancendash Select provider with best performance per prefix
ndash Requires active probing
bull Costndash Select provider per prefix over time to minimize the
total financial cost
65
Inbound Traffic Control
bull More difficult no control over neighborsrsquo decisions
bull Three common techniques (previously discussed)ndash AS path prependingndash Communities and local preferencendash Prefix splitting
How does todayrsquos paper (MONET) control inbound traffic
66
How many links are enough
K upstream ISPs
Not much benefit beyond 4 ISPs
Akella et al ldquoPerformance Benefits of Multihomingrdquo SIGCOMM 2003
67
Problems with Multihoming in IPv4
bull Routing table growthndash Provider-based addressingndash Advertising prefix out multiple ISPs ndash canrsquot aggregate
bull Poor control over inbound trafficndash Existing mechanisms do not allow hosts to control
inbound traffic
68
GeorgiaTech
Internet Routing Overview
bull Intradomain (ie ldquointra-ASrdquo) routingbull Interdomain routing
Comcast
Abilene
ATampT Cogent
Autonomous Systems (ASes)
69
Configuration Problems ldquoAS 7007rdquoldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
UUNet
Florida InternetBarn
Sprint
70
Diagnosis and Troubleshootingldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
ldquoMicrosofts websites were offline for up to 23 hoursbecause of a [router] misconfigurationhellipit took nearly a day to determine what was wrong and undo the changesrdquo -- wiredcom January 25 2001
ldquoWorldCom Inchellipsuffered a widespread outage on its Internet backbone that affected roughly 20 percent of its US customer base The network problemshellipaffected millions of computer users worldwide A spokeswoman attributed the outage to a route table issue -- cnncom October 3 2002
A number of Covad customers went out from 5pm today due to supposedly a DDOS (distributed denial of service attack) on a key Level3 data center which later was described as a route leak (misconfiguration)ldquo
-- dslreportscom February 23 2004
71
Operator Mailing List (NANOG)Date Mon 18 Oct 2004 091515 -0700Subject Level 3 US east coast issues
Level 3 experiencing widespread unspecified routing issues on the US east coast Master ticket 1086844 Anyone have more specific information
Date Mon 18 Oct 2004 122034 -0400 (EDT)Subject Re Level 3 US east coast issues
Level 3 is currently experiencing a backbone outage causing routing instability and packet loss We are working to restore and will be sending hourly updateshellip
72
Operator Mailing List
0102030405060708090
Filtering RouteLeaks
RouteHijacks
RouteInstability
RoutingLoops
Blackholes
Occurr
ences o
ver
10 Y
ears
1995-1997 1998-2001 2001-2004
Note Only includes problems openly discussed on this list
Compare 83 power outages 1 fire
73
Routing Configuration
Ranking route selection
Dissemination internal route advertisement
Filtering route advertisement
Customer
Competitor
Primary
Backup
74
Internet Business Model (Simplified)
bull CustomerProvider One AS pays another for reachability to some set of destinations
bull ldquoSettlement-freerdquo Peering Bartering Two ASes exchange routes with one another
Provider
Peer
Customer
Preferences implemented with local preference manipulation
Destination
Pay to use
Get paid to use
Free to use
75
Peering Contracts Consistent Export
bull Rules of settlement-free peeringndash Advertise routes at all peering pointsndash Advertised routes must have equal ldquoAS path lengthrdquo
Sprint
ATampT
Enables ldquohot potatordquo routing
ldquoequally goodrdquoroutes
76
Consistent Export
bull Malicedeceptionbull iBGP signaling partitionbull Inconsistent export policy
neighbor 10123route-map PEER permit 10 set prepend 123
neighbor 10456route-map PEER permit 10 set prepend 123 123
Possible Causes
10123 456 1
10456 456 2
Neighbor AS Export
1 1 123
Export Clause Prepend
2 1 123 123
Two different Export Policies
77
Inconsistent Export in Practice
Feamster et al ldquoBorderGuard Detecting Cold Potatoes from Peersrdquo ACM IMC October 2004
78
Blackholes
Date Thu 18 Jul 2002 060510 -0400 (EDT)From Chad Oleary ltcolpoboxcomgtSubject Re problems with 701To ltnanogmeritedugt
Were starting to see the same issues with UUNet again Anyone elseseeing this Trying to reach Qwest
traceroute to 631461901 (631461901) 30 hops max 38 byte packets 1 esc-lp2-gwe-solutionscorpcom (631182201) 1167 ms 1163 ms 1142 ms 2 500Serial2-10GW1TPA2ALTERNET (1571301499) 1097 ms 1059 ms 1044 ms 3 161at-1-0-0XL4ATL1ALTERNET (1526381190) 13839 ms 14108 ms 16638 ms 4 0so-3-1-0XL2ATL5ALTERNET (152630238) 14370 ms 14587 ms 14553 ms 5 POS7-0BR2ATL5ALTERNET (1526382193) 13928 ms 14099 ms 14053 ms 6 7 hellip
79
Security
80
Security ldquoBogonrdquo Routes
Feamster et al ldquoAn Empirical Study of lsquoBogonrsquo Route Advertisementsrdquo ACM CCR January 2005
81
Spam Phishing etc
bull Unsolicited commercial emailbull As of about August 2008 estimates indicate that
about 95 of all email is spambull Common spam filtering techniques
ndash Content-based filtersndash DNS Blacklist (DNSBL) lookups Significant fraction of
todayrsquos DNS traffic
Can IP addresses from which spam is received be spoofed
82
Spam and Routing
83
Worms and Botnets
84
What is a Worm
bull Code that replicates and propagates across the networkndash Often carries a ldquopayloadrdquo
bull Usually spread via exploiting flaws in open servicesndash ldquoVirusesrdquo require user action to spread
bull First worm Robert Morris November 1988ndash 6-10 of all Internet hosts infected ()
bull Many more since but none on that scale until July 2001
85
Example Worm Code Red
bull Initial version July 13 2001
bull Exploited known ISAPI vulnerability in Microsoft IIS Web servers
bull 1st through 20th of each month spread20th through end of each month attack
bull Payload Web site defacementbull Scanning Random IP addressesbull Bug failure to seed random number generator
86
Code Red Revisions
bull Released July 19 2001
bull Payload flooding attack on wwwwhitehousegovndash Attack was mounted at the IP address of the Web site
bull Bug died after 20th of each month
bull Random number generator for IP scanning fixed
87
Code Red Host Infection Rate
Exponential infection rate
Measured using backscatter technique
88
Designing Fast-Spreading Worms
bull Hit-list scanningndash Time to infect first 10k hosts dominates infection timendash Solution Reconnaissance (stealthy scans etc)
bull Permutation scanningndash Observation Most scanning is redundantndash Idea Shared permutation of address space Start scanning
from own IP address Re-randomize when another infected machine is found
bull Internet-scale hit listsndash Flash worm complete infection within 30 seconds
89
Botnets
bull Bots Autonomous programs performing tasksbull Plenty of ldquobenignrdquo bots
ndash eg weatherbug
bull Botnets group of bots ndash Typically carries malicious connotationndash Large numbers of infected machinesndash Machines ldquoenlistedrdquo with infection vectors like worms
(last lecture)
bull Available for simultaneous control by a masterbull Size up to 350000 nodes (from todayrsquos paper)
90
ldquoRallyingrdquo the Botnet
bull Easy to combine worm backdoor functionalitybull Problem how to learn about successfully
infected machines
bull Optionsndash Emailndash Hard-coded email address
91
Botnet Control
bull Botnet master typically runs some IRC server on a well-known port (eg 6667)
bull Infected machine contacts botnet with pre-programmed DNS name (eg big-botde)
bull Dynamic DNS allows controller to move about freely
Infected Machine
DynamicDNS
BotnetController
(IRC server)
92
Some Defenses
93
Idea 1 Ingress Filtering
bull RFC 2827 Routers install filters to drop packets from networks that are not downstream
bull Feasible at edgesbull Difficult to configure closer to network ldquocorerdquo
20469207024 Internet
Drop all packets with source address other than 20469207024
94
Idea 2 uRPF Checks
bull Unicast Reverse Path Forwardingndash Cisco ldquoip verify unicast reverse-pathrdquo
bull Requires symmetric routing
Accept packet from interface only if forwarding table entry for source IP address matches ingress interface
100183
A10018
10015
101203
100181 24
10011 24
Strict Mode uRPF
Enabled
ldquoArdquo Routing TableDestination Next Hop1001024 Int 110018024 Int 2
100183 from wrong interface
95
Problems with uRPF
bull Asymmetric routing
96
S-BGP
bull Address-based PKI validate signaturesndash Authentication of
bull ownership for IP address blocks bull AS number bull an ASs identity and bull a BGP routers identity
ndash Use existing infrastructure (Internet registries etc)ndash Routing origination is digitally signedndash BGP updates are digitally signed
1048708 bull Route attestations A new optional BGP transitive path attribute
ndash carries digital signatures covering the routing information in updates
97
Practical Problems with S-BGP
bull Requires Public-Key Infrastructure
bull Lots of digital signatures to calculate and verifyndash Message overheadndash CPU overhead
bull Calculation expense is greatest when topology is changingndash Caching can help
bull Route aggregation is problematic (maybe thatrsquos OK)
bull Secure route withdrawals when link or node fails
bull Address ownership data out of date
bull Deployment
- Networking
- Goal of This Tutorial
- Todayrsquos Networks
- Business Models
- Billing for Internet Usage
- Net Neutrality
- Network Operations
- Point-of-Presence (PoP)
- Example Abilene Network Topology
- Another Example Backbone
- Internet Routing Overview
- Internet Routing Protocol BGP
- Two Flavors of BGP
- IPv4 Addresses Networks of Networks
- Pre-1994 Classful Addressing
- Classless Interdomain Routing (CIDR)
- Benefits of CIDR
- Growth of IP Prefixes
- 1994-1998 Linear Growth
- Around 2000 Fast Growth Resumes
- Fast growth resumes
- The Address Allocation Process
- Common Problems
- What can go wrong
- Measurement and Monitoring
- Passive vs Active Measurement
- Slide 27
- Passive Traffic Data Measurement
- Simple Network Management Protocol
- SNMP (Passive)
- Packet-level Monitoring
- Full Packet Capture (Passive)
- What is a flow
- Cisco NetFlow
- Flow Record Contents
- Aggregating Packets into Flows
- Reducing Measurement Overhead
- Packet Sampling for Flow Monitoring
- Sampling Flow-Level Sampling
- Two Main Approaches
- Packet Capture on High-Speed Links
- Characteristics of Packet Capture
- Data Measurement Repositories
- Multihoming and Traffic Engineering
- What is Multihoming
- Why Multihome
- Redundancy
- Performance
- Multihoming in IP Networks Today
- BGP or no
- Same Provider or Multiple
- Multihomed Stub One Link
- Multihomed Stub Multiple Links
- Multihomed Stub Multiple ISPs
- Multihomed Transit Network
- Interdomain Traffic Engineering
- Outbound Traffic Control
- Outbound Traffic Load Balancing
- Traffic Engineering Goals
- Managing Scale
- Achieving Predictability
- Challenges to Predictability
- Inter-AS Negotiation
- Outbound Multihoming Goals
- Inbound Traffic Control
- How many links are enough
- Problems with Multihoming in IPv4
- Slide 68
- Slide 69
- Diagnosis and Troubleshooting
- Operator Mailing List (NANOG)
- Operator Mailing List
- Routing Configuration
- Internet Business Model (Simplified)
- Peering Contracts Consistent Export
- Consistent Export
- Inconsistent Export in Practice
- Blackholes
- Security
- Security ldquoBogonrdquo Routes
- Spam Phishing etc
- Spam and Routing
- Worms and Botnets
- What is a Worm
- Example Worm Code Red
- Code Red Revisions
- Code Red Host Infection Rate
- Designing Fast-Spreading Worms
- Botnets
- ldquoRallyingrdquo the Botnet
- Botnet Control
- Some Defenses
- Idea 1 Ingress Filtering
- Idea 2 uRPF Checks
- Problems with uRPF
- S-BGP
- Practical Problems with S-BGP
-
17
Benefits of CIDR
bull Efficiency Can allocate blocks of prefixes on a finer granularity
bull Hierarchy Prefixes can be aggregated into supernets (Not always done Typically not in fact)
Customer 1
Customer 2
ATampT Internet
1220249024
1220231024120008
18
Growth of IP Prefixes
19
1994-1998 Linear Growth
bull About 10000 new entries per yearbull In theory less instability at the edges (why)
Source Geoff Huston
20
Around 2000 Fast Growth Resumes
Claim remaining 8s will be exhausted within the next 5-10 years
T Hain ldquoA Pragmatic Report on IPv4 Address Space Consumptionrdquo Cisco IPJ September 2005
21
Fast growth resumes
Rapid growth in routing tables
Dot-Bomb Hiccup
Significant contributor Multihoming
Source Geoff Huston
22
The Address Allocation Process
bull Allocation policies of RIRs affect pressure on IPv4 address space
IANA
AfriNIC APNIC ARIN LACNIC RIPE
httpwwwianaorgassignmentsipv4-address-space
Georgia Tech
23
Common Problems
bull Diagnosis and troubleshooting (hence measurement)bull Traffic engineeringbull Securitybull Design and capacity planning
24
What can go wrong
Two-thirds of the problems are caused by configuration of the routing protocol
Some downtime is very hard to protect againsthellip
25
Measurement and Monitoring
26
Passive vs Active Measurement
bull Passive Measurement Collection of packets flow statistics of traffic that is already flowing on the networkndash Packet tracesndash Flow statisticsndash Application-level logs
bull Active Measurement Inject ldquoprobingrdquo traffic to measure various characteristicsndash Traceroutendash Pingndash Application-level probes (eg Web downloads)
27
Billing for Internet Usage
bull 95th Percentile billingndash Customer network pays for ldquocommitted information
raterdquo (CIR)ndash Throughput measured every 5 minutes (typically with
SNMP flow statistics also can be used for billing)ndash Customer billed based on 95th percentile
28
Passive Traffic Data Measurement
bull SNMP bytepacket counts everywhere
bull Packet monitoring selected locations
bull Flow monitoring typically at edges (if possible)ndash Direct computation of the traffic matrixndash Input to denial-of-service attack detection
bull Deep Packet Inspection also at edge where possible
29
Simple Network Management Protocol
bull Management Information Base (MIB)ndash Information storendash Unique variables named by
OIDsndash Accessed with SNMP
bull Specific MIBs for bytepacket counts (per link)
Manager Agent
SNMP
DB
ManagedObjects
30
SNMP (Passive)
bull Advantage ubiquitousndash Supported on all networking equipmentndash Multiple products for polling and analyzing data
bull Disadvantages ndash Coarse granularityndash Cannot express complex queries on the datandash Unreliable delivery of the data using UDP
bull Utilityndash Link utilization (billing)ndash Traffic matrix inference
31
Packet-level Monitoring
bull Passive monitoring to collect full packet contents (or at least headers)
bull Advantages lots of detailed informationndash Precise timing informationndash Information in packet headers
bull Disadvantages overheadndash Hard to keep up with high-speed linksndash Often requires a separate monitoring device
32
Full Packet Capture (Passive)
Example Georgia Tech OC3Mon
bull Rack-mounted PCbull Optical splitterbull Data Acquisition and
Generation (DAG) card
Source endacecom
33
What is a flow
bull Source IP addressbull Destination IP addressbull Source portbull Destination portbull Layer 3 protocol typebull TOS byte (DSCP)bull Input logical interface (ifIndex)
34
Cisco NetFlow
bull Basic output ldquoFlow recordrdquondash Most common version is v5
bull Current version (9) is being standardized in the IETF (template-based)ndash More flexible record formatndash Much easier to add new flow record types
Core Network
Collection and Aggregation
Collector (PC)Approximately 1500 bytes
20-50 flow recordsSent more frequently if traffic increases
35
Flow Record Contents
bull Source and Destination IP address and portbull Packet and byte countsbull Start and end timesbull ToS TCP flags
Basic information about the flowhellip
hellipplus information related to routing
bull Next-hop IP addressbull Source and destination ASbull Source and destination prefix
36
flow 1 flow 2 flow 3 flow 4
Aggregating Packets into Flows
bull Criteria 1 Set of packets that ldquobelong togetherrdquondash Sourcedestination IP addresses and port numbersndash Same protocol ToS bits hellip ndash Same inputoutput interfaces at a router (if known)
bull Criteria 2 Packets that are ldquocloserdquo together in timendash Maximum inter-packet spacing (eg 15 sec 30 sec)ndash Example flows 2 and 4 are different flows due to time
37
Reducing Measurement Overhead
bull Filtering on interfacendash destination prefix for a customerndash port number for an application (eg 80 for Web)
bull Sampling before insertion into flow cachendash Random deterministic or hash-based samplingndash 1-out-of-n or stratified based on packetflow sizendash Two types packet-level and flow-level
bull Aggregation after cache evictionndash packetsflows with same next-hop ASndash packetsflows destined to a particular service
38
Packet Sampling for Flow Monitoring
bull Packet sampling before flow creation (Sampled Netflow)ndash 1-out-of-m sampling of individual packets (eg m=100)ndash Create of flow records over the sampled packets
bull Reducing overheadndash Avoid per-packet overhead on (m-1)m packetsndash Avoid creating records for a large number of small flows
bull Increasing overhead (in some cases)ndash May split some long transfers into multiple flow records ndash hellip due to larger time gaps between successive packets
time
not sampled
two flowstimeout
39
Sampling Flow-Level Sampling
bull Sampling of flow records evicted from flow cachendash When evicting flows from table or when analyzing flows
bull Stratified sampling to put weight on ldquoheavyrdquo flowsndash Select all long flows and sample the short flows
bull Reduces the number of flow records ndash Still measures the vast majority of the traffic
Flow 1 40 bytesFlow 2 15580 bytesFlow 3 8196 bytesFlow 4 5350789 bytesFlow 5 532 bytesFlow 6 7432 bytes
sample with 100 probability
sample with 01 probability
sample with 10 probability
40
Two Main Approaches
bull Packet-level Monitoringndash Keep packet-level statisticsndash Examine (and potentially log) variety of packet-level
statistics Essentially anything in the packetndash Timing
bull Flow-level Monitoringndash Monitor packet-by-packet (though sometimes
sampled)ndash Keep aggregate statistics on a flow
41
Packet Capture on High-Speed Links
Example Georgia Tech ldquoOC3Monrdquo
bull Rack-mounted PCbull Optical splitterbull Data Acquisition and
Generation (DAG) card
Source endacecom
42
Characteristics of Packet Capture
bull Allows inspection on every packet on 10G links
bull Disadvantagesndash Costlyndash Requires splitting optical fibersndash Must be able to filterstore data
43
Data Measurement Repositories
bull AbileneInternet 2 Observatoryndash Configuration examplesndash SNMP datandash ISIS BGP routing data NetFlow traffic data
bull RouteViewsndash BGP updatesndash BGP table snapshots
44
Multihoming and Traffic Engineering
45
What is Multihoming
bull The use of redundant network links for the purposes of external connectivity
bull Can be achieved at many layers of the protocol stack and many places in the networkndash Multiple network interfaces in a PC
ndash An ISP with multiple upstream interfaces
bull Can refer to having multiple connections tondash The same ISP
ndash Multiple ISPs
46
Why Multihome
bull Redundancybull Availabilitybull Performancebull Cost
Interdomain traffic engineering the process by which a multihomed network configures its
network to achieve these goals
47
Redundancy
bull Maintain connectivity in the face ofndash Physical connectivity problems (fiber cut device
failures etc)ndash Failures in upstream ISP
48
Performance
bull Use multiple network links at once to achieve higher throughput than just over a single link
bull Allows incoming traffic to be load-balanced
70 of traffic30 of traffic
49
Multihoming in IP Networks Today
bull Stub AS no transit service for other ASesndash No need to use BGP
bull Multi-homed stub AS has connectivity to multiple immediate upstream ISPsndash Need BGPndash No need for a public AS numberndash No need for IP prefix allocation
bull Multi-homed transit AS connectivity to multiple ASes and transit servicendash Need BGP public AS number IP prefix allocation
50
BGP or no
bull Advantages of static routingndash Cheapersmaller routers (less true nowadays)ndash Simpler to configure
bull Advantages of BGPndash More control of your destiny (have providers stop
announcing you)ndash Fastermore intelligent selection of where to send
outbound packetsndash Better debugging of net problems (you can see the
Internet topology now)
51
Same Provider or Multiple
bull If your provider is reliable and fast and affordably and offers good tech-support you may want to multi-home initially to them via some backup path (slow is better than dead)
bull Eventually yoursquoll want to multi-home to different providers to avoid failure modes due to one providerrsquos architecture decisions
52
Multihomed Stub One Link
bull Downstream ISPrsquos routers configure default (ldquostaticrdquo) routes pointing to border router
bull Upstream ISP advertises reachability
Upstream ISP
Multiple links between same pair of routers
Default routes to ldquoborderrdquo
ldquoStubrdquoISP
53
Multihomed Stub Multiple Links
bull Use BGP to share loadbull Use private AS number (why is this OK)bull As before upstream ISP advertises prefix
Upstream ISP
Multiple links to different upstream routers
ldquoStubrdquoISP
Internal routing for ldquohot potatordquo
BGP for load balance at edge
54
Multihomed Stub Multiple ISPs
bull Many possibilitiesndash Load sharingndash Primary-backupndash Selective use of different ISPs
bull Requires BGP public AS number etc
ldquoStubrdquoISP
Upstream
ISP 1
Upstream
ISP 2
55
Multihomed Transit Network
bull BGP everywherebull Incoming and outcoming trafficbull Challenge balancing load on intradomain and egress
links given an offered traffic load
TransitISP
ISP 1
ISP 2
ISP 3
56
Interdomain Traffic Engineering
bull The process by which a network operator configures the network to achievendash Traffic load balancendash Redundancy (primarybackup) etc
bull Two tasksndash Outbound traffic controlndash Inbound traffic control
bull Key Problems Predictability and Scalability
57
Outbound Traffic Control
bull Easier to control than inbound trafficndash Destination-based routing sender determines where
the packets go
bull Control over next-hop AS onlyndash Cannot control selection of the entire path
Provider 1 Provider 2
Control with local preference
58
Outbound Traffic Load Balancingbull Control routes to provider per-prefix
ndash Assign local preference across destination prefixesndash Change the local preference assignments over time
bull Useful inputs to load balancingndash End-to-end path performance datandash Outbound traffic statistics per destination prefix
bull Challenge Getting from traffic volumes to groups of prefixes that should be assigned to each link
Premise of ldquointelligent route controlrdquo preoducts
59
Traffic Engineering Goals
bull Predictabilityndash Ensure the BGP decision process is deterministicndash Assume that BGP updates are (relatively) stable
bull Limit overhead introduced by routing changesndash Minimize frequency of changes to routing policiesndash Limit number of prefixes affected by changes
bull Limit impact on how traffic enters the networkndash Avoid new routes that might change neighborrsquos mindndash Select route with same attributes or at least path length
60
Managing Scalebull Destination prefixes
ndash More than 90000 destination prefixes
bull Donrsquot want to have per-prefix routing policies
ndash Small fraction of prefixes contribute most of the traffic
bull Focus on the small number of heavy hitters
ndash Define routing policies for selected prefixes
bull Routing choicesndash About 27000 unique ldquorouting choicesrdquo
bull Help in reducing the scale of the problem
ndash Small fraction of ldquorouting choicesrdquo contribute most traffic
bull Focus on the very small number of ldquorouting choicesrdquo
ndash Define routing policies on common attributes
61
Achieving Predictability
bull Route prediction with static analysisndash Helpful to know effects before deploymentndash Static analysis can help
TopologyBGP policy
configuration
eBGP routes
Offered traffic
BGP routingmodel
Flow of traffic through the network
62
Challenges to Predictabilitybull For transit ISPs effects on incoming traffic
ndash Lack of coordination strikes again
63
ldquoHot Potatordquo routing
Inter-AS Negotiation
bull Coordination aids predictabilityndash Negotiate where to sendndash Inbound and outboundndash Mutual benefits
bull How to implementndash What info to exchangendash Protecting privacyndash How to prioritize choicesndash How to prevent cheating
Destination 2
Destination 1
multiplepeeringpoints
Provider A
Provider B
64
Outbound Multihoming Goals
bull Redundancyndash Dynamic routing will failover to backup link
bull Performancendash Select provider with best performance per prefix
ndash Requires active probing
bull Costndash Select provider per prefix over time to minimize the
total financial cost
65
Inbound Traffic Control
bull More difficult no control over neighborsrsquo decisions
bull Three common techniques (previously discussed)ndash AS path prependingndash Communities and local preferencendash Prefix splitting
How does todayrsquos paper (MONET) control inbound traffic
66
How many links are enough
K upstream ISPs
Not much benefit beyond 4 ISPs
Akella et al ldquoPerformance Benefits of Multihomingrdquo SIGCOMM 2003
67
Problems with Multihoming in IPv4
bull Routing table growthndash Provider-based addressingndash Advertising prefix out multiple ISPs ndash canrsquot aggregate
bull Poor control over inbound trafficndash Existing mechanisms do not allow hosts to control
inbound traffic
68
GeorgiaTech
Internet Routing Overview
bull Intradomain (ie ldquointra-ASrdquo) routingbull Interdomain routing
Comcast
Abilene
ATampT Cogent
Autonomous Systems (ASes)
69
Configuration Problems ldquoAS 7007rdquoldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
UUNet
Florida InternetBarn
Sprint
70
Diagnosis and Troubleshootingldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
ldquoMicrosofts websites were offline for up to 23 hoursbecause of a [router] misconfigurationhellipit took nearly a day to determine what was wrong and undo the changesrdquo -- wiredcom January 25 2001
ldquoWorldCom Inchellipsuffered a widespread outage on its Internet backbone that affected roughly 20 percent of its US customer base The network problemshellipaffected millions of computer users worldwide A spokeswoman attributed the outage to a route table issue -- cnncom October 3 2002
A number of Covad customers went out from 5pm today due to supposedly a DDOS (distributed denial of service attack) on a key Level3 data center which later was described as a route leak (misconfiguration)ldquo
-- dslreportscom February 23 2004
71
Operator Mailing List (NANOG)Date Mon 18 Oct 2004 091515 -0700Subject Level 3 US east coast issues
Level 3 experiencing widespread unspecified routing issues on the US east coast Master ticket 1086844 Anyone have more specific information
Date Mon 18 Oct 2004 122034 -0400 (EDT)Subject Re Level 3 US east coast issues
Level 3 is currently experiencing a backbone outage causing routing instability and packet loss We are working to restore and will be sending hourly updateshellip
72
Operator Mailing List
0102030405060708090
Filtering RouteLeaks
RouteHijacks
RouteInstability
RoutingLoops
Blackholes
Occurr
ences o
ver
10 Y
ears
1995-1997 1998-2001 2001-2004
Note Only includes problems openly discussed on this list
Compare 83 power outages 1 fire
73
Routing Configuration
Ranking route selection
Dissemination internal route advertisement
Filtering route advertisement
Customer
Competitor
Primary
Backup
74
Internet Business Model (Simplified)
bull CustomerProvider One AS pays another for reachability to some set of destinations
bull ldquoSettlement-freerdquo Peering Bartering Two ASes exchange routes with one another
Provider
Peer
Customer
Preferences implemented with local preference manipulation
Destination
Pay to use
Get paid to use
Free to use
75
Peering Contracts Consistent Export
bull Rules of settlement-free peeringndash Advertise routes at all peering pointsndash Advertised routes must have equal ldquoAS path lengthrdquo
Sprint
ATampT
Enables ldquohot potatordquo routing
ldquoequally goodrdquoroutes
76
Consistent Export
bull Malicedeceptionbull iBGP signaling partitionbull Inconsistent export policy
neighbor 10123route-map PEER permit 10 set prepend 123
neighbor 10456route-map PEER permit 10 set prepend 123 123
Possible Causes
10123 456 1
10456 456 2
Neighbor AS Export
1 1 123
Export Clause Prepend
2 1 123 123
Two different Export Policies
77
Inconsistent Export in Practice
Feamster et al ldquoBorderGuard Detecting Cold Potatoes from Peersrdquo ACM IMC October 2004
78
Blackholes
Date Thu 18 Jul 2002 060510 -0400 (EDT)From Chad Oleary ltcolpoboxcomgtSubject Re problems with 701To ltnanogmeritedugt
Were starting to see the same issues with UUNet again Anyone elseseeing this Trying to reach Qwest
traceroute to 631461901 (631461901) 30 hops max 38 byte packets 1 esc-lp2-gwe-solutionscorpcom (631182201) 1167 ms 1163 ms 1142 ms 2 500Serial2-10GW1TPA2ALTERNET (1571301499) 1097 ms 1059 ms 1044 ms 3 161at-1-0-0XL4ATL1ALTERNET (1526381190) 13839 ms 14108 ms 16638 ms 4 0so-3-1-0XL2ATL5ALTERNET (152630238) 14370 ms 14587 ms 14553 ms 5 POS7-0BR2ATL5ALTERNET (1526382193) 13928 ms 14099 ms 14053 ms 6 7 hellip
79
Security
80
Security ldquoBogonrdquo Routes
Feamster et al ldquoAn Empirical Study of lsquoBogonrsquo Route Advertisementsrdquo ACM CCR January 2005
81
Spam Phishing etc
bull Unsolicited commercial emailbull As of about August 2008 estimates indicate that
about 95 of all email is spambull Common spam filtering techniques
ndash Content-based filtersndash DNS Blacklist (DNSBL) lookups Significant fraction of
todayrsquos DNS traffic
Can IP addresses from which spam is received be spoofed
82
Spam and Routing
83
Worms and Botnets
84
What is a Worm
bull Code that replicates and propagates across the networkndash Often carries a ldquopayloadrdquo
bull Usually spread via exploiting flaws in open servicesndash ldquoVirusesrdquo require user action to spread
bull First worm Robert Morris November 1988ndash 6-10 of all Internet hosts infected ()
bull Many more since but none on that scale until July 2001
85
Example Worm Code Red
bull Initial version July 13 2001
bull Exploited known ISAPI vulnerability in Microsoft IIS Web servers
bull 1st through 20th of each month spread20th through end of each month attack
bull Payload Web site defacementbull Scanning Random IP addressesbull Bug failure to seed random number generator
86
Code Red Revisions
bull Released July 19 2001
bull Payload flooding attack on wwwwhitehousegovndash Attack was mounted at the IP address of the Web site
bull Bug died after 20th of each month
bull Random number generator for IP scanning fixed
87
Code Red Host Infection Rate
Exponential infection rate
Measured using backscatter technique
88
Designing Fast-Spreading Worms
bull Hit-list scanningndash Time to infect first 10k hosts dominates infection timendash Solution Reconnaissance (stealthy scans etc)
bull Permutation scanningndash Observation Most scanning is redundantndash Idea Shared permutation of address space Start scanning
from own IP address Re-randomize when another infected machine is found
bull Internet-scale hit listsndash Flash worm complete infection within 30 seconds
89
Botnets
bull Bots Autonomous programs performing tasksbull Plenty of ldquobenignrdquo bots
ndash eg weatherbug
bull Botnets group of bots ndash Typically carries malicious connotationndash Large numbers of infected machinesndash Machines ldquoenlistedrdquo with infection vectors like worms
(last lecture)
bull Available for simultaneous control by a masterbull Size up to 350000 nodes (from todayrsquos paper)
90
ldquoRallyingrdquo the Botnet
bull Easy to combine worm backdoor functionalitybull Problem how to learn about successfully
infected machines
bull Optionsndash Emailndash Hard-coded email address
91
Botnet Control
bull Botnet master typically runs some IRC server on a well-known port (eg 6667)
bull Infected machine contacts botnet with pre-programmed DNS name (eg big-botde)
bull Dynamic DNS allows controller to move about freely
Infected Machine
DynamicDNS
BotnetController
(IRC server)
92
Some Defenses
93
Idea 1 Ingress Filtering
bull RFC 2827 Routers install filters to drop packets from networks that are not downstream
bull Feasible at edgesbull Difficult to configure closer to network ldquocorerdquo
20469207024 Internet
Drop all packets with source address other than 20469207024
94
Idea 2 uRPF Checks
bull Unicast Reverse Path Forwardingndash Cisco ldquoip verify unicast reverse-pathrdquo
bull Requires symmetric routing
Accept packet from interface only if forwarding table entry for source IP address matches ingress interface
100183
A10018
10015
101203
100181 24
10011 24
Strict Mode uRPF
Enabled
ldquoArdquo Routing TableDestination Next Hop1001024 Int 110018024 Int 2
100183 from wrong interface
95
Problems with uRPF
bull Asymmetric routing
96
S-BGP
bull Address-based PKI validate signaturesndash Authentication of
bull ownership for IP address blocks bull AS number bull an ASs identity and bull a BGP routers identity
ndash Use existing infrastructure (Internet registries etc)ndash Routing origination is digitally signedndash BGP updates are digitally signed
1048708 bull Route attestations A new optional BGP transitive path attribute
ndash carries digital signatures covering the routing information in updates
97
Practical Problems with S-BGP
bull Requires Public-Key Infrastructure
bull Lots of digital signatures to calculate and verifyndash Message overheadndash CPU overhead
bull Calculation expense is greatest when topology is changingndash Caching can help
bull Route aggregation is problematic (maybe thatrsquos OK)
bull Secure route withdrawals when link or node fails
bull Address ownership data out of date
bull Deployment
- Networking
- Goal of This Tutorial
- Todayrsquos Networks
- Business Models
- Billing for Internet Usage
- Net Neutrality
- Network Operations
- Point-of-Presence (PoP)
- Example Abilene Network Topology
- Another Example Backbone
- Internet Routing Overview
- Internet Routing Protocol BGP
- Two Flavors of BGP
- IPv4 Addresses Networks of Networks
- Pre-1994 Classful Addressing
- Classless Interdomain Routing (CIDR)
- Benefits of CIDR
- Growth of IP Prefixes
- 1994-1998 Linear Growth
- Around 2000 Fast Growth Resumes
- Fast growth resumes
- The Address Allocation Process
- Common Problems
- What can go wrong
- Measurement and Monitoring
- Passive vs Active Measurement
- Slide 27
- Passive Traffic Data Measurement
- Simple Network Management Protocol
- SNMP (Passive)
- Packet-level Monitoring
- Full Packet Capture (Passive)
- What is a flow
- Cisco NetFlow
- Flow Record Contents
- Aggregating Packets into Flows
- Reducing Measurement Overhead
- Packet Sampling for Flow Monitoring
- Sampling Flow-Level Sampling
- Two Main Approaches
- Packet Capture on High-Speed Links
- Characteristics of Packet Capture
- Data Measurement Repositories
- Multihoming and Traffic Engineering
- What is Multihoming
- Why Multihome
- Redundancy
- Performance
- Multihoming in IP Networks Today
- BGP or no
- Same Provider or Multiple
- Multihomed Stub One Link
- Multihomed Stub Multiple Links
- Multihomed Stub Multiple ISPs
- Multihomed Transit Network
- Interdomain Traffic Engineering
- Outbound Traffic Control
- Outbound Traffic Load Balancing
- Traffic Engineering Goals
- Managing Scale
- Achieving Predictability
- Challenges to Predictability
- Inter-AS Negotiation
- Outbound Multihoming Goals
- Inbound Traffic Control
- How many links are enough
- Problems with Multihoming in IPv4
- Slide 68
- Slide 69
- Diagnosis and Troubleshooting
- Operator Mailing List (NANOG)
- Operator Mailing List
- Routing Configuration
- Internet Business Model (Simplified)
- Peering Contracts Consistent Export
- Consistent Export
- Inconsistent Export in Practice
- Blackholes
- Security
- Security ldquoBogonrdquo Routes
- Spam Phishing etc
- Spam and Routing
- Worms and Botnets
- What is a Worm
- Example Worm Code Red
- Code Red Revisions
- Code Red Host Infection Rate
- Designing Fast-Spreading Worms
- Botnets
- ldquoRallyingrdquo the Botnet
- Botnet Control
- Some Defenses
- Idea 1 Ingress Filtering
- Idea 2 uRPF Checks
- Problems with uRPF
- S-BGP
- Practical Problems with S-BGP
-
18
Growth of IP Prefixes
19
1994-1998 Linear Growth
bull About 10000 new entries per yearbull In theory less instability at the edges (why)
Source Geoff Huston
20
Around 2000 Fast Growth Resumes
Claim remaining 8s will be exhausted within the next 5-10 years
T Hain ldquoA Pragmatic Report on IPv4 Address Space Consumptionrdquo Cisco IPJ September 2005
21
Fast growth resumes
Rapid growth in routing tables
Dot-Bomb Hiccup
Significant contributor Multihoming
Source Geoff Huston
22
The Address Allocation Process
bull Allocation policies of RIRs affect pressure on IPv4 address space
IANA
AfriNIC APNIC ARIN LACNIC RIPE
httpwwwianaorgassignmentsipv4-address-space
Georgia Tech
23
Common Problems
bull Diagnosis and troubleshooting (hence measurement)bull Traffic engineeringbull Securitybull Design and capacity planning
24
What can go wrong
Two-thirds of the problems are caused by configuration of the routing protocol
Some downtime is very hard to protect againsthellip
25
Measurement and Monitoring
26
Passive vs Active Measurement
bull Passive Measurement Collection of packets flow statistics of traffic that is already flowing on the networkndash Packet tracesndash Flow statisticsndash Application-level logs
bull Active Measurement Inject ldquoprobingrdquo traffic to measure various characteristicsndash Traceroutendash Pingndash Application-level probes (eg Web downloads)
27
Billing for Internet Usage
bull 95th Percentile billingndash Customer network pays for ldquocommitted information
raterdquo (CIR)ndash Throughput measured every 5 minutes (typically with
SNMP flow statistics also can be used for billing)ndash Customer billed based on 95th percentile
28
Passive Traffic Data Measurement
bull SNMP bytepacket counts everywhere
bull Packet monitoring selected locations
bull Flow monitoring typically at edges (if possible)ndash Direct computation of the traffic matrixndash Input to denial-of-service attack detection
bull Deep Packet Inspection also at edge where possible
29
Simple Network Management Protocol
bull Management Information Base (MIB)ndash Information storendash Unique variables named by
OIDsndash Accessed with SNMP
bull Specific MIBs for bytepacket counts (per link)
Manager Agent
SNMP
DB
ManagedObjects
30
SNMP (Passive)
bull Advantage ubiquitousndash Supported on all networking equipmentndash Multiple products for polling and analyzing data
bull Disadvantages ndash Coarse granularityndash Cannot express complex queries on the datandash Unreliable delivery of the data using UDP
bull Utilityndash Link utilization (billing)ndash Traffic matrix inference
31
Packet-level Monitoring
bull Passive monitoring to collect full packet contents (or at least headers)
bull Advantages lots of detailed informationndash Precise timing informationndash Information in packet headers
bull Disadvantages overheadndash Hard to keep up with high-speed linksndash Often requires a separate monitoring device
32
Full Packet Capture (Passive)
Example Georgia Tech OC3Mon
bull Rack-mounted PCbull Optical splitterbull Data Acquisition and
Generation (DAG) card
Source endacecom
33
What is a flow
bull Source IP addressbull Destination IP addressbull Source portbull Destination portbull Layer 3 protocol typebull TOS byte (DSCP)bull Input logical interface (ifIndex)
34
Cisco NetFlow
bull Basic output ldquoFlow recordrdquondash Most common version is v5
bull Current version (9) is being standardized in the IETF (template-based)ndash More flexible record formatndash Much easier to add new flow record types
Core Network
Collection and Aggregation
Collector (PC)Approximately 1500 bytes
20-50 flow recordsSent more frequently if traffic increases
35
Flow Record Contents
bull Source and Destination IP address and portbull Packet and byte countsbull Start and end timesbull ToS TCP flags
Basic information about the flowhellip
hellipplus information related to routing
bull Next-hop IP addressbull Source and destination ASbull Source and destination prefix
36
flow 1 flow 2 flow 3 flow 4
Aggregating Packets into Flows
bull Criteria 1 Set of packets that ldquobelong togetherrdquondash Sourcedestination IP addresses and port numbersndash Same protocol ToS bits hellip ndash Same inputoutput interfaces at a router (if known)
bull Criteria 2 Packets that are ldquocloserdquo together in timendash Maximum inter-packet spacing (eg 15 sec 30 sec)ndash Example flows 2 and 4 are different flows due to time
37
Reducing Measurement Overhead
bull Filtering on interfacendash destination prefix for a customerndash port number for an application (eg 80 for Web)
bull Sampling before insertion into flow cachendash Random deterministic or hash-based samplingndash 1-out-of-n or stratified based on packetflow sizendash Two types packet-level and flow-level
bull Aggregation after cache evictionndash packetsflows with same next-hop ASndash packetsflows destined to a particular service
38
Packet Sampling for Flow Monitoring
bull Packet sampling before flow creation (Sampled Netflow)ndash 1-out-of-m sampling of individual packets (eg m=100)ndash Create of flow records over the sampled packets
bull Reducing overheadndash Avoid per-packet overhead on (m-1)m packetsndash Avoid creating records for a large number of small flows
bull Increasing overhead (in some cases)ndash May split some long transfers into multiple flow records ndash hellip due to larger time gaps between successive packets
time
not sampled
two flowstimeout
39
Sampling Flow-Level Sampling
bull Sampling of flow records evicted from flow cachendash When evicting flows from table or when analyzing flows
bull Stratified sampling to put weight on ldquoheavyrdquo flowsndash Select all long flows and sample the short flows
bull Reduces the number of flow records ndash Still measures the vast majority of the traffic
Flow 1 40 bytesFlow 2 15580 bytesFlow 3 8196 bytesFlow 4 5350789 bytesFlow 5 532 bytesFlow 6 7432 bytes
sample with 100 probability
sample with 01 probability
sample with 10 probability
40
Two Main Approaches
bull Packet-level Monitoringndash Keep packet-level statisticsndash Examine (and potentially log) variety of packet-level
statistics Essentially anything in the packetndash Timing
bull Flow-level Monitoringndash Monitor packet-by-packet (though sometimes
sampled)ndash Keep aggregate statistics on a flow
41
Packet Capture on High-Speed Links
Example Georgia Tech ldquoOC3Monrdquo
bull Rack-mounted PCbull Optical splitterbull Data Acquisition and
Generation (DAG) card
Source endacecom
42
Characteristics of Packet Capture
bull Allows inspection on every packet on 10G links
bull Disadvantagesndash Costlyndash Requires splitting optical fibersndash Must be able to filterstore data
43
Data Measurement Repositories
bull AbileneInternet 2 Observatoryndash Configuration examplesndash SNMP datandash ISIS BGP routing data NetFlow traffic data
bull RouteViewsndash BGP updatesndash BGP table snapshots
44
Multihoming and Traffic Engineering
45
What is Multihoming
bull The use of redundant network links for the purposes of external connectivity
bull Can be achieved at many layers of the protocol stack and many places in the networkndash Multiple network interfaces in a PC
ndash An ISP with multiple upstream interfaces
bull Can refer to having multiple connections tondash The same ISP
ndash Multiple ISPs
46
Why Multihome
bull Redundancybull Availabilitybull Performancebull Cost
Interdomain traffic engineering the process by which a multihomed network configures its
network to achieve these goals
47
Redundancy
bull Maintain connectivity in the face ofndash Physical connectivity problems (fiber cut device
failures etc)ndash Failures in upstream ISP
48
Performance
bull Use multiple network links at once to achieve higher throughput than just over a single link
bull Allows incoming traffic to be load-balanced
70 of traffic30 of traffic
49
Multihoming in IP Networks Today
bull Stub AS no transit service for other ASesndash No need to use BGP
bull Multi-homed stub AS has connectivity to multiple immediate upstream ISPsndash Need BGPndash No need for a public AS numberndash No need for IP prefix allocation
bull Multi-homed transit AS connectivity to multiple ASes and transit servicendash Need BGP public AS number IP prefix allocation
50
BGP or no
bull Advantages of static routingndash Cheapersmaller routers (less true nowadays)ndash Simpler to configure
bull Advantages of BGPndash More control of your destiny (have providers stop
announcing you)ndash Fastermore intelligent selection of where to send
outbound packetsndash Better debugging of net problems (you can see the
Internet topology now)
51
Same Provider or Multiple
bull If your provider is reliable and fast and affordably and offers good tech-support you may want to multi-home initially to them via some backup path (slow is better than dead)
bull Eventually yoursquoll want to multi-home to different providers to avoid failure modes due to one providerrsquos architecture decisions
52
Multihomed Stub One Link
bull Downstream ISPrsquos routers configure default (ldquostaticrdquo) routes pointing to border router
bull Upstream ISP advertises reachability
Upstream ISP
Multiple links between same pair of routers
Default routes to ldquoborderrdquo
ldquoStubrdquoISP
53
Multihomed Stub Multiple Links
bull Use BGP to share loadbull Use private AS number (why is this OK)bull As before upstream ISP advertises prefix
Upstream ISP
Multiple links to different upstream routers
ldquoStubrdquoISP
Internal routing for ldquohot potatordquo
BGP for load balance at edge
54
Multihomed Stub Multiple ISPs
bull Many possibilitiesndash Load sharingndash Primary-backupndash Selective use of different ISPs
bull Requires BGP public AS number etc
ldquoStubrdquoISP
Upstream
ISP 1
Upstream
ISP 2
55
Multihomed Transit Network
bull BGP everywherebull Incoming and outcoming trafficbull Challenge balancing load on intradomain and egress
links given an offered traffic load
TransitISP
ISP 1
ISP 2
ISP 3
56
Interdomain Traffic Engineering
bull The process by which a network operator configures the network to achievendash Traffic load balancendash Redundancy (primarybackup) etc
bull Two tasksndash Outbound traffic controlndash Inbound traffic control
bull Key Problems Predictability and Scalability
57
Outbound Traffic Control
bull Easier to control than inbound trafficndash Destination-based routing sender determines where
the packets go
bull Control over next-hop AS onlyndash Cannot control selection of the entire path
Provider 1 Provider 2
Control with local preference
58
Outbound Traffic Load Balancingbull Control routes to provider per-prefix
ndash Assign local preference across destination prefixesndash Change the local preference assignments over time
bull Useful inputs to load balancingndash End-to-end path performance datandash Outbound traffic statistics per destination prefix
bull Challenge Getting from traffic volumes to groups of prefixes that should be assigned to each link
Premise of ldquointelligent route controlrdquo preoducts
59
Traffic Engineering Goals
bull Predictabilityndash Ensure the BGP decision process is deterministicndash Assume that BGP updates are (relatively) stable
bull Limit overhead introduced by routing changesndash Minimize frequency of changes to routing policiesndash Limit number of prefixes affected by changes
bull Limit impact on how traffic enters the networkndash Avoid new routes that might change neighborrsquos mindndash Select route with same attributes or at least path length
60
Managing Scalebull Destination prefixes
ndash More than 90000 destination prefixes
bull Donrsquot want to have per-prefix routing policies
ndash Small fraction of prefixes contribute most of the traffic
bull Focus on the small number of heavy hitters
ndash Define routing policies for selected prefixes
bull Routing choicesndash About 27000 unique ldquorouting choicesrdquo
bull Help in reducing the scale of the problem
ndash Small fraction of ldquorouting choicesrdquo contribute most traffic
bull Focus on the very small number of ldquorouting choicesrdquo
ndash Define routing policies on common attributes
61
Achieving Predictability
bull Route prediction with static analysisndash Helpful to know effects before deploymentndash Static analysis can help
TopologyBGP policy
configuration
eBGP routes
Offered traffic
BGP routingmodel
Flow of traffic through the network
62
Challenges to Predictabilitybull For transit ISPs effects on incoming traffic
ndash Lack of coordination strikes again
63
ldquoHot Potatordquo routing
Inter-AS Negotiation
bull Coordination aids predictabilityndash Negotiate where to sendndash Inbound and outboundndash Mutual benefits
bull How to implementndash What info to exchangendash Protecting privacyndash How to prioritize choicesndash How to prevent cheating
Destination 2
Destination 1
multiplepeeringpoints
Provider A
Provider B
64
Outbound Multihoming Goals
bull Redundancyndash Dynamic routing will failover to backup link
bull Performancendash Select provider with best performance per prefix
ndash Requires active probing
bull Costndash Select provider per prefix over time to minimize the
total financial cost
65
Inbound Traffic Control
bull More difficult no control over neighborsrsquo decisions
bull Three common techniques (previously discussed)ndash AS path prependingndash Communities and local preferencendash Prefix splitting
How does todayrsquos paper (MONET) control inbound traffic
66
How many links are enough
K upstream ISPs
Not much benefit beyond 4 ISPs
Akella et al ldquoPerformance Benefits of Multihomingrdquo SIGCOMM 2003
67
Problems with Multihoming in IPv4
bull Routing table growthndash Provider-based addressingndash Advertising prefix out multiple ISPs ndash canrsquot aggregate
bull Poor control over inbound trafficndash Existing mechanisms do not allow hosts to control
inbound traffic
68
GeorgiaTech
Internet Routing Overview
bull Intradomain (ie ldquointra-ASrdquo) routingbull Interdomain routing
Comcast
Abilene
ATampT Cogent
Autonomous Systems (ASes)
69
Configuration Problems ldquoAS 7007rdquoldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
UUNet
Florida InternetBarn
Sprint
70
Diagnosis and Troubleshootingldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
ldquoMicrosofts websites were offline for up to 23 hoursbecause of a [router] misconfigurationhellipit took nearly a day to determine what was wrong and undo the changesrdquo -- wiredcom January 25 2001
ldquoWorldCom Inchellipsuffered a widespread outage on its Internet backbone that affected roughly 20 percent of its US customer base The network problemshellipaffected millions of computer users worldwide A spokeswoman attributed the outage to a route table issue -- cnncom October 3 2002
A number of Covad customers went out from 5pm today due to supposedly a DDOS (distributed denial of service attack) on a key Level3 data center which later was described as a route leak (misconfiguration)ldquo
-- dslreportscom February 23 2004
71
Operator Mailing List (NANOG)Date Mon 18 Oct 2004 091515 -0700Subject Level 3 US east coast issues
Level 3 experiencing widespread unspecified routing issues on the US east coast Master ticket 1086844 Anyone have more specific information
Date Mon 18 Oct 2004 122034 -0400 (EDT)Subject Re Level 3 US east coast issues
Level 3 is currently experiencing a backbone outage causing routing instability and packet loss We are working to restore and will be sending hourly updateshellip
72
Operator Mailing List
0102030405060708090
Filtering RouteLeaks
RouteHijacks
RouteInstability
RoutingLoops
Blackholes
Occurr
ences o
ver
10 Y
ears
1995-1997 1998-2001 2001-2004
Note Only includes problems openly discussed on this list
Compare 83 power outages 1 fire
73
Routing Configuration
Ranking route selection
Dissemination internal route advertisement
Filtering route advertisement
Customer
Competitor
Primary
Backup
74
Internet Business Model (Simplified)
bull CustomerProvider One AS pays another for reachability to some set of destinations
bull ldquoSettlement-freerdquo Peering Bartering Two ASes exchange routes with one another
Provider
Peer
Customer
Preferences implemented with local preference manipulation
Destination
Pay to use
Get paid to use
Free to use
75
Peering Contracts Consistent Export
bull Rules of settlement-free peeringndash Advertise routes at all peering pointsndash Advertised routes must have equal ldquoAS path lengthrdquo
Sprint
ATampT
Enables ldquohot potatordquo routing
ldquoequally goodrdquoroutes
76
Consistent Export
bull Malicedeceptionbull iBGP signaling partitionbull Inconsistent export policy
neighbor 10123route-map PEER permit 10 set prepend 123
neighbor 10456route-map PEER permit 10 set prepend 123 123
Possible Causes
10123 456 1
10456 456 2
Neighbor AS Export
1 1 123
Export Clause Prepend
2 1 123 123
Two different Export Policies
77
Inconsistent Export in Practice
Feamster et al ldquoBorderGuard Detecting Cold Potatoes from Peersrdquo ACM IMC October 2004
78
Blackholes
Date Thu 18 Jul 2002 060510 -0400 (EDT)From Chad Oleary ltcolpoboxcomgtSubject Re problems with 701To ltnanogmeritedugt
Were starting to see the same issues with UUNet again Anyone elseseeing this Trying to reach Qwest
traceroute to 631461901 (631461901) 30 hops max 38 byte packets 1 esc-lp2-gwe-solutionscorpcom (631182201) 1167 ms 1163 ms 1142 ms 2 500Serial2-10GW1TPA2ALTERNET (1571301499) 1097 ms 1059 ms 1044 ms 3 161at-1-0-0XL4ATL1ALTERNET (1526381190) 13839 ms 14108 ms 16638 ms 4 0so-3-1-0XL2ATL5ALTERNET (152630238) 14370 ms 14587 ms 14553 ms 5 POS7-0BR2ATL5ALTERNET (1526382193) 13928 ms 14099 ms 14053 ms 6 7 hellip
79
Security
80
Security ldquoBogonrdquo Routes
Feamster et al ldquoAn Empirical Study of lsquoBogonrsquo Route Advertisementsrdquo ACM CCR January 2005
81
Spam Phishing etc
bull Unsolicited commercial emailbull As of about August 2008 estimates indicate that
about 95 of all email is spambull Common spam filtering techniques
ndash Content-based filtersndash DNS Blacklist (DNSBL) lookups Significant fraction of
todayrsquos DNS traffic
Can IP addresses from which spam is received be spoofed
82
Spam and Routing
83
Worms and Botnets
84
What is a Worm
bull Code that replicates and propagates across the networkndash Often carries a ldquopayloadrdquo
bull Usually spread via exploiting flaws in open servicesndash ldquoVirusesrdquo require user action to spread
bull First worm Robert Morris November 1988ndash 6-10 of all Internet hosts infected ()
bull Many more since but none on that scale until July 2001
85
Example Worm Code Red
bull Initial version July 13 2001
bull Exploited known ISAPI vulnerability in Microsoft IIS Web servers
bull 1st through 20th of each month spread20th through end of each month attack
bull Payload Web site defacementbull Scanning Random IP addressesbull Bug failure to seed random number generator
86
Code Red Revisions
bull Released July 19 2001
bull Payload flooding attack on wwwwhitehousegovndash Attack was mounted at the IP address of the Web site
bull Bug died after 20th of each month
bull Random number generator for IP scanning fixed
87
Code Red Host Infection Rate
Exponential infection rate
Measured using backscatter technique
88
Designing Fast-Spreading Worms
bull Hit-list scanningndash Time to infect first 10k hosts dominates infection timendash Solution Reconnaissance (stealthy scans etc)
bull Permutation scanningndash Observation Most scanning is redundantndash Idea Shared permutation of address space Start scanning
from own IP address Re-randomize when another infected machine is found
bull Internet-scale hit listsndash Flash worm complete infection within 30 seconds
89
Botnets
bull Bots Autonomous programs performing tasksbull Plenty of ldquobenignrdquo bots
ndash eg weatherbug
bull Botnets group of bots ndash Typically carries malicious connotationndash Large numbers of infected machinesndash Machines ldquoenlistedrdquo with infection vectors like worms
(last lecture)
bull Available for simultaneous control by a masterbull Size up to 350000 nodes (from todayrsquos paper)
90
ldquoRallyingrdquo the Botnet
bull Easy to combine worm backdoor functionalitybull Problem how to learn about successfully
infected machines
bull Optionsndash Emailndash Hard-coded email address
91
Botnet Control
bull Botnet master typically runs some IRC server on a well-known port (eg 6667)
bull Infected machine contacts botnet with pre-programmed DNS name (eg big-botde)
bull Dynamic DNS allows controller to move about freely
Infected Machine
DynamicDNS
BotnetController
(IRC server)
92
Some Defenses
93
Idea 1 Ingress Filtering
bull RFC 2827 Routers install filters to drop packets from networks that are not downstream
bull Feasible at edgesbull Difficult to configure closer to network ldquocorerdquo
20469207024 Internet
Drop all packets with source address other than 20469207024
94
Idea 2 uRPF Checks
bull Unicast Reverse Path Forwardingndash Cisco ldquoip verify unicast reverse-pathrdquo
bull Requires symmetric routing
Accept packet from interface only if forwarding table entry for source IP address matches ingress interface
100183
A10018
10015
101203
100181 24
10011 24
Strict Mode uRPF
Enabled
ldquoArdquo Routing TableDestination Next Hop1001024 Int 110018024 Int 2
100183 from wrong interface
95
Problems with uRPF
bull Asymmetric routing
96
S-BGP
bull Address-based PKI validate signaturesndash Authentication of
bull ownership for IP address blocks bull AS number bull an ASs identity and bull a BGP routers identity
ndash Use existing infrastructure (Internet registries etc)ndash Routing origination is digitally signedndash BGP updates are digitally signed
1048708 bull Route attestations A new optional BGP transitive path attribute
ndash carries digital signatures covering the routing information in updates
97
Practical Problems with S-BGP
bull Requires Public-Key Infrastructure
bull Lots of digital signatures to calculate and verifyndash Message overheadndash CPU overhead
bull Calculation expense is greatest when topology is changingndash Caching can help
bull Route aggregation is problematic (maybe thatrsquos OK)
bull Secure route withdrawals when link or node fails
bull Address ownership data out of date
bull Deployment
- Networking
- Goal of This Tutorial
- Todayrsquos Networks
- Business Models
- Billing for Internet Usage
- Net Neutrality
- Network Operations
- Point-of-Presence (PoP)
- Example Abilene Network Topology
- Another Example Backbone
- Internet Routing Overview
- Internet Routing Protocol BGP
- Two Flavors of BGP
- IPv4 Addresses Networks of Networks
- Pre-1994 Classful Addressing
- Classless Interdomain Routing (CIDR)
- Benefits of CIDR
- Growth of IP Prefixes
- 1994-1998 Linear Growth
- Around 2000 Fast Growth Resumes
- Fast growth resumes
- The Address Allocation Process
- Common Problems
- What can go wrong
- Measurement and Monitoring
- Passive vs Active Measurement
- Slide 27
- Passive Traffic Data Measurement
- Simple Network Management Protocol
- SNMP (Passive)
- Packet-level Monitoring
- Full Packet Capture (Passive)
- What is a flow
- Cisco NetFlow
- Flow Record Contents
- Aggregating Packets into Flows
- Reducing Measurement Overhead
- Packet Sampling for Flow Monitoring
- Sampling Flow-Level Sampling
- Two Main Approaches
- Packet Capture on High-Speed Links
- Characteristics of Packet Capture
- Data Measurement Repositories
- Multihoming and Traffic Engineering
- What is Multihoming
- Why Multihome
- Redundancy
- Performance
- Multihoming in IP Networks Today
- BGP or no
- Same Provider or Multiple
- Multihomed Stub One Link
- Multihomed Stub Multiple Links
- Multihomed Stub Multiple ISPs
- Multihomed Transit Network
- Interdomain Traffic Engineering
- Outbound Traffic Control
- Outbound Traffic Load Balancing
- Traffic Engineering Goals
- Managing Scale
- Achieving Predictability
- Challenges to Predictability
- Inter-AS Negotiation
- Outbound Multihoming Goals
- Inbound Traffic Control
- How many links are enough
- Problems with Multihoming in IPv4
- Slide 68
- Slide 69
- Diagnosis and Troubleshooting
- Operator Mailing List (NANOG)
- Operator Mailing List
- Routing Configuration
- Internet Business Model (Simplified)
- Peering Contracts Consistent Export
- Consistent Export
- Inconsistent Export in Practice
- Blackholes
- Security
- Security ldquoBogonrdquo Routes
- Spam Phishing etc
- Spam and Routing
- Worms and Botnets
- What is a Worm
- Example Worm Code Red
- Code Red Revisions
- Code Red Host Infection Rate
- Designing Fast-Spreading Worms
- Botnets
- ldquoRallyingrdquo the Botnet
- Botnet Control
- Some Defenses
- Idea 1 Ingress Filtering
- Idea 2 uRPF Checks
- Problems with uRPF
- S-BGP
- Practical Problems with S-BGP
-
19
1994-1998 Linear Growth
bull About 10000 new entries per yearbull In theory less instability at the edges (why)
Source Geoff Huston
20
Around 2000 Fast Growth Resumes
Claim remaining 8s will be exhausted within the next 5-10 years
T Hain ldquoA Pragmatic Report on IPv4 Address Space Consumptionrdquo Cisco IPJ September 2005
21
Fast growth resumes
Rapid growth in routing tables
Dot-Bomb Hiccup
Significant contributor Multihoming
Source Geoff Huston
22
The Address Allocation Process
bull Allocation policies of RIRs affect pressure on IPv4 address space
IANA
AfriNIC APNIC ARIN LACNIC RIPE
httpwwwianaorgassignmentsipv4-address-space
Georgia Tech
23
Common Problems
bull Diagnosis and troubleshooting (hence measurement)bull Traffic engineeringbull Securitybull Design and capacity planning
24
What can go wrong
Two-thirds of the problems are caused by configuration of the routing protocol
Some downtime is very hard to protect againsthellip
25
Measurement and Monitoring
26
Passive vs Active Measurement
bull Passive Measurement Collection of packets flow statistics of traffic that is already flowing on the networkndash Packet tracesndash Flow statisticsndash Application-level logs
bull Active Measurement Inject ldquoprobingrdquo traffic to measure various characteristicsndash Traceroutendash Pingndash Application-level probes (eg Web downloads)
27
Billing for Internet Usage
bull 95th Percentile billingndash Customer network pays for ldquocommitted information
raterdquo (CIR)ndash Throughput measured every 5 minutes (typically with
SNMP flow statistics also can be used for billing)ndash Customer billed based on 95th percentile
28
Passive Traffic Data Measurement
bull SNMP bytepacket counts everywhere
bull Packet monitoring selected locations
bull Flow monitoring typically at edges (if possible)ndash Direct computation of the traffic matrixndash Input to denial-of-service attack detection
bull Deep Packet Inspection also at edge where possible
29
Simple Network Management Protocol
bull Management Information Base (MIB)ndash Information storendash Unique variables named by
OIDsndash Accessed with SNMP
bull Specific MIBs for bytepacket counts (per link)
Manager Agent
SNMP
DB
ManagedObjects
30
SNMP (Passive)
bull Advantage ubiquitousndash Supported on all networking equipmentndash Multiple products for polling and analyzing data
bull Disadvantages ndash Coarse granularityndash Cannot express complex queries on the datandash Unreliable delivery of the data using UDP
bull Utilityndash Link utilization (billing)ndash Traffic matrix inference
31
Packet-level Monitoring
bull Passive monitoring to collect full packet contents (or at least headers)
bull Advantages lots of detailed informationndash Precise timing informationndash Information in packet headers
bull Disadvantages overheadndash Hard to keep up with high-speed linksndash Often requires a separate monitoring device
32
Full Packet Capture (Passive)
Example Georgia Tech OC3Mon
bull Rack-mounted PCbull Optical splitterbull Data Acquisition and
Generation (DAG) card
Source endacecom
33
What is a flow
bull Source IP addressbull Destination IP addressbull Source portbull Destination portbull Layer 3 protocol typebull TOS byte (DSCP)bull Input logical interface (ifIndex)
34
Cisco NetFlow
bull Basic output ldquoFlow recordrdquondash Most common version is v5
bull Current version (9) is being standardized in the IETF (template-based)ndash More flexible record formatndash Much easier to add new flow record types
Core Network
Collection and Aggregation
Collector (PC)Approximately 1500 bytes
20-50 flow recordsSent more frequently if traffic increases
35
Flow Record Contents
bull Source and Destination IP address and portbull Packet and byte countsbull Start and end timesbull ToS TCP flags
Basic information about the flowhellip
hellipplus information related to routing
bull Next-hop IP addressbull Source and destination ASbull Source and destination prefix
36
flow 1 flow 2 flow 3 flow 4
Aggregating Packets into Flows
bull Criteria 1 Set of packets that ldquobelong togetherrdquondash Sourcedestination IP addresses and port numbersndash Same protocol ToS bits hellip ndash Same inputoutput interfaces at a router (if known)
bull Criteria 2 Packets that are ldquocloserdquo together in timendash Maximum inter-packet spacing (eg 15 sec 30 sec)ndash Example flows 2 and 4 are different flows due to time
37
Reducing Measurement Overhead
bull Filtering on interfacendash destination prefix for a customerndash port number for an application (eg 80 for Web)
bull Sampling before insertion into flow cachendash Random deterministic or hash-based samplingndash 1-out-of-n or stratified based on packetflow sizendash Two types packet-level and flow-level
bull Aggregation after cache evictionndash packetsflows with same next-hop ASndash packetsflows destined to a particular service
38
Packet Sampling for Flow Monitoring
bull Packet sampling before flow creation (Sampled Netflow)ndash 1-out-of-m sampling of individual packets (eg m=100)ndash Create of flow records over the sampled packets
bull Reducing overheadndash Avoid per-packet overhead on (m-1)m packetsndash Avoid creating records for a large number of small flows
bull Increasing overhead (in some cases)ndash May split some long transfers into multiple flow records ndash hellip due to larger time gaps between successive packets
time
not sampled
two flowstimeout
39
Sampling Flow-Level Sampling
bull Sampling of flow records evicted from flow cachendash When evicting flows from table or when analyzing flows
bull Stratified sampling to put weight on ldquoheavyrdquo flowsndash Select all long flows and sample the short flows
bull Reduces the number of flow records ndash Still measures the vast majority of the traffic
Flow 1 40 bytesFlow 2 15580 bytesFlow 3 8196 bytesFlow 4 5350789 bytesFlow 5 532 bytesFlow 6 7432 bytes
sample with 100 probability
sample with 01 probability
sample with 10 probability
40
Two Main Approaches
bull Packet-level Monitoringndash Keep packet-level statisticsndash Examine (and potentially log) variety of packet-level
statistics Essentially anything in the packetndash Timing
bull Flow-level Monitoringndash Monitor packet-by-packet (though sometimes
sampled)ndash Keep aggregate statistics on a flow
41
Packet Capture on High-Speed Links
Example Georgia Tech ldquoOC3Monrdquo
bull Rack-mounted PCbull Optical splitterbull Data Acquisition and
Generation (DAG) card
Source endacecom
42
Characteristics of Packet Capture
bull Allows inspection on every packet on 10G links
bull Disadvantagesndash Costlyndash Requires splitting optical fibersndash Must be able to filterstore data
43
Data Measurement Repositories
bull AbileneInternet 2 Observatoryndash Configuration examplesndash SNMP datandash ISIS BGP routing data NetFlow traffic data
bull RouteViewsndash BGP updatesndash BGP table snapshots
44
Multihoming and Traffic Engineering
45
What is Multihoming
bull The use of redundant network links for the purposes of external connectivity
bull Can be achieved at many layers of the protocol stack and many places in the networkndash Multiple network interfaces in a PC
ndash An ISP with multiple upstream interfaces
bull Can refer to having multiple connections tondash The same ISP
ndash Multiple ISPs
46
Why Multihome
bull Redundancybull Availabilitybull Performancebull Cost
Interdomain traffic engineering the process by which a multihomed network configures its
network to achieve these goals
47
Redundancy
bull Maintain connectivity in the face ofndash Physical connectivity problems (fiber cut device
failures etc)ndash Failures in upstream ISP
48
Performance
bull Use multiple network links at once to achieve higher throughput than just over a single link
bull Allows incoming traffic to be load-balanced
70 of traffic30 of traffic
49
Multihoming in IP Networks Today
bull Stub AS no transit service for other ASesndash No need to use BGP
bull Multi-homed stub AS has connectivity to multiple immediate upstream ISPsndash Need BGPndash No need for a public AS numberndash No need for IP prefix allocation
bull Multi-homed transit AS connectivity to multiple ASes and transit servicendash Need BGP public AS number IP prefix allocation
50
BGP or no
bull Advantages of static routingndash Cheapersmaller routers (less true nowadays)ndash Simpler to configure
bull Advantages of BGPndash More control of your destiny (have providers stop
announcing you)ndash Fastermore intelligent selection of where to send
outbound packetsndash Better debugging of net problems (you can see the
Internet topology now)
51
Same Provider or Multiple
bull If your provider is reliable and fast and affordably and offers good tech-support you may want to multi-home initially to them via some backup path (slow is better than dead)
bull Eventually yoursquoll want to multi-home to different providers to avoid failure modes due to one providerrsquos architecture decisions
52
Multihomed Stub One Link
bull Downstream ISPrsquos routers configure default (ldquostaticrdquo) routes pointing to border router
bull Upstream ISP advertises reachability
Upstream ISP
Multiple links between same pair of routers
Default routes to ldquoborderrdquo
ldquoStubrdquoISP
53
Multihomed Stub Multiple Links
bull Use BGP to share loadbull Use private AS number (why is this OK)bull As before upstream ISP advertises prefix
Upstream ISP
Multiple links to different upstream routers
ldquoStubrdquoISP
Internal routing for ldquohot potatordquo
BGP for load balance at edge
54
Multihomed Stub Multiple ISPs
bull Many possibilitiesndash Load sharingndash Primary-backupndash Selective use of different ISPs
bull Requires BGP public AS number etc
ldquoStubrdquoISP
Upstream
ISP 1
Upstream
ISP 2
55
Multihomed Transit Network
bull BGP everywherebull Incoming and outcoming trafficbull Challenge balancing load on intradomain and egress
links given an offered traffic load
TransitISP
ISP 1
ISP 2
ISP 3
56
Interdomain Traffic Engineering
bull The process by which a network operator configures the network to achievendash Traffic load balancendash Redundancy (primarybackup) etc
bull Two tasksndash Outbound traffic controlndash Inbound traffic control
bull Key Problems Predictability and Scalability
57
Outbound Traffic Control
bull Easier to control than inbound trafficndash Destination-based routing sender determines where
the packets go
bull Control over next-hop AS onlyndash Cannot control selection of the entire path
Provider 1 Provider 2
Control with local preference
58
Outbound Traffic Load Balancingbull Control routes to provider per-prefix
ndash Assign local preference across destination prefixesndash Change the local preference assignments over time
bull Useful inputs to load balancingndash End-to-end path performance datandash Outbound traffic statistics per destination prefix
bull Challenge Getting from traffic volumes to groups of prefixes that should be assigned to each link
Premise of ldquointelligent route controlrdquo preoducts
59
Traffic Engineering Goals
bull Predictabilityndash Ensure the BGP decision process is deterministicndash Assume that BGP updates are (relatively) stable
bull Limit overhead introduced by routing changesndash Minimize frequency of changes to routing policiesndash Limit number of prefixes affected by changes
bull Limit impact on how traffic enters the networkndash Avoid new routes that might change neighborrsquos mindndash Select route with same attributes or at least path length
60
Managing Scalebull Destination prefixes
ndash More than 90000 destination prefixes
bull Donrsquot want to have per-prefix routing policies
ndash Small fraction of prefixes contribute most of the traffic
bull Focus on the small number of heavy hitters
ndash Define routing policies for selected prefixes
bull Routing choicesndash About 27000 unique ldquorouting choicesrdquo
bull Help in reducing the scale of the problem
ndash Small fraction of ldquorouting choicesrdquo contribute most traffic
bull Focus on the very small number of ldquorouting choicesrdquo
ndash Define routing policies on common attributes
61
Achieving Predictability
bull Route prediction with static analysisndash Helpful to know effects before deploymentndash Static analysis can help
TopologyBGP policy
configuration
eBGP routes
Offered traffic
BGP routingmodel
Flow of traffic through the network
62
Challenges to Predictabilitybull For transit ISPs effects on incoming traffic
ndash Lack of coordination strikes again
63
ldquoHot Potatordquo routing
Inter-AS Negotiation
bull Coordination aids predictabilityndash Negotiate where to sendndash Inbound and outboundndash Mutual benefits
bull How to implementndash What info to exchangendash Protecting privacyndash How to prioritize choicesndash How to prevent cheating
Destination 2
Destination 1
multiplepeeringpoints
Provider A
Provider B
64
Outbound Multihoming Goals
bull Redundancyndash Dynamic routing will failover to backup link
bull Performancendash Select provider with best performance per prefix
ndash Requires active probing
bull Costndash Select provider per prefix over time to minimize the
total financial cost
65
Inbound Traffic Control
bull More difficult no control over neighborsrsquo decisions
bull Three common techniques (previously discussed)ndash AS path prependingndash Communities and local preferencendash Prefix splitting
How does todayrsquos paper (MONET) control inbound traffic
66
How many links are enough
K upstream ISPs
Not much benefit beyond 4 ISPs
Akella et al ldquoPerformance Benefits of Multihomingrdquo SIGCOMM 2003
67
Problems with Multihoming in IPv4
bull Routing table growthndash Provider-based addressingndash Advertising prefix out multiple ISPs ndash canrsquot aggregate
bull Poor control over inbound trafficndash Existing mechanisms do not allow hosts to control
inbound traffic
68
GeorgiaTech
Internet Routing Overview
bull Intradomain (ie ldquointra-ASrdquo) routingbull Interdomain routing
Comcast
Abilene
ATampT Cogent
Autonomous Systems (ASes)
69
Configuration Problems ldquoAS 7007rdquoldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
UUNet
Florida InternetBarn
Sprint
70
Diagnosis and Troubleshootingldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
ldquoMicrosofts websites were offline for up to 23 hoursbecause of a [router] misconfigurationhellipit took nearly a day to determine what was wrong and undo the changesrdquo -- wiredcom January 25 2001
ldquoWorldCom Inchellipsuffered a widespread outage on its Internet backbone that affected roughly 20 percent of its US customer base The network problemshellipaffected millions of computer users worldwide A spokeswoman attributed the outage to a route table issue -- cnncom October 3 2002
A number of Covad customers went out from 5pm today due to supposedly a DDOS (distributed denial of service attack) on a key Level3 data center which later was described as a route leak (misconfiguration)ldquo
-- dslreportscom February 23 2004
71
Operator Mailing List (NANOG)Date Mon 18 Oct 2004 091515 -0700Subject Level 3 US east coast issues
Level 3 experiencing widespread unspecified routing issues on the US east coast Master ticket 1086844 Anyone have more specific information
Date Mon 18 Oct 2004 122034 -0400 (EDT)Subject Re Level 3 US east coast issues
Level 3 is currently experiencing a backbone outage causing routing instability and packet loss We are working to restore and will be sending hourly updateshellip
72
Operator Mailing List
0102030405060708090
Filtering RouteLeaks
RouteHijacks
RouteInstability
RoutingLoops
Blackholes
Occurr
ences o
ver
10 Y
ears
1995-1997 1998-2001 2001-2004
Note Only includes problems openly discussed on this list
Compare 83 power outages 1 fire
73
Routing Configuration
Ranking route selection
Dissemination internal route advertisement
Filtering route advertisement
Customer
Competitor
Primary
Backup
74
Internet Business Model (Simplified)
bull CustomerProvider One AS pays another for reachability to some set of destinations
bull ldquoSettlement-freerdquo Peering Bartering Two ASes exchange routes with one another
Provider
Peer
Customer
Preferences implemented with local preference manipulation
Destination
Pay to use
Get paid to use
Free to use
75
Peering Contracts Consistent Export
bull Rules of settlement-free peeringndash Advertise routes at all peering pointsndash Advertised routes must have equal ldquoAS path lengthrdquo
Sprint
ATampT
Enables ldquohot potatordquo routing
ldquoequally goodrdquoroutes
76
Consistent Export
bull Malicedeceptionbull iBGP signaling partitionbull Inconsistent export policy
neighbor 10123route-map PEER permit 10 set prepend 123
neighbor 10456route-map PEER permit 10 set prepend 123 123
Possible Causes
10123 456 1
10456 456 2
Neighbor AS Export
1 1 123
Export Clause Prepend
2 1 123 123
Two different Export Policies
77
Inconsistent Export in Practice
Feamster et al ldquoBorderGuard Detecting Cold Potatoes from Peersrdquo ACM IMC October 2004
78
Blackholes
Date Thu 18 Jul 2002 060510 -0400 (EDT)From Chad Oleary ltcolpoboxcomgtSubject Re problems with 701To ltnanogmeritedugt
Were starting to see the same issues with UUNet again Anyone elseseeing this Trying to reach Qwest
traceroute to 631461901 (631461901) 30 hops max 38 byte packets 1 esc-lp2-gwe-solutionscorpcom (631182201) 1167 ms 1163 ms 1142 ms 2 500Serial2-10GW1TPA2ALTERNET (1571301499) 1097 ms 1059 ms 1044 ms 3 161at-1-0-0XL4ATL1ALTERNET (1526381190) 13839 ms 14108 ms 16638 ms 4 0so-3-1-0XL2ATL5ALTERNET (152630238) 14370 ms 14587 ms 14553 ms 5 POS7-0BR2ATL5ALTERNET (1526382193) 13928 ms 14099 ms 14053 ms 6 7 hellip
79
Security
80
Security ldquoBogonrdquo Routes
Feamster et al ldquoAn Empirical Study of lsquoBogonrsquo Route Advertisementsrdquo ACM CCR January 2005
81
Spam Phishing etc
bull Unsolicited commercial emailbull As of about August 2008 estimates indicate that
about 95 of all email is spambull Common spam filtering techniques
ndash Content-based filtersndash DNS Blacklist (DNSBL) lookups Significant fraction of
todayrsquos DNS traffic
Can IP addresses from which spam is received be spoofed
82
Spam and Routing
83
Worms and Botnets
84
What is a Worm
bull Code that replicates and propagates across the networkndash Often carries a ldquopayloadrdquo
bull Usually spread via exploiting flaws in open servicesndash ldquoVirusesrdquo require user action to spread
bull First worm Robert Morris November 1988ndash 6-10 of all Internet hosts infected ()
bull Many more since but none on that scale until July 2001
85
Example Worm Code Red
bull Initial version July 13 2001
bull Exploited known ISAPI vulnerability in Microsoft IIS Web servers
bull 1st through 20th of each month spread20th through end of each month attack
bull Payload Web site defacementbull Scanning Random IP addressesbull Bug failure to seed random number generator
86
Code Red Revisions
bull Released July 19 2001
bull Payload flooding attack on wwwwhitehousegovndash Attack was mounted at the IP address of the Web site
bull Bug died after 20th of each month
bull Random number generator for IP scanning fixed
87
Code Red Host Infection Rate
Exponential infection rate
Measured using backscatter technique
88
Designing Fast-Spreading Worms
bull Hit-list scanningndash Time to infect first 10k hosts dominates infection timendash Solution Reconnaissance (stealthy scans etc)
bull Permutation scanningndash Observation Most scanning is redundantndash Idea Shared permutation of address space Start scanning
from own IP address Re-randomize when another infected machine is found
bull Internet-scale hit listsndash Flash worm complete infection within 30 seconds
89
Botnets
bull Bots Autonomous programs performing tasksbull Plenty of ldquobenignrdquo bots
ndash eg weatherbug
bull Botnets group of bots ndash Typically carries malicious connotationndash Large numbers of infected machinesndash Machines ldquoenlistedrdquo with infection vectors like worms
(last lecture)
bull Available for simultaneous control by a masterbull Size up to 350000 nodes (from todayrsquos paper)
90
ldquoRallyingrdquo the Botnet
bull Easy to combine worm backdoor functionalitybull Problem how to learn about successfully
infected machines
bull Optionsndash Emailndash Hard-coded email address
91
Botnet Control
bull Botnet master typically runs some IRC server on a well-known port (eg 6667)
bull Infected machine contacts botnet with pre-programmed DNS name (eg big-botde)
bull Dynamic DNS allows controller to move about freely
Infected Machine
DynamicDNS
BotnetController
(IRC server)
92
Some Defenses
93
Idea 1 Ingress Filtering
bull RFC 2827 Routers install filters to drop packets from networks that are not downstream
bull Feasible at edgesbull Difficult to configure closer to network ldquocorerdquo
20469207024 Internet
Drop all packets with source address other than 20469207024
94
Idea 2 uRPF Checks
bull Unicast Reverse Path Forwardingndash Cisco ldquoip verify unicast reverse-pathrdquo
bull Requires symmetric routing
Accept packet from interface only if forwarding table entry for source IP address matches ingress interface
100183
A10018
10015
101203
100181 24
10011 24
Strict Mode uRPF
Enabled
ldquoArdquo Routing TableDestination Next Hop1001024 Int 110018024 Int 2
100183 from wrong interface
95
Problems with uRPF
bull Asymmetric routing
96
S-BGP
bull Address-based PKI validate signaturesndash Authentication of
bull ownership for IP address blocks bull AS number bull an ASs identity and bull a BGP routers identity
ndash Use existing infrastructure (Internet registries etc)ndash Routing origination is digitally signedndash BGP updates are digitally signed
1048708 bull Route attestations A new optional BGP transitive path attribute
ndash carries digital signatures covering the routing information in updates
97
Practical Problems with S-BGP
bull Requires Public-Key Infrastructure
bull Lots of digital signatures to calculate and verifyndash Message overheadndash CPU overhead
bull Calculation expense is greatest when topology is changingndash Caching can help
bull Route aggregation is problematic (maybe thatrsquos OK)
bull Secure route withdrawals when link or node fails
bull Address ownership data out of date
bull Deployment
- Networking
- Goal of This Tutorial
- Todayrsquos Networks
- Business Models
- Billing for Internet Usage
- Net Neutrality
- Network Operations
- Point-of-Presence (PoP)
- Example Abilene Network Topology
- Another Example Backbone
- Internet Routing Overview
- Internet Routing Protocol BGP
- Two Flavors of BGP
- IPv4 Addresses Networks of Networks
- Pre-1994 Classful Addressing
- Classless Interdomain Routing (CIDR)
- Benefits of CIDR
- Growth of IP Prefixes
- 1994-1998 Linear Growth
- Around 2000 Fast Growth Resumes
- Fast growth resumes
- The Address Allocation Process
- Common Problems
- What can go wrong
- Measurement and Monitoring
- Passive vs Active Measurement
- Slide 27
- Passive Traffic Data Measurement
- Simple Network Management Protocol
- SNMP (Passive)
- Packet-level Monitoring
- Full Packet Capture (Passive)
- What is a flow
- Cisco NetFlow
- Flow Record Contents
- Aggregating Packets into Flows
- Reducing Measurement Overhead
- Packet Sampling for Flow Monitoring
- Sampling Flow-Level Sampling
- Two Main Approaches
- Packet Capture on High-Speed Links
- Characteristics of Packet Capture
- Data Measurement Repositories
- Multihoming and Traffic Engineering
- What is Multihoming
- Why Multihome
- Redundancy
- Performance
- Multihoming in IP Networks Today
- BGP or no
- Same Provider or Multiple
- Multihomed Stub One Link
- Multihomed Stub Multiple Links
- Multihomed Stub Multiple ISPs
- Multihomed Transit Network
- Interdomain Traffic Engineering
- Outbound Traffic Control
- Outbound Traffic Load Balancing
- Traffic Engineering Goals
- Managing Scale
- Achieving Predictability
- Challenges to Predictability
- Inter-AS Negotiation
- Outbound Multihoming Goals
- Inbound Traffic Control
- How many links are enough
- Problems with Multihoming in IPv4
- Slide 68
- Slide 69
- Diagnosis and Troubleshooting
- Operator Mailing List (NANOG)
- Operator Mailing List
- Routing Configuration
- Internet Business Model (Simplified)
- Peering Contracts Consistent Export
- Consistent Export
- Inconsistent Export in Practice
- Blackholes
- Security
- Security ldquoBogonrdquo Routes
- Spam Phishing etc
- Spam and Routing
- Worms and Botnets
- What is a Worm
- Example Worm Code Red
- Code Red Revisions
- Code Red Host Infection Rate
- Designing Fast-Spreading Worms
- Botnets
- ldquoRallyingrdquo the Botnet
- Botnet Control
- Some Defenses
- Idea 1 Ingress Filtering
- Idea 2 uRPF Checks
- Problems with uRPF
- S-BGP
- Practical Problems with S-BGP
-
20
Around 2000 Fast Growth Resumes
Claim remaining 8s will be exhausted within the next 5-10 years
T Hain ldquoA Pragmatic Report on IPv4 Address Space Consumptionrdquo Cisco IPJ September 2005
21
Fast growth resumes
Rapid growth in routing tables
Dot-Bomb Hiccup
Significant contributor Multihoming
Source Geoff Huston
22
The Address Allocation Process
bull Allocation policies of RIRs affect pressure on IPv4 address space
IANA
AfriNIC APNIC ARIN LACNIC RIPE
httpwwwianaorgassignmentsipv4-address-space
Georgia Tech
23
Common Problems
bull Diagnosis and troubleshooting (hence measurement)bull Traffic engineeringbull Securitybull Design and capacity planning
24
What can go wrong
Two-thirds of the problems are caused by configuration of the routing protocol
Some downtime is very hard to protect againsthellip
25
Measurement and Monitoring
26
Passive vs Active Measurement
bull Passive Measurement Collection of packets flow statistics of traffic that is already flowing on the networkndash Packet tracesndash Flow statisticsndash Application-level logs
bull Active Measurement Inject ldquoprobingrdquo traffic to measure various characteristicsndash Traceroutendash Pingndash Application-level probes (eg Web downloads)
27
Billing for Internet Usage
bull 95th Percentile billingndash Customer network pays for ldquocommitted information
raterdquo (CIR)ndash Throughput measured every 5 minutes (typically with
SNMP flow statistics also can be used for billing)ndash Customer billed based on 95th percentile
28
Passive Traffic Data Measurement
bull SNMP bytepacket counts everywhere
bull Packet monitoring selected locations
bull Flow monitoring typically at edges (if possible)ndash Direct computation of the traffic matrixndash Input to denial-of-service attack detection
bull Deep Packet Inspection also at edge where possible
29
Simple Network Management Protocol
bull Management Information Base (MIB)ndash Information storendash Unique variables named by
OIDsndash Accessed with SNMP
bull Specific MIBs for bytepacket counts (per link)
Manager Agent
SNMP
DB
ManagedObjects
30
SNMP (Passive)
bull Advantage ubiquitousndash Supported on all networking equipmentndash Multiple products for polling and analyzing data
bull Disadvantages ndash Coarse granularityndash Cannot express complex queries on the datandash Unreliable delivery of the data using UDP
bull Utilityndash Link utilization (billing)ndash Traffic matrix inference
31
Packet-level Monitoring
bull Passive monitoring to collect full packet contents (or at least headers)
bull Advantages lots of detailed informationndash Precise timing informationndash Information in packet headers
bull Disadvantages overheadndash Hard to keep up with high-speed linksndash Often requires a separate monitoring device
32
Full Packet Capture (Passive)
Example Georgia Tech OC3Mon
bull Rack-mounted PCbull Optical splitterbull Data Acquisition and
Generation (DAG) card
Source endacecom
33
What is a flow
bull Source IP addressbull Destination IP addressbull Source portbull Destination portbull Layer 3 protocol typebull TOS byte (DSCP)bull Input logical interface (ifIndex)
34
Cisco NetFlow
bull Basic output ldquoFlow recordrdquondash Most common version is v5
bull Current version (9) is being standardized in the IETF (template-based)ndash More flexible record formatndash Much easier to add new flow record types
Core Network
Collection and Aggregation
Collector (PC)Approximately 1500 bytes
20-50 flow recordsSent more frequently if traffic increases
35
Flow Record Contents
bull Source and Destination IP address and portbull Packet and byte countsbull Start and end timesbull ToS TCP flags
Basic information about the flowhellip
hellipplus information related to routing
bull Next-hop IP addressbull Source and destination ASbull Source and destination prefix
36
flow 1 flow 2 flow 3 flow 4
Aggregating Packets into Flows
bull Criteria 1 Set of packets that ldquobelong togetherrdquondash Sourcedestination IP addresses and port numbersndash Same protocol ToS bits hellip ndash Same inputoutput interfaces at a router (if known)
bull Criteria 2 Packets that are ldquocloserdquo together in timendash Maximum inter-packet spacing (eg 15 sec 30 sec)ndash Example flows 2 and 4 are different flows due to time
37
Reducing Measurement Overhead
bull Filtering on interfacendash destination prefix for a customerndash port number for an application (eg 80 for Web)
bull Sampling before insertion into flow cachendash Random deterministic or hash-based samplingndash 1-out-of-n or stratified based on packetflow sizendash Two types packet-level and flow-level
bull Aggregation after cache evictionndash packetsflows with same next-hop ASndash packetsflows destined to a particular service
38
Packet Sampling for Flow Monitoring
bull Packet sampling before flow creation (Sampled Netflow)ndash 1-out-of-m sampling of individual packets (eg m=100)ndash Create of flow records over the sampled packets
bull Reducing overheadndash Avoid per-packet overhead on (m-1)m packetsndash Avoid creating records for a large number of small flows
bull Increasing overhead (in some cases)ndash May split some long transfers into multiple flow records ndash hellip due to larger time gaps between successive packets
time
not sampled
two flowstimeout
39
Sampling Flow-Level Sampling
bull Sampling of flow records evicted from flow cachendash When evicting flows from table or when analyzing flows
bull Stratified sampling to put weight on ldquoheavyrdquo flowsndash Select all long flows and sample the short flows
bull Reduces the number of flow records ndash Still measures the vast majority of the traffic
Flow 1 40 bytesFlow 2 15580 bytesFlow 3 8196 bytesFlow 4 5350789 bytesFlow 5 532 bytesFlow 6 7432 bytes
sample with 100 probability
sample with 01 probability
sample with 10 probability
40
Two Main Approaches
bull Packet-level Monitoringndash Keep packet-level statisticsndash Examine (and potentially log) variety of packet-level
statistics Essentially anything in the packetndash Timing
bull Flow-level Monitoringndash Monitor packet-by-packet (though sometimes
sampled)ndash Keep aggregate statistics on a flow
41
Packet Capture on High-Speed Links
Example Georgia Tech ldquoOC3Monrdquo
bull Rack-mounted PCbull Optical splitterbull Data Acquisition and
Generation (DAG) card
Source endacecom
42
Characteristics of Packet Capture
bull Allows inspection on every packet on 10G links
bull Disadvantagesndash Costlyndash Requires splitting optical fibersndash Must be able to filterstore data
43
Data Measurement Repositories
bull AbileneInternet 2 Observatoryndash Configuration examplesndash SNMP datandash ISIS BGP routing data NetFlow traffic data
bull RouteViewsndash BGP updatesndash BGP table snapshots
44
Multihoming and Traffic Engineering
45
What is Multihoming
bull The use of redundant network links for the purposes of external connectivity
bull Can be achieved at many layers of the protocol stack and many places in the networkndash Multiple network interfaces in a PC
ndash An ISP with multiple upstream interfaces
bull Can refer to having multiple connections tondash The same ISP
ndash Multiple ISPs
46
Why Multihome
bull Redundancybull Availabilitybull Performancebull Cost
Interdomain traffic engineering the process by which a multihomed network configures its
network to achieve these goals
47
Redundancy
bull Maintain connectivity in the face ofndash Physical connectivity problems (fiber cut device
failures etc)ndash Failures in upstream ISP
48
Performance
bull Use multiple network links at once to achieve higher throughput than just over a single link
bull Allows incoming traffic to be load-balanced
70 of traffic30 of traffic
49
Multihoming in IP Networks Today
bull Stub AS no transit service for other ASesndash No need to use BGP
bull Multi-homed stub AS has connectivity to multiple immediate upstream ISPsndash Need BGPndash No need for a public AS numberndash No need for IP prefix allocation
bull Multi-homed transit AS connectivity to multiple ASes and transit servicendash Need BGP public AS number IP prefix allocation
50
BGP or no
bull Advantages of static routingndash Cheapersmaller routers (less true nowadays)ndash Simpler to configure
bull Advantages of BGPndash More control of your destiny (have providers stop
announcing you)ndash Fastermore intelligent selection of where to send
outbound packetsndash Better debugging of net problems (you can see the
Internet topology now)
51
Same Provider or Multiple
bull If your provider is reliable and fast and affordably and offers good tech-support you may want to multi-home initially to them via some backup path (slow is better than dead)
bull Eventually yoursquoll want to multi-home to different providers to avoid failure modes due to one providerrsquos architecture decisions
52
Multihomed Stub One Link
bull Downstream ISPrsquos routers configure default (ldquostaticrdquo) routes pointing to border router
bull Upstream ISP advertises reachability
Upstream ISP
Multiple links between same pair of routers
Default routes to ldquoborderrdquo
ldquoStubrdquoISP
53
Multihomed Stub Multiple Links
bull Use BGP to share loadbull Use private AS number (why is this OK)bull As before upstream ISP advertises prefix
Upstream ISP
Multiple links to different upstream routers
ldquoStubrdquoISP
Internal routing for ldquohot potatordquo
BGP for load balance at edge
54
Multihomed Stub Multiple ISPs
bull Many possibilitiesndash Load sharingndash Primary-backupndash Selective use of different ISPs
bull Requires BGP public AS number etc
ldquoStubrdquoISP
Upstream
ISP 1
Upstream
ISP 2
55
Multihomed Transit Network
bull BGP everywherebull Incoming and outcoming trafficbull Challenge balancing load on intradomain and egress
links given an offered traffic load
TransitISP
ISP 1
ISP 2
ISP 3
56
Interdomain Traffic Engineering
bull The process by which a network operator configures the network to achievendash Traffic load balancendash Redundancy (primarybackup) etc
bull Two tasksndash Outbound traffic controlndash Inbound traffic control
bull Key Problems Predictability and Scalability
57
Outbound Traffic Control
bull Easier to control than inbound trafficndash Destination-based routing sender determines where
the packets go
bull Control over next-hop AS onlyndash Cannot control selection of the entire path
Provider 1 Provider 2
Control with local preference
58
Outbound Traffic Load Balancingbull Control routes to provider per-prefix
ndash Assign local preference across destination prefixesndash Change the local preference assignments over time
bull Useful inputs to load balancingndash End-to-end path performance datandash Outbound traffic statistics per destination prefix
bull Challenge Getting from traffic volumes to groups of prefixes that should be assigned to each link
Premise of ldquointelligent route controlrdquo preoducts
59
Traffic Engineering Goals
bull Predictabilityndash Ensure the BGP decision process is deterministicndash Assume that BGP updates are (relatively) stable
bull Limit overhead introduced by routing changesndash Minimize frequency of changes to routing policiesndash Limit number of prefixes affected by changes
bull Limit impact on how traffic enters the networkndash Avoid new routes that might change neighborrsquos mindndash Select route with same attributes or at least path length
60
Managing Scalebull Destination prefixes
ndash More than 90000 destination prefixes
bull Donrsquot want to have per-prefix routing policies
ndash Small fraction of prefixes contribute most of the traffic
bull Focus on the small number of heavy hitters
ndash Define routing policies for selected prefixes
bull Routing choicesndash About 27000 unique ldquorouting choicesrdquo
bull Help in reducing the scale of the problem
ndash Small fraction of ldquorouting choicesrdquo contribute most traffic
bull Focus on the very small number of ldquorouting choicesrdquo
ndash Define routing policies on common attributes
61
Achieving Predictability
bull Route prediction with static analysisndash Helpful to know effects before deploymentndash Static analysis can help
TopologyBGP policy
configuration
eBGP routes
Offered traffic
BGP routingmodel
Flow of traffic through the network
62
Challenges to Predictabilitybull For transit ISPs effects on incoming traffic
ndash Lack of coordination strikes again
63
ldquoHot Potatordquo routing
Inter-AS Negotiation
bull Coordination aids predictabilityndash Negotiate where to sendndash Inbound and outboundndash Mutual benefits
bull How to implementndash What info to exchangendash Protecting privacyndash How to prioritize choicesndash How to prevent cheating
Destination 2
Destination 1
multiplepeeringpoints
Provider A
Provider B
64
Outbound Multihoming Goals
bull Redundancyndash Dynamic routing will failover to backup link
bull Performancendash Select provider with best performance per prefix
ndash Requires active probing
bull Costndash Select provider per prefix over time to minimize the
total financial cost
65
Inbound Traffic Control
bull More difficult no control over neighborsrsquo decisions
bull Three common techniques (previously discussed)ndash AS path prependingndash Communities and local preferencendash Prefix splitting
How does todayrsquos paper (MONET) control inbound traffic
66
How many links are enough
K upstream ISPs
Not much benefit beyond 4 ISPs
Akella et al ldquoPerformance Benefits of Multihomingrdquo SIGCOMM 2003
67
Problems with Multihoming in IPv4
bull Routing table growthndash Provider-based addressingndash Advertising prefix out multiple ISPs ndash canrsquot aggregate
bull Poor control over inbound trafficndash Existing mechanisms do not allow hosts to control
inbound traffic
68
GeorgiaTech
Internet Routing Overview
bull Intradomain (ie ldquointra-ASrdquo) routingbull Interdomain routing
Comcast
Abilene
ATampT Cogent
Autonomous Systems (ASes)
69
Configuration Problems ldquoAS 7007rdquoldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
UUNet
Florida InternetBarn
Sprint
70
Diagnosis and Troubleshootingldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
ldquoMicrosofts websites were offline for up to 23 hoursbecause of a [router] misconfigurationhellipit took nearly a day to determine what was wrong and undo the changesrdquo -- wiredcom January 25 2001
ldquoWorldCom Inchellipsuffered a widespread outage on its Internet backbone that affected roughly 20 percent of its US customer base The network problemshellipaffected millions of computer users worldwide A spokeswoman attributed the outage to a route table issue -- cnncom October 3 2002
A number of Covad customers went out from 5pm today due to supposedly a DDOS (distributed denial of service attack) on a key Level3 data center which later was described as a route leak (misconfiguration)ldquo
-- dslreportscom February 23 2004
71
Operator Mailing List (NANOG)Date Mon 18 Oct 2004 091515 -0700Subject Level 3 US east coast issues
Level 3 experiencing widespread unspecified routing issues on the US east coast Master ticket 1086844 Anyone have more specific information
Date Mon 18 Oct 2004 122034 -0400 (EDT)Subject Re Level 3 US east coast issues
Level 3 is currently experiencing a backbone outage causing routing instability and packet loss We are working to restore and will be sending hourly updateshellip
72
Operator Mailing List
0102030405060708090
Filtering RouteLeaks
RouteHijacks
RouteInstability
RoutingLoops
Blackholes
Occurr
ences o
ver
10 Y
ears
1995-1997 1998-2001 2001-2004
Note Only includes problems openly discussed on this list
Compare 83 power outages 1 fire
73
Routing Configuration
Ranking route selection
Dissemination internal route advertisement
Filtering route advertisement
Customer
Competitor
Primary
Backup
74
Internet Business Model (Simplified)
bull CustomerProvider One AS pays another for reachability to some set of destinations
bull ldquoSettlement-freerdquo Peering Bartering Two ASes exchange routes with one another
Provider
Peer
Customer
Preferences implemented with local preference manipulation
Destination
Pay to use
Get paid to use
Free to use
75
Peering Contracts Consistent Export
bull Rules of settlement-free peeringndash Advertise routes at all peering pointsndash Advertised routes must have equal ldquoAS path lengthrdquo
Sprint
ATampT
Enables ldquohot potatordquo routing
ldquoequally goodrdquoroutes
76
Consistent Export
bull Malicedeceptionbull iBGP signaling partitionbull Inconsistent export policy
neighbor 10123route-map PEER permit 10 set prepend 123
neighbor 10456route-map PEER permit 10 set prepend 123 123
Possible Causes
10123 456 1
10456 456 2
Neighbor AS Export
1 1 123
Export Clause Prepend
2 1 123 123
Two different Export Policies
77
Inconsistent Export in Practice
Feamster et al ldquoBorderGuard Detecting Cold Potatoes from Peersrdquo ACM IMC October 2004
78
Blackholes
Date Thu 18 Jul 2002 060510 -0400 (EDT)From Chad Oleary ltcolpoboxcomgtSubject Re problems with 701To ltnanogmeritedugt
Were starting to see the same issues with UUNet again Anyone elseseeing this Trying to reach Qwest
traceroute to 631461901 (631461901) 30 hops max 38 byte packets 1 esc-lp2-gwe-solutionscorpcom (631182201) 1167 ms 1163 ms 1142 ms 2 500Serial2-10GW1TPA2ALTERNET (1571301499) 1097 ms 1059 ms 1044 ms 3 161at-1-0-0XL4ATL1ALTERNET (1526381190) 13839 ms 14108 ms 16638 ms 4 0so-3-1-0XL2ATL5ALTERNET (152630238) 14370 ms 14587 ms 14553 ms 5 POS7-0BR2ATL5ALTERNET (1526382193) 13928 ms 14099 ms 14053 ms 6 7 hellip
79
Security
80
Security ldquoBogonrdquo Routes
Feamster et al ldquoAn Empirical Study of lsquoBogonrsquo Route Advertisementsrdquo ACM CCR January 2005
81
Spam Phishing etc
bull Unsolicited commercial emailbull As of about August 2008 estimates indicate that
about 95 of all email is spambull Common spam filtering techniques
ndash Content-based filtersndash DNS Blacklist (DNSBL) lookups Significant fraction of
todayrsquos DNS traffic
Can IP addresses from which spam is received be spoofed
82
Spam and Routing
83
Worms and Botnets
84
What is a Worm
bull Code that replicates and propagates across the networkndash Often carries a ldquopayloadrdquo
bull Usually spread via exploiting flaws in open servicesndash ldquoVirusesrdquo require user action to spread
bull First worm Robert Morris November 1988ndash 6-10 of all Internet hosts infected ()
bull Many more since but none on that scale until July 2001
85
Example Worm Code Red
bull Initial version July 13 2001
bull Exploited known ISAPI vulnerability in Microsoft IIS Web servers
bull 1st through 20th of each month spread20th through end of each month attack
bull Payload Web site defacementbull Scanning Random IP addressesbull Bug failure to seed random number generator
86
Code Red Revisions
bull Released July 19 2001
bull Payload flooding attack on wwwwhitehousegovndash Attack was mounted at the IP address of the Web site
bull Bug died after 20th of each month
bull Random number generator for IP scanning fixed
87
Code Red Host Infection Rate
Exponential infection rate
Measured using backscatter technique
88
Designing Fast-Spreading Worms
bull Hit-list scanningndash Time to infect first 10k hosts dominates infection timendash Solution Reconnaissance (stealthy scans etc)
bull Permutation scanningndash Observation Most scanning is redundantndash Idea Shared permutation of address space Start scanning
from own IP address Re-randomize when another infected machine is found
bull Internet-scale hit listsndash Flash worm complete infection within 30 seconds
89
Botnets
bull Bots Autonomous programs performing tasksbull Plenty of ldquobenignrdquo bots
ndash eg weatherbug
bull Botnets group of bots ndash Typically carries malicious connotationndash Large numbers of infected machinesndash Machines ldquoenlistedrdquo with infection vectors like worms
(last lecture)
bull Available for simultaneous control by a masterbull Size up to 350000 nodes (from todayrsquos paper)
90
ldquoRallyingrdquo the Botnet
bull Easy to combine worm backdoor functionalitybull Problem how to learn about successfully
infected machines
bull Optionsndash Emailndash Hard-coded email address
91
Botnet Control
bull Botnet master typically runs some IRC server on a well-known port (eg 6667)
bull Infected machine contacts botnet with pre-programmed DNS name (eg big-botde)
bull Dynamic DNS allows controller to move about freely
Infected Machine
DynamicDNS
BotnetController
(IRC server)
92
Some Defenses
93
Idea 1 Ingress Filtering
bull RFC 2827 Routers install filters to drop packets from networks that are not downstream
bull Feasible at edgesbull Difficult to configure closer to network ldquocorerdquo
20469207024 Internet
Drop all packets with source address other than 20469207024
94
Idea 2 uRPF Checks
bull Unicast Reverse Path Forwardingndash Cisco ldquoip verify unicast reverse-pathrdquo
bull Requires symmetric routing
Accept packet from interface only if forwarding table entry for source IP address matches ingress interface
100183
A10018
10015
101203
100181 24
10011 24
Strict Mode uRPF
Enabled
ldquoArdquo Routing TableDestination Next Hop1001024 Int 110018024 Int 2
100183 from wrong interface
95
Problems with uRPF
bull Asymmetric routing
96
S-BGP
bull Address-based PKI validate signaturesndash Authentication of
bull ownership for IP address blocks bull AS number bull an ASs identity and bull a BGP routers identity
ndash Use existing infrastructure (Internet registries etc)ndash Routing origination is digitally signedndash BGP updates are digitally signed
1048708 bull Route attestations A new optional BGP transitive path attribute
ndash carries digital signatures covering the routing information in updates
97
Practical Problems with S-BGP
bull Requires Public-Key Infrastructure
bull Lots of digital signatures to calculate and verifyndash Message overheadndash CPU overhead
bull Calculation expense is greatest when topology is changingndash Caching can help
bull Route aggregation is problematic (maybe thatrsquos OK)
bull Secure route withdrawals when link or node fails
bull Address ownership data out of date
bull Deployment
- Networking
- Goal of This Tutorial
- Todayrsquos Networks
- Business Models
- Billing for Internet Usage
- Net Neutrality
- Network Operations
- Point-of-Presence (PoP)
- Example Abilene Network Topology
- Another Example Backbone
- Internet Routing Overview
- Internet Routing Protocol BGP
- Two Flavors of BGP
- IPv4 Addresses Networks of Networks
- Pre-1994 Classful Addressing
- Classless Interdomain Routing (CIDR)
- Benefits of CIDR
- Growth of IP Prefixes
- 1994-1998 Linear Growth
- Around 2000 Fast Growth Resumes
- Fast growth resumes
- The Address Allocation Process
- Common Problems
- What can go wrong
- Measurement and Monitoring
- Passive vs Active Measurement
- Slide 27
- Passive Traffic Data Measurement
- Simple Network Management Protocol
- SNMP (Passive)
- Packet-level Monitoring
- Full Packet Capture (Passive)
- What is a flow
- Cisco NetFlow
- Flow Record Contents
- Aggregating Packets into Flows
- Reducing Measurement Overhead
- Packet Sampling for Flow Monitoring
- Sampling Flow-Level Sampling
- Two Main Approaches
- Packet Capture on High-Speed Links
- Characteristics of Packet Capture
- Data Measurement Repositories
- Multihoming and Traffic Engineering
- What is Multihoming
- Why Multihome
- Redundancy
- Performance
- Multihoming in IP Networks Today
- BGP or no
- Same Provider or Multiple
- Multihomed Stub One Link
- Multihomed Stub Multiple Links
- Multihomed Stub Multiple ISPs
- Multihomed Transit Network
- Interdomain Traffic Engineering
- Outbound Traffic Control
- Outbound Traffic Load Balancing
- Traffic Engineering Goals
- Managing Scale
- Achieving Predictability
- Challenges to Predictability
- Inter-AS Negotiation
- Outbound Multihoming Goals
- Inbound Traffic Control
- How many links are enough
- Problems with Multihoming in IPv4
- Slide 68
- Slide 69
- Diagnosis and Troubleshooting
- Operator Mailing List (NANOG)
- Operator Mailing List
- Routing Configuration
- Internet Business Model (Simplified)
- Peering Contracts Consistent Export
- Consistent Export
- Inconsistent Export in Practice
- Blackholes
- Security
- Security ldquoBogonrdquo Routes
- Spam Phishing etc
- Spam and Routing
- Worms and Botnets
- What is a Worm
- Example Worm Code Red
- Code Red Revisions
- Code Red Host Infection Rate
- Designing Fast-Spreading Worms
- Botnets
- ldquoRallyingrdquo the Botnet
- Botnet Control
- Some Defenses
- Idea 1 Ingress Filtering
- Idea 2 uRPF Checks
- Problems with uRPF
- S-BGP
- Practical Problems with S-BGP
-
21
Fast growth resumes
Rapid growth in routing tables
Dot-Bomb Hiccup
Significant contributor Multihoming
Source Geoff Huston
22
The Address Allocation Process
bull Allocation policies of RIRs affect pressure on IPv4 address space
IANA
AfriNIC APNIC ARIN LACNIC RIPE
httpwwwianaorgassignmentsipv4-address-space
Georgia Tech
23
Common Problems
bull Diagnosis and troubleshooting (hence measurement)bull Traffic engineeringbull Securitybull Design and capacity planning
24
What can go wrong
Two-thirds of the problems are caused by configuration of the routing protocol
Some downtime is very hard to protect againsthellip
25
Measurement and Monitoring
26
Passive vs Active Measurement
bull Passive Measurement Collection of packets flow statistics of traffic that is already flowing on the networkndash Packet tracesndash Flow statisticsndash Application-level logs
bull Active Measurement Inject ldquoprobingrdquo traffic to measure various characteristicsndash Traceroutendash Pingndash Application-level probes (eg Web downloads)
27
Billing for Internet Usage
bull 95th Percentile billingndash Customer network pays for ldquocommitted information
raterdquo (CIR)ndash Throughput measured every 5 minutes (typically with
SNMP flow statistics also can be used for billing)ndash Customer billed based on 95th percentile
28
Passive Traffic Data Measurement
bull SNMP bytepacket counts everywhere
bull Packet monitoring selected locations
bull Flow monitoring typically at edges (if possible)ndash Direct computation of the traffic matrixndash Input to denial-of-service attack detection
bull Deep Packet Inspection also at edge where possible
29
Simple Network Management Protocol
bull Management Information Base (MIB)ndash Information storendash Unique variables named by
OIDsndash Accessed with SNMP
bull Specific MIBs for bytepacket counts (per link)
Manager Agent
SNMP
DB
ManagedObjects
30
SNMP (Passive)
bull Advantage ubiquitousndash Supported on all networking equipmentndash Multiple products for polling and analyzing data
bull Disadvantages ndash Coarse granularityndash Cannot express complex queries on the datandash Unreliable delivery of the data using UDP
bull Utilityndash Link utilization (billing)ndash Traffic matrix inference
31
Packet-level Monitoring
bull Passive monitoring to collect full packet contents (or at least headers)
bull Advantages lots of detailed informationndash Precise timing informationndash Information in packet headers
bull Disadvantages overheadndash Hard to keep up with high-speed linksndash Often requires a separate monitoring device
32
Full Packet Capture (Passive)
Example Georgia Tech OC3Mon
bull Rack-mounted PCbull Optical splitterbull Data Acquisition and
Generation (DAG) card
Source endacecom
33
What is a flow
bull Source IP addressbull Destination IP addressbull Source portbull Destination portbull Layer 3 protocol typebull TOS byte (DSCP)bull Input logical interface (ifIndex)
34
Cisco NetFlow
bull Basic output ldquoFlow recordrdquondash Most common version is v5
bull Current version (9) is being standardized in the IETF (template-based)ndash More flexible record formatndash Much easier to add new flow record types
Core Network
Collection and Aggregation
Collector (PC)Approximately 1500 bytes
20-50 flow recordsSent more frequently if traffic increases
35
Flow Record Contents
bull Source and Destination IP address and portbull Packet and byte countsbull Start and end timesbull ToS TCP flags
Basic information about the flowhellip
hellipplus information related to routing
bull Next-hop IP addressbull Source and destination ASbull Source and destination prefix
36
flow 1 flow 2 flow 3 flow 4
Aggregating Packets into Flows
bull Criteria 1 Set of packets that ldquobelong togetherrdquondash Sourcedestination IP addresses and port numbersndash Same protocol ToS bits hellip ndash Same inputoutput interfaces at a router (if known)
bull Criteria 2 Packets that are ldquocloserdquo together in timendash Maximum inter-packet spacing (eg 15 sec 30 sec)ndash Example flows 2 and 4 are different flows due to time
37
Reducing Measurement Overhead
bull Filtering on interfacendash destination prefix for a customerndash port number for an application (eg 80 for Web)
bull Sampling before insertion into flow cachendash Random deterministic or hash-based samplingndash 1-out-of-n or stratified based on packetflow sizendash Two types packet-level and flow-level
bull Aggregation after cache evictionndash packetsflows with same next-hop ASndash packetsflows destined to a particular service
38
Packet Sampling for Flow Monitoring
bull Packet sampling before flow creation (Sampled Netflow)ndash 1-out-of-m sampling of individual packets (eg m=100)ndash Create of flow records over the sampled packets
bull Reducing overheadndash Avoid per-packet overhead on (m-1)m packetsndash Avoid creating records for a large number of small flows
bull Increasing overhead (in some cases)ndash May split some long transfers into multiple flow records ndash hellip due to larger time gaps between successive packets
time
not sampled
two flowstimeout
39
Sampling Flow-Level Sampling
bull Sampling of flow records evicted from flow cachendash When evicting flows from table or when analyzing flows
bull Stratified sampling to put weight on ldquoheavyrdquo flowsndash Select all long flows and sample the short flows
bull Reduces the number of flow records ndash Still measures the vast majority of the traffic
Flow 1 40 bytesFlow 2 15580 bytesFlow 3 8196 bytesFlow 4 5350789 bytesFlow 5 532 bytesFlow 6 7432 bytes
sample with 100 probability
sample with 01 probability
sample with 10 probability
40
Two Main Approaches
bull Packet-level Monitoringndash Keep packet-level statisticsndash Examine (and potentially log) variety of packet-level
statistics Essentially anything in the packetndash Timing
bull Flow-level Monitoringndash Monitor packet-by-packet (though sometimes
sampled)ndash Keep aggregate statistics on a flow
41
Packet Capture on High-Speed Links
Example Georgia Tech ldquoOC3Monrdquo
bull Rack-mounted PCbull Optical splitterbull Data Acquisition and
Generation (DAG) card
Source endacecom
42
Characteristics of Packet Capture
bull Allows inspection on every packet on 10G links
bull Disadvantagesndash Costlyndash Requires splitting optical fibersndash Must be able to filterstore data
43
Data Measurement Repositories
bull AbileneInternet 2 Observatoryndash Configuration examplesndash SNMP datandash ISIS BGP routing data NetFlow traffic data
bull RouteViewsndash BGP updatesndash BGP table snapshots
44
Multihoming and Traffic Engineering
45
What is Multihoming
bull The use of redundant network links for the purposes of external connectivity
bull Can be achieved at many layers of the protocol stack and many places in the networkndash Multiple network interfaces in a PC
ndash An ISP with multiple upstream interfaces
bull Can refer to having multiple connections tondash The same ISP
ndash Multiple ISPs
46
Why Multihome
bull Redundancybull Availabilitybull Performancebull Cost
Interdomain traffic engineering the process by which a multihomed network configures its
network to achieve these goals
47
Redundancy
bull Maintain connectivity in the face ofndash Physical connectivity problems (fiber cut device
failures etc)ndash Failures in upstream ISP
48
Performance
bull Use multiple network links at once to achieve higher throughput than just over a single link
bull Allows incoming traffic to be load-balanced
70 of traffic30 of traffic
49
Multihoming in IP Networks Today
bull Stub AS no transit service for other ASesndash No need to use BGP
bull Multi-homed stub AS has connectivity to multiple immediate upstream ISPsndash Need BGPndash No need for a public AS numberndash No need for IP prefix allocation
bull Multi-homed transit AS connectivity to multiple ASes and transit servicendash Need BGP public AS number IP prefix allocation
50
BGP or no
bull Advantages of static routingndash Cheapersmaller routers (less true nowadays)ndash Simpler to configure
bull Advantages of BGPndash More control of your destiny (have providers stop
announcing you)ndash Fastermore intelligent selection of where to send
outbound packetsndash Better debugging of net problems (you can see the
Internet topology now)
51
Same Provider or Multiple
bull If your provider is reliable and fast and affordably and offers good tech-support you may want to multi-home initially to them via some backup path (slow is better than dead)
bull Eventually yoursquoll want to multi-home to different providers to avoid failure modes due to one providerrsquos architecture decisions
52
Multihomed Stub One Link
bull Downstream ISPrsquos routers configure default (ldquostaticrdquo) routes pointing to border router
bull Upstream ISP advertises reachability
Upstream ISP
Multiple links between same pair of routers
Default routes to ldquoborderrdquo
ldquoStubrdquoISP
53
Multihomed Stub Multiple Links
bull Use BGP to share loadbull Use private AS number (why is this OK)bull As before upstream ISP advertises prefix
Upstream ISP
Multiple links to different upstream routers
ldquoStubrdquoISP
Internal routing for ldquohot potatordquo
BGP for load balance at edge
54
Multihomed Stub Multiple ISPs
bull Many possibilitiesndash Load sharingndash Primary-backupndash Selective use of different ISPs
bull Requires BGP public AS number etc
ldquoStubrdquoISP
Upstream
ISP 1
Upstream
ISP 2
55
Multihomed Transit Network
bull BGP everywherebull Incoming and outcoming trafficbull Challenge balancing load on intradomain and egress
links given an offered traffic load
TransitISP
ISP 1
ISP 2
ISP 3
56
Interdomain Traffic Engineering
bull The process by which a network operator configures the network to achievendash Traffic load balancendash Redundancy (primarybackup) etc
bull Two tasksndash Outbound traffic controlndash Inbound traffic control
bull Key Problems Predictability and Scalability
57
Outbound Traffic Control
bull Easier to control than inbound trafficndash Destination-based routing sender determines where
the packets go
bull Control over next-hop AS onlyndash Cannot control selection of the entire path
Provider 1 Provider 2
Control with local preference
58
Outbound Traffic Load Balancingbull Control routes to provider per-prefix
ndash Assign local preference across destination prefixesndash Change the local preference assignments over time
bull Useful inputs to load balancingndash End-to-end path performance datandash Outbound traffic statistics per destination prefix
bull Challenge Getting from traffic volumes to groups of prefixes that should be assigned to each link
Premise of ldquointelligent route controlrdquo preoducts
59
Traffic Engineering Goals
bull Predictabilityndash Ensure the BGP decision process is deterministicndash Assume that BGP updates are (relatively) stable
bull Limit overhead introduced by routing changesndash Minimize frequency of changes to routing policiesndash Limit number of prefixes affected by changes
bull Limit impact on how traffic enters the networkndash Avoid new routes that might change neighborrsquos mindndash Select route with same attributes or at least path length
60
Managing Scalebull Destination prefixes
ndash More than 90000 destination prefixes
bull Donrsquot want to have per-prefix routing policies
ndash Small fraction of prefixes contribute most of the traffic
bull Focus on the small number of heavy hitters
ndash Define routing policies for selected prefixes
bull Routing choicesndash About 27000 unique ldquorouting choicesrdquo
bull Help in reducing the scale of the problem
ndash Small fraction of ldquorouting choicesrdquo contribute most traffic
bull Focus on the very small number of ldquorouting choicesrdquo
ndash Define routing policies on common attributes
61
Achieving Predictability
bull Route prediction with static analysisndash Helpful to know effects before deploymentndash Static analysis can help
TopologyBGP policy
configuration
eBGP routes
Offered traffic
BGP routingmodel
Flow of traffic through the network
62
Challenges to Predictabilitybull For transit ISPs effects on incoming traffic
ndash Lack of coordination strikes again
63
ldquoHot Potatordquo routing
Inter-AS Negotiation
bull Coordination aids predictabilityndash Negotiate where to sendndash Inbound and outboundndash Mutual benefits
bull How to implementndash What info to exchangendash Protecting privacyndash How to prioritize choicesndash How to prevent cheating
Destination 2
Destination 1
multiplepeeringpoints
Provider A
Provider B
64
Outbound Multihoming Goals
bull Redundancyndash Dynamic routing will failover to backup link
bull Performancendash Select provider with best performance per prefix
ndash Requires active probing
bull Costndash Select provider per prefix over time to minimize the
total financial cost
65
Inbound Traffic Control
bull More difficult no control over neighborsrsquo decisions
bull Three common techniques (previously discussed)ndash AS path prependingndash Communities and local preferencendash Prefix splitting
How does todayrsquos paper (MONET) control inbound traffic
66
How many links are enough
K upstream ISPs
Not much benefit beyond 4 ISPs
Akella et al ldquoPerformance Benefits of Multihomingrdquo SIGCOMM 2003
67
Problems with Multihoming in IPv4
bull Routing table growthndash Provider-based addressingndash Advertising prefix out multiple ISPs ndash canrsquot aggregate
bull Poor control over inbound trafficndash Existing mechanisms do not allow hosts to control
inbound traffic
68
GeorgiaTech
Internet Routing Overview
bull Intradomain (ie ldquointra-ASrdquo) routingbull Interdomain routing
Comcast
Abilene
ATampT Cogent
Autonomous Systems (ASes)
69
Configuration Problems ldquoAS 7007rdquoldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
UUNet
Florida InternetBarn
Sprint
70
Diagnosis and Troubleshootingldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
ldquoMicrosofts websites were offline for up to 23 hoursbecause of a [router] misconfigurationhellipit took nearly a day to determine what was wrong and undo the changesrdquo -- wiredcom January 25 2001
ldquoWorldCom Inchellipsuffered a widespread outage on its Internet backbone that affected roughly 20 percent of its US customer base The network problemshellipaffected millions of computer users worldwide A spokeswoman attributed the outage to a route table issue -- cnncom October 3 2002
A number of Covad customers went out from 5pm today due to supposedly a DDOS (distributed denial of service attack) on a key Level3 data center which later was described as a route leak (misconfiguration)ldquo
-- dslreportscom February 23 2004
71
Operator Mailing List (NANOG)Date Mon 18 Oct 2004 091515 -0700Subject Level 3 US east coast issues
Level 3 experiencing widespread unspecified routing issues on the US east coast Master ticket 1086844 Anyone have more specific information
Date Mon 18 Oct 2004 122034 -0400 (EDT)Subject Re Level 3 US east coast issues
Level 3 is currently experiencing a backbone outage causing routing instability and packet loss We are working to restore and will be sending hourly updateshellip
72
Operator Mailing List
0102030405060708090
Filtering RouteLeaks
RouteHijacks
RouteInstability
RoutingLoops
Blackholes
Occurr
ences o
ver
10 Y
ears
1995-1997 1998-2001 2001-2004
Note Only includes problems openly discussed on this list
Compare 83 power outages 1 fire
73
Routing Configuration
Ranking route selection
Dissemination internal route advertisement
Filtering route advertisement
Customer
Competitor
Primary
Backup
74
Internet Business Model (Simplified)
bull CustomerProvider One AS pays another for reachability to some set of destinations
bull ldquoSettlement-freerdquo Peering Bartering Two ASes exchange routes with one another
Provider
Peer
Customer
Preferences implemented with local preference manipulation
Destination
Pay to use
Get paid to use
Free to use
75
Peering Contracts Consistent Export
bull Rules of settlement-free peeringndash Advertise routes at all peering pointsndash Advertised routes must have equal ldquoAS path lengthrdquo
Sprint
ATampT
Enables ldquohot potatordquo routing
ldquoequally goodrdquoroutes
76
Consistent Export
bull Malicedeceptionbull iBGP signaling partitionbull Inconsistent export policy
neighbor 10123route-map PEER permit 10 set prepend 123
neighbor 10456route-map PEER permit 10 set prepend 123 123
Possible Causes
10123 456 1
10456 456 2
Neighbor AS Export
1 1 123
Export Clause Prepend
2 1 123 123
Two different Export Policies
77
Inconsistent Export in Practice
Feamster et al ldquoBorderGuard Detecting Cold Potatoes from Peersrdquo ACM IMC October 2004
78
Blackholes
Date Thu 18 Jul 2002 060510 -0400 (EDT)From Chad Oleary ltcolpoboxcomgtSubject Re problems with 701To ltnanogmeritedugt
Were starting to see the same issues with UUNet again Anyone elseseeing this Trying to reach Qwest
traceroute to 631461901 (631461901) 30 hops max 38 byte packets 1 esc-lp2-gwe-solutionscorpcom (631182201) 1167 ms 1163 ms 1142 ms 2 500Serial2-10GW1TPA2ALTERNET (1571301499) 1097 ms 1059 ms 1044 ms 3 161at-1-0-0XL4ATL1ALTERNET (1526381190) 13839 ms 14108 ms 16638 ms 4 0so-3-1-0XL2ATL5ALTERNET (152630238) 14370 ms 14587 ms 14553 ms 5 POS7-0BR2ATL5ALTERNET (1526382193) 13928 ms 14099 ms 14053 ms 6 7 hellip
79
Security
80
Security ldquoBogonrdquo Routes
Feamster et al ldquoAn Empirical Study of lsquoBogonrsquo Route Advertisementsrdquo ACM CCR January 2005
81
Spam Phishing etc
bull Unsolicited commercial emailbull As of about August 2008 estimates indicate that
about 95 of all email is spambull Common spam filtering techniques
ndash Content-based filtersndash DNS Blacklist (DNSBL) lookups Significant fraction of
todayrsquos DNS traffic
Can IP addresses from which spam is received be spoofed
82
Spam and Routing
83
Worms and Botnets
84
What is a Worm
bull Code that replicates and propagates across the networkndash Often carries a ldquopayloadrdquo
bull Usually spread via exploiting flaws in open servicesndash ldquoVirusesrdquo require user action to spread
bull First worm Robert Morris November 1988ndash 6-10 of all Internet hosts infected ()
bull Many more since but none on that scale until July 2001
85
Example Worm Code Red
bull Initial version July 13 2001
bull Exploited known ISAPI vulnerability in Microsoft IIS Web servers
bull 1st through 20th of each month spread20th through end of each month attack
bull Payload Web site defacementbull Scanning Random IP addressesbull Bug failure to seed random number generator
86
Code Red Revisions
bull Released July 19 2001
bull Payload flooding attack on wwwwhitehousegovndash Attack was mounted at the IP address of the Web site
bull Bug died after 20th of each month
bull Random number generator for IP scanning fixed
87
Code Red Host Infection Rate
Exponential infection rate
Measured using backscatter technique
88
Designing Fast-Spreading Worms
bull Hit-list scanningndash Time to infect first 10k hosts dominates infection timendash Solution Reconnaissance (stealthy scans etc)
bull Permutation scanningndash Observation Most scanning is redundantndash Idea Shared permutation of address space Start scanning
from own IP address Re-randomize when another infected machine is found
bull Internet-scale hit listsndash Flash worm complete infection within 30 seconds
89
Botnets
bull Bots Autonomous programs performing tasksbull Plenty of ldquobenignrdquo bots
ndash eg weatherbug
bull Botnets group of bots ndash Typically carries malicious connotationndash Large numbers of infected machinesndash Machines ldquoenlistedrdquo with infection vectors like worms
(last lecture)
bull Available for simultaneous control by a masterbull Size up to 350000 nodes (from todayrsquos paper)
90
ldquoRallyingrdquo the Botnet
bull Easy to combine worm backdoor functionalitybull Problem how to learn about successfully
infected machines
bull Optionsndash Emailndash Hard-coded email address
91
Botnet Control
bull Botnet master typically runs some IRC server on a well-known port (eg 6667)
bull Infected machine contacts botnet with pre-programmed DNS name (eg big-botde)
bull Dynamic DNS allows controller to move about freely
Infected Machine
DynamicDNS
BotnetController
(IRC server)
92
Some Defenses
93
Idea 1 Ingress Filtering
bull RFC 2827 Routers install filters to drop packets from networks that are not downstream
bull Feasible at edgesbull Difficult to configure closer to network ldquocorerdquo
20469207024 Internet
Drop all packets with source address other than 20469207024
94
Idea 2 uRPF Checks
bull Unicast Reverse Path Forwardingndash Cisco ldquoip verify unicast reverse-pathrdquo
bull Requires symmetric routing
Accept packet from interface only if forwarding table entry for source IP address matches ingress interface
100183
A10018
10015
101203
100181 24
10011 24
Strict Mode uRPF
Enabled
ldquoArdquo Routing TableDestination Next Hop1001024 Int 110018024 Int 2
100183 from wrong interface
95
Problems with uRPF
bull Asymmetric routing
96
S-BGP
bull Address-based PKI validate signaturesndash Authentication of
bull ownership for IP address blocks bull AS number bull an ASs identity and bull a BGP routers identity
ndash Use existing infrastructure (Internet registries etc)ndash Routing origination is digitally signedndash BGP updates are digitally signed
1048708 bull Route attestations A new optional BGP transitive path attribute
ndash carries digital signatures covering the routing information in updates
97
Practical Problems with S-BGP
bull Requires Public-Key Infrastructure
bull Lots of digital signatures to calculate and verifyndash Message overheadndash CPU overhead
bull Calculation expense is greatest when topology is changingndash Caching can help
bull Route aggregation is problematic (maybe thatrsquos OK)
bull Secure route withdrawals when link or node fails
bull Address ownership data out of date
bull Deployment
- Networking
- Goal of This Tutorial
- Todayrsquos Networks
- Business Models
- Billing for Internet Usage
- Net Neutrality
- Network Operations
- Point-of-Presence (PoP)
- Example Abilene Network Topology
- Another Example Backbone
- Internet Routing Overview
- Internet Routing Protocol BGP
- Two Flavors of BGP
- IPv4 Addresses Networks of Networks
- Pre-1994 Classful Addressing
- Classless Interdomain Routing (CIDR)
- Benefits of CIDR
- Growth of IP Prefixes
- 1994-1998 Linear Growth
- Around 2000 Fast Growth Resumes
- Fast growth resumes
- The Address Allocation Process
- Common Problems
- What can go wrong
- Measurement and Monitoring
- Passive vs Active Measurement
- Slide 27
- Passive Traffic Data Measurement
- Simple Network Management Protocol
- SNMP (Passive)
- Packet-level Monitoring
- Full Packet Capture (Passive)
- What is a flow
- Cisco NetFlow
- Flow Record Contents
- Aggregating Packets into Flows
- Reducing Measurement Overhead
- Packet Sampling for Flow Monitoring
- Sampling Flow-Level Sampling
- Two Main Approaches
- Packet Capture on High-Speed Links
- Characteristics of Packet Capture
- Data Measurement Repositories
- Multihoming and Traffic Engineering
- What is Multihoming
- Why Multihome
- Redundancy
- Performance
- Multihoming in IP Networks Today
- BGP or no
- Same Provider or Multiple
- Multihomed Stub One Link
- Multihomed Stub Multiple Links
- Multihomed Stub Multiple ISPs
- Multihomed Transit Network
- Interdomain Traffic Engineering
- Outbound Traffic Control
- Outbound Traffic Load Balancing
- Traffic Engineering Goals
- Managing Scale
- Achieving Predictability
- Challenges to Predictability
- Inter-AS Negotiation
- Outbound Multihoming Goals
- Inbound Traffic Control
- How many links are enough
- Problems with Multihoming in IPv4
- Slide 68
- Slide 69
- Diagnosis and Troubleshooting
- Operator Mailing List (NANOG)
- Operator Mailing List
- Routing Configuration
- Internet Business Model (Simplified)
- Peering Contracts Consistent Export
- Consistent Export
- Inconsistent Export in Practice
- Blackholes
- Security
- Security ldquoBogonrdquo Routes
- Spam Phishing etc
- Spam and Routing
- Worms and Botnets
- What is a Worm
- Example Worm Code Red
- Code Red Revisions
- Code Red Host Infection Rate
- Designing Fast-Spreading Worms
- Botnets
- ldquoRallyingrdquo the Botnet
- Botnet Control
- Some Defenses
- Idea 1 Ingress Filtering
- Idea 2 uRPF Checks
- Problems with uRPF
- S-BGP
- Practical Problems with S-BGP
-
22
The Address Allocation Process
bull Allocation policies of RIRs affect pressure on IPv4 address space
IANA
AfriNIC APNIC ARIN LACNIC RIPE
httpwwwianaorgassignmentsipv4-address-space
Georgia Tech
23
Common Problems
bull Diagnosis and troubleshooting (hence measurement)bull Traffic engineeringbull Securitybull Design and capacity planning
24
What can go wrong
Two-thirds of the problems are caused by configuration of the routing protocol
Some downtime is very hard to protect againsthellip
25
Measurement and Monitoring
26
Passive vs Active Measurement
bull Passive Measurement Collection of packets flow statistics of traffic that is already flowing on the networkndash Packet tracesndash Flow statisticsndash Application-level logs
bull Active Measurement Inject ldquoprobingrdquo traffic to measure various characteristicsndash Traceroutendash Pingndash Application-level probes (eg Web downloads)
27
Billing for Internet Usage
bull 95th Percentile billingndash Customer network pays for ldquocommitted information
raterdquo (CIR)ndash Throughput measured every 5 minutes (typically with
SNMP flow statistics also can be used for billing)ndash Customer billed based on 95th percentile
28
Passive Traffic Data Measurement
bull SNMP bytepacket counts everywhere
bull Packet monitoring selected locations
bull Flow monitoring typically at edges (if possible)ndash Direct computation of the traffic matrixndash Input to denial-of-service attack detection
bull Deep Packet Inspection also at edge where possible
29
Simple Network Management Protocol
bull Management Information Base (MIB)ndash Information storendash Unique variables named by
OIDsndash Accessed with SNMP
bull Specific MIBs for bytepacket counts (per link)
Manager Agent
SNMP
DB
ManagedObjects
30
SNMP (Passive)
bull Advantage ubiquitousndash Supported on all networking equipmentndash Multiple products for polling and analyzing data
bull Disadvantages ndash Coarse granularityndash Cannot express complex queries on the datandash Unreliable delivery of the data using UDP
bull Utilityndash Link utilization (billing)ndash Traffic matrix inference
31
Packet-level Monitoring
bull Passive monitoring to collect full packet contents (or at least headers)
bull Advantages lots of detailed informationndash Precise timing informationndash Information in packet headers
bull Disadvantages overheadndash Hard to keep up with high-speed linksndash Often requires a separate monitoring device
32
Full Packet Capture (Passive)
Example Georgia Tech OC3Mon
bull Rack-mounted PCbull Optical splitterbull Data Acquisition and
Generation (DAG) card
Source endacecom
33
What is a flow
bull Source IP addressbull Destination IP addressbull Source portbull Destination portbull Layer 3 protocol typebull TOS byte (DSCP)bull Input logical interface (ifIndex)
34
Cisco NetFlow
bull Basic output ldquoFlow recordrdquondash Most common version is v5
bull Current version (9) is being standardized in the IETF (template-based)ndash More flexible record formatndash Much easier to add new flow record types
Core Network
Collection and Aggregation
Collector (PC)Approximately 1500 bytes
20-50 flow recordsSent more frequently if traffic increases
35
Flow Record Contents
bull Source and Destination IP address and portbull Packet and byte countsbull Start and end timesbull ToS TCP flags
Basic information about the flowhellip
hellipplus information related to routing
bull Next-hop IP addressbull Source and destination ASbull Source and destination prefix
36
flow 1 flow 2 flow 3 flow 4
Aggregating Packets into Flows
bull Criteria 1 Set of packets that ldquobelong togetherrdquondash Sourcedestination IP addresses and port numbersndash Same protocol ToS bits hellip ndash Same inputoutput interfaces at a router (if known)
bull Criteria 2 Packets that are ldquocloserdquo together in timendash Maximum inter-packet spacing (eg 15 sec 30 sec)ndash Example flows 2 and 4 are different flows due to time
37
Reducing Measurement Overhead
bull Filtering on interfacendash destination prefix for a customerndash port number for an application (eg 80 for Web)
bull Sampling before insertion into flow cachendash Random deterministic or hash-based samplingndash 1-out-of-n or stratified based on packetflow sizendash Two types packet-level and flow-level
bull Aggregation after cache evictionndash packetsflows with same next-hop ASndash packetsflows destined to a particular service
38
Packet Sampling for Flow Monitoring
bull Packet sampling before flow creation (Sampled Netflow)ndash 1-out-of-m sampling of individual packets (eg m=100)ndash Create of flow records over the sampled packets
bull Reducing overheadndash Avoid per-packet overhead on (m-1)m packetsndash Avoid creating records for a large number of small flows
bull Increasing overhead (in some cases)ndash May split some long transfers into multiple flow records ndash hellip due to larger time gaps between successive packets
time
not sampled
two flowstimeout
39
Sampling Flow-Level Sampling
bull Sampling of flow records evicted from flow cachendash When evicting flows from table or when analyzing flows
bull Stratified sampling to put weight on ldquoheavyrdquo flowsndash Select all long flows and sample the short flows
bull Reduces the number of flow records ndash Still measures the vast majority of the traffic
Flow 1 40 bytesFlow 2 15580 bytesFlow 3 8196 bytesFlow 4 5350789 bytesFlow 5 532 bytesFlow 6 7432 bytes
sample with 100 probability
sample with 01 probability
sample with 10 probability
40
Two Main Approaches
bull Packet-level Monitoringndash Keep packet-level statisticsndash Examine (and potentially log) variety of packet-level
statistics Essentially anything in the packetndash Timing
bull Flow-level Monitoringndash Monitor packet-by-packet (though sometimes
sampled)ndash Keep aggregate statistics on a flow
41
Packet Capture on High-Speed Links
Example Georgia Tech ldquoOC3Monrdquo
bull Rack-mounted PCbull Optical splitterbull Data Acquisition and
Generation (DAG) card
Source endacecom
42
Characteristics of Packet Capture
bull Allows inspection on every packet on 10G links
bull Disadvantagesndash Costlyndash Requires splitting optical fibersndash Must be able to filterstore data
43
Data Measurement Repositories
bull AbileneInternet 2 Observatoryndash Configuration examplesndash SNMP datandash ISIS BGP routing data NetFlow traffic data
bull RouteViewsndash BGP updatesndash BGP table snapshots
44
Multihoming and Traffic Engineering
45
What is Multihoming
bull The use of redundant network links for the purposes of external connectivity
bull Can be achieved at many layers of the protocol stack and many places in the networkndash Multiple network interfaces in a PC
ndash An ISP with multiple upstream interfaces
bull Can refer to having multiple connections tondash The same ISP
ndash Multiple ISPs
46
Why Multihome
bull Redundancybull Availabilitybull Performancebull Cost
Interdomain traffic engineering the process by which a multihomed network configures its
network to achieve these goals
47
Redundancy
bull Maintain connectivity in the face ofndash Physical connectivity problems (fiber cut device
failures etc)ndash Failures in upstream ISP
48
Performance
bull Use multiple network links at once to achieve higher throughput than just over a single link
bull Allows incoming traffic to be load-balanced
70 of traffic30 of traffic
49
Multihoming in IP Networks Today
bull Stub AS no transit service for other ASesndash No need to use BGP
bull Multi-homed stub AS has connectivity to multiple immediate upstream ISPsndash Need BGPndash No need for a public AS numberndash No need for IP prefix allocation
bull Multi-homed transit AS connectivity to multiple ASes and transit servicendash Need BGP public AS number IP prefix allocation
50
BGP or no
bull Advantages of static routingndash Cheapersmaller routers (less true nowadays)ndash Simpler to configure
bull Advantages of BGPndash More control of your destiny (have providers stop
announcing you)ndash Fastermore intelligent selection of where to send
outbound packetsndash Better debugging of net problems (you can see the
Internet topology now)
51
Same Provider or Multiple
bull If your provider is reliable and fast and affordably and offers good tech-support you may want to multi-home initially to them via some backup path (slow is better than dead)
bull Eventually yoursquoll want to multi-home to different providers to avoid failure modes due to one providerrsquos architecture decisions
52
Multihomed Stub One Link
bull Downstream ISPrsquos routers configure default (ldquostaticrdquo) routes pointing to border router
bull Upstream ISP advertises reachability
Upstream ISP
Multiple links between same pair of routers
Default routes to ldquoborderrdquo
ldquoStubrdquoISP
53
Multihomed Stub Multiple Links
bull Use BGP to share loadbull Use private AS number (why is this OK)bull As before upstream ISP advertises prefix
Upstream ISP
Multiple links to different upstream routers
ldquoStubrdquoISP
Internal routing for ldquohot potatordquo
BGP for load balance at edge
54
Multihomed Stub Multiple ISPs
bull Many possibilitiesndash Load sharingndash Primary-backupndash Selective use of different ISPs
bull Requires BGP public AS number etc
ldquoStubrdquoISP
Upstream
ISP 1
Upstream
ISP 2
55
Multihomed Transit Network
bull BGP everywherebull Incoming and outcoming trafficbull Challenge balancing load on intradomain and egress
links given an offered traffic load
TransitISP
ISP 1
ISP 2
ISP 3
56
Interdomain Traffic Engineering
bull The process by which a network operator configures the network to achievendash Traffic load balancendash Redundancy (primarybackup) etc
bull Two tasksndash Outbound traffic controlndash Inbound traffic control
bull Key Problems Predictability and Scalability
57
Outbound Traffic Control
bull Easier to control than inbound trafficndash Destination-based routing sender determines where
the packets go
bull Control over next-hop AS onlyndash Cannot control selection of the entire path
Provider 1 Provider 2
Control with local preference
58
Outbound Traffic Load Balancingbull Control routes to provider per-prefix
ndash Assign local preference across destination prefixesndash Change the local preference assignments over time
bull Useful inputs to load balancingndash End-to-end path performance datandash Outbound traffic statistics per destination prefix
bull Challenge Getting from traffic volumes to groups of prefixes that should be assigned to each link
Premise of ldquointelligent route controlrdquo preoducts
59
Traffic Engineering Goals
bull Predictabilityndash Ensure the BGP decision process is deterministicndash Assume that BGP updates are (relatively) stable
bull Limit overhead introduced by routing changesndash Minimize frequency of changes to routing policiesndash Limit number of prefixes affected by changes
bull Limit impact on how traffic enters the networkndash Avoid new routes that might change neighborrsquos mindndash Select route with same attributes or at least path length
60
Managing Scalebull Destination prefixes
ndash More than 90000 destination prefixes
bull Donrsquot want to have per-prefix routing policies
ndash Small fraction of prefixes contribute most of the traffic
bull Focus on the small number of heavy hitters
ndash Define routing policies for selected prefixes
bull Routing choicesndash About 27000 unique ldquorouting choicesrdquo
bull Help in reducing the scale of the problem
ndash Small fraction of ldquorouting choicesrdquo contribute most traffic
bull Focus on the very small number of ldquorouting choicesrdquo
ndash Define routing policies on common attributes
61
Achieving Predictability
bull Route prediction with static analysisndash Helpful to know effects before deploymentndash Static analysis can help
TopologyBGP policy
configuration
eBGP routes
Offered traffic
BGP routingmodel
Flow of traffic through the network
62
Challenges to Predictabilitybull For transit ISPs effects on incoming traffic
ndash Lack of coordination strikes again
63
ldquoHot Potatordquo routing
Inter-AS Negotiation
bull Coordination aids predictabilityndash Negotiate where to sendndash Inbound and outboundndash Mutual benefits
bull How to implementndash What info to exchangendash Protecting privacyndash How to prioritize choicesndash How to prevent cheating
Destination 2
Destination 1
multiplepeeringpoints
Provider A
Provider B
64
Outbound Multihoming Goals
bull Redundancyndash Dynamic routing will failover to backup link
bull Performancendash Select provider with best performance per prefix
ndash Requires active probing
bull Costndash Select provider per prefix over time to minimize the
total financial cost
65
Inbound Traffic Control
bull More difficult no control over neighborsrsquo decisions
bull Three common techniques (previously discussed)ndash AS path prependingndash Communities and local preferencendash Prefix splitting
How does todayrsquos paper (MONET) control inbound traffic
66
How many links are enough
K upstream ISPs
Not much benefit beyond 4 ISPs
Akella et al ldquoPerformance Benefits of Multihomingrdquo SIGCOMM 2003
67
Problems with Multihoming in IPv4
bull Routing table growthndash Provider-based addressingndash Advertising prefix out multiple ISPs ndash canrsquot aggregate
bull Poor control over inbound trafficndash Existing mechanisms do not allow hosts to control
inbound traffic
68
GeorgiaTech
Internet Routing Overview
bull Intradomain (ie ldquointra-ASrdquo) routingbull Interdomain routing
Comcast
Abilene
ATampT Cogent
Autonomous Systems (ASes)
69
Configuration Problems ldquoAS 7007rdquoldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
UUNet
Florida InternetBarn
Sprint
70
Diagnosis and Troubleshootingldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
ldquoMicrosofts websites were offline for up to 23 hoursbecause of a [router] misconfigurationhellipit took nearly a day to determine what was wrong and undo the changesrdquo -- wiredcom January 25 2001
ldquoWorldCom Inchellipsuffered a widespread outage on its Internet backbone that affected roughly 20 percent of its US customer base The network problemshellipaffected millions of computer users worldwide A spokeswoman attributed the outage to a route table issue -- cnncom October 3 2002
A number of Covad customers went out from 5pm today due to supposedly a DDOS (distributed denial of service attack) on a key Level3 data center which later was described as a route leak (misconfiguration)ldquo
-- dslreportscom February 23 2004
71
Operator Mailing List (NANOG)Date Mon 18 Oct 2004 091515 -0700Subject Level 3 US east coast issues
Level 3 experiencing widespread unspecified routing issues on the US east coast Master ticket 1086844 Anyone have more specific information
Date Mon 18 Oct 2004 122034 -0400 (EDT)Subject Re Level 3 US east coast issues
Level 3 is currently experiencing a backbone outage causing routing instability and packet loss We are working to restore and will be sending hourly updateshellip
72
Operator Mailing List
0102030405060708090
Filtering RouteLeaks
RouteHijacks
RouteInstability
RoutingLoops
Blackholes
Occurr
ences o
ver
10 Y
ears
1995-1997 1998-2001 2001-2004
Note Only includes problems openly discussed on this list
Compare 83 power outages 1 fire
73
Routing Configuration
Ranking route selection
Dissemination internal route advertisement
Filtering route advertisement
Customer
Competitor
Primary
Backup
74
Internet Business Model (Simplified)
bull CustomerProvider One AS pays another for reachability to some set of destinations
bull ldquoSettlement-freerdquo Peering Bartering Two ASes exchange routes with one another
Provider
Peer
Customer
Preferences implemented with local preference manipulation
Destination
Pay to use
Get paid to use
Free to use
75
Peering Contracts Consistent Export
bull Rules of settlement-free peeringndash Advertise routes at all peering pointsndash Advertised routes must have equal ldquoAS path lengthrdquo
Sprint
ATampT
Enables ldquohot potatordquo routing
ldquoequally goodrdquoroutes
76
Consistent Export
bull Malicedeceptionbull iBGP signaling partitionbull Inconsistent export policy
neighbor 10123route-map PEER permit 10 set prepend 123
neighbor 10456route-map PEER permit 10 set prepend 123 123
Possible Causes
10123 456 1
10456 456 2
Neighbor AS Export
1 1 123
Export Clause Prepend
2 1 123 123
Two different Export Policies
77
Inconsistent Export in Practice
Feamster et al ldquoBorderGuard Detecting Cold Potatoes from Peersrdquo ACM IMC October 2004
78
Blackholes
Date Thu 18 Jul 2002 060510 -0400 (EDT)From Chad Oleary ltcolpoboxcomgtSubject Re problems with 701To ltnanogmeritedugt
Were starting to see the same issues with UUNet again Anyone elseseeing this Trying to reach Qwest
traceroute to 631461901 (631461901) 30 hops max 38 byte packets 1 esc-lp2-gwe-solutionscorpcom (631182201) 1167 ms 1163 ms 1142 ms 2 500Serial2-10GW1TPA2ALTERNET (1571301499) 1097 ms 1059 ms 1044 ms 3 161at-1-0-0XL4ATL1ALTERNET (1526381190) 13839 ms 14108 ms 16638 ms 4 0so-3-1-0XL2ATL5ALTERNET (152630238) 14370 ms 14587 ms 14553 ms 5 POS7-0BR2ATL5ALTERNET (1526382193) 13928 ms 14099 ms 14053 ms 6 7 hellip
79
Security
80
Security ldquoBogonrdquo Routes
Feamster et al ldquoAn Empirical Study of lsquoBogonrsquo Route Advertisementsrdquo ACM CCR January 2005
81
Spam Phishing etc
bull Unsolicited commercial emailbull As of about August 2008 estimates indicate that
about 95 of all email is spambull Common spam filtering techniques
ndash Content-based filtersndash DNS Blacklist (DNSBL) lookups Significant fraction of
todayrsquos DNS traffic
Can IP addresses from which spam is received be spoofed
82
Spam and Routing
83
Worms and Botnets
84
What is a Worm
bull Code that replicates and propagates across the networkndash Often carries a ldquopayloadrdquo
bull Usually spread via exploiting flaws in open servicesndash ldquoVirusesrdquo require user action to spread
bull First worm Robert Morris November 1988ndash 6-10 of all Internet hosts infected ()
bull Many more since but none on that scale until July 2001
85
Example Worm Code Red
bull Initial version July 13 2001
bull Exploited known ISAPI vulnerability in Microsoft IIS Web servers
bull 1st through 20th of each month spread20th through end of each month attack
bull Payload Web site defacementbull Scanning Random IP addressesbull Bug failure to seed random number generator
86
Code Red Revisions
bull Released July 19 2001
bull Payload flooding attack on wwwwhitehousegovndash Attack was mounted at the IP address of the Web site
bull Bug died after 20th of each month
bull Random number generator for IP scanning fixed
87
Code Red Host Infection Rate
Exponential infection rate
Measured using backscatter technique
88
Designing Fast-Spreading Worms
bull Hit-list scanningndash Time to infect first 10k hosts dominates infection timendash Solution Reconnaissance (stealthy scans etc)
bull Permutation scanningndash Observation Most scanning is redundantndash Idea Shared permutation of address space Start scanning
from own IP address Re-randomize when another infected machine is found
bull Internet-scale hit listsndash Flash worm complete infection within 30 seconds
89
Botnets
bull Bots Autonomous programs performing tasksbull Plenty of ldquobenignrdquo bots
ndash eg weatherbug
bull Botnets group of bots ndash Typically carries malicious connotationndash Large numbers of infected machinesndash Machines ldquoenlistedrdquo with infection vectors like worms
(last lecture)
bull Available for simultaneous control by a masterbull Size up to 350000 nodes (from todayrsquos paper)
90
ldquoRallyingrdquo the Botnet
bull Easy to combine worm backdoor functionalitybull Problem how to learn about successfully
infected machines
bull Optionsndash Emailndash Hard-coded email address
91
Botnet Control
bull Botnet master typically runs some IRC server on a well-known port (eg 6667)
bull Infected machine contacts botnet with pre-programmed DNS name (eg big-botde)
bull Dynamic DNS allows controller to move about freely
Infected Machine
DynamicDNS
BotnetController
(IRC server)
92
Some Defenses
93
Idea 1 Ingress Filtering
bull RFC 2827 Routers install filters to drop packets from networks that are not downstream
bull Feasible at edgesbull Difficult to configure closer to network ldquocorerdquo
20469207024 Internet
Drop all packets with source address other than 20469207024
94
Idea 2 uRPF Checks
bull Unicast Reverse Path Forwardingndash Cisco ldquoip verify unicast reverse-pathrdquo
bull Requires symmetric routing
Accept packet from interface only if forwarding table entry for source IP address matches ingress interface
100183
A10018
10015
101203
100181 24
10011 24
Strict Mode uRPF
Enabled
ldquoArdquo Routing TableDestination Next Hop1001024 Int 110018024 Int 2
100183 from wrong interface
95
Problems with uRPF
bull Asymmetric routing
96
S-BGP
bull Address-based PKI validate signaturesndash Authentication of
bull ownership for IP address blocks bull AS number bull an ASs identity and bull a BGP routers identity
ndash Use existing infrastructure (Internet registries etc)ndash Routing origination is digitally signedndash BGP updates are digitally signed
1048708 bull Route attestations A new optional BGP transitive path attribute
ndash carries digital signatures covering the routing information in updates
97
Practical Problems with S-BGP
bull Requires Public-Key Infrastructure
bull Lots of digital signatures to calculate and verifyndash Message overheadndash CPU overhead
bull Calculation expense is greatest when topology is changingndash Caching can help
bull Route aggregation is problematic (maybe thatrsquos OK)
bull Secure route withdrawals when link or node fails
bull Address ownership data out of date
bull Deployment
- Networking
- Goal of This Tutorial
- Todayrsquos Networks
- Business Models
- Billing for Internet Usage
- Net Neutrality
- Network Operations
- Point-of-Presence (PoP)
- Example Abilene Network Topology
- Another Example Backbone
- Internet Routing Overview
- Internet Routing Protocol BGP
- Two Flavors of BGP
- IPv4 Addresses Networks of Networks
- Pre-1994 Classful Addressing
- Classless Interdomain Routing (CIDR)
- Benefits of CIDR
- Growth of IP Prefixes
- 1994-1998 Linear Growth
- Around 2000 Fast Growth Resumes
- Fast growth resumes
- The Address Allocation Process
- Common Problems
- What can go wrong
- Measurement and Monitoring
- Passive vs Active Measurement
- Slide 27
- Passive Traffic Data Measurement
- Simple Network Management Protocol
- SNMP (Passive)
- Packet-level Monitoring
- Full Packet Capture (Passive)
- What is a flow
- Cisco NetFlow
- Flow Record Contents
- Aggregating Packets into Flows
- Reducing Measurement Overhead
- Packet Sampling for Flow Monitoring
- Sampling Flow-Level Sampling
- Two Main Approaches
- Packet Capture on High-Speed Links
- Characteristics of Packet Capture
- Data Measurement Repositories
- Multihoming and Traffic Engineering
- What is Multihoming
- Why Multihome
- Redundancy
- Performance
- Multihoming in IP Networks Today
- BGP or no
- Same Provider or Multiple
- Multihomed Stub One Link
- Multihomed Stub Multiple Links
- Multihomed Stub Multiple ISPs
- Multihomed Transit Network
- Interdomain Traffic Engineering
- Outbound Traffic Control
- Outbound Traffic Load Balancing
- Traffic Engineering Goals
- Managing Scale
- Achieving Predictability
- Challenges to Predictability
- Inter-AS Negotiation
- Outbound Multihoming Goals
- Inbound Traffic Control
- How many links are enough
- Problems with Multihoming in IPv4
- Slide 68
- Slide 69
- Diagnosis and Troubleshooting
- Operator Mailing List (NANOG)
- Operator Mailing List
- Routing Configuration
- Internet Business Model (Simplified)
- Peering Contracts Consistent Export
- Consistent Export
- Inconsistent Export in Practice
- Blackholes
- Security
- Security ldquoBogonrdquo Routes
- Spam Phishing etc
- Spam and Routing
- Worms and Botnets
- What is a Worm
- Example Worm Code Red
- Code Red Revisions
- Code Red Host Infection Rate
- Designing Fast-Spreading Worms
- Botnets
- ldquoRallyingrdquo the Botnet
- Botnet Control
- Some Defenses
- Idea 1 Ingress Filtering
- Idea 2 uRPF Checks
- Problems with uRPF
- S-BGP
- Practical Problems with S-BGP
-
23
Common Problems
bull Diagnosis and troubleshooting (hence measurement)bull Traffic engineeringbull Securitybull Design and capacity planning
24
What can go wrong
Two-thirds of the problems are caused by configuration of the routing protocol
Some downtime is very hard to protect againsthellip
25
Measurement and Monitoring
26
Passive vs Active Measurement
bull Passive Measurement Collection of packets flow statistics of traffic that is already flowing on the networkndash Packet tracesndash Flow statisticsndash Application-level logs
bull Active Measurement Inject ldquoprobingrdquo traffic to measure various characteristicsndash Traceroutendash Pingndash Application-level probes (eg Web downloads)
27
Billing for Internet Usage
bull 95th Percentile billingndash Customer network pays for ldquocommitted information
raterdquo (CIR)ndash Throughput measured every 5 minutes (typically with
SNMP flow statistics also can be used for billing)ndash Customer billed based on 95th percentile
28
Passive Traffic Data Measurement
bull SNMP bytepacket counts everywhere
bull Packet monitoring selected locations
bull Flow monitoring typically at edges (if possible)ndash Direct computation of the traffic matrixndash Input to denial-of-service attack detection
bull Deep Packet Inspection also at edge where possible
29
Simple Network Management Protocol
bull Management Information Base (MIB)ndash Information storendash Unique variables named by
OIDsndash Accessed with SNMP
bull Specific MIBs for bytepacket counts (per link)
Manager Agent
SNMP
DB
ManagedObjects
30
SNMP (Passive)
bull Advantage ubiquitousndash Supported on all networking equipmentndash Multiple products for polling and analyzing data
bull Disadvantages ndash Coarse granularityndash Cannot express complex queries on the datandash Unreliable delivery of the data using UDP
bull Utilityndash Link utilization (billing)ndash Traffic matrix inference
31
Packet-level Monitoring
bull Passive monitoring to collect full packet contents (or at least headers)
bull Advantages lots of detailed informationndash Precise timing informationndash Information in packet headers
bull Disadvantages overheadndash Hard to keep up with high-speed linksndash Often requires a separate monitoring device
32
Full Packet Capture (Passive)
Example Georgia Tech OC3Mon
bull Rack-mounted PCbull Optical splitterbull Data Acquisition and
Generation (DAG) card
Source endacecom
33
What is a flow
bull Source IP addressbull Destination IP addressbull Source portbull Destination portbull Layer 3 protocol typebull TOS byte (DSCP)bull Input logical interface (ifIndex)
34
Cisco NetFlow
bull Basic output ldquoFlow recordrdquondash Most common version is v5
bull Current version (9) is being standardized in the IETF (template-based)ndash More flexible record formatndash Much easier to add new flow record types
Core Network
Collection and Aggregation
Collector (PC)Approximately 1500 bytes
20-50 flow recordsSent more frequently if traffic increases
35
Flow Record Contents
bull Source and Destination IP address and portbull Packet and byte countsbull Start and end timesbull ToS TCP flags
Basic information about the flowhellip
hellipplus information related to routing
bull Next-hop IP addressbull Source and destination ASbull Source and destination prefix
36
flow 1 flow 2 flow 3 flow 4
Aggregating Packets into Flows
bull Criteria 1 Set of packets that ldquobelong togetherrdquondash Sourcedestination IP addresses and port numbersndash Same protocol ToS bits hellip ndash Same inputoutput interfaces at a router (if known)
bull Criteria 2 Packets that are ldquocloserdquo together in timendash Maximum inter-packet spacing (eg 15 sec 30 sec)ndash Example flows 2 and 4 are different flows due to time
37
Reducing Measurement Overhead
bull Filtering on interfacendash destination prefix for a customerndash port number for an application (eg 80 for Web)
bull Sampling before insertion into flow cachendash Random deterministic or hash-based samplingndash 1-out-of-n or stratified based on packetflow sizendash Two types packet-level and flow-level
bull Aggregation after cache evictionndash packetsflows with same next-hop ASndash packetsflows destined to a particular service
38
Packet Sampling for Flow Monitoring
bull Packet sampling before flow creation (Sampled Netflow)ndash 1-out-of-m sampling of individual packets (eg m=100)ndash Create of flow records over the sampled packets
bull Reducing overheadndash Avoid per-packet overhead on (m-1)m packetsndash Avoid creating records for a large number of small flows
bull Increasing overhead (in some cases)ndash May split some long transfers into multiple flow records ndash hellip due to larger time gaps between successive packets
time
not sampled
two flowstimeout
39
Sampling Flow-Level Sampling
bull Sampling of flow records evicted from flow cachendash When evicting flows from table or when analyzing flows
bull Stratified sampling to put weight on ldquoheavyrdquo flowsndash Select all long flows and sample the short flows
bull Reduces the number of flow records ndash Still measures the vast majority of the traffic
Flow 1 40 bytesFlow 2 15580 bytesFlow 3 8196 bytesFlow 4 5350789 bytesFlow 5 532 bytesFlow 6 7432 bytes
sample with 100 probability
sample with 01 probability
sample with 10 probability
40
Two Main Approaches
bull Packet-level Monitoringndash Keep packet-level statisticsndash Examine (and potentially log) variety of packet-level
statistics Essentially anything in the packetndash Timing
bull Flow-level Monitoringndash Monitor packet-by-packet (though sometimes
sampled)ndash Keep aggregate statistics on a flow
41
Packet Capture on High-Speed Links
Example Georgia Tech ldquoOC3Monrdquo
bull Rack-mounted PCbull Optical splitterbull Data Acquisition and
Generation (DAG) card
Source endacecom
42
Characteristics of Packet Capture
bull Allows inspection on every packet on 10G links
bull Disadvantagesndash Costlyndash Requires splitting optical fibersndash Must be able to filterstore data
43
Data Measurement Repositories
bull AbileneInternet 2 Observatoryndash Configuration examplesndash SNMP datandash ISIS BGP routing data NetFlow traffic data
bull RouteViewsndash BGP updatesndash BGP table snapshots
44
Multihoming and Traffic Engineering
45
What is Multihoming
bull The use of redundant network links for the purposes of external connectivity
bull Can be achieved at many layers of the protocol stack and many places in the networkndash Multiple network interfaces in a PC
ndash An ISP with multiple upstream interfaces
bull Can refer to having multiple connections tondash The same ISP
ndash Multiple ISPs
46
Why Multihome
bull Redundancybull Availabilitybull Performancebull Cost
Interdomain traffic engineering the process by which a multihomed network configures its
network to achieve these goals
47
Redundancy
bull Maintain connectivity in the face ofndash Physical connectivity problems (fiber cut device
failures etc)ndash Failures in upstream ISP
48
Performance
bull Use multiple network links at once to achieve higher throughput than just over a single link
bull Allows incoming traffic to be load-balanced
70 of traffic30 of traffic
49
Multihoming in IP Networks Today
bull Stub AS no transit service for other ASesndash No need to use BGP
bull Multi-homed stub AS has connectivity to multiple immediate upstream ISPsndash Need BGPndash No need for a public AS numberndash No need for IP prefix allocation
bull Multi-homed transit AS connectivity to multiple ASes and transit servicendash Need BGP public AS number IP prefix allocation
50
BGP or no
bull Advantages of static routingndash Cheapersmaller routers (less true nowadays)ndash Simpler to configure
bull Advantages of BGPndash More control of your destiny (have providers stop
announcing you)ndash Fastermore intelligent selection of where to send
outbound packetsndash Better debugging of net problems (you can see the
Internet topology now)
51
Same Provider or Multiple
bull If your provider is reliable and fast and affordably and offers good tech-support you may want to multi-home initially to them via some backup path (slow is better than dead)
bull Eventually yoursquoll want to multi-home to different providers to avoid failure modes due to one providerrsquos architecture decisions
52
Multihomed Stub One Link
bull Downstream ISPrsquos routers configure default (ldquostaticrdquo) routes pointing to border router
bull Upstream ISP advertises reachability
Upstream ISP
Multiple links between same pair of routers
Default routes to ldquoborderrdquo
ldquoStubrdquoISP
53
Multihomed Stub Multiple Links
bull Use BGP to share loadbull Use private AS number (why is this OK)bull As before upstream ISP advertises prefix
Upstream ISP
Multiple links to different upstream routers
ldquoStubrdquoISP
Internal routing for ldquohot potatordquo
BGP for load balance at edge
54
Multihomed Stub Multiple ISPs
bull Many possibilitiesndash Load sharingndash Primary-backupndash Selective use of different ISPs
bull Requires BGP public AS number etc
ldquoStubrdquoISP
Upstream
ISP 1
Upstream
ISP 2
55
Multihomed Transit Network
bull BGP everywherebull Incoming and outcoming trafficbull Challenge balancing load on intradomain and egress
links given an offered traffic load
TransitISP
ISP 1
ISP 2
ISP 3
56
Interdomain Traffic Engineering
bull The process by which a network operator configures the network to achievendash Traffic load balancendash Redundancy (primarybackup) etc
bull Two tasksndash Outbound traffic controlndash Inbound traffic control
bull Key Problems Predictability and Scalability
57
Outbound Traffic Control
bull Easier to control than inbound trafficndash Destination-based routing sender determines where
the packets go
bull Control over next-hop AS onlyndash Cannot control selection of the entire path
Provider 1 Provider 2
Control with local preference
58
Outbound Traffic Load Balancingbull Control routes to provider per-prefix
ndash Assign local preference across destination prefixesndash Change the local preference assignments over time
bull Useful inputs to load balancingndash End-to-end path performance datandash Outbound traffic statistics per destination prefix
bull Challenge Getting from traffic volumes to groups of prefixes that should be assigned to each link
Premise of ldquointelligent route controlrdquo preoducts
59
Traffic Engineering Goals
bull Predictabilityndash Ensure the BGP decision process is deterministicndash Assume that BGP updates are (relatively) stable
bull Limit overhead introduced by routing changesndash Minimize frequency of changes to routing policiesndash Limit number of prefixes affected by changes
bull Limit impact on how traffic enters the networkndash Avoid new routes that might change neighborrsquos mindndash Select route with same attributes or at least path length
60
Managing Scalebull Destination prefixes
ndash More than 90000 destination prefixes
bull Donrsquot want to have per-prefix routing policies
ndash Small fraction of prefixes contribute most of the traffic
bull Focus on the small number of heavy hitters
ndash Define routing policies for selected prefixes
bull Routing choicesndash About 27000 unique ldquorouting choicesrdquo
bull Help in reducing the scale of the problem
ndash Small fraction of ldquorouting choicesrdquo contribute most traffic
bull Focus on the very small number of ldquorouting choicesrdquo
ndash Define routing policies on common attributes
61
Achieving Predictability
bull Route prediction with static analysisndash Helpful to know effects before deploymentndash Static analysis can help
TopologyBGP policy
configuration
eBGP routes
Offered traffic
BGP routingmodel
Flow of traffic through the network
62
Challenges to Predictabilitybull For transit ISPs effects on incoming traffic
ndash Lack of coordination strikes again
63
ldquoHot Potatordquo routing
Inter-AS Negotiation
bull Coordination aids predictabilityndash Negotiate where to sendndash Inbound and outboundndash Mutual benefits
bull How to implementndash What info to exchangendash Protecting privacyndash How to prioritize choicesndash How to prevent cheating
Destination 2
Destination 1
multiplepeeringpoints
Provider A
Provider B
64
Outbound Multihoming Goals
bull Redundancyndash Dynamic routing will failover to backup link
bull Performancendash Select provider with best performance per prefix
ndash Requires active probing
bull Costndash Select provider per prefix over time to minimize the
total financial cost
65
Inbound Traffic Control
bull More difficult no control over neighborsrsquo decisions
bull Three common techniques (previously discussed)ndash AS path prependingndash Communities and local preferencendash Prefix splitting
How does todayrsquos paper (MONET) control inbound traffic
66
How many links are enough
K upstream ISPs
Not much benefit beyond 4 ISPs
Akella et al ldquoPerformance Benefits of Multihomingrdquo SIGCOMM 2003
67
Problems with Multihoming in IPv4
bull Routing table growthndash Provider-based addressingndash Advertising prefix out multiple ISPs ndash canrsquot aggregate
bull Poor control over inbound trafficndash Existing mechanisms do not allow hosts to control
inbound traffic
68
GeorgiaTech
Internet Routing Overview
bull Intradomain (ie ldquointra-ASrdquo) routingbull Interdomain routing
Comcast
Abilene
ATampT Cogent
Autonomous Systems (ASes)
69
Configuration Problems ldquoAS 7007rdquoldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
UUNet
Florida InternetBarn
Sprint
70
Diagnosis and Troubleshootingldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
ldquoMicrosofts websites were offline for up to 23 hoursbecause of a [router] misconfigurationhellipit took nearly a day to determine what was wrong and undo the changesrdquo -- wiredcom January 25 2001
ldquoWorldCom Inchellipsuffered a widespread outage on its Internet backbone that affected roughly 20 percent of its US customer base The network problemshellipaffected millions of computer users worldwide A spokeswoman attributed the outage to a route table issue -- cnncom October 3 2002
A number of Covad customers went out from 5pm today due to supposedly a DDOS (distributed denial of service attack) on a key Level3 data center which later was described as a route leak (misconfiguration)ldquo
-- dslreportscom February 23 2004
71
Operator Mailing List (NANOG)Date Mon 18 Oct 2004 091515 -0700Subject Level 3 US east coast issues
Level 3 experiencing widespread unspecified routing issues on the US east coast Master ticket 1086844 Anyone have more specific information
Date Mon 18 Oct 2004 122034 -0400 (EDT)Subject Re Level 3 US east coast issues
Level 3 is currently experiencing a backbone outage causing routing instability and packet loss We are working to restore and will be sending hourly updateshellip
72
Operator Mailing List
0102030405060708090
Filtering RouteLeaks
RouteHijacks
RouteInstability
RoutingLoops
Blackholes
Occurr
ences o
ver
10 Y
ears
1995-1997 1998-2001 2001-2004
Note Only includes problems openly discussed on this list
Compare 83 power outages 1 fire
73
Routing Configuration
Ranking route selection
Dissemination internal route advertisement
Filtering route advertisement
Customer
Competitor
Primary
Backup
74
Internet Business Model (Simplified)
bull CustomerProvider One AS pays another for reachability to some set of destinations
bull ldquoSettlement-freerdquo Peering Bartering Two ASes exchange routes with one another
Provider
Peer
Customer
Preferences implemented with local preference manipulation
Destination
Pay to use
Get paid to use
Free to use
75
Peering Contracts Consistent Export
bull Rules of settlement-free peeringndash Advertise routes at all peering pointsndash Advertised routes must have equal ldquoAS path lengthrdquo
Sprint
ATampT
Enables ldquohot potatordquo routing
ldquoequally goodrdquoroutes
76
Consistent Export
bull Malicedeceptionbull iBGP signaling partitionbull Inconsistent export policy
neighbor 10123route-map PEER permit 10 set prepend 123
neighbor 10456route-map PEER permit 10 set prepend 123 123
Possible Causes
10123 456 1
10456 456 2
Neighbor AS Export
1 1 123
Export Clause Prepend
2 1 123 123
Two different Export Policies
77
Inconsistent Export in Practice
Feamster et al ldquoBorderGuard Detecting Cold Potatoes from Peersrdquo ACM IMC October 2004
78
Blackholes
Date Thu 18 Jul 2002 060510 -0400 (EDT)From Chad Oleary ltcolpoboxcomgtSubject Re problems with 701To ltnanogmeritedugt
Were starting to see the same issues with UUNet again Anyone elseseeing this Trying to reach Qwest
traceroute to 631461901 (631461901) 30 hops max 38 byte packets 1 esc-lp2-gwe-solutionscorpcom (631182201) 1167 ms 1163 ms 1142 ms 2 500Serial2-10GW1TPA2ALTERNET (1571301499) 1097 ms 1059 ms 1044 ms 3 161at-1-0-0XL4ATL1ALTERNET (1526381190) 13839 ms 14108 ms 16638 ms 4 0so-3-1-0XL2ATL5ALTERNET (152630238) 14370 ms 14587 ms 14553 ms 5 POS7-0BR2ATL5ALTERNET (1526382193) 13928 ms 14099 ms 14053 ms 6 7 hellip
79
Security
80
Security ldquoBogonrdquo Routes
Feamster et al ldquoAn Empirical Study of lsquoBogonrsquo Route Advertisementsrdquo ACM CCR January 2005
81
Spam Phishing etc
bull Unsolicited commercial emailbull As of about August 2008 estimates indicate that
about 95 of all email is spambull Common spam filtering techniques
ndash Content-based filtersndash DNS Blacklist (DNSBL) lookups Significant fraction of
todayrsquos DNS traffic
Can IP addresses from which spam is received be spoofed
82
Spam and Routing
83
Worms and Botnets
84
What is a Worm
bull Code that replicates and propagates across the networkndash Often carries a ldquopayloadrdquo
bull Usually spread via exploiting flaws in open servicesndash ldquoVirusesrdquo require user action to spread
bull First worm Robert Morris November 1988ndash 6-10 of all Internet hosts infected ()
bull Many more since but none on that scale until July 2001
85
Example Worm Code Red
bull Initial version July 13 2001
bull Exploited known ISAPI vulnerability in Microsoft IIS Web servers
bull 1st through 20th of each month spread20th through end of each month attack
bull Payload Web site defacementbull Scanning Random IP addressesbull Bug failure to seed random number generator
86
Code Red Revisions
bull Released July 19 2001
bull Payload flooding attack on wwwwhitehousegovndash Attack was mounted at the IP address of the Web site
bull Bug died after 20th of each month
bull Random number generator for IP scanning fixed
87
Code Red Host Infection Rate
Exponential infection rate
Measured using backscatter technique
88
Designing Fast-Spreading Worms
bull Hit-list scanningndash Time to infect first 10k hosts dominates infection timendash Solution Reconnaissance (stealthy scans etc)
bull Permutation scanningndash Observation Most scanning is redundantndash Idea Shared permutation of address space Start scanning
from own IP address Re-randomize when another infected machine is found
bull Internet-scale hit listsndash Flash worm complete infection within 30 seconds
89
Botnets
bull Bots Autonomous programs performing tasksbull Plenty of ldquobenignrdquo bots
ndash eg weatherbug
bull Botnets group of bots ndash Typically carries malicious connotationndash Large numbers of infected machinesndash Machines ldquoenlistedrdquo with infection vectors like worms
(last lecture)
bull Available for simultaneous control by a masterbull Size up to 350000 nodes (from todayrsquos paper)
90
ldquoRallyingrdquo the Botnet
bull Easy to combine worm backdoor functionalitybull Problem how to learn about successfully
infected machines
bull Optionsndash Emailndash Hard-coded email address
91
Botnet Control
bull Botnet master typically runs some IRC server on a well-known port (eg 6667)
bull Infected machine contacts botnet with pre-programmed DNS name (eg big-botde)
bull Dynamic DNS allows controller to move about freely
Infected Machine
DynamicDNS
BotnetController
(IRC server)
92
Some Defenses
93
Idea 1 Ingress Filtering
bull RFC 2827 Routers install filters to drop packets from networks that are not downstream
bull Feasible at edgesbull Difficult to configure closer to network ldquocorerdquo
20469207024 Internet
Drop all packets with source address other than 20469207024
94
Idea 2 uRPF Checks
bull Unicast Reverse Path Forwardingndash Cisco ldquoip verify unicast reverse-pathrdquo
bull Requires symmetric routing
Accept packet from interface only if forwarding table entry for source IP address matches ingress interface
100183
A10018
10015
101203
100181 24
10011 24
Strict Mode uRPF
Enabled
ldquoArdquo Routing TableDestination Next Hop1001024 Int 110018024 Int 2
100183 from wrong interface
95
Problems with uRPF
bull Asymmetric routing
96
S-BGP
bull Address-based PKI validate signaturesndash Authentication of
bull ownership for IP address blocks bull AS number bull an ASs identity and bull a BGP routers identity
ndash Use existing infrastructure (Internet registries etc)ndash Routing origination is digitally signedndash BGP updates are digitally signed
1048708 bull Route attestations A new optional BGP transitive path attribute
ndash carries digital signatures covering the routing information in updates
97
Practical Problems with S-BGP
bull Requires Public-Key Infrastructure
bull Lots of digital signatures to calculate and verifyndash Message overheadndash CPU overhead
bull Calculation expense is greatest when topology is changingndash Caching can help
bull Route aggregation is problematic (maybe thatrsquos OK)
bull Secure route withdrawals when link or node fails
bull Address ownership data out of date
bull Deployment
- Networking
- Goal of This Tutorial
- Todayrsquos Networks
- Business Models
- Billing for Internet Usage
- Net Neutrality
- Network Operations
- Point-of-Presence (PoP)
- Example Abilene Network Topology
- Another Example Backbone
- Internet Routing Overview
- Internet Routing Protocol BGP
- Two Flavors of BGP
- IPv4 Addresses Networks of Networks
- Pre-1994 Classful Addressing
- Classless Interdomain Routing (CIDR)
- Benefits of CIDR
- Growth of IP Prefixes
- 1994-1998 Linear Growth
- Around 2000 Fast Growth Resumes
- Fast growth resumes
- The Address Allocation Process
- Common Problems
- What can go wrong
- Measurement and Monitoring
- Passive vs Active Measurement
- Slide 27
- Passive Traffic Data Measurement
- Simple Network Management Protocol
- SNMP (Passive)
- Packet-level Monitoring
- Full Packet Capture (Passive)
- What is a flow
- Cisco NetFlow
- Flow Record Contents
- Aggregating Packets into Flows
- Reducing Measurement Overhead
- Packet Sampling for Flow Monitoring
- Sampling Flow-Level Sampling
- Two Main Approaches
- Packet Capture on High-Speed Links
- Characteristics of Packet Capture
- Data Measurement Repositories
- Multihoming and Traffic Engineering
- What is Multihoming
- Why Multihome
- Redundancy
- Performance
- Multihoming in IP Networks Today
- BGP or no
- Same Provider or Multiple
- Multihomed Stub One Link
- Multihomed Stub Multiple Links
- Multihomed Stub Multiple ISPs
- Multihomed Transit Network
- Interdomain Traffic Engineering
- Outbound Traffic Control
- Outbound Traffic Load Balancing
- Traffic Engineering Goals
- Managing Scale
- Achieving Predictability
- Challenges to Predictability
- Inter-AS Negotiation
- Outbound Multihoming Goals
- Inbound Traffic Control
- How many links are enough
- Problems with Multihoming in IPv4
- Slide 68
- Slide 69
- Diagnosis and Troubleshooting
- Operator Mailing List (NANOG)
- Operator Mailing List
- Routing Configuration
- Internet Business Model (Simplified)
- Peering Contracts Consistent Export
- Consistent Export
- Inconsistent Export in Practice
- Blackholes
- Security
- Security ldquoBogonrdquo Routes
- Spam Phishing etc
- Spam and Routing
- Worms and Botnets
- What is a Worm
- Example Worm Code Red
- Code Red Revisions
- Code Red Host Infection Rate
- Designing Fast-Spreading Worms
- Botnets
- ldquoRallyingrdquo the Botnet
- Botnet Control
- Some Defenses
- Idea 1 Ingress Filtering
- Idea 2 uRPF Checks
- Problems with uRPF
- S-BGP
- Practical Problems with S-BGP
-
24
What can go wrong
Two-thirds of the problems are caused by configuration of the routing protocol
Some downtime is very hard to protect againsthellip
25
Measurement and Monitoring
26
Passive vs Active Measurement
bull Passive Measurement Collection of packets flow statistics of traffic that is already flowing on the networkndash Packet tracesndash Flow statisticsndash Application-level logs
bull Active Measurement Inject ldquoprobingrdquo traffic to measure various characteristicsndash Traceroutendash Pingndash Application-level probes (eg Web downloads)
27
Billing for Internet Usage
bull 95th Percentile billingndash Customer network pays for ldquocommitted information
raterdquo (CIR)ndash Throughput measured every 5 minutes (typically with
SNMP flow statistics also can be used for billing)ndash Customer billed based on 95th percentile
28
Passive Traffic Data Measurement
bull SNMP bytepacket counts everywhere
bull Packet monitoring selected locations
bull Flow monitoring typically at edges (if possible)ndash Direct computation of the traffic matrixndash Input to denial-of-service attack detection
bull Deep Packet Inspection also at edge where possible
29
Simple Network Management Protocol
bull Management Information Base (MIB)ndash Information storendash Unique variables named by
OIDsndash Accessed with SNMP
bull Specific MIBs for bytepacket counts (per link)
Manager Agent
SNMP
DB
ManagedObjects
30
SNMP (Passive)
bull Advantage ubiquitousndash Supported on all networking equipmentndash Multiple products for polling and analyzing data
bull Disadvantages ndash Coarse granularityndash Cannot express complex queries on the datandash Unreliable delivery of the data using UDP
bull Utilityndash Link utilization (billing)ndash Traffic matrix inference
31
Packet-level Monitoring
bull Passive monitoring to collect full packet contents (or at least headers)
bull Advantages lots of detailed informationndash Precise timing informationndash Information in packet headers
bull Disadvantages overheadndash Hard to keep up with high-speed linksndash Often requires a separate monitoring device
32
Full Packet Capture (Passive)
Example Georgia Tech OC3Mon
bull Rack-mounted PCbull Optical splitterbull Data Acquisition and
Generation (DAG) card
Source endacecom
33
What is a flow
bull Source IP addressbull Destination IP addressbull Source portbull Destination portbull Layer 3 protocol typebull TOS byte (DSCP)bull Input logical interface (ifIndex)
34
Cisco NetFlow
bull Basic output ldquoFlow recordrdquondash Most common version is v5
bull Current version (9) is being standardized in the IETF (template-based)ndash More flexible record formatndash Much easier to add new flow record types
Core Network
Collection and Aggregation
Collector (PC)Approximately 1500 bytes
20-50 flow recordsSent more frequently if traffic increases
35
Flow Record Contents
bull Source and Destination IP address and portbull Packet and byte countsbull Start and end timesbull ToS TCP flags
Basic information about the flowhellip
hellipplus information related to routing
bull Next-hop IP addressbull Source and destination ASbull Source and destination prefix
36
flow 1 flow 2 flow 3 flow 4
Aggregating Packets into Flows
bull Criteria 1 Set of packets that ldquobelong togetherrdquondash Sourcedestination IP addresses and port numbersndash Same protocol ToS bits hellip ndash Same inputoutput interfaces at a router (if known)
bull Criteria 2 Packets that are ldquocloserdquo together in timendash Maximum inter-packet spacing (eg 15 sec 30 sec)ndash Example flows 2 and 4 are different flows due to time
37
Reducing Measurement Overhead
bull Filtering on interfacendash destination prefix for a customerndash port number for an application (eg 80 for Web)
bull Sampling before insertion into flow cachendash Random deterministic or hash-based samplingndash 1-out-of-n or stratified based on packetflow sizendash Two types packet-level and flow-level
bull Aggregation after cache evictionndash packetsflows with same next-hop ASndash packetsflows destined to a particular service
38
Packet Sampling for Flow Monitoring
bull Packet sampling before flow creation (Sampled Netflow)ndash 1-out-of-m sampling of individual packets (eg m=100)ndash Create of flow records over the sampled packets
bull Reducing overheadndash Avoid per-packet overhead on (m-1)m packetsndash Avoid creating records for a large number of small flows
bull Increasing overhead (in some cases)ndash May split some long transfers into multiple flow records ndash hellip due to larger time gaps between successive packets
time
not sampled
two flowstimeout
39
Sampling Flow-Level Sampling
bull Sampling of flow records evicted from flow cachendash When evicting flows from table or when analyzing flows
bull Stratified sampling to put weight on ldquoheavyrdquo flowsndash Select all long flows and sample the short flows
bull Reduces the number of flow records ndash Still measures the vast majority of the traffic
Flow 1 40 bytesFlow 2 15580 bytesFlow 3 8196 bytesFlow 4 5350789 bytesFlow 5 532 bytesFlow 6 7432 bytes
sample with 100 probability
sample with 01 probability
sample with 10 probability
40
Two Main Approaches
bull Packet-level Monitoringndash Keep packet-level statisticsndash Examine (and potentially log) variety of packet-level
statistics Essentially anything in the packetndash Timing
bull Flow-level Monitoringndash Monitor packet-by-packet (though sometimes
sampled)ndash Keep aggregate statistics on a flow
41
Packet Capture on High-Speed Links
Example Georgia Tech ldquoOC3Monrdquo
bull Rack-mounted PCbull Optical splitterbull Data Acquisition and
Generation (DAG) card
Source endacecom
42
Characteristics of Packet Capture
bull Allows inspection on every packet on 10G links
bull Disadvantagesndash Costlyndash Requires splitting optical fibersndash Must be able to filterstore data
43
Data Measurement Repositories
bull AbileneInternet 2 Observatoryndash Configuration examplesndash SNMP datandash ISIS BGP routing data NetFlow traffic data
bull RouteViewsndash BGP updatesndash BGP table snapshots
44
Multihoming and Traffic Engineering
45
What is Multihoming
bull The use of redundant network links for the purposes of external connectivity
bull Can be achieved at many layers of the protocol stack and many places in the networkndash Multiple network interfaces in a PC
ndash An ISP with multiple upstream interfaces
bull Can refer to having multiple connections tondash The same ISP
ndash Multiple ISPs
46
Why Multihome
bull Redundancybull Availabilitybull Performancebull Cost
Interdomain traffic engineering the process by which a multihomed network configures its
network to achieve these goals
47
Redundancy
bull Maintain connectivity in the face ofndash Physical connectivity problems (fiber cut device
failures etc)ndash Failures in upstream ISP
48
Performance
bull Use multiple network links at once to achieve higher throughput than just over a single link
bull Allows incoming traffic to be load-balanced
70 of traffic30 of traffic
49
Multihoming in IP Networks Today
bull Stub AS no transit service for other ASesndash No need to use BGP
bull Multi-homed stub AS has connectivity to multiple immediate upstream ISPsndash Need BGPndash No need for a public AS numberndash No need for IP prefix allocation
bull Multi-homed transit AS connectivity to multiple ASes and transit servicendash Need BGP public AS number IP prefix allocation
50
BGP or no
bull Advantages of static routingndash Cheapersmaller routers (less true nowadays)ndash Simpler to configure
bull Advantages of BGPndash More control of your destiny (have providers stop
announcing you)ndash Fastermore intelligent selection of where to send
outbound packetsndash Better debugging of net problems (you can see the
Internet topology now)
51
Same Provider or Multiple
bull If your provider is reliable and fast and affordably and offers good tech-support you may want to multi-home initially to them via some backup path (slow is better than dead)
bull Eventually yoursquoll want to multi-home to different providers to avoid failure modes due to one providerrsquos architecture decisions
52
Multihomed Stub One Link
bull Downstream ISPrsquos routers configure default (ldquostaticrdquo) routes pointing to border router
bull Upstream ISP advertises reachability
Upstream ISP
Multiple links between same pair of routers
Default routes to ldquoborderrdquo
ldquoStubrdquoISP
53
Multihomed Stub Multiple Links
bull Use BGP to share loadbull Use private AS number (why is this OK)bull As before upstream ISP advertises prefix
Upstream ISP
Multiple links to different upstream routers
ldquoStubrdquoISP
Internal routing for ldquohot potatordquo
BGP for load balance at edge
54
Multihomed Stub Multiple ISPs
bull Many possibilitiesndash Load sharingndash Primary-backupndash Selective use of different ISPs
bull Requires BGP public AS number etc
ldquoStubrdquoISP
Upstream
ISP 1
Upstream
ISP 2
55
Multihomed Transit Network
bull BGP everywherebull Incoming and outcoming trafficbull Challenge balancing load on intradomain and egress
links given an offered traffic load
TransitISP
ISP 1
ISP 2
ISP 3
56
Interdomain Traffic Engineering
bull The process by which a network operator configures the network to achievendash Traffic load balancendash Redundancy (primarybackup) etc
bull Two tasksndash Outbound traffic controlndash Inbound traffic control
bull Key Problems Predictability and Scalability
57
Outbound Traffic Control
bull Easier to control than inbound trafficndash Destination-based routing sender determines where
the packets go
bull Control over next-hop AS onlyndash Cannot control selection of the entire path
Provider 1 Provider 2
Control with local preference
58
Outbound Traffic Load Balancingbull Control routes to provider per-prefix
ndash Assign local preference across destination prefixesndash Change the local preference assignments over time
bull Useful inputs to load balancingndash End-to-end path performance datandash Outbound traffic statistics per destination prefix
bull Challenge Getting from traffic volumes to groups of prefixes that should be assigned to each link
Premise of ldquointelligent route controlrdquo preoducts
59
Traffic Engineering Goals
bull Predictabilityndash Ensure the BGP decision process is deterministicndash Assume that BGP updates are (relatively) stable
bull Limit overhead introduced by routing changesndash Minimize frequency of changes to routing policiesndash Limit number of prefixes affected by changes
bull Limit impact on how traffic enters the networkndash Avoid new routes that might change neighborrsquos mindndash Select route with same attributes or at least path length
60
Managing Scalebull Destination prefixes
ndash More than 90000 destination prefixes
bull Donrsquot want to have per-prefix routing policies
ndash Small fraction of prefixes contribute most of the traffic
bull Focus on the small number of heavy hitters
ndash Define routing policies for selected prefixes
bull Routing choicesndash About 27000 unique ldquorouting choicesrdquo
bull Help in reducing the scale of the problem
ndash Small fraction of ldquorouting choicesrdquo contribute most traffic
bull Focus on the very small number of ldquorouting choicesrdquo
ndash Define routing policies on common attributes
61
Achieving Predictability
bull Route prediction with static analysisndash Helpful to know effects before deploymentndash Static analysis can help
TopologyBGP policy
configuration
eBGP routes
Offered traffic
BGP routingmodel
Flow of traffic through the network
62
Challenges to Predictabilitybull For transit ISPs effects on incoming traffic
ndash Lack of coordination strikes again
63
ldquoHot Potatordquo routing
Inter-AS Negotiation
bull Coordination aids predictabilityndash Negotiate where to sendndash Inbound and outboundndash Mutual benefits
bull How to implementndash What info to exchangendash Protecting privacyndash How to prioritize choicesndash How to prevent cheating
Destination 2
Destination 1
multiplepeeringpoints
Provider A
Provider B
64
Outbound Multihoming Goals
bull Redundancyndash Dynamic routing will failover to backup link
bull Performancendash Select provider with best performance per prefix
ndash Requires active probing
bull Costndash Select provider per prefix over time to minimize the
total financial cost
65
Inbound Traffic Control
bull More difficult no control over neighborsrsquo decisions
bull Three common techniques (previously discussed)ndash AS path prependingndash Communities and local preferencendash Prefix splitting
How does todayrsquos paper (MONET) control inbound traffic
66
How many links are enough
K upstream ISPs
Not much benefit beyond 4 ISPs
Akella et al ldquoPerformance Benefits of Multihomingrdquo SIGCOMM 2003
67
Problems with Multihoming in IPv4
bull Routing table growthndash Provider-based addressingndash Advertising prefix out multiple ISPs ndash canrsquot aggregate
bull Poor control over inbound trafficndash Existing mechanisms do not allow hosts to control
inbound traffic
68
GeorgiaTech
Internet Routing Overview
bull Intradomain (ie ldquointra-ASrdquo) routingbull Interdomain routing
Comcast
Abilene
ATampT Cogent
Autonomous Systems (ASes)
69
Configuration Problems ldquoAS 7007rdquoldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
UUNet
Florida InternetBarn
Sprint
70
Diagnosis and Troubleshootingldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
ldquoMicrosofts websites were offline for up to 23 hoursbecause of a [router] misconfigurationhellipit took nearly a day to determine what was wrong and undo the changesrdquo -- wiredcom January 25 2001
ldquoWorldCom Inchellipsuffered a widespread outage on its Internet backbone that affected roughly 20 percent of its US customer base The network problemshellipaffected millions of computer users worldwide A spokeswoman attributed the outage to a route table issue -- cnncom October 3 2002
A number of Covad customers went out from 5pm today due to supposedly a DDOS (distributed denial of service attack) on a key Level3 data center which later was described as a route leak (misconfiguration)ldquo
-- dslreportscom February 23 2004
71
Operator Mailing List (NANOG)Date Mon 18 Oct 2004 091515 -0700Subject Level 3 US east coast issues
Level 3 experiencing widespread unspecified routing issues on the US east coast Master ticket 1086844 Anyone have more specific information
Date Mon 18 Oct 2004 122034 -0400 (EDT)Subject Re Level 3 US east coast issues
Level 3 is currently experiencing a backbone outage causing routing instability and packet loss We are working to restore and will be sending hourly updateshellip
72
Operator Mailing List
0102030405060708090
Filtering RouteLeaks
RouteHijacks
RouteInstability
RoutingLoops
Blackholes
Occurr
ences o
ver
10 Y
ears
1995-1997 1998-2001 2001-2004
Note Only includes problems openly discussed on this list
Compare 83 power outages 1 fire
73
Routing Configuration
Ranking route selection
Dissemination internal route advertisement
Filtering route advertisement
Customer
Competitor
Primary
Backup
74
Internet Business Model (Simplified)
bull CustomerProvider One AS pays another for reachability to some set of destinations
bull ldquoSettlement-freerdquo Peering Bartering Two ASes exchange routes with one another
Provider
Peer
Customer
Preferences implemented with local preference manipulation
Destination
Pay to use
Get paid to use
Free to use
75
Peering Contracts Consistent Export
bull Rules of settlement-free peeringndash Advertise routes at all peering pointsndash Advertised routes must have equal ldquoAS path lengthrdquo
Sprint
ATampT
Enables ldquohot potatordquo routing
ldquoequally goodrdquoroutes
76
Consistent Export
bull Malicedeceptionbull iBGP signaling partitionbull Inconsistent export policy
neighbor 10123route-map PEER permit 10 set prepend 123
neighbor 10456route-map PEER permit 10 set prepend 123 123
Possible Causes
10123 456 1
10456 456 2
Neighbor AS Export
1 1 123
Export Clause Prepend
2 1 123 123
Two different Export Policies
77
Inconsistent Export in Practice
Feamster et al ldquoBorderGuard Detecting Cold Potatoes from Peersrdquo ACM IMC October 2004
78
Blackholes
Date Thu 18 Jul 2002 060510 -0400 (EDT)From Chad Oleary ltcolpoboxcomgtSubject Re problems with 701To ltnanogmeritedugt
Were starting to see the same issues with UUNet again Anyone elseseeing this Trying to reach Qwest
traceroute to 631461901 (631461901) 30 hops max 38 byte packets 1 esc-lp2-gwe-solutionscorpcom (631182201) 1167 ms 1163 ms 1142 ms 2 500Serial2-10GW1TPA2ALTERNET (1571301499) 1097 ms 1059 ms 1044 ms 3 161at-1-0-0XL4ATL1ALTERNET (1526381190) 13839 ms 14108 ms 16638 ms 4 0so-3-1-0XL2ATL5ALTERNET (152630238) 14370 ms 14587 ms 14553 ms 5 POS7-0BR2ATL5ALTERNET (1526382193) 13928 ms 14099 ms 14053 ms 6 7 hellip
79
Security
80
Security ldquoBogonrdquo Routes
Feamster et al ldquoAn Empirical Study of lsquoBogonrsquo Route Advertisementsrdquo ACM CCR January 2005
81
Spam Phishing etc
bull Unsolicited commercial emailbull As of about August 2008 estimates indicate that
about 95 of all email is spambull Common spam filtering techniques
ndash Content-based filtersndash DNS Blacklist (DNSBL) lookups Significant fraction of
todayrsquos DNS traffic
Can IP addresses from which spam is received be spoofed
82
Spam and Routing
83
Worms and Botnets
84
What is a Worm
bull Code that replicates and propagates across the networkndash Often carries a ldquopayloadrdquo
bull Usually spread via exploiting flaws in open servicesndash ldquoVirusesrdquo require user action to spread
bull First worm Robert Morris November 1988ndash 6-10 of all Internet hosts infected ()
bull Many more since but none on that scale until July 2001
85
Example Worm Code Red
bull Initial version July 13 2001
bull Exploited known ISAPI vulnerability in Microsoft IIS Web servers
bull 1st through 20th of each month spread20th through end of each month attack
bull Payload Web site defacementbull Scanning Random IP addressesbull Bug failure to seed random number generator
86
Code Red Revisions
bull Released July 19 2001
bull Payload flooding attack on wwwwhitehousegovndash Attack was mounted at the IP address of the Web site
bull Bug died after 20th of each month
bull Random number generator for IP scanning fixed
87
Code Red Host Infection Rate
Exponential infection rate
Measured using backscatter technique
88
Designing Fast-Spreading Worms
bull Hit-list scanningndash Time to infect first 10k hosts dominates infection timendash Solution Reconnaissance (stealthy scans etc)
bull Permutation scanningndash Observation Most scanning is redundantndash Idea Shared permutation of address space Start scanning
from own IP address Re-randomize when another infected machine is found
bull Internet-scale hit listsndash Flash worm complete infection within 30 seconds
89
Botnets
bull Bots Autonomous programs performing tasksbull Plenty of ldquobenignrdquo bots
ndash eg weatherbug
bull Botnets group of bots ndash Typically carries malicious connotationndash Large numbers of infected machinesndash Machines ldquoenlistedrdquo with infection vectors like worms
(last lecture)
bull Available for simultaneous control by a masterbull Size up to 350000 nodes (from todayrsquos paper)
90
ldquoRallyingrdquo the Botnet
bull Easy to combine worm backdoor functionalitybull Problem how to learn about successfully
infected machines
bull Optionsndash Emailndash Hard-coded email address
91
Botnet Control
bull Botnet master typically runs some IRC server on a well-known port (eg 6667)
bull Infected machine contacts botnet with pre-programmed DNS name (eg big-botde)
bull Dynamic DNS allows controller to move about freely
Infected Machine
DynamicDNS
BotnetController
(IRC server)
92
Some Defenses
93
Idea 1 Ingress Filtering
bull RFC 2827 Routers install filters to drop packets from networks that are not downstream
bull Feasible at edgesbull Difficult to configure closer to network ldquocorerdquo
20469207024 Internet
Drop all packets with source address other than 20469207024
94
Idea 2 uRPF Checks
bull Unicast Reverse Path Forwardingndash Cisco ldquoip verify unicast reverse-pathrdquo
bull Requires symmetric routing
Accept packet from interface only if forwarding table entry for source IP address matches ingress interface
100183
A10018
10015
101203
100181 24
10011 24
Strict Mode uRPF
Enabled
ldquoArdquo Routing TableDestination Next Hop1001024 Int 110018024 Int 2
100183 from wrong interface
95
Problems with uRPF
bull Asymmetric routing
96
S-BGP
bull Address-based PKI validate signaturesndash Authentication of
bull ownership for IP address blocks bull AS number bull an ASs identity and bull a BGP routers identity
ndash Use existing infrastructure (Internet registries etc)ndash Routing origination is digitally signedndash BGP updates are digitally signed
1048708 bull Route attestations A new optional BGP transitive path attribute
ndash carries digital signatures covering the routing information in updates
97
Practical Problems with S-BGP
bull Requires Public-Key Infrastructure
bull Lots of digital signatures to calculate and verifyndash Message overheadndash CPU overhead
bull Calculation expense is greatest when topology is changingndash Caching can help
bull Route aggregation is problematic (maybe thatrsquos OK)
bull Secure route withdrawals when link or node fails
bull Address ownership data out of date
bull Deployment
- Networking
- Goal of This Tutorial
- Todayrsquos Networks
- Business Models
- Billing for Internet Usage
- Net Neutrality
- Network Operations
- Point-of-Presence (PoP)
- Example Abilene Network Topology
- Another Example Backbone
- Internet Routing Overview
- Internet Routing Protocol BGP
- Two Flavors of BGP
- IPv4 Addresses Networks of Networks
- Pre-1994 Classful Addressing
- Classless Interdomain Routing (CIDR)
- Benefits of CIDR
- Growth of IP Prefixes
- 1994-1998 Linear Growth
- Around 2000 Fast Growth Resumes
- Fast growth resumes
- The Address Allocation Process
- Common Problems
- What can go wrong
- Measurement and Monitoring
- Passive vs Active Measurement
- Slide 27
- Passive Traffic Data Measurement
- Simple Network Management Protocol
- SNMP (Passive)
- Packet-level Monitoring
- Full Packet Capture (Passive)
- What is a flow
- Cisco NetFlow
- Flow Record Contents
- Aggregating Packets into Flows
- Reducing Measurement Overhead
- Packet Sampling for Flow Monitoring
- Sampling Flow-Level Sampling
- Two Main Approaches
- Packet Capture on High-Speed Links
- Characteristics of Packet Capture
- Data Measurement Repositories
- Multihoming and Traffic Engineering
- What is Multihoming
- Why Multihome
- Redundancy
- Performance
- Multihoming in IP Networks Today
- BGP or no
- Same Provider or Multiple
- Multihomed Stub One Link
- Multihomed Stub Multiple Links
- Multihomed Stub Multiple ISPs
- Multihomed Transit Network
- Interdomain Traffic Engineering
- Outbound Traffic Control
- Outbound Traffic Load Balancing
- Traffic Engineering Goals
- Managing Scale
- Achieving Predictability
- Challenges to Predictability
- Inter-AS Negotiation
- Outbound Multihoming Goals
- Inbound Traffic Control
- How many links are enough
- Problems with Multihoming in IPv4
- Slide 68
- Slide 69
- Diagnosis and Troubleshooting
- Operator Mailing List (NANOG)
- Operator Mailing List
- Routing Configuration
- Internet Business Model (Simplified)
- Peering Contracts Consistent Export
- Consistent Export
- Inconsistent Export in Practice
- Blackholes
- Security
- Security ldquoBogonrdquo Routes
- Spam Phishing etc
- Spam and Routing
- Worms and Botnets
- What is a Worm
- Example Worm Code Red
- Code Red Revisions
- Code Red Host Infection Rate
- Designing Fast-Spreading Worms
- Botnets
- ldquoRallyingrdquo the Botnet
- Botnet Control
- Some Defenses
- Idea 1 Ingress Filtering
- Idea 2 uRPF Checks
- Problems with uRPF
- S-BGP
- Practical Problems with S-BGP
-
25
Measurement and Monitoring
26
Passive vs Active Measurement
bull Passive Measurement Collection of packets flow statistics of traffic that is already flowing on the networkndash Packet tracesndash Flow statisticsndash Application-level logs
bull Active Measurement Inject ldquoprobingrdquo traffic to measure various characteristicsndash Traceroutendash Pingndash Application-level probes (eg Web downloads)
27
Billing for Internet Usage
bull 95th Percentile billingndash Customer network pays for ldquocommitted information
raterdquo (CIR)ndash Throughput measured every 5 minutes (typically with
SNMP flow statistics also can be used for billing)ndash Customer billed based on 95th percentile
28
Passive Traffic Data Measurement
bull SNMP bytepacket counts everywhere
bull Packet monitoring selected locations
bull Flow monitoring typically at edges (if possible)ndash Direct computation of the traffic matrixndash Input to denial-of-service attack detection
bull Deep Packet Inspection also at edge where possible
29
Simple Network Management Protocol
bull Management Information Base (MIB)ndash Information storendash Unique variables named by
OIDsndash Accessed with SNMP
bull Specific MIBs for bytepacket counts (per link)
Manager Agent
SNMP
DB
ManagedObjects
30
SNMP (Passive)
bull Advantage ubiquitousndash Supported on all networking equipmentndash Multiple products for polling and analyzing data
bull Disadvantages ndash Coarse granularityndash Cannot express complex queries on the datandash Unreliable delivery of the data using UDP
bull Utilityndash Link utilization (billing)ndash Traffic matrix inference
31
Packet-level Monitoring
bull Passive monitoring to collect full packet contents (or at least headers)
bull Advantages lots of detailed informationndash Precise timing informationndash Information in packet headers
bull Disadvantages overheadndash Hard to keep up with high-speed linksndash Often requires a separate monitoring device
32
Full Packet Capture (Passive)
Example Georgia Tech OC3Mon
bull Rack-mounted PCbull Optical splitterbull Data Acquisition and
Generation (DAG) card
Source endacecom
33
What is a flow
bull Source IP addressbull Destination IP addressbull Source portbull Destination portbull Layer 3 protocol typebull TOS byte (DSCP)bull Input logical interface (ifIndex)
34
Cisco NetFlow
bull Basic output ldquoFlow recordrdquondash Most common version is v5
bull Current version (9) is being standardized in the IETF (template-based)ndash More flexible record formatndash Much easier to add new flow record types
Core Network
Collection and Aggregation
Collector (PC)Approximately 1500 bytes
20-50 flow recordsSent more frequently if traffic increases
35
Flow Record Contents
bull Source and Destination IP address and portbull Packet and byte countsbull Start and end timesbull ToS TCP flags
Basic information about the flowhellip
hellipplus information related to routing
bull Next-hop IP addressbull Source and destination ASbull Source and destination prefix
36
flow 1 flow 2 flow 3 flow 4
Aggregating Packets into Flows
bull Criteria 1 Set of packets that ldquobelong togetherrdquondash Sourcedestination IP addresses and port numbersndash Same protocol ToS bits hellip ndash Same inputoutput interfaces at a router (if known)
bull Criteria 2 Packets that are ldquocloserdquo together in timendash Maximum inter-packet spacing (eg 15 sec 30 sec)ndash Example flows 2 and 4 are different flows due to time
37
Reducing Measurement Overhead
bull Filtering on interfacendash destination prefix for a customerndash port number for an application (eg 80 for Web)
bull Sampling before insertion into flow cachendash Random deterministic or hash-based samplingndash 1-out-of-n or stratified based on packetflow sizendash Two types packet-level and flow-level
bull Aggregation after cache evictionndash packetsflows with same next-hop ASndash packetsflows destined to a particular service
38
Packet Sampling for Flow Monitoring
bull Packet sampling before flow creation (Sampled Netflow)ndash 1-out-of-m sampling of individual packets (eg m=100)ndash Create of flow records over the sampled packets
bull Reducing overheadndash Avoid per-packet overhead on (m-1)m packetsndash Avoid creating records for a large number of small flows
bull Increasing overhead (in some cases)ndash May split some long transfers into multiple flow records ndash hellip due to larger time gaps between successive packets
time
not sampled
two flowstimeout
39
Sampling Flow-Level Sampling
bull Sampling of flow records evicted from flow cachendash When evicting flows from table or when analyzing flows
bull Stratified sampling to put weight on ldquoheavyrdquo flowsndash Select all long flows and sample the short flows
bull Reduces the number of flow records ndash Still measures the vast majority of the traffic
Flow 1 40 bytesFlow 2 15580 bytesFlow 3 8196 bytesFlow 4 5350789 bytesFlow 5 532 bytesFlow 6 7432 bytes
sample with 100 probability
sample with 01 probability
sample with 10 probability
40
Two Main Approaches
bull Packet-level Monitoringndash Keep packet-level statisticsndash Examine (and potentially log) variety of packet-level
statistics Essentially anything in the packetndash Timing
bull Flow-level Monitoringndash Monitor packet-by-packet (though sometimes
sampled)ndash Keep aggregate statistics on a flow
41
Packet Capture on High-Speed Links
Example Georgia Tech ldquoOC3Monrdquo
bull Rack-mounted PCbull Optical splitterbull Data Acquisition and
Generation (DAG) card
Source endacecom
42
Characteristics of Packet Capture
bull Allows inspection on every packet on 10G links
bull Disadvantagesndash Costlyndash Requires splitting optical fibersndash Must be able to filterstore data
43
Data Measurement Repositories
bull AbileneInternet 2 Observatoryndash Configuration examplesndash SNMP datandash ISIS BGP routing data NetFlow traffic data
bull RouteViewsndash BGP updatesndash BGP table snapshots
44
Multihoming and Traffic Engineering
45
What is Multihoming
bull The use of redundant network links for the purposes of external connectivity
bull Can be achieved at many layers of the protocol stack and many places in the networkndash Multiple network interfaces in a PC
ndash An ISP with multiple upstream interfaces
bull Can refer to having multiple connections tondash The same ISP
ndash Multiple ISPs
46
Why Multihome
bull Redundancybull Availabilitybull Performancebull Cost
Interdomain traffic engineering the process by which a multihomed network configures its
network to achieve these goals
47
Redundancy
bull Maintain connectivity in the face ofndash Physical connectivity problems (fiber cut device
failures etc)ndash Failures in upstream ISP
48
Performance
bull Use multiple network links at once to achieve higher throughput than just over a single link
bull Allows incoming traffic to be load-balanced
70 of traffic30 of traffic
49
Multihoming in IP Networks Today
bull Stub AS no transit service for other ASesndash No need to use BGP
bull Multi-homed stub AS has connectivity to multiple immediate upstream ISPsndash Need BGPndash No need for a public AS numberndash No need for IP prefix allocation
bull Multi-homed transit AS connectivity to multiple ASes and transit servicendash Need BGP public AS number IP prefix allocation
50
BGP or no
bull Advantages of static routingndash Cheapersmaller routers (less true nowadays)ndash Simpler to configure
bull Advantages of BGPndash More control of your destiny (have providers stop
announcing you)ndash Fastermore intelligent selection of where to send
outbound packetsndash Better debugging of net problems (you can see the
Internet topology now)
51
Same Provider or Multiple
bull If your provider is reliable and fast and affordably and offers good tech-support you may want to multi-home initially to them via some backup path (slow is better than dead)
bull Eventually yoursquoll want to multi-home to different providers to avoid failure modes due to one providerrsquos architecture decisions
52
Multihomed Stub One Link
bull Downstream ISPrsquos routers configure default (ldquostaticrdquo) routes pointing to border router
bull Upstream ISP advertises reachability
Upstream ISP
Multiple links between same pair of routers
Default routes to ldquoborderrdquo
ldquoStubrdquoISP
53
Multihomed Stub Multiple Links
bull Use BGP to share loadbull Use private AS number (why is this OK)bull As before upstream ISP advertises prefix
Upstream ISP
Multiple links to different upstream routers
ldquoStubrdquoISP
Internal routing for ldquohot potatordquo
BGP for load balance at edge
54
Multihomed Stub Multiple ISPs
bull Many possibilitiesndash Load sharingndash Primary-backupndash Selective use of different ISPs
bull Requires BGP public AS number etc
ldquoStubrdquoISP
Upstream
ISP 1
Upstream
ISP 2
55
Multihomed Transit Network
bull BGP everywherebull Incoming and outcoming trafficbull Challenge balancing load on intradomain and egress
links given an offered traffic load
TransitISP
ISP 1
ISP 2
ISP 3
56
Interdomain Traffic Engineering
bull The process by which a network operator configures the network to achievendash Traffic load balancendash Redundancy (primarybackup) etc
bull Two tasksndash Outbound traffic controlndash Inbound traffic control
bull Key Problems Predictability and Scalability
57
Outbound Traffic Control
bull Easier to control than inbound trafficndash Destination-based routing sender determines where
the packets go
bull Control over next-hop AS onlyndash Cannot control selection of the entire path
Provider 1 Provider 2
Control with local preference
58
Outbound Traffic Load Balancingbull Control routes to provider per-prefix
ndash Assign local preference across destination prefixesndash Change the local preference assignments over time
bull Useful inputs to load balancingndash End-to-end path performance datandash Outbound traffic statistics per destination prefix
bull Challenge Getting from traffic volumes to groups of prefixes that should be assigned to each link
Premise of ldquointelligent route controlrdquo preoducts
59
Traffic Engineering Goals
bull Predictabilityndash Ensure the BGP decision process is deterministicndash Assume that BGP updates are (relatively) stable
bull Limit overhead introduced by routing changesndash Minimize frequency of changes to routing policiesndash Limit number of prefixes affected by changes
bull Limit impact on how traffic enters the networkndash Avoid new routes that might change neighborrsquos mindndash Select route with same attributes or at least path length
60
Managing Scalebull Destination prefixes
ndash More than 90000 destination prefixes
bull Donrsquot want to have per-prefix routing policies
ndash Small fraction of prefixes contribute most of the traffic
bull Focus on the small number of heavy hitters
ndash Define routing policies for selected prefixes
bull Routing choicesndash About 27000 unique ldquorouting choicesrdquo
bull Help in reducing the scale of the problem
ndash Small fraction of ldquorouting choicesrdquo contribute most traffic
bull Focus on the very small number of ldquorouting choicesrdquo
ndash Define routing policies on common attributes
61
Achieving Predictability
bull Route prediction with static analysisndash Helpful to know effects before deploymentndash Static analysis can help
TopologyBGP policy
configuration
eBGP routes
Offered traffic
BGP routingmodel
Flow of traffic through the network
62
Challenges to Predictabilitybull For transit ISPs effects on incoming traffic
ndash Lack of coordination strikes again
63
ldquoHot Potatordquo routing
Inter-AS Negotiation
bull Coordination aids predictabilityndash Negotiate where to sendndash Inbound and outboundndash Mutual benefits
bull How to implementndash What info to exchangendash Protecting privacyndash How to prioritize choicesndash How to prevent cheating
Destination 2
Destination 1
multiplepeeringpoints
Provider A
Provider B
64
Outbound Multihoming Goals
bull Redundancyndash Dynamic routing will failover to backup link
bull Performancendash Select provider with best performance per prefix
ndash Requires active probing
bull Costndash Select provider per prefix over time to minimize the
total financial cost
65
Inbound Traffic Control
bull More difficult no control over neighborsrsquo decisions
bull Three common techniques (previously discussed)ndash AS path prependingndash Communities and local preferencendash Prefix splitting
How does todayrsquos paper (MONET) control inbound traffic
66
How many links are enough
K upstream ISPs
Not much benefit beyond 4 ISPs
Akella et al ldquoPerformance Benefits of Multihomingrdquo SIGCOMM 2003
67
Problems with Multihoming in IPv4
bull Routing table growthndash Provider-based addressingndash Advertising prefix out multiple ISPs ndash canrsquot aggregate
bull Poor control over inbound trafficndash Existing mechanisms do not allow hosts to control
inbound traffic
68
GeorgiaTech
Internet Routing Overview
bull Intradomain (ie ldquointra-ASrdquo) routingbull Interdomain routing
Comcast
Abilene
ATampT Cogent
Autonomous Systems (ASes)
69
Configuration Problems ldquoAS 7007rdquoldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
UUNet
Florida InternetBarn
Sprint
70
Diagnosis and Troubleshootingldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
ldquoMicrosofts websites were offline for up to 23 hoursbecause of a [router] misconfigurationhellipit took nearly a day to determine what was wrong and undo the changesrdquo -- wiredcom January 25 2001
ldquoWorldCom Inchellipsuffered a widespread outage on its Internet backbone that affected roughly 20 percent of its US customer base The network problemshellipaffected millions of computer users worldwide A spokeswoman attributed the outage to a route table issue -- cnncom October 3 2002
A number of Covad customers went out from 5pm today due to supposedly a DDOS (distributed denial of service attack) on a key Level3 data center which later was described as a route leak (misconfiguration)ldquo
-- dslreportscom February 23 2004
71
Operator Mailing List (NANOG)Date Mon 18 Oct 2004 091515 -0700Subject Level 3 US east coast issues
Level 3 experiencing widespread unspecified routing issues on the US east coast Master ticket 1086844 Anyone have more specific information
Date Mon 18 Oct 2004 122034 -0400 (EDT)Subject Re Level 3 US east coast issues
Level 3 is currently experiencing a backbone outage causing routing instability and packet loss We are working to restore and will be sending hourly updateshellip
72
Operator Mailing List
0102030405060708090
Filtering RouteLeaks
RouteHijacks
RouteInstability
RoutingLoops
Blackholes
Occurr
ences o
ver
10 Y
ears
1995-1997 1998-2001 2001-2004
Note Only includes problems openly discussed on this list
Compare 83 power outages 1 fire
73
Routing Configuration
Ranking route selection
Dissemination internal route advertisement
Filtering route advertisement
Customer
Competitor
Primary
Backup
74
Internet Business Model (Simplified)
bull CustomerProvider One AS pays another for reachability to some set of destinations
bull ldquoSettlement-freerdquo Peering Bartering Two ASes exchange routes with one another
Provider
Peer
Customer
Preferences implemented with local preference manipulation
Destination
Pay to use
Get paid to use
Free to use
75
Peering Contracts Consistent Export
bull Rules of settlement-free peeringndash Advertise routes at all peering pointsndash Advertised routes must have equal ldquoAS path lengthrdquo
Sprint
ATampT
Enables ldquohot potatordquo routing
ldquoequally goodrdquoroutes
76
Consistent Export
bull Malicedeceptionbull iBGP signaling partitionbull Inconsistent export policy
neighbor 10123route-map PEER permit 10 set prepend 123
neighbor 10456route-map PEER permit 10 set prepend 123 123
Possible Causes
10123 456 1
10456 456 2
Neighbor AS Export
1 1 123
Export Clause Prepend
2 1 123 123
Two different Export Policies
77
Inconsistent Export in Practice
Feamster et al ldquoBorderGuard Detecting Cold Potatoes from Peersrdquo ACM IMC October 2004
78
Blackholes
Date Thu 18 Jul 2002 060510 -0400 (EDT)From Chad Oleary ltcolpoboxcomgtSubject Re problems with 701To ltnanogmeritedugt
Were starting to see the same issues with UUNet again Anyone elseseeing this Trying to reach Qwest
traceroute to 631461901 (631461901) 30 hops max 38 byte packets 1 esc-lp2-gwe-solutionscorpcom (631182201) 1167 ms 1163 ms 1142 ms 2 500Serial2-10GW1TPA2ALTERNET (1571301499) 1097 ms 1059 ms 1044 ms 3 161at-1-0-0XL4ATL1ALTERNET (1526381190) 13839 ms 14108 ms 16638 ms 4 0so-3-1-0XL2ATL5ALTERNET (152630238) 14370 ms 14587 ms 14553 ms 5 POS7-0BR2ATL5ALTERNET (1526382193) 13928 ms 14099 ms 14053 ms 6 7 hellip
79
Security
80
Security ldquoBogonrdquo Routes
Feamster et al ldquoAn Empirical Study of lsquoBogonrsquo Route Advertisementsrdquo ACM CCR January 2005
81
Spam Phishing etc
bull Unsolicited commercial emailbull As of about August 2008 estimates indicate that
about 95 of all email is spambull Common spam filtering techniques
ndash Content-based filtersndash DNS Blacklist (DNSBL) lookups Significant fraction of
todayrsquos DNS traffic
Can IP addresses from which spam is received be spoofed
82
Spam and Routing
83
Worms and Botnets
84
What is a Worm
bull Code that replicates and propagates across the networkndash Often carries a ldquopayloadrdquo
bull Usually spread via exploiting flaws in open servicesndash ldquoVirusesrdquo require user action to spread
bull First worm Robert Morris November 1988ndash 6-10 of all Internet hosts infected ()
bull Many more since but none on that scale until July 2001
85
Example Worm Code Red
bull Initial version July 13 2001
bull Exploited known ISAPI vulnerability in Microsoft IIS Web servers
bull 1st through 20th of each month spread20th through end of each month attack
bull Payload Web site defacementbull Scanning Random IP addressesbull Bug failure to seed random number generator
86
Code Red Revisions
bull Released July 19 2001
bull Payload flooding attack on wwwwhitehousegovndash Attack was mounted at the IP address of the Web site
bull Bug died after 20th of each month
bull Random number generator for IP scanning fixed
87
Code Red Host Infection Rate
Exponential infection rate
Measured using backscatter technique
88
Designing Fast-Spreading Worms
bull Hit-list scanningndash Time to infect first 10k hosts dominates infection timendash Solution Reconnaissance (stealthy scans etc)
bull Permutation scanningndash Observation Most scanning is redundantndash Idea Shared permutation of address space Start scanning
from own IP address Re-randomize when another infected machine is found
bull Internet-scale hit listsndash Flash worm complete infection within 30 seconds
89
Botnets
bull Bots Autonomous programs performing tasksbull Plenty of ldquobenignrdquo bots
ndash eg weatherbug
bull Botnets group of bots ndash Typically carries malicious connotationndash Large numbers of infected machinesndash Machines ldquoenlistedrdquo with infection vectors like worms
(last lecture)
bull Available for simultaneous control by a masterbull Size up to 350000 nodes (from todayrsquos paper)
90
ldquoRallyingrdquo the Botnet
bull Easy to combine worm backdoor functionalitybull Problem how to learn about successfully
infected machines
bull Optionsndash Emailndash Hard-coded email address
91
Botnet Control
bull Botnet master typically runs some IRC server on a well-known port (eg 6667)
bull Infected machine contacts botnet with pre-programmed DNS name (eg big-botde)
bull Dynamic DNS allows controller to move about freely
Infected Machine
DynamicDNS
BotnetController
(IRC server)
92
Some Defenses
93
Idea 1 Ingress Filtering
bull RFC 2827 Routers install filters to drop packets from networks that are not downstream
bull Feasible at edgesbull Difficult to configure closer to network ldquocorerdquo
20469207024 Internet
Drop all packets with source address other than 20469207024
94
Idea 2 uRPF Checks
bull Unicast Reverse Path Forwardingndash Cisco ldquoip verify unicast reverse-pathrdquo
bull Requires symmetric routing
Accept packet from interface only if forwarding table entry for source IP address matches ingress interface
100183
A10018
10015
101203
100181 24
10011 24
Strict Mode uRPF
Enabled
ldquoArdquo Routing TableDestination Next Hop1001024 Int 110018024 Int 2
100183 from wrong interface
95
Problems with uRPF
bull Asymmetric routing
96
S-BGP
bull Address-based PKI validate signaturesndash Authentication of
bull ownership for IP address blocks bull AS number bull an ASs identity and bull a BGP routers identity
ndash Use existing infrastructure (Internet registries etc)ndash Routing origination is digitally signedndash BGP updates are digitally signed
1048708 bull Route attestations A new optional BGP transitive path attribute
ndash carries digital signatures covering the routing information in updates
97
Practical Problems with S-BGP
bull Requires Public-Key Infrastructure
bull Lots of digital signatures to calculate and verifyndash Message overheadndash CPU overhead
bull Calculation expense is greatest when topology is changingndash Caching can help
bull Route aggregation is problematic (maybe thatrsquos OK)
bull Secure route withdrawals when link or node fails
bull Address ownership data out of date
bull Deployment
- Networking
- Goal of This Tutorial
- Todayrsquos Networks
- Business Models
- Billing for Internet Usage
- Net Neutrality
- Network Operations
- Point-of-Presence (PoP)
- Example Abilene Network Topology
- Another Example Backbone
- Internet Routing Overview
- Internet Routing Protocol BGP
- Two Flavors of BGP
- IPv4 Addresses Networks of Networks
- Pre-1994 Classful Addressing
- Classless Interdomain Routing (CIDR)
- Benefits of CIDR
- Growth of IP Prefixes
- 1994-1998 Linear Growth
- Around 2000 Fast Growth Resumes
- Fast growth resumes
- The Address Allocation Process
- Common Problems
- What can go wrong
- Measurement and Monitoring
- Passive vs Active Measurement
- Slide 27
- Passive Traffic Data Measurement
- Simple Network Management Protocol
- SNMP (Passive)
- Packet-level Monitoring
- Full Packet Capture (Passive)
- What is a flow
- Cisco NetFlow
- Flow Record Contents
- Aggregating Packets into Flows
- Reducing Measurement Overhead
- Packet Sampling for Flow Monitoring
- Sampling Flow-Level Sampling
- Two Main Approaches
- Packet Capture on High-Speed Links
- Characteristics of Packet Capture
- Data Measurement Repositories
- Multihoming and Traffic Engineering
- What is Multihoming
- Why Multihome
- Redundancy
- Performance
- Multihoming in IP Networks Today
- BGP or no
- Same Provider or Multiple
- Multihomed Stub One Link
- Multihomed Stub Multiple Links
- Multihomed Stub Multiple ISPs
- Multihomed Transit Network
- Interdomain Traffic Engineering
- Outbound Traffic Control
- Outbound Traffic Load Balancing
- Traffic Engineering Goals
- Managing Scale
- Achieving Predictability
- Challenges to Predictability
- Inter-AS Negotiation
- Outbound Multihoming Goals
- Inbound Traffic Control
- How many links are enough
- Problems with Multihoming in IPv4
- Slide 68
- Slide 69
- Diagnosis and Troubleshooting
- Operator Mailing List (NANOG)
- Operator Mailing List
- Routing Configuration
- Internet Business Model (Simplified)
- Peering Contracts Consistent Export
- Consistent Export
- Inconsistent Export in Practice
- Blackholes
- Security
- Security ldquoBogonrdquo Routes
- Spam Phishing etc
- Spam and Routing
- Worms and Botnets
- What is a Worm
- Example Worm Code Red
- Code Red Revisions
- Code Red Host Infection Rate
- Designing Fast-Spreading Worms
- Botnets
- ldquoRallyingrdquo the Botnet
- Botnet Control
- Some Defenses
- Idea 1 Ingress Filtering
- Idea 2 uRPF Checks
- Problems with uRPF
- S-BGP
- Practical Problems with S-BGP
-
26
Passive vs Active Measurement
bull Passive Measurement Collection of packets flow statistics of traffic that is already flowing on the networkndash Packet tracesndash Flow statisticsndash Application-level logs
bull Active Measurement Inject ldquoprobingrdquo traffic to measure various characteristicsndash Traceroutendash Pingndash Application-level probes (eg Web downloads)
27
Billing for Internet Usage
bull 95th Percentile billingndash Customer network pays for ldquocommitted information
raterdquo (CIR)ndash Throughput measured every 5 minutes (typically with
SNMP flow statistics also can be used for billing)ndash Customer billed based on 95th percentile
28
Passive Traffic Data Measurement
bull SNMP bytepacket counts everywhere
bull Packet monitoring selected locations
bull Flow monitoring typically at edges (if possible)ndash Direct computation of the traffic matrixndash Input to denial-of-service attack detection
bull Deep Packet Inspection also at edge where possible
29
Simple Network Management Protocol
bull Management Information Base (MIB)ndash Information storendash Unique variables named by
OIDsndash Accessed with SNMP
bull Specific MIBs for bytepacket counts (per link)
Manager Agent
SNMP
DB
ManagedObjects
30
SNMP (Passive)
bull Advantage ubiquitousndash Supported on all networking equipmentndash Multiple products for polling and analyzing data
bull Disadvantages ndash Coarse granularityndash Cannot express complex queries on the datandash Unreliable delivery of the data using UDP
bull Utilityndash Link utilization (billing)ndash Traffic matrix inference
31
Packet-level Monitoring
bull Passive monitoring to collect full packet contents (or at least headers)
bull Advantages lots of detailed informationndash Precise timing informationndash Information in packet headers
bull Disadvantages overheadndash Hard to keep up with high-speed linksndash Often requires a separate monitoring device
32
Full Packet Capture (Passive)
Example Georgia Tech OC3Mon
bull Rack-mounted PCbull Optical splitterbull Data Acquisition and
Generation (DAG) card
Source endacecom
33
What is a flow
bull Source IP addressbull Destination IP addressbull Source portbull Destination portbull Layer 3 protocol typebull TOS byte (DSCP)bull Input logical interface (ifIndex)
34
Cisco NetFlow
bull Basic output ldquoFlow recordrdquondash Most common version is v5
bull Current version (9) is being standardized in the IETF (template-based)ndash More flexible record formatndash Much easier to add new flow record types
Core Network
Collection and Aggregation
Collector (PC)Approximately 1500 bytes
20-50 flow recordsSent more frequently if traffic increases
35
Flow Record Contents
bull Source and Destination IP address and portbull Packet and byte countsbull Start and end timesbull ToS TCP flags
Basic information about the flowhellip
hellipplus information related to routing
bull Next-hop IP addressbull Source and destination ASbull Source and destination prefix
36
flow 1 flow 2 flow 3 flow 4
Aggregating Packets into Flows
bull Criteria 1 Set of packets that ldquobelong togetherrdquondash Sourcedestination IP addresses and port numbersndash Same protocol ToS bits hellip ndash Same inputoutput interfaces at a router (if known)
bull Criteria 2 Packets that are ldquocloserdquo together in timendash Maximum inter-packet spacing (eg 15 sec 30 sec)ndash Example flows 2 and 4 are different flows due to time
37
Reducing Measurement Overhead
bull Filtering on interfacendash destination prefix for a customerndash port number for an application (eg 80 for Web)
bull Sampling before insertion into flow cachendash Random deterministic or hash-based samplingndash 1-out-of-n or stratified based on packetflow sizendash Two types packet-level and flow-level
bull Aggregation after cache evictionndash packetsflows with same next-hop ASndash packetsflows destined to a particular service
38
Packet Sampling for Flow Monitoring
bull Packet sampling before flow creation (Sampled Netflow)ndash 1-out-of-m sampling of individual packets (eg m=100)ndash Create of flow records over the sampled packets
bull Reducing overheadndash Avoid per-packet overhead on (m-1)m packetsndash Avoid creating records for a large number of small flows
bull Increasing overhead (in some cases)ndash May split some long transfers into multiple flow records ndash hellip due to larger time gaps between successive packets
time
not sampled
two flowstimeout
39
Sampling Flow-Level Sampling
bull Sampling of flow records evicted from flow cachendash When evicting flows from table or when analyzing flows
bull Stratified sampling to put weight on ldquoheavyrdquo flowsndash Select all long flows and sample the short flows
bull Reduces the number of flow records ndash Still measures the vast majority of the traffic
Flow 1 40 bytesFlow 2 15580 bytesFlow 3 8196 bytesFlow 4 5350789 bytesFlow 5 532 bytesFlow 6 7432 bytes
sample with 100 probability
sample with 01 probability
sample with 10 probability
40
Two Main Approaches
bull Packet-level Monitoringndash Keep packet-level statisticsndash Examine (and potentially log) variety of packet-level
statistics Essentially anything in the packetndash Timing
bull Flow-level Monitoringndash Monitor packet-by-packet (though sometimes
sampled)ndash Keep aggregate statistics on a flow
41
Packet Capture on High-Speed Links
Example Georgia Tech ldquoOC3Monrdquo
bull Rack-mounted PCbull Optical splitterbull Data Acquisition and
Generation (DAG) card
Source endacecom
42
Characteristics of Packet Capture
bull Allows inspection on every packet on 10G links
bull Disadvantagesndash Costlyndash Requires splitting optical fibersndash Must be able to filterstore data
43
Data Measurement Repositories
bull AbileneInternet 2 Observatoryndash Configuration examplesndash SNMP datandash ISIS BGP routing data NetFlow traffic data
bull RouteViewsndash BGP updatesndash BGP table snapshots
44
Multihoming and Traffic Engineering
45
What is Multihoming
bull The use of redundant network links for the purposes of external connectivity
bull Can be achieved at many layers of the protocol stack and many places in the networkndash Multiple network interfaces in a PC
ndash An ISP with multiple upstream interfaces
bull Can refer to having multiple connections tondash The same ISP
ndash Multiple ISPs
46
Why Multihome
bull Redundancybull Availabilitybull Performancebull Cost
Interdomain traffic engineering the process by which a multihomed network configures its
network to achieve these goals
47
Redundancy
bull Maintain connectivity in the face ofndash Physical connectivity problems (fiber cut device
failures etc)ndash Failures in upstream ISP
48
Performance
bull Use multiple network links at once to achieve higher throughput than just over a single link
bull Allows incoming traffic to be load-balanced
70 of traffic30 of traffic
49
Multihoming in IP Networks Today
bull Stub AS no transit service for other ASesndash No need to use BGP
bull Multi-homed stub AS has connectivity to multiple immediate upstream ISPsndash Need BGPndash No need for a public AS numberndash No need for IP prefix allocation
bull Multi-homed transit AS connectivity to multiple ASes and transit servicendash Need BGP public AS number IP prefix allocation
50
BGP or no
bull Advantages of static routingndash Cheapersmaller routers (less true nowadays)ndash Simpler to configure
bull Advantages of BGPndash More control of your destiny (have providers stop
announcing you)ndash Fastermore intelligent selection of where to send
outbound packetsndash Better debugging of net problems (you can see the
Internet topology now)
51
Same Provider or Multiple
bull If your provider is reliable and fast and affordably and offers good tech-support you may want to multi-home initially to them via some backup path (slow is better than dead)
bull Eventually yoursquoll want to multi-home to different providers to avoid failure modes due to one providerrsquos architecture decisions
52
Multihomed Stub One Link
bull Downstream ISPrsquos routers configure default (ldquostaticrdquo) routes pointing to border router
bull Upstream ISP advertises reachability
Upstream ISP
Multiple links between same pair of routers
Default routes to ldquoborderrdquo
ldquoStubrdquoISP
53
Multihomed Stub Multiple Links
bull Use BGP to share loadbull Use private AS number (why is this OK)bull As before upstream ISP advertises prefix
Upstream ISP
Multiple links to different upstream routers
ldquoStubrdquoISP
Internal routing for ldquohot potatordquo
BGP for load balance at edge
54
Multihomed Stub Multiple ISPs
bull Many possibilitiesndash Load sharingndash Primary-backupndash Selective use of different ISPs
bull Requires BGP public AS number etc
ldquoStubrdquoISP
Upstream
ISP 1
Upstream
ISP 2
55
Multihomed Transit Network
bull BGP everywherebull Incoming and outcoming trafficbull Challenge balancing load on intradomain and egress
links given an offered traffic load
TransitISP
ISP 1
ISP 2
ISP 3
56
Interdomain Traffic Engineering
bull The process by which a network operator configures the network to achievendash Traffic load balancendash Redundancy (primarybackup) etc
bull Two tasksndash Outbound traffic controlndash Inbound traffic control
bull Key Problems Predictability and Scalability
57
Outbound Traffic Control
bull Easier to control than inbound trafficndash Destination-based routing sender determines where
the packets go
bull Control over next-hop AS onlyndash Cannot control selection of the entire path
Provider 1 Provider 2
Control with local preference
58
Outbound Traffic Load Balancingbull Control routes to provider per-prefix
ndash Assign local preference across destination prefixesndash Change the local preference assignments over time
bull Useful inputs to load balancingndash End-to-end path performance datandash Outbound traffic statistics per destination prefix
bull Challenge Getting from traffic volumes to groups of prefixes that should be assigned to each link
Premise of ldquointelligent route controlrdquo preoducts
59
Traffic Engineering Goals
bull Predictabilityndash Ensure the BGP decision process is deterministicndash Assume that BGP updates are (relatively) stable
bull Limit overhead introduced by routing changesndash Minimize frequency of changes to routing policiesndash Limit number of prefixes affected by changes
bull Limit impact on how traffic enters the networkndash Avoid new routes that might change neighborrsquos mindndash Select route with same attributes or at least path length
60
Managing Scalebull Destination prefixes
ndash More than 90000 destination prefixes
bull Donrsquot want to have per-prefix routing policies
ndash Small fraction of prefixes contribute most of the traffic
bull Focus on the small number of heavy hitters
ndash Define routing policies for selected prefixes
bull Routing choicesndash About 27000 unique ldquorouting choicesrdquo
bull Help in reducing the scale of the problem
ndash Small fraction of ldquorouting choicesrdquo contribute most traffic
bull Focus on the very small number of ldquorouting choicesrdquo
ndash Define routing policies on common attributes
61
Achieving Predictability
bull Route prediction with static analysisndash Helpful to know effects before deploymentndash Static analysis can help
TopologyBGP policy
configuration
eBGP routes
Offered traffic
BGP routingmodel
Flow of traffic through the network
62
Challenges to Predictabilitybull For transit ISPs effects on incoming traffic
ndash Lack of coordination strikes again
63
ldquoHot Potatordquo routing
Inter-AS Negotiation
bull Coordination aids predictabilityndash Negotiate where to sendndash Inbound and outboundndash Mutual benefits
bull How to implementndash What info to exchangendash Protecting privacyndash How to prioritize choicesndash How to prevent cheating
Destination 2
Destination 1
multiplepeeringpoints
Provider A
Provider B
64
Outbound Multihoming Goals
bull Redundancyndash Dynamic routing will failover to backup link
bull Performancendash Select provider with best performance per prefix
ndash Requires active probing
bull Costndash Select provider per prefix over time to minimize the
total financial cost
65
Inbound Traffic Control
bull More difficult no control over neighborsrsquo decisions
bull Three common techniques (previously discussed)ndash AS path prependingndash Communities and local preferencendash Prefix splitting
How does todayrsquos paper (MONET) control inbound traffic
66
How many links are enough
K upstream ISPs
Not much benefit beyond 4 ISPs
Akella et al ldquoPerformance Benefits of Multihomingrdquo SIGCOMM 2003
67
Problems with Multihoming in IPv4
bull Routing table growthndash Provider-based addressingndash Advertising prefix out multiple ISPs ndash canrsquot aggregate
bull Poor control over inbound trafficndash Existing mechanisms do not allow hosts to control
inbound traffic
68
GeorgiaTech
Internet Routing Overview
bull Intradomain (ie ldquointra-ASrdquo) routingbull Interdomain routing
Comcast
Abilene
ATampT Cogent
Autonomous Systems (ASes)
69
Configuration Problems ldquoAS 7007rdquoldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
UUNet
Florida InternetBarn
Sprint
70
Diagnosis and Troubleshootingldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
ldquoMicrosofts websites were offline for up to 23 hoursbecause of a [router] misconfigurationhellipit took nearly a day to determine what was wrong and undo the changesrdquo -- wiredcom January 25 2001
ldquoWorldCom Inchellipsuffered a widespread outage on its Internet backbone that affected roughly 20 percent of its US customer base The network problemshellipaffected millions of computer users worldwide A spokeswoman attributed the outage to a route table issue -- cnncom October 3 2002
A number of Covad customers went out from 5pm today due to supposedly a DDOS (distributed denial of service attack) on a key Level3 data center which later was described as a route leak (misconfiguration)ldquo
-- dslreportscom February 23 2004
71
Operator Mailing List (NANOG)Date Mon 18 Oct 2004 091515 -0700Subject Level 3 US east coast issues
Level 3 experiencing widespread unspecified routing issues on the US east coast Master ticket 1086844 Anyone have more specific information
Date Mon 18 Oct 2004 122034 -0400 (EDT)Subject Re Level 3 US east coast issues
Level 3 is currently experiencing a backbone outage causing routing instability and packet loss We are working to restore and will be sending hourly updateshellip
72
Operator Mailing List
0102030405060708090
Filtering RouteLeaks
RouteHijacks
RouteInstability
RoutingLoops
Blackholes
Occurr
ences o
ver
10 Y
ears
1995-1997 1998-2001 2001-2004
Note Only includes problems openly discussed on this list
Compare 83 power outages 1 fire
73
Routing Configuration
Ranking route selection
Dissemination internal route advertisement
Filtering route advertisement
Customer
Competitor
Primary
Backup
74
Internet Business Model (Simplified)
bull CustomerProvider One AS pays another for reachability to some set of destinations
bull ldquoSettlement-freerdquo Peering Bartering Two ASes exchange routes with one another
Provider
Peer
Customer
Preferences implemented with local preference manipulation
Destination
Pay to use
Get paid to use
Free to use
75
Peering Contracts Consistent Export
bull Rules of settlement-free peeringndash Advertise routes at all peering pointsndash Advertised routes must have equal ldquoAS path lengthrdquo
Sprint
ATampT
Enables ldquohot potatordquo routing
ldquoequally goodrdquoroutes
76
Consistent Export
bull Malicedeceptionbull iBGP signaling partitionbull Inconsistent export policy
neighbor 10123route-map PEER permit 10 set prepend 123
neighbor 10456route-map PEER permit 10 set prepend 123 123
Possible Causes
10123 456 1
10456 456 2
Neighbor AS Export
1 1 123
Export Clause Prepend
2 1 123 123
Two different Export Policies
77
Inconsistent Export in Practice
Feamster et al ldquoBorderGuard Detecting Cold Potatoes from Peersrdquo ACM IMC October 2004
78
Blackholes
Date Thu 18 Jul 2002 060510 -0400 (EDT)From Chad Oleary ltcolpoboxcomgtSubject Re problems with 701To ltnanogmeritedugt
Were starting to see the same issues with UUNet again Anyone elseseeing this Trying to reach Qwest
traceroute to 631461901 (631461901) 30 hops max 38 byte packets 1 esc-lp2-gwe-solutionscorpcom (631182201) 1167 ms 1163 ms 1142 ms 2 500Serial2-10GW1TPA2ALTERNET (1571301499) 1097 ms 1059 ms 1044 ms 3 161at-1-0-0XL4ATL1ALTERNET (1526381190) 13839 ms 14108 ms 16638 ms 4 0so-3-1-0XL2ATL5ALTERNET (152630238) 14370 ms 14587 ms 14553 ms 5 POS7-0BR2ATL5ALTERNET (1526382193) 13928 ms 14099 ms 14053 ms 6 7 hellip
79
Security
80
Security ldquoBogonrdquo Routes
Feamster et al ldquoAn Empirical Study of lsquoBogonrsquo Route Advertisementsrdquo ACM CCR January 2005
81
Spam Phishing etc
bull Unsolicited commercial emailbull As of about August 2008 estimates indicate that
about 95 of all email is spambull Common spam filtering techniques
ndash Content-based filtersndash DNS Blacklist (DNSBL) lookups Significant fraction of
todayrsquos DNS traffic
Can IP addresses from which spam is received be spoofed
82
Spam and Routing
83
Worms and Botnets
84
What is a Worm
bull Code that replicates and propagates across the networkndash Often carries a ldquopayloadrdquo
bull Usually spread via exploiting flaws in open servicesndash ldquoVirusesrdquo require user action to spread
bull First worm Robert Morris November 1988ndash 6-10 of all Internet hosts infected ()
bull Many more since but none on that scale until July 2001
85
Example Worm Code Red
bull Initial version July 13 2001
bull Exploited known ISAPI vulnerability in Microsoft IIS Web servers
bull 1st through 20th of each month spread20th through end of each month attack
bull Payload Web site defacementbull Scanning Random IP addressesbull Bug failure to seed random number generator
86
Code Red Revisions
bull Released July 19 2001
bull Payload flooding attack on wwwwhitehousegovndash Attack was mounted at the IP address of the Web site
bull Bug died after 20th of each month
bull Random number generator for IP scanning fixed
87
Code Red Host Infection Rate
Exponential infection rate
Measured using backscatter technique
88
Designing Fast-Spreading Worms
bull Hit-list scanningndash Time to infect first 10k hosts dominates infection timendash Solution Reconnaissance (stealthy scans etc)
bull Permutation scanningndash Observation Most scanning is redundantndash Idea Shared permutation of address space Start scanning
from own IP address Re-randomize when another infected machine is found
bull Internet-scale hit listsndash Flash worm complete infection within 30 seconds
89
Botnets
bull Bots Autonomous programs performing tasksbull Plenty of ldquobenignrdquo bots
ndash eg weatherbug
bull Botnets group of bots ndash Typically carries malicious connotationndash Large numbers of infected machinesndash Machines ldquoenlistedrdquo with infection vectors like worms
(last lecture)
bull Available for simultaneous control by a masterbull Size up to 350000 nodes (from todayrsquos paper)
90
ldquoRallyingrdquo the Botnet
bull Easy to combine worm backdoor functionalitybull Problem how to learn about successfully
infected machines
bull Optionsndash Emailndash Hard-coded email address
91
Botnet Control
bull Botnet master typically runs some IRC server on a well-known port (eg 6667)
bull Infected machine contacts botnet with pre-programmed DNS name (eg big-botde)
bull Dynamic DNS allows controller to move about freely
Infected Machine
DynamicDNS
BotnetController
(IRC server)
92
Some Defenses
93
Idea 1 Ingress Filtering
bull RFC 2827 Routers install filters to drop packets from networks that are not downstream
bull Feasible at edgesbull Difficult to configure closer to network ldquocorerdquo
20469207024 Internet
Drop all packets with source address other than 20469207024
94
Idea 2 uRPF Checks
bull Unicast Reverse Path Forwardingndash Cisco ldquoip verify unicast reverse-pathrdquo
bull Requires symmetric routing
Accept packet from interface only if forwarding table entry for source IP address matches ingress interface
100183
A10018
10015
101203
100181 24
10011 24
Strict Mode uRPF
Enabled
ldquoArdquo Routing TableDestination Next Hop1001024 Int 110018024 Int 2
100183 from wrong interface
95
Problems with uRPF
bull Asymmetric routing
96
S-BGP
bull Address-based PKI validate signaturesndash Authentication of
bull ownership for IP address blocks bull AS number bull an ASs identity and bull a BGP routers identity
ndash Use existing infrastructure (Internet registries etc)ndash Routing origination is digitally signedndash BGP updates are digitally signed
1048708 bull Route attestations A new optional BGP transitive path attribute
ndash carries digital signatures covering the routing information in updates
97
Practical Problems with S-BGP
bull Requires Public-Key Infrastructure
bull Lots of digital signatures to calculate and verifyndash Message overheadndash CPU overhead
bull Calculation expense is greatest when topology is changingndash Caching can help
bull Route aggregation is problematic (maybe thatrsquos OK)
bull Secure route withdrawals when link or node fails
bull Address ownership data out of date
bull Deployment
- Networking
- Goal of This Tutorial
- Todayrsquos Networks
- Business Models
- Billing for Internet Usage
- Net Neutrality
- Network Operations
- Point-of-Presence (PoP)
- Example Abilene Network Topology
- Another Example Backbone
- Internet Routing Overview
- Internet Routing Protocol BGP
- Two Flavors of BGP
- IPv4 Addresses Networks of Networks
- Pre-1994 Classful Addressing
- Classless Interdomain Routing (CIDR)
- Benefits of CIDR
- Growth of IP Prefixes
- 1994-1998 Linear Growth
- Around 2000 Fast Growth Resumes
- Fast growth resumes
- The Address Allocation Process
- Common Problems
- What can go wrong
- Measurement and Monitoring
- Passive vs Active Measurement
- Slide 27
- Passive Traffic Data Measurement
- Simple Network Management Protocol
- SNMP (Passive)
- Packet-level Monitoring
- Full Packet Capture (Passive)
- What is a flow
- Cisco NetFlow
- Flow Record Contents
- Aggregating Packets into Flows
- Reducing Measurement Overhead
- Packet Sampling for Flow Monitoring
- Sampling Flow-Level Sampling
- Two Main Approaches
- Packet Capture on High-Speed Links
- Characteristics of Packet Capture
- Data Measurement Repositories
- Multihoming and Traffic Engineering
- What is Multihoming
- Why Multihome
- Redundancy
- Performance
- Multihoming in IP Networks Today
- BGP or no
- Same Provider or Multiple
- Multihomed Stub One Link
- Multihomed Stub Multiple Links
- Multihomed Stub Multiple ISPs
- Multihomed Transit Network
- Interdomain Traffic Engineering
- Outbound Traffic Control
- Outbound Traffic Load Balancing
- Traffic Engineering Goals
- Managing Scale
- Achieving Predictability
- Challenges to Predictability
- Inter-AS Negotiation
- Outbound Multihoming Goals
- Inbound Traffic Control
- How many links are enough
- Problems with Multihoming in IPv4
- Slide 68
- Slide 69
- Diagnosis and Troubleshooting
- Operator Mailing List (NANOG)
- Operator Mailing List
- Routing Configuration
- Internet Business Model (Simplified)
- Peering Contracts Consistent Export
- Consistent Export
- Inconsistent Export in Practice
- Blackholes
- Security
- Security ldquoBogonrdquo Routes
- Spam Phishing etc
- Spam and Routing
- Worms and Botnets
- What is a Worm
- Example Worm Code Red
- Code Red Revisions
- Code Red Host Infection Rate
- Designing Fast-Spreading Worms
- Botnets
- ldquoRallyingrdquo the Botnet
- Botnet Control
- Some Defenses
- Idea 1 Ingress Filtering
- Idea 2 uRPF Checks
- Problems with uRPF
- S-BGP
- Practical Problems with S-BGP
-
27
Billing for Internet Usage
bull 95th Percentile billingndash Customer network pays for ldquocommitted information
raterdquo (CIR)ndash Throughput measured every 5 minutes (typically with
SNMP flow statistics also can be used for billing)ndash Customer billed based on 95th percentile
28
Passive Traffic Data Measurement
bull SNMP bytepacket counts everywhere
bull Packet monitoring selected locations
bull Flow monitoring typically at edges (if possible)ndash Direct computation of the traffic matrixndash Input to denial-of-service attack detection
bull Deep Packet Inspection also at edge where possible
29
Simple Network Management Protocol
bull Management Information Base (MIB)ndash Information storendash Unique variables named by
OIDsndash Accessed with SNMP
bull Specific MIBs for bytepacket counts (per link)
Manager Agent
SNMP
DB
ManagedObjects
30
SNMP (Passive)
bull Advantage ubiquitousndash Supported on all networking equipmentndash Multiple products for polling and analyzing data
bull Disadvantages ndash Coarse granularityndash Cannot express complex queries on the datandash Unreliable delivery of the data using UDP
bull Utilityndash Link utilization (billing)ndash Traffic matrix inference
31
Packet-level Monitoring
bull Passive monitoring to collect full packet contents (or at least headers)
bull Advantages lots of detailed informationndash Precise timing informationndash Information in packet headers
bull Disadvantages overheadndash Hard to keep up with high-speed linksndash Often requires a separate monitoring device
32
Full Packet Capture (Passive)
Example Georgia Tech OC3Mon
bull Rack-mounted PCbull Optical splitterbull Data Acquisition and
Generation (DAG) card
Source endacecom
33
What is a flow
bull Source IP addressbull Destination IP addressbull Source portbull Destination portbull Layer 3 protocol typebull TOS byte (DSCP)bull Input logical interface (ifIndex)
34
Cisco NetFlow
bull Basic output ldquoFlow recordrdquondash Most common version is v5
bull Current version (9) is being standardized in the IETF (template-based)ndash More flexible record formatndash Much easier to add new flow record types
Core Network
Collection and Aggregation
Collector (PC)Approximately 1500 bytes
20-50 flow recordsSent more frequently if traffic increases
35
Flow Record Contents
bull Source and Destination IP address and portbull Packet and byte countsbull Start and end timesbull ToS TCP flags
Basic information about the flowhellip
hellipplus information related to routing
bull Next-hop IP addressbull Source and destination ASbull Source and destination prefix
36
flow 1 flow 2 flow 3 flow 4
Aggregating Packets into Flows
bull Criteria 1 Set of packets that ldquobelong togetherrdquondash Sourcedestination IP addresses and port numbersndash Same protocol ToS bits hellip ndash Same inputoutput interfaces at a router (if known)
bull Criteria 2 Packets that are ldquocloserdquo together in timendash Maximum inter-packet spacing (eg 15 sec 30 sec)ndash Example flows 2 and 4 are different flows due to time
37
Reducing Measurement Overhead
bull Filtering on interfacendash destination prefix for a customerndash port number for an application (eg 80 for Web)
bull Sampling before insertion into flow cachendash Random deterministic or hash-based samplingndash 1-out-of-n or stratified based on packetflow sizendash Two types packet-level and flow-level
bull Aggregation after cache evictionndash packetsflows with same next-hop ASndash packetsflows destined to a particular service
38
Packet Sampling for Flow Monitoring
bull Packet sampling before flow creation (Sampled Netflow)ndash 1-out-of-m sampling of individual packets (eg m=100)ndash Create of flow records over the sampled packets
bull Reducing overheadndash Avoid per-packet overhead on (m-1)m packetsndash Avoid creating records for a large number of small flows
bull Increasing overhead (in some cases)ndash May split some long transfers into multiple flow records ndash hellip due to larger time gaps between successive packets
time
not sampled
two flowstimeout
39
Sampling Flow-Level Sampling
bull Sampling of flow records evicted from flow cachendash When evicting flows from table or when analyzing flows
bull Stratified sampling to put weight on ldquoheavyrdquo flowsndash Select all long flows and sample the short flows
bull Reduces the number of flow records ndash Still measures the vast majority of the traffic
Flow 1 40 bytesFlow 2 15580 bytesFlow 3 8196 bytesFlow 4 5350789 bytesFlow 5 532 bytesFlow 6 7432 bytes
sample with 100 probability
sample with 01 probability
sample with 10 probability
40
Two Main Approaches
bull Packet-level Monitoringndash Keep packet-level statisticsndash Examine (and potentially log) variety of packet-level
statistics Essentially anything in the packetndash Timing
bull Flow-level Monitoringndash Monitor packet-by-packet (though sometimes
sampled)ndash Keep aggregate statistics on a flow
41
Packet Capture on High-Speed Links
Example Georgia Tech ldquoOC3Monrdquo
bull Rack-mounted PCbull Optical splitterbull Data Acquisition and
Generation (DAG) card
Source endacecom
42
Characteristics of Packet Capture
bull Allows inspection on every packet on 10G links
bull Disadvantagesndash Costlyndash Requires splitting optical fibersndash Must be able to filterstore data
43
Data Measurement Repositories
bull AbileneInternet 2 Observatoryndash Configuration examplesndash SNMP datandash ISIS BGP routing data NetFlow traffic data
bull RouteViewsndash BGP updatesndash BGP table snapshots
44
Multihoming and Traffic Engineering
45
What is Multihoming
bull The use of redundant network links for the purposes of external connectivity
bull Can be achieved at many layers of the protocol stack and many places in the networkndash Multiple network interfaces in a PC
ndash An ISP with multiple upstream interfaces
bull Can refer to having multiple connections tondash The same ISP
ndash Multiple ISPs
46
Why Multihome
bull Redundancybull Availabilitybull Performancebull Cost
Interdomain traffic engineering the process by which a multihomed network configures its
network to achieve these goals
47
Redundancy
bull Maintain connectivity in the face ofndash Physical connectivity problems (fiber cut device
failures etc)ndash Failures in upstream ISP
48
Performance
bull Use multiple network links at once to achieve higher throughput than just over a single link
bull Allows incoming traffic to be load-balanced
70 of traffic30 of traffic
49
Multihoming in IP Networks Today
bull Stub AS no transit service for other ASesndash No need to use BGP
bull Multi-homed stub AS has connectivity to multiple immediate upstream ISPsndash Need BGPndash No need for a public AS numberndash No need for IP prefix allocation
bull Multi-homed transit AS connectivity to multiple ASes and transit servicendash Need BGP public AS number IP prefix allocation
50
BGP or no
bull Advantages of static routingndash Cheapersmaller routers (less true nowadays)ndash Simpler to configure
bull Advantages of BGPndash More control of your destiny (have providers stop
announcing you)ndash Fastermore intelligent selection of where to send
outbound packetsndash Better debugging of net problems (you can see the
Internet topology now)
51
Same Provider or Multiple
bull If your provider is reliable and fast and affordably and offers good tech-support you may want to multi-home initially to them via some backup path (slow is better than dead)
bull Eventually yoursquoll want to multi-home to different providers to avoid failure modes due to one providerrsquos architecture decisions
52
Multihomed Stub One Link
bull Downstream ISPrsquos routers configure default (ldquostaticrdquo) routes pointing to border router
bull Upstream ISP advertises reachability
Upstream ISP
Multiple links between same pair of routers
Default routes to ldquoborderrdquo
ldquoStubrdquoISP
53
Multihomed Stub Multiple Links
bull Use BGP to share loadbull Use private AS number (why is this OK)bull As before upstream ISP advertises prefix
Upstream ISP
Multiple links to different upstream routers
ldquoStubrdquoISP
Internal routing for ldquohot potatordquo
BGP for load balance at edge
54
Multihomed Stub Multiple ISPs
bull Many possibilitiesndash Load sharingndash Primary-backupndash Selective use of different ISPs
bull Requires BGP public AS number etc
ldquoStubrdquoISP
Upstream
ISP 1
Upstream
ISP 2
55
Multihomed Transit Network
bull BGP everywherebull Incoming and outcoming trafficbull Challenge balancing load on intradomain and egress
links given an offered traffic load
TransitISP
ISP 1
ISP 2
ISP 3
56
Interdomain Traffic Engineering
bull The process by which a network operator configures the network to achievendash Traffic load balancendash Redundancy (primarybackup) etc
bull Two tasksndash Outbound traffic controlndash Inbound traffic control
bull Key Problems Predictability and Scalability
57
Outbound Traffic Control
bull Easier to control than inbound trafficndash Destination-based routing sender determines where
the packets go
bull Control over next-hop AS onlyndash Cannot control selection of the entire path
Provider 1 Provider 2
Control with local preference
58
Outbound Traffic Load Balancingbull Control routes to provider per-prefix
ndash Assign local preference across destination prefixesndash Change the local preference assignments over time
bull Useful inputs to load balancingndash End-to-end path performance datandash Outbound traffic statistics per destination prefix
bull Challenge Getting from traffic volumes to groups of prefixes that should be assigned to each link
Premise of ldquointelligent route controlrdquo preoducts
59
Traffic Engineering Goals
bull Predictabilityndash Ensure the BGP decision process is deterministicndash Assume that BGP updates are (relatively) stable
bull Limit overhead introduced by routing changesndash Minimize frequency of changes to routing policiesndash Limit number of prefixes affected by changes
bull Limit impact on how traffic enters the networkndash Avoid new routes that might change neighborrsquos mindndash Select route with same attributes or at least path length
60
Managing Scalebull Destination prefixes
ndash More than 90000 destination prefixes
bull Donrsquot want to have per-prefix routing policies
ndash Small fraction of prefixes contribute most of the traffic
bull Focus on the small number of heavy hitters
ndash Define routing policies for selected prefixes
bull Routing choicesndash About 27000 unique ldquorouting choicesrdquo
bull Help in reducing the scale of the problem
ndash Small fraction of ldquorouting choicesrdquo contribute most traffic
bull Focus on the very small number of ldquorouting choicesrdquo
ndash Define routing policies on common attributes
61
Achieving Predictability
bull Route prediction with static analysisndash Helpful to know effects before deploymentndash Static analysis can help
TopologyBGP policy
configuration
eBGP routes
Offered traffic
BGP routingmodel
Flow of traffic through the network
62
Challenges to Predictabilitybull For transit ISPs effects on incoming traffic
ndash Lack of coordination strikes again
63
ldquoHot Potatordquo routing
Inter-AS Negotiation
bull Coordination aids predictabilityndash Negotiate where to sendndash Inbound and outboundndash Mutual benefits
bull How to implementndash What info to exchangendash Protecting privacyndash How to prioritize choicesndash How to prevent cheating
Destination 2
Destination 1
multiplepeeringpoints
Provider A
Provider B
64
Outbound Multihoming Goals
bull Redundancyndash Dynamic routing will failover to backup link
bull Performancendash Select provider with best performance per prefix
ndash Requires active probing
bull Costndash Select provider per prefix over time to minimize the
total financial cost
65
Inbound Traffic Control
bull More difficult no control over neighborsrsquo decisions
bull Three common techniques (previously discussed)ndash AS path prependingndash Communities and local preferencendash Prefix splitting
How does todayrsquos paper (MONET) control inbound traffic
66
How many links are enough
K upstream ISPs
Not much benefit beyond 4 ISPs
Akella et al ldquoPerformance Benefits of Multihomingrdquo SIGCOMM 2003
67
Problems with Multihoming in IPv4
bull Routing table growthndash Provider-based addressingndash Advertising prefix out multiple ISPs ndash canrsquot aggregate
bull Poor control over inbound trafficndash Existing mechanisms do not allow hosts to control
inbound traffic
68
GeorgiaTech
Internet Routing Overview
bull Intradomain (ie ldquointra-ASrdquo) routingbull Interdomain routing
Comcast
Abilene
ATampT Cogent
Autonomous Systems (ASes)
69
Configuration Problems ldquoAS 7007rdquoldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
UUNet
Florida InternetBarn
Sprint
70
Diagnosis and Troubleshootingldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
ldquoMicrosofts websites were offline for up to 23 hoursbecause of a [router] misconfigurationhellipit took nearly a day to determine what was wrong and undo the changesrdquo -- wiredcom January 25 2001
ldquoWorldCom Inchellipsuffered a widespread outage on its Internet backbone that affected roughly 20 percent of its US customer base The network problemshellipaffected millions of computer users worldwide A spokeswoman attributed the outage to a route table issue -- cnncom October 3 2002
A number of Covad customers went out from 5pm today due to supposedly a DDOS (distributed denial of service attack) on a key Level3 data center which later was described as a route leak (misconfiguration)ldquo
-- dslreportscom February 23 2004
71
Operator Mailing List (NANOG)Date Mon 18 Oct 2004 091515 -0700Subject Level 3 US east coast issues
Level 3 experiencing widespread unspecified routing issues on the US east coast Master ticket 1086844 Anyone have more specific information
Date Mon 18 Oct 2004 122034 -0400 (EDT)Subject Re Level 3 US east coast issues
Level 3 is currently experiencing a backbone outage causing routing instability and packet loss We are working to restore and will be sending hourly updateshellip
72
Operator Mailing List
0102030405060708090
Filtering RouteLeaks
RouteHijacks
RouteInstability
RoutingLoops
Blackholes
Occurr
ences o
ver
10 Y
ears
1995-1997 1998-2001 2001-2004
Note Only includes problems openly discussed on this list
Compare 83 power outages 1 fire
73
Routing Configuration
Ranking route selection
Dissemination internal route advertisement
Filtering route advertisement
Customer
Competitor
Primary
Backup
74
Internet Business Model (Simplified)
bull CustomerProvider One AS pays another for reachability to some set of destinations
bull ldquoSettlement-freerdquo Peering Bartering Two ASes exchange routes with one another
Provider
Peer
Customer
Preferences implemented with local preference manipulation
Destination
Pay to use
Get paid to use
Free to use
75
Peering Contracts Consistent Export
bull Rules of settlement-free peeringndash Advertise routes at all peering pointsndash Advertised routes must have equal ldquoAS path lengthrdquo
Sprint
ATampT
Enables ldquohot potatordquo routing
ldquoequally goodrdquoroutes
76
Consistent Export
bull Malicedeceptionbull iBGP signaling partitionbull Inconsistent export policy
neighbor 10123route-map PEER permit 10 set prepend 123
neighbor 10456route-map PEER permit 10 set prepend 123 123
Possible Causes
10123 456 1
10456 456 2
Neighbor AS Export
1 1 123
Export Clause Prepend
2 1 123 123
Two different Export Policies
77
Inconsistent Export in Practice
Feamster et al ldquoBorderGuard Detecting Cold Potatoes from Peersrdquo ACM IMC October 2004
78
Blackholes
Date Thu 18 Jul 2002 060510 -0400 (EDT)From Chad Oleary ltcolpoboxcomgtSubject Re problems with 701To ltnanogmeritedugt
Were starting to see the same issues with UUNet again Anyone elseseeing this Trying to reach Qwest
traceroute to 631461901 (631461901) 30 hops max 38 byte packets 1 esc-lp2-gwe-solutionscorpcom (631182201) 1167 ms 1163 ms 1142 ms 2 500Serial2-10GW1TPA2ALTERNET (1571301499) 1097 ms 1059 ms 1044 ms 3 161at-1-0-0XL4ATL1ALTERNET (1526381190) 13839 ms 14108 ms 16638 ms 4 0so-3-1-0XL2ATL5ALTERNET (152630238) 14370 ms 14587 ms 14553 ms 5 POS7-0BR2ATL5ALTERNET (1526382193) 13928 ms 14099 ms 14053 ms 6 7 hellip
79
Security
80
Security ldquoBogonrdquo Routes
Feamster et al ldquoAn Empirical Study of lsquoBogonrsquo Route Advertisementsrdquo ACM CCR January 2005
81
Spam Phishing etc
bull Unsolicited commercial emailbull As of about August 2008 estimates indicate that
about 95 of all email is spambull Common spam filtering techniques
ndash Content-based filtersndash DNS Blacklist (DNSBL) lookups Significant fraction of
todayrsquos DNS traffic
Can IP addresses from which spam is received be spoofed
82
Spam and Routing
83
Worms and Botnets
84
What is a Worm
bull Code that replicates and propagates across the networkndash Often carries a ldquopayloadrdquo
bull Usually spread via exploiting flaws in open servicesndash ldquoVirusesrdquo require user action to spread
bull First worm Robert Morris November 1988ndash 6-10 of all Internet hosts infected ()
bull Many more since but none on that scale until July 2001
85
Example Worm Code Red
bull Initial version July 13 2001
bull Exploited known ISAPI vulnerability in Microsoft IIS Web servers
bull 1st through 20th of each month spread20th through end of each month attack
bull Payload Web site defacementbull Scanning Random IP addressesbull Bug failure to seed random number generator
86
Code Red Revisions
bull Released July 19 2001
bull Payload flooding attack on wwwwhitehousegovndash Attack was mounted at the IP address of the Web site
bull Bug died after 20th of each month
bull Random number generator for IP scanning fixed
87
Code Red Host Infection Rate
Exponential infection rate
Measured using backscatter technique
88
Designing Fast-Spreading Worms
bull Hit-list scanningndash Time to infect first 10k hosts dominates infection timendash Solution Reconnaissance (stealthy scans etc)
bull Permutation scanningndash Observation Most scanning is redundantndash Idea Shared permutation of address space Start scanning
from own IP address Re-randomize when another infected machine is found
bull Internet-scale hit listsndash Flash worm complete infection within 30 seconds
89
Botnets
bull Bots Autonomous programs performing tasksbull Plenty of ldquobenignrdquo bots
ndash eg weatherbug
bull Botnets group of bots ndash Typically carries malicious connotationndash Large numbers of infected machinesndash Machines ldquoenlistedrdquo with infection vectors like worms
(last lecture)
bull Available for simultaneous control by a masterbull Size up to 350000 nodes (from todayrsquos paper)
90
ldquoRallyingrdquo the Botnet
bull Easy to combine worm backdoor functionalitybull Problem how to learn about successfully
infected machines
bull Optionsndash Emailndash Hard-coded email address
91
Botnet Control
bull Botnet master typically runs some IRC server on a well-known port (eg 6667)
bull Infected machine contacts botnet with pre-programmed DNS name (eg big-botde)
bull Dynamic DNS allows controller to move about freely
Infected Machine
DynamicDNS
BotnetController
(IRC server)
92
Some Defenses
93
Idea 1 Ingress Filtering
bull RFC 2827 Routers install filters to drop packets from networks that are not downstream
bull Feasible at edgesbull Difficult to configure closer to network ldquocorerdquo
20469207024 Internet
Drop all packets with source address other than 20469207024
94
Idea 2 uRPF Checks
bull Unicast Reverse Path Forwardingndash Cisco ldquoip verify unicast reverse-pathrdquo
bull Requires symmetric routing
Accept packet from interface only if forwarding table entry for source IP address matches ingress interface
100183
A10018
10015
101203
100181 24
10011 24
Strict Mode uRPF
Enabled
ldquoArdquo Routing TableDestination Next Hop1001024 Int 110018024 Int 2
100183 from wrong interface
95
Problems with uRPF
bull Asymmetric routing
96
S-BGP
bull Address-based PKI validate signaturesndash Authentication of
bull ownership for IP address blocks bull AS number bull an ASs identity and bull a BGP routers identity
ndash Use existing infrastructure (Internet registries etc)ndash Routing origination is digitally signedndash BGP updates are digitally signed
1048708 bull Route attestations A new optional BGP transitive path attribute
ndash carries digital signatures covering the routing information in updates
97
Practical Problems with S-BGP
bull Requires Public-Key Infrastructure
bull Lots of digital signatures to calculate and verifyndash Message overheadndash CPU overhead
bull Calculation expense is greatest when topology is changingndash Caching can help
bull Route aggregation is problematic (maybe thatrsquos OK)
bull Secure route withdrawals when link or node fails
bull Address ownership data out of date
bull Deployment
- Networking
- Goal of This Tutorial
- Todayrsquos Networks
- Business Models
- Billing for Internet Usage
- Net Neutrality
- Network Operations
- Point-of-Presence (PoP)
- Example Abilene Network Topology
- Another Example Backbone
- Internet Routing Overview
- Internet Routing Protocol BGP
- Two Flavors of BGP
- IPv4 Addresses Networks of Networks
- Pre-1994 Classful Addressing
- Classless Interdomain Routing (CIDR)
- Benefits of CIDR
- Growth of IP Prefixes
- 1994-1998 Linear Growth
- Around 2000 Fast Growth Resumes
- Fast growth resumes
- The Address Allocation Process
- Common Problems
- What can go wrong
- Measurement and Monitoring
- Passive vs Active Measurement
- Slide 27
- Passive Traffic Data Measurement
- Simple Network Management Protocol
- SNMP (Passive)
- Packet-level Monitoring
- Full Packet Capture (Passive)
- What is a flow
- Cisco NetFlow
- Flow Record Contents
- Aggregating Packets into Flows
- Reducing Measurement Overhead
- Packet Sampling for Flow Monitoring
- Sampling Flow-Level Sampling
- Two Main Approaches
- Packet Capture on High-Speed Links
- Characteristics of Packet Capture
- Data Measurement Repositories
- Multihoming and Traffic Engineering
- What is Multihoming
- Why Multihome
- Redundancy
- Performance
- Multihoming in IP Networks Today
- BGP or no
- Same Provider or Multiple
- Multihomed Stub One Link
- Multihomed Stub Multiple Links
- Multihomed Stub Multiple ISPs
- Multihomed Transit Network
- Interdomain Traffic Engineering
- Outbound Traffic Control
- Outbound Traffic Load Balancing
- Traffic Engineering Goals
- Managing Scale
- Achieving Predictability
- Challenges to Predictability
- Inter-AS Negotiation
- Outbound Multihoming Goals
- Inbound Traffic Control
- How many links are enough
- Problems with Multihoming in IPv4
- Slide 68
- Slide 69
- Diagnosis and Troubleshooting
- Operator Mailing List (NANOG)
- Operator Mailing List
- Routing Configuration
- Internet Business Model (Simplified)
- Peering Contracts Consistent Export
- Consistent Export
- Inconsistent Export in Practice
- Blackholes
- Security
- Security ldquoBogonrdquo Routes
- Spam Phishing etc
- Spam and Routing
- Worms and Botnets
- What is a Worm
- Example Worm Code Red
- Code Red Revisions
- Code Red Host Infection Rate
- Designing Fast-Spreading Worms
- Botnets
- ldquoRallyingrdquo the Botnet
- Botnet Control
- Some Defenses
- Idea 1 Ingress Filtering
- Idea 2 uRPF Checks
- Problems with uRPF
- S-BGP
- Practical Problems with S-BGP
-
28
Passive Traffic Data Measurement
bull SNMP bytepacket counts everywhere
bull Packet monitoring selected locations
bull Flow monitoring typically at edges (if possible)ndash Direct computation of the traffic matrixndash Input to denial-of-service attack detection
bull Deep Packet Inspection also at edge where possible
29
Simple Network Management Protocol
bull Management Information Base (MIB)ndash Information storendash Unique variables named by
OIDsndash Accessed with SNMP
bull Specific MIBs for bytepacket counts (per link)
Manager Agent
SNMP
DB
ManagedObjects
30
SNMP (Passive)
bull Advantage ubiquitousndash Supported on all networking equipmentndash Multiple products for polling and analyzing data
bull Disadvantages ndash Coarse granularityndash Cannot express complex queries on the datandash Unreliable delivery of the data using UDP
bull Utilityndash Link utilization (billing)ndash Traffic matrix inference
31
Packet-level Monitoring
bull Passive monitoring to collect full packet contents (or at least headers)
bull Advantages lots of detailed informationndash Precise timing informationndash Information in packet headers
bull Disadvantages overheadndash Hard to keep up with high-speed linksndash Often requires a separate monitoring device
32
Full Packet Capture (Passive)
Example Georgia Tech OC3Mon
bull Rack-mounted PCbull Optical splitterbull Data Acquisition and
Generation (DAG) card
Source endacecom
33
What is a flow
bull Source IP addressbull Destination IP addressbull Source portbull Destination portbull Layer 3 protocol typebull TOS byte (DSCP)bull Input logical interface (ifIndex)
34
Cisco NetFlow
bull Basic output ldquoFlow recordrdquondash Most common version is v5
bull Current version (9) is being standardized in the IETF (template-based)ndash More flexible record formatndash Much easier to add new flow record types
Core Network
Collection and Aggregation
Collector (PC)Approximately 1500 bytes
20-50 flow recordsSent more frequently if traffic increases
35
Flow Record Contents
bull Source and Destination IP address and portbull Packet and byte countsbull Start and end timesbull ToS TCP flags
Basic information about the flowhellip
hellipplus information related to routing
bull Next-hop IP addressbull Source and destination ASbull Source and destination prefix
36
flow 1 flow 2 flow 3 flow 4
Aggregating Packets into Flows
bull Criteria 1 Set of packets that ldquobelong togetherrdquondash Sourcedestination IP addresses and port numbersndash Same protocol ToS bits hellip ndash Same inputoutput interfaces at a router (if known)
bull Criteria 2 Packets that are ldquocloserdquo together in timendash Maximum inter-packet spacing (eg 15 sec 30 sec)ndash Example flows 2 and 4 are different flows due to time
37
Reducing Measurement Overhead
bull Filtering on interfacendash destination prefix for a customerndash port number for an application (eg 80 for Web)
bull Sampling before insertion into flow cachendash Random deterministic or hash-based samplingndash 1-out-of-n or stratified based on packetflow sizendash Two types packet-level and flow-level
bull Aggregation after cache evictionndash packetsflows with same next-hop ASndash packetsflows destined to a particular service
38
Packet Sampling for Flow Monitoring
bull Packet sampling before flow creation (Sampled Netflow)ndash 1-out-of-m sampling of individual packets (eg m=100)ndash Create of flow records over the sampled packets
bull Reducing overheadndash Avoid per-packet overhead on (m-1)m packetsndash Avoid creating records for a large number of small flows
bull Increasing overhead (in some cases)ndash May split some long transfers into multiple flow records ndash hellip due to larger time gaps between successive packets
time
not sampled
two flowstimeout
39
Sampling Flow-Level Sampling
bull Sampling of flow records evicted from flow cachendash When evicting flows from table or when analyzing flows
bull Stratified sampling to put weight on ldquoheavyrdquo flowsndash Select all long flows and sample the short flows
bull Reduces the number of flow records ndash Still measures the vast majority of the traffic
Flow 1 40 bytesFlow 2 15580 bytesFlow 3 8196 bytesFlow 4 5350789 bytesFlow 5 532 bytesFlow 6 7432 bytes
sample with 100 probability
sample with 01 probability
sample with 10 probability
40
Two Main Approaches
bull Packet-level Monitoringndash Keep packet-level statisticsndash Examine (and potentially log) variety of packet-level
statistics Essentially anything in the packetndash Timing
bull Flow-level Monitoringndash Monitor packet-by-packet (though sometimes
sampled)ndash Keep aggregate statistics on a flow
41
Packet Capture on High-Speed Links
Example Georgia Tech ldquoOC3Monrdquo
bull Rack-mounted PCbull Optical splitterbull Data Acquisition and
Generation (DAG) card
Source endacecom
42
Characteristics of Packet Capture
bull Allows inspection on every packet on 10G links
bull Disadvantagesndash Costlyndash Requires splitting optical fibersndash Must be able to filterstore data
43
Data Measurement Repositories
bull AbileneInternet 2 Observatoryndash Configuration examplesndash SNMP datandash ISIS BGP routing data NetFlow traffic data
bull RouteViewsndash BGP updatesndash BGP table snapshots
44
Multihoming and Traffic Engineering
45
What is Multihoming
bull The use of redundant network links for the purposes of external connectivity
bull Can be achieved at many layers of the protocol stack and many places in the networkndash Multiple network interfaces in a PC
ndash An ISP with multiple upstream interfaces
bull Can refer to having multiple connections tondash The same ISP
ndash Multiple ISPs
46
Why Multihome
bull Redundancybull Availabilitybull Performancebull Cost
Interdomain traffic engineering the process by which a multihomed network configures its
network to achieve these goals
47
Redundancy
bull Maintain connectivity in the face ofndash Physical connectivity problems (fiber cut device
failures etc)ndash Failures in upstream ISP
48
Performance
bull Use multiple network links at once to achieve higher throughput than just over a single link
bull Allows incoming traffic to be load-balanced
70 of traffic30 of traffic
49
Multihoming in IP Networks Today
bull Stub AS no transit service for other ASesndash No need to use BGP
bull Multi-homed stub AS has connectivity to multiple immediate upstream ISPsndash Need BGPndash No need for a public AS numberndash No need for IP prefix allocation
bull Multi-homed transit AS connectivity to multiple ASes and transit servicendash Need BGP public AS number IP prefix allocation
50
BGP or no
bull Advantages of static routingndash Cheapersmaller routers (less true nowadays)ndash Simpler to configure
bull Advantages of BGPndash More control of your destiny (have providers stop
announcing you)ndash Fastermore intelligent selection of where to send
outbound packetsndash Better debugging of net problems (you can see the
Internet topology now)
51
Same Provider or Multiple
bull If your provider is reliable and fast and affordably and offers good tech-support you may want to multi-home initially to them via some backup path (slow is better than dead)
bull Eventually yoursquoll want to multi-home to different providers to avoid failure modes due to one providerrsquos architecture decisions
52
Multihomed Stub One Link
bull Downstream ISPrsquos routers configure default (ldquostaticrdquo) routes pointing to border router
bull Upstream ISP advertises reachability
Upstream ISP
Multiple links between same pair of routers
Default routes to ldquoborderrdquo
ldquoStubrdquoISP
53
Multihomed Stub Multiple Links
bull Use BGP to share loadbull Use private AS number (why is this OK)bull As before upstream ISP advertises prefix
Upstream ISP
Multiple links to different upstream routers
ldquoStubrdquoISP
Internal routing for ldquohot potatordquo
BGP for load balance at edge
54
Multihomed Stub Multiple ISPs
bull Many possibilitiesndash Load sharingndash Primary-backupndash Selective use of different ISPs
bull Requires BGP public AS number etc
ldquoStubrdquoISP
Upstream
ISP 1
Upstream
ISP 2
55
Multihomed Transit Network
bull BGP everywherebull Incoming and outcoming trafficbull Challenge balancing load on intradomain and egress
links given an offered traffic load
TransitISP
ISP 1
ISP 2
ISP 3
56
Interdomain Traffic Engineering
bull The process by which a network operator configures the network to achievendash Traffic load balancendash Redundancy (primarybackup) etc
bull Two tasksndash Outbound traffic controlndash Inbound traffic control
bull Key Problems Predictability and Scalability
57
Outbound Traffic Control
bull Easier to control than inbound trafficndash Destination-based routing sender determines where
the packets go
bull Control over next-hop AS onlyndash Cannot control selection of the entire path
Provider 1 Provider 2
Control with local preference
58
Outbound Traffic Load Balancingbull Control routes to provider per-prefix
ndash Assign local preference across destination prefixesndash Change the local preference assignments over time
bull Useful inputs to load balancingndash End-to-end path performance datandash Outbound traffic statistics per destination prefix
bull Challenge Getting from traffic volumes to groups of prefixes that should be assigned to each link
Premise of ldquointelligent route controlrdquo preoducts
59
Traffic Engineering Goals
bull Predictabilityndash Ensure the BGP decision process is deterministicndash Assume that BGP updates are (relatively) stable
bull Limit overhead introduced by routing changesndash Minimize frequency of changes to routing policiesndash Limit number of prefixes affected by changes
bull Limit impact on how traffic enters the networkndash Avoid new routes that might change neighborrsquos mindndash Select route with same attributes or at least path length
60
Managing Scalebull Destination prefixes
ndash More than 90000 destination prefixes
bull Donrsquot want to have per-prefix routing policies
ndash Small fraction of prefixes contribute most of the traffic
bull Focus on the small number of heavy hitters
ndash Define routing policies for selected prefixes
bull Routing choicesndash About 27000 unique ldquorouting choicesrdquo
bull Help in reducing the scale of the problem
ndash Small fraction of ldquorouting choicesrdquo contribute most traffic
bull Focus on the very small number of ldquorouting choicesrdquo
ndash Define routing policies on common attributes
61
Achieving Predictability
bull Route prediction with static analysisndash Helpful to know effects before deploymentndash Static analysis can help
TopologyBGP policy
configuration
eBGP routes
Offered traffic
BGP routingmodel
Flow of traffic through the network
62
Challenges to Predictabilitybull For transit ISPs effects on incoming traffic
ndash Lack of coordination strikes again
63
ldquoHot Potatordquo routing
Inter-AS Negotiation
bull Coordination aids predictabilityndash Negotiate where to sendndash Inbound and outboundndash Mutual benefits
bull How to implementndash What info to exchangendash Protecting privacyndash How to prioritize choicesndash How to prevent cheating
Destination 2
Destination 1
multiplepeeringpoints
Provider A
Provider B
64
Outbound Multihoming Goals
bull Redundancyndash Dynamic routing will failover to backup link
bull Performancendash Select provider with best performance per prefix
ndash Requires active probing
bull Costndash Select provider per prefix over time to minimize the
total financial cost
65
Inbound Traffic Control
bull More difficult no control over neighborsrsquo decisions
bull Three common techniques (previously discussed)ndash AS path prependingndash Communities and local preferencendash Prefix splitting
How does todayrsquos paper (MONET) control inbound traffic
66
How many links are enough
K upstream ISPs
Not much benefit beyond 4 ISPs
Akella et al ldquoPerformance Benefits of Multihomingrdquo SIGCOMM 2003
67
Problems with Multihoming in IPv4
bull Routing table growthndash Provider-based addressingndash Advertising prefix out multiple ISPs ndash canrsquot aggregate
bull Poor control over inbound trafficndash Existing mechanisms do not allow hosts to control
inbound traffic
68
GeorgiaTech
Internet Routing Overview
bull Intradomain (ie ldquointra-ASrdquo) routingbull Interdomain routing
Comcast
Abilene
ATampT Cogent
Autonomous Systems (ASes)
69
Configuration Problems ldquoAS 7007rdquoldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
UUNet
Florida InternetBarn
Sprint
70
Diagnosis and Troubleshootingldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
ldquoMicrosofts websites were offline for up to 23 hoursbecause of a [router] misconfigurationhellipit took nearly a day to determine what was wrong and undo the changesrdquo -- wiredcom January 25 2001
ldquoWorldCom Inchellipsuffered a widespread outage on its Internet backbone that affected roughly 20 percent of its US customer base The network problemshellipaffected millions of computer users worldwide A spokeswoman attributed the outage to a route table issue -- cnncom October 3 2002
A number of Covad customers went out from 5pm today due to supposedly a DDOS (distributed denial of service attack) on a key Level3 data center which later was described as a route leak (misconfiguration)ldquo
-- dslreportscom February 23 2004
71
Operator Mailing List (NANOG)Date Mon 18 Oct 2004 091515 -0700Subject Level 3 US east coast issues
Level 3 experiencing widespread unspecified routing issues on the US east coast Master ticket 1086844 Anyone have more specific information
Date Mon 18 Oct 2004 122034 -0400 (EDT)Subject Re Level 3 US east coast issues
Level 3 is currently experiencing a backbone outage causing routing instability and packet loss We are working to restore and will be sending hourly updateshellip
72
Operator Mailing List
0102030405060708090
Filtering RouteLeaks
RouteHijacks
RouteInstability
RoutingLoops
Blackholes
Occurr
ences o
ver
10 Y
ears
1995-1997 1998-2001 2001-2004
Note Only includes problems openly discussed on this list
Compare 83 power outages 1 fire
73
Routing Configuration
Ranking route selection
Dissemination internal route advertisement
Filtering route advertisement
Customer
Competitor
Primary
Backup
74
Internet Business Model (Simplified)
bull CustomerProvider One AS pays another for reachability to some set of destinations
bull ldquoSettlement-freerdquo Peering Bartering Two ASes exchange routes with one another
Provider
Peer
Customer
Preferences implemented with local preference manipulation
Destination
Pay to use
Get paid to use
Free to use
75
Peering Contracts Consistent Export
bull Rules of settlement-free peeringndash Advertise routes at all peering pointsndash Advertised routes must have equal ldquoAS path lengthrdquo
Sprint
ATampT
Enables ldquohot potatordquo routing
ldquoequally goodrdquoroutes
76
Consistent Export
bull Malicedeceptionbull iBGP signaling partitionbull Inconsistent export policy
neighbor 10123route-map PEER permit 10 set prepend 123
neighbor 10456route-map PEER permit 10 set prepend 123 123
Possible Causes
10123 456 1
10456 456 2
Neighbor AS Export
1 1 123
Export Clause Prepend
2 1 123 123
Two different Export Policies
77
Inconsistent Export in Practice
Feamster et al ldquoBorderGuard Detecting Cold Potatoes from Peersrdquo ACM IMC October 2004
78
Blackholes
Date Thu 18 Jul 2002 060510 -0400 (EDT)From Chad Oleary ltcolpoboxcomgtSubject Re problems with 701To ltnanogmeritedugt
Were starting to see the same issues with UUNet again Anyone elseseeing this Trying to reach Qwest
traceroute to 631461901 (631461901) 30 hops max 38 byte packets 1 esc-lp2-gwe-solutionscorpcom (631182201) 1167 ms 1163 ms 1142 ms 2 500Serial2-10GW1TPA2ALTERNET (1571301499) 1097 ms 1059 ms 1044 ms 3 161at-1-0-0XL4ATL1ALTERNET (1526381190) 13839 ms 14108 ms 16638 ms 4 0so-3-1-0XL2ATL5ALTERNET (152630238) 14370 ms 14587 ms 14553 ms 5 POS7-0BR2ATL5ALTERNET (1526382193) 13928 ms 14099 ms 14053 ms 6 7 hellip
79
Security
80
Security ldquoBogonrdquo Routes
Feamster et al ldquoAn Empirical Study of lsquoBogonrsquo Route Advertisementsrdquo ACM CCR January 2005
81
Spam Phishing etc
bull Unsolicited commercial emailbull As of about August 2008 estimates indicate that
about 95 of all email is spambull Common spam filtering techniques
ndash Content-based filtersndash DNS Blacklist (DNSBL) lookups Significant fraction of
todayrsquos DNS traffic
Can IP addresses from which spam is received be spoofed
82
Spam and Routing
83
Worms and Botnets
84
What is a Worm
bull Code that replicates and propagates across the networkndash Often carries a ldquopayloadrdquo
bull Usually spread via exploiting flaws in open servicesndash ldquoVirusesrdquo require user action to spread
bull First worm Robert Morris November 1988ndash 6-10 of all Internet hosts infected ()
bull Many more since but none on that scale until July 2001
85
Example Worm Code Red
bull Initial version July 13 2001
bull Exploited known ISAPI vulnerability in Microsoft IIS Web servers
bull 1st through 20th of each month spread20th through end of each month attack
bull Payload Web site defacementbull Scanning Random IP addressesbull Bug failure to seed random number generator
86
Code Red Revisions
bull Released July 19 2001
bull Payload flooding attack on wwwwhitehousegovndash Attack was mounted at the IP address of the Web site
bull Bug died after 20th of each month
bull Random number generator for IP scanning fixed
87
Code Red Host Infection Rate
Exponential infection rate
Measured using backscatter technique
88
Designing Fast-Spreading Worms
bull Hit-list scanningndash Time to infect first 10k hosts dominates infection timendash Solution Reconnaissance (stealthy scans etc)
bull Permutation scanningndash Observation Most scanning is redundantndash Idea Shared permutation of address space Start scanning
from own IP address Re-randomize when another infected machine is found
bull Internet-scale hit listsndash Flash worm complete infection within 30 seconds
89
Botnets
bull Bots Autonomous programs performing tasksbull Plenty of ldquobenignrdquo bots
ndash eg weatherbug
bull Botnets group of bots ndash Typically carries malicious connotationndash Large numbers of infected machinesndash Machines ldquoenlistedrdquo with infection vectors like worms
(last lecture)
bull Available for simultaneous control by a masterbull Size up to 350000 nodes (from todayrsquos paper)
90
ldquoRallyingrdquo the Botnet
bull Easy to combine worm backdoor functionalitybull Problem how to learn about successfully
infected machines
bull Optionsndash Emailndash Hard-coded email address
91
Botnet Control
bull Botnet master typically runs some IRC server on a well-known port (eg 6667)
bull Infected machine contacts botnet with pre-programmed DNS name (eg big-botde)
bull Dynamic DNS allows controller to move about freely
Infected Machine
DynamicDNS
BotnetController
(IRC server)
92
Some Defenses
93
Idea 1 Ingress Filtering
bull RFC 2827 Routers install filters to drop packets from networks that are not downstream
bull Feasible at edgesbull Difficult to configure closer to network ldquocorerdquo
20469207024 Internet
Drop all packets with source address other than 20469207024
94
Idea 2 uRPF Checks
bull Unicast Reverse Path Forwardingndash Cisco ldquoip verify unicast reverse-pathrdquo
bull Requires symmetric routing
Accept packet from interface only if forwarding table entry for source IP address matches ingress interface
100183
A10018
10015
101203
100181 24
10011 24
Strict Mode uRPF
Enabled
ldquoArdquo Routing TableDestination Next Hop1001024 Int 110018024 Int 2
100183 from wrong interface
95
Problems with uRPF
bull Asymmetric routing
96
S-BGP
bull Address-based PKI validate signaturesndash Authentication of
bull ownership for IP address blocks bull AS number bull an ASs identity and bull a BGP routers identity
ndash Use existing infrastructure (Internet registries etc)ndash Routing origination is digitally signedndash BGP updates are digitally signed
1048708 bull Route attestations A new optional BGP transitive path attribute
ndash carries digital signatures covering the routing information in updates
97
Practical Problems with S-BGP
bull Requires Public-Key Infrastructure
bull Lots of digital signatures to calculate and verifyndash Message overheadndash CPU overhead
bull Calculation expense is greatest when topology is changingndash Caching can help
bull Route aggregation is problematic (maybe thatrsquos OK)
bull Secure route withdrawals when link or node fails
bull Address ownership data out of date
bull Deployment
- Networking
- Goal of This Tutorial
- Todayrsquos Networks
- Business Models
- Billing for Internet Usage
- Net Neutrality
- Network Operations
- Point-of-Presence (PoP)
- Example Abilene Network Topology
- Another Example Backbone
- Internet Routing Overview
- Internet Routing Protocol BGP
- Two Flavors of BGP
- IPv4 Addresses Networks of Networks
- Pre-1994 Classful Addressing
- Classless Interdomain Routing (CIDR)
- Benefits of CIDR
- Growth of IP Prefixes
- 1994-1998 Linear Growth
- Around 2000 Fast Growth Resumes
- Fast growth resumes
- The Address Allocation Process
- Common Problems
- What can go wrong
- Measurement and Monitoring
- Passive vs Active Measurement
- Slide 27
- Passive Traffic Data Measurement
- Simple Network Management Protocol
- SNMP (Passive)
- Packet-level Monitoring
- Full Packet Capture (Passive)
- What is a flow
- Cisco NetFlow
- Flow Record Contents
- Aggregating Packets into Flows
- Reducing Measurement Overhead
- Packet Sampling for Flow Monitoring
- Sampling Flow-Level Sampling
- Two Main Approaches
- Packet Capture on High-Speed Links
- Characteristics of Packet Capture
- Data Measurement Repositories
- Multihoming and Traffic Engineering
- What is Multihoming
- Why Multihome
- Redundancy
- Performance
- Multihoming in IP Networks Today
- BGP or no
- Same Provider or Multiple
- Multihomed Stub One Link
- Multihomed Stub Multiple Links
- Multihomed Stub Multiple ISPs
- Multihomed Transit Network
- Interdomain Traffic Engineering
- Outbound Traffic Control
- Outbound Traffic Load Balancing
- Traffic Engineering Goals
- Managing Scale
- Achieving Predictability
- Challenges to Predictability
- Inter-AS Negotiation
- Outbound Multihoming Goals
- Inbound Traffic Control
- How many links are enough
- Problems with Multihoming in IPv4
- Slide 68
- Slide 69
- Diagnosis and Troubleshooting
- Operator Mailing List (NANOG)
- Operator Mailing List
- Routing Configuration
- Internet Business Model (Simplified)
- Peering Contracts Consistent Export
- Consistent Export
- Inconsistent Export in Practice
- Blackholes
- Security
- Security ldquoBogonrdquo Routes
- Spam Phishing etc
- Spam and Routing
- Worms and Botnets
- What is a Worm
- Example Worm Code Red
- Code Red Revisions
- Code Red Host Infection Rate
- Designing Fast-Spreading Worms
- Botnets
- ldquoRallyingrdquo the Botnet
- Botnet Control
- Some Defenses
- Idea 1 Ingress Filtering
- Idea 2 uRPF Checks
- Problems with uRPF
- S-BGP
- Practical Problems with S-BGP
-
29
Simple Network Management Protocol
bull Management Information Base (MIB)ndash Information storendash Unique variables named by
OIDsndash Accessed with SNMP
bull Specific MIBs for bytepacket counts (per link)
Manager Agent
SNMP
DB
ManagedObjects
30
SNMP (Passive)
bull Advantage ubiquitousndash Supported on all networking equipmentndash Multiple products for polling and analyzing data
bull Disadvantages ndash Coarse granularityndash Cannot express complex queries on the datandash Unreliable delivery of the data using UDP
bull Utilityndash Link utilization (billing)ndash Traffic matrix inference
31
Packet-level Monitoring
bull Passive monitoring to collect full packet contents (or at least headers)
bull Advantages lots of detailed informationndash Precise timing informationndash Information in packet headers
bull Disadvantages overheadndash Hard to keep up with high-speed linksndash Often requires a separate monitoring device
32
Full Packet Capture (Passive)
Example Georgia Tech OC3Mon
bull Rack-mounted PCbull Optical splitterbull Data Acquisition and
Generation (DAG) card
Source endacecom
33
What is a flow
bull Source IP addressbull Destination IP addressbull Source portbull Destination portbull Layer 3 protocol typebull TOS byte (DSCP)bull Input logical interface (ifIndex)
34
Cisco NetFlow
bull Basic output ldquoFlow recordrdquondash Most common version is v5
bull Current version (9) is being standardized in the IETF (template-based)ndash More flexible record formatndash Much easier to add new flow record types
Core Network
Collection and Aggregation
Collector (PC)Approximately 1500 bytes
20-50 flow recordsSent more frequently if traffic increases
35
Flow Record Contents
bull Source and Destination IP address and portbull Packet and byte countsbull Start and end timesbull ToS TCP flags
Basic information about the flowhellip
hellipplus information related to routing
bull Next-hop IP addressbull Source and destination ASbull Source and destination prefix
36
flow 1 flow 2 flow 3 flow 4
Aggregating Packets into Flows
bull Criteria 1 Set of packets that ldquobelong togetherrdquondash Sourcedestination IP addresses and port numbersndash Same protocol ToS bits hellip ndash Same inputoutput interfaces at a router (if known)
bull Criteria 2 Packets that are ldquocloserdquo together in timendash Maximum inter-packet spacing (eg 15 sec 30 sec)ndash Example flows 2 and 4 are different flows due to time
37
Reducing Measurement Overhead
bull Filtering on interfacendash destination prefix for a customerndash port number for an application (eg 80 for Web)
bull Sampling before insertion into flow cachendash Random deterministic or hash-based samplingndash 1-out-of-n or stratified based on packetflow sizendash Two types packet-level and flow-level
bull Aggregation after cache evictionndash packetsflows with same next-hop ASndash packetsflows destined to a particular service
38
Packet Sampling for Flow Monitoring
bull Packet sampling before flow creation (Sampled Netflow)ndash 1-out-of-m sampling of individual packets (eg m=100)ndash Create of flow records over the sampled packets
bull Reducing overheadndash Avoid per-packet overhead on (m-1)m packetsndash Avoid creating records for a large number of small flows
bull Increasing overhead (in some cases)ndash May split some long transfers into multiple flow records ndash hellip due to larger time gaps between successive packets
time
not sampled
two flowstimeout
39
Sampling Flow-Level Sampling
bull Sampling of flow records evicted from flow cachendash When evicting flows from table or when analyzing flows
bull Stratified sampling to put weight on ldquoheavyrdquo flowsndash Select all long flows and sample the short flows
bull Reduces the number of flow records ndash Still measures the vast majority of the traffic
Flow 1 40 bytesFlow 2 15580 bytesFlow 3 8196 bytesFlow 4 5350789 bytesFlow 5 532 bytesFlow 6 7432 bytes
sample with 100 probability
sample with 01 probability
sample with 10 probability
40
Two Main Approaches
bull Packet-level Monitoringndash Keep packet-level statisticsndash Examine (and potentially log) variety of packet-level
statistics Essentially anything in the packetndash Timing
bull Flow-level Monitoringndash Monitor packet-by-packet (though sometimes
sampled)ndash Keep aggregate statistics on a flow
41
Packet Capture on High-Speed Links
Example Georgia Tech ldquoOC3Monrdquo
bull Rack-mounted PCbull Optical splitterbull Data Acquisition and
Generation (DAG) card
Source endacecom
42
Characteristics of Packet Capture
bull Allows inspection on every packet on 10G links
bull Disadvantagesndash Costlyndash Requires splitting optical fibersndash Must be able to filterstore data
43
Data Measurement Repositories
bull AbileneInternet 2 Observatoryndash Configuration examplesndash SNMP datandash ISIS BGP routing data NetFlow traffic data
bull RouteViewsndash BGP updatesndash BGP table snapshots
44
Multihoming and Traffic Engineering
45
What is Multihoming
bull The use of redundant network links for the purposes of external connectivity
bull Can be achieved at many layers of the protocol stack and many places in the networkndash Multiple network interfaces in a PC
ndash An ISP with multiple upstream interfaces
bull Can refer to having multiple connections tondash The same ISP
ndash Multiple ISPs
46
Why Multihome
bull Redundancybull Availabilitybull Performancebull Cost
Interdomain traffic engineering the process by which a multihomed network configures its
network to achieve these goals
47
Redundancy
bull Maintain connectivity in the face ofndash Physical connectivity problems (fiber cut device
failures etc)ndash Failures in upstream ISP
48
Performance
bull Use multiple network links at once to achieve higher throughput than just over a single link
bull Allows incoming traffic to be load-balanced
70 of traffic30 of traffic
49
Multihoming in IP Networks Today
bull Stub AS no transit service for other ASesndash No need to use BGP
bull Multi-homed stub AS has connectivity to multiple immediate upstream ISPsndash Need BGPndash No need for a public AS numberndash No need for IP prefix allocation
bull Multi-homed transit AS connectivity to multiple ASes and transit servicendash Need BGP public AS number IP prefix allocation
50
BGP or no
bull Advantages of static routingndash Cheapersmaller routers (less true nowadays)ndash Simpler to configure
bull Advantages of BGPndash More control of your destiny (have providers stop
announcing you)ndash Fastermore intelligent selection of where to send
outbound packetsndash Better debugging of net problems (you can see the
Internet topology now)
51
Same Provider or Multiple
bull If your provider is reliable and fast and affordably and offers good tech-support you may want to multi-home initially to them via some backup path (slow is better than dead)
bull Eventually yoursquoll want to multi-home to different providers to avoid failure modes due to one providerrsquos architecture decisions
52
Multihomed Stub One Link
bull Downstream ISPrsquos routers configure default (ldquostaticrdquo) routes pointing to border router
bull Upstream ISP advertises reachability
Upstream ISP
Multiple links between same pair of routers
Default routes to ldquoborderrdquo
ldquoStubrdquoISP
53
Multihomed Stub Multiple Links
bull Use BGP to share loadbull Use private AS number (why is this OK)bull As before upstream ISP advertises prefix
Upstream ISP
Multiple links to different upstream routers
ldquoStubrdquoISP
Internal routing for ldquohot potatordquo
BGP for load balance at edge
54
Multihomed Stub Multiple ISPs
bull Many possibilitiesndash Load sharingndash Primary-backupndash Selective use of different ISPs
bull Requires BGP public AS number etc
ldquoStubrdquoISP
Upstream
ISP 1
Upstream
ISP 2
55
Multihomed Transit Network
bull BGP everywherebull Incoming and outcoming trafficbull Challenge balancing load on intradomain and egress
links given an offered traffic load
TransitISP
ISP 1
ISP 2
ISP 3
56
Interdomain Traffic Engineering
bull The process by which a network operator configures the network to achievendash Traffic load balancendash Redundancy (primarybackup) etc
bull Two tasksndash Outbound traffic controlndash Inbound traffic control
bull Key Problems Predictability and Scalability
57
Outbound Traffic Control
bull Easier to control than inbound trafficndash Destination-based routing sender determines where
the packets go
bull Control over next-hop AS onlyndash Cannot control selection of the entire path
Provider 1 Provider 2
Control with local preference
58
Outbound Traffic Load Balancingbull Control routes to provider per-prefix
ndash Assign local preference across destination prefixesndash Change the local preference assignments over time
bull Useful inputs to load balancingndash End-to-end path performance datandash Outbound traffic statistics per destination prefix
bull Challenge Getting from traffic volumes to groups of prefixes that should be assigned to each link
Premise of ldquointelligent route controlrdquo preoducts
59
Traffic Engineering Goals
bull Predictabilityndash Ensure the BGP decision process is deterministicndash Assume that BGP updates are (relatively) stable
bull Limit overhead introduced by routing changesndash Minimize frequency of changes to routing policiesndash Limit number of prefixes affected by changes
bull Limit impact on how traffic enters the networkndash Avoid new routes that might change neighborrsquos mindndash Select route with same attributes or at least path length
60
Managing Scalebull Destination prefixes
ndash More than 90000 destination prefixes
bull Donrsquot want to have per-prefix routing policies
ndash Small fraction of prefixes contribute most of the traffic
bull Focus on the small number of heavy hitters
ndash Define routing policies for selected prefixes
bull Routing choicesndash About 27000 unique ldquorouting choicesrdquo
bull Help in reducing the scale of the problem
ndash Small fraction of ldquorouting choicesrdquo contribute most traffic
bull Focus on the very small number of ldquorouting choicesrdquo
ndash Define routing policies on common attributes
61
Achieving Predictability
bull Route prediction with static analysisndash Helpful to know effects before deploymentndash Static analysis can help
TopologyBGP policy
configuration
eBGP routes
Offered traffic
BGP routingmodel
Flow of traffic through the network
62
Challenges to Predictabilitybull For transit ISPs effects on incoming traffic
ndash Lack of coordination strikes again
63
ldquoHot Potatordquo routing
Inter-AS Negotiation
bull Coordination aids predictabilityndash Negotiate where to sendndash Inbound and outboundndash Mutual benefits
bull How to implementndash What info to exchangendash Protecting privacyndash How to prioritize choicesndash How to prevent cheating
Destination 2
Destination 1
multiplepeeringpoints
Provider A
Provider B
64
Outbound Multihoming Goals
bull Redundancyndash Dynamic routing will failover to backup link
bull Performancendash Select provider with best performance per prefix
ndash Requires active probing
bull Costndash Select provider per prefix over time to minimize the
total financial cost
65
Inbound Traffic Control
bull More difficult no control over neighborsrsquo decisions
bull Three common techniques (previously discussed)ndash AS path prependingndash Communities and local preferencendash Prefix splitting
How does todayrsquos paper (MONET) control inbound traffic
66
How many links are enough
K upstream ISPs
Not much benefit beyond 4 ISPs
Akella et al ldquoPerformance Benefits of Multihomingrdquo SIGCOMM 2003
67
Problems with Multihoming in IPv4
bull Routing table growthndash Provider-based addressingndash Advertising prefix out multiple ISPs ndash canrsquot aggregate
bull Poor control over inbound trafficndash Existing mechanisms do not allow hosts to control
inbound traffic
68
GeorgiaTech
Internet Routing Overview
bull Intradomain (ie ldquointra-ASrdquo) routingbull Interdomain routing
Comcast
Abilene
ATampT Cogent
Autonomous Systems (ASes)
69
Configuration Problems ldquoAS 7007rdquoldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
UUNet
Florida InternetBarn
Sprint
70
Diagnosis and Troubleshootingldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
ldquoMicrosofts websites were offline for up to 23 hoursbecause of a [router] misconfigurationhellipit took nearly a day to determine what was wrong and undo the changesrdquo -- wiredcom January 25 2001
ldquoWorldCom Inchellipsuffered a widespread outage on its Internet backbone that affected roughly 20 percent of its US customer base The network problemshellipaffected millions of computer users worldwide A spokeswoman attributed the outage to a route table issue -- cnncom October 3 2002
A number of Covad customers went out from 5pm today due to supposedly a DDOS (distributed denial of service attack) on a key Level3 data center which later was described as a route leak (misconfiguration)ldquo
-- dslreportscom February 23 2004
71
Operator Mailing List (NANOG)Date Mon 18 Oct 2004 091515 -0700Subject Level 3 US east coast issues
Level 3 experiencing widespread unspecified routing issues on the US east coast Master ticket 1086844 Anyone have more specific information
Date Mon 18 Oct 2004 122034 -0400 (EDT)Subject Re Level 3 US east coast issues
Level 3 is currently experiencing a backbone outage causing routing instability and packet loss We are working to restore and will be sending hourly updateshellip
72
Operator Mailing List
0102030405060708090
Filtering RouteLeaks
RouteHijacks
RouteInstability
RoutingLoops
Blackholes
Occurr
ences o
ver
10 Y
ears
1995-1997 1998-2001 2001-2004
Note Only includes problems openly discussed on this list
Compare 83 power outages 1 fire
73
Routing Configuration
Ranking route selection
Dissemination internal route advertisement
Filtering route advertisement
Customer
Competitor
Primary
Backup
74
Internet Business Model (Simplified)
bull CustomerProvider One AS pays another for reachability to some set of destinations
bull ldquoSettlement-freerdquo Peering Bartering Two ASes exchange routes with one another
Provider
Peer
Customer
Preferences implemented with local preference manipulation
Destination
Pay to use
Get paid to use
Free to use
75
Peering Contracts Consistent Export
bull Rules of settlement-free peeringndash Advertise routes at all peering pointsndash Advertised routes must have equal ldquoAS path lengthrdquo
Sprint
ATampT
Enables ldquohot potatordquo routing
ldquoequally goodrdquoroutes
76
Consistent Export
bull Malicedeceptionbull iBGP signaling partitionbull Inconsistent export policy
neighbor 10123route-map PEER permit 10 set prepend 123
neighbor 10456route-map PEER permit 10 set prepend 123 123
Possible Causes
10123 456 1
10456 456 2
Neighbor AS Export
1 1 123
Export Clause Prepend
2 1 123 123
Two different Export Policies
77
Inconsistent Export in Practice
Feamster et al ldquoBorderGuard Detecting Cold Potatoes from Peersrdquo ACM IMC October 2004
78
Blackholes
Date Thu 18 Jul 2002 060510 -0400 (EDT)From Chad Oleary ltcolpoboxcomgtSubject Re problems with 701To ltnanogmeritedugt
Were starting to see the same issues with UUNet again Anyone elseseeing this Trying to reach Qwest
traceroute to 631461901 (631461901) 30 hops max 38 byte packets 1 esc-lp2-gwe-solutionscorpcom (631182201) 1167 ms 1163 ms 1142 ms 2 500Serial2-10GW1TPA2ALTERNET (1571301499) 1097 ms 1059 ms 1044 ms 3 161at-1-0-0XL4ATL1ALTERNET (1526381190) 13839 ms 14108 ms 16638 ms 4 0so-3-1-0XL2ATL5ALTERNET (152630238) 14370 ms 14587 ms 14553 ms 5 POS7-0BR2ATL5ALTERNET (1526382193) 13928 ms 14099 ms 14053 ms 6 7 hellip
79
Security
80
Security ldquoBogonrdquo Routes
Feamster et al ldquoAn Empirical Study of lsquoBogonrsquo Route Advertisementsrdquo ACM CCR January 2005
81
Spam Phishing etc
bull Unsolicited commercial emailbull As of about August 2008 estimates indicate that
about 95 of all email is spambull Common spam filtering techniques
ndash Content-based filtersndash DNS Blacklist (DNSBL) lookups Significant fraction of
todayrsquos DNS traffic
Can IP addresses from which spam is received be spoofed
82
Spam and Routing
83
Worms and Botnets
84
What is a Worm
bull Code that replicates and propagates across the networkndash Often carries a ldquopayloadrdquo
bull Usually spread via exploiting flaws in open servicesndash ldquoVirusesrdquo require user action to spread
bull First worm Robert Morris November 1988ndash 6-10 of all Internet hosts infected ()
bull Many more since but none on that scale until July 2001
85
Example Worm Code Red
bull Initial version July 13 2001
bull Exploited known ISAPI vulnerability in Microsoft IIS Web servers
bull 1st through 20th of each month spread20th through end of each month attack
bull Payload Web site defacementbull Scanning Random IP addressesbull Bug failure to seed random number generator
86
Code Red Revisions
bull Released July 19 2001
bull Payload flooding attack on wwwwhitehousegovndash Attack was mounted at the IP address of the Web site
bull Bug died after 20th of each month
bull Random number generator for IP scanning fixed
87
Code Red Host Infection Rate
Exponential infection rate
Measured using backscatter technique
88
Designing Fast-Spreading Worms
bull Hit-list scanningndash Time to infect first 10k hosts dominates infection timendash Solution Reconnaissance (stealthy scans etc)
bull Permutation scanningndash Observation Most scanning is redundantndash Idea Shared permutation of address space Start scanning
from own IP address Re-randomize when another infected machine is found
bull Internet-scale hit listsndash Flash worm complete infection within 30 seconds
89
Botnets
bull Bots Autonomous programs performing tasksbull Plenty of ldquobenignrdquo bots
ndash eg weatherbug
bull Botnets group of bots ndash Typically carries malicious connotationndash Large numbers of infected machinesndash Machines ldquoenlistedrdquo with infection vectors like worms
(last lecture)
bull Available for simultaneous control by a masterbull Size up to 350000 nodes (from todayrsquos paper)
90
ldquoRallyingrdquo the Botnet
bull Easy to combine worm backdoor functionalitybull Problem how to learn about successfully
infected machines
bull Optionsndash Emailndash Hard-coded email address
91
Botnet Control
bull Botnet master typically runs some IRC server on a well-known port (eg 6667)
bull Infected machine contacts botnet with pre-programmed DNS name (eg big-botde)
bull Dynamic DNS allows controller to move about freely
Infected Machine
DynamicDNS
BotnetController
(IRC server)
92
Some Defenses
93
Idea 1 Ingress Filtering
bull RFC 2827 Routers install filters to drop packets from networks that are not downstream
bull Feasible at edgesbull Difficult to configure closer to network ldquocorerdquo
20469207024 Internet
Drop all packets with source address other than 20469207024
94
Idea 2 uRPF Checks
bull Unicast Reverse Path Forwardingndash Cisco ldquoip verify unicast reverse-pathrdquo
bull Requires symmetric routing
Accept packet from interface only if forwarding table entry for source IP address matches ingress interface
100183
A10018
10015
101203
100181 24
10011 24
Strict Mode uRPF
Enabled
ldquoArdquo Routing TableDestination Next Hop1001024 Int 110018024 Int 2
100183 from wrong interface
95
Problems with uRPF
bull Asymmetric routing
96
S-BGP
bull Address-based PKI validate signaturesndash Authentication of
bull ownership for IP address blocks bull AS number bull an ASs identity and bull a BGP routers identity
ndash Use existing infrastructure (Internet registries etc)ndash Routing origination is digitally signedndash BGP updates are digitally signed
1048708 bull Route attestations A new optional BGP transitive path attribute
ndash carries digital signatures covering the routing information in updates
97
Practical Problems with S-BGP
bull Requires Public-Key Infrastructure
bull Lots of digital signatures to calculate and verifyndash Message overheadndash CPU overhead
bull Calculation expense is greatest when topology is changingndash Caching can help
bull Route aggregation is problematic (maybe thatrsquos OK)
bull Secure route withdrawals when link or node fails
bull Address ownership data out of date
bull Deployment
- Networking
- Goal of This Tutorial
- Todayrsquos Networks
- Business Models
- Billing for Internet Usage
- Net Neutrality
- Network Operations
- Point-of-Presence (PoP)
- Example Abilene Network Topology
- Another Example Backbone
- Internet Routing Overview
- Internet Routing Protocol BGP
- Two Flavors of BGP
- IPv4 Addresses Networks of Networks
- Pre-1994 Classful Addressing
- Classless Interdomain Routing (CIDR)
- Benefits of CIDR
- Growth of IP Prefixes
- 1994-1998 Linear Growth
- Around 2000 Fast Growth Resumes
- Fast growth resumes
- The Address Allocation Process
- Common Problems
- What can go wrong
- Measurement and Monitoring
- Passive vs Active Measurement
- Slide 27
- Passive Traffic Data Measurement
- Simple Network Management Protocol
- SNMP (Passive)
- Packet-level Monitoring
- Full Packet Capture (Passive)
- What is a flow
- Cisco NetFlow
- Flow Record Contents
- Aggregating Packets into Flows
- Reducing Measurement Overhead
- Packet Sampling for Flow Monitoring
- Sampling Flow-Level Sampling
- Two Main Approaches
- Packet Capture on High-Speed Links
- Characteristics of Packet Capture
- Data Measurement Repositories
- Multihoming and Traffic Engineering
- What is Multihoming
- Why Multihome
- Redundancy
- Performance
- Multihoming in IP Networks Today
- BGP or no
- Same Provider or Multiple
- Multihomed Stub One Link
- Multihomed Stub Multiple Links
- Multihomed Stub Multiple ISPs
- Multihomed Transit Network
- Interdomain Traffic Engineering
- Outbound Traffic Control
- Outbound Traffic Load Balancing
- Traffic Engineering Goals
- Managing Scale
- Achieving Predictability
- Challenges to Predictability
- Inter-AS Negotiation
- Outbound Multihoming Goals
- Inbound Traffic Control
- How many links are enough
- Problems with Multihoming in IPv4
- Slide 68
- Slide 69
- Diagnosis and Troubleshooting
- Operator Mailing List (NANOG)
- Operator Mailing List
- Routing Configuration
- Internet Business Model (Simplified)
- Peering Contracts Consistent Export
- Consistent Export
- Inconsistent Export in Practice
- Blackholes
- Security
- Security ldquoBogonrdquo Routes
- Spam Phishing etc
- Spam and Routing
- Worms and Botnets
- What is a Worm
- Example Worm Code Red
- Code Red Revisions
- Code Red Host Infection Rate
- Designing Fast-Spreading Worms
- Botnets
- ldquoRallyingrdquo the Botnet
- Botnet Control
- Some Defenses
- Idea 1 Ingress Filtering
- Idea 2 uRPF Checks
- Problems with uRPF
- S-BGP
- Practical Problems with S-BGP
-
30
SNMP (Passive)
bull Advantage ubiquitousndash Supported on all networking equipmentndash Multiple products for polling and analyzing data
bull Disadvantages ndash Coarse granularityndash Cannot express complex queries on the datandash Unreliable delivery of the data using UDP
bull Utilityndash Link utilization (billing)ndash Traffic matrix inference
31
Packet-level Monitoring
bull Passive monitoring to collect full packet contents (or at least headers)
bull Advantages lots of detailed informationndash Precise timing informationndash Information in packet headers
bull Disadvantages overheadndash Hard to keep up with high-speed linksndash Often requires a separate monitoring device
32
Full Packet Capture (Passive)
Example Georgia Tech OC3Mon
bull Rack-mounted PCbull Optical splitterbull Data Acquisition and
Generation (DAG) card
Source endacecom
33
What is a flow
bull Source IP addressbull Destination IP addressbull Source portbull Destination portbull Layer 3 protocol typebull TOS byte (DSCP)bull Input logical interface (ifIndex)
34
Cisco NetFlow
bull Basic output ldquoFlow recordrdquondash Most common version is v5
bull Current version (9) is being standardized in the IETF (template-based)ndash More flexible record formatndash Much easier to add new flow record types
Core Network
Collection and Aggregation
Collector (PC)Approximately 1500 bytes
20-50 flow recordsSent more frequently if traffic increases
35
Flow Record Contents
bull Source and Destination IP address and portbull Packet and byte countsbull Start and end timesbull ToS TCP flags
Basic information about the flowhellip
hellipplus information related to routing
bull Next-hop IP addressbull Source and destination ASbull Source and destination prefix
36
flow 1 flow 2 flow 3 flow 4
Aggregating Packets into Flows
bull Criteria 1 Set of packets that ldquobelong togetherrdquondash Sourcedestination IP addresses and port numbersndash Same protocol ToS bits hellip ndash Same inputoutput interfaces at a router (if known)
bull Criteria 2 Packets that are ldquocloserdquo together in timendash Maximum inter-packet spacing (eg 15 sec 30 sec)ndash Example flows 2 and 4 are different flows due to time
37
Reducing Measurement Overhead
bull Filtering on interfacendash destination prefix for a customerndash port number for an application (eg 80 for Web)
bull Sampling before insertion into flow cachendash Random deterministic or hash-based samplingndash 1-out-of-n or stratified based on packetflow sizendash Two types packet-level and flow-level
bull Aggregation after cache evictionndash packetsflows with same next-hop ASndash packetsflows destined to a particular service
38
Packet Sampling for Flow Monitoring
bull Packet sampling before flow creation (Sampled Netflow)ndash 1-out-of-m sampling of individual packets (eg m=100)ndash Create of flow records over the sampled packets
bull Reducing overheadndash Avoid per-packet overhead on (m-1)m packetsndash Avoid creating records for a large number of small flows
bull Increasing overhead (in some cases)ndash May split some long transfers into multiple flow records ndash hellip due to larger time gaps between successive packets
time
not sampled
two flowstimeout
39
Sampling Flow-Level Sampling
bull Sampling of flow records evicted from flow cachendash When evicting flows from table or when analyzing flows
bull Stratified sampling to put weight on ldquoheavyrdquo flowsndash Select all long flows and sample the short flows
bull Reduces the number of flow records ndash Still measures the vast majority of the traffic
Flow 1 40 bytesFlow 2 15580 bytesFlow 3 8196 bytesFlow 4 5350789 bytesFlow 5 532 bytesFlow 6 7432 bytes
sample with 100 probability
sample with 01 probability
sample with 10 probability
40
Two Main Approaches
bull Packet-level Monitoringndash Keep packet-level statisticsndash Examine (and potentially log) variety of packet-level
statistics Essentially anything in the packetndash Timing
bull Flow-level Monitoringndash Monitor packet-by-packet (though sometimes
sampled)ndash Keep aggregate statistics on a flow
41
Packet Capture on High-Speed Links
Example Georgia Tech ldquoOC3Monrdquo
bull Rack-mounted PCbull Optical splitterbull Data Acquisition and
Generation (DAG) card
Source endacecom
42
Characteristics of Packet Capture
bull Allows inspection on every packet on 10G links
bull Disadvantagesndash Costlyndash Requires splitting optical fibersndash Must be able to filterstore data
43
Data Measurement Repositories
bull AbileneInternet 2 Observatoryndash Configuration examplesndash SNMP datandash ISIS BGP routing data NetFlow traffic data
bull RouteViewsndash BGP updatesndash BGP table snapshots
44
Multihoming and Traffic Engineering
45
What is Multihoming
bull The use of redundant network links for the purposes of external connectivity
bull Can be achieved at many layers of the protocol stack and many places in the networkndash Multiple network interfaces in a PC
ndash An ISP with multiple upstream interfaces
bull Can refer to having multiple connections tondash The same ISP
ndash Multiple ISPs
46
Why Multihome
bull Redundancybull Availabilitybull Performancebull Cost
Interdomain traffic engineering the process by which a multihomed network configures its
network to achieve these goals
47
Redundancy
bull Maintain connectivity in the face ofndash Physical connectivity problems (fiber cut device
failures etc)ndash Failures in upstream ISP
48
Performance
bull Use multiple network links at once to achieve higher throughput than just over a single link
bull Allows incoming traffic to be load-balanced
70 of traffic30 of traffic
49
Multihoming in IP Networks Today
bull Stub AS no transit service for other ASesndash No need to use BGP
bull Multi-homed stub AS has connectivity to multiple immediate upstream ISPsndash Need BGPndash No need for a public AS numberndash No need for IP prefix allocation
bull Multi-homed transit AS connectivity to multiple ASes and transit servicendash Need BGP public AS number IP prefix allocation
50
BGP or no
bull Advantages of static routingndash Cheapersmaller routers (less true nowadays)ndash Simpler to configure
bull Advantages of BGPndash More control of your destiny (have providers stop
announcing you)ndash Fastermore intelligent selection of where to send
outbound packetsndash Better debugging of net problems (you can see the
Internet topology now)
51
Same Provider or Multiple
bull If your provider is reliable and fast and affordably and offers good tech-support you may want to multi-home initially to them via some backup path (slow is better than dead)
bull Eventually yoursquoll want to multi-home to different providers to avoid failure modes due to one providerrsquos architecture decisions
52
Multihomed Stub One Link
bull Downstream ISPrsquos routers configure default (ldquostaticrdquo) routes pointing to border router
bull Upstream ISP advertises reachability
Upstream ISP
Multiple links between same pair of routers
Default routes to ldquoborderrdquo
ldquoStubrdquoISP
53
Multihomed Stub Multiple Links
bull Use BGP to share loadbull Use private AS number (why is this OK)bull As before upstream ISP advertises prefix
Upstream ISP
Multiple links to different upstream routers
ldquoStubrdquoISP
Internal routing for ldquohot potatordquo
BGP for load balance at edge
54
Multihomed Stub Multiple ISPs
bull Many possibilitiesndash Load sharingndash Primary-backupndash Selective use of different ISPs
bull Requires BGP public AS number etc
ldquoStubrdquoISP
Upstream
ISP 1
Upstream
ISP 2
55
Multihomed Transit Network
bull BGP everywherebull Incoming and outcoming trafficbull Challenge balancing load on intradomain and egress
links given an offered traffic load
TransitISP
ISP 1
ISP 2
ISP 3
56
Interdomain Traffic Engineering
bull The process by which a network operator configures the network to achievendash Traffic load balancendash Redundancy (primarybackup) etc
bull Two tasksndash Outbound traffic controlndash Inbound traffic control
bull Key Problems Predictability and Scalability
57
Outbound Traffic Control
bull Easier to control than inbound trafficndash Destination-based routing sender determines where
the packets go
bull Control over next-hop AS onlyndash Cannot control selection of the entire path
Provider 1 Provider 2
Control with local preference
58
Outbound Traffic Load Balancingbull Control routes to provider per-prefix
ndash Assign local preference across destination prefixesndash Change the local preference assignments over time
bull Useful inputs to load balancingndash End-to-end path performance datandash Outbound traffic statistics per destination prefix
bull Challenge Getting from traffic volumes to groups of prefixes that should be assigned to each link
Premise of ldquointelligent route controlrdquo preoducts
59
Traffic Engineering Goals
bull Predictabilityndash Ensure the BGP decision process is deterministicndash Assume that BGP updates are (relatively) stable
bull Limit overhead introduced by routing changesndash Minimize frequency of changes to routing policiesndash Limit number of prefixes affected by changes
bull Limit impact on how traffic enters the networkndash Avoid new routes that might change neighborrsquos mindndash Select route with same attributes or at least path length
60
Managing Scalebull Destination prefixes
ndash More than 90000 destination prefixes
bull Donrsquot want to have per-prefix routing policies
ndash Small fraction of prefixes contribute most of the traffic
bull Focus on the small number of heavy hitters
ndash Define routing policies for selected prefixes
bull Routing choicesndash About 27000 unique ldquorouting choicesrdquo
bull Help in reducing the scale of the problem
ndash Small fraction of ldquorouting choicesrdquo contribute most traffic
bull Focus on the very small number of ldquorouting choicesrdquo
ndash Define routing policies on common attributes
61
Achieving Predictability
bull Route prediction with static analysisndash Helpful to know effects before deploymentndash Static analysis can help
TopologyBGP policy
configuration
eBGP routes
Offered traffic
BGP routingmodel
Flow of traffic through the network
62
Challenges to Predictabilitybull For transit ISPs effects on incoming traffic
ndash Lack of coordination strikes again
63
ldquoHot Potatordquo routing
Inter-AS Negotiation
bull Coordination aids predictabilityndash Negotiate where to sendndash Inbound and outboundndash Mutual benefits
bull How to implementndash What info to exchangendash Protecting privacyndash How to prioritize choicesndash How to prevent cheating
Destination 2
Destination 1
multiplepeeringpoints
Provider A
Provider B
64
Outbound Multihoming Goals
bull Redundancyndash Dynamic routing will failover to backup link
bull Performancendash Select provider with best performance per prefix
ndash Requires active probing
bull Costndash Select provider per prefix over time to minimize the
total financial cost
65
Inbound Traffic Control
bull More difficult no control over neighborsrsquo decisions
bull Three common techniques (previously discussed)ndash AS path prependingndash Communities and local preferencendash Prefix splitting
How does todayrsquos paper (MONET) control inbound traffic
66
How many links are enough
K upstream ISPs
Not much benefit beyond 4 ISPs
Akella et al ldquoPerformance Benefits of Multihomingrdquo SIGCOMM 2003
67
Problems with Multihoming in IPv4
bull Routing table growthndash Provider-based addressingndash Advertising prefix out multiple ISPs ndash canrsquot aggregate
bull Poor control over inbound trafficndash Existing mechanisms do not allow hosts to control
inbound traffic
68
GeorgiaTech
Internet Routing Overview
bull Intradomain (ie ldquointra-ASrdquo) routingbull Interdomain routing
Comcast
Abilene
ATampT Cogent
Autonomous Systems (ASes)
69
Configuration Problems ldquoAS 7007rdquoldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
UUNet
Florida InternetBarn
Sprint
70
Diagnosis and Troubleshootingldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
ldquoMicrosofts websites were offline for up to 23 hoursbecause of a [router] misconfigurationhellipit took nearly a day to determine what was wrong and undo the changesrdquo -- wiredcom January 25 2001
ldquoWorldCom Inchellipsuffered a widespread outage on its Internet backbone that affected roughly 20 percent of its US customer base The network problemshellipaffected millions of computer users worldwide A spokeswoman attributed the outage to a route table issue -- cnncom October 3 2002
A number of Covad customers went out from 5pm today due to supposedly a DDOS (distributed denial of service attack) on a key Level3 data center which later was described as a route leak (misconfiguration)ldquo
-- dslreportscom February 23 2004
71
Operator Mailing List (NANOG)Date Mon 18 Oct 2004 091515 -0700Subject Level 3 US east coast issues
Level 3 experiencing widespread unspecified routing issues on the US east coast Master ticket 1086844 Anyone have more specific information
Date Mon 18 Oct 2004 122034 -0400 (EDT)Subject Re Level 3 US east coast issues
Level 3 is currently experiencing a backbone outage causing routing instability and packet loss We are working to restore and will be sending hourly updateshellip
72
Operator Mailing List
0102030405060708090
Filtering RouteLeaks
RouteHijacks
RouteInstability
RoutingLoops
Blackholes
Occurr
ences o
ver
10 Y
ears
1995-1997 1998-2001 2001-2004
Note Only includes problems openly discussed on this list
Compare 83 power outages 1 fire
73
Routing Configuration
Ranking route selection
Dissemination internal route advertisement
Filtering route advertisement
Customer
Competitor
Primary
Backup
74
Internet Business Model (Simplified)
bull CustomerProvider One AS pays another for reachability to some set of destinations
bull ldquoSettlement-freerdquo Peering Bartering Two ASes exchange routes with one another
Provider
Peer
Customer
Preferences implemented with local preference manipulation
Destination
Pay to use
Get paid to use
Free to use
75
Peering Contracts Consistent Export
bull Rules of settlement-free peeringndash Advertise routes at all peering pointsndash Advertised routes must have equal ldquoAS path lengthrdquo
Sprint
ATampT
Enables ldquohot potatordquo routing
ldquoequally goodrdquoroutes
76
Consistent Export
bull Malicedeceptionbull iBGP signaling partitionbull Inconsistent export policy
neighbor 10123route-map PEER permit 10 set prepend 123
neighbor 10456route-map PEER permit 10 set prepend 123 123
Possible Causes
10123 456 1
10456 456 2
Neighbor AS Export
1 1 123
Export Clause Prepend
2 1 123 123
Two different Export Policies
77
Inconsistent Export in Practice
Feamster et al ldquoBorderGuard Detecting Cold Potatoes from Peersrdquo ACM IMC October 2004
78
Blackholes
Date Thu 18 Jul 2002 060510 -0400 (EDT)From Chad Oleary ltcolpoboxcomgtSubject Re problems with 701To ltnanogmeritedugt
Were starting to see the same issues with UUNet again Anyone elseseeing this Trying to reach Qwest
traceroute to 631461901 (631461901) 30 hops max 38 byte packets 1 esc-lp2-gwe-solutionscorpcom (631182201) 1167 ms 1163 ms 1142 ms 2 500Serial2-10GW1TPA2ALTERNET (1571301499) 1097 ms 1059 ms 1044 ms 3 161at-1-0-0XL4ATL1ALTERNET (1526381190) 13839 ms 14108 ms 16638 ms 4 0so-3-1-0XL2ATL5ALTERNET (152630238) 14370 ms 14587 ms 14553 ms 5 POS7-0BR2ATL5ALTERNET (1526382193) 13928 ms 14099 ms 14053 ms 6 7 hellip
79
Security
80
Security ldquoBogonrdquo Routes
Feamster et al ldquoAn Empirical Study of lsquoBogonrsquo Route Advertisementsrdquo ACM CCR January 2005
81
Spam Phishing etc
bull Unsolicited commercial emailbull As of about August 2008 estimates indicate that
about 95 of all email is spambull Common spam filtering techniques
ndash Content-based filtersndash DNS Blacklist (DNSBL) lookups Significant fraction of
todayrsquos DNS traffic
Can IP addresses from which spam is received be spoofed
82
Spam and Routing
83
Worms and Botnets
84
What is a Worm
bull Code that replicates and propagates across the networkndash Often carries a ldquopayloadrdquo
bull Usually spread via exploiting flaws in open servicesndash ldquoVirusesrdquo require user action to spread
bull First worm Robert Morris November 1988ndash 6-10 of all Internet hosts infected ()
bull Many more since but none on that scale until July 2001
85
Example Worm Code Red
bull Initial version July 13 2001
bull Exploited known ISAPI vulnerability in Microsoft IIS Web servers
bull 1st through 20th of each month spread20th through end of each month attack
bull Payload Web site defacementbull Scanning Random IP addressesbull Bug failure to seed random number generator
86
Code Red Revisions
bull Released July 19 2001
bull Payload flooding attack on wwwwhitehousegovndash Attack was mounted at the IP address of the Web site
bull Bug died after 20th of each month
bull Random number generator for IP scanning fixed
87
Code Red Host Infection Rate
Exponential infection rate
Measured using backscatter technique
88
Designing Fast-Spreading Worms
bull Hit-list scanningndash Time to infect first 10k hosts dominates infection timendash Solution Reconnaissance (stealthy scans etc)
bull Permutation scanningndash Observation Most scanning is redundantndash Idea Shared permutation of address space Start scanning
from own IP address Re-randomize when another infected machine is found
bull Internet-scale hit listsndash Flash worm complete infection within 30 seconds
89
Botnets
bull Bots Autonomous programs performing tasksbull Plenty of ldquobenignrdquo bots
ndash eg weatherbug
bull Botnets group of bots ndash Typically carries malicious connotationndash Large numbers of infected machinesndash Machines ldquoenlistedrdquo with infection vectors like worms
(last lecture)
bull Available for simultaneous control by a masterbull Size up to 350000 nodes (from todayrsquos paper)
90
ldquoRallyingrdquo the Botnet
bull Easy to combine worm backdoor functionalitybull Problem how to learn about successfully
infected machines
bull Optionsndash Emailndash Hard-coded email address
91
Botnet Control
bull Botnet master typically runs some IRC server on a well-known port (eg 6667)
bull Infected machine contacts botnet with pre-programmed DNS name (eg big-botde)
bull Dynamic DNS allows controller to move about freely
Infected Machine
DynamicDNS
BotnetController
(IRC server)
92
Some Defenses
93
Idea 1 Ingress Filtering
bull RFC 2827 Routers install filters to drop packets from networks that are not downstream
bull Feasible at edgesbull Difficult to configure closer to network ldquocorerdquo
20469207024 Internet
Drop all packets with source address other than 20469207024
94
Idea 2 uRPF Checks
bull Unicast Reverse Path Forwardingndash Cisco ldquoip verify unicast reverse-pathrdquo
bull Requires symmetric routing
Accept packet from interface only if forwarding table entry for source IP address matches ingress interface
100183
A10018
10015
101203
100181 24
10011 24
Strict Mode uRPF
Enabled
ldquoArdquo Routing TableDestination Next Hop1001024 Int 110018024 Int 2
100183 from wrong interface
95
Problems with uRPF
bull Asymmetric routing
96
S-BGP
bull Address-based PKI validate signaturesndash Authentication of
bull ownership for IP address blocks bull AS number bull an ASs identity and bull a BGP routers identity
ndash Use existing infrastructure (Internet registries etc)ndash Routing origination is digitally signedndash BGP updates are digitally signed
1048708 bull Route attestations A new optional BGP transitive path attribute
ndash carries digital signatures covering the routing information in updates
97
Practical Problems with S-BGP
bull Requires Public-Key Infrastructure
bull Lots of digital signatures to calculate and verifyndash Message overheadndash CPU overhead
bull Calculation expense is greatest when topology is changingndash Caching can help
bull Route aggregation is problematic (maybe thatrsquos OK)
bull Secure route withdrawals when link or node fails
bull Address ownership data out of date
bull Deployment
- Networking
- Goal of This Tutorial
- Todayrsquos Networks
- Business Models
- Billing for Internet Usage
- Net Neutrality
- Network Operations
- Point-of-Presence (PoP)
- Example Abilene Network Topology
- Another Example Backbone
- Internet Routing Overview
- Internet Routing Protocol BGP
- Two Flavors of BGP
- IPv4 Addresses Networks of Networks
- Pre-1994 Classful Addressing
- Classless Interdomain Routing (CIDR)
- Benefits of CIDR
- Growth of IP Prefixes
- 1994-1998 Linear Growth
- Around 2000 Fast Growth Resumes
- Fast growth resumes
- The Address Allocation Process
- Common Problems
- What can go wrong
- Measurement and Monitoring
- Passive vs Active Measurement
- Slide 27
- Passive Traffic Data Measurement
- Simple Network Management Protocol
- SNMP (Passive)
- Packet-level Monitoring
- Full Packet Capture (Passive)
- What is a flow
- Cisco NetFlow
- Flow Record Contents
- Aggregating Packets into Flows
- Reducing Measurement Overhead
- Packet Sampling for Flow Monitoring
- Sampling Flow-Level Sampling
- Two Main Approaches
- Packet Capture on High-Speed Links
- Characteristics of Packet Capture
- Data Measurement Repositories
- Multihoming and Traffic Engineering
- What is Multihoming
- Why Multihome
- Redundancy
- Performance
- Multihoming in IP Networks Today
- BGP or no
- Same Provider or Multiple
- Multihomed Stub One Link
- Multihomed Stub Multiple Links
- Multihomed Stub Multiple ISPs
- Multihomed Transit Network
- Interdomain Traffic Engineering
- Outbound Traffic Control
- Outbound Traffic Load Balancing
- Traffic Engineering Goals
- Managing Scale
- Achieving Predictability
- Challenges to Predictability
- Inter-AS Negotiation
- Outbound Multihoming Goals
- Inbound Traffic Control
- How many links are enough
- Problems with Multihoming in IPv4
- Slide 68
- Slide 69
- Diagnosis and Troubleshooting
- Operator Mailing List (NANOG)
- Operator Mailing List
- Routing Configuration
- Internet Business Model (Simplified)
- Peering Contracts Consistent Export
- Consistent Export
- Inconsistent Export in Practice
- Blackholes
- Security
- Security ldquoBogonrdquo Routes
- Spam Phishing etc
- Spam and Routing
- Worms and Botnets
- What is a Worm
- Example Worm Code Red
- Code Red Revisions
- Code Red Host Infection Rate
- Designing Fast-Spreading Worms
- Botnets
- ldquoRallyingrdquo the Botnet
- Botnet Control
- Some Defenses
- Idea 1 Ingress Filtering
- Idea 2 uRPF Checks
- Problems with uRPF
- S-BGP
- Practical Problems with S-BGP
-
31
Packet-level Monitoring
bull Passive monitoring to collect full packet contents (or at least headers)
bull Advantages lots of detailed informationndash Precise timing informationndash Information in packet headers
bull Disadvantages overheadndash Hard to keep up with high-speed linksndash Often requires a separate monitoring device
32
Full Packet Capture (Passive)
Example Georgia Tech OC3Mon
bull Rack-mounted PCbull Optical splitterbull Data Acquisition and
Generation (DAG) card
Source endacecom
33
What is a flow
bull Source IP addressbull Destination IP addressbull Source portbull Destination portbull Layer 3 protocol typebull TOS byte (DSCP)bull Input logical interface (ifIndex)
34
Cisco NetFlow
bull Basic output ldquoFlow recordrdquondash Most common version is v5
bull Current version (9) is being standardized in the IETF (template-based)ndash More flexible record formatndash Much easier to add new flow record types
Core Network
Collection and Aggregation
Collector (PC)Approximately 1500 bytes
20-50 flow recordsSent more frequently if traffic increases
35
Flow Record Contents
bull Source and Destination IP address and portbull Packet and byte countsbull Start and end timesbull ToS TCP flags
Basic information about the flowhellip
hellipplus information related to routing
bull Next-hop IP addressbull Source and destination ASbull Source and destination prefix
36
flow 1 flow 2 flow 3 flow 4
Aggregating Packets into Flows
bull Criteria 1 Set of packets that ldquobelong togetherrdquondash Sourcedestination IP addresses and port numbersndash Same protocol ToS bits hellip ndash Same inputoutput interfaces at a router (if known)
bull Criteria 2 Packets that are ldquocloserdquo together in timendash Maximum inter-packet spacing (eg 15 sec 30 sec)ndash Example flows 2 and 4 are different flows due to time
37
Reducing Measurement Overhead
bull Filtering on interfacendash destination prefix for a customerndash port number for an application (eg 80 for Web)
bull Sampling before insertion into flow cachendash Random deterministic or hash-based samplingndash 1-out-of-n or stratified based on packetflow sizendash Two types packet-level and flow-level
bull Aggregation after cache evictionndash packetsflows with same next-hop ASndash packetsflows destined to a particular service
38
Packet Sampling for Flow Monitoring
bull Packet sampling before flow creation (Sampled Netflow)ndash 1-out-of-m sampling of individual packets (eg m=100)ndash Create of flow records over the sampled packets
bull Reducing overheadndash Avoid per-packet overhead on (m-1)m packetsndash Avoid creating records for a large number of small flows
bull Increasing overhead (in some cases)ndash May split some long transfers into multiple flow records ndash hellip due to larger time gaps between successive packets
time
not sampled
two flowstimeout
39
Sampling Flow-Level Sampling
bull Sampling of flow records evicted from flow cachendash When evicting flows from table or when analyzing flows
bull Stratified sampling to put weight on ldquoheavyrdquo flowsndash Select all long flows and sample the short flows
bull Reduces the number of flow records ndash Still measures the vast majority of the traffic
Flow 1 40 bytesFlow 2 15580 bytesFlow 3 8196 bytesFlow 4 5350789 bytesFlow 5 532 bytesFlow 6 7432 bytes
sample with 100 probability
sample with 01 probability
sample with 10 probability
40
Two Main Approaches
bull Packet-level Monitoringndash Keep packet-level statisticsndash Examine (and potentially log) variety of packet-level
statistics Essentially anything in the packetndash Timing
bull Flow-level Monitoringndash Monitor packet-by-packet (though sometimes
sampled)ndash Keep aggregate statistics on a flow
41
Packet Capture on High-Speed Links
Example Georgia Tech ldquoOC3Monrdquo
bull Rack-mounted PCbull Optical splitterbull Data Acquisition and
Generation (DAG) card
Source endacecom
42
Characteristics of Packet Capture
bull Allows inspection on every packet on 10G links
bull Disadvantagesndash Costlyndash Requires splitting optical fibersndash Must be able to filterstore data
43
Data Measurement Repositories
bull AbileneInternet 2 Observatoryndash Configuration examplesndash SNMP datandash ISIS BGP routing data NetFlow traffic data
bull RouteViewsndash BGP updatesndash BGP table snapshots
44
Multihoming and Traffic Engineering
45
What is Multihoming
bull The use of redundant network links for the purposes of external connectivity
bull Can be achieved at many layers of the protocol stack and many places in the networkndash Multiple network interfaces in a PC
ndash An ISP with multiple upstream interfaces
bull Can refer to having multiple connections tondash The same ISP
ndash Multiple ISPs
46
Why Multihome
bull Redundancybull Availabilitybull Performancebull Cost
Interdomain traffic engineering the process by which a multihomed network configures its
network to achieve these goals
47
Redundancy
bull Maintain connectivity in the face ofndash Physical connectivity problems (fiber cut device
failures etc)ndash Failures in upstream ISP
48
Performance
bull Use multiple network links at once to achieve higher throughput than just over a single link
bull Allows incoming traffic to be load-balanced
70 of traffic30 of traffic
49
Multihoming in IP Networks Today
bull Stub AS no transit service for other ASesndash No need to use BGP
bull Multi-homed stub AS has connectivity to multiple immediate upstream ISPsndash Need BGPndash No need for a public AS numberndash No need for IP prefix allocation
bull Multi-homed transit AS connectivity to multiple ASes and transit servicendash Need BGP public AS number IP prefix allocation
50
BGP or no
bull Advantages of static routingndash Cheapersmaller routers (less true nowadays)ndash Simpler to configure
bull Advantages of BGPndash More control of your destiny (have providers stop
announcing you)ndash Fastermore intelligent selection of where to send
outbound packetsndash Better debugging of net problems (you can see the
Internet topology now)
51
Same Provider or Multiple
bull If your provider is reliable and fast and affordably and offers good tech-support you may want to multi-home initially to them via some backup path (slow is better than dead)
bull Eventually yoursquoll want to multi-home to different providers to avoid failure modes due to one providerrsquos architecture decisions
52
Multihomed Stub One Link
bull Downstream ISPrsquos routers configure default (ldquostaticrdquo) routes pointing to border router
bull Upstream ISP advertises reachability
Upstream ISP
Multiple links between same pair of routers
Default routes to ldquoborderrdquo
ldquoStubrdquoISP
53
Multihomed Stub Multiple Links
bull Use BGP to share loadbull Use private AS number (why is this OK)bull As before upstream ISP advertises prefix
Upstream ISP
Multiple links to different upstream routers
ldquoStubrdquoISP
Internal routing for ldquohot potatordquo
BGP for load balance at edge
54
Multihomed Stub Multiple ISPs
bull Many possibilitiesndash Load sharingndash Primary-backupndash Selective use of different ISPs
bull Requires BGP public AS number etc
ldquoStubrdquoISP
Upstream
ISP 1
Upstream
ISP 2
55
Multihomed Transit Network
bull BGP everywherebull Incoming and outcoming trafficbull Challenge balancing load on intradomain and egress
links given an offered traffic load
TransitISP
ISP 1
ISP 2
ISP 3
56
Interdomain Traffic Engineering
bull The process by which a network operator configures the network to achievendash Traffic load balancendash Redundancy (primarybackup) etc
bull Two tasksndash Outbound traffic controlndash Inbound traffic control
bull Key Problems Predictability and Scalability
57
Outbound Traffic Control
bull Easier to control than inbound trafficndash Destination-based routing sender determines where
the packets go
bull Control over next-hop AS onlyndash Cannot control selection of the entire path
Provider 1 Provider 2
Control with local preference
58
Outbound Traffic Load Balancingbull Control routes to provider per-prefix
ndash Assign local preference across destination prefixesndash Change the local preference assignments over time
bull Useful inputs to load balancingndash End-to-end path performance datandash Outbound traffic statistics per destination prefix
bull Challenge Getting from traffic volumes to groups of prefixes that should be assigned to each link
Premise of ldquointelligent route controlrdquo preoducts
59
Traffic Engineering Goals
bull Predictabilityndash Ensure the BGP decision process is deterministicndash Assume that BGP updates are (relatively) stable
bull Limit overhead introduced by routing changesndash Minimize frequency of changes to routing policiesndash Limit number of prefixes affected by changes
bull Limit impact on how traffic enters the networkndash Avoid new routes that might change neighborrsquos mindndash Select route with same attributes or at least path length
60
Managing Scalebull Destination prefixes
ndash More than 90000 destination prefixes
bull Donrsquot want to have per-prefix routing policies
ndash Small fraction of prefixes contribute most of the traffic
bull Focus on the small number of heavy hitters
ndash Define routing policies for selected prefixes
bull Routing choicesndash About 27000 unique ldquorouting choicesrdquo
bull Help in reducing the scale of the problem
ndash Small fraction of ldquorouting choicesrdquo contribute most traffic
bull Focus on the very small number of ldquorouting choicesrdquo
ndash Define routing policies on common attributes
61
Achieving Predictability
bull Route prediction with static analysisndash Helpful to know effects before deploymentndash Static analysis can help
TopologyBGP policy
configuration
eBGP routes
Offered traffic
BGP routingmodel
Flow of traffic through the network
62
Challenges to Predictabilitybull For transit ISPs effects on incoming traffic
ndash Lack of coordination strikes again
63
ldquoHot Potatordquo routing
Inter-AS Negotiation
bull Coordination aids predictabilityndash Negotiate where to sendndash Inbound and outboundndash Mutual benefits
bull How to implementndash What info to exchangendash Protecting privacyndash How to prioritize choicesndash How to prevent cheating
Destination 2
Destination 1
multiplepeeringpoints
Provider A
Provider B
64
Outbound Multihoming Goals
bull Redundancyndash Dynamic routing will failover to backup link
bull Performancendash Select provider with best performance per prefix
ndash Requires active probing
bull Costndash Select provider per prefix over time to minimize the
total financial cost
65
Inbound Traffic Control
bull More difficult no control over neighborsrsquo decisions
bull Three common techniques (previously discussed)ndash AS path prependingndash Communities and local preferencendash Prefix splitting
How does todayrsquos paper (MONET) control inbound traffic
66
How many links are enough
K upstream ISPs
Not much benefit beyond 4 ISPs
Akella et al ldquoPerformance Benefits of Multihomingrdquo SIGCOMM 2003
67
Problems with Multihoming in IPv4
bull Routing table growthndash Provider-based addressingndash Advertising prefix out multiple ISPs ndash canrsquot aggregate
bull Poor control over inbound trafficndash Existing mechanisms do not allow hosts to control
inbound traffic
68
GeorgiaTech
Internet Routing Overview
bull Intradomain (ie ldquointra-ASrdquo) routingbull Interdomain routing
Comcast
Abilene
ATampT Cogent
Autonomous Systems (ASes)
69
Configuration Problems ldquoAS 7007rdquoldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
UUNet
Florida InternetBarn
Sprint
70
Diagnosis and Troubleshootingldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
ldquoMicrosofts websites were offline for up to 23 hoursbecause of a [router] misconfigurationhellipit took nearly a day to determine what was wrong and undo the changesrdquo -- wiredcom January 25 2001
ldquoWorldCom Inchellipsuffered a widespread outage on its Internet backbone that affected roughly 20 percent of its US customer base The network problemshellipaffected millions of computer users worldwide A spokeswoman attributed the outage to a route table issue -- cnncom October 3 2002
A number of Covad customers went out from 5pm today due to supposedly a DDOS (distributed denial of service attack) on a key Level3 data center which later was described as a route leak (misconfiguration)ldquo
-- dslreportscom February 23 2004
71
Operator Mailing List (NANOG)Date Mon 18 Oct 2004 091515 -0700Subject Level 3 US east coast issues
Level 3 experiencing widespread unspecified routing issues on the US east coast Master ticket 1086844 Anyone have more specific information
Date Mon 18 Oct 2004 122034 -0400 (EDT)Subject Re Level 3 US east coast issues
Level 3 is currently experiencing a backbone outage causing routing instability and packet loss We are working to restore and will be sending hourly updateshellip
72
Operator Mailing List
0102030405060708090
Filtering RouteLeaks
RouteHijacks
RouteInstability
RoutingLoops
Blackholes
Occurr
ences o
ver
10 Y
ears
1995-1997 1998-2001 2001-2004
Note Only includes problems openly discussed on this list
Compare 83 power outages 1 fire
73
Routing Configuration
Ranking route selection
Dissemination internal route advertisement
Filtering route advertisement
Customer
Competitor
Primary
Backup
74
Internet Business Model (Simplified)
bull CustomerProvider One AS pays another for reachability to some set of destinations
bull ldquoSettlement-freerdquo Peering Bartering Two ASes exchange routes with one another
Provider
Peer
Customer
Preferences implemented with local preference manipulation
Destination
Pay to use
Get paid to use
Free to use
75
Peering Contracts Consistent Export
bull Rules of settlement-free peeringndash Advertise routes at all peering pointsndash Advertised routes must have equal ldquoAS path lengthrdquo
Sprint
ATampT
Enables ldquohot potatordquo routing
ldquoequally goodrdquoroutes
76
Consistent Export
bull Malicedeceptionbull iBGP signaling partitionbull Inconsistent export policy
neighbor 10123route-map PEER permit 10 set prepend 123
neighbor 10456route-map PEER permit 10 set prepend 123 123
Possible Causes
10123 456 1
10456 456 2
Neighbor AS Export
1 1 123
Export Clause Prepend
2 1 123 123
Two different Export Policies
77
Inconsistent Export in Practice
Feamster et al ldquoBorderGuard Detecting Cold Potatoes from Peersrdquo ACM IMC October 2004
78
Blackholes
Date Thu 18 Jul 2002 060510 -0400 (EDT)From Chad Oleary ltcolpoboxcomgtSubject Re problems with 701To ltnanogmeritedugt
Were starting to see the same issues with UUNet again Anyone elseseeing this Trying to reach Qwest
traceroute to 631461901 (631461901) 30 hops max 38 byte packets 1 esc-lp2-gwe-solutionscorpcom (631182201) 1167 ms 1163 ms 1142 ms 2 500Serial2-10GW1TPA2ALTERNET (1571301499) 1097 ms 1059 ms 1044 ms 3 161at-1-0-0XL4ATL1ALTERNET (1526381190) 13839 ms 14108 ms 16638 ms 4 0so-3-1-0XL2ATL5ALTERNET (152630238) 14370 ms 14587 ms 14553 ms 5 POS7-0BR2ATL5ALTERNET (1526382193) 13928 ms 14099 ms 14053 ms 6 7 hellip
79
Security
80
Security ldquoBogonrdquo Routes
Feamster et al ldquoAn Empirical Study of lsquoBogonrsquo Route Advertisementsrdquo ACM CCR January 2005
81
Spam Phishing etc
bull Unsolicited commercial emailbull As of about August 2008 estimates indicate that
about 95 of all email is spambull Common spam filtering techniques
ndash Content-based filtersndash DNS Blacklist (DNSBL) lookups Significant fraction of
todayrsquos DNS traffic
Can IP addresses from which spam is received be spoofed
82
Spam and Routing
83
Worms and Botnets
84
What is a Worm
bull Code that replicates and propagates across the networkndash Often carries a ldquopayloadrdquo
bull Usually spread via exploiting flaws in open servicesndash ldquoVirusesrdquo require user action to spread
bull First worm Robert Morris November 1988ndash 6-10 of all Internet hosts infected ()
bull Many more since but none on that scale until July 2001
85
Example Worm Code Red
bull Initial version July 13 2001
bull Exploited known ISAPI vulnerability in Microsoft IIS Web servers
bull 1st through 20th of each month spread20th through end of each month attack
bull Payload Web site defacementbull Scanning Random IP addressesbull Bug failure to seed random number generator
86
Code Red Revisions
bull Released July 19 2001
bull Payload flooding attack on wwwwhitehousegovndash Attack was mounted at the IP address of the Web site
bull Bug died after 20th of each month
bull Random number generator for IP scanning fixed
87
Code Red Host Infection Rate
Exponential infection rate
Measured using backscatter technique
88
Designing Fast-Spreading Worms
bull Hit-list scanningndash Time to infect first 10k hosts dominates infection timendash Solution Reconnaissance (stealthy scans etc)
bull Permutation scanningndash Observation Most scanning is redundantndash Idea Shared permutation of address space Start scanning
from own IP address Re-randomize when another infected machine is found
bull Internet-scale hit listsndash Flash worm complete infection within 30 seconds
89
Botnets
bull Bots Autonomous programs performing tasksbull Plenty of ldquobenignrdquo bots
ndash eg weatherbug
bull Botnets group of bots ndash Typically carries malicious connotationndash Large numbers of infected machinesndash Machines ldquoenlistedrdquo with infection vectors like worms
(last lecture)
bull Available for simultaneous control by a masterbull Size up to 350000 nodes (from todayrsquos paper)
90
ldquoRallyingrdquo the Botnet
bull Easy to combine worm backdoor functionalitybull Problem how to learn about successfully
infected machines
bull Optionsndash Emailndash Hard-coded email address
91
Botnet Control
bull Botnet master typically runs some IRC server on a well-known port (eg 6667)
bull Infected machine contacts botnet with pre-programmed DNS name (eg big-botde)
bull Dynamic DNS allows controller to move about freely
Infected Machine
DynamicDNS
BotnetController
(IRC server)
92
Some Defenses
93
Idea 1 Ingress Filtering
bull RFC 2827 Routers install filters to drop packets from networks that are not downstream
bull Feasible at edgesbull Difficult to configure closer to network ldquocorerdquo
20469207024 Internet
Drop all packets with source address other than 20469207024
94
Idea 2 uRPF Checks
bull Unicast Reverse Path Forwardingndash Cisco ldquoip verify unicast reverse-pathrdquo
bull Requires symmetric routing
Accept packet from interface only if forwarding table entry for source IP address matches ingress interface
100183
A10018
10015
101203
100181 24
10011 24
Strict Mode uRPF
Enabled
ldquoArdquo Routing TableDestination Next Hop1001024 Int 110018024 Int 2
100183 from wrong interface
95
Problems with uRPF
bull Asymmetric routing
96
S-BGP
bull Address-based PKI validate signaturesndash Authentication of
bull ownership for IP address blocks bull AS number bull an ASs identity and bull a BGP routers identity
ndash Use existing infrastructure (Internet registries etc)ndash Routing origination is digitally signedndash BGP updates are digitally signed
1048708 bull Route attestations A new optional BGP transitive path attribute
ndash carries digital signatures covering the routing information in updates
97
Practical Problems with S-BGP
bull Requires Public-Key Infrastructure
bull Lots of digital signatures to calculate and verifyndash Message overheadndash CPU overhead
bull Calculation expense is greatest when topology is changingndash Caching can help
bull Route aggregation is problematic (maybe thatrsquos OK)
bull Secure route withdrawals when link or node fails
bull Address ownership data out of date
bull Deployment
- Networking
- Goal of This Tutorial
- Todayrsquos Networks
- Business Models
- Billing for Internet Usage
- Net Neutrality
- Network Operations
- Point-of-Presence (PoP)
- Example Abilene Network Topology
- Another Example Backbone
- Internet Routing Overview
- Internet Routing Protocol BGP
- Two Flavors of BGP
- IPv4 Addresses Networks of Networks
- Pre-1994 Classful Addressing
- Classless Interdomain Routing (CIDR)
- Benefits of CIDR
- Growth of IP Prefixes
- 1994-1998 Linear Growth
- Around 2000 Fast Growth Resumes
- Fast growth resumes
- The Address Allocation Process
- Common Problems
- What can go wrong
- Measurement and Monitoring
- Passive vs Active Measurement
- Slide 27
- Passive Traffic Data Measurement
- Simple Network Management Protocol
- SNMP (Passive)
- Packet-level Monitoring
- Full Packet Capture (Passive)
- What is a flow
- Cisco NetFlow
- Flow Record Contents
- Aggregating Packets into Flows
- Reducing Measurement Overhead
- Packet Sampling for Flow Monitoring
- Sampling Flow-Level Sampling
- Two Main Approaches
- Packet Capture on High-Speed Links
- Characteristics of Packet Capture
- Data Measurement Repositories
- Multihoming and Traffic Engineering
- What is Multihoming
- Why Multihome
- Redundancy
- Performance
- Multihoming in IP Networks Today
- BGP or no
- Same Provider or Multiple
- Multihomed Stub One Link
- Multihomed Stub Multiple Links
- Multihomed Stub Multiple ISPs
- Multihomed Transit Network
- Interdomain Traffic Engineering
- Outbound Traffic Control
- Outbound Traffic Load Balancing
- Traffic Engineering Goals
- Managing Scale
- Achieving Predictability
- Challenges to Predictability
- Inter-AS Negotiation
- Outbound Multihoming Goals
- Inbound Traffic Control
- How many links are enough
- Problems with Multihoming in IPv4
- Slide 68
- Slide 69
- Diagnosis and Troubleshooting
- Operator Mailing List (NANOG)
- Operator Mailing List
- Routing Configuration
- Internet Business Model (Simplified)
- Peering Contracts Consistent Export
- Consistent Export
- Inconsistent Export in Practice
- Blackholes
- Security
- Security ldquoBogonrdquo Routes
- Spam Phishing etc
- Spam and Routing
- Worms and Botnets
- What is a Worm
- Example Worm Code Red
- Code Red Revisions
- Code Red Host Infection Rate
- Designing Fast-Spreading Worms
- Botnets
- ldquoRallyingrdquo the Botnet
- Botnet Control
- Some Defenses
- Idea 1 Ingress Filtering
- Idea 2 uRPF Checks
- Problems with uRPF
- S-BGP
- Practical Problems with S-BGP
-
32
Full Packet Capture (Passive)
Example Georgia Tech OC3Mon
bull Rack-mounted PCbull Optical splitterbull Data Acquisition and
Generation (DAG) card
Source endacecom
33
What is a flow
bull Source IP addressbull Destination IP addressbull Source portbull Destination portbull Layer 3 protocol typebull TOS byte (DSCP)bull Input logical interface (ifIndex)
34
Cisco NetFlow
bull Basic output ldquoFlow recordrdquondash Most common version is v5
bull Current version (9) is being standardized in the IETF (template-based)ndash More flexible record formatndash Much easier to add new flow record types
Core Network
Collection and Aggregation
Collector (PC)Approximately 1500 bytes
20-50 flow recordsSent more frequently if traffic increases
35
Flow Record Contents
bull Source and Destination IP address and portbull Packet and byte countsbull Start and end timesbull ToS TCP flags
Basic information about the flowhellip
hellipplus information related to routing
bull Next-hop IP addressbull Source and destination ASbull Source and destination prefix
36
flow 1 flow 2 flow 3 flow 4
Aggregating Packets into Flows
bull Criteria 1 Set of packets that ldquobelong togetherrdquondash Sourcedestination IP addresses and port numbersndash Same protocol ToS bits hellip ndash Same inputoutput interfaces at a router (if known)
bull Criteria 2 Packets that are ldquocloserdquo together in timendash Maximum inter-packet spacing (eg 15 sec 30 sec)ndash Example flows 2 and 4 are different flows due to time
37
Reducing Measurement Overhead
bull Filtering on interfacendash destination prefix for a customerndash port number for an application (eg 80 for Web)
bull Sampling before insertion into flow cachendash Random deterministic or hash-based samplingndash 1-out-of-n or stratified based on packetflow sizendash Two types packet-level and flow-level
bull Aggregation after cache evictionndash packetsflows with same next-hop ASndash packetsflows destined to a particular service
38
Packet Sampling for Flow Monitoring
bull Packet sampling before flow creation (Sampled Netflow)ndash 1-out-of-m sampling of individual packets (eg m=100)ndash Create of flow records over the sampled packets
bull Reducing overheadndash Avoid per-packet overhead on (m-1)m packetsndash Avoid creating records for a large number of small flows
bull Increasing overhead (in some cases)ndash May split some long transfers into multiple flow records ndash hellip due to larger time gaps between successive packets
time
not sampled
two flowstimeout
39
Sampling Flow-Level Sampling
bull Sampling of flow records evicted from flow cachendash When evicting flows from table or when analyzing flows
bull Stratified sampling to put weight on ldquoheavyrdquo flowsndash Select all long flows and sample the short flows
bull Reduces the number of flow records ndash Still measures the vast majority of the traffic
Flow 1 40 bytesFlow 2 15580 bytesFlow 3 8196 bytesFlow 4 5350789 bytesFlow 5 532 bytesFlow 6 7432 bytes
sample with 100 probability
sample with 01 probability
sample with 10 probability
40
Two Main Approaches
bull Packet-level Monitoringndash Keep packet-level statisticsndash Examine (and potentially log) variety of packet-level
statistics Essentially anything in the packetndash Timing
bull Flow-level Monitoringndash Monitor packet-by-packet (though sometimes
sampled)ndash Keep aggregate statistics on a flow
41
Packet Capture on High-Speed Links
Example Georgia Tech ldquoOC3Monrdquo
bull Rack-mounted PCbull Optical splitterbull Data Acquisition and
Generation (DAG) card
Source endacecom
42
Characteristics of Packet Capture
bull Allows inspection on every packet on 10G links
bull Disadvantagesndash Costlyndash Requires splitting optical fibersndash Must be able to filterstore data
43
Data Measurement Repositories
bull AbileneInternet 2 Observatoryndash Configuration examplesndash SNMP datandash ISIS BGP routing data NetFlow traffic data
bull RouteViewsndash BGP updatesndash BGP table snapshots
44
Multihoming and Traffic Engineering
45
What is Multihoming
bull The use of redundant network links for the purposes of external connectivity
bull Can be achieved at many layers of the protocol stack and many places in the networkndash Multiple network interfaces in a PC
ndash An ISP with multiple upstream interfaces
bull Can refer to having multiple connections tondash The same ISP
ndash Multiple ISPs
46
Why Multihome
bull Redundancybull Availabilitybull Performancebull Cost
Interdomain traffic engineering the process by which a multihomed network configures its
network to achieve these goals
47
Redundancy
bull Maintain connectivity in the face ofndash Physical connectivity problems (fiber cut device
failures etc)ndash Failures in upstream ISP
48
Performance
bull Use multiple network links at once to achieve higher throughput than just over a single link
bull Allows incoming traffic to be load-balanced
70 of traffic30 of traffic
49
Multihoming in IP Networks Today
bull Stub AS no transit service for other ASesndash No need to use BGP
bull Multi-homed stub AS has connectivity to multiple immediate upstream ISPsndash Need BGPndash No need for a public AS numberndash No need for IP prefix allocation
bull Multi-homed transit AS connectivity to multiple ASes and transit servicendash Need BGP public AS number IP prefix allocation
50
BGP or no
bull Advantages of static routingndash Cheapersmaller routers (less true nowadays)ndash Simpler to configure
bull Advantages of BGPndash More control of your destiny (have providers stop
announcing you)ndash Fastermore intelligent selection of where to send
outbound packetsndash Better debugging of net problems (you can see the
Internet topology now)
51
Same Provider or Multiple
bull If your provider is reliable and fast and affordably and offers good tech-support you may want to multi-home initially to them via some backup path (slow is better than dead)
bull Eventually yoursquoll want to multi-home to different providers to avoid failure modes due to one providerrsquos architecture decisions
52
Multihomed Stub One Link
bull Downstream ISPrsquos routers configure default (ldquostaticrdquo) routes pointing to border router
bull Upstream ISP advertises reachability
Upstream ISP
Multiple links between same pair of routers
Default routes to ldquoborderrdquo
ldquoStubrdquoISP
53
Multihomed Stub Multiple Links
bull Use BGP to share loadbull Use private AS number (why is this OK)bull As before upstream ISP advertises prefix
Upstream ISP
Multiple links to different upstream routers
ldquoStubrdquoISP
Internal routing for ldquohot potatordquo
BGP for load balance at edge
54
Multihomed Stub Multiple ISPs
bull Many possibilitiesndash Load sharingndash Primary-backupndash Selective use of different ISPs
bull Requires BGP public AS number etc
ldquoStubrdquoISP
Upstream
ISP 1
Upstream
ISP 2
55
Multihomed Transit Network
bull BGP everywherebull Incoming and outcoming trafficbull Challenge balancing load on intradomain and egress
links given an offered traffic load
TransitISP
ISP 1
ISP 2
ISP 3
56
Interdomain Traffic Engineering
bull The process by which a network operator configures the network to achievendash Traffic load balancendash Redundancy (primarybackup) etc
bull Two tasksndash Outbound traffic controlndash Inbound traffic control
bull Key Problems Predictability and Scalability
57
Outbound Traffic Control
bull Easier to control than inbound trafficndash Destination-based routing sender determines where
the packets go
bull Control over next-hop AS onlyndash Cannot control selection of the entire path
Provider 1 Provider 2
Control with local preference
58
Outbound Traffic Load Balancingbull Control routes to provider per-prefix
ndash Assign local preference across destination prefixesndash Change the local preference assignments over time
bull Useful inputs to load balancingndash End-to-end path performance datandash Outbound traffic statistics per destination prefix
bull Challenge Getting from traffic volumes to groups of prefixes that should be assigned to each link
Premise of ldquointelligent route controlrdquo preoducts
59
Traffic Engineering Goals
bull Predictabilityndash Ensure the BGP decision process is deterministicndash Assume that BGP updates are (relatively) stable
bull Limit overhead introduced by routing changesndash Minimize frequency of changes to routing policiesndash Limit number of prefixes affected by changes
bull Limit impact on how traffic enters the networkndash Avoid new routes that might change neighborrsquos mindndash Select route with same attributes or at least path length
60
Managing Scalebull Destination prefixes
ndash More than 90000 destination prefixes
bull Donrsquot want to have per-prefix routing policies
ndash Small fraction of prefixes contribute most of the traffic
bull Focus on the small number of heavy hitters
ndash Define routing policies for selected prefixes
bull Routing choicesndash About 27000 unique ldquorouting choicesrdquo
bull Help in reducing the scale of the problem
ndash Small fraction of ldquorouting choicesrdquo contribute most traffic
bull Focus on the very small number of ldquorouting choicesrdquo
ndash Define routing policies on common attributes
61
Achieving Predictability
bull Route prediction with static analysisndash Helpful to know effects before deploymentndash Static analysis can help
TopologyBGP policy
configuration
eBGP routes
Offered traffic
BGP routingmodel
Flow of traffic through the network
62
Challenges to Predictabilitybull For transit ISPs effects on incoming traffic
ndash Lack of coordination strikes again
63
ldquoHot Potatordquo routing
Inter-AS Negotiation
bull Coordination aids predictabilityndash Negotiate where to sendndash Inbound and outboundndash Mutual benefits
bull How to implementndash What info to exchangendash Protecting privacyndash How to prioritize choicesndash How to prevent cheating
Destination 2
Destination 1
multiplepeeringpoints
Provider A
Provider B
64
Outbound Multihoming Goals
bull Redundancyndash Dynamic routing will failover to backup link
bull Performancendash Select provider with best performance per prefix
ndash Requires active probing
bull Costndash Select provider per prefix over time to minimize the
total financial cost
65
Inbound Traffic Control
bull More difficult no control over neighborsrsquo decisions
bull Three common techniques (previously discussed)ndash AS path prependingndash Communities and local preferencendash Prefix splitting
How does todayrsquos paper (MONET) control inbound traffic
66
How many links are enough
K upstream ISPs
Not much benefit beyond 4 ISPs
Akella et al ldquoPerformance Benefits of Multihomingrdquo SIGCOMM 2003
67
Problems with Multihoming in IPv4
bull Routing table growthndash Provider-based addressingndash Advertising prefix out multiple ISPs ndash canrsquot aggregate
bull Poor control over inbound trafficndash Existing mechanisms do not allow hosts to control
inbound traffic
68
GeorgiaTech
Internet Routing Overview
bull Intradomain (ie ldquointra-ASrdquo) routingbull Interdomain routing
Comcast
Abilene
ATampT Cogent
Autonomous Systems (ASes)
69
Configuration Problems ldquoAS 7007rdquoldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
UUNet
Florida InternetBarn
Sprint
70
Diagnosis and Troubleshootingldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
ldquoMicrosofts websites were offline for up to 23 hoursbecause of a [router] misconfigurationhellipit took nearly a day to determine what was wrong and undo the changesrdquo -- wiredcom January 25 2001
ldquoWorldCom Inchellipsuffered a widespread outage on its Internet backbone that affected roughly 20 percent of its US customer base The network problemshellipaffected millions of computer users worldwide A spokeswoman attributed the outage to a route table issue -- cnncom October 3 2002
A number of Covad customers went out from 5pm today due to supposedly a DDOS (distributed denial of service attack) on a key Level3 data center which later was described as a route leak (misconfiguration)ldquo
-- dslreportscom February 23 2004
71
Operator Mailing List (NANOG)Date Mon 18 Oct 2004 091515 -0700Subject Level 3 US east coast issues
Level 3 experiencing widespread unspecified routing issues on the US east coast Master ticket 1086844 Anyone have more specific information
Date Mon 18 Oct 2004 122034 -0400 (EDT)Subject Re Level 3 US east coast issues
Level 3 is currently experiencing a backbone outage causing routing instability and packet loss We are working to restore and will be sending hourly updateshellip
72
Operator Mailing List
0102030405060708090
Filtering RouteLeaks
RouteHijacks
RouteInstability
RoutingLoops
Blackholes
Occurr
ences o
ver
10 Y
ears
1995-1997 1998-2001 2001-2004
Note Only includes problems openly discussed on this list
Compare 83 power outages 1 fire
73
Routing Configuration
Ranking route selection
Dissemination internal route advertisement
Filtering route advertisement
Customer
Competitor
Primary
Backup
74
Internet Business Model (Simplified)
bull CustomerProvider One AS pays another for reachability to some set of destinations
bull ldquoSettlement-freerdquo Peering Bartering Two ASes exchange routes with one another
Provider
Peer
Customer
Preferences implemented with local preference manipulation
Destination
Pay to use
Get paid to use
Free to use
75
Peering Contracts Consistent Export
bull Rules of settlement-free peeringndash Advertise routes at all peering pointsndash Advertised routes must have equal ldquoAS path lengthrdquo
Sprint
ATampT
Enables ldquohot potatordquo routing
ldquoequally goodrdquoroutes
76
Consistent Export
bull Malicedeceptionbull iBGP signaling partitionbull Inconsistent export policy
neighbor 10123route-map PEER permit 10 set prepend 123
neighbor 10456route-map PEER permit 10 set prepend 123 123
Possible Causes
10123 456 1
10456 456 2
Neighbor AS Export
1 1 123
Export Clause Prepend
2 1 123 123
Two different Export Policies
77
Inconsistent Export in Practice
Feamster et al ldquoBorderGuard Detecting Cold Potatoes from Peersrdquo ACM IMC October 2004
78
Blackholes
Date Thu 18 Jul 2002 060510 -0400 (EDT)From Chad Oleary ltcolpoboxcomgtSubject Re problems with 701To ltnanogmeritedugt
Were starting to see the same issues with UUNet again Anyone elseseeing this Trying to reach Qwest
traceroute to 631461901 (631461901) 30 hops max 38 byte packets 1 esc-lp2-gwe-solutionscorpcom (631182201) 1167 ms 1163 ms 1142 ms 2 500Serial2-10GW1TPA2ALTERNET (1571301499) 1097 ms 1059 ms 1044 ms 3 161at-1-0-0XL4ATL1ALTERNET (1526381190) 13839 ms 14108 ms 16638 ms 4 0so-3-1-0XL2ATL5ALTERNET (152630238) 14370 ms 14587 ms 14553 ms 5 POS7-0BR2ATL5ALTERNET (1526382193) 13928 ms 14099 ms 14053 ms 6 7 hellip
79
Security
80
Security ldquoBogonrdquo Routes
Feamster et al ldquoAn Empirical Study of lsquoBogonrsquo Route Advertisementsrdquo ACM CCR January 2005
81
Spam Phishing etc
bull Unsolicited commercial emailbull As of about August 2008 estimates indicate that
about 95 of all email is spambull Common spam filtering techniques
ndash Content-based filtersndash DNS Blacklist (DNSBL) lookups Significant fraction of
todayrsquos DNS traffic
Can IP addresses from which spam is received be spoofed
82
Spam and Routing
83
Worms and Botnets
84
What is a Worm
bull Code that replicates and propagates across the networkndash Often carries a ldquopayloadrdquo
bull Usually spread via exploiting flaws in open servicesndash ldquoVirusesrdquo require user action to spread
bull First worm Robert Morris November 1988ndash 6-10 of all Internet hosts infected ()
bull Many more since but none on that scale until July 2001
85
Example Worm Code Red
bull Initial version July 13 2001
bull Exploited known ISAPI vulnerability in Microsoft IIS Web servers
bull 1st through 20th of each month spread20th through end of each month attack
bull Payload Web site defacementbull Scanning Random IP addressesbull Bug failure to seed random number generator
86
Code Red Revisions
bull Released July 19 2001
bull Payload flooding attack on wwwwhitehousegovndash Attack was mounted at the IP address of the Web site
bull Bug died after 20th of each month
bull Random number generator for IP scanning fixed
87
Code Red Host Infection Rate
Exponential infection rate
Measured using backscatter technique
88
Designing Fast-Spreading Worms
bull Hit-list scanningndash Time to infect first 10k hosts dominates infection timendash Solution Reconnaissance (stealthy scans etc)
bull Permutation scanningndash Observation Most scanning is redundantndash Idea Shared permutation of address space Start scanning
from own IP address Re-randomize when another infected machine is found
bull Internet-scale hit listsndash Flash worm complete infection within 30 seconds
89
Botnets
bull Bots Autonomous programs performing tasksbull Plenty of ldquobenignrdquo bots
ndash eg weatherbug
bull Botnets group of bots ndash Typically carries malicious connotationndash Large numbers of infected machinesndash Machines ldquoenlistedrdquo with infection vectors like worms
(last lecture)
bull Available for simultaneous control by a masterbull Size up to 350000 nodes (from todayrsquos paper)
90
ldquoRallyingrdquo the Botnet
bull Easy to combine worm backdoor functionalitybull Problem how to learn about successfully
infected machines
bull Optionsndash Emailndash Hard-coded email address
91
Botnet Control
bull Botnet master typically runs some IRC server on a well-known port (eg 6667)
bull Infected machine contacts botnet with pre-programmed DNS name (eg big-botde)
bull Dynamic DNS allows controller to move about freely
Infected Machine
DynamicDNS
BotnetController
(IRC server)
92
Some Defenses
93
Idea 1 Ingress Filtering
bull RFC 2827 Routers install filters to drop packets from networks that are not downstream
bull Feasible at edgesbull Difficult to configure closer to network ldquocorerdquo
20469207024 Internet
Drop all packets with source address other than 20469207024
94
Idea 2 uRPF Checks
bull Unicast Reverse Path Forwardingndash Cisco ldquoip verify unicast reverse-pathrdquo
bull Requires symmetric routing
Accept packet from interface only if forwarding table entry for source IP address matches ingress interface
100183
A10018
10015
101203
100181 24
10011 24
Strict Mode uRPF
Enabled
ldquoArdquo Routing TableDestination Next Hop1001024 Int 110018024 Int 2
100183 from wrong interface
95
Problems with uRPF
bull Asymmetric routing
96
S-BGP
bull Address-based PKI validate signaturesndash Authentication of
bull ownership for IP address blocks bull AS number bull an ASs identity and bull a BGP routers identity
ndash Use existing infrastructure (Internet registries etc)ndash Routing origination is digitally signedndash BGP updates are digitally signed
1048708 bull Route attestations A new optional BGP transitive path attribute
ndash carries digital signatures covering the routing information in updates
97
Practical Problems with S-BGP
bull Requires Public-Key Infrastructure
bull Lots of digital signatures to calculate and verifyndash Message overheadndash CPU overhead
bull Calculation expense is greatest when topology is changingndash Caching can help
bull Route aggregation is problematic (maybe thatrsquos OK)
bull Secure route withdrawals when link or node fails
bull Address ownership data out of date
bull Deployment
- Networking
- Goal of This Tutorial
- Todayrsquos Networks
- Business Models
- Billing for Internet Usage
- Net Neutrality
- Network Operations
- Point-of-Presence (PoP)
- Example Abilene Network Topology
- Another Example Backbone
- Internet Routing Overview
- Internet Routing Protocol BGP
- Two Flavors of BGP
- IPv4 Addresses Networks of Networks
- Pre-1994 Classful Addressing
- Classless Interdomain Routing (CIDR)
- Benefits of CIDR
- Growth of IP Prefixes
- 1994-1998 Linear Growth
- Around 2000 Fast Growth Resumes
- Fast growth resumes
- The Address Allocation Process
- Common Problems
- What can go wrong
- Measurement and Monitoring
- Passive vs Active Measurement
- Slide 27
- Passive Traffic Data Measurement
- Simple Network Management Protocol
- SNMP (Passive)
- Packet-level Monitoring
- Full Packet Capture (Passive)
- What is a flow
- Cisco NetFlow
- Flow Record Contents
- Aggregating Packets into Flows
- Reducing Measurement Overhead
- Packet Sampling for Flow Monitoring
- Sampling Flow-Level Sampling
- Two Main Approaches
- Packet Capture on High-Speed Links
- Characteristics of Packet Capture
- Data Measurement Repositories
- Multihoming and Traffic Engineering
- What is Multihoming
- Why Multihome
- Redundancy
- Performance
- Multihoming in IP Networks Today
- BGP or no
- Same Provider or Multiple
- Multihomed Stub One Link
- Multihomed Stub Multiple Links
- Multihomed Stub Multiple ISPs
- Multihomed Transit Network
- Interdomain Traffic Engineering
- Outbound Traffic Control
- Outbound Traffic Load Balancing
- Traffic Engineering Goals
- Managing Scale
- Achieving Predictability
- Challenges to Predictability
- Inter-AS Negotiation
- Outbound Multihoming Goals
- Inbound Traffic Control
- How many links are enough
- Problems with Multihoming in IPv4
- Slide 68
- Slide 69
- Diagnosis and Troubleshooting
- Operator Mailing List (NANOG)
- Operator Mailing List
- Routing Configuration
- Internet Business Model (Simplified)
- Peering Contracts Consistent Export
- Consistent Export
- Inconsistent Export in Practice
- Blackholes
- Security
- Security ldquoBogonrdquo Routes
- Spam Phishing etc
- Spam and Routing
- Worms and Botnets
- What is a Worm
- Example Worm Code Red
- Code Red Revisions
- Code Red Host Infection Rate
- Designing Fast-Spreading Worms
- Botnets
- ldquoRallyingrdquo the Botnet
- Botnet Control
- Some Defenses
- Idea 1 Ingress Filtering
- Idea 2 uRPF Checks
- Problems with uRPF
- S-BGP
- Practical Problems with S-BGP
-
33
What is a flow
bull Source IP addressbull Destination IP addressbull Source portbull Destination portbull Layer 3 protocol typebull TOS byte (DSCP)bull Input logical interface (ifIndex)
34
Cisco NetFlow
bull Basic output ldquoFlow recordrdquondash Most common version is v5
bull Current version (9) is being standardized in the IETF (template-based)ndash More flexible record formatndash Much easier to add new flow record types
Core Network
Collection and Aggregation
Collector (PC)Approximately 1500 bytes
20-50 flow recordsSent more frequently if traffic increases
35
Flow Record Contents
bull Source and Destination IP address and portbull Packet and byte countsbull Start and end timesbull ToS TCP flags
Basic information about the flowhellip
hellipplus information related to routing
bull Next-hop IP addressbull Source and destination ASbull Source and destination prefix
36
flow 1 flow 2 flow 3 flow 4
Aggregating Packets into Flows
bull Criteria 1 Set of packets that ldquobelong togetherrdquondash Sourcedestination IP addresses and port numbersndash Same protocol ToS bits hellip ndash Same inputoutput interfaces at a router (if known)
bull Criteria 2 Packets that are ldquocloserdquo together in timendash Maximum inter-packet spacing (eg 15 sec 30 sec)ndash Example flows 2 and 4 are different flows due to time
37
Reducing Measurement Overhead
bull Filtering on interfacendash destination prefix for a customerndash port number for an application (eg 80 for Web)
bull Sampling before insertion into flow cachendash Random deterministic or hash-based samplingndash 1-out-of-n or stratified based on packetflow sizendash Two types packet-level and flow-level
bull Aggregation after cache evictionndash packetsflows with same next-hop ASndash packetsflows destined to a particular service
38
Packet Sampling for Flow Monitoring
bull Packet sampling before flow creation (Sampled Netflow)ndash 1-out-of-m sampling of individual packets (eg m=100)ndash Create of flow records over the sampled packets
bull Reducing overheadndash Avoid per-packet overhead on (m-1)m packetsndash Avoid creating records for a large number of small flows
bull Increasing overhead (in some cases)ndash May split some long transfers into multiple flow records ndash hellip due to larger time gaps between successive packets
time
not sampled
two flowstimeout
39
Sampling Flow-Level Sampling
bull Sampling of flow records evicted from flow cachendash When evicting flows from table or when analyzing flows
bull Stratified sampling to put weight on ldquoheavyrdquo flowsndash Select all long flows and sample the short flows
bull Reduces the number of flow records ndash Still measures the vast majority of the traffic
Flow 1 40 bytesFlow 2 15580 bytesFlow 3 8196 bytesFlow 4 5350789 bytesFlow 5 532 bytesFlow 6 7432 bytes
sample with 100 probability
sample with 01 probability
sample with 10 probability
40
Two Main Approaches
bull Packet-level Monitoringndash Keep packet-level statisticsndash Examine (and potentially log) variety of packet-level
statistics Essentially anything in the packetndash Timing
bull Flow-level Monitoringndash Monitor packet-by-packet (though sometimes
sampled)ndash Keep aggregate statistics on a flow
41
Packet Capture on High-Speed Links
Example Georgia Tech ldquoOC3Monrdquo
bull Rack-mounted PCbull Optical splitterbull Data Acquisition and
Generation (DAG) card
Source endacecom
42
Characteristics of Packet Capture
bull Allows inspection on every packet on 10G links
bull Disadvantagesndash Costlyndash Requires splitting optical fibersndash Must be able to filterstore data
43
Data Measurement Repositories
bull AbileneInternet 2 Observatoryndash Configuration examplesndash SNMP datandash ISIS BGP routing data NetFlow traffic data
bull RouteViewsndash BGP updatesndash BGP table snapshots
44
Multihoming and Traffic Engineering
45
What is Multihoming
bull The use of redundant network links for the purposes of external connectivity
bull Can be achieved at many layers of the protocol stack and many places in the networkndash Multiple network interfaces in a PC
ndash An ISP with multiple upstream interfaces
bull Can refer to having multiple connections tondash The same ISP
ndash Multiple ISPs
46
Why Multihome
bull Redundancybull Availabilitybull Performancebull Cost
Interdomain traffic engineering the process by which a multihomed network configures its
network to achieve these goals
47
Redundancy
bull Maintain connectivity in the face ofndash Physical connectivity problems (fiber cut device
failures etc)ndash Failures in upstream ISP
48
Performance
bull Use multiple network links at once to achieve higher throughput than just over a single link
bull Allows incoming traffic to be load-balanced
70 of traffic30 of traffic
49
Multihoming in IP Networks Today
bull Stub AS no transit service for other ASesndash No need to use BGP
bull Multi-homed stub AS has connectivity to multiple immediate upstream ISPsndash Need BGPndash No need for a public AS numberndash No need for IP prefix allocation
bull Multi-homed transit AS connectivity to multiple ASes and transit servicendash Need BGP public AS number IP prefix allocation
50
BGP or no
bull Advantages of static routingndash Cheapersmaller routers (less true nowadays)ndash Simpler to configure
bull Advantages of BGPndash More control of your destiny (have providers stop
announcing you)ndash Fastermore intelligent selection of where to send
outbound packetsndash Better debugging of net problems (you can see the
Internet topology now)
51
Same Provider or Multiple
bull If your provider is reliable and fast and affordably and offers good tech-support you may want to multi-home initially to them via some backup path (slow is better than dead)
bull Eventually yoursquoll want to multi-home to different providers to avoid failure modes due to one providerrsquos architecture decisions
52
Multihomed Stub One Link
bull Downstream ISPrsquos routers configure default (ldquostaticrdquo) routes pointing to border router
bull Upstream ISP advertises reachability
Upstream ISP
Multiple links between same pair of routers
Default routes to ldquoborderrdquo
ldquoStubrdquoISP
53
Multihomed Stub Multiple Links
bull Use BGP to share loadbull Use private AS number (why is this OK)bull As before upstream ISP advertises prefix
Upstream ISP
Multiple links to different upstream routers
ldquoStubrdquoISP
Internal routing for ldquohot potatordquo
BGP for load balance at edge
54
Multihomed Stub Multiple ISPs
bull Many possibilitiesndash Load sharingndash Primary-backupndash Selective use of different ISPs
bull Requires BGP public AS number etc
ldquoStubrdquoISP
Upstream
ISP 1
Upstream
ISP 2
55
Multihomed Transit Network
bull BGP everywherebull Incoming and outcoming trafficbull Challenge balancing load on intradomain and egress
links given an offered traffic load
TransitISP
ISP 1
ISP 2
ISP 3
56
Interdomain Traffic Engineering
bull The process by which a network operator configures the network to achievendash Traffic load balancendash Redundancy (primarybackup) etc
bull Two tasksndash Outbound traffic controlndash Inbound traffic control
bull Key Problems Predictability and Scalability
57
Outbound Traffic Control
bull Easier to control than inbound trafficndash Destination-based routing sender determines where
the packets go
bull Control over next-hop AS onlyndash Cannot control selection of the entire path
Provider 1 Provider 2
Control with local preference
58
Outbound Traffic Load Balancingbull Control routes to provider per-prefix
ndash Assign local preference across destination prefixesndash Change the local preference assignments over time
bull Useful inputs to load balancingndash End-to-end path performance datandash Outbound traffic statistics per destination prefix
bull Challenge Getting from traffic volumes to groups of prefixes that should be assigned to each link
Premise of ldquointelligent route controlrdquo preoducts
59
Traffic Engineering Goals
bull Predictabilityndash Ensure the BGP decision process is deterministicndash Assume that BGP updates are (relatively) stable
bull Limit overhead introduced by routing changesndash Minimize frequency of changes to routing policiesndash Limit number of prefixes affected by changes
bull Limit impact on how traffic enters the networkndash Avoid new routes that might change neighborrsquos mindndash Select route with same attributes or at least path length
60
Managing Scalebull Destination prefixes
ndash More than 90000 destination prefixes
bull Donrsquot want to have per-prefix routing policies
ndash Small fraction of prefixes contribute most of the traffic
bull Focus on the small number of heavy hitters
ndash Define routing policies for selected prefixes
bull Routing choicesndash About 27000 unique ldquorouting choicesrdquo
bull Help in reducing the scale of the problem
ndash Small fraction of ldquorouting choicesrdquo contribute most traffic
bull Focus on the very small number of ldquorouting choicesrdquo
ndash Define routing policies on common attributes
61
Achieving Predictability
bull Route prediction with static analysisndash Helpful to know effects before deploymentndash Static analysis can help
TopologyBGP policy
configuration
eBGP routes
Offered traffic
BGP routingmodel
Flow of traffic through the network
62
Challenges to Predictabilitybull For transit ISPs effects on incoming traffic
ndash Lack of coordination strikes again
63
ldquoHot Potatordquo routing
Inter-AS Negotiation
bull Coordination aids predictabilityndash Negotiate where to sendndash Inbound and outboundndash Mutual benefits
bull How to implementndash What info to exchangendash Protecting privacyndash How to prioritize choicesndash How to prevent cheating
Destination 2
Destination 1
multiplepeeringpoints
Provider A
Provider B
64
Outbound Multihoming Goals
bull Redundancyndash Dynamic routing will failover to backup link
bull Performancendash Select provider with best performance per prefix
ndash Requires active probing
bull Costndash Select provider per prefix over time to minimize the
total financial cost
65
Inbound Traffic Control
bull More difficult no control over neighborsrsquo decisions
bull Three common techniques (previously discussed)ndash AS path prependingndash Communities and local preferencendash Prefix splitting
How does todayrsquos paper (MONET) control inbound traffic
66
How many links are enough
K upstream ISPs
Not much benefit beyond 4 ISPs
Akella et al ldquoPerformance Benefits of Multihomingrdquo SIGCOMM 2003
67
Problems with Multihoming in IPv4
bull Routing table growthndash Provider-based addressingndash Advertising prefix out multiple ISPs ndash canrsquot aggregate
bull Poor control over inbound trafficndash Existing mechanisms do not allow hosts to control
inbound traffic
68
GeorgiaTech
Internet Routing Overview
bull Intradomain (ie ldquointra-ASrdquo) routingbull Interdomain routing
Comcast
Abilene
ATampT Cogent
Autonomous Systems (ASes)
69
Configuration Problems ldquoAS 7007rdquoldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
UUNet
Florida InternetBarn
Sprint
70
Diagnosis and Troubleshootingldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
ldquoMicrosofts websites were offline for up to 23 hoursbecause of a [router] misconfigurationhellipit took nearly a day to determine what was wrong and undo the changesrdquo -- wiredcom January 25 2001
ldquoWorldCom Inchellipsuffered a widespread outage on its Internet backbone that affected roughly 20 percent of its US customer base The network problemshellipaffected millions of computer users worldwide A spokeswoman attributed the outage to a route table issue -- cnncom October 3 2002
A number of Covad customers went out from 5pm today due to supposedly a DDOS (distributed denial of service attack) on a key Level3 data center which later was described as a route leak (misconfiguration)ldquo
-- dslreportscom February 23 2004
71
Operator Mailing List (NANOG)Date Mon 18 Oct 2004 091515 -0700Subject Level 3 US east coast issues
Level 3 experiencing widespread unspecified routing issues on the US east coast Master ticket 1086844 Anyone have more specific information
Date Mon 18 Oct 2004 122034 -0400 (EDT)Subject Re Level 3 US east coast issues
Level 3 is currently experiencing a backbone outage causing routing instability and packet loss We are working to restore and will be sending hourly updateshellip
72
Operator Mailing List
0102030405060708090
Filtering RouteLeaks
RouteHijacks
RouteInstability
RoutingLoops
Blackholes
Occurr
ences o
ver
10 Y
ears
1995-1997 1998-2001 2001-2004
Note Only includes problems openly discussed on this list
Compare 83 power outages 1 fire
73
Routing Configuration
Ranking route selection
Dissemination internal route advertisement
Filtering route advertisement
Customer
Competitor
Primary
Backup
74
Internet Business Model (Simplified)
bull CustomerProvider One AS pays another for reachability to some set of destinations
bull ldquoSettlement-freerdquo Peering Bartering Two ASes exchange routes with one another
Provider
Peer
Customer
Preferences implemented with local preference manipulation
Destination
Pay to use
Get paid to use
Free to use
75
Peering Contracts Consistent Export
bull Rules of settlement-free peeringndash Advertise routes at all peering pointsndash Advertised routes must have equal ldquoAS path lengthrdquo
Sprint
ATampT
Enables ldquohot potatordquo routing
ldquoequally goodrdquoroutes
76
Consistent Export
bull Malicedeceptionbull iBGP signaling partitionbull Inconsistent export policy
neighbor 10123route-map PEER permit 10 set prepend 123
neighbor 10456route-map PEER permit 10 set prepend 123 123
Possible Causes
10123 456 1
10456 456 2
Neighbor AS Export
1 1 123
Export Clause Prepend
2 1 123 123
Two different Export Policies
77
Inconsistent Export in Practice
Feamster et al ldquoBorderGuard Detecting Cold Potatoes from Peersrdquo ACM IMC October 2004
78
Blackholes
Date Thu 18 Jul 2002 060510 -0400 (EDT)From Chad Oleary ltcolpoboxcomgtSubject Re problems with 701To ltnanogmeritedugt
Were starting to see the same issues with UUNet again Anyone elseseeing this Trying to reach Qwest
traceroute to 631461901 (631461901) 30 hops max 38 byte packets 1 esc-lp2-gwe-solutionscorpcom (631182201) 1167 ms 1163 ms 1142 ms 2 500Serial2-10GW1TPA2ALTERNET (1571301499) 1097 ms 1059 ms 1044 ms 3 161at-1-0-0XL4ATL1ALTERNET (1526381190) 13839 ms 14108 ms 16638 ms 4 0so-3-1-0XL2ATL5ALTERNET (152630238) 14370 ms 14587 ms 14553 ms 5 POS7-0BR2ATL5ALTERNET (1526382193) 13928 ms 14099 ms 14053 ms 6 7 hellip
79
Security
80
Security ldquoBogonrdquo Routes
Feamster et al ldquoAn Empirical Study of lsquoBogonrsquo Route Advertisementsrdquo ACM CCR January 2005
81
Spam Phishing etc
bull Unsolicited commercial emailbull As of about August 2008 estimates indicate that
about 95 of all email is spambull Common spam filtering techniques
ndash Content-based filtersndash DNS Blacklist (DNSBL) lookups Significant fraction of
todayrsquos DNS traffic
Can IP addresses from which spam is received be spoofed
82
Spam and Routing
83
Worms and Botnets
84
What is a Worm
bull Code that replicates and propagates across the networkndash Often carries a ldquopayloadrdquo
bull Usually spread via exploiting flaws in open servicesndash ldquoVirusesrdquo require user action to spread
bull First worm Robert Morris November 1988ndash 6-10 of all Internet hosts infected ()
bull Many more since but none on that scale until July 2001
85
Example Worm Code Red
bull Initial version July 13 2001
bull Exploited known ISAPI vulnerability in Microsoft IIS Web servers
bull 1st through 20th of each month spread20th through end of each month attack
bull Payload Web site defacementbull Scanning Random IP addressesbull Bug failure to seed random number generator
86
Code Red Revisions
bull Released July 19 2001
bull Payload flooding attack on wwwwhitehousegovndash Attack was mounted at the IP address of the Web site
bull Bug died after 20th of each month
bull Random number generator for IP scanning fixed
87
Code Red Host Infection Rate
Exponential infection rate
Measured using backscatter technique
88
Designing Fast-Spreading Worms
bull Hit-list scanningndash Time to infect first 10k hosts dominates infection timendash Solution Reconnaissance (stealthy scans etc)
bull Permutation scanningndash Observation Most scanning is redundantndash Idea Shared permutation of address space Start scanning
from own IP address Re-randomize when another infected machine is found
bull Internet-scale hit listsndash Flash worm complete infection within 30 seconds
89
Botnets
bull Bots Autonomous programs performing tasksbull Plenty of ldquobenignrdquo bots
ndash eg weatherbug
bull Botnets group of bots ndash Typically carries malicious connotationndash Large numbers of infected machinesndash Machines ldquoenlistedrdquo with infection vectors like worms
(last lecture)
bull Available for simultaneous control by a masterbull Size up to 350000 nodes (from todayrsquos paper)
90
ldquoRallyingrdquo the Botnet
bull Easy to combine worm backdoor functionalitybull Problem how to learn about successfully
infected machines
bull Optionsndash Emailndash Hard-coded email address
91
Botnet Control
bull Botnet master typically runs some IRC server on a well-known port (eg 6667)
bull Infected machine contacts botnet with pre-programmed DNS name (eg big-botde)
bull Dynamic DNS allows controller to move about freely
Infected Machine
DynamicDNS
BotnetController
(IRC server)
92
Some Defenses
93
Idea 1 Ingress Filtering
bull RFC 2827 Routers install filters to drop packets from networks that are not downstream
bull Feasible at edgesbull Difficult to configure closer to network ldquocorerdquo
20469207024 Internet
Drop all packets with source address other than 20469207024
94
Idea 2 uRPF Checks
bull Unicast Reverse Path Forwardingndash Cisco ldquoip verify unicast reverse-pathrdquo
bull Requires symmetric routing
Accept packet from interface only if forwarding table entry for source IP address matches ingress interface
100183
A10018
10015
101203
100181 24
10011 24
Strict Mode uRPF
Enabled
ldquoArdquo Routing TableDestination Next Hop1001024 Int 110018024 Int 2
100183 from wrong interface
95
Problems with uRPF
bull Asymmetric routing
96
S-BGP
bull Address-based PKI validate signaturesndash Authentication of
bull ownership for IP address blocks bull AS number bull an ASs identity and bull a BGP routers identity
ndash Use existing infrastructure (Internet registries etc)ndash Routing origination is digitally signedndash BGP updates are digitally signed
1048708 bull Route attestations A new optional BGP transitive path attribute
ndash carries digital signatures covering the routing information in updates
97
Practical Problems with S-BGP
bull Requires Public-Key Infrastructure
bull Lots of digital signatures to calculate and verifyndash Message overheadndash CPU overhead
bull Calculation expense is greatest when topology is changingndash Caching can help
bull Route aggregation is problematic (maybe thatrsquos OK)
bull Secure route withdrawals when link or node fails
bull Address ownership data out of date
bull Deployment
- Networking
- Goal of This Tutorial
- Todayrsquos Networks
- Business Models
- Billing for Internet Usage
- Net Neutrality
- Network Operations
- Point-of-Presence (PoP)
- Example Abilene Network Topology
- Another Example Backbone
- Internet Routing Overview
- Internet Routing Protocol BGP
- Two Flavors of BGP
- IPv4 Addresses Networks of Networks
- Pre-1994 Classful Addressing
- Classless Interdomain Routing (CIDR)
- Benefits of CIDR
- Growth of IP Prefixes
- 1994-1998 Linear Growth
- Around 2000 Fast Growth Resumes
- Fast growth resumes
- The Address Allocation Process
- Common Problems
- What can go wrong
- Measurement and Monitoring
- Passive vs Active Measurement
- Slide 27
- Passive Traffic Data Measurement
- Simple Network Management Protocol
- SNMP (Passive)
- Packet-level Monitoring
- Full Packet Capture (Passive)
- What is a flow
- Cisco NetFlow
- Flow Record Contents
- Aggregating Packets into Flows
- Reducing Measurement Overhead
- Packet Sampling for Flow Monitoring
- Sampling Flow-Level Sampling
- Two Main Approaches
- Packet Capture on High-Speed Links
- Characteristics of Packet Capture
- Data Measurement Repositories
- Multihoming and Traffic Engineering
- What is Multihoming
- Why Multihome
- Redundancy
- Performance
- Multihoming in IP Networks Today
- BGP or no
- Same Provider or Multiple
- Multihomed Stub One Link
- Multihomed Stub Multiple Links
- Multihomed Stub Multiple ISPs
- Multihomed Transit Network
- Interdomain Traffic Engineering
- Outbound Traffic Control
- Outbound Traffic Load Balancing
- Traffic Engineering Goals
- Managing Scale
- Achieving Predictability
- Challenges to Predictability
- Inter-AS Negotiation
- Outbound Multihoming Goals
- Inbound Traffic Control
- How many links are enough
- Problems with Multihoming in IPv4
- Slide 68
- Slide 69
- Diagnosis and Troubleshooting
- Operator Mailing List (NANOG)
- Operator Mailing List
- Routing Configuration
- Internet Business Model (Simplified)
- Peering Contracts Consistent Export
- Consistent Export
- Inconsistent Export in Practice
- Blackholes
- Security
- Security ldquoBogonrdquo Routes
- Spam Phishing etc
- Spam and Routing
- Worms and Botnets
- What is a Worm
- Example Worm Code Red
- Code Red Revisions
- Code Red Host Infection Rate
- Designing Fast-Spreading Worms
- Botnets
- ldquoRallyingrdquo the Botnet
- Botnet Control
- Some Defenses
- Idea 1 Ingress Filtering
- Idea 2 uRPF Checks
- Problems with uRPF
- S-BGP
- Practical Problems with S-BGP
-
34
Cisco NetFlow
bull Basic output ldquoFlow recordrdquondash Most common version is v5
bull Current version (9) is being standardized in the IETF (template-based)ndash More flexible record formatndash Much easier to add new flow record types
Core Network
Collection and Aggregation
Collector (PC)Approximately 1500 bytes
20-50 flow recordsSent more frequently if traffic increases
35
Flow Record Contents
bull Source and Destination IP address and portbull Packet and byte countsbull Start and end timesbull ToS TCP flags
Basic information about the flowhellip
hellipplus information related to routing
bull Next-hop IP addressbull Source and destination ASbull Source and destination prefix
36
flow 1 flow 2 flow 3 flow 4
Aggregating Packets into Flows
bull Criteria 1 Set of packets that ldquobelong togetherrdquondash Sourcedestination IP addresses and port numbersndash Same protocol ToS bits hellip ndash Same inputoutput interfaces at a router (if known)
bull Criteria 2 Packets that are ldquocloserdquo together in timendash Maximum inter-packet spacing (eg 15 sec 30 sec)ndash Example flows 2 and 4 are different flows due to time
37
Reducing Measurement Overhead
bull Filtering on interfacendash destination prefix for a customerndash port number for an application (eg 80 for Web)
bull Sampling before insertion into flow cachendash Random deterministic or hash-based samplingndash 1-out-of-n or stratified based on packetflow sizendash Two types packet-level and flow-level
bull Aggregation after cache evictionndash packetsflows with same next-hop ASndash packetsflows destined to a particular service
38
Packet Sampling for Flow Monitoring
bull Packet sampling before flow creation (Sampled Netflow)ndash 1-out-of-m sampling of individual packets (eg m=100)ndash Create of flow records over the sampled packets
bull Reducing overheadndash Avoid per-packet overhead on (m-1)m packetsndash Avoid creating records for a large number of small flows
bull Increasing overhead (in some cases)ndash May split some long transfers into multiple flow records ndash hellip due to larger time gaps between successive packets
time
not sampled
two flowstimeout
39
Sampling Flow-Level Sampling
bull Sampling of flow records evicted from flow cachendash When evicting flows from table or when analyzing flows
bull Stratified sampling to put weight on ldquoheavyrdquo flowsndash Select all long flows and sample the short flows
bull Reduces the number of flow records ndash Still measures the vast majority of the traffic
Flow 1 40 bytesFlow 2 15580 bytesFlow 3 8196 bytesFlow 4 5350789 bytesFlow 5 532 bytesFlow 6 7432 bytes
sample with 100 probability
sample with 01 probability
sample with 10 probability
40
Two Main Approaches
bull Packet-level Monitoringndash Keep packet-level statisticsndash Examine (and potentially log) variety of packet-level
statistics Essentially anything in the packetndash Timing
bull Flow-level Monitoringndash Monitor packet-by-packet (though sometimes
sampled)ndash Keep aggregate statistics on a flow
41
Packet Capture on High-Speed Links
Example Georgia Tech ldquoOC3Monrdquo
bull Rack-mounted PCbull Optical splitterbull Data Acquisition and
Generation (DAG) card
Source endacecom
42
Characteristics of Packet Capture
bull Allows inspection on every packet on 10G links
bull Disadvantagesndash Costlyndash Requires splitting optical fibersndash Must be able to filterstore data
43
Data Measurement Repositories
bull AbileneInternet 2 Observatoryndash Configuration examplesndash SNMP datandash ISIS BGP routing data NetFlow traffic data
bull RouteViewsndash BGP updatesndash BGP table snapshots
44
Multihoming and Traffic Engineering
45
What is Multihoming
bull The use of redundant network links for the purposes of external connectivity
bull Can be achieved at many layers of the protocol stack and many places in the networkndash Multiple network interfaces in a PC
ndash An ISP with multiple upstream interfaces
bull Can refer to having multiple connections tondash The same ISP
ndash Multiple ISPs
46
Why Multihome
bull Redundancybull Availabilitybull Performancebull Cost
Interdomain traffic engineering the process by which a multihomed network configures its
network to achieve these goals
47
Redundancy
bull Maintain connectivity in the face ofndash Physical connectivity problems (fiber cut device
failures etc)ndash Failures in upstream ISP
48
Performance
bull Use multiple network links at once to achieve higher throughput than just over a single link
bull Allows incoming traffic to be load-balanced
70 of traffic30 of traffic
49
Multihoming in IP Networks Today
bull Stub AS no transit service for other ASesndash No need to use BGP
bull Multi-homed stub AS has connectivity to multiple immediate upstream ISPsndash Need BGPndash No need for a public AS numberndash No need for IP prefix allocation
bull Multi-homed transit AS connectivity to multiple ASes and transit servicendash Need BGP public AS number IP prefix allocation
50
BGP or no
bull Advantages of static routingndash Cheapersmaller routers (less true nowadays)ndash Simpler to configure
bull Advantages of BGPndash More control of your destiny (have providers stop
announcing you)ndash Fastermore intelligent selection of where to send
outbound packetsndash Better debugging of net problems (you can see the
Internet topology now)
51
Same Provider or Multiple
bull If your provider is reliable and fast and affordably and offers good tech-support you may want to multi-home initially to them via some backup path (slow is better than dead)
bull Eventually yoursquoll want to multi-home to different providers to avoid failure modes due to one providerrsquos architecture decisions
52
Multihomed Stub One Link
bull Downstream ISPrsquos routers configure default (ldquostaticrdquo) routes pointing to border router
bull Upstream ISP advertises reachability
Upstream ISP
Multiple links between same pair of routers
Default routes to ldquoborderrdquo
ldquoStubrdquoISP
53
Multihomed Stub Multiple Links
bull Use BGP to share loadbull Use private AS number (why is this OK)bull As before upstream ISP advertises prefix
Upstream ISP
Multiple links to different upstream routers
ldquoStubrdquoISP
Internal routing for ldquohot potatordquo
BGP for load balance at edge
54
Multihomed Stub Multiple ISPs
bull Many possibilitiesndash Load sharingndash Primary-backupndash Selective use of different ISPs
bull Requires BGP public AS number etc
ldquoStubrdquoISP
Upstream
ISP 1
Upstream
ISP 2
55
Multihomed Transit Network
bull BGP everywherebull Incoming and outcoming trafficbull Challenge balancing load on intradomain and egress
links given an offered traffic load
TransitISP
ISP 1
ISP 2
ISP 3
56
Interdomain Traffic Engineering
bull The process by which a network operator configures the network to achievendash Traffic load balancendash Redundancy (primarybackup) etc
bull Two tasksndash Outbound traffic controlndash Inbound traffic control
bull Key Problems Predictability and Scalability
57
Outbound Traffic Control
bull Easier to control than inbound trafficndash Destination-based routing sender determines where
the packets go
bull Control over next-hop AS onlyndash Cannot control selection of the entire path
Provider 1 Provider 2
Control with local preference
58
Outbound Traffic Load Balancingbull Control routes to provider per-prefix
ndash Assign local preference across destination prefixesndash Change the local preference assignments over time
bull Useful inputs to load balancingndash End-to-end path performance datandash Outbound traffic statistics per destination prefix
bull Challenge Getting from traffic volumes to groups of prefixes that should be assigned to each link
Premise of ldquointelligent route controlrdquo preoducts
59
Traffic Engineering Goals
bull Predictabilityndash Ensure the BGP decision process is deterministicndash Assume that BGP updates are (relatively) stable
bull Limit overhead introduced by routing changesndash Minimize frequency of changes to routing policiesndash Limit number of prefixes affected by changes
bull Limit impact on how traffic enters the networkndash Avoid new routes that might change neighborrsquos mindndash Select route with same attributes or at least path length
60
Managing Scalebull Destination prefixes
ndash More than 90000 destination prefixes
bull Donrsquot want to have per-prefix routing policies
ndash Small fraction of prefixes contribute most of the traffic
bull Focus on the small number of heavy hitters
ndash Define routing policies for selected prefixes
bull Routing choicesndash About 27000 unique ldquorouting choicesrdquo
bull Help in reducing the scale of the problem
ndash Small fraction of ldquorouting choicesrdquo contribute most traffic
bull Focus on the very small number of ldquorouting choicesrdquo
ndash Define routing policies on common attributes
61
Achieving Predictability
bull Route prediction with static analysisndash Helpful to know effects before deploymentndash Static analysis can help
TopologyBGP policy
configuration
eBGP routes
Offered traffic
BGP routingmodel
Flow of traffic through the network
62
Challenges to Predictabilitybull For transit ISPs effects on incoming traffic
ndash Lack of coordination strikes again
63
ldquoHot Potatordquo routing
Inter-AS Negotiation
bull Coordination aids predictabilityndash Negotiate where to sendndash Inbound and outboundndash Mutual benefits
bull How to implementndash What info to exchangendash Protecting privacyndash How to prioritize choicesndash How to prevent cheating
Destination 2
Destination 1
multiplepeeringpoints
Provider A
Provider B
64
Outbound Multihoming Goals
bull Redundancyndash Dynamic routing will failover to backup link
bull Performancendash Select provider with best performance per prefix
ndash Requires active probing
bull Costndash Select provider per prefix over time to minimize the
total financial cost
65
Inbound Traffic Control
bull More difficult no control over neighborsrsquo decisions
bull Three common techniques (previously discussed)ndash AS path prependingndash Communities and local preferencendash Prefix splitting
How does todayrsquos paper (MONET) control inbound traffic
66
How many links are enough
K upstream ISPs
Not much benefit beyond 4 ISPs
Akella et al ldquoPerformance Benefits of Multihomingrdquo SIGCOMM 2003
67
Problems with Multihoming in IPv4
bull Routing table growthndash Provider-based addressingndash Advertising prefix out multiple ISPs ndash canrsquot aggregate
bull Poor control over inbound trafficndash Existing mechanisms do not allow hosts to control
inbound traffic
68
GeorgiaTech
Internet Routing Overview
bull Intradomain (ie ldquointra-ASrdquo) routingbull Interdomain routing
Comcast
Abilene
ATampT Cogent
Autonomous Systems (ASes)
69
Configuration Problems ldquoAS 7007rdquoldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
UUNet
Florida InternetBarn
Sprint
70
Diagnosis and Troubleshootingldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
ldquoMicrosofts websites were offline for up to 23 hoursbecause of a [router] misconfigurationhellipit took nearly a day to determine what was wrong and undo the changesrdquo -- wiredcom January 25 2001
ldquoWorldCom Inchellipsuffered a widespread outage on its Internet backbone that affected roughly 20 percent of its US customer base The network problemshellipaffected millions of computer users worldwide A spokeswoman attributed the outage to a route table issue -- cnncom October 3 2002
A number of Covad customers went out from 5pm today due to supposedly a DDOS (distributed denial of service attack) on a key Level3 data center which later was described as a route leak (misconfiguration)ldquo
-- dslreportscom February 23 2004
71
Operator Mailing List (NANOG)Date Mon 18 Oct 2004 091515 -0700Subject Level 3 US east coast issues
Level 3 experiencing widespread unspecified routing issues on the US east coast Master ticket 1086844 Anyone have more specific information
Date Mon 18 Oct 2004 122034 -0400 (EDT)Subject Re Level 3 US east coast issues
Level 3 is currently experiencing a backbone outage causing routing instability and packet loss We are working to restore and will be sending hourly updateshellip
72
Operator Mailing List
0102030405060708090
Filtering RouteLeaks
RouteHijacks
RouteInstability
RoutingLoops
Blackholes
Occurr
ences o
ver
10 Y
ears
1995-1997 1998-2001 2001-2004
Note Only includes problems openly discussed on this list
Compare 83 power outages 1 fire
73
Routing Configuration
Ranking route selection
Dissemination internal route advertisement
Filtering route advertisement
Customer
Competitor
Primary
Backup
74
Internet Business Model (Simplified)
bull CustomerProvider One AS pays another for reachability to some set of destinations
bull ldquoSettlement-freerdquo Peering Bartering Two ASes exchange routes with one another
Provider
Peer
Customer
Preferences implemented with local preference manipulation
Destination
Pay to use
Get paid to use
Free to use
75
Peering Contracts Consistent Export
bull Rules of settlement-free peeringndash Advertise routes at all peering pointsndash Advertised routes must have equal ldquoAS path lengthrdquo
Sprint
ATampT
Enables ldquohot potatordquo routing
ldquoequally goodrdquoroutes
76
Consistent Export
bull Malicedeceptionbull iBGP signaling partitionbull Inconsistent export policy
neighbor 10123route-map PEER permit 10 set prepend 123
neighbor 10456route-map PEER permit 10 set prepend 123 123
Possible Causes
10123 456 1
10456 456 2
Neighbor AS Export
1 1 123
Export Clause Prepend
2 1 123 123
Two different Export Policies
77
Inconsistent Export in Practice
Feamster et al ldquoBorderGuard Detecting Cold Potatoes from Peersrdquo ACM IMC October 2004
78
Blackholes
Date Thu 18 Jul 2002 060510 -0400 (EDT)From Chad Oleary ltcolpoboxcomgtSubject Re problems with 701To ltnanogmeritedugt
Were starting to see the same issues with UUNet again Anyone elseseeing this Trying to reach Qwest
traceroute to 631461901 (631461901) 30 hops max 38 byte packets 1 esc-lp2-gwe-solutionscorpcom (631182201) 1167 ms 1163 ms 1142 ms 2 500Serial2-10GW1TPA2ALTERNET (1571301499) 1097 ms 1059 ms 1044 ms 3 161at-1-0-0XL4ATL1ALTERNET (1526381190) 13839 ms 14108 ms 16638 ms 4 0so-3-1-0XL2ATL5ALTERNET (152630238) 14370 ms 14587 ms 14553 ms 5 POS7-0BR2ATL5ALTERNET (1526382193) 13928 ms 14099 ms 14053 ms 6 7 hellip
79
Security
80
Security ldquoBogonrdquo Routes
Feamster et al ldquoAn Empirical Study of lsquoBogonrsquo Route Advertisementsrdquo ACM CCR January 2005
81
Spam Phishing etc
bull Unsolicited commercial emailbull As of about August 2008 estimates indicate that
about 95 of all email is spambull Common spam filtering techniques
ndash Content-based filtersndash DNS Blacklist (DNSBL) lookups Significant fraction of
todayrsquos DNS traffic
Can IP addresses from which spam is received be spoofed
82
Spam and Routing
83
Worms and Botnets
84
What is a Worm
bull Code that replicates and propagates across the networkndash Often carries a ldquopayloadrdquo
bull Usually spread via exploiting flaws in open servicesndash ldquoVirusesrdquo require user action to spread
bull First worm Robert Morris November 1988ndash 6-10 of all Internet hosts infected ()
bull Many more since but none on that scale until July 2001
85
Example Worm Code Red
bull Initial version July 13 2001
bull Exploited known ISAPI vulnerability in Microsoft IIS Web servers
bull 1st through 20th of each month spread20th through end of each month attack
bull Payload Web site defacementbull Scanning Random IP addressesbull Bug failure to seed random number generator
86
Code Red Revisions
bull Released July 19 2001
bull Payload flooding attack on wwwwhitehousegovndash Attack was mounted at the IP address of the Web site
bull Bug died after 20th of each month
bull Random number generator for IP scanning fixed
87
Code Red Host Infection Rate
Exponential infection rate
Measured using backscatter technique
88
Designing Fast-Spreading Worms
bull Hit-list scanningndash Time to infect first 10k hosts dominates infection timendash Solution Reconnaissance (stealthy scans etc)
bull Permutation scanningndash Observation Most scanning is redundantndash Idea Shared permutation of address space Start scanning
from own IP address Re-randomize when another infected machine is found
bull Internet-scale hit listsndash Flash worm complete infection within 30 seconds
89
Botnets
bull Bots Autonomous programs performing tasksbull Plenty of ldquobenignrdquo bots
ndash eg weatherbug
bull Botnets group of bots ndash Typically carries malicious connotationndash Large numbers of infected machinesndash Machines ldquoenlistedrdquo with infection vectors like worms
(last lecture)
bull Available for simultaneous control by a masterbull Size up to 350000 nodes (from todayrsquos paper)
90
ldquoRallyingrdquo the Botnet
bull Easy to combine worm backdoor functionalitybull Problem how to learn about successfully
infected machines
bull Optionsndash Emailndash Hard-coded email address
91
Botnet Control
bull Botnet master typically runs some IRC server on a well-known port (eg 6667)
bull Infected machine contacts botnet with pre-programmed DNS name (eg big-botde)
bull Dynamic DNS allows controller to move about freely
Infected Machine
DynamicDNS
BotnetController
(IRC server)
92
Some Defenses
93
Idea 1 Ingress Filtering
bull RFC 2827 Routers install filters to drop packets from networks that are not downstream
bull Feasible at edgesbull Difficult to configure closer to network ldquocorerdquo
20469207024 Internet
Drop all packets with source address other than 20469207024
94
Idea 2 uRPF Checks
bull Unicast Reverse Path Forwardingndash Cisco ldquoip verify unicast reverse-pathrdquo
bull Requires symmetric routing
Accept packet from interface only if forwarding table entry for source IP address matches ingress interface
100183
A10018
10015
101203
100181 24
10011 24
Strict Mode uRPF
Enabled
ldquoArdquo Routing TableDestination Next Hop1001024 Int 110018024 Int 2
100183 from wrong interface
95
Problems with uRPF
bull Asymmetric routing
96
S-BGP
bull Address-based PKI validate signaturesndash Authentication of
bull ownership for IP address blocks bull AS number bull an ASs identity and bull a BGP routers identity
ndash Use existing infrastructure (Internet registries etc)ndash Routing origination is digitally signedndash BGP updates are digitally signed
1048708 bull Route attestations A new optional BGP transitive path attribute
ndash carries digital signatures covering the routing information in updates
97
Practical Problems with S-BGP
bull Requires Public-Key Infrastructure
bull Lots of digital signatures to calculate and verifyndash Message overheadndash CPU overhead
bull Calculation expense is greatest when topology is changingndash Caching can help
bull Route aggregation is problematic (maybe thatrsquos OK)
bull Secure route withdrawals when link or node fails
bull Address ownership data out of date
bull Deployment
- Networking
- Goal of This Tutorial
- Todayrsquos Networks
- Business Models
- Billing for Internet Usage
- Net Neutrality
- Network Operations
- Point-of-Presence (PoP)
- Example Abilene Network Topology
- Another Example Backbone
- Internet Routing Overview
- Internet Routing Protocol BGP
- Two Flavors of BGP
- IPv4 Addresses Networks of Networks
- Pre-1994 Classful Addressing
- Classless Interdomain Routing (CIDR)
- Benefits of CIDR
- Growth of IP Prefixes
- 1994-1998 Linear Growth
- Around 2000 Fast Growth Resumes
- Fast growth resumes
- The Address Allocation Process
- Common Problems
- What can go wrong
- Measurement and Monitoring
- Passive vs Active Measurement
- Slide 27
- Passive Traffic Data Measurement
- Simple Network Management Protocol
- SNMP (Passive)
- Packet-level Monitoring
- Full Packet Capture (Passive)
- What is a flow
- Cisco NetFlow
- Flow Record Contents
- Aggregating Packets into Flows
- Reducing Measurement Overhead
- Packet Sampling for Flow Monitoring
- Sampling Flow-Level Sampling
- Two Main Approaches
- Packet Capture on High-Speed Links
- Characteristics of Packet Capture
- Data Measurement Repositories
- Multihoming and Traffic Engineering
- What is Multihoming
- Why Multihome
- Redundancy
- Performance
- Multihoming in IP Networks Today
- BGP or no
- Same Provider or Multiple
- Multihomed Stub One Link
- Multihomed Stub Multiple Links
- Multihomed Stub Multiple ISPs
- Multihomed Transit Network
- Interdomain Traffic Engineering
- Outbound Traffic Control
- Outbound Traffic Load Balancing
- Traffic Engineering Goals
- Managing Scale
- Achieving Predictability
- Challenges to Predictability
- Inter-AS Negotiation
- Outbound Multihoming Goals
- Inbound Traffic Control
- How many links are enough
- Problems with Multihoming in IPv4
- Slide 68
- Slide 69
- Diagnosis and Troubleshooting
- Operator Mailing List (NANOG)
- Operator Mailing List
- Routing Configuration
- Internet Business Model (Simplified)
- Peering Contracts Consistent Export
- Consistent Export
- Inconsistent Export in Practice
- Blackholes
- Security
- Security ldquoBogonrdquo Routes
- Spam Phishing etc
- Spam and Routing
- Worms and Botnets
- What is a Worm
- Example Worm Code Red
- Code Red Revisions
- Code Red Host Infection Rate
- Designing Fast-Spreading Worms
- Botnets
- ldquoRallyingrdquo the Botnet
- Botnet Control
- Some Defenses
- Idea 1 Ingress Filtering
- Idea 2 uRPF Checks
- Problems with uRPF
- S-BGP
- Practical Problems with S-BGP
-
35
Flow Record Contents
bull Source and Destination IP address and portbull Packet and byte countsbull Start and end timesbull ToS TCP flags
Basic information about the flowhellip
hellipplus information related to routing
bull Next-hop IP addressbull Source and destination ASbull Source and destination prefix
36
flow 1 flow 2 flow 3 flow 4
Aggregating Packets into Flows
bull Criteria 1 Set of packets that ldquobelong togetherrdquondash Sourcedestination IP addresses and port numbersndash Same protocol ToS bits hellip ndash Same inputoutput interfaces at a router (if known)
bull Criteria 2 Packets that are ldquocloserdquo together in timendash Maximum inter-packet spacing (eg 15 sec 30 sec)ndash Example flows 2 and 4 are different flows due to time
37
Reducing Measurement Overhead
bull Filtering on interfacendash destination prefix for a customerndash port number for an application (eg 80 for Web)
bull Sampling before insertion into flow cachendash Random deterministic or hash-based samplingndash 1-out-of-n or stratified based on packetflow sizendash Two types packet-level and flow-level
bull Aggregation after cache evictionndash packetsflows with same next-hop ASndash packetsflows destined to a particular service
38
Packet Sampling for Flow Monitoring
bull Packet sampling before flow creation (Sampled Netflow)ndash 1-out-of-m sampling of individual packets (eg m=100)ndash Create of flow records over the sampled packets
bull Reducing overheadndash Avoid per-packet overhead on (m-1)m packetsndash Avoid creating records for a large number of small flows
bull Increasing overhead (in some cases)ndash May split some long transfers into multiple flow records ndash hellip due to larger time gaps between successive packets
time
not sampled
two flowstimeout
39
Sampling Flow-Level Sampling
bull Sampling of flow records evicted from flow cachendash When evicting flows from table or when analyzing flows
bull Stratified sampling to put weight on ldquoheavyrdquo flowsndash Select all long flows and sample the short flows
bull Reduces the number of flow records ndash Still measures the vast majority of the traffic
Flow 1 40 bytesFlow 2 15580 bytesFlow 3 8196 bytesFlow 4 5350789 bytesFlow 5 532 bytesFlow 6 7432 bytes
sample with 100 probability
sample with 01 probability
sample with 10 probability
40
Two Main Approaches
bull Packet-level Monitoringndash Keep packet-level statisticsndash Examine (and potentially log) variety of packet-level
statistics Essentially anything in the packetndash Timing
bull Flow-level Monitoringndash Monitor packet-by-packet (though sometimes
sampled)ndash Keep aggregate statistics on a flow
41
Packet Capture on High-Speed Links
Example Georgia Tech ldquoOC3Monrdquo
bull Rack-mounted PCbull Optical splitterbull Data Acquisition and
Generation (DAG) card
Source endacecom
42
Characteristics of Packet Capture
bull Allows inspection on every packet on 10G links
bull Disadvantagesndash Costlyndash Requires splitting optical fibersndash Must be able to filterstore data
43
Data Measurement Repositories
bull AbileneInternet 2 Observatoryndash Configuration examplesndash SNMP datandash ISIS BGP routing data NetFlow traffic data
bull RouteViewsndash BGP updatesndash BGP table snapshots
44
Multihoming and Traffic Engineering
45
What is Multihoming
bull The use of redundant network links for the purposes of external connectivity
bull Can be achieved at many layers of the protocol stack and many places in the networkndash Multiple network interfaces in a PC
ndash An ISP with multiple upstream interfaces
bull Can refer to having multiple connections tondash The same ISP
ndash Multiple ISPs
46
Why Multihome
bull Redundancybull Availabilitybull Performancebull Cost
Interdomain traffic engineering the process by which a multihomed network configures its
network to achieve these goals
47
Redundancy
bull Maintain connectivity in the face ofndash Physical connectivity problems (fiber cut device
failures etc)ndash Failures in upstream ISP
48
Performance
bull Use multiple network links at once to achieve higher throughput than just over a single link
bull Allows incoming traffic to be load-balanced
70 of traffic30 of traffic
49
Multihoming in IP Networks Today
bull Stub AS no transit service for other ASesndash No need to use BGP
bull Multi-homed stub AS has connectivity to multiple immediate upstream ISPsndash Need BGPndash No need for a public AS numberndash No need for IP prefix allocation
bull Multi-homed transit AS connectivity to multiple ASes and transit servicendash Need BGP public AS number IP prefix allocation
50
BGP or no
bull Advantages of static routingndash Cheapersmaller routers (less true nowadays)ndash Simpler to configure
bull Advantages of BGPndash More control of your destiny (have providers stop
announcing you)ndash Fastermore intelligent selection of where to send
outbound packetsndash Better debugging of net problems (you can see the
Internet topology now)
51
Same Provider or Multiple
bull If your provider is reliable and fast and affordably and offers good tech-support you may want to multi-home initially to them via some backup path (slow is better than dead)
bull Eventually yoursquoll want to multi-home to different providers to avoid failure modes due to one providerrsquos architecture decisions
52
Multihomed Stub One Link
bull Downstream ISPrsquos routers configure default (ldquostaticrdquo) routes pointing to border router
bull Upstream ISP advertises reachability
Upstream ISP
Multiple links between same pair of routers
Default routes to ldquoborderrdquo
ldquoStubrdquoISP
53
Multihomed Stub Multiple Links
bull Use BGP to share loadbull Use private AS number (why is this OK)bull As before upstream ISP advertises prefix
Upstream ISP
Multiple links to different upstream routers
ldquoStubrdquoISP
Internal routing for ldquohot potatordquo
BGP for load balance at edge
54
Multihomed Stub Multiple ISPs
bull Many possibilitiesndash Load sharingndash Primary-backupndash Selective use of different ISPs
bull Requires BGP public AS number etc
ldquoStubrdquoISP
Upstream
ISP 1
Upstream
ISP 2
55
Multihomed Transit Network
bull BGP everywherebull Incoming and outcoming trafficbull Challenge balancing load on intradomain and egress
links given an offered traffic load
TransitISP
ISP 1
ISP 2
ISP 3
56
Interdomain Traffic Engineering
bull The process by which a network operator configures the network to achievendash Traffic load balancendash Redundancy (primarybackup) etc
bull Two tasksndash Outbound traffic controlndash Inbound traffic control
bull Key Problems Predictability and Scalability
57
Outbound Traffic Control
bull Easier to control than inbound trafficndash Destination-based routing sender determines where
the packets go
bull Control over next-hop AS onlyndash Cannot control selection of the entire path
Provider 1 Provider 2
Control with local preference
58
Outbound Traffic Load Balancingbull Control routes to provider per-prefix
ndash Assign local preference across destination prefixesndash Change the local preference assignments over time
bull Useful inputs to load balancingndash End-to-end path performance datandash Outbound traffic statistics per destination prefix
bull Challenge Getting from traffic volumes to groups of prefixes that should be assigned to each link
Premise of ldquointelligent route controlrdquo preoducts
59
Traffic Engineering Goals
bull Predictabilityndash Ensure the BGP decision process is deterministicndash Assume that BGP updates are (relatively) stable
bull Limit overhead introduced by routing changesndash Minimize frequency of changes to routing policiesndash Limit number of prefixes affected by changes
bull Limit impact on how traffic enters the networkndash Avoid new routes that might change neighborrsquos mindndash Select route with same attributes or at least path length
60
Managing Scalebull Destination prefixes
ndash More than 90000 destination prefixes
bull Donrsquot want to have per-prefix routing policies
ndash Small fraction of prefixes contribute most of the traffic
bull Focus on the small number of heavy hitters
ndash Define routing policies for selected prefixes
bull Routing choicesndash About 27000 unique ldquorouting choicesrdquo
bull Help in reducing the scale of the problem
ndash Small fraction of ldquorouting choicesrdquo contribute most traffic
bull Focus on the very small number of ldquorouting choicesrdquo
ndash Define routing policies on common attributes
61
Achieving Predictability
bull Route prediction with static analysisndash Helpful to know effects before deploymentndash Static analysis can help
TopologyBGP policy
configuration
eBGP routes
Offered traffic
BGP routingmodel
Flow of traffic through the network
62
Challenges to Predictabilitybull For transit ISPs effects on incoming traffic
ndash Lack of coordination strikes again
63
ldquoHot Potatordquo routing
Inter-AS Negotiation
bull Coordination aids predictabilityndash Negotiate where to sendndash Inbound and outboundndash Mutual benefits
bull How to implementndash What info to exchangendash Protecting privacyndash How to prioritize choicesndash How to prevent cheating
Destination 2
Destination 1
multiplepeeringpoints
Provider A
Provider B
64
Outbound Multihoming Goals
bull Redundancyndash Dynamic routing will failover to backup link
bull Performancendash Select provider with best performance per prefix
ndash Requires active probing
bull Costndash Select provider per prefix over time to minimize the
total financial cost
65
Inbound Traffic Control
bull More difficult no control over neighborsrsquo decisions
bull Three common techniques (previously discussed)ndash AS path prependingndash Communities and local preferencendash Prefix splitting
How does todayrsquos paper (MONET) control inbound traffic
66
How many links are enough
K upstream ISPs
Not much benefit beyond 4 ISPs
Akella et al ldquoPerformance Benefits of Multihomingrdquo SIGCOMM 2003
67
Problems with Multihoming in IPv4
bull Routing table growthndash Provider-based addressingndash Advertising prefix out multiple ISPs ndash canrsquot aggregate
bull Poor control over inbound trafficndash Existing mechanisms do not allow hosts to control
inbound traffic
68
GeorgiaTech
Internet Routing Overview
bull Intradomain (ie ldquointra-ASrdquo) routingbull Interdomain routing
Comcast
Abilene
ATampT Cogent
Autonomous Systems (ASes)
69
Configuration Problems ldquoAS 7007rdquoldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
UUNet
Florida InternetBarn
Sprint
70
Diagnosis and Troubleshootingldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
ldquoMicrosofts websites were offline for up to 23 hoursbecause of a [router] misconfigurationhellipit took nearly a day to determine what was wrong and undo the changesrdquo -- wiredcom January 25 2001
ldquoWorldCom Inchellipsuffered a widespread outage on its Internet backbone that affected roughly 20 percent of its US customer base The network problemshellipaffected millions of computer users worldwide A spokeswoman attributed the outage to a route table issue -- cnncom October 3 2002
A number of Covad customers went out from 5pm today due to supposedly a DDOS (distributed denial of service attack) on a key Level3 data center which later was described as a route leak (misconfiguration)ldquo
-- dslreportscom February 23 2004
71
Operator Mailing List (NANOG)Date Mon 18 Oct 2004 091515 -0700Subject Level 3 US east coast issues
Level 3 experiencing widespread unspecified routing issues on the US east coast Master ticket 1086844 Anyone have more specific information
Date Mon 18 Oct 2004 122034 -0400 (EDT)Subject Re Level 3 US east coast issues
Level 3 is currently experiencing a backbone outage causing routing instability and packet loss We are working to restore and will be sending hourly updateshellip
72
Operator Mailing List
0102030405060708090
Filtering RouteLeaks
RouteHijacks
RouteInstability
RoutingLoops
Blackholes
Occurr
ences o
ver
10 Y
ears
1995-1997 1998-2001 2001-2004
Note Only includes problems openly discussed on this list
Compare 83 power outages 1 fire
73
Routing Configuration
Ranking route selection
Dissemination internal route advertisement
Filtering route advertisement
Customer
Competitor
Primary
Backup
74
Internet Business Model (Simplified)
bull CustomerProvider One AS pays another for reachability to some set of destinations
bull ldquoSettlement-freerdquo Peering Bartering Two ASes exchange routes with one another
Provider
Peer
Customer
Preferences implemented with local preference manipulation
Destination
Pay to use
Get paid to use
Free to use
75
Peering Contracts Consistent Export
bull Rules of settlement-free peeringndash Advertise routes at all peering pointsndash Advertised routes must have equal ldquoAS path lengthrdquo
Sprint
ATampT
Enables ldquohot potatordquo routing
ldquoequally goodrdquoroutes
76
Consistent Export
bull Malicedeceptionbull iBGP signaling partitionbull Inconsistent export policy
neighbor 10123route-map PEER permit 10 set prepend 123
neighbor 10456route-map PEER permit 10 set prepend 123 123
Possible Causes
10123 456 1
10456 456 2
Neighbor AS Export
1 1 123
Export Clause Prepend
2 1 123 123
Two different Export Policies
77
Inconsistent Export in Practice
Feamster et al ldquoBorderGuard Detecting Cold Potatoes from Peersrdquo ACM IMC October 2004
78
Blackholes
Date Thu 18 Jul 2002 060510 -0400 (EDT)From Chad Oleary ltcolpoboxcomgtSubject Re problems with 701To ltnanogmeritedugt
Were starting to see the same issues with UUNet again Anyone elseseeing this Trying to reach Qwest
traceroute to 631461901 (631461901) 30 hops max 38 byte packets 1 esc-lp2-gwe-solutionscorpcom (631182201) 1167 ms 1163 ms 1142 ms 2 500Serial2-10GW1TPA2ALTERNET (1571301499) 1097 ms 1059 ms 1044 ms 3 161at-1-0-0XL4ATL1ALTERNET (1526381190) 13839 ms 14108 ms 16638 ms 4 0so-3-1-0XL2ATL5ALTERNET (152630238) 14370 ms 14587 ms 14553 ms 5 POS7-0BR2ATL5ALTERNET (1526382193) 13928 ms 14099 ms 14053 ms 6 7 hellip
79
Security
80
Security ldquoBogonrdquo Routes
Feamster et al ldquoAn Empirical Study of lsquoBogonrsquo Route Advertisementsrdquo ACM CCR January 2005
81
Spam Phishing etc
bull Unsolicited commercial emailbull As of about August 2008 estimates indicate that
about 95 of all email is spambull Common spam filtering techniques
ndash Content-based filtersndash DNS Blacklist (DNSBL) lookups Significant fraction of
todayrsquos DNS traffic
Can IP addresses from which spam is received be spoofed
82
Spam and Routing
83
Worms and Botnets
84
What is a Worm
bull Code that replicates and propagates across the networkndash Often carries a ldquopayloadrdquo
bull Usually spread via exploiting flaws in open servicesndash ldquoVirusesrdquo require user action to spread
bull First worm Robert Morris November 1988ndash 6-10 of all Internet hosts infected ()
bull Many more since but none on that scale until July 2001
85
Example Worm Code Red
bull Initial version July 13 2001
bull Exploited known ISAPI vulnerability in Microsoft IIS Web servers
bull 1st through 20th of each month spread20th through end of each month attack
bull Payload Web site defacementbull Scanning Random IP addressesbull Bug failure to seed random number generator
86
Code Red Revisions
bull Released July 19 2001
bull Payload flooding attack on wwwwhitehousegovndash Attack was mounted at the IP address of the Web site
bull Bug died after 20th of each month
bull Random number generator for IP scanning fixed
87
Code Red Host Infection Rate
Exponential infection rate
Measured using backscatter technique
88
Designing Fast-Spreading Worms
bull Hit-list scanningndash Time to infect first 10k hosts dominates infection timendash Solution Reconnaissance (stealthy scans etc)
bull Permutation scanningndash Observation Most scanning is redundantndash Idea Shared permutation of address space Start scanning
from own IP address Re-randomize when another infected machine is found
bull Internet-scale hit listsndash Flash worm complete infection within 30 seconds
89
Botnets
bull Bots Autonomous programs performing tasksbull Plenty of ldquobenignrdquo bots
ndash eg weatherbug
bull Botnets group of bots ndash Typically carries malicious connotationndash Large numbers of infected machinesndash Machines ldquoenlistedrdquo with infection vectors like worms
(last lecture)
bull Available for simultaneous control by a masterbull Size up to 350000 nodes (from todayrsquos paper)
90
ldquoRallyingrdquo the Botnet
bull Easy to combine worm backdoor functionalitybull Problem how to learn about successfully
infected machines
bull Optionsndash Emailndash Hard-coded email address
91
Botnet Control
bull Botnet master typically runs some IRC server on a well-known port (eg 6667)
bull Infected machine contacts botnet with pre-programmed DNS name (eg big-botde)
bull Dynamic DNS allows controller to move about freely
Infected Machine
DynamicDNS
BotnetController
(IRC server)
92
Some Defenses
93
Idea 1 Ingress Filtering
bull RFC 2827 Routers install filters to drop packets from networks that are not downstream
bull Feasible at edgesbull Difficult to configure closer to network ldquocorerdquo
20469207024 Internet
Drop all packets with source address other than 20469207024
94
Idea 2 uRPF Checks
bull Unicast Reverse Path Forwardingndash Cisco ldquoip verify unicast reverse-pathrdquo
bull Requires symmetric routing
Accept packet from interface only if forwarding table entry for source IP address matches ingress interface
100183
A10018
10015
101203
100181 24
10011 24
Strict Mode uRPF
Enabled
ldquoArdquo Routing TableDestination Next Hop1001024 Int 110018024 Int 2
100183 from wrong interface
95
Problems with uRPF
bull Asymmetric routing
96
S-BGP
bull Address-based PKI validate signaturesndash Authentication of
bull ownership for IP address blocks bull AS number bull an ASs identity and bull a BGP routers identity
ndash Use existing infrastructure (Internet registries etc)ndash Routing origination is digitally signedndash BGP updates are digitally signed
1048708 bull Route attestations A new optional BGP transitive path attribute
ndash carries digital signatures covering the routing information in updates
97
Practical Problems with S-BGP
bull Requires Public-Key Infrastructure
bull Lots of digital signatures to calculate and verifyndash Message overheadndash CPU overhead
bull Calculation expense is greatest when topology is changingndash Caching can help
bull Route aggregation is problematic (maybe thatrsquos OK)
bull Secure route withdrawals when link or node fails
bull Address ownership data out of date
bull Deployment
- Networking
- Goal of This Tutorial
- Todayrsquos Networks
- Business Models
- Billing for Internet Usage
- Net Neutrality
- Network Operations
- Point-of-Presence (PoP)
- Example Abilene Network Topology
- Another Example Backbone
- Internet Routing Overview
- Internet Routing Protocol BGP
- Two Flavors of BGP
- IPv4 Addresses Networks of Networks
- Pre-1994 Classful Addressing
- Classless Interdomain Routing (CIDR)
- Benefits of CIDR
- Growth of IP Prefixes
- 1994-1998 Linear Growth
- Around 2000 Fast Growth Resumes
- Fast growth resumes
- The Address Allocation Process
- Common Problems
- What can go wrong
- Measurement and Monitoring
- Passive vs Active Measurement
- Slide 27
- Passive Traffic Data Measurement
- Simple Network Management Protocol
- SNMP (Passive)
- Packet-level Monitoring
- Full Packet Capture (Passive)
- What is a flow
- Cisco NetFlow
- Flow Record Contents
- Aggregating Packets into Flows
- Reducing Measurement Overhead
- Packet Sampling for Flow Monitoring
- Sampling Flow-Level Sampling
- Two Main Approaches
- Packet Capture on High-Speed Links
- Characteristics of Packet Capture
- Data Measurement Repositories
- Multihoming and Traffic Engineering
- What is Multihoming
- Why Multihome
- Redundancy
- Performance
- Multihoming in IP Networks Today
- BGP or no
- Same Provider or Multiple
- Multihomed Stub One Link
- Multihomed Stub Multiple Links
- Multihomed Stub Multiple ISPs
- Multihomed Transit Network
- Interdomain Traffic Engineering
- Outbound Traffic Control
- Outbound Traffic Load Balancing
- Traffic Engineering Goals
- Managing Scale
- Achieving Predictability
- Challenges to Predictability
- Inter-AS Negotiation
- Outbound Multihoming Goals
- Inbound Traffic Control
- How many links are enough
- Problems with Multihoming in IPv4
- Slide 68
- Slide 69
- Diagnosis and Troubleshooting
- Operator Mailing List (NANOG)
- Operator Mailing List
- Routing Configuration
- Internet Business Model (Simplified)
- Peering Contracts Consistent Export
- Consistent Export
- Inconsistent Export in Practice
- Blackholes
- Security
- Security ldquoBogonrdquo Routes
- Spam Phishing etc
- Spam and Routing
- Worms and Botnets
- What is a Worm
- Example Worm Code Red
- Code Red Revisions
- Code Red Host Infection Rate
- Designing Fast-Spreading Worms
- Botnets
- ldquoRallyingrdquo the Botnet
- Botnet Control
- Some Defenses
- Idea 1 Ingress Filtering
- Idea 2 uRPF Checks
- Problems with uRPF
- S-BGP
- Practical Problems with S-BGP
-
36
flow 1 flow 2 flow 3 flow 4
Aggregating Packets into Flows
bull Criteria 1 Set of packets that ldquobelong togetherrdquondash Sourcedestination IP addresses and port numbersndash Same protocol ToS bits hellip ndash Same inputoutput interfaces at a router (if known)
bull Criteria 2 Packets that are ldquocloserdquo together in timendash Maximum inter-packet spacing (eg 15 sec 30 sec)ndash Example flows 2 and 4 are different flows due to time
37
Reducing Measurement Overhead
bull Filtering on interfacendash destination prefix for a customerndash port number for an application (eg 80 for Web)
bull Sampling before insertion into flow cachendash Random deterministic or hash-based samplingndash 1-out-of-n or stratified based on packetflow sizendash Two types packet-level and flow-level
bull Aggregation after cache evictionndash packetsflows with same next-hop ASndash packetsflows destined to a particular service
38
Packet Sampling for Flow Monitoring
bull Packet sampling before flow creation (Sampled Netflow)ndash 1-out-of-m sampling of individual packets (eg m=100)ndash Create of flow records over the sampled packets
bull Reducing overheadndash Avoid per-packet overhead on (m-1)m packetsndash Avoid creating records for a large number of small flows
bull Increasing overhead (in some cases)ndash May split some long transfers into multiple flow records ndash hellip due to larger time gaps between successive packets
time
not sampled
two flowstimeout
39
Sampling Flow-Level Sampling
bull Sampling of flow records evicted from flow cachendash When evicting flows from table or when analyzing flows
bull Stratified sampling to put weight on ldquoheavyrdquo flowsndash Select all long flows and sample the short flows
bull Reduces the number of flow records ndash Still measures the vast majority of the traffic
Flow 1 40 bytesFlow 2 15580 bytesFlow 3 8196 bytesFlow 4 5350789 bytesFlow 5 532 bytesFlow 6 7432 bytes
sample with 100 probability
sample with 01 probability
sample with 10 probability
40
Two Main Approaches
bull Packet-level Monitoringndash Keep packet-level statisticsndash Examine (and potentially log) variety of packet-level
statistics Essentially anything in the packetndash Timing
bull Flow-level Monitoringndash Monitor packet-by-packet (though sometimes
sampled)ndash Keep aggregate statistics on a flow
41
Packet Capture on High-Speed Links
Example Georgia Tech ldquoOC3Monrdquo
bull Rack-mounted PCbull Optical splitterbull Data Acquisition and
Generation (DAG) card
Source endacecom
42
Characteristics of Packet Capture
bull Allows inspection on every packet on 10G links
bull Disadvantagesndash Costlyndash Requires splitting optical fibersndash Must be able to filterstore data
43
Data Measurement Repositories
bull AbileneInternet 2 Observatoryndash Configuration examplesndash SNMP datandash ISIS BGP routing data NetFlow traffic data
bull RouteViewsndash BGP updatesndash BGP table snapshots
44
Multihoming and Traffic Engineering
45
What is Multihoming
bull The use of redundant network links for the purposes of external connectivity
bull Can be achieved at many layers of the protocol stack and many places in the networkndash Multiple network interfaces in a PC
ndash An ISP with multiple upstream interfaces
bull Can refer to having multiple connections tondash The same ISP
ndash Multiple ISPs
46
Why Multihome
bull Redundancybull Availabilitybull Performancebull Cost
Interdomain traffic engineering the process by which a multihomed network configures its
network to achieve these goals
47
Redundancy
bull Maintain connectivity in the face ofndash Physical connectivity problems (fiber cut device
failures etc)ndash Failures in upstream ISP
48
Performance
bull Use multiple network links at once to achieve higher throughput than just over a single link
bull Allows incoming traffic to be load-balanced
70 of traffic30 of traffic
49
Multihoming in IP Networks Today
bull Stub AS no transit service for other ASesndash No need to use BGP
bull Multi-homed stub AS has connectivity to multiple immediate upstream ISPsndash Need BGPndash No need for a public AS numberndash No need for IP prefix allocation
bull Multi-homed transit AS connectivity to multiple ASes and transit servicendash Need BGP public AS number IP prefix allocation
50
BGP or no
bull Advantages of static routingndash Cheapersmaller routers (less true nowadays)ndash Simpler to configure
bull Advantages of BGPndash More control of your destiny (have providers stop
announcing you)ndash Fastermore intelligent selection of where to send
outbound packetsndash Better debugging of net problems (you can see the
Internet topology now)
51
Same Provider or Multiple
bull If your provider is reliable and fast and affordably and offers good tech-support you may want to multi-home initially to them via some backup path (slow is better than dead)
bull Eventually yoursquoll want to multi-home to different providers to avoid failure modes due to one providerrsquos architecture decisions
52
Multihomed Stub One Link
bull Downstream ISPrsquos routers configure default (ldquostaticrdquo) routes pointing to border router
bull Upstream ISP advertises reachability
Upstream ISP
Multiple links between same pair of routers
Default routes to ldquoborderrdquo
ldquoStubrdquoISP
53
Multihomed Stub Multiple Links
bull Use BGP to share loadbull Use private AS number (why is this OK)bull As before upstream ISP advertises prefix
Upstream ISP
Multiple links to different upstream routers
ldquoStubrdquoISP
Internal routing for ldquohot potatordquo
BGP for load balance at edge
54
Multihomed Stub Multiple ISPs
bull Many possibilitiesndash Load sharingndash Primary-backupndash Selective use of different ISPs
bull Requires BGP public AS number etc
ldquoStubrdquoISP
Upstream
ISP 1
Upstream
ISP 2
55
Multihomed Transit Network
bull BGP everywherebull Incoming and outcoming trafficbull Challenge balancing load on intradomain and egress
links given an offered traffic load
TransitISP
ISP 1
ISP 2
ISP 3
56
Interdomain Traffic Engineering
bull The process by which a network operator configures the network to achievendash Traffic load balancendash Redundancy (primarybackup) etc
bull Two tasksndash Outbound traffic controlndash Inbound traffic control
bull Key Problems Predictability and Scalability
57
Outbound Traffic Control
bull Easier to control than inbound trafficndash Destination-based routing sender determines where
the packets go
bull Control over next-hop AS onlyndash Cannot control selection of the entire path
Provider 1 Provider 2
Control with local preference
58
Outbound Traffic Load Balancingbull Control routes to provider per-prefix
ndash Assign local preference across destination prefixesndash Change the local preference assignments over time
bull Useful inputs to load balancingndash End-to-end path performance datandash Outbound traffic statistics per destination prefix
bull Challenge Getting from traffic volumes to groups of prefixes that should be assigned to each link
Premise of ldquointelligent route controlrdquo preoducts
59
Traffic Engineering Goals
bull Predictabilityndash Ensure the BGP decision process is deterministicndash Assume that BGP updates are (relatively) stable
bull Limit overhead introduced by routing changesndash Minimize frequency of changes to routing policiesndash Limit number of prefixes affected by changes
bull Limit impact on how traffic enters the networkndash Avoid new routes that might change neighborrsquos mindndash Select route with same attributes or at least path length
60
Managing Scalebull Destination prefixes
ndash More than 90000 destination prefixes
bull Donrsquot want to have per-prefix routing policies
ndash Small fraction of prefixes contribute most of the traffic
bull Focus on the small number of heavy hitters
ndash Define routing policies for selected prefixes
bull Routing choicesndash About 27000 unique ldquorouting choicesrdquo
bull Help in reducing the scale of the problem
ndash Small fraction of ldquorouting choicesrdquo contribute most traffic
bull Focus on the very small number of ldquorouting choicesrdquo
ndash Define routing policies on common attributes
61
Achieving Predictability
bull Route prediction with static analysisndash Helpful to know effects before deploymentndash Static analysis can help
TopologyBGP policy
configuration
eBGP routes
Offered traffic
BGP routingmodel
Flow of traffic through the network
62
Challenges to Predictabilitybull For transit ISPs effects on incoming traffic
ndash Lack of coordination strikes again
63
ldquoHot Potatordquo routing
Inter-AS Negotiation
bull Coordination aids predictabilityndash Negotiate where to sendndash Inbound and outboundndash Mutual benefits
bull How to implementndash What info to exchangendash Protecting privacyndash How to prioritize choicesndash How to prevent cheating
Destination 2
Destination 1
multiplepeeringpoints
Provider A
Provider B
64
Outbound Multihoming Goals
bull Redundancyndash Dynamic routing will failover to backup link
bull Performancendash Select provider with best performance per prefix
ndash Requires active probing
bull Costndash Select provider per prefix over time to minimize the
total financial cost
65
Inbound Traffic Control
bull More difficult no control over neighborsrsquo decisions
bull Three common techniques (previously discussed)ndash AS path prependingndash Communities and local preferencendash Prefix splitting
How does todayrsquos paper (MONET) control inbound traffic
66
How many links are enough
K upstream ISPs
Not much benefit beyond 4 ISPs
Akella et al ldquoPerformance Benefits of Multihomingrdquo SIGCOMM 2003
67
Problems with Multihoming in IPv4
bull Routing table growthndash Provider-based addressingndash Advertising prefix out multiple ISPs ndash canrsquot aggregate
bull Poor control over inbound trafficndash Existing mechanisms do not allow hosts to control
inbound traffic
68
GeorgiaTech
Internet Routing Overview
bull Intradomain (ie ldquointra-ASrdquo) routingbull Interdomain routing
Comcast
Abilene
ATampT Cogent
Autonomous Systems (ASes)
69
Configuration Problems ldquoAS 7007rdquoldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
UUNet
Florida InternetBarn
Sprint
70
Diagnosis and Troubleshootingldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
ldquoMicrosofts websites were offline for up to 23 hoursbecause of a [router] misconfigurationhellipit took nearly a day to determine what was wrong and undo the changesrdquo -- wiredcom January 25 2001
ldquoWorldCom Inchellipsuffered a widespread outage on its Internet backbone that affected roughly 20 percent of its US customer base The network problemshellipaffected millions of computer users worldwide A spokeswoman attributed the outage to a route table issue -- cnncom October 3 2002
A number of Covad customers went out from 5pm today due to supposedly a DDOS (distributed denial of service attack) on a key Level3 data center which later was described as a route leak (misconfiguration)ldquo
-- dslreportscom February 23 2004
71
Operator Mailing List (NANOG)Date Mon 18 Oct 2004 091515 -0700Subject Level 3 US east coast issues
Level 3 experiencing widespread unspecified routing issues on the US east coast Master ticket 1086844 Anyone have more specific information
Date Mon 18 Oct 2004 122034 -0400 (EDT)Subject Re Level 3 US east coast issues
Level 3 is currently experiencing a backbone outage causing routing instability and packet loss We are working to restore and will be sending hourly updateshellip
72
Operator Mailing List
0102030405060708090
Filtering RouteLeaks
RouteHijacks
RouteInstability
RoutingLoops
Blackholes
Occurr
ences o
ver
10 Y
ears
1995-1997 1998-2001 2001-2004
Note Only includes problems openly discussed on this list
Compare 83 power outages 1 fire
73
Routing Configuration
Ranking route selection
Dissemination internal route advertisement
Filtering route advertisement
Customer
Competitor
Primary
Backup
74
Internet Business Model (Simplified)
bull CustomerProvider One AS pays another for reachability to some set of destinations
bull ldquoSettlement-freerdquo Peering Bartering Two ASes exchange routes with one another
Provider
Peer
Customer
Preferences implemented with local preference manipulation
Destination
Pay to use
Get paid to use
Free to use
75
Peering Contracts Consistent Export
bull Rules of settlement-free peeringndash Advertise routes at all peering pointsndash Advertised routes must have equal ldquoAS path lengthrdquo
Sprint
ATampT
Enables ldquohot potatordquo routing
ldquoequally goodrdquoroutes
76
Consistent Export
bull Malicedeceptionbull iBGP signaling partitionbull Inconsistent export policy
neighbor 10123route-map PEER permit 10 set prepend 123
neighbor 10456route-map PEER permit 10 set prepend 123 123
Possible Causes
10123 456 1
10456 456 2
Neighbor AS Export
1 1 123
Export Clause Prepend
2 1 123 123
Two different Export Policies
77
Inconsistent Export in Practice
Feamster et al ldquoBorderGuard Detecting Cold Potatoes from Peersrdquo ACM IMC October 2004
78
Blackholes
Date Thu 18 Jul 2002 060510 -0400 (EDT)From Chad Oleary ltcolpoboxcomgtSubject Re problems with 701To ltnanogmeritedugt
Were starting to see the same issues with UUNet again Anyone elseseeing this Trying to reach Qwest
traceroute to 631461901 (631461901) 30 hops max 38 byte packets 1 esc-lp2-gwe-solutionscorpcom (631182201) 1167 ms 1163 ms 1142 ms 2 500Serial2-10GW1TPA2ALTERNET (1571301499) 1097 ms 1059 ms 1044 ms 3 161at-1-0-0XL4ATL1ALTERNET (1526381190) 13839 ms 14108 ms 16638 ms 4 0so-3-1-0XL2ATL5ALTERNET (152630238) 14370 ms 14587 ms 14553 ms 5 POS7-0BR2ATL5ALTERNET (1526382193) 13928 ms 14099 ms 14053 ms 6 7 hellip
79
Security
80
Security ldquoBogonrdquo Routes
Feamster et al ldquoAn Empirical Study of lsquoBogonrsquo Route Advertisementsrdquo ACM CCR January 2005
81
Spam Phishing etc
bull Unsolicited commercial emailbull As of about August 2008 estimates indicate that
about 95 of all email is spambull Common spam filtering techniques
ndash Content-based filtersndash DNS Blacklist (DNSBL) lookups Significant fraction of
todayrsquos DNS traffic
Can IP addresses from which spam is received be spoofed
82
Spam and Routing
83
Worms and Botnets
84
What is a Worm
bull Code that replicates and propagates across the networkndash Often carries a ldquopayloadrdquo
bull Usually spread via exploiting flaws in open servicesndash ldquoVirusesrdquo require user action to spread
bull First worm Robert Morris November 1988ndash 6-10 of all Internet hosts infected ()
bull Many more since but none on that scale until July 2001
85
Example Worm Code Red
bull Initial version July 13 2001
bull Exploited known ISAPI vulnerability in Microsoft IIS Web servers
bull 1st through 20th of each month spread20th through end of each month attack
bull Payload Web site defacementbull Scanning Random IP addressesbull Bug failure to seed random number generator
86
Code Red Revisions
bull Released July 19 2001
bull Payload flooding attack on wwwwhitehousegovndash Attack was mounted at the IP address of the Web site
bull Bug died after 20th of each month
bull Random number generator for IP scanning fixed
87
Code Red Host Infection Rate
Exponential infection rate
Measured using backscatter technique
88
Designing Fast-Spreading Worms
bull Hit-list scanningndash Time to infect first 10k hosts dominates infection timendash Solution Reconnaissance (stealthy scans etc)
bull Permutation scanningndash Observation Most scanning is redundantndash Idea Shared permutation of address space Start scanning
from own IP address Re-randomize when another infected machine is found
bull Internet-scale hit listsndash Flash worm complete infection within 30 seconds
89
Botnets
bull Bots Autonomous programs performing tasksbull Plenty of ldquobenignrdquo bots
ndash eg weatherbug
bull Botnets group of bots ndash Typically carries malicious connotationndash Large numbers of infected machinesndash Machines ldquoenlistedrdquo with infection vectors like worms
(last lecture)
bull Available for simultaneous control by a masterbull Size up to 350000 nodes (from todayrsquos paper)
90
ldquoRallyingrdquo the Botnet
bull Easy to combine worm backdoor functionalitybull Problem how to learn about successfully
infected machines
bull Optionsndash Emailndash Hard-coded email address
91
Botnet Control
bull Botnet master typically runs some IRC server on a well-known port (eg 6667)
bull Infected machine contacts botnet with pre-programmed DNS name (eg big-botde)
bull Dynamic DNS allows controller to move about freely
Infected Machine
DynamicDNS
BotnetController
(IRC server)
92
Some Defenses
93
Idea 1 Ingress Filtering
bull RFC 2827 Routers install filters to drop packets from networks that are not downstream
bull Feasible at edgesbull Difficult to configure closer to network ldquocorerdquo
20469207024 Internet
Drop all packets with source address other than 20469207024
94
Idea 2 uRPF Checks
bull Unicast Reverse Path Forwardingndash Cisco ldquoip verify unicast reverse-pathrdquo
bull Requires symmetric routing
Accept packet from interface only if forwarding table entry for source IP address matches ingress interface
100183
A10018
10015
101203
100181 24
10011 24
Strict Mode uRPF
Enabled
ldquoArdquo Routing TableDestination Next Hop1001024 Int 110018024 Int 2
100183 from wrong interface
95
Problems with uRPF
bull Asymmetric routing
96
S-BGP
bull Address-based PKI validate signaturesndash Authentication of
bull ownership for IP address blocks bull AS number bull an ASs identity and bull a BGP routers identity
ndash Use existing infrastructure (Internet registries etc)ndash Routing origination is digitally signedndash BGP updates are digitally signed
1048708 bull Route attestations A new optional BGP transitive path attribute
ndash carries digital signatures covering the routing information in updates
97
Practical Problems with S-BGP
bull Requires Public-Key Infrastructure
bull Lots of digital signatures to calculate and verifyndash Message overheadndash CPU overhead
bull Calculation expense is greatest when topology is changingndash Caching can help
bull Route aggregation is problematic (maybe thatrsquos OK)
bull Secure route withdrawals when link or node fails
bull Address ownership data out of date
bull Deployment
- Networking
- Goal of This Tutorial
- Todayrsquos Networks
- Business Models
- Billing for Internet Usage
- Net Neutrality
- Network Operations
- Point-of-Presence (PoP)
- Example Abilene Network Topology
- Another Example Backbone
- Internet Routing Overview
- Internet Routing Protocol BGP
- Two Flavors of BGP
- IPv4 Addresses Networks of Networks
- Pre-1994 Classful Addressing
- Classless Interdomain Routing (CIDR)
- Benefits of CIDR
- Growth of IP Prefixes
- 1994-1998 Linear Growth
- Around 2000 Fast Growth Resumes
- Fast growth resumes
- The Address Allocation Process
- Common Problems
- What can go wrong
- Measurement and Monitoring
- Passive vs Active Measurement
- Slide 27
- Passive Traffic Data Measurement
- Simple Network Management Protocol
- SNMP (Passive)
- Packet-level Monitoring
- Full Packet Capture (Passive)
- What is a flow
- Cisco NetFlow
- Flow Record Contents
- Aggregating Packets into Flows
- Reducing Measurement Overhead
- Packet Sampling for Flow Monitoring
- Sampling Flow-Level Sampling
- Two Main Approaches
- Packet Capture on High-Speed Links
- Characteristics of Packet Capture
- Data Measurement Repositories
- Multihoming and Traffic Engineering
- What is Multihoming
- Why Multihome
- Redundancy
- Performance
- Multihoming in IP Networks Today
- BGP or no
- Same Provider or Multiple
- Multihomed Stub One Link
- Multihomed Stub Multiple Links
- Multihomed Stub Multiple ISPs
- Multihomed Transit Network
- Interdomain Traffic Engineering
- Outbound Traffic Control
- Outbound Traffic Load Balancing
- Traffic Engineering Goals
- Managing Scale
- Achieving Predictability
- Challenges to Predictability
- Inter-AS Negotiation
- Outbound Multihoming Goals
- Inbound Traffic Control
- How many links are enough
- Problems with Multihoming in IPv4
- Slide 68
- Slide 69
- Diagnosis and Troubleshooting
- Operator Mailing List (NANOG)
- Operator Mailing List
- Routing Configuration
- Internet Business Model (Simplified)
- Peering Contracts Consistent Export
- Consistent Export
- Inconsistent Export in Practice
- Blackholes
- Security
- Security ldquoBogonrdquo Routes
- Spam Phishing etc
- Spam and Routing
- Worms and Botnets
- What is a Worm
- Example Worm Code Red
- Code Red Revisions
- Code Red Host Infection Rate
- Designing Fast-Spreading Worms
- Botnets
- ldquoRallyingrdquo the Botnet
- Botnet Control
- Some Defenses
- Idea 1 Ingress Filtering
- Idea 2 uRPF Checks
- Problems with uRPF
- S-BGP
- Practical Problems with S-BGP
-
37
Reducing Measurement Overhead
bull Filtering on interfacendash destination prefix for a customerndash port number for an application (eg 80 for Web)
bull Sampling before insertion into flow cachendash Random deterministic or hash-based samplingndash 1-out-of-n or stratified based on packetflow sizendash Two types packet-level and flow-level
bull Aggregation after cache evictionndash packetsflows with same next-hop ASndash packetsflows destined to a particular service
38
Packet Sampling for Flow Monitoring
bull Packet sampling before flow creation (Sampled Netflow)ndash 1-out-of-m sampling of individual packets (eg m=100)ndash Create of flow records over the sampled packets
bull Reducing overheadndash Avoid per-packet overhead on (m-1)m packetsndash Avoid creating records for a large number of small flows
bull Increasing overhead (in some cases)ndash May split some long transfers into multiple flow records ndash hellip due to larger time gaps between successive packets
time
not sampled
two flowstimeout
39
Sampling Flow-Level Sampling
bull Sampling of flow records evicted from flow cachendash When evicting flows from table or when analyzing flows
bull Stratified sampling to put weight on ldquoheavyrdquo flowsndash Select all long flows and sample the short flows
bull Reduces the number of flow records ndash Still measures the vast majority of the traffic
Flow 1 40 bytesFlow 2 15580 bytesFlow 3 8196 bytesFlow 4 5350789 bytesFlow 5 532 bytesFlow 6 7432 bytes
sample with 100 probability
sample with 01 probability
sample with 10 probability
40
Two Main Approaches
bull Packet-level Monitoringndash Keep packet-level statisticsndash Examine (and potentially log) variety of packet-level
statistics Essentially anything in the packetndash Timing
bull Flow-level Monitoringndash Monitor packet-by-packet (though sometimes
sampled)ndash Keep aggregate statistics on a flow
41
Packet Capture on High-Speed Links
Example Georgia Tech ldquoOC3Monrdquo
bull Rack-mounted PCbull Optical splitterbull Data Acquisition and
Generation (DAG) card
Source endacecom
42
Characteristics of Packet Capture
bull Allows inspection on every packet on 10G links
bull Disadvantagesndash Costlyndash Requires splitting optical fibersndash Must be able to filterstore data
43
Data Measurement Repositories
bull AbileneInternet 2 Observatoryndash Configuration examplesndash SNMP datandash ISIS BGP routing data NetFlow traffic data
bull RouteViewsndash BGP updatesndash BGP table snapshots
44
Multihoming and Traffic Engineering
45
What is Multihoming
bull The use of redundant network links for the purposes of external connectivity
bull Can be achieved at many layers of the protocol stack and many places in the networkndash Multiple network interfaces in a PC
ndash An ISP with multiple upstream interfaces
bull Can refer to having multiple connections tondash The same ISP
ndash Multiple ISPs
46
Why Multihome
bull Redundancybull Availabilitybull Performancebull Cost
Interdomain traffic engineering the process by which a multihomed network configures its
network to achieve these goals
47
Redundancy
bull Maintain connectivity in the face ofndash Physical connectivity problems (fiber cut device
failures etc)ndash Failures in upstream ISP
48
Performance
bull Use multiple network links at once to achieve higher throughput than just over a single link
bull Allows incoming traffic to be load-balanced
70 of traffic30 of traffic
49
Multihoming in IP Networks Today
bull Stub AS no transit service for other ASesndash No need to use BGP
bull Multi-homed stub AS has connectivity to multiple immediate upstream ISPsndash Need BGPndash No need for a public AS numberndash No need for IP prefix allocation
bull Multi-homed transit AS connectivity to multiple ASes and transit servicendash Need BGP public AS number IP prefix allocation
50
BGP or no
bull Advantages of static routingndash Cheapersmaller routers (less true nowadays)ndash Simpler to configure
bull Advantages of BGPndash More control of your destiny (have providers stop
announcing you)ndash Fastermore intelligent selection of where to send
outbound packetsndash Better debugging of net problems (you can see the
Internet topology now)
51
Same Provider or Multiple
bull If your provider is reliable and fast and affordably and offers good tech-support you may want to multi-home initially to them via some backup path (slow is better than dead)
bull Eventually yoursquoll want to multi-home to different providers to avoid failure modes due to one providerrsquos architecture decisions
52
Multihomed Stub One Link
bull Downstream ISPrsquos routers configure default (ldquostaticrdquo) routes pointing to border router
bull Upstream ISP advertises reachability
Upstream ISP
Multiple links between same pair of routers
Default routes to ldquoborderrdquo
ldquoStubrdquoISP
53
Multihomed Stub Multiple Links
bull Use BGP to share loadbull Use private AS number (why is this OK)bull As before upstream ISP advertises prefix
Upstream ISP
Multiple links to different upstream routers
ldquoStubrdquoISP
Internal routing for ldquohot potatordquo
BGP for load balance at edge
54
Multihomed Stub Multiple ISPs
bull Many possibilitiesndash Load sharingndash Primary-backupndash Selective use of different ISPs
bull Requires BGP public AS number etc
ldquoStubrdquoISP
Upstream
ISP 1
Upstream
ISP 2
55
Multihomed Transit Network
bull BGP everywherebull Incoming and outcoming trafficbull Challenge balancing load on intradomain and egress
links given an offered traffic load
TransitISP
ISP 1
ISP 2
ISP 3
56
Interdomain Traffic Engineering
bull The process by which a network operator configures the network to achievendash Traffic load balancendash Redundancy (primarybackup) etc
bull Two tasksndash Outbound traffic controlndash Inbound traffic control
bull Key Problems Predictability and Scalability
57
Outbound Traffic Control
bull Easier to control than inbound trafficndash Destination-based routing sender determines where
the packets go
bull Control over next-hop AS onlyndash Cannot control selection of the entire path
Provider 1 Provider 2
Control with local preference
58
Outbound Traffic Load Balancingbull Control routes to provider per-prefix
ndash Assign local preference across destination prefixesndash Change the local preference assignments over time
bull Useful inputs to load balancingndash End-to-end path performance datandash Outbound traffic statistics per destination prefix
bull Challenge Getting from traffic volumes to groups of prefixes that should be assigned to each link
Premise of ldquointelligent route controlrdquo preoducts
59
Traffic Engineering Goals
bull Predictabilityndash Ensure the BGP decision process is deterministicndash Assume that BGP updates are (relatively) stable
bull Limit overhead introduced by routing changesndash Minimize frequency of changes to routing policiesndash Limit number of prefixes affected by changes
bull Limit impact on how traffic enters the networkndash Avoid new routes that might change neighborrsquos mindndash Select route with same attributes or at least path length
60
Managing Scalebull Destination prefixes
ndash More than 90000 destination prefixes
bull Donrsquot want to have per-prefix routing policies
ndash Small fraction of prefixes contribute most of the traffic
bull Focus on the small number of heavy hitters
ndash Define routing policies for selected prefixes
bull Routing choicesndash About 27000 unique ldquorouting choicesrdquo
bull Help in reducing the scale of the problem
ndash Small fraction of ldquorouting choicesrdquo contribute most traffic
bull Focus on the very small number of ldquorouting choicesrdquo
ndash Define routing policies on common attributes
61
Achieving Predictability
bull Route prediction with static analysisndash Helpful to know effects before deploymentndash Static analysis can help
TopologyBGP policy
configuration
eBGP routes
Offered traffic
BGP routingmodel
Flow of traffic through the network
62
Challenges to Predictabilitybull For transit ISPs effects on incoming traffic
ndash Lack of coordination strikes again
63
ldquoHot Potatordquo routing
Inter-AS Negotiation
bull Coordination aids predictabilityndash Negotiate where to sendndash Inbound and outboundndash Mutual benefits
bull How to implementndash What info to exchangendash Protecting privacyndash How to prioritize choicesndash How to prevent cheating
Destination 2
Destination 1
multiplepeeringpoints
Provider A
Provider B
64
Outbound Multihoming Goals
bull Redundancyndash Dynamic routing will failover to backup link
bull Performancendash Select provider with best performance per prefix
ndash Requires active probing
bull Costndash Select provider per prefix over time to minimize the
total financial cost
65
Inbound Traffic Control
bull More difficult no control over neighborsrsquo decisions
bull Three common techniques (previously discussed)ndash AS path prependingndash Communities and local preferencendash Prefix splitting
How does todayrsquos paper (MONET) control inbound traffic
66
How many links are enough
K upstream ISPs
Not much benefit beyond 4 ISPs
Akella et al ldquoPerformance Benefits of Multihomingrdquo SIGCOMM 2003
67
Problems with Multihoming in IPv4
bull Routing table growthndash Provider-based addressingndash Advertising prefix out multiple ISPs ndash canrsquot aggregate
bull Poor control over inbound trafficndash Existing mechanisms do not allow hosts to control
inbound traffic
68
GeorgiaTech
Internet Routing Overview
bull Intradomain (ie ldquointra-ASrdquo) routingbull Interdomain routing
Comcast
Abilene
ATampT Cogent
Autonomous Systems (ASes)
69
Configuration Problems ldquoAS 7007rdquoldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
UUNet
Florida InternetBarn
Sprint
70
Diagnosis and Troubleshootingldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
ldquoMicrosofts websites were offline for up to 23 hoursbecause of a [router] misconfigurationhellipit took nearly a day to determine what was wrong and undo the changesrdquo -- wiredcom January 25 2001
ldquoWorldCom Inchellipsuffered a widespread outage on its Internet backbone that affected roughly 20 percent of its US customer base The network problemshellipaffected millions of computer users worldwide A spokeswoman attributed the outage to a route table issue -- cnncom October 3 2002
A number of Covad customers went out from 5pm today due to supposedly a DDOS (distributed denial of service attack) on a key Level3 data center which later was described as a route leak (misconfiguration)ldquo
-- dslreportscom February 23 2004
71
Operator Mailing List (NANOG)Date Mon 18 Oct 2004 091515 -0700Subject Level 3 US east coast issues
Level 3 experiencing widespread unspecified routing issues on the US east coast Master ticket 1086844 Anyone have more specific information
Date Mon 18 Oct 2004 122034 -0400 (EDT)Subject Re Level 3 US east coast issues
Level 3 is currently experiencing a backbone outage causing routing instability and packet loss We are working to restore and will be sending hourly updateshellip
72
Operator Mailing List
0102030405060708090
Filtering RouteLeaks
RouteHijacks
RouteInstability
RoutingLoops
Blackholes
Occurr
ences o
ver
10 Y
ears
1995-1997 1998-2001 2001-2004
Note Only includes problems openly discussed on this list
Compare 83 power outages 1 fire
73
Routing Configuration
Ranking route selection
Dissemination internal route advertisement
Filtering route advertisement
Customer
Competitor
Primary
Backup
74
Internet Business Model (Simplified)
bull CustomerProvider One AS pays another for reachability to some set of destinations
bull ldquoSettlement-freerdquo Peering Bartering Two ASes exchange routes with one another
Provider
Peer
Customer
Preferences implemented with local preference manipulation
Destination
Pay to use
Get paid to use
Free to use
75
Peering Contracts Consistent Export
bull Rules of settlement-free peeringndash Advertise routes at all peering pointsndash Advertised routes must have equal ldquoAS path lengthrdquo
Sprint
ATampT
Enables ldquohot potatordquo routing
ldquoequally goodrdquoroutes
76
Consistent Export
bull Malicedeceptionbull iBGP signaling partitionbull Inconsistent export policy
neighbor 10123route-map PEER permit 10 set prepend 123
neighbor 10456route-map PEER permit 10 set prepend 123 123
Possible Causes
10123 456 1
10456 456 2
Neighbor AS Export
1 1 123
Export Clause Prepend
2 1 123 123
Two different Export Policies
77
Inconsistent Export in Practice
Feamster et al ldquoBorderGuard Detecting Cold Potatoes from Peersrdquo ACM IMC October 2004
78
Blackholes
Date Thu 18 Jul 2002 060510 -0400 (EDT)From Chad Oleary ltcolpoboxcomgtSubject Re problems with 701To ltnanogmeritedugt
Were starting to see the same issues with UUNet again Anyone elseseeing this Trying to reach Qwest
traceroute to 631461901 (631461901) 30 hops max 38 byte packets 1 esc-lp2-gwe-solutionscorpcom (631182201) 1167 ms 1163 ms 1142 ms 2 500Serial2-10GW1TPA2ALTERNET (1571301499) 1097 ms 1059 ms 1044 ms 3 161at-1-0-0XL4ATL1ALTERNET (1526381190) 13839 ms 14108 ms 16638 ms 4 0so-3-1-0XL2ATL5ALTERNET (152630238) 14370 ms 14587 ms 14553 ms 5 POS7-0BR2ATL5ALTERNET (1526382193) 13928 ms 14099 ms 14053 ms 6 7 hellip
79
Security
80
Security ldquoBogonrdquo Routes
Feamster et al ldquoAn Empirical Study of lsquoBogonrsquo Route Advertisementsrdquo ACM CCR January 2005
81
Spam Phishing etc
bull Unsolicited commercial emailbull As of about August 2008 estimates indicate that
about 95 of all email is spambull Common spam filtering techniques
ndash Content-based filtersndash DNS Blacklist (DNSBL) lookups Significant fraction of
todayrsquos DNS traffic
Can IP addresses from which spam is received be spoofed
82
Spam and Routing
83
Worms and Botnets
84
What is a Worm
bull Code that replicates and propagates across the networkndash Often carries a ldquopayloadrdquo
bull Usually spread via exploiting flaws in open servicesndash ldquoVirusesrdquo require user action to spread
bull First worm Robert Morris November 1988ndash 6-10 of all Internet hosts infected ()
bull Many more since but none on that scale until July 2001
85
Example Worm Code Red
bull Initial version July 13 2001
bull Exploited known ISAPI vulnerability in Microsoft IIS Web servers
bull 1st through 20th of each month spread20th through end of each month attack
bull Payload Web site defacementbull Scanning Random IP addressesbull Bug failure to seed random number generator
86
Code Red Revisions
bull Released July 19 2001
bull Payload flooding attack on wwwwhitehousegovndash Attack was mounted at the IP address of the Web site
bull Bug died after 20th of each month
bull Random number generator for IP scanning fixed
87
Code Red Host Infection Rate
Exponential infection rate
Measured using backscatter technique
88
Designing Fast-Spreading Worms
bull Hit-list scanningndash Time to infect first 10k hosts dominates infection timendash Solution Reconnaissance (stealthy scans etc)
bull Permutation scanningndash Observation Most scanning is redundantndash Idea Shared permutation of address space Start scanning
from own IP address Re-randomize when another infected machine is found
bull Internet-scale hit listsndash Flash worm complete infection within 30 seconds
89
Botnets
bull Bots Autonomous programs performing tasksbull Plenty of ldquobenignrdquo bots
ndash eg weatherbug
bull Botnets group of bots ndash Typically carries malicious connotationndash Large numbers of infected machinesndash Machines ldquoenlistedrdquo with infection vectors like worms
(last lecture)
bull Available for simultaneous control by a masterbull Size up to 350000 nodes (from todayrsquos paper)
90
ldquoRallyingrdquo the Botnet
bull Easy to combine worm backdoor functionalitybull Problem how to learn about successfully
infected machines
bull Optionsndash Emailndash Hard-coded email address
91
Botnet Control
bull Botnet master typically runs some IRC server on a well-known port (eg 6667)
bull Infected machine contacts botnet with pre-programmed DNS name (eg big-botde)
bull Dynamic DNS allows controller to move about freely
Infected Machine
DynamicDNS
BotnetController
(IRC server)
92
Some Defenses
93
Idea 1 Ingress Filtering
bull RFC 2827 Routers install filters to drop packets from networks that are not downstream
bull Feasible at edgesbull Difficult to configure closer to network ldquocorerdquo
20469207024 Internet
Drop all packets with source address other than 20469207024
94
Idea 2 uRPF Checks
bull Unicast Reverse Path Forwardingndash Cisco ldquoip verify unicast reverse-pathrdquo
bull Requires symmetric routing
Accept packet from interface only if forwarding table entry for source IP address matches ingress interface
100183
A10018
10015
101203
100181 24
10011 24
Strict Mode uRPF
Enabled
ldquoArdquo Routing TableDestination Next Hop1001024 Int 110018024 Int 2
100183 from wrong interface
95
Problems with uRPF
bull Asymmetric routing
96
S-BGP
bull Address-based PKI validate signaturesndash Authentication of
bull ownership for IP address blocks bull AS number bull an ASs identity and bull a BGP routers identity
ndash Use existing infrastructure (Internet registries etc)ndash Routing origination is digitally signedndash BGP updates are digitally signed
1048708 bull Route attestations A new optional BGP transitive path attribute
ndash carries digital signatures covering the routing information in updates
97
Practical Problems with S-BGP
bull Requires Public-Key Infrastructure
bull Lots of digital signatures to calculate and verifyndash Message overheadndash CPU overhead
bull Calculation expense is greatest when topology is changingndash Caching can help
bull Route aggregation is problematic (maybe thatrsquos OK)
bull Secure route withdrawals when link or node fails
bull Address ownership data out of date
bull Deployment
- Networking
- Goal of This Tutorial
- Todayrsquos Networks
- Business Models
- Billing for Internet Usage
- Net Neutrality
- Network Operations
- Point-of-Presence (PoP)
- Example Abilene Network Topology
- Another Example Backbone
- Internet Routing Overview
- Internet Routing Protocol BGP
- Two Flavors of BGP
- IPv4 Addresses Networks of Networks
- Pre-1994 Classful Addressing
- Classless Interdomain Routing (CIDR)
- Benefits of CIDR
- Growth of IP Prefixes
- 1994-1998 Linear Growth
- Around 2000 Fast Growth Resumes
- Fast growth resumes
- The Address Allocation Process
- Common Problems
- What can go wrong
- Measurement and Monitoring
- Passive vs Active Measurement
- Slide 27
- Passive Traffic Data Measurement
- Simple Network Management Protocol
- SNMP (Passive)
- Packet-level Monitoring
- Full Packet Capture (Passive)
- What is a flow
- Cisco NetFlow
- Flow Record Contents
- Aggregating Packets into Flows
- Reducing Measurement Overhead
- Packet Sampling for Flow Monitoring
- Sampling Flow-Level Sampling
- Two Main Approaches
- Packet Capture on High-Speed Links
- Characteristics of Packet Capture
- Data Measurement Repositories
- Multihoming and Traffic Engineering
- What is Multihoming
- Why Multihome
- Redundancy
- Performance
- Multihoming in IP Networks Today
- BGP or no
- Same Provider or Multiple
- Multihomed Stub One Link
- Multihomed Stub Multiple Links
- Multihomed Stub Multiple ISPs
- Multihomed Transit Network
- Interdomain Traffic Engineering
- Outbound Traffic Control
- Outbound Traffic Load Balancing
- Traffic Engineering Goals
- Managing Scale
- Achieving Predictability
- Challenges to Predictability
- Inter-AS Negotiation
- Outbound Multihoming Goals
- Inbound Traffic Control
- How many links are enough
- Problems with Multihoming in IPv4
- Slide 68
- Slide 69
- Diagnosis and Troubleshooting
- Operator Mailing List (NANOG)
- Operator Mailing List
- Routing Configuration
- Internet Business Model (Simplified)
- Peering Contracts Consistent Export
- Consistent Export
- Inconsistent Export in Practice
- Blackholes
- Security
- Security ldquoBogonrdquo Routes
- Spam Phishing etc
- Spam and Routing
- Worms and Botnets
- What is a Worm
- Example Worm Code Red
- Code Red Revisions
- Code Red Host Infection Rate
- Designing Fast-Spreading Worms
- Botnets
- ldquoRallyingrdquo the Botnet
- Botnet Control
- Some Defenses
- Idea 1 Ingress Filtering
- Idea 2 uRPF Checks
- Problems with uRPF
- S-BGP
- Practical Problems with S-BGP
-
38
Packet Sampling for Flow Monitoring
bull Packet sampling before flow creation (Sampled Netflow)ndash 1-out-of-m sampling of individual packets (eg m=100)ndash Create of flow records over the sampled packets
bull Reducing overheadndash Avoid per-packet overhead on (m-1)m packetsndash Avoid creating records for a large number of small flows
bull Increasing overhead (in some cases)ndash May split some long transfers into multiple flow records ndash hellip due to larger time gaps between successive packets
time
not sampled
two flowstimeout
39
Sampling Flow-Level Sampling
bull Sampling of flow records evicted from flow cachendash When evicting flows from table or when analyzing flows
bull Stratified sampling to put weight on ldquoheavyrdquo flowsndash Select all long flows and sample the short flows
bull Reduces the number of flow records ndash Still measures the vast majority of the traffic
Flow 1 40 bytesFlow 2 15580 bytesFlow 3 8196 bytesFlow 4 5350789 bytesFlow 5 532 bytesFlow 6 7432 bytes
sample with 100 probability
sample with 01 probability
sample with 10 probability
40
Two Main Approaches
bull Packet-level Monitoringndash Keep packet-level statisticsndash Examine (and potentially log) variety of packet-level
statistics Essentially anything in the packetndash Timing
bull Flow-level Monitoringndash Monitor packet-by-packet (though sometimes
sampled)ndash Keep aggregate statistics on a flow
41
Packet Capture on High-Speed Links
Example Georgia Tech ldquoOC3Monrdquo
bull Rack-mounted PCbull Optical splitterbull Data Acquisition and
Generation (DAG) card
Source endacecom
42
Characteristics of Packet Capture
bull Allows inspection on every packet on 10G links
bull Disadvantagesndash Costlyndash Requires splitting optical fibersndash Must be able to filterstore data
43
Data Measurement Repositories
bull AbileneInternet 2 Observatoryndash Configuration examplesndash SNMP datandash ISIS BGP routing data NetFlow traffic data
bull RouteViewsndash BGP updatesndash BGP table snapshots
44
Multihoming and Traffic Engineering
45
What is Multihoming
bull The use of redundant network links for the purposes of external connectivity
bull Can be achieved at many layers of the protocol stack and many places in the networkndash Multiple network interfaces in a PC
ndash An ISP with multiple upstream interfaces
bull Can refer to having multiple connections tondash The same ISP
ndash Multiple ISPs
46
Why Multihome
bull Redundancybull Availabilitybull Performancebull Cost
Interdomain traffic engineering the process by which a multihomed network configures its
network to achieve these goals
47
Redundancy
bull Maintain connectivity in the face ofndash Physical connectivity problems (fiber cut device
failures etc)ndash Failures in upstream ISP
48
Performance
bull Use multiple network links at once to achieve higher throughput than just over a single link
bull Allows incoming traffic to be load-balanced
70 of traffic30 of traffic
49
Multihoming in IP Networks Today
bull Stub AS no transit service for other ASesndash No need to use BGP
bull Multi-homed stub AS has connectivity to multiple immediate upstream ISPsndash Need BGPndash No need for a public AS numberndash No need for IP prefix allocation
bull Multi-homed transit AS connectivity to multiple ASes and transit servicendash Need BGP public AS number IP prefix allocation
50
BGP or no
bull Advantages of static routingndash Cheapersmaller routers (less true nowadays)ndash Simpler to configure
bull Advantages of BGPndash More control of your destiny (have providers stop
announcing you)ndash Fastermore intelligent selection of where to send
outbound packetsndash Better debugging of net problems (you can see the
Internet topology now)
51
Same Provider or Multiple
bull If your provider is reliable and fast and affordably and offers good tech-support you may want to multi-home initially to them via some backup path (slow is better than dead)
bull Eventually yoursquoll want to multi-home to different providers to avoid failure modes due to one providerrsquos architecture decisions
52
Multihomed Stub One Link
bull Downstream ISPrsquos routers configure default (ldquostaticrdquo) routes pointing to border router
bull Upstream ISP advertises reachability
Upstream ISP
Multiple links between same pair of routers
Default routes to ldquoborderrdquo
ldquoStubrdquoISP
53
Multihomed Stub Multiple Links
bull Use BGP to share loadbull Use private AS number (why is this OK)bull As before upstream ISP advertises prefix
Upstream ISP
Multiple links to different upstream routers
ldquoStubrdquoISP
Internal routing for ldquohot potatordquo
BGP for load balance at edge
54
Multihomed Stub Multiple ISPs
bull Many possibilitiesndash Load sharingndash Primary-backupndash Selective use of different ISPs
bull Requires BGP public AS number etc
ldquoStubrdquoISP
Upstream
ISP 1
Upstream
ISP 2
55
Multihomed Transit Network
bull BGP everywherebull Incoming and outcoming trafficbull Challenge balancing load on intradomain and egress
links given an offered traffic load
TransitISP
ISP 1
ISP 2
ISP 3
56
Interdomain Traffic Engineering
bull The process by which a network operator configures the network to achievendash Traffic load balancendash Redundancy (primarybackup) etc
bull Two tasksndash Outbound traffic controlndash Inbound traffic control
bull Key Problems Predictability and Scalability
57
Outbound Traffic Control
bull Easier to control than inbound trafficndash Destination-based routing sender determines where
the packets go
bull Control over next-hop AS onlyndash Cannot control selection of the entire path
Provider 1 Provider 2
Control with local preference
58
Outbound Traffic Load Balancingbull Control routes to provider per-prefix
ndash Assign local preference across destination prefixesndash Change the local preference assignments over time
bull Useful inputs to load balancingndash End-to-end path performance datandash Outbound traffic statistics per destination prefix
bull Challenge Getting from traffic volumes to groups of prefixes that should be assigned to each link
Premise of ldquointelligent route controlrdquo preoducts
59
Traffic Engineering Goals
bull Predictabilityndash Ensure the BGP decision process is deterministicndash Assume that BGP updates are (relatively) stable
bull Limit overhead introduced by routing changesndash Minimize frequency of changes to routing policiesndash Limit number of prefixes affected by changes
bull Limit impact on how traffic enters the networkndash Avoid new routes that might change neighborrsquos mindndash Select route with same attributes or at least path length
60
Managing Scalebull Destination prefixes
ndash More than 90000 destination prefixes
bull Donrsquot want to have per-prefix routing policies
ndash Small fraction of prefixes contribute most of the traffic
bull Focus on the small number of heavy hitters
ndash Define routing policies for selected prefixes
bull Routing choicesndash About 27000 unique ldquorouting choicesrdquo
bull Help in reducing the scale of the problem
ndash Small fraction of ldquorouting choicesrdquo contribute most traffic
bull Focus on the very small number of ldquorouting choicesrdquo
ndash Define routing policies on common attributes
61
Achieving Predictability
bull Route prediction with static analysisndash Helpful to know effects before deploymentndash Static analysis can help
TopologyBGP policy
configuration
eBGP routes
Offered traffic
BGP routingmodel
Flow of traffic through the network
62
Challenges to Predictabilitybull For transit ISPs effects on incoming traffic
ndash Lack of coordination strikes again
63
ldquoHot Potatordquo routing
Inter-AS Negotiation
bull Coordination aids predictabilityndash Negotiate where to sendndash Inbound and outboundndash Mutual benefits
bull How to implementndash What info to exchangendash Protecting privacyndash How to prioritize choicesndash How to prevent cheating
Destination 2
Destination 1
multiplepeeringpoints
Provider A
Provider B
64
Outbound Multihoming Goals
bull Redundancyndash Dynamic routing will failover to backup link
bull Performancendash Select provider with best performance per prefix
ndash Requires active probing
bull Costndash Select provider per prefix over time to minimize the
total financial cost
65
Inbound Traffic Control
bull More difficult no control over neighborsrsquo decisions
bull Three common techniques (previously discussed)ndash AS path prependingndash Communities and local preferencendash Prefix splitting
How does todayrsquos paper (MONET) control inbound traffic
66
How many links are enough
K upstream ISPs
Not much benefit beyond 4 ISPs
Akella et al ldquoPerformance Benefits of Multihomingrdquo SIGCOMM 2003
67
Problems with Multihoming in IPv4
bull Routing table growthndash Provider-based addressingndash Advertising prefix out multiple ISPs ndash canrsquot aggregate
bull Poor control over inbound trafficndash Existing mechanisms do not allow hosts to control
inbound traffic
68
GeorgiaTech
Internet Routing Overview
bull Intradomain (ie ldquointra-ASrdquo) routingbull Interdomain routing
Comcast
Abilene
ATampT Cogent
Autonomous Systems (ASes)
69
Configuration Problems ldquoAS 7007rdquoldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
UUNet
Florida InternetBarn
Sprint
70
Diagnosis and Troubleshootingldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
ldquoMicrosofts websites were offline for up to 23 hoursbecause of a [router] misconfigurationhellipit took nearly a day to determine what was wrong and undo the changesrdquo -- wiredcom January 25 2001
ldquoWorldCom Inchellipsuffered a widespread outage on its Internet backbone that affected roughly 20 percent of its US customer base The network problemshellipaffected millions of computer users worldwide A spokeswoman attributed the outage to a route table issue -- cnncom October 3 2002
A number of Covad customers went out from 5pm today due to supposedly a DDOS (distributed denial of service attack) on a key Level3 data center which later was described as a route leak (misconfiguration)ldquo
-- dslreportscom February 23 2004
71
Operator Mailing List (NANOG)Date Mon 18 Oct 2004 091515 -0700Subject Level 3 US east coast issues
Level 3 experiencing widespread unspecified routing issues on the US east coast Master ticket 1086844 Anyone have more specific information
Date Mon 18 Oct 2004 122034 -0400 (EDT)Subject Re Level 3 US east coast issues
Level 3 is currently experiencing a backbone outage causing routing instability and packet loss We are working to restore and will be sending hourly updateshellip
72
Operator Mailing List
0102030405060708090
Filtering RouteLeaks
RouteHijacks
RouteInstability
RoutingLoops
Blackholes
Occurr
ences o
ver
10 Y
ears
1995-1997 1998-2001 2001-2004
Note Only includes problems openly discussed on this list
Compare 83 power outages 1 fire
73
Routing Configuration
Ranking route selection
Dissemination internal route advertisement
Filtering route advertisement
Customer
Competitor
Primary
Backup
74
Internet Business Model (Simplified)
bull CustomerProvider One AS pays another for reachability to some set of destinations
bull ldquoSettlement-freerdquo Peering Bartering Two ASes exchange routes with one another
Provider
Peer
Customer
Preferences implemented with local preference manipulation
Destination
Pay to use
Get paid to use
Free to use
75
Peering Contracts Consistent Export
bull Rules of settlement-free peeringndash Advertise routes at all peering pointsndash Advertised routes must have equal ldquoAS path lengthrdquo
Sprint
ATampT
Enables ldquohot potatordquo routing
ldquoequally goodrdquoroutes
76
Consistent Export
bull Malicedeceptionbull iBGP signaling partitionbull Inconsistent export policy
neighbor 10123route-map PEER permit 10 set prepend 123
neighbor 10456route-map PEER permit 10 set prepend 123 123
Possible Causes
10123 456 1
10456 456 2
Neighbor AS Export
1 1 123
Export Clause Prepend
2 1 123 123
Two different Export Policies
77
Inconsistent Export in Practice
Feamster et al ldquoBorderGuard Detecting Cold Potatoes from Peersrdquo ACM IMC October 2004
78
Blackholes
Date Thu 18 Jul 2002 060510 -0400 (EDT)From Chad Oleary ltcolpoboxcomgtSubject Re problems with 701To ltnanogmeritedugt
Were starting to see the same issues with UUNet again Anyone elseseeing this Trying to reach Qwest
traceroute to 631461901 (631461901) 30 hops max 38 byte packets 1 esc-lp2-gwe-solutionscorpcom (631182201) 1167 ms 1163 ms 1142 ms 2 500Serial2-10GW1TPA2ALTERNET (1571301499) 1097 ms 1059 ms 1044 ms 3 161at-1-0-0XL4ATL1ALTERNET (1526381190) 13839 ms 14108 ms 16638 ms 4 0so-3-1-0XL2ATL5ALTERNET (152630238) 14370 ms 14587 ms 14553 ms 5 POS7-0BR2ATL5ALTERNET (1526382193) 13928 ms 14099 ms 14053 ms 6 7 hellip
79
Security
80
Security ldquoBogonrdquo Routes
Feamster et al ldquoAn Empirical Study of lsquoBogonrsquo Route Advertisementsrdquo ACM CCR January 2005
81
Spam Phishing etc
bull Unsolicited commercial emailbull As of about August 2008 estimates indicate that
about 95 of all email is spambull Common spam filtering techniques
ndash Content-based filtersndash DNS Blacklist (DNSBL) lookups Significant fraction of
todayrsquos DNS traffic
Can IP addresses from which spam is received be spoofed
82
Spam and Routing
83
Worms and Botnets
84
What is a Worm
bull Code that replicates and propagates across the networkndash Often carries a ldquopayloadrdquo
bull Usually spread via exploiting flaws in open servicesndash ldquoVirusesrdquo require user action to spread
bull First worm Robert Morris November 1988ndash 6-10 of all Internet hosts infected ()
bull Many more since but none on that scale until July 2001
85
Example Worm Code Red
bull Initial version July 13 2001
bull Exploited known ISAPI vulnerability in Microsoft IIS Web servers
bull 1st through 20th of each month spread20th through end of each month attack
bull Payload Web site defacementbull Scanning Random IP addressesbull Bug failure to seed random number generator
86
Code Red Revisions
bull Released July 19 2001
bull Payload flooding attack on wwwwhitehousegovndash Attack was mounted at the IP address of the Web site
bull Bug died after 20th of each month
bull Random number generator for IP scanning fixed
87
Code Red Host Infection Rate
Exponential infection rate
Measured using backscatter technique
88
Designing Fast-Spreading Worms
bull Hit-list scanningndash Time to infect first 10k hosts dominates infection timendash Solution Reconnaissance (stealthy scans etc)
bull Permutation scanningndash Observation Most scanning is redundantndash Idea Shared permutation of address space Start scanning
from own IP address Re-randomize when another infected machine is found
bull Internet-scale hit listsndash Flash worm complete infection within 30 seconds
89
Botnets
bull Bots Autonomous programs performing tasksbull Plenty of ldquobenignrdquo bots
ndash eg weatherbug
bull Botnets group of bots ndash Typically carries malicious connotationndash Large numbers of infected machinesndash Machines ldquoenlistedrdquo with infection vectors like worms
(last lecture)
bull Available for simultaneous control by a masterbull Size up to 350000 nodes (from todayrsquos paper)
90
ldquoRallyingrdquo the Botnet
bull Easy to combine worm backdoor functionalitybull Problem how to learn about successfully
infected machines
bull Optionsndash Emailndash Hard-coded email address
91
Botnet Control
bull Botnet master typically runs some IRC server on a well-known port (eg 6667)
bull Infected machine contacts botnet with pre-programmed DNS name (eg big-botde)
bull Dynamic DNS allows controller to move about freely
Infected Machine
DynamicDNS
BotnetController
(IRC server)
92
Some Defenses
93
Idea 1 Ingress Filtering
bull RFC 2827 Routers install filters to drop packets from networks that are not downstream
bull Feasible at edgesbull Difficult to configure closer to network ldquocorerdquo
20469207024 Internet
Drop all packets with source address other than 20469207024
94
Idea 2 uRPF Checks
bull Unicast Reverse Path Forwardingndash Cisco ldquoip verify unicast reverse-pathrdquo
bull Requires symmetric routing
Accept packet from interface only if forwarding table entry for source IP address matches ingress interface
100183
A10018
10015
101203
100181 24
10011 24
Strict Mode uRPF
Enabled
ldquoArdquo Routing TableDestination Next Hop1001024 Int 110018024 Int 2
100183 from wrong interface
95
Problems with uRPF
bull Asymmetric routing
96
S-BGP
bull Address-based PKI validate signaturesndash Authentication of
bull ownership for IP address blocks bull AS number bull an ASs identity and bull a BGP routers identity
ndash Use existing infrastructure (Internet registries etc)ndash Routing origination is digitally signedndash BGP updates are digitally signed
1048708 bull Route attestations A new optional BGP transitive path attribute
ndash carries digital signatures covering the routing information in updates
97
Practical Problems with S-BGP
bull Requires Public-Key Infrastructure
bull Lots of digital signatures to calculate and verifyndash Message overheadndash CPU overhead
bull Calculation expense is greatest when topology is changingndash Caching can help
bull Route aggregation is problematic (maybe thatrsquos OK)
bull Secure route withdrawals when link or node fails
bull Address ownership data out of date
bull Deployment
- Networking
- Goal of This Tutorial
- Todayrsquos Networks
- Business Models
- Billing for Internet Usage
- Net Neutrality
- Network Operations
- Point-of-Presence (PoP)
- Example Abilene Network Topology
- Another Example Backbone
- Internet Routing Overview
- Internet Routing Protocol BGP
- Two Flavors of BGP
- IPv4 Addresses Networks of Networks
- Pre-1994 Classful Addressing
- Classless Interdomain Routing (CIDR)
- Benefits of CIDR
- Growth of IP Prefixes
- 1994-1998 Linear Growth
- Around 2000 Fast Growth Resumes
- Fast growth resumes
- The Address Allocation Process
- Common Problems
- What can go wrong
- Measurement and Monitoring
- Passive vs Active Measurement
- Slide 27
- Passive Traffic Data Measurement
- Simple Network Management Protocol
- SNMP (Passive)
- Packet-level Monitoring
- Full Packet Capture (Passive)
- What is a flow
- Cisco NetFlow
- Flow Record Contents
- Aggregating Packets into Flows
- Reducing Measurement Overhead
- Packet Sampling for Flow Monitoring
- Sampling Flow-Level Sampling
- Two Main Approaches
- Packet Capture on High-Speed Links
- Characteristics of Packet Capture
- Data Measurement Repositories
- Multihoming and Traffic Engineering
- What is Multihoming
- Why Multihome
- Redundancy
- Performance
- Multihoming in IP Networks Today
- BGP or no
- Same Provider or Multiple
- Multihomed Stub One Link
- Multihomed Stub Multiple Links
- Multihomed Stub Multiple ISPs
- Multihomed Transit Network
- Interdomain Traffic Engineering
- Outbound Traffic Control
- Outbound Traffic Load Balancing
- Traffic Engineering Goals
- Managing Scale
- Achieving Predictability
- Challenges to Predictability
- Inter-AS Negotiation
- Outbound Multihoming Goals
- Inbound Traffic Control
- How many links are enough
- Problems with Multihoming in IPv4
- Slide 68
- Slide 69
- Diagnosis and Troubleshooting
- Operator Mailing List (NANOG)
- Operator Mailing List
- Routing Configuration
- Internet Business Model (Simplified)
- Peering Contracts Consistent Export
- Consistent Export
- Inconsistent Export in Practice
- Blackholes
- Security
- Security ldquoBogonrdquo Routes
- Spam Phishing etc
- Spam and Routing
- Worms and Botnets
- What is a Worm
- Example Worm Code Red
- Code Red Revisions
- Code Red Host Infection Rate
- Designing Fast-Spreading Worms
- Botnets
- ldquoRallyingrdquo the Botnet
- Botnet Control
- Some Defenses
- Idea 1 Ingress Filtering
- Idea 2 uRPF Checks
- Problems with uRPF
- S-BGP
- Practical Problems with S-BGP
-
39
Sampling Flow-Level Sampling
bull Sampling of flow records evicted from flow cachendash When evicting flows from table or when analyzing flows
bull Stratified sampling to put weight on ldquoheavyrdquo flowsndash Select all long flows and sample the short flows
bull Reduces the number of flow records ndash Still measures the vast majority of the traffic
Flow 1 40 bytesFlow 2 15580 bytesFlow 3 8196 bytesFlow 4 5350789 bytesFlow 5 532 bytesFlow 6 7432 bytes
sample with 100 probability
sample with 01 probability
sample with 10 probability
40
Two Main Approaches
bull Packet-level Monitoringndash Keep packet-level statisticsndash Examine (and potentially log) variety of packet-level
statistics Essentially anything in the packetndash Timing
bull Flow-level Monitoringndash Monitor packet-by-packet (though sometimes
sampled)ndash Keep aggregate statistics on a flow
41
Packet Capture on High-Speed Links
Example Georgia Tech ldquoOC3Monrdquo
bull Rack-mounted PCbull Optical splitterbull Data Acquisition and
Generation (DAG) card
Source endacecom
42
Characteristics of Packet Capture
bull Allows inspection on every packet on 10G links
bull Disadvantagesndash Costlyndash Requires splitting optical fibersndash Must be able to filterstore data
43
Data Measurement Repositories
bull AbileneInternet 2 Observatoryndash Configuration examplesndash SNMP datandash ISIS BGP routing data NetFlow traffic data
bull RouteViewsndash BGP updatesndash BGP table snapshots
44
Multihoming and Traffic Engineering
45
What is Multihoming
bull The use of redundant network links for the purposes of external connectivity
bull Can be achieved at many layers of the protocol stack and many places in the networkndash Multiple network interfaces in a PC
ndash An ISP with multiple upstream interfaces
bull Can refer to having multiple connections tondash The same ISP
ndash Multiple ISPs
46
Why Multihome
bull Redundancybull Availabilitybull Performancebull Cost
Interdomain traffic engineering the process by which a multihomed network configures its
network to achieve these goals
47
Redundancy
bull Maintain connectivity in the face ofndash Physical connectivity problems (fiber cut device
failures etc)ndash Failures in upstream ISP
48
Performance
bull Use multiple network links at once to achieve higher throughput than just over a single link
bull Allows incoming traffic to be load-balanced
70 of traffic30 of traffic
49
Multihoming in IP Networks Today
bull Stub AS no transit service for other ASesndash No need to use BGP
bull Multi-homed stub AS has connectivity to multiple immediate upstream ISPsndash Need BGPndash No need for a public AS numberndash No need for IP prefix allocation
bull Multi-homed transit AS connectivity to multiple ASes and transit servicendash Need BGP public AS number IP prefix allocation
50
BGP or no
bull Advantages of static routingndash Cheapersmaller routers (less true nowadays)ndash Simpler to configure
bull Advantages of BGPndash More control of your destiny (have providers stop
announcing you)ndash Fastermore intelligent selection of where to send
outbound packetsndash Better debugging of net problems (you can see the
Internet topology now)
51
Same Provider or Multiple
bull If your provider is reliable and fast and affordably and offers good tech-support you may want to multi-home initially to them via some backup path (slow is better than dead)
bull Eventually yoursquoll want to multi-home to different providers to avoid failure modes due to one providerrsquos architecture decisions
52
Multihomed Stub One Link
bull Downstream ISPrsquos routers configure default (ldquostaticrdquo) routes pointing to border router
bull Upstream ISP advertises reachability
Upstream ISP
Multiple links between same pair of routers
Default routes to ldquoborderrdquo
ldquoStubrdquoISP
53
Multihomed Stub Multiple Links
bull Use BGP to share loadbull Use private AS number (why is this OK)bull As before upstream ISP advertises prefix
Upstream ISP
Multiple links to different upstream routers
ldquoStubrdquoISP
Internal routing for ldquohot potatordquo
BGP for load balance at edge
54
Multihomed Stub Multiple ISPs
bull Many possibilitiesndash Load sharingndash Primary-backupndash Selective use of different ISPs
bull Requires BGP public AS number etc
ldquoStubrdquoISP
Upstream
ISP 1
Upstream
ISP 2
55
Multihomed Transit Network
bull BGP everywherebull Incoming and outcoming trafficbull Challenge balancing load on intradomain and egress
links given an offered traffic load
TransitISP
ISP 1
ISP 2
ISP 3
56
Interdomain Traffic Engineering
bull The process by which a network operator configures the network to achievendash Traffic load balancendash Redundancy (primarybackup) etc
bull Two tasksndash Outbound traffic controlndash Inbound traffic control
bull Key Problems Predictability and Scalability
57
Outbound Traffic Control
bull Easier to control than inbound trafficndash Destination-based routing sender determines where
the packets go
bull Control over next-hop AS onlyndash Cannot control selection of the entire path
Provider 1 Provider 2
Control with local preference
58
Outbound Traffic Load Balancingbull Control routes to provider per-prefix
ndash Assign local preference across destination prefixesndash Change the local preference assignments over time
bull Useful inputs to load balancingndash End-to-end path performance datandash Outbound traffic statistics per destination prefix
bull Challenge Getting from traffic volumes to groups of prefixes that should be assigned to each link
Premise of ldquointelligent route controlrdquo preoducts
59
Traffic Engineering Goals
bull Predictabilityndash Ensure the BGP decision process is deterministicndash Assume that BGP updates are (relatively) stable
bull Limit overhead introduced by routing changesndash Minimize frequency of changes to routing policiesndash Limit number of prefixes affected by changes
bull Limit impact on how traffic enters the networkndash Avoid new routes that might change neighborrsquos mindndash Select route with same attributes or at least path length
60
Managing Scalebull Destination prefixes
ndash More than 90000 destination prefixes
bull Donrsquot want to have per-prefix routing policies
ndash Small fraction of prefixes contribute most of the traffic
bull Focus on the small number of heavy hitters
ndash Define routing policies for selected prefixes
bull Routing choicesndash About 27000 unique ldquorouting choicesrdquo
bull Help in reducing the scale of the problem
ndash Small fraction of ldquorouting choicesrdquo contribute most traffic
bull Focus on the very small number of ldquorouting choicesrdquo
ndash Define routing policies on common attributes
61
Achieving Predictability
bull Route prediction with static analysisndash Helpful to know effects before deploymentndash Static analysis can help
TopologyBGP policy
configuration
eBGP routes
Offered traffic
BGP routingmodel
Flow of traffic through the network
62
Challenges to Predictabilitybull For transit ISPs effects on incoming traffic
ndash Lack of coordination strikes again
63
ldquoHot Potatordquo routing
Inter-AS Negotiation
bull Coordination aids predictabilityndash Negotiate where to sendndash Inbound and outboundndash Mutual benefits
bull How to implementndash What info to exchangendash Protecting privacyndash How to prioritize choicesndash How to prevent cheating
Destination 2
Destination 1
multiplepeeringpoints
Provider A
Provider B
64
Outbound Multihoming Goals
bull Redundancyndash Dynamic routing will failover to backup link
bull Performancendash Select provider with best performance per prefix
ndash Requires active probing
bull Costndash Select provider per prefix over time to minimize the
total financial cost
65
Inbound Traffic Control
bull More difficult no control over neighborsrsquo decisions
bull Three common techniques (previously discussed)ndash AS path prependingndash Communities and local preferencendash Prefix splitting
How does todayrsquos paper (MONET) control inbound traffic
66
How many links are enough
K upstream ISPs
Not much benefit beyond 4 ISPs
Akella et al ldquoPerformance Benefits of Multihomingrdquo SIGCOMM 2003
67
Problems with Multihoming in IPv4
bull Routing table growthndash Provider-based addressingndash Advertising prefix out multiple ISPs ndash canrsquot aggregate
bull Poor control over inbound trafficndash Existing mechanisms do not allow hosts to control
inbound traffic
68
GeorgiaTech
Internet Routing Overview
bull Intradomain (ie ldquointra-ASrdquo) routingbull Interdomain routing
Comcast
Abilene
ATampT Cogent
Autonomous Systems (ASes)
69
Configuration Problems ldquoAS 7007rdquoldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
UUNet
Florida InternetBarn
Sprint
70
Diagnosis and Troubleshootingldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
ldquoMicrosofts websites were offline for up to 23 hoursbecause of a [router] misconfigurationhellipit took nearly a day to determine what was wrong and undo the changesrdquo -- wiredcom January 25 2001
ldquoWorldCom Inchellipsuffered a widespread outage on its Internet backbone that affected roughly 20 percent of its US customer base The network problemshellipaffected millions of computer users worldwide A spokeswoman attributed the outage to a route table issue -- cnncom October 3 2002
A number of Covad customers went out from 5pm today due to supposedly a DDOS (distributed denial of service attack) on a key Level3 data center which later was described as a route leak (misconfiguration)ldquo
-- dslreportscom February 23 2004
71
Operator Mailing List (NANOG)Date Mon 18 Oct 2004 091515 -0700Subject Level 3 US east coast issues
Level 3 experiencing widespread unspecified routing issues on the US east coast Master ticket 1086844 Anyone have more specific information
Date Mon 18 Oct 2004 122034 -0400 (EDT)Subject Re Level 3 US east coast issues
Level 3 is currently experiencing a backbone outage causing routing instability and packet loss We are working to restore and will be sending hourly updateshellip
72
Operator Mailing List
0102030405060708090
Filtering RouteLeaks
RouteHijacks
RouteInstability
RoutingLoops
Blackholes
Occurr
ences o
ver
10 Y
ears
1995-1997 1998-2001 2001-2004
Note Only includes problems openly discussed on this list
Compare 83 power outages 1 fire
73
Routing Configuration
Ranking route selection
Dissemination internal route advertisement
Filtering route advertisement
Customer
Competitor
Primary
Backup
74
Internet Business Model (Simplified)
bull CustomerProvider One AS pays another for reachability to some set of destinations
bull ldquoSettlement-freerdquo Peering Bartering Two ASes exchange routes with one another
Provider
Peer
Customer
Preferences implemented with local preference manipulation
Destination
Pay to use
Get paid to use
Free to use
75
Peering Contracts Consistent Export
bull Rules of settlement-free peeringndash Advertise routes at all peering pointsndash Advertised routes must have equal ldquoAS path lengthrdquo
Sprint
ATampT
Enables ldquohot potatordquo routing
ldquoequally goodrdquoroutes
76
Consistent Export
bull Malicedeceptionbull iBGP signaling partitionbull Inconsistent export policy
neighbor 10123route-map PEER permit 10 set prepend 123
neighbor 10456route-map PEER permit 10 set prepend 123 123
Possible Causes
10123 456 1
10456 456 2
Neighbor AS Export
1 1 123
Export Clause Prepend
2 1 123 123
Two different Export Policies
77
Inconsistent Export in Practice
Feamster et al ldquoBorderGuard Detecting Cold Potatoes from Peersrdquo ACM IMC October 2004
78
Blackholes
Date Thu 18 Jul 2002 060510 -0400 (EDT)From Chad Oleary ltcolpoboxcomgtSubject Re problems with 701To ltnanogmeritedugt
Were starting to see the same issues with UUNet again Anyone elseseeing this Trying to reach Qwest
traceroute to 631461901 (631461901) 30 hops max 38 byte packets 1 esc-lp2-gwe-solutionscorpcom (631182201) 1167 ms 1163 ms 1142 ms 2 500Serial2-10GW1TPA2ALTERNET (1571301499) 1097 ms 1059 ms 1044 ms 3 161at-1-0-0XL4ATL1ALTERNET (1526381190) 13839 ms 14108 ms 16638 ms 4 0so-3-1-0XL2ATL5ALTERNET (152630238) 14370 ms 14587 ms 14553 ms 5 POS7-0BR2ATL5ALTERNET (1526382193) 13928 ms 14099 ms 14053 ms 6 7 hellip
79
Security
80
Security ldquoBogonrdquo Routes
Feamster et al ldquoAn Empirical Study of lsquoBogonrsquo Route Advertisementsrdquo ACM CCR January 2005
81
Spam Phishing etc
bull Unsolicited commercial emailbull As of about August 2008 estimates indicate that
about 95 of all email is spambull Common spam filtering techniques
ndash Content-based filtersndash DNS Blacklist (DNSBL) lookups Significant fraction of
todayrsquos DNS traffic
Can IP addresses from which spam is received be spoofed
82
Spam and Routing
83
Worms and Botnets
84
What is a Worm
bull Code that replicates and propagates across the networkndash Often carries a ldquopayloadrdquo
bull Usually spread via exploiting flaws in open servicesndash ldquoVirusesrdquo require user action to spread
bull First worm Robert Morris November 1988ndash 6-10 of all Internet hosts infected ()
bull Many more since but none on that scale until July 2001
85
Example Worm Code Red
bull Initial version July 13 2001
bull Exploited known ISAPI vulnerability in Microsoft IIS Web servers
bull 1st through 20th of each month spread20th through end of each month attack
bull Payload Web site defacementbull Scanning Random IP addressesbull Bug failure to seed random number generator
86
Code Red Revisions
bull Released July 19 2001
bull Payload flooding attack on wwwwhitehousegovndash Attack was mounted at the IP address of the Web site
bull Bug died after 20th of each month
bull Random number generator for IP scanning fixed
87
Code Red Host Infection Rate
Exponential infection rate
Measured using backscatter technique
88
Designing Fast-Spreading Worms
bull Hit-list scanningndash Time to infect first 10k hosts dominates infection timendash Solution Reconnaissance (stealthy scans etc)
bull Permutation scanningndash Observation Most scanning is redundantndash Idea Shared permutation of address space Start scanning
from own IP address Re-randomize when another infected machine is found
bull Internet-scale hit listsndash Flash worm complete infection within 30 seconds
89
Botnets
bull Bots Autonomous programs performing tasksbull Plenty of ldquobenignrdquo bots
ndash eg weatherbug
bull Botnets group of bots ndash Typically carries malicious connotationndash Large numbers of infected machinesndash Machines ldquoenlistedrdquo with infection vectors like worms
(last lecture)
bull Available for simultaneous control by a masterbull Size up to 350000 nodes (from todayrsquos paper)
90
ldquoRallyingrdquo the Botnet
bull Easy to combine worm backdoor functionalitybull Problem how to learn about successfully
infected machines
bull Optionsndash Emailndash Hard-coded email address
91
Botnet Control
bull Botnet master typically runs some IRC server on a well-known port (eg 6667)
bull Infected machine contacts botnet with pre-programmed DNS name (eg big-botde)
bull Dynamic DNS allows controller to move about freely
Infected Machine
DynamicDNS
BotnetController
(IRC server)
92
Some Defenses
93
Idea 1 Ingress Filtering
bull RFC 2827 Routers install filters to drop packets from networks that are not downstream
bull Feasible at edgesbull Difficult to configure closer to network ldquocorerdquo
20469207024 Internet
Drop all packets with source address other than 20469207024
94
Idea 2 uRPF Checks
bull Unicast Reverse Path Forwardingndash Cisco ldquoip verify unicast reverse-pathrdquo
bull Requires symmetric routing
Accept packet from interface only if forwarding table entry for source IP address matches ingress interface
100183
A10018
10015
101203
100181 24
10011 24
Strict Mode uRPF
Enabled
ldquoArdquo Routing TableDestination Next Hop1001024 Int 110018024 Int 2
100183 from wrong interface
95
Problems with uRPF
bull Asymmetric routing
96
S-BGP
bull Address-based PKI validate signaturesndash Authentication of
bull ownership for IP address blocks bull AS number bull an ASs identity and bull a BGP routers identity
ndash Use existing infrastructure (Internet registries etc)ndash Routing origination is digitally signedndash BGP updates are digitally signed
1048708 bull Route attestations A new optional BGP transitive path attribute
ndash carries digital signatures covering the routing information in updates
97
Practical Problems with S-BGP
bull Requires Public-Key Infrastructure
bull Lots of digital signatures to calculate and verifyndash Message overheadndash CPU overhead
bull Calculation expense is greatest when topology is changingndash Caching can help
bull Route aggregation is problematic (maybe thatrsquos OK)
bull Secure route withdrawals when link or node fails
bull Address ownership data out of date
bull Deployment
- Networking
- Goal of This Tutorial
- Todayrsquos Networks
- Business Models
- Billing for Internet Usage
- Net Neutrality
- Network Operations
- Point-of-Presence (PoP)
- Example Abilene Network Topology
- Another Example Backbone
- Internet Routing Overview
- Internet Routing Protocol BGP
- Two Flavors of BGP
- IPv4 Addresses Networks of Networks
- Pre-1994 Classful Addressing
- Classless Interdomain Routing (CIDR)
- Benefits of CIDR
- Growth of IP Prefixes
- 1994-1998 Linear Growth
- Around 2000 Fast Growth Resumes
- Fast growth resumes
- The Address Allocation Process
- Common Problems
- What can go wrong
- Measurement and Monitoring
- Passive vs Active Measurement
- Slide 27
- Passive Traffic Data Measurement
- Simple Network Management Protocol
- SNMP (Passive)
- Packet-level Monitoring
- Full Packet Capture (Passive)
- What is a flow
- Cisco NetFlow
- Flow Record Contents
- Aggregating Packets into Flows
- Reducing Measurement Overhead
- Packet Sampling for Flow Monitoring
- Sampling Flow-Level Sampling
- Two Main Approaches
- Packet Capture on High-Speed Links
- Characteristics of Packet Capture
- Data Measurement Repositories
- Multihoming and Traffic Engineering
- What is Multihoming
- Why Multihome
- Redundancy
- Performance
- Multihoming in IP Networks Today
- BGP or no
- Same Provider or Multiple
- Multihomed Stub One Link
- Multihomed Stub Multiple Links
- Multihomed Stub Multiple ISPs
- Multihomed Transit Network
- Interdomain Traffic Engineering
- Outbound Traffic Control
- Outbound Traffic Load Balancing
- Traffic Engineering Goals
- Managing Scale
- Achieving Predictability
- Challenges to Predictability
- Inter-AS Negotiation
- Outbound Multihoming Goals
- Inbound Traffic Control
- How many links are enough
- Problems with Multihoming in IPv4
- Slide 68
- Slide 69
- Diagnosis and Troubleshooting
- Operator Mailing List (NANOG)
- Operator Mailing List
- Routing Configuration
- Internet Business Model (Simplified)
- Peering Contracts Consistent Export
- Consistent Export
- Inconsistent Export in Practice
- Blackholes
- Security
- Security ldquoBogonrdquo Routes
- Spam Phishing etc
- Spam and Routing
- Worms and Botnets
- What is a Worm
- Example Worm Code Red
- Code Red Revisions
- Code Red Host Infection Rate
- Designing Fast-Spreading Worms
- Botnets
- ldquoRallyingrdquo the Botnet
- Botnet Control
- Some Defenses
- Idea 1 Ingress Filtering
- Idea 2 uRPF Checks
- Problems with uRPF
- S-BGP
- Practical Problems with S-BGP
-
40
Two Main Approaches
bull Packet-level Monitoringndash Keep packet-level statisticsndash Examine (and potentially log) variety of packet-level
statistics Essentially anything in the packetndash Timing
bull Flow-level Monitoringndash Monitor packet-by-packet (though sometimes
sampled)ndash Keep aggregate statistics on a flow
41
Packet Capture on High-Speed Links
Example Georgia Tech ldquoOC3Monrdquo
bull Rack-mounted PCbull Optical splitterbull Data Acquisition and
Generation (DAG) card
Source endacecom
42
Characteristics of Packet Capture
bull Allows inspection on every packet on 10G links
bull Disadvantagesndash Costlyndash Requires splitting optical fibersndash Must be able to filterstore data
43
Data Measurement Repositories
bull AbileneInternet 2 Observatoryndash Configuration examplesndash SNMP datandash ISIS BGP routing data NetFlow traffic data
bull RouteViewsndash BGP updatesndash BGP table snapshots
44
Multihoming and Traffic Engineering
45
What is Multihoming
bull The use of redundant network links for the purposes of external connectivity
bull Can be achieved at many layers of the protocol stack and many places in the networkndash Multiple network interfaces in a PC
ndash An ISP with multiple upstream interfaces
bull Can refer to having multiple connections tondash The same ISP
ndash Multiple ISPs
46
Why Multihome
bull Redundancybull Availabilitybull Performancebull Cost
Interdomain traffic engineering the process by which a multihomed network configures its
network to achieve these goals
47
Redundancy
bull Maintain connectivity in the face ofndash Physical connectivity problems (fiber cut device
failures etc)ndash Failures in upstream ISP
48
Performance
bull Use multiple network links at once to achieve higher throughput than just over a single link
bull Allows incoming traffic to be load-balanced
70 of traffic30 of traffic
49
Multihoming in IP Networks Today
bull Stub AS no transit service for other ASesndash No need to use BGP
bull Multi-homed stub AS has connectivity to multiple immediate upstream ISPsndash Need BGPndash No need for a public AS numberndash No need for IP prefix allocation
bull Multi-homed transit AS connectivity to multiple ASes and transit servicendash Need BGP public AS number IP prefix allocation
50
BGP or no
bull Advantages of static routingndash Cheapersmaller routers (less true nowadays)ndash Simpler to configure
bull Advantages of BGPndash More control of your destiny (have providers stop
announcing you)ndash Fastermore intelligent selection of where to send
outbound packetsndash Better debugging of net problems (you can see the
Internet topology now)
51
Same Provider or Multiple
bull If your provider is reliable and fast and affordably and offers good tech-support you may want to multi-home initially to them via some backup path (slow is better than dead)
bull Eventually yoursquoll want to multi-home to different providers to avoid failure modes due to one providerrsquos architecture decisions
52
Multihomed Stub One Link
bull Downstream ISPrsquos routers configure default (ldquostaticrdquo) routes pointing to border router
bull Upstream ISP advertises reachability
Upstream ISP
Multiple links between same pair of routers
Default routes to ldquoborderrdquo
ldquoStubrdquoISP
53
Multihomed Stub Multiple Links
bull Use BGP to share loadbull Use private AS number (why is this OK)bull As before upstream ISP advertises prefix
Upstream ISP
Multiple links to different upstream routers
ldquoStubrdquoISP
Internal routing for ldquohot potatordquo
BGP for load balance at edge
54
Multihomed Stub Multiple ISPs
bull Many possibilitiesndash Load sharingndash Primary-backupndash Selective use of different ISPs
bull Requires BGP public AS number etc
ldquoStubrdquoISP
Upstream
ISP 1
Upstream
ISP 2
55
Multihomed Transit Network
bull BGP everywherebull Incoming and outcoming trafficbull Challenge balancing load on intradomain and egress
links given an offered traffic load
TransitISP
ISP 1
ISP 2
ISP 3
56
Interdomain Traffic Engineering
bull The process by which a network operator configures the network to achievendash Traffic load balancendash Redundancy (primarybackup) etc
bull Two tasksndash Outbound traffic controlndash Inbound traffic control
bull Key Problems Predictability and Scalability
57
Outbound Traffic Control
bull Easier to control than inbound trafficndash Destination-based routing sender determines where
the packets go
bull Control over next-hop AS onlyndash Cannot control selection of the entire path
Provider 1 Provider 2
Control with local preference
58
Outbound Traffic Load Balancingbull Control routes to provider per-prefix
ndash Assign local preference across destination prefixesndash Change the local preference assignments over time
bull Useful inputs to load balancingndash End-to-end path performance datandash Outbound traffic statistics per destination prefix
bull Challenge Getting from traffic volumes to groups of prefixes that should be assigned to each link
Premise of ldquointelligent route controlrdquo preoducts
59
Traffic Engineering Goals
bull Predictabilityndash Ensure the BGP decision process is deterministicndash Assume that BGP updates are (relatively) stable
bull Limit overhead introduced by routing changesndash Minimize frequency of changes to routing policiesndash Limit number of prefixes affected by changes
bull Limit impact on how traffic enters the networkndash Avoid new routes that might change neighborrsquos mindndash Select route with same attributes or at least path length
60
Managing Scalebull Destination prefixes
ndash More than 90000 destination prefixes
bull Donrsquot want to have per-prefix routing policies
ndash Small fraction of prefixes contribute most of the traffic
bull Focus on the small number of heavy hitters
ndash Define routing policies for selected prefixes
bull Routing choicesndash About 27000 unique ldquorouting choicesrdquo
bull Help in reducing the scale of the problem
ndash Small fraction of ldquorouting choicesrdquo contribute most traffic
bull Focus on the very small number of ldquorouting choicesrdquo
ndash Define routing policies on common attributes
61
Achieving Predictability
bull Route prediction with static analysisndash Helpful to know effects before deploymentndash Static analysis can help
TopologyBGP policy
configuration
eBGP routes
Offered traffic
BGP routingmodel
Flow of traffic through the network
62
Challenges to Predictabilitybull For transit ISPs effects on incoming traffic
ndash Lack of coordination strikes again
63
ldquoHot Potatordquo routing
Inter-AS Negotiation
bull Coordination aids predictabilityndash Negotiate where to sendndash Inbound and outboundndash Mutual benefits
bull How to implementndash What info to exchangendash Protecting privacyndash How to prioritize choicesndash How to prevent cheating
Destination 2
Destination 1
multiplepeeringpoints
Provider A
Provider B
64
Outbound Multihoming Goals
bull Redundancyndash Dynamic routing will failover to backup link
bull Performancendash Select provider with best performance per prefix
ndash Requires active probing
bull Costndash Select provider per prefix over time to minimize the
total financial cost
65
Inbound Traffic Control
bull More difficult no control over neighborsrsquo decisions
bull Three common techniques (previously discussed)ndash AS path prependingndash Communities and local preferencendash Prefix splitting
How does todayrsquos paper (MONET) control inbound traffic
66
How many links are enough
K upstream ISPs
Not much benefit beyond 4 ISPs
Akella et al ldquoPerformance Benefits of Multihomingrdquo SIGCOMM 2003
67
Problems with Multihoming in IPv4
bull Routing table growthndash Provider-based addressingndash Advertising prefix out multiple ISPs ndash canrsquot aggregate
bull Poor control over inbound trafficndash Existing mechanisms do not allow hosts to control
inbound traffic
68
GeorgiaTech
Internet Routing Overview
bull Intradomain (ie ldquointra-ASrdquo) routingbull Interdomain routing
Comcast
Abilene
ATampT Cogent
Autonomous Systems (ASes)
69
Configuration Problems ldquoAS 7007rdquoldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
UUNet
Florida InternetBarn
Sprint
70
Diagnosis and Troubleshootingldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
ldquoMicrosofts websites were offline for up to 23 hoursbecause of a [router] misconfigurationhellipit took nearly a day to determine what was wrong and undo the changesrdquo -- wiredcom January 25 2001
ldquoWorldCom Inchellipsuffered a widespread outage on its Internet backbone that affected roughly 20 percent of its US customer base The network problemshellipaffected millions of computer users worldwide A spokeswoman attributed the outage to a route table issue -- cnncom October 3 2002
A number of Covad customers went out from 5pm today due to supposedly a DDOS (distributed denial of service attack) on a key Level3 data center which later was described as a route leak (misconfiguration)ldquo
-- dslreportscom February 23 2004
71
Operator Mailing List (NANOG)Date Mon 18 Oct 2004 091515 -0700Subject Level 3 US east coast issues
Level 3 experiencing widespread unspecified routing issues on the US east coast Master ticket 1086844 Anyone have more specific information
Date Mon 18 Oct 2004 122034 -0400 (EDT)Subject Re Level 3 US east coast issues
Level 3 is currently experiencing a backbone outage causing routing instability and packet loss We are working to restore and will be sending hourly updateshellip
72
Operator Mailing List
0102030405060708090
Filtering RouteLeaks
RouteHijacks
RouteInstability
RoutingLoops
Blackholes
Occurr
ences o
ver
10 Y
ears
1995-1997 1998-2001 2001-2004
Note Only includes problems openly discussed on this list
Compare 83 power outages 1 fire
73
Routing Configuration
Ranking route selection
Dissemination internal route advertisement
Filtering route advertisement
Customer
Competitor
Primary
Backup
74
Internet Business Model (Simplified)
bull CustomerProvider One AS pays another for reachability to some set of destinations
bull ldquoSettlement-freerdquo Peering Bartering Two ASes exchange routes with one another
Provider
Peer
Customer
Preferences implemented with local preference manipulation
Destination
Pay to use
Get paid to use
Free to use
75
Peering Contracts Consistent Export
bull Rules of settlement-free peeringndash Advertise routes at all peering pointsndash Advertised routes must have equal ldquoAS path lengthrdquo
Sprint
ATampT
Enables ldquohot potatordquo routing
ldquoequally goodrdquoroutes
76
Consistent Export
bull Malicedeceptionbull iBGP signaling partitionbull Inconsistent export policy
neighbor 10123route-map PEER permit 10 set prepend 123
neighbor 10456route-map PEER permit 10 set prepend 123 123
Possible Causes
10123 456 1
10456 456 2
Neighbor AS Export
1 1 123
Export Clause Prepend
2 1 123 123
Two different Export Policies
77
Inconsistent Export in Practice
Feamster et al ldquoBorderGuard Detecting Cold Potatoes from Peersrdquo ACM IMC October 2004
78
Blackholes
Date Thu 18 Jul 2002 060510 -0400 (EDT)From Chad Oleary ltcolpoboxcomgtSubject Re problems with 701To ltnanogmeritedugt
Were starting to see the same issues with UUNet again Anyone elseseeing this Trying to reach Qwest
traceroute to 631461901 (631461901) 30 hops max 38 byte packets 1 esc-lp2-gwe-solutionscorpcom (631182201) 1167 ms 1163 ms 1142 ms 2 500Serial2-10GW1TPA2ALTERNET (1571301499) 1097 ms 1059 ms 1044 ms 3 161at-1-0-0XL4ATL1ALTERNET (1526381190) 13839 ms 14108 ms 16638 ms 4 0so-3-1-0XL2ATL5ALTERNET (152630238) 14370 ms 14587 ms 14553 ms 5 POS7-0BR2ATL5ALTERNET (1526382193) 13928 ms 14099 ms 14053 ms 6 7 hellip
79
Security
80
Security ldquoBogonrdquo Routes
Feamster et al ldquoAn Empirical Study of lsquoBogonrsquo Route Advertisementsrdquo ACM CCR January 2005
81
Spam Phishing etc
bull Unsolicited commercial emailbull As of about August 2008 estimates indicate that
about 95 of all email is spambull Common spam filtering techniques
ndash Content-based filtersndash DNS Blacklist (DNSBL) lookups Significant fraction of
todayrsquos DNS traffic
Can IP addresses from which spam is received be spoofed
82
Spam and Routing
83
Worms and Botnets
84
What is a Worm
bull Code that replicates and propagates across the networkndash Often carries a ldquopayloadrdquo
bull Usually spread via exploiting flaws in open servicesndash ldquoVirusesrdquo require user action to spread
bull First worm Robert Morris November 1988ndash 6-10 of all Internet hosts infected ()
bull Many more since but none on that scale until July 2001
85
Example Worm Code Red
bull Initial version July 13 2001
bull Exploited known ISAPI vulnerability in Microsoft IIS Web servers
bull 1st through 20th of each month spread20th through end of each month attack
bull Payload Web site defacementbull Scanning Random IP addressesbull Bug failure to seed random number generator
86
Code Red Revisions
bull Released July 19 2001
bull Payload flooding attack on wwwwhitehousegovndash Attack was mounted at the IP address of the Web site
bull Bug died after 20th of each month
bull Random number generator for IP scanning fixed
87
Code Red Host Infection Rate
Exponential infection rate
Measured using backscatter technique
88
Designing Fast-Spreading Worms
bull Hit-list scanningndash Time to infect first 10k hosts dominates infection timendash Solution Reconnaissance (stealthy scans etc)
bull Permutation scanningndash Observation Most scanning is redundantndash Idea Shared permutation of address space Start scanning
from own IP address Re-randomize when another infected machine is found
bull Internet-scale hit listsndash Flash worm complete infection within 30 seconds
89
Botnets
bull Bots Autonomous programs performing tasksbull Plenty of ldquobenignrdquo bots
ndash eg weatherbug
bull Botnets group of bots ndash Typically carries malicious connotationndash Large numbers of infected machinesndash Machines ldquoenlistedrdquo with infection vectors like worms
(last lecture)
bull Available for simultaneous control by a masterbull Size up to 350000 nodes (from todayrsquos paper)
90
ldquoRallyingrdquo the Botnet
bull Easy to combine worm backdoor functionalitybull Problem how to learn about successfully
infected machines
bull Optionsndash Emailndash Hard-coded email address
91
Botnet Control
bull Botnet master typically runs some IRC server on a well-known port (eg 6667)
bull Infected machine contacts botnet with pre-programmed DNS name (eg big-botde)
bull Dynamic DNS allows controller to move about freely
Infected Machine
DynamicDNS
BotnetController
(IRC server)
92
Some Defenses
93
Idea 1 Ingress Filtering
bull RFC 2827 Routers install filters to drop packets from networks that are not downstream
bull Feasible at edgesbull Difficult to configure closer to network ldquocorerdquo
20469207024 Internet
Drop all packets with source address other than 20469207024
94
Idea 2 uRPF Checks
bull Unicast Reverse Path Forwardingndash Cisco ldquoip verify unicast reverse-pathrdquo
bull Requires symmetric routing
Accept packet from interface only if forwarding table entry for source IP address matches ingress interface
100183
A10018
10015
101203
100181 24
10011 24
Strict Mode uRPF
Enabled
ldquoArdquo Routing TableDestination Next Hop1001024 Int 110018024 Int 2
100183 from wrong interface
95
Problems with uRPF
bull Asymmetric routing
96
S-BGP
bull Address-based PKI validate signaturesndash Authentication of
bull ownership for IP address blocks bull AS number bull an ASs identity and bull a BGP routers identity
ndash Use existing infrastructure (Internet registries etc)ndash Routing origination is digitally signedndash BGP updates are digitally signed
1048708 bull Route attestations A new optional BGP transitive path attribute
ndash carries digital signatures covering the routing information in updates
97
Practical Problems with S-BGP
bull Requires Public-Key Infrastructure
bull Lots of digital signatures to calculate and verifyndash Message overheadndash CPU overhead
bull Calculation expense is greatest when topology is changingndash Caching can help
bull Route aggregation is problematic (maybe thatrsquos OK)
bull Secure route withdrawals when link or node fails
bull Address ownership data out of date
bull Deployment
- Networking
- Goal of This Tutorial
- Todayrsquos Networks
- Business Models
- Billing for Internet Usage
- Net Neutrality
- Network Operations
- Point-of-Presence (PoP)
- Example Abilene Network Topology
- Another Example Backbone
- Internet Routing Overview
- Internet Routing Protocol BGP
- Two Flavors of BGP
- IPv4 Addresses Networks of Networks
- Pre-1994 Classful Addressing
- Classless Interdomain Routing (CIDR)
- Benefits of CIDR
- Growth of IP Prefixes
- 1994-1998 Linear Growth
- Around 2000 Fast Growth Resumes
- Fast growth resumes
- The Address Allocation Process
- Common Problems
- What can go wrong
- Measurement and Monitoring
- Passive vs Active Measurement
- Slide 27
- Passive Traffic Data Measurement
- Simple Network Management Protocol
- SNMP (Passive)
- Packet-level Monitoring
- Full Packet Capture (Passive)
- What is a flow
- Cisco NetFlow
- Flow Record Contents
- Aggregating Packets into Flows
- Reducing Measurement Overhead
- Packet Sampling for Flow Monitoring
- Sampling Flow-Level Sampling
- Two Main Approaches
- Packet Capture on High-Speed Links
- Characteristics of Packet Capture
- Data Measurement Repositories
- Multihoming and Traffic Engineering
- What is Multihoming
- Why Multihome
- Redundancy
- Performance
- Multihoming in IP Networks Today
- BGP or no
- Same Provider or Multiple
- Multihomed Stub One Link
- Multihomed Stub Multiple Links
- Multihomed Stub Multiple ISPs
- Multihomed Transit Network
- Interdomain Traffic Engineering
- Outbound Traffic Control
- Outbound Traffic Load Balancing
- Traffic Engineering Goals
- Managing Scale
- Achieving Predictability
- Challenges to Predictability
- Inter-AS Negotiation
- Outbound Multihoming Goals
- Inbound Traffic Control
- How many links are enough
- Problems with Multihoming in IPv4
- Slide 68
- Slide 69
- Diagnosis and Troubleshooting
- Operator Mailing List (NANOG)
- Operator Mailing List
- Routing Configuration
- Internet Business Model (Simplified)
- Peering Contracts Consistent Export
- Consistent Export
- Inconsistent Export in Practice
- Blackholes
- Security
- Security ldquoBogonrdquo Routes
- Spam Phishing etc
- Spam and Routing
- Worms and Botnets
- What is a Worm
- Example Worm Code Red
- Code Red Revisions
- Code Red Host Infection Rate
- Designing Fast-Spreading Worms
- Botnets
- ldquoRallyingrdquo the Botnet
- Botnet Control
- Some Defenses
- Idea 1 Ingress Filtering
- Idea 2 uRPF Checks
- Problems with uRPF
- S-BGP
- Practical Problems with S-BGP
-
41
Packet Capture on High-Speed Links
Example Georgia Tech ldquoOC3Monrdquo
bull Rack-mounted PCbull Optical splitterbull Data Acquisition and
Generation (DAG) card
Source endacecom
42
Characteristics of Packet Capture
bull Allows inspection on every packet on 10G links
bull Disadvantagesndash Costlyndash Requires splitting optical fibersndash Must be able to filterstore data
43
Data Measurement Repositories
bull AbileneInternet 2 Observatoryndash Configuration examplesndash SNMP datandash ISIS BGP routing data NetFlow traffic data
bull RouteViewsndash BGP updatesndash BGP table snapshots
44
Multihoming and Traffic Engineering
45
What is Multihoming
bull The use of redundant network links for the purposes of external connectivity
bull Can be achieved at many layers of the protocol stack and many places in the networkndash Multiple network interfaces in a PC
ndash An ISP with multiple upstream interfaces
bull Can refer to having multiple connections tondash The same ISP
ndash Multiple ISPs
46
Why Multihome
bull Redundancybull Availabilitybull Performancebull Cost
Interdomain traffic engineering the process by which a multihomed network configures its
network to achieve these goals
47
Redundancy
bull Maintain connectivity in the face ofndash Physical connectivity problems (fiber cut device
failures etc)ndash Failures in upstream ISP
48
Performance
bull Use multiple network links at once to achieve higher throughput than just over a single link
bull Allows incoming traffic to be load-balanced
70 of traffic30 of traffic
49
Multihoming in IP Networks Today
bull Stub AS no transit service for other ASesndash No need to use BGP
bull Multi-homed stub AS has connectivity to multiple immediate upstream ISPsndash Need BGPndash No need for a public AS numberndash No need for IP prefix allocation
bull Multi-homed transit AS connectivity to multiple ASes and transit servicendash Need BGP public AS number IP prefix allocation
50
BGP or no
bull Advantages of static routingndash Cheapersmaller routers (less true nowadays)ndash Simpler to configure
bull Advantages of BGPndash More control of your destiny (have providers stop
announcing you)ndash Fastermore intelligent selection of where to send
outbound packetsndash Better debugging of net problems (you can see the
Internet topology now)
51
Same Provider or Multiple
bull If your provider is reliable and fast and affordably and offers good tech-support you may want to multi-home initially to them via some backup path (slow is better than dead)
bull Eventually yoursquoll want to multi-home to different providers to avoid failure modes due to one providerrsquos architecture decisions
52
Multihomed Stub One Link
bull Downstream ISPrsquos routers configure default (ldquostaticrdquo) routes pointing to border router
bull Upstream ISP advertises reachability
Upstream ISP
Multiple links between same pair of routers
Default routes to ldquoborderrdquo
ldquoStubrdquoISP
53
Multihomed Stub Multiple Links
bull Use BGP to share loadbull Use private AS number (why is this OK)bull As before upstream ISP advertises prefix
Upstream ISP
Multiple links to different upstream routers
ldquoStubrdquoISP
Internal routing for ldquohot potatordquo
BGP for load balance at edge
54
Multihomed Stub Multiple ISPs
bull Many possibilitiesndash Load sharingndash Primary-backupndash Selective use of different ISPs
bull Requires BGP public AS number etc
ldquoStubrdquoISP
Upstream
ISP 1
Upstream
ISP 2
55
Multihomed Transit Network
bull BGP everywherebull Incoming and outcoming trafficbull Challenge balancing load on intradomain and egress
links given an offered traffic load
TransitISP
ISP 1
ISP 2
ISP 3
56
Interdomain Traffic Engineering
bull The process by which a network operator configures the network to achievendash Traffic load balancendash Redundancy (primarybackup) etc
bull Two tasksndash Outbound traffic controlndash Inbound traffic control
bull Key Problems Predictability and Scalability
57
Outbound Traffic Control
bull Easier to control than inbound trafficndash Destination-based routing sender determines where
the packets go
bull Control over next-hop AS onlyndash Cannot control selection of the entire path
Provider 1 Provider 2
Control with local preference
58
Outbound Traffic Load Balancingbull Control routes to provider per-prefix
ndash Assign local preference across destination prefixesndash Change the local preference assignments over time
bull Useful inputs to load balancingndash End-to-end path performance datandash Outbound traffic statistics per destination prefix
bull Challenge Getting from traffic volumes to groups of prefixes that should be assigned to each link
Premise of ldquointelligent route controlrdquo preoducts
59
Traffic Engineering Goals
bull Predictabilityndash Ensure the BGP decision process is deterministicndash Assume that BGP updates are (relatively) stable
bull Limit overhead introduced by routing changesndash Minimize frequency of changes to routing policiesndash Limit number of prefixes affected by changes
bull Limit impact on how traffic enters the networkndash Avoid new routes that might change neighborrsquos mindndash Select route with same attributes or at least path length
60
Managing Scalebull Destination prefixes
ndash More than 90000 destination prefixes
bull Donrsquot want to have per-prefix routing policies
ndash Small fraction of prefixes contribute most of the traffic
bull Focus on the small number of heavy hitters
ndash Define routing policies for selected prefixes
bull Routing choicesndash About 27000 unique ldquorouting choicesrdquo
bull Help in reducing the scale of the problem
ndash Small fraction of ldquorouting choicesrdquo contribute most traffic
bull Focus on the very small number of ldquorouting choicesrdquo
ndash Define routing policies on common attributes
61
Achieving Predictability
bull Route prediction with static analysisndash Helpful to know effects before deploymentndash Static analysis can help
TopologyBGP policy
configuration
eBGP routes
Offered traffic
BGP routingmodel
Flow of traffic through the network
62
Challenges to Predictabilitybull For transit ISPs effects on incoming traffic
ndash Lack of coordination strikes again
63
ldquoHot Potatordquo routing
Inter-AS Negotiation
bull Coordination aids predictabilityndash Negotiate where to sendndash Inbound and outboundndash Mutual benefits
bull How to implementndash What info to exchangendash Protecting privacyndash How to prioritize choicesndash How to prevent cheating
Destination 2
Destination 1
multiplepeeringpoints
Provider A
Provider B
64
Outbound Multihoming Goals
bull Redundancyndash Dynamic routing will failover to backup link
bull Performancendash Select provider with best performance per prefix
ndash Requires active probing
bull Costndash Select provider per prefix over time to minimize the
total financial cost
65
Inbound Traffic Control
bull More difficult no control over neighborsrsquo decisions
bull Three common techniques (previously discussed)ndash AS path prependingndash Communities and local preferencendash Prefix splitting
How does todayrsquos paper (MONET) control inbound traffic
66
How many links are enough
K upstream ISPs
Not much benefit beyond 4 ISPs
Akella et al ldquoPerformance Benefits of Multihomingrdquo SIGCOMM 2003
67
Problems with Multihoming in IPv4
bull Routing table growthndash Provider-based addressingndash Advertising prefix out multiple ISPs ndash canrsquot aggregate
bull Poor control over inbound trafficndash Existing mechanisms do not allow hosts to control
inbound traffic
68
GeorgiaTech
Internet Routing Overview
bull Intradomain (ie ldquointra-ASrdquo) routingbull Interdomain routing
Comcast
Abilene
ATampT Cogent
Autonomous Systems (ASes)
69
Configuration Problems ldquoAS 7007rdquoldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
UUNet
Florida InternetBarn
Sprint
70
Diagnosis and Troubleshootingldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
ldquoMicrosofts websites were offline for up to 23 hoursbecause of a [router] misconfigurationhellipit took nearly a day to determine what was wrong and undo the changesrdquo -- wiredcom January 25 2001
ldquoWorldCom Inchellipsuffered a widespread outage on its Internet backbone that affected roughly 20 percent of its US customer base The network problemshellipaffected millions of computer users worldwide A spokeswoman attributed the outage to a route table issue -- cnncom October 3 2002
A number of Covad customers went out from 5pm today due to supposedly a DDOS (distributed denial of service attack) on a key Level3 data center which later was described as a route leak (misconfiguration)ldquo
-- dslreportscom February 23 2004
71
Operator Mailing List (NANOG)Date Mon 18 Oct 2004 091515 -0700Subject Level 3 US east coast issues
Level 3 experiencing widespread unspecified routing issues on the US east coast Master ticket 1086844 Anyone have more specific information
Date Mon 18 Oct 2004 122034 -0400 (EDT)Subject Re Level 3 US east coast issues
Level 3 is currently experiencing a backbone outage causing routing instability and packet loss We are working to restore and will be sending hourly updateshellip
72
Operator Mailing List
0102030405060708090
Filtering RouteLeaks
RouteHijacks
RouteInstability
RoutingLoops
Blackholes
Occurr
ences o
ver
10 Y
ears
1995-1997 1998-2001 2001-2004
Note Only includes problems openly discussed on this list
Compare 83 power outages 1 fire
73
Routing Configuration
Ranking route selection
Dissemination internal route advertisement
Filtering route advertisement
Customer
Competitor
Primary
Backup
74
Internet Business Model (Simplified)
bull CustomerProvider One AS pays another for reachability to some set of destinations
bull ldquoSettlement-freerdquo Peering Bartering Two ASes exchange routes with one another
Provider
Peer
Customer
Preferences implemented with local preference manipulation
Destination
Pay to use
Get paid to use
Free to use
75
Peering Contracts Consistent Export
bull Rules of settlement-free peeringndash Advertise routes at all peering pointsndash Advertised routes must have equal ldquoAS path lengthrdquo
Sprint
ATampT
Enables ldquohot potatordquo routing
ldquoequally goodrdquoroutes
76
Consistent Export
bull Malicedeceptionbull iBGP signaling partitionbull Inconsistent export policy
neighbor 10123route-map PEER permit 10 set prepend 123
neighbor 10456route-map PEER permit 10 set prepend 123 123
Possible Causes
10123 456 1
10456 456 2
Neighbor AS Export
1 1 123
Export Clause Prepend
2 1 123 123
Two different Export Policies
77
Inconsistent Export in Practice
Feamster et al ldquoBorderGuard Detecting Cold Potatoes from Peersrdquo ACM IMC October 2004
78
Blackholes
Date Thu 18 Jul 2002 060510 -0400 (EDT)From Chad Oleary ltcolpoboxcomgtSubject Re problems with 701To ltnanogmeritedugt
Were starting to see the same issues with UUNet again Anyone elseseeing this Trying to reach Qwest
traceroute to 631461901 (631461901) 30 hops max 38 byte packets 1 esc-lp2-gwe-solutionscorpcom (631182201) 1167 ms 1163 ms 1142 ms 2 500Serial2-10GW1TPA2ALTERNET (1571301499) 1097 ms 1059 ms 1044 ms 3 161at-1-0-0XL4ATL1ALTERNET (1526381190) 13839 ms 14108 ms 16638 ms 4 0so-3-1-0XL2ATL5ALTERNET (152630238) 14370 ms 14587 ms 14553 ms 5 POS7-0BR2ATL5ALTERNET (1526382193) 13928 ms 14099 ms 14053 ms 6 7 hellip
79
Security
80
Security ldquoBogonrdquo Routes
Feamster et al ldquoAn Empirical Study of lsquoBogonrsquo Route Advertisementsrdquo ACM CCR January 2005
81
Spam Phishing etc
bull Unsolicited commercial emailbull As of about August 2008 estimates indicate that
about 95 of all email is spambull Common spam filtering techniques
ndash Content-based filtersndash DNS Blacklist (DNSBL) lookups Significant fraction of
todayrsquos DNS traffic
Can IP addresses from which spam is received be spoofed
82
Spam and Routing
83
Worms and Botnets
84
What is a Worm
bull Code that replicates and propagates across the networkndash Often carries a ldquopayloadrdquo
bull Usually spread via exploiting flaws in open servicesndash ldquoVirusesrdquo require user action to spread
bull First worm Robert Morris November 1988ndash 6-10 of all Internet hosts infected ()
bull Many more since but none on that scale until July 2001
85
Example Worm Code Red
bull Initial version July 13 2001
bull Exploited known ISAPI vulnerability in Microsoft IIS Web servers
bull 1st through 20th of each month spread20th through end of each month attack
bull Payload Web site defacementbull Scanning Random IP addressesbull Bug failure to seed random number generator
86
Code Red Revisions
bull Released July 19 2001
bull Payload flooding attack on wwwwhitehousegovndash Attack was mounted at the IP address of the Web site
bull Bug died after 20th of each month
bull Random number generator for IP scanning fixed
87
Code Red Host Infection Rate
Exponential infection rate
Measured using backscatter technique
88
Designing Fast-Spreading Worms
bull Hit-list scanningndash Time to infect first 10k hosts dominates infection timendash Solution Reconnaissance (stealthy scans etc)
bull Permutation scanningndash Observation Most scanning is redundantndash Idea Shared permutation of address space Start scanning
from own IP address Re-randomize when another infected machine is found
bull Internet-scale hit listsndash Flash worm complete infection within 30 seconds
89
Botnets
bull Bots Autonomous programs performing tasksbull Plenty of ldquobenignrdquo bots
ndash eg weatherbug
bull Botnets group of bots ndash Typically carries malicious connotationndash Large numbers of infected machinesndash Machines ldquoenlistedrdquo with infection vectors like worms
(last lecture)
bull Available for simultaneous control by a masterbull Size up to 350000 nodes (from todayrsquos paper)
90
ldquoRallyingrdquo the Botnet
bull Easy to combine worm backdoor functionalitybull Problem how to learn about successfully
infected machines
bull Optionsndash Emailndash Hard-coded email address
91
Botnet Control
bull Botnet master typically runs some IRC server on a well-known port (eg 6667)
bull Infected machine contacts botnet with pre-programmed DNS name (eg big-botde)
bull Dynamic DNS allows controller to move about freely
Infected Machine
DynamicDNS
BotnetController
(IRC server)
92
Some Defenses
93
Idea 1 Ingress Filtering
bull RFC 2827 Routers install filters to drop packets from networks that are not downstream
bull Feasible at edgesbull Difficult to configure closer to network ldquocorerdquo
20469207024 Internet
Drop all packets with source address other than 20469207024
94
Idea 2 uRPF Checks
bull Unicast Reverse Path Forwardingndash Cisco ldquoip verify unicast reverse-pathrdquo
bull Requires symmetric routing
Accept packet from interface only if forwarding table entry for source IP address matches ingress interface
100183
A10018
10015
101203
100181 24
10011 24
Strict Mode uRPF
Enabled
ldquoArdquo Routing TableDestination Next Hop1001024 Int 110018024 Int 2
100183 from wrong interface
95
Problems with uRPF
bull Asymmetric routing
96
S-BGP
bull Address-based PKI validate signaturesndash Authentication of
bull ownership for IP address blocks bull AS number bull an ASs identity and bull a BGP routers identity
ndash Use existing infrastructure (Internet registries etc)ndash Routing origination is digitally signedndash BGP updates are digitally signed
1048708 bull Route attestations A new optional BGP transitive path attribute
ndash carries digital signatures covering the routing information in updates
97
Practical Problems with S-BGP
bull Requires Public-Key Infrastructure
bull Lots of digital signatures to calculate and verifyndash Message overheadndash CPU overhead
bull Calculation expense is greatest when topology is changingndash Caching can help
bull Route aggregation is problematic (maybe thatrsquos OK)
bull Secure route withdrawals when link or node fails
bull Address ownership data out of date
bull Deployment
- Networking
- Goal of This Tutorial
- Todayrsquos Networks
- Business Models
- Billing for Internet Usage
- Net Neutrality
- Network Operations
- Point-of-Presence (PoP)
- Example Abilene Network Topology
- Another Example Backbone
- Internet Routing Overview
- Internet Routing Protocol BGP
- Two Flavors of BGP
- IPv4 Addresses Networks of Networks
- Pre-1994 Classful Addressing
- Classless Interdomain Routing (CIDR)
- Benefits of CIDR
- Growth of IP Prefixes
- 1994-1998 Linear Growth
- Around 2000 Fast Growth Resumes
- Fast growth resumes
- The Address Allocation Process
- Common Problems
- What can go wrong
- Measurement and Monitoring
- Passive vs Active Measurement
- Slide 27
- Passive Traffic Data Measurement
- Simple Network Management Protocol
- SNMP (Passive)
- Packet-level Monitoring
- Full Packet Capture (Passive)
- What is a flow
- Cisco NetFlow
- Flow Record Contents
- Aggregating Packets into Flows
- Reducing Measurement Overhead
- Packet Sampling for Flow Monitoring
- Sampling Flow-Level Sampling
- Two Main Approaches
- Packet Capture on High-Speed Links
- Characteristics of Packet Capture
- Data Measurement Repositories
- Multihoming and Traffic Engineering
- What is Multihoming
- Why Multihome
- Redundancy
- Performance
- Multihoming in IP Networks Today
- BGP or no
- Same Provider or Multiple
- Multihomed Stub One Link
- Multihomed Stub Multiple Links
- Multihomed Stub Multiple ISPs
- Multihomed Transit Network
- Interdomain Traffic Engineering
- Outbound Traffic Control
- Outbound Traffic Load Balancing
- Traffic Engineering Goals
- Managing Scale
- Achieving Predictability
- Challenges to Predictability
- Inter-AS Negotiation
- Outbound Multihoming Goals
- Inbound Traffic Control
- How many links are enough
- Problems with Multihoming in IPv4
- Slide 68
- Slide 69
- Diagnosis and Troubleshooting
- Operator Mailing List (NANOG)
- Operator Mailing List
- Routing Configuration
- Internet Business Model (Simplified)
- Peering Contracts Consistent Export
- Consistent Export
- Inconsistent Export in Practice
- Blackholes
- Security
- Security ldquoBogonrdquo Routes
- Spam Phishing etc
- Spam and Routing
- Worms and Botnets
- What is a Worm
- Example Worm Code Red
- Code Red Revisions
- Code Red Host Infection Rate
- Designing Fast-Spreading Worms
- Botnets
- ldquoRallyingrdquo the Botnet
- Botnet Control
- Some Defenses
- Idea 1 Ingress Filtering
- Idea 2 uRPF Checks
- Problems with uRPF
- S-BGP
- Practical Problems with S-BGP
-
42
Characteristics of Packet Capture
bull Allows inspection on every packet on 10G links
bull Disadvantagesndash Costlyndash Requires splitting optical fibersndash Must be able to filterstore data
43
Data Measurement Repositories
bull AbileneInternet 2 Observatoryndash Configuration examplesndash SNMP datandash ISIS BGP routing data NetFlow traffic data
bull RouteViewsndash BGP updatesndash BGP table snapshots
44
Multihoming and Traffic Engineering
45
What is Multihoming
bull The use of redundant network links for the purposes of external connectivity
bull Can be achieved at many layers of the protocol stack and many places in the networkndash Multiple network interfaces in a PC
ndash An ISP with multiple upstream interfaces
bull Can refer to having multiple connections tondash The same ISP
ndash Multiple ISPs
46
Why Multihome
bull Redundancybull Availabilitybull Performancebull Cost
Interdomain traffic engineering the process by which a multihomed network configures its
network to achieve these goals
47
Redundancy
bull Maintain connectivity in the face ofndash Physical connectivity problems (fiber cut device
failures etc)ndash Failures in upstream ISP
48
Performance
bull Use multiple network links at once to achieve higher throughput than just over a single link
bull Allows incoming traffic to be load-balanced
70 of traffic30 of traffic
49
Multihoming in IP Networks Today
bull Stub AS no transit service for other ASesndash No need to use BGP
bull Multi-homed stub AS has connectivity to multiple immediate upstream ISPsndash Need BGPndash No need for a public AS numberndash No need for IP prefix allocation
bull Multi-homed transit AS connectivity to multiple ASes and transit servicendash Need BGP public AS number IP prefix allocation
50
BGP or no
bull Advantages of static routingndash Cheapersmaller routers (less true nowadays)ndash Simpler to configure
bull Advantages of BGPndash More control of your destiny (have providers stop
announcing you)ndash Fastermore intelligent selection of where to send
outbound packetsndash Better debugging of net problems (you can see the
Internet topology now)
51
Same Provider or Multiple
bull If your provider is reliable and fast and affordably and offers good tech-support you may want to multi-home initially to them via some backup path (slow is better than dead)
bull Eventually yoursquoll want to multi-home to different providers to avoid failure modes due to one providerrsquos architecture decisions
52
Multihomed Stub One Link
bull Downstream ISPrsquos routers configure default (ldquostaticrdquo) routes pointing to border router
bull Upstream ISP advertises reachability
Upstream ISP
Multiple links between same pair of routers
Default routes to ldquoborderrdquo
ldquoStubrdquoISP
53
Multihomed Stub Multiple Links
bull Use BGP to share loadbull Use private AS number (why is this OK)bull As before upstream ISP advertises prefix
Upstream ISP
Multiple links to different upstream routers
ldquoStubrdquoISP
Internal routing for ldquohot potatordquo
BGP for load balance at edge
54
Multihomed Stub Multiple ISPs
bull Many possibilitiesndash Load sharingndash Primary-backupndash Selective use of different ISPs
bull Requires BGP public AS number etc
ldquoStubrdquoISP
Upstream
ISP 1
Upstream
ISP 2
55
Multihomed Transit Network
bull BGP everywherebull Incoming and outcoming trafficbull Challenge balancing load on intradomain and egress
links given an offered traffic load
TransitISP
ISP 1
ISP 2
ISP 3
56
Interdomain Traffic Engineering
bull The process by which a network operator configures the network to achievendash Traffic load balancendash Redundancy (primarybackup) etc
bull Two tasksndash Outbound traffic controlndash Inbound traffic control
bull Key Problems Predictability and Scalability
57
Outbound Traffic Control
bull Easier to control than inbound trafficndash Destination-based routing sender determines where
the packets go
bull Control over next-hop AS onlyndash Cannot control selection of the entire path
Provider 1 Provider 2
Control with local preference
58
Outbound Traffic Load Balancingbull Control routes to provider per-prefix
ndash Assign local preference across destination prefixesndash Change the local preference assignments over time
bull Useful inputs to load balancingndash End-to-end path performance datandash Outbound traffic statistics per destination prefix
bull Challenge Getting from traffic volumes to groups of prefixes that should be assigned to each link
Premise of ldquointelligent route controlrdquo preoducts
59
Traffic Engineering Goals
bull Predictabilityndash Ensure the BGP decision process is deterministicndash Assume that BGP updates are (relatively) stable
bull Limit overhead introduced by routing changesndash Minimize frequency of changes to routing policiesndash Limit number of prefixes affected by changes
bull Limit impact on how traffic enters the networkndash Avoid new routes that might change neighborrsquos mindndash Select route with same attributes or at least path length
60
Managing Scalebull Destination prefixes
ndash More than 90000 destination prefixes
bull Donrsquot want to have per-prefix routing policies
ndash Small fraction of prefixes contribute most of the traffic
bull Focus on the small number of heavy hitters
ndash Define routing policies for selected prefixes
bull Routing choicesndash About 27000 unique ldquorouting choicesrdquo
bull Help in reducing the scale of the problem
ndash Small fraction of ldquorouting choicesrdquo contribute most traffic
bull Focus on the very small number of ldquorouting choicesrdquo
ndash Define routing policies on common attributes
61
Achieving Predictability
bull Route prediction with static analysisndash Helpful to know effects before deploymentndash Static analysis can help
TopologyBGP policy
configuration
eBGP routes
Offered traffic
BGP routingmodel
Flow of traffic through the network
62
Challenges to Predictabilitybull For transit ISPs effects on incoming traffic
ndash Lack of coordination strikes again
63
ldquoHot Potatordquo routing
Inter-AS Negotiation
bull Coordination aids predictabilityndash Negotiate where to sendndash Inbound and outboundndash Mutual benefits
bull How to implementndash What info to exchangendash Protecting privacyndash How to prioritize choicesndash How to prevent cheating
Destination 2
Destination 1
multiplepeeringpoints
Provider A
Provider B
64
Outbound Multihoming Goals
bull Redundancyndash Dynamic routing will failover to backup link
bull Performancendash Select provider with best performance per prefix
ndash Requires active probing
bull Costndash Select provider per prefix over time to minimize the
total financial cost
65
Inbound Traffic Control
bull More difficult no control over neighborsrsquo decisions
bull Three common techniques (previously discussed)ndash AS path prependingndash Communities and local preferencendash Prefix splitting
How does todayrsquos paper (MONET) control inbound traffic
66
How many links are enough
K upstream ISPs
Not much benefit beyond 4 ISPs
Akella et al ldquoPerformance Benefits of Multihomingrdquo SIGCOMM 2003
67
Problems with Multihoming in IPv4
bull Routing table growthndash Provider-based addressingndash Advertising prefix out multiple ISPs ndash canrsquot aggregate
bull Poor control over inbound trafficndash Existing mechanisms do not allow hosts to control
inbound traffic
68
GeorgiaTech
Internet Routing Overview
bull Intradomain (ie ldquointra-ASrdquo) routingbull Interdomain routing
Comcast
Abilene
ATampT Cogent
Autonomous Systems (ASes)
69
Configuration Problems ldquoAS 7007rdquoldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
UUNet
Florida InternetBarn
Sprint
70
Diagnosis and Troubleshootingldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
ldquoMicrosofts websites were offline for up to 23 hoursbecause of a [router] misconfigurationhellipit took nearly a day to determine what was wrong and undo the changesrdquo -- wiredcom January 25 2001
ldquoWorldCom Inchellipsuffered a widespread outage on its Internet backbone that affected roughly 20 percent of its US customer base The network problemshellipaffected millions of computer users worldwide A spokeswoman attributed the outage to a route table issue -- cnncom October 3 2002
A number of Covad customers went out from 5pm today due to supposedly a DDOS (distributed denial of service attack) on a key Level3 data center which later was described as a route leak (misconfiguration)ldquo
-- dslreportscom February 23 2004
71
Operator Mailing List (NANOG)Date Mon 18 Oct 2004 091515 -0700Subject Level 3 US east coast issues
Level 3 experiencing widespread unspecified routing issues on the US east coast Master ticket 1086844 Anyone have more specific information
Date Mon 18 Oct 2004 122034 -0400 (EDT)Subject Re Level 3 US east coast issues
Level 3 is currently experiencing a backbone outage causing routing instability and packet loss We are working to restore and will be sending hourly updateshellip
72
Operator Mailing List
0102030405060708090
Filtering RouteLeaks
RouteHijacks
RouteInstability
RoutingLoops
Blackholes
Occurr
ences o
ver
10 Y
ears
1995-1997 1998-2001 2001-2004
Note Only includes problems openly discussed on this list
Compare 83 power outages 1 fire
73
Routing Configuration
Ranking route selection
Dissemination internal route advertisement
Filtering route advertisement
Customer
Competitor
Primary
Backup
74
Internet Business Model (Simplified)
bull CustomerProvider One AS pays another for reachability to some set of destinations
bull ldquoSettlement-freerdquo Peering Bartering Two ASes exchange routes with one another
Provider
Peer
Customer
Preferences implemented with local preference manipulation
Destination
Pay to use
Get paid to use
Free to use
75
Peering Contracts Consistent Export
bull Rules of settlement-free peeringndash Advertise routes at all peering pointsndash Advertised routes must have equal ldquoAS path lengthrdquo
Sprint
ATampT
Enables ldquohot potatordquo routing
ldquoequally goodrdquoroutes
76
Consistent Export
bull Malicedeceptionbull iBGP signaling partitionbull Inconsistent export policy
neighbor 10123route-map PEER permit 10 set prepend 123
neighbor 10456route-map PEER permit 10 set prepend 123 123
Possible Causes
10123 456 1
10456 456 2
Neighbor AS Export
1 1 123
Export Clause Prepend
2 1 123 123
Two different Export Policies
77
Inconsistent Export in Practice
Feamster et al ldquoBorderGuard Detecting Cold Potatoes from Peersrdquo ACM IMC October 2004
78
Blackholes
Date Thu 18 Jul 2002 060510 -0400 (EDT)From Chad Oleary ltcolpoboxcomgtSubject Re problems with 701To ltnanogmeritedugt
Were starting to see the same issues with UUNet again Anyone elseseeing this Trying to reach Qwest
traceroute to 631461901 (631461901) 30 hops max 38 byte packets 1 esc-lp2-gwe-solutionscorpcom (631182201) 1167 ms 1163 ms 1142 ms 2 500Serial2-10GW1TPA2ALTERNET (1571301499) 1097 ms 1059 ms 1044 ms 3 161at-1-0-0XL4ATL1ALTERNET (1526381190) 13839 ms 14108 ms 16638 ms 4 0so-3-1-0XL2ATL5ALTERNET (152630238) 14370 ms 14587 ms 14553 ms 5 POS7-0BR2ATL5ALTERNET (1526382193) 13928 ms 14099 ms 14053 ms 6 7 hellip
79
Security
80
Security ldquoBogonrdquo Routes
Feamster et al ldquoAn Empirical Study of lsquoBogonrsquo Route Advertisementsrdquo ACM CCR January 2005
81
Spam Phishing etc
bull Unsolicited commercial emailbull As of about August 2008 estimates indicate that
about 95 of all email is spambull Common spam filtering techniques
ndash Content-based filtersndash DNS Blacklist (DNSBL) lookups Significant fraction of
todayrsquos DNS traffic
Can IP addresses from which spam is received be spoofed
82
Spam and Routing
83
Worms and Botnets
84
What is a Worm
bull Code that replicates and propagates across the networkndash Often carries a ldquopayloadrdquo
bull Usually spread via exploiting flaws in open servicesndash ldquoVirusesrdquo require user action to spread
bull First worm Robert Morris November 1988ndash 6-10 of all Internet hosts infected ()
bull Many more since but none on that scale until July 2001
85
Example Worm Code Red
bull Initial version July 13 2001
bull Exploited known ISAPI vulnerability in Microsoft IIS Web servers
bull 1st through 20th of each month spread20th through end of each month attack
bull Payload Web site defacementbull Scanning Random IP addressesbull Bug failure to seed random number generator
86
Code Red Revisions
bull Released July 19 2001
bull Payload flooding attack on wwwwhitehousegovndash Attack was mounted at the IP address of the Web site
bull Bug died after 20th of each month
bull Random number generator for IP scanning fixed
87
Code Red Host Infection Rate
Exponential infection rate
Measured using backscatter technique
88
Designing Fast-Spreading Worms
bull Hit-list scanningndash Time to infect first 10k hosts dominates infection timendash Solution Reconnaissance (stealthy scans etc)
bull Permutation scanningndash Observation Most scanning is redundantndash Idea Shared permutation of address space Start scanning
from own IP address Re-randomize when another infected machine is found
bull Internet-scale hit listsndash Flash worm complete infection within 30 seconds
89
Botnets
bull Bots Autonomous programs performing tasksbull Plenty of ldquobenignrdquo bots
ndash eg weatherbug
bull Botnets group of bots ndash Typically carries malicious connotationndash Large numbers of infected machinesndash Machines ldquoenlistedrdquo with infection vectors like worms
(last lecture)
bull Available for simultaneous control by a masterbull Size up to 350000 nodes (from todayrsquos paper)
90
ldquoRallyingrdquo the Botnet
bull Easy to combine worm backdoor functionalitybull Problem how to learn about successfully
infected machines
bull Optionsndash Emailndash Hard-coded email address
91
Botnet Control
bull Botnet master typically runs some IRC server on a well-known port (eg 6667)
bull Infected machine contacts botnet with pre-programmed DNS name (eg big-botde)
bull Dynamic DNS allows controller to move about freely
Infected Machine
DynamicDNS
BotnetController
(IRC server)
92
Some Defenses
93
Idea 1 Ingress Filtering
bull RFC 2827 Routers install filters to drop packets from networks that are not downstream
bull Feasible at edgesbull Difficult to configure closer to network ldquocorerdquo
20469207024 Internet
Drop all packets with source address other than 20469207024
94
Idea 2 uRPF Checks
bull Unicast Reverse Path Forwardingndash Cisco ldquoip verify unicast reverse-pathrdquo
bull Requires symmetric routing
Accept packet from interface only if forwarding table entry for source IP address matches ingress interface
100183
A10018
10015
101203
100181 24
10011 24
Strict Mode uRPF
Enabled
ldquoArdquo Routing TableDestination Next Hop1001024 Int 110018024 Int 2
100183 from wrong interface
95
Problems with uRPF
bull Asymmetric routing
96
S-BGP
bull Address-based PKI validate signaturesndash Authentication of
bull ownership for IP address blocks bull AS number bull an ASs identity and bull a BGP routers identity
ndash Use existing infrastructure (Internet registries etc)ndash Routing origination is digitally signedndash BGP updates are digitally signed
1048708 bull Route attestations A new optional BGP transitive path attribute
ndash carries digital signatures covering the routing information in updates
97
Practical Problems with S-BGP
bull Requires Public-Key Infrastructure
bull Lots of digital signatures to calculate and verifyndash Message overheadndash CPU overhead
bull Calculation expense is greatest when topology is changingndash Caching can help
bull Route aggregation is problematic (maybe thatrsquos OK)
bull Secure route withdrawals when link or node fails
bull Address ownership data out of date
bull Deployment
- Networking
- Goal of This Tutorial
- Todayrsquos Networks
- Business Models
- Billing for Internet Usage
- Net Neutrality
- Network Operations
- Point-of-Presence (PoP)
- Example Abilene Network Topology
- Another Example Backbone
- Internet Routing Overview
- Internet Routing Protocol BGP
- Two Flavors of BGP
- IPv4 Addresses Networks of Networks
- Pre-1994 Classful Addressing
- Classless Interdomain Routing (CIDR)
- Benefits of CIDR
- Growth of IP Prefixes
- 1994-1998 Linear Growth
- Around 2000 Fast Growth Resumes
- Fast growth resumes
- The Address Allocation Process
- Common Problems
- What can go wrong
- Measurement and Monitoring
- Passive vs Active Measurement
- Slide 27
- Passive Traffic Data Measurement
- Simple Network Management Protocol
- SNMP (Passive)
- Packet-level Monitoring
- Full Packet Capture (Passive)
- What is a flow
- Cisco NetFlow
- Flow Record Contents
- Aggregating Packets into Flows
- Reducing Measurement Overhead
- Packet Sampling for Flow Monitoring
- Sampling Flow-Level Sampling
- Two Main Approaches
- Packet Capture on High-Speed Links
- Characteristics of Packet Capture
- Data Measurement Repositories
- Multihoming and Traffic Engineering
- What is Multihoming
- Why Multihome
- Redundancy
- Performance
- Multihoming in IP Networks Today
- BGP or no
- Same Provider or Multiple
- Multihomed Stub One Link
- Multihomed Stub Multiple Links
- Multihomed Stub Multiple ISPs
- Multihomed Transit Network
- Interdomain Traffic Engineering
- Outbound Traffic Control
- Outbound Traffic Load Balancing
- Traffic Engineering Goals
- Managing Scale
- Achieving Predictability
- Challenges to Predictability
- Inter-AS Negotiation
- Outbound Multihoming Goals
- Inbound Traffic Control
- How many links are enough
- Problems with Multihoming in IPv4
- Slide 68
- Slide 69
- Diagnosis and Troubleshooting
- Operator Mailing List (NANOG)
- Operator Mailing List
- Routing Configuration
- Internet Business Model (Simplified)
- Peering Contracts Consistent Export
- Consistent Export
- Inconsistent Export in Practice
- Blackholes
- Security
- Security ldquoBogonrdquo Routes
- Spam Phishing etc
- Spam and Routing
- Worms and Botnets
- What is a Worm
- Example Worm Code Red
- Code Red Revisions
- Code Red Host Infection Rate
- Designing Fast-Spreading Worms
- Botnets
- ldquoRallyingrdquo the Botnet
- Botnet Control
- Some Defenses
- Idea 1 Ingress Filtering
- Idea 2 uRPF Checks
- Problems with uRPF
- S-BGP
- Practical Problems with S-BGP
-
43
Data Measurement Repositories
bull AbileneInternet 2 Observatoryndash Configuration examplesndash SNMP datandash ISIS BGP routing data NetFlow traffic data
bull RouteViewsndash BGP updatesndash BGP table snapshots
44
Multihoming and Traffic Engineering
45
What is Multihoming
bull The use of redundant network links for the purposes of external connectivity
bull Can be achieved at many layers of the protocol stack and many places in the networkndash Multiple network interfaces in a PC
ndash An ISP with multiple upstream interfaces
bull Can refer to having multiple connections tondash The same ISP
ndash Multiple ISPs
46
Why Multihome
bull Redundancybull Availabilitybull Performancebull Cost
Interdomain traffic engineering the process by which a multihomed network configures its
network to achieve these goals
47
Redundancy
bull Maintain connectivity in the face ofndash Physical connectivity problems (fiber cut device
failures etc)ndash Failures in upstream ISP
48
Performance
bull Use multiple network links at once to achieve higher throughput than just over a single link
bull Allows incoming traffic to be load-balanced
70 of traffic30 of traffic
49
Multihoming in IP Networks Today
bull Stub AS no transit service for other ASesndash No need to use BGP
bull Multi-homed stub AS has connectivity to multiple immediate upstream ISPsndash Need BGPndash No need for a public AS numberndash No need for IP prefix allocation
bull Multi-homed transit AS connectivity to multiple ASes and transit servicendash Need BGP public AS number IP prefix allocation
50
BGP or no
bull Advantages of static routingndash Cheapersmaller routers (less true nowadays)ndash Simpler to configure
bull Advantages of BGPndash More control of your destiny (have providers stop
announcing you)ndash Fastermore intelligent selection of where to send
outbound packetsndash Better debugging of net problems (you can see the
Internet topology now)
51
Same Provider or Multiple
bull If your provider is reliable and fast and affordably and offers good tech-support you may want to multi-home initially to them via some backup path (slow is better than dead)
bull Eventually yoursquoll want to multi-home to different providers to avoid failure modes due to one providerrsquos architecture decisions
52
Multihomed Stub One Link
bull Downstream ISPrsquos routers configure default (ldquostaticrdquo) routes pointing to border router
bull Upstream ISP advertises reachability
Upstream ISP
Multiple links between same pair of routers
Default routes to ldquoborderrdquo
ldquoStubrdquoISP
53
Multihomed Stub Multiple Links
bull Use BGP to share loadbull Use private AS number (why is this OK)bull As before upstream ISP advertises prefix
Upstream ISP
Multiple links to different upstream routers
ldquoStubrdquoISP
Internal routing for ldquohot potatordquo
BGP for load balance at edge
54
Multihomed Stub Multiple ISPs
bull Many possibilitiesndash Load sharingndash Primary-backupndash Selective use of different ISPs
bull Requires BGP public AS number etc
ldquoStubrdquoISP
Upstream
ISP 1
Upstream
ISP 2
55
Multihomed Transit Network
bull BGP everywherebull Incoming and outcoming trafficbull Challenge balancing load on intradomain and egress
links given an offered traffic load
TransitISP
ISP 1
ISP 2
ISP 3
56
Interdomain Traffic Engineering
bull The process by which a network operator configures the network to achievendash Traffic load balancendash Redundancy (primarybackup) etc
bull Two tasksndash Outbound traffic controlndash Inbound traffic control
bull Key Problems Predictability and Scalability
57
Outbound Traffic Control
bull Easier to control than inbound trafficndash Destination-based routing sender determines where
the packets go
bull Control over next-hop AS onlyndash Cannot control selection of the entire path
Provider 1 Provider 2
Control with local preference
58
Outbound Traffic Load Balancingbull Control routes to provider per-prefix
ndash Assign local preference across destination prefixesndash Change the local preference assignments over time
bull Useful inputs to load balancingndash End-to-end path performance datandash Outbound traffic statistics per destination prefix
bull Challenge Getting from traffic volumes to groups of prefixes that should be assigned to each link
Premise of ldquointelligent route controlrdquo preoducts
59
Traffic Engineering Goals
bull Predictabilityndash Ensure the BGP decision process is deterministicndash Assume that BGP updates are (relatively) stable
bull Limit overhead introduced by routing changesndash Minimize frequency of changes to routing policiesndash Limit number of prefixes affected by changes
bull Limit impact on how traffic enters the networkndash Avoid new routes that might change neighborrsquos mindndash Select route with same attributes or at least path length
60
Managing Scalebull Destination prefixes
ndash More than 90000 destination prefixes
bull Donrsquot want to have per-prefix routing policies
ndash Small fraction of prefixes contribute most of the traffic
bull Focus on the small number of heavy hitters
ndash Define routing policies for selected prefixes
bull Routing choicesndash About 27000 unique ldquorouting choicesrdquo
bull Help in reducing the scale of the problem
ndash Small fraction of ldquorouting choicesrdquo contribute most traffic
bull Focus on the very small number of ldquorouting choicesrdquo
ndash Define routing policies on common attributes
61
Achieving Predictability
bull Route prediction with static analysisndash Helpful to know effects before deploymentndash Static analysis can help
TopologyBGP policy
configuration
eBGP routes
Offered traffic
BGP routingmodel
Flow of traffic through the network
62
Challenges to Predictabilitybull For transit ISPs effects on incoming traffic
ndash Lack of coordination strikes again
63
ldquoHot Potatordquo routing
Inter-AS Negotiation
bull Coordination aids predictabilityndash Negotiate where to sendndash Inbound and outboundndash Mutual benefits
bull How to implementndash What info to exchangendash Protecting privacyndash How to prioritize choicesndash How to prevent cheating
Destination 2
Destination 1
multiplepeeringpoints
Provider A
Provider B
64
Outbound Multihoming Goals
bull Redundancyndash Dynamic routing will failover to backup link
bull Performancendash Select provider with best performance per prefix
ndash Requires active probing
bull Costndash Select provider per prefix over time to minimize the
total financial cost
65
Inbound Traffic Control
bull More difficult no control over neighborsrsquo decisions
bull Three common techniques (previously discussed)ndash AS path prependingndash Communities and local preferencendash Prefix splitting
How does todayrsquos paper (MONET) control inbound traffic
66
How many links are enough
K upstream ISPs
Not much benefit beyond 4 ISPs
Akella et al ldquoPerformance Benefits of Multihomingrdquo SIGCOMM 2003
67
Problems with Multihoming in IPv4
bull Routing table growthndash Provider-based addressingndash Advertising prefix out multiple ISPs ndash canrsquot aggregate
bull Poor control over inbound trafficndash Existing mechanisms do not allow hosts to control
inbound traffic
68
GeorgiaTech
Internet Routing Overview
bull Intradomain (ie ldquointra-ASrdquo) routingbull Interdomain routing
Comcast
Abilene
ATampT Cogent
Autonomous Systems (ASes)
69
Configuration Problems ldquoAS 7007rdquoldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
UUNet
Florida InternetBarn
Sprint
70
Diagnosis and Troubleshootingldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
ldquoMicrosofts websites were offline for up to 23 hoursbecause of a [router] misconfigurationhellipit took nearly a day to determine what was wrong and undo the changesrdquo -- wiredcom January 25 2001
ldquoWorldCom Inchellipsuffered a widespread outage on its Internet backbone that affected roughly 20 percent of its US customer base The network problemshellipaffected millions of computer users worldwide A spokeswoman attributed the outage to a route table issue -- cnncom October 3 2002
A number of Covad customers went out from 5pm today due to supposedly a DDOS (distributed denial of service attack) on a key Level3 data center which later was described as a route leak (misconfiguration)ldquo
-- dslreportscom February 23 2004
71
Operator Mailing List (NANOG)Date Mon 18 Oct 2004 091515 -0700Subject Level 3 US east coast issues
Level 3 experiencing widespread unspecified routing issues on the US east coast Master ticket 1086844 Anyone have more specific information
Date Mon 18 Oct 2004 122034 -0400 (EDT)Subject Re Level 3 US east coast issues
Level 3 is currently experiencing a backbone outage causing routing instability and packet loss We are working to restore and will be sending hourly updateshellip
72
Operator Mailing List
0102030405060708090
Filtering RouteLeaks
RouteHijacks
RouteInstability
RoutingLoops
Blackholes
Occurr
ences o
ver
10 Y
ears
1995-1997 1998-2001 2001-2004
Note Only includes problems openly discussed on this list
Compare 83 power outages 1 fire
73
Routing Configuration
Ranking route selection
Dissemination internal route advertisement
Filtering route advertisement
Customer
Competitor
Primary
Backup
74
Internet Business Model (Simplified)
bull CustomerProvider One AS pays another for reachability to some set of destinations
bull ldquoSettlement-freerdquo Peering Bartering Two ASes exchange routes with one another
Provider
Peer
Customer
Preferences implemented with local preference manipulation
Destination
Pay to use
Get paid to use
Free to use
75
Peering Contracts Consistent Export
bull Rules of settlement-free peeringndash Advertise routes at all peering pointsndash Advertised routes must have equal ldquoAS path lengthrdquo
Sprint
ATampT
Enables ldquohot potatordquo routing
ldquoequally goodrdquoroutes
76
Consistent Export
bull Malicedeceptionbull iBGP signaling partitionbull Inconsistent export policy
neighbor 10123route-map PEER permit 10 set prepend 123
neighbor 10456route-map PEER permit 10 set prepend 123 123
Possible Causes
10123 456 1
10456 456 2
Neighbor AS Export
1 1 123
Export Clause Prepend
2 1 123 123
Two different Export Policies
77
Inconsistent Export in Practice
Feamster et al ldquoBorderGuard Detecting Cold Potatoes from Peersrdquo ACM IMC October 2004
78
Blackholes
Date Thu 18 Jul 2002 060510 -0400 (EDT)From Chad Oleary ltcolpoboxcomgtSubject Re problems with 701To ltnanogmeritedugt
Were starting to see the same issues with UUNet again Anyone elseseeing this Trying to reach Qwest
traceroute to 631461901 (631461901) 30 hops max 38 byte packets 1 esc-lp2-gwe-solutionscorpcom (631182201) 1167 ms 1163 ms 1142 ms 2 500Serial2-10GW1TPA2ALTERNET (1571301499) 1097 ms 1059 ms 1044 ms 3 161at-1-0-0XL4ATL1ALTERNET (1526381190) 13839 ms 14108 ms 16638 ms 4 0so-3-1-0XL2ATL5ALTERNET (152630238) 14370 ms 14587 ms 14553 ms 5 POS7-0BR2ATL5ALTERNET (1526382193) 13928 ms 14099 ms 14053 ms 6 7 hellip
79
Security
80
Security ldquoBogonrdquo Routes
Feamster et al ldquoAn Empirical Study of lsquoBogonrsquo Route Advertisementsrdquo ACM CCR January 2005
81
Spam Phishing etc
bull Unsolicited commercial emailbull As of about August 2008 estimates indicate that
about 95 of all email is spambull Common spam filtering techniques
ndash Content-based filtersndash DNS Blacklist (DNSBL) lookups Significant fraction of
todayrsquos DNS traffic
Can IP addresses from which spam is received be spoofed
82
Spam and Routing
83
Worms and Botnets
84
What is a Worm
bull Code that replicates and propagates across the networkndash Often carries a ldquopayloadrdquo
bull Usually spread via exploiting flaws in open servicesndash ldquoVirusesrdquo require user action to spread
bull First worm Robert Morris November 1988ndash 6-10 of all Internet hosts infected ()
bull Many more since but none on that scale until July 2001
85
Example Worm Code Red
bull Initial version July 13 2001
bull Exploited known ISAPI vulnerability in Microsoft IIS Web servers
bull 1st through 20th of each month spread20th through end of each month attack
bull Payload Web site defacementbull Scanning Random IP addressesbull Bug failure to seed random number generator
86
Code Red Revisions
bull Released July 19 2001
bull Payload flooding attack on wwwwhitehousegovndash Attack was mounted at the IP address of the Web site
bull Bug died after 20th of each month
bull Random number generator for IP scanning fixed
87
Code Red Host Infection Rate
Exponential infection rate
Measured using backscatter technique
88
Designing Fast-Spreading Worms
bull Hit-list scanningndash Time to infect first 10k hosts dominates infection timendash Solution Reconnaissance (stealthy scans etc)
bull Permutation scanningndash Observation Most scanning is redundantndash Idea Shared permutation of address space Start scanning
from own IP address Re-randomize when another infected machine is found
bull Internet-scale hit listsndash Flash worm complete infection within 30 seconds
89
Botnets
bull Bots Autonomous programs performing tasksbull Plenty of ldquobenignrdquo bots
ndash eg weatherbug
bull Botnets group of bots ndash Typically carries malicious connotationndash Large numbers of infected machinesndash Machines ldquoenlistedrdquo with infection vectors like worms
(last lecture)
bull Available for simultaneous control by a masterbull Size up to 350000 nodes (from todayrsquos paper)
90
ldquoRallyingrdquo the Botnet
bull Easy to combine worm backdoor functionalitybull Problem how to learn about successfully
infected machines
bull Optionsndash Emailndash Hard-coded email address
91
Botnet Control
bull Botnet master typically runs some IRC server on a well-known port (eg 6667)
bull Infected machine contacts botnet with pre-programmed DNS name (eg big-botde)
bull Dynamic DNS allows controller to move about freely
Infected Machine
DynamicDNS
BotnetController
(IRC server)
92
Some Defenses
93
Idea 1 Ingress Filtering
bull RFC 2827 Routers install filters to drop packets from networks that are not downstream
bull Feasible at edgesbull Difficult to configure closer to network ldquocorerdquo
20469207024 Internet
Drop all packets with source address other than 20469207024
94
Idea 2 uRPF Checks
bull Unicast Reverse Path Forwardingndash Cisco ldquoip verify unicast reverse-pathrdquo
bull Requires symmetric routing
Accept packet from interface only if forwarding table entry for source IP address matches ingress interface
100183
A10018
10015
101203
100181 24
10011 24
Strict Mode uRPF
Enabled
ldquoArdquo Routing TableDestination Next Hop1001024 Int 110018024 Int 2
100183 from wrong interface
95
Problems with uRPF
bull Asymmetric routing
96
S-BGP
bull Address-based PKI validate signaturesndash Authentication of
bull ownership for IP address blocks bull AS number bull an ASs identity and bull a BGP routers identity
ndash Use existing infrastructure (Internet registries etc)ndash Routing origination is digitally signedndash BGP updates are digitally signed
1048708 bull Route attestations A new optional BGP transitive path attribute
ndash carries digital signatures covering the routing information in updates
97
Practical Problems with S-BGP
bull Requires Public-Key Infrastructure
bull Lots of digital signatures to calculate and verifyndash Message overheadndash CPU overhead
bull Calculation expense is greatest when topology is changingndash Caching can help
bull Route aggregation is problematic (maybe thatrsquos OK)
bull Secure route withdrawals when link or node fails
bull Address ownership data out of date
bull Deployment
- Networking
- Goal of This Tutorial
- Todayrsquos Networks
- Business Models
- Billing for Internet Usage
- Net Neutrality
- Network Operations
- Point-of-Presence (PoP)
- Example Abilene Network Topology
- Another Example Backbone
- Internet Routing Overview
- Internet Routing Protocol BGP
- Two Flavors of BGP
- IPv4 Addresses Networks of Networks
- Pre-1994 Classful Addressing
- Classless Interdomain Routing (CIDR)
- Benefits of CIDR
- Growth of IP Prefixes
- 1994-1998 Linear Growth
- Around 2000 Fast Growth Resumes
- Fast growth resumes
- The Address Allocation Process
- Common Problems
- What can go wrong
- Measurement and Monitoring
- Passive vs Active Measurement
- Slide 27
- Passive Traffic Data Measurement
- Simple Network Management Protocol
- SNMP (Passive)
- Packet-level Monitoring
- Full Packet Capture (Passive)
- What is a flow
- Cisco NetFlow
- Flow Record Contents
- Aggregating Packets into Flows
- Reducing Measurement Overhead
- Packet Sampling for Flow Monitoring
- Sampling Flow-Level Sampling
- Two Main Approaches
- Packet Capture on High-Speed Links
- Characteristics of Packet Capture
- Data Measurement Repositories
- Multihoming and Traffic Engineering
- What is Multihoming
- Why Multihome
- Redundancy
- Performance
- Multihoming in IP Networks Today
- BGP or no
- Same Provider or Multiple
- Multihomed Stub One Link
- Multihomed Stub Multiple Links
- Multihomed Stub Multiple ISPs
- Multihomed Transit Network
- Interdomain Traffic Engineering
- Outbound Traffic Control
- Outbound Traffic Load Balancing
- Traffic Engineering Goals
- Managing Scale
- Achieving Predictability
- Challenges to Predictability
- Inter-AS Negotiation
- Outbound Multihoming Goals
- Inbound Traffic Control
- How many links are enough
- Problems with Multihoming in IPv4
- Slide 68
- Slide 69
- Diagnosis and Troubleshooting
- Operator Mailing List (NANOG)
- Operator Mailing List
- Routing Configuration
- Internet Business Model (Simplified)
- Peering Contracts Consistent Export
- Consistent Export
- Inconsistent Export in Practice
- Blackholes
- Security
- Security ldquoBogonrdquo Routes
- Spam Phishing etc
- Spam and Routing
- Worms and Botnets
- What is a Worm
- Example Worm Code Red
- Code Red Revisions
- Code Red Host Infection Rate
- Designing Fast-Spreading Worms
- Botnets
- ldquoRallyingrdquo the Botnet
- Botnet Control
- Some Defenses
- Idea 1 Ingress Filtering
- Idea 2 uRPF Checks
- Problems with uRPF
- S-BGP
- Practical Problems with S-BGP
-
44
Multihoming and Traffic Engineering
45
What is Multihoming
bull The use of redundant network links for the purposes of external connectivity
bull Can be achieved at many layers of the protocol stack and many places in the networkndash Multiple network interfaces in a PC
ndash An ISP with multiple upstream interfaces
bull Can refer to having multiple connections tondash The same ISP
ndash Multiple ISPs
46
Why Multihome
bull Redundancybull Availabilitybull Performancebull Cost
Interdomain traffic engineering the process by which a multihomed network configures its
network to achieve these goals
47
Redundancy
bull Maintain connectivity in the face ofndash Physical connectivity problems (fiber cut device
failures etc)ndash Failures in upstream ISP
48
Performance
bull Use multiple network links at once to achieve higher throughput than just over a single link
bull Allows incoming traffic to be load-balanced
70 of traffic30 of traffic
49
Multihoming in IP Networks Today
bull Stub AS no transit service for other ASesndash No need to use BGP
bull Multi-homed stub AS has connectivity to multiple immediate upstream ISPsndash Need BGPndash No need for a public AS numberndash No need for IP prefix allocation
bull Multi-homed transit AS connectivity to multiple ASes and transit servicendash Need BGP public AS number IP prefix allocation
50
BGP or no
bull Advantages of static routingndash Cheapersmaller routers (less true nowadays)ndash Simpler to configure
bull Advantages of BGPndash More control of your destiny (have providers stop
announcing you)ndash Fastermore intelligent selection of where to send
outbound packetsndash Better debugging of net problems (you can see the
Internet topology now)
51
Same Provider or Multiple
bull If your provider is reliable and fast and affordably and offers good tech-support you may want to multi-home initially to them via some backup path (slow is better than dead)
bull Eventually yoursquoll want to multi-home to different providers to avoid failure modes due to one providerrsquos architecture decisions
52
Multihomed Stub One Link
bull Downstream ISPrsquos routers configure default (ldquostaticrdquo) routes pointing to border router
bull Upstream ISP advertises reachability
Upstream ISP
Multiple links between same pair of routers
Default routes to ldquoborderrdquo
ldquoStubrdquoISP
53
Multihomed Stub Multiple Links
bull Use BGP to share loadbull Use private AS number (why is this OK)bull As before upstream ISP advertises prefix
Upstream ISP
Multiple links to different upstream routers
ldquoStubrdquoISP
Internal routing for ldquohot potatordquo
BGP for load balance at edge
54
Multihomed Stub Multiple ISPs
bull Many possibilitiesndash Load sharingndash Primary-backupndash Selective use of different ISPs
bull Requires BGP public AS number etc
ldquoStubrdquoISP
Upstream
ISP 1
Upstream
ISP 2
55
Multihomed Transit Network
bull BGP everywherebull Incoming and outcoming trafficbull Challenge balancing load on intradomain and egress
links given an offered traffic load
TransitISP
ISP 1
ISP 2
ISP 3
56
Interdomain Traffic Engineering
bull The process by which a network operator configures the network to achievendash Traffic load balancendash Redundancy (primarybackup) etc
bull Two tasksndash Outbound traffic controlndash Inbound traffic control
bull Key Problems Predictability and Scalability
57
Outbound Traffic Control
bull Easier to control than inbound trafficndash Destination-based routing sender determines where
the packets go
bull Control over next-hop AS onlyndash Cannot control selection of the entire path
Provider 1 Provider 2
Control with local preference
58
Outbound Traffic Load Balancingbull Control routes to provider per-prefix
ndash Assign local preference across destination prefixesndash Change the local preference assignments over time
bull Useful inputs to load balancingndash End-to-end path performance datandash Outbound traffic statistics per destination prefix
bull Challenge Getting from traffic volumes to groups of prefixes that should be assigned to each link
Premise of ldquointelligent route controlrdquo preoducts
59
Traffic Engineering Goals
bull Predictabilityndash Ensure the BGP decision process is deterministicndash Assume that BGP updates are (relatively) stable
bull Limit overhead introduced by routing changesndash Minimize frequency of changes to routing policiesndash Limit number of prefixes affected by changes
bull Limit impact on how traffic enters the networkndash Avoid new routes that might change neighborrsquos mindndash Select route with same attributes or at least path length
60
Managing Scalebull Destination prefixes
ndash More than 90000 destination prefixes
bull Donrsquot want to have per-prefix routing policies
ndash Small fraction of prefixes contribute most of the traffic
bull Focus on the small number of heavy hitters
ndash Define routing policies for selected prefixes
bull Routing choicesndash About 27000 unique ldquorouting choicesrdquo
bull Help in reducing the scale of the problem
ndash Small fraction of ldquorouting choicesrdquo contribute most traffic
bull Focus on the very small number of ldquorouting choicesrdquo
ndash Define routing policies on common attributes
61
Achieving Predictability
bull Route prediction with static analysisndash Helpful to know effects before deploymentndash Static analysis can help
TopologyBGP policy
configuration
eBGP routes
Offered traffic
BGP routingmodel
Flow of traffic through the network
62
Challenges to Predictabilitybull For transit ISPs effects on incoming traffic
ndash Lack of coordination strikes again
63
ldquoHot Potatordquo routing
Inter-AS Negotiation
bull Coordination aids predictabilityndash Negotiate where to sendndash Inbound and outboundndash Mutual benefits
bull How to implementndash What info to exchangendash Protecting privacyndash How to prioritize choicesndash How to prevent cheating
Destination 2
Destination 1
multiplepeeringpoints
Provider A
Provider B
64
Outbound Multihoming Goals
bull Redundancyndash Dynamic routing will failover to backup link
bull Performancendash Select provider with best performance per prefix
ndash Requires active probing
bull Costndash Select provider per prefix over time to minimize the
total financial cost
65
Inbound Traffic Control
bull More difficult no control over neighborsrsquo decisions
bull Three common techniques (previously discussed)ndash AS path prependingndash Communities and local preferencendash Prefix splitting
How does todayrsquos paper (MONET) control inbound traffic
66
How many links are enough
K upstream ISPs
Not much benefit beyond 4 ISPs
Akella et al ldquoPerformance Benefits of Multihomingrdquo SIGCOMM 2003
67
Problems with Multihoming in IPv4
bull Routing table growthndash Provider-based addressingndash Advertising prefix out multiple ISPs ndash canrsquot aggregate
bull Poor control over inbound trafficndash Existing mechanisms do not allow hosts to control
inbound traffic
68
GeorgiaTech
Internet Routing Overview
bull Intradomain (ie ldquointra-ASrdquo) routingbull Interdomain routing
Comcast
Abilene
ATampT Cogent
Autonomous Systems (ASes)
69
Configuration Problems ldquoAS 7007rdquoldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
UUNet
Florida InternetBarn
Sprint
70
Diagnosis and Troubleshootingldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
ldquoMicrosofts websites were offline for up to 23 hoursbecause of a [router] misconfigurationhellipit took nearly a day to determine what was wrong and undo the changesrdquo -- wiredcom January 25 2001
ldquoWorldCom Inchellipsuffered a widespread outage on its Internet backbone that affected roughly 20 percent of its US customer base The network problemshellipaffected millions of computer users worldwide A spokeswoman attributed the outage to a route table issue -- cnncom October 3 2002
A number of Covad customers went out from 5pm today due to supposedly a DDOS (distributed denial of service attack) on a key Level3 data center which later was described as a route leak (misconfiguration)ldquo
-- dslreportscom February 23 2004
71
Operator Mailing List (NANOG)Date Mon 18 Oct 2004 091515 -0700Subject Level 3 US east coast issues
Level 3 experiencing widespread unspecified routing issues on the US east coast Master ticket 1086844 Anyone have more specific information
Date Mon 18 Oct 2004 122034 -0400 (EDT)Subject Re Level 3 US east coast issues
Level 3 is currently experiencing a backbone outage causing routing instability and packet loss We are working to restore and will be sending hourly updateshellip
72
Operator Mailing List
0102030405060708090
Filtering RouteLeaks
RouteHijacks
RouteInstability
RoutingLoops
Blackholes
Occurr
ences o
ver
10 Y
ears
1995-1997 1998-2001 2001-2004
Note Only includes problems openly discussed on this list
Compare 83 power outages 1 fire
73
Routing Configuration
Ranking route selection
Dissemination internal route advertisement
Filtering route advertisement
Customer
Competitor
Primary
Backup
74
Internet Business Model (Simplified)
bull CustomerProvider One AS pays another for reachability to some set of destinations
bull ldquoSettlement-freerdquo Peering Bartering Two ASes exchange routes with one another
Provider
Peer
Customer
Preferences implemented with local preference manipulation
Destination
Pay to use
Get paid to use
Free to use
75
Peering Contracts Consistent Export
bull Rules of settlement-free peeringndash Advertise routes at all peering pointsndash Advertised routes must have equal ldquoAS path lengthrdquo
Sprint
ATampT
Enables ldquohot potatordquo routing
ldquoequally goodrdquoroutes
76
Consistent Export
bull Malicedeceptionbull iBGP signaling partitionbull Inconsistent export policy
neighbor 10123route-map PEER permit 10 set prepend 123
neighbor 10456route-map PEER permit 10 set prepend 123 123
Possible Causes
10123 456 1
10456 456 2
Neighbor AS Export
1 1 123
Export Clause Prepend
2 1 123 123
Two different Export Policies
77
Inconsistent Export in Practice
Feamster et al ldquoBorderGuard Detecting Cold Potatoes from Peersrdquo ACM IMC October 2004
78
Blackholes
Date Thu 18 Jul 2002 060510 -0400 (EDT)From Chad Oleary ltcolpoboxcomgtSubject Re problems with 701To ltnanogmeritedugt
Were starting to see the same issues with UUNet again Anyone elseseeing this Trying to reach Qwest
traceroute to 631461901 (631461901) 30 hops max 38 byte packets 1 esc-lp2-gwe-solutionscorpcom (631182201) 1167 ms 1163 ms 1142 ms 2 500Serial2-10GW1TPA2ALTERNET (1571301499) 1097 ms 1059 ms 1044 ms 3 161at-1-0-0XL4ATL1ALTERNET (1526381190) 13839 ms 14108 ms 16638 ms 4 0so-3-1-0XL2ATL5ALTERNET (152630238) 14370 ms 14587 ms 14553 ms 5 POS7-0BR2ATL5ALTERNET (1526382193) 13928 ms 14099 ms 14053 ms 6 7 hellip
79
Security
80
Security ldquoBogonrdquo Routes
Feamster et al ldquoAn Empirical Study of lsquoBogonrsquo Route Advertisementsrdquo ACM CCR January 2005
81
Spam Phishing etc
bull Unsolicited commercial emailbull As of about August 2008 estimates indicate that
about 95 of all email is spambull Common spam filtering techniques
ndash Content-based filtersndash DNS Blacklist (DNSBL) lookups Significant fraction of
todayrsquos DNS traffic
Can IP addresses from which spam is received be spoofed
82
Spam and Routing
83
Worms and Botnets
84
What is a Worm
bull Code that replicates and propagates across the networkndash Often carries a ldquopayloadrdquo
bull Usually spread via exploiting flaws in open servicesndash ldquoVirusesrdquo require user action to spread
bull First worm Robert Morris November 1988ndash 6-10 of all Internet hosts infected ()
bull Many more since but none on that scale until July 2001
85
Example Worm Code Red
bull Initial version July 13 2001
bull Exploited known ISAPI vulnerability in Microsoft IIS Web servers
bull 1st through 20th of each month spread20th through end of each month attack
bull Payload Web site defacementbull Scanning Random IP addressesbull Bug failure to seed random number generator
86
Code Red Revisions
bull Released July 19 2001
bull Payload flooding attack on wwwwhitehousegovndash Attack was mounted at the IP address of the Web site
bull Bug died after 20th of each month
bull Random number generator for IP scanning fixed
87
Code Red Host Infection Rate
Exponential infection rate
Measured using backscatter technique
88
Designing Fast-Spreading Worms
bull Hit-list scanningndash Time to infect first 10k hosts dominates infection timendash Solution Reconnaissance (stealthy scans etc)
bull Permutation scanningndash Observation Most scanning is redundantndash Idea Shared permutation of address space Start scanning
from own IP address Re-randomize when another infected machine is found
bull Internet-scale hit listsndash Flash worm complete infection within 30 seconds
89
Botnets
bull Bots Autonomous programs performing tasksbull Plenty of ldquobenignrdquo bots
ndash eg weatherbug
bull Botnets group of bots ndash Typically carries malicious connotationndash Large numbers of infected machinesndash Machines ldquoenlistedrdquo with infection vectors like worms
(last lecture)
bull Available for simultaneous control by a masterbull Size up to 350000 nodes (from todayrsquos paper)
90
ldquoRallyingrdquo the Botnet
bull Easy to combine worm backdoor functionalitybull Problem how to learn about successfully
infected machines
bull Optionsndash Emailndash Hard-coded email address
91
Botnet Control
bull Botnet master typically runs some IRC server on a well-known port (eg 6667)
bull Infected machine contacts botnet with pre-programmed DNS name (eg big-botde)
bull Dynamic DNS allows controller to move about freely
Infected Machine
DynamicDNS
BotnetController
(IRC server)
92
Some Defenses
93
Idea 1 Ingress Filtering
bull RFC 2827 Routers install filters to drop packets from networks that are not downstream
bull Feasible at edgesbull Difficult to configure closer to network ldquocorerdquo
20469207024 Internet
Drop all packets with source address other than 20469207024
94
Idea 2 uRPF Checks
bull Unicast Reverse Path Forwardingndash Cisco ldquoip verify unicast reverse-pathrdquo
bull Requires symmetric routing
Accept packet from interface only if forwarding table entry for source IP address matches ingress interface
100183
A10018
10015
101203
100181 24
10011 24
Strict Mode uRPF
Enabled
ldquoArdquo Routing TableDestination Next Hop1001024 Int 110018024 Int 2
100183 from wrong interface
95
Problems with uRPF
bull Asymmetric routing
96
S-BGP
bull Address-based PKI validate signaturesndash Authentication of
bull ownership for IP address blocks bull AS number bull an ASs identity and bull a BGP routers identity
ndash Use existing infrastructure (Internet registries etc)ndash Routing origination is digitally signedndash BGP updates are digitally signed
1048708 bull Route attestations A new optional BGP transitive path attribute
ndash carries digital signatures covering the routing information in updates
97
Practical Problems with S-BGP
bull Requires Public-Key Infrastructure
bull Lots of digital signatures to calculate and verifyndash Message overheadndash CPU overhead
bull Calculation expense is greatest when topology is changingndash Caching can help
bull Route aggregation is problematic (maybe thatrsquos OK)
bull Secure route withdrawals when link or node fails
bull Address ownership data out of date
bull Deployment
- Networking
- Goal of This Tutorial
- Todayrsquos Networks
- Business Models
- Billing for Internet Usage
- Net Neutrality
- Network Operations
- Point-of-Presence (PoP)
- Example Abilene Network Topology
- Another Example Backbone
- Internet Routing Overview
- Internet Routing Protocol BGP
- Two Flavors of BGP
- IPv4 Addresses Networks of Networks
- Pre-1994 Classful Addressing
- Classless Interdomain Routing (CIDR)
- Benefits of CIDR
- Growth of IP Prefixes
- 1994-1998 Linear Growth
- Around 2000 Fast Growth Resumes
- Fast growth resumes
- The Address Allocation Process
- Common Problems
- What can go wrong
- Measurement and Monitoring
- Passive vs Active Measurement
- Slide 27
- Passive Traffic Data Measurement
- Simple Network Management Protocol
- SNMP (Passive)
- Packet-level Monitoring
- Full Packet Capture (Passive)
- What is a flow
- Cisco NetFlow
- Flow Record Contents
- Aggregating Packets into Flows
- Reducing Measurement Overhead
- Packet Sampling for Flow Monitoring
- Sampling Flow-Level Sampling
- Two Main Approaches
- Packet Capture on High-Speed Links
- Characteristics of Packet Capture
- Data Measurement Repositories
- Multihoming and Traffic Engineering
- What is Multihoming
- Why Multihome
- Redundancy
- Performance
- Multihoming in IP Networks Today
- BGP or no
- Same Provider or Multiple
- Multihomed Stub One Link
- Multihomed Stub Multiple Links
- Multihomed Stub Multiple ISPs
- Multihomed Transit Network
- Interdomain Traffic Engineering
- Outbound Traffic Control
- Outbound Traffic Load Balancing
- Traffic Engineering Goals
- Managing Scale
- Achieving Predictability
- Challenges to Predictability
- Inter-AS Negotiation
- Outbound Multihoming Goals
- Inbound Traffic Control
- How many links are enough
- Problems with Multihoming in IPv4
- Slide 68
- Slide 69
- Diagnosis and Troubleshooting
- Operator Mailing List (NANOG)
- Operator Mailing List
- Routing Configuration
- Internet Business Model (Simplified)
- Peering Contracts Consistent Export
- Consistent Export
- Inconsistent Export in Practice
- Blackholes
- Security
- Security ldquoBogonrdquo Routes
- Spam Phishing etc
- Spam and Routing
- Worms and Botnets
- What is a Worm
- Example Worm Code Red
- Code Red Revisions
- Code Red Host Infection Rate
- Designing Fast-Spreading Worms
- Botnets
- ldquoRallyingrdquo the Botnet
- Botnet Control
- Some Defenses
- Idea 1 Ingress Filtering
- Idea 2 uRPF Checks
- Problems with uRPF
- S-BGP
- Practical Problems with S-BGP
-
45
What is Multihoming
bull The use of redundant network links for the purposes of external connectivity
bull Can be achieved at many layers of the protocol stack and many places in the networkndash Multiple network interfaces in a PC
ndash An ISP with multiple upstream interfaces
bull Can refer to having multiple connections tondash The same ISP
ndash Multiple ISPs
46
Why Multihome
bull Redundancybull Availabilitybull Performancebull Cost
Interdomain traffic engineering the process by which a multihomed network configures its
network to achieve these goals
47
Redundancy
bull Maintain connectivity in the face ofndash Physical connectivity problems (fiber cut device
failures etc)ndash Failures in upstream ISP
48
Performance
bull Use multiple network links at once to achieve higher throughput than just over a single link
bull Allows incoming traffic to be load-balanced
70 of traffic30 of traffic
49
Multihoming in IP Networks Today
bull Stub AS no transit service for other ASesndash No need to use BGP
bull Multi-homed stub AS has connectivity to multiple immediate upstream ISPsndash Need BGPndash No need for a public AS numberndash No need for IP prefix allocation
bull Multi-homed transit AS connectivity to multiple ASes and transit servicendash Need BGP public AS number IP prefix allocation
50
BGP or no
bull Advantages of static routingndash Cheapersmaller routers (less true nowadays)ndash Simpler to configure
bull Advantages of BGPndash More control of your destiny (have providers stop
announcing you)ndash Fastermore intelligent selection of where to send
outbound packetsndash Better debugging of net problems (you can see the
Internet topology now)
51
Same Provider or Multiple
bull If your provider is reliable and fast and affordably and offers good tech-support you may want to multi-home initially to them via some backup path (slow is better than dead)
bull Eventually yoursquoll want to multi-home to different providers to avoid failure modes due to one providerrsquos architecture decisions
52
Multihomed Stub One Link
bull Downstream ISPrsquos routers configure default (ldquostaticrdquo) routes pointing to border router
bull Upstream ISP advertises reachability
Upstream ISP
Multiple links between same pair of routers
Default routes to ldquoborderrdquo
ldquoStubrdquoISP
53
Multihomed Stub Multiple Links
bull Use BGP to share loadbull Use private AS number (why is this OK)bull As before upstream ISP advertises prefix
Upstream ISP
Multiple links to different upstream routers
ldquoStubrdquoISP
Internal routing for ldquohot potatordquo
BGP for load balance at edge
54
Multihomed Stub Multiple ISPs
bull Many possibilitiesndash Load sharingndash Primary-backupndash Selective use of different ISPs
bull Requires BGP public AS number etc
ldquoStubrdquoISP
Upstream
ISP 1
Upstream
ISP 2
55
Multihomed Transit Network
bull BGP everywherebull Incoming and outcoming trafficbull Challenge balancing load on intradomain and egress
links given an offered traffic load
TransitISP
ISP 1
ISP 2
ISP 3
56
Interdomain Traffic Engineering
bull The process by which a network operator configures the network to achievendash Traffic load balancendash Redundancy (primarybackup) etc
bull Two tasksndash Outbound traffic controlndash Inbound traffic control
bull Key Problems Predictability and Scalability
57
Outbound Traffic Control
bull Easier to control than inbound trafficndash Destination-based routing sender determines where
the packets go
bull Control over next-hop AS onlyndash Cannot control selection of the entire path
Provider 1 Provider 2
Control with local preference
58
Outbound Traffic Load Balancingbull Control routes to provider per-prefix
ndash Assign local preference across destination prefixesndash Change the local preference assignments over time
bull Useful inputs to load balancingndash End-to-end path performance datandash Outbound traffic statistics per destination prefix
bull Challenge Getting from traffic volumes to groups of prefixes that should be assigned to each link
Premise of ldquointelligent route controlrdquo preoducts
59
Traffic Engineering Goals
bull Predictabilityndash Ensure the BGP decision process is deterministicndash Assume that BGP updates are (relatively) stable
bull Limit overhead introduced by routing changesndash Minimize frequency of changes to routing policiesndash Limit number of prefixes affected by changes
bull Limit impact on how traffic enters the networkndash Avoid new routes that might change neighborrsquos mindndash Select route with same attributes or at least path length
60
Managing Scalebull Destination prefixes
ndash More than 90000 destination prefixes
bull Donrsquot want to have per-prefix routing policies
ndash Small fraction of prefixes contribute most of the traffic
bull Focus on the small number of heavy hitters
ndash Define routing policies for selected prefixes
bull Routing choicesndash About 27000 unique ldquorouting choicesrdquo
bull Help in reducing the scale of the problem
ndash Small fraction of ldquorouting choicesrdquo contribute most traffic
bull Focus on the very small number of ldquorouting choicesrdquo
ndash Define routing policies on common attributes
61
Achieving Predictability
bull Route prediction with static analysisndash Helpful to know effects before deploymentndash Static analysis can help
TopologyBGP policy
configuration
eBGP routes
Offered traffic
BGP routingmodel
Flow of traffic through the network
62
Challenges to Predictabilitybull For transit ISPs effects on incoming traffic
ndash Lack of coordination strikes again
63
ldquoHot Potatordquo routing
Inter-AS Negotiation
bull Coordination aids predictabilityndash Negotiate where to sendndash Inbound and outboundndash Mutual benefits
bull How to implementndash What info to exchangendash Protecting privacyndash How to prioritize choicesndash How to prevent cheating
Destination 2
Destination 1
multiplepeeringpoints
Provider A
Provider B
64
Outbound Multihoming Goals
bull Redundancyndash Dynamic routing will failover to backup link
bull Performancendash Select provider with best performance per prefix
ndash Requires active probing
bull Costndash Select provider per prefix over time to minimize the
total financial cost
65
Inbound Traffic Control
bull More difficult no control over neighborsrsquo decisions
bull Three common techniques (previously discussed)ndash AS path prependingndash Communities and local preferencendash Prefix splitting
How does todayrsquos paper (MONET) control inbound traffic
66
How many links are enough
K upstream ISPs
Not much benefit beyond 4 ISPs
Akella et al ldquoPerformance Benefits of Multihomingrdquo SIGCOMM 2003
67
Problems with Multihoming in IPv4
bull Routing table growthndash Provider-based addressingndash Advertising prefix out multiple ISPs ndash canrsquot aggregate
bull Poor control over inbound trafficndash Existing mechanisms do not allow hosts to control
inbound traffic
68
GeorgiaTech
Internet Routing Overview
bull Intradomain (ie ldquointra-ASrdquo) routingbull Interdomain routing
Comcast
Abilene
ATampT Cogent
Autonomous Systems (ASes)
69
Configuration Problems ldquoAS 7007rdquoldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
UUNet
Florida InternetBarn
Sprint
70
Diagnosis and Troubleshootingldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
ldquoMicrosofts websites were offline for up to 23 hoursbecause of a [router] misconfigurationhellipit took nearly a day to determine what was wrong and undo the changesrdquo -- wiredcom January 25 2001
ldquoWorldCom Inchellipsuffered a widespread outage on its Internet backbone that affected roughly 20 percent of its US customer base The network problemshellipaffected millions of computer users worldwide A spokeswoman attributed the outage to a route table issue -- cnncom October 3 2002
A number of Covad customers went out from 5pm today due to supposedly a DDOS (distributed denial of service attack) on a key Level3 data center which later was described as a route leak (misconfiguration)ldquo
-- dslreportscom February 23 2004
71
Operator Mailing List (NANOG)Date Mon 18 Oct 2004 091515 -0700Subject Level 3 US east coast issues
Level 3 experiencing widespread unspecified routing issues on the US east coast Master ticket 1086844 Anyone have more specific information
Date Mon 18 Oct 2004 122034 -0400 (EDT)Subject Re Level 3 US east coast issues
Level 3 is currently experiencing a backbone outage causing routing instability and packet loss We are working to restore and will be sending hourly updateshellip
72
Operator Mailing List
0102030405060708090
Filtering RouteLeaks
RouteHijacks
RouteInstability
RoutingLoops
Blackholes
Occurr
ences o
ver
10 Y
ears
1995-1997 1998-2001 2001-2004
Note Only includes problems openly discussed on this list
Compare 83 power outages 1 fire
73
Routing Configuration
Ranking route selection
Dissemination internal route advertisement
Filtering route advertisement
Customer
Competitor
Primary
Backup
74
Internet Business Model (Simplified)
bull CustomerProvider One AS pays another for reachability to some set of destinations
bull ldquoSettlement-freerdquo Peering Bartering Two ASes exchange routes with one another
Provider
Peer
Customer
Preferences implemented with local preference manipulation
Destination
Pay to use
Get paid to use
Free to use
75
Peering Contracts Consistent Export
bull Rules of settlement-free peeringndash Advertise routes at all peering pointsndash Advertised routes must have equal ldquoAS path lengthrdquo
Sprint
ATampT
Enables ldquohot potatordquo routing
ldquoequally goodrdquoroutes
76
Consistent Export
bull Malicedeceptionbull iBGP signaling partitionbull Inconsistent export policy
neighbor 10123route-map PEER permit 10 set prepend 123
neighbor 10456route-map PEER permit 10 set prepend 123 123
Possible Causes
10123 456 1
10456 456 2
Neighbor AS Export
1 1 123
Export Clause Prepend
2 1 123 123
Two different Export Policies
77
Inconsistent Export in Practice
Feamster et al ldquoBorderGuard Detecting Cold Potatoes from Peersrdquo ACM IMC October 2004
78
Blackholes
Date Thu 18 Jul 2002 060510 -0400 (EDT)From Chad Oleary ltcolpoboxcomgtSubject Re problems with 701To ltnanogmeritedugt
Were starting to see the same issues with UUNet again Anyone elseseeing this Trying to reach Qwest
traceroute to 631461901 (631461901) 30 hops max 38 byte packets 1 esc-lp2-gwe-solutionscorpcom (631182201) 1167 ms 1163 ms 1142 ms 2 500Serial2-10GW1TPA2ALTERNET (1571301499) 1097 ms 1059 ms 1044 ms 3 161at-1-0-0XL4ATL1ALTERNET (1526381190) 13839 ms 14108 ms 16638 ms 4 0so-3-1-0XL2ATL5ALTERNET (152630238) 14370 ms 14587 ms 14553 ms 5 POS7-0BR2ATL5ALTERNET (1526382193) 13928 ms 14099 ms 14053 ms 6 7 hellip
79
Security
80
Security ldquoBogonrdquo Routes
Feamster et al ldquoAn Empirical Study of lsquoBogonrsquo Route Advertisementsrdquo ACM CCR January 2005
81
Spam Phishing etc
bull Unsolicited commercial emailbull As of about August 2008 estimates indicate that
about 95 of all email is spambull Common spam filtering techniques
ndash Content-based filtersndash DNS Blacklist (DNSBL) lookups Significant fraction of
todayrsquos DNS traffic
Can IP addresses from which spam is received be spoofed
82
Spam and Routing
83
Worms and Botnets
84
What is a Worm
bull Code that replicates and propagates across the networkndash Often carries a ldquopayloadrdquo
bull Usually spread via exploiting flaws in open servicesndash ldquoVirusesrdquo require user action to spread
bull First worm Robert Morris November 1988ndash 6-10 of all Internet hosts infected ()
bull Many more since but none on that scale until July 2001
85
Example Worm Code Red
bull Initial version July 13 2001
bull Exploited known ISAPI vulnerability in Microsoft IIS Web servers
bull 1st through 20th of each month spread20th through end of each month attack
bull Payload Web site defacementbull Scanning Random IP addressesbull Bug failure to seed random number generator
86
Code Red Revisions
bull Released July 19 2001
bull Payload flooding attack on wwwwhitehousegovndash Attack was mounted at the IP address of the Web site
bull Bug died after 20th of each month
bull Random number generator for IP scanning fixed
87
Code Red Host Infection Rate
Exponential infection rate
Measured using backscatter technique
88
Designing Fast-Spreading Worms
bull Hit-list scanningndash Time to infect first 10k hosts dominates infection timendash Solution Reconnaissance (stealthy scans etc)
bull Permutation scanningndash Observation Most scanning is redundantndash Idea Shared permutation of address space Start scanning
from own IP address Re-randomize when another infected machine is found
bull Internet-scale hit listsndash Flash worm complete infection within 30 seconds
89
Botnets
bull Bots Autonomous programs performing tasksbull Plenty of ldquobenignrdquo bots
ndash eg weatherbug
bull Botnets group of bots ndash Typically carries malicious connotationndash Large numbers of infected machinesndash Machines ldquoenlistedrdquo with infection vectors like worms
(last lecture)
bull Available for simultaneous control by a masterbull Size up to 350000 nodes (from todayrsquos paper)
90
ldquoRallyingrdquo the Botnet
bull Easy to combine worm backdoor functionalitybull Problem how to learn about successfully
infected machines
bull Optionsndash Emailndash Hard-coded email address
91
Botnet Control
bull Botnet master typically runs some IRC server on a well-known port (eg 6667)
bull Infected machine contacts botnet with pre-programmed DNS name (eg big-botde)
bull Dynamic DNS allows controller to move about freely
Infected Machine
DynamicDNS
BotnetController
(IRC server)
92
Some Defenses
93
Idea 1 Ingress Filtering
bull RFC 2827 Routers install filters to drop packets from networks that are not downstream
bull Feasible at edgesbull Difficult to configure closer to network ldquocorerdquo
20469207024 Internet
Drop all packets with source address other than 20469207024
94
Idea 2 uRPF Checks
bull Unicast Reverse Path Forwardingndash Cisco ldquoip verify unicast reverse-pathrdquo
bull Requires symmetric routing
Accept packet from interface only if forwarding table entry for source IP address matches ingress interface
100183
A10018
10015
101203
100181 24
10011 24
Strict Mode uRPF
Enabled
ldquoArdquo Routing TableDestination Next Hop1001024 Int 110018024 Int 2
100183 from wrong interface
95
Problems with uRPF
bull Asymmetric routing
96
S-BGP
bull Address-based PKI validate signaturesndash Authentication of
bull ownership for IP address blocks bull AS number bull an ASs identity and bull a BGP routers identity
ndash Use existing infrastructure (Internet registries etc)ndash Routing origination is digitally signedndash BGP updates are digitally signed
1048708 bull Route attestations A new optional BGP transitive path attribute
ndash carries digital signatures covering the routing information in updates
97
Practical Problems with S-BGP
bull Requires Public-Key Infrastructure
bull Lots of digital signatures to calculate and verifyndash Message overheadndash CPU overhead
bull Calculation expense is greatest when topology is changingndash Caching can help
bull Route aggregation is problematic (maybe thatrsquos OK)
bull Secure route withdrawals when link or node fails
bull Address ownership data out of date
bull Deployment
- Networking
- Goal of This Tutorial
- Todayrsquos Networks
- Business Models
- Billing for Internet Usage
- Net Neutrality
- Network Operations
- Point-of-Presence (PoP)
- Example Abilene Network Topology
- Another Example Backbone
- Internet Routing Overview
- Internet Routing Protocol BGP
- Two Flavors of BGP
- IPv4 Addresses Networks of Networks
- Pre-1994 Classful Addressing
- Classless Interdomain Routing (CIDR)
- Benefits of CIDR
- Growth of IP Prefixes
- 1994-1998 Linear Growth
- Around 2000 Fast Growth Resumes
- Fast growth resumes
- The Address Allocation Process
- Common Problems
- What can go wrong
- Measurement and Monitoring
- Passive vs Active Measurement
- Slide 27
- Passive Traffic Data Measurement
- Simple Network Management Protocol
- SNMP (Passive)
- Packet-level Monitoring
- Full Packet Capture (Passive)
- What is a flow
- Cisco NetFlow
- Flow Record Contents
- Aggregating Packets into Flows
- Reducing Measurement Overhead
- Packet Sampling for Flow Monitoring
- Sampling Flow-Level Sampling
- Two Main Approaches
- Packet Capture on High-Speed Links
- Characteristics of Packet Capture
- Data Measurement Repositories
- Multihoming and Traffic Engineering
- What is Multihoming
- Why Multihome
- Redundancy
- Performance
- Multihoming in IP Networks Today
- BGP or no
- Same Provider or Multiple
- Multihomed Stub One Link
- Multihomed Stub Multiple Links
- Multihomed Stub Multiple ISPs
- Multihomed Transit Network
- Interdomain Traffic Engineering
- Outbound Traffic Control
- Outbound Traffic Load Balancing
- Traffic Engineering Goals
- Managing Scale
- Achieving Predictability
- Challenges to Predictability
- Inter-AS Negotiation
- Outbound Multihoming Goals
- Inbound Traffic Control
- How many links are enough
- Problems with Multihoming in IPv4
- Slide 68
- Slide 69
- Diagnosis and Troubleshooting
- Operator Mailing List (NANOG)
- Operator Mailing List
- Routing Configuration
- Internet Business Model (Simplified)
- Peering Contracts Consistent Export
- Consistent Export
- Inconsistent Export in Practice
- Blackholes
- Security
- Security ldquoBogonrdquo Routes
- Spam Phishing etc
- Spam and Routing
- Worms and Botnets
- What is a Worm
- Example Worm Code Red
- Code Red Revisions
- Code Red Host Infection Rate
- Designing Fast-Spreading Worms
- Botnets
- ldquoRallyingrdquo the Botnet
- Botnet Control
- Some Defenses
- Idea 1 Ingress Filtering
- Idea 2 uRPF Checks
- Problems with uRPF
- S-BGP
- Practical Problems with S-BGP
-
46
Why Multihome
bull Redundancybull Availabilitybull Performancebull Cost
Interdomain traffic engineering the process by which a multihomed network configures its
network to achieve these goals
47
Redundancy
bull Maintain connectivity in the face ofndash Physical connectivity problems (fiber cut device
failures etc)ndash Failures in upstream ISP
48
Performance
bull Use multiple network links at once to achieve higher throughput than just over a single link
bull Allows incoming traffic to be load-balanced
70 of traffic30 of traffic
49
Multihoming in IP Networks Today
bull Stub AS no transit service for other ASesndash No need to use BGP
bull Multi-homed stub AS has connectivity to multiple immediate upstream ISPsndash Need BGPndash No need for a public AS numberndash No need for IP prefix allocation
bull Multi-homed transit AS connectivity to multiple ASes and transit servicendash Need BGP public AS number IP prefix allocation
50
BGP or no
bull Advantages of static routingndash Cheapersmaller routers (less true nowadays)ndash Simpler to configure
bull Advantages of BGPndash More control of your destiny (have providers stop
announcing you)ndash Fastermore intelligent selection of where to send
outbound packetsndash Better debugging of net problems (you can see the
Internet topology now)
51
Same Provider or Multiple
bull If your provider is reliable and fast and affordably and offers good tech-support you may want to multi-home initially to them via some backup path (slow is better than dead)
bull Eventually yoursquoll want to multi-home to different providers to avoid failure modes due to one providerrsquos architecture decisions
52
Multihomed Stub One Link
bull Downstream ISPrsquos routers configure default (ldquostaticrdquo) routes pointing to border router
bull Upstream ISP advertises reachability
Upstream ISP
Multiple links between same pair of routers
Default routes to ldquoborderrdquo
ldquoStubrdquoISP
53
Multihomed Stub Multiple Links
bull Use BGP to share loadbull Use private AS number (why is this OK)bull As before upstream ISP advertises prefix
Upstream ISP
Multiple links to different upstream routers
ldquoStubrdquoISP
Internal routing for ldquohot potatordquo
BGP for load balance at edge
54
Multihomed Stub Multiple ISPs
bull Many possibilitiesndash Load sharingndash Primary-backupndash Selective use of different ISPs
bull Requires BGP public AS number etc
ldquoStubrdquoISP
Upstream
ISP 1
Upstream
ISP 2
55
Multihomed Transit Network
bull BGP everywherebull Incoming and outcoming trafficbull Challenge balancing load on intradomain and egress
links given an offered traffic load
TransitISP
ISP 1
ISP 2
ISP 3
56
Interdomain Traffic Engineering
bull The process by which a network operator configures the network to achievendash Traffic load balancendash Redundancy (primarybackup) etc
bull Two tasksndash Outbound traffic controlndash Inbound traffic control
bull Key Problems Predictability and Scalability
57
Outbound Traffic Control
bull Easier to control than inbound trafficndash Destination-based routing sender determines where
the packets go
bull Control over next-hop AS onlyndash Cannot control selection of the entire path
Provider 1 Provider 2
Control with local preference
58
Outbound Traffic Load Balancingbull Control routes to provider per-prefix
ndash Assign local preference across destination prefixesndash Change the local preference assignments over time
bull Useful inputs to load balancingndash End-to-end path performance datandash Outbound traffic statistics per destination prefix
bull Challenge Getting from traffic volumes to groups of prefixes that should be assigned to each link
Premise of ldquointelligent route controlrdquo preoducts
59
Traffic Engineering Goals
bull Predictabilityndash Ensure the BGP decision process is deterministicndash Assume that BGP updates are (relatively) stable
bull Limit overhead introduced by routing changesndash Minimize frequency of changes to routing policiesndash Limit number of prefixes affected by changes
bull Limit impact on how traffic enters the networkndash Avoid new routes that might change neighborrsquos mindndash Select route with same attributes or at least path length
60
Managing Scalebull Destination prefixes
ndash More than 90000 destination prefixes
bull Donrsquot want to have per-prefix routing policies
ndash Small fraction of prefixes contribute most of the traffic
bull Focus on the small number of heavy hitters
ndash Define routing policies for selected prefixes
bull Routing choicesndash About 27000 unique ldquorouting choicesrdquo
bull Help in reducing the scale of the problem
ndash Small fraction of ldquorouting choicesrdquo contribute most traffic
bull Focus on the very small number of ldquorouting choicesrdquo
ndash Define routing policies on common attributes
61
Achieving Predictability
bull Route prediction with static analysisndash Helpful to know effects before deploymentndash Static analysis can help
TopologyBGP policy
configuration
eBGP routes
Offered traffic
BGP routingmodel
Flow of traffic through the network
62
Challenges to Predictabilitybull For transit ISPs effects on incoming traffic
ndash Lack of coordination strikes again
63
ldquoHot Potatordquo routing
Inter-AS Negotiation
bull Coordination aids predictabilityndash Negotiate where to sendndash Inbound and outboundndash Mutual benefits
bull How to implementndash What info to exchangendash Protecting privacyndash How to prioritize choicesndash How to prevent cheating
Destination 2
Destination 1
multiplepeeringpoints
Provider A
Provider B
64
Outbound Multihoming Goals
bull Redundancyndash Dynamic routing will failover to backup link
bull Performancendash Select provider with best performance per prefix
ndash Requires active probing
bull Costndash Select provider per prefix over time to minimize the
total financial cost
65
Inbound Traffic Control
bull More difficult no control over neighborsrsquo decisions
bull Three common techniques (previously discussed)ndash AS path prependingndash Communities and local preferencendash Prefix splitting
How does todayrsquos paper (MONET) control inbound traffic
66
How many links are enough
K upstream ISPs
Not much benefit beyond 4 ISPs
Akella et al ldquoPerformance Benefits of Multihomingrdquo SIGCOMM 2003
67
Problems with Multihoming in IPv4
bull Routing table growthndash Provider-based addressingndash Advertising prefix out multiple ISPs ndash canrsquot aggregate
bull Poor control over inbound trafficndash Existing mechanisms do not allow hosts to control
inbound traffic
68
GeorgiaTech
Internet Routing Overview
bull Intradomain (ie ldquointra-ASrdquo) routingbull Interdomain routing
Comcast
Abilene
ATampT Cogent
Autonomous Systems (ASes)
69
Configuration Problems ldquoAS 7007rdquoldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
UUNet
Florida InternetBarn
Sprint
70
Diagnosis and Troubleshootingldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
ldquoMicrosofts websites were offline for up to 23 hoursbecause of a [router] misconfigurationhellipit took nearly a day to determine what was wrong and undo the changesrdquo -- wiredcom January 25 2001
ldquoWorldCom Inchellipsuffered a widespread outage on its Internet backbone that affected roughly 20 percent of its US customer base The network problemshellipaffected millions of computer users worldwide A spokeswoman attributed the outage to a route table issue -- cnncom October 3 2002
A number of Covad customers went out from 5pm today due to supposedly a DDOS (distributed denial of service attack) on a key Level3 data center which later was described as a route leak (misconfiguration)ldquo
-- dslreportscom February 23 2004
71
Operator Mailing List (NANOG)Date Mon 18 Oct 2004 091515 -0700Subject Level 3 US east coast issues
Level 3 experiencing widespread unspecified routing issues on the US east coast Master ticket 1086844 Anyone have more specific information
Date Mon 18 Oct 2004 122034 -0400 (EDT)Subject Re Level 3 US east coast issues
Level 3 is currently experiencing a backbone outage causing routing instability and packet loss We are working to restore and will be sending hourly updateshellip
72
Operator Mailing List
0102030405060708090
Filtering RouteLeaks
RouteHijacks
RouteInstability
RoutingLoops
Blackholes
Occurr
ences o
ver
10 Y
ears
1995-1997 1998-2001 2001-2004
Note Only includes problems openly discussed on this list
Compare 83 power outages 1 fire
73
Routing Configuration
Ranking route selection
Dissemination internal route advertisement
Filtering route advertisement
Customer
Competitor
Primary
Backup
74
Internet Business Model (Simplified)
bull CustomerProvider One AS pays another for reachability to some set of destinations
bull ldquoSettlement-freerdquo Peering Bartering Two ASes exchange routes with one another
Provider
Peer
Customer
Preferences implemented with local preference manipulation
Destination
Pay to use
Get paid to use
Free to use
75
Peering Contracts Consistent Export
bull Rules of settlement-free peeringndash Advertise routes at all peering pointsndash Advertised routes must have equal ldquoAS path lengthrdquo
Sprint
ATampT
Enables ldquohot potatordquo routing
ldquoequally goodrdquoroutes
76
Consistent Export
bull Malicedeceptionbull iBGP signaling partitionbull Inconsistent export policy
neighbor 10123route-map PEER permit 10 set prepend 123
neighbor 10456route-map PEER permit 10 set prepend 123 123
Possible Causes
10123 456 1
10456 456 2
Neighbor AS Export
1 1 123
Export Clause Prepend
2 1 123 123
Two different Export Policies
77
Inconsistent Export in Practice
Feamster et al ldquoBorderGuard Detecting Cold Potatoes from Peersrdquo ACM IMC October 2004
78
Blackholes
Date Thu 18 Jul 2002 060510 -0400 (EDT)From Chad Oleary ltcolpoboxcomgtSubject Re problems with 701To ltnanogmeritedugt
Were starting to see the same issues with UUNet again Anyone elseseeing this Trying to reach Qwest
traceroute to 631461901 (631461901) 30 hops max 38 byte packets 1 esc-lp2-gwe-solutionscorpcom (631182201) 1167 ms 1163 ms 1142 ms 2 500Serial2-10GW1TPA2ALTERNET (1571301499) 1097 ms 1059 ms 1044 ms 3 161at-1-0-0XL4ATL1ALTERNET (1526381190) 13839 ms 14108 ms 16638 ms 4 0so-3-1-0XL2ATL5ALTERNET (152630238) 14370 ms 14587 ms 14553 ms 5 POS7-0BR2ATL5ALTERNET (1526382193) 13928 ms 14099 ms 14053 ms 6 7 hellip
79
Security
80
Security ldquoBogonrdquo Routes
Feamster et al ldquoAn Empirical Study of lsquoBogonrsquo Route Advertisementsrdquo ACM CCR January 2005
81
Spam Phishing etc
bull Unsolicited commercial emailbull As of about August 2008 estimates indicate that
about 95 of all email is spambull Common spam filtering techniques
ndash Content-based filtersndash DNS Blacklist (DNSBL) lookups Significant fraction of
todayrsquos DNS traffic
Can IP addresses from which spam is received be spoofed
82
Spam and Routing
83
Worms and Botnets
84
What is a Worm
bull Code that replicates and propagates across the networkndash Often carries a ldquopayloadrdquo
bull Usually spread via exploiting flaws in open servicesndash ldquoVirusesrdquo require user action to spread
bull First worm Robert Morris November 1988ndash 6-10 of all Internet hosts infected ()
bull Many more since but none on that scale until July 2001
85
Example Worm Code Red
bull Initial version July 13 2001
bull Exploited known ISAPI vulnerability in Microsoft IIS Web servers
bull 1st through 20th of each month spread20th through end of each month attack
bull Payload Web site defacementbull Scanning Random IP addressesbull Bug failure to seed random number generator
86
Code Red Revisions
bull Released July 19 2001
bull Payload flooding attack on wwwwhitehousegovndash Attack was mounted at the IP address of the Web site
bull Bug died after 20th of each month
bull Random number generator for IP scanning fixed
87
Code Red Host Infection Rate
Exponential infection rate
Measured using backscatter technique
88
Designing Fast-Spreading Worms
bull Hit-list scanningndash Time to infect first 10k hosts dominates infection timendash Solution Reconnaissance (stealthy scans etc)
bull Permutation scanningndash Observation Most scanning is redundantndash Idea Shared permutation of address space Start scanning
from own IP address Re-randomize when another infected machine is found
bull Internet-scale hit listsndash Flash worm complete infection within 30 seconds
89
Botnets
bull Bots Autonomous programs performing tasksbull Plenty of ldquobenignrdquo bots
ndash eg weatherbug
bull Botnets group of bots ndash Typically carries malicious connotationndash Large numbers of infected machinesndash Machines ldquoenlistedrdquo with infection vectors like worms
(last lecture)
bull Available for simultaneous control by a masterbull Size up to 350000 nodes (from todayrsquos paper)
90
ldquoRallyingrdquo the Botnet
bull Easy to combine worm backdoor functionalitybull Problem how to learn about successfully
infected machines
bull Optionsndash Emailndash Hard-coded email address
91
Botnet Control
bull Botnet master typically runs some IRC server on a well-known port (eg 6667)
bull Infected machine contacts botnet with pre-programmed DNS name (eg big-botde)
bull Dynamic DNS allows controller to move about freely
Infected Machine
DynamicDNS
BotnetController
(IRC server)
92
Some Defenses
93
Idea 1 Ingress Filtering
bull RFC 2827 Routers install filters to drop packets from networks that are not downstream
bull Feasible at edgesbull Difficult to configure closer to network ldquocorerdquo
20469207024 Internet
Drop all packets with source address other than 20469207024
94
Idea 2 uRPF Checks
bull Unicast Reverse Path Forwardingndash Cisco ldquoip verify unicast reverse-pathrdquo
bull Requires symmetric routing
Accept packet from interface only if forwarding table entry for source IP address matches ingress interface
100183
A10018
10015
101203
100181 24
10011 24
Strict Mode uRPF
Enabled
ldquoArdquo Routing TableDestination Next Hop1001024 Int 110018024 Int 2
100183 from wrong interface
95
Problems with uRPF
bull Asymmetric routing
96
S-BGP
bull Address-based PKI validate signaturesndash Authentication of
bull ownership for IP address blocks bull AS number bull an ASs identity and bull a BGP routers identity
ndash Use existing infrastructure (Internet registries etc)ndash Routing origination is digitally signedndash BGP updates are digitally signed
1048708 bull Route attestations A new optional BGP transitive path attribute
ndash carries digital signatures covering the routing information in updates
97
Practical Problems with S-BGP
bull Requires Public-Key Infrastructure
bull Lots of digital signatures to calculate and verifyndash Message overheadndash CPU overhead
bull Calculation expense is greatest when topology is changingndash Caching can help
bull Route aggregation is problematic (maybe thatrsquos OK)
bull Secure route withdrawals when link or node fails
bull Address ownership data out of date
bull Deployment
- Networking
- Goal of This Tutorial
- Todayrsquos Networks
- Business Models
- Billing for Internet Usage
- Net Neutrality
- Network Operations
- Point-of-Presence (PoP)
- Example Abilene Network Topology
- Another Example Backbone
- Internet Routing Overview
- Internet Routing Protocol BGP
- Two Flavors of BGP
- IPv4 Addresses Networks of Networks
- Pre-1994 Classful Addressing
- Classless Interdomain Routing (CIDR)
- Benefits of CIDR
- Growth of IP Prefixes
- 1994-1998 Linear Growth
- Around 2000 Fast Growth Resumes
- Fast growth resumes
- The Address Allocation Process
- Common Problems
- What can go wrong
- Measurement and Monitoring
- Passive vs Active Measurement
- Slide 27
- Passive Traffic Data Measurement
- Simple Network Management Protocol
- SNMP (Passive)
- Packet-level Monitoring
- Full Packet Capture (Passive)
- What is a flow
- Cisco NetFlow
- Flow Record Contents
- Aggregating Packets into Flows
- Reducing Measurement Overhead
- Packet Sampling for Flow Monitoring
- Sampling Flow-Level Sampling
- Two Main Approaches
- Packet Capture on High-Speed Links
- Characteristics of Packet Capture
- Data Measurement Repositories
- Multihoming and Traffic Engineering
- What is Multihoming
- Why Multihome
- Redundancy
- Performance
- Multihoming in IP Networks Today
- BGP or no
- Same Provider or Multiple
- Multihomed Stub One Link
- Multihomed Stub Multiple Links
- Multihomed Stub Multiple ISPs
- Multihomed Transit Network
- Interdomain Traffic Engineering
- Outbound Traffic Control
- Outbound Traffic Load Balancing
- Traffic Engineering Goals
- Managing Scale
- Achieving Predictability
- Challenges to Predictability
- Inter-AS Negotiation
- Outbound Multihoming Goals
- Inbound Traffic Control
- How many links are enough
- Problems with Multihoming in IPv4
- Slide 68
- Slide 69
- Diagnosis and Troubleshooting
- Operator Mailing List (NANOG)
- Operator Mailing List
- Routing Configuration
- Internet Business Model (Simplified)
- Peering Contracts Consistent Export
- Consistent Export
- Inconsistent Export in Practice
- Blackholes
- Security
- Security ldquoBogonrdquo Routes
- Spam Phishing etc
- Spam and Routing
- Worms and Botnets
- What is a Worm
- Example Worm Code Red
- Code Red Revisions
- Code Red Host Infection Rate
- Designing Fast-Spreading Worms
- Botnets
- ldquoRallyingrdquo the Botnet
- Botnet Control
- Some Defenses
- Idea 1 Ingress Filtering
- Idea 2 uRPF Checks
- Problems with uRPF
- S-BGP
- Practical Problems with S-BGP
-
47
Redundancy
bull Maintain connectivity in the face ofndash Physical connectivity problems (fiber cut device
failures etc)ndash Failures in upstream ISP
48
Performance
bull Use multiple network links at once to achieve higher throughput than just over a single link
bull Allows incoming traffic to be load-balanced
70 of traffic30 of traffic
49
Multihoming in IP Networks Today
bull Stub AS no transit service for other ASesndash No need to use BGP
bull Multi-homed stub AS has connectivity to multiple immediate upstream ISPsndash Need BGPndash No need for a public AS numberndash No need for IP prefix allocation
bull Multi-homed transit AS connectivity to multiple ASes and transit servicendash Need BGP public AS number IP prefix allocation
50
BGP or no
bull Advantages of static routingndash Cheapersmaller routers (less true nowadays)ndash Simpler to configure
bull Advantages of BGPndash More control of your destiny (have providers stop
announcing you)ndash Fastermore intelligent selection of where to send
outbound packetsndash Better debugging of net problems (you can see the
Internet topology now)
51
Same Provider or Multiple
bull If your provider is reliable and fast and affordably and offers good tech-support you may want to multi-home initially to them via some backup path (slow is better than dead)
bull Eventually yoursquoll want to multi-home to different providers to avoid failure modes due to one providerrsquos architecture decisions
52
Multihomed Stub One Link
bull Downstream ISPrsquos routers configure default (ldquostaticrdquo) routes pointing to border router
bull Upstream ISP advertises reachability
Upstream ISP
Multiple links between same pair of routers
Default routes to ldquoborderrdquo
ldquoStubrdquoISP
53
Multihomed Stub Multiple Links
bull Use BGP to share loadbull Use private AS number (why is this OK)bull As before upstream ISP advertises prefix
Upstream ISP
Multiple links to different upstream routers
ldquoStubrdquoISP
Internal routing for ldquohot potatordquo
BGP for load balance at edge
54
Multihomed Stub Multiple ISPs
bull Many possibilitiesndash Load sharingndash Primary-backupndash Selective use of different ISPs
bull Requires BGP public AS number etc
ldquoStubrdquoISP
Upstream
ISP 1
Upstream
ISP 2
55
Multihomed Transit Network
bull BGP everywherebull Incoming and outcoming trafficbull Challenge balancing load on intradomain and egress
links given an offered traffic load
TransitISP
ISP 1
ISP 2
ISP 3
56
Interdomain Traffic Engineering
bull The process by which a network operator configures the network to achievendash Traffic load balancendash Redundancy (primarybackup) etc
bull Two tasksndash Outbound traffic controlndash Inbound traffic control
bull Key Problems Predictability and Scalability
57
Outbound Traffic Control
bull Easier to control than inbound trafficndash Destination-based routing sender determines where
the packets go
bull Control over next-hop AS onlyndash Cannot control selection of the entire path
Provider 1 Provider 2
Control with local preference
58
Outbound Traffic Load Balancingbull Control routes to provider per-prefix
ndash Assign local preference across destination prefixesndash Change the local preference assignments over time
bull Useful inputs to load balancingndash End-to-end path performance datandash Outbound traffic statistics per destination prefix
bull Challenge Getting from traffic volumes to groups of prefixes that should be assigned to each link
Premise of ldquointelligent route controlrdquo preoducts
59
Traffic Engineering Goals
bull Predictabilityndash Ensure the BGP decision process is deterministicndash Assume that BGP updates are (relatively) stable
bull Limit overhead introduced by routing changesndash Minimize frequency of changes to routing policiesndash Limit number of prefixes affected by changes
bull Limit impact on how traffic enters the networkndash Avoid new routes that might change neighborrsquos mindndash Select route with same attributes or at least path length
60
Managing Scalebull Destination prefixes
ndash More than 90000 destination prefixes
bull Donrsquot want to have per-prefix routing policies
ndash Small fraction of prefixes contribute most of the traffic
bull Focus on the small number of heavy hitters
ndash Define routing policies for selected prefixes
bull Routing choicesndash About 27000 unique ldquorouting choicesrdquo
bull Help in reducing the scale of the problem
ndash Small fraction of ldquorouting choicesrdquo contribute most traffic
bull Focus on the very small number of ldquorouting choicesrdquo
ndash Define routing policies on common attributes
61
Achieving Predictability
bull Route prediction with static analysisndash Helpful to know effects before deploymentndash Static analysis can help
TopologyBGP policy
configuration
eBGP routes
Offered traffic
BGP routingmodel
Flow of traffic through the network
62
Challenges to Predictabilitybull For transit ISPs effects on incoming traffic
ndash Lack of coordination strikes again
63
ldquoHot Potatordquo routing
Inter-AS Negotiation
bull Coordination aids predictabilityndash Negotiate where to sendndash Inbound and outboundndash Mutual benefits
bull How to implementndash What info to exchangendash Protecting privacyndash How to prioritize choicesndash How to prevent cheating
Destination 2
Destination 1
multiplepeeringpoints
Provider A
Provider B
64
Outbound Multihoming Goals
bull Redundancyndash Dynamic routing will failover to backup link
bull Performancendash Select provider with best performance per prefix
ndash Requires active probing
bull Costndash Select provider per prefix over time to minimize the
total financial cost
65
Inbound Traffic Control
bull More difficult no control over neighborsrsquo decisions
bull Three common techniques (previously discussed)ndash AS path prependingndash Communities and local preferencendash Prefix splitting
How does todayrsquos paper (MONET) control inbound traffic
66
How many links are enough
K upstream ISPs
Not much benefit beyond 4 ISPs
Akella et al ldquoPerformance Benefits of Multihomingrdquo SIGCOMM 2003
67
Problems with Multihoming in IPv4
bull Routing table growthndash Provider-based addressingndash Advertising prefix out multiple ISPs ndash canrsquot aggregate
bull Poor control over inbound trafficndash Existing mechanisms do not allow hosts to control
inbound traffic
68
GeorgiaTech
Internet Routing Overview
bull Intradomain (ie ldquointra-ASrdquo) routingbull Interdomain routing
Comcast
Abilene
ATampT Cogent
Autonomous Systems (ASes)
69
Configuration Problems ldquoAS 7007rdquoldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
UUNet
Florida InternetBarn
Sprint
70
Diagnosis and Troubleshootingldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
ldquoMicrosofts websites were offline for up to 23 hoursbecause of a [router] misconfigurationhellipit took nearly a day to determine what was wrong and undo the changesrdquo -- wiredcom January 25 2001
ldquoWorldCom Inchellipsuffered a widespread outage on its Internet backbone that affected roughly 20 percent of its US customer base The network problemshellipaffected millions of computer users worldwide A spokeswoman attributed the outage to a route table issue -- cnncom October 3 2002
A number of Covad customers went out from 5pm today due to supposedly a DDOS (distributed denial of service attack) on a key Level3 data center which later was described as a route leak (misconfiguration)ldquo
-- dslreportscom February 23 2004
71
Operator Mailing List (NANOG)Date Mon 18 Oct 2004 091515 -0700Subject Level 3 US east coast issues
Level 3 experiencing widespread unspecified routing issues on the US east coast Master ticket 1086844 Anyone have more specific information
Date Mon 18 Oct 2004 122034 -0400 (EDT)Subject Re Level 3 US east coast issues
Level 3 is currently experiencing a backbone outage causing routing instability and packet loss We are working to restore and will be sending hourly updateshellip
72
Operator Mailing List
0102030405060708090
Filtering RouteLeaks
RouteHijacks
RouteInstability
RoutingLoops
Blackholes
Occurr
ences o
ver
10 Y
ears
1995-1997 1998-2001 2001-2004
Note Only includes problems openly discussed on this list
Compare 83 power outages 1 fire
73
Routing Configuration
Ranking route selection
Dissemination internal route advertisement
Filtering route advertisement
Customer
Competitor
Primary
Backup
74
Internet Business Model (Simplified)
bull CustomerProvider One AS pays another for reachability to some set of destinations
bull ldquoSettlement-freerdquo Peering Bartering Two ASes exchange routes with one another
Provider
Peer
Customer
Preferences implemented with local preference manipulation
Destination
Pay to use
Get paid to use
Free to use
75
Peering Contracts Consistent Export
bull Rules of settlement-free peeringndash Advertise routes at all peering pointsndash Advertised routes must have equal ldquoAS path lengthrdquo
Sprint
ATampT
Enables ldquohot potatordquo routing
ldquoequally goodrdquoroutes
76
Consistent Export
bull Malicedeceptionbull iBGP signaling partitionbull Inconsistent export policy
neighbor 10123route-map PEER permit 10 set prepend 123
neighbor 10456route-map PEER permit 10 set prepend 123 123
Possible Causes
10123 456 1
10456 456 2
Neighbor AS Export
1 1 123
Export Clause Prepend
2 1 123 123
Two different Export Policies
77
Inconsistent Export in Practice
Feamster et al ldquoBorderGuard Detecting Cold Potatoes from Peersrdquo ACM IMC October 2004
78
Blackholes
Date Thu 18 Jul 2002 060510 -0400 (EDT)From Chad Oleary ltcolpoboxcomgtSubject Re problems with 701To ltnanogmeritedugt
Were starting to see the same issues with UUNet again Anyone elseseeing this Trying to reach Qwest
traceroute to 631461901 (631461901) 30 hops max 38 byte packets 1 esc-lp2-gwe-solutionscorpcom (631182201) 1167 ms 1163 ms 1142 ms 2 500Serial2-10GW1TPA2ALTERNET (1571301499) 1097 ms 1059 ms 1044 ms 3 161at-1-0-0XL4ATL1ALTERNET (1526381190) 13839 ms 14108 ms 16638 ms 4 0so-3-1-0XL2ATL5ALTERNET (152630238) 14370 ms 14587 ms 14553 ms 5 POS7-0BR2ATL5ALTERNET (1526382193) 13928 ms 14099 ms 14053 ms 6 7 hellip
79
Security
80
Security ldquoBogonrdquo Routes
Feamster et al ldquoAn Empirical Study of lsquoBogonrsquo Route Advertisementsrdquo ACM CCR January 2005
81
Spam Phishing etc
bull Unsolicited commercial emailbull As of about August 2008 estimates indicate that
about 95 of all email is spambull Common spam filtering techniques
ndash Content-based filtersndash DNS Blacklist (DNSBL) lookups Significant fraction of
todayrsquos DNS traffic
Can IP addresses from which spam is received be spoofed
82
Spam and Routing
83
Worms and Botnets
84
What is a Worm
bull Code that replicates and propagates across the networkndash Often carries a ldquopayloadrdquo
bull Usually spread via exploiting flaws in open servicesndash ldquoVirusesrdquo require user action to spread
bull First worm Robert Morris November 1988ndash 6-10 of all Internet hosts infected ()
bull Many more since but none on that scale until July 2001
85
Example Worm Code Red
bull Initial version July 13 2001
bull Exploited known ISAPI vulnerability in Microsoft IIS Web servers
bull 1st through 20th of each month spread20th through end of each month attack
bull Payload Web site defacementbull Scanning Random IP addressesbull Bug failure to seed random number generator
86
Code Red Revisions
bull Released July 19 2001
bull Payload flooding attack on wwwwhitehousegovndash Attack was mounted at the IP address of the Web site
bull Bug died after 20th of each month
bull Random number generator for IP scanning fixed
87
Code Red Host Infection Rate
Exponential infection rate
Measured using backscatter technique
88
Designing Fast-Spreading Worms
bull Hit-list scanningndash Time to infect first 10k hosts dominates infection timendash Solution Reconnaissance (stealthy scans etc)
bull Permutation scanningndash Observation Most scanning is redundantndash Idea Shared permutation of address space Start scanning
from own IP address Re-randomize when another infected machine is found
bull Internet-scale hit listsndash Flash worm complete infection within 30 seconds
89
Botnets
bull Bots Autonomous programs performing tasksbull Plenty of ldquobenignrdquo bots
ndash eg weatherbug
bull Botnets group of bots ndash Typically carries malicious connotationndash Large numbers of infected machinesndash Machines ldquoenlistedrdquo with infection vectors like worms
(last lecture)
bull Available for simultaneous control by a masterbull Size up to 350000 nodes (from todayrsquos paper)
90
ldquoRallyingrdquo the Botnet
bull Easy to combine worm backdoor functionalitybull Problem how to learn about successfully
infected machines
bull Optionsndash Emailndash Hard-coded email address
91
Botnet Control
bull Botnet master typically runs some IRC server on a well-known port (eg 6667)
bull Infected machine contacts botnet with pre-programmed DNS name (eg big-botde)
bull Dynamic DNS allows controller to move about freely
Infected Machine
DynamicDNS
BotnetController
(IRC server)
92
Some Defenses
93
Idea 1 Ingress Filtering
bull RFC 2827 Routers install filters to drop packets from networks that are not downstream
bull Feasible at edgesbull Difficult to configure closer to network ldquocorerdquo
20469207024 Internet
Drop all packets with source address other than 20469207024
94
Idea 2 uRPF Checks
bull Unicast Reverse Path Forwardingndash Cisco ldquoip verify unicast reverse-pathrdquo
bull Requires symmetric routing
Accept packet from interface only if forwarding table entry for source IP address matches ingress interface
100183
A10018
10015
101203
100181 24
10011 24
Strict Mode uRPF
Enabled
ldquoArdquo Routing TableDestination Next Hop1001024 Int 110018024 Int 2
100183 from wrong interface
95
Problems with uRPF
bull Asymmetric routing
96
S-BGP
bull Address-based PKI validate signaturesndash Authentication of
bull ownership for IP address blocks bull AS number bull an ASs identity and bull a BGP routers identity
ndash Use existing infrastructure (Internet registries etc)ndash Routing origination is digitally signedndash BGP updates are digitally signed
1048708 bull Route attestations A new optional BGP transitive path attribute
ndash carries digital signatures covering the routing information in updates
97
Practical Problems with S-BGP
bull Requires Public-Key Infrastructure
bull Lots of digital signatures to calculate and verifyndash Message overheadndash CPU overhead
bull Calculation expense is greatest when topology is changingndash Caching can help
bull Route aggregation is problematic (maybe thatrsquos OK)
bull Secure route withdrawals when link or node fails
bull Address ownership data out of date
bull Deployment
- Networking
- Goal of This Tutorial
- Todayrsquos Networks
- Business Models
- Billing for Internet Usage
- Net Neutrality
- Network Operations
- Point-of-Presence (PoP)
- Example Abilene Network Topology
- Another Example Backbone
- Internet Routing Overview
- Internet Routing Protocol BGP
- Two Flavors of BGP
- IPv4 Addresses Networks of Networks
- Pre-1994 Classful Addressing
- Classless Interdomain Routing (CIDR)
- Benefits of CIDR
- Growth of IP Prefixes
- 1994-1998 Linear Growth
- Around 2000 Fast Growth Resumes
- Fast growth resumes
- The Address Allocation Process
- Common Problems
- What can go wrong
- Measurement and Monitoring
- Passive vs Active Measurement
- Slide 27
- Passive Traffic Data Measurement
- Simple Network Management Protocol
- SNMP (Passive)
- Packet-level Monitoring
- Full Packet Capture (Passive)
- What is a flow
- Cisco NetFlow
- Flow Record Contents
- Aggregating Packets into Flows
- Reducing Measurement Overhead
- Packet Sampling for Flow Monitoring
- Sampling Flow-Level Sampling
- Two Main Approaches
- Packet Capture on High-Speed Links
- Characteristics of Packet Capture
- Data Measurement Repositories
- Multihoming and Traffic Engineering
- What is Multihoming
- Why Multihome
- Redundancy
- Performance
- Multihoming in IP Networks Today
- BGP or no
- Same Provider or Multiple
- Multihomed Stub One Link
- Multihomed Stub Multiple Links
- Multihomed Stub Multiple ISPs
- Multihomed Transit Network
- Interdomain Traffic Engineering
- Outbound Traffic Control
- Outbound Traffic Load Balancing
- Traffic Engineering Goals
- Managing Scale
- Achieving Predictability
- Challenges to Predictability
- Inter-AS Negotiation
- Outbound Multihoming Goals
- Inbound Traffic Control
- How many links are enough
- Problems with Multihoming in IPv4
- Slide 68
- Slide 69
- Diagnosis and Troubleshooting
- Operator Mailing List (NANOG)
- Operator Mailing List
- Routing Configuration
- Internet Business Model (Simplified)
- Peering Contracts Consistent Export
- Consistent Export
- Inconsistent Export in Practice
- Blackholes
- Security
- Security ldquoBogonrdquo Routes
- Spam Phishing etc
- Spam and Routing
- Worms and Botnets
- What is a Worm
- Example Worm Code Red
- Code Red Revisions
- Code Red Host Infection Rate
- Designing Fast-Spreading Worms
- Botnets
- ldquoRallyingrdquo the Botnet
- Botnet Control
- Some Defenses
- Idea 1 Ingress Filtering
- Idea 2 uRPF Checks
- Problems with uRPF
- S-BGP
- Practical Problems with S-BGP
-
48
Performance
bull Use multiple network links at once to achieve higher throughput than just over a single link
bull Allows incoming traffic to be load-balanced
70 of traffic30 of traffic
49
Multihoming in IP Networks Today
bull Stub AS no transit service for other ASesndash No need to use BGP
bull Multi-homed stub AS has connectivity to multiple immediate upstream ISPsndash Need BGPndash No need for a public AS numberndash No need for IP prefix allocation
bull Multi-homed transit AS connectivity to multiple ASes and transit servicendash Need BGP public AS number IP prefix allocation
50
BGP or no
bull Advantages of static routingndash Cheapersmaller routers (less true nowadays)ndash Simpler to configure
bull Advantages of BGPndash More control of your destiny (have providers stop
announcing you)ndash Fastermore intelligent selection of where to send
outbound packetsndash Better debugging of net problems (you can see the
Internet topology now)
51
Same Provider or Multiple
bull If your provider is reliable and fast and affordably and offers good tech-support you may want to multi-home initially to them via some backup path (slow is better than dead)
bull Eventually yoursquoll want to multi-home to different providers to avoid failure modes due to one providerrsquos architecture decisions
52
Multihomed Stub One Link
bull Downstream ISPrsquos routers configure default (ldquostaticrdquo) routes pointing to border router
bull Upstream ISP advertises reachability
Upstream ISP
Multiple links between same pair of routers
Default routes to ldquoborderrdquo
ldquoStubrdquoISP
53
Multihomed Stub Multiple Links
bull Use BGP to share loadbull Use private AS number (why is this OK)bull As before upstream ISP advertises prefix
Upstream ISP
Multiple links to different upstream routers
ldquoStubrdquoISP
Internal routing for ldquohot potatordquo
BGP for load balance at edge
54
Multihomed Stub Multiple ISPs
bull Many possibilitiesndash Load sharingndash Primary-backupndash Selective use of different ISPs
bull Requires BGP public AS number etc
ldquoStubrdquoISP
Upstream
ISP 1
Upstream
ISP 2
55
Multihomed Transit Network
bull BGP everywherebull Incoming and outcoming trafficbull Challenge balancing load on intradomain and egress
links given an offered traffic load
TransitISP
ISP 1
ISP 2
ISP 3
56
Interdomain Traffic Engineering
bull The process by which a network operator configures the network to achievendash Traffic load balancendash Redundancy (primarybackup) etc
bull Two tasksndash Outbound traffic controlndash Inbound traffic control
bull Key Problems Predictability and Scalability
57
Outbound Traffic Control
bull Easier to control than inbound trafficndash Destination-based routing sender determines where
the packets go
bull Control over next-hop AS onlyndash Cannot control selection of the entire path
Provider 1 Provider 2
Control with local preference
58
Outbound Traffic Load Balancingbull Control routes to provider per-prefix
ndash Assign local preference across destination prefixesndash Change the local preference assignments over time
bull Useful inputs to load balancingndash End-to-end path performance datandash Outbound traffic statistics per destination prefix
bull Challenge Getting from traffic volumes to groups of prefixes that should be assigned to each link
Premise of ldquointelligent route controlrdquo preoducts
59
Traffic Engineering Goals
bull Predictabilityndash Ensure the BGP decision process is deterministicndash Assume that BGP updates are (relatively) stable
bull Limit overhead introduced by routing changesndash Minimize frequency of changes to routing policiesndash Limit number of prefixes affected by changes
bull Limit impact on how traffic enters the networkndash Avoid new routes that might change neighborrsquos mindndash Select route with same attributes or at least path length
60
Managing Scalebull Destination prefixes
ndash More than 90000 destination prefixes
bull Donrsquot want to have per-prefix routing policies
ndash Small fraction of prefixes contribute most of the traffic
bull Focus on the small number of heavy hitters
ndash Define routing policies for selected prefixes
bull Routing choicesndash About 27000 unique ldquorouting choicesrdquo
bull Help in reducing the scale of the problem
ndash Small fraction of ldquorouting choicesrdquo contribute most traffic
bull Focus on the very small number of ldquorouting choicesrdquo
ndash Define routing policies on common attributes
61
Achieving Predictability
bull Route prediction with static analysisndash Helpful to know effects before deploymentndash Static analysis can help
TopologyBGP policy
configuration
eBGP routes
Offered traffic
BGP routingmodel
Flow of traffic through the network
62
Challenges to Predictabilitybull For transit ISPs effects on incoming traffic
ndash Lack of coordination strikes again
63
ldquoHot Potatordquo routing
Inter-AS Negotiation
bull Coordination aids predictabilityndash Negotiate where to sendndash Inbound and outboundndash Mutual benefits
bull How to implementndash What info to exchangendash Protecting privacyndash How to prioritize choicesndash How to prevent cheating
Destination 2
Destination 1
multiplepeeringpoints
Provider A
Provider B
64
Outbound Multihoming Goals
bull Redundancyndash Dynamic routing will failover to backup link
bull Performancendash Select provider with best performance per prefix
ndash Requires active probing
bull Costndash Select provider per prefix over time to minimize the
total financial cost
65
Inbound Traffic Control
bull More difficult no control over neighborsrsquo decisions
bull Three common techniques (previously discussed)ndash AS path prependingndash Communities and local preferencendash Prefix splitting
How does todayrsquos paper (MONET) control inbound traffic
66
How many links are enough
K upstream ISPs
Not much benefit beyond 4 ISPs
Akella et al ldquoPerformance Benefits of Multihomingrdquo SIGCOMM 2003
67
Problems with Multihoming in IPv4
bull Routing table growthndash Provider-based addressingndash Advertising prefix out multiple ISPs ndash canrsquot aggregate
bull Poor control over inbound trafficndash Existing mechanisms do not allow hosts to control
inbound traffic
68
GeorgiaTech
Internet Routing Overview
bull Intradomain (ie ldquointra-ASrdquo) routingbull Interdomain routing
Comcast
Abilene
ATampT Cogent
Autonomous Systems (ASes)
69
Configuration Problems ldquoAS 7007rdquoldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
UUNet
Florida InternetBarn
Sprint
70
Diagnosis and Troubleshootingldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
ldquoMicrosofts websites were offline for up to 23 hoursbecause of a [router] misconfigurationhellipit took nearly a day to determine what was wrong and undo the changesrdquo -- wiredcom January 25 2001
ldquoWorldCom Inchellipsuffered a widespread outage on its Internet backbone that affected roughly 20 percent of its US customer base The network problemshellipaffected millions of computer users worldwide A spokeswoman attributed the outage to a route table issue -- cnncom October 3 2002
A number of Covad customers went out from 5pm today due to supposedly a DDOS (distributed denial of service attack) on a key Level3 data center which later was described as a route leak (misconfiguration)ldquo
-- dslreportscom February 23 2004
71
Operator Mailing List (NANOG)Date Mon 18 Oct 2004 091515 -0700Subject Level 3 US east coast issues
Level 3 experiencing widespread unspecified routing issues on the US east coast Master ticket 1086844 Anyone have more specific information
Date Mon 18 Oct 2004 122034 -0400 (EDT)Subject Re Level 3 US east coast issues
Level 3 is currently experiencing a backbone outage causing routing instability and packet loss We are working to restore and will be sending hourly updateshellip
72
Operator Mailing List
0102030405060708090
Filtering RouteLeaks
RouteHijacks
RouteInstability
RoutingLoops
Blackholes
Occurr
ences o
ver
10 Y
ears
1995-1997 1998-2001 2001-2004
Note Only includes problems openly discussed on this list
Compare 83 power outages 1 fire
73
Routing Configuration
Ranking route selection
Dissemination internal route advertisement
Filtering route advertisement
Customer
Competitor
Primary
Backup
74
Internet Business Model (Simplified)
bull CustomerProvider One AS pays another for reachability to some set of destinations
bull ldquoSettlement-freerdquo Peering Bartering Two ASes exchange routes with one another
Provider
Peer
Customer
Preferences implemented with local preference manipulation
Destination
Pay to use
Get paid to use
Free to use
75
Peering Contracts Consistent Export
bull Rules of settlement-free peeringndash Advertise routes at all peering pointsndash Advertised routes must have equal ldquoAS path lengthrdquo
Sprint
ATampT
Enables ldquohot potatordquo routing
ldquoequally goodrdquoroutes
76
Consistent Export
bull Malicedeceptionbull iBGP signaling partitionbull Inconsistent export policy
neighbor 10123route-map PEER permit 10 set prepend 123
neighbor 10456route-map PEER permit 10 set prepend 123 123
Possible Causes
10123 456 1
10456 456 2
Neighbor AS Export
1 1 123
Export Clause Prepend
2 1 123 123
Two different Export Policies
77
Inconsistent Export in Practice
Feamster et al ldquoBorderGuard Detecting Cold Potatoes from Peersrdquo ACM IMC October 2004
78
Blackholes
Date Thu 18 Jul 2002 060510 -0400 (EDT)From Chad Oleary ltcolpoboxcomgtSubject Re problems with 701To ltnanogmeritedugt
Were starting to see the same issues with UUNet again Anyone elseseeing this Trying to reach Qwest
traceroute to 631461901 (631461901) 30 hops max 38 byte packets 1 esc-lp2-gwe-solutionscorpcom (631182201) 1167 ms 1163 ms 1142 ms 2 500Serial2-10GW1TPA2ALTERNET (1571301499) 1097 ms 1059 ms 1044 ms 3 161at-1-0-0XL4ATL1ALTERNET (1526381190) 13839 ms 14108 ms 16638 ms 4 0so-3-1-0XL2ATL5ALTERNET (152630238) 14370 ms 14587 ms 14553 ms 5 POS7-0BR2ATL5ALTERNET (1526382193) 13928 ms 14099 ms 14053 ms 6 7 hellip
79
Security
80
Security ldquoBogonrdquo Routes
Feamster et al ldquoAn Empirical Study of lsquoBogonrsquo Route Advertisementsrdquo ACM CCR January 2005
81
Spam Phishing etc
bull Unsolicited commercial emailbull As of about August 2008 estimates indicate that
about 95 of all email is spambull Common spam filtering techniques
ndash Content-based filtersndash DNS Blacklist (DNSBL) lookups Significant fraction of
todayrsquos DNS traffic
Can IP addresses from which spam is received be spoofed
82
Spam and Routing
83
Worms and Botnets
84
What is a Worm
bull Code that replicates and propagates across the networkndash Often carries a ldquopayloadrdquo
bull Usually spread via exploiting flaws in open servicesndash ldquoVirusesrdquo require user action to spread
bull First worm Robert Morris November 1988ndash 6-10 of all Internet hosts infected ()
bull Many more since but none on that scale until July 2001
85
Example Worm Code Red
bull Initial version July 13 2001
bull Exploited known ISAPI vulnerability in Microsoft IIS Web servers
bull 1st through 20th of each month spread20th through end of each month attack
bull Payload Web site defacementbull Scanning Random IP addressesbull Bug failure to seed random number generator
86
Code Red Revisions
bull Released July 19 2001
bull Payload flooding attack on wwwwhitehousegovndash Attack was mounted at the IP address of the Web site
bull Bug died after 20th of each month
bull Random number generator for IP scanning fixed
87
Code Red Host Infection Rate
Exponential infection rate
Measured using backscatter technique
88
Designing Fast-Spreading Worms
bull Hit-list scanningndash Time to infect first 10k hosts dominates infection timendash Solution Reconnaissance (stealthy scans etc)
bull Permutation scanningndash Observation Most scanning is redundantndash Idea Shared permutation of address space Start scanning
from own IP address Re-randomize when another infected machine is found
bull Internet-scale hit listsndash Flash worm complete infection within 30 seconds
89
Botnets
bull Bots Autonomous programs performing tasksbull Plenty of ldquobenignrdquo bots
ndash eg weatherbug
bull Botnets group of bots ndash Typically carries malicious connotationndash Large numbers of infected machinesndash Machines ldquoenlistedrdquo with infection vectors like worms
(last lecture)
bull Available for simultaneous control by a masterbull Size up to 350000 nodes (from todayrsquos paper)
90
ldquoRallyingrdquo the Botnet
bull Easy to combine worm backdoor functionalitybull Problem how to learn about successfully
infected machines
bull Optionsndash Emailndash Hard-coded email address
91
Botnet Control
bull Botnet master typically runs some IRC server on a well-known port (eg 6667)
bull Infected machine contacts botnet with pre-programmed DNS name (eg big-botde)
bull Dynamic DNS allows controller to move about freely
Infected Machine
DynamicDNS
BotnetController
(IRC server)
92
Some Defenses
93
Idea 1 Ingress Filtering
bull RFC 2827 Routers install filters to drop packets from networks that are not downstream
bull Feasible at edgesbull Difficult to configure closer to network ldquocorerdquo
20469207024 Internet
Drop all packets with source address other than 20469207024
94
Idea 2 uRPF Checks
bull Unicast Reverse Path Forwardingndash Cisco ldquoip verify unicast reverse-pathrdquo
bull Requires symmetric routing
Accept packet from interface only if forwarding table entry for source IP address matches ingress interface
100183
A10018
10015
101203
100181 24
10011 24
Strict Mode uRPF
Enabled
ldquoArdquo Routing TableDestination Next Hop1001024 Int 110018024 Int 2
100183 from wrong interface
95
Problems with uRPF
bull Asymmetric routing
96
S-BGP
bull Address-based PKI validate signaturesndash Authentication of
bull ownership for IP address blocks bull AS number bull an ASs identity and bull a BGP routers identity
ndash Use existing infrastructure (Internet registries etc)ndash Routing origination is digitally signedndash BGP updates are digitally signed
1048708 bull Route attestations A new optional BGP transitive path attribute
ndash carries digital signatures covering the routing information in updates
97
Practical Problems with S-BGP
bull Requires Public-Key Infrastructure
bull Lots of digital signatures to calculate and verifyndash Message overheadndash CPU overhead
bull Calculation expense is greatest when topology is changingndash Caching can help
bull Route aggregation is problematic (maybe thatrsquos OK)
bull Secure route withdrawals when link or node fails
bull Address ownership data out of date
bull Deployment
- Networking
- Goal of This Tutorial
- Todayrsquos Networks
- Business Models
- Billing for Internet Usage
- Net Neutrality
- Network Operations
- Point-of-Presence (PoP)
- Example Abilene Network Topology
- Another Example Backbone
- Internet Routing Overview
- Internet Routing Protocol BGP
- Two Flavors of BGP
- IPv4 Addresses Networks of Networks
- Pre-1994 Classful Addressing
- Classless Interdomain Routing (CIDR)
- Benefits of CIDR
- Growth of IP Prefixes
- 1994-1998 Linear Growth
- Around 2000 Fast Growth Resumes
- Fast growth resumes
- The Address Allocation Process
- Common Problems
- What can go wrong
- Measurement and Monitoring
- Passive vs Active Measurement
- Slide 27
- Passive Traffic Data Measurement
- Simple Network Management Protocol
- SNMP (Passive)
- Packet-level Monitoring
- Full Packet Capture (Passive)
- What is a flow
- Cisco NetFlow
- Flow Record Contents
- Aggregating Packets into Flows
- Reducing Measurement Overhead
- Packet Sampling for Flow Monitoring
- Sampling Flow-Level Sampling
- Two Main Approaches
- Packet Capture on High-Speed Links
- Characteristics of Packet Capture
- Data Measurement Repositories
- Multihoming and Traffic Engineering
- What is Multihoming
- Why Multihome
- Redundancy
- Performance
- Multihoming in IP Networks Today
- BGP or no
- Same Provider or Multiple
- Multihomed Stub One Link
- Multihomed Stub Multiple Links
- Multihomed Stub Multiple ISPs
- Multihomed Transit Network
- Interdomain Traffic Engineering
- Outbound Traffic Control
- Outbound Traffic Load Balancing
- Traffic Engineering Goals
- Managing Scale
- Achieving Predictability
- Challenges to Predictability
- Inter-AS Negotiation
- Outbound Multihoming Goals
- Inbound Traffic Control
- How many links are enough
- Problems with Multihoming in IPv4
- Slide 68
- Slide 69
- Diagnosis and Troubleshooting
- Operator Mailing List (NANOG)
- Operator Mailing List
- Routing Configuration
- Internet Business Model (Simplified)
- Peering Contracts Consistent Export
- Consistent Export
- Inconsistent Export in Practice
- Blackholes
- Security
- Security ldquoBogonrdquo Routes
- Spam Phishing etc
- Spam and Routing
- Worms and Botnets
- What is a Worm
- Example Worm Code Red
- Code Red Revisions
- Code Red Host Infection Rate
- Designing Fast-Spreading Worms
- Botnets
- ldquoRallyingrdquo the Botnet
- Botnet Control
- Some Defenses
- Idea 1 Ingress Filtering
- Idea 2 uRPF Checks
- Problems with uRPF
- S-BGP
- Practical Problems with S-BGP
-
49
Multihoming in IP Networks Today
bull Stub AS no transit service for other ASesndash No need to use BGP
bull Multi-homed stub AS has connectivity to multiple immediate upstream ISPsndash Need BGPndash No need for a public AS numberndash No need for IP prefix allocation
bull Multi-homed transit AS connectivity to multiple ASes and transit servicendash Need BGP public AS number IP prefix allocation
50
BGP or no
bull Advantages of static routingndash Cheapersmaller routers (less true nowadays)ndash Simpler to configure
bull Advantages of BGPndash More control of your destiny (have providers stop
announcing you)ndash Fastermore intelligent selection of where to send
outbound packetsndash Better debugging of net problems (you can see the
Internet topology now)
51
Same Provider or Multiple
bull If your provider is reliable and fast and affordably and offers good tech-support you may want to multi-home initially to them via some backup path (slow is better than dead)
bull Eventually yoursquoll want to multi-home to different providers to avoid failure modes due to one providerrsquos architecture decisions
52
Multihomed Stub One Link
bull Downstream ISPrsquos routers configure default (ldquostaticrdquo) routes pointing to border router
bull Upstream ISP advertises reachability
Upstream ISP
Multiple links between same pair of routers
Default routes to ldquoborderrdquo
ldquoStubrdquoISP
53
Multihomed Stub Multiple Links
bull Use BGP to share loadbull Use private AS number (why is this OK)bull As before upstream ISP advertises prefix
Upstream ISP
Multiple links to different upstream routers
ldquoStubrdquoISP
Internal routing for ldquohot potatordquo
BGP for load balance at edge
54
Multihomed Stub Multiple ISPs
bull Many possibilitiesndash Load sharingndash Primary-backupndash Selective use of different ISPs
bull Requires BGP public AS number etc
ldquoStubrdquoISP
Upstream
ISP 1
Upstream
ISP 2
55
Multihomed Transit Network
bull BGP everywherebull Incoming and outcoming trafficbull Challenge balancing load on intradomain and egress
links given an offered traffic load
TransitISP
ISP 1
ISP 2
ISP 3
56
Interdomain Traffic Engineering
bull The process by which a network operator configures the network to achievendash Traffic load balancendash Redundancy (primarybackup) etc
bull Two tasksndash Outbound traffic controlndash Inbound traffic control
bull Key Problems Predictability and Scalability
57
Outbound Traffic Control
bull Easier to control than inbound trafficndash Destination-based routing sender determines where
the packets go
bull Control over next-hop AS onlyndash Cannot control selection of the entire path
Provider 1 Provider 2
Control with local preference
58
Outbound Traffic Load Balancingbull Control routes to provider per-prefix
ndash Assign local preference across destination prefixesndash Change the local preference assignments over time
bull Useful inputs to load balancingndash End-to-end path performance datandash Outbound traffic statistics per destination prefix
bull Challenge Getting from traffic volumes to groups of prefixes that should be assigned to each link
Premise of ldquointelligent route controlrdquo preoducts
59
Traffic Engineering Goals
bull Predictabilityndash Ensure the BGP decision process is deterministicndash Assume that BGP updates are (relatively) stable
bull Limit overhead introduced by routing changesndash Minimize frequency of changes to routing policiesndash Limit number of prefixes affected by changes
bull Limit impact on how traffic enters the networkndash Avoid new routes that might change neighborrsquos mindndash Select route with same attributes or at least path length
60
Managing Scalebull Destination prefixes
ndash More than 90000 destination prefixes
bull Donrsquot want to have per-prefix routing policies
ndash Small fraction of prefixes contribute most of the traffic
bull Focus on the small number of heavy hitters
ndash Define routing policies for selected prefixes
bull Routing choicesndash About 27000 unique ldquorouting choicesrdquo
bull Help in reducing the scale of the problem
ndash Small fraction of ldquorouting choicesrdquo contribute most traffic
bull Focus on the very small number of ldquorouting choicesrdquo
ndash Define routing policies on common attributes
61
Achieving Predictability
bull Route prediction with static analysisndash Helpful to know effects before deploymentndash Static analysis can help
TopologyBGP policy
configuration
eBGP routes
Offered traffic
BGP routingmodel
Flow of traffic through the network
62
Challenges to Predictabilitybull For transit ISPs effects on incoming traffic
ndash Lack of coordination strikes again
63
ldquoHot Potatordquo routing
Inter-AS Negotiation
bull Coordination aids predictabilityndash Negotiate where to sendndash Inbound and outboundndash Mutual benefits
bull How to implementndash What info to exchangendash Protecting privacyndash How to prioritize choicesndash How to prevent cheating
Destination 2
Destination 1
multiplepeeringpoints
Provider A
Provider B
64
Outbound Multihoming Goals
bull Redundancyndash Dynamic routing will failover to backup link
bull Performancendash Select provider with best performance per prefix
ndash Requires active probing
bull Costndash Select provider per prefix over time to minimize the
total financial cost
65
Inbound Traffic Control
bull More difficult no control over neighborsrsquo decisions
bull Three common techniques (previously discussed)ndash AS path prependingndash Communities and local preferencendash Prefix splitting
How does todayrsquos paper (MONET) control inbound traffic
66
How many links are enough
K upstream ISPs
Not much benefit beyond 4 ISPs
Akella et al ldquoPerformance Benefits of Multihomingrdquo SIGCOMM 2003
67
Problems with Multihoming in IPv4
bull Routing table growthndash Provider-based addressingndash Advertising prefix out multiple ISPs ndash canrsquot aggregate
bull Poor control over inbound trafficndash Existing mechanisms do not allow hosts to control
inbound traffic
68
GeorgiaTech
Internet Routing Overview
bull Intradomain (ie ldquointra-ASrdquo) routingbull Interdomain routing
Comcast
Abilene
ATampT Cogent
Autonomous Systems (ASes)
69
Configuration Problems ldquoAS 7007rdquoldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
UUNet
Florida InternetBarn
Sprint
70
Diagnosis and Troubleshootingldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
ldquoMicrosofts websites were offline for up to 23 hoursbecause of a [router] misconfigurationhellipit took nearly a day to determine what was wrong and undo the changesrdquo -- wiredcom January 25 2001
ldquoWorldCom Inchellipsuffered a widespread outage on its Internet backbone that affected roughly 20 percent of its US customer base The network problemshellipaffected millions of computer users worldwide A spokeswoman attributed the outage to a route table issue -- cnncom October 3 2002
A number of Covad customers went out from 5pm today due to supposedly a DDOS (distributed denial of service attack) on a key Level3 data center which later was described as a route leak (misconfiguration)ldquo
-- dslreportscom February 23 2004
71
Operator Mailing List (NANOG)Date Mon 18 Oct 2004 091515 -0700Subject Level 3 US east coast issues
Level 3 experiencing widespread unspecified routing issues on the US east coast Master ticket 1086844 Anyone have more specific information
Date Mon 18 Oct 2004 122034 -0400 (EDT)Subject Re Level 3 US east coast issues
Level 3 is currently experiencing a backbone outage causing routing instability and packet loss We are working to restore and will be sending hourly updateshellip
72
Operator Mailing List
0102030405060708090
Filtering RouteLeaks
RouteHijacks
RouteInstability
RoutingLoops
Blackholes
Occurr
ences o
ver
10 Y
ears
1995-1997 1998-2001 2001-2004
Note Only includes problems openly discussed on this list
Compare 83 power outages 1 fire
73
Routing Configuration
Ranking route selection
Dissemination internal route advertisement
Filtering route advertisement
Customer
Competitor
Primary
Backup
74
Internet Business Model (Simplified)
bull CustomerProvider One AS pays another for reachability to some set of destinations
bull ldquoSettlement-freerdquo Peering Bartering Two ASes exchange routes with one another
Provider
Peer
Customer
Preferences implemented with local preference manipulation
Destination
Pay to use
Get paid to use
Free to use
75
Peering Contracts Consistent Export
bull Rules of settlement-free peeringndash Advertise routes at all peering pointsndash Advertised routes must have equal ldquoAS path lengthrdquo
Sprint
ATampT
Enables ldquohot potatordquo routing
ldquoequally goodrdquoroutes
76
Consistent Export
bull Malicedeceptionbull iBGP signaling partitionbull Inconsistent export policy
neighbor 10123route-map PEER permit 10 set prepend 123
neighbor 10456route-map PEER permit 10 set prepend 123 123
Possible Causes
10123 456 1
10456 456 2
Neighbor AS Export
1 1 123
Export Clause Prepend
2 1 123 123
Two different Export Policies
77
Inconsistent Export in Practice
Feamster et al ldquoBorderGuard Detecting Cold Potatoes from Peersrdquo ACM IMC October 2004
78
Blackholes
Date Thu 18 Jul 2002 060510 -0400 (EDT)From Chad Oleary ltcolpoboxcomgtSubject Re problems with 701To ltnanogmeritedugt
Were starting to see the same issues with UUNet again Anyone elseseeing this Trying to reach Qwest
traceroute to 631461901 (631461901) 30 hops max 38 byte packets 1 esc-lp2-gwe-solutionscorpcom (631182201) 1167 ms 1163 ms 1142 ms 2 500Serial2-10GW1TPA2ALTERNET (1571301499) 1097 ms 1059 ms 1044 ms 3 161at-1-0-0XL4ATL1ALTERNET (1526381190) 13839 ms 14108 ms 16638 ms 4 0so-3-1-0XL2ATL5ALTERNET (152630238) 14370 ms 14587 ms 14553 ms 5 POS7-0BR2ATL5ALTERNET (1526382193) 13928 ms 14099 ms 14053 ms 6 7 hellip
79
Security
80
Security ldquoBogonrdquo Routes
Feamster et al ldquoAn Empirical Study of lsquoBogonrsquo Route Advertisementsrdquo ACM CCR January 2005
81
Spam Phishing etc
bull Unsolicited commercial emailbull As of about August 2008 estimates indicate that
about 95 of all email is spambull Common spam filtering techniques
ndash Content-based filtersndash DNS Blacklist (DNSBL) lookups Significant fraction of
todayrsquos DNS traffic
Can IP addresses from which spam is received be spoofed
82
Spam and Routing
83
Worms and Botnets
84
What is a Worm
bull Code that replicates and propagates across the networkndash Often carries a ldquopayloadrdquo
bull Usually spread via exploiting flaws in open servicesndash ldquoVirusesrdquo require user action to spread
bull First worm Robert Morris November 1988ndash 6-10 of all Internet hosts infected ()
bull Many more since but none on that scale until July 2001
85
Example Worm Code Red
bull Initial version July 13 2001
bull Exploited known ISAPI vulnerability in Microsoft IIS Web servers
bull 1st through 20th of each month spread20th through end of each month attack
bull Payload Web site defacementbull Scanning Random IP addressesbull Bug failure to seed random number generator
86
Code Red Revisions
bull Released July 19 2001
bull Payload flooding attack on wwwwhitehousegovndash Attack was mounted at the IP address of the Web site
bull Bug died after 20th of each month
bull Random number generator for IP scanning fixed
87
Code Red Host Infection Rate
Exponential infection rate
Measured using backscatter technique
88
Designing Fast-Spreading Worms
bull Hit-list scanningndash Time to infect first 10k hosts dominates infection timendash Solution Reconnaissance (stealthy scans etc)
bull Permutation scanningndash Observation Most scanning is redundantndash Idea Shared permutation of address space Start scanning
from own IP address Re-randomize when another infected machine is found
bull Internet-scale hit listsndash Flash worm complete infection within 30 seconds
89
Botnets
bull Bots Autonomous programs performing tasksbull Plenty of ldquobenignrdquo bots
ndash eg weatherbug
bull Botnets group of bots ndash Typically carries malicious connotationndash Large numbers of infected machinesndash Machines ldquoenlistedrdquo with infection vectors like worms
(last lecture)
bull Available for simultaneous control by a masterbull Size up to 350000 nodes (from todayrsquos paper)
90
ldquoRallyingrdquo the Botnet
bull Easy to combine worm backdoor functionalitybull Problem how to learn about successfully
infected machines
bull Optionsndash Emailndash Hard-coded email address
91
Botnet Control
bull Botnet master typically runs some IRC server on a well-known port (eg 6667)
bull Infected machine contacts botnet with pre-programmed DNS name (eg big-botde)
bull Dynamic DNS allows controller to move about freely
Infected Machine
DynamicDNS
BotnetController
(IRC server)
92
Some Defenses
93
Idea 1 Ingress Filtering
bull RFC 2827 Routers install filters to drop packets from networks that are not downstream
bull Feasible at edgesbull Difficult to configure closer to network ldquocorerdquo
20469207024 Internet
Drop all packets with source address other than 20469207024
94
Idea 2 uRPF Checks
bull Unicast Reverse Path Forwardingndash Cisco ldquoip verify unicast reverse-pathrdquo
bull Requires symmetric routing
Accept packet from interface only if forwarding table entry for source IP address matches ingress interface
100183
A10018
10015
101203
100181 24
10011 24
Strict Mode uRPF
Enabled
ldquoArdquo Routing TableDestination Next Hop1001024 Int 110018024 Int 2
100183 from wrong interface
95
Problems with uRPF
bull Asymmetric routing
96
S-BGP
bull Address-based PKI validate signaturesndash Authentication of
bull ownership for IP address blocks bull AS number bull an ASs identity and bull a BGP routers identity
ndash Use existing infrastructure (Internet registries etc)ndash Routing origination is digitally signedndash BGP updates are digitally signed
1048708 bull Route attestations A new optional BGP transitive path attribute
ndash carries digital signatures covering the routing information in updates
97
Practical Problems with S-BGP
bull Requires Public-Key Infrastructure
bull Lots of digital signatures to calculate and verifyndash Message overheadndash CPU overhead
bull Calculation expense is greatest when topology is changingndash Caching can help
bull Route aggregation is problematic (maybe thatrsquos OK)
bull Secure route withdrawals when link or node fails
bull Address ownership data out of date
bull Deployment
- Networking
- Goal of This Tutorial
- Todayrsquos Networks
- Business Models
- Billing for Internet Usage
- Net Neutrality
- Network Operations
- Point-of-Presence (PoP)
- Example Abilene Network Topology
- Another Example Backbone
- Internet Routing Overview
- Internet Routing Protocol BGP
- Two Flavors of BGP
- IPv4 Addresses Networks of Networks
- Pre-1994 Classful Addressing
- Classless Interdomain Routing (CIDR)
- Benefits of CIDR
- Growth of IP Prefixes
- 1994-1998 Linear Growth
- Around 2000 Fast Growth Resumes
- Fast growth resumes
- The Address Allocation Process
- Common Problems
- What can go wrong
- Measurement and Monitoring
- Passive vs Active Measurement
- Slide 27
- Passive Traffic Data Measurement
- Simple Network Management Protocol
- SNMP (Passive)
- Packet-level Monitoring
- Full Packet Capture (Passive)
- What is a flow
- Cisco NetFlow
- Flow Record Contents
- Aggregating Packets into Flows
- Reducing Measurement Overhead
- Packet Sampling for Flow Monitoring
- Sampling Flow-Level Sampling
- Two Main Approaches
- Packet Capture on High-Speed Links
- Characteristics of Packet Capture
- Data Measurement Repositories
- Multihoming and Traffic Engineering
- What is Multihoming
- Why Multihome
- Redundancy
- Performance
- Multihoming in IP Networks Today
- BGP or no
- Same Provider or Multiple
- Multihomed Stub One Link
- Multihomed Stub Multiple Links
- Multihomed Stub Multiple ISPs
- Multihomed Transit Network
- Interdomain Traffic Engineering
- Outbound Traffic Control
- Outbound Traffic Load Balancing
- Traffic Engineering Goals
- Managing Scale
- Achieving Predictability
- Challenges to Predictability
- Inter-AS Negotiation
- Outbound Multihoming Goals
- Inbound Traffic Control
- How many links are enough
- Problems with Multihoming in IPv4
- Slide 68
- Slide 69
- Diagnosis and Troubleshooting
- Operator Mailing List (NANOG)
- Operator Mailing List
- Routing Configuration
- Internet Business Model (Simplified)
- Peering Contracts Consistent Export
- Consistent Export
- Inconsistent Export in Practice
- Blackholes
- Security
- Security ldquoBogonrdquo Routes
- Spam Phishing etc
- Spam and Routing
- Worms and Botnets
- What is a Worm
- Example Worm Code Red
- Code Red Revisions
- Code Red Host Infection Rate
- Designing Fast-Spreading Worms
- Botnets
- ldquoRallyingrdquo the Botnet
- Botnet Control
- Some Defenses
- Idea 1 Ingress Filtering
- Idea 2 uRPF Checks
- Problems with uRPF
- S-BGP
- Practical Problems with S-BGP
-
50
BGP or no
bull Advantages of static routingndash Cheapersmaller routers (less true nowadays)ndash Simpler to configure
bull Advantages of BGPndash More control of your destiny (have providers stop
announcing you)ndash Fastermore intelligent selection of where to send
outbound packetsndash Better debugging of net problems (you can see the
Internet topology now)
51
Same Provider or Multiple
bull If your provider is reliable and fast and affordably and offers good tech-support you may want to multi-home initially to them via some backup path (slow is better than dead)
bull Eventually yoursquoll want to multi-home to different providers to avoid failure modes due to one providerrsquos architecture decisions
52
Multihomed Stub One Link
bull Downstream ISPrsquos routers configure default (ldquostaticrdquo) routes pointing to border router
bull Upstream ISP advertises reachability
Upstream ISP
Multiple links between same pair of routers
Default routes to ldquoborderrdquo
ldquoStubrdquoISP
53
Multihomed Stub Multiple Links
bull Use BGP to share loadbull Use private AS number (why is this OK)bull As before upstream ISP advertises prefix
Upstream ISP
Multiple links to different upstream routers
ldquoStubrdquoISP
Internal routing for ldquohot potatordquo
BGP for load balance at edge
54
Multihomed Stub Multiple ISPs
bull Many possibilitiesndash Load sharingndash Primary-backupndash Selective use of different ISPs
bull Requires BGP public AS number etc
ldquoStubrdquoISP
Upstream
ISP 1
Upstream
ISP 2
55
Multihomed Transit Network
bull BGP everywherebull Incoming and outcoming trafficbull Challenge balancing load on intradomain and egress
links given an offered traffic load
TransitISP
ISP 1
ISP 2
ISP 3
56
Interdomain Traffic Engineering
bull The process by which a network operator configures the network to achievendash Traffic load balancendash Redundancy (primarybackup) etc
bull Two tasksndash Outbound traffic controlndash Inbound traffic control
bull Key Problems Predictability and Scalability
57
Outbound Traffic Control
bull Easier to control than inbound trafficndash Destination-based routing sender determines where
the packets go
bull Control over next-hop AS onlyndash Cannot control selection of the entire path
Provider 1 Provider 2
Control with local preference
58
Outbound Traffic Load Balancingbull Control routes to provider per-prefix
ndash Assign local preference across destination prefixesndash Change the local preference assignments over time
bull Useful inputs to load balancingndash End-to-end path performance datandash Outbound traffic statistics per destination prefix
bull Challenge Getting from traffic volumes to groups of prefixes that should be assigned to each link
Premise of ldquointelligent route controlrdquo preoducts
59
Traffic Engineering Goals
bull Predictabilityndash Ensure the BGP decision process is deterministicndash Assume that BGP updates are (relatively) stable
bull Limit overhead introduced by routing changesndash Minimize frequency of changes to routing policiesndash Limit number of prefixes affected by changes
bull Limit impact on how traffic enters the networkndash Avoid new routes that might change neighborrsquos mindndash Select route with same attributes or at least path length
60
Managing Scalebull Destination prefixes
ndash More than 90000 destination prefixes
bull Donrsquot want to have per-prefix routing policies
ndash Small fraction of prefixes contribute most of the traffic
bull Focus on the small number of heavy hitters
ndash Define routing policies for selected prefixes
bull Routing choicesndash About 27000 unique ldquorouting choicesrdquo
bull Help in reducing the scale of the problem
ndash Small fraction of ldquorouting choicesrdquo contribute most traffic
bull Focus on the very small number of ldquorouting choicesrdquo
ndash Define routing policies on common attributes
61
Achieving Predictability
bull Route prediction with static analysisndash Helpful to know effects before deploymentndash Static analysis can help
TopologyBGP policy
configuration
eBGP routes
Offered traffic
BGP routingmodel
Flow of traffic through the network
62
Challenges to Predictabilitybull For transit ISPs effects on incoming traffic
ndash Lack of coordination strikes again
63
ldquoHot Potatordquo routing
Inter-AS Negotiation
bull Coordination aids predictabilityndash Negotiate where to sendndash Inbound and outboundndash Mutual benefits
bull How to implementndash What info to exchangendash Protecting privacyndash How to prioritize choicesndash How to prevent cheating
Destination 2
Destination 1
multiplepeeringpoints
Provider A
Provider B
64
Outbound Multihoming Goals
bull Redundancyndash Dynamic routing will failover to backup link
bull Performancendash Select provider with best performance per prefix
ndash Requires active probing
bull Costndash Select provider per prefix over time to minimize the
total financial cost
65
Inbound Traffic Control
bull More difficult no control over neighborsrsquo decisions
bull Three common techniques (previously discussed)ndash AS path prependingndash Communities and local preferencendash Prefix splitting
How does todayrsquos paper (MONET) control inbound traffic
66
How many links are enough
K upstream ISPs
Not much benefit beyond 4 ISPs
Akella et al ldquoPerformance Benefits of Multihomingrdquo SIGCOMM 2003
67
Problems with Multihoming in IPv4
bull Routing table growthndash Provider-based addressingndash Advertising prefix out multiple ISPs ndash canrsquot aggregate
bull Poor control over inbound trafficndash Existing mechanisms do not allow hosts to control
inbound traffic
68
GeorgiaTech
Internet Routing Overview
bull Intradomain (ie ldquointra-ASrdquo) routingbull Interdomain routing
Comcast
Abilene
ATampT Cogent
Autonomous Systems (ASes)
69
Configuration Problems ldquoAS 7007rdquoldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
UUNet
Florida InternetBarn
Sprint
70
Diagnosis and Troubleshootingldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
ldquoMicrosofts websites were offline for up to 23 hoursbecause of a [router] misconfigurationhellipit took nearly a day to determine what was wrong and undo the changesrdquo -- wiredcom January 25 2001
ldquoWorldCom Inchellipsuffered a widespread outage on its Internet backbone that affected roughly 20 percent of its US customer base The network problemshellipaffected millions of computer users worldwide A spokeswoman attributed the outage to a route table issue -- cnncom October 3 2002
A number of Covad customers went out from 5pm today due to supposedly a DDOS (distributed denial of service attack) on a key Level3 data center which later was described as a route leak (misconfiguration)ldquo
-- dslreportscom February 23 2004
71
Operator Mailing List (NANOG)Date Mon 18 Oct 2004 091515 -0700Subject Level 3 US east coast issues
Level 3 experiencing widespread unspecified routing issues on the US east coast Master ticket 1086844 Anyone have more specific information
Date Mon 18 Oct 2004 122034 -0400 (EDT)Subject Re Level 3 US east coast issues
Level 3 is currently experiencing a backbone outage causing routing instability and packet loss We are working to restore and will be sending hourly updateshellip
72
Operator Mailing List
0102030405060708090
Filtering RouteLeaks
RouteHijacks
RouteInstability
RoutingLoops
Blackholes
Occurr
ences o
ver
10 Y
ears
1995-1997 1998-2001 2001-2004
Note Only includes problems openly discussed on this list
Compare 83 power outages 1 fire
73
Routing Configuration
Ranking route selection
Dissemination internal route advertisement
Filtering route advertisement
Customer
Competitor
Primary
Backup
74
Internet Business Model (Simplified)
bull CustomerProvider One AS pays another for reachability to some set of destinations
bull ldquoSettlement-freerdquo Peering Bartering Two ASes exchange routes with one another
Provider
Peer
Customer
Preferences implemented with local preference manipulation
Destination
Pay to use
Get paid to use
Free to use
75
Peering Contracts Consistent Export
bull Rules of settlement-free peeringndash Advertise routes at all peering pointsndash Advertised routes must have equal ldquoAS path lengthrdquo
Sprint
ATampT
Enables ldquohot potatordquo routing
ldquoequally goodrdquoroutes
76
Consistent Export
bull Malicedeceptionbull iBGP signaling partitionbull Inconsistent export policy
neighbor 10123route-map PEER permit 10 set prepend 123
neighbor 10456route-map PEER permit 10 set prepend 123 123
Possible Causes
10123 456 1
10456 456 2
Neighbor AS Export
1 1 123
Export Clause Prepend
2 1 123 123
Two different Export Policies
77
Inconsistent Export in Practice
Feamster et al ldquoBorderGuard Detecting Cold Potatoes from Peersrdquo ACM IMC October 2004
78
Blackholes
Date Thu 18 Jul 2002 060510 -0400 (EDT)From Chad Oleary ltcolpoboxcomgtSubject Re problems with 701To ltnanogmeritedugt
Were starting to see the same issues with UUNet again Anyone elseseeing this Trying to reach Qwest
traceroute to 631461901 (631461901) 30 hops max 38 byte packets 1 esc-lp2-gwe-solutionscorpcom (631182201) 1167 ms 1163 ms 1142 ms 2 500Serial2-10GW1TPA2ALTERNET (1571301499) 1097 ms 1059 ms 1044 ms 3 161at-1-0-0XL4ATL1ALTERNET (1526381190) 13839 ms 14108 ms 16638 ms 4 0so-3-1-0XL2ATL5ALTERNET (152630238) 14370 ms 14587 ms 14553 ms 5 POS7-0BR2ATL5ALTERNET (1526382193) 13928 ms 14099 ms 14053 ms 6 7 hellip
79
Security
80
Security ldquoBogonrdquo Routes
Feamster et al ldquoAn Empirical Study of lsquoBogonrsquo Route Advertisementsrdquo ACM CCR January 2005
81
Spam Phishing etc
bull Unsolicited commercial emailbull As of about August 2008 estimates indicate that
about 95 of all email is spambull Common spam filtering techniques
ndash Content-based filtersndash DNS Blacklist (DNSBL) lookups Significant fraction of
todayrsquos DNS traffic
Can IP addresses from which spam is received be spoofed
82
Spam and Routing
83
Worms and Botnets
84
What is a Worm
bull Code that replicates and propagates across the networkndash Often carries a ldquopayloadrdquo
bull Usually spread via exploiting flaws in open servicesndash ldquoVirusesrdquo require user action to spread
bull First worm Robert Morris November 1988ndash 6-10 of all Internet hosts infected ()
bull Many more since but none on that scale until July 2001
85
Example Worm Code Red
bull Initial version July 13 2001
bull Exploited known ISAPI vulnerability in Microsoft IIS Web servers
bull 1st through 20th of each month spread20th through end of each month attack
bull Payload Web site defacementbull Scanning Random IP addressesbull Bug failure to seed random number generator
86
Code Red Revisions
bull Released July 19 2001
bull Payload flooding attack on wwwwhitehousegovndash Attack was mounted at the IP address of the Web site
bull Bug died after 20th of each month
bull Random number generator for IP scanning fixed
87
Code Red Host Infection Rate
Exponential infection rate
Measured using backscatter technique
88
Designing Fast-Spreading Worms
bull Hit-list scanningndash Time to infect first 10k hosts dominates infection timendash Solution Reconnaissance (stealthy scans etc)
bull Permutation scanningndash Observation Most scanning is redundantndash Idea Shared permutation of address space Start scanning
from own IP address Re-randomize when another infected machine is found
bull Internet-scale hit listsndash Flash worm complete infection within 30 seconds
89
Botnets
bull Bots Autonomous programs performing tasksbull Plenty of ldquobenignrdquo bots
ndash eg weatherbug
bull Botnets group of bots ndash Typically carries malicious connotationndash Large numbers of infected machinesndash Machines ldquoenlistedrdquo with infection vectors like worms
(last lecture)
bull Available for simultaneous control by a masterbull Size up to 350000 nodes (from todayrsquos paper)
90
ldquoRallyingrdquo the Botnet
bull Easy to combine worm backdoor functionalitybull Problem how to learn about successfully
infected machines
bull Optionsndash Emailndash Hard-coded email address
91
Botnet Control
bull Botnet master typically runs some IRC server on a well-known port (eg 6667)
bull Infected machine contacts botnet with pre-programmed DNS name (eg big-botde)
bull Dynamic DNS allows controller to move about freely
Infected Machine
DynamicDNS
BotnetController
(IRC server)
92
Some Defenses
93
Idea 1 Ingress Filtering
bull RFC 2827 Routers install filters to drop packets from networks that are not downstream
bull Feasible at edgesbull Difficult to configure closer to network ldquocorerdquo
20469207024 Internet
Drop all packets with source address other than 20469207024
94
Idea 2 uRPF Checks
bull Unicast Reverse Path Forwardingndash Cisco ldquoip verify unicast reverse-pathrdquo
bull Requires symmetric routing
Accept packet from interface only if forwarding table entry for source IP address matches ingress interface
100183
A10018
10015
101203
100181 24
10011 24
Strict Mode uRPF
Enabled
ldquoArdquo Routing TableDestination Next Hop1001024 Int 110018024 Int 2
100183 from wrong interface
95
Problems with uRPF
bull Asymmetric routing
96
S-BGP
bull Address-based PKI validate signaturesndash Authentication of
bull ownership for IP address blocks bull AS number bull an ASs identity and bull a BGP routers identity
ndash Use existing infrastructure (Internet registries etc)ndash Routing origination is digitally signedndash BGP updates are digitally signed
1048708 bull Route attestations A new optional BGP transitive path attribute
ndash carries digital signatures covering the routing information in updates
97
Practical Problems with S-BGP
bull Requires Public-Key Infrastructure
bull Lots of digital signatures to calculate and verifyndash Message overheadndash CPU overhead
bull Calculation expense is greatest when topology is changingndash Caching can help
bull Route aggregation is problematic (maybe thatrsquos OK)
bull Secure route withdrawals when link or node fails
bull Address ownership data out of date
bull Deployment
- Networking
- Goal of This Tutorial
- Todayrsquos Networks
- Business Models
- Billing for Internet Usage
- Net Neutrality
- Network Operations
- Point-of-Presence (PoP)
- Example Abilene Network Topology
- Another Example Backbone
- Internet Routing Overview
- Internet Routing Protocol BGP
- Two Flavors of BGP
- IPv4 Addresses Networks of Networks
- Pre-1994 Classful Addressing
- Classless Interdomain Routing (CIDR)
- Benefits of CIDR
- Growth of IP Prefixes
- 1994-1998 Linear Growth
- Around 2000 Fast Growth Resumes
- Fast growth resumes
- The Address Allocation Process
- Common Problems
- What can go wrong
- Measurement and Monitoring
- Passive vs Active Measurement
- Slide 27
- Passive Traffic Data Measurement
- Simple Network Management Protocol
- SNMP (Passive)
- Packet-level Monitoring
- Full Packet Capture (Passive)
- What is a flow
- Cisco NetFlow
- Flow Record Contents
- Aggregating Packets into Flows
- Reducing Measurement Overhead
- Packet Sampling for Flow Monitoring
- Sampling Flow-Level Sampling
- Two Main Approaches
- Packet Capture on High-Speed Links
- Characteristics of Packet Capture
- Data Measurement Repositories
- Multihoming and Traffic Engineering
- What is Multihoming
- Why Multihome
- Redundancy
- Performance
- Multihoming in IP Networks Today
- BGP or no
- Same Provider or Multiple
- Multihomed Stub One Link
- Multihomed Stub Multiple Links
- Multihomed Stub Multiple ISPs
- Multihomed Transit Network
- Interdomain Traffic Engineering
- Outbound Traffic Control
- Outbound Traffic Load Balancing
- Traffic Engineering Goals
- Managing Scale
- Achieving Predictability
- Challenges to Predictability
- Inter-AS Negotiation
- Outbound Multihoming Goals
- Inbound Traffic Control
- How many links are enough
- Problems with Multihoming in IPv4
- Slide 68
- Slide 69
- Diagnosis and Troubleshooting
- Operator Mailing List (NANOG)
- Operator Mailing List
- Routing Configuration
- Internet Business Model (Simplified)
- Peering Contracts Consistent Export
- Consistent Export
- Inconsistent Export in Practice
- Blackholes
- Security
- Security ldquoBogonrdquo Routes
- Spam Phishing etc
- Spam and Routing
- Worms and Botnets
- What is a Worm
- Example Worm Code Red
- Code Red Revisions
- Code Red Host Infection Rate
- Designing Fast-Spreading Worms
- Botnets
- ldquoRallyingrdquo the Botnet
- Botnet Control
- Some Defenses
- Idea 1 Ingress Filtering
- Idea 2 uRPF Checks
- Problems with uRPF
- S-BGP
- Practical Problems with S-BGP
-
51
Same Provider or Multiple
bull If your provider is reliable and fast and affordably and offers good tech-support you may want to multi-home initially to them via some backup path (slow is better than dead)
bull Eventually yoursquoll want to multi-home to different providers to avoid failure modes due to one providerrsquos architecture decisions
52
Multihomed Stub One Link
bull Downstream ISPrsquos routers configure default (ldquostaticrdquo) routes pointing to border router
bull Upstream ISP advertises reachability
Upstream ISP
Multiple links between same pair of routers
Default routes to ldquoborderrdquo
ldquoStubrdquoISP
53
Multihomed Stub Multiple Links
bull Use BGP to share loadbull Use private AS number (why is this OK)bull As before upstream ISP advertises prefix
Upstream ISP
Multiple links to different upstream routers
ldquoStubrdquoISP
Internal routing for ldquohot potatordquo
BGP for load balance at edge
54
Multihomed Stub Multiple ISPs
bull Many possibilitiesndash Load sharingndash Primary-backupndash Selective use of different ISPs
bull Requires BGP public AS number etc
ldquoStubrdquoISP
Upstream
ISP 1
Upstream
ISP 2
55
Multihomed Transit Network
bull BGP everywherebull Incoming and outcoming trafficbull Challenge balancing load on intradomain and egress
links given an offered traffic load
TransitISP
ISP 1
ISP 2
ISP 3
56
Interdomain Traffic Engineering
bull The process by which a network operator configures the network to achievendash Traffic load balancendash Redundancy (primarybackup) etc
bull Two tasksndash Outbound traffic controlndash Inbound traffic control
bull Key Problems Predictability and Scalability
57
Outbound Traffic Control
bull Easier to control than inbound trafficndash Destination-based routing sender determines where
the packets go
bull Control over next-hop AS onlyndash Cannot control selection of the entire path
Provider 1 Provider 2
Control with local preference
58
Outbound Traffic Load Balancingbull Control routes to provider per-prefix
ndash Assign local preference across destination prefixesndash Change the local preference assignments over time
bull Useful inputs to load balancingndash End-to-end path performance datandash Outbound traffic statistics per destination prefix
bull Challenge Getting from traffic volumes to groups of prefixes that should be assigned to each link
Premise of ldquointelligent route controlrdquo preoducts
59
Traffic Engineering Goals
bull Predictabilityndash Ensure the BGP decision process is deterministicndash Assume that BGP updates are (relatively) stable
bull Limit overhead introduced by routing changesndash Minimize frequency of changes to routing policiesndash Limit number of prefixes affected by changes
bull Limit impact on how traffic enters the networkndash Avoid new routes that might change neighborrsquos mindndash Select route with same attributes or at least path length
60
Managing Scalebull Destination prefixes
ndash More than 90000 destination prefixes
bull Donrsquot want to have per-prefix routing policies
ndash Small fraction of prefixes contribute most of the traffic
bull Focus on the small number of heavy hitters
ndash Define routing policies for selected prefixes
bull Routing choicesndash About 27000 unique ldquorouting choicesrdquo
bull Help in reducing the scale of the problem
ndash Small fraction of ldquorouting choicesrdquo contribute most traffic
bull Focus on the very small number of ldquorouting choicesrdquo
ndash Define routing policies on common attributes
61
Achieving Predictability
bull Route prediction with static analysisndash Helpful to know effects before deploymentndash Static analysis can help
TopologyBGP policy
configuration
eBGP routes
Offered traffic
BGP routingmodel
Flow of traffic through the network
62
Challenges to Predictabilitybull For transit ISPs effects on incoming traffic
ndash Lack of coordination strikes again
63
ldquoHot Potatordquo routing
Inter-AS Negotiation
bull Coordination aids predictabilityndash Negotiate where to sendndash Inbound and outboundndash Mutual benefits
bull How to implementndash What info to exchangendash Protecting privacyndash How to prioritize choicesndash How to prevent cheating
Destination 2
Destination 1
multiplepeeringpoints
Provider A
Provider B
64
Outbound Multihoming Goals
bull Redundancyndash Dynamic routing will failover to backup link
bull Performancendash Select provider with best performance per prefix
ndash Requires active probing
bull Costndash Select provider per prefix over time to minimize the
total financial cost
65
Inbound Traffic Control
bull More difficult no control over neighborsrsquo decisions
bull Three common techniques (previously discussed)ndash AS path prependingndash Communities and local preferencendash Prefix splitting
How does todayrsquos paper (MONET) control inbound traffic
66
How many links are enough
K upstream ISPs
Not much benefit beyond 4 ISPs
Akella et al ldquoPerformance Benefits of Multihomingrdquo SIGCOMM 2003
67
Problems with Multihoming in IPv4
bull Routing table growthndash Provider-based addressingndash Advertising prefix out multiple ISPs ndash canrsquot aggregate
bull Poor control over inbound trafficndash Existing mechanisms do not allow hosts to control
inbound traffic
68
GeorgiaTech
Internet Routing Overview
bull Intradomain (ie ldquointra-ASrdquo) routingbull Interdomain routing
Comcast
Abilene
ATampT Cogent
Autonomous Systems (ASes)
69
Configuration Problems ldquoAS 7007rdquoldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
UUNet
Florida InternetBarn
Sprint
70
Diagnosis and Troubleshootingldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
ldquoMicrosofts websites were offline for up to 23 hoursbecause of a [router] misconfigurationhellipit took nearly a day to determine what was wrong and undo the changesrdquo -- wiredcom January 25 2001
ldquoWorldCom Inchellipsuffered a widespread outage on its Internet backbone that affected roughly 20 percent of its US customer base The network problemshellipaffected millions of computer users worldwide A spokeswoman attributed the outage to a route table issue -- cnncom October 3 2002
A number of Covad customers went out from 5pm today due to supposedly a DDOS (distributed denial of service attack) on a key Level3 data center which later was described as a route leak (misconfiguration)ldquo
-- dslreportscom February 23 2004
71
Operator Mailing List (NANOG)Date Mon 18 Oct 2004 091515 -0700Subject Level 3 US east coast issues
Level 3 experiencing widespread unspecified routing issues on the US east coast Master ticket 1086844 Anyone have more specific information
Date Mon 18 Oct 2004 122034 -0400 (EDT)Subject Re Level 3 US east coast issues
Level 3 is currently experiencing a backbone outage causing routing instability and packet loss We are working to restore and will be sending hourly updateshellip
72
Operator Mailing List
0102030405060708090
Filtering RouteLeaks
RouteHijacks
RouteInstability
RoutingLoops
Blackholes
Occurr
ences o
ver
10 Y
ears
1995-1997 1998-2001 2001-2004
Note Only includes problems openly discussed on this list
Compare 83 power outages 1 fire
73
Routing Configuration
Ranking route selection
Dissemination internal route advertisement
Filtering route advertisement
Customer
Competitor
Primary
Backup
74
Internet Business Model (Simplified)
bull CustomerProvider One AS pays another for reachability to some set of destinations
bull ldquoSettlement-freerdquo Peering Bartering Two ASes exchange routes with one another
Provider
Peer
Customer
Preferences implemented with local preference manipulation
Destination
Pay to use
Get paid to use
Free to use
75
Peering Contracts Consistent Export
bull Rules of settlement-free peeringndash Advertise routes at all peering pointsndash Advertised routes must have equal ldquoAS path lengthrdquo
Sprint
ATampT
Enables ldquohot potatordquo routing
ldquoequally goodrdquoroutes
76
Consistent Export
bull Malicedeceptionbull iBGP signaling partitionbull Inconsistent export policy
neighbor 10123route-map PEER permit 10 set prepend 123
neighbor 10456route-map PEER permit 10 set prepend 123 123
Possible Causes
10123 456 1
10456 456 2
Neighbor AS Export
1 1 123
Export Clause Prepend
2 1 123 123
Two different Export Policies
77
Inconsistent Export in Practice
Feamster et al ldquoBorderGuard Detecting Cold Potatoes from Peersrdquo ACM IMC October 2004
78
Blackholes
Date Thu 18 Jul 2002 060510 -0400 (EDT)From Chad Oleary ltcolpoboxcomgtSubject Re problems with 701To ltnanogmeritedugt
Were starting to see the same issues with UUNet again Anyone elseseeing this Trying to reach Qwest
traceroute to 631461901 (631461901) 30 hops max 38 byte packets 1 esc-lp2-gwe-solutionscorpcom (631182201) 1167 ms 1163 ms 1142 ms 2 500Serial2-10GW1TPA2ALTERNET (1571301499) 1097 ms 1059 ms 1044 ms 3 161at-1-0-0XL4ATL1ALTERNET (1526381190) 13839 ms 14108 ms 16638 ms 4 0so-3-1-0XL2ATL5ALTERNET (152630238) 14370 ms 14587 ms 14553 ms 5 POS7-0BR2ATL5ALTERNET (1526382193) 13928 ms 14099 ms 14053 ms 6 7 hellip
79
Security
80
Security ldquoBogonrdquo Routes
Feamster et al ldquoAn Empirical Study of lsquoBogonrsquo Route Advertisementsrdquo ACM CCR January 2005
81
Spam Phishing etc
bull Unsolicited commercial emailbull As of about August 2008 estimates indicate that
about 95 of all email is spambull Common spam filtering techniques
ndash Content-based filtersndash DNS Blacklist (DNSBL) lookups Significant fraction of
todayrsquos DNS traffic
Can IP addresses from which spam is received be spoofed
82
Spam and Routing
83
Worms and Botnets
84
What is a Worm
bull Code that replicates and propagates across the networkndash Often carries a ldquopayloadrdquo
bull Usually spread via exploiting flaws in open servicesndash ldquoVirusesrdquo require user action to spread
bull First worm Robert Morris November 1988ndash 6-10 of all Internet hosts infected ()
bull Many more since but none on that scale until July 2001
85
Example Worm Code Red
bull Initial version July 13 2001
bull Exploited known ISAPI vulnerability in Microsoft IIS Web servers
bull 1st through 20th of each month spread20th through end of each month attack
bull Payload Web site defacementbull Scanning Random IP addressesbull Bug failure to seed random number generator
86
Code Red Revisions
bull Released July 19 2001
bull Payload flooding attack on wwwwhitehousegovndash Attack was mounted at the IP address of the Web site
bull Bug died after 20th of each month
bull Random number generator for IP scanning fixed
87
Code Red Host Infection Rate
Exponential infection rate
Measured using backscatter technique
88
Designing Fast-Spreading Worms
bull Hit-list scanningndash Time to infect first 10k hosts dominates infection timendash Solution Reconnaissance (stealthy scans etc)
bull Permutation scanningndash Observation Most scanning is redundantndash Idea Shared permutation of address space Start scanning
from own IP address Re-randomize when another infected machine is found
bull Internet-scale hit listsndash Flash worm complete infection within 30 seconds
89
Botnets
bull Bots Autonomous programs performing tasksbull Plenty of ldquobenignrdquo bots
ndash eg weatherbug
bull Botnets group of bots ndash Typically carries malicious connotationndash Large numbers of infected machinesndash Machines ldquoenlistedrdquo with infection vectors like worms
(last lecture)
bull Available for simultaneous control by a masterbull Size up to 350000 nodes (from todayrsquos paper)
90
ldquoRallyingrdquo the Botnet
bull Easy to combine worm backdoor functionalitybull Problem how to learn about successfully
infected machines
bull Optionsndash Emailndash Hard-coded email address
91
Botnet Control
bull Botnet master typically runs some IRC server on a well-known port (eg 6667)
bull Infected machine contacts botnet with pre-programmed DNS name (eg big-botde)
bull Dynamic DNS allows controller to move about freely
Infected Machine
DynamicDNS
BotnetController
(IRC server)
92
Some Defenses
93
Idea 1 Ingress Filtering
bull RFC 2827 Routers install filters to drop packets from networks that are not downstream
bull Feasible at edgesbull Difficult to configure closer to network ldquocorerdquo
20469207024 Internet
Drop all packets with source address other than 20469207024
94
Idea 2 uRPF Checks
bull Unicast Reverse Path Forwardingndash Cisco ldquoip verify unicast reverse-pathrdquo
bull Requires symmetric routing
Accept packet from interface only if forwarding table entry for source IP address matches ingress interface
100183
A10018
10015
101203
100181 24
10011 24
Strict Mode uRPF
Enabled
ldquoArdquo Routing TableDestination Next Hop1001024 Int 110018024 Int 2
100183 from wrong interface
95
Problems with uRPF
bull Asymmetric routing
96
S-BGP
bull Address-based PKI validate signaturesndash Authentication of
bull ownership for IP address blocks bull AS number bull an ASs identity and bull a BGP routers identity
ndash Use existing infrastructure (Internet registries etc)ndash Routing origination is digitally signedndash BGP updates are digitally signed
1048708 bull Route attestations A new optional BGP transitive path attribute
ndash carries digital signatures covering the routing information in updates
97
Practical Problems with S-BGP
bull Requires Public-Key Infrastructure
bull Lots of digital signatures to calculate and verifyndash Message overheadndash CPU overhead
bull Calculation expense is greatest when topology is changingndash Caching can help
bull Route aggregation is problematic (maybe thatrsquos OK)
bull Secure route withdrawals when link or node fails
bull Address ownership data out of date
bull Deployment
- Networking
- Goal of This Tutorial
- Todayrsquos Networks
- Business Models
- Billing for Internet Usage
- Net Neutrality
- Network Operations
- Point-of-Presence (PoP)
- Example Abilene Network Topology
- Another Example Backbone
- Internet Routing Overview
- Internet Routing Protocol BGP
- Two Flavors of BGP
- IPv4 Addresses Networks of Networks
- Pre-1994 Classful Addressing
- Classless Interdomain Routing (CIDR)
- Benefits of CIDR
- Growth of IP Prefixes
- 1994-1998 Linear Growth
- Around 2000 Fast Growth Resumes
- Fast growth resumes
- The Address Allocation Process
- Common Problems
- What can go wrong
- Measurement and Monitoring
- Passive vs Active Measurement
- Slide 27
- Passive Traffic Data Measurement
- Simple Network Management Protocol
- SNMP (Passive)
- Packet-level Monitoring
- Full Packet Capture (Passive)
- What is a flow
- Cisco NetFlow
- Flow Record Contents
- Aggregating Packets into Flows
- Reducing Measurement Overhead
- Packet Sampling for Flow Monitoring
- Sampling Flow-Level Sampling
- Two Main Approaches
- Packet Capture on High-Speed Links
- Characteristics of Packet Capture
- Data Measurement Repositories
- Multihoming and Traffic Engineering
- What is Multihoming
- Why Multihome
- Redundancy
- Performance
- Multihoming in IP Networks Today
- BGP or no
- Same Provider or Multiple
- Multihomed Stub One Link
- Multihomed Stub Multiple Links
- Multihomed Stub Multiple ISPs
- Multihomed Transit Network
- Interdomain Traffic Engineering
- Outbound Traffic Control
- Outbound Traffic Load Balancing
- Traffic Engineering Goals
- Managing Scale
- Achieving Predictability
- Challenges to Predictability
- Inter-AS Negotiation
- Outbound Multihoming Goals
- Inbound Traffic Control
- How many links are enough
- Problems with Multihoming in IPv4
- Slide 68
- Slide 69
- Diagnosis and Troubleshooting
- Operator Mailing List (NANOG)
- Operator Mailing List
- Routing Configuration
- Internet Business Model (Simplified)
- Peering Contracts Consistent Export
- Consistent Export
- Inconsistent Export in Practice
- Blackholes
- Security
- Security ldquoBogonrdquo Routes
- Spam Phishing etc
- Spam and Routing
- Worms and Botnets
- What is a Worm
- Example Worm Code Red
- Code Red Revisions
- Code Red Host Infection Rate
- Designing Fast-Spreading Worms
- Botnets
- ldquoRallyingrdquo the Botnet
- Botnet Control
- Some Defenses
- Idea 1 Ingress Filtering
- Idea 2 uRPF Checks
- Problems with uRPF
- S-BGP
- Practical Problems with S-BGP
-
52
Multihomed Stub One Link
bull Downstream ISPrsquos routers configure default (ldquostaticrdquo) routes pointing to border router
bull Upstream ISP advertises reachability
Upstream ISP
Multiple links between same pair of routers
Default routes to ldquoborderrdquo
ldquoStubrdquoISP
53
Multihomed Stub Multiple Links
bull Use BGP to share loadbull Use private AS number (why is this OK)bull As before upstream ISP advertises prefix
Upstream ISP
Multiple links to different upstream routers
ldquoStubrdquoISP
Internal routing for ldquohot potatordquo
BGP for load balance at edge
54
Multihomed Stub Multiple ISPs
bull Many possibilitiesndash Load sharingndash Primary-backupndash Selective use of different ISPs
bull Requires BGP public AS number etc
ldquoStubrdquoISP
Upstream
ISP 1
Upstream
ISP 2
55
Multihomed Transit Network
bull BGP everywherebull Incoming and outcoming trafficbull Challenge balancing load on intradomain and egress
links given an offered traffic load
TransitISP
ISP 1
ISP 2
ISP 3
56
Interdomain Traffic Engineering
bull The process by which a network operator configures the network to achievendash Traffic load balancendash Redundancy (primarybackup) etc
bull Two tasksndash Outbound traffic controlndash Inbound traffic control
bull Key Problems Predictability and Scalability
57
Outbound Traffic Control
bull Easier to control than inbound trafficndash Destination-based routing sender determines where
the packets go
bull Control over next-hop AS onlyndash Cannot control selection of the entire path
Provider 1 Provider 2
Control with local preference
58
Outbound Traffic Load Balancingbull Control routes to provider per-prefix
ndash Assign local preference across destination prefixesndash Change the local preference assignments over time
bull Useful inputs to load balancingndash End-to-end path performance datandash Outbound traffic statistics per destination prefix
bull Challenge Getting from traffic volumes to groups of prefixes that should be assigned to each link
Premise of ldquointelligent route controlrdquo preoducts
59
Traffic Engineering Goals
bull Predictabilityndash Ensure the BGP decision process is deterministicndash Assume that BGP updates are (relatively) stable
bull Limit overhead introduced by routing changesndash Minimize frequency of changes to routing policiesndash Limit number of prefixes affected by changes
bull Limit impact on how traffic enters the networkndash Avoid new routes that might change neighborrsquos mindndash Select route with same attributes or at least path length
60
Managing Scalebull Destination prefixes
ndash More than 90000 destination prefixes
bull Donrsquot want to have per-prefix routing policies
ndash Small fraction of prefixes contribute most of the traffic
bull Focus on the small number of heavy hitters
ndash Define routing policies for selected prefixes
bull Routing choicesndash About 27000 unique ldquorouting choicesrdquo
bull Help in reducing the scale of the problem
ndash Small fraction of ldquorouting choicesrdquo contribute most traffic
bull Focus on the very small number of ldquorouting choicesrdquo
ndash Define routing policies on common attributes
61
Achieving Predictability
bull Route prediction with static analysisndash Helpful to know effects before deploymentndash Static analysis can help
TopologyBGP policy
configuration
eBGP routes
Offered traffic
BGP routingmodel
Flow of traffic through the network
62
Challenges to Predictabilitybull For transit ISPs effects on incoming traffic
ndash Lack of coordination strikes again
63
ldquoHot Potatordquo routing
Inter-AS Negotiation
bull Coordination aids predictabilityndash Negotiate where to sendndash Inbound and outboundndash Mutual benefits
bull How to implementndash What info to exchangendash Protecting privacyndash How to prioritize choicesndash How to prevent cheating
Destination 2
Destination 1
multiplepeeringpoints
Provider A
Provider B
64
Outbound Multihoming Goals
bull Redundancyndash Dynamic routing will failover to backup link
bull Performancendash Select provider with best performance per prefix
ndash Requires active probing
bull Costndash Select provider per prefix over time to minimize the
total financial cost
65
Inbound Traffic Control
bull More difficult no control over neighborsrsquo decisions
bull Three common techniques (previously discussed)ndash AS path prependingndash Communities and local preferencendash Prefix splitting
How does todayrsquos paper (MONET) control inbound traffic
66
How many links are enough
K upstream ISPs
Not much benefit beyond 4 ISPs
Akella et al ldquoPerformance Benefits of Multihomingrdquo SIGCOMM 2003
67
Problems with Multihoming in IPv4
bull Routing table growthndash Provider-based addressingndash Advertising prefix out multiple ISPs ndash canrsquot aggregate
bull Poor control over inbound trafficndash Existing mechanisms do not allow hosts to control
inbound traffic
68
GeorgiaTech
Internet Routing Overview
bull Intradomain (ie ldquointra-ASrdquo) routingbull Interdomain routing
Comcast
Abilene
ATampT Cogent
Autonomous Systems (ASes)
69
Configuration Problems ldquoAS 7007rdquoldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
UUNet
Florida InternetBarn
Sprint
70
Diagnosis and Troubleshootingldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
ldquoMicrosofts websites were offline for up to 23 hoursbecause of a [router] misconfigurationhellipit took nearly a day to determine what was wrong and undo the changesrdquo -- wiredcom January 25 2001
ldquoWorldCom Inchellipsuffered a widespread outage on its Internet backbone that affected roughly 20 percent of its US customer base The network problemshellipaffected millions of computer users worldwide A spokeswoman attributed the outage to a route table issue -- cnncom October 3 2002
A number of Covad customers went out from 5pm today due to supposedly a DDOS (distributed denial of service attack) on a key Level3 data center which later was described as a route leak (misconfiguration)ldquo
-- dslreportscom February 23 2004
71
Operator Mailing List (NANOG)Date Mon 18 Oct 2004 091515 -0700Subject Level 3 US east coast issues
Level 3 experiencing widespread unspecified routing issues on the US east coast Master ticket 1086844 Anyone have more specific information
Date Mon 18 Oct 2004 122034 -0400 (EDT)Subject Re Level 3 US east coast issues
Level 3 is currently experiencing a backbone outage causing routing instability and packet loss We are working to restore and will be sending hourly updateshellip
72
Operator Mailing List
0102030405060708090
Filtering RouteLeaks
RouteHijacks
RouteInstability
RoutingLoops
Blackholes
Occurr
ences o
ver
10 Y
ears
1995-1997 1998-2001 2001-2004
Note Only includes problems openly discussed on this list
Compare 83 power outages 1 fire
73
Routing Configuration
Ranking route selection
Dissemination internal route advertisement
Filtering route advertisement
Customer
Competitor
Primary
Backup
74
Internet Business Model (Simplified)
bull CustomerProvider One AS pays another for reachability to some set of destinations
bull ldquoSettlement-freerdquo Peering Bartering Two ASes exchange routes with one another
Provider
Peer
Customer
Preferences implemented with local preference manipulation
Destination
Pay to use
Get paid to use
Free to use
75
Peering Contracts Consistent Export
bull Rules of settlement-free peeringndash Advertise routes at all peering pointsndash Advertised routes must have equal ldquoAS path lengthrdquo
Sprint
ATampT
Enables ldquohot potatordquo routing
ldquoequally goodrdquoroutes
76
Consistent Export
bull Malicedeceptionbull iBGP signaling partitionbull Inconsistent export policy
neighbor 10123route-map PEER permit 10 set prepend 123
neighbor 10456route-map PEER permit 10 set prepend 123 123
Possible Causes
10123 456 1
10456 456 2
Neighbor AS Export
1 1 123
Export Clause Prepend
2 1 123 123
Two different Export Policies
77
Inconsistent Export in Practice
Feamster et al ldquoBorderGuard Detecting Cold Potatoes from Peersrdquo ACM IMC October 2004
78
Blackholes
Date Thu 18 Jul 2002 060510 -0400 (EDT)From Chad Oleary ltcolpoboxcomgtSubject Re problems with 701To ltnanogmeritedugt
Were starting to see the same issues with UUNet again Anyone elseseeing this Trying to reach Qwest
traceroute to 631461901 (631461901) 30 hops max 38 byte packets 1 esc-lp2-gwe-solutionscorpcom (631182201) 1167 ms 1163 ms 1142 ms 2 500Serial2-10GW1TPA2ALTERNET (1571301499) 1097 ms 1059 ms 1044 ms 3 161at-1-0-0XL4ATL1ALTERNET (1526381190) 13839 ms 14108 ms 16638 ms 4 0so-3-1-0XL2ATL5ALTERNET (152630238) 14370 ms 14587 ms 14553 ms 5 POS7-0BR2ATL5ALTERNET (1526382193) 13928 ms 14099 ms 14053 ms 6 7 hellip
79
Security
80
Security ldquoBogonrdquo Routes
Feamster et al ldquoAn Empirical Study of lsquoBogonrsquo Route Advertisementsrdquo ACM CCR January 2005
81
Spam Phishing etc
bull Unsolicited commercial emailbull As of about August 2008 estimates indicate that
about 95 of all email is spambull Common spam filtering techniques
ndash Content-based filtersndash DNS Blacklist (DNSBL) lookups Significant fraction of
todayrsquos DNS traffic
Can IP addresses from which spam is received be spoofed
82
Spam and Routing
83
Worms and Botnets
84
What is a Worm
bull Code that replicates and propagates across the networkndash Often carries a ldquopayloadrdquo
bull Usually spread via exploiting flaws in open servicesndash ldquoVirusesrdquo require user action to spread
bull First worm Robert Morris November 1988ndash 6-10 of all Internet hosts infected ()
bull Many more since but none on that scale until July 2001
85
Example Worm Code Red
bull Initial version July 13 2001
bull Exploited known ISAPI vulnerability in Microsoft IIS Web servers
bull 1st through 20th of each month spread20th through end of each month attack
bull Payload Web site defacementbull Scanning Random IP addressesbull Bug failure to seed random number generator
86
Code Red Revisions
bull Released July 19 2001
bull Payload flooding attack on wwwwhitehousegovndash Attack was mounted at the IP address of the Web site
bull Bug died after 20th of each month
bull Random number generator for IP scanning fixed
87
Code Red Host Infection Rate
Exponential infection rate
Measured using backscatter technique
88
Designing Fast-Spreading Worms
bull Hit-list scanningndash Time to infect first 10k hosts dominates infection timendash Solution Reconnaissance (stealthy scans etc)
bull Permutation scanningndash Observation Most scanning is redundantndash Idea Shared permutation of address space Start scanning
from own IP address Re-randomize when another infected machine is found
bull Internet-scale hit listsndash Flash worm complete infection within 30 seconds
89
Botnets
bull Bots Autonomous programs performing tasksbull Plenty of ldquobenignrdquo bots
ndash eg weatherbug
bull Botnets group of bots ndash Typically carries malicious connotationndash Large numbers of infected machinesndash Machines ldquoenlistedrdquo with infection vectors like worms
(last lecture)
bull Available for simultaneous control by a masterbull Size up to 350000 nodes (from todayrsquos paper)
90
ldquoRallyingrdquo the Botnet
bull Easy to combine worm backdoor functionalitybull Problem how to learn about successfully
infected machines
bull Optionsndash Emailndash Hard-coded email address
91
Botnet Control
bull Botnet master typically runs some IRC server on a well-known port (eg 6667)
bull Infected machine contacts botnet with pre-programmed DNS name (eg big-botde)
bull Dynamic DNS allows controller to move about freely
Infected Machine
DynamicDNS
BotnetController
(IRC server)
92
Some Defenses
93
Idea 1 Ingress Filtering
bull RFC 2827 Routers install filters to drop packets from networks that are not downstream
bull Feasible at edgesbull Difficult to configure closer to network ldquocorerdquo
20469207024 Internet
Drop all packets with source address other than 20469207024
94
Idea 2 uRPF Checks
bull Unicast Reverse Path Forwardingndash Cisco ldquoip verify unicast reverse-pathrdquo
bull Requires symmetric routing
Accept packet from interface only if forwarding table entry for source IP address matches ingress interface
100183
A10018
10015
101203
100181 24
10011 24
Strict Mode uRPF
Enabled
ldquoArdquo Routing TableDestination Next Hop1001024 Int 110018024 Int 2
100183 from wrong interface
95
Problems with uRPF
bull Asymmetric routing
96
S-BGP
bull Address-based PKI validate signaturesndash Authentication of
bull ownership for IP address blocks bull AS number bull an ASs identity and bull a BGP routers identity
ndash Use existing infrastructure (Internet registries etc)ndash Routing origination is digitally signedndash BGP updates are digitally signed
1048708 bull Route attestations A new optional BGP transitive path attribute
ndash carries digital signatures covering the routing information in updates
97
Practical Problems with S-BGP
bull Requires Public-Key Infrastructure
bull Lots of digital signatures to calculate and verifyndash Message overheadndash CPU overhead
bull Calculation expense is greatest when topology is changingndash Caching can help
bull Route aggregation is problematic (maybe thatrsquos OK)
bull Secure route withdrawals when link or node fails
bull Address ownership data out of date
bull Deployment
- Networking
- Goal of This Tutorial
- Todayrsquos Networks
- Business Models
- Billing for Internet Usage
- Net Neutrality
- Network Operations
- Point-of-Presence (PoP)
- Example Abilene Network Topology
- Another Example Backbone
- Internet Routing Overview
- Internet Routing Protocol BGP
- Two Flavors of BGP
- IPv4 Addresses Networks of Networks
- Pre-1994 Classful Addressing
- Classless Interdomain Routing (CIDR)
- Benefits of CIDR
- Growth of IP Prefixes
- 1994-1998 Linear Growth
- Around 2000 Fast Growth Resumes
- Fast growth resumes
- The Address Allocation Process
- Common Problems
- What can go wrong
- Measurement and Monitoring
- Passive vs Active Measurement
- Slide 27
- Passive Traffic Data Measurement
- Simple Network Management Protocol
- SNMP (Passive)
- Packet-level Monitoring
- Full Packet Capture (Passive)
- What is a flow
- Cisco NetFlow
- Flow Record Contents
- Aggregating Packets into Flows
- Reducing Measurement Overhead
- Packet Sampling for Flow Monitoring
- Sampling Flow-Level Sampling
- Two Main Approaches
- Packet Capture on High-Speed Links
- Characteristics of Packet Capture
- Data Measurement Repositories
- Multihoming and Traffic Engineering
- What is Multihoming
- Why Multihome
- Redundancy
- Performance
- Multihoming in IP Networks Today
- BGP or no
- Same Provider or Multiple
- Multihomed Stub One Link
- Multihomed Stub Multiple Links
- Multihomed Stub Multiple ISPs
- Multihomed Transit Network
- Interdomain Traffic Engineering
- Outbound Traffic Control
- Outbound Traffic Load Balancing
- Traffic Engineering Goals
- Managing Scale
- Achieving Predictability
- Challenges to Predictability
- Inter-AS Negotiation
- Outbound Multihoming Goals
- Inbound Traffic Control
- How many links are enough
- Problems with Multihoming in IPv4
- Slide 68
- Slide 69
- Diagnosis and Troubleshooting
- Operator Mailing List (NANOG)
- Operator Mailing List
- Routing Configuration
- Internet Business Model (Simplified)
- Peering Contracts Consistent Export
- Consistent Export
- Inconsistent Export in Practice
- Blackholes
- Security
- Security ldquoBogonrdquo Routes
- Spam Phishing etc
- Spam and Routing
- Worms and Botnets
- What is a Worm
- Example Worm Code Red
- Code Red Revisions
- Code Red Host Infection Rate
- Designing Fast-Spreading Worms
- Botnets
- ldquoRallyingrdquo the Botnet
- Botnet Control
- Some Defenses
- Idea 1 Ingress Filtering
- Idea 2 uRPF Checks
- Problems with uRPF
- S-BGP
- Practical Problems with S-BGP
-
53
Multihomed Stub Multiple Links
bull Use BGP to share loadbull Use private AS number (why is this OK)bull As before upstream ISP advertises prefix
Upstream ISP
Multiple links to different upstream routers
ldquoStubrdquoISP
Internal routing for ldquohot potatordquo
BGP for load balance at edge
54
Multihomed Stub Multiple ISPs
bull Many possibilitiesndash Load sharingndash Primary-backupndash Selective use of different ISPs
bull Requires BGP public AS number etc
ldquoStubrdquoISP
Upstream
ISP 1
Upstream
ISP 2
55
Multihomed Transit Network
bull BGP everywherebull Incoming and outcoming trafficbull Challenge balancing load on intradomain and egress
links given an offered traffic load
TransitISP
ISP 1
ISP 2
ISP 3
56
Interdomain Traffic Engineering
bull The process by which a network operator configures the network to achievendash Traffic load balancendash Redundancy (primarybackup) etc
bull Two tasksndash Outbound traffic controlndash Inbound traffic control
bull Key Problems Predictability and Scalability
57
Outbound Traffic Control
bull Easier to control than inbound trafficndash Destination-based routing sender determines where
the packets go
bull Control over next-hop AS onlyndash Cannot control selection of the entire path
Provider 1 Provider 2
Control with local preference
58
Outbound Traffic Load Balancingbull Control routes to provider per-prefix
ndash Assign local preference across destination prefixesndash Change the local preference assignments over time
bull Useful inputs to load balancingndash End-to-end path performance datandash Outbound traffic statistics per destination prefix
bull Challenge Getting from traffic volumes to groups of prefixes that should be assigned to each link
Premise of ldquointelligent route controlrdquo preoducts
59
Traffic Engineering Goals
bull Predictabilityndash Ensure the BGP decision process is deterministicndash Assume that BGP updates are (relatively) stable
bull Limit overhead introduced by routing changesndash Minimize frequency of changes to routing policiesndash Limit number of prefixes affected by changes
bull Limit impact on how traffic enters the networkndash Avoid new routes that might change neighborrsquos mindndash Select route with same attributes or at least path length
60
Managing Scalebull Destination prefixes
ndash More than 90000 destination prefixes
bull Donrsquot want to have per-prefix routing policies
ndash Small fraction of prefixes contribute most of the traffic
bull Focus on the small number of heavy hitters
ndash Define routing policies for selected prefixes
bull Routing choicesndash About 27000 unique ldquorouting choicesrdquo
bull Help in reducing the scale of the problem
ndash Small fraction of ldquorouting choicesrdquo contribute most traffic
bull Focus on the very small number of ldquorouting choicesrdquo
ndash Define routing policies on common attributes
61
Achieving Predictability
bull Route prediction with static analysisndash Helpful to know effects before deploymentndash Static analysis can help
TopologyBGP policy
configuration
eBGP routes
Offered traffic
BGP routingmodel
Flow of traffic through the network
62
Challenges to Predictabilitybull For transit ISPs effects on incoming traffic
ndash Lack of coordination strikes again
63
ldquoHot Potatordquo routing
Inter-AS Negotiation
bull Coordination aids predictabilityndash Negotiate where to sendndash Inbound and outboundndash Mutual benefits
bull How to implementndash What info to exchangendash Protecting privacyndash How to prioritize choicesndash How to prevent cheating
Destination 2
Destination 1
multiplepeeringpoints
Provider A
Provider B
64
Outbound Multihoming Goals
bull Redundancyndash Dynamic routing will failover to backup link
bull Performancendash Select provider with best performance per prefix
ndash Requires active probing
bull Costndash Select provider per prefix over time to minimize the
total financial cost
65
Inbound Traffic Control
bull More difficult no control over neighborsrsquo decisions
bull Three common techniques (previously discussed)ndash AS path prependingndash Communities and local preferencendash Prefix splitting
How does todayrsquos paper (MONET) control inbound traffic
66
How many links are enough
K upstream ISPs
Not much benefit beyond 4 ISPs
Akella et al ldquoPerformance Benefits of Multihomingrdquo SIGCOMM 2003
67
Problems with Multihoming in IPv4
bull Routing table growthndash Provider-based addressingndash Advertising prefix out multiple ISPs ndash canrsquot aggregate
bull Poor control over inbound trafficndash Existing mechanisms do not allow hosts to control
inbound traffic
68
GeorgiaTech
Internet Routing Overview
bull Intradomain (ie ldquointra-ASrdquo) routingbull Interdomain routing
Comcast
Abilene
ATampT Cogent
Autonomous Systems (ASes)
69
Configuration Problems ldquoAS 7007rdquoldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
UUNet
Florida InternetBarn
Sprint
70
Diagnosis and Troubleshootingldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
ldquoMicrosofts websites were offline for up to 23 hoursbecause of a [router] misconfigurationhellipit took nearly a day to determine what was wrong and undo the changesrdquo -- wiredcom January 25 2001
ldquoWorldCom Inchellipsuffered a widespread outage on its Internet backbone that affected roughly 20 percent of its US customer base The network problemshellipaffected millions of computer users worldwide A spokeswoman attributed the outage to a route table issue -- cnncom October 3 2002
A number of Covad customers went out from 5pm today due to supposedly a DDOS (distributed denial of service attack) on a key Level3 data center which later was described as a route leak (misconfiguration)ldquo
-- dslreportscom February 23 2004
71
Operator Mailing List (NANOG)Date Mon 18 Oct 2004 091515 -0700Subject Level 3 US east coast issues
Level 3 experiencing widespread unspecified routing issues on the US east coast Master ticket 1086844 Anyone have more specific information
Date Mon 18 Oct 2004 122034 -0400 (EDT)Subject Re Level 3 US east coast issues
Level 3 is currently experiencing a backbone outage causing routing instability and packet loss We are working to restore and will be sending hourly updateshellip
72
Operator Mailing List
0102030405060708090
Filtering RouteLeaks
RouteHijacks
RouteInstability
RoutingLoops
Blackholes
Occurr
ences o
ver
10 Y
ears
1995-1997 1998-2001 2001-2004
Note Only includes problems openly discussed on this list
Compare 83 power outages 1 fire
73
Routing Configuration
Ranking route selection
Dissemination internal route advertisement
Filtering route advertisement
Customer
Competitor
Primary
Backup
74
Internet Business Model (Simplified)
bull CustomerProvider One AS pays another for reachability to some set of destinations
bull ldquoSettlement-freerdquo Peering Bartering Two ASes exchange routes with one another
Provider
Peer
Customer
Preferences implemented with local preference manipulation
Destination
Pay to use
Get paid to use
Free to use
75
Peering Contracts Consistent Export
bull Rules of settlement-free peeringndash Advertise routes at all peering pointsndash Advertised routes must have equal ldquoAS path lengthrdquo
Sprint
ATampT
Enables ldquohot potatordquo routing
ldquoequally goodrdquoroutes
76
Consistent Export
bull Malicedeceptionbull iBGP signaling partitionbull Inconsistent export policy
neighbor 10123route-map PEER permit 10 set prepend 123
neighbor 10456route-map PEER permit 10 set prepend 123 123
Possible Causes
10123 456 1
10456 456 2
Neighbor AS Export
1 1 123
Export Clause Prepend
2 1 123 123
Two different Export Policies
77
Inconsistent Export in Practice
Feamster et al ldquoBorderGuard Detecting Cold Potatoes from Peersrdquo ACM IMC October 2004
78
Blackholes
Date Thu 18 Jul 2002 060510 -0400 (EDT)From Chad Oleary ltcolpoboxcomgtSubject Re problems with 701To ltnanogmeritedugt
Were starting to see the same issues with UUNet again Anyone elseseeing this Trying to reach Qwest
traceroute to 631461901 (631461901) 30 hops max 38 byte packets 1 esc-lp2-gwe-solutionscorpcom (631182201) 1167 ms 1163 ms 1142 ms 2 500Serial2-10GW1TPA2ALTERNET (1571301499) 1097 ms 1059 ms 1044 ms 3 161at-1-0-0XL4ATL1ALTERNET (1526381190) 13839 ms 14108 ms 16638 ms 4 0so-3-1-0XL2ATL5ALTERNET (152630238) 14370 ms 14587 ms 14553 ms 5 POS7-0BR2ATL5ALTERNET (1526382193) 13928 ms 14099 ms 14053 ms 6 7 hellip
79
Security
80
Security ldquoBogonrdquo Routes
Feamster et al ldquoAn Empirical Study of lsquoBogonrsquo Route Advertisementsrdquo ACM CCR January 2005
81
Spam Phishing etc
bull Unsolicited commercial emailbull As of about August 2008 estimates indicate that
about 95 of all email is spambull Common spam filtering techniques
ndash Content-based filtersndash DNS Blacklist (DNSBL) lookups Significant fraction of
todayrsquos DNS traffic
Can IP addresses from which spam is received be spoofed
82
Spam and Routing
83
Worms and Botnets
84
What is a Worm
bull Code that replicates and propagates across the networkndash Often carries a ldquopayloadrdquo
bull Usually spread via exploiting flaws in open servicesndash ldquoVirusesrdquo require user action to spread
bull First worm Robert Morris November 1988ndash 6-10 of all Internet hosts infected ()
bull Many more since but none on that scale until July 2001
85
Example Worm Code Red
bull Initial version July 13 2001
bull Exploited known ISAPI vulnerability in Microsoft IIS Web servers
bull 1st through 20th of each month spread20th through end of each month attack
bull Payload Web site defacementbull Scanning Random IP addressesbull Bug failure to seed random number generator
86
Code Red Revisions
bull Released July 19 2001
bull Payload flooding attack on wwwwhitehousegovndash Attack was mounted at the IP address of the Web site
bull Bug died after 20th of each month
bull Random number generator for IP scanning fixed
87
Code Red Host Infection Rate
Exponential infection rate
Measured using backscatter technique
88
Designing Fast-Spreading Worms
bull Hit-list scanningndash Time to infect first 10k hosts dominates infection timendash Solution Reconnaissance (stealthy scans etc)
bull Permutation scanningndash Observation Most scanning is redundantndash Idea Shared permutation of address space Start scanning
from own IP address Re-randomize when another infected machine is found
bull Internet-scale hit listsndash Flash worm complete infection within 30 seconds
89
Botnets
bull Bots Autonomous programs performing tasksbull Plenty of ldquobenignrdquo bots
ndash eg weatherbug
bull Botnets group of bots ndash Typically carries malicious connotationndash Large numbers of infected machinesndash Machines ldquoenlistedrdquo with infection vectors like worms
(last lecture)
bull Available for simultaneous control by a masterbull Size up to 350000 nodes (from todayrsquos paper)
90
ldquoRallyingrdquo the Botnet
bull Easy to combine worm backdoor functionalitybull Problem how to learn about successfully
infected machines
bull Optionsndash Emailndash Hard-coded email address
91
Botnet Control
bull Botnet master typically runs some IRC server on a well-known port (eg 6667)
bull Infected machine contacts botnet with pre-programmed DNS name (eg big-botde)
bull Dynamic DNS allows controller to move about freely
Infected Machine
DynamicDNS
BotnetController
(IRC server)
92
Some Defenses
93
Idea 1 Ingress Filtering
bull RFC 2827 Routers install filters to drop packets from networks that are not downstream
bull Feasible at edgesbull Difficult to configure closer to network ldquocorerdquo
20469207024 Internet
Drop all packets with source address other than 20469207024
94
Idea 2 uRPF Checks
bull Unicast Reverse Path Forwardingndash Cisco ldquoip verify unicast reverse-pathrdquo
bull Requires symmetric routing
Accept packet from interface only if forwarding table entry for source IP address matches ingress interface
100183
A10018
10015
101203
100181 24
10011 24
Strict Mode uRPF
Enabled
ldquoArdquo Routing TableDestination Next Hop1001024 Int 110018024 Int 2
100183 from wrong interface
95
Problems with uRPF
bull Asymmetric routing
96
S-BGP
bull Address-based PKI validate signaturesndash Authentication of
bull ownership for IP address blocks bull AS number bull an ASs identity and bull a BGP routers identity
ndash Use existing infrastructure (Internet registries etc)ndash Routing origination is digitally signedndash BGP updates are digitally signed
1048708 bull Route attestations A new optional BGP transitive path attribute
ndash carries digital signatures covering the routing information in updates
97
Practical Problems with S-BGP
bull Requires Public-Key Infrastructure
bull Lots of digital signatures to calculate and verifyndash Message overheadndash CPU overhead
bull Calculation expense is greatest when topology is changingndash Caching can help
bull Route aggregation is problematic (maybe thatrsquos OK)
bull Secure route withdrawals when link or node fails
bull Address ownership data out of date
bull Deployment
- Networking
- Goal of This Tutorial
- Todayrsquos Networks
- Business Models
- Billing for Internet Usage
- Net Neutrality
- Network Operations
- Point-of-Presence (PoP)
- Example Abilene Network Topology
- Another Example Backbone
- Internet Routing Overview
- Internet Routing Protocol BGP
- Two Flavors of BGP
- IPv4 Addresses Networks of Networks
- Pre-1994 Classful Addressing
- Classless Interdomain Routing (CIDR)
- Benefits of CIDR
- Growth of IP Prefixes
- 1994-1998 Linear Growth
- Around 2000 Fast Growth Resumes
- Fast growth resumes
- The Address Allocation Process
- Common Problems
- What can go wrong
- Measurement and Monitoring
- Passive vs Active Measurement
- Slide 27
- Passive Traffic Data Measurement
- Simple Network Management Protocol
- SNMP (Passive)
- Packet-level Monitoring
- Full Packet Capture (Passive)
- What is a flow
- Cisco NetFlow
- Flow Record Contents
- Aggregating Packets into Flows
- Reducing Measurement Overhead
- Packet Sampling for Flow Monitoring
- Sampling Flow-Level Sampling
- Two Main Approaches
- Packet Capture on High-Speed Links
- Characteristics of Packet Capture
- Data Measurement Repositories
- Multihoming and Traffic Engineering
- What is Multihoming
- Why Multihome
- Redundancy
- Performance
- Multihoming in IP Networks Today
- BGP or no
- Same Provider or Multiple
- Multihomed Stub One Link
- Multihomed Stub Multiple Links
- Multihomed Stub Multiple ISPs
- Multihomed Transit Network
- Interdomain Traffic Engineering
- Outbound Traffic Control
- Outbound Traffic Load Balancing
- Traffic Engineering Goals
- Managing Scale
- Achieving Predictability
- Challenges to Predictability
- Inter-AS Negotiation
- Outbound Multihoming Goals
- Inbound Traffic Control
- How many links are enough
- Problems with Multihoming in IPv4
- Slide 68
- Slide 69
- Diagnosis and Troubleshooting
- Operator Mailing List (NANOG)
- Operator Mailing List
- Routing Configuration
- Internet Business Model (Simplified)
- Peering Contracts Consistent Export
- Consistent Export
- Inconsistent Export in Practice
- Blackholes
- Security
- Security ldquoBogonrdquo Routes
- Spam Phishing etc
- Spam and Routing
- Worms and Botnets
- What is a Worm
- Example Worm Code Red
- Code Red Revisions
- Code Red Host Infection Rate
- Designing Fast-Spreading Worms
- Botnets
- ldquoRallyingrdquo the Botnet
- Botnet Control
- Some Defenses
- Idea 1 Ingress Filtering
- Idea 2 uRPF Checks
- Problems with uRPF
- S-BGP
- Practical Problems with S-BGP
-
54
Multihomed Stub Multiple ISPs
bull Many possibilitiesndash Load sharingndash Primary-backupndash Selective use of different ISPs
bull Requires BGP public AS number etc
ldquoStubrdquoISP
Upstream
ISP 1
Upstream
ISP 2
55
Multihomed Transit Network
bull BGP everywherebull Incoming and outcoming trafficbull Challenge balancing load on intradomain and egress
links given an offered traffic load
TransitISP
ISP 1
ISP 2
ISP 3
56
Interdomain Traffic Engineering
bull The process by which a network operator configures the network to achievendash Traffic load balancendash Redundancy (primarybackup) etc
bull Two tasksndash Outbound traffic controlndash Inbound traffic control
bull Key Problems Predictability and Scalability
57
Outbound Traffic Control
bull Easier to control than inbound trafficndash Destination-based routing sender determines where
the packets go
bull Control over next-hop AS onlyndash Cannot control selection of the entire path
Provider 1 Provider 2
Control with local preference
58
Outbound Traffic Load Balancingbull Control routes to provider per-prefix
ndash Assign local preference across destination prefixesndash Change the local preference assignments over time
bull Useful inputs to load balancingndash End-to-end path performance datandash Outbound traffic statistics per destination prefix
bull Challenge Getting from traffic volumes to groups of prefixes that should be assigned to each link
Premise of ldquointelligent route controlrdquo preoducts
59
Traffic Engineering Goals
bull Predictabilityndash Ensure the BGP decision process is deterministicndash Assume that BGP updates are (relatively) stable
bull Limit overhead introduced by routing changesndash Minimize frequency of changes to routing policiesndash Limit number of prefixes affected by changes
bull Limit impact on how traffic enters the networkndash Avoid new routes that might change neighborrsquos mindndash Select route with same attributes or at least path length
60
Managing Scalebull Destination prefixes
ndash More than 90000 destination prefixes
bull Donrsquot want to have per-prefix routing policies
ndash Small fraction of prefixes contribute most of the traffic
bull Focus on the small number of heavy hitters
ndash Define routing policies for selected prefixes
bull Routing choicesndash About 27000 unique ldquorouting choicesrdquo
bull Help in reducing the scale of the problem
ndash Small fraction of ldquorouting choicesrdquo contribute most traffic
bull Focus on the very small number of ldquorouting choicesrdquo
ndash Define routing policies on common attributes
61
Achieving Predictability
bull Route prediction with static analysisndash Helpful to know effects before deploymentndash Static analysis can help
TopologyBGP policy
configuration
eBGP routes
Offered traffic
BGP routingmodel
Flow of traffic through the network
62
Challenges to Predictabilitybull For transit ISPs effects on incoming traffic
ndash Lack of coordination strikes again
63
ldquoHot Potatordquo routing
Inter-AS Negotiation
bull Coordination aids predictabilityndash Negotiate where to sendndash Inbound and outboundndash Mutual benefits
bull How to implementndash What info to exchangendash Protecting privacyndash How to prioritize choicesndash How to prevent cheating
Destination 2
Destination 1
multiplepeeringpoints
Provider A
Provider B
64
Outbound Multihoming Goals
bull Redundancyndash Dynamic routing will failover to backup link
bull Performancendash Select provider with best performance per prefix
ndash Requires active probing
bull Costndash Select provider per prefix over time to minimize the
total financial cost
65
Inbound Traffic Control
bull More difficult no control over neighborsrsquo decisions
bull Three common techniques (previously discussed)ndash AS path prependingndash Communities and local preferencendash Prefix splitting
How does todayrsquos paper (MONET) control inbound traffic
66
How many links are enough
K upstream ISPs
Not much benefit beyond 4 ISPs
Akella et al ldquoPerformance Benefits of Multihomingrdquo SIGCOMM 2003
67
Problems with Multihoming in IPv4
bull Routing table growthndash Provider-based addressingndash Advertising prefix out multiple ISPs ndash canrsquot aggregate
bull Poor control over inbound trafficndash Existing mechanisms do not allow hosts to control
inbound traffic
68
GeorgiaTech
Internet Routing Overview
bull Intradomain (ie ldquointra-ASrdquo) routingbull Interdomain routing
Comcast
Abilene
ATampT Cogent
Autonomous Systems (ASes)
69
Configuration Problems ldquoAS 7007rdquoldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
UUNet
Florida InternetBarn
Sprint
70
Diagnosis and Troubleshootingldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
ldquoMicrosofts websites were offline for up to 23 hoursbecause of a [router] misconfigurationhellipit took nearly a day to determine what was wrong and undo the changesrdquo -- wiredcom January 25 2001
ldquoWorldCom Inchellipsuffered a widespread outage on its Internet backbone that affected roughly 20 percent of its US customer base The network problemshellipaffected millions of computer users worldwide A spokeswoman attributed the outage to a route table issue -- cnncom October 3 2002
A number of Covad customers went out from 5pm today due to supposedly a DDOS (distributed denial of service attack) on a key Level3 data center which later was described as a route leak (misconfiguration)ldquo
-- dslreportscom February 23 2004
71
Operator Mailing List (NANOG)Date Mon 18 Oct 2004 091515 -0700Subject Level 3 US east coast issues
Level 3 experiencing widespread unspecified routing issues on the US east coast Master ticket 1086844 Anyone have more specific information
Date Mon 18 Oct 2004 122034 -0400 (EDT)Subject Re Level 3 US east coast issues
Level 3 is currently experiencing a backbone outage causing routing instability and packet loss We are working to restore and will be sending hourly updateshellip
72
Operator Mailing List
0102030405060708090
Filtering RouteLeaks
RouteHijacks
RouteInstability
RoutingLoops
Blackholes
Occurr
ences o
ver
10 Y
ears
1995-1997 1998-2001 2001-2004
Note Only includes problems openly discussed on this list
Compare 83 power outages 1 fire
73
Routing Configuration
Ranking route selection
Dissemination internal route advertisement
Filtering route advertisement
Customer
Competitor
Primary
Backup
74
Internet Business Model (Simplified)
bull CustomerProvider One AS pays another for reachability to some set of destinations
bull ldquoSettlement-freerdquo Peering Bartering Two ASes exchange routes with one another
Provider
Peer
Customer
Preferences implemented with local preference manipulation
Destination
Pay to use
Get paid to use
Free to use
75
Peering Contracts Consistent Export
bull Rules of settlement-free peeringndash Advertise routes at all peering pointsndash Advertised routes must have equal ldquoAS path lengthrdquo
Sprint
ATampT
Enables ldquohot potatordquo routing
ldquoequally goodrdquoroutes
76
Consistent Export
bull Malicedeceptionbull iBGP signaling partitionbull Inconsistent export policy
neighbor 10123route-map PEER permit 10 set prepend 123
neighbor 10456route-map PEER permit 10 set prepend 123 123
Possible Causes
10123 456 1
10456 456 2
Neighbor AS Export
1 1 123
Export Clause Prepend
2 1 123 123
Two different Export Policies
77
Inconsistent Export in Practice
Feamster et al ldquoBorderGuard Detecting Cold Potatoes from Peersrdquo ACM IMC October 2004
78
Blackholes
Date Thu 18 Jul 2002 060510 -0400 (EDT)From Chad Oleary ltcolpoboxcomgtSubject Re problems with 701To ltnanogmeritedugt
Were starting to see the same issues with UUNet again Anyone elseseeing this Trying to reach Qwest
traceroute to 631461901 (631461901) 30 hops max 38 byte packets 1 esc-lp2-gwe-solutionscorpcom (631182201) 1167 ms 1163 ms 1142 ms 2 500Serial2-10GW1TPA2ALTERNET (1571301499) 1097 ms 1059 ms 1044 ms 3 161at-1-0-0XL4ATL1ALTERNET (1526381190) 13839 ms 14108 ms 16638 ms 4 0so-3-1-0XL2ATL5ALTERNET (152630238) 14370 ms 14587 ms 14553 ms 5 POS7-0BR2ATL5ALTERNET (1526382193) 13928 ms 14099 ms 14053 ms 6 7 hellip
79
Security
80
Security ldquoBogonrdquo Routes
Feamster et al ldquoAn Empirical Study of lsquoBogonrsquo Route Advertisementsrdquo ACM CCR January 2005
81
Spam Phishing etc
bull Unsolicited commercial emailbull As of about August 2008 estimates indicate that
about 95 of all email is spambull Common spam filtering techniques
ndash Content-based filtersndash DNS Blacklist (DNSBL) lookups Significant fraction of
todayrsquos DNS traffic
Can IP addresses from which spam is received be spoofed
82
Spam and Routing
83
Worms and Botnets
84
What is a Worm
bull Code that replicates and propagates across the networkndash Often carries a ldquopayloadrdquo
bull Usually spread via exploiting flaws in open servicesndash ldquoVirusesrdquo require user action to spread
bull First worm Robert Morris November 1988ndash 6-10 of all Internet hosts infected ()
bull Many more since but none on that scale until July 2001
85
Example Worm Code Red
bull Initial version July 13 2001
bull Exploited known ISAPI vulnerability in Microsoft IIS Web servers
bull 1st through 20th of each month spread20th through end of each month attack
bull Payload Web site defacementbull Scanning Random IP addressesbull Bug failure to seed random number generator
86
Code Red Revisions
bull Released July 19 2001
bull Payload flooding attack on wwwwhitehousegovndash Attack was mounted at the IP address of the Web site
bull Bug died after 20th of each month
bull Random number generator for IP scanning fixed
87
Code Red Host Infection Rate
Exponential infection rate
Measured using backscatter technique
88
Designing Fast-Spreading Worms
bull Hit-list scanningndash Time to infect first 10k hosts dominates infection timendash Solution Reconnaissance (stealthy scans etc)
bull Permutation scanningndash Observation Most scanning is redundantndash Idea Shared permutation of address space Start scanning
from own IP address Re-randomize when another infected machine is found
bull Internet-scale hit listsndash Flash worm complete infection within 30 seconds
89
Botnets
bull Bots Autonomous programs performing tasksbull Plenty of ldquobenignrdquo bots
ndash eg weatherbug
bull Botnets group of bots ndash Typically carries malicious connotationndash Large numbers of infected machinesndash Machines ldquoenlistedrdquo with infection vectors like worms
(last lecture)
bull Available for simultaneous control by a masterbull Size up to 350000 nodes (from todayrsquos paper)
90
ldquoRallyingrdquo the Botnet
bull Easy to combine worm backdoor functionalitybull Problem how to learn about successfully
infected machines
bull Optionsndash Emailndash Hard-coded email address
91
Botnet Control
bull Botnet master typically runs some IRC server on a well-known port (eg 6667)
bull Infected machine contacts botnet with pre-programmed DNS name (eg big-botde)
bull Dynamic DNS allows controller to move about freely
Infected Machine
DynamicDNS
BotnetController
(IRC server)
92
Some Defenses
93
Idea 1 Ingress Filtering
bull RFC 2827 Routers install filters to drop packets from networks that are not downstream
bull Feasible at edgesbull Difficult to configure closer to network ldquocorerdquo
20469207024 Internet
Drop all packets with source address other than 20469207024
94
Idea 2 uRPF Checks
bull Unicast Reverse Path Forwardingndash Cisco ldquoip verify unicast reverse-pathrdquo
bull Requires symmetric routing
Accept packet from interface only if forwarding table entry for source IP address matches ingress interface
100183
A10018
10015
101203
100181 24
10011 24
Strict Mode uRPF
Enabled
ldquoArdquo Routing TableDestination Next Hop1001024 Int 110018024 Int 2
100183 from wrong interface
95
Problems with uRPF
bull Asymmetric routing
96
S-BGP
bull Address-based PKI validate signaturesndash Authentication of
bull ownership for IP address blocks bull AS number bull an ASs identity and bull a BGP routers identity
ndash Use existing infrastructure (Internet registries etc)ndash Routing origination is digitally signedndash BGP updates are digitally signed
1048708 bull Route attestations A new optional BGP transitive path attribute
ndash carries digital signatures covering the routing information in updates
97
Practical Problems with S-BGP
bull Requires Public-Key Infrastructure
bull Lots of digital signatures to calculate and verifyndash Message overheadndash CPU overhead
bull Calculation expense is greatest when topology is changingndash Caching can help
bull Route aggregation is problematic (maybe thatrsquos OK)
bull Secure route withdrawals when link or node fails
bull Address ownership data out of date
bull Deployment
- Networking
- Goal of This Tutorial
- Todayrsquos Networks
- Business Models
- Billing for Internet Usage
- Net Neutrality
- Network Operations
- Point-of-Presence (PoP)
- Example Abilene Network Topology
- Another Example Backbone
- Internet Routing Overview
- Internet Routing Protocol BGP
- Two Flavors of BGP
- IPv4 Addresses Networks of Networks
- Pre-1994 Classful Addressing
- Classless Interdomain Routing (CIDR)
- Benefits of CIDR
- Growth of IP Prefixes
- 1994-1998 Linear Growth
- Around 2000 Fast Growth Resumes
- Fast growth resumes
- The Address Allocation Process
- Common Problems
- What can go wrong
- Measurement and Monitoring
- Passive vs Active Measurement
- Slide 27
- Passive Traffic Data Measurement
- Simple Network Management Protocol
- SNMP (Passive)
- Packet-level Monitoring
- Full Packet Capture (Passive)
- What is a flow
- Cisco NetFlow
- Flow Record Contents
- Aggregating Packets into Flows
- Reducing Measurement Overhead
- Packet Sampling for Flow Monitoring
- Sampling Flow-Level Sampling
- Two Main Approaches
- Packet Capture on High-Speed Links
- Characteristics of Packet Capture
- Data Measurement Repositories
- Multihoming and Traffic Engineering
- What is Multihoming
- Why Multihome
- Redundancy
- Performance
- Multihoming in IP Networks Today
- BGP or no
- Same Provider or Multiple
- Multihomed Stub One Link
- Multihomed Stub Multiple Links
- Multihomed Stub Multiple ISPs
- Multihomed Transit Network
- Interdomain Traffic Engineering
- Outbound Traffic Control
- Outbound Traffic Load Balancing
- Traffic Engineering Goals
- Managing Scale
- Achieving Predictability
- Challenges to Predictability
- Inter-AS Negotiation
- Outbound Multihoming Goals
- Inbound Traffic Control
- How many links are enough
- Problems with Multihoming in IPv4
- Slide 68
- Slide 69
- Diagnosis and Troubleshooting
- Operator Mailing List (NANOG)
- Operator Mailing List
- Routing Configuration
- Internet Business Model (Simplified)
- Peering Contracts Consistent Export
- Consistent Export
- Inconsistent Export in Practice
- Blackholes
- Security
- Security ldquoBogonrdquo Routes
- Spam Phishing etc
- Spam and Routing
- Worms and Botnets
- What is a Worm
- Example Worm Code Red
- Code Red Revisions
- Code Red Host Infection Rate
- Designing Fast-Spreading Worms
- Botnets
- ldquoRallyingrdquo the Botnet
- Botnet Control
- Some Defenses
- Idea 1 Ingress Filtering
- Idea 2 uRPF Checks
- Problems with uRPF
- S-BGP
- Practical Problems with S-BGP
-
55
Multihomed Transit Network
bull BGP everywherebull Incoming and outcoming trafficbull Challenge balancing load on intradomain and egress
links given an offered traffic load
TransitISP
ISP 1
ISP 2
ISP 3
56
Interdomain Traffic Engineering
bull The process by which a network operator configures the network to achievendash Traffic load balancendash Redundancy (primarybackup) etc
bull Two tasksndash Outbound traffic controlndash Inbound traffic control
bull Key Problems Predictability and Scalability
57
Outbound Traffic Control
bull Easier to control than inbound trafficndash Destination-based routing sender determines where
the packets go
bull Control over next-hop AS onlyndash Cannot control selection of the entire path
Provider 1 Provider 2
Control with local preference
58
Outbound Traffic Load Balancingbull Control routes to provider per-prefix
ndash Assign local preference across destination prefixesndash Change the local preference assignments over time
bull Useful inputs to load balancingndash End-to-end path performance datandash Outbound traffic statistics per destination prefix
bull Challenge Getting from traffic volumes to groups of prefixes that should be assigned to each link
Premise of ldquointelligent route controlrdquo preoducts
59
Traffic Engineering Goals
bull Predictabilityndash Ensure the BGP decision process is deterministicndash Assume that BGP updates are (relatively) stable
bull Limit overhead introduced by routing changesndash Minimize frequency of changes to routing policiesndash Limit number of prefixes affected by changes
bull Limit impact on how traffic enters the networkndash Avoid new routes that might change neighborrsquos mindndash Select route with same attributes or at least path length
60
Managing Scalebull Destination prefixes
ndash More than 90000 destination prefixes
bull Donrsquot want to have per-prefix routing policies
ndash Small fraction of prefixes contribute most of the traffic
bull Focus on the small number of heavy hitters
ndash Define routing policies for selected prefixes
bull Routing choicesndash About 27000 unique ldquorouting choicesrdquo
bull Help in reducing the scale of the problem
ndash Small fraction of ldquorouting choicesrdquo contribute most traffic
bull Focus on the very small number of ldquorouting choicesrdquo
ndash Define routing policies on common attributes
61
Achieving Predictability
bull Route prediction with static analysisndash Helpful to know effects before deploymentndash Static analysis can help
TopologyBGP policy
configuration
eBGP routes
Offered traffic
BGP routingmodel
Flow of traffic through the network
62
Challenges to Predictabilitybull For transit ISPs effects on incoming traffic
ndash Lack of coordination strikes again
63
ldquoHot Potatordquo routing
Inter-AS Negotiation
bull Coordination aids predictabilityndash Negotiate where to sendndash Inbound and outboundndash Mutual benefits
bull How to implementndash What info to exchangendash Protecting privacyndash How to prioritize choicesndash How to prevent cheating
Destination 2
Destination 1
multiplepeeringpoints
Provider A
Provider B
64
Outbound Multihoming Goals
bull Redundancyndash Dynamic routing will failover to backup link
bull Performancendash Select provider with best performance per prefix
ndash Requires active probing
bull Costndash Select provider per prefix over time to minimize the
total financial cost
65
Inbound Traffic Control
bull More difficult no control over neighborsrsquo decisions
bull Three common techniques (previously discussed)ndash AS path prependingndash Communities and local preferencendash Prefix splitting
How does todayrsquos paper (MONET) control inbound traffic
66
How many links are enough
K upstream ISPs
Not much benefit beyond 4 ISPs
Akella et al ldquoPerformance Benefits of Multihomingrdquo SIGCOMM 2003
67
Problems with Multihoming in IPv4
bull Routing table growthndash Provider-based addressingndash Advertising prefix out multiple ISPs ndash canrsquot aggregate
bull Poor control over inbound trafficndash Existing mechanisms do not allow hosts to control
inbound traffic
68
GeorgiaTech
Internet Routing Overview
bull Intradomain (ie ldquointra-ASrdquo) routingbull Interdomain routing
Comcast
Abilene
ATampT Cogent
Autonomous Systems (ASes)
69
Configuration Problems ldquoAS 7007rdquoldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
UUNet
Florida InternetBarn
Sprint
70
Diagnosis and Troubleshootingldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
ldquoMicrosofts websites were offline for up to 23 hoursbecause of a [router] misconfigurationhellipit took nearly a day to determine what was wrong and undo the changesrdquo -- wiredcom January 25 2001
ldquoWorldCom Inchellipsuffered a widespread outage on its Internet backbone that affected roughly 20 percent of its US customer base The network problemshellipaffected millions of computer users worldwide A spokeswoman attributed the outage to a route table issue -- cnncom October 3 2002
A number of Covad customers went out from 5pm today due to supposedly a DDOS (distributed denial of service attack) on a key Level3 data center which later was described as a route leak (misconfiguration)ldquo
-- dslreportscom February 23 2004
71
Operator Mailing List (NANOG)Date Mon 18 Oct 2004 091515 -0700Subject Level 3 US east coast issues
Level 3 experiencing widespread unspecified routing issues on the US east coast Master ticket 1086844 Anyone have more specific information
Date Mon 18 Oct 2004 122034 -0400 (EDT)Subject Re Level 3 US east coast issues
Level 3 is currently experiencing a backbone outage causing routing instability and packet loss We are working to restore and will be sending hourly updateshellip
72
Operator Mailing List
0102030405060708090
Filtering RouteLeaks
RouteHijacks
RouteInstability
RoutingLoops
Blackholes
Occurr
ences o
ver
10 Y
ears
1995-1997 1998-2001 2001-2004
Note Only includes problems openly discussed on this list
Compare 83 power outages 1 fire
73
Routing Configuration
Ranking route selection
Dissemination internal route advertisement
Filtering route advertisement
Customer
Competitor
Primary
Backup
74
Internet Business Model (Simplified)
bull CustomerProvider One AS pays another for reachability to some set of destinations
bull ldquoSettlement-freerdquo Peering Bartering Two ASes exchange routes with one another
Provider
Peer
Customer
Preferences implemented with local preference manipulation
Destination
Pay to use
Get paid to use
Free to use
75
Peering Contracts Consistent Export
bull Rules of settlement-free peeringndash Advertise routes at all peering pointsndash Advertised routes must have equal ldquoAS path lengthrdquo
Sprint
ATampT
Enables ldquohot potatordquo routing
ldquoequally goodrdquoroutes
76
Consistent Export
bull Malicedeceptionbull iBGP signaling partitionbull Inconsistent export policy
neighbor 10123route-map PEER permit 10 set prepend 123
neighbor 10456route-map PEER permit 10 set prepend 123 123
Possible Causes
10123 456 1
10456 456 2
Neighbor AS Export
1 1 123
Export Clause Prepend
2 1 123 123
Two different Export Policies
77
Inconsistent Export in Practice
Feamster et al ldquoBorderGuard Detecting Cold Potatoes from Peersrdquo ACM IMC October 2004
78
Blackholes
Date Thu 18 Jul 2002 060510 -0400 (EDT)From Chad Oleary ltcolpoboxcomgtSubject Re problems with 701To ltnanogmeritedugt
Were starting to see the same issues with UUNet again Anyone elseseeing this Trying to reach Qwest
traceroute to 631461901 (631461901) 30 hops max 38 byte packets 1 esc-lp2-gwe-solutionscorpcom (631182201) 1167 ms 1163 ms 1142 ms 2 500Serial2-10GW1TPA2ALTERNET (1571301499) 1097 ms 1059 ms 1044 ms 3 161at-1-0-0XL4ATL1ALTERNET (1526381190) 13839 ms 14108 ms 16638 ms 4 0so-3-1-0XL2ATL5ALTERNET (152630238) 14370 ms 14587 ms 14553 ms 5 POS7-0BR2ATL5ALTERNET (1526382193) 13928 ms 14099 ms 14053 ms 6 7 hellip
79
Security
80
Security ldquoBogonrdquo Routes
Feamster et al ldquoAn Empirical Study of lsquoBogonrsquo Route Advertisementsrdquo ACM CCR January 2005
81
Spam Phishing etc
bull Unsolicited commercial emailbull As of about August 2008 estimates indicate that
about 95 of all email is spambull Common spam filtering techniques
ndash Content-based filtersndash DNS Blacklist (DNSBL) lookups Significant fraction of
todayrsquos DNS traffic
Can IP addresses from which spam is received be spoofed
82
Spam and Routing
83
Worms and Botnets
84
What is a Worm
bull Code that replicates and propagates across the networkndash Often carries a ldquopayloadrdquo
bull Usually spread via exploiting flaws in open servicesndash ldquoVirusesrdquo require user action to spread
bull First worm Robert Morris November 1988ndash 6-10 of all Internet hosts infected ()
bull Many more since but none on that scale until July 2001
85
Example Worm Code Red
bull Initial version July 13 2001
bull Exploited known ISAPI vulnerability in Microsoft IIS Web servers
bull 1st through 20th of each month spread20th through end of each month attack
bull Payload Web site defacementbull Scanning Random IP addressesbull Bug failure to seed random number generator
86
Code Red Revisions
bull Released July 19 2001
bull Payload flooding attack on wwwwhitehousegovndash Attack was mounted at the IP address of the Web site
bull Bug died after 20th of each month
bull Random number generator for IP scanning fixed
87
Code Red Host Infection Rate
Exponential infection rate
Measured using backscatter technique
88
Designing Fast-Spreading Worms
bull Hit-list scanningndash Time to infect first 10k hosts dominates infection timendash Solution Reconnaissance (stealthy scans etc)
bull Permutation scanningndash Observation Most scanning is redundantndash Idea Shared permutation of address space Start scanning
from own IP address Re-randomize when another infected machine is found
bull Internet-scale hit listsndash Flash worm complete infection within 30 seconds
89
Botnets
bull Bots Autonomous programs performing tasksbull Plenty of ldquobenignrdquo bots
ndash eg weatherbug
bull Botnets group of bots ndash Typically carries malicious connotationndash Large numbers of infected machinesndash Machines ldquoenlistedrdquo with infection vectors like worms
(last lecture)
bull Available for simultaneous control by a masterbull Size up to 350000 nodes (from todayrsquos paper)
90
ldquoRallyingrdquo the Botnet
bull Easy to combine worm backdoor functionalitybull Problem how to learn about successfully
infected machines
bull Optionsndash Emailndash Hard-coded email address
91
Botnet Control
bull Botnet master typically runs some IRC server on a well-known port (eg 6667)
bull Infected machine contacts botnet with pre-programmed DNS name (eg big-botde)
bull Dynamic DNS allows controller to move about freely
Infected Machine
DynamicDNS
BotnetController
(IRC server)
92
Some Defenses
93
Idea 1 Ingress Filtering
bull RFC 2827 Routers install filters to drop packets from networks that are not downstream
bull Feasible at edgesbull Difficult to configure closer to network ldquocorerdquo
20469207024 Internet
Drop all packets with source address other than 20469207024
94
Idea 2 uRPF Checks
bull Unicast Reverse Path Forwardingndash Cisco ldquoip verify unicast reverse-pathrdquo
bull Requires symmetric routing
Accept packet from interface only if forwarding table entry for source IP address matches ingress interface
100183
A10018
10015
101203
100181 24
10011 24
Strict Mode uRPF
Enabled
ldquoArdquo Routing TableDestination Next Hop1001024 Int 110018024 Int 2
100183 from wrong interface
95
Problems with uRPF
bull Asymmetric routing
96
S-BGP
bull Address-based PKI validate signaturesndash Authentication of
bull ownership for IP address blocks bull AS number bull an ASs identity and bull a BGP routers identity
ndash Use existing infrastructure (Internet registries etc)ndash Routing origination is digitally signedndash BGP updates are digitally signed
1048708 bull Route attestations A new optional BGP transitive path attribute
ndash carries digital signatures covering the routing information in updates
97
Practical Problems with S-BGP
bull Requires Public-Key Infrastructure
bull Lots of digital signatures to calculate and verifyndash Message overheadndash CPU overhead
bull Calculation expense is greatest when topology is changingndash Caching can help
bull Route aggregation is problematic (maybe thatrsquos OK)
bull Secure route withdrawals when link or node fails
bull Address ownership data out of date
bull Deployment
- Networking
- Goal of This Tutorial
- Todayrsquos Networks
- Business Models
- Billing for Internet Usage
- Net Neutrality
- Network Operations
- Point-of-Presence (PoP)
- Example Abilene Network Topology
- Another Example Backbone
- Internet Routing Overview
- Internet Routing Protocol BGP
- Two Flavors of BGP
- IPv4 Addresses Networks of Networks
- Pre-1994 Classful Addressing
- Classless Interdomain Routing (CIDR)
- Benefits of CIDR
- Growth of IP Prefixes
- 1994-1998 Linear Growth
- Around 2000 Fast Growth Resumes
- Fast growth resumes
- The Address Allocation Process
- Common Problems
- What can go wrong
- Measurement and Monitoring
- Passive vs Active Measurement
- Slide 27
- Passive Traffic Data Measurement
- Simple Network Management Protocol
- SNMP (Passive)
- Packet-level Monitoring
- Full Packet Capture (Passive)
- What is a flow
- Cisco NetFlow
- Flow Record Contents
- Aggregating Packets into Flows
- Reducing Measurement Overhead
- Packet Sampling for Flow Monitoring
- Sampling Flow-Level Sampling
- Two Main Approaches
- Packet Capture on High-Speed Links
- Characteristics of Packet Capture
- Data Measurement Repositories
- Multihoming and Traffic Engineering
- What is Multihoming
- Why Multihome
- Redundancy
- Performance
- Multihoming in IP Networks Today
- BGP or no
- Same Provider or Multiple
- Multihomed Stub One Link
- Multihomed Stub Multiple Links
- Multihomed Stub Multiple ISPs
- Multihomed Transit Network
- Interdomain Traffic Engineering
- Outbound Traffic Control
- Outbound Traffic Load Balancing
- Traffic Engineering Goals
- Managing Scale
- Achieving Predictability
- Challenges to Predictability
- Inter-AS Negotiation
- Outbound Multihoming Goals
- Inbound Traffic Control
- How many links are enough
- Problems with Multihoming in IPv4
- Slide 68
- Slide 69
- Diagnosis and Troubleshooting
- Operator Mailing List (NANOG)
- Operator Mailing List
- Routing Configuration
- Internet Business Model (Simplified)
- Peering Contracts Consistent Export
- Consistent Export
- Inconsistent Export in Practice
- Blackholes
- Security
- Security ldquoBogonrdquo Routes
- Spam Phishing etc
- Spam and Routing
- Worms and Botnets
- What is a Worm
- Example Worm Code Red
- Code Red Revisions
- Code Red Host Infection Rate
- Designing Fast-Spreading Worms
- Botnets
- ldquoRallyingrdquo the Botnet
- Botnet Control
- Some Defenses
- Idea 1 Ingress Filtering
- Idea 2 uRPF Checks
- Problems with uRPF
- S-BGP
- Practical Problems with S-BGP
-
56
Interdomain Traffic Engineering
bull The process by which a network operator configures the network to achievendash Traffic load balancendash Redundancy (primarybackup) etc
bull Two tasksndash Outbound traffic controlndash Inbound traffic control
bull Key Problems Predictability and Scalability
57
Outbound Traffic Control
bull Easier to control than inbound trafficndash Destination-based routing sender determines where
the packets go
bull Control over next-hop AS onlyndash Cannot control selection of the entire path
Provider 1 Provider 2
Control with local preference
58
Outbound Traffic Load Balancingbull Control routes to provider per-prefix
ndash Assign local preference across destination prefixesndash Change the local preference assignments over time
bull Useful inputs to load balancingndash End-to-end path performance datandash Outbound traffic statistics per destination prefix
bull Challenge Getting from traffic volumes to groups of prefixes that should be assigned to each link
Premise of ldquointelligent route controlrdquo preoducts
59
Traffic Engineering Goals
bull Predictabilityndash Ensure the BGP decision process is deterministicndash Assume that BGP updates are (relatively) stable
bull Limit overhead introduced by routing changesndash Minimize frequency of changes to routing policiesndash Limit number of prefixes affected by changes
bull Limit impact on how traffic enters the networkndash Avoid new routes that might change neighborrsquos mindndash Select route with same attributes or at least path length
60
Managing Scalebull Destination prefixes
ndash More than 90000 destination prefixes
bull Donrsquot want to have per-prefix routing policies
ndash Small fraction of prefixes contribute most of the traffic
bull Focus on the small number of heavy hitters
ndash Define routing policies for selected prefixes
bull Routing choicesndash About 27000 unique ldquorouting choicesrdquo
bull Help in reducing the scale of the problem
ndash Small fraction of ldquorouting choicesrdquo contribute most traffic
bull Focus on the very small number of ldquorouting choicesrdquo
ndash Define routing policies on common attributes
61
Achieving Predictability
bull Route prediction with static analysisndash Helpful to know effects before deploymentndash Static analysis can help
TopologyBGP policy
configuration
eBGP routes
Offered traffic
BGP routingmodel
Flow of traffic through the network
62
Challenges to Predictabilitybull For transit ISPs effects on incoming traffic
ndash Lack of coordination strikes again
63
ldquoHot Potatordquo routing
Inter-AS Negotiation
bull Coordination aids predictabilityndash Negotiate where to sendndash Inbound and outboundndash Mutual benefits
bull How to implementndash What info to exchangendash Protecting privacyndash How to prioritize choicesndash How to prevent cheating
Destination 2
Destination 1
multiplepeeringpoints
Provider A
Provider B
64
Outbound Multihoming Goals
bull Redundancyndash Dynamic routing will failover to backup link
bull Performancendash Select provider with best performance per prefix
ndash Requires active probing
bull Costndash Select provider per prefix over time to minimize the
total financial cost
65
Inbound Traffic Control
bull More difficult no control over neighborsrsquo decisions
bull Three common techniques (previously discussed)ndash AS path prependingndash Communities and local preferencendash Prefix splitting
How does todayrsquos paper (MONET) control inbound traffic
66
How many links are enough
K upstream ISPs
Not much benefit beyond 4 ISPs
Akella et al ldquoPerformance Benefits of Multihomingrdquo SIGCOMM 2003
67
Problems with Multihoming in IPv4
bull Routing table growthndash Provider-based addressingndash Advertising prefix out multiple ISPs ndash canrsquot aggregate
bull Poor control over inbound trafficndash Existing mechanisms do not allow hosts to control
inbound traffic
68
GeorgiaTech
Internet Routing Overview
bull Intradomain (ie ldquointra-ASrdquo) routingbull Interdomain routing
Comcast
Abilene
ATampT Cogent
Autonomous Systems (ASes)
69
Configuration Problems ldquoAS 7007rdquoldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
UUNet
Florida InternetBarn
Sprint
70
Diagnosis and Troubleshootingldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
ldquoMicrosofts websites were offline for up to 23 hoursbecause of a [router] misconfigurationhellipit took nearly a day to determine what was wrong and undo the changesrdquo -- wiredcom January 25 2001
ldquoWorldCom Inchellipsuffered a widespread outage on its Internet backbone that affected roughly 20 percent of its US customer base The network problemshellipaffected millions of computer users worldwide A spokeswoman attributed the outage to a route table issue -- cnncom October 3 2002
A number of Covad customers went out from 5pm today due to supposedly a DDOS (distributed denial of service attack) on a key Level3 data center which later was described as a route leak (misconfiguration)ldquo
-- dslreportscom February 23 2004
71
Operator Mailing List (NANOG)Date Mon 18 Oct 2004 091515 -0700Subject Level 3 US east coast issues
Level 3 experiencing widespread unspecified routing issues on the US east coast Master ticket 1086844 Anyone have more specific information
Date Mon 18 Oct 2004 122034 -0400 (EDT)Subject Re Level 3 US east coast issues
Level 3 is currently experiencing a backbone outage causing routing instability and packet loss We are working to restore and will be sending hourly updateshellip
72
Operator Mailing List
0102030405060708090
Filtering RouteLeaks
RouteHijacks
RouteInstability
RoutingLoops
Blackholes
Occurr
ences o
ver
10 Y
ears
1995-1997 1998-2001 2001-2004
Note Only includes problems openly discussed on this list
Compare 83 power outages 1 fire
73
Routing Configuration
Ranking route selection
Dissemination internal route advertisement
Filtering route advertisement
Customer
Competitor
Primary
Backup
74
Internet Business Model (Simplified)
bull CustomerProvider One AS pays another for reachability to some set of destinations
bull ldquoSettlement-freerdquo Peering Bartering Two ASes exchange routes with one another
Provider
Peer
Customer
Preferences implemented with local preference manipulation
Destination
Pay to use
Get paid to use
Free to use
75
Peering Contracts Consistent Export
bull Rules of settlement-free peeringndash Advertise routes at all peering pointsndash Advertised routes must have equal ldquoAS path lengthrdquo
Sprint
ATampT
Enables ldquohot potatordquo routing
ldquoequally goodrdquoroutes
76
Consistent Export
bull Malicedeceptionbull iBGP signaling partitionbull Inconsistent export policy
neighbor 10123route-map PEER permit 10 set prepend 123
neighbor 10456route-map PEER permit 10 set prepend 123 123
Possible Causes
10123 456 1
10456 456 2
Neighbor AS Export
1 1 123
Export Clause Prepend
2 1 123 123
Two different Export Policies
77
Inconsistent Export in Practice
Feamster et al ldquoBorderGuard Detecting Cold Potatoes from Peersrdquo ACM IMC October 2004
78
Blackholes
Date Thu 18 Jul 2002 060510 -0400 (EDT)From Chad Oleary ltcolpoboxcomgtSubject Re problems with 701To ltnanogmeritedugt
Were starting to see the same issues with UUNet again Anyone elseseeing this Trying to reach Qwest
traceroute to 631461901 (631461901) 30 hops max 38 byte packets 1 esc-lp2-gwe-solutionscorpcom (631182201) 1167 ms 1163 ms 1142 ms 2 500Serial2-10GW1TPA2ALTERNET (1571301499) 1097 ms 1059 ms 1044 ms 3 161at-1-0-0XL4ATL1ALTERNET (1526381190) 13839 ms 14108 ms 16638 ms 4 0so-3-1-0XL2ATL5ALTERNET (152630238) 14370 ms 14587 ms 14553 ms 5 POS7-0BR2ATL5ALTERNET (1526382193) 13928 ms 14099 ms 14053 ms 6 7 hellip
79
Security
80
Security ldquoBogonrdquo Routes
Feamster et al ldquoAn Empirical Study of lsquoBogonrsquo Route Advertisementsrdquo ACM CCR January 2005
81
Spam Phishing etc
bull Unsolicited commercial emailbull As of about August 2008 estimates indicate that
about 95 of all email is spambull Common spam filtering techniques
ndash Content-based filtersndash DNS Blacklist (DNSBL) lookups Significant fraction of
todayrsquos DNS traffic
Can IP addresses from which spam is received be spoofed
82
Spam and Routing
83
Worms and Botnets
84
What is a Worm
bull Code that replicates and propagates across the networkndash Often carries a ldquopayloadrdquo
bull Usually spread via exploiting flaws in open servicesndash ldquoVirusesrdquo require user action to spread
bull First worm Robert Morris November 1988ndash 6-10 of all Internet hosts infected ()
bull Many more since but none on that scale until July 2001
85
Example Worm Code Red
bull Initial version July 13 2001
bull Exploited known ISAPI vulnerability in Microsoft IIS Web servers
bull 1st through 20th of each month spread20th through end of each month attack
bull Payload Web site defacementbull Scanning Random IP addressesbull Bug failure to seed random number generator
86
Code Red Revisions
bull Released July 19 2001
bull Payload flooding attack on wwwwhitehousegovndash Attack was mounted at the IP address of the Web site
bull Bug died after 20th of each month
bull Random number generator for IP scanning fixed
87
Code Red Host Infection Rate
Exponential infection rate
Measured using backscatter technique
88
Designing Fast-Spreading Worms
bull Hit-list scanningndash Time to infect first 10k hosts dominates infection timendash Solution Reconnaissance (stealthy scans etc)
bull Permutation scanningndash Observation Most scanning is redundantndash Idea Shared permutation of address space Start scanning
from own IP address Re-randomize when another infected machine is found
bull Internet-scale hit listsndash Flash worm complete infection within 30 seconds
89
Botnets
bull Bots Autonomous programs performing tasksbull Plenty of ldquobenignrdquo bots
ndash eg weatherbug
bull Botnets group of bots ndash Typically carries malicious connotationndash Large numbers of infected machinesndash Machines ldquoenlistedrdquo with infection vectors like worms
(last lecture)
bull Available for simultaneous control by a masterbull Size up to 350000 nodes (from todayrsquos paper)
90
ldquoRallyingrdquo the Botnet
bull Easy to combine worm backdoor functionalitybull Problem how to learn about successfully
infected machines
bull Optionsndash Emailndash Hard-coded email address
91
Botnet Control
bull Botnet master typically runs some IRC server on a well-known port (eg 6667)
bull Infected machine contacts botnet with pre-programmed DNS name (eg big-botde)
bull Dynamic DNS allows controller to move about freely
Infected Machine
DynamicDNS
BotnetController
(IRC server)
92
Some Defenses
93
Idea 1 Ingress Filtering
bull RFC 2827 Routers install filters to drop packets from networks that are not downstream
bull Feasible at edgesbull Difficult to configure closer to network ldquocorerdquo
20469207024 Internet
Drop all packets with source address other than 20469207024
94
Idea 2 uRPF Checks
bull Unicast Reverse Path Forwardingndash Cisco ldquoip verify unicast reverse-pathrdquo
bull Requires symmetric routing
Accept packet from interface only if forwarding table entry for source IP address matches ingress interface
100183
A10018
10015
101203
100181 24
10011 24
Strict Mode uRPF
Enabled
ldquoArdquo Routing TableDestination Next Hop1001024 Int 110018024 Int 2
100183 from wrong interface
95
Problems with uRPF
bull Asymmetric routing
96
S-BGP
bull Address-based PKI validate signaturesndash Authentication of
bull ownership for IP address blocks bull AS number bull an ASs identity and bull a BGP routers identity
ndash Use existing infrastructure (Internet registries etc)ndash Routing origination is digitally signedndash BGP updates are digitally signed
1048708 bull Route attestations A new optional BGP transitive path attribute
ndash carries digital signatures covering the routing information in updates
97
Practical Problems with S-BGP
bull Requires Public-Key Infrastructure
bull Lots of digital signatures to calculate and verifyndash Message overheadndash CPU overhead
bull Calculation expense is greatest when topology is changingndash Caching can help
bull Route aggregation is problematic (maybe thatrsquos OK)
bull Secure route withdrawals when link or node fails
bull Address ownership data out of date
bull Deployment
- Networking
- Goal of This Tutorial
- Todayrsquos Networks
- Business Models
- Billing for Internet Usage
- Net Neutrality
- Network Operations
- Point-of-Presence (PoP)
- Example Abilene Network Topology
- Another Example Backbone
- Internet Routing Overview
- Internet Routing Protocol BGP
- Two Flavors of BGP
- IPv4 Addresses Networks of Networks
- Pre-1994 Classful Addressing
- Classless Interdomain Routing (CIDR)
- Benefits of CIDR
- Growth of IP Prefixes
- 1994-1998 Linear Growth
- Around 2000 Fast Growth Resumes
- Fast growth resumes
- The Address Allocation Process
- Common Problems
- What can go wrong
- Measurement and Monitoring
- Passive vs Active Measurement
- Slide 27
- Passive Traffic Data Measurement
- Simple Network Management Protocol
- SNMP (Passive)
- Packet-level Monitoring
- Full Packet Capture (Passive)
- What is a flow
- Cisco NetFlow
- Flow Record Contents
- Aggregating Packets into Flows
- Reducing Measurement Overhead
- Packet Sampling for Flow Monitoring
- Sampling Flow-Level Sampling
- Two Main Approaches
- Packet Capture on High-Speed Links
- Characteristics of Packet Capture
- Data Measurement Repositories
- Multihoming and Traffic Engineering
- What is Multihoming
- Why Multihome
- Redundancy
- Performance
- Multihoming in IP Networks Today
- BGP or no
- Same Provider or Multiple
- Multihomed Stub One Link
- Multihomed Stub Multiple Links
- Multihomed Stub Multiple ISPs
- Multihomed Transit Network
- Interdomain Traffic Engineering
- Outbound Traffic Control
- Outbound Traffic Load Balancing
- Traffic Engineering Goals
- Managing Scale
- Achieving Predictability
- Challenges to Predictability
- Inter-AS Negotiation
- Outbound Multihoming Goals
- Inbound Traffic Control
- How many links are enough
- Problems with Multihoming in IPv4
- Slide 68
- Slide 69
- Diagnosis and Troubleshooting
- Operator Mailing List (NANOG)
- Operator Mailing List
- Routing Configuration
- Internet Business Model (Simplified)
- Peering Contracts Consistent Export
- Consistent Export
- Inconsistent Export in Practice
- Blackholes
- Security
- Security ldquoBogonrdquo Routes
- Spam Phishing etc
- Spam and Routing
- Worms and Botnets
- What is a Worm
- Example Worm Code Red
- Code Red Revisions
- Code Red Host Infection Rate
- Designing Fast-Spreading Worms
- Botnets
- ldquoRallyingrdquo the Botnet
- Botnet Control
- Some Defenses
- Idea 1 Ingress Filtering
- Idea 2 uRPF Checks
- Problems with uRPF
- S-BGP
- Practical Problems with S-BGP
-
57
Outbound Traffic Control
bull Easier to control than inbound trafficndash Destination-based routing sender determines where
the packets go
bull Control over next-hop AS onlyndash Cannot control selection of the entire path
Provider 1 Provider 2
Control with local preference
58
Outbound Traffic Load Balancingbull Control routes to provider per-prefix
ndash Assign local preference across destination prefixesndash Change the local preference assignments over time
bull Useful inputs to load balancingndash End-to-end path performance datandash Outbound traffic statistics per destination prefix
bull Challenge Getting from traffic volumes to groups of prefixes that should be assigned to each link
Premise of ldquointelligent route controlrdquo preoducts
59
Traffic Engineering Goals
bull Predictabilityndash Ensure the BGP decision process is deterministicndash Assume that BGP updates are (relatively) stable
bull Limit overhead introduced by routing changesndash Minimize frequency of changes to routing policiesndash Limit number of prefixes affected by changes
bull Limit impact on how traffic enters the networkndash Avoid new routes that might change neighborrsquos mindndash Select route with same attributes or at least path length
60
Managing Scalebull Destination prefixes
ndash More than 90000 destination prefixes
bull Donrsquot want to have per-prefix routing policies
ndash Small fraction of prefixes contribute most of the traffic
bull Focus on the small number of heavy hitters
ndash Define routing policies for selected prefixes
bull Routing choicesndash About 27000 unique ldquorouting choicesrdquo
bull Help in reducing the scale of the problem
ndash Small fraction of ldquorouting choicesrdquo contribute most traffic
bull Focus on the very small number of ldquorouting choicesrdquo
ndash Define routing policies on common attributes
61
Achieving Predictability
bull Route prediction with static analysisndash Helpful to know effects before deploymentndash Static analysis can help
TopologyBGP policy
configuration
eBGP routes
Offered traffic
BGP routingmodel
Flow of traffic through the network
62
Challenges to Predictabilitybull For transit ISPs effects on incoming traffic
ndash Lack of coordination strikes again
63
ldquoHot Potatordquo routing
Inter-AS Negotiation
bull Coordination aids predictabilityndash Negotiate where to sendndash Inbound and outboundndash Mutual benefits
bull How to implementndash What info to exchangendash Protecting privacyndash How to prioritize choicesndash How to prevent cheating
Destination 2
Destination 1
multiplepeeringpoints
Provider A
Provider B
64
Outbound Multihoming Goals
bull Redundancyndash Dynamic routing will failover to backup link
bull Performancendash Select provider with best performance per prefix
ndash Requires active probing
bull Costndash Select provider per prefix over time to minimize the
total financial cost
65
Inbound Traffic Control
bull More difficult no control over neighborsrsquo decisions
bull Three common techniques (previously discussed)ndash AS path prependingndash Communities and local preferencendash Prefix splitting
How does todayrsquos paper (MONET) control inbound traffic
66
How many links are enough
K upstream ISPs
Not much benefit beyond 4 ISPs
Akella et al ldquoPerformance Benefits of Multihomingrdquo SIGCOMM 2003
67
Problems with Multihoming in IPv4
bull Routing table growthndash Provider-based addressingndash Advertising prefix out multiple ISPs ndash canrsquot aggregate
bull Poor control over inbound trafficndash Existing mechanisms do not allow hosts to control
inbound traffic
68
GeorgiaTech
Internet Routing Overview
bull Intradomain (ie ldquointra-ASrdquo) routingbull Interdomain routing
Comcast
Abilene
ATampT Cogent
Autonomous Systems (ASes)
69
Configuration Problems ldquoAS 7007rdquoldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
UUNet
Florida InternetBarn
Sprint
70
Diagnosis and Troubleshootingldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
ldquoMicrosofts websites were offline for up to 23 hoursbecause of a [router] misconfigurationhellipit took nearly a day to determine what was wrong and undo the changesrdquo -- wiredcom January 25 2001
ldquoWorldCom Inchellipsuffered a widespread outage on its Internet backbone that affected roughly 20 percent of its US customer base The network problemshellipaffected millions of computer users worldwide A spokeswoman attributed the outage to a route table issue -- cnncom October 3 2002
A number of Covad customers went out from 5pm today due to supposedly a DDOS (distributed denial of service attack) on a key Level3 data center which later was described as a route leak (misconfiguration)ldquo
-- dslreportscom February 23 2004
71
Operator Mailing List (NANOG)Date Mon 18 Oct 2004 091515 -0700Subject Level 3 US east coast issues
Level 3 experiencing widespread unspecified routing issues on the US east coast Master ticket 1086844 Anyone have more specific information
Date Mon 18 Oct 2004 122034 -0400 (EDT)Subject Re Level 3 US east coast issues
Level 3 is currently experiencing a backbone outage causing routing instability and packet loss We are working to restore and will be sending hourly updateshellip
72
Operator Mailing List
0102030405060708090
Filtering RouteLeaks
RouteHijacks
RouteInstability
RoutingLoops
Blackholes
Occurr
ences o
ver
10 Y
ears
1995-1997 1998-2001 2001-2004
Note Only includes problems openly discussed on this list
Compare 83 power outages 1 fire
73
Routing Configuration
Ranking route selection
Dissemination internal route advertisement
Filtering route advertisement
Customer
Competitor
Primary
Backup
74
Internet Business Model (Simplified)
bull CustomerProvider One AS pays another for reachability to some set of destinations
bull ldquoSettlement-freerdquo Peering Bartering Two ASes exchange routes with one another
Provider
Peer
Customer
Preferences implemented with local preference manipulation
Destination
Pay to use
Get paid to use
Free to use
75
Peering Contracts Consistent Export
bull Rules of settlement-free peeringndash Advertise routes at all peering pointsndash Advertised routes must have equal ldquoAS path lengthrdquo
Sprint
ATampT
Enables ldquohot potatordquo routing
ldquoequally goodrdquoroutes
76
Consistent Export
bull Malicedeceptionbull iBGP signaling partitionbull Inconsistent export policy
neighbor 10123route-map PEER permit 10 set prepend 123
neighbor 10456route-map PEER permit 10 set prepend 123 123
Possible Causes
10123 456 1
10456 456 2
Neighbor AS Export
1 1 123
Export Clause Prepend
2 1 123 123
Two different Export Policies
77
Inconsistent Export in Practice
Feamster et al ldquoBorderGuard Detecting Cold Potatoes from Peersrdquo ACM IMC October 2004
78
Blackholes
Date Thu 18 Jul 2002 060510 -0400 (EDT)From Chad Oleary ltcolpoboxcomgtSubject Re problems with 701To ltnanogmeritedugt
Were starting to see the same issues with UUNet again Anyone elseseeing this Trying to reach Qwest
traceroute to 631461901 (631461901) 30 hops max 38 byte packets 1 esc-lp2-gwe-solutionscorpcom (631182201) 1167 ms 1163 ms 1142 ms 2 500Serial2-10GW1TPA2ALTERNET (1571301499) 1097 ms 1059 ms 1044 ms 3 161at-1-0-0XL4ATL1ALTERNET (1526381190) 13839 ms 14108 ms 16638 ms 4 0so-3-1-0XL2ATL5ALTERNET (152630238) 14370 ms 14587 ms 14553 ms 5 POS7-0BR2ATL5ALTERNET (1526382193) 13928 ms 14099 ms 14053 ms 6 7 hellip
79
Security
80
Security ldquoBogonrdquo Routes
Feamster et al ldquoAn Empirical Study of lsquoBogonrsquo Route Advertisementsrdquo ACM CCR January 2005
81
Spam Phishing etc
bull Unsolicited commercial emailbull As of about August 2008 estimates indicate that
about 95 of all email is spambull Common spam filtering techniques
ndash Content-based filtersndash DNS Blacklist (DNSBL) lookups Significant fraction of
todayrsquos DNS traffic
Can IP addresses from which spam is received be spoofed
82
Spam and Routing
83
Worms and Botnets
84
What is a Worm
bull Code that replicates and propagates across the networkndash Often carries a ldquopayloadrdquo
bull Usually spread via exploiting flaws in open servicesndash ldquoVirusesrdquo require user action to spread
bull First worm Robert Morris November 1988ndash 6-10 of all Internet hosts infected ()
bull Many more since but none on that scale until July 2001
85
Example Worm Code Red
bull Initial version July 13 2001
bull Exploited known ISAPI vulnerability in Microsoft IIS Web servers
bull 1st through 20th of each month spread20th through end of each month attack
bull Payload Web site defacementbull Scanning Random IP addressesbull Bug failure to seed random number generator
86
Code Red Revisions
bull Released July 19 2001
bull Payload flooding attack on wwwwhitehousegovndash Attack was mounted at the IP address of the Web site
bull Bug died after 20th of each month
bull Random number generator for IP scanning fixed
87
Code Red Host Infection Rate
Exponential infection rate
Measured using backscatter technique
88
Designing Fast-Spreading Worms
bull Hit-list scanningndash Time to infect first 10k hosts dominates infection timendash Solution Reconnaissance (stealthy scans etc)
bull Permutation scanningndash Observation Most scanning is redundantndash Idea Shared permutation of address space Start scanning
from own IP address Re-randomize when another infected machine is found
bull Internet-scale hit listsndash Flash worm complete infection within 30 seconds
89
Botnets
bull Bots Autonomous programs performing tasksbull Plenty of ldquobenignrdquo bots
ndash eg weatherbug
bull Botnets group of bots ndash Typically carries malicious connotationndash Large numbers of infected machinesndash Machines ldquoenlistedrdquo with infection vectors like worms
(last lecture)
bull Available for simultaneous control by a masterbull Size up to 350000 nodes (from todayrsquos paper)
90
ldquoRallyingrdquo the Botnet
bull Easy to combine worm backdoor functionalitybull Problem how to learn about successfully
infected machines
bull Optionsndash Emailndash Hard-coded email address
91
Botnet Control
bull Botnet master typically runs some IRC server on a well-known port (eg 6667)
bull Infected machine contacts botnet with pre-programmed DNS name (eg big-botde)
bull Dynamic DNS allows controller to move about freely
Infected Machine
DynamicDNS
BotnetController
(IRC server)
92
Some Defenses
93
Idea 1 Ingress Filtering
bull RFC 2827 Routers install filters to drop packets from networks that are not downstream
bull Feasible at edgesbull Difficult to configure closer to network ldquocorerdquo
20469207024 Internet
Drop all packets with source address other than 20469207024
94
Idea 2 uRPF Checks
bull Unicast Reverse Path Forwardingndash Cisco ldquoip verify unicast reverse-pathrdquo
bull Requires symmetric routing
Accept packet from interface only if forwarding table entry for source IP address matches ingress interface
100183
A10018
10015
101203
100181 24
10011 24
Strict Mode uRPF
Enabled
ldquoArdquo Routing TableDestination Next Hop1001024 Int 110018024 Int 2
100183 from wrong interface
95
Problems with uRPF
bull Asymmetric routing
96
S-BGP
bull Address-based PKI validate signaturesndash Authentication of
bull ownership for IP address blocks bull AS number bull an ASs identity and bull a BGP routers identity
ndash Use existing infrastructure (Internet registries etc)ndash Routing origination is digitally signedndash BGP updates are digitally signed
1048708 bull Route attestations A new optional BGP transitive path attribute
ndash carries digital signatures covering the routing information in updates
97
Practical Problems with S-BGP
bull Requires Public-Key Infrastructure
bull Lots of digital signatures to calculate and verifyndash Message overheadndash CPU overhead
bull Calculation expense is greatest when topology is changingndash Caching can help
bull Route aggregation is problematic (maybe thatrsquos OK)
bull Secure route withdrawals when link or node fails
bull Address ownership data out of date
bull Deployment
- Networking
- Goal of This Tutorial
- Todayrsquos Networks
- Business Models
- Billing for Internet Usage
- Net Neutrality
- Network Operations
- Point-of-Presence (PoP)
- Example Abilene Network Topology
- Another Example Backbone
- Internet Routing Overview
- Internet Routing Protocol BGP
- Two Flavors of BGP
- IPv4 Addresses Networks of Networks
- Pre-1994 Classful Addressing
- Classless Interdomain Routing (CIDR)
- Benefits of CIDR
- Growth of IP Prefixes
- 1994-1998 Linear Growth
- Around 2000 Fast Growth Resumes
- Fast growth resumes
- The Address Allocation Process
- Common Problems
- What can go wrong
- Measurement and Monitoring
- Passive vs Active Measurement
- Slide 27
- Passive Traffic Data Measurement
- Simple Network Management Protocol
- SNMP (Passive)
- Packet-level Monitoring
- Full Packet Capture (Passive)
- What is a flow
- Cisco NetFlow
- Flow Record Contents
- Aggregating Packets into Flows
- Reducing Measurement Overhead
- Packet Sampling for Flow Monitoring
- Sampling Flow-Level Sampling
- Two Main Approaches
- Packet Capture on High-Speed Links
- Characteristics of Packet Capture
- Data Measurement Repositories
- Multihoming and Traffic Engineering
- What is Multihoming
- Why Multihome
- Redundancy
- Performance
- Multihoming in IP Networks Today
- BGP or no
- Same Provider or Multiple
- Multihomed Stub One Link
- Multihomed Stub Multiple Links
- Multihomed Stub Multiple ISPs
- Multihomed Transit Network
- Interdomain Traffic Engineering
- Outbound Traffic Control
- Outbound Traffic Load Balancing
- Traffic Engineering Goals
- Managing Scale
- Achieving Predictability
- Challenges to Predictability
- Inter-AS Negotiation
- Outbound Multihoming Goals
- Inbound Traffic Control
- How many links are enough
- Problems with Multihoming in IPv4
- Slide 68
- Slide 69
- Diagnosis and Troubleshooting
- Operator Mailing List (NANOG)
- Operator Mailing List
- Routing Configuration
- Internet Business Model (Simplified)
- Peering Contracts Consistent Export
- Consistent Export
- Inconsistent Export in Practice
- Blackholes
- Security
- Security ldquoBogonrdquo Routes
- Spam Phishing etc
- Spam and Routing
- Worms and Botnets
- What is a Worm
- Example Worm Code Red
- Code Red Revisions
- Code Red Host Infection Rate
- Designing Fast-Spreading Worms
- Botnets
- ldquoRallyingrdquo the Botnet
- Botnet Control
- Some Defenses
- Idea 1 Ingress Filtering
- Idea 2 uRPF Checks
- Problems with uRPF
- S-BGP
- Practical Problems with S-BGP
-
58
Outbound Traffic Load Balancingbull Control routes to provider per-prefix
ndash Assign local preference across destination prefixesndash Change the local preference assignments over time
bull Useful inputs to load balancingndash End-to-end path performance datandash Outbound traffic statistics per destination prefix
bull Challenge Getting from traffic volumes to groups of prefixes that should be assigned to each link
Premise of ldquointelligent route controlrdquo preoducts
59
Traffic Engineering Goals
bull Predictabilityndash Ensure the BGP decision process is deterministicndash Assume that BGP updates are (relatively) stable
bull Limit overhead introduced by routing changesndash Minimize frequency of changes to routing policiesndash Limit number of prefixes affected by changes
bull Limit impact on how traffic enters the networkndash Avoid new routes that might change neighborrsquos mindndash Select route with same attributes or at least path length
60
Managing Scalebull Destination prefixes
ndash More than 90000 destination prefixes
bull Donrsquot want to have per-prefix routing policies
ndash Small fraction of prefixes contribute most of the traffic
bull Focus on the small number of heavy hitters
ndash Define routing policies for selected prefixes
bull Routing choicesndash About 27000 unique ldquorouting choicesrdquo
bull Help in reducing the scale of the problem
ndash Small fraction of ldquorouting choicesrdquo contribute most traffic
bull Focus on the very small number of ldquorouting choicesrdquo
ndash Define routing policies on common attributes
61
Achieving Predictability
bull Route prediction with static analysisndash Helpful to know effects before deploymentndash Static analysis can help
TopologyBGP policy
configuration
eBGP routes
Offered traffic
BGP routingmodel
Flow of traffic through the network
62
Challenges to Predictabilitybull For transit ISPs effects on incoming traffic
ndash Lack of coordination strikes again
63
ldquoHot Potatordquo routing
Inter-AS Negotiation
bull Coordination aids predictabilityndash Negotiate where to sendndash Inbound and outboundndash Mutual benefits
bull How to implementndash What info to exchangendash Protecting privacyndash How to prioritize choicesndash How to prevent cheating
Destination 2
Destination 1
multiplepeeringpoints
Provider A
Provider B
64
Outbound Multihoming Goals
bull Redundancyndash Dynamic routing will failover to backup link
bull Performancendash Select provider with best performance per prefix
ndash Requires active probing
bull Costndash Select provider per prefix over time to minimize the
total financial cost
65
Inbound Traffic Control
bull More difficult no control over neighborsrsquo decisions
bull Three common techniques (previously discussed)ndash AS path prependingndash Communities and local preferencendash Prefix splitting
How does todayrsquos paper (MONET) control inbound traffic
66
How many links are enough
K upstream ISPs
Not much benefit beyond 4 ISPs
Akella et al ldquoPerformance Benefits of Multihomingrdquo SIGCOMM 2003
67
Problems with Multihoming in IPv4
bull Routing table growthndash Provider-based addressingndash Advertising prefix out multiple ISPs ndash canrsquot aggregate
bull Poor control over inbound trafficndash Existing mechanisms do not allow hosts to control
inbound traffic
68
GeorgiaTech
Internet Routing Overview
bull Intradomain (ie ldquointra-ASrdquo) routingbull Interdomain routing
Comcast
Abilene
ATampT Cogent
Autonomous Systems (ASes)
69
Configuration Problems ldquoAS 7007rdquoldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
UUNet
Florida InternetBarn
Sprint
70
Diagnosis and Troubleshootingldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
ldquoMicrosofts websites were offline for up to 23 hoursbecause of a [router] misconfigurationhellipit took nearly a day to determine what was wrong and undo the changesrdquo -- wiredcom January 25 2001
ldquoWorldCom Inchellipsuffered a widespread outage on its Internet backbone that affected roughly 20 percent of its US customer base The network problemshellipaffected millions of computer users worldwide A spokeswoman attributed the outage to a route table issue -- cnncom October 3 2002
A number of Covad customers went out from 5pm today due to supposedly a DDOS (distributed denial of service attack) on a key Level3 data center which later was described as a route leak (misconfiguration)ldquo
-- dslreportscom February 23 2004
71
Operator Mailing List (NANOG)Date Mon 18 Oct 2004 091515 -0700Subject Level 3 US east coast issues
Level 3 experiencing widespread unspecified routing issues on the US east coast Master ticket 1086844 Anyone have more specific information
Date Mon 18 Oct 2004 122034 -0400 (EDT)Subject Re Level 3 US east coast issues
Level 3 is currently experiencing a backbone outage causing routing instability and packet loss We are working to restore and will be sending hourly updateshellip
72
Operator Mailing List
0102030405060708090
Filtering RouteLeaks
RouteHijacks
RouteInstability
RoutingLoops
Blackholes
Occurr
ences o
ver
10 Y
ears
1995-1997 1998-2001 2001-2004
Note Only includes problems openly discussed on this list
Compare 83 power outages 1 fire
73
Routing Configuration
Ranking route selection
Dissemination internal route advertisement
Filtering route advertisement
Customer
Competitor
Primary
Backup
74
Internet Business Model (Simplified)
bull CustomerProvider One AS pays another for reachability to some set of destinations
bull ldquoSettlement-freerdquo Peering Bartering Two ASes exchange routes with one another
Provider
Peer
Customer
Preferences implemented with local preference manipulation
Destination
Pay to use
Get paid to use
Free to use
75
Peering Contracts Consistent Export
bull Rules of settlement-free peeringndash Advertise routes at all peering pointsndash Advertised routes must have equal ldquoAS path lengthrdquo
Sprint
ATampT
Enables ldquohot potatordquo routing
ldquoequally goodrdquoroutes
76
Consistent Export
bull Malicedeceptionbull iBGP signaling partitionbull Inconsistent export policy
neighbor 10123route-map PEER permit 10 set prepend 123
neighbor 10456route-map PEER permit 10 set prepend 123 123
Possible Causes
10123 456 1
10456 456 2
Neighbor AS Export
1 1 123
Export Clause Prepend
2 1 123 123
Two different Export Policies
77
Inconsistent Export in Practice
Feamster et al ldquoBorderGuard Detecting Cold Potatoes from Peersrdquo ACM IMC October 2004
78
Blackholes
Date Thu 18 Jul 2002 060510 -0400 (EDT)From Chad Oleary ltcolpoboxcomgtSubject Re problems with 701To ltnanogmeritedugt
Were starting to see the same issues with UUNet again Anyone elseseeing this Trying to reach Qwest
traceroute to 631461901 (631461901) 30 hops max 38 byte packets 1 esc-lp2-gwe-solutionscorpcom (631182201) 1167 ms 1163 ms 1142 ms 2 500Serial2-10GW1TPA2ALTERNET (1571301499) 1097 ms 1059 ms 1044 ms 3 161at-1-0-0XL4ATL1ALTERNET (1526381190) 13839 ms 14108 ms 16638 ms 4 0so-3-1-0XL2ATL5ALTERNET (152630238) 14370 ms 14587 ms 14553 ms 5 POS7-0BR2ATL5ALTERNET (1526382193) 13928 ms 14099 ms 14053 ms 6 7 hellip
79
Security
80
Security ldquoBogonrdquo Routes
Feamster et al ldquoAn Empirical Study of lsquoBogonrsquo Route Advertisementsrdquo ACM CCR January 2005
81
Spam Phishing etc
bull Unsolicited commercial emailbull As of about August 2008 estimates indicate that
about 95 of all email is spambull Common spam filtering techniques
ndash Content-based filtersndash DNS Blacklist (DNSBL) lookups Significant fraction of
todayrsquos DNS traffic
Can IP addresses from which spam is received be spoofed
82
Spam and Routing
83
Worms and Botnets
84
What is a Worm
bull Code that replicates and propagates across the networkndash Often carries a ldquopayloadrdquo
bull Usually spread via exploiting flaws in open servicesndash ldquoVirusesrdquo require user action to spread
bull First worm Robert Morris November 1988ndash 6-10 of all Internet hosts infected ()
bull Many more since but none on that scale until July 2001
85
Example Worm Code Red
bull Initial version July 13 2001
bull Exploited known ISAPI vulnerability in Microsoft IIS Web servers
bull 1st through 20th of each month spread20th through end of each month attack
bull Payload Web site defacementbull Scanning Random IP addressesbull Bug failure to seed random number generator
86
Code Red Revisions
bull Released July 19 2001
bull Payload flooding attack on wwwwhitehousegovndash Attack was mounted at the IP address of the Web site
bull Bug died after 20th of each month
bull Random number generator for IP scanning fixed
87
Code Red Host Infection Rate
Exponential infection rate
Measured using backscatter technique
88
Designing Fast-Spreading Worms
bull Hit-list scanningndash Time to infect first 10k hosts dominates infection timendash Solution Reconnaissance (stealthy scans etc)
bull Permutation scanningndash Observation Most scanning is redundantndash Idea Shared permutation of address space Start scanning
from own IP address Re-randomize when another infected machine is found
bull Internet-scale hit listsndash Flash worm complete infection within 30 seconds
89
Botnets
bull Bots Autonomous programs performing tasksbull Plenty of ldquobenignrdquo bots
ndash eg weatherbug
bull Botnets group of bots ndash Typically carries malicious connotationndash Large numbers of infected machinesndash Machines ldquoenlistedrdquo with infection vectors like worms
(last lecture)
bull Available for simultaneous control by a masterbull Size up to 350000 nodes (from todayrsquos paper)
90
ldquoRallyingrdquo the Botnet
bull Easy to combine worm backdoor functionalitybull Problem how to learn about successfully
infected machines
bull Optionsndash Emailndash Hard-coded email address
91
Botnet Control
bull Botnet master typically runs some IRC server on a well-known port (eg 6667)
bull Infected machine contacts botnet with pre-programmed DNS name (eg big-botde)
bull Dynamic DNS allows controller to move about freely
Infected Machine
DynamicDNS
BotnetController
(IRC server)
92
Some Defenses
93
Idea 1 Ingress Filtering
bull RFC 2827 Routers install filters to drop packets from networks that are not downstream
bull Feasible at edgesbull Difficult to configure closer to network ldquocorerdquo
20469207024 Internet
Drop all packets with source address other than 20469207024
94
Idea 2 uRPF Checks
bull Unicast Reverse Path Forwardingndash Cisco ldquoip verify unicast reverse-pathrdquo
bull Requires symmetric routing
Accept packet from interface only if forwarding table entry for source IP address matches ingress interface
100183
A10018
10015
101203
100181 24
10011 24
Strict Mode uRPF
Enabled
ldquoArdquo Routing TableDestination Next Hop1001024 Int 110018024 Int 2
100183 from wrong interface
95
Problems with uRPF
bull Asymmetric routing
96
S-BGP
bull Address-based PKI validate signaturesndash Authentication of
bull ownership for IP address blocks bull AS number bull an ASs identity and bull a BGP routers identity
ndash Use existing infrastructure (Internet registries etc)ndash Routing origination is digitally signedndash BGP updates are digitally signed
1048708 bull Route attestations A new optional BGP transitive path attribute
ndash carries digital signatures covering the routing information in updates
97
Practical Problems with S-BGP
bull Requires Public-Key Infrastructure
bull Lots of digital signatures to calculate and verifyndash Message overheadndash CPU overhead
bull Calculation expense is greatest when topology is changingndash Caching can help
bull Route aggregation is problematic (maybe thatrsquos OK)
bull Secure route withdrawals when link or node fails
bull Address ownership data out of date
bull Deployment
- Networking
- Goal of This Tutorial
- Todayrsquos Networks
- Business Models
- Billing for Internet Usage
- Net Neutrality
- Network Operations
- Point-of-Presence (PoP)
- Example Abilene Network Topology
- Another Example Backbone
- Internet Routing Overview
- Internet Routing Protocol BGP
- Two Flavors of BGP
- IPv4 Addresses Networks of Networks
- Pre-1994 Classful Addressing
- Classless Interdomain Routing (CIDR)
- Benefits of CIDR
- Growth of IP Prefixes
- 1994-1998 Linear Growth
- Around 2000 Fast Growth Resumes
- Fast growth resumes
- The Address Allocation Process
- Common Problems
- What can go wrong
- Measurement and Monitoring
- Passive vs Active Measurement
- Slide 27
- Passive Traffic Data Measurement
- Simple Network Management Protocol
- SNMP (Passive)
- Packet-level Monitoring
- Full Packet Capture (Passive)
- What is a flow
- Cisco NetFlow
- Flow Record Contents
- Aggregating Packets into Flows
- Reducing Measurement Overhead
- Packet Sampling for Flow Monitoring
- Sampling Flow-Level Sampling
- Two Main Approaches
- Packet Capture on High-Speed Links
- Characteristics of Packet Capture
- Data Measurement Repositories
- Multihoming and Traffic Engineering
- What is Multihoming
- Why Multihome
- Redundancy
- Performance
- Multihoming in IP Networks Today
- BGP or no
- Same Provider or Multiple
- Multihomed Stub One Link
- Multihomed Stub Multiple Links
- Multihomed Stub Multiple ISPs
- Multihomed Transit Network
- Interdomain Traffic Engineering
- Outbound Traffic Control
- Outbound Traffic Load Balancing
- Traffic Engineering Goals
- Managing Scale
- Achieving Predictability
- Challenges to Predictability
- Inter-AS Negotiation
- Outbound Multihoming Goals
- Inbound Traffic Control
- How many links are enough
- Problems with Multihoming in IPv4
- Slide 68
- Slide 69
- Diagnosis and Troubleshooting
- Operator Mailing List (NANOG)
- Operator Mailing List
- Routing Configuration
- Internet Business Model (Simplified)
- Peering Contracts Consistent Export
- Consistent Export
- Inconsistent Export in Practice
- Blackholes
- Security
- Security ldquoBogonrdquo Routes
- Spam Phishing etc
- Spam and Routing
- Worms and Botnets
- What is a Worm
- Example Worm Code Red
- Code Red Revisions
- Code Red Host Infection Rate
- Designing Fast-Spreading Worms
- Botnets
- ldquoRallyingrdquo the Botnet
- Botnet Control
- Some Defenses
- Idea 1 Ingress Filtering
- Idea 2 uRPF Checks
- Problems with uRPF
- S-BGP
- Practical Problems with S-BGP
-
59
Traffic Engineering Goals
bull Predictabilityndash Ensure the BGP decision process is deterministicndash Assume that BGP updates are (relatively) stable
bull Limit overhead introduced by routing changesndash Minimize frequency of changes to routing policiesndash Limit number of prefixes affected by changes
bull Limit impact on how traffic enters the networkndash Avoid new routes that might change neighborrsquos mindndash Select route with same attributes or at least path length
60
Managing Scalebull Destination prefixes
ndash More than 90000 destination prefixes
bull Donrsquot want to have per-prefix routing policies
ndash Small fraction of prefixes contribute most of the traffic
bull Focus on the small number of heavy hitters
ndash Define routing policies for selected prefixes
bull Routing choicesndash About 27000 unique ldquorouting choicesrdquo
bull Help in reducing the scale of the problem
ndash Small fraction of ldquorouting choicesrdquo contribute most traffic
bull Focus on the very small number of ldquorouting choicesrdquo
ndash Define routing policies on common attributes
61
Achieving Predictability
bull Route prediction with static analysisndash Helpful to know effects before deploymentndash Static analysis can help
TopologyBGP policy
configuration
eBGP routes
Offered traffic
BGP routingmodel
Flow of traffic through the network
62
Challenges to Predictabilitybull For transit ISPs effects on incoming traffic
ndash Lack of coordination strikes again
63
ldquoHot Potatordquo routing
Inter-AS Negotiation
bull Coordination aids predictabilityndash Negotiate where to sendndash Inbound and outboundndash Mutual benefits
bull How to implementndash What info to exchangendash Protecting privacyndash How to prioritize choicesndash How to prevent cheating
Destination 2
Destination 1
multiplepeeringpoints
Provider A
Provider B
64
Outbound Multihoming Goals
bull Redundancyndash Dynamic routing will failover to backup link
bull Performancendash Select provider with best performance per prefix
ndash Requires active probing
bull Costndash Select provider per prefix over time to minimize the
total financial cost
65
Inbound Traffic Control
bull More difficult no control over neighborsrsquo decisions
bull Three common techniques (previously discussed)ndash AS path prependingndash Communities and local preferencendash Prefix splitting
How does todayrsquos paper (MONET) control inbound traffic
66
How many links are enough
K upstream ISPs
Not much benefit beyond 4 ISPs
Akella et al ldquoPerformance Benefits of Multihomingrdquo SIGCOMM 2003
67
Problems with Multihoming in IPv4
bull Routing table growthndash Provider-based addressingndash Advertising prefix out multiple ISPs ndash canrsquot aggregate
bull Poor control over inbound trafficndash Existing mechanisms do not allow hosts to control
inbound traffic
68
GeorgiaTech
Internet Routing Overview
bull Intradomain (ie ldquointra-ASrdquo) routingbull Interdomain routing
Comcast
Abilene
ATampT Cogent
Autonomous Systems (ASes)
69
Configuration Problems ldquoAS 7007rdquoldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
UUNet
Florida InternetBarn
Sprint
70
Diagnosis and Troubleshootingldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
ldquoMicrosofts websites were offline for up to 23 hoursbecause of a [router] misconfigurationhellipit took nearly a day to determine what was wrong and undo the changesrdquo -- wiredcom January 25 2001
ldquoWorldCom Inchellipsuffered a widespread outage on its Internet backbone that affected roughly 20 percent of its US customer base The network problemshellipaffected millions of computer users worldwide A spokeswoman attributed the outage to a route table issue -- cnncom October 3 2002
A number of Covad customers went out from 5pm today due to supposedly a DDOS (distributed denial of service attack) on a key Level3 data center which later was described as a route leak (misconfiguration)ldquo
-- dslreportscom February 23 2004
71
Operator Mailing List (NANOG)Date Mon 18 Oct 2004 091515 -0700Subject Level 3 US east coast issues
Level 3 experiencing widespread unspecified routing issues on the US east coast Master ticket 1086844 Anyone have more specific information
Date Mon 18 Oct 2004 122034 -0400 (EDT)Subject Re Level 3 US east coast issues
Level 3 is currently experiencing a backbone outage causing routing instability and packet loss We are working to restore and will be sending hourly updateshellip
72
Operator Mailing List
0102030405060708090
Filtering RouteLeaks
RouteHijacks
RouteInstability
RoutingLoops
Blackholes
Occurr
ences o
ver
10 Y
ears
1995-1997 1998-2001 2001-2004
Note Only includes problems openly discussed on this list
Compare 83 power outages 1 fire
73
Routing Configuration
Ranking route selection
Dissemination internal route advertisement
Filtering route advertisement
Customer
Competitor
Primary
Backup
74
Internet Business Model (Simplified)
bull CustomerProvider One AS pays another for reachability to some set of destinations
bull ldquoSettlement-freerdquo Peering Bartering Two ASes exchange routes with one another
Provider
Peer
Customer
Preferences implemented with local preference manipulation
Destination
Pay to use
Get paid to use
Free to use
75
Peering Contracts Consistent Export
bull Rules of settlement-free peeringndash Advertise routes at all peering pointsndash Advertised routes must have equal ldquoAS path lengthrdquo
Sprint
ATampT
Enables ldquohot potatordquo routing
ldquoequally goodrdquoroutes
76
Consistent Export
bull Malicedeceptionbull iBGP signaling partitionbull Inconsistent export policy
neighbor 10123route-map PEER permit 10 set prepend 123
neighbor 10456route-map PEER permit 10 set prepend 123 123
Possible Causes
10123 456 1
10456 456 2
Neighbor AS Export
1 1 123
Export Clause Prepend
2 1 123 123
Two different Export Policies
77
Inconsistent Export in Practice
Feamster et al ldquoBorderGuard Detecting Cold Potatoes from Peersrdquo ACM IMC October 2004
78
Blackholes
Date Thu 18 Jul 2002 060510 -0400 (EDT)From Chad Oleary ltcolpoboxcomgtSubject Re problems with 701To ltnanogmeritedugt
Were starting to see the same issues with UUNet again Anyone elseseeing this Trying to reach Qwest
traceroute to 631461901 (631461901) 30 hops max 38 byte packets 1 esc-lp2-gwe-solutionscorpcom (631182201) 1167 ms 1163 ms 1142 ms 2 500Serial2-10GW1TPA2ALTERNET (1571301499) 1097 ms 1059 ms 1044 ms 3 161at-1-0-0XL4ATL1ALTERNET (1526381190) 13839 ms 14108 ms 16638 ms 4 0so-3-1-0XL2ATL5ALTERNET (152630238) 14370 ms 14587 ms 14553 ms 5 POS7-0BR2ATL5ALTERNET (1526382193) 13928 ms 14099 ms 14053 ms 6 7 hellip
79
Security
80
Security ldquoBogonrdquo Routes
Feamster et al ldquoAn Empirical Study of lsquoBogonrsquo Route Advertisementsrdquo ACM CCR January 2005
81
Spam Phishing etc
bull Unsolicited commercial emailbull As of about August 2008 estimates indicate that
about 95 of all email is spambull Common spam filtering techniques
ndash Content-based filtersndash DNS Blacklist (DNSBL) lookups Significant fraction of
todayrsquos DNS traffic
Can IP addresses from which spam is received be spoofed
82
Spam and Routing
83
Worms and Botnets
84
What is a Worm
bull Code that replicates and propagates across the networkndash Often carries a ldquopayloadrdquo
bull Usually spread via exploiting flaws in open servicesndash ldquoVirusesrdquo require user action to spread
bull First worm Robert Morris November 1988ndash 6-10 of all Internet hosts infected ()
bull Many more since but none on that scale until July 2001
85
Example Worm Code Red
bull Initial version July 13 2001
bull Exploited known ISAPI vulnerability in Microsoft IIS Web servers
bull 1st through 20th of each month spread20th through end of each month attack
bull Payload Web site defacementbull Scanning Random IP addressesbull Bug failure to seed random number generator
86
Code Red Revisions
bull Released July 19 2001
bull Payload flooding attack on wwwwhitehousegovndash Attack was mounted at the IP address of the Web site
bull Bug died after 20th of each month
bull Random number generator for IP scanning fixed
87
Code Red Host Infection Rate
Exponential infection rate
Measured using backscatter technique
88
Designing Fast-Spreading Worms
bull Hit-list scanningndash Time to infect first 10k hosts dominates infection timendash Solution Reconnaissance (stealthy scans etc)
bull Permutation scanningndash Observation Most scanning is redundantndash Idea Shared permutation of address space Start scanning
from own IP address Re-randomize when another infected machine is found
bull Internet-scale hit listsndash Flash worm complete infection within 30 seconds
89
Botnets
bull Bots Autonomous programs performing tasksbull Plenty of ldquobenignrdquo bots
ndash eg weatherbug
bull Botnets group of bots ndash Typically carries malicious connotationndash Large numbers of infected machinesndash Machines ldquoenlistedrdquo with infection vectors like worms
(last lecture)
bull Available for simultaneous control by a masterbull Size up to 350000 nodes (from todayrsquos paper)
90
ldquoRallyingrdquo the Botnet
bull Easy to combine worm backdoor functionalitybull Problem how to learn about successfully
infected machines
bull Optionsndash Emailndash Hard-coded email address
91
Botnet Control
bull Botnet master typically runs some IRC server on a well-known port (eg 6667)
bull Infected machine contacts botnet with pre-programmed DNS name (eg big-botde)
bull Dynamic DNS allows controller to move about freely
Infected Machine
DynamicDNS
BotnetController
(IRC server)
92
Some Defenses
93
Idea 1 Ingress Filtering
bull RFC 2827 Routers install filters to drop packets from networks that are not downstream
bull Feasible at edgesbull Difficult to configure closer to network ldquocorerdquo
20469207024 Internet
Drop all packets with source address other than 20469207024
94
Idea 2 uRPF Checks
bull Unicast Reverse Path Forwardingndash Cisco ldquoip verify unicast reverse-pathrdquo
bull Requires symmetric routing
Accept packet from interface only if forwarding table entry for source IP address matches ingress interface
100183
A10018
10015
101203
100181 24
10011 24
Strict Mode uRPF
Enabled
ldquoArdquo Routing TableDestination Next Hop1001024 Int 110018024 Int 2
100183 from wrong interface
95
Problems with uRPF
bull Asymmetric routing
96
S-BGP
bull Address-based PKI validate signaturesndash Authentication of
bull ownership for IP address blocks bull AS number bull an ASs identity and bull a BGP routers identity
ndash Use existing infrastructure (Internet registries etc)ndash Routing origination is digitally signedndash BGP updates are digitally signed
1048708 bull Route attestations A new optional BGP transitive path attribute
ndash carries digital signatures covering the routing information in updates
97
Practical Problems with S-BGP
bull Requires Public-Key Infrastructure
bull Lots of digital signatures to calculate and verifyndash Message overheadndash CPU overhead
bull Calculation expense is greatest when topology is changingndash Caching can help
bull Route aggregation is problematic (maybe thatrsquos OK)
bull Secure route withdrawals when link or node fails
bull Address ownership data out of date
bull Deployment
- Networking
- Goal of This Tutorial
- Todayrsquos Networks
- Business Models
- Billing for Internet Usage
- Net Neutrality
- Network Operations
- Point-of-Presence (PoP)
- Example Abilene Network Topology
- Another Example Backbone
- Internet Routing Overview
- Internet Routing Protocol BGP
- Two Flavors of BGP
- IPv4 Addresses Networks of Networks
- Pre-1994 Classful Addressing
- Classless Interdomain Routing (CIDR)
- Benefits of CIDR
- Growth of IP Prefixes
- 1994-1998 Linear Growth
- Around 2000 Fast Growth Resumes
- Fast growth resumes
- The Address Allocation Process
- Common Problems
- What can go wrong
- Measurement and Monitoring
- Passive vs Active Measurement
- Slide 27
- Passive Traffic Data Measurement
- Simple Network Management Protocol
- SNMP (Passive)
- Packet-level Monitoring
- Full Packet Capture (Passive)
- What is a flow
- Cisco NetFlow
- Flow Record Contents
- Aggregating Packets into Flows
- Reducing Measurement Overhead
- Packet Sampling for Flow Monitoring
- Sampling Flow-Level Sampling
- Two Main Approaches
- Packet Capture on High-Speed Links
- Characteristics of Packet Capture
- Data Measurement Repositories
- Multihoming and Traffic Engineering
- What is Multihoming
- Why Multihome
- Redundancy
- Performance
- Multihoming in IP Networks Today
- BGP or no
- Same Provider or Multiple
- Multihomed Stub One Link
- Multihomed Stub Multiple Links
- Multihomed Stub Multiple ISPs
- Multihomed Transit Network
- Interdomain Traffic Engineering
- Outbound Traffic Control
- Outbound Traffic Load Balancing
- Traffic Engineering Goals
- Managing Scale
- Achieving Predictability
- Challenges to Predictability
- Inter-AS Negotiation
- Outbound Multihoming Goals
- Inbound Traffic Control
- How many links are enough
- Problems with Multihoming in IPv4
- Slide 68
- Slide 69
- Diagnosis and Troubleshooting
- Operator Mailing List (NANOG)
- Operator Mailing List
- Routing Configuration
- Internet Business Model (Simplified)
- Peering Contracts Consistent Export
- Consistent Export
- Inconsistent Export in Practice
- Blackholes
- Security
- Security ldquoBogonrdquo Routes
- Spam Phishing etc
- Spam and Routing
- Worms and Botnets
- What is a Worm
- Example Worm Code Red
- Code Red Revisions
- Code Red Host Infection Rate
- Designing Fast-Spreading Worms
- Botnets
- ldquoRallyingrdquo the Botnet
- Botnet Control
- Some Defenses
- Idea 1 Ingress Filtering
- Idea 2 uRPF Checks
- Problems with uRPF
- S-BGP
- Practical Problems with S-BGP
-
60
Managing Scalebull Destination prefixes
ndash More than 90000 destination prefixes
bull Donrsquot want to have per-prefix routing policies
ndash Small fraction of prefixes contribute most of the traffic
bull Focus on the small number of heavy hitters
ndash Define routing policies for selected prefixes
bull Routing choicesndash About 27000 unique ldquorouting choicesrdquo
bull Help in reducing the scale of the problem
ndash Small fraction of ldquorouting choicesrdquo contribute most traffic
bull Focus on the very small number of ldquorouting choicesrdquo
ndash Define routing policies on common attributes
61
Achieving Predictability
bull Route prediction with static analysisndash Helpful to know effects before deploymentndash Static analysis can help
TopologyBGP policy
configuration
eBGP routes
Offered traffic
BGP routingmodel
Flow of traffic through the network
62
Challenges to Predictabilitybull For transit ISPs effects on incoming traffic
ndash Lack of coordination strikes again
63
ldquoHot Potatordquo routing
Inter-AS Negotiation
bull Coordination aids predictabilityndash Negotiate where to sendndash Inbound and outboundndash Mutual benefits
bull How to implementndash What info to exchangendash Protecting privacyndash How to prioritize choicesndash How to prevent cheating
Destination 2
Destination 1
multiplepeeringpoints
Provider A
Provider B
64
Outbound Multihoming Goals
bull Redundancyndash Dynamic routing will failover to backup link
bull Performancendash Select provider with best performance per prefix
ndash Requires active probing
bull Costndash Select provider per prefix over time to minimize the
total financial cost
65
Inbound Traffic Control
bull More difficult no control over neighborsrsquo decisions
bull Three common techniques (previously discussed)ndash AS path prependingndash Communities and local preferencendash Prefix splitting
How does todayrsquos paper (MONET) control inbound traffic
66
How many links are enough
K upstream ISPs
Not much benefit beyond 4 ISPs
Akella et al ldquoPerformance Benefits of Multihomingrdquo SIGCOMM 2003
67
Problems with Multihoming in IPv4
bull Routing table growthndash Provider-based addressingndash Advertising prefix out multiple ISPs ndash canrsquot aggregate
bull Poor control over inbound trafficndash Existing mechanisms do not allow hosts to control
inbound traffic
68
GeorgiaTech
Internet Routing Overview
bull Intradomain (ie ldquointra-ASrdquo) routingbull Interdomain routing
Comcast
Abilene
ATampT Cogent
Autonomous Systems (ASes)
69
Configuration Problems ldquoAS 7007rdquoldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
UUNet
Florida InternetBarn
Sprint
70
Diagnosis and Troubleshootingldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
ldquoMicrosofts websites were offline for up to 23 hoursbecause of a [router] misconfigurationhellipit took nearly a day to determine what was wrong and undo the changesrdquo -- wiredcom January 25 2001
ldquoWorldCom Inchellipsuffered a widespread outage on its Internet backbone that affected roughly 20 percent of its US customer base The network problemshellipaffected millions of computer users worldwide A spokeswoman attributed the outage to a route table issue -- cnncom October 3 2002
A number of Covad customers went out from 5pm today due to supposedly a DDOS (distributed denial of service attack) on a key Level3 data center which later was described as a route leak (misconfiguration)ldquo
-- dslreportscom February 23 2004
71
Operator Mailing List (NANOG)Date Mon 18 Oct 2004 091515 -0700Subject Level 3 US east coast issues
Level 3 experiencing widespread unspecified routing issues on the US east coast Master ticket 1086844 Anyone have more specific information
Date Mon 18 Oct 2004 122034 -0400 (EDT)Subject Re Level 3 US east coast issues
Level 3 is currently experiencing a backbone outage causing routing instability and packet loss We are working to restore and will be sending hourly updateshellip
72
Operator Mailing List
0102030405060708090
Filtering RouteLeaks
RouteHijacks
RouteInstability
RoutingLoops
Blackholes
Occurr
ences o
ver
10 Y
ears
1995-1997 1998-2001 2001-2004
Note Only includes problems openly discussed on this list
Compare 83 power outages 1 fire
73
Routing Configuration
Ranking route selection
Dissemination internal route advertisement
Filtering route advertisement
Customer
Competitor
Primary
Backup
74
Internet Business Model (Simplified)
bull CustomerProvider One AS pays another for reachability to some set of destinations
bull ldquoSettlement-freerdquo Peering Bartering Two ASes exchange routes with one another
Provider
Peer
Customer
Preferences implemented with local preference manipulation
Destination
Pay to use
Get paid to use
Free to use
75
Peering Contracts Consistent Export
bull Rules of settlement-free peeringndash Advertise routes at all peering pointsndash Advertised routes must have equal ldquoAS path lengthrdquo
Sprint
ATampT
Enables ldquohot potatordquo routing
ldquoequally goodrdquoroutes
76
Consistent Export
bull Malicedeceptionbull iBGP signaling partitionbull Inconsistent export policy
neighbor 10123route-map PEER permit 10 set prepend 123
neighbor 10456route-map PEER permit 10 set prepend 123 123
Possible Causes
10123 456 1
10456 456 2
Neighbor AS Export
1 1 123
Export Clause Prepend
2 1 123 123
Two different Export Policies
77
Inconsistent Export in Practice
Feamster et al ldquoBorderGuard Detecting Cold Potatoes from Peersrdquo ACM IMC October 2004
78
Blackholes
Date Thu 18 Jul 2002 060510 -0400 (EDT)From Chad Oleary ltcolpoboxcomgtSubject Re problems with 701To ltnanogmeritedugt
Were starting to see the same issues with UUNet again Anyone elseseeing this Trying to reach Qwest
traceroute to 631461901 (631461901) 30 hops max 38 byte packets 1 esc-lp2-gwe-solutionscorpcom (631182201) 1167 ms 1163 ms 1142 ms 2 500Serial2-10GW1TPA2ALTERNET (1571301499) 1097 ms 1059 ms 1044 ms 3 161at-1-0-0XL4ATL1ALTERNET (1526381190) 13839 ms 14108 ms 16638 ms 4 0so-3-1-0XL2ATL5ALTERNET (152630238) 14370 ms 14587 ms 14553 ms 5 POS7-0BR2ATL5ALTERNET (1526382193) 13928 ms 14099 ms 14053 ms 6 7 hellip
79
Security
80
Security ldquoBogonrdquo Routes
Feamster et al ldquoAn Empirical Study of lsquoBogonrsquo Route Advertisementsrdquo ACM CCR January 2005
81
Spam Phishing etc
bull Unsolicited commercial emailbull As of about August 2008 estimates indicate that
about 95 of all email is spambull Common spam filtering techniques
ndash Content-based filtersndash DNS Blacklist (DNSBL) lookups Significant fraction of
todayrsquos DNS traffic
Can IP addresses from which spam is received be spoofed
82
Spam and Routing
83
Worms and Botnets
84
What is a Worm
bull Code that replicates and propagates across the networkndash Often carries a ldquopayloadrdquo
bull Usually spread via exploiting flaws in open servicesndash ldquoVirusesrdquo require user action to spread
bull First worm Robert Morris November 1988ndash 6-10 of all Internet hosts infected ()
bull Many more since but none on that scale until July 2001
85
Example Worm Code Red
bull Initial version July 13 2001
bull Exploited known ISAPI vulnerability in Microsoft IIS Web servers
bull 1st through 20th of each month spread20th through end of each month attack
bull Payload Web site defacementbull Scanning Random IP addressesbull Bug failure to seed random number generator
86
Code Red Revisions
bull Released July 19 2001
bull Payload flooding attack on wwwwhitehousegovndash Attack was mounted at the IP address of the Web site
bull Bug died after 20th of each month
bull Random number generator for IP scanning fixed
87
Code Red Host Infection Rate
Exponential infection rate
Measured using backscatter technique
88
Designing Fast-Spreading Worms
bull Hit-list scanningndash Time to infect first 10k hosts dominates infection timendash Solution Reconnaissance (stealthy scans etc)
bull Permutation scanningndash Observation Most scanning is redundantndash Idea Shared permutation of address space Start scanning
from own IP address Re-randomize when another infected machine is found
bull Internet-scale hit listsndash Flash worm complete infection within 30 seconds
89
Botnets
bull Bots Autonomous programs performing tasksbull Plenty of ldquobenignrdquo bots
ndash eg weatherbug
bull Botnets group of bots ndash Typically carries malicious connotationndash Large numbers of infected machinesndash Machines ldquoenlistedrdquo with infection vectors like worms
(last lecture)
bull Available for simultaneous control by a masterbull Size up to 350000 nodes (from todayrsquos paper)
90
ldquoRallyingrdquo the Botnet
bull Easy to combine worm backdoor functionalitybull Problem how to learn about successfully
infected machines
bull Optionsndash Emailndash Hard-coded email address
91
Botnet Control
bull Botnet master typically runs some IRC server on a well-known port (eg 6667)
bull Infected machine contacts botnet with pre-programmed DNS name (eg big-botde)
bull Dynamic DNS allows controller to move about freely
Infected Machine
DynamicDNS
BotnetController
(IRC server)
92
Some Defenses
93
Idea 1 Ingress Filtering
bull RFC 2827 Routers install filters to drop packets from networks that are not downstream
bull Feasible at edgesbull Difficult to configure closer to network ldquocorerdquo
20469207024 Internet
Drop all packets with source address other than 20469207024
94
Idea 2 uRPF Checks
bull Unicast Reverse Path Forwardingndash Cisco ldquoip verify unicast reverse-pathrdquo
bull Requires symmetric routing
Accept packet from interface only if forwarding table entry for source IP address matches ingress interface
100183
A10018
10015
101203
100181 24
10011 24
Strict Mode uRPF
Enabled
ldquoArdquo Routing TableDestination Next Hop1001024 Int 110018024 Int 2
100183 from wrong interface
95
Problems with uRPF
bull Asymmetric routing
96
S-BGP
bull Address-based PKI validate signaturesndash Authentication of
bull ownership for IP address blocks bull AS number bull an ASs identity and bull a BGP routers identity
ndash Use existing infrastructure (Internet registries etc)ndash Routing origination is digitally signedndash BGP updates are digitally signed
1048708 bull Route attestations A new optional BGP transitive path attribute
ndash carries digital signatures covering the routing information in updates
97
Practical Problems with S-BGP
bull Requires Public-Key Infrastructure
bull Lots of digital signatures to calculate and verifyndash Message overheadndash CPU overhead
bull Calculation expense is greatest when topology is changingndash Caching can help
bull Route aggregation is problematic (maybe thatrsquos OK)
bull Secure route withdrawals when link or node fails
bull Address ownership data out of date
bull Deployment
- Networking
- Goal of This Tutorial
- Todayrsquos Networks
- Business Models
- Billing for Internet Usage
- Net Neutrality
- Network Operations
- Point-of-Presence (PoP)
- Example Abilene Network Topology
- Another Example Backbone
- Internet Routing Overview
- Internet Routing Protocol BGP
- Two Flavors of BGP
- IPv4 Addresses Networks of Networks
- Pre-1994 Classful Addressing
- Classless Interdomain Routing (CIDR)
- Benefits of CIDR
- Growth of IP Prefixes
- 1994-1998 Linear Growth
- Around 2000 Fast Growth Resumes
- Fast growth resumes
- The Address Allocation Process
- Common Problems
- What can go wrong
- Measurement and Monitoring
- Passive vs Active Measurement
- Slide 27
- Passive Traffic Data Measurement
- Simple Network Management Protocol
- SNMP (Passive)
- Packet-level Monitoring
- Full Packet Capture (Passive)
- What is a flow
- Cisco NetFlow
- Flow Record Contents
- Aggregating Packets into Flows
- Reducing Measurement Overhead
- Packet Sampling for Flow Monitoring
- Sampling Flow-Level Sampling
- Two Main Approaches
- Packet Capture on High-Speed Links
- Characteristics of Packet Capture
- Data Measurement Repositories
- Multihoming and Traffic Engineering
- What is Multihoming
- Why Multihome
- Redundancy
- Performance
- Multihoming in IP Networks Today
- BGP or no
- Same Provider or Multiple
- Multihomed Stub One Link
- Multihomed Stub Multiple Links
- Multihomed Stub Multiple ISPs
- Multihomed Transit Network
- Interdomain Traffic Engineering
- Outbound Traffic Control
- Outbound Traffic Load Balancing
- Traffic Engineering Goals
- Managing Scale
- Achieving Predictability
- Challenges to Predictability
- Inter-AS Negotiation
- Outbound Multihoming Goals
- Inbound Traffic Control
- How many links are enough
- Problems with Multihoming in IPv4
- Slide 68
- Slide 69
- Diagnosis and Troubleshooting
- Operator Mailing List (NANOG)
- Operator Mailing List
- Routing Configuration
- Internet Business Model (Simplified)
- Peering Contracts Consistent Export
- Consistent Export
- Inconsistent Export in Practice
- Blackholes
- Security
- Security ldquoBogonrdquo Routes
- Spam Phishing etc
- Spam and Routing
- Worms and Botnets
- What is a Worm
- Example Worm Code Red
- Code Red Revisions
- Code Red Host Infection Rate
- Designing Fast-Spreading Worms
- Botnets
- ldquoRallyingrdquo the Botnet
- Botnet Control
- Some Defenses
- Idea 1 Ingress Filtering
- Idea 2 uRPF Checks
- Problems with uRPF
- S-BGP
- Practical Problems with S-BGP
-
61
Achieving Predictability
bull Route prediction with static analysisndash Helpful to know effects before deploymentndash Static analysis can help
TopologyBGP policy
configuration
eBGP routes
Offered traffic
BGP routingmodel
Flow of traffic through the network
62
Challenges to Predictabilitybull For transit ISPs effects on incoming traffic
ndash Lack of coordination strikes again
63
ldquoHot Potatordquo routing
Inter-AS Negotiation
bull Coordination aids predictabilityndash Negotiate where to sendndash Inbound and outboundndash Mutual benefits
bull How to implementndash What info to exchangendash Protecting privacyndash How to prioritize choicesndash How to prevent cheating
Destination 2
Destination 1
multiplepeeringpoints
Provider A
Provider B
64
Outbound Multihoming Goals
bull Redundancyndash Dynamic routing will failover to backup link
bull Performancendash Select provider with best performance per prefix
ndash Requires active probing
bull Costndash Select provider per prefix over time to minimize the
total financial cost
65
Inbound Traffic Control
bull More difficult no control over neighborsrsquo decisions
bull Three common techniques (previously discussed)ndash AS path prependingndash Communities and local preferencendash Prefix splitting
How does todayrsquos paper (MONET) control inbound traffic
66
How many links are enough
K upstream ISPs
Not much benefit beyond 4 ISPs
Akella et al ldquoPerformance Benefits of Multihomingrdquo SIGCOMM 2003
67
Problems with Multihoming in IPv4
bull Routing table growthndash Provider-based addressingndash Advertising prefix out multiple ISPs ndash canrsquot aggregate
bull Poor control over inbound trafficndash Existing mechanisms do not allow hosts to control
inbound traffic
68
GeorgiaTech
Internet Routing Overview
bull Intradomain (ie ldquointra-ASrdquo) routingbull Interdomain routing
Comcast
Abilene
ATampT Cogent
Autonomous Systems (ASes)
69
Configuration Problems ldquoAS 7007rdquoldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
UUNet
Florida InternetBarn
Sprint
70
Diagnosis and Troubleshootingldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
ldquoMicrosofts websites were offline for up to 23 hoursbecause of a [router] misconfigurationhellipit took nearly a day to determine what was wrong and undo the changesrdquo -- wiredcom January 25 2001
ldquoWorldCom Inchellipsuffered a widespread outage on its Internet backbone that affected roughly 20 percent of its US customer base The network problemshellipaffected millions of computer users worldwide A spokeswoman attributed the outage to a route table issue -- cnncom October 3 2002
A number of Covad customers went out from 5pm today due to supposedly a DDOS (distributed denial of service attack) on a key Level3 data center which later was described as a route leak (misconfiguration)ldquo
-- dslreportscom February 23 2004
71
Operator Mailing List (NANOG)Date Mon 18 Oct 2004 091515 -0700Subject Level 3 US east coast issues
Level 3 experiencing widespread unspecified routing issues on the US east coast Master ticket 1086844 Anyone have more specific information
Date Mon 18 Oct 2004 122034 -0400 (EDT)Subject Re Level 3 US east coast issues
Level 3 is currently experiencing a backbone outage causing routing instability and packet loss We are working to restore and will be sending hourly updateshellip
72
Operator Mailing List
0102030405060708090
Filtering RouteLeaks
RouteHijacks
RouteInstability
RoutingLoops
Blackholes
Occurr
ences o
ver
10 Y
ears
1995-1997 1998-2001 2001-2004
Note Only includes problems openly discussed on this list
Compare 83 power outages 1 fire
73
Routing Configuration
Ranking route selection
Dissemination internal route advertisement
Filtering route advertisement
Customer
Competitor
Primary
Backup
74
Internet Business Model (Simplified)
bull CustomerProvider One AS pays another for reachability to some set of destinations
bull ldquoSettlement-freerdquo Peering Bartering Two ASes exchange routes with one another
Provider
Peer
Customer
Preferences implemented with local preference manipulation
Destination
Pay to use
Get paid to use
Free to use
75
Peering Contracts Consistent Export
bull Rules of settlement-free peeringndash Advertise routes at all peering pointsndash Advertised routes must have equal ldquoAS path lengthrdquo
Sprint
ATampT
Enables ldquohot potatordquo routing
ldquoequally goodrdquoroutes
76
Consistent Export
bull Malicedeceptionbull iBGP signaling partitionbull Inconsistent export policy
neighbor 10123route-map PEER permit 10 set prepend 123
neighbor 10456route-map PEER permit 10 set prepend 123 123
Possible Causes
10123 456 1
10456 456 2
Neighbor AS Export
1 1 123
Export Clause Prepend
2 1 123 123
Two different Export Policies
77
Inconsistent Export in Practice
Feamster et al ldquoBorderGuard Detecting Cold Potatoes from Peersrdquo ACM IMC October 2004
78
Blackholes
Date Thu 18 Jul 2002 060510 -0400 (EDT)From Chad Oleary ltcolpoboxcomgtSubject Re problems with 701To ltnanogmeritedugt
Were starting to see the same issues with UUNet again Anyone elseseeing this Trying to reach Qwest
traceroute to 631461901 (631461901) 30 hops max 38 byte packets 1 esc-lp2-gwe-solutionscorpcom (631182201) 1167 ms 1163 ms 1142 ms 2 500Serial2-10GW1TPA2ALTERNET (1571301499) 1097 ms 1059 ms 1044 ms 3 161at-1-0-0XL4ATL1ALTERNET (1526381190) 13839 ms 14108 ms 16638 ms 4 0so-3-1-0XL2ATL5ALTERNET (152630238) 14370 ms 14587 ms 14553 ms 5 POS7-0BR2ATL5ALTERNET (1526382193) 13928 ms 14099 ms 14053 ms 6 7 hellip
79
Security
80
Security ldquoBogonrdquo Routes
Feamster et al ldquoAn Empirical Study of lsquoBogonrsquo Route Advertisementsrdquo ACM CCR January 2005
81
Spam Phishing etc
bull Unsolicited commercial emailbull As of about August 2008 estimates indicate that
about 95 of all email is spambull Common spam filtering techniques
ndash Content-based filtersndash DNS Blacklist (DNSBL) lookups Significant fraction of
todayrsquos DNS traffic
Can IP addresses from which spam is received be spoofed
82
Spam and Routing
83
Worms and Botnets
84
What is a Worm
bull Code that replicates and propagates across the networkndash Often carries a ldquopayloadrdquo
bull Usually spread via exploiting flaws in open servicesndash ldquoVirusesrdquo require user action to spread
bull First worm Robert Morris November 1988ndash 6-10 of all Internet hosts infected ()
bull Many more since but none on that scale until July 2001
85
Example Worm Code Red
bull Initial version July 13 2001
bull Exploited known ISAPI vulnerability in Microsoft IIS Web servers
bull 1st through 20th of each month spread20th through end of each month attack
bull Payload Web site defacementbull Scanning Random IP addressesbull Bug failure to seed random number generator
86
Code Red Revisions
bull Released July 19 2001
bull Payload flooding attack on wwwwhitehousegovndash Attack was mounted at the IP address of the Web site
bull Bug died after 20th of each month
bull Random number generator for IP scanning fixed
87
Code Red Host Infection Rate
Exponential infection rate
Measured using backscatter technique
88
Designing Fast-Spreading Worms
bull Hit-list scanningndash Time to infect first 10k hosts dominates infection timendash Solution Reconnaissance (stealthy scans etc)
bull Permutation scanningndash Observation Most scanning is redundantndash Idea Shared permutation of address space Start scanning
from own IP address Re-randomize when another infected machine is found
bull Internet-scale hit listsndash Flash worm complete infection within 30 seconds
89
Botnets
bull Bots Autonomous programs performing tasksbull Plenty of ldquobenignrdquo bots
ndash eg weatherbug
bull Botnets group of bots ndash Typically carries malicious connotationndash Large numbers of infected machinesndash Machines ldquoenlistedrdquo with infection vectors like worms
(last lecture)
bull Available for simultaneous control by a masterbull Size up to 350000 nodes (from todayrsquos paper)
90
ldquoRallyingrdquo the Botnet
bull Easy to combine worm backdoor functionalitybull Problem how to learn about successfully
infected machines
bull Optionsndash Emailndash Hard-coded email address
91
Botnet Control
bull Botnet master typically runs some IRC server on a well-known port (eg 6667)
bull Infected machine contacts botnet with pre-programmed DNS name (eg big-botde)
bull Dynamic DNS allows controller to move about freely
Infected Machine
DynamicDNS
BotnetController
(IRC server)
92
Some Defenses
93
Idea 1 Ingress Filtering
bull RFC 2827 Routers install filters to drop packets from networks that are not downstream
bull Feasible at edgesbull Difficult to configure closer to network ldquocorerdquo
20469207024 Internet
Drop all packets with source address other than 20469207024
94
Idea 2 uRPF Checks
bull Unicast Reverse Path Forwardingndash Cisco ldquoip verify unicast reverse-pathrdquo
bull Requires symmetric routing
Accept packet from interface only if forwarding table entry for source IP address matches ingress interface
100183
A10018
10015
101203
100181 24
10011 24
Strict Mode uRPF
Enabled
ldquoArdquo Routing TableDestination Next Hop1001024 Int 110018024 Int 2
100183 from wrong interface
95
Problems with uRPF
bull Asymmetric routing
96
S-BGP
bull Address-based PKI validate signaturesndash Authentication of
bull ownership for IP address blocks bull AS number bull an ASs identity and bull a BGP routers identity
ndash Use existing infrastructure (Internet registries etc)ndash Routing origination is digitally signedndash BGP updates are digitally signed
1048708 bull Route attestations A new optional BGP transitive path attribute
ndash carries digital signatures covering the routing information in updates
97
Practical Problems with S-BGP
bull Requires Public-Key Infrastructure
bull Lots of digital signatures to calculate and verifyndash Message overheadndash CPU overhead
bull Calculation expense is greatest when topology is changingndash Caching can help
bull Route aggregation is problematic (maybe thatrsquos OK)
bull Secure route withdrawals when link or node fails
bull Address ownership data out of date
bull Deployment
- Networking
- Goal of This Tutorial
- Todayrsquos Networks
- Business Models
- Billing for Internet Usage
- Net Neutrality
- Network Operations
- Point-of-Presence (PoP)
- Example Abilene Network Topology
- Another Example Backbone
- Internet Routing Overview
- Internet Routing Protocol BGP
- Two Flavors of BGP
- IPv4 Addresses Networks of Networks
- Pre-1994 Classful Addressing
- Classless Interdomain Routing (CIDR)
- Benefits of CIDR
- Growth of IP Prefixes
- 1994-1998 Linear Growth
- Around 2000 Fast Growth Resumes
- Fast growth resumes
- The Address Allocation Process
- Common Problems
- What can go wrong
- Measurement and Monitoring
- Passive vs Active Measurement
- Slide 27
- Passive Traffic Data Measurement
- Simple Network Management Protocol
- SNMP (Passive)
- Packet-level Monitoring
- Full Packet Capture (Passive)
- What is a flow
- Cisco NetFlow
- Flow Record Contents
- Aggregating Packets into Flows
- Reducing Measurement Overhead
- Packet Sampling for Flow Monitoring
- Sampling Flow-Level Sampling
- Two Main Approaches
- Packet Capture on High-Speed Links
- Characteristics of Packet Capture
- Data Measurement Repositories
- Multihoming and Traffic Engineering
- What is Multihoming
- Why Multihome
- Redundancy
- Performance
- Multihoming in IP Networks Today
- BGP or no
- Same Provider or Multiple
- Multihomed Stub One Link
- Multihomed Stub Multiple Links
- Multihomed Stub Multiple ISPs
- Multihomed Transit Network
- Interdomain Traffic Engineering
- Outbound Traffic Control
- Outbound Traffic Load Balancing
- Traffic Engineering Goals
- Managing Scale
- Achieving Predictability
- Challenges to Predictability
- Inter-AS Negotiation
- Outbound Multihoming Goals
- Inbound Traffic Control
- How many links are enough
- Problems with Multihoming in IPv4
- Slide 68
- Slide 69
- Diagnosis and Troubleshooting
- Operator Mailing List (NANOG)
- Operator Mailing List
- Routing Configuration
- Internet Business Model (Simplified)
- Peering Contracts Consistent Export
- Consistent Export
- Inconsistent Export in Practice
- Blackholes
- Security
- Security ldquoBogonrdquo Routes
- Spam Phishing etc
- Spam and Routing
- Worms and Botnets
- What is a Worm
- Example Worm Code Red
- Code Red Revisions
- Code Red Host Infection Rate
- Designing Fast-Spreading Worms
- Botnets
- ldquoRallyingrdquo the Botnet
- Botnet Control
- Some Defenses
- Idea 1 Ingress Filtering
- Idea 2 uRPF Checks
- Problems with uRPF
- S-BGP
- Practical Problems with S-BGP
-
62
Challenges to Predictabilitybull For transit ISPs effects on incoming traffic
ndash Lack of coordination strikes again
63
ldquoHot Potatordquo routing
Inter-AS Negotiation
bull Coordination aids predictabilityndash Negotiate where to sendndash Inbound and outboundndash Mutual benefits
bull How to implementndash What info to exchangendash Protecting privacyndash How to prioritize choicesndash How to prevent cheating
Destination 2
Destination 1
multiplepeeringpoints
Provider A
Provider B
64
Outbound Multihoming Goals
bull Redundancyndash Dynamic routing will failover to backup link
bull Performancendash Select provider with best performance per prefix
ndash Requires active probing
bull Costndash Select provider per prefix over time to minimize the
total financial cost
65
Inbound Traffic Control
bull More difficult no control over neighborsrsquo decisions
bull Three common techniques (previously discussed)ndash AS path prependingndash Communities and local preferencendash Prefix splitting
How does todayrsquos paper (MONET) control inbound traffic
66
How many links are enough
K upstream ISPs
Not much benefit beyond 4 ISPs
Akella et al ldquoPerformance Benefits of Multihomingrdquo SIGCOMM 2003
67
Problems with Multihoming in IPv4
bull Routing table growthndash Provider-based addressingndash Advertising prefix out multiple ISPs ndash canrsquot aggregate
bull Poor control over inbound trafficndash Existing mechanisms do not allow hosts to control
inbound traffic
68
GeorgiaTech
Internet Routing Overview
bull Intradomain (ie ldquointra-ASrdquo) routingbull Interdomain routing
Comcast
Abilene
ATampT Cogent
Autonomous Systems (ASes)
69
Configuration Problems ldquoAS 7007rdquoldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
UUNet
Florida InternetBarn
Sprint
70
Diagnosis and Troubleshootingldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
ldquoMicrosofts websites were offline for up to 23 hoursbecause of a [router] misconfigurationhellipit took nearly a day to determine what was wrong and undo the changesrdquo -- wiredcom January 25 2001
ldquoWorldCom Inchellipsuffered a widespread outage on its Internet backbone that affected roughly 20 percent of its US customer base The network problemshellipaffected millions of computer users worldwide A spokeswoman attributed the outage to a route table issue -- cnncom October 3 2002
A number of Covad customers went out from 5pm today due to supposedly a DDOS (distributed denial of service attack) on a key Level3 data center which later was described as a route leak (misconfiguration)ldquo
-- dslreportscom February 23 2004
71
Operator Mailing List (NANOG)Date Mon 18 Oct 2004 091515 -0700Subject Level 3 US east coast issues
Level 3 experiencing widespread unspecified routing issues on the US east coast Master ticket 1086844 Anyone have more specific information
Date Mon 18 Oct 2004 122034 -0400 (EDT)Subject Re Level 3 US east coast issues
Level 3 is currently experiencing a backbone outage causing routing instability and packet loss We are working to restore and will be sending hourly updateshellip
72
Operator Mailing List
0102030405060708090
Filtering RouteLeaks
RouteHijacks
RouteInstability
RoutingLoops
Blackholes
Occurr
ences o
ver
10 Y
ears
1995-1997 1998-2001 2001-2004
Note Only includes problems openly discussed on this list
Compare 83 power outages 1 fire
73
Routing Configuration
Ranking route selection
Dissemination internal route advertisement
Filtering route advertisement
Customer
Competitor
Primary
Backup
74
Internet Business Model (Simplified)
bull CustomerProvider One AS pays another for reachability to some set of destinations
bull ldquoSettlement-freerdquo Peering Bartering Two ASes exchange routes with one another
Provider
Peer
Customer
Preferences implemented with local preference manipulation
Destination
Pay to use
Get paid to use
Free to use
75
Peering Contracts Consistent Export
bull Rules of settlement-free peeringndash Advertise routes at all peering pointsndash Advertised routes must have equal ldquoAS path lengthrdquo
Sprint
ATampT
Enables ldquohot potatordquo routing
ldquoequally goodrdquoroutes
76
Consistent Export
bull Malicedeceptionbull iBGP signaling partitionbull Inconsistent export policy
neighbor 10123route-map PEER permit 10 set prepend 123
neighbor 10456route-map PEER permit 10 set prepend 123 123
Possible Causes
10123 456 1
10456 456 2
Neighbor AS Export
1 1 123
Export Clause Prepend
2 1 123 123
Two different Export Policies
77
Inconsistent Export in Practice
Feamster et al ldquoBorderGuard Detecting Cold Potatoes from Peersrdquo ACM IMC October 2004
78
Blackholes
Date Thu 18 Jul 2002 060510 -0400 (EDT)From Chad Oleary ltcolpoboxcomgtSubject Re problems with 701To ltnanogmeritedugt
Were starting to see the same issues with UUNet again Anyone elseseeing this Trying to reach Qwest
traceroute to 631461901 (631461901) 30 hops max 38 byte packets 1 esc-lp2-gwe-solutionscorpcom (631182201) 1167 ms 1163 ms 1142 ms 2 500Serial2-10GW1TPA2ALTERNET (1571301499) 1097 ms 1059 ms 1044 ms 3 161at-1-0-0XL4ATL1ALTERNET (1526381190) 13839 ms 14108 ms 16638 ms 4 0so-3-1-0XL2ATL5ALTERNET (152630238) 14370 ms 14587 ms 14553 ms 5 POS7-0BR2ATL5ALTERNET (1526382193) 13928 ms 14099 ms 14053 ms 6 7 hellip
79
Security
80
Security ldquoBogonrdquo Routes
Feamster et al ldquoAn Empirical Study of lsquoBogonrsquo Route Advertisementsrdquo ACM CCR January 2005
81
Spam Phishing etc
bull Unsolicited commercial emailbull As of about August 2008 estimates indicate that
about 95 of all email is spambull Common spam filtering techniques
ndash Content-based filtersndash DNS Blacklist (DNSBL) lookups Significant fraction of
todayrsquos DNS traffic
Can IP addresses from which spam is received be spoofed
82
Spam and Routing
83
Worms and Botnets
84
What is a Worm
bull Code that replicates and propagates across the networkndash Often carries a ldquopayloadrdquo
bull Usually spread via exploiting flaws in open servicesndash ldquoVirusesrdquo require user action to spread
bull First worm Robert Morris November 1988ndash 6-10 of all Internet hosts infected ()
bull Many more since but none on that scale until July 2001
85
Example Worm Code Red
bull Initial version July 13 2001
bull Exploited known ISAPI vulnerability in Microsoft IIS Web servers
bull 1st through 20th of each month spread20th through end of each month attack
bull Payload Web site defacementbull Scanning Random IP addressesbull Bug failure to seed random number generator
86
Code Red Revisions
bull Released July 19 2001
bull Payload flooding attack on wwwwhitehousegovndash Attack was mounted at the IP address of the Web site
bull Bug died after 20th of each month
bull Random number generator for IP scanning fixed
87
Code Red Host Infection Rate
Exponential infection rate
Measured using backscatter technique
88
Designing Fast-Spreading Worms
bull Hit-list scanningndash Time to infect first 10k hosts dominates infection timendash Solution Reconnaissance (stealthy scans etc)
bull Permutation scanningndash Observation Most scanning is redundantndash Idea Shared permutation of address space Start scanning
from own IP address Re-randomize when another infected machine is found
bull Internet-scale hit listsndash Flash worm complete infection within 30 seconds
89
Botnets
bull Bots Autonomous programs performing tasksbull Plenty of ldquobenignrdquo bots
ndash eg weatherbug
bull Botnets group of bots ndash Typically carries malicious connotationndash Large numbers of infected machinesndash Machines ldquoenlistedrdquo with infection vectors like worms
(last lecture)
bull Available for simultaneous control by a masterbull Size up to 350000 nodes (from todayrsquos paper)
90
ldquoRallyingrdquo the Botnet
bull Easy to combine worm backdoor functionalitybull Problem how to learn about successfully
infected machines
bull Optionsndash Emailndash Hard-coded email address
91
Botnet Control
bull Botnet master typically runs some IRC server on a well-known port (eg 6667)
bull Infected machine contacts botnet with pre-programmed DNS name (eg big-botde)
bull Dynamic DNS allows controller to move about freely
Infected Machine
DynamicDNS
BotnetController
(IRC server)
92
Some Defenses
93
Idea 1 Ingress Filtering
bull RFC 2827 Routers install filters to drop packets from networks that are not downstream
bull Feasible at edgesbull Difficult to configure closer to network ldquocorerdquo
20469207024 Internet
Drop all packets with source address other than 20469207024
94
Idea 2 uRPF Checks
bull Unicast Reverse Path Forwardingndash Cisco ldquoip verify unicast reverse-pathrdquo
bull Requires symmetric routing
Accept packet from interface only if forwarding table entry for source IP address matches ingress interface
100183
A10018
10015
101203
100181 24
10011 24
Strict Mode uRPF
Enabled
ldquoArdquo Routing TableDestination Next Hop1001024 Int 110018024 Int 2
100183 from wrong interface
95
Problems with uRPF
bull Asymmetric routing
96
S-BGP
bull Address-based PKI validate signaturesndash Authentication of
bull ownership for IP address blocks bull AS number bull an ASs identity and bull a BGP routers identity
ndash Use existing infrastructure (Internet registries etc)ndash Routing origination is digitally signedndash BGP updates are digitally signed
1048708 bull Route attestations A new optional BGP transitive path attribute
ndash carries digital signatures covering the routing information in updates
97
Practical Problems with S-BGP
bull Requires Public-Key Infrastructure
bull Lots of digital signatures to calculate and verifyndash Message overheadndash CPU overhead
bull Calculation expense is greatest when topology is changingndash Caching can help
bull Route aggregation is problematic (maybe thatrsquos OK)
bull Secure route withdrawals when link or node fails
bull Address ownership data out of date
bull Deployment
- Networking
- Goal of This Tutorial
- Todayrsquos Networks
- Business Models
- Billing for Internet Usage
- Net Neutrality
- Network Operations
- Point-of-Presence (PoP)
- Example Abilene Network Topology
- Another Example Backbone
- Internet Routing Overview
- Internet Routing Protocol BGP
- Two Flavors of BGP
- IPv4 Addresses Networks of Networks
- Pre-1994 Classful Addressing
- Classless Interdomain Routing (CIDR)
- Benefits of CIDR
- Growth of IP Prefixes
- 1994-1998 Linear Growth
- Around 2000 Fast Growth Resumes
- Fast growth resumes
- The Address Allocation Process
- Common Problems
- What can go wrong
- Measurement and Monitoring
- Passive vs Active Measurement
- Slide 27
- Passive Traffic Data Measurement
- Simple Network Management Protocol
- SNMP (Passive)
- Packet-level Monitoring
- Full Packet Capture (Passive)
- What is a flow
- Cisco NetFlow
- Flow Record Contents
- Aggregating Packets into Flows
- Reducing Measurement Overhead
- Packet Sampling for Flow Monitoring
- Sampling Flow-Level Sampling
- Two Main Approaches
- Packet Capture on High-Speed Links
- Characteristics of Packet Capture
- Data Measurement Repositories
- Multihoming and Traffic Engineering
- What is Multihoming
- Why Multihome
- Redundancy
- Performance
- Multihoming in IP Networks Today
- BGP or no
- Same Provider or Multiple
- Multihomed Stub One Link
- Multihomed Stub Multiple Links
- Multihomed Stub Multiple ISPs
- Multihomed Transit Network
- Interdomain Traffic Engineering
- Outbound Traffic Control
- Outbound Traffic Load Balancing
- Traffic Engineering Goals
- Managing Scale
- Achieving Predictability
- Challenges to Predictability
- Inter-AS Negotiation
- Outbound Multihoming Goals
- Inbound Traffic Control
- How many links are enough
- Problems with Multihoming in IPv4
- Slide 68
- Slide 69
- Diagnosis and Troubleshooting
- Operator Mailing List (NANOG)
- Operator Mailing List
- Routing Configuration
- Internet Business Model (Simplified)
- Peering Contracts Consistent Export
- Consistent Export
- Inconsistent Export in Practice
- Blackholes
- Security
- Security ldquoBogonrdquo Routes
- Spam Phishing etc
- Spam and Routing
- Worms and Botnets
- What is a Worm
- Example Worm Code Red
- Code Red Revisions
- Code Red Host Infection Rate
- Designing Fast-Spreading Worms
- Botnets
- ldquoRallyingrdquo the Botnet
- Botnet Control
- Some Defenses
- Idea 1 Ingress Filtering
- Idea 2 uRPF Checks
- Problems with uRPF
- S-BGP
- Practical Problems with S-BGP
-
63
ldquoHot Potatordquo routing
Inter-AS Negotiation
bull Coordination aids predictabilityndash Negotiate where to sendndash Inbound and outboundndash Mutual benefits
bull How to implementndash What info to exchangendash Protecting privacyndash How to prioritize choicesndash How to prevent cheating
Destination 2
Destination 1
multiplepeeringpoints
Provider A
Provider B
64
Outbound Multihoming Goals
bull Redundancyndash Dynamic routing will failover to backup link
bull Performancendash Select provider with best performance per prefix
ndash Requires active probing
bull Costndash Select provider per prefix over time to minimize the
total financial cost
65
Inbound Traffic Control
bull More difficult no control over neighborsrsquo decisions
bull Three common techniques (previously discussed)ndash AS path prependingndash Communities and local preferencendash Prefix splitting
How does todayrsquos paper (MONET) control inbound traffic
66
How many links are enough
K upstream ISPs
Not much benefit beyond 4 ISPs
Akella et al ldquoPerformance Benefits of Multihomingrdquo SIGCOMM 2003
67
Problems with Multihoming in IPv4
bull Routing table growthndash Provider-based addressingndash Advertising prefix out multiple ISPs ndash canrsquot aggregate
bull Poor control over inbound trafficndash Existing mechanisms do not allow hosts to control
inbound traffic
68
GeorgiaTech
Internet Routing Overview
bull Intradomain (ie ldquointra-ASrdquo) routingbull Interdomain routing
Comcast
Abilene
ATampT Cogent
Autonomous Systems (ASes)
69
Configuration Problems ldquoAS 7007rdquoldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
UUNet
Florida InternetBarn
Sprint
70
Diagnosis and Troubleshootingldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
ldquoMicrosofts websites were offline for up to 23 hoursbecause of a [router] misconfigurationhellipit took nearly a day to determine what was wrong and undo the changesrdquo -- wiredcom January 25 2001
ldquoWorldCom Inchellipsuffered a widespread outage on its Internet backbone that affected roughly 20 percent of its US customer base The network problemshellipaffected millions of computer users worldwide A spokeswoman attributed the outage to a route table issue -- cnncom October 3 2002
A number of Covad customers went out from 5pm today due to supposedly a DDOS (distributed denial of service attack) on a key Level3 data center which later was described as a route leak (misconfiguration)ldquo
-- dslreportscom February 23 2004
71
Operator Mailing List (NANOG)Date Mon 18 Oct 2004 091515 -0700Subject Level 3 US east coast issues
Level 3 experiencing widespread unspecified routing issues on the US east coast Master ticket 1086844 Anyone have more specific information
Date Mon 18 Oct 2004 122034 -0400 (EDT)Subject Re Level 3 US east coast issues
Level 3 is currently experiencing a backbone outage causing routing instability and packet loss We are working to restore and will be sending hourly updateshellip
72
Operator Mailing List
0102030405060708090
Filtering RouteLeaks
RouteHijacks
RouteInstability
RoutingLoops
Blackholes
Occurr
ences o
ver
10 Y
ears
1995-1997 1998-2001 2001-2004
Note Only includes problems openly discussed on this list
Compare 83 power outages 1 fire
73
Routing Configuration
Ranking route selection
Dissemination internal route advertisement
Filtering route advertisement
Customer
Competitor
Primary
Backup
74
Internet Business Model (Simplified)
bull CustomerProvider One AS pays another for reachability to some set of destinations
bull ldquoSettlement-freerdquo Peering Bartering Two ASes exchange routes with one another
Provider
Peer
Customer
Preferences implemented with local preference manipulation
Destination
Pay to use
Get paid to use
Free to use
75
Peering Contracts Consistent Export
bull Rules of settlement-free peeringndash Advertise routes at all peering pointsndash Advertised routes must have equal ldquoAS path lengthrdquo
Sprint
ATampT
Enables ldquohot potatordquo routing
ldquoequally goodrdquoroutes
76
Consistent Export
bull Malicedeceptionbull iBGP signaling partitionbull Inconsistent export policy
neighbor 10123route-map PEER permit 10 set prepend 123
neighbor 10456route-map PEER permit 10 set prepend 123 123
Possible Causes
10123 456 1
10456 456 2
Neighbor AS Export
1 1 123
Export Clause Prepend
2 1 123 123
Two different Export Policies
77
Inconsistent Export in Practice
Feamster et al ldquoBorderGuard Detecting Cold Potatoes from Peersrdquo ACM IMC October 2004
78
Blackholes
Date Thu 18 Jul 2002 060510 -0400 (EDT)From Chad Oleary ltcolpoboxcomgtSubject Re problems with 701To ltnanogmeritedugt
Were starting to see the same issues with UUNet again Anyone elseseeing this Trying to reach Qwest
traceroute to 631461901 (631461901) 30 hops max 38 byte packets 1 esc-lp2-gwe-solutionscorpcom (631182201) 1167 ms 1163 ms 1142 ms 2 500Serial2-10GW1TPA2ALTERNET (1571301499) 1097 ms 1059 ms 1044 ms 3 161at-1-0-0XL4ATL1ALTERNET (1526381190) 13839 ms 14108 ms 16638 ms 4 0so-3-1-0XL2ATL5ALTERNET (152630238) 14370 ms 14587 ms 14553 ms 5 POS7-0BR2ATL5ALTERNET (1526382193) 13928 ms 14099 ms 14053 ms 6 7 hellip
79
Security
80
Security ldquoBogonrdquo Routes
Feamster et al ldquoAn Empirical Study of lsquoBogonrsquo Route Advertisementsrdquo ACM CCR January 2005
81
Spam Phishing etc
bull Unsolicited commercial emailbull As of about August 2008 estimates indicate that
about 95 of all email is spambull Common spam filtering techniques
ndash Content-based filtersndash DNS Blacklist (DNSBL) lookups Significant fraction of
todayrsquos DNS traffic
Can IP addresses from which spam is received be spoofed
82
Spam and Routing
83
Worms and Botnets
84
What is a Worm
bull Code that replicates and propagates across the networkndash Often carries a ldquopayloadrdquo
bull Usually spread via exploiting flaws in open servicesndash ldquoVirusesrdquo require user action to spread
bull First worm Robert Morris November 1988ndash 6-10 of all Internet hosts infected ()
bull Many more since but none on that scale until July 2001
85
Example Worm Code Red
bull Initial version July 13 2001
bull Exploited known ISAPI vulnerability in Microsoft IIS Web servers
bull 1st through 20th of each month spread20th through end of each month attack
bull Payload Web site defacementbull Scanning Random IP addressesbull Bug failure to seed random number generator
86
Code Red Revisions
bull Released July 19 2001
bull Payload flooding attack on wwwwhitehousegovndash Attack was mounted at the IP address of the Web site
bull Bug died after 20th of each month
bull Random number generator for IP scanning fixed
87
Code Red Host Infection Rate
Exponential infection rate
Measured using backscatter technique
88
Designing Fast-Spreading Worms
bull Hit-list scanningndash Time to infect first 10k hosts dominates infection timendash Solution Reconnaissance (stealthy scans etc)
bull Permutation scanningndash Observation Most scanning is redundantndash Idea Shared permutation of address space Start scanning
from own IP address Re-randomize when another infected machine is found
bull Internet-scale hit listsndash Flash worm complete infection within 30 seconds
89
Botnets
bull Bots Autonomous programs performing tasksbull Plenty of ldquobenignrdquo bots
ndash eg weatherbug
bull Botnets group of bots ndash Typically carries malicious connotationndash Large numbers of infected machinesndash Machines ldquoenlistedrdquo with infection vectors like worms
(last lecture)
bull Available for simultaneous control by a masterbull Size up to 350000 nodes (from todayrsquos paper)
90
ldquoRallyingrdquo the Botnet
bull Easy to combine worm backdoor functionalitybull Problem how to learn about successfully
infected machines
bull Optionsndash Emailndash Hard-coded email address
91
Botnet Control
bull Botnet master typically runs some IRC server on a well-known port (eg 6667)
bull Infected machine contacts botnet with pre-programmed DNS name (eg big-botde)
bull Dynamic DNS allows controller to move about freely
Infected Machine
DynamicDNS
BotnetController
(IRC server)
92
Some Defenses
93
Idea 1 Ingress Filtering
bull RFC 2827 Routers install filters to drop packets from networks that are not downstream
bull Feasible at edgesbull Difficult to configure closer to network ldquocorerdquo
20469207024 Internet
Drop all packets with source address other than 20469207024
94
Idea 2 uRPF Checks
bull Unicast Reverse Path Forwardingndash Cisco ldquoip verify unicast reverse-pathrdquo
bull Requires symmetric routing
Accept packet from interface only if forwarding table entry for source IP address matches ingress interface
100183
A10018
10015
101203
100181 24
10011 24
Strict Mode uRPF
Enabled
ldquoArdquo Routing TableDestination Next Hop1001024 Int 110018024 Int 2
100183 from wrong interface
95
Problems with uRPF
bull Asymmetric routing
96
S-BGP
bull Address-based PKI validate signaturesndash Authentication of
bull ownership for IP address blocks bull AS number bull an ASs identity and bull a BGP routers identity
ndash Use existing infrastructure (Internet registries etc)ndash Routing origination is digitally signedndash BGP updates are digitally signed
1048708 bull Route attestations A new optional BGP transitive path attribute
ndash carries digital signatures covering the routing information in updates
97
Practical Problems with S-BGP
bull Requires Public-Key Infrastructure
bull Lots of digital signatures to calculate and verifyndash Message overheadndash CPU overhead
bull Calculation expense is greatest when topology is changingndash Caching can help
bull Route aggregation is problematic (maybe thatrsquos OK)
bull Secure route withdrawals when link or node fails
bull Address ownership data out of date
bull Deployment
- Networking
- Goal of This Tutorial
- Todayrsquos Networks
- Business Models
- Billing for Internet Usage
- Net Neutrality
- Network Operations
- Point-of-Presence (PoP)
- Example Abilene Network Topology
- Another Example Backbone
- Internet Routing Overview
- Internet Routing Protocol BGP
- Two Flavors of BGP
- IPv4 Addresses Networks of Networks
- Pre-1994 Classful Addressing
- Classless Interdomain Routing (CIDR)
- Benefits of CIDR
- Growth of IP Prefixes
- 1994-1998 Linear Growth
- Around 2000 Fast Growth Resumes
- Fast growth resumes
- The Address Allocation Process
- Common Problems
- What can go wrong
- Measurement and Monitoring
- Passive vs Active Measurement
- Slide 27
- Passive Traffic Data Measurement
- Simple Network Management Protocol
- SNMP (Passive)
- Packet-level Monitoring
- Full Packet Capture (Passive)
- What is a flow
- Cisco NetFlow
- Flow Record Contents
- Aggregating Packets into Flows
- Reducing Measurement Overhead
- Packet Sampling for Flow Monitoring
- Sampling Flow-Level Sampling
- Two Main Approaches
- Packet Capture on High-Speed Links
- Characteristics of Packet Capture
- Data Measurement Repositories
- Multihoming and Traffic Engineering
- What is Multihoming
- Why Multihome
- Redundancy
- Performance
- Multihoming in IP Networks Today
- BGP or no
- Same Provider or Multiple
- Multihomed Stub One Link
- Multihomed Stub Multiple Links
- Multihomed Stub Multiple ISPs
- Multihomed Transit Network
- Interdomain Traffic Engineering
- Outbound Traffic Control
- Outbound Traffic Load Balancing
- Traffic Engineering Goals
- Managing Scale
- Achieving Predictability
- Challenges to Predictability
- Inter-AS Negotiation
- Outbound Multihoming Goals
- Inbound Traffic Control
- How many links are enough
- Problems with Multihoming in IPv4
- Slide 68
- Slide 69
- Diagnosis and Troubleshooting
- Operator Mailing List (NANOG)
- Operator Mailing List
- Routing Configuration
- Internet Business Model (Simplified)
- Peering Contracts Consistent Export
- Consistent Export
- Inconsistent Export in Practice
- Blackholes
- Security
- Security ldquoBogonrdquo Routes
- Spam Phishing etc
- Spam and Routing
- Worms and Botnets
- What is a Worm
- Example Worm Code Red
- Code Red Revisions
- Code Red Host Infection Rate
- Designing Fast-Spreading Worms
- Botnets
- ldquoRallyingrdquo the Botnet
- Botnet Control
- Some Defenses
- Idea 1 Ingress Filtering
- Idea 2 uRPF Checks
- Problems with uRPF
- S-BGP
- Practical Problems with S-BGP
-
64
Outbound Multihoming Goals
bull Redundancyndash Dynamic routing will failover to backup link
bull Performancendash Select provider with best performance per prefix
ndash Requires active probing
bull Costndash Select provider per prefix over time to minimize the
total financial cost
65
Inbound Traffic Control
bull More difficult no control over neighborsrsquo decisions
bull Three common techniques (previously discussed)ndash AS path prependingndash Communities and local preferencendash Prefix splitting
How does todayrsquos paper (MONET) control inbound traffic
66
How many links are enough
K upstream ISPs
Not much benefit beyond 4 ISPs
Akella et al ldquoPerformance Benefits of Multihomingrdquo SIGCOMM 2003
67
Problems with Multihoming in IPv4
bull Routing table growthndash Provider-based addressingndash Advertising prefix out multiple ISPs ndash canrsquot aggregate
bull Poor control over inbound trafficndash Existing mechanisms do not allow hosts to control
inbound traffic
68
GeorgiaTech
Internet Routing Overview
bull Intradomain (ie ldquointra-ASrdquo) routingbull Interdomain routing
Comcast
Abilene
ATampT Cogent
Autonomous Systems (ASes)
69
Configuration Problems ldquoAS 7007rdquoldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
UUNet
Florida InternetBarn
Sprint
70
Diagnosis and Troubleshootingldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
ldquoMicrosofts websites were offline for up to 23 hoursbecause of a [router] misconfigurationhellipit took nearly a day to determine what was wrong and undo the changesrdquo -- wiredcom January 25 2001
ldquoWorldCom Inchellipsuffered a widespread outage on its Internet backbone that affected roughly 20 percent of its US customer base The network problemshellipaffected millions of computer users worldwide A spokeswoman attributed the outage to a route table issue -- cnncom October 3 2002
A number of Covad customers went out from 5pm today due to supposedly a DDOS (distributed denial of service attack) on a key Level3 data center which later was described as a route leak (misconfiguration)ldquo
-- dslreportscom February 23 2004
71
Operator Mailing List (NANOG)Date Mon 18 Oct 2004 091515 -0700Subject Level 3 US east coast issues
Level 3 experiencing widespread unspecified routing issues on the US east coast Master ticket 1086844 Anyone have more specific information
Date Mon 18 Oct 2004 122034 -0400 (EDT)Subject Re Level 3 US east coast issues
Level 3 is currently experiencing a backbone outage causing routing instability and packet loss We are working to restore and will be sending hourly updateshellip
72
Operator Mailing List
0102030405060708090
Filtering RouteLeaks
RouteHijacks
RouteInstability
RoutingLoops
Blackholes
Occurr
ences o
ver
10 Y
ears
1995-1997 1998-2001 2001-2004
Note Only includes problems openly discussed on this list
Compare 83 power outages 1 fire
73
Routing Configuration
Ranking route selection
Dissemination internal route advertisement
Filtering route advertisement
Customer
Competitor
Primary
Backup
74
Internet Business Model (Simplified)
bull CustomerProvider One AS pays another for reachability to some set of destinations
bull ldquoSettlement-freerdquo Peering Bartering Two ASes exchange routes with one another
Provider
Peer
Customer
Preferences implemented with local preference manipulation
Destination
Pay to use
Get paid to use
Free to use
75
Peering Contracts Consistent Export
bull Rules of settlement-free peeringndash Advertise routes at all peering pointsndash Advertised routes must have equal ldquoAS path lengthrdquo
Sprint
ATampT
Enables ldquohot potatordquo routing
ldquoequally goodrdquoroutes
76
Consistent Export
bull Malicedeceptionbull iBGP signaling partitionbull Inconsistent export policy
neighbor 10123route-map PEER permit 10 set prepend 123
neighbor 10456route-map PEER permit 10 set prepend 123 123
Possible Causes
10123 456 1
10456 456 2
Neighbor AS Export
1 1 123
Export Clause Prepend
2 1 123 123
Two different Export Policies
77
Inconsistent Export in Practice
Feamster et al ldquoBorderGuard Detecting Cold Potatoes from Peersrdquo ACM IMC October 2004
78
Blackholes
Date Thu 18 Jul 2002 060510 -0400 (EDT)From Chad Oleary ltcolpoboxcomgtSubject Re problems with 701To ltnanogmeritedugt
Were starting to see the same issues with UUNet again Anyone elseseeing this Trying to reach Qwest
traceroute to 631461901 (631461901) 30 hops max 38 byte packets 1 esc-lp2-gwe-solutionscorpcom (631182201) 1167 ms 1163 ms 1142 ms 2 500Serial2-10GW1TPA2ALTERNET (1571301499) 1097 ms 1059 ms 1044 ms 3 161at-1-0-0XL4ATL1ALTERNET (1526381190) 13839 ms 14108 ms 16638 ms 4 0so-3-1-0XL2ATL5ALTERNET (152630238) 14370 ms 14587 ms 14553 ms 5 POS7-0BR2ATL5ALTERNET (1526382193) 13928 ms 14099 ms 14053 ms 6 7 hellip
79
Security
80
Security ldquoBogonrdquo Routes
Feamster et al ldquoAn Empirical Study of lsquoBogonrsquo Route Advertisementsrdquo ACM CCR January 2005
81
Spam Phishing etc
bull Unsolicited commercial emailbull As of about August 2008 estimates indicate that
about 95 of all email is spambull Common spam filtering techniques
ndash Content-based filtersndash DNS Blacklist (DNSBL) lookups Significant fraction of
todayrsquos DNS traffic
Can IP addresses from which spam is received be spoofed
82
Spam and Routing
83
Worms and Botnets
84
What is a Worm
bull Code that replicates and propagates across the networkndash Often carries a ldquopayloadrdquo
bull Usually spread via exploiting flaws in open servicesndash ldquoVirusesrdquo require user action to spread
bull First worm Robert Morris November 1988ndash 6-10 of all Internet hosts infected ()
bull Many more since but none on that scale until July 2001
85
Example Worm Code Red
bull Initial version July 13 2001
bull Exploited known ISAPI vulnerability in Microsoft IIS Web servers
bull 1st through 20th of each month spread20th through end of each month attack
bull Payload Web site defacementbull Scanning Random IP addressesbull Bug failure to seed random number generator
86
Code Red Revisions
bull Released July 19 2001
bull Payload flooding attack on wwwwhitehousegovndash Attack was mounted at the IP address of the Web site
bull Bug died after 20th of each month
bull Random number generator for IP scanning fixed
87
Code Red Host Infection Rate
Exponential infection rate
Measured using backscatter technique
88
Designing Fast-Spreading Worms
bull Hit-list scanningndash Time to infect first 10k hosts dominates infection timendash Solution Reconnaissance (stealthy scans etc)
bull Permutation scanningndash Observation Most scanning is redundantndash Idea Shared permutation of address space Start scanning
from own IP address Re-randomize when another infected machine is found
bull Internet-scale hit listsndash Flash worm complete infection within 30 seconds
89
Botnets
bull Bots Autonomous programs performing tasksbull Plenty of ldquobenignrdquo bots
ndash eg weatherbug
bull Botnets group of bots ndash Typically carries malicious connotationndash Large numbers of infected machinesndash Machines ldquoenlistedrdquo with infection vectors like worms
(last lecture)
bull Available for simultaneous control by a masterbull Size up to 350000 nodes (from todayrsquos paper)
90
ldquoRallyingrdquo the Botnet
bull Easy to combine worm backdoor functionalitybull Problem how to learn about successfully
infected machines
bull Optionsndash Emailndash Hard-coded email address
91
Botnet Control
bull Botnet master typically runs some IRC server on a well-known port (eg 6667)
bull Infected machine contacts botnet with pre-programmed DNS name (eg big-botde)
bull Dynamic DNS allows controller to move about freely
Infected Machine
DynamicDNS
BotnetController
(IRC server)
92
Some Defenses
93
Idea 1 Ingress Filtering
bull RFC 2827 Routers install filters to drop packets from networks that are not downstream
bull Feasible at edgesbull Difficult to configure closer to network ldquocorerdquo
20469207024 Internet
Drop all packets with source address other than 20469207024
94
Idea 2 uRPF Checks
bull Unicast Reverse Path Forwardingndash Cisco ldquoip verify unicast reverse-pathrdquo
bull Requires symmetric routing
Accept packet from interface only if forwarding table entry for source IP address matches ingress interface
100183
A10018
10015
101203
100181 24
10011 24
Strict Mode uRPF
Enabled
ldquoArdquo Routing TableDestination Next Hop1001024 Int 110018024 Int 2
100183 from wrong interface
95
Problems with uRPF
bull Asymmetric routing
96
S-BGP
bull Address-based PKI validate signaturesndash Authentication of
bull ownership for IP address blocks bull AS number bull an ASs identity and bull a BGP routers identity
ndash Use existing infrastructure (Internet registries etc)ndash Routing origination is digitally signedndash BGP updates are digitally signed
1048708 bull Route attestations A new optional BGP transitive path attribute
ndash carries digital signatures covering the routing information in updates
97
Practical Problems with S-BGP
bull Requires Public-Key Infrastructure
bull Lots of digital signatures to calculate and verifyndash Message overheadndash CPU overhead
bull Calculation expense is greatest when topology is changingndash Caching can help
bull Route aggregation is problematic (maybe thatrsquos OK)
bull Secure route withdrawals when link or node fails
bull Address ownership data out of date
bull Deployment
- Networking
- Goal of This Tutorial
- Todayrsquos Networks
- Business Models
- Billing for Internet Usage
- Net Neutrality
- Network Operations
- Point-of-Presence (PoP)
- Example Abilene Network Topology
- Another Example Backbone
- Internet Routing Overview
- Internet Routing Protocol BGP
- Two Flavors of BGP
- IPv4 Addresses Networks of Networks
- Pre-1994 Classful Addressing
- Classless Interdomain Routing (CIDR)
- Benefits of CIDR
- Growth of IP Prefixes
- 1994-1998 Linear Growth
- Around 2000 Fast Growth Resumes
- Fast growth resumes
- The Address Allocation Process
- Common Problems
- What can go wrong
- Measurement and Monitoring
- Passive vs Active Measurement
- Slide 27
- Passive Traffic Data Measurement
- Simple Network Management Protocol
- SNMP (Passive)
- Packet-level Monitoring
- Full Packet Capture (Passive)
- What is a flow
- Cisco NetFlow
- Flow Record Contents
- Aggregating Packets into Flows
- Reducing Measurement Overhead
- Packet Sampling for Flow Monitoring
- Sampling Flow-Level Sampling
- Two Main Approaches
- Packet Capture on High-Speed Links
- Characteristics of Packet Capture
- Data Measurement Repositories
- Multihoming and Traffic Engineering
- What is Multihoming
- Why Multihome
- Redundancy
- Performance
- Multihoming in IP Networks Today
- BGP or no
- Same Provider or Multiple
- Multihomed Stub One Link
- Multihomed Stub Multiple Links
- Multihomed Stub Multiple ISPs
- Multihomed Transit Network
- Interdomain Traffic Engineering
- Outbound Traffic Control
- Outbound Traffic Load Balancing
- Traffic Engineering Goals
- Managing Scale
- Achieving Predictability
- Challenges to Predictability
- Inter-AS Negotiation
- Outbound Multihoming Goals
- Inbound Traffic Control
- How many links are enough
- Problems with Multihoming in IPv4
- Slide 68
- Slide 69
- Diagnosis and Troubleshooting
- Operator Mailing List (NANOG)
- Operator Mailing List
- Routing Configuration
- Internet Business Model (Simplified)
- Peering Contracts Consistent Export
- Consistent Export
- Inconsistent Export in Practice
- Blackholes
- Security
- Security ldquoBogonrdquo Routes
- Spam Phishing etc
- Spam and Routing
- Worms and Botnets
- What is a Worm
- Example Worm Code Red
- Code Red Revisions
- Code Red Host Infection Rate
- Designing Fast-Spreading Worms
- Botnets
- ldquoRallyingrdquo the Botnet
- Botnet Control
- Some Defenses
- Idea 1 Ingress Filtering
- Idea 2 uRPF Checks
- Problems with uRPF
- S-BGP
- Practical Problems with S-BGP
-
65
Inbound Traffic Control
bull More difficult no control over neighborsrsquo decisions
bull Three common techniques (previously discussed)ndash AS path prependingndash Communities and local preferencendash Prefix splitting
How does todayrsquos paper (MONET) control inbound traffic
66
How many links are enough
K upstream ISPs
Not much benefit beyond 4 ISPs
Akella et al ldquoPerformance Benefits of Multihomingrdquo SIGCOMM 2003
67
Problems with Multihoming in IPv4
bull Routing table growthndash Provider-based addressingndash Advertising prefix out multiple ISPs ndash canrsquot aggregate
bull Poor control over inbound trafficndash Existing mechanisms do not allow hosts to control
inbound traffic
68
GeorgiaTech
Internet Routing Overview
bull Intradomain (ie ldquointra-ASrdquo) routingbull Interdomain routing
Comcast
Abilene
ATampT Cogent
Autonomous Systems (ASes)
69
Configuration Problems ldquoAS 7007rdquoldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
UUNet
Florida InternetBarn
Sprint
70
Diagnosis and Troubleshootingldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
ldquoMicrosofts websites were offline for up to 23 hoursbecause of a [router] misconfigurationhellipit took nearly a day to determine what was wrong and undo the changesrdquo -- wiredcom January 25 2001
ldquoWorldCom Inchellipsuffered a widespread outage on its Internet backbone that affected roughly 20 percent of its US customer base The network problemshellipaffected millions of computer users worldwide A spokeswoman attributed the outage to a route table issue -- cnncom October 3 2002
A number of Covad customers went out from 5pm today due to supposedly a DDOS (distributed denial of service attack) on a key Level3 data center which later was described as a route leak (misconfiguration)ldquo
-- dslreportscom February 23 2004
71
Operator Mailing List (NANOG)Date Mon 18 Oct 2004 091515 -0700Subject Level 3 US east coast issues
Level 3 experiencing widespread unspecified routing issues on the US east coast Master ticket 1086844 Anyone have more specific information
Date Mon 18 Oct 2004 122034 -0400 (EDT)Subject Re Level 3 US east coast issues
Level 3 is currently experiencing a backbone outage causing routing instability and packet loss We are working to restore and will be sending hourly updateshellip
72
Operator Mailing List
0102030405060708090
Filtering RouteLeaks
RouteHijacks
RouteInstability
RoutingLoops
Blackholes
Occurr
ences o
ver
10 Y
ears
1995-1997 1998-2001 2001-2004
Note Only includes problems openly discussed on this list
Compare 83 power outages 1 fire
73
Routing Configuration
Ranking route selection
Dissemination internal route advertisement
Filtering route advertisement
Customer
Competitor
Primary
Backup
74
Internet Business Model (Simplified)
bull CustomerProvider One AS pays another for reachability to some set of destinations
bull ldquoSettlement-freerdquo Peering Bartering Two ASes exchange routes with one another
Provider
Peer
Customer
Preferences implemented with local preference manipulation
Destination
Pay to use
Get paid to use
Free to use
75
Peering Contracts Consistent Export
bull Rules of settlement-free peeringndash Advertise routes at all peering pointsndash Advertised routes must have equal ldquoAS path lengthrdquo
Sprint
ATampT
Enables ldquohot potatordquo routing
ldquoequally goodrdquoroutes
76
Consistent Export
bull Malicedeceptionbull iBGP signaling partitionbull Inconsistent export policy
neighbor 10123route-map PEER permit 10 set prepend 123
neighbor 10456route-map PEER permit 10 set prepend 123 123
Possible Causes
10123 456 1
10456 456 2
Neighbor AS Export
1 1 123
Export Clause Prepend
2 1 123 123
Two different Export Policies
77
Inconsistent Export in Practice
Feamster et al ldquoBorderGuard Detecting Cold Potatoes from Peersrdquo ACM IMC October 2004
78
Blackholes
Date Thu 18 Jul 2002 060510 -0400 (EDT)From Chad Oleary ltcolpoboxcomgtSubject Re problems with 701To ltnanogmeritedugt
Were starting to see the same issues with UUNet again Anyone elseseeing this Trying to reach Qwest
traceroute to 631461901 (631461901) 30 hops max 38 byte packets 1 esc-lp2-gwe-solutionscorpcom (631182201) 1167 ms 1163 ms 1142 ms 2 500Serial2-10GW1TPA2ALTERNET (1571301499) 1097 ms 1059 ms 1044 ms 3 161at-1-0-0XL4ATL1ALTERNET (1526381190) 13839 ms 14108 ms 16638 ms 4 0so-3-1-0XL2ATL5ALTERNET (152630238) 14370 ms 14587 ms 14553 ms 5 POS7-0BR2ATL5ALTERNET (1526382193) 13928 ms 14099 ms 14053 ms 6 7 hellip
79
Security
80
Security ldquoBogonrdquo Routes
Feamster et al ldquoAn Empirical Study of lsquoBogonrsquo Route Advertisementsrdquo ACM CCR January 2005
81
Spam Phishing etc
bull Unsolicited commercial emailbull As of about August 2008 estimates indicate that
about 95 of all email is spambull Common spam filtering techniques
ndash Content-based filtersndash DNS Blacklist (DNSBL) lookups Significant fraction of
todayrsquos DNS traffic
Can IP addresses from which spam is received be spoofed
82
Spam and Routing
83
Worms and Botnets
84
What is a Worm
bull Code that replicates and propagates across the networkndash Often carries a ldquopayloadrdquo
bull Usually spread via exploiting flaws in open servicesndash ldquoVirusesrdquo require user action to spread
bull First worm Robert Morris November 1988ndash 6-10 of all Internet hosts infected ()
bull Many more since but none on that scale until July 2001
85
Example Worm Code Red
bull Initial version July 13 2001
bull Exploited known ISAPI vulnerability in Microsoft IIS Web servers
bull 1st through 20th of each month spread20th through end of each month attack
bull Payload Web site defacementbull Scanning Random IP addressesbull Bug failure to seed random number generator
86
Code Red Revisions
bull Released July 19 2001
bull Payload flooding attack on wwwwhitehousegovndash Attack was mounted at the IP address of the Web site
bull Bug died after 20th of each month
bull Random number generator for IP scanning fixed
87
Code Red Host Infection Rate
Exponential infection rate
Measured using backscatter technique
88
Designing Fast-Spreading Worms
bull Hit-list scanningndash Time to infect first 10k hosts dominates infection timendash Solution Reconnaissance (stealthy scans etc)
bull Permutation scanningndash Observation Most scanning is redundantndash Idea Shared permutation of address space Start scanning
from own IP address Re-randomize when another infected machine is found
bull Internet-scale hit listsndash Flash worm complete infection within 30 seconds
89
Botnets
bull Bots Autonomous programs performing tasksbull Plenty of ldquobenignrdquo bots
ndash eg weatherbug
bull Botnets group of bots ndash Typically carries malicious connotationndash Large numbers of infected machinesndash Machines ldquoenlistedrdquo with infection vectors like worms
(last lecture)
bull Available for simultaneous control by a masterbull Size up to 350000 nodes (from todayrsquos paper)
90
ldquoRallyingrdquo the Botnet
bull Easy to combine worm backdoor functionalitybull Problem how to learn about successfully
infected machines
bull Optionsndash Emailndash Hard-coded email address
91
Botnet Control
bull Botnet master typically runs some IRC server on a well-known port (eg 6667)
bull Infected machine contacts botnet with pre-programmed DNS name (eg big-botde)
bull Dynamic DNS allows controller to move about freely
Infected Machine
DynamicDNS
BotnetController
(IRC server)
92
Some Defenses
93
Idea 1 Ingress Filtering
bull RFC 2827 Routers install filters to drop packets from networks that are not downstream
bull Feasible at edgesbull Difficult to configure closer to network ldquocorerdquo
20469207024 Internet
Drop all packets with source address other than 20469207024
94
Idea 2 uRPF Checks
bull Unicast Reverse Path Forwardingndash Cisco ldquoip verify unicast reverse-pathrdquo
bull Requires symmetric routing
Accept packet from interface only if forwarding table entry for source IP address matches ingress interface
100183
A10018
10015
101203
100181 24
10011 24
Strict Mode uRPF
Enabled
ldquoArdquo Routing TableDestination Next Hop1001024 Int 110018024 Int 2
100183 from wrong interface
95
Problems with uRPF
bull Asymmetric routing
96
S-BGP
bull Address-based PKI validate signaturesndash Authentication of
bull ownership for IP address blocks bull AS number bull an ASs identity and bull a BGP routers identity
ndash Use existing infrastructure (Internet registries etc)ndash Routing origination is digitally signedndash BGP updates are digitally signed
1048708 bull Route attestations A new optional BGP transitive path attribute
ndash carries digital signatures covering the routing information in updates
97
Practical Problems with S-BGP
bull Requires Public-Key Infrastructure
bull Lots of digital signatures to calculate and verifyndash Message overheadndash CPU overhead
bull Calculation expense is greatest when topology is changingndash Caching can help
bull Route aggregation is problematic (maybe thatrsquos OK)
bull Secure route withdrawals when link or node fails
bull Address ownership data out of date
bull Deployment
- Networking
- Goal of This Tutorial
- Todayrsquos Networks
- Business Models
- Billing for Internet Usage
- Net Neutrality
- Network Operations
- Point-of-Presence (PoP)
- Example Abilene Network Topology
- Another Example Backbone
- Internet Routing Overview
- Internet Routing Protocol BGP
- Two Flavors of BGP
- IPv4 Addresses Networks of Networks
- Pre-1994 Classful Addressing
- Classless Interdomain Routing (CIDR)
- Benefits of CIDR
- Growth of IP Prefixes
- 1994-1998 Linear Growth
- Around 2000 Fast Growth Resumes
- Fast growth resumes
- The Address Allocation Process
- Common Problems
- What can go wrong
- Measurement and Monitoring
- Passive vs Active Measurement
- Slide 27
- Passive Traffic Data Measurement
- Simple Network Management Protocol
- SNMP (Passive)
- Packet-level Monitoring
- Full Packet Capture (Passive)
- What is a flow
- Cisco NetFlow
- Flow Record Contents
- Aggregating Packets into Flows
- Reducing Measurement Overhead
- Packet Sampling for Flow Monitoring
- Sampling Flow-Level Sampling
- Two Main Approaches
- Packet Capture on High-Speed Links
- Characteristics of Packet Capture
- Data Measurement Repositories
- Multihoming and Traffic Engineering
- What is Multihoming
- Why Multihome
- Redundancy
- Performance
- Multihoming in IP Networks Today
- BGP or no
- Same Provider or Multiple
- Multihomed Stub One Link
- Multihomed Stub Multiple Links
- Multihomed Stub Multiple ISPs
- Multihomed Transit Network
- Interdomain Traffic Engineering
- Outbound Traffic Control
- Outbound Traffic Load Balancing
- Traffic Engineering Goals
- Managing Scale
- Achieving Predictability
- Challenges to Predictability
- Inter-AS Negotiation
- Outbound Multihoming Goals
- Inbound Traffic Control
- How many links are enough
- Problems with Multihoming in IPv4
- Slide 68
- Slide 69
- Diagnosis and Troubleshooting
- Operator Mailing List (NANOG)
- Operator Mailing List
- Routing Configuration
- Internet Business Model (Simplified)
- Peering Contracts Consistent Export
- Consistent Export
- Inconsistent Export in Practice
- Blackholes
- Security
- Security ldquoBogonrdquo Routes
- Spam Phishing etc
- Spam and Routing
- Worms and Botnets
- What is a Worm
- Example Worm Code Red
- Code Red Revisions
- Code Red Host Infection Rate
- Designing Fast-Spreading Worms
- Botnets
- ldquoRallyingrdquo the Botnet
- Botnet Control
- Some Defenses
- Idea 1 Ingress Filtering
- Idea 2 uRPF Checks
- Problems with uRPF
- S-BGP
- Practical Problems with S-BGP
-
66
How many links are enough
K upstream ISPs
Not much benefit beyond 4 ISPs
Akella et al ldquoPerformance Benefits of Multihomingrdquo SIGCOMM 2003
67
Problems with Multihoming in IPv4
bull Routing table growthndash Provider-based addressingndash Advertising prefix out multiple ISPs ndash canrsquot aggregate
bull Poor control over inbound trafficndash Existing mechanisms do not allow hosts to control
inbound traffic
68
GeorgiaTech
Internet Routing Overview
bull Intradomain (ie ldquointra-ASrdquo) routingbull Interdomain routing
Comcast
Abilene
ATampT Cogent
Autonomous Systems (ASes)
69
Configuration Problems ldquoAS 7007rdquoldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
UUNet
Florida InternetBarn
Sprint
70
Diagnosis and Troubleshootingldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
ldquoMicrosofts websites were offline for up to 23 hoursbecause of a [router] misconfigurationhellipit took nearly a day to determine what was wrong and undo the changesrdquo -- wiredcom January 25 2001
ldquoWorldCom Inchellipsuffered a widespread outage on its Internet backbone that affected roughly 20 percent of its US customer base The network problemshellipaffected millions of computer users worldwide A spokeswoman attributed the outage to a route table issue -- cnncom October 3 2002
A number of Covad customers went out from 5pm today due to supposedly a DDOS (distributed denial of service attack) on a key Level3 data center which later was described as a route leak (misconfiguration)ldquo
-- dslreportscom February 23 2004
71
Operator Mailing List (NANOG)Date Mon 18 Oct 2004 091515 -0700Subject Level 3 US east coast issues
Level 3 experiencing widespread unspecified routing issues on the US east coast Master ticket 1086844 Anyone have more specific information
Date Mon 18 Oct 2004 122034 -0400 (EDT)Subject Re Level 3 US east coast issues
Level 3 is currently experiencing a backbone outage causing routing instability and packet loss We are working to restore and will be sending hourly updateshellip
72
Operator Mailing List
0102030405060708090
Filtering RouteLeaks
RouteHijacks
RouteInstability
RoutingLoops
Blackholes
Occurr
ences o
ver
10 Y
ears
1995-1997 1998-2001 2001-2004
Note Only includes problems openly discussed on this list
Compare 83 power outages 1 fire
73
Routing Configuration
Ranking route selection
Dissemination internal route advertisement
Filtering route advertisement
Customer
Competitor
Primary
Backup
74
Internet Business Model (Simplified)
bull CustomerProvider One AS pays another for reachability to some set of destinations
bull ldquoSettlement-freerdquo Peering Bartering Two ASes exchange routes with one another
Provider
Peer
Customer
Preferences implemented with local preference manipulation
Destination
Pay to use
Get paid to use
Free to use
75
Peering Contracts Consistent Export
bull Rules of settlement-free peeringndash Advertise routes at all peering pointsndash Advertised routes must have equal ldquoAS path lengthrdquo
Sprint
ATampT
Enables ldquohot potatordquo routing
ldquoequally goodrdquoroutes
76
Consistent Export
bull Malicedeceptionbull iBGP signaling partitionbull Inconsistent export policy
neighbor 10123route-map PEER permit 10 set prepend 123
neighbor 10456route-map PEER permit 10 set prepend 123 123
Possible Causes
10123 456 1
10456 456 2
Neighbor AS Export
1 1 123
Export Clause Prepend
2 1 123 123
Two different Export Policies
77
Inconsistent Export in Practice
Feamster et al ldquoBorderGuard Detecting Cold Potatoes from Peersrdquo ACM IMC October 2004
78
Blackholes
Date Thu 18 Jul 2002 060510 -0400 (EDT)From Chad Oleary ltcolpoboxcomgtSubject Re problems with 701To ltnanogmeritedugt
Were starting to see the same issues with UUNet again Anyone elseseeing this Trying to reach Qwest
traceroute to 631461901 (631461901) 30 hops max 38 byte packets 1 esc-lp2-gwe-solutionscorpcom (631182201) 1167 ms 1163 ms 1142 ms 2 500Serial2-10GW1TPA2ALTERNET (1571301499) 1097 ms 1059 ms 1044 ms 3 161at-1-0-0XL4ATL1ALTERNET (1526381190) 13839 ms 14108 ms 16638 ms 4 0so-3-1-0XL2ATL5ALTERNET (152630238) 14370 ms 14587 ms 14553 ms 5 POS7-0BR2ATL5ALTERNET (1526382193) 13928 ms 14099 ms 14053 ms 6 7 hellip
79
Security
80
Security ldquoBogonrdquo Routes
Feamster et al ldquoAn Empirical Study of lsquoBogonrsquo Route Advertisementsrdquo ACM CCR January 2005
81
Spam Phishing etc
bull Unsolicited commercial emailbull As of about August 2008 estimates indicate that
about 95 of all email is spambull Common spam filtering techniques
ndash Content-based filtersndash DNS Blacklist (DNSBL) lookups Significant fraction of
todayrsquos DNS traffic
Can IP addresses from which spam is received be spoofed
82
Spam and Routing
83
Worms and Botnets
84
What is a Worm
bull Code that replicates and propagates across the networkndash Often carries a ldquopayloadrdquo
bull Usually spread via exploiting flaws in open servicesndash ldquoVirusesrdquo require user action to spread
bull First worm Robert Morris November 1988ndash 6-10 of all Internet hosts infected ()
bull Many more since but none on that scale until July 2001
85
Example Worm Code Red
bull Initial version July 13 2001
bull Exploited known ISAPI vulnerability in Microsoft IIS Web servers
bull 1st through 20th of each month spread20th through end of each month attack
bull Payload Web site defacementbull Scanning Random IP addressesbull Bug failure to seed random number generator
86
Code Red Revisions
bull Released July 19 2001
bull Payload flooding attack on wwwwhitehousegovndash Attack was mounted at the IP address of the Web site
bull Bug died after 20th of each month
bull Random number generator for IP scanning fixed
87
Code Red Host Infection Rate
Exponential infection rate
Measured using backscatter technique
88
Designing Fast-Spreading Worms
bull Hit-list scanningndash Time to infect first 10k hosts dominates infection timendash Solution Reconnaissance (stealthy scans etc)
bull Permutation scanningndash Observation Most scanning is redundantndash Idea Shared permutation of address space Start scanning
from own IP address Re-randomize when another infected machine is found
bull Internet-scale hit listsndash Flash worm complete infection within 30 seconds
89
Botnets
bull Bots Autonomous programs performing tasksbull Plenty of ldquobenignrdquo bots
ndash eg weatherbug
bull Botnets group of bots ndash Typically carries malicious connotationndash Large numbers of infected machinesndash Machines ldquoenlistedrdquo with infection vectors like worms
(last lecture)
bull Available for simultaneous control by a masterbull Size up to 350000 nodes (from todayrsquos paper)
90
ldquoRallyingrdquo the Botnet
bull Easy to combine worm backdoor functionalitybull Problem how to learn about successfully
infected machines
bull Optionsndash Emailndash Hard-coded email address
91
Botnet Control
bull Botnet master typically runs some IRC server on a well-known port (eg 6667)
bull Infected machine contacts botnet with pre-programmed DNS name (eg big-botde)
bull Dynamic DNS allows controller to move about freely
Infected Machine
DynamicDNS
BotnetController
(IRC server)
92
Some Defenses
93
Idea 1 Ingress Filtering
bull RFC 2827 Routers install filters to drop packets from networks that are not downstream
bull Feasible at edgesbull Difficult to configure closer to network ldquocorerdquo
20469207024 Internet
Drop all packets with source address other than 20469207024
94
Idea 2 uRPF Checks
bull Unicast Reverse Path Forwardingndash Cisco ldquoip verify unicast reverse-pathrdquo
bull Requires symmetric routing
Accept packet from interface only if forwarding table entry for source IP address matches ingress interface
100183
A10018
10015
101203
100181 24
10011 24
Strict Mode uRPF
Enabled
ldquoArdquo Routing TableDestination Next Hop1001024 Int 110018024 Int 2
100183 from wrong interface
95
Problems with uRPF
bull Asymmetric routing
96
S-BGP
bull Address-based PKI validate signaturesndash Authentication of
bull ownership for IP address blocks bull AS number bull an ASs identity and bull a BGP routers identity
ndash Use existing infrastructure (Internet registries etc)ndash Routing origination is digitally signedndash BGP updates are digitally signed
1048708 bull Route attestations A new optional BGP transitive path attribute
ndash carries digital signatures covering the routing information in updates
97
Practical Problems with S-BGP
bull Requires Public-Key Infrastructure
bull Lots of digital signatures to calculate and verifyndash Message overheadndash CPU overhead
bull Calculation expense is greatest when topology is changingndash Caching can help
bull Route aggregation is problematic (maybe thatrsquos OK)
bull Secure route withdrawals when link or node fails
bull Address ownership data out of date
bull Deployment
- Networking
- Goal of This Tutorial
- Todayrsquos Networks
- Business Models
- Billing for Internet Usage
- Net Neutrality
- Network Operations
- Point-of-Presence (PoP)
- Example Abilene Network Topology
- Another Example Backbone
- Internet Routing Overview
- Internet Routing Protocol BGP
- Two Flavors of BGP
- IPv4 Addresses Networks of Networks
- Pre-1994 Classful Addressing
- Classless Interdomain Routing (CIDR)
- Benefits of CIDR
- Growth of IP Prefixes
- 1994-1998 Linear Growth
- Around 2000 Fast Growth Resumes
- Fast growth resumes
- The Address Allocation Process
- Common Problems
- What can go wrong
- Measurement and Monitoring
- Passive vs Active Measurement
- Slide 27
- Passive Traffic Data Measurement
- Simple Network Management Protocol
- SNMP (Passive)
- Packet-level Monitoring
- Full Packet Capture (Passive)
- What is a flow
- Cisco NetFlow
- Flow Record Contents
- Aggregating Packets into Flows
- Reducing Measurement Overhead
- Packet Sampling for Flow Monitoring
- Sampling Flow-Level Sampling
- Two Main Approaches
- Packet Capture on High-Speed Links
- Characteristics of Packet Capture
- Data Measurement Repositories
- Multihoming and Traffic Engineering
- What is Multihoming
- Why Multihome
- Redundancy
- Performance
- Multihoming in IP Networks Today
- BGP or no
- Same Provider or Multiple
- Multihomed Stub One Link
- Multihomed Stub Multiple Links
- Multihomed Stub Multiple ISPs
- Multihomed Transit Network
- Interdomain Traffic Engineering
- Outbound Traffic Control
- Outbound Traffic Load Balancing
- Traffic Engineering Goals
- Managing Scale
- Achieving Predictability
- Challenges to Predictability
- Inter-AS Negotiation
- Outbound Multihoming Goals
- Inbound Traffic Control
- How many links are enough
- Problems with Multihoming in IPv4
- Slide 68
- Slide 69
- Diagnosis and Troubleshooting
- Operator Mailing List (NANOG)
- Operator Mailing List
- Routing Configuration
- Internet Business Model (Simplified)
- Peering Contracts Consistent Export
- Consistent Export
- Inconsistent Export in Practice
- Blackholes
- Security
- Security ldquoBogonrdquo Routes
- Spam Phishing etc
- Spam and Routing
- Worms and Botnets
- What is a Worm
- Example Worm Code Red
- Code Red Revisions
- Code Red Host Infection Rate
- Designing Fast-Spreading Worms
- Botnets
- ldquoRallyingrdquo the Botnet
- Botnet Control
- Some Defenses
- Idea 1 Ingress Filtering
- Idea 2 uRPF Checks
- Problems with uRPF
- S-BGP
- Practical Problems with S-BGP
-
67
Problems with Multihoming in IPv4
bull Routing table growthndash Provider-based addressingndash Advertising prefix out multiple ISPs ndash canrsquot aggregate
bull Poor control over inbound trafficndash Existing mechanisms do not allow hosts to control
inbound traffic
68
GeorgiaTech
Internet Routing Overview
bull Intradomain (ie ldquointra-ASrdquo) routingbull Interdomain routing
Comcast
Abilene
ATampT Cogent
Autonomous Systems (ASes)
69
Configuration Problems ldquoAS 7007rdquoldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
UUNet
Florida InternetBarn
Sprint
70
Diagnosis and Troubleshootingldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
ldquoMicrosofts websites were offline for up to 23 hoursbecause of a [router] misconfigurationhellipit took nearly a day to determine what was wrong and undo the changesrdquo -- wiredcom January 25 2001
ldquoWorldCom Inchellipsuffered a widespread outage on its Internet backbone that affected roughly 20 percent of its US customer base The network problemshellipaffected millions of computer users worldwide A spokeswoman attributed the outage to a route table issue -- cnncom October 3 2002
A number of Covad customers went out from 5pm today due to supposedly a DDOS (distributed denial of service attack) on a key Level3 data center which later was described as a route leak (misconfiguration)ldquo
-- dslreportscom February 23 2004
71
Operator Mailing List (NANOG)Date Mon 18 Oct 2004 091515 -0700Subject Level 3 US east coast issues
Level 3 experiencing widespread unspecified routing issues on the US east coast Master ticket 1086844 Anyone have more specific information
Date Mon 18 Oct 2004 122034 -0400 (EDT)Subject Re Level 3 US east coast issues
Level 3 is currently experiencing a backbone outage causing routing instability and packet loss We are working to restore and will be sending hourly updateshellip
72
Operator Mailing List
0102030405060708090
Filtering RouteLeaks
RouteHijacks
RouteInstability
RoutingLoops
Blackholes
Occurr
ences o
ver
10 Y
ears
1995-1997 1998-2001 2001-2004
Note Only includes problems openly discussed on this list
Compare 83 power outages 1 fire
73
Routing Configuration
Ranking route selection
Dissemination internal route advertisement
Filtering route advertisement
Customer
Competitor
Primary
Backup
74
Internet Business Model (Simplified)
bull CustomerProvider One AS pays another for reachability to some set of destinations
bull ldquoSettlement-freerdquo Peering Bartering Two ASes exchange routes with one another
Provider
Peer
Customer
Preferences implemented with local preference manipulation
Destination
Pay to use
Get paid to use
Free to use
75
Peering Contracts Consistent Export
bull Rules of settlement-free peeringndash Advertise routes at all peering pointsndash Advertised routes must have equal ldquoAS path lengthrdquo
Sprint
ATampT
Enables ldquohot potatordquo routing
ldquoequally goodrdquoroutes
76
Consistent Export
bull Malicedeceptionbull iBGP signaling partitionbull Inconsistent export policy
neighbor 10123route-map PEER permit 10 set prepend 123
neighbor 10456route-map PEER permit 10 set prepend 123 123
Possible Causes
10123 456 1
10456 456 2
Neighbor AS Export
1 1 123
Export Clause Prepend
2 1 123 123
Two different Export Policies
77
Inconsistent Export in Practice
Feamster et al ldquoBorderGuard Detecting Cold Potatoes from Peersrdquo ACM IMC October 2004
78
Blackholes
Date Thu 18 Jul 2002 060510 -0400 (EDT)From Chad Oleary ltcolpoboxcomgtSubject Re problems with 701To ltnanogmeritedugt
Were starting to see the same issues with UUNet again Anyone elseseeing this Trying to reach Qwest
traceroute to 631461901 (631461901) 30 hops max 38 byte packets 1 esc-lp2-gwe-solutionscorpcom (631182201) 1167 ms 1163 ms 1142 ms 2 500Serial2-10GW1TPA2ALTERNET (1571301499) 1097 ms 1059 ms 1044 ms 3 161at-1-0-0XL4ATL1ALTERNET (1526381190) 13839 ms 14108 ms 16638 ms 4 0so-3-1-0XL2ATL5ALTERNET (152630238) 14370 ms 14587 ms 14553 ms 5 POS7-0BR2ATL5ALTERNET (1526382193) 13928 ms 14099 ms 14053 ms 6 7 hellip
79
Security
80
Security ldquoBogonrdquo Routes
Feamster et al ldquoAn Empirical Study of lsquoBogonrsquo Route Advertisementsrdquo ACM CCR January 2005
81
Spam Phishing etc
bull Unsolicited commercial emailbull As of about August 2008 estimates indicate that
about 95 of all email is spambull Common spam filtering techniques
ndash Content-based filtersndash DNS Blacklist (DNSBL) lookups Significant fraction of
todayrsquos DNS traffic
Can IP addresses from which spam is received be spoofed
82
Spam and Routing
83
Worms and Botnets
84
What is a Worm
bull Code that replicates and propagates across the networkndash Often carries a ldquopayloadrdquo
bull Usually spread via exploiting flaws in open servicesndash ldquoVirusesrdquo require user action to spread
bull First worm Robert Morris November 1988ndash 6-10 of all Internet hosts infected ()
bull Many more since but none on that scale until July 2001
85
Example Worm Code Red
bull Initial version July 13 2001
bull Exploited known ISAPI vulnerability in Microsoft IIS Web servers
bull 1st through 20th of each month spread20th through end of each month attack
bull Payload Web site defacementbull Scanning Random IP addressesbull Bug failure to seed random number generator
86
Code Red Revisions
bull Released July 19 2001
bull Payload flooding attack on wwwwhitehousegovndash Attack was mounted at the IP address of the Web site
bull Bug died after 20th of each month
bull Random number generator for IP scanning fixed
87
Code Red Host Infection Rate
Exponential infection rate
Measured using backscatter technique
88
Designing Fast-Spreading Worms
bull Hit-list scanningndash Time to infect first 10k hosts dominates infection timendash Solution Reconnaissance (stealthy scans etc)
bull Permutation scanningndash Observation Most scanning is redundantndash Idea Shared permutation of address space Start scanning
from own IP address Re-randomize when another infected machine is found
bull Internet-scale hit listsndash Flash worm complete infection within 30 seconds
89
Botnets
bull Bots Autonomous programs performing tasksbull Plenty of ldquobenignrdquo bots
ndash eg weatherbug
bull Botnets group of bots ndash Typically carries malicious connotationndash Large numbers of infected machinesndash Machines ldquoenlistedrdquo with infection vectors like worms
(last lecture)
bull Available for simultaneous control by a masterbull Size up to 350000 nodes (from todayrsquos paper)
90
ldquoRallyingrdquo the Botnet
bull Easy to combine worm backdoor functionalitybull Problem how to learn about successfully
infected machines
bull Optionsndash Emailndash Hard-coded email address
91
Botnet Control
bull Botnet master typically runs some IRC server on a well-known port (eg 6667)
bull Infected machine contacts botnet with pre-programmed DNS name (eg big-botde)
bull Dynamic DNS allows controller to move about freely
Infected Machine
DynamicDNS
BotnetController
(IRC server)
92
Some Defenses
93
Idea 1 Ingress Filtering
bull RFC 2827 Routers install filters to drop packets from networks that are not downstream
bull Feasible at edgesbull Difficult to configure closer to network ldquocorerdquo
20469207024 Internet
Drop all packets with source address other than 20469207024
94
Idea 2 uRPF Checks
bull Unicast Reverse Path Forwardingndash Cisco ldquoip verify unicast reverse-pathrdquo
bull Requires symmetric routing
Accept packet from interface only if forwarding table entry for source IP address matches ingress interface
100183
A10018
10015
101203
100181 24
10011 24
Strict Mode uRPF
Enabled
ldquoArdquo Routing TableDestination Next Hop1001024 Int 110018024 Int 2
100183 from wrong interface
95
Problems with uRPF
bull Asymmetric routing
96
S-BGP
bull Address-based PKI validate signaturesndash Authentication of
bull ownership for IP address blocks bull AS number bull an ASs identity and bull a BGP routers identity
ndash Use existing infrastructure (Internet registries etc)ndash Routing origination is digitally signedndash BGP updates are digitally signed
1048708 bull Route attestations A new optional BGP transitive path attribute
ndash carries digital signatures covering the routing information in updates
97
Practical Problems with S-BGP
bull Requires Public-Key Infrastructure
bull Lots of digital signatures to calculate and verifyndash Message overheadndash CPU overhead
bull Calculation expense is greatest when topology is changingndash Caching can help
bull Route aggregation is problematic (maybe thatrsquos OK)
bull Secure route withdrawals when link or node fails
bull Address ownership data out of date
bull Deployment
- Networking
- Goal of This Tutorial
- Todayrsquos Networks
- Business Models
- Billing for Internet Usage
- Net Neutrality
- Network Operations
- Point-of-Presence (PoP)
- Example Abilene Network Topology
- Another Example Backbone
- Internet Routing Overview
- Internet Routing Protocol BGP
- Two Flavors of BGP
- IPv4 Addresses Networks of Networks
- Pre-1994 Classful Addressing
- Classless Interdomain Routing (CIDR)
- Benefits of CIDR
- Growth of IP Prefixes
- 1994-1998 Linear Growth
- Around 2000 Fast Growth Resumes
- Fast growth resumes
- The Address Allocation Process
- Common Problems
- What can go wrong
- Measurement and Monitoring
- Passive vs Active Measurement
- Slide 27
- Passive Traffic Data Measurement
- Simple Network Management Protocol
- SNMP (Passive)
- Packet-level Monitoring
- Full Packet Capture (Passive)
- What is a flow
- Cisco NetFlow
- Flow Record Contents
- Aggregating Packets into Flows
- Reducing Measurement Overhead
- Packet Sampling for Flow Monitoring
- Sampling Flow-Level Sampling
- Two Main Approaches
- Packet Capture on High-Speed Links
- Characteristics of Packet Capture
- Data Measurement Repositories
- Multihoming and Traffic Engineering
- What is Multihoming
- Why Multihome
- Redundancy
- Performance
- Multihoming in IP Networks Today
- BGP or no
- Same Provider or Multiple
- Multihomed Stub One Link
- Multihomed Stub Multiple Links
- Multihomed Stub Multiple ISPs
- Multihomed Transit Network
- Interdomain Traffic Engineering
- Outbound Traffic Control
- Outbound Traffic Load Balancing
- Traffic Engineering Goals
- Managing Scale
- Achieving Predictability
- Challenges to Predictability
- Inter-AS Negotiation
- Outbound Multihoming Goals
- Inbound Traffic Control
- How many links are enough
- Problems with Multihoming in IPv4
- Slide 68
- Slide 69
- Diagnosis and Troubleshooting
- Operator Mailing List (NANOG)
- Operator Mailing List
- Routing Configuration
- Internet Business Model (Simplified)
- Peering Contracts Consistent Export
- Consistent Export
- Inconsistent Export in Practice
- Blackholes
- Security
- Security ldquoBogonrdquo Routes
- Spam Phishing etc
- Spam and Routing
- Worms and Botnets
- What is a Worm
- Example Worm Code Red
- Code Red Revisions
- Code Red Host Infection Rate
- Designing Fast-Spreading Worms
- Botnets
- ldquoRallyingrdquo the Botnet
- Botnet Control
- Some Defenses
- Idea 1 Ingress Filtering
- Idea 2 uRPF Checks
- Problems with uRPF
- S-BGP
- Practical Problems with S-BGP
-
68
GeorgiaTech
Internet Routing Overview
bull Intradomain (ie ldquointra-ASrdquo) routingbull Interdomain routing
Comcast
Abilene
ATampT Cogent
Autonomous Systems (ASes)
69
Configuration Problems ldquoAS 7007rdquoldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
UUNet
Florida InternetBarn
Sprint
70
Diagnosis and Troubleshootingldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
ldquoMicrosofts websites were offline for up to 23 hoursbecause of a [router] misconfigurationhellipit took nearly a day to determine what was wrong and undo the changesrdquo -- wiredcom January 25 2001
ldquoWorldCom Inchellipsuffered a widespread outage on its Internet backbone that affected roughly 20 percent of its US customer base The network problemshellipaffected millions of computer users worldwide A spokeswoman attributed the outage to a route table issue -- cnncom October 3 2002
A number of Covad customers went out from 5pm today due to supposedly a DDOS (distributed denial of service attack) on a key Level3 data center which later was described as a route leak (misconfiguration)ldquo
-- dslreportscom February 23 2004
71
Operator Mailing List (NANOG)Date Mon 18 Oct 2004 091515 -0700Subject Level 3 US east coast issues
Level 3 experiencing widespread unspecified routing issues on the US east coast Master ticket 1086844 Anyone have more specific information
Date Mon 18 Oct 2004 122034 -0400 (EDT)Subject Re Level 3 US east coast issues
Level 3 is currently experiencing a backbone outage causing routing instability and packet loss We are working to restore and will be sending hourly updateshellip
72
Operator Mailing List
0102030405060708090
Filtering RouteLeaks
RouteHijacks
RouteInstability
RoutingLoops
Blackholes
Occurr
ences o
ver
10 Y
ears
1995-1997 1998-2001 2001-2004
Note Only includes problems openly discussed on this list
Compare 83 power outages 1 fire
73
Routing Configuration
Ranking route selection
Dissemination internal route advertisement
Filtering route advertisement
Customer
Competitor
Primary
Backup
74
Internet Business Model (Simplified)
bull CustomerProvider One AS pays another for reachability to some set of destinations
bull ldquoSettlement-freerdquo Peering Bartering Two ASes exchange routes with one another
Provider
Peer
Customer
Preferences implemented with local preference manipulation
Destination
Pay to use
Get paid to use
Free to use
75
Peering Contracts Consistent Export
bull Rules of settlement-free peeringndash Advertise routes at all peering pointsndash Advertised routes must have equal ldquoAS path lengthrdquo
Sprint
ATampT
Enables ldquohot potatordquo routing
ldquoequally goodrdquoroutes
76
Consistent Export
bull Malicedeceptionbull iBGP signaling partitionbull Inconsistent export policy
neighbor 10123route-map PEER permit 10 set prepend 123
neighbor 10456route-map PEER permit 10 set prepend 123 123
Possible Causes
10123 456 1
10456 456 2
Neighbor AS Export
1 1 123
Export Clause Prepend
2 1 123 123
Two different Export Policies
77
Inconsistent Export in Practice
Feamster et al ldquoBorderGuard Detecting Cold Potatoes from Peersrdquo ACM IMC October 2004
78
Blackholes
Date Thu 18 Jul 2002 060510 -0400 (EDT)From Chad Oleary ltcolpoboxcomgtSubject Re problems with 701To ltnanogmeritedugt
Were starting to see the same issues with UUNet again Anyone elseseeing this Trying to reach Qwest
traceroute to 631461901 (631461901) 30 hops max 38 byte packets 1 esc-lp2-gwe-solutionscorpcom (631182201) 1167 ms 1163 ms 1142 ms 2 500Serial2-10GW1TPA2ALTERNET (1571301499) 1097 ms 1059 ms 1044 ms 3 161at-1-0-0XL4ATL1ALTERNET (1526381190) 13839 ms 14108 ms 16638 ms 4 0so-3-1-0XL2ATL5ALTERNET (152630238) 14370 ms 14587 ms 14553 ms 5 POS7-0BR2ATL5ALTERNET (1526382193) 13928 ms 14099 ms 14053 ms 6 7 hellip
79
Security
80
Security ldquoBogonrdquo Routes
Feamster et al ldquoAn Empirical Study of lsquoBogonrsquo Route Advertisementsrdquo ACM CCR January 2005
81
Spam Phishing etc
bull Unsolicited commercial emailbull As of about August 2008 estimates indicate that
about 95 of all email is spambull Common spam filtering techniques
ndash Content-based filtersndash DNS Blacklist (DNSBL) lookups Significant fraction of
todayrsquos DNS traffic
Can IP addresses from which spam is received be spoofed
82
Spam and Routing
83
Worms and Botnets
84
What is a Worm
bull Code that replicates and propagates across the networkndash Often carries a ldquopayloadrdquo
bull Usually spread via exploiting flaws in open servicesndash ldquoVirusesrdquo require user action to spread
bull First worm Robert Morris November 1988ndash 6-10 of all Internet hosts infected ()
bull Many more since but none on that scale until July 2001
85
Example Worm Code Red
bull Initial version July 13 2001
bull Exploited known ISAPI vulnerability in Microsoft IIS Web servers
bull 1st through 20th of each month spread20th through end of each month attack
bull Payload Web site defacementbull Scanning Random IP addressesbull Bug failure to seed random number generator
86
Code Red Revisions
bull Released July 19 2001
bull Payload flooding attack on wwwwhitehousegovndash Attack was mounted at the IP address of the Web site
bull Bug died after 20th of each month
bull Random number generator for IP scanning fixed
87
Code Red Host Infection Rate
Exponential infection rate
Measured using backscatter technique
88
Designing Fast-Spreading Worms
bull Hit-list scanningndash Time to infect first 10k hosts dominates infection timendash Solution Reconnaissance (stealthy scans etc)
bull Permutation scanningndash Observation Most scanning is redundantndash Idea Shared permutation of address space Start scanning
from own IP address Re-randomize when another infected machine is found
bull Internet-scale hit listsndash Flash worm complete infection within 30 seconds
89
Botnets
bull Bots Autonomous programs performing tasksbull Plenty of ldquobenignrdquo bots
ndash eg weatherbug
bull Botnets group of bots ndash Typically carries malicious connotationndash Large numbers of infected machinesndash Machines ldquoenlistedrdquo with infection vectors like worms
(last lecture)
bull Available for simultaneous control by a masterbull Size up to 350000 nodes (from todayrsquos paper)
90
ldquoRallyingrdquo the Botnet
bull Easy to combine worm backdoor functionalitybull Problem how to learn about successfully
infected machines
bull Optionsndash Emailndash Hard-coded email address
91
Botnet Control
bull Botnet master typically runs some IRC server on a well-known port (eg 6667)
bull Infected machine contacts botnet with pre-programmed DNS name (eg big-botde)
bull Dynamic DNS allows controller to move about freely
Infected Machine
DynamicDNS
BotnetController
(IRC server)
92
Some Defenses
93
Idea 1 Ingress Filtering
bull RFC 2827 Routers install filters to drop packets from networks that are not downstream
bull Feasible at edgesbull Difficult to configure closer to network ldquocorerdquo
20469207024 Internet
Drop all packets with source address other than 20469207024
94
Idea 2 uRPF Checks
bull Unicast Reverse Path Forwardingndash Cisco ldquoip verify unicast reverse-pathrdquo
bull Requires symmetric routing
Accept packet from interface only if forwarding table entry for source IP address matches ingress interface
100183
A10018
10015
101203
100181 24
10011 24
Strict Mode uRPF
Enabled
ldquoArdquo Routing TableDestination Next Hop1001024 Int 110018024 Int 2
100183 from wrong interface
95
Problems with uRPF
bull Asymmetric routing
96
S-BGP
bull Address-based PKI validate signaturesndash Authentication of
bull ownership for IP address blocks bull AS number bull an ASs identity and bull a BGP routers identity
ndash Use existing infrastructure (Internet registries etc)ndash Routing origination is digitally signedndash BGP updates are digitally signed
1048708 bull Route attestations A new optional BGP transitive path attribute
ndash carries digital signatures covering the routing information in updates
97
Practical Problems with S-BGP
bull Requires Public-Key Infrastructure
bull Lots of digital signatures to calculate and verifyndash Message overheadndash CPU overhead
bull Calculation expense is greatest when topology is changingndash Caching can help
bull Route aggregation is problematic (maybe thatrsquos OK)
bull Secure route withdrawals when link or node fails
bull Address ownership data out of date
bull Deployment
- Networking
- Goal of This Tutorial
- Todayrsquos Networks
- Business Models
- Billing for Internet Usage
- Net Neutrality
- Network Operations
- Point-of-Presence (PoP)
- Example Abilene Network Topology
- Another Example Backbone
- Internet Routing Overview
- Internet Routing Protocol BGP
- Two Flavors of BGP
- IPv4 Addresses Networks of Networks
- Pre-1994 Classful Addressing
- Classless Interdomain Routing (CIDR)
- Benefits of CIDR
- Growth of IP Prefixes
- 1994-1998 Linear Growth
- Around 2000 Fast Growth Resumes
- Fast growth resumes
- The Address Allocation Process
- Common Problems
- What can go wrong
- Measurement and Monitoring
- Passive vs Active Measurement
- Slide 27
- Passive Traffic Data Measurement
- Simple Network Management Protocol
- SNMP (Passive)
- Packet-level Monitoring
- Full Packet Capture (Passive)
- What is a flow
- Cisco NetFlow
- Flow Record Contents
- Aggregating Packets into Flows
- Reducing Measurement Overhead
- Packet Sampling for Flow Monitoring
- Sampling Flow-Level Sampling
- Two Main Approaches
- Packet Capture on High-Speed Links
- Characteristics of Packet Capture
- Data Measurement Repositories
- Multihoming and Traffic Engineering
- What is Multihoming
- Why Multihome
- Redundancy
- Performance
- Multihoming in IP Networks Today
- BGP or no
- Same Provider or Multiple
- Multihomed Stub One Link
- Multihomed Stub Multiple Links
- Multihomed Stub Multiple ISPs
- Multihomed Transit Network
- Interdomain Traffic Engineering
- Outbound Traffic Control
- Outbound Traffic Load Balancing
- Traffic Engineering Goals
- Managing Scale
- Achieving Predictability
- Challenges to Predictability
- Inter-AS Negotiation
- Outbound Multihoming Goals
- Inbound Traffic Control
- How many links are enough
- Problems with Multihoming in IPv4
- Slide 68
- Slide 69
- Diagnosis and Troubleshooting
- Operator Mailing List (NANOG)
- Operator Mailing List
- Routing Configuration
- Internet Business Model (Simplified)
- Peering Contracts Consistent Export
- Consistent Export
- Inconsistent Export in Practice
- Blackholes
- Security
- Security ldquoBogonrdquo Routes
- Spam Phishing etc
- Spam and Routing
- Worms and Botnets
- What is a Worm
- Example Worm Code Red
- Code Red Revisions
- Code Red Host Infection Rate
- Designing Fast-Spreading Worms
- Botnets
- ldquoRallyingrdquo the Botnet
- Botnet Control
- Some Defenses
- Idea 1 Ingress Filtering
- Idea 2 uRPF Checks
- Problems with uRPF
- S-BGP
- Practical Problems with S-BGP
-
69
Configuration Problems ldquoAS 7007rdquoldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
UUNet
Florida InternetBarn
Sprint
70
Diagnosis and Troubleshootingldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
ldquoMicrosofts websites were offline for up to 23 hoursbecause of a [router] misconfigurationhellipit took nearly a day to determine what was wrong and undo the changesrdquo -- wiredcom January 25 2001
ldquoWorldCom Inchellipsuffered a widespread outage on its Internet backbone that affected roughly 20 percent of its US customer base The network problemshellipaffected millions of computer users worldwide A spokeswoman attributed the outage to a route table issue -- cnncom October 3 2002
A number of Covad customers went out from 5pm today due to supposedly a DDOS (distributed denial of service attack) on a key Level3 data center which later was described as a route leak (misconfiguration)ldquo
-- dslreportscom February 23 2004
71
Operator Mailing List (NANOG)Date Mon 18 Oct 2004 091515 -0700Subject Level 3 US east coast issues
Level 3 experiencing widespread unspecified routing issues on the US east coast Master ticket 1086844 Anyone have more specific information
Date Mon 18 Oct 2004 122034 -0400 (EDT)Subject Re Level 3 US east coast issues
Level 3 is currently experiencing a backbone outage causing routing instability and packet loss We are working to restore and will be sending hourly updateshellip
72
Operator Mailing List
0102030405060708090
Filtering RouteLeaks
RouteHijacks
RouteInstability
RoutingLoops
Blackholes
Occurr
ences o
ver
10 Y
ears
1995-1997 1998-2001 2001-2004
Note Only includes problems openly discussed on this list
Compare 83 power outages 1 fire
73
Routing Configuration
Ranking route selection
Dissemination internal route advertisement
Filtering route advertisement
Customer
Competitor
Primary
Backup
74
Internet Business Model (Simplified)
bull CustomerProvider One AS pays another for reachability to some set of destinations
bull ldquoSettlement-freerdquo Peering Bartering Two ASes exchange routes with one another
Provider
Peer
Customer
Preferences implemented with local preference manipulation
Destination
Pay to use
Get paid to use
Free to use
75
Peering Contracts Consistent Export
bull Rules of settlement-free peeringndash Advertise routes at all peering pointsndash Advertised routes must have equal ldquoAS path lengthrdquo
Sprint
ATampT
Enables ldquohot potatordquo routing
ldquoequally goodrdquoroutes
76
Consistent Export
bull Malicedeceptionbull iBGP signaling partitionbull Inconsistent export policy
neighbor 10123route-map PEER permit 10 set prepend 123
neighbor 10456route-map PEER permit 10 set prepend 123 123
Possible Causes
10123 456 1
10456 456 2
Neighbor AS Export
1 1 123
Export Clause Prepend
2 1 123 123
Two different Export Policies
77
Inconsistent Export in Practice
Feamster et al ldquoBorderGuard Detecting Cold Potatoes from Peersrdquo ACM IMC October 2004
78
Blackholes
Date Thu 18 Jul 2002 060510 -0400 (EDT)From Chad Oleary ltcolpoboxcomgtSubject Re problems with 701To ltnanogmeritedugt
Were starting to see the same issues with UUNet again Anyone elseseeing this Trying to reach Qwest
traceroute to 631461901 (631461901) 30 hops max 38 byte packets 1 esc-lp2-gwe-solutionscorpcom (631182201) 1167 ms 1163 ms 1142 ms 2 500Serial2-10GW1TPA2ALTERNET (1571301499) 1097 ms 1059 ms 1044 ms 3 161at-1-0-0XL4ATL1ALTERNET (1526381190) 13839 ms 14108 ms 16638 ms 4 0so-3-1-0XL2ATL5ALTERNET (152630238) 14370 ms 14587 ms 14553 ms 5 POS7-0BR2ATL5ALTERNET (1526382193) 13928 ms 14099 ms 14053 ms 6 7 hellip
79
Security
80
Security ldquoBogonrdquo Routes
Feamster et al ldquoAn Empirical Study of lsquoBogonrsquo Route Advertisementsrdquo ACM CCR January 2005
81
Spam Phishing etc
bull Unsolicited commercial emailbull As of about August 2008 estimates indicate that
about 95 of all email is spambull Common spam filtering techniques
ndash Content-based filtersndash DNS Blacklist (DNSBL) lookups Significant fraction of
todayrsquos DNS traffic
Can IP addresses from which spam is received be spoofed
82
Spam and Routing
83
Worms and Botnets
84
What is a Worm
bull Code that replicates and propagates across the networkndash Often carries a ldquopayloadrdquo
bull Usually spread via exploiting flaws in open servicesndash ldquoVirusesrdquo require user action to spread
bull First worm Robert Morris November 1988ndash 6-10 of all Internet hosts infected ()
bull Many more since but none on that scale until July 2001
85
Example Worm Code Red
bull Initial version July 13 2001
bull Exploited known ISAPI vulnerability in Microsoft IIS Web servers
bull 1st through 20th of each month spread20th through end of each month attack
bull Payload Web site defacementbull Scanning Random IP addressesbull Bug failure to seed random number generator
86
Code Red Revisions
bull Released July 19 2001
bull Payload flooding attack on wwwwhitehousegovndash Attack was mounted at the IP address of the Web site
bull Bug died after 20th of each month
bull Random number generator for IP scanning fixed
87
Code Red Host Infection Rate
Exponential infection rate
Measured using backscatter technique
88
Designing Fast-Spreading Worms
bull Hit-list scanningndash Time to infect first 10k hosts dominates infection timendash Solution Reconnaissance (stealthy scans etc)
bull Permutation scanningndash Observation Most scanning is redundantndash Idea Shared permutation of address space Start scanning
from own IP address Re-randomize when another infected machine is found
bull Internet-scale hit listsndash Flash worm complete infection within 30 seconds
89
Botnets
bull Bots Autonomous programs performing tasksbull Plenty of ldquobenignrdquo bots
ndash eg weatherbug
bull Botnets group of bots ndash Typically carries malicious connotationndash Large numbers of infected machinesndash Machines ldquoenlistedrdquo with infection vectors like worms
(last lecture)
bull Available for simultaneous control by a masterbull Size up to 350000 nodes (from todayrsquos paper)
90
ldquoRallyingrdquo the Botnet
bull Easy to combine worm backdoor functionalitybull Problem how to learn about successfully
infected machines
bull Optionsndash Emailndash Hard-coded email address
91
Botnet Control
bull Botnet master typically runs some IRC server on a well-known port (eg 6667)
bull Infected machine contacts botnet with pre-programmed DNS name (eg big-botde)
bull Dynamic DNS allows controller to move about freely
Infected Machine
DynamicDNS
BotnetController
(IRC server)
92
Some Defenses
93
Idea 1 Ingress Filtering
bull RFC 2827 Routers install filters to drop packets from networks that are not downstream
bull Feasible at edgesbull Difficult to configure closer to network ldquocorerdquo
20469207024 Internet
Drop all packets with source address other than 20469207024
94
Idea 2 uRPF Checks
bull Unicast Reverse Path Forwardingndash Cisco ldquoip verify unicast reverse-pathrdquo
bull Requires symmetric routing
Accept packet from interface only if forwarding table entry for source IP address matches ingress interface
100183
A10018
10015
101203
100181 24
10011 24
Strict Mode uRPF
Enabled
ldquoArdquo Routing TableDestination Next Hop1001024 Int 110018024 Int 2
100183 from wrong interface
95
Problems with uRPF
bull Asymmetric routing
96
S-BGP
bull Address-based PKI validate signaturesndash Authentication of
bull ownership for IP address blocks bull AS number bull an ASs identity and bull a BGP routers identity
ndash Use existing infrastructure (Internet registries etc)ndash Routing origination is digitally signedndash BGP updates are digitally signed
1048708 bull Route attestations A new optional BGP transitive path attribute
ndash carries digital signatures covering the routing information in updates
97
Practical Problems with S-BGP
bull Requires Public-Key Infrastructure
bull Lots of digital signatures to calculate and verifyndash Message overheadndash CPU overhead
bull Calculation expense is greatest when topology is changingndash Caching can help
bull Route aggregation is problematic (maybe thatrsquos OK)
bull Secure route withdrawals when link or node fails
bull Address ownership data out of date
bull Deployment
- Networking
- Goal of This Tutorial
- Todayrsquos Networks
- Business Models
- Billing for Internet Usage
- Net Neutrality
- Network Operations
- Point-of-Presence (PoP)
- Example Abilene Network Topology
- Another Example Backbone
- Internet Routing Overview
- Internet Routing Protocol BGP
- Two Flavors of BGP
- IPv4 Addresses Networks of Networks
- Pre-1994 Classful Addressing
- Classless Interdomain Routing (CIDR)
- Benefits of CIDR
- Growth of IP Prefixes
- 1994-1998 Linear Growth
- Around 2000 Fast Growth Resumes
- Fast growth resumes
- The Address Allocation Process
- Common Problems
- What can go wrong
- Measurement and Monitoring
- Passive vs Active Measurement
- Slide 27
- Passive Traffic Data Measurement
- Simple Network Management Protocol
- SNMP (Passive)
- Packet-level Monitoring
- Full Packet Capture (Passive)
- What is a flow
- Cisco NetFlow
- Flow Record Contents
- Aggregating Packets into Flows
- Reducing Measurement Overhead
- Packet Sampling for Flow Monitoring
- Sampling Flow-Level Sampling
- Two Main Approaches
- Packet Capture on High-Speed Links
- Characteristics of Packet Capture
- Data Measurement Repositories
- Multihoming and Traffic Engineering
- What is Multihoming
- Why Multihome
- Redundancy
- Performance
- Multihoming in IP Networks Today
- BGP or no
- Same Provider or Multiple
- Multihomed Stub One Link
- Multihomed Stub Multiple Links
- Multihomed Stub Multiple ISPs
- Multihomed Transit Network
- Interdomain Traffic Engineering
- Outbound Traffic Control
- Outbound Traffic Load Balancing
- Traffic Engineering Goals
- Managing Scale
- Achieving Predictability
- Challenges to Predictability
- Inter-AS Negotiation
- Outbound Multihoming Goals
- Inbound Traffic Control
- How many links are enough
- Problems with Multihoming in IPv4
- Slide 68
- Slide 69
- Diagnosis and Troubleshooting
- Operator Mailing List (NANOG)
- Operator Mailing List
- Routing Configuration
- Internet Business Model (Simplified)
- Peering Contracts Consistent Export
- Consistent Export
- Inconsistent Export in Practice
- Blackholes
- Security
- Security ldquoBogonrdquo Routes
- Spam Phishing etc
- Spam and Routing
- Worms and Botnets
- What is a Worm
- Example Worm Code Red
- Code Red Revisions
- Code Red Host Infection Rate
- Designing Fast-Spreading Worms
- Botnets
- ldquoRallyingrdquo the Botnet
- Botnet Control
- Some Defenses
- Idea 1 Ingress Filtering
- Idea 2 uRPF Checks
- Problems with uRPF
- S-BGP
- Practical Problems with S-BGP
-
70
Diagnosis and Troubleshootingldquohellipa glitch at a small ISPhellip triggered a major outage in Internet access across the country The problem started when MAI Network Servicespassed bad router information from one of its customers onto Sprintrdquo
-- newscom April 25 1997
ldquoMicrosofts websites were offline for up to 23 hoursbecause of a [router] misconfigurationhellipit took nearly a day to determine what was wrong and undo the changesrdquo -- wiredcom January 25 2001
ldquoWorldCom Inchellipsuffered a widespread outage on its Internet backbone that affected roughly 20 percent of its US customer base The network problemshellipaffected millions of computer users worldwide A spokeswoman attributed the outage to a route table issue -- cnncom October 3 2002
A number of Covad customers went out from 5pm today due to supposedly a DDOS (distributed denial of service attack) on a key Level3 data center which later was described as a route leak (misconfiguration)ldquo
-- dslreportscom February 23 2004
71
Operator Mailing List (NANOG)Date Mon 18 Oct 2004 091515 -0700Subject Level 3 US east coast issues
Level 3 experiencing widespread unspecified routing issues on the US east coast Master ticket 1086844 Anyone have more specific information
Date Mon 18 Oct 2004 122034 -0400 (EDT)Subject Re Level 3 US east coast issues
Level 3 is currently experiencing a backbone outage causing routing instability and packet loss We are working to restore and will be sending hourly updateshellip
72
Operator Mailing List
0102030405060708090
Filtering RouteLeaks
RouteHijacks
RouteInstability
RoutingLoops
Blackholes
Occurr
ences o
ver
10 Y
ears
1995-1997 1998-2001 2001-2004
Note Only includes problems openly discussed on this list
Compare 83 power outages 1 fire
73
Routing Configuration
Ranking route selection
Dissemination internal route advertisement
Filtering route advertisement
Customer
Competitor
Primary
Backup
74
Internet Business Model (Simplified)
bull CustomerProvider One AS pays another for reachability to some set of destinations
bull ldquoSettlement-freerdquo Peering Bartering Two ASes exchange routes with one another
Provider
Peer
Customer
Preferences implemented with local preference manipulation
Destination
Pay to use
Get paid to use
Free to use
75
Peering Contracts Consistent Export
bull Rules of settlement-free peeringndash Advertise routes at all peering pointsndash Advertised routes must have equal ldquoAS path lengthrdquo
Sprint
ATampT
Enables ldquohot potatordquo routing
ldquoequally goodrdquoroutes
76
Consistent Export
bull Malicedeceptionbull iBGP signaling partitionbull Inconsistent export policy
neighbor 10123route-map PEER permit 10 set prepend 123
neighbor 10456route-map PEER permit 10 set prepend 123 123
Possible Causes
10123 456 1
10456 456 2
Neighbor AS Export
1 1 123
Export Clause Prepend
2 1 123 123
Two different Export Policies
77
Inconsistent Export in Practice
Feamster et al ldquoBorderGuard Detecting Cold Potatoes from Peersrdquo ACM IMC October 2004
78
Blackholes
Date Thu 18 Jul 2002 060510 -0400 (EDT)From Chad Oleary ltcolpoboxcomgtSubject Re problems with 701To ltnanogmeritedugt
Were starting to see the same issues with UUNet again Anyone elseseeing this Trying to reach Qwest
traceroute to 631461901 (631461901) 30 hops max 38 byte packets 1 esc-lp2-gwe-solutionscorpcom (631182201) 1167 ms 1163 ms 1142 ms 2 500Serial2-10GW1TPA2ALTERNET (1571301499) 1097 ms 1059 ms 1044 ms 3 161at-1-0-0XL4ATL1ALTERNET (1526381190) 13839 ms 14108 ms 16638 ms 4 0so-3-1-0XL2ATL5ALTERNET (152630238) 14370 ms 14587 ms 14553 ms 5 POS7-0BR2ATL5ALTERNET (1526382193) 13928 ms 14099 ms 14053 ms 6 7 hellip
79
Security
80
Security ldquoBogonrdquo Routes
Feamster et al ldquoAn Empirical Study of lsquoBogonrsquo Route Advertisementsrdquo ACM CCR January 2005
81
Spam Phishing etc
bull Unsolicited commercial emailbull As of about August 2008 estimates indicate that
about 95 of all email is spambull Common spam filtering techniques
ndash Content-based filtersndash DNS Blacklist (DNSBL) lookups Significant fraction of
todayrsquos DNS traffic
Can IP addresses from which spam is received be spoofed
82
Spam and Routing
83
Worms and Botnets
84
What is a Worm
bull Code that replicates and propagates across the networkndash Often carries a ldquopayloadrdquo
bull Usually spread via exploiting flaws in open servicesndash ldquoVirusesrdquo require user action to spread
bull First worm Robert Morris November 1988ndash 6-10 of all Internet hosts infected ()
bull Many more since but none on that scale until July 2001
85
Example Worm Code Red
bull Initial version July 13 2001
bull Exploited known ISAPI vulnerability in Microsoft IIS Web servers
bull 1st through 20th of each month spread20th through end of each month attack
bull Payload Web site defacementbull Scanning Random IP addressesbull Bug failure to seed random number generator
86
Code Red Revisions
bull Released July 19 2001
bull Payload flooding attack on wwwwhitehousegovndash Attack was mounted at the IP address of the Web site
bull Bug died after 20th of each month
bull Random number generator for IP scanning fixed
87
Code Red Host Infection Rate
Exponential infection rate
Measured using backscatter technique
88
Designing Fast-Spreading Worms
bull Hit-list scanningndash Time to infect first 10k hosts dominates infection timendash Solution Reconnaissance (stealthy scans etc)
bull Permutation scanningndash Observation Most scanning is redundantndash Idea Shared permutation of address space Start scanning
from own IP address Re-randomize when another infected machine is found
bull Internet-scale hit listsndash Flash worm complete infection within 30 seconds
89
Botnets
bull Bots Autonomous programs performing tasksbull Plenty of ldquobenignrdquo bots
ndash eg weatherbug
bull Botnets group of bots ndash Typically carries malicious connotationndash Large numbers of infected machinesndash Machines ldquoenlistedrdquo with infection vectors like worms
(last lecture)
bull Available for simultaneous control by a masterbull Size up to 350000 nodes (from todayrsquos paper)
90
ldquoRallyingrdquo the Botnet
bull Easy to combine worm backdoor functionalitybull Problem how to learn about successfully
infected machines
bull Optionsndash Emailndash Hard-coded email address
91
Botnet Control
bull Botnet master typically runs some IRC server on a well-known port (eg 6667)
bull Infected machine contacts botnet with pre-programmed DNS name (eg big-botde)
bull Dynamic DNS allows controller to move about freely
Infected Machine
DynamicDNS
BotnetController
(IRC server)
92
Some Defenses
93
Idea 1 Ingress Filtering
bull RFC 2827 Routers install filters to drop packets from networks that are not downstream
bull Feasible at edgesbull Difficult to configure closer to network ldquocorerdquo
20469207024 Internet
Drop all packets with source address other than 20469207024
94
Idea 2 uRPF Checks
bull Unicast Reverse Path Forwardingndash Cisco ldquoip verify unicast reverse-pathrdquo
bull Requires symmetric routing
Accept packet from interface only if forwarding table entry for source IP address matches ingress interface
100183
A10018
10015
101203
100181 24
10011 24
Strict Mode uRPF
Enabled
ldquoArdquo Routing TableDestination Next Hop1001024 Int 110018024 Int 2
100183 from wrong interface
95
Problems with uRPF
bull Asymmetric routing
96
S-BGP
bull Address-based PKI validate signaturesndash Authentication of
bull ownership for IP address blocks bull AS number bull an ASs identity and bull a BGP routers identity
ndash Use existing infrastructure (Internet registries etc)ndash Routing origination is digitally signedndash BGP updates are digitally signed
1048708 bull Route attestations A new optional BGP transitive path attribute
ndash carries digital signatures covering the routing information in updates
97
Practical Problems with S-BGP
bull Requires Public-Key Infrastructure
bull Lots of digital signatures to calculate and verifyndash Message overheadndash CPU overhead
bull Calculation expense is greatest when topology is changingndash Caching can help
bull Route aggregation is problematic (maybe thatrsquos OK)
bull Secure route withdrawals when link or node fails
bull Address ownership data out of date
bull Deployment
- Networking
- Goal of This Tutorial
- Todayrsquos Networks
- Business Models
- Billing for Internet Usage
- Net Neutrality
- Network Operations
- Point-of-Presence (PoP)
- Example Abilene Network Topology
- Another Example Backbone
- Internet Routing Overview
- Internet Routing Protocol BGP
- Two Flavors of BGP
- IPv4 Addresses Networks of Networks
- Pre-1994 Classful Addressing
- Classless Interdomain Routing (CIDR)
- Benefits of CIDR
- Growth of IP Prefixes
- 1994-1998 Linear Growth
- Around 2000 Fast Growth Resumes
- Fast growth resumes
- The Address Allocation Process
- Common Problems
- What can go wrong
- Measurement and Monitoring
- Passive vs Active Measurement
- Slide 27
- Passive Traffic Data Measurement
- Simple Network Management Protocol
- SNMP (Passive)
- Packet-level Monitoring
- Full Packet Capture (Passive)
- What is a flow
- Cisco NetFlow
- Flow Record Contents
- Aggregating Packets into Flows
- Reducing Measurement Overhead
- Packet Sampling for Flow Monitoring
- Sampling Flow-Level Sampling
- Two Main Approaches
- Packet Capture on High-Speed Links
- Characteristics of Packet Capture
- Data Measurement Repositories
- Multihoming and Traffic Engineering
- What is Multihoming
- Why Multihome
- Redundancy
- Performance
- Multihoming in IP Networks Today
- BGP or no
- Same Provider or Multiple
- Multihomed Stub One Link
- Multihomed Stub Multiple Links
- Multihomed Stub Multiple ISPs
- Multihomed Transit Network
- Interdomain Traffic Engineering
- Outbound Traffic Control
- Outbound Traffic Load Balancing
- Traffic Engineering Goals
- Managing Scale
- Achieving Predictability
- Challenges to Predictability
- Inter-AS Negotiation
- Outbound Multihoming Goals
- Inbound Traffic Control
- How many links are enough
- Problems with Multihoming in IPv4
- Slide 68
- Slide 69
- Diagnosis and Troubleshooting
- Operator Mailing List (NANOG)
- Operator Mailing List
- Routing Configuration
- Internet Business Model (Simplified)
- Peering Contracts Consistent Export
- Consistent Export
- Inconsistent Export in Practice
- Blackholes
- Security
- Security ldquoBogonrdquo Routes
- Spam Phishing etc
- Spam and Routing
- Worms and Botnets
- What is a Worm
- Example Worm Code Red
- Code Red Revisions
- Code Red Host Infection Rate
- Designing Fast-Spreading Worms
- Botnets
- ldquoRallyingrdquo the Botnet
- Botnet Control
- Some Defenses
- Idea 1 Ingress Filtering
- Idea 2 uRPF Checks
- Problems with uRPF
- S-BGP
- Practical Problems with S-BGP
-
71
Operator Mailing List (NANOG)Date Mon 18 Oct 2004 091515 -0700Subject Level 3 US east coast issues
Level 3 experiencing widespread unspecified routing issues on the US east coast Master ticket 1086844 Anyone have more specific information
Date Mon 18 Oct 2004 122034 -0400 (EDT)Subject Re Level 3 US east coast issues
Level 3 is currently experiencing a backbone outage causing routing instability and packet loss We are working to restore and will be sending hourly updateshellip
72
Operator Mailing List
0102030405060708090
Filtering RouteLeaks
RouteHijacks
RouteInstability
RoutingLoops
Blackholes
Occurr
ences o
ver
10 Y
ears
1995-1997 1998-2001 2001-2004
Note Only includes problems openly discussed on this list
Compare 83 power outages 1 fire
73
Routing Configuration
Ranking route selection
Dissemination internal route advertisement
Filtering route advertisement
Customer
Competitor
Primary
Backup
74
Internet Business Model (Simplified)
bull CustomerProvider One AS pays another for reachability to some set of destinations
bull ldquoSettlement-freerdquo Peering Bartering Two ASes exchange routes with one another
Provider
Peer
Customer
Preferences implemented with local preference manipulation
Destination
Pay to use
Get paid to use
Free to use
75
Peering Contracts Consistent Export
bull Rules of settlement-free peeringndash Advertise routes at all peering pointsndash Advertised routes must have equal ldquoAS path lengthrdquo
Sprint
ATampT
Enables ldquohot potatordquo routing
ldquoequally goodrdquoroutes
76
Consistent Export
bull Malicedeceptionbull iBGP signaling partitionbull Inconsistent export policy
neighbor 10123route-map PEER permit 10 set prepend 123
neighbor 10456route-map PEER permit 10 set prepend 123 123
Possible Causes
10123 456 1
10456 456 2
Neighbor AS Export
1 1 123
Export Clause Prepend
2 1 123 123
Two different Export Policies
77
Inconsistent Export in Practice
Feamster et al ldquoBorderGuard Detecting Cold Potatoes from Peersrdquo ACM IMC October 2004
78
Blackholes
Date Thu 18 Jul 2002 060510 -0400 (EDT)From Chad Oleary ltcolpoboxcomgtSubject Re problems with 701To ltnanogmeritedugt
Were starting to see the same issues with UUNet again Anyone elseseeing this Trying to reach Qwest
traceroute to 631461901 (631461901) 30 hops max 38 byte packets 1 esc-lp2-gwe-solutionscorpcom (631182201) 1167 ms 1163 ms 1142 ms 2 500Serial2-10GW1TPA2ALTERNET (1571301499) 1097 ms 1059 ms 1044 ms 3 161at-1-0-0XL4ATL1ALTERNET (1526381190) 13839 ms 14108 ms 16638 ms 4 0so-3-1-0XL2ATL5ALTERNET (152630238) 14370 ms 14587 ms 14553 ms 5 POS7-0BR2ATL5ALTERNET (1526382193) 13928 ms 14099 ms 14053 ms 6 7 hellip
79
Security
80
Security ldquoBogonrdquo Routes
Feamster et al ldquoAn Empirical Study of lsquoBogonrsquo Route Advertisementsrdquo ACM CCR January 2005
81
Spam Phishing etc
bull Unsolicited commercial emailbull As of about August 2008 estimates indicate that
about 95 of all email is spambull Common spam filtering techniques
ndash Content-based filtersndash DNS Blacklist (DNSBL) lookups Significant fraction of
todayrsquos DNS traffic
Can IP addresses from which spam is received be spoofed
82
Spam and Routing
83
Worms and Botnets
84
What is a Worm
bull Code that replicates and propagates across the networkndash Often carries a ldquopayloadrdquo
bull Usually spread via exploiting flaws in open servicesndash ldquoVirusesrdquo require user action to spread
bull First worm Robert Morris November 1988ndash 6-10 of all Internet hosts infected ()
bull Many more since but none on that scale until July 2001
85
Example Worm Code Red
bull Initial version July 13 2001
bull Exploited known ISAPI vulnerability in Microsoft IIS Web servers
bull 1st through 20th of each month spread20th through end of each month attack
bull Payload Web site defacementbull Scanning Random IP addressesbull Bug failure to seed random number generator
86
Code Red Revisions
bull Released July 19 2001
bull Payload flooding attack on wwwwhitehousegovndash Attack was mounted at the IP address of the Web site
bull Bug died after 20th of each month
bull Random number generator for IP scanning fixed
87
Code Red Host Infection Rate
Exponential infection rate
Measured using backscatter technique
88
Designing Fast-Spreading Worms
bull Hit-list scanningndash Time to infect first 10k hosts dominates infection timendash Solution Reconnaissance (stealthy scans etc)
bull Permutation scanningndash Observation Most scanning is redundantndash Idea Shared permutation of address space Start scanning
from own IP address Re-randomize when another infected machine is found
bull Internet-scale hit listsndash Flash worm complete infection within 30 seconds
89
Botnets
bull Bots Autonomous programs performing tasksbull Plenty of ldquobenignrdquo bots
ndash eg weatherbug
bull Botnets group of bots ndash Typically carries malicious connotationndash Large numbers of infected machinesndash Machines ldquoenlistedrdquo with infection vectors like worms
(last lecture)
bull Available for simultaneous control by a masterbull Size up to 350000 nodes (from todayrsquos paper)
90
ldquoRallyingrdquo the Botnet
bull Easy to combine worm backdoor functionalitybull Problem how to learn about successfully
infected machines
bull Optionsndash Emailndash Hard-coded email address
91
Botnet Control
bull Botnet master typically runs some IRC server on a well-known port (eg 6667)
bull Infected machine contacts botnet with pre-programmed DNS name (eg big-botde)
bull Dynamic DNS allows controller to move about freely
Infected Machine
DynamicDNS
BotnetController
(IRC server)
92
Some Defenses
93
Idea 1 Ingress Filtering
bull RFC 2827 Routers install filters to drop packets from networks that are not downstream
bull Feasible at edgesbull Difficult to configure closer to network ldquocorerdquo
20469207024 Internet
Drop all packets with source address other than 20469207024
94
Idea 2 uRPF Checks
bull Unicast Reverse Path Forwardingndash Cisco ldquoip verify unicast reverse-pathrdquo
bull Requires symmetric routing
Accept packet from interface only if forwarding table entry for source IP address matches ingress interface
100183
A10018
10015
101203
100181 24
10011 24
Strict Mode uRPF
Enabled
ldquoArdquo Routing TableDestination Next Hop1001024 Int 110018024 Int 2
100183 from wrong interface
95
Problems with uRPF
bull Asymmetric routing
96
S-BGP
bull Address-based PKI validate signaturesndash Authentication of
bull ownership for IP address blocks bull AS number bull an ASs identity and bull a BGP routers identity
ndash Use existing infrastructure (Internet registries etc)ndash Routing origination is digitally signedndash BGP updates are digitally signed
1048708 bull Route attestations A new optional BGP transitive path attribute
ndash carries digital signatures covering the routing information in updates
97
Practical Problems with S-BGP
bull Requires Public-Key Infrastructure
bull Lots of digital signatures to calculate and verifyndash Message overheadndash CPU overhead
bull Calculation expense is greatest when topology is changingndash Caching can help
bull Route aggregation is problematic (maybe thatrsquos OK)
bull Secure route withdrawals when link or node fails
bull Address ownership data out of date
bull Deployment
- Networking
- Goal of This Tutorial
- Todayrsquos Networks
- Business Models
- Billing for Internet Usage
- Net Neutrality
- Network Operations
- Point-of-Presence (PoP)
- Example Abilene Network Topology
- Another Example Backbone
- Internet Routing Overview
- Internet Routing Protocol BGP
- Two Flavors of BGP
- IPv4 Addresses Networks of Networks
- Pre-1994 Classful Addressing
- Classless Interdomain Routing (CIDR)
- Benefits of CIDR
- Growth of IP Prefixes
- 1994-1998 Linear Growth
- Around 2000 Fast Growth Resumes
- Fast growth resumes
- The Address Allocation Process
- Common Problems
- What can go wrong
- Measurement and Monitoring
- Passive vs Active Measurement
- Slide 27
- Passive Traffic Data Measurement
- Simple Network Management Protocol
- SNMP (Passive)
- Packet-level Monitoring
- Full Packet Capture (Passive)
- What is a flow
- Cisco NetFlow
- Flow Record Contents
- Aggregating Packets into Flows
- Reducing Measurement Overhead
- Packet Sampling for Flow Monitoring
- Sampling Flow-Level Sampling
- Two Main Approaches
- Packet Capture on High-Speed Links
- Characteristics of Packet Capture
- Data Measurement Repositories
- Multihoming and Traffic Engineering
- What is Multihoming
- Why Multihome
- Redundancy
- Performance
- Multihoming in IP Networks Today
- BGP or no
- Same Provider or Multiple
- Multihomed Stub One Link
- Multihomed Stub Multiple Links
- Multihomed Stub Multiple ISPs
- Multihomed Transit Network
- Interdomain Traffic Engineering
- Outbound Traffic Control
- Outbound Traffic Load Balancing
- Traffic Engineering Goals
- Managing Scale
- Achieving Predictability
- Challenges to Predictability
- Inter-AS Negotiation
- Outbound Multihoming Goals
- Inbound Traffic Control
- How many links are enough
- Problems with Multihoming in IPv4
- Slide 68
- Slide 69
- Diagnosis and Troubleshooting
- Operator Mailing List (NANOG)
- Operator Mailing List
- Routing Configuration
- Internet Business Model (Simplified)
- Peering Contracts Consistent Export
- Consistent Export
- Inconsistent Export in Practice
- Blackholes
- Security
- Security ldquoBogonrdquo Routes
- Spam Phishing etc
- Spam and Routing
- Worms and Botnets
- What is a Worm
- Example Worm Code Red
- Code Red Revisions
- Code Red Host Infection Rate
- Designing Fast-Spreading Worms
- Botnets
- ldquoRallyingrdquo the Botnet
- Botnet Control
- Some Defenses
- Idea 1 Ingress Filtering
- Idea 2 uRPF Checks
- Problems with uRPF
- S-BGP
- Practical Problems with S-BGP
-
72
Operator Mailing List
0102030405060708090
Filtering RouteLeaks
RouteHijacks
RouteInstability
RoutingLoops
Blackholes
Occurr
ences o
ver
10 Y
ears
1995-1997 1998-2001 2001-2004
Note Only includes problems openly discussed on this list
Compare 83 power outages 1 fire
73
Routing Configuration
Ranking route selection
Dissemination internal route advertisement
Filtering route advertisement
Customer
Competitor
Primary
Backup
74
Internet Business Model (Simplified)
bull CustomerProvider One AS pays another for reachability to some set of destinations
bull ldquoSettlement-freerdquo Peering Bartering Two ASes exchange routes with one another
Provider
Peer
Customer
Preferences implemented with local preference manipulation
Destination
Pay to use
Get paid to use
Free to use
75
Peering Contracts Consistent Export
bull Rules of settlement-free peeringndash Advertise routes at all peering pointsndash Advertised routes must have equal ldquoAS path lengthrdquo
Sprint
ATampT
Enables ldquohot potatordquo routing
ldquoequally goodrdquoroutes
76
Consistent Export
bull Malicedeceptionbull iBGP signaling partitionbull Inconsistent export policy
neighbor 10123route-map PEER permit 10 set prepend 123
neighbor 10456route-map PEER permit 10 set prepend 123 123
Possible Causes
10123 456 1
10456 456 2
Neighbor AS Export
1 1 123
Export Clause Prepend
2 1 123 123
Two different Export Policies
77
Inconsistent Export in Practice
Feamster et al ldquoBorderGuard Detecting Cold Potatoes from Peersrdquo ACM IMC October 2004
78
Blackholes
Date Thu 18 Jul 2002 060510 -0400 (EDT)From Chad Oleary ltcolpoboxcomgtSubject Re problems with 701To ltnanogmeritedugt
Were starting to see the same issues with UUNet again Anyone elseseeing this Trying to reach Qwest
traceroute to 631461901 (631461901) 30 hops max 38 byte packets 1 esc-lp2-gwe-solutionscorpcom (631182201) 1167 ms 1163 ms 1142 ms 2 500Serial2-10GW1TPA2ALTERNET (1571301499) 1097 ms 1059 ms 1044 ms 3 161at-1-0-0XL4ATL1ALTERNET (1526381190) 13839 ms 14108 ms 16638 ms 4 0so-3-1-0XL2ATL5ALTERNET (152630238) 14370 ms 14587 ms 14553 ms 5 POS7-0BR2ATL5ALTERNET (1526382193) 13928 ms 14099 ms 14053 ms 6 7 hellip
79
Security
80
Security ldquoBogonrdquo Routes
Feamster et al ldquoAn Empirical Study of lsquoBogonrsquo Route Advertisementsrdquo ACM CCR January 2005
81
Spam Phishing etc
bull Unsolicited commercial emailbull As of about August 2008 estimates indicate that
about 95 of all email is spambull Common spam filtering techniques
ndash Content-based filtersndash DNS Blacklist (DNSBL) lookups Significant fraction of
todayrsquos DNS traffic
Can IP addresses from which spam is received be spoofed
82
Spam and Routing
83
Worms and Botnets
84
What is a Worm
bull Code that replicates and propagates across the networkndash Often carries a ldquopayloadrdquo
bull Usually spread via exploiting flaws in open servicesndash ldquoVirusesrdquo require user action to spread
bull First worm Robert Morris November 1988ndash 6-10 of all Internet hosts infected ()
bull Many more since but none on that scale until July 2001
85
Example Worm Code Red
bull Initial version July 13 2001
bull Exploited known ISAPI vulnerability in Microsoft IIS Web servers
bull 1st through 20th of each month spread20th through end of each month attack
bull Payload Web site defacementbull Scanning Random IP addressesbull Bug failure to seed random number generator
86
Code Red Revisions
bull Released July 19 2001
bull Payload flooding attack on wwwwhitehousegovndash Attack was mounted at the IP address of the Web site
bull Bug died after 20th of each month
bull Random number generator for IP scanning fixed
87
Code Red Host Infection Rate
Exponential infection rate
Measured using backscatter technique
88
Designing Fast-Spreading Worms
bull Hit-list scanningndash Time to infect first 10k hosts dominates infection timendash Solution Reconnaissance (stealthy scans etc)
bull Permutation scanningndash Observation Most scanning is redundantndash Idea Shared permutation of address space Start scanning
from own IP address Re-randomize when another infected machine is found
bull Internet-scale hit listsndash Flash worm complete infection within 30 seconds
89
Botnets
bull Bots Autonomous programs performing tasksbull Plenty of ldquobenignrdquo bots
ndash eg weatherbug
bull Botnets group of bots ndash Typically carries malicious connotationndash Large numbers of infected machinesndash Machines ldquoenlistedrdquo with infection vectors like worms
(last lecture)
bull Available for simultaneous control by a masterbull Size up to 350000 nodes (from todayrsquos paper)
90
ldquoRallyingrdquo the Botnet
bull Easy to combine worm backdoor functionalitybull Problem how to learn about successfully
infected machines
bull Optionsndash Emailndash Hard-coded email address
91
Botnet Control
bull Botnet master typically runs some IRC server on a well-known port (eg 6667)
bull Infected machine contacts botnet with pre-programmed DNS name (eg big-botde)
bull Dynamic DNS allows controller to move about freely
Infected Machine
DynamicDNS
BotnetController
(IRC server)
92
Some Defenses
93
Idea 1 Ingress Filtering
bull RFC 2827 Routers install filters to drop packets from networks that are not downstream
bull Feasible at edgesbull Difficult to configure closer to network ldquocorerdquo
20469207024 Internet
Drop all packets with source address other than 20469207024
94
Idea 2 uRPF Checks
bull Unicast Reverse Path Forwardingndash Cisco ldquoip verify unicast reverse-pathrdquo
bull Requires symmetric routing
Accept packet from interface only if forwarding table entry for source IP address matches ingress interface
100183
A10018
10015
101203
100181 24
10011 24
Strict Mode uRPF
Enabled
ldquoArdquo Routing TableDestination Next Hop1001024 Int 110018024 Int 2
100183 from wrong interface
95
Problems with uRPF
bull Asymmetric routing
96
S-BGP
bull Address-based PKI validate signaturesndash Authentication of
bull ownership for IP address blocks bull AS number bull an ASs identity and bull a BGP routers identity
ndash Use existing infrastructure (Internet registries etc)ndash Routing origination is digitally signedndash BGP updates are digitally signed
1048708 bull Route attestations A new optional BGP transitive path attribute
ndash carries digital signatures covering the routing information in updates
97
Practical Problems with S-BGP
bull Requires Public-Key Infrastructure
bull Lots of digital signatures to calculate and verifyndash Message overheadndash CPU overhead
bull Calculation expense is greatest when topology is changingndash Caching can help
bull Route aggregation is problematic (maybe thatrsquos OK)
bull Secure route withdrawals when link or node fails
bull Address ownership data out of date
bull Deployment
- Networking
- Goal of This Tutorial
- Todayrsquos Networks
- Business Models
- Billing for Internet Usage
- Net Neutrality
- Network Operations
- Point-of-Presence (PoP)
- Example Abilene Network Topology
- Another Example Backbone
- Internet Routing Overview
- Internet Routing Protocol BGP
- Two Flavors of BGP
- IPv4 Addresses Networks of Networks
- Pre-1994 Classful Addressing
- Classless Interdomain Routing (CIDR)
- Benefits of CIDR
- Growth of IP Prefixes
- 1994-1998 Linear Growth
- Around 2000 Fast Growth Resumes
- Fast growth resumes
- The Address Allocation Process
- Common Problems
- What can go wrong
- Measurement and Monitoring
- Passive vs Active Measurement
- Slide 27
- Passive Traffic Data Measurement
- Simple Network Management Protocol
- SNMP (Passive)
- Packet-level Monitoring
- Full Packet Capture (Passive)
- What is a flow
- Cisco NetFlow
- Flow Record Contents
- Aggregating Packets into Flows
- Reducing Measurement Overhead
- Packet Sampling for Flow Monitoring
- Sampling Flow-Level Sampling
- Two Main Approaches
- Packet Capture on High-Speed Links
- Characteristics of Packet Capture
- Data Measurement Repositories
- Multihoming and Traffic Engineering
- What is Multihoming
- Why Multihome
- Redundancy
- Performance
- Multihoming in IP Networks Today
- BGP or no
- Same Provider or Multiple
- Multihomed Stub One Link
- Multihomed Stub Multiple Links
- Multihomed Stub Multiple ISPs
- Multihomed Transit Network
- Interdomain Traffic Engineering
- Outbound Traffic Control
- Outbound Traffic Load Balancing
- Traffic Engineering Goals
- Managing Scale
- Achieving Predictability
- Challenges to Predictability
- Inter-AS Negotiation
- Outbound Multihoming Goals
- Inbound Traffic Control
- How many links are enough
- Problems with Multihoming in IPv4
- Slide 68
- Slide 69
- Diagnosis and Troubleshooting
- Operator Mailing List (NANOG)
- Operator Mailing List
- Routing Configuration
- Internet Business Model (Simplified)
- Peering Contracts Consistent Export
- Consistent Export
- Inconsistent Export in Practice
- Blackholes
- Security
- Security ldquoBogonrdquo Routes
- Spam Phishing etc
- Spam and Routing
- Worms and Botnets
- What is a Worm
- Example Worm Code Red
- Code Red Revisions
- Code Red Host Infection Rate
- Designing Fast-Spreading Worms
- Botnets
- ldquoRallyingrdquo the Botnet
- Botnet Control
- Some Defenses
- Idea 1 Ingress Filtering
- Idea 2 uRPF Checks
- Problems with uRPF
- S-BGP
- Practical Problems with S-BGP
-
73
Routing Configuration
Ranking route selection
Dissemination internal route advertisement
Filtering route advertisement
Customer
Competitor
Primary
Backup
74
Internet Business Model (Simplified)
bull CustomerProvider One AS pays another for reachability to some set of destinations
bull ldquoSettlement-freerdquo Peering Bartering Two ASes exchange routes with one another
Provider
Peer
Customer
Preferences implemented with local preference manipulation
Destination
Pay to use
Get paid to use
Free to use
75
Peering Contracts Consistent Export
bull Rules of settlement-free peeringndash Advertise routes at all peering pointsndash Advertised routes must have equal ldquoAS path lengthrdquo
Sprint
ATampT
Enables ldquohot potatordquo routing
ldquoequally goodrdquoroutes
76
Consistent Export
bull Malicedeceptionbull iBGP signaling partitionbull Inconsistent export policy
neighbor 10123route-map PEER permit 10 set prepend 123
neighbor 10456route-map PEER permit 10 set prepend 123 123
Possible Causes
10123 456 1
10456 456 2
Neighbor AS Export
1 1 123
Export Clause Prepend
2 1 123 123
Two different Export Policies
77
Inconsistent Export in Practice
Feamster et al ldquoBorderGuard Detecting Cold Potatoes from Peersrdquo ACM IMC October 2004
78
Blackholes
Date Thu 18 Jul 2002 060510 -0400 (EDT)From Chad Oleary ltcolpoboxcomgtSubject Re problems with 701To ltnanogmeritedugt
Were starting to see the same issues with UUNet again Anyone elseseeing this Trying to reach Qwest
traceroute to 631461901 (631461901) 30 hops max 38 byte packets 1 esc-lp2-gwe-solutionscorpcom (631182201) 1167 ms 1163 ms 1142 ms 2 500Serial2-10GW1TPA2ALTERNET (1571301499) 1097 ms 1059 ms 1044 ms 3 161at-1-0-0XL4ATL1ALTERNET (1526381190) 13839 ms 14108 ms 16638 ms 4 0so-3-1-0XL2ATL5ALTERNET (152630238) 14370 ms 14587 ms 14553 ms 5 POS7-0BR2ATL5ALTERNET (1526382193) 13928 ms 14099 ms 14053 ms 6 7 hellip
79
Security
80
Security ldquoBogonrdquo Routes
Feamster et al ldquoAn Empirical Study of lsquoBogonrsquo Route Advertisementsrdquo ACM CCR January 2005
81
Spam Phishing etc
bull Unsolicited commercial emailbull As of about August 2008 estimates indicate that
about 95 of all email is spambull Common spam filtering techniques
ndash Content-based filtersndash DNS Blacklist (DNSBL) lookups Significant fraction of
todayrsquos DNS traffic
Can IP addresses from which spam is received be spoofed
82
Spam and Routing
83
Worms and Botnets
84
What is a Worm
bull Code that replicates and propagates across the networkndash Often carries a ldquopayloadrdquo
bull Usually spread via exploiting flaws in open servicesndash ldquoVirusesrdquo require user action to spread
bull First worm Robert Morris November 1988ndash 6-10 of all Internet hosts infected ()
bull Many more since but none on that scale until July 2001
85
Example Worm Code Red
bull Initial version July 13 2001
bull Exploited known ISAPI vulnerability in Microsoft IIS Web servers
bull 1st through 20th of each month spread20th through end of each month attack
bull Payload Web site defacementbull Scanning Random IP addressesbull Bug failure to seed random number generator
86
Code Red Revisions
bull Released July 19 2001
bull Payload flooding attack on wwwwhitehousegovndash Attack was mounted at the IP address of the Web site
bull Bug died after 20th of each month
bull Random number generator for IP scanning fixed
87
Code Red Host Infection Rate
Exponential infection rate
Measured using backscatter technique
88
Designing Fast-Spreading Worms
bull Hit-list scanningndash Time to infect first 10k hosts dominates infection timendash Solution Reconnaissance (stealthy scans etc)
bull Permutation scanningndash Observation Most scanning is redundantndash Idea Shared permutation of address space Start scanning
from own IP address Re-randomize when another infected machine is found
bull Internet-scale hit listsndash Flash worm complete infection within 30 seconds
89
Botnets
bull Bots Autonomous programs performing tasksbull Plenty of ldquobenignrdquo bots
ndash eg weatherbug
bull Botnets group of bots ndash Typically carries malicious connotationndash Large numbers of infected machinesndash Machines ldquoenlistedrdquo with infection vectors like worms
(last lecture)
bull Available for simultaneous control by a masterbull Size up to 350000 nodes (from todayrsquos paper)
90
ldquoRallyingrdquo the Botnet
bull Easy to combine worm backdoor functionalitybull Problem how to learn about successfully
infected machines
bull Optionsndash Emailndash Hard-coded email address
91
Botnet Control
bull Botnet master typically runs some IRC server on a well-known port (eg 6667)
bull Infected machine contacts botnet with pre-programmed DNS name (eg big-botde)
bull Dynamic DNS allows controller to move about freely
Infected Machine
DynamicDNS
BotnetController
(IRC server)
92
Some Defenses
93
Idea 1 Ingress Filtering
bull RFC 2827 Routers install filters to drop packets from networks that are not downstream
bull Feasible at edgesbull Difficult to configure closer to network ldquocorerdquo
20469207024 Internet
Drop all packets with source address other than 20469207024
94
Idea 2 uRPF Checks
bull Unicast Reverse Path Forwardingndash Cisco ldquoip verify unicast reverse-pathrdquo
bull Requires symmetric routing
Accept packet from interface only if forwarding table entry for source IP address matches ingress interface
100183
A10018
10015
101203
100181 24
10011 24
Strict Mode uRPF
Enabled
ldquoArdquo Routing TableDestination Next Hop1001024 Int 110018024 Int 2
100183 from wrong interface
95
Problems with uRPF
bull Asymmetric routing
96
S-BGP
bull Address-based PKI validate signaturesndash Authentication of
bull ownership for IP address blocks bull AS number bull an ASs identity and bull a BGP routers identity
ndash Use existing infrastructure (Internet registries etc)ndash Routing origination is digitally signedndash BGP updates are digitally signed
1048708 bull Route attestations A new optional BGP transitive path attribute
ndash carries digital signatures covering the routing information in updates
97
Practical Problems with S-BGP
bull Requires Public-Key Infrastructure
bull Lots of digital signatures to calculate and verifyndash Message overheadndash CPU overhead
bull Calculation expense is greatest when topology is changingndash Caching can help
bull Route aggregation is problematic (maybe thatrsquos OK)
bull Secure route withdrawals when link or node fails
bull Address ownership data out of date
bull Deployment
- Networking
- Goal of This Tutorial
- Todayrsquos Networks
- Business Models
- Billing for Internet Usage
- Net Neutrality
- Network Operations
- Point-of-Presence (PoP)
- Example Abilene Network Topology
- Another Example Backbone
- Internet Routing Overview
- Internet Routing Protocol BGP
- Two Flavors of BGP
- IPv4 Addresses Networks of Networks
- Pre-1994 Classful Addressing
- Classless Interdomain Routing (CIDR)
- Benefits of CIDR
- Growth of IP Prefixes
- 1994-1998 Linear Growth
- Around 2000 Fast Growth Resumes
- Fast growth resumes
- The Address Allocation Process
- Common Problems
- What can go wrong
- Measurement and Monitoring
- Passive vs Active Measurement
- Slide 27
- Passive Traffic Data Measurement
- Simple Network Management Protocol
- SNMP (Passive)
- Packet-level Monitoring
- Full Packet Capture (Passive)
- What is a flow
- Cisco NetFlow
- Flow Record Contents
- Aggregating Packets into Flows
- Reducing Measurement Overhead
- Packet Sampling for Flow Monitoring
- Sampling Flow-Level Sampling
- Two Main Approaches
- Packet Capture on High-Speed Links
- Characteristics of Packet Capture
- Data Measurement Repositories
- Multihoming and Traffic Engineering
- What is Multihoming
- Why Multihome
- Redundancy
- Performance
- Multihoming in IP Networks Today
- BGP or no
- Same Provider or Multiple
- Multihomed Stub One Link
- Multihomed Stub Multiple Links
- Multihomed Stub Multiple ISPs
- Multihomed Transit Network
- Interdomain Traffic Engineering
- Outbound Traffic Control
- Outbound Traffic Load Balancing
- Traffic Engineering Goals
- Managing Scale
- Achieving Predictability
- Challenges to Predictability
- Inter-AS Negotiation
- Outbound Multihoming Goals
- Inbound Traffic Control
- How many links are enough
- Problems with Multihoming in IPv4
- Slide 68
- Slide 69
- Diagnosis and Troubleshooting
- Operator Mailing List (NANOG)
- Operator Mailing List
- Routing Configuration
- Internet Business Model (Simplified)
- Peering Contracts Consistent Export
- Consistent Export
- Inconsistent Export in Practice
- Blackholes
- Security
- Security ldquoBogonrdquo Routes
- Spam Phishing etc
- Spam and Routing
- Worms and Botnets
- What is a Worm
- Example Worm Code Red
- Code Red Revisions
- Code Red Host Infection Rate
- Designing Fast-Spreading Worms
- Botnets
- ldquoRallyingrdquo the Botnet
- Botnet Control
- Some Defenses
- Idea 1 Ingress Filtering
- Idea 2 uRPF Checks
- Problems with uRPF
- S-BGP
- Practical Problems with S-BGP
-
74
Internet Business Model (Simplified)
bull CustomerProvider One AS pays another for reachability to some set of destinations
bull ldquoSettlement-freerdquo Peering Bartering Two ASes exchange routes with one another
Provider
Peer
Customer
Preferences implemented with local preference manipulation
Destination
Pay to use
Get paid to use
Free to use
75
Peering Contracts Consistent Export
bull Rules of settlement-free peeringndash Advertise routes at all peering pointsndash Advertised routes must have equal ldquoAS path lengthrdquo
Sprint
ATampT
Enables ldquohot potatordquo routing
ldquoequally goodrdquoroutes
76
Consistent Export
bull Malicedeceptionbull iBGP signaling partitionbull Inconsistent export policy
neighbor 10123route-map PEER permit 10 set prepend 123
neighbor 10456route-map PEER permit 10 set prepend 123 123
Possible Causes
10123 456 1
10456 456 2
Neighbor AS Export
1 1 123
Export Clause Prepend
2 1 123 123
Two different Export Policies
77
Inconsistent Export in Practice
Feamster et al ldquoBorderGuard Detecting Cold Potatoes from Peersrdquo ACM IMC October 2004
78
Blackholes
Date Thu 18 Jul 2002 060510 -0400 (EDT)From Chad Oleary ltcolpoboxcomgtSubject Re problems with 701To ltnanogmeritedugt
Were starting to see the same issues with UUNet again Anyone elseseeing this Trying to reach Qwest
traceroute to 631461901 (631461901) 30 hops max 38 byte packets 1 esc-lp2-gwe-solutionscorpcom (631182201) 1167 ms 1163 ms 1142 ms 2 500Serial2-10GW1TPA2ALTERNET (1571301499) 1097 ms 1059 ms 1044 ms 3 161at-1-0-0XL4ATL1ALTERNET (1526381190) 13839 ms 14108 ms 16638 ms 4 0so-3-1-0XL2ATL5ALTERNET (152630238) 14370 ms 14587 ms 14553 ms 5 POS7-0BR2ATL5ALTERNET (1526382193) 13928 ms 14099 ms 14053 ms 6 7 hellip
79
Security
80
Security ldquoBogonrdquo Routes
Feamster et al ldquoAn Empirical Study of lsquoBogonrsquo Route Advertisementsrdquo ACM CCR January 2005
81
Spam Phishing etc
bull Unsolicited commercial emailbull As of about August 2008 estimates indicate that
about 95 of all email is spambull Common spam filtering techniques
ndash Content-based filtersndash DNS Blacklist (DNSBL) lookups Significant fraction of
todayrsquos DNS traffic
Can IP addresses from which spam is received be spoofed
82
Spam and Routing
83
Worms and Botnets
84
What is a Worm
bull Code that replicates and propagates across the networkndash Often carries a ldquopayloadrdquo
bull Usually spread via exploiting flaws in open servicesndash ldquoVirusesrdquo require user action to spread
bull First worm Robert Morris November 1988ndash 6-10 of all Internet hosts infected ()
bull Many more since but none on that scale until July 2001
85
Example Worm Code Red
bull Initial version July 13 2001
bull Exploited known ISAPI vulnerability in Microsoft IIS Web servers
bull 1st through 20th of each month spread20th through end of each month attack
bull Payload Web site defacementbull Scanning Random IP addressesbull Bug failure to seed random number generator
86
Code Red Revisions
bull Released July 19 2001
bull Payload flooding attack on wwwwhitehousegovndash Attack was mounted at the IP address of the Web site
bull Bug died after 20th of each month
bull Random number generator for IP scanning fixed
87
Code Red Host Infection Rate
Exponential infection rate
Measured using backscatter technique
88
Designing Fast-Spreading Worms
bull Hit-list scanningndash Time to infect first 10k hosts dominates infection timendash Solution Reconnaissance (stealthy scans etc)
bull Permutation scanningndash Observation Most scanning is redundantndash Idea Shared permutation of address space Start scanning
from own IP address Re-randomize when another infected machine is found
bull Internet-scale hit listsndash Flash worm complete infection within 30 seconds
89
Botnets
bull Bots Autonomous programs performing tasksbull Plenty of ldquobenignrdquo bots
ndash eg weatherbug
bull Botnets group of bots ndash Typically carries malicious connotationndash Large numbers of infected machinesndash Machines ldquoenlistedrdquo with infection vectors like worms
(last lecture)
bull Available for simultaneous control by a masterbull Size up to 350000 nodes (from todayrsquos paper)
90
ldquoRallyingrdquo the Botnet
bull Easy to combine worm backdoor functionalitybull Problem how to learn about successfully
infected machines
bull Optionsndash Emailndash Hard-coded email address
91
Botnet Control
bull Botnet master typically runs some IRC server on a well-known port (eg 6667)
bull Infected machine contacts botnet with pre-programmed DNS name (eg big-botde)
bull Dynamic DNS allows controller to move about freely
Infected Machine
DynamicDNS
BotnetController
(IRC server)
92
Some Defenses
93
Idea 1 Ingress Filtering
bull RFC 2827 Routers install filters to drop packets from networks that are not downstream
bull Feasible at edgesbull Difficult to configure closer to network ldquocorerdquo
20469207024 Internet
Drop all packets with source address other than 20469207024
94
Idea 2 uRPF Checks
bull Unicast Reverse Path Forwardingndash Cisco ldquoip verify unicast reverse-pathrdquo
bull Requires symmetric routing
Accept packet from interface only if forwarding table entry for source IP address matches ingress interface
100183
A10018
10015
101203
100181 24
10011 24
Strict Mode uRPF
Enabled
ldquoArdquo Routing TableDestination Next Hop1001024 Int 110018024 Int 2
100183 from wrong interface
95
Problems with uRPF
bull Asymmetric routing
96
S-BGP
bull Address-based PKI validate signaturesndash Authentication of
bull ownership for IP address blocks bull AS number bull an ASs identity and bull a BGP routers identity
ndash Use existing infrastructure (Internet registries etc)ndash Routing origination is digitally signedndash BGP updates are digitally signed
1048708 bull Route attestations A new optional BGP transitive path attribute
ndash carries digital signatures covering the routing information in updates
97
Practical Problems with S-BGP
bull Requires Public-Key Infrastructure
bull Lots of digital signatures to calculate and verifyndash Message overheadndash CPU overhead
bull Calculation expense is greatest when topology is changingndash Caching can help
bull Route aggregation is problematic (maybe thatrsquos OK)
bull Secure route withdrawals when link or node fails
bull Address ownership data out of date
bull Deployment
- Networking
- Goal of This Tutorial
- Todayrsquos Networks
- Business Models
- Billing for Internet Usage
- Net Neutrality
- Network Operations
- Point-of-Presence (PoP)
- Example Abilene Network Topology
- Another Example Backbone
- Internet Routing Overview
- Internet Routing Protocol BGP
- Two Flavors of BGP
- IPv4 Addresses Networks of Networks
- Pre-1994 Classful Addressing
- Classless Interdomain Routing (CIDR)
- Benefits of CIDR
- Growth of IP Prefixes
- 1994-1998 Linear Growth
- Around 2000 Fast Growth Resumes
- Fast growth resumes
- The Address Allocation Process
- Common Problems
- What can go wrong
- Measurement and Monitoring
- Passive vs Active Measurement
- Slide 27
- Passive Traffic Data Measurement
- Simple Network Management Protocol
- SNMP (Passive)
- Packet-level Monitoring
- Full Packet Capture (Passive)
- What is a flow
- Cisco NetFlow
- Flow Record Contents
- Aggregating Packets into Flows
- Reducing Measurement Overhead
- Packet Sampling for Flow Monitoring
- Sampling Flow-Level Sampling
- Two Main Approaches
- Packet Capture on High-Speed Links
- Characteristics of Packet Capture
- Data Measurement Repositories
- Multihoming and Traffic Engineering
- What is Multihoming
- Why Multihome
- Redundancy
- Performance
- Multihoming in IP Networks Today
- BGP or no
- Same Provider or Multiple
- Multihomed Stub One Link
- Multihomed Stub Multiple Links
- Multihomed Stub Multiple ISPs
- Multihomed Transit Network
- Interdomain Traffic Engineering
- Outbound Traffic Control
- Outbound Traffic Load Balancing
- Traffic Engineering Goals
- Managing Scale
- Achieving Predictability
- Challenges to Predictability
- Inter-AS Negotiation
- Outbound Multihoming Goals
- Inbound Traffic Control
- How many links are enough
- Problems with Multihoming in IPv4
- Slide 68
- Slide 69
- Diagnosis and Troubleshooting
- Operator Mailing List (NANOG)
- Operator Mailing List
- Routing Configuration
- Internet Business Model (Simplified)
- Peering Contracts Consistent Export
- Consistent Export
- Inconsistent Export in Practice
- Blackholes
- Security
- Security ldquoBogonrdquo Routes
- Spam Phishing etc
- Spam and Routing
- Worms and Botnets
- What is a Worm
- Example Worm Code Red
- Code Red Revisions
- Code Red Host Infection Rate
- Designing Fast-Spreading Worms
- Botnets
- ldquoRallyingrdquo the Botnet
- Botnet Control
- Some Defenses
- Idea 1 Ingress Filtering
- Idea 2 uRPF Checks
- Problems with uRPF
- S-BGP
- Practical Problems with S-BGP
-
75
Peering Contracts Consistent Export
bull Rules of settlement-free peeringndash Advertise routes at all peering pointsndash Advertised routes must have equal ldquoAS path lengthrdquo
Sprint
ATampT
Enables ldquohot potatordquo routing
ldquoequally goodrdquoroutes
76
Consistent Export
bull Malicedeceptionbull iBGP signaling partitionbull Inconsistent export policy
neighbor 10123route-map PEER permit 10 set prepend 123
neighbor 10456route-map PEER permit 10 set prepend 123 123
Possible Causes
10123 456 1
10456 456 2
Neighbor AS Export
1 1 123
Export Clause Prepend
2 1 123 123
Two different Export Policies
77
Inconsistent Export in Practice
Feamster et al ldquoBorderGuard Detecting Cold Potatoes from Peersrdquo ACM IMC October 2004
78
Blackholes
Date Thu 18 Jul 2002 060510 -0400 (EDT)From Chad Oleary ltcolpoboxcomgtSubject Re problems with 701To ltnanogmeritedugt
Were starting to see the same issues with UUNet again Anyone elseseeing this Trying to reach Qwest
traceroute to 631461901 (631461901) 30 hops max 38 byte packets 1 esc-lp2-gwe-solutionscorpcom (631182201) 1167 ms 1163 ms 1142 ms 2 500Serial2-10GW1TPA2ALTERNET (1571301499) 1097 ms 1059 ms 1044 ms 3 161at-1-0-0XL4ATL1ALTERNET (1526381190) 13839 ms 14108 ms 16638 ms 4 0so-3-1-0XL2ATL5ALTERNET (152630238) 14370 ms 14587 ms 14553 ms 5 POS7-0BR2ATL5ALTERNET (1526382193) 13928 ms 14099 ms 14053 ms 6 7 hellip
79
Security
80
Security ldquoBogonrdquo Routes
Feamster et al ldquoAn Empirical Study of lsquoBogonrsquo Route Advertisementsrdquo ACM CCR January 2005
81
Spam Phishing etc
bull Unsolicited commercial emailbull As of about August 2008 estimates indicate that
about 95 of all email is spambull Common spam filtering techniques
ndash Content-based filtersndash DNS Blacklist (DNSBL) lookups Significant fraction of
todayrsquos DNS traffic
Can IP addresses from which spam is received be spoofed
82
Spam and Routing
83
Worms and Botnets
84
What is a Worm
bull Code that replicates and propagates across the networkndash Often carries a ldquopayloadrdquo
bull Usually spread via exploiting flaws in open servicesndash ldquoVirusesrdquo require user action to spread
bull First worm Robert Morris November 1988ndash 6-10 of all Internet hosts infected ()
bull Many more since but none on that scale until July 2001
85
Example Worm Code Red
bull Initial version July 13 2001
bull Exploited known ISAPI vulnerability in Microsoft IIS Web servers
bull 1st through 20th of each month spread20th through end of each month attack
bull Payload Web site defacementbull Scanning Random IP addressesbull Bug failure to seed random number generator
86
Code Red Revisions
bull Released July 19 2001
bull Payload flooding attack on wwwwhitehousegovndash Attack was mounted at the IP address of the Web site
bull Bug died after 20th of each month
bull Random number generator for IP scanning fixed
87
Code Red Host Infection Rate
Exponential infection rate
Measured using backscatter technique
88
Designing Fast-Spreading Worms
bull Hit-list scanningndash Time to infect first 10k hosts dominates infection timendash Solution Reconnaissance (stealthy scans etc)
bull Permutation scanningndash Observation Most scanning is redundantndash Idea Shared permutation of address space Start scanning
from own IP address Re-randomize when another infected machine is found
bull Internet-scale hit listsndash Flash worm complete infection within 30 seconds
89
Botnets
bull Bots Autonomous programs performing tasksbull Plenty of ldquobenignrdquo bots
ndash eg weatherbug
bull Botnets group of bots ndash Typically carries malicious connotationndash Large numbers of infected machinesndash Machines ldquoenlistedrdquo with infection vectors like worms
(last lecture)
bull Available for simultaneous control by a masterbull Size up to 350000 nodes (from todayrsquos paper)
90
ldquoRallyingrdquo the Botnet
bull Easy to combine worm backdoor functionalitybull Problem how to learn about successfully
infected machines
bull Optionsndash Emailndash Hard-coded email address
91
Botnet Control
bull Botnet master typically runs some IRC server on a well-known port (eg 6667)
bull Infected machine contacts botnet with pre-programmed DNS name (eg big-botde)
bull Dynamic DNS allows controller to move about freely
Infected Machine
DynamicDNS
BotnetController
(IRC server)
92
Some Defenses
93
Idea 1 Ingress Filtering
bull RFC 2827 Routers install filters to drop packets from networks that are not downstream
bull Feasible at edgesbull Difficult to configure closer to network ldquocorerdquo
20469207024 Internet
Drop all packets with source address other than 20469207024
94
Idea 2 uRPF Checks
bull Unicast Reverse Path Forwardingndash Cisco ldquoip verify unicast reverse-pathrdquo
bull Requires symmetric routing
Accept packet from interface only if forwarding table entry for source IP address matches ingress interface
100183
A10018
10015
101203
100181 24
10011 24
Strict Mode uRPF
Enabled
ldquoArdquo Routing TableDestination Next Hop1001024 Int 110018024 Int 2
100183 from wrong interface
95
Problems with uRPF
bull Asymmetric routing
96
S-BGP
bull Address-based PKI validate signaturesndash Authentication of
bull ownership for IP address blocks bull AS number bull an ASs identity and bull a BGP routers identity
ndash Use existing infrastructure (Internet registries etc)ndash Routing origination is digitally signedndash BGP updates are digitally signed
1048708 bull Route attestations A new optional BGP transitive path attribute
ndash carries digital signatures covering the routing information in updates
97
Practical Problems with S-BGP
bull Requires Public-Key Infrastructure
bull Lots of digital signatures to calculate and verifyndash Message overheadndash CPU overhead
bull Calculation expense is greatest when topology is changingndash Caching can help
bull Route aggregation is problematic (maybe thatrsquos OK)
bull Secure route withdrawals when link or node fails
bull Address ownership data out of date
bull Deployment
- Networking
- Goal of This Tutorial
- Todayrsquos Networks
- Business Models
- Billing for Internet Usage
- Net Neutrality
- Network Operations
- Point-of-Presence (PoP)
- Example Abilene Network Topology
- Another Example Backbone
- Internet Routing Overview
- Internet Routing Protocol BGP
- Two Flavors of BGP
- IPv4 Addresses Networks of Networks
- Pre-1994 Classful Addressing
- Classless Interdomain Routing (CIDR)
- Benefits of CIDR
- Growth of IP Prefixes
- 1994-1998 Linear Growth
- Around 2000 Fast Growth Resumes
- Fast growth resumes
- The Address Allocation Process
- Common Problems
- What can go wrong
- Measurement and Monitoring
- Passive vs Active Measurement
- Slide 27
- Passive Traffic Data Measurement
- Simple Network Management Protocol
- SNMP (Passive)
- Packet-level Monitoring
- Full Packet Capture (Passive)
- What is a flow
- Cisco NetFlow
- Flow Record Contents
- Aggregating Packets into Flows
- Reducing Measurement Overhead
- Packet Sampling for Flow Monitoring
- Sampling Flow-Level Sampling
- Two Main Approaches
- Packet Capture on High-Speed Links
- Characteristics of Packet Capture
- Data Measurement Repositories
- Multihoming and Traffic Engineering
- What is Multihoming
- Why Multihome
- Redundancy
- Performance
- Multihoming in IP Networks Today
- BGP or no
- Same Provider or Multiple
- Multihomed Stub One Link
- Multihomed Stub Multiple Links
- Multihomed Stub Multiple ISPs
- Multihomed Transit Network
- Interdomain Traffic Engineering
- Outbound Traffic Control
- Outbound Traffic Load Balancing
- Traffic Engineering Goals
- Managing Scale
- Achieving Predictability
- Challenges to Predictability
- Inter-AS Negotiation
- Outbound Multihoming Goals
- Inbound Traffic Control
- How many links are enough
- Problems with Multihoming in IPv4
- Slide 68
- Slide 69
- Diagnosis and Troubleshooting
- Operator Mailing List (NANOG)
- Operator Mailing List
- Routing Configuration
- Internet Business Model (Simplified)
- Peering Contracts Consistent Export
- Consistent Export
- Inconsistent Export in Practice
- Blackholes
- Security
- Security ldquoBogonrdquo Routes
- Spam Phishing etc
- Spam and Routing
- Worms and Botnets
- What is a Worm
- Example Worm Code Red
- Code Red Revisions
- Code Red Host Infection Rate
- Designing Fast-Spreading Worms
- Botnets
- ldquoRallyingrdquo the Botnet
- Botnet Control
- Some Defenses
- Idea 1 Ingress Filtering
- Idea 2 uRPF Checks
- Problems with uRPF
- S-BGP
- Practical Problems with S-BGP
-
76
Consistent Export
bull Malicedeceptionbull iBGP signaling partitionbull Inconsistent export policy
neighbor 10123route-map PEER permit 10 set prepend 123
neighbor 10456route-map PEER permit 10 set prepend 123 123
Possible Causes
10123 456 1
10456 456 2
Neighbor AS Export
1 1 123
Export Clause Prepend
2 1 123 123
Two different Export Policies
77
Inconsistent Export in Practice
Feamster et al ldquoBorderGuard Detecting Cold Potatoes from Peersrdquo ACM IMC October 2004
78
Blackholes
Date Thu 18 Jul 2002 060510 -0400 (EDT)From Chad Oleary ltcolpoboxcomgtSubject Re problems with 701To ltnanogmeritedugt
Were starting to see the same issues with UUNet again Anyone elseseeing this Trying to reach Qwest
traceroute to 631461901 (631461901) 30 hops max 38 byte packets 1 esc-lp2-gwe-solutionscorpcom (631182201) 1167 ms 1163 ms 1142 ms 2 500Serial2-10GW1TPA2ALTERNET (1571301499) 1097 ms 1059 ms 1044 ms 3 161at-1-0-0XL4ATL1ALTERNET (1526381190) 13839 ms 14108 ms 16638 ms 4 0so-3-1-0XL2ATL5ALTERNET (152630238) 14370 ms 14587 ms 14553 ms 5 POS7-0BR2ATL5ALTERNET (1526382193) 13928 ms 14099 ms 14053 ms 6 7 hellip
79
Security
80
Security ldquoBogonrdquo Routes
Feamster et al ldquoAn Empirical Study of lsquoBogonrsquo Route Advertisementsrdquo ACM CCR January 2005
81
Spam Phishing etc
bull Unsolicited commercial emailbull As of about August 2008 estimates indicate that
about 95 of all email is spambull Common spam filtering techniques
ndash Content-based filtersndash DNS Blacklist (DNSBL) lookups Significant fraction of
todayrsquos DNS traffic
Can IP addresses from which spam is received be spoofed
82
Spam and Routing
83
Worms and Botnets
84
What is a Worm
bull Code that replicates and propagates across the networkndash Often carries a ldquopayloadrdquo
bull Usually spread via exploiting flaws in open servicesndash ldquoVirusesrdquo require user action to spread
bull First worm Robert Morris November 1988ndash 6-10 of all Internet hosts infected ()
bull Many more since but none on that scale until July 2001
85
Example Worm Code Red
bull Initial version July 13 2001
bull Exploited known ISAPI vulnerability in Microsoft IIS Web servers
bull 1st through 20th of each month spread20th through end of each month attack
bull Payload Web site defacementbull Scanning Random IP addressesbull Bug failure to seed random number generator
86
Code Red Revisions
bull Released July 19 2001
bull Payload flooding attack on wwwwhitehousegovndash Attack was mounted at the IP address of the Web site
bull Bug died after 20th of each month
bull Random number generator for IP scanning fixed
87
Code Red Host Infection Rate
Exponential infection rate
Measured using backscatter technique
88
Designing Fast-Spreading Worms
bull Hit-list scanningndash Time to infect first 10k hosts dominates infection timendash Solution Reconnaissance (stealthy scans etc)
bull Permutation scanningndash Observation Most scanning is redundantndash Idea Shared permutation of address space Start scanning
from own IP address Re-randomize when another infected machine is found
bull Internet-scale hit listsndash Flash worm complete infection within 30 seconds
89
Botnets
bull Bots Autonomous programs performing tasksbull Plenty of ldquobenignrdquo bots
ndash eg weatherbug
bull Botnets group of bots ndash Typically carries malicious connotationndash Large numbers of infected machinesndash Machines ldquoenlistedrdquo with infection vectors like worms
(last lecture)
bull Available for simultaneous control by a masterbull Size up to 350000 nodes (from todayrsquos paper)
90
ldquoRallyingrdquo the Botnet
bull Easy to combine worm backdoor functionalitybull Problem how to learn about successfully
infected machines
bull Optionsndash Emailndash Hard-coded email address
91
Botnet Control
bull Botnet master typically runs some IRC server on a well-known port (eg 6667)
bull Infected machine contacts botnet with pre-programmed DNS name (eg big-botde)
bull Dynamic DNS allows controller to move about freely
Infected Machine
DynamicDNS
BotnetController
(IRC server)
92
Some Defenses
93
Idea 1 Ingress Filtering
bull RFC 2827 Routers install filters to drop packets from networks that are not downstream
bull Feasible at edgesbull Difficult to configure closer to network ldquocorerdquo
20469207024 Internet
Drop all packets with source address other than 20469207024
94
Idea 2 uRPF Checks
bull Unicast Reverse Path Forwardingndash Cisco ldquoip verify unicast reverse-pathrdquo
bull Requires symmetric routing
Accept packet from interface only if forwarding table entry for source IP address matches ingress interface
100183
A10018
10015
101203
100181 24
10011 24
Strict Mode uRPF
Enabled
ldquoArdquo Routing TableDestination Next Hop1001024 Int 110018024 Int 2
100183 from wrong interface
95
Problems with uRPF
bull Asymmetric routing
96
S-BGP
bull Address-based PKI validate signaturesndash Authentication of
bull ownership for IP address blocks bull AS number bull an ASs identity and bull a BGP routers identity
ndash Use existing infrastructure (Internet registries etc)ndash Routing origination is digitally signedndash BGP updates are digitally signed
1048708 bull Route attestations A new optional BGP transitive path attribute
ndash carries digital signatures covering the routing information in updates
97
Practical Problems with S-BGP
bull Requires Public-Key Infrastructure
bull Lots of digital signatures to calculate and verifyndash Message overheadndash CPU overhead
bull Calculation expense is greatest when topology is changingndash Caching can help
bull Route aggregation is problematic (maybe thatrsquos OK)
bull Secure route withdrawals when link or node fails
bull Address ownership data out of date
bull Deployment
- Networking
- Goal of This Tutorial
- Todayrsquos Networks
- Business Models
- Billing for Internet Usage
- Net Neutrality
- Network Operations
- Point-of-Presence (PoP)
- Example Abilene Network Topology
- Another Example Backbone
- Internet Routing Overview
- Internet Routing Protocol BGP
- Two Flavors of BGP
- IPv4 Addresses Networks of Networks
- Pre-1994 Classful Addressing
- Classless Interdomain Routing (CIDR)
- Benefits of CIDR
- Growth of IP Prefixes
- 1994-1998 Linear Growth
- Around 2000 Fast Growth Resumes
- Fast growth resumes
- The Address Allocation Process
- Common Problems
- What can go wrong
- Measurement and Monitoring
- Passive vs Active Measurement
- Slide 27
- Passive Traffic Data Measurement
- Simple Network Management Protocol
- SNMP (Passive)
- Packet-level Monitoring
- Full Packet Capture (Passive)
- What is a flow
- Cisco NetFlow
- Flow Record Contents
- Aggregating Packets into Flows
- Reducing Measurement Overhead
- Packet Sampling for Flow Monitoring
- Sampling Flow-Level Sampling
- Two Main Approaches
- Packet Capture on High-Speed Links
- Characteristics of Packet Capture
- Data Measurement Repositories
- Multihoming and Traffic Engineering
- What is Multihoming
- Why Multihome
- Redundancy
- Performance
- Multihoming in IP Networks Today
- BGP or no
- Same Provider or Multiple
- Multihomed Stub One Link
- Multihomed Stub Multiple Links
- Multihomed Stub Multiple ISPs
- Multihomed Transit Network
- Interdomain Traffic Engineering
- Outbound Traffic Control
- Outbound Traffic Load Balancing
- Traffic Engineering Goals
- Managing Scale
- Achieving Predictability
- Challenges to Predictability
- Inter-AS Negotiation
- Outbound Multihoming Goals
- Inbound Traffic Control
- How many links are enough
- Problems with Multihoming in IPv4
- Slide 68
- Slide 69
- Diagnosis and Troubleshooting
- Operator Mailing List (NANOG)
- Operator Mailing List
- Routing Configuration
- Internet Business Model (Simplified)
- Peering Contracts Consistent Export
- Consistent Export
- Inconsistent Export in Practice
- Blackholes
- Security
- Security ldquoBogonrdquo Routes
- Spam Phishing etc
- Spam and Routing
- Worms and Botnets
- What is a Worm
- Example Worm Code Red
- Code Red Revisions
- Code Red Host Infection Rate
- Designing Fast-Spreading Worms
- Botnets
- ldquoRallyingrdquo the Botnet
- Botnet Control
- Some Defenses
- Idea 1 Ingress Filtering
- Idea 2 uRPF Checks
- Problems with uRPF
- S-BGP
- Practical Problems with S-BGP
-
77
Inconsistent Export in Practice
Feamster et al ldquoBorderGuard Detecting Cold Potatoes from Peersrdquo ACM IMC October 2004
78
Blackholes
Date Thu 18 Jul 2002 060510 -0400 (EDT)From Chad Oleary ltcolpoboxcomgtSubject Re problems with 701To ltnanogmeritedugt
Were starting to see the same issues with UUNet again Anyone elseseeing this Trying to reach Qwest
traceroute to 631461901 (631461901) 30 hops max 38 byte packets 1 esc-lp2-gwe-solutionscorpcom (631182201) 1167 ms 1163 ms 1142 ms 2 500Serial2-10GW1TPA2ALTERNET (1571301499) 1097 ms 1059 ms 1044 ms 3 161at-1-0-0XL4ATL1ALTERNET (1526381190) 13839 ms 14108 ms 16638 ms 4 0so-3-1-0XL2ATL5ALTERNET (152630238) 14370 ms 14587 ms 14553 ms 5 POS7-0BR2ATL5ALTERNET (1526382193) 13928 ms 14099 ms 14053 ms 6 7 hellip
79
Security
80
Security ldquoBogonrdquo Routes
Feamster et al ldquoAn Empirical Study of lsquoBogonrsquo Route Advertisementsrdquo ACM CCR January 2005
81
Spam Phishing etc
bull Unsolicited commercial emailbull As of about August 2008 estimates indicate that
about 95 of all email is spambull Common spam filtering techniques
ndash Content-based filtersndash DNS Blacklist (DNSBL) lookups Significant fraction of
todayrsquos DNS traffic
Can IP addresses from which spam is received be spoofed
82
Spam and Routing
83
Worms and Botnets
84
What is a Worm
bull Code that replicates and propagates across the networkndash Often carries a ldquopayloadrdquo
bull Usually spread via exploiting flaws in open servicesndash ldquoVirusesrdquo require user action to spread
bull First worm Robert Morris November 1988ndash 6-10 of all Internet hosts infected ()
bull Many more since but none on that scale until July 2001
85
Example Worm Code Red
bull Initial version July 13 2001
bull Exploited known ISAPI vulnerability in Microsoft IIS Web servers
bull 1st through 20th of each month spread20th through end of each month attack
bull Payload Web site defacementbull Scanning Random IP addressesbull Bug failure to seed random number generator
86
Code Red Revisions
bull Released July 19 2001
bull Payload flooding attack on wwwwhitehousegovndash Attack was mounted at the IP address of the Web site
bull Bug died after 20th of each month
bull Random number generator for IP scanning fixed
87
Code Red Host Infection Rate
Exponential infection rate
Measured using backscatter technique
88
Designing Fast-Spreading Worms
bull Hit-list scanningndash Time to infect first 10k hosts dominates infection timendash Solution Reconnaissance (stealthy scans etc)
bull Permutation scanningndash Observation Most scanning is redundantndash Idea Shared permutation of address space Start scanning
from own IP address Re-randomize when another infected machine is found
bull Internet-scale hit listsndash Flash worm complete infection within 30 seconds
89
Botnets
bull Bots Autonomous programs performing tasksbull Plenty of ldquobenignrdquo bots
ndash eg weatherbug
bull Botnets group of bots ndash Typically carries malicious connotationndash Large numbers of infected machinesndash Machines ldquoenlistedrdquo with infection vectors like worms
(last lecture)
bull Available for simultaneous control by a masterbull Size up to 350000 nodes (from todayrsquos paper)
90
ldquoRallyingrdquo the Botnet
bull Easy to combine worm backdoor functionalitybull Problem how to learn about successfully
infected machines
bull Optionsndash Emailndash Hard-coded email address
91
Botnet Control
bull Botnet master typically runs some IRC server on a well-known port (eg 6667)
bull Infected machine contacts botnet with pre-programmed DNS name (eg big-botde)
bull Dynamic DNS allows controller to move about freely
Infected Machine
DynamicDNS
BotnetController
(IRC server)
92
Some Defenses
93
Idea 1 Ingress Filtering
bull RFC 2827 Routers install filters to drop packets from networks that are not downstream
bull Feasible at edgesbull Difficult to configure closer to network ldquocorerdquo
20469207024 Internet
Drop all packets with source address other than 20469207024
94
Idea 2 uRPF Checks
bull Unicast Reverse Path Forwardingndash Cisco ldquoip verify unicast reverse-pathrdquo
bull Requires symmetric routing
Accept packet from interface only if forwarding table entry for source IP address matches ingress interface
100183
A10018
10015
101203
100181 24
10011 24
Strict Mode uRPF
Enabled
ldquoArdquo Routing TableDestination Next Hop1001024 Int 110018024 Int 2
100183 from wrong interface
95
Problems with uRPF
bull Asymmetric routing
96
S-BGP
bull Address-based PKI validate signaturesndash Authentication of
bull ownership for IP address blocks bull AS number bull an ASs identity and bull a BGP routers identity
ndash Use existing infrastructure (Internet registries etc)ndash Routing origination is digitally signedndash BGP updates are digitally signed
1048708 bull Route attestations A new optional BGP transitive path attribute
ndash carries digital signatures covering the routing information in updates
97
Practical Problems with S-BGP
bull Requires Public-Key Infrastructure
bull Lots of digital signatures to calculate and verifyndash Message overheadndash CPU overhead
bull Calculation expense is greatest when topology is changingndash Caching can help
bull Route aggregation is problematic (maybe thatrsquos OK)
bull Secure route withdrawals when link or node fails
bull Address ownership data out of date
bull Deployment
- Networking
- Goal of This Tutorial
- Todayrsquos Networks
- Business Models
- Billing for Internet Usage
- Net Neutrality
- Network Operations
- Point-of-Presence (PoP)
- Example Abilene Network Topology
- Another Example Backbone
- Internet Routing Overview
- Internet Routing Protocol BGP
- Two Flavors of BGP
- IPv4 Addresses Networks of Networks
- Pre-1994 Classful Addressing
- Classless Interdomain Routing (CIDR)
- Benefits of CIDR
- Growth of IP Prefixes
- 1994-1998 Linear Growth
- Around 2000 Fast Growth Resumes
- Fast growth resumes
- The Address Allocation Process
- Common Problems
- What can go wrong
- Measurement and Monitoring
- Passive vs Active Measurement
- Slide 27
- Passive Traffic Data Measurement
- Simple Network Management Protocol
- SNMP (Passive)
- Packet-level Monitoring
- Full Packet Capture (Passive)
- What is a flow
- Cisco NetFlow
- Flow Record Contents
- Aggregating Packets into Flows
- Reducing Measurement Overhead
- Packet Sampling for Flow Monitoring
- Sampling Flow-Level Sampling
- Two Main Approaches
- Packet Capture on High-Speed Links
- Characteristics of Packet Capture
- Data Measurement Repositories
- Multihoming and Traffic Engineering
- What is Multihoming
- Why Multihome
- Redundancy
- Performance
- Multihoming in IP Networks Today
- BGP or no
- Same Provider or Multiple
- Multihomed Stub One Link
- Multihomed Stub Multiple Links
- Multihomed Stub Multiple ISPs
- Multihomed Transit Network
- Interdomain Traffic Engineering
- Outbound Traffic Control
- Outbound Traffic Load Balancing
- Traffic Engineering Goals
- Managing Scale
- Achieving Predictability
- Challenges to Predictability
- Inter-AS Negotiation
- Outbound Multihoming Goals
- Inbound Traffic Control
- How many links are enough
- Problems with Multihoming in IPv4
- Slide 68
- Slide 69
- Diagnosis and Troubleshooting
- Operator Mailing List (NANOG)
- Operator Mailing List
- Routing Configuration
- Internet Business Model (Simplified)
- Peering Contracts Consistent Export
- Consistent Export
- Inconsistent Export in Practice
- Blackholes
- Security
- Security ldquoBogonrdquo Routes
- Spam Phishing etc
- Spam and Routing
- Worms and Botnets
- What is a Worm
- Example Worm Code Red
- Code Red Revisions
- Code Red Host Infection Rate
- Designing Fast-Spreading Worms
- Botnets
- ldquoRallyingrdquo the Botnet
- Botnet Control
- Some Defenses
- Idea 1 Ingress Filtering
- Idea 2 uRPF Checks
- Problems with uRPF
- S-BGP
- Practical Problems with S-BGP
-
78
Blackholes
Date Thu 18 Jul 2002 060510 -0400 (EDT)From Chad Oleary ltcolpoboxcomgtSubject Re problems with 701To ltnanogmeritedugt
Were starting to see the same issues with UUNet again Anyone elseseeing this Trying to reach Qwest
traceroute to 631461901 (631461901) 30 hops max 38 byte packets 1 esc-lp2-gwe-solutionscorpcom (631182201) 1167 ms 1163 ms 1142 ms 2 500Serial2-10GW1TPA2ALTERNET (1571301499) 1097 ms 1059 ms 1044 ms 3 161at-1-0-0XL4ATL1ALTERNET (1526381190) 13839 ms 14108 ms 16638 ms 4 0so-3-1-0XL2ATL5ALTERNET (152630238) 14370 ms 14587 ms 14553 ms 5 POS7-0BR2ATL5ALTERNET (1526382193) 13928 ms 14099 ms 14053 ms 6 7 hellip
79
Security
80
Security ldquoBogonrdquo Routes
Feamster et al ldquoAn Empirical Study of lsquoBogonrsquo Route Advertisementsrdquo ACM CCR January 2005
81
Spam Phishing etc
bull Unsolicited commercial emailbull As of about August 2008 estimates indicate that
about 95 of all email is spambull Common spam filtering techniques
ndash Content-based filtersndash DNS Blacklist (DNSBL) lookups Significant fraction of
todayrsquos DNS traffic
Can IP addresses from which spam is received be spoofed
82
Spam and Routing
83
Worms and Botnets
84
What is a Worm
bull Code that replicates and propagates across the networkndash Often carries a ldquopayloadrdquo
bull Usually spread via exploiting flaws in open servicesndash ldquoVirusesrdquo require user action to spread
bull First worm Robert Morris November 1988ndash 6-10 of all Internet hosts infected ()
bull Many more since but none on that scale until July 2001
85
Example Worm Code Red
bull Initial version July 13 2001
bull Exploited known ISAPI vulnerability in Microsoft IIS Web servers
bull 1st through 20th of each month spread20th through end of each month attack
bull Payload Web site defacementbull Scanning Random IP addressesbull Bug failure to seed random number generator
86
Code Red Revisions
bull Released July 19 2001
bull Payload flooding attack on wwwwhitehousegovndash Attack was mounted at the IP address of the Web site
bull Bug died after 20th of each month
bull Random number generator for IP scanning fixed
87
Code Red Host Infection Rate
Exponential infection rate
Measured using backscatter technique
88
Designing Fast-Spreading Worms
bull Hit-list scanningndash Time to infect first 10k hosts dominates infection timendash Solution Reconnaissance (stealthy scans etc)
bull Permutation scanningndash Observation Most scanning is redundantndash Idea Shared permutation of address space Start scanning
from own IP address Re-randomize when another infected machine is found
bull Internet-scale hit listsndash Flash worm complete infection within 30 seconds
89
Botnets
bull Bots Autonomous programs performing tasksbull Plenty of ldquobenignrdquo bots
ndash eg weatherbug
bull Botnets group of bots ndash Typically carries malicious connotationndash Large numbers of infected machinesndash Machines ldquoenlistedrdquo with infection vectors like worms
(last lecture)
bull Available for simultaneous control by a masterbull Size up to 350000 nodes (from todayrsquos paper)
90
ldquoRallyingrdquo the Botnet
bull Easy to combine worm backdoor functionalitybull Problem how to learn about successfully
infected machines
bull Optionsndash Emailndash Hard-coded email address
91
Botnet Control
bull Botnet master typically runs some IRC server on a well-known port (eg 6667)
bull Infected machine contacts botnet with pre-programmed DNS name (eg big-botde)
bull Dynamic DNS allows controller to move about freely
Infected Machine
DynamicDNS
BotnetController
(IRC server)
92
Some Defenses
93
Idea 1 Ingress Filtering
bull RFC 2827 Routers install filters to drop packets from networks that are not downstream
bull Feasible at edgesbull Difficult to configure closer to network ldquocorerdquo
20469207024 Internet
Drop all packets with source address other than 20469207024
94
Idea 2 uRPF Checks
bull Unicast Reverse Path Forwardingndash Cisco ldquoip verify unicast reverse-pathrdquo
bull Requires symmetric routing
Accept packet from interface only if forwarding table entry for source IP address matches ingress interface
100183
A10018
10015
101203
100181 24
10011 24
Strict Mode uRPF
Enabled
ldquoArdquo Routing TableDestination Next Hop1001024 Int 110018024 Int 2
100183 from wrong interface
95
Problems with uRPF
bull Asymmetric routing
96
S-BGP
bull Address-based PKI validate signaturesndash Authentication of
bull ownership for IP address blocks bull AS number bull an ASs identity and bull a BGP routers identity
ndash Use existing infrastructure (Internet registries etc)ndash Routing origination is digitally signedndash BGP updates are digitally signed
1048708 bull Route attestations A new optional BGP transitive path attribute
ndash carries digital signatures covering the routing information in updates
97
Practical Problems with S-BGP
bull Requires Public-Key Infrastructure
bull Lots of digital signatures to calculate and verifyndash Message overheadndash CPU overhead
bull Calculation expense is greatest when topology is changingndash Caching can help
bull Route aggregation is problematic (maybe thatrsquos OK)
bull Secure route withdrawals when link or node fails
bull Address ownership data out of date
bull Deployment
- Networking
- Goal of This Tutorial
- Todayrsquos Networks
- Business Models
- Billing for Internet Usage
- Net Neutrality
- Network Operations
- Point-of-Presence (PoP)
- Example Abilene Network Topology
- Another Example Backbone
- Internet Routing Overview
- Internet Routing Protocol BGP
- Two Flavors of BGP
- IPv4 Addresses Networks of Networks
- Pre-1994 Classful Addressing
- Classless Interdomain Routing (CIDR)
- Benefits of CIDR
- Growth of IP Prefixes
- 1994-1998 Linear Growth
- Around 2000 Fast Growth Resumes
- Fast growth resumes
- The Address Allocation Process
- Common Problems
- What can go wrong
- Measurement and Monitoring
- Passive vs Active Measurement
- Slide 27
- Passive Traffic Data Measurement
- Simple Network Management Protocol
- SNMP (Passive)
- Packet-level Monitoring
- Full Packet Capture (Passive)
- What is a flow
- Cisco NetFlow
- Flow Record Contents
- Aggregating Packets into Flows
- Reducing Measurement Overhead
- Packet Sampling for Flow Monitoring
- Sampling Flow-Level Sampling
- Two Main Approaches
- Packet Capture on High-Speed Links
- Characteristics of Packet Capture
- Data Measurement Repositories
- Multihoming and Traffic Engineering
- What is Multihoming
- Why Multihome
- Redundancy
- Performance
- Multihoming in IP Networks Today
- BGP or no
- Same Provider or Multiple
- Multihomed Stub One Link
- Multihomed Stub Multiple Links
- Multihomed Stub Multiple ISPs
- Multihomed Transit Network
- Interdomain Traffic Engineering
- Outbound Traffic Control
- Outbound Traffic Load Balancing
- Traffic Engineering Goals
- Managing Scale
- Achieving Predictability
- Challenges to Predictability
- Inter-AS Negotiation
- Outbound Multihoming Goals
- Inbound Traffic Control
- How many links are enough
- Problems with Multihoming in IPv4
- Slide 68
- Slide 69
- Diagnosis and Troubleshooting
- Operator Mailing List (NANOG)
- Operator Mailing List
- Routing Configuration
- Internet Business Model (Simplified)
- Peering Contracts Consistent Export
- Consistent Export
- Inconsistent Export in Practice
- Blackholes
- Security
- Security ldquoBogonrdquo Routes
- Spam Phishing etc
- Spam and Routing
- Worms and Botnets
- What is a Worm
- Example Worm Code Red
- Code Red Revisions
- Code Red Host Infection Rate
- Designing Fast-Spreading Worms
- Botnets
- ldquoRallyingrdquo the Botnet
- Botnet Control
- Some Defenses
- Idea 1 Ingress Filtering
- Idea 2 uRPF Checks
- Problems with uRPF
- S-BGP
- Practical Problems with S-BGP
-
79
Security
80
Security ldquoBogonrdquo Routes
Feamster et al ldquoAn Empirical Study of lsquoBogonrsquo Route Advertisementsrdquo ACM CCR January 2005
81
Spam Phishing etc
bull Unsolicited commercial emailbull As of about August 2008 estimates indicate that
about 95 of all email is spambull Common spam filtering techniques
ndash Content-based filtersndash DNS Blacklist (DNSBL) lookups Significant fraction of
todayrsquos DNS traffic
Can IP addresses from which spam is received be spoofed
82
Spam and Routing
83
Worms and Botnets
84
What is a Worm
bull Code that replicates and propagates across the networkndash Often carries a ldquopayloadrdquo
bull Usually spread via exploiting flaws in open servicesndash ldquoVirusesrdquo require user action to spread
bull First worm Robert Morris November 1988ndash 6-10 of all Internet hosts infected ()
bull Many more since but none on that scale until July 2001
85
Example Worm Code Red
bull Initial version July 13 2001
bull Exploited known ISAPI vulnerability in Microsoft IIS Web servers
bull 1st through 20th of each month spread20th through end of each month attack
bull Payload Web site defacementbull Scanning Random IP addressesbull Bug failure to seed random number generator
86
Code Red Revisions
bull Released July 19 2001
bull Payload flooding attack on wwwwhitehousegovndash Attack was mounted at the IP address of the Web site
bull Bug died after 20th of each month
bull Random number generator for IP scanning fixed
87
Code Red Host Infection Rate
Exponential infection rate
Measured using backscatter technique
88
Designing Fast-Spreading Worms
bull Hit-list scanningndash Time to infect first 10k hosts dominates infection timendash Solution Reconnaissance (stealthy scans etc)
bull Permutation scanningndash Observation Most scanning is redundantndash Idea Shared permutation of address space Start scanning
from own IP address Re-randomize when another infected machine is found
bull Internet-scale hit listsndash Flash worm complete infection within 30 seconds
89
Botnets
bull Bots Autonomous programs performing tasksbull Plenty of ldquobenignrdquo bots
ndash eg weatherbug
bull Botnets group of bots ndash Typically carries malicious connotationndash Large numbers of infected machinesndash Machines ldquoenlistedrdquo with infection vectors like worms
(last lecture)
bull Available for simultaneous control by a masterbull Size up to 350000 nodes (from todayrsquos paper)
90
ldquoRallyingrdquo the Botnet
bull Easy to combine worm backdoor functionalitybull Problem how to learn about successfully
infected machines
bull Optionsndash Emailndash Hard-coded email address
91
Botnet Control
bull Botnet master typically runs some IRC server on a well-known port (eg 6667)
bull Infected machine contacts botnet with pre-programmed DNS name (eg big-botde)
bull Dynamic DNS allows controller to move about freely
Infected Machine
DynamicDNS
BotnetController
(IRC server)
92
Some Defenses
93
Idea 1 Ingress Filtering
bull RFC 2827 Routers install filters to drop packets from networks that are not downstream
bull Feasible at edgesbull Difficult to configure closer to network ldquocorerdquo
20469207024 Internet
Drop all packets with source address other than 20469207024
94
Idea 2 uRPF Checks
bull Unicast Reverse Path Forwardingndash Cisco ldquoip verify unicast reverse-pathrdquo
bull Requires symmetric routing
Accept packet from interface only if forwarding table entry for source IP address matches ingress interface
100183
A10018
10015
101203
100181 24
10011 24
Strict Mode uRPF
Enabled
ldquoArdquo Routing TableDestination Next Hop1001024 Int 110018024 Int 2
100183 from wrong interface
95
Problems with uRPF
bull Asymmetric routing
96
S-BGP
bull Address-based PKI validate signaturesndash Authentication of
bull ownership for IP address blocks bull AS number bull an ASs identity and bull a BGP routers identity
ndash Use existing infrastructure (Internet registries etc)ndash Routing origination is digitally signedndash BGP updates are digitally signed
1048708 bull Route attestations A new optional BGP transitive path attribute
ndash carries digital signatures covering the routing information in updates
97
Practical Problems with S-BGP
bull Requires Public-Key Infrastructure
bull Lots of digital signatures to calculate and verifyndash Message overheadndash CPU overhead
bull Calculation expense is greatest when topology is changingndash Caching can help
bull Route aggregation is problematic (maybe thatrsquos OK)
bull Secure route withdrawals when link or node fails
bull Address ownership data out of date
bull Deployment
- Networking
- Goal of This Tutorial
- Todayrsquos Networks
- Business Models
- Billing for Internet Usage
- Net Neutrality
- Network Operations
- Point-of-Presence (PoP)
- Example Abilene Network Topology
- Another Example Backbone
- Internet Routing Overview
- Internet Routing Protocol BGP
- Two Flavors of BGP
- IPv4 Addresses Networks of Networks
- Pre-1994 Classful Addressing
- Classless Interdomain Routing (CIDR)
- Benefits of CIDR
- Growth of IP Prefixes
- 1994-1998 Linear Growth
- Around 2000 Fast Growth Resumes
- Fast growth resumes
- The Address Allocation Process
- Common Problems
- What can go wrong
- Measurement and Monitoring
- Passive vs Active Measurement
- Slide 27
- Passive Traffic Data Measurement
- Simple Network Management Protocol
- SNMP (Passive)
- Packet-level Monitoring
- Full Packet Capture (Passive)
- What is a flow
- Cisco NetFlow
- Flow Record Contents
- Aggregating Packets into Flows
- Reducing Measurement Overhead
- Packet Sampling for Flow Monitoring
- Sampling Flow-Level Sampling
- Two Main Approaches
- Packet Capture on High-Speed Links
- Characteristics of Packet Capture
- Data Measurement Repositories
- Multihoming and Traffic Engineering
- What is Multihoming
- Why Multihome
- Redundancy
- Performance
- Multihoming in IP Networks Today
- BGP or no
- Same Provider or Multiple
- Multihomed Stub One Link
- Multihomed Stub Multiple Links
- Multihomed Stub Multiple ISPs
- Multihomed Transit Network
- Interdomain Traffic Engineering
- Outbound Traffic Control
- Outbound Traffic Load Balancing
- Traffic Engineering Goals
- Managing Scale
- Achieving Predictability
- Challenges to Predictability
- Inter-AS Negotiation
- Outbound Multihoming Goals
- Inbound Traffic Control
- How many links are enough
- Problems with Multihoming in IPv4
- Slide 68
- Slide 69
- Diagnosis and Troubleshooting
- Operator Mailing List (NANOG)
- Operator Mailing List
- Routing Configuration
- Internet Business Model (Simplified)
- Peering Contracts Consistent Export
- Consistent Export
- Inconsistent Export in Practice
- Blackholes
- Security
- Security ldquoBogonrdquo Routes
- Spam Phishing etc
- Spam and Routing
- Worms and Botnets
- What is a Worm
- Example Worm Code Red
- Code Red Revisions
- Code Red Host Infection Rate
- Designing Fast-Spreading Worms
- Botnets
- ldquoRallyingrdquo the Botnet
- Botnet Control
- Some Defenses
- Idea 1 Ingress Filtering
- Idea 2 uRPF Checks
- Problems with uRPF
- S-BGP
- Practical Problems with S-BGP
-
80
Security ldquoBogonrdquo Routes
Feamster et al ldquoAn Empirical Study of lsquoBogonrsquo Route Advertisementsrdquo ACM CCR January 2005
81
Spam Phishing etc
bull Unsolicited commercial emailbull As of about August 2008 estimates indicate that
about 95 of all email is spambull Common spam filtering techniques
ndash Content-based filtersndash DNS Blacklist (DNSBL) lookups Significant fraction of
todayrsquos DNS traffic
Can IP addresses from which spam is received be spoofed
82
Spam and Routing
83
Worms and Botnets
84
What is a Worm
bull Code that replicates and propagates across the networkndash Often carries a ldquopayloadrdquo
bull Usually spread via exploiting flaws in open servicesndash ldquoVirusesrdquo require user action to spread
bull First worm Robert Morris November 1988ndash 6-10 of all Internet hosts infected ()
bull Many more since but none on that scale until July 2001
85
Example Worm Code Red
bull Initial version July 13 2001
bull Exploited known ISAPI vulnerability in Microsoft IIS Web servers
bull 1st through 20th of each month spread20th through end of each month attack
bull Payload Web site defacementbull Scanning Random IP addressesbull Bug failure to seed random number generator
86
Code Red Revisions
bull Released July 19 2001
bull Payload flooding attack on wwwwhitehousegovndash Attack was mounted at the IP address of the Web site
bull Bug died after 20th of each month
bull Random number generator for IP scanning fixed
87
Code Red Host Infection Rate
Exponential infection rate
Measured using backscatter technique
88
Designing Fast-Spreading Worms
bull Hit-list scanningndash Time to infect first 10k hosts dominates infection timendash Solution Reconnaissance (stealthy scans etc)
bull Permutation scanningndash Observation Most scanning is redundantndash Idea Shared permutation of address space Start scanning
from own IP address Re-randomize when another infected machine is found
bull Internet-scale hit listsndash Flash worm complete infection within 30 seconds
89
Botnets
bull Bots Autonomous programs performing tasksbull Plenty of ldquobenignrdquo bots
ndash eg weatherbug
bull Botnets group of bots ndash Typically carries malicious connotationndash Large numbers of infected machinesndash Machines ldquoenlistedrdquo with infection vectors like worms
(last lecture)
bull Available for simultaneous control by a masterbull Size up to 350000 nodes (from todayrsquos paper)
90
ldquoRallyingrdquo the Botnet
bull Easy to combine worm backdoor functionalitybull Problem how to learn about successfully
infected machines
bull Optionsndash Emailndash Hard-coded email address
91
Botnet Control
bull Botnet master typically runs some IRC server on a well-known port (eg 6667)
bull Infected machine contacts botnet with pre-programmed DNS name (eg big-botde)
bull Dynamic DNS allows controller to move about freely
Infected Machine
DynamicDNS
BotnetController
(IRC server)
92
Some Defenses
93
Idea 1 Ingress Filtering
bull RFC 2827 Routers install filters to drop packets from networks that are not downstream
bull Feasible at edgesbull Difficult to configure closer to network ldquocorerdquo
20469207024 Internet
Drop all packets with source address other than 20469207024
94
Idea 2 uRPF Checks
bull Unicast Reverse Path Forwardingndash Cisco ldquoip verify unicast reverse-pathrdquo
bull Requires symmetric routing
Accept packet from interface only if forwarding table entry for source IP address matches ingress interface
100183
A10018
10015
101203
100181 24
10011 24
Strict Mode uRPF
Enabled
ldquoArdquo Routing TableDestination Next Hop1001024 Int 110018024 Int 2
100183 from wrong interface
95
Problems with uRPF
bull Asymmetric routing
96
S-BGP
bull Address-based PKI validate signaturesndash Authentication of
bull ownership for IP address blocks bull AS number bull an ASs identity and bull a BGP routers identity
ndash Use existing infrastructure (Internet registries etc)ndash Routing origination is digitally signedndash BGP updates are digitally signed
1048708 bull Route attestations A new optional BGP transitive path attribute
ndash carries digital signatures covering the routing information in updates
97
Practical Problems with S-BGP
bull Requires Public-Key Infrastructure
bull Lots of digital signatures to calculate and verifyndash Message overheadndash CPU overhead
bull Calculation expense is greatest when topology is changingndash Caching can help
bull Route aggregation is problematic (maybe thatrsquos OK)
bull Secure route withdrawals when link or node fails
bull Address ownership data out of date
bull Deployment
- Networking
- Goal of This Tutorial
- Todayrsquos Networks
- Business Models
- Billing for Internet Usage
- Net Neutrality
- Network Operations
- Point-of-Presence (PoP)
- Example Abilene Network Topology
- Another Example Backbone
- Internet Routing Overview
- Internet Routing Protocol BGP
- Two Flavors of BGP
- IPv4 Addresses Networks of Networks
- Pre-1994 Classful Addressing
- Classless Interdomain Routing (CIDR)
- Benefits of CIDR
- Growth of IP Prefixes
- 1994-1998 Linear Growth
- Around 2000 Fast Growth Resumes
- Fast growth resumes
- The Address Allocation Process
- Common Problems
- What can go wrong
- Measurement and Monitoring
- Passive vs Active Measurement
- Slide 27
- Passive Traffic Data Measurement
- Simple Network Management Protocol
- SNMP (Passive)
- Packet-level Monitoring
- Full Packet Capture (Passive)
- What is a flow
- Cisco NetFlow
- Flow Record Contents
- Aggregating Packets into Flows
- Reducing Measurement Overhead
- Packet Sampling for Flow Monitoring
- Sampling Flow-Level Sampling
- Two Main Approaches
- Packet Capture on High-Speed Links
- Characteristics of Packet Capture
- Data Measurement Repositories
- Multihoming and Traffic Engineering
- What is Multihoming
- Why Multihome
- Redundancy
- Performance
- Multihoming in IP Networks Today
- BGP or no
- Same Provider or Multiple
- Multihomed Stub One Link
- Multihomed Stub Multiple Links
- Multihomed Stub Multiple ISPs
- Multihomed Transit Network
- Interdomain Traffic Engineering
- Outbound Traffic Control
- Outbound Traffic Load Balancing
- Traffic Engineering Goals
- Managing Scale
- Achieving Predictability
- Challenges to Predictability
- Inter-AS Negotiation
- Outbound Multihoming Goals
- Inbound Traffic Control
- How many links are enough
- Problems with Multihoming in IPv4
- Slide 68
- Slide 69
- Diagnosis and Troubleshooting
- Operator Mailing List (NANOG)
- Operator Mailing List
- Routing Configuration
- Internet Business Model (Simplified)
- Peering Contracts Consistent Export
- Consistent Export
- Inconsistent Export in Practice
- Blackholes
- Security
- Security ldquoBogonrdquo Routes
- Spam Phishing etc
- Spam and Routing
- Worms and Botnets
- What is a Worm
- Example Worm Code Red
- Code Red Revisions
- Code Red Host Infection Rate
- Designing Fast-Spreading Worms
- Botnets
- ldquoRallyingrdquo the Botnet
- Botnet Control
- Some Defenses
- Idea 1 Ingress Filtering
- Idea 2 uRPF Checks
- Problems with uRPF
- S-BGP
- Practical Problems with S-BGP
-
81
Spam Phishing etc
bull Unsolicited commercial emailbull As of about August 2008 estimates indicate that
about 95 of all email is spambull Common spam filtering techniques
ndash Content-based filtersndash DNS Blacklist (DNSBL) lookups Significant fraction of
todayrsquos DNS traffic
Can IP addresses from which spam is received be spoofed
82
Spam and Routing
83
Worms and Botnets
84
What is a Worm
bull Code that replicates and propagates across the networkndash Often carries a ldquopayloadrdquo
bull Usually spread via exploiting flaws in open servicesndash ldquoVirusesrdquo require user action to spread
bull First worm Robert Morris November 1988ndash 6-10 of all Internet hosts infected ()
bull Many more since but none on that scale until July 2001
85
Example Worm Code Red
bull Initial version July 13 2001
bull Exploited known ISAPI vulnerability in Microsoft IIS Web servers
bull 1st through 20th of each month spread20th through end of each month attack
bull Payload Web site defacementbull Scanning Random IP addressesbull Bug failure to seed random number generator
86
Code Red Revisions
bull Released July 19 2001
bull Payload flooding attack on wwwwhitehousegovndash Attack was mounted at the IP address of the Web site
bull Bug died after 20th of each month
bull Random number generator for IP scanning fixed
87
Code Red Host Infection Rate
Exponential infection rate
Measured using backscatter technique
88
Designing Fast-Spreading Worms
bull Hit-list scanningndash Time to infect first 10k hosts dominates infection timendash Solution Reconnaissance (stealthy scans etc)
bull Permutation scanningndash Observation Most scanning is redundantndash Idea Shared permutation of address space Start scanning
from own IP address Re-randomize when another infected machine is found
bull Internet-scale hit listsndash Flash worm complete infection within 30 seconds
89
Botnets
bull Bots Autonomous programs performing tasksbull Plenty of ldquobenignrdquo bots
ndash eg weatherbug
bull Botnets group of bots ndash Typically carries malicious connotationndash Large numbers of infected machinesndash Machines ldquoenlistedrdquo with infection vectors like worms
(last lecture)
bull Available for simultaneous control by a masterbull Size up to 350000 nodes (from todayrsquos paper)
90
ldquoRallyingrdquo the Botnet
bull Easy to combine worm backdoor functionalitybull Problem how to learn about successfully
infected machines
bull Optionsndash Emailndash Hard-coded email address
91
Botnet Control
bull Botnet master typically runs some IRC server on a well-known port (eg 6667)
bull Infected machine contacts botnet with pre-programmed DNS name (eg big-botde)
bull Dynamic DNS allows controller to move about freely
Infected Machine
DynamicDNS
BotnetController
(IRC server)
92
Some Defenses
93
Idea 1 Ingress Filtering
bull RFC 2827 Routers install filters to drop packets from networks that are not downstream
bull Feasible at edgesbull Difficult to configure closer to network ldquocorerdquo
20469207024 Internet
Drop all packets with source address other than 20469207024
94
Idea 2 uRPF Checks
bull Unicast Reverse Path Forwardingndash Cisco ldquoip verify unicast reverse-pathrdquo
bull Requires symmetric routing
Accept packet from interface only if forwarding table entry for source IP address matches ingress interface
100183
A10018
10015
101203
100181 24
10011 24
Strict Mode uRPF
Enabled
ldquoArdquo Routing TableDestination Next Hop1001024 Int 110018024 Int 2
100183 from wrong interface
95
Problems with uRPF
bull Asymmetric routing
96
S-BGP
bull Address-based PKI validate signaturesndash Authentication of
bull ownership for IP address blocks bull AS number bull an ASs identity and bull a BGP routers identity
ndash Use existing infrastructure (Internet registries etc)ndash Routing origination is digitally signedndash BGP updates are digitally signed
1048708 bull Route attestations A new optional BGP transitive path attribute
ndash carries digital signatures covering the routing information in updates
97
Practical Problems with S-BGP
bull Requires Public-Key Infrastructure
bull Lots of digital signatures to calculate and verifyndash Message overheadndash CPU overhead
bull Calculation expense is greatest when topology is changingndash Caching can help
bull Route aggregation is problematic (maybe thatrsquos OK)
bull Secure route withdrawals when link or node fails
bull Address ownership data out of date
bull Deployment
- Networking
- Goal of This Tutorial
- Todayrsquos Networks
- Business Models
- Billing for Internet Usage
- Net Neutrality
- Network Operations
- Point-of-Presence (PoP)
- Example Abilene Network Topology
- Another Example Backbone
- Internet Routing Overview
- Internet Routing Protocol BGP
- Two Flavors of BGP
- IPv4 Addresses Networks of Networks
- Pre-1994 Classful Addressing
- Classless Interdomain Routing (CIDR)
- Benefits of CIDR
- Growth of IP Prefixes
- 1994-1998 Linear Growth
- Around 2000 Fast Growth Resumes
- Fast growth resumes
- The Address Allocation Process
- Common Problems
- What can go wrong
- Measurement and Monitoring
- Passive vs Active Measurement
- Slide 27
- Passive Traffic Data Measurement
- Simple Network Management Protocol
- SNMP (Passive)
- Packet-level Monitoring
- Full Packet Capture (Passive)
- What is a flow
- Cisco NetFlow
- Flow Record Contents
- Aggregating Packets into Flows
- Reducing Measurement Overhead
- Packet Sampling for Flow Monitoring
- Sampling Flow-Level Sampling
- Two Main Approaches
- Packet Capture on High-Speed Links
- Characteristics of Packet Capture
- Data Measurement Repositories
- Multihoming and Traffic Engineering
- What is Multihoming
- Why Multihome
- Redundancy
- Performance
- Multihoming in IP Networks Today
- BGP or no
- Same Provider or Multiple
- Multihomed Stub One Link
- Multihomed Stub Multiple Links
- Multihomed Stub Multiple ISPs
- Multihomed Transit Network
- Interdomain Traffic Engineering
- Outbound Traffic Control
- Outbound Traffic Load Balancing
- Traffic Engineering Goals
- Managing Scale
- Achieving Predictability
- Challenges to Predictability
- Inter-AS Negotiation
- Outbound Multihoming Goals
- Inbound Traffic Control
- How many links are enough
- Problems with Multihoming in IPv4
- Slide 68
- Slide 69
- Diagnosis and Troubleshooting
- Operator Mailing List (NANOG)
- Operator Mailing List
- Routing Configuration
- Internet Business Model (Simplified)
- Peering Contracts Consistent Export
- Consistent Export
- Inconsistent Export in Practice
- Blackholes
- Security
- Security ldquoBogonrdquo Routes
- Spam Phishing etc
- Spam and Routing
- Worms and Botnets
- What is a Worm
- Example Worm Code Red
- Code Red Revisions
- Code Red Host Infection Rate
- Designing Fast-Spreading Worms
- Botnets
- ldquoRallyingrdquo the Botnet
- Botnet Control
- Some Defenses
- Idea 1 Ingress Filtering
- Idea 2 uRPF Checks
- Problems with uRPF
- S-BGP
- Practical Problems with S-BGP
-
82
Spam and Routing
83
Worms and Botnets
84
What is a Worm
bull Code that replicates and propagates across the networkndash Often carries a ldquopayloadrdquo
bull Usually spread via exploiting flaws in open servicesndash ldquoVirusesrdquo require user action to spread
bull First worm Robert Morris November 1988ndash 6-10 of all Internet hosts infected ()
bull Many more since but none on that scale until July 2001
85
Example Worm Code Red
bull Initial version July 13 2001
bull Exploited known ISAPI vulnerability in Microsoft IIS Web servers
bull 1st through 20th of each month spread20th through end of each month attack
bull Payload Web site defacementbull Scanning Random IP addressesbull Bug failure to seed random number generator
86
Code Red Revisions
bull Released July 19 2001
bull Payload flooding attack on wwwwhitehousegovndash Attack was mounted at the IP address of the Web site
bull Bug died after 20th of each month
bull Random number generator for IP scanning fixed
87
Code Red Host Infection Rate
Exponential infection rate
Measured using backscatter technique
88
Designing Fast-Spreading Worms
bull Hit-list scanningndash Time to infect first 10k hosts dominates infection timendash Solution Reconnaissance (stealthy scans etc)
bull Permutation scanningndash Observation Most scanning is redundantndash Idea Shared permutation of address space Start scanning
from own IP address Re-randomize when another infected machine is found
bull Internet-scale hit listsndash Flash worm complete infection within 30 seconds
89
Botnets
bull Bots Autonomous programs performing tasksbull Plenty of ldquobenignrdquo bots
ndash eg weatherbug
bull Botnets group of bots ndash Typically carries malicious connotationndash Large numbers of infected machinesndash Machines ldquoenlistedrdquo with infection vectors like worms
(last lecture)
bull Available for simultaneous control by a masterbull Size up to 350000 nodes (from todayrsquos paper)
90
ldquoRallyingrdquo the Botnet
bull Easy to combine worm backdoor functionalitybull Problem how to learn about successfully
infected machines
bull Optionsndash Emailndash Hard-coded email address
91
Botnet Control
bull Botnet master typically runs some IRC server on a well-known port (eg 6667)
bull Infected machine contacts botnet with pre-programmed DNS name (eg big-botde)
bull Dynamic DNS allows controller to move about freely
Infected Machine
DynamicDNS
BotnetController
(IRC server)
92
Some Defenses
93
Idea 1 Ingress Filtering
bull RFC 2827 Routers install filters to drop packets from networks that are not downstream
bull Feasible at edgesbull Difficult to configure closer to network ldquocorerdquo
20469207024 Internet
Drop all packets with source address other than 20469207024
94
Idea 2 uRPF Checks
bull Unicast Reverse Path Forwardingndash Cisco ldquoip verify unicast reverse-pathrdquo
bull Requires symmetric routing
Accept packet from interface only if forwarding table entry for source IP address matches ingress interface
100183
A10018
10015
101203
100181 24
10011 24
Strict Mode uRPF
Enabled
ldquoArdquo Routing TableDestination Next Hop1001024 Int 110018024 Int 2
100183 from wrong interface
95
Problems with uRPF
bull Asymmetric routing
96
S-BGP
bull Address-based PKI validate signaturesndash Authentication of
bull ownership for IP address blocks bull AS number bull an ASs identity and bull a BGP routers identity
ndash Use existing infrastructure (Internet registries etc)ndash Routing origination is digitally signedndash BGP updates are digitally signed
1048708 bull Route attestations A new optional BGP transitive path attribute
ndash carries digital signatures covering the routing information in updates
97
Practical Problems with S-BGP
bull Requires Public-Key Infrastructure
bull Lots of digital signatures to calculate and verifyndash Message overheadndash CPU overhead
bull Calculation expense is greatest when topology is changingndash Caching can help
bull Route aggregation is problematic (maybe thatrsquos OK)
bull Secure route withdrawals when link or node fails
bull Address ownership data out of date
bull Deployment
- Networking
- Goal of This Tutorial
- Todayrsquos Networks
- Business Models
- Billing for Internet Usage
- Net Neutrality
- Network Operations
- Point-of-Presence (PoP)
- Example Abilene Network Topology
- Another Example Backbone
- Internet Routing Overview
- Internet Routing Protocol BGP
- Two Flavors of BGP
- IPv4 Addresses Networks of Networks
- Pre-1994 Classful Addressing
- Classless Interdomain Routing (CIDR)
- Benefits of CIDR
- Growth of IP Prefixes
- 1994-1998 Linear Growth
- Around 2000 Fast Growth Resumes
- Fast growth resumes
- The Address Allocation Process
- Common Problems
- What can go wrong
- Measurement and Monitoring
- Passive vs Active Measurement
- Slide 27
- Passive Traffic Data Measurement
- Simple Network Management Protocol
- SNMP (Passive)
- Packet-level Monitoring
- Full Packet Capture (Passive)
- What is a flow
- Cisco NetFlow
- Flow Record Contents
- Aggregating Packets into Flows
- Reducing Measurement Overhead
- Packet Sampling for Flow Monitoring
- Sampling Flow-Level Sampling
- Two Main Approaches
- Packet Capture on High-Speed Links
- Characteristics of Packet Capture
- Data Measurement Repositories
- Multihoming and Traffic Engineering
- What is Multihoming
- Why Multihome
- Redundancy
- Performance
- Multihoming in IP Networks Today
- BGP or no
- Same Provider or Multiple
- Multihomed Stub One Link
- Multihomed Stub Multiple Links
- Multihomed Stub Multiple ISPs
- Multihomed Transit Network
- Interdomain Traffic Engineering
- Outbound Traffic Control
- Outbound Traffic Load Balancing
- Traffic Engineering Goals
- Managing Scale
- Achieving Predictability
- Challenges to Predictability
- Inter-AS Negotiation
- Outbound Multihoming Goals
- Inbound Traffic Control
- How many links are enough
- Problems with Multihoming in IPv4
- Slide 68
- Slide 69
- Diagnosis and Troubleshooting
- Operator Mailing List (NANOG)
- Operator Mailing List
- Routing Configuration
- Internet Business Model (Simplified)
- Peering Contracts Consistent Export
- Consistent Export
- Inconsistent Export in Practice
- Blackholes
- Security
- Security ldquoBogonrdquo Routes
- Spam Phishing etc
- Spam and Routing
- Worms and Botnets
- What is a Worm
- Example Worm Code Red
- Code Red Revisions
- Code Red Host Infection Rate
- Designing Fast-Spreading Worms
- Botnets
- ldquoRallyingrdquo the Botnet
- Botnet Control
- Some Defenses
- Idea 1 Ingress Filtering
- Idea 2 uRPF Checks
- Problems with uRPF
- S-BGP
- Practical Problems with S-BGP
-
83
Worms and Botnets
84
What is a Worm
bull Code that replicates and propagates across the networkndash Often carries a ldquopayloadrdquo
bull Usually spread via exploiting flaws in open servicesndash ldquoVirusesrdquo require user action to spread
bull First worm Robert Morris November 1988ndash 6-10 of all Internet hosts infected ()
bull Many more since but none on that scale until July 2001
85
Example Worm Code Red
bull Initial version July 13 2001
bull Exploited known ISAPI vulnerability in Microsoft IIS Web servers
bull 1st through 20th of each month spread20th through end of each month attack
bull Payload Web site defacementbull Scanning Random IP addressesbull Bug failure to seed random number generator
86
Code Red Revisions
bull Released July 19 2001
bull Payload flooding attack on wwwwhitehousegovndash Attack was mounted at the IP address of the Web site
bull Bug died after 20th of each month
bull Random number generator for IP scanning fixed
87
Code Red Host Infection Rate
Exponential infection rate
Measured using backscatter technique
88
Designing Fast-Spreading Worms
bull Hit-list scanningndash Time to infect first 10k hosts dominates infection timendash Solution Reconnaissance (stealthy scans etc)
bull Permutation scanningndash Observation Most scanning is redundantndash Idea Shared permutation of address space Start scanning
from own IP address Re-randomize when another infected machine is found
bull Internet-scale hit listsndash Flash worm complete infection within 30 seconds
89
Botnets
bull Bots Autonomous programs performing tasksbull Plenty of ldquobenignrdquo bots
ndash eg weatherbug
bull Botnets group of bots ndash Typically carries malicious connotationndash Large numbers of infected machinesndash Machines ldquoenlistedrdquo with infection vectors like worms
(last lecture)
bull Available for simultaneous control by a masterbull Size up to 350000 nodes (from todayrsquos paper)
90
ldquoRallyingrdquo the Botnet
bull Easy to combine worm backdoor functionalitybull Problem how to learn about successfully
infected machines
bull Optionsndash Emailndash Hard-coded email address
91
Botnet Control
bull Botnet master typically runs some IRC server on a well-known port (eg 6667)
bull Infected machine contacts botnet with pre-programmed DNS name (eg big-botde)
bull Dynamic DNS allows controller to move about freely
Infected Machine
DynamicDNS
BotnetController
(IRC server)
92
Some Defenses
93
Idea 1 Ingress Filtering
bull RFC 2827 Routers install filters to drop packets from networks that are not downstream
bull Feasible at edgesbull Difficult to configure closer to network ldquocorerdquo
20469207024 Internet
Drop all packets with source address other than 20469207024
94
Idea 2 uRPF Checks
bull Unicast Reverse Path Forwardingndash Cisco ldquoip verify unicast reverse-pathrdquo
bull Requires symmetric routing
Accept packet from interface only if forwarding table entry for source IP address matches ingress interface
100183
A10018
10015
101203
100181 24
10011 24
Strict Mode uRPF
Enabled
ldquoArdquo Routing TableDestination Next Hop1001024 Int 110018024 Int 2
100183 from wrong interface
95
Problems with uRPF
bull Asymmetric routing
96
S-BGP
bull Address-based PKI validate signaturesndash Authentication of
bull ownership for IP address blocks bull AS number bull an ASs identity and bull a BGP routers identity
ndash Use existing infrastructure (Internet registries etc)ndash Routing origination is digitally signedndash BGP updates are digitally signed
1048708 bull Route attestations A new optional BGP transitive path attribute
ndash carries digital signatures covering the routing information in updates
97
Practical Problems with S-BGP
bull Requires Public-Key Infrastructure
bull Lots of digital signatures to calculate and verifyndash Message overheadndash CPU overhead
bull Calculation expense is greatest when topology is changingndash Caching can help
bull Route aggregation is problematic (maybe thatrsquos OK)
bull Secure route withdrawals when link or node fails
bull Address ownership data out of date
bull Deployment
- Networking
- Goal of This Tutorial
- Todayrsquos Networks
- Business Models
- Billing for Internet Usage
- Net Neutrality
- Network Operations
- Point-of-Presence (PoP)
- Example Abilene Network Topology
- Another Example Backbone
- Internet Routing Overview
- Internet Routing Protocol BGP
- Two Flavors of BGP
- IPv4 Addresses Networks of Networks
- Pre-1994 Classful Addressing
- Classless Interdomain Routing (CIDR)
- Benefits of CIDR
- Growth of IP Prefixes
- 1994-1998 Linear Growth
- Around 2000 Fast Growth Resumes
- Fast growth resumes
- The Address Allocation Process
- Common Problems
- What can go wrong
- Measurement and Monitoring
- Passive vs Active Measurement
- Slide 27
- Passive Traffic Data Measurement
- Simple Network Management Protocol
- SNMP (Passive)
- Packet-level Monitoring
- Full Packet Capture (Passive)
- What is a flow
- Cisco NetFlow
- Flow Record Contents
- Aggregating Packets into Flows
- Reducing Measurement Overhead
- Packet Sampling for Flow Monitoring
- Sampling Flow-Level Sampling
- Two Main Approaches
- Packet Capture on High-Speed Links
- Characteristics of Packet Capture
- Data Measurement Repositories
- Multihoming and Traffic Engineering
- What is Multihoming
- Why Multihome
- Redundancy
- Performance
- Multihoming in IP Networks Today
- BGP or no
- Same Provider or Multiple
- Multihomed Stub One Link
- Multihomed Stub Multiple Links
- Multihomed Stub Multiple ISPs
- Multihomed Transit Network
- Interdomain Traffic Engineering
- Outbound Traffic Control
- Outbound Traffic Load Balancing
- Traffic Engineering Goals
- Managing Scale
- Achieving Predictability
- Challenges to Predictability
- Inter-AS Negotiation
- Outbound Multihoming Goals
- Inbound Traffic Control
- How many links are enough
- Problems with Multihoming in IPv4
- Slide 68
- Slide 69
- Diagnosis and Troubleshooting
- Operator Mailing List (NANOG)
- Operator Mailing List
- Routing Configuration
- Internet Business Model (Simplified)
- Peering Contracts Consistent Export
- Consistent Export
- Inconsistent Export in Practice
- Blackholes
- Security
- Security ldquoBogonrdquo Routes
- Spam Phishing etc
- Spam and Routing
- Worms and Botnets
- What is a Worm
- Example Worm Code Red
- Code Red Revisions
- Code Red Host Infection Rate
- Designing Fast-Spreading Worms
- Botnets
- ldquoRallyingrdquo the Botnet
- Botnet Control
- Some Defenses
- Idea 1 Ingress Filtering
- Idea 2 uRPF Checks
- Problems with uRPF
- S-BGP
- Practical Problems with S-BGP
-
84
What is a Worm
bull Code that replicates and propagates across the networkndash Often carries a ldquopayloadrdquo
bull Usually spread via exploiting flaws in open servicesndash ldquoVirusesrdquo require user action to spread
bull First worm Robert Morris November 1988ndash 6-10 of all Internet hosts infected ()
bull Many more since but none on that scale until July 2001
85
Example Worm Code Red
bull Initial version July 13 2001
bull Exploited known ISAPI vulnerability in Microsoft IIS Web servers
bull 1st through 20th of each month spread20th through end of each month attack
bull Payload Web site defacementbull Scanning Random IP addressesbull Bug failure to seed random number generator
86
Code Red Revisions
bull Released July 19 2001
bull Payload flooding attack on wwwwhitehousegovndash Attack was mounted at the IP address of the Web site
bull Bug died after 20th of each month
bull Random number generator for IP scanning fixed
87
Code Red Host Infection Rate
Exponential infection rate
Measured using backscatter technique
88
Designing Fast-Spreading Worms
bull Hit-list scanningndash Time to infect first 10k hosts dominates infection timendash Solution Reconnaissance (stealthy scans etc)
bull Permutation scanningndash Observation Most scanning is redundantndash Idea Shared permutation of address space Start scanning
from own IP address Re-randomize when another infected machine is found
bull Internet-scale hit listsndash Flash worm complete infection within 30 seconds
89
Botnets
bull Bots Autonomous programs performing tasksbull Plenty of ldquobenignrdquo bots
ndash eg weatherbug
bull Botnets group of bots ndash Typically carries malicious connotationndash Large numbers of infected machinesndash Machines ldquoenlistedrdquo with infection vectors like worms
(last lecture)
bull Available for simultaneous control by a masterbull Size up to 350000 nodes (from todayrsquos paper)
90
ldquoRallyingrdquo the Botnet
bull Easy to combine worm backdoor functionalitybull Problem how to learn about successfully
infected machines
bull Optionsndash Emailndash Hard-coded email address
91
Botnet Control
bull Botnet master typically runs some IRC server on a well-known port (eg 6667)
bull Infected machine contacts botnet with pre-programmed DNS name (eg big-botde)
bull Dynamic DNS allows controller to move about freely
Infected Machine
DynamicDNS
BotnetController
(IRC server)
92
Some Defenses
93
Idea 1 Ingress Filtering
bull RFC 2827 Routers install filters to drop packets from networks that are not downstream
bull Feasible at edgesbull Difficult to configure closer to network ldquocorerdquo
20469207024 Internet
Drop all packets with source address other than 20469207024
94
Idea 2 uRPF Checks
bull Unicast Reverse Path Forwardingndash Cisco ldquoip verify unicast reverse-pathrdquo
bull Requires symmetric routing
Accept packet from interface only if forwarding table entry for source IP address matches ingress interface
100183
A10018
10015
101203
100181 24
10011 24
Strict Mode uRPF
Enabled
ldquoArdquo Routing TableDestination Next Hop1001024 Int 110018024 Int 2
100183 from wrong interface
95
Problems with uRPF
bull Asymmetric routing
96
S-BGP
bull Address-based PKI validate signaturesndash Authentication of
bull ownership for IP address blocks bull AS number bull an ASs identity and bull a BGP routers identity
ndash Use existing infrastructure (Internet registries etc)ndash Routing origination is digitally signedndash BGP updates are digitally signed
1048708 bull Route attestations A new optional BGP transitive path attribute
ndash carries digital signatures covering the routing information in updates
97
Practical Problems with S-BGP
bull Requires Public-Key Infrastructure
bull Lots of digital signatures to calculate and verifyndash Message overheadndash CPU overhead
bull Calculation expense is greatest when topology is changingndash Caching can help
bull Route aggregation is problematic (maybe thatrsquos OK)
bull Secure route withdrawals when link or node fails
bull Address ownership data out of date
bull Deployment
- Networking
- Goal of This Tutorial
- Todayrsquos Networks
- Business Models
- Billing for Internet Usage
- Net Neutrality
- Network Operations
- Point-of-Presence (PoP)
- Example Abilene Network Topology
- Another Example Backbone
- Internet Routing Overview
- Internet Routing Protocol BGP
- Two Flavors of BGP
- IPv4 Addresses Networks of Networks
- Pre-1994 Classful Addressing
- Classless Interdomain Routing (CIDR)
- Benefits of CIDR
- Growth of IP Prefixes
- 1994-1998 Linear Growth
- Around 2000 Fast Growth Resumes
- Fast growth resumes
- The Address Allocation Process
- Common Problems
- What can go wrong
- Measurement and Monitoring
- Passive vs Active Measurement
- Slide 27
- Passive Traffic Data Measurement
- Simple Network Management Protocol
- SNMP (Passive)
- Packet-level Monitoring
- Full Packet Capture (Passive)
- What is a flow
- Cisco NetFlow
- Flow Record Contents
- Aggregating Packets into Flows
- Reducing Measurement Overhead
- Packet Sampling for Flow Monitoring
- Sampling Flow-Level Sampling
- Two Main Approaches
- Packet Capture on High-Speed Links
- Characteristics of Packet Capture
- Data Measurement Repositories
- Multihoming and Traffic Engineering
- What is Multihoming
- Why Multihome
- Redundancy
- Performance
- Multihoming in IP Networks Today
- BGP or no
- Same Provider or Multiple
- Multihomed Stub One Link
- Multihomed Stub Multiple Links
- Multihomed Stub Multiple ISPs
- Multihomed Transit Network
- Interdomain Traffic Engineering
- Outbound Traffic Control
- Outbound Traffic Load Balancing
- Traffic Engineering Goals
- Managing Scale
- Achieving Predictability
- Challenges to Predictability
- Inter-AS Negotiation
- Outbound Multihoming Goals
- Inbound Traffic Control
- How many links are enough
- Problems with Multihoming in IPv4
- Slide 68
- Slide 69
- Diagnosis and Troubleshooting
- Operator Mailing List (NANOG)
- Operator Mailing List
- Routing Configuration
- Internet Business Model (Simplified)
- Peering Contracts Consistent Export
- Consistent Export
- Inconsistent Export in Practice
- Blackholes
- Security
- Security ldquoBogonrdquo Routes
- Spam Phishing etc
- Spam and Routing
- Worms and Botnets
- What is a Worm
- Example Worm Code Red
- Code Red Revisions
- Code Red Host Infection Rate
- Designing Fast-Spreading Worms
- Botnets
- ldquoRallyingrdquo the Botnet
- Botnet Control
- Some Defenses
- Idea 1 Ingress Filtering
- Idea 2 uRPF Checks
- Problems with uRPF
- S-BGP
- Practical Problems with S-BGP
-
85
Example Worm Code Red
bull Initial version July 13 2001
bull Exploited known ISAPI vulnerability in Microsoft IIS Web servers
bull 1st through 20th of each month spread20th through end of each month attack
bull Payload Web site defacementbull Scanning Random IP addressesbull Bug failure to seed random number generator
86
Code Red Revisions
bull Released July 19 2001
bull Payload flooding attack on wwwwhitehousegovndash Attack was mounted at the IP address of the Web site
bull Bug died after 20th of each month
bull Random number generator for IP scanning fixed
87
Code Red Host Infection Rate
Exponential infection rate
Measured using backscatter technique
88
Designing Fast-Spreading Worms
bull Hit-list scanningndash Time to infect first 10k hosts dominates infection timendash Solution Reconnaissance (stealthy scans etc)
bull Permutation scanningndash Observation Most scanning is redundantndash Idea Shared permutation of address space Start scanning
from own IP address Re-randomize when another infected machine is found
bull Internet-scale hit listsndash Flash worm complete infection within 30 seconds
89
Botnets
bull Bots Autonomous programs performing tasksbull Plenty of ldquobenignrdquo bots
ndash eg weatherbug
bull Botnets group of bots ndash Typically carries malicious connotationndash Large numbers of infected machinesndash Machines ldquoenlistedrdquo with infection vectors like worms
(last lecture)
bull Available for simultaneous control by a masterbull Size up to 350000 nodes (from todayrsquos paper)
90
ldquoRallyingrdquo the Botnet
bull Easy to combine worm backdoor functionalitybull Problem how to learn about successfully
infected machines
bull Optionsndash Emailndash Hard-coded email address
91
Botnet Control
bull Botnet master typically runs some IRC server on a well-known port (eg 6667)
bull Infected machine contacts botnet with pre-programmed DNS name (eg big-botde)
bull Dynamic DNS allows controller to move about freely
Infected Machine
DynamicDNS
BotnetController
(IRC server)
92
Some Defenses
93
Idea 1 Ingress Filtering
bull RFC 2827 Routers install filters to drop packets from networks that are not downstream
bull Feasible at edgesbull Difficult to configure closer to network ldquocorerdquo
20469207024 Internet
Drop all packets with source address other than 20469207024
94
Idea 2 uRPF Checks
bull Unicast Reverse Path Forwardingndash Cisco ldquoip verify unicast reverse-pathrdquo
bull Requires symmetric routing
Accept packet from interface only if forwarding table entry for source IP address matches ingress interface
100183
A10018
10015
101203
100181 24
10011 24
Strict Mode uRPF
Enabled
ldquoArdquo Routing TableDestination Next Hop1001024 Int 110018024 Int 2
100183 from wrong interface
95
Problems with uRPF
bull Asymmetric routing
96
S-BGP
bull Address-based PKI validate signaturesndash Authentication of
bull ownership for IP address blocks bull AS number bull an ASs identity and bull a BGP routers identity
ndash Use existing infrastructure (Internet registries etc)ndash Routing origination is digitally signedndash BGP updates are digitally signed
1048708 bull Route attestations A new optional BGP transitive path attribute
ndash carries digital signatures covering the routing information in updates
97
Practical Problems with S-BGP
bull Requires Public-Key Infrastructure
bull Lots of digital signatures to calculate and verifyndash Message overheadndash CPU overhead
bull Calculation expense is greatest when topology is changingndash Caching can help
bull Route aggregation is problematic (maybe thatrsquos OK)
bull Secure route withdrawals when link or node fails
bull Address ownership data out of date
bull Deployment
- Networking
- Goal of This Tutorial
- Todayrsquos Networks
- Business Models
- Billing for Internet Usage
- Net Neutrality
- Network Operations
- Point-of-Presence (PoP)
- Example Abilene Network Topology
- Another Example Backbone
- Internet Routing Overview
- Internet Routing Protocol BGP
- Two Flavors of BGP
- IPv4 Addresses Networks of Networks
- Pre-1994 Classful Addressing
- Classless Interdomain Routing (CIDR)
- Benefits of CIDR
- Growth of IP Prefixes
- 1994-1998 Linear Growth
- Around 2000 Fast Growth Resumes
- Fast growth resumes
- The Address Allocation Process
- Common Problems
- What can go wrong
- Measurement and Monitoring
- Passive vs Active Measurement
- Slide 27
- Passive Traffic Data Measurement
- Simple Network Management Protocol
- SNMP (Passive)
- Packet-level Monitoring
- Full Packet Capture (Passive)
- What is a flow
- Cisco NetFlow
- Flow Record Contents
- Aggregating Packets into Flows
- Reducing Measurement Overhead
- Packet Sampling for Flow Monitoring
- Sampling Flow-Level Sampling
- Two Main Approaches
- Packet Capture on High-Speed Links
- Characteristics of Packet Capture
- Data Measurement Repositories
- Multihoming and Traffic Engineering
- What is Multihoming
- Why Multihome
- Redundancy
- Performance
- Multihoming in IP Networks Today
- BGP or no
- Same Provider or Multiple
- Multihomed Stub One Link
- Multihomed Stub Multiple Links
- Multihomed Stub Multiple ISPs
- Multihomed Transit Network
- Interdomain Traffic Engineering
- Outbound Traffic Control
- Outbound Traffic Load Balancing
- Traffic Engineering Goals
- Managing Scale
- Achieving Predictability
- Challenges to Predictability
- Inter-AS Negotiation
- Outbound Multihoming Goals
- Inbound Traffic Control
- How many links are enough
- Problems with Multihoming in IPv4
- Slide 68
- Slide 69
- Diagnosis and Troubleshooting
- Operator Mailing List (NANOG)
- Operator Mailing List
- Routing Configuration
- Internet Business Model (Simplified)
- Peering Contracts Consistent Export
- Consistent Export
- Inconsistent Export in Practice
- Blackholes
- Security
- Security ldquoBogonrdquo Routes
- Spam Phishing etc
- Spam and Routing
- Worms and Botnets
- What is a Worm
- Example Worm Code Red
- Code Red Revisions
- Code Red Host Infection Rate
- Designing Fast-Spreading Worms
- Botnets
- ldquoRallyingrdquo the Botnet
- Botnet Control
- Some Defenses
- Idea 1 Ingress Filtering
- Idea 2 uRPF Checks
- Problems with uRPF
- S-BGP
- Practical Problems with S-BGP
-
86
Code Red Revisions
bull Released July 19 2001
bull Payload flooding attack on wwwwhitehousegovndash Attack was mounted at the IP address of the Web site
bull Bug died after 20th of each month
bull Random number generator for IP scanning fixed
87
Code Red Host Infection Rate
Exponential infection rate
Measured using backscatter technique
88
Designing Fast-Spreading Worms
bull Hit-list scanningndash Time to infect first 10k hosts dominates infection timendash Solution Reconnaissance (stealthy scans etc)
bull Permutation scanningndash Observation Most scanning is redundantndash Idea Shared permutation of address space Start scanning
from own IP address Re-randomize when another infected machine is found
bull Internet-scale hit listsndash Flash worm complete infection within 30 seconds
89
Botnets
bull Bots Autonomous programs performing tasksbull Plenty of ldquobenignrdquo bots
ndash eg weatherbug
bull Botnets group of bots ndash Typically carries malicious connotationndash Large numbers of infected machinesndash Machines ldquoenlistedrdquo with infection vectors like worms
(last lecture)
bull Available for simultaneous control by a masterbull Size up to 350000 nodes (from todayrsquos paper)
90
ldquoRallyingrdquo the Botnet
bull Easy to combine worm backdoor functionalitybull Problem how to learn about successfully
infected machines
bull Optionsndash Emailndash Hard-coded email address
91
Botnet Control
bull Botnet master typically runs some IRC server on a well-known port (eg 6667)
bull Infected machine contacts botnet with pre-programmed DNS name (eg big-botde)
bull Dynamic DNS allows controller to move about freely
Infected Machine
DynamicDNS
BotnetController
(IRC server)
92
Some Defenses
93
Idea 1 Ingress Filtering
bull RFC 2827 Routers install filters to drop packets from networks that are not downstream
bull Feasible at edgesbull Difficult to configure closer to network ldquocorerdquo
20469207024 Internet
Drop all packets with source address other than 20469207024
94
Idea 2 uRPF Checks
bull Unicast Reverse Path Forwardingndash Cisco ldquoip verify unicast reverse-pathrdquo
bull Requires symmetric routing
Accept packet from interface only if forwarding table entry for source IP address matches ingress interface
100183
A10018
10015
101203
100181 24
10011 24
Strict Mode uRPF
Enabled
ldquoArdquo Routing TableDestination Next Hop1001024 Int 110018024 Int 2
100183 from wrong interface
95
Problems with uRPF
bull Asymmetric routing
96
S-BGP
bull Address-based PKI validate signaturesndash Authentication of
bull ownership for IP address blocks bull AS number bull an ASs identity and bull a BGP routers identity
ndash Use existing infrastructure (Internet registries etc)ndash Routing origination is digitally signedndash BGP updates are digitally signed
1048708 bull Route attestations A new optional BGP transitive path attribute
ndash carries digital signatures covering the routing information in updates
97
Practical Problems with S-BGP
bull Requires Public-Key Infrastructure
bull Lots of digital signatures to calculate and verifyndash Message overheadndash CPU overhead
bull Calculation expense is greatest when topology is changingndash Caching can help
bull Route aggregation is problematic (maybe thatrsquos OK)
bull Secure route withdrawals when link or node fails
bull Address ownership data out of date
bull Deployment
- Networking
- Goal of This Tutorial
- Todayrsquos Networks
- Business Models
- Billing for Internet Usage
- Net Neutrality
- Network Operations
- Point-of-Presence (PoP)
- Example Abilene Network Topology
- Another Example Backbone
- Internet Routing Overview
- Internet Routing Protocol BGP
- Two Flavors of BGP
- IPv4 Addresses Networks of Networks
- Pre-1994 Classful Addressing
- Classless Interdomain Routing (CIDR)
- Benefits of CIDR
- Growth of IP Prefixes
- 1994-1998 Linear Growth
- Around 2000 Fast Growth Resumes
- Fast growth resumes
- The Address Allocation Process
- Common Problems
- What can go wrong
- Measurement and Monitoring
- Passive vs Active Measurement
- Slide 27
- Passive Traffic Data Measurement
- Simple Network Management Protocol
- SNMP (Passive)
- Packet-level Monitoring
- Full Packet Capture (Passive)
- What is a flow
- Cisco NetFlow
- Flow Record Contents
- Aggregating Packets into Flows
- Reducing Measurement Overhead
- Packet Sampling for Flow Monitoring
- Sampling Flow-Level Sampling
- Two Main Approaches
- Packet Capture on High-Speed Links
- Characteristics of Packet Capture
- Data Measurement Repositories
- Multihoming and Traffic Engineering
- What is Multihoming
- Why Multihome
- Redundancy
- Performance
- Multihoming in IP Networks Today
- BGP or no
- Same Provider or Multiple
- Multihomed Stub One Link
- Multihomed Stub Multiple Links
- Multihomed Stub Multiple ISPs
- Multihomed Transit Network
- Interdomain Traffic Engineering
- Outbound Traffic Control
- Outbound Traffic Load Balancing
- Traffic Engineering Goals
- Managing Scale
- Achieving Predictability
- Challenges to Predictability
- Inter-AS Negotiation
- Outbound Multihoming Goals
- Inbound Traffic Control
- How many links are enough
- Problems with Multihoming in IPv4
- Slide 68
- Slide 69
- Diagnosis and Troubleshooting
- Operator Mailing List (NANOG)
- Operator Mailing List
- Routing Configuration
- Internet Business Model (Simplified)
- Peering Contracts Consistent Export
- Consistent Export
- Inconsistent Export in Practice
- Blackholes
- Security
- Security ldquoBogonrdquo Routes
- Spam Phishing etc
- Spam and Routing
- Worms and Botnets
- What is a Worm
- Example Worm Code Red
- Code Red Revisions
- Code Red Host Infection Rate
- Designing Fast-Spreading Worms
- Botnets
- ldquoRallyingrdquo the Botnet
- Botnet Control
- Some Defenses
- Idea 1 Ingress Filtering
- Idea 2 uRPF Checks
- Problems with uRPF
- S-BGP
- Practical Problems with S-BGP
-
87
Code Red Host Infection Rate
Exponential infection rate
Measured using backscatter technique
88
Designing Fast-Spreading Worms
bull Hit-list scanningndash Time to infect first 10k hosts dominates infection timendash Solution Reconnaissance (stealthy scans etc)
bull Permutation scanningndash Observation Most scanning is redundantndash Idea Shared permutation of address space Start scanning
from own IP address Re-randomize when another infected machine is found
bull Internet-scale hit listsndash Flash worm complete infection within 30 seconds
89
Botnets
bull Bots Autonomous programs performing tasksbull Plenty of ldquobenignrdquo bots
ndash eg weatherbug
bull Botnets group of bots ndash Typically carries malicious connotationndash Large numbers of infected machinesndash Machines ldquoenlistedrdquo with infection vectors like worms
(last lecture)
bull Available for simultaneous control by a masterbull Size up to 350000 nodes (from todayrsquos paper)
90
ldquoRallyingrdquo the Botnet
bull Easy to combine worm backdoor functionalitybull Problem how to learn about successfully
infected machines
bull Optionsndash Emailndash Hard-coded email address
91
Botnet Control
bull Botnet master typically runs some IRC server on a well-known port (eg 6667)
bull Infected machine contacts botnet with pre-programmed DNS name (eg big-botde)
bull Dynamic DNS allows controller to move about freely
Infected Machine
DynamicDNS
BotnetController
(IRC server)
92
Some Defenses
93
Idea 1 Ingress Filtering
bull RFC 2827 Routers install filters to drop packets from networks that are not downstream
bull Feasible at edgesbull Difficult to configure closer to network ldquocorerdquo
20469207024 Internet
Drop all packets with source address other than 20469207024
94
Idea 2 uRPF Checks
bull Unicast Reverse Path Forwardingndash Cisco ldquoip verify unicast reverse-pathrdquo
bull Requires symmetric routing
Accept packet from interface only if forwarding table entry for source IP address matches ingress interface
100183
A10018
10015
101203
100181 24
10011 24
Strict Mode uRPF
Enabled
ldquoArdquo Routing TableDestination Next Hop1001024 Int 110018024 Int 2
100183 from wrong interface
95
Problems with uRPF
bull Asymmetric routing
96
S-BGP
bull Address-based PKI validate signaturesndash Authentication of
bull ownership for IP address blocks bull AS number bull an ASs identity and bull a BGP routers identity
ndash Use existing infrastructure (Internet registries etc)ndash Routing origination is digitally signedndash BGP updates are digitally signed
1048708 bull Route attestations A new optional BGP transitive path attribute
ndash carries digital signatures covering the routing information in updates
97
Practical Problems with S-BGP
bull Requires Public-Key Infrastructure
bull Lots of digital signatures to calculate and verifyndash Message overheadndash CPU overhead
bull Calculation expense is greatest when topology is changingndash Caching can help
bull Route aggregation is problematic (maybe thatrsquos OK)
bull Secure route withdrawals when link or node fails
bull Address ownership data out of date
bull Deployment
- Networking
- Goal of This Tutorial
- Todayrsquos Networks
- Business Models
- Billing for Internet Usage
- Net Neutrality
- Network Operations
- Point-of-Presence (PoP)
- Example Abilene Network Topology
- Another Example Backbone
- Internet Routing Overview
- Internet Routing Protocol BGP
- Two Flavors of BGP
- IPv4 Addresses Networks of Networks
- Pre-1994 Classful Addressing
- Classless Interdomain Routing (CIDR)
- Benefits of CIDR
- Growth of IP Prefixes
- 1994-1998 Linear Growth
- Around 2000 Fast Growth Resumes
- Fast growth resumes
- The Address Allocation Process
- Common Problems
- What can go wrong
- Measurement and Monitoring
- Passive vs Active Measurement
- Slide 27
- Passive Traffic Data Measurement
- Simple Network Management Protocol
- SNMP (Passive)
- Packet-level Monitoring
- Full Packet Capture (Passive)
- What is a flow
- Cisco NetFlow
- Flow Record Contents
- Aggregating Packets into Flows
- Reducing Measurement Overhead
- Packet Sampling for Flow Monitoring
- Sampling Flow-Level Sampling
- Two Main Approaches
- Packet Capture on High-Speed Links
- Characteristics of Packet Capture
- Data Measurement Repositories
- Multihoming and Traffic Engineering
- What is Multihoming
- Why Multihome
- Redundancy
- Performance
- Multihoming in IP Networks Today
- BGP or no
- Same Provider or Multiple
- Multihomed Stub One Link
- Multihomed Stub Multiple Links
- Multihomed Stub Multiple ISPs
- Multihomed Transit Network
- Interdomain Traffic Engineering
- Outbound Traffic Control
- Outbound Traffic Load Balancing
- Traffic Engineering Goals
- Managing Scale
- Achieving Predictability
- Challenges to Predictability
- Inter-AS Negotiation
- Outbound Multihoming Goals
- Inbound Traffic Control
- How many links are enough
- Problems with Multihoming in IPv4
- Slide 68
- Slide 69
- Diagnosis and Troubleshooting
- Operator Mailing List (NANOG)
- Operator Mailing List
- Routing Configuration
- Internet Business Model (Simplified)
- Peering Contracts Consistent Export
- Consistent Export
- Inconsistent Export in Practice
- Blackholes
- Security
- Security ldquoBogonrdquo Routes
- Spam Phishing etc
- Spam and Routing
- Worms and Botnets
- What is a Worm
- Example Worm Code Red
- Code Red Revisions
- Code Red Host Infection Rate
- Designing Fast-Spreading Worms
- Botnets
- ldquoRallyingrdquo the Botnet
- Botnet Control
- Some Defenses
- Idea 1 Ingress Filtering
- Idea 2 uRPF Checks
- Problems with uRPF
- S-BGP
- Practical Problems with S-BGP
-
88
Designing Fast-Spreading Worms
bull Hit-list scanningndash Time to infect first 10k hosts dominates infection timendash Solution Reconnaissance (stealthy scans etc)
bull Permutation scanningndash Observation Most scanning is redundantndash Idea Shared permutation of address space Start scanning
from own IP address Re-randomize when another infected machine is found
bull Internet-scale hit listsndash Flash worm complete infection within 30 seconds
89
Botnets
bull Bots Autonomous programs performing tasksbull Plenty of ldquobenignrdquo bots
ndash eg weatherbug
bull Botnets group of bots ndash Typically carries malicious connotationndash Large numbers of infected machinesndash Machines ldquoenlistedrdquo with infection vectors like worms
(last lecture)
bull Available for simultaneous control by a masterbull Size up to 350000 nodes (from todayrsquos paper)
90
ldquoRallyingrdquo the Botnet
bull Easy to combine worm backdoor functionalitybull Problem how to learn about successfully
infected machines
bull Optionsndash Emailndash Hard-coded email address
91
Botnet Control
bull Botnet master typically runs some IRC server on a well-known port (eg 6667)
bull Infected machine contacts botnet with pre-programmed DNS name (eg big-botde)
bull Dynamic DNS allows controller to move about freely
Infected Machine
DynamicDNS
BotnetController
(IRC server)
92
Some Defenses
93
Idea 1 Ingress Filtering
bull RFC 2827 Routers install filters to drop packets from networks that are not downstream
bull Feasible at edgesbull Difficult to configure closer to network ldquocorerdquo
20469207024 Internet
Drop all packets with source address other than 20469207024
94
Idea 2 uRPF Checks
bull Unicast Reverse Path Forwardingndash Cisco ldquoip verify unicast reverse-pathrdquo
bull Requires symmetric routing
Accept packet from interface only if forwarding table entry for source IP address matches ingress interface
100183
A10018
10015
101203
100181 24
10011 24
Strict Mode uRPF
Enabled
ldquoArdquo Routing TableDestination Next Hop1001024 Int 110018024 Int 2
100183 from wrong interface
95
Problems with uRPF
bull Asymmetric routing
96
S-BGP
bull Address-based PKI validate signaturesndash Authentication of
bull ownership for IP address blocks bull AS number bull an ASs identity and bull a BGP routers identity
ndash Use existing infrastructure (Internet registries etc)ndash Routing origination is digitally signedndash BGP updates are digitally signed
1048708 bull Route attestations A new optional BGP transitive path attribute
ndash carries digital signatures covering the routing information in updates
97
Practical Problems with S-BGP
bull Requires Public-Key Infrastructure
bull Lots of digital signatures to calculate and verifyndash Message overheadndash CPU overhead
bull Calculation expense is greatest when topology is changingndash Caching can help
bull Route aggregation is problematic (maybe thatrsquos OK)
bull Secure route withdrawals when link or node fails
bull Address ownership data out of date
bull Deployment
- Networking
- Goal of This Tutorial
- Todayrsquos Networks
- Business Models
- Billing for Internet Usage
- Net Neutrality
- Network Operations
- Point-of-Presence (PoP)
- Example Abilene Network Topology
- Another Example Backbone
- Internet Routing Overview
- Internet Routing Protocol BGP
- Two Flavors of BGP
- IPv4 Addresses Networks of Networks
- Pre-1994 Classful Addressing
- Classless Interdomain Routing (CIDR)
- Benefits of CIDR
- Growth of IP Prefixes
- 1994-1998 Linear Growth
- Around 2000 Fast Growth Resumes
- Fast growth resumes
- The Address Allocation Process
- Common Problems
- What can go wrong
- Measurement and Monitoring
- Passive vs Active Measurement
- Slide 27
- Passive Traffic Data Measurement
- Simple Network Management Protocol
- SNMP (Passive)
- Packet-level Monitoring
- Full Packet Capture (Passive)
- What is a flow
- Cisco NetFlow
- Flow Record Contents
- Aggregating Packets into Flows
- Reducing Measurement Overhead
- Packet Sampling for Flow Monitoring
- Sampling Flow-Level Sampling
- Two Main Approaches
- Packet Capture on High-Speed Links
- Characteristics of Packet Capture
- Data Measurement Repositories
- Multihoming and Traffic Engineering
- What is Multihoming
- Why Multihome
- Redundancy
- Performance
- Multihoming in IP Networks Today
- BGP or no
- Same Provider or Multiple
- Multihomed Stub One Link
- Multihomed Stub Multiple Links
- Multihomed Stub Multiple ISPs
- Multihomed Transit Network
- Interdomain Traffic Engineering
- Outbound Traffic Control
- Outbound Traffic Load Balancing
- Traffic Engineering Goals
- Managing Scale
- Achieving Predictability
- Challenges to Predictability
- Inter-AS Negotiation
- Outbound Multihoming Goals
- Inbound Traffic Control
- How many links are enough
- Problems with Multihoming in IPv4
- Slide 68
- Slide 69
- Diagnosis and Troubleshooting
- Operator Mailing List (NANOG)
- Operator Mailing List
- Routing Configuration
- Internet Business Model (Simplified)
- Peering Contracts Consistent Export
- Consistent Export
- Inconsistent Export in Practice
- Blackholes
- Security
- Security ldquoBogonrdquo Routes
- Spam Phishing etc
- Spam and Routing
- Worms and Botnets
- What is a Worm
- Example Worm Code Red
- Code Red Revisions
- Code Red Host Infection Rate
- Designing Fast-Spreading Worms
- Botnets
- ldquoRallyingrdquo the Botnet
- Botnet Control
- Some Defenses
- Idea 1 Ingress Filtering
- Idea 2 uRPF Checks
- Problems with uRPF
- S-BGP
- Practical Problems with S-BGP
-
89
Botnets
bull Bots Autonomous programs performing tasksbull Plenty of ldquobenignrdquo bots
ndash eg weatherbug
bull Botnets group of bots ndash Typically carries malicious connotationndash Large numbers of infected machinesndash Machines ldquoenlistedrdquo with infection vectors like worms
(last lecture)
bull Available for simultaneous control by a masterbull Size up to 350000 nodes (from todayrsquos paper)
90
ldquoRallyingrdquo the Botnet
bull Easy to combine worm backdoor functionalitybull Problem how to learn about successfully
infected machines
bull Optionsndash Emailndash Hard-coded email address
91
Botnet Control
bull Botnet master typically runs some IRC server on a well-known port (eg 6667)
bull Infected machine contacts botnet with pre-programmed DNS name (eg big-botde)
bull Dynamic DNS allows controller to move about freely
Infected Machine
DynamicDNS
BotnetController
(IRC server)
92
Some Defenses
93
Idea 1 Ingress Filtering
bull RFC 2827 Routers install filters to drop packets from networks that are not downstream
bull Feasible at edgesbull Difficult to configure closer to network ldquocorerdquo
20469207024 Internet
Drop all packets with source address other than 20469207024
94
Idea 2 uRPF Checks
bull Unicast Reverse Path Forwardingndash Cisco ldquoip verify unicast reverse-pathrdquo
bull Requires symmetric routing
Accept packet from interface only if forwarding table entry for source IP address matches ingress interface
100183
A10018
10015
101203
100181 24
10011 24
Strict Mode uRPF
Enabled
ldquoArdquo Routing TableDestination Next Hop1001024 Int 110018024 Int 2
100183 from wrong interface
95
Problems with uRPF
bull Asymmetric routing
96
S-BGP
bull Address-based PKI validate signaturesndash Authentication of
bull ownership for IP address blocks bull AS number bull an ASs identity and bull a BGP routers identity
ndash Use existing infrastructure (Internet registries etc)ndash Routing origination is digitally signedndash BGP updates are digitally signed
1048708 bull Route attestations A new optional BGP transitive path attribute
ndash carries digital signatures covering the routing information in updates
97
Practical Problems with S-BGP
bull Requires Public-Key Infrastructure
bull Lots of digital signatures to calculate and verifyndash Message overheadndash CPU overhead
bull Calculation expense is greatest when topology is changingndash Caching can help
bull Route aggregation is problematic (maybe thatrsquos OK)
bull Secure route withdrawals when link or node fails
bull Address ownership data out of date
bull Deployment
- Networking
- Goal of This Tutorial
- Todayrsquos Networks
- Business Models
- Billing for Internet Usage
- Net Neutrality
- Network Operations
- Point-of-Presence (PoP)
- Example Abilene Network Topology
- Another Example Backbone
- Internet Routing Overview
- Internet Routing Protocol BGP
- Two Flavors of BGP
- IPv4 Addresses Networks of Networks
- Pre-1994 Classful Addressing
- Classless Interdomain Routing (CIDR)
- Benefits of CIDR
- Growth of IP Prefixes
- 1994-1998 Linear Growth
- Around 2000 Fast Growth Resumes
- Fast growth resumes
- The Address Allocation Process
- Common Problems
- What can go wrong
- Measurement and Monitoring
- Passive vs Active Measurement
- Slide 27
- Passive Traffic Data Measurement
- Simple Network Management Protocol
- SNMP (Passive)
- Packet-level Monitoring
- Full Packet Capture (Passive)
- What is a flow
- Cisco NetFlow
- Flow Record Contents
- Aggregating Packets into Flows
- Reducing Measurement Overhead
- Packet Sampling for Flow Monitoring
- Sampling Flow-Level Sampling
- Two Main Approaches
- Packet Capture on High-Speed Links
- Characteristics of Packet Capture
- Data Measurement Repositories
- Multihoming and Traffic Engineering
- What is Multihoming
- Why Multihome
- Redundancy
- Performance
- Multihoming in IP Networks Today
- BGP or no
- Same Provider or Multiple
- Multihomed Stub One Link
- Multihomed Stub Multiple Links
- Multihomed Stub Multiple ISPs
- Multihomed Transit Network
- Interdomain Traffic Engineering
- Outbound Traffic Control
- Outbound Traffic Load Balancing
- Traffic Engineering Goals
- Managing Scale
- Achieving Predictability
- Challenges to Predictability
- Inter-AS Negotiation
- Outbound Multihoming Goals
- Inbound Traffic Control
- How many links are enough
- Problems with Multihoming in IPv4
- Slide 68
- Slide 69
- Diagnosis and Troubleshooting
- Operator Mailing List (NANOG)
- Operator Mailing List
- Routing Configuration
- Internet Business Model (Simplified)
- Peering Contracts Consistent Export
- Consistent Export
- Inconsistent Export in Practice
- Blackholes
- Security
- Security ldquoBogonrdquo Routes
- Spam Phishing etc
- Spam and Routing
- Worms and Botnets
- What is a Worm
- Example Worm Code Red
- Code Red Revisions
- Code Red Host Infection Rate
- Designing Fast-Spreading Worms
- Botnets
- ldquoRallyingrdquo the Botnet
- Botnet Control
- Some Defenses
- Idea 1 Ingress Filtering
- Idea 2 uRPF Checks
- Problems with uRPF
- S-BGP
- Practical Problems with S-BGP
-
90
ldquoRallyingrdquo the Botnet
bull Easy to combine worm backdoor functionalitybull Problem how to learn about successfully
infected machines
bull Optionsndash Emailndash Hard-coded email address
91
Botnet Control
bull Botnet master typically runs some IRC server on a well-known port (eg 6667)
bull Infected machine contacts botnet with pre-programmed DNS name (eg big-botde)
bull Dynamic DNS allows controller to move about freely
Infected Machine
DynamicDNS
BotnetController
(IRC server)
92
Some Defenses
93
Idea 1 Ingress Filtering
bull RFC 2827 Routers install filters to drop packets from networks that are not downstream
bull Feasible at edgesbull Difficult to configure closer to network ldquocorerdquo
20469207024 Internet
Drop all packets with source address other than 20469207024
94
Idea 2 uRPF Checks
bull Unicast Reverse Path Forwardingndash Cisco ldquoip verify unicast reverse-pathrdquo
bull Requires symmetric routing
Accept packet from interface only if forwarding table entry for source IP address matches ingress interface
100183
A10018
10015
101203
100181 24
10011 24
Strict Mode uRPF
Enabled
ldquoArdquo Routing TableDestination Next Hop1001024 Int 110018024 Int 2
100183 from wrong interface
95
Problems with uRPF
bull Asymmetric routing
96
S-BGP
bull Address-based PKI validate signaturesndash Authentication of
bull ownership for IP address blocks bull AS number bull an ASs identity and bull a BGP routers identity
ndash Use existing infrastructure (Internet registries etc)ndash Routing origination is digitally signedndash BGP updates are digitally signed
1048708 bull Route attestations A new optional BGP transitive path attribute
ndash carries digital signatures covering the routing information in updates
97
Practical Problems with S-BGP
bull Requires Public-Key Infrastructure
bull Lots of digital signatures to calculate and verifyndash Message overheadndash CPU overhead
bull Calculation expense is greatest when topology is changingndash Caching can help
bull Route aggregation is problematic (maybe thatrsquos OK)
bull Secure route withdrawals when link or node fails
bull Address ownership data out of date
bull Deployment
- Networking
- Goal of This Tutorial
- Todayrsquos Networks
- Business Models
- Billing for Internet Usage
- Net Neutrality
- Network Operations
- Point-of-Presence (PoP)
- Example Abilene Network Topology
- Another Example Backbone
- Internet Routing Overview
- Internet Routing Protocol BGP
- Two Flavors of BGP
- IPv4 Addresses Networks of Networks
- Pre-1994 Classful Addressing
- Classless Interdomain Routing (CIDR)
- Benefits of CIDR
- Growth of IP Prefixes
- 1994-1998 Linear Growth
- Around 2000 Fast Growth Resumes
- Fast growth resumes
- The Address Allocation Process
- Common Problems
- What can go wrong
- Measurement and Monitoring
- Passive vs Active Measurement
- Slide 27
- Passive Traffic Data Measurement
- Simple Network Management Protocol
- SNMP (Passive)
- Packet-level Monitoring
- Full Packet Capture (Passive)
- What is a flow
- Cisco NetFlow
- Flow Record Contents
- Aggregating Packets into Flows
- Reducing Measurement Overhead
- Packet Sampling for Flow Monitoring
- Sampling Flow-Level Sampling
- Two Main Approaches
- Packet Capture on High-Speed Links
- Characteristics of Packet Capture
- Data Measurement Repositories
- Multihoming and Traffic Engineering
- What is Multihoming
- Why Multihome
- Redundancy
- Performance
- Multihoming in IP Networks Today
- BGP or no
- Same Provider or Multiple
- Multihomed Stub One Link
- Multihomed Stub Multiple Links
- Multihomed Stub Multiple ISPs
- Multihomed Transit Network
- Interdomain Traffic Engineering
- Outbound Traffic Control
- Outbound Traffic Load Balancing
- Traffic Engineering Goals
- Managing Scale
- Achieving Predictability
- Challenges to Predictability
- Inter-AS Negotiation
- Outbound Multihoming Goals
- Inbound Traffic Control
- How many links are enough
- Problems with Multihoming in IPv4
- Slide 68
- Slide 69
- Diagnosis and Troubleshooting
- Operator Mailing List (NANOG)
- Operator Mailing List
- Routing Configuration
- Internet Business Model (Simplified)
- Peering Contracts Consistent Export
- Consistent Export
- Inconsistent Export in Practice
- Blackholes
- Security
- Security ldquoBogonrdquo Routes
- Spam Phishing etc
- Spam and Routing
- Worms and Botnets
- What is a Worm
- Example Worm Code Red
- Code Red Revisions
- Code Red Host Infection Rate
- Designing Fast-Spreading Worms
- Botnets
- ldquoRallyingrdquo the Botnet
- Botnet Control
- Some Defenses
- Idea 1 Ingress Filtering
- Idea 2 uRPF Checks
- Problems with uRPF
- S-BGP
- Practical Problems with S-BGP
-
91
Botnet Control
bull Botnet master typically runs some IRC server on a well-known port (eg 6667)
bull Infected machine contacts botnet with pre-programmed DNS name (eg big-botde)
bull Dynamic DNS allows controller to move about freely
Infected Machine
DynamicDNS
BotnetController
(IRC server)
92
Some Defenses
93
Idea 1 Ingress Filtering
bull RFC 2827 Routers install filters to drop packets from networks that are not downstream
bull Feasible at edgesbull Difficult to configure closer to network ldquocorerdquo
20469207024 Internet
Drop all packets with source address other than 20469207024
94
Idea 2 uRPF Checks
bull Unicast Reverse Path Forwardingndash Cisco ldquoip verify unicast reverse-pathrdquo
bull Requires symmetric routing
Accept packet from interface only if forwarding table entry for source IP address matches ingress interface
100183
A10018
10015
101203
100181 24
10011 24
Strict Mode uRPF
Enabled
ldquoArdquo Routing TableDestination Next Hop1001024 Int 110018024 Int 2
100183 from wrong interface
95
Problems with uRPF
bull Asymmetric routing
96
S-BGP
bull Address-based PKI validate signaturesndash Authentication of
bull ownership for IP address blocks bull AS number bull an ASs identity and bull a BGP routers identity
ndash Use existing infrastructure (Internet registries etc)ndash Routing origination is digitally signedndash BGP updates are digitally signed
1048708 bull Route attestations A new optional BGP transitive path attribute
ndash carries digital signatures covering the routing information in updates
97
Practical Problems with S-BGP
bull Requires Public-Key Infrastructure
bull Lots of digital signatures to calculate and verifyndash Message overheadndash CPU overhead
bull Calculation expense is greatest when topology is changingndash Caching can help
bull Route aggregation is problematic (maybe thatrsquos OK)
bull Secure route withdrawals when link or node fails
bull Address ownership data out of date
bull Deployment
- Networking
- Goal of This Tutorial
- Todayrsquos Networks
- Business Models
- Billing for Internet Usage
- Net Neutrality
- Network Operations
- Point-of-Presence (PoP)
- Example Abilene Network Topology
- Another Example Backbone
- Internet Routing Overview
- Internet Routing Protocol BGP
- Two Flavors of BGP
- IPv4 Addresses Networks of Networks
- Pre-1994 Classful Addressing
- Classless Interdomain Routing (CIDR)
- Benefits of CIDR
- Growth of IP Prefixes
- 1994-1998 Linear Growth
- Around 2000 Fast Growth Resumes
- Fast growth resumes
- The Address Allocation Process
- Common Problems
- What can go wrong
- Measurement and Monitoring
- Passive vs Active Measurement
- Slide 27
- Passive Traffic Data Measurement
- Simple Network Management Protocol
- SNMP (Passive)
- Packet-level Monitoring
- Full Packet Capture (Passive)
- What is a flow
- Cisco NetFlow
- Flow Record Contents
- Aggregating Packets into Flows
- Reducing Measurement Overhead
- Packet Sampling for Flow Monitoring
- Sampling Flow-Level Sampling
- Two Main Approaches
- Packet Capture on High-Speed Links
- Characteristics of Packet Capture
- Data Measurement Repositories
- Multihoming and Traffic Engineering
- What is Multihoming
- Why Multihome
- Redundancy
- Performance
- Multihoming in IP Networks Today
- BGP or no
- Same Provider or Multiple
- Multihomed Stub One Link
- Multihomed Stub Multiple Links
- Multihomed Stub Multiple ISPs
- Multihomed Transit Network
- Interdomain Traffic Engineering
- Outbound Traffic Control
- Outbound Traffic Load Balancing
- Traffic Engineering Goals
- Managing Scale
- Achieving Predictability
- Challenges to Predictability
- Inter-AS Negotiation
- Outbound Multihoming Goals
- Inbound Traffic Control
- How many links are enough
- Problems with Multihoming in IPv4
- Slide 68
- Slide 69
- Diagnosis and Troubleshooting
- Operator Mailing List (NANOG)
- Operator Mailing List
- Routing Configuration
- Internet Business Model (Simplified)
- Peering Contracts Consistent Export
- Consistent Export
- Inconsistent Export in Practice
- Blackholes
- Security
- Security ldquoBogonrdquo Routes
- Spam Phishing etc
- Spam and Routing
- Worms and Botnets
- What is a Worm
- Example Worm Code Red
- Code Red Revisions
- Code Red Host Infection Rate
- Designing Fast-Spreading Worms
- Botnets
- ldquoRallyingrdquo the Botnet
- Botnet Control
- Some Defenses
- Idea 1 Ingress Filtering
- Idea 2 uRPF Checks
- Problems with uRPF
- S-BGP
- Practical Problems with S-BGP
-
92
Some Defenses
93
Idea 1 Ingress Filtering
bull RFC 2827 Routers install filters to drop packets from networks that are not downstream
bull Feasible at edgesbull Difficult to configure closer to network ldquocorerdquo
20469207024 Internet
Drop all packets with source address other than 20469207024
94
Idea 2 uRPF Checks
bull Unicast Reverse Path Forwardingndash Cisco ldquoip verify unicast reverse-pathrdquo
bull Requires symmetric routing
Accept packet from interface only if forwarding table entry for source IP address matches ingress interface
100183
A10018
10015
101203
100181 24
10011 24
Strict Mode uRPF
Enabled
ldquoArdquo Routing TableDestination Next Hop1001024 Int 110018024 Int 2
100183 from wrong interface
95
Problems with uRPF
bull Asymmetric routing
96
S-BGP
bull Address-based PKI validate signaturesndash Authentication of
bull ownership for IP address blocks bull AS number bull an ASs identity and bull a BGP routers identity
ndash Use existing infrastructure (Internet registries etc)ndash Routing origination is digitally signedndash BGP updates are digitally signed
1048708 bull Route attestations A new optional BGP transitive path attribute
ndash carries digital signatures covering the routing information in updates
97
Practical Problems with S-BGP
bull Requires Public-Key Infrastructure
bull Lots of digital signatures to calculate and verifyndash Message overheadndash CPU overhead
bull Calculation expense is greatest when topology is changingndash Caching can help
bull Route aggregation is problematic (maybe thatrsquos OK)
bull Secure route withdrawals when link or node fails
bull Address ownership data out of date
bull Deployment
- Networking
- Goal of This Tutorial
- Todayrsquos Networks
- Business Models
- Billing for Internet Usage
- Net Neutrality
- Network Operations
- Point-of-Presence (PoP)
- Example Abilene Network Topology
- Another Example Backbone
- Internet Routing Overview
- Internet Routing Protocol BGP
- Two Flavors of BGP
- IPv4 Addresses Networks of Networks
- Pre-1994 Classful Addressing
- Classless Interdomain Routing (CIDR)
- Benefits of CIDR
- Growth of IP Prefixes
- 1994-1998 Linear Growth
- Around 2000 Fast Growth Resumes
- Fast growth resumes
- The Address Allocation Process
- Common Problems
- What can go wrong
- Measurement and Monitoring
- Passive vs Active Measurement
- Slide 27
- Passive Traffic Data Measurement
- Simple Network Management Protocol
- SNMP (Passive)
- Packet-level Monitoring
- Full Packet Capture (Passive)
- What is a flow
- Cisco NetFlow
- Flow Record Contents
- Aggregating Packets into Flows
- Reducing Measurement Overhead
- Packet Sampling for Flow Monitoring
- Sampling Flow-Level Sampling
- Two Main Approaches
- Packet Capture on High-Speed Links
- Characteristics of Packet Capture
- Data Measurement Repositories
- Multihoming and Traffic Engineering
- What is Multihoming
- Why Multihome
- Redundancy
- Performance
- Multihoming in IP Networks Today
- BGP or no
- Same Provider or Multiple
- Multihomed Stub One Link
- Multihomed Stub Multiple Links
- Multihomed Stub Multiple ISPs
- Multihomed Transit Network
- Interdomain Traffic Engineering
- Outbound Traffic Control
- Outbound Traffic Load Balancing
- Traffic Engineering Goals
- Managing Scale
- Achieving Predictability
- Challenges to Predictability
- Inter-AS Negotiation
- Outbound Multihoming Goals
- Inbound Traffic Control
- How many links are enough
- Problems with Multihoming in IPv4
- Slide 68
- Slide 69
- Diagnosis and Troubleshooting
- Operator Mailing List (NANOG)
- Operator Mailing List
- Routing Configuration
- Internet Business Model (Simplified)
- Peering Contracts Consistent Export
- Consistent Export
- Inconsistent Export in Practice
- Blackholes
- Security
- Security ldquoBogonrdquo Routes
- Spam Phishing etc
- Spam and Routing
- Worms and Botnets
- What is a Worm
- Example Worm Code Red
- Code Red Revisions
- Code Red Host Infection Rate
- Designing Fast-Spreading Worms
- Botnets
- ldquoRallyingrdquo the Botnet
- Botnet Control
- Some Defenses
- Idea 1 Ingress Filtering
- Idea 2 uRPF Checks
- Problems with uRPF
- S-BGP
- Practical Problems with S-BGP
-
93
Idea 1 Ingress Filtering
bull RFC 2827 Routers install filters to drop packets from networks that are not downstream
bull Feasible at edgesbull Difficult to configure closer to network ldquocorerdquo
20469207024 Internet
Drop all packets with source address other than 20469207024
94
Idea 2 uRPF Checks
bull Unicast Reverse Path Forwardingndash Cisco ldquoip verify unicast reverse-pathrdquo
bull Requires symmetric routing
Accept packet from interface only if forwarding table entry for source IP address matches ingress interface
100183
A10018
10015
101203
100181 24
10011 24
Strict Mode uRPF
Enabled
ldquoArdquo Routing TableDestination Next Hop1001024 Int 110018024 Int 2
100183 from wrong interface
95
Problems with uRPF
bull Asymmetric routing
96
S-BGP
bull Address-based PKI validate signaturesndash Authentication of
bull ownership for IP address blocks bull AS number bull an ASs identity and bull a BGP routers identity
ndash Use existing infrastructure (Internet registries etc)ndash Routing origination is digitally signedndash BGP updates are digitally signed
1048708 bull Route attestations A new optional BGP transitive path attribute
ndash carries digital signatures covering the routing information in updates
97
Practical Problems with S-BGP
bull Requires Public-Key Infrastructure
bull Lots of digital signatures to calculate and verifyndash Message overheadndash CPU overhead
bull Calculation expense is greatest when topology is changingndash Caching can help
bull Route aggregation is problematic (maybe thatrsquos OK)
bull Secure route withdrawals when link or node fails
bull Address ownership data out of date
bull Deployment
- Networking
- Goal of This Tutorial
- Todayrsquos Networks
- Business Models
- Billing for Internet Usage
- Net Neutrality
- Network Operations
- Point-of-Presence (PoP)
- Example Abilene Network Topology
- Another Example Backbone
- Internet Routing Overview
- Internet Routing Protocol BGP
- Two Flavors of BGP
- IPv4 Addresses Networks of Networks
- Pre-1994 Classful Addressing
- Classless Interdomain Routing (CIDR)
- Benefits of CIDR
- Growth of IP Prefixes
- 1994-1998 Linear Growth
- Around 2000 Fast Growth Resumes
- Fast growth resumes
- The Address Allocation Process
- Common Problems
- What can go wrong
- Measurement and Monitoring
- Passive vs Active Measurement
- Slide 27
- Passive Traffic Data Measurement
- Simple Network Management Protocol
- SNMP (Passive)
- Packet-level Monitoring
- Full Packet Capture (Passive)
- What is a flow
- Cisco NetFlow
- Flow Record Contents
- Aggregating Packets into Flows
- Reducing Measurement Overhead
- Packet Sampling for Flow Monitoring
- Sampling Flow-Level Sampling
- Two Main Approaches
- Packet Capture on High-Speed Links
- Characteristics of Packet Capture
- Data Measurement Repositories
- Multihoming and Traffic Engineering
- What is Multihoming
- Why Multihome
- Redundancy
- Performance
- Multihoming in IP Networks Today
- BGP or no
- Same Provider or Multiple
- Multihomed Stub One Link
- Multihomed Stub Multiple Links
- Multihomed Stub Multiple ISPs
- Multihomed Transit Network
- Interdomain Traffic Engineering
- Outbound Traffic Control
- Outbound Traffic Load Balancing
- Traffic Engineering Goals
- Managing Scale
- Achieving Predictability
- Challenges to Predictability
- Inter-AS Negotiation
- Outbound Multihoming Goals
- Inbound Traffic Control
- How many links are enough
- Problems with Multihoming in IPv4
- Slide 68
- Slide 69
- Diagnosis and Troubleshooting
- Operator Mailing List (NANOG)
- Operator Mailing List
- Routing Configuration
- Internet Business Model (Simplified)
- Peering Contracts Consistent Export
- Consistent Export
- Inconsistent Export in Practice
- Blackholes
- Security
- Security ldquoBogonrdquo Routes
- Spam Phishing etc
- Spam and Routing
- Worms and Botnets
- What is a Worm
- Example Worm Code Red
- Code Red Revisions
- Code Red Host Infection Rate
- Designing Fast-Spreading Worms
- Botnets
- ldquoRallyingrdquo the Botnet
- Botnet Control
- Some Defenses
- Idea 1 Ingress Filtering
- Idea 2 uRPF Checks
- Problems with uRPF
- S-BGP
- Practical Problems with S-BGP
-
94
Idea 2 uRPF Checks
bull Unicast Reverse Path Forwardingndash Cisco ldquoip verify unicast reverse-pathrdquo
bull Requires symmetric routing
Accept packet from interface only if forwarding table entry for source IP address matches ingress interface
100183
A10018
10015
101203
100181 24
10011 24
Strict Mode uRPF
Enabled
ldquoArdquo Routing TableDestination Next Hop1001024 Int 110018024 Int 2
100183 from wrong interface
95
Problems with uRPF
bull Asymmetric routing
96
S-BGP
bull Address-based PKI validate signaturesndash Authentication of
bull ownership for IP address blocks bull AS number bull an ASs identity and bull a BGP routers identity
ndash Use existing infrastructure (Internet registries etc)ndash Routing origination is digitally signedndash BGP updates are digitally signed
1048708 bull Route attestations A new optional BGP transitive path attribute
ndash carries digital signatures covering the routing information in updates
97
Practical Problems with S-BGP
bull Requires Public-Key Infrastructure
bull Lots of digital signatures to calculate and verifyndash Message overheadndash CPU overhead
bull Calculation expense is greatest when topology is changingndash Caching can help
bull Route aggregation is problematic (maybe thatrsquos OK)
bull Secure route withdrawals when link or node fails
bull Address ownership data out of date
bull Deployment
- Networking
- Goal of This Tutorial
- Todayrsquos Networks
- Business Models
- Billing for Internet Usage
- Net Neutrality
- Network Operations
- Point-of-Presence (PoP)
- Example Abilene Network Topology
- Another Example Backbone
- Internet Routing Overview
- Internet Routing Protocol BGP
- Two Flavors of BGP
- IPv4 Addresses Networks of Networks
- Pre-1994 Classful Addressing
- Classless Interdomain Routing (CIDR)
- Benefits of CIDR
- Growth of IP Prefixes
- 1994-1998 Linear Growth
- Around 2000 Fast Growth Resumes
- Fast growth resumes
- The Address Allocation Process
- Common Problems
- What can go wrong
- Measurement and Monitoring
- Passive vs Active Measurement
- Slide 27
- Passive Traffic Data Measurement
- Simple Network Management Protocol
- SNMP (Passive)
- Packet-level Monitoring
- Full Packet Capture (Passive)
- What is a flow
- Cisco NetFlow
- Flow Record Contents
- Aggregating Packets into Flows
- Reducing Measurement Overhead
- Packet Sampling for Flow Monitoring
- Sampling Flow-Level Sampling
- Two Main Approaches
- Packet Capture on High-Speed Links
- Characteristics of Packet Capture
- Data Measurement Repositories
- Multihoming and Traffic Engineering
- What is Multihoming
- Why Multihome
- Redundancy
- Performance
- Multihoming in IP Networks Today
- BGP or no
- Same Provider or Multiple
- Multihomed Stub One Link
- Multihomed Stub Multiple Links
- Multihomed Stub Multiple ISPs
- Multihomed Transit Network
- Interdomain Traffic Engineering
- Outbound Traffic Control
- Outbound Traffic Load Balancing
- Traffic Engineering Goals
- Managing Scale
- Achieving Predictability
- Challenges to Predictability
- Inter-AS Negotiation
- Outbound Multihoming Goals
- Inbound Traffic Control
- How many links are enough
- Problems with Multihoming in IPv4
- Slide 68
- Slide 69
- Diagnosis and Troubleshooting
- Operator Mailing List (NANOG)
- Operator Mailing List
- Routing Configuration
- Internet Business Model (Simplified)
- Peering Contracts Consistent Export
- Consistent Export
- Inconsistent Export in Practice
- Blackholes
- Security
- Security ldquoBogonrdquo Routes
- Spam Phishing etc
- Spam and Routing
- Worms and Botnets
- What is a Worm
- Example Worm Code Red
- Code Red Revisions
- Code Red Host Infection Rate
- Designing Fast-Spreading Worms
- Botnets
- ldquoRallyingrdquo the Botnet
- Botnet Control
- Some Defenses
- Idea 1 Ingress Filtering
- Idea 2 uRPF Checks
- Problems with uRPF
- S-BGP
- Practical Problems with S-BGP
-
95
Problems with uRPF
bull Asymmetric routing
96
S-BGP
bull Address-based PKI validate signaturesndash Authentication of
bull ownership for IP address blocks bull AS number bull an ASs identity and bull a BGP routers identity
ndash Use existing infrastructure (Internet registries etc)ndash Routing origination is digitally signedndash BGP updates are digitally signed
1048708 bull Route attestations A new optional BGP transitive path attribute
ndash carries digital signatures covering the routing information in updates
97
Practical Problems with S-BGP
bull Requires Public-Key Infrastructure
bull Lots of digital signatures to calculate and verifyndash Message overheadndash CPU overhead
bull Calculation expense is greatest when topology is changingndash Caching can help
bull Route aggregation is problematic (maybe thatrsquos OK)
bull Secure route withdrawals when link or node fails
bull Address ownership data out of date
bull Deployment
- Networking
- Goal of This Tutorial
- Todayrsquos Networks
- Business Models
- Billing for Internet Usage
- Net Neutrality
- Network Operations
- Point-of-Presence (PoP)
- Example Abilene Network Topology
- Another Example Backbone
- Internet Routing Overview
- Internet Routing Protocol BGP
- Two Flavors of BGP
- IPv4 Addresses Networks of Networks
- Pre-1994 Classful Addressing
- Classless Interdomain Routing (CIDR)
- Benefits of CIDR
- Growth of IP Prefixes
- 1994-1998 Linear Growth
- Around 2000 Fast Growth Resumes
- Fast growth resumes
- The Address Allocation Process
- Common Problems
- What can go wrong
- Measurement and Monitoring
- Passive vs Active Measurement
- Slide 27
- Passive Traffic Data Measurement
- Simple Network Management Protocol
- SNMP (Passive)
- Packet-level Monitoring
- Full Packet Capture (Passive)
- What is a flow
- Cisco NetFlow
- Flow Record Contents
- Aggregating Packets into Flows
- Reducing Measurement Overhead
- Packet Sampling for Flow Monitoring
- Sampling Flow-Level Sampling
- Two Main Approaches
- Packet Capture on High-Speed Links
- Characteristics of Packet Capture
- Data Measurement Repositories
- Multihoming and Traffic Engineering
- What is Multihoming
- Why Multihome
- Redundancy
- Performance
- Multihoming in IP Networks Today
- BGP or no
- Same Provider or Multiple
- Multihomed Stub One Link
- Multihomed Stub Multiple Links
- Multihomed Stub Multiple ISPs
- Multihomed Transit Network
- Interdomain Traffic Engineering
- Outbound Traffic Control
- Outbound Traffic Load Balancing
- Traffic Engineering Goals
- Managing Scale
- Achieving Predictability
- Challenges to Predictability
- Inter-AS Negotiation
- Outbound Multihoming Goals
- Inbound Traffic Control
- How many links are enough
- Problems with Multihoming in IPv4
- Slide 68
- Slide 69
- Diagnosis and Troubleshooting
- Operator Mailing List (NANOG)
- Operator Mailing List
- Routing Configuration
- Internet Business Model (Simplified)
- Peering Contracts Consistent Export
- Consistent Export
- Inconsistent Export in Practice
- Blackholes
- Security
- Security ldquoBogonrdquo Routes
- Spam Phishing etc
- Spam and Routing
- Worms and Botnets
- What is a Worm
- Example Worm Code Red
- Code Red Revisions
- Code Red Host Infection Rate
- Designing Fast-Spreading Worms
- Botnets
- ldquoRallyingrdquo the Botnet
- Botnet Control
- Some Defenses
- Idea 1 Ingress Filtering
- Idea 2 uRPF Checks
- Problems with uRPF
- S-BGP
- Practical Problems with S-BGP
-
96
S-BGP
bull Address-based PKI validate signaturesndash Authentication of
bull ownership for IP address blocks bull AS number bull an ASs identity and bull a BGP routers identity
ndash Use existing infrastructure (Internet registries etc)ndash Routing origination is digitally signedndash BGP updates are digitally signed
1048708 bull Route attestations A new optional BGP transitive path attribute
ndash carries digital signatures covering the routing information in updates
97
Practical Problems with S-BGP
bull Requires Public-Key Infrastructure
bull Lots of digital signatures to calculate and verifyndash Message overheadndash CPU overhead
bull Calculation expense is greatest when topology is changingndash Caching can help
bull Route aggregation is problematic (maybe thatrsquos OK)
bull Secure route withdrawals when link or node fails
bull Address ownership data out of date
bull Deployment
- Networking
- Goal of This Tutorial
- Todayrsquos Networks
- Business Models
- Billing for Internet Usage
- Net Neutrality
- Network Operations
- Point-of-Presence (PoP)
- Example Abilene Network Topology
- Another Example Backbone
- Internet Routing Overview
- Internet Routing Protocol BGP
- Two Flavors of BGP
- IPv4 Addresses Networks of Networks
- Pre-1994 Classful Addressing
- Classless Interdomain Routing (CIDR)
- Benefits of CIDR
- Growth of IP Prefixes
- 1994-1998 Linear Growth
- Around 2000 Fast Growth Resumes
- Fast growth resumes
- The Address Allocation Process
- Common Problems
- What can go wrong
- Measurement and Monitoring
- Passive vs Active Measurement
- Slide 27
- Passive Traffic Data Measurement
- Simple Network Management Protocol
- SNMP (Passive)
- Packet-level Monitoring
- Full Packet Capture (Passive)
- What is a flow
- Cisco NetFlow
- Flow Record Contents
- Aggregating Packets into Flows
- Reducing Measurement Overhead
- Packet Sampling for Flow Monitoring
- Sampling Flow-Level Sampling
- Two Main Approaches
- Packet Capture on High-Speed Links
- Characteristics of Packet Capture
- Data Measurement Repositories
- Multihoming and Traffic Engineering
- What is Multihoming
- Why Multihome
- Redundancy
- Performance
- Multihoming in IP Networks Today
- BGP or no
- Same Provider or Multiple
- Multihomed Stub One Link
- Multihomed Stub Multiple Links
- Multihomed Stub Multiple ISPs
- Multihomed Transit Network
- Interdomain Traffic Engineering
- Outbound Traffic Control
- Outbound Traffic Load Balancing
- Traffic Engineering Goals
- Managing Scale
- Achieving Predictability
- Challenges to Predictability
- Inter-AS Negotiation
- Outbound Multihoming Goals
- Inbound Traffic Control
- How many links are enough
- Problems with Multihoming in IPv4
- Slide 68
- Slide 69
- Diagnosis and Troubleshooting
- Operator Mailing List (NANOG)
- Operator Mailing List
- Routing Configuration
- Internet Business Model (Simplified)
- Peering Contracts Consistent Export
- Consistent Export
- Inconsistent Export in Practice
- Blackholes
- Security
- Security ldquoBogonrdquo Routes
- Spam Phishing etc
- Spam and Routing
- Worms and Botnets
- What is a Worm
- Example Worm Code Red
- Code Red Revisions
- Code Red Host Infection Rate
- Designing Fast-Spreading Worms
- Botnets
- ldquoRallyingrdquo the Botnet
- Botnet Control
- Some Defenses
- Idea 1 Ingress Filtering
- Idea 2 uRPF Checks
- Problems with uRPF
- S-BGP
- Practical Problems with S-BGP
-
97
Practical Problems with S-BGP
bull Requires Public-Key Infrastructure
bull Lots of digital signatures to calculate and verifyndash Message overheadndash CPU overhead
bull Calculation expense is greatest when topology is changingndash Caching can help
bull Route aggregation is problematic (maybe thatrsquos OK)
bull Secure route withdrawals when link or node fails
bull Address ownership data out of date
bull Deployment
- Networking
- Goal of This Tutorial
- Todayrsquos Networks
- Business Models
- Billing for Internet Usage
- Net Neutrality
- Network Operations
- Point-of-Presence (PoP)
- Example Abilene Network Topology
- Another Example Backbone
- Internet Routing Overview
- Internet Routing Protocol BGP
- Two Flavors of BGP
- IPv4 Addresses Networks of Networks
- Pre-1994 Classful Addressing
- Classless Interdomain Routing (CIDR)
- Benefits of CIDR
- Growth of IP Prefixes
- 1994-1998 Linear Growth
- Around 2000 Fast Growth Resumes
- Fast growth resumes
- The Address Allocation Process
- Common Problems
- What can go wrong
- Measurement and Monitoring
- Passive vs Active Measurement
- Slide 27
- Passive Traffic Data Measurement
- Simple Network Management Protocol
- SNMP (Passive)
- Packet-level Monitoring
- Full Packet Capture (Passive)
- What is a flow
- Cisco NetFlow
- Flow Record Contents
- Aggregating Packets into Flows
- Reducing Measurement Overhead
- Packet Sampling for Flow Monitoring
- Sampling Flow-Level Sampling
- Two Main Approaches
- Packet Capture on High-Speed Links
- Characteristics of Packet Capture
- Data Measurement Repositories
- Multihoming and Traffic Engineering
- What is Multihoming
- Why Multihome
- Redundancy
- Performance
- Multihoming in IP Networks Today
- BGP or no
- Same Provider or Multiple
- Multihomed Stub One Link
- Multihomed Stub Multiple Links
- Multihomed Stub Multiple ISPs
- Multihomed Transit Network
- Interdomain Traffic Engineering
- Outbound Traffic Control
- Outbound Traffic Load Balancing
- Traffic Engineering Goals
- Managing Scale
- Achieving Predictability
- Challenges to Predictability
- Inter-AS Negotiation
- Outbound Multihoming Goals
- Inbound Traffic Control
- How many links are enough
- Problems with Multihoming in IPv4
- Slide 68
- Slide 69
- Diagnosis and Troubleshooting
- Operator Mailing List (NANOG)
- Operator Mailing List
- Routing Configuration
- Internet Business Model (Simplified)
- Peering Contracts Consistent Export
- Consistent Export
- Inconsistent Export in Practice
- Blackholes
- Security
- Security ldquoBogonrdquo Routes
- Spam Phishing etc
- Spam and Routing
- Worms and Botnets
- What is a Worm
- Example Worm Code Red
- Code Red Revisions
- Code Red Host Infection Rate
- Designing Fast-Spreading Worms
- Botnets
- ldquoRallyingrdquo the Botnet
- Botnet Control
- Some Defenses
- Idea 1 Ingress Filtering
- Idea 2 uRPF Checks
- Problems with uRPF
- S-BGP
- Practical Problems with S-BGP
-