Traffic Engineering With BGP and Level3

1

© 2004-2008 Level 3 Communications, Inc. All rights reserved. The Level 3 logo is a registered service mark of Level 3 Communications, Inc. in the United States, other countries and/or both. Other trademarks or service marks herein are the property of their respective owners. Proprietary and confidential.

Traffic Engineering with BGP on the Level 3 Backbone

Version 1.8 July 18, 2008

2


Table of Contents TABLE OF CONTENTS 2

INTRODUCTION 3 CONVENTIONS 3 OVERVIEW OF PEERING 3 OVERVIEW OF THE ROUTING DECISION PROCESS 4

OUTBOUND TRAFFIC ENGINEERING 6 AS PATH 6 MULTI-EXIT DISCRIMINATOR (MED) 7 MULTI-HOMING AND BACKUP 8 LEVEL 3 COMMUNITIES 9 NO-EXPORT COMMUNITY ATTRIBUTE 11

INBOUND TRAFFIC ENGINEERING 12 CUSTOMER PREPEND 12 PREPEND/SUPPRESS TO ALL PEERS 13 PER-PEER PREPEND 14 CUSTOMER MEDS 16 LOCAL PREFERENCE COMMUNITIES 17

SUMMARY 19 OUTBOUND TRAFFIC 19 INBOUND TRAFFIC 19

GLOSSARY 20

REFERENCES 23

APPENDIX – RIPE ENTRY 24

3


Introduction This document gives guidance on using Border Gateway Protocol (BGP) for the purpose of traffic engineering on the Level 3 Network.

The purpose of this document is to provide Level 3 customers with the information they need to know in order to properly control the flow of their IP traffic.

Conventions This document contains a number of diagrams describing the interaction between networks. The two main flows of information this document is concerned with are BGP route announcements (with possible additional attributes) and the actual flow of traffic based on those announcements. The following conventions will clarify each of these flows:

Overview of Peering Because the Internet is a collection of networks, no one company or organization owns the entire Internet. As such, in order for the Internet to work, all of these networks must connect to each other directly or through other networks.

A simplified diagram of how other BGP Autonomous Systems (ASes) relate to Level 3 is shown in Figure 1. We send customer routes to the rest of the Internet and allow their traffic to transit our network.

4


Figure 1: Examples of AS connectivity (simplified)

The full Internet routing table is sent to customers. Global peers receive all Level 3 customer routes and regional peers receive only Level 3 customer routes for the region in which they peer.

Overview of the Routing Decision Process BGP can use several factors to determine where a packet will be sent. If there is a tie, the next rule down is used. The following are the most important factors used for selecting a prefix:

1. Longest match. If 212.113.0.0/19 and 212.113.1.0/24 are advertised, then the route chosen is always the most specific, whatever local preference values, AS path lengths, etc. are configured. This is why it important to filter out more-specific routes, which can lead to unusual routing decisions if they are released haphazardly. For identical prefixes, the other criteria are used.

2. Local preference. Local preference can be set to prefer some routes over others. In Level 3’s backbone, local preference has been set to prefer customer routes over peering routes. Only if the local preference is the same are other factors like AS-path length considered.

3. AS-path length. AS-path length is used conventionally as a decision process between routes with the same local preference.

4. Multi-Exit Discriminators (MEDs.) MEDs have a relatively weak influence on routing. They can be used by customers to steer traffic, and are also used to help with traffic engineering (i.e., balance transatlantic traffic.) We do not currently listen to MEDs from peers. We accept MEDs from customers by default.

5. Closest exit. Closest exit (i.e., lowest OSPF metric) is used for conventional “hot-potato” routing.

5


Figure 2: Example of how traffic can take unusual paths because of a more specific route announcement.

6


Outbound Traffic Engineering Outbound traffic from a customer to Level 3 and other providers can be affected in several ways:

1. Choosing the route with the smallest AS Path

2. Listening to Level 3 MEDs

3. Using local preference to prefer routes over a particular link

4. Creating route-maps that assign a local-pref to particular routes

AS Path This is the simplest way of steering traffic by affecting routing decisions. No special configuration is needed, and this is more or less “letting BGP do its thing.”

For multi-homed customers to Level 3 in one continent, the AS path will be the same at all interconnection points. Therefore the BGP decision process will be closest exit (“hot potato”) routing, which is simple and effective.

For customers multi-homed to more than one provider, allowing BGP to select a route based on the number of AS hops is the recommended way to make sensible routing decisions.

Figure 3: Traffic will exit towards Level 3 because the identical route has fewer hops from Level 3

7


Multi-Exit Discriminator (MED) Customers have the option of having Level 3 send MEDs to them. By listening to MEDs, the customer can send traffic to Level 3 closest to where the traffic will actually be delivered. For each prefix Level 3 sends a MED based on the internal metric to the customer. When MEDs are accepted properly, traffic will tend to flow across the customer network to the exit nearest the destination on the Level 3’s Network. This is sometimes called “cold potato” routing.

The MEDs shown are taken from the OSPF metrics on access routers in London and Amsterdam. If the customer listens to MEDs they will send all traffic to 209.244.0.0/14 across their network to the London Level 3 connection. All traffic to prefixes advertised in Amsterdam (e.g. 212.72.32.0/19) will be sent on the Amsterdam connection.

Figure 4: Customer routing based on MEDs sent by Level 3

8


Multi-homing and Backup Many times a customer is connected to Level 3 in more than one place, or the customer is connected to Level 3 and another provider for backup. In these scenarios, the customer can choose which link they would like to use for outgoing traffic by “pref-ing up” the routes on that link, or similarly by “pref-ing down” the routes on all the other links. The advantage of using local preference to determine outbound traffic flow is that all links remain active, and if for some reason there’s a problem with the primary link being used, traffic will automatically fail over to the other active links.

There are certain situations when a link is purchased for the sole purpose of being used as a backup connection to the Internet, in the event that there is a failure with the primary connection. In this situation, the backup link always needs to be available, but traffic should only flow over it when the primary link has failed. This can be accomplished by receiving the full set of routes over both primary and backup links, but using local preference to prefer the primary link over the backup link. If the primary circuit were to go down, the backup becomes the only path available and will be used. Note that this will affect outgoing traffic only.

Figure 5: Outbound customer traffic controlled by local-pref

9


Figure 6: How traffic will flow in a failure scenario when using local-pref on multi-homed links

Level 3 Communities Communities can be used to allow networks to communicate specific attributes about routes to each other. For the Level 3 Network, the customer must specifically indicate they would like to receive communities on the BGP form. If communities are exchanged, the network can then build route maps based on those communities. A route map allows the network to perform a specific action on the route (usually adjusting the local pref) based on matching the community. For example, Level 3 routes with the community 3356:2010 indicate the routes are from New York City in the US. A customer multi-homed to Level 3 would hear all routes (New York included) over both connections. The customer could chose to send New York traffic down one connection and the rest of the traffic down the other.

10


Figure 7: Outbound traffic can be controlled based on the communities Level 3 sends to the customer

The following properties are tagged at the ingress to the Level 3 Network and may be useful in setting up route maps to steer traffic:

• city of origin

• country of origin

• peer or customer route

The full list is given in the appendix.

11


No-Export Community Attribute Community attribute provides a way of grouping destinations, called communities, to which routing decisions (such as acceptance, preference, and redistribution) can be applied. Route maps are used to set the community attribute. A predefined community attribute accepted within the Level 3 Network is the no-export attribute (meaning do not advertise this route to EBGP peers). The important item to note is that this will not work when using Level 3 address space, as the larger aggregate prefixes will still be advertised by Level 3 to customers and peers. An ASN that wants to take advantage of this community must use its own address space.

The no-export community functions as seen below. AS 10753 advertises 64.16.1.0/24 to AS 3356 with the community attribute no-export. AS 3356 will propagate the route throughout AS 3356 but will not send this route to AS X (any other external AS outside of internal communication).

Figure 8: Example of BGP no-export Community Attribute

12


Inbound Traffic Engineering

Inbound traffic to a customer from Level 3 and other providers can be affected in several ways:

a) Prepending AS hops to route announcements

b) Sending MEDs to Level 3

c) Sending communities to Level 3 to have Level 3 change its local preference.

Customer Prepend Customers can traffic engineer between multiple upstream providers by prepending on a per-prefix basis. By adding AS hops to specific routes, traffic will flow through the provider who advertises the route with the fewer number of hops. This technique cannot guarantee traffic flow beyond the first upstream provider.

Figure 9: Customer prepending to steer incoming traffic

13


Prepend/suppress to all peers Prepending toward Level 3 may give sub-optimal routing from other Level 3 customers in some cases. Another approach to reducing transit traffic is to send communities to prepend or suppress toward non-customer peers. For example, if AS50505 sends community 65000:2 then the AS paths seen will be:

• Level 3 customer sees AS path “3356 50505”

• Level 3 peer sees AS path “3356 3356 3356 50505.”

The following possibilities are available: Community Effect

64960:XXX Override the effect of 65000:0 and advertise normally to peer XXX

65000:0 Do not advertise this prefix to any non-customer peers

65000:1 Prepend once to non-customer peers

65000:2 Prepend twice to non-customer peers

65000:3 Prepend three times to non-customer peers

65000:4 Prepend four times to non-customer peers

The use of community 64960:XXX is as shown below. Although prefix advertisements are suppressed toward all other non-customer peers, the use of 64960:XXX will mean that the prefix is advertised to peer XXX by Level 3. This feature should be used with caution.

14


Figure 10: Remove route suppression towards a single peer

There are also regional communities: 6498x:0 and 6499x:0 (x = 0,1,2,3,4). These will prepend or suppress to most non-customer peers in Europe or North America. The communities do not affect advertisement to global peers (e.g. Sprint, C&W, etc.).

Per-Peer Prepend Some customers multi-homed to several providers may have special requirements that cannot be met by the above techniques. For detailed traffic control, Level 3 also defines customer-sent communities that steer incoming traffic on a per-peer basis. Prefixes marked with these communities are suppressed or prepended on the peering sessions with individually chosen peers as follows:

Community Effect

65000:44444 Do not advertise this prefix to AS44444

65001:44444 Prepend once to AS44444

65002:44444 Prepend twice to AS44444

65003:44444 Prepend three times to AS44444

65004:44444 Prepend four times to AS44444

15


There are a number of points to bear in mind when using these communities:

• If AS44444 is a Level 3 customer (rather than a peer) these communities cannot be used

• If AS44444 does not directly peer with Level 3 then these communities will have no effect

• If the target AS also receives Level 3 routes via another AS then traffic engineering may become more complicated. For example, we interconnect with UUnet in the US (AS701) and Europe (AS702). In this case it would be necessary to label routes with two communities - 65001:701 and 65001:702 - to ensure sensible routing

• The prepend is applied to all interconnection points with that peer equally, i.e. prepends must be applied at Amsterdam and London (not just Amsterdam)

Note that this is a powerful feature with the potential to cause unusual traffic paths. It can cause traffic to take inefficient routes which could cause it to traverse extra networks. This could introduce unwanted amounts of latency and possibly even packet loss. It should be used with great care to avoid these undesired effects.

An example of per-peer prepending is shown below. The customer is sending out a single prefix and is trying to balance traffic across two upstream providers. By selectively prepending to AS50505 the customer can increase the relative proportion of incoming traffic from AS30303.

Figure 11: Per-peer prepend targeted at AS50505

16


A more detailed list is given in the Appendix. The above communities are unlikely to be used much. They may potentially be useful when the same target AS peers with Level 3 in two or three continents, but there are only a very small number of peers where this is the case.

Customer MEDs Level 3 listens to customer MEDs by default. A customer multi-homed to Level 3 can set MEDs individually for each prefix at each connection point. This allows traffic to flow in a “cold potato” scenario, where traffic is pulled across the Level 3 network to the lowest MED exit for that prefix. The figure below gives an example of applying MEDs at two customer connections.

Figure 12: Customer steers incoming traffic by sending MEDs

Tagging the same prefix with MEDs at one interconnection point but not all interconnection points is inconsistent and not recommended. The current implementations of BGP prefer routes without MEDs over those that have a value set. This is not a defined standard, and as new versions of router code are released, this rule may change.

17


Therefore if MEDs are applied to the customer’s outgoing routes, every route must have a MED set at each entry point into the Level 3 Network.

MEDs are only compared from the same AS. Therefore they have no effect on upstream connections to two different service providers.

There are however specific scenarios where MEDs will not work as expected. The Level 3 IP Network has been physically deployed at both core sites and remote sites. A flat IGP is running between all these IP nodes however BGP route-reflection was split into two categories: top level route reflector (all core gateway sites) and route reflector client (non-core gateway sites and remote sites).

As a result a penalty has been added to the internal metric between gateway and remote to prevent traffic transiting through the latter under failure of the link between the two upstream gateways.

By the nature of the hierarchical route reflector topology, with in mind BGP deterministic-med enabled by default on all Level 3 routers, a routing loop may occur if two customers of Level 3 are to announce the same prefix with equal AS-path length – typically from a common multi-homed end-user - but different MED value at both gateway and remote sites.

Careful consideration should therefore be taken when applying MED settings to dual-homed customers which are already customers of Level 3 at diverse sites. It is recommended to engage the Level 3 NOC ([email protected]) and account team to verify a specific MEDs implementation will steer traffic as expected.

Local preference communities Local preference is an extremely powerful tool in BGP, as it is the first criterion used to choose between two identical prefixes. For this reason it must be used very carefully. Typically it is not necessary to set any local preference communities, as BGP will choose the most efficient route. The following communities allow customers to set local preference: Community Effect

3356:70 set local preference 70



A simple application of these communities is to steer incoming traffic by marking prefixes on backup links with community 3356:90 – preferred links will be at the default local preference of 100.

18


Figure 13: Local preference to steer incoming traffic

There is no particular advantage in using local preference in this case. MEDs are the recommended way of steering incoming traffic.

If customers peer in Europe and have a transit connection in North America then local preference is adjusted by default in Level 3. Broadly speaking, routes from customers are set to a local preference of 100 and peer routes are set to a local preference of 86, but in some special cases these values are reduced when moving from one AS to another. The “peer in Europe, customer in US” case is dealt with by reducing the local preference of US-originated customer routes down to 86 within the European part of Level 3’s network – i.e. making them the same local preference as European peer prefixes.

19


Summary

Outbound Traffic Scenario Desire Solution

Multi-homed to Level 3 and multiple providers

Allow traffic to flow in the most efficient manner

Let BGP choose the route based on AS hops

Multi-homed to Level 3 Cold potato route on customer network

Request and then listen to Level 3 MEDs


Control which link traffic exits and allow for a backup link

Use local-preference to “prefer” one link over the others

Multi-homed to Level 3 Control how traffic exits based on Level 3 specific attributes

Request and then use Level 3 communities to local-pref prefixes from specific links

Inbound Traffic

Scenario Desire Solution


Control which link traffic enters the network

Prepend AS hops of the prefix on links facing providers it is NOT desired to receive traffic


Control traffic from Level 3 peers

Send Level 3 communities to prepend prefixes to certain peers

Multi-homed to Level 3 Control which links traffic enters the network

Send MEDs to Level 3

20


Glossary

Acronym/Term Definition

AS Autonomous System—a set of routers with a single routing policy, running under a single technical administration

ASN Autonomous System Number—a two-byte number that uniquely identifies an AS

BGPv4 Border Gateway Protocol Version 4—regulates traffic between Autonomous Systems to provide loop-free connectivity

CIDR Classless Inter-Domain Routing. Replaced the A, B, and C (“Class” network terms indicating how many hosts in a network require unique IP address assignment for routing across the Internet) address assignments/allocations designation

CPE Customer Premises Equipment

Community An attribute added to BGP that is used to identify a route as belonging to a category of routes, all of which are treated the same with respect to a connection identifier

Customer BGP Questionnaire Level 3 information form required before service activation

EGP External Gateway Protocol—a broad type of routing protocol used to exchange routing information between ASes

E-BGP External BGP—a use of BGP for communicating routing information between routers in different Ases

Gbps Gigabits per second

IGP Internal Gateway Protocol—a broad type of routing protocol used between routers within the same AS

I-BGP Internal BGP—a use of BGP between routers within the same AS used to distribute routes within the AS that were learned from other sources (e.g. E-BGP)

IP Internet Protocol

IP Address Unique 32-bit number for specific TCP/IP host on the Internet

IP Address Block A group of numbers assigned to a domain by the Internet’s Central Registry

21



ISP Internet Service Provider

LAN Local Area Network

Level 3 BGP Policy Level 3 minimum requirements for customers to run BGP

Local Preference A BGP attribute that allows a router to prefer one prefix announcement over the announcement of the same prefix from a different location

Longest Match The process of finding an entry in a forwarding or routing table associated with a particular address so that the entry matches more bits in the destination address than any other entry

Mbps Megabits per second

Metrics (also cost) Values applied to routes and/or links in a routing protocol that are used to select the best route or path

Multi-homing A host that has multiple interfaces and is connected to more than one network, or a network that is connected to more than one ISP

Next Hop The next node to which a packet should be sent in order to advance the packet closer to the destination

PING Packet Internet Groper—used to test reachability of destinations

Prepend The act of adding additional AS hops to a prefix to make that route less desirable

Policy Routing The ability of a router (and an AS) to control the routes that it accepts from and advertises to other routers (and ASes) as well as the ability to modify the attributes associated with the routes accepted and advertised

Prefix A group of contiguous bits, from 0 to 32 bits in length, that defines a set of addresses

Route Coloring The act of applying a BGP community attribute with a particular value

Route Map A filter placed on a BGP session to change attributes of specific prefixes

Routing Mesh Network architecture in which each node has dedicated connection to all other nodes

RPSL Routing Policy Specification Language

22



Traceroute A widely used utility to evaluate the forward path a packet traverses

WAN Wide Area Network

23


References

Stewart III, John W. “BGP 4 Inter-Domain Routing in the Internet.” Addison-Wesley 1999.

Chen, E. and Bates, T. “An Application of the BGP Community Attribute in Multi-home Routing,” RFC 1998, August 1996.

Cisco Systems: “Using the Border Gateway Protocol for Interdomain Routing.”

http://www.cisco.com/univercd/cc/td/doc/cisintwk/ics/icsbgp4.htm#xtocid276510

[BGP FAQ: http://www.cisco.com/warp/public/459/bgpfaq_5816.pdf ]

Rekhter, Y. and Li, T. “A Border Gateway Protocol 4 (BGP-4)” RFC 1771, March 1995.

24


Appendix – RIPE entry

The RIPE entry for AS3356 is shown below. The latest version is available at http://www.ripe.net/perl/whois?AS3356

remarks: ======================================================== remarks: remarks: Operational issues to [email protected] remarks: Abuse reports to [email protected] remarks: Peering contact is [email protected] remarks: remarks: ======================================================== remarks: remarks: Level 3 does not allow any part of 4.0.0.0/8 to be remarks: multihomed. remarks: remarks: If you are a customer who is currently using address remarks: space from 4.0.0.0/8 and if you need to multihome your remarks: network to another provider, please contact Level 3 for remarks: new address space which can be multihomed. remarks: remarks: If you are a provider who has been asked to route a remarks: more-specific network block that is part of 4.0.0.0/8, remarks: please ask your customer to contact Level 3 to obtain remarks: a new network block. remarks: remarks: Level 3 will not accept advertisements for any networks remarks: that are part of 4.0.0.0/8 from any non-customer peer. remarks: remarks: Note that the import/export designations above are a remarks: simplification of our actual policies which cannot remarks: be properly described given the limitations of RPSL remarks: They would also be rather tedious to read if we remarks: denoted all of the common policy bits applied to every remarks: peer, so we have only noted the peer-specific bits. remarks: remarks: We have also only documented our customer peers in this remarks: object at this time. remarks: remarks: The following import actions are common to every remarks: Level 3 customer peering session: remarks: remarks: - RFC1918 and other reserved networks and subnets are remarks: not permitted. remarks: remarks: - Advertisements with reserved ASes in the path remarks: (ie 64512 - 65535) are not permitted. remarks: remarks: - Prefixes shorter than /8 are not permitted. remarks: remarks: - Advertisements containing the AS of a network with

25


remarks: which we have a non-customer peering relationship remarks: are not permitted (ie customers are not allowed remarks: to advertise transit for Sprint, AT&T, etc). remarks: remarks: - Advertisements tagged with our own "internal use only" remarks: communities (ie the city/country/region/etc communities remarks: outlined below) will have all of their communities remarks: stripped from them at ingress, and any communities remarks: meant to affect localpref, prepending, etc are thus remarks: ignored. remarks: remarks: - All customer peering sessions are prefix-filtered remarks: inbound using a customer specified import policy remarks: pulled from any known public registry. remarks: remarks: - Prefixes not matching the per-peer import policy remarks: as documented above are not permitted. remarks: remarks: - Localpref will be set to 100 by default, subject remarks: to modification based on received communities as remarks: outlined below. remarks: remarks: - A hard limit is placed on the number of routes a peer remarks: is allowed to announce to us. This number is based remarks: on their registered routes plus a bit of extra remarks: overhead. remarks: remarks: The following import actions are common to every remarks: Level 3 non-customer peering session: remarks: remarks: - RFC1918 and other reserved networks and subnets are remarks: not permitted. remarks: remarks: - Advertisements with reserved ASes in the path remarks: (ie 64512 - 65535) are not permitted. remarks: remarks: - Prefixes shorter than /8 or longer than /24 are remarks: not permitted. remarks: remarks: - Any subnet of any Level 3 CIDR block (as documented remarks: in rs-Level3) is not accepted unless routing for the remarks: subnet in question has been explicitly requested by remarks: the multihomed customer in question. If you are a remarks: customer wishing to multihome a Level 3 CIDR block remarks: please contact [email protected] remarks: remarks: - Advertisements tagged with our own "internal use" remarks: or "customer use" communities (as outlined below) remarks: will have all of their communities stripped from them remarks: at ingress. remarks: remarks: - MEDs are overwritten unless special arrangements remarks: have been made. remarks: remarks: - Peers who register their routes with meaningful remarks: policies may have a prefix filter applied based on

26


remarks: this policy. remarks: remarks: - Localpref for non-customer peers is generally set remarks: to 86. remarks: remarks: - A hard limit is placed on the number of routes a peer remarks: is allowed to announce to us. This number is based remarks: on their registered routes or on a historical view of remarks: the number of routes they have been advertising plus remarks: a bit of extra overhead. remarks: remarks: The following export actions are common to every remarks: Level 3 peering session: remarks: remarks: - RFC1918 and other reserved networks and subnets are remarks: not announced. remarks: remarks: - Advertisements with reserved ASes in the path remarks: (ie 64512 - 65535) are not announced. remarks: remarks: - Prefixes shorter than /8 or longer than /24 are remarks: not announced. remarks: remarks: - Any subnet of any Level 3 CIDR block (as documented remarks: in rs-Level3) is not announced unless routing for the remarks: subnet in question has been explicitly requested by remarks: the multihomed customer in question. remarks: remarks: - Suppression and prepend actions as outlined in the remarks: community list below are taken for announcements to remarks: non-customer peers. remarks: remarks: ======================================================== remarks: remarks: Communities applied at ingress remarks: remarks: ======================================================== remarks: remarks: -------------------------------------------------------- remarks: prefix type communities remarks: -------------------------------------------------------- remarks: 3356:123 - Customer route remarks: 3356:666 - Peer route remarks: -------------------------------------------------------- remarks: error type communities remarks: -------------------------------------------------------- remarks: 3356:911 - "internal use" communities received from peer remarks: -------------------------------------------------------- remarks: city communities (some cities not listed as they home off remarks: one of the below) remarks: -------------------------------------------------------- remarks: 3356:2001 - CHI1 - Chicago remarks: 3356:2002 - SDG1 - San Diego remarks: 3356:2003 - LAX1 - Los Angeles remarks: 3356:2004 - DEN1 - Denver remarks: 3356:2005 - PHI1 - Philadelphia

27


remarks: 3356:2006 - WDC1 - Washington DC remarks: 3356:2007 - DET1 - Detroit remarks: 3356:2008 - DAL1 - Dallas remarks: 3356:2009 - SFO1 - San Francisco remarks: 3356:2010 - NYC1 - New York City remarks: 3356:2011 - SJO1 - San Jose remarks: 3356:2012 - SEA1 - Seattle remarks: 3356:2013 - ATL1 - Atlanta remarks: 3356:2014 - HOU1 - Houston remarks: 3356:2015 - BOS1 - Boston remarks: 3356:2016 - MIN1 - Minneapolis remarks: 3356:2017 - HON1 - Honolulu remarks: 3356:2018 - WEE1 - Weehawken remarks: 3356:2019 - BAL1 - Baltimore remarks: 3356:2020 - CIN1 - Cincinnati remarks: 3356:2021 - STM1 - Stamford remarks: 3356:2022 - MIA1 - Miami remarks: 3356:2023 - TMP1 - Tampa remarks: 3356:2024 - ORL1 - Orlando remarks: 3356:2025 - STL1 - St Louis remarks: 3356:2026 - PHX1 - Phoenix remarks: 3356:2027 - RCH1 - Richmond remarks: 3356:2028 - MEM1 - Memphis remarks: 3356:2029 - SLC1 - Salt Lake City remarks: 3356:2030 - SAC1 - Sacramento remarks: 3356:2031 - RAL1 - Raleigh remarks: 3356:2032 - SAT1 - San Antonio remarks: 3356:2033 - LVG1 - Las Vegas remarks: 3356:2034 - CLT1 - Charlotte remarks: 3356:2035 - CLE1 - Cleveland remarks: 3356:2036 - OAK1 - Oakland remarks: 3356:2037 - NVL1 - Nashville remarks: 3356:2038 - TUS1 - Tustin remarks: 3356:2039 - NWR1 - Newark remarks: 3356:2040 - PIT1 - Pittsburgh remarks: 3356:2041 - KAC1 - Kansas City remarks: 3356:2042 - CHI2 - Chicago 2 remarks: 3356:2043 - NYC2 - New York City 2 remarks: 3356:2064 - LON1 - London remarks: 3356:2065 - FRF1 - Frankfurt remarks: 3356:2066 - PAR1 - Paris remarks: 3356:2067 - AMS1 - Amsterdam remarks: 3356:2068 - BRU1 - Brussels remarks: 3356:2069 - DUS1 - Dusseldorf remarks: 3356:2070 - HAM1 - Hamburg remarks: 3356:2071 - BER1 - Berlin remarks: 3356:2072 - MUN1 - Munich remarks: 3356:2073 - LON2 - London 2 remarks: 3356:2074 - MLN1 - Milan remarks: 3356:2075 - ZUR1 - Zurich remarks: 3356:2076 - STK1 - Stockholm remarks: 3356:2077 - MAD1 - Madrid remarks: 3356:2078 - GEN1 - Geneva remarks: 3356:2079 - MAN1 - Manchester remarks: 3356:2080 - DUB1 - Dublin remarks: 3356:2081 - COP1 - Copenhagen

28


remarks: 3356:2082 - VIE1 - Vienna remarks: 3356:2083 - PRG1 - Prague remarks: 3356:2084 - WAW1 - Warsaw remarks: -------------------------------------------------------- remarks: country communities remarks: -------------------------------------------------------- remarks: 3356:500 - UK remarks: 3356:501 - Germany remarks: 3356:502 - France remarks: 3356:503 - Netherlands remarks: 3356:504 - Belgium remarks: 3356:505 - Italy remarks: 3356:506 - Switzerland remarks: 3356:507 - Sweden remarks: 3356:508 - Spain remarks: 3356:509 - Ireland remarks: 3356:510 - Denmark remarks: 3356:511 - Austria remarks: 3356:512 - Czech Republic remarks: 3356:513 - Poland remarks: 3356:575 - USA remarks: 3356:576 - Canada remarks: -------------------------------------------------------- remarks: regional communities remarks: -------------------------------------------------------- remarks: 3356:2 - Europe remarks: 3356:3 - North America remarks: remarks: ======================================================== remarks: remarks: Communities accepted from customers remarks: remarks: ======================================================== remarks: remarks: -------------------------------------------------------- remarks: customer traffic engineering (TE) notes remarks: -------------------------------------------------------- remarks: Communities allow suppress or prepend to peer AS, where remarks: - peer AS has a peering connection to Level 3 remarks: - peer AS is not a customer of Level 3 remarks: -------------------------------------------------------- remarks: customer traffic engineering communities - Suppression remarks: -------------------------------------------------------- remarks: 64960:XXX - announce to AS XXX if 65000:0 remarks: 65000:0 - announce to customers but not to peers remarks: 65000:XXX - do not announce at peerings to AS XXX remarks: -------------------------------------------------------- remarks: customer traffic engineering communities - Prepending remarks: -------------------------------------------------------- remarks: 65001:0 - prepend once to all peers remarks: 65001:XXX - prepend once at peerings to AS XXX remarks: 65002:0 - prepend twice to all peers remarks: 65002:XXX - prepend twice at peerings to AS XXX remarks: 65003:0 - prepend 3x to all peers remarks: 65003:XXX - prepend 3x at peerings to AS XXX remarks: 65004:0 - prepend 4x to all peers

29


remarks: 65004:XXX - prepend 4x at peerings to AS XXX remarks: -------------------------------------------------------- remarks: customer traffic engineering communities - Regional remarks: -------------------------------------------------------- remarks: Will only work for regional peers remarks: 64980:0 - announce to customers but not to EU peers remarks: 64981:0 - prepend once to all EU peers remarks: 64982:0 - prepend twice to all EU peers remarks: 64983:0 - prepend 3x to all EU peers remarks: 64984:0 - prepend 4x to all EU peers remarks: -------------------------------------------------------- remarks: customer traffic engineering communities - LocalPref remarks: -------------------------------------------------------- remarks: 3356:70 - set local preference to 70 remarks: 3356:80 - set local preference to 80 remarks: 3356:90 - set local preference to 90 remarks: -------------------------------------------------------- remarks: customer traffic engineering communities - Blackhole remarks: -------------------------------------------------------- remarks: 3356:9999 - blackhole (discard) traffic remarks: remarks: Traffic destined for any prefixes tagged with this remarks: community will be discarded at ingress to the Level 3 remarks: network. The prefix must be one permitted by the remarks: customer's existing ingress BGP filter. remarks: [email protected] may need to be contacted to allow remarks: in some cases. For some router vendors the peering remarks: must be changed to an eBGP multihop session on the Level remarks: 3 side of the connection. remarks: remarks: Please contact [email protected] with any questions remarks: regarding this functionality. remarks: ---------------------------------------------------------

Traffic Engineering With BGP and Level3

Documents

Transcript of Traffic Engineering With BGP and Level3