NOC

Lessons LearnedTEIN2 and CERNET

Xing Li

2007-01-22

NOC

Outline

• Introduction

• TEIN2 routing policy

• CERNET BGP Experience

• Lessons learned

NOC

Simple Case (where BGP can handle things easily)

• Global transit – To tier 1 or tier 2 commodity networks

• Care the aggregation • Care the load balancing • Don’t care the symmetry

• Peering (no transit, except for the down streams)– To domestic ISPs (bi-literal or via IX)

• Care the business model

– To academic partners• Care the performance• Care the symmetry

NOC

Complicated Case (where BGP cannot handle things easily)

• Global transit – To tier 1 or tier 2 commodity networks

• Care the aggregation • Care the load balancing • Don’t care the symmetry

• Academic transit– To multiple transit backbones within academic scope

• Care the aggregation • Care the load balancing • Care the performance• Care the symmetry• Etc.

• Peering (no transit, except for the down streams)– To domestic ISPs (bi-literal or via IX)

• Care the business model– To academic partners

• Care the performance• Care the symmetry

NOC

Two Steps to Implement the Policy

• Identification– IP prefix– AS path regular expression– Community tag

• Path selection– AS path (inbound and outbound)– Local-preference (outbound)– More specific (inbound)

NOC

For Transit NetworkTEIN2 Example

NOC

TEIN2 Topology

NOC

The Principle of Routing Design for the TEIN2 network

• To provide interconnection among TEIN2 partners and between TEIN2 partners and EU NRENs.

• To provide back-up paths within the TEIN2 network and/or via partner networks for service resilience when possible.

• To provide a flexible and transparent routing policy to TEIN2 NRENs.

• To avoid being selected by GÉANT, Abilene and other R&E networks outside TEIN2 as the preferred transit network.

• To minimize the adjustment of the external peers’ routing policy outside TEIN2 networks, e.g. GÉANT and APAN.

NOC

TEIN2 Routing Policy

• Enable additive community tagging to mark the prefix announcements.

• Adopt AS number prepending as the preferred BGP policy for TEIN2 traffic adjustment within TEIN2 backbone.– Use ingress AS number prepending for outbound

traffic adjustment, including traffic from TEIN2 POP to NRENs, GÉANT and APAN.

– Use egress AS number prepending for inbound traffic adjustment, including traffic from NRENs, GÉANT and APAN to TEIN2 POP.

• May use Local-Preference amendment as the last resort of mechanism for fine tuning on TEIN2 traffic over the backbone.

NOC

For NRNCERNET Example

NOC

CERNET Topology

NOC

CERNET Peering

CERNET

TEIN2

STARLIGHT

HARNET

ASNET

DRAGONLIGHT CNGI-BJIX

DRAGONTAP

CERNET 2Internet

Domestic Peering

CNGI Peering

APAN KOREN

3 G

12G

155M

100M 1G 155M 2x155M 622M

10G

NOC

CERNET Routing Policy

• Outbound– Use AS number prepending if possible– Heavily use Local-Preference– Enable additive community tagging to mark

the prefixes

• Inbound– Use AS number prepending if possible– Announce more specifics– Enable additive community tagging to mark

the prefixes

NOC

Case 1TAIWAN Earthquake

NOC

Earthquake on 26th DEC 2006

NOC

Why did not include this policy before the earthquake?

NOC

Case 2Routing and End-to-end

performance

NOC

Ping and dvping beacons

NOC

Here in the APAN venue WLAN

NOC

Lessons Learned (1)• The nature of BGP is reachability

– Stupid routing happen– Policy based routing makes thing very complicated– The routing and topology are very dynamic environment

• The key words are: simple, open and controllability – For transit network

• Simple• Open

– For NRN• Simple• Controllability

• Why did not include this policy before the earthquake?– Because it is a NP problem and there are many contradict

requirements– Mission impossible– What should be the solution?

NOC

Lessons Learned (2)

• It seems that we still need to do a lot manual BGP policy adjustment, case by case with the help of– Multi-site collaborations– Routeviews

• We have to compare the routing table with the end-to-end performance matrix– dvping tool