Best Practices for ISPs

By Ahmer Ghazi, Sysnet

Introduction

• POP Hierarchy• Scaling IGP

– Not running IGP with the customer– No customer routes in IGP– No Redistribution from BGP to IGP– IGP Summarization– Partitioning into areas

• Scaling BGP– Aggregation– Filtering Policy– Route Reflector Design

Other POPsOther POPs

Peering ISPsPeering ISPs

CustomersCustomers

Secondary POPSecondary POPPrimary POPPrimary POP

DistributionDistribution

CoreCore

AccessAccess

POP Hierarchy

Scaling IGP

• Never run IGP with the customer– Increases the database size and difficult to control– Routing instability within the customers’ network will

be propagated to the ISP.

• If non-BGP customer, ISP should put static routes for customers address blocks.

• Customer static routes should not be redistributed into IGP.

• Redistribute customer static routes directly into BGP; send towards core using iBGP.

Scaling IGP (Managing Edge links)

– Edge links, if required should be summarized at the AS Boundary router

– Edge link IP addresses should be assigned from contiguous address block

– Externals create Type-5 LSA in OSPF and flooded everywhere

– If not summarized at ASBR, externals cannot be summarized at ABR

Scaling IGP (No Redistribution from BGP to IGP)

• Goal of the IGP design should be to minimize the number of prefixes and only carry the BGP Next-Hops.

• Never redistribute BGP into IGP

• Try to bring customers in stub area; that way redistribution (even if configured) won’t work.

Scaling IGP (Summarization)

• IGP Summarization at POP boundary

• Manageable address allocation– Edge Links– Customer addresses– Internal (to the POP) addresses

• No backhaul connections by-passing the core

• Not summarizing Loopbacks

Scaling IGP (Area Partitioning)

• Scalable but complex

• Less Load on routers

• Instability is hidden

BGP Prefix Aggregation

• Aggregation at Multiple points• Following is captured from a router server.• 202.125.128.0 17233 7018 1 7473 17557 ?• 202.125.159.0 17233 7018 5400 17557 17557

17557 17557 i• AS 17557 owns 202.125.128.0/19• Engineer traffic by advertising different MEDs.• Advertise more specifics with “no-export”

community, in addition to the aggregate.

Filtering Policy

• Filter to prevent the injection of invalid routes into global tables.

• Prefixes should not be accepted as smaller blocks.

• ISP should only accept prefixes originated from the customers’ AS and those for which the customer is offering transit services.

• Avoid becoming an unintended transit.

Route Reflectors

• Route reflector• Client• Non-client• Cluster• Cluster ID• Cluster List• Originator-ID• Hierarchical RRs• Normal BGP peer

RRRR

RRRR

RRRR

RRCRRC

RRCRRC

Route Reflector

• RR advertises iBGP learned routes from its clients to other iBGP routers.

• A Cluster consists of RR + Clients; Cluster, Cluster-ID, Cluster List

• RR does not modify any of the BGP attributes. • Route Reflector might not come in the actual

traffic path.• The RR discards the route if it sees its own

cluster-id in the cluster list

Multiple RR

• Two RRs per POP

RR1 RR2

RRCRRC

RRC

RR Designs

• Follow physical topology• Larger POPs can have two levels of route

reflection. • Core (RR), Distribution (RR & Client), Edge

routers (Client) • RR of smaller POPs will be clients of the top

level RR of the larger POPs.• All the RR should either be iBGP fully meshed or

connected to another layer of RR.

Other POPsOther POPs

Peering ISPsPeering ISPs

CustomersCustomers

Secondary POPSecondary POPPrimary POPPrimary POP

RRRR RRRR

RR/RRCRR/RRCRR/RRCRR/RRC

RRCRRCRRCRRC

RRRR RRRR

RRCRRC RRCRRC RRCRRC

DistributionDistribution

CoreCore

AccessAccess

Hierarchical RR

RRCRRC RRCRRC

Summary

• Scaling IGP– Not running IGP with the customer– No customer routes in IGP– No Redistribution from BGP to IGP– IGP Summarization– Partitioning into areas

• Scaling BGP– Prefix Aggregation– Filtering Policy– Route Reflector Design