NX-OS Multicast Design and Best...

© 2012 Cisco and/or its affiliates. All rights reserved. BRKIPM-3062 Cisco Public

NX-OS Multicast Design and Recommended

Practices BRKIPM-3062

Ron Fuller– CCIE #5851 (R&S/Storage) Technical Marketing Engineer, Nexus 7000 [email protected]


Housekeeping

We value your feedback- don't forget to complete your online session

evaluations after each session & the Overall Conference Evaluation which

will be available online from Thursday

Visit the World of Solutions and Meet the Engineer

Visit the Cisco Store to purchase your recommended readings

After the event don’t forget to visit Cisco Live Virtual:

www.ciscolivevirtual.com

Please switch off your mobile phones

Follow us on Twitter for real time updates of the event:

@ciscolive

3

http://www.ciscolivevirtual.com/

http://www.ciscolivevirtual.com/


Course Objective

What you will learn…..

NX-OS Multicast Features – Platform Independent

Platform Specific Information

Recommended Practices with NX-OS

Summary

4


Technical Breakout Sessions

Session-ID Session Name

BRKARC-3470 Cisco Nexus 7000 Switch Architecture

BRKARC-3452 Cisco Nexus 5000/5500 and 2000 Switch Architecture

BRKARC-3471 Cisco NX-OS Software Architecture

BRKARC-3472 Cisco NX-OS Routing and Layer 3 Switching

BRKDCT-2121 Virtual Device Context (VDC) Design and Implementation Considerations with

Nexus 7000

TECRST-3190 Advanced IP Routing Fast Convergence

BRKIPM-3062 Nexus Multicast Design Best Practices

TECDCT-3297 Operating and Deploying NX-OS Nexus Devices in the Network Infrastructure

TECVIR-2003 Enterprise Network Virtualization

Related Cisco Live 2012 Events

5


Agenda

NX-OS Multicast Features – Platform Independent

Nexus 7000 Platform Specifics




Summary

Q&A

6

NX-OS Multicast Features – Platform

Independent


NX-OS Multicast Architecture

NX-OS is a modular operating system

‒ Modularity helps with high availability, resource allocation and scale

Some software components are always loaded

‒ Others may be conditional

Modularity includes multicast components

Multicast in NX-OS is ―VRF-Aware‖

NX-OS has unique features which can change traditional multicast models

NX-OS does not support PIM Dense Mode

8


NX-OS IGMP Snooping

IGMP Snooping on by default and supports v1/v2/v3

224.0.0.X are reserved for protocol use. All switches should flood the frame with destination IP address 224.0.0.X

All frames with destination MAC 0100.5E00.00XX will be flooded. Avoid using IP multicast groups that map to this MAC address range

Packets destined to unknown IGMP groups are dropped (except 224.0.0.x)

Detect mrouter ports via IGMP query and PIM hello

Can be configured as IGMP V3 querier. Support hosts running all IGMP version with backward compatibility

Fast leave is disabled by default

IGMP v3 explicit tracking is on. Track joins from individual host

9


PIM Sparse Mode

General purpose multicast routing protocol

Data-driven multicast state

Automatic source discovery

Efficient on-demand packet delivery

Uses both shared and source-based trees

‒ Distribution trees are unidirectional

Can support arbitrary source and receiver distribution

Group membership tracked via IGMPv1, v2, or v3

PIM-SM

RP

Shared Tree a.k.a RPT, rooted at the rendezvous point

Source Tree a.k.a SPT, rooted at the source

Source

Receivers

10


Source2

PIM Source-Specific Multicast

Simplifies one-to-many multicast delivery— uses source trees only

Control-plane multicast state

Assumes one-to-many model

‒ Internet/inter-domain multicast

‒ Video distribution

Hosts responsible for source discovery—

‒ Typically via some out-of-band mechanism (web page, content server, etc.)

‒ Eliminates need for RP and shared trees

‒ Eliminates need for MSDP

Group membership tracked via IGMPv3

‒ SSM mapping also supported

PIM-SSM Source Tree Rooted at Source1

Source1

S1,G1

Source Tree Rooted at Source2

S2,G1 S1,G1 S1,G1

Receivers

S2,G1

11


Bidirectional PIM

Massively scalable—ideal for many-to-many applications

Data-flow independent—no registers, asserts, non-RPF issues

Drastically reduces network mroute state

‒ Eliminates ALL (S,G) state in the network for Bidir groups

‒ Shortest path trees from sources to RP eliminated

‒ Source traffic flows both up and down shared RP tree

‒ Permits virtually unlimited sources

Bidir-PIM

RP

Shared Tree Bidirectional tree rooted at the rendezvous point

Source

Sources/Receivers

12

© 2012 Cisco and/or its affiliates. All rights reserved. BRKIPM-3062 Cisco Public 13

PIM

Conditional service

‒ Parser available after you enable via feature pim command

‒ Process runs after you configure an interface for PIM

Single PIM process runs any/all PIM flavors for VDC

PIM process responsibilities:

‒ Form PIM neighborships

‒ Processes inbound and outbound PIM protocol packets

‒ Encapsulate and transmit PIM registers, process PIM Register Stop messages (first-hop router)

‒ Decapsulate and process PIM registers, send Register Stop messages (PIM RP)

‒ Interface with MRIB to provide/learn multicast routes


IGMP

IGMP process always running

Single IGMP process provides both Layer 3 IGMP processing, and Layer 2 IGMP snooping functions

Layer 3 IGMP functions include:

‒ Send IGMP queries on PIM-enabled interfaces

‒ Process IGMP reports (joins) and leaves received from multicast receivers


Layer 2 IGMP snooping functions include:

‒ Process snooped multicast router packets (PIM Hellos, IGMP queries)

‒ Process IGMP reports and leaves sent by receivers

‒ Interface with MFDM to provide snooping entries

‒ Send special IGMP Leave messages to mrouters on TCN if switch is STP root


MSDP

Conditional service

‒ Parser available after you enable via feature pim command

‒ Process runs after you configure at least one MSDP peer

Functions of MSDP process include:

‒ Establish MSDP peering relationships

‒ Transmit source-active messages to configured peers

‒ Receive source-active messages from peers


MSDP SA cache enabled by default (non configurable)


MRIB

Multicast routing information base (MRIB) process always running

Functions include:

‒ Interface with various client processes to provide/learn multicast routes

‒ Combine information from various sources into single multicast routing table (the

MRIB)

‒ Interface with MFDM to provide routes for hardware programming IPv4 (M4RIB)

and IPv6 (M6RIB)

‒ RPF services via U4RIB / U6RIB

‒ Client-driven and designed for easy insertion of new clients

‒ Detailed traffic statistics

16


MFDM

Multicast Forwarding Distribution Manager (MFDM) process always running

Acts as interface between platform-independent Supervisor Engine processes, and platform-specific I/O module processes

Translates MRIB data into data structures required by hardware

Distributes that platform-specific information to I/O modules

17


IPFIB and L2MCAST (Nexus 7000)

Processes run on each I/O module

‒ Single instance of each handles requests from all configured VDCs

Both processes

‒ Interface with MFDM to receive platform-specific data structures required to program multicast forwarding entries into hardware

‒ Interact with hardware drivers to program ASIC forwarding tables

IPFIB – Responsible for programming (*,G) and (S,G) entries in FIB/ADJ, OILs in MET

L2MCAST – Responsible for programming IGMP snooping entries in MAC table

18


PIXM/PIXMC (Nexus 7000 Only)

Port Index Manager (PIXM) process handles index table management for all VDCs

‒ Interfaces with MFDM in each configured VDC

Serves critical function of allocating and managing index tables (LTL and FPOE) for system

Pushes table information to PIXM Client (PIXMC) process running on each I/O module

PIXMC interfaces with hardware drivers to program hardware


Multicast State Creation

How is state created?

Depends on:

‒ Where router sits relative to sources, receivers, and RP (if applicable)

‒ What flavor of PIM used


PIM-SM State Creation

(*,G) state driven by control plane protocols

‒ On last-hop router, IGMP joins create (*,G) state

‒ On upstream routers, PIM joins create (*,G) state all the way to the RP

(S,G) state frequently driven by data packets

‒ On first-hop router, packets punted to Supervisor Engine CPU to create (S,G) state, trigger PIM registers

‒ On last-hop router, packets received on shared tree punted to Supervisor Engine CPU to create (S,G) state, trigger SPT switchover

‒ On RP, PIM registers sent to Supervisor Engine CPU to create (S,G) state, trigger register stops and SPT switchover


PIM-SSM and PIM-Bidir State Creation

PIM-SSM (S,G) state driven by control plane protocols

‒ On last-hop router, IGMPv3 joins create (S,G) state

‒ On upstream routers, PIM-SSM joins create (S,G) state all the way to the first-hop

router

PIM-Bidir (*,G) state driven by control plane protocols

‒ On last-hop router, IGMP joins create (*,G) state

‒ On upstream routers, PIM joins create (*,G) state all the way to the RP

‒ On source-only branches, control plane installs (*,G/m) entries to enable data

forwarding toward bidir RP


Factors Influencing Control-Plane State

Creation Data-driven state creation consumes inband bandwidth and requires CPU cycles

Nexus 7000 implements hardware rate limiters and CoPP by default to protect these resources

Default values may not be appropriate/optimized for all environments

Examples:

‒ Multicast data packets (i.e., outside 224.0.0.0/24 range) match CoPP class-default

‒ copp-system-class-important lumps PIM registers with other traffic (FHRPs etc.)

‒ Directly-connected and local-groups rate limiters set to 3000 pps each

Default rate for mcast-snooping rate limiter (10000 pps) should be plenty


Multicast Packet Flow with VPC (1) Source in L3, Remote RP

1. Receiver sends IGMP join, EtherChannel hash on access switch happens to select link to vpc-peer2

Creates snooping, IGMP, and (*,G) mroute state with VPC VLAN as OIF

2. vpc-peer2 sends IGMP packet encapsulated in CFS to vpc-peer1

Creates identical state to vpc-peer2

3. Both VPC peers send PIM (*,G) joins to the RP to join the RPT

If ECMP to RP, hash selects the RPF interface

Source

Receiver

vpc-peer1 VPC Pri PIM-DR

vpc-peer2 VPC Sec Proxy-DR

IGMP Join

IGMP in CFS

1

2

(*,G) (*,G) PIM Join

3

4

core1 Anycast-RP

core2 Anycast-RP

24


State After IGMP Join from Receiver (1)

Source

Receiver



(*,G) (*,G)

vpc-peer1# sh ip igmp snooping groups vlan 101

Type: S - Static, D - Dynamic, R - Router port, F - Fabricpath core port

Vlan Group Address Ver Type Port list

101 */* - R Vlan101 Po1

101 239.1.1.1 v2 D Po100

vpc-peer1# sh ip igmp groups vlan 101

IGMP Connected Group Membership for Interface "Vlan101" - 1 total entries

Type: S - Static, D - Dynamic, L - Local, T - SSM Translated

Group Address Type Interface Uptime Expires Last Reporter

239.1.1.1 D Vlan101 00:00:33 00:04:11 10.100.101.100

vpc-peer1# sh ip mroute

IP Multicast Routing Table for VRF "default"

(*, 239.1.1.1/32), uptime: 00:00:39, igmp ip pim

Incoming interface: port-channel51, RPF nbr: 10.1.1.1

Outgoing interface list: (count: 1)

Vlan101, uptime: 00:00:39, igmp

vpc-peer1#

vpc-peer2# sh ip igmp snooping groups vlan 101

Type: S - Static, D - Dynamic, R - Router port, F - Fabricpath core port

Vlan Group Address Ver Type Port list

101 */* - R Vlan101 Po1

101 239.1.1.1 v2 D Po100

vpc-peer2# sh ip igmp groups vlan 101

IGMP Connected Group Membership for Interface "Vlan101" - 1 total entries

Type: S - Static, D - Dynamic, L - Local, T - SSM Translated

Group Address Type Interface Uptime Expires Last Reporter

239.1.1.1 D Vlan101 00:01:19 00:03:26 10.100.101.100







vpc-peer2#

vlan101

po100 po100

po1 po51 po52

po1 po2

core1 Anycast-RP

core2 Anycast-RP


State After IGMP Join from Receiver (2)

Source

Receiver



(*,G) (*,G)

core1# sh ip mroute


(*, 239.1.1.1/32), uptime: 00:01:48, pim ip

Incoming interface: loopback2, RPF nbr: 200.200.200.200


port-channel1, uptime: 00:01:48, pim

core1#

core2# sh ip mroute


(*, 239.1.1.1/32), uptime: 00:02:02, pim ip




core2# vlan101

po100 po100

po1

po51 po52

po1 po2

core1 Anycast-RP

core2 Anycast-RP


(*,G)

Multicast Packet Flow with VPC (2) Source in L3, Remote RP

5. Source begins transmitting

FHR registers source to RPs, etc.

6. One or both VPC peers receive (S,G) traffic on shared tree

Depends on upstream state

7. VPC peer switches negotiate for forwarder role

CFS messages exchanged to determine forwarder

Best routing metric, with VPC role as tie-breaker

8. Elected forwarder for (S,G), sends PIM (S,G) joins toward source

Joins SPT, prunes RPT

Adds VPC VLAN as L3 OIF

9. Data traffic flows down source tree to forwarding peer

Traffic also forwarded on peer link, dropped by other peer

Source

Receiver



CFS

6 + (S,G) (*,G) PIM Join

9

Data

7

8 PIM Prune

5

+ (S,G)

core1 Anycast-RP

core2 Anycast-RP

27


State After Peer Joins SPT

Source

Receiver



(*,G) + (S,G)







(10.200.0.100/32, 239.1.1.1/32), uptime: 00:14:49, ip pim mrib



Vlan101, uptime: 00:14:48, mrib

vpc-peer1#







(10.200.0.100/32, 239.1.1.1/32), uptime: 00:05:01, ip pim



vpc-peer2#

vlan101

po100 po100

po1 po51 po52

po1 po2

(*,G) + (S,G)

core1 Anycast-RP

core2 Anycast-RP


Multicast Packet Flow with VPC (1) Source in L3, Local RP

1. Receiver sends IGMP join, EtherChannel hash on access switch happens to select link to vpc-peer2

Creates snooping, IGMP, and (*,G) mroute state with VPC VLAN as OIF

2. vpc-peer2 sends IGMP packet encapsulated in CFS to vpc-peer1

Creates identical state to vpc-peer2

3. VPC peers are Anycast-RPs so no further PIM

activity

Source

Receiver

vpc-peer1 VPC Pri PIM-DR Anycast-RP

vpc-peer2 VPC Sec Proxy-DR Anycast-RP

IGMP Join IGMP in CFS 1

2

(*,G) (*,G)

core1 core2

29


State After IGMP Join from Receiver

Source

Receiver

(*,G) (*,G)



(*, 239.1.1.1/32), uptime: 00:00:15, igmp pim ip




vpc-peer1#







vpc-peer2# vlan101

po100 po100

po1 po51 po52

po1 po2



core1 core2


(*,G)

Multicast Packet Flow with VPC (2) Source in L3, Local RP

4. Source begins transmitting

FHR registers source to the RP

One VPC peer receives PIM registers

5. VPC peer switches negotiate for forwarder role

CFS messages exchanged to determine forwarder

Best routing metric, with VPC role as tie-breaker

6. Elected forwarder for (S,G) joins SPT, sends register

stops

7. Data traffic flows down source tree to forwarding peer

Traffic forwarded on peer link, dropped by other peer

Source

Receiver

CFS

5 + (S,G) (*,G)

PIM Join + Register Stop

6



Data

4

core1 core2

+ (S,G)

7

31


State After Peer Joins SPT (1)

Source

Receiver

(*,G) + (S,G)







(10.200.0.100/32, 239.1.1.1/32), uptime: 00:00:22, ip msdp pim mrib




vpc-peer1#







(10.200.0.100/32, 239.1.1.1/32), uptime: 00:00:40, pim ip

Incoming interface: port-channel51, RPF nbr: 10.1.1.5, internal


vpc-peer2#

vlan101

po100 po100

po1 po51

po51

po1 po2

(*,G) + (S,G) vpc-peer1 VPC Pri PIM-DR Anycast-RP


core1 core2


State After Peer Joins SPT (2)

core1# sh ip mroute IP Multicast Routing Table for VRF "default"

(10.200.0.100/32, 239.1.1.1/32), uptime: 00:00:55, ip pim

Incoming interface: Vlan200, RPF nbr: 10.200.0.100



core1#

core2# sh ip mroute


(10.200.0.100/32, 239.1.1.1/32), uptime: 00:01:12, ip pim



core2#

Source

Receiver

(*,G) + (S,G)

vlan101

po100 po100

po1 po51 po52

po1 po2



core1 core2


(*,G) (*,G)

Pre-Build SPT Option

ip pim pre-build-spt

1. Both forwarder and non-forwarder peer switches join SPT for new sources

2. Data traffic flows down source tree to both Peer-1 and Peer-2

3. On failure of forwarder (Peer-1), new forwarder (Peer-2) already has (S,G) state, is receiving traffic, and only needs to add OIFs

Pre-build SPT considerations:

Creates Live/Live data stream

Consumes bandwidth and replication capacity on primary and secondary data path in steady state

Decreases reconvergence time on failure (no need to create upstream state)

Source

Receiver



2 + (S,G) + (S,G)

PIM Join

3 Data

1 PIM Join 1

2

X 4

Add OIFs

core1 core2

34


State After Both Peers Join SPT (pre-build-

spt option) Source

Receiver

(*,G) + (S,G)

core1# sh ip mroute


(10.200.0.100/32, 239.1.1.1/32), uptime: 00:00:11, ip pim





core1#

core2# sh ip mroute


(10.200.0.100/32, 239.1.1.1/32), uptime: 00:00:23, ip pim



core2#

vlan101

po100 po100

po1 po51 po52

po2



po1

core1 core2


(*,G)

Source Receiver

+ (S,G) (*,G)

Multicast Packet Flow with VPC Source in L2, Remote RP

1. Source traffic arrives on vpc-peer1, creates

(S,G) state, initiates PIM regsiters etc.

2. Data traffic flows down VPC to receiver

3. vpc-peer1 also forwards data traffic over peer

link to vpc-peer2, which creates (S,G) state

Traffic dropped by vpc-peer2

core1 Anycast-RP

core2 Anycast-RP

Data

+ (S,G)

1 2

3



36


State After Source Starts Sending (1)











vpc-peer1#











vpc-peer2#

(*,G)

Source Receiver

+ (S,G) (*,G)

core1 Anycast-RP

core2 Anycast-RP

+ (S,G)

vlan101 vlan112

po51 po52

po2 po1




State After Source Starts Sending (2)

core1# sh ip mroute


(*, 239.1.1.1/32), uptime: 00:02:29, pim ip




(10.100.112.100/32, 239.1.1.1/32), uptime: 00:01:29, pim mrib ip



core1#

core2# sh ip mroute


(*, 239.1.1.1/32), uptime: 00:02:43, pim ip







core2#

(*,G)

Source Receiver

+ (S,G) (*,G)

core1 Anycast-RP

core2 Anycast-RP

+ (S,G)

vlan101 vlan112

po51 po52

po2 po1




(*,G)

Multicast Packet Flow with VPC Source in L2, Local RP

1. Source traffic arrives on vpc-peer1, creates

(S,G) state

VPC peers are Anycast-RPs so no further PIM activity

2. Data traffic flows down VPC to receiver

3. vpc-peer1 also forwards data traffic over peer

link to vpc-peer2, which creates (S,G) state

Traffic dropped by vpc-peer2

Source Receiver

+ (S,G) (*,G)

Data

1



+ (S,G)

2

3

39


State After Source Starts Sending








Incoming interface: Vlan112, RPF nbr: 10.100.112.100, internal



vpc-peer1#







(10.100.112.100/32, 239.1.1.1/32), uptime: 00:00:21, ip msdp pim mrib




vpc-peer2#

(*,G)

Source Receiver

+ (S,G) (*,G)



+ (S,G)


Source

(*,G)

Source Receiver

+ (S,G) (*,G)

Multicast Packet Flow with VPC Source in L2 and L3, Remote RP

State is combination of previous examples

(*,G) and (S,G) state on both VPC peers

For source in L2, either VPC peer can forward

For source in L3, one peer chosen to forward

+ (S,G)

Data

Data



core1 Anycast-RP

core2 Anycast-RP

41


State After Sources Start Sending (1)















vpc-peer1#











(10.200.0.100/32, 239.1.1.1/32), uptime: 00:03:08, ip pim



vpc-peer2#

Source

(*,G)

Source Receiver

+ (S,G) (*,G) + (S,G) vpc-peer1 VPC Pri PIM-DR


core1 Anycast-RP

core2 Anycast-RP

vlan101 vlan112

po52

po2 po1

po51 po51

vlan200

po2


State After Sources Start Sending (2)

core1# sh ip mroute


(*, 239.1.1.1/32), uptime: 02:14:29, pim ip







(10.200.0.100/32, 239.1.1.1/32), uptime: 02:11:13, ip mrib pim



port-channel1, uptime: 02:11:13, mrib, pim

core1#

core2# sh ip mroute


(*, 239.1.1.1/32), uptime: 02:14:50, pim ip







(10.200.0.100/32, 239.1.1.1/32), uptime: 02:11:34, ip mrib pim



core2#

Source

(*,G)

Source Receiver

+ (S,G) (*,G) + (S,G) vpc-peer1 VPC Pri PIM-DR


core1 Anycast-RP

core2 Anycast-RP

vlan101 vlan112

po52

po2 po1

po51 po51

vlan200

po2


Scoping Multicast Group Ranges

You can specify multiple RPs

‒ Static RP configuration always overrides dynamically learned RP information

Two options for specifying groups to map to each RP

‒ ―Inline‖ group-lists

‒ Specify route-map containing groups

RP used for particular group based on longest-match mask length

Highest RP IP address used for tie-breaks


“Inline” RP Group-Lists

ip pim rp-address <ip> group-list <ip>/<mask>

Useful if group ranges are contiguous and relatively simple

You can specify multiple lines for the same RP ‒ ip pim rp-address 100.100.100.100 group-list 239.1.2.1/32

‒ ip pim rp-address 100.100.100.100 group-list 239.1.3.1/32




Route-Maps for RP Scoping

ip pim rp-address <ip> route-map <name>

Useful for more complex scoping configurations

NOTE: As of 4.2(6)/5.0(3), ―deny‖ semantics are ignored in RP-scoping route-maps

‒ Can make non-contiguous scoping difficult

Interim solution: use static route to Null0 and define a ―blackhole RP‖ for the unneeded groups

‒ ip pim rp-address 100.100.100.100 route-map real-rp

‒ ip pim rp-address 255.255.255.254 route-map blackhole-rp

‒ route-map blackhole-rp permit 10

‒ match ip multicast group 239.1.4.1/32

‒ route-map blackhole-rp permit 20


‒ route-map real-rp permit 10


‒ ip route 255.255.255.254/32 Null0


RP Scoping and Source State on First-Hop

Router RP scoping does not prevent state creation on the first-hop router

To completely prevent state creation, use a RACL denying the groups in

question


Controlling (S,G) Expiration Time

Default (S,G) expiration time is 3 minutes

―Intermittent sources‖ in PIM-SM can cause latency/loss issues

Change expiration time with: ‒ ip pim sg-expiry-timer <sec> [sg-list <route-map>]

Route-map defines list of (S,G) entries to which the timer applies

Technically only needed on last-hop routers ‒ PIM joins keep state alive on upstream routers

7010-1# sh run pim | in sg

ip pim sg-expiry-timer 36000 sg-list sg-expiry

7010-1# sh route-map sg-expiry

route-map sg-expiry, permit, sequence 10

Match clauses:

ip multicast: group 239.1.2.0/23

Set clauses:

7010-1# sh ip pim route | eg -v \* | in expires

(10.200.200.3/32, 239.1.2.1/32), expires 09:59:25

(10.200.200.4/32, 239.1.3.1/32), expires 09:59:25

(10.200.200.5/32, 239.1.4.1/32), expires 00:02:25

(10.200.200.6/32, 239.1.5.1/32), expires 00:02:25

7010-1#

36000 seconds = 10 hours


PIM Join/Prune Policies

Define route-map to control PIM join/prune policy

‒ ―Permit‖ or ―deny‖ stanzas define for which groups PIM join/prunes are processed

‒ Implicit deny for unmatched groups

Use ―ip pim jp-policy‖ to apply to interface

‒ Can apply inbound, outbound, or both

Example: ‒ route-map pim-policy deny 10


‒ route-map pim-policy permit 20


‒ interface port-channel300

‒ ip pim jp-policy pim-policy in


“Multicast Boundary” – Combining Control-Plane Policies and

ACLs

NX-OS does not provide ―ip multicast-boundary‖ command

Each control-plane protocol controlled independently using policy configuration

Data-plane traffic controlled using IP ACLs

Example: prevent PIM, Auto-RP/BSR, and data-plane traffic from entering an interface

7010-1# sh run int po300

interface port-channel300

ip access-group no-mcast-data in

ip address 10.18.0.2/30

ip ospf network point-to-point

ip router ospf 10 area 0.0.0.0

ip pim sparse-mode

ip pim border

ip pim jp-policy pim-policy in

7010-1# sh route-map pim-policy

route-map pim-policy, deny, sequence 10

Match clauses:

ip multicast: group 224.0.0.0/4

Set clauses:

Limits data-plane traffic

Limits PIM joins

Limits BSR/Auto-RP

7010-1# sh ip access no-mcast-data

IP access list no-mcast-data

10 permit ip any 224.0.0.0/24

20 deny ip any 224.0.0.0/4

7010-1#


I/O Module

Supervisor

NX-OS and Nexus 7000 Multicast Routing

Architecture

• Push routes to platform

• Route download

• Translate routes to hardware format

• Program hardware forwarding and

replication engines

RPF updates

Multicast Routing Information Base (mRIB)

m4RIB m6RIB

FIB Manager

Forwarding Hardware

mFDM

uRIB

PIM MSDP IGMP PIM6 ICMPv6 / MLD

Add (*,G) & (S,G) from reports Add (S,G) from SAs Add (*,G) & (S,G) from Join/Prune

& Register/Assert Add (*,G) & (S,G) from reports

• Add routes, OIFs

• Update when RPF changes

52


Where Are Multicast Routes Stored?

sh ip pim route sh ip igmp route [sh ip igmp groups] sh ip igmp snooping groups sh ip msdp route [sh ip msdp sa-cache] etc.

sh routing ip multicast [sh ip mroute]

sh forwarding distribution ip multicast route sh forwarding distribution ip igmp snooping

sh forwarding ip multicast route

sh system internal forwarding ip multicast route sh system internal ip igmp snooping

ADJ Table MET

MAC Table FIB TCAM

I/O Module

Supervisor Engine

PIM MSDP

MRIB

State Database (PSS)

IGMP

URIB

STP

MFDM

PIXMC

PIXM

IP FIB

Other HW

Hardware Drivers

L2MCAST

53


Nexus 7000 Hardware IP Multicast

Fully distributed Layer 2 multicast hardware switching

Fully distributed Layer 3 IPv4 multicast hardware switching

(S,G), (*,G), and (*,G/m) mroute forwarding in hardware

Distributed Layer 2 and Layer 3 multicast packet replication using egress

replication

Up to 8 Bidir RPs per VRF

IGMPv2/IGMPv3 snooping with IP-based traffic constraint

VRF-lite for multicast


Multicast with vPC

vPC supports PIM-SM only

vPC uses CFS to sync IGMP state

For sources in vPC domain, both vPC peers are

forwarders

‒ Duplicates avoided via vPC loop-avoidance logic

For sources in Layer 3 cloud, unicast best metric

determines active forwarder (vPC operational primary in

case of tie)

‒ CFS used to negotiate active forwarder role on per-source

basis

Source

Receivers Source

55


Multicast Software Architecture with VDCs

Core multicast services always present in each VDC

‒ MRIB, MFDM, IGMP

PIM/MSDP spawned as configured on per-VDC basis

Other lower-level processes run in global space on Supervisor Engine and I/O modules

‒ PIXM on Supervisor Engine, IP FIB, ASIC drivers on I/O modules

Be aware of other global services that affect multicast, such as hardware rate limiters and CoPP


VDC1 VDC2 VDC3

Multicast Software Architecture with VDCs

I/O Module I/O Module I/O Module

Hardware

IP FIB

Hardware Hardware

IP FIB IP FIB

Supervisor Engine

PIM etc.

MRIB

PIM etc.

MRIB

PIM etc.

MRIB

MFDM MFDM MFDM

57


BFD for PIM

NX-OS version 5.0(2a) introduces BFD support

PIM process can be BFD client

Enable globally for PIM, disable per interface if desired

‒ ip pim bfd (global)

‒ ip pim bfd-instance disable (interface)

7010-1# sh run pim | in bfd

ip pim bfd

7010-1# sh ip pim neighbor vlan 102

PIM Neighbor Status for VRF "default"

Neighbor Interface Uptime Expires DR Bidir- BFD

Priority Capable State

10.100.102.3 Vlan102 09:42:38 00:01:19 1 yes Up

7010-1# sh bfd neighbor int vlan 102 detail | in pim

Registered protocols: hsrp_engine pim

7010-1#


Clearing Mroutes

What happens when you ―clear ip mroute‖?

NOT what happens in traditional Cisco IOS router

‒ Cisco IOS stores multicast routing table in monolithic data structure

In NX-OS, clear ip mroute == clear routing ip multicast

In other words, command removes routes from the MRIB

MRIB notifies MFDM, MFDM removes route, notifies MFIB (IPFIB) on I/O modules, MFIB removes route from hardware

MRIB immediately requests client processes (PIM, IGMP, MSDP) to repopulate the MRIB

MRIB adds routes back, notifies MFDM, MFDM notifies MFIB, MFIB reprograms hardware


Clearing a Route from the MRIB 7010-1# clear ip mroute 239.1.18.1

7010-1# 2010 Jul 29 15:53:59.332105 mrib: [4600] (default-base) Schedule route removal from mrib

2010 Jul 29 15:53:59.332191 mrib: [4600] (default-base) Route removed from mrib

2010 Jul 29 15:53:59.332235 mrib: [4600] (default-base) Schedule route removal from mrib

2010 Jul 29 15:53:59.332270 mrib: [4600] (default-base) Route removed from mrib

2010 Jul 29 15:53:59.332973 igmp: Received repopulate route notification for VRF default(1)

2010 Jul 29 15:53:59.333058 pim: Received repopulate mroute notification from MRIB for VRF default

2010 Jul 29 15:53:59.334172 igmp: Processing repopulate route request for igmp mpib, for VRF default (*, 239.1.18.1/32)

2010 Jul 29 15:53:59.334291 pim: repopulate (*, 239.1.18.1/32)

<etc.>

debug ip pim internal debug ip igmp internal debug ip mrouting summary


Clearing PIM Routes

Use clear ip pim route to remove route entries from PIM

Only PIM routes created by periodic PIM join messages removed

PIM routes created to trigger upstream joins not removed

‒ Example: IGMP join from directly connected receiver causes MRIB to notify PIM to create a

PIM route

If PIM route is ―mixed‖ (e.g., mroute has both PIM and IGMP OIFs), PIM OIFs

removed but route remains in PIM database


Clearing Data-Created Mroutes

Use clear ip mroute data-driven to remove route entries created by netstack

‒ E.g., (S,G) entries created on FHR

Supported in NX-OS 4.2(6) and 5.1(1) and later


Multicast Enhancement in Nexus 5500

Increased IGMP groups to 4K groups

Lower latency ~2.1-2.2us

Better throughput and latency with complex traffic

pattern with more multicast VOQ

Improved multicast load sharing over PortChannel

Drop multicast traffic for congested egress ports

Supports PIM BiDir in vPC

64


Multicast with vPC

vPC supports PIM-SM, PIM-SSM and PIM BiDir

vPC uses CFS to sync IGMP state

For sources in vPC domain, both vPC peers are

forwarders

‒ Duplicates avoided via vPC loop-avoidance logic

For sources in Layer 3 cloud, unicast best metric

determines active forwarder (vPC operational primary in

case of tie)

‒ CFS used to negotiate active forwarder role on per-source

basis

Source

Receivers Source

65


Multicast Superframing

Multicast frame sharing same fanout are superframed

automatically

When the switch fabric access grant is received the

multicast packets in VOQ that has same fanout as the

first packet in the queue will be packed and sent to

egress ports within one scheduling cycle

Superframing improves throughput and reduce latency

Up to 10KB per superframe.

No waiting period. Only the complete frames in the queue can

be packed.

Superframing in on by default for both unicast and

multicast. Hardware always packs the frames

whenever it is possible

66


Multicast Optimization Configuration Multicast optimization is turned on by default for ―class-default‖. It means all multi-

destination traffic will be assigned to multicast VOQ according to their fanout

Multi-destination traffic include

‒IP multicast

‒Unknown unicast flooding

‒Broadcast traffic

‒L2 multicast traffic

Multicast optimization can only be turned on for one system class.

8 multicast VOQ reserved for QoS queuing. The rest of 120 queues for multicast

optimization

N5k(config-cmap-qos)# policy-map type qos Mcast_optimize

N5k(config-pmap-qos)# class type qos class-ip-multicast

N5k(config-pmap-c-qos)# set qos-group 2

N5k(config-pmap-c-qos)# exit

N5k(config-pmap-qos)# class type network-qos IP_mcast

N5k(config-cmap-nq)# match qos-group 2

N5k(config-cmap-nq)# policy-map type network-qos Mcast_optimize

N5k(config-pmap-nq)# class type network-qos IP_mcast

N5k(config-pmap-nq-c)# multicast-optimize

N5k(config-pmap-nq-c)# queue-limit 170000

67


Line rate multicast forwarding and replication for all ports and

all frame size with features on

Ultra low latency even with mesh traffic pattern

4K IGMP snooping entries

4K IP multicast routes

Multicast Performance/Scalability

69


Reduce Burstiness of IGMP with MRT

IGMP packets are rate limited to 400pps at hardware as of 5.0(3)U1

Host intended to join the group has to response within the time specified

by the MRT(Maximum Response Time) in query message.

Reduce burstiness of IGMP message with larger MRT value. Hosts will

delay the IGMP for a random amount of time which is less than MRT

Recommend to increase MRT value with large number of IGMP join

Default MRT value 10s. Configure at IGMP querier or IGMP snooping

querier

N3k-1(config)#interface vlan 101

N3k-1(config-vlan)# ip igmmp query-max-response-time 25

N3k-1(config-vlan)# ip igmp last-member-query-response-time 25


PIM Features

Support PIM-SM and PIM-SSM

RP selection: static RP, BSR, Auto-RP

Anycast RP with PIM. Anycast RP with MSDP

VRF-aware

PIM policies: Neighbor policy, Join/prune policy, registration policy

No Support for PIM-BiDir


L3 Multicast Table Size Host Table and LPM(Longest Prefix Match) in Nexus 3064 routing engine

Host Table stores ARP entries, host routes (/32) and multicast routes-(*,G) and (S,G). One

multicast route((*,G) or (S,G)) consume two HW entries

LPM table stores summarized routes, ARP and host routes

CLI to increase IP multicast region to support up to 4000 mroutes

Default Table Partition Increased multicast Table size

IP Multicast

(2k routes

4K HW entries)

Host Table

(8K entries)

ARP /32 routes 4k entries ARP and

routes

LPM Table

(8K entries)

IP Multicast

(4k routes

8K HW entries)

ARP and

routes

Host Table

(8K entries)

LPM Table

(8K entries)


L3 Multicast Table Size By default Nexus 3064 supports 2K multicast routes. The following CLI increases that to 4K at the

expense of ARP and routing table size. The CLI will fail if there are pre-existing ARP entries or /32

host route. Recommend to configure the CLI at the beginning.

N3k-1(config)# hardware profile multicast max-limit 4000

N3k-1(config)#

• Hardware space allocated for IP multicast are reserved for IP multicast and can’t be shared with ARP

• Check the hardware resource utilization

N3k-1# sh hardware profile status

Reserved LPM Entries = 1024.

Reserved Host Entries = 96.

Reserved Mcast Entries = 4000.

Used LPM Entries = 3.

Used Host Entries in LPM = 0.

Used Mcast Entries = 3500.

Used Host Entries in Host = 15.

N3k-1#


This presentation will use

the topology at the right to

illustrate designs and use

cases for multicast.

Typical 3-layer hierarchical

network design

Classical Ethernet

Topologies

Multicast and Places in the Network

Layer 3

Layer 2

75


Control Plane Policing (CoPP) enabled by default

May wish to tune for multicast

Multi-step process

‒ Monitor existing CoPP policy

‒ Adjust specific attributes (PIM, IGMP, MSDP, etc)

‒ Monitor and tune as needed

As much science as an art – network requirements change!

Control Plane Policing - CoPP

76


Show policy-map interface control-plane

<snip>

set cos 7

police cir 39600 kbps , bc 250 ms

module 4 :

conformed 1187892 bytes; action: transmit

violated 0 bytes; action: drop

*NOTE* CoPP is enforced per module on Nexus 7000

Tune policy

N7K-1# copp copy profile strict prefix tuned

N7K-1(config)# policy-map type control-plane tuned-copp-policy-strict

N7K-1(config-pmap)# class tuned-copp-class-critical

N7K-1(config-pmap-c)# police cir 64000 kbps bc 250 ms conform transmit violate drop

N7K-1(config-pmap-c)# end

N7K-1# show run copp

N7K-1# show policy-map int control-plane

<snip>

set cos 7

police cir 64000 kbps , bc 250 ms

module 4 :

conformed 172 bytes; action: transmit

violated 0 bytes; action: drop

CoPP - Example

77

Look for increments

here

Monitor for more

increments


Lightweight hello protocol over different data protocols

‒IPv4, IPv6, MPLS

Used for fast (often sub-second) communication failure detection

Single, common & standardized mechanism

Independent of specific routing, FHRP and other client protocols using BFD

Any ―interested application‖ (OSPF, BGP, EIGRP, PIM, etc.) registers with BFD and

is notified as soon as BFD recognizes a neighbor loss

UDP port 3784 / 3785 (for echo)

RFC 5880

Available on Nexus 7000 and 3000

Bidirectional Forwarding Detection (BFD)

78


Enable BFD

Enable BFD for PIM

Verify BFD Configuration

BFD - Example

79

Config

N7K-1-Core1# (config) feature bfd

N7K-1-Core1# (config) pim bfd

N7K-1-Core1# (config) end

N7K-1-Core1# show bfd neighbors application pim

OurAddr NeighAddr LD/RD RH/RS Holdown(mult) State Int Vrf

10.1.0.29 10.1.0.30 1124073473/0 Down N/A(3) Down Eth4/7 default

10.1.0.1 10.1.0.2 1124073474/1090519042 Up 4954(3) Up Eth4/1 default

10.1.0.5 10.1.0.6 1124073475/1107296259 Up 4520(3) Up Eth4/2 default

N7K-1-Core1#


Recommendations for RL/CoPP for

Multicast On first-hop router:

‒ Add CoPP class(es) for multicast data groups – match all data groups, or define multiple classes for different group ranges (critical groups, important groups, best-effort groups)

‒ Tweak directly-connected rate limiter – may want to increase rate, understanding potential implications

On last-hop router:

‒ Add CoPP class(es) for multicast data groups

‒ Tweak local-groups rate limiter

On RP:

‒ Consider creating separate PIM register class


Enable PIM BFD

‒ Don’t forget to disable ip redirects

‒ For non-BFD devices, timers may be tuned – test impact!

Tune Control Plane Policing Policy

Typical RP protocols can be used

Consider anycast-rp commands (RFC 4610)

Multicast in L3 Core

81


Enable PIM BFD

‒ Don’t forget to disable ip redirects

‒ For non-BFD devices, timers may be tuned – test impact!

Tune Control Plane Policing Policy

Typical RP protocols can be used

Consider anycast-rp commands (RFC 4610)

Multicast at L2/L3 Boundary

82


NX-OS versus Cisco IOS Multicast

Configuration No need to enable multicast globally

Must enable each feature (PIM, MSDP) explicitly before configuring

No support for PIM-DM

No support for PIM sparse-dense mode

‒ Use Auto-RP listener/forwarder configuration instead

MSDP SA cache enabled by default (non configurable)

Static RP configuration based on longest-match prefix length

Support for PIM-based Anycast-RP (RFC 4610)

Uses multicast multipath RPF by default for ECMP prefixes (non configurable)

Supports PIM neighbor authentication

No direct support for multicast boundary

‒ Use protocol-specific filtering policies and/or data-plane RACLs


NX-OS/Nexus 7000 versus Cisco

IOS/Catalyst 6500 Multicast Operation Supports egress local Layer 3 multicast replication only

Clearing mroute state behaves differently

‒ ―clear ip mroute‖ clears state from MRIB down to hardware – does not clear protocol state – use clear ip pim route, clear ip igmp route, etc.

Software replication disabled by default:

‒ Controls whether software routes initial leaked multicast packets – ip routing multicast software-replicate

Use show ip mroute summary count instead of show ip mroute count


Migration from IOS PIM Auto-RP

IOS

interface Loopback10 ip address 172.16.1.1

255.255.255.255

ip pim sparse-mode

ip pim send-rp-announce Loopback10 scope 32

ip pim send-rp-discovery Loopback10 scope 32

ip pim autorp listener

NX-OS

interface loopback10 ip address 172.16.1.1/32

ip pim sparse-mode

ip pim auto-rp rp-candidate loopback10 group-

list 224.0.0.0/4

ip pim auto-rp mapping-agent loopback10

ip pim auto-rp forward listen

or

ip pim send-rp-announce loopback10 group-list

224.0.0.0/4

ip pim send-rp-discovery loopback10

ip pim auto-rp forward listen

85


Migration from IOS PIM Anycast RP – RFC 4610

Cisco IOS Software does not have the

ability to enable the PIM Anycast RP

feature.

NX-OS

interface loopback0 ip address 192.168.10.1/32

ip pim sparse-mode

interface loopback10

description Anycast-RP-Address

ip address 172.16.1.1/32

ip pim sparse-mode

ip pim anycast-rp 172.16.1.1 192.168.10.1

ip pim anycast-rp 172.16.1.1 192.168.10.2

86


Migration from IOS Configuring PIM in a non-default VRF Instance

IOS

ip vrf production ip multicast-routing vrf

production

interface Loopback10

ip vrf forwarding production

ip address 172.16.1.1 255.255.255.255

ip pim sparse-mode

interface TenGigabitEthernet1/1

ip vrf forwarding production

ip address 192.168.10.1 255.255.255.0

ip pim sparse-mode

ip pim vrf production rp-address 172.16.1.1

NX-OS

vrf context production ip pim rp-address

172.16.1.1 group-list 224.0.0.0/4


vrf member production

ip address 172.16.1.1/32

interface Ethernet1/1

vrf member production

ip address 192.168.10.1/24

ip pim sparse-mode

87


Migration from IOS Configuring MSDP with Anycast-RP

IOS

interface Loopback0 description MSDP Peer

Address

ip address 192.168.1.1 255.255.255.255

interface Loopback10

description PIM RP Address

ip address 1.1.1.1 255.255.255.255

ip pim rp-address 1.1.1.1

ip msdp peer 192.168.2.1 connect-source

Loopback0

ip msdp cache-sa-state

NX-OS

feature msdp

interface loopback0 description MSDP Peer

Address

ip address 192.168.1.1/32


description PIM RP Address

ip address 1.1.1.1/32

ip pim rp-address 1.1.1.1 group-list

224.0.0.0/4

ip msdp peer 192.168.2.1 connect-source

loopback0

88

Summary


Summary

NX-OS supports a wide variety of multicast technologies across the

different family members

Common architecture with small platform specific components

Flexible options to address different needs in the network

90


Complete Your Online

Session Evaluation Give us your feedback and you could

win fabulous prizes.

Winners announced daily.

Receive 20 Passport points for each

session evaluation you complete.

Complete your session evaluation

online now (open a browser through

our wireless network to access our

portal) or visit one of the Internet

stations throughout the Convention

Center.

Don’t forget to activate your

Cisco Live Virtual account for access to

all session material, communities, and

on-demand and live activities throughout

the year. Activate your account at the

Cisco booth in the World of Solutions or visit

www.ciscolive.com.

91

http://www.ciscolive.com


Final Thoughts

Get hands-on experience with the Walk-in Labs located in World of

Solutions, booth 1042

Come see demos of many key solutions and products in the main Cisco

booth 2924

Visit www.ciscoLive365.com after the event for updated PDFs, on-

demand session videos, networking, and more!

Follow Cisco Live! using social media:

‒ Facebook: https://www.facebook.com/ciscoliveus

‒ Twitter: https://twitter.com/#!/CiscoLive

‒ LinkedIn Group: http://linkd.in/CiscoLI

92

http://www.ciscolive365.com/

https://www.facebook.com/ciscoliveus

https://twitter.com/

http://linkd.in/CiscoLI

Reference Slides

94

NX-OS Multicast Design and Best...

Documents

Transcript of NX-OS Multicast Design and Best...