Post on 17-Jul-2020
Catalyst 4500 Bootcamp
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential4500 Bootcamp 1
Chapter 5: High Availability
Richard Licon
Technical Marketing Engineer
Campus Switching Solutions Technology Group
February 2009
Agenda
• High Availability - Brief Intro / Evolution
• Catalyst 4500 System Resiliency – Redundancy Components
• Supervisor Uplinks / Shared Backplane Mode
• RPR and SSO Mode
• NSF – Non Stop Forwarding
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential4500 Bootcamp 2
• NSF – Non Stop Forwarding
• ISSU – In Service Software Upgrade
• Isolated CPU Queues / CoPP – Control Plane Policing
• FHRP – VRRP / HSRP / GLBP
• GOLD – Generic Online Diagnostics
• EEM – Embedded Event Manager
High Availability – What is achieved?
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential4500 Bootcamp 3
VTP Domain CampusVTP Domain Campus
High Availability – What is achieved?
Provide High Availability at Critical Sites to:Provide High Availability at Critical Sites to:
Reduce outage for Supervisor Hardware Failures Reduce outage for Supervisor Hardware Failures
Reduce outage for Software Failures
Mitigate planned outages by providing “Hitless” “Hitless” Software Updates
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential4500 Bootcamp 4
VTP Domain CampusVTP Domain Campus
Mitigate planned outages by providing “Hitless” “Hitless” Software Updates
Provide Power Supply Redundancy
Provide Network Level Redundancy
Introduced RPR
SupIV w/ 2 x GE Active Uplinks –4507R Chassis
LACP- 12.1(13)EW
12.1(12c)EW 12.1(19)EW
SupV w/ up to 4 x GE Active Uplinks4510R Chassis
12.2(18)EW 12.2(20)EWA 12.2(25)EWA 12.2(25)SG
SupII+10GE 2 x 10GbE + 4 x GE Active Uplnksconcurrenty
SupV-10GE - 2 x
12.2(31)SGA
Introduced ISSU
12.2(4x)SG
Catalyst 4500 – Evolution of HA Features
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential4500 Bootcamp 5
SupII+ w/ 2 x GE Active Uplinks
Introduced SSO for L2 Ports
Enhanced SSO Support for L3 interfaces
SupV-10GE w/ 2 x 10GbE or 4 x GE Active Uplinks
SupV-10GE - 2 x 10GbE + 4 x GE uplinks active concurrently
12.2(44)SG –
E-Series Chassis SSO Support
Supervisor 6-E -ISSU
Shared-Backplane Mode
Agenda
• High Availability - Brief Intro / Evolution
• Catalyst 4500 System Resiliency – Redundancy Components
• Supervisor Uplinks / Shared Backplane Mode
• RPR and SSO Mode
• NSF – Non Stop Forwarding
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential4500 Bootcamp 6
• NSF – Non Stop Forwarding
• ISSU – In Service Software Upgrade
• Isolated CPU Queues / CoPP – Control Plane Policing
• FHRP – VRRP / HSRP / GLBP
• GOLD – Generic Online Diagnostics
• EEM – Embedded Event Manager
Chassis Redundancy Components
Fabric Redundancy Modules
Line Card Slot 1
Line Card Slot 2
Line Card Slot 3
Line Card Slot 4
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential4500 Bootcamp 7
Fan
Tra
yFa
nTr
ay
Clock Module
Modules
Backplane Connectors
Supervisor Slot 5
Supervisor Slot 6
Line Card Slot 7
Line Card Slot 8
Line Card Slot 9
Line Card Slot 10
Chassis Redundancy Components
Tra
yT
ray
Fabric Redundancy Modules
Supervisor Slot 5
Line Card Slot 1
Line Card Slot 2
Line Card Slot 3
Line Card Slot 4
Fabric Redundancy ModulePID - WS-X4590-E=One Mux-Buffer required per line cardMuxes Signal from two supervisors to single line cardOnly used in Redundant Chassis
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential4500 Bootcamp 8
Fa
nT
ray
Fa
nT
ray
Clock Module
Backplane Connectors
Supervisor Slot 5
Supervisor Slot 6
Line Card Slot 7
Line Card Slot 8
Line Card Slot 9
Line Card Slot 10
Clock Modules –PID - WS-X4K-CLOCK-E= Clock for line cards, Fabric Redundancy Modules, Supervisors3 x Redundant Oscillators Only used in Redundant Chassis
Redundant Supervisor Communication
Active Supervisor
EOBC MAC
Synchronizes:Startup ConfigurationRunning Configuration
VLAN DatabaseBootVariables
Config-Register
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential4500 Bootcamp 9
Keep-Alive
Standby Supervisor
EOBC MAC
Keep-Alive - ACK
S2S (Sup2Sup) ConnectionFull-Duplex Gigabit Link
Chassis + Sup – Packet Flow
FPGA
FPGA
Active Supervisor
SERDESPacket
Processor
CPU
ForwardingEngine
CPU
S2WBUS
PacketFlow
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential4500 Bootcamp 10
FPGA
FPGA
Standby Supervisor
Line Card
PHYSTUB RJ45SERDES
SERDESPacket
Processor
FabricRedundancy
Module
CPU
ForwardingEngine
Agenda
• High Availability - Brief Intro / Evolution
• Catalyst 4500 System Resiliency – Redundancy Components
• Supervisor Uplinks / Shared Backplane Mode
• RPR and SSO Mode
• NSF – Non Stop Forwarding
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential4500 Bootcamp 11
• NSF – Non Stop Forwarding
• ISSU – In Service Software Upgrade
• Isolated CPU Queues / CoPP – Control Plane Policing
• FHRP – VRRP / HSRP / GLBP
• GOLD – Generic Online Diagnostics
• EEM – Embedded Event Manager
Redundant Supervisor Uplinks
Prior to Cisco IOS Release 12.2(44)SGPrior to Cisco IOS Release 12.2(44)SG-- Sup6Sup6--E allowed one to enable either dual E allowed one to enable either dual wire speed 10GbE ports or four Twinwire speed 10GbE ports or four Twin--Gigabit converter based GbE SFP uplink ports Gigabit converter based GbE SFP uplink ports when operating in redundancy modewhen operating in redundancy mode
Active Supervisor
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential4500 Bootcamp 12
Standby Supervisor
Only Left-most Ports Enabled
Ports Inactive
Redundant Supervisor 6-E Uplinks
Cisco IOS 12.2(46)SGCisco IOS 12.2(46)SG-- SharedShared--Backplane ModeBackplane Mode enables all frontenables all front--panel ports on both panel ports on both Active and Standby Supervisor Engines. Capability supported on all Catalyst 4500 and Active and Standby Supervisor Engines. Capability supported on all Catalyst 4500 and 4500E4500E Series Chassis with Supervisor 6Series Chassis with Supervisor 6--E. E.
Standby
Active Supervisor
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential4500 Bootcamp 13
Standby Supervisor
All Ports Are Active
4510R-E-1(config)#hw-module uplink mode shared-backplane
A 'redundancy reload shelf' or power-cycle of chassis is required toapply the new configuration
4510R-E-1# redundancy reload shelf
Redundant Supervisor Uplinks
XGStub ASIC
X2 or Twin-Gig
XGStub ASIC
X2 or Twin-Gig
IPP Fabric Ports Active Supervisor
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential4500 Bootcamp 14
X2 or Twin-GigModule
X2 or Twin-GigModule
XGStub ASIC
X2 or Twin-GigModule
XGStub ASIC
X2 or Twin-GigModule
IPP Fabric Ports
2.5 Gbps SuperPort(10 Gbps per
Stub ASIC)
Standby Supervisor
Redundant Supervisor Uplinks
XGStub ASIC
X2 or Twin-Gig
XGStub ASIC
X2 or Twin-Gig
IPP Fabric Ports Active Supervisor
Active Ports Disabled Ports
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential4500 Bootcamp 15
X2 or Twin-GigModule
X2 or Twin-GigModule
XGStub ASIC
X2 or Twin-GigModule
XGStub ASIC
X2 or Twin-GigModule
IPP Fabric PortsStandby Supervisor
Active Ports Disabled Ports
1+1 Uplink Mode
Redundant Supervisor Uplinks
XGStub ASIC
X2 or Twin-Gig
XGStub ASIC
X2 or Twin-Gig
IPP Fabric Ports Active Supervisor
Shared Backplane-Mode
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential4500 Bootcamp 16
X2 or Twin-GigModule
X2 or Twin-GigModule
XGStub ASIC
X2 or Twin-GigModule
XGStub ASIC
X2 or Twin-GigModule
IPP Fabric Ports
Front-Panel PortsRouted via Single
XGStub ASIC
Standby Supervisor
Shared Backplane-Mode
Redundant Uplinks Configurations
Supervisor
Configurations
Front-Panel
Port 1
Front-Panel
Port 2
Number of
Uplink Ports
Oversubscribed
Active Supervisor
Standby Supervisor
10GbE (X2)
10GbE (X2)
10GbE (X2)
10GbE (X2) 4 x 10GbE
Yes (2:1)
Yes (2:1)
Active Supervisor 10GbE (X2) 10GbE (X2) 2 x 10GbE Yes (2:1)
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential4500 Bootcamp 17
Active Supervisor
Standby Supervisor
10GbE (X2)
2 x 1GbE (SFP)
10GbE (X2)
2 x 1GbE (SFP)
2 x 10GbE
4 x GbE
Yes (2:1)
NO
Active Supervisor
Standby Supervisor
2 x 1GbE (SFP)
10GbE (X2)
2 x 1GbE (SFP)
10GbE (X2)
4 x GbE
2 x 10GbE
NO
Yes (2:1)
Active Supervisor
Standby Supervisor
2 x 1GbE (SFP)
2 x 1GbE (SFP)
2 x 1GbE (SFP)
2 x 1GbE (SFP) 8 x GbE
NO
NO
Agenda
� High Availability - Brief Intro / Evolution
� Catalyst 4500 System Resiliency – Redundancy Components
� Supervisor Uplinks / Shared Backplane Mode
� RPR and SSO Mode
� NSF – Non Stop Forwarding
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential4500 Bootcamp 18
� NSF – Non Stop Forwarding
� ISSU – In Service Software Upgrade
� Isolated CPU Queues / CoPP – Control Plane Policing
� FHRP – VRRP / HSRP / GLBP
� GOLD – Generic Online Diagnostics
� EEM – Embedded Event Manager
Redundant Systems – 4507R-E / 4510R-ESupervisor Engine redundancy is enabled by running the redundant supervisor Supervisor Engine redundancy is enabled by running the redundant supervisor engine in engine in RRoute oute PProcessor rocessor RRedundancy edundancy (RPR) (RPR) or or SStateful tateful SSwitchwitchOOverver (SSO) (SSO) modemode
4507R-E 4510R-E
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential4500 Bootcamp 19
The minimum ROMMON requirement for running SSO is The minimum ROMMON requirement for running SSO is Cisco IOS Release 12.1(Cisco IOS Release 12.1(20r20r))EW1EW1 or Cisco IOS Release 12.2(or Cisco IOS Release 12.2(20r20r))EW1EW1
Supervisors must be identicalSupervisors must be identical
Supervisors located in Slots 3 and 4Supported Supervisors: SupII+, SupII+10GE, SupIV, SupV, SupV-10GE, Sup6-E
Supervisors located in Slots 5 and 6Supported Supervisors: SupV, SupV, SupV-10GE, Sup6-E
RPR – Route Processor Redundancy
RPRRPR – Basic Mode of Redundancy
Simple algorithm determines which Supervisor is active
What is not Synchronized? What is not Synchronized? -- Running Configurations, Routing Table, FIB/Adjacency Running Configurations, Routing Table, FIB/Adjacency Table, MACTable, MAC--Address Table, Port StatisticsAddress Table, Port Statistics
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential4500 Bootcamp 20
Reset
Reset
Reset
Reset
Reset
Supervisor is active
Standby Supervisor does partial boot and suspends at the IOS init process
Upon Switchover Line Cards are Reset; Line Cards are Reset;
Traffic Interruption: 3030--40 seconds40 seconds
Fall Back Fall Back mode for SSO/ISSUmode for SSO/ISSU
Redundancy Configuration Check - RPR
4510R-E-1#sh moduleChassis Type : WS-C4510R-E
Power consumed by backplane : 40 Watts
Mod Ports Card Type Model Serial No.---+-----+--------------------------------------+------------------+-----------1 18 10GE (X2), 1000BaseX (SFP) WS-X4606-X2-E JAB1122023L2 48 10/100/1000BaseT POE E Series WS-X4648-RJ45V-E JAB1122021V3 48 10/100/1000BaseT Premium POE E Series WS-X4648-RJ45V+E JAE1129QM6P5 6 Sup 6-E 10GE (X2), 1000BaseX (SFP) WS-X45-SUP6-E JAB112500EF
4510R-E-1(config)#redundancy4510R-E-1(config-red)#mode rpr
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential4500 Bootcamp 21
5 6 Sup 6-E 10GE (X2), 1000BaseX (SFP) WS-X45-SUP6-E JAB112500EF6 6 Sup 6-E 10GE (X2), 1000BaseX (SFP) WS-X45-SUP6-E JAB112500EL9 24 10/100/1000BaseT (RJ45) WS-X4424-GB-RJ45 JAE10265HWV
M MAC addresses Hw Fw Sw Status--+--------------------------------+---+------------+----------------+---------1 001b.54fc.339d to 001b.54fc.33ae 0.3 Ok2 001b.54fc.4996 to 001b.54fc.49c5 0.3 Ok3 001c.58f8.1250 to 001c.58f8.127f 0.3 Ok5 001b.2a68.0080 to 001b.2a68.0085 0.3 12.2(33r)SG( 12.2(44)SG Ok6 001b.2a68.0086 to 001b.2a68.008b 0.3 12.2(33r)SG( 12.2(44)SG Ok9 0016.c73d.7860 to 0016.c73d.7877 1.8 Ok
Mod Redundancy role Operating mode Redundancy status----+-------------------+-------------------+----------------------------------5 Active Supervisor RPR Active6 Standby Supervisor RPR Standby cold
SSO – Stateful SwitchOver
SSOSSO – supported in Cisco IOS Release 12.2(46)SG with Sup6-E
SSOSSO allows Redundant Supervisors to run a allows Redundant Supervisors to run a statefulstateful IOS IOS and and statefulstateful applications applications to to exchange state in order to minimize outage at the time of switchover from Active to exchange state in order to minimize outage at the time of switchover from Active to Standby Supervisor.Standby Supervisor.
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential4500 Bootcamp 22
Default Redundancy Mode –Redundant Supervisor fully initialized
Upon Switchover Physical Links stay up Physical Links stay up -Protocols do not reset
Traffic Interruption: SubSub--Second (<Second (<150ms150ms))
IOS Images need to be identicalIOS Images need to be identical
Redundancy Configuration Check - SSO
4510R-E-1#sh moduleChassis Type : WS-C4510R-E
Power consumed by backplane : 40 Watts
Mod Ports Card Type Model Serial No.---+-----+--------------------------------------+------------------+-----------1 18 10GE (X2), 1000BaseX (SFP) WS-X4606-X2-E JAB1122023L2 48 10/100/1000BaseT POE E Series WS-X4648-RJ45V-E JAB1122021V3 48 10/100/1000BaseT Premium POE E Series WS-X4648-RJ45V+E JAE1129QM6P5 6 Sup 6-E 10GE (X2), 1000BaseX (SFP) WS-X45-SUP6-E JAB112500EF
4510R-E-1(config)#redundancy4510R-E-1(config-red)#mode sso
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential4500 Bootcamp 23
5 6 Sup 6-E 10GE (X2), 1000BaseX (SFP) WS-X45-SUP6-E JAB112500EF6 6 Sup 6-E 10GE (X2), 1000BaseX (SFP) WS-X45-SUP6-E JAB112500EL9 24 10/100/1000BaseT (RJ45) WS-X4424-GB-RJ45 JAE10265HWV
M MAC addresses Hw Fw Sw Status--+--------------------------------+---+------------+----------------+---------1 001b.54fc.339d to 001b.54fc.33ae 0.3 Ok2 001b.54fc.4996 to 001b.54fc.49c5 0.3 Ok3 001c.58f8.1250 to 001c.58f8.127f 0.3 Ok5 001b.2a68.0080 to 001b.2a68.0085 0.3 12.2(33r)SG( 12.2(44)SG Ok6 001b.2a68.0086 to 001b.2a68.008b 0.3 12.2(33r)SG( 12.2(44)SG Ok9 0016.c73d.7860 to 0016.c73d.7877 1.8 Ok
Mod Redundancy role Operating mode Redundancy status----+-------------------+-------------------+----------------------------------5 Active Supervisor SSO Active6 Standby Supervisor SSO Standby hot
SSO-Aware Features
Non-Exhaustive list of SSO-Aware Features
802.3 802.1p VTP HSRP
802.3u 802.1q Dynamic ARP Inspection MST/ PVST+
802.3x 802.1X DHCP Snooping Rapid-PVST
SSOSSO supports supports statefulstateful switchover of the following Layer 2 features. The state of switchover of the following Layer 2 features. The state of the features are preserved between both Active and Standby Supervisor Enginesthe features are preserved between both Active and Standby Supervisor Engines
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential4500 Bootcamp 24
802.3x 802.1X DHCP Snooping Rapid-PVST
802.3ab 802.1D IP Source Guard Spanning Tree Toolkit
802.3z 802.3af IGMP Snooping v1 / v2 Voice VLAN
802.3ad PAgP DTP (802.1q and ISL) Port Security
Unicast MAC
Filtering
ACL (VACLs,
PACLs, RACLs)
Multicast/Broadcast Storm
Control
QoS (DBL)
Agenda
� High Availability - Brief Intro / Evolution
� Catalyst 4500 System Resiliency – Redundancy Components
� Supervisor Uplinks / Shared Backplane Mode
� RPR and SSO Mode
� NSF – Non Stop Forwarding
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential4500 Bootcamp 25
� NSF – Non Stop Forwarding
� ISSU – In Service Software Upgrade
� Isolated CPU Queues / CoPP – Control Plane Policing
� FHRP – VRRP / HSRP / GLBP
� GOLD – Generic Online Diagnostics
� EEM – Embedded Event Manager
NSF – Non-Stop Forwarding
Following SSO Recovery and activation of Standby Supervisor – Synchronized ForwardingEngine continues to forward traffic continues to forward traffic based on existing Hardware FIB entries
Supervisor re-establishes adjacency indicating this is an NSF restartNSF restart
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential4500 Bootcamp 26
Peer Updates newly active Supervisor Routingprocesses with it’s routing information
Newly active Supervisor sends routing updates to the peer
RIB reattaches to FIB and Packet ProcessorForwarding Engine TCAMs updated with new FIB entries
No Route Flaps During RecoveryNo Route Flaps During Recovery
NSF – Non-Stop Forwarding
NSFNSF--awareness awareness –– if routing peers detect if routing peers detect than an NSF router can still forward than an NSF router can still forward packets when a supervisor switchover packets when a supervisor switchover occurs, this capability is referred to as NSFoccurs, this capability is referred to as NSF--aware. aware.
The The NSF aware NSF aware switch helps to send switch helps to send NSF-Aware
NSF-Aware
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential4500 Bootcamp 27
The The NSF aware NSF aware switch helps to send switch helps to send routing protocol information to the NSF routing protocol information to the NSF peer router.peer router.
NSFNSF--capable capable –– NSF works with SSO to NSF works with SSO to minimize the amount of time that a L3 minimize the amount of time that a L3 network is unavailable following a network is unavailable following a supervisor switchover by continuing to supervisor switchover by continuing to forward IP packets. forward IP packets.
The routing protocols recover routing The routing protocols recover routing information from peers and rebuild information from peers and rebuild CEFCEFtable. table.
No Route Flaps During RecoveryNo Route Flaps During Recovery
NSF-Capable /NSF-Aware
Enabling NSF Configuration – Routing
4510R-E-1(config)#router eigrp 1004510R-E-1(config-router)#nsf4510R-E-1(config-router)#timers nsf ?converge EIGRP time limit for convergence after switchoverroute-hold EIGRP hold time for routes learned from nsf peersignal EIGRP time limit for signaling NSF restart
4510R-E-1(config)#router ospf 1004510R-E-1(config-router)#nsf4510R-E-1(config-router)#nsf ?
EIGRP ExampleEIGRP Example
OSPF ExampleOSPF Example
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential4500 Bootcamp 28
4510R-E-1(config-router)#nsf ?cisco Cisco Non-stop forwardingietf IETF graceful restart
4510R-E-1(config-router)#nsf cisco ?enforce Cancel NSF restart when non-NSF-aware neighbors detectedhelper helper support
4510R-E-1(config-router)#nsf ietf ?helper helper supportrestart-interval Graceful restart interval
4510R-E-1(config-router)#bgp graceful-restart ?restart-time Set the max time needed to restart and come back upstalepath-time Set the max time to hold onto restarting peer's stale paths
BGP ExampleBGP Example
Agenda
• High Availability - Brief Intro / Evolution
• Catalyst 4500 System Resiliency – Redundancy Components
• Supervisor Uplinks / Shared Backplane Mode
• RPR and SSO Mode
• NSF – Non Stop Forwarding
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential4500 Bootcamp 29
• NSF – Non Stop Forwarding
• ISSU – In Service Software Upgrade
• Isolated CPU Queues / CoPP – Control Plane Policing
• FHRP – VRRP / HSRP / GLBP
• GOLD – Generic Online Diagnostics
• EEM – Embedded Event Manager
Catalyst 4500 – In Service Software Upgrade
12.2(46)SG
12.2(50)SG
Software Maintenance Windows are Software Maintenance Windows are significant case of downtimesignificant case of downtime
On redundant systems, the On redundant systems, the ISSUISSU process process allows the running IOS software to be allows the running IOS software to be upgraded while packet forwarding upgraded while packet forwarding continuescontinues
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential4500 Bootcamp 30
continuescontinues
ISSU mechanism leverages architecture ISSU mechanism leverages architecture for High Availability for High Availability -- NSF / SSONSF / SSO
Catalyst 4500 utilizes full image upgrades Catalyst 4500 utilizes full image upgrades for the addition of new features, defects, for the addition of new features, defects, and and PSIRTsPSIRTs
Increases network availability and reduces Increases network availability and reduces downtime caused by planned upgradesdowntime caused by planned upgrades
Targets Planned DowntimeDue to Software Upgrades
Catalyst 4500 - ISSU Stages
12.2(46)SG
12.2(46)SG
12.2(46)SG
12.2(50)SG
12.2(46)SG
12.2(50)SG
12.2(46)SG
12.2(50)SG
12.2(50)SG
12.2(50)SG
Initial State Final StateAbortVersionAbortVersion
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential4500 Bootcamp 31
LoadVersionLoadVersion RunVersion AcceptVersion CommitVersion
ISSU ISSU is a Four Step Processis a Four Step ProcessPossible to rollback (abort) up until you complete 4Possible to rollback (abort) up until you complete 4thth step (commit to final State)step (commit to final State)
Leverages NSF/SSO to implement Supervisor transitionLeverages NSF/SSO to implement Supervisor transitionRequires both images to be compatible for upgrade/downgradesRequires both images to be compatible for upgrade/downgrades
ISSU Stages: Step 1 - LoadVersion
12.2(46)SG
12.2(46)SG
12.2(46)SG
12.2(50)SGissu issu loadversionloadversion
Active SupervisorActive Supervisor
Old Image
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential4500 Bootcamp 32
Issue Issue issu issu loadversionloadversion to reboot Standby Supervisor with new imageto reboot Standby Supervisor with new image
If an incompatible image is detected and SSO mode is not achievable the switch If an incompatible image is detected and SSO mode is not achievable the switch automatically aborts the ISSU process and reboots Standby with previous versionautomatically aborts the ISSU process and reboots Standby with previous version
Standby Supervisor
Standby SupervisorNew Image
ISSU Stages: Step 2 - RunVersion
12.2(46)SG
12.2(50)SG
12.2(46)SG
12.2(50)SGissu runversionissu runversion
Active SupervisorStandby Supervisor
Old Image
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential4500 Bootcamp 33
Issue the Issue the issu runversion issu runversion to initiate an SSO failover to the to initiate an SSO failover to the Standby Supervisor running new image Standby Supervisor running new image
Old Active Supervisor reboots with the old image into standby mode Old Active Supervisor reboots with the old image into standby mode –– System still System still in SSO mode and rollback timer is startedin SSO mode and rollback timer is started
Standby Supervisor
Active SupervisorNew Image
ISSU Stages: Step 3 - AcceptVersion
12.2(46)SG
12.2(50)SG
12.2(46)SG
12.2(50)SGissu acceptversionissu acceptversion
Standby Supervisor
Standby Supervisor
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential4500 Bootcamp 34
Prior to issuing the Prior to issuing the issu acceptversion issu acceptversion the system will be counting down the the system will be counting down the rollback timerrollback timer
If If issu acceptversion issu acceptversion is not completed before rollback timer expires an automatic is not completed before rollback timer expires an automatic abort will occurabort will occur
Active Supervisor
Active Supervisor
ISSU Stages: Step 4 - CommitVersion
12.2(46)SG
12.2(50)SG
12.2(50)SG
12.2(50)SGissu issu commitversioncommitversion
Active Supervisor
Standby Supervisor
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential4500 Bootcamp 35
Once network is confirmed stable and change evaluation criteria are met issue the Once network is confirmed stable and change evaluation criteria are met issue the issu issu commitversioncommitversion commandcommand
OnOn committversioncommittversion the Standby Supervisor reboots and loads the new image the Standby Supervisor reboots and loads the new image coming up in Standby Mode coming up in Standby Mode –– New IOS Features are enabled at this pointNew IOS Features are enabled at this point
Standby Supervisor
Active Supervisor
ISSU: Rollback-Timer
12.2(46)SG
12.2(50)SG
Active Supervisor
On issuing the issu runversion issu runversion command the system activates the rollback timer
The Rollback-timer provides a mechanism to trigger an abort to return the switch to its original state if you lose
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential4500 Bootcamp 36
Standby Supervisor
original state if you lose connectivity during an ISSU
You can disable the rollback timer by setting the timer to “0”
4510R-E-1#show issu rollback-timer
Rollback Process State = Not in progressConfigured Rollback Time = 45:00
4510R-E-1(config)#issu set rollback-timer ?<0-7200> Rollback timer value
Medical Grade Network - ISSU Case Study
High Availability Testing
• GE Unity VLAN + Catalyst 4500 / 6500
• Mission Critical Data – Patient Monitors, Nursing Stations
– Monitors Communicate to each other via Same Broadcast Domain
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential4500 Bootcamp 37
other via Same Broadcast Domain
– Monitors Communicate with Central Nurse Station
– Nurses View Patient Monitors Simultaneously
• Goal - Characterize Application Impact during Supervisor Switchover for both Wired / Wireless connections
ISSU Case Study Topology
SiSi
SiSiSiSiSiSi
SiSiSiSiSiSi
SiSi SiSi
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential4500 Bootcamp 38
SiSi
SiSi
SiSi SiSiSiSi SiSi
Medical Grade Network -ISSU Case Study
High Availability Testing
Results
• Code upgrade without loss of vital sign history
• ~150ms switchover
• Clear Monitors (no glitch)
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential4500 Bootcamp 39
Customer Testimony
• Managed Planned Time Solution for Critical Sites (Nurse Station)
Agenda
• High Availability - Brief Intro / Evolution
• Catalyst 4500 System Resiliency – Redundancy Components
• Supervisor Uplinks / Shared Backplane Mode
• RPR and SSO Mode
• NSF – Non Stop Forwarding
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential4500 Bootcamp 40
• NSF – Non Stop Forwarding
• ISSU – In Service Software Upgrade
• Isolated CPU Queues / CoPP – Control Plane Policing
• FHRP – VRRP / HSRP / GLBP
• GOLD – Generic Online Diagnostics
• EEM – Embedded Event Manager
Packet Processing – Hardware / SoftwareHardware Process Features are defined as Data Plane Features, while Software processed features are defined as Control Plane Features
CPU Sub-System
Switching Sub-System
CPU
Packet Memory
CPU FPGA
SystemDRAM
Control-Plane
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential4500 Bootcamp 41
Switching Sub-System
Packet Processor
Forwarding Engine
Packet Memory
Line Card
Port
Data Traffic Forwarded via
Switching ASICs(Data-Plane)
CPU Packet ProcessingControl Plane Features are processed by the switch CPU so there is a limited amount of processing power available for these tasks
CPU Sub-System
Switching Sub-System
CPU
Packet Memory
CPU FPGA
SystemDRAM
Control-Plane
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential4500 Bootcamp 42
Switching Sub-System
Packet Processor
Forwarding Engine
Packet Memory
Line Card
Port
Control PacketsKeep-alives
Protocol UpdatesTelnet, ICMP, ARP,
IP Options
Packets Requiring CPU Processing –
Example:Host
Learning…
Data-Plane
CPU Queue IsolationSupervisor 6-E has 64 internal CPU queues. Packets of different events go to different queues. This architecture contributes to event isolation.
CPU Sub-System
Switching Sub-System
CPU
Packet Memory
CPU FPGA
SystemDRAM
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential4500 Bootcamp 43
Switching Sub-System
Packet Processor
Forwarding Engine
Packet Memory
Line Card
Port
All Packets destined to CPU need to be queued
Example: Telnet, CDP,
ACL Log, RPF Failure,
MTU Failure…
CPU Port on Packet ProcessorLogically divided
in 8 Subports each containing 8
queues (64)
Control Plane Policing (CoPP) – Why? If the CPU is heavily stressed all other processes stop. This can have a severe Impact on the switch if the CPU is overwhelmed for a period of time
CPU Sub-System
Switching Sub-System
CPU
Packet Memory
CPU FPGA
SystemDRAM
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential4500 Bootcamp 44
Switching Sub-System
Packet Processor
Forwarding Engine
Packet Memory
Line Card
Port
CPU Overload
Drop Routing PeersFailure to send Protocol Updates
CLI Locks UpNo ARPs Processed
Control Plane Policing CoPP Function Hardware-based mechanisms built to rate-limit and protect the CPU bound traffic
CPU Sub-System
Switching Sub-System
CPU
Packet Memory
CPU FPGA
SystemDRAM
Control-Plane Separate Entity
Packets Conform to Control-Plane
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential4500 Bootcamp 45
Switching Sub-System
Packet Processor
Forwarding Engine
Packet Memory
Line Card
Port
Function handled via Forwarding EngineUses available Policers from QoS CAM
Output Policers applied to packets destined to CPU port
Apply Policy via Pre-configured System Traffic
Types and User Configurable
Traffic Types
to Control-Plane Service Policy
Agenda
• High Availability - Brief Intro / Evolution
• Catalyst 4500 System Resiliency – Redundancy Components
• Supervisor Uplinks / Shared Backplane Mode
• RPR and SSO Mode
• NSF – Non Stop Forwarding
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential4500 Bootcamp 46
• NSF – Non Stop Forwarding
• ISSU – In Service Software Upgrade
• Isolated CPU Queues / CoPP – Control Plane Policing
• FHRP – VRRP / HSRP / GLBP
• GOLD – Generic Online Diagnostics
• EEM – Embedded Event Manager
FHRP – First Hop Redundancy Protocols
HSRP – Hot Standby Router Protocol
IP Routing Redundancy - Allows for transparent-failover at the first hop IP Router for end stations
VRRP – Virtual Router Redundancy ProtocolMulti-Vendor OperabilityUses a default hello timer of 1 second with a hold timer of 3 seconds
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential4500 Bootcamp 47
HSRP – Hot Standby Router ProtocolUses a default hello timer of 3 seconds with a hold timer of 10 seconds.
GLBP - Gateway Load-Balancing ProtocolIdentical Features to HSRP, but allows an active-active connection that adds load-balancing features. Optimized Load-Sharing across all uplink interfaces
First Hop Redundancy
FHRP using VRRP – Functionality
IP: 10.0.0.254MAC: 0000.0c12.3456vIP: 10.0.0.10vMAC: 0000.5e00.0101
IP: 10.0.0.253MAC: 0000.0c78.9abcvIP: 10.0.0.10vMAC: 0000.5e00.0101
VRRP Active VRRP Backup
A group of routers function as one VirtualRouter by sharing ONE virtual IP address and ONE virtual MAC address
One (master) router performs packet forwarding for local hosts
Remaining routers act as a “back-up” in case the master router fails
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential4500 Bootcamp 48
First Hop Redundancy
IP: 10.0.0.1MAC: aaaa.aaaa.aa01GW: 10.0.0.10ARP: 0000.5e00.0101
IP: 10.0.0.2MAC: aaaa.aaaa.aa02GW: 10.0.0.10ARP: 0000.5e00.0101
IP: 10.0.0.3MAC: aaaa.aaaa.aa03GW: 10.0.0.10ARP: 0000.5e00.0101
case the master router fails
Backup Routers remain idleMaster
interface Vlan4ip address 10.120.4.1 255.255.255.0ip helper-address 10.121.0.5no ip redirectsvrrp 1 description Master VRRPvrrp 1 ip 10.0.0.10vrrp 1 timers advertise msec 250vrrp 1 preemt delay minimum 180
FHRP using HSRP – Functionality
IP: 10.0.0.254MAC: 0000.0c12.3456vIP: 10.0.0.10vMAC: 0000.0c07.ac00
IP: 10.0.0.253MAC: 0000.0c78.9abcvIP: 10.0.0.10vMAC: 0000.0c07.ac00
HSRP Active HSRP Standby
A group of routers function as one VirtualRouter by sharing ONE virtual IP address and ONE virtual MAC address
One (active) router performs packet forwarding for local hosts
Remaining routers act as a “Standby” in case the active router fails
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential4500 Bootcamp 49
First Hop Redundancy
IP: 10.0.0.1MAC: aaaa.aaaa.aa01GW: 10.0.0.10ARP: 0000.0c07.ac00
IP: 10.0.0.2MAC: aaaa.aaaa.aa02GW: 10.0.0.10ARP: 0000.0c07.ac00
IP: 10.0.0.3MAC: aaaa.aaaa.aa03GW: 10.0.0.10ARP: 0000.0c07.ac00
case the active router fails
Standby Routers remain idleActive
interface Vlan5description Data VLAN for Accessip address 10.1.5.3 255.255.255.0ip helper-address 10.5.10.20standby 1 ip 10.1.5.1standby 1 timers msec 250 msec 750standby 1 priority 150standby 1 preemptstandby 1 preempt delay minimum 180
FHRP using GLBP – Functionality
IP: 10.120.4.2MAC: 0000.0c12.3456vIP: 10.120.4.1vMAC: 0007.b400.0101
IP: 10.120.4.3MAC: 0000.0c78.9abcvIP: 10.120.4.1vMAC: 0007.b400.0102
GLBP - AVG/AVF/SVF
Multiple Modes:AVG – Active Virtual GatewayAVF – Active Virtual ForwarderSVF – Secondary Virtual ForwarderGLBP - AVF/SVF
A group of routers function as one Virtual Router by sharing ONE virtual IP address and Multiple virtual MAC address
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential4500 Bootcamp 50
First Hop Redundancy
IP: 10.120.4.101MAC: aaaa.aaaa.aa01GW: 10.120.4.1ARP: 0007.b400.0101
IP: 10.120.4.102MAC: aaaa.aaaa.aa02GW: 10.120.4.1ARP: 0007.b400.0102
IP: 10.120.4.103MAC: aaaa.aaaa.aa03GW: 10.120.4.1ARP: 0007.b400.0101
AVG balances client ARP requests between Virtual Forwarders
Active
interface Vlan4ip address 10.120.4.2 255.255.255.0ip helper-address 10.5.10.20glbp 1 ip 10.120.4.1glbp 1 timers msec 250 msec 750glbp 1 priority 150glbp 1 preemptglbp 1 preempt delay minimum 180
Active
Agenda
• High Availability - Brief Intro / Evolution
• Catalyst 4500 System Resiliency – Redundancy Components
• Supervisor Uplinks / Shared Backplane Mode
• RPR and SSO Mode
• NSF – Non Stop Forwarding
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential4500 Bootcamp 51
• NSF – Non Stop Forwarding
• ISSU – In Service Software Upgrade
• Isolated CPU Queues / CoPP – Control Plane Policing
• FHRP – VRRP / HSRP / GLBP
• GOLD – Generic Online Diagnostics
• EEM – Embedded Event Manager
Generic Online Diagnostics – What is it?GOLD defines a common framework for diagnostics operations across Cisco Platforms running IOS software. The goal is to check the health of hardware components and verify proper operation of the system control and data plane at run-time and boot…
Runtime DiagnosticsLine Card Module, Temperature, Power Supply, Fan Tray
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential4500 Bootcamp 52
Power-On DiagnosticsSupervisor, BackplaneL2 ASIC, L3 ASICMemory, CPU, Port
Generic Online Diagnostics – GOLD (POST)
Module 5
CPUMemory
Packet Processor
Forwarding Engine
CPU Subsystem TestSEEPROM
1
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential4500 Bootcamp 53
CPUMemory
Packet Processor
Forwarding Engine
Module 6
Generic Online Diagnostics – GOLD (POST)
Module 5
CPUMemory
Packet Processor
Forwarding Engine
CPU Subsystem TestSEEPROM
1
Traffic: L3 Loopback Test
2
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential4500 Bootcamp 54
CPUMemory
Packet Processor
Forwarding Engine
Module 6
Generic Online Diagnostics – GOLD (POST)
Module 5
CPUMemory
Packet Processor
Forwarding Engine
CPU Subsystem TestSEEPROM
1
Traffic: L3 Loopback Test
2
3
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential4500 Bootcamp 55
CPUMemory
Packet Processor
Forwarding Engine
Module 6
3
Traffic: L2 Loopback Test
Generic Online Diagnostics – GOLD (POST)
Module 5
CPUMemory
Packet Processor
Forwarding Engine
CPU Subsystem TestSEEPROM
1
Traffic: L3 Loopback Test
2
3
4
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential4500 Bootcamp 56
CPUMemory
Packet Processor
Forwarding Engine
Module 6
3
Traffic: L2 Loopback Test
Switching Sub-SystemPacket Memory Test
Generic Online Diagnostics – GOLD (POST)
Module 5
CPUMemory
Packet Processor
Forwarding Engine
CPU Subsystem TestSEEPROM
1
Traffic: L3 Loopback Test
2
3
4Power-on-self-test for Module 5: WS-X45-SUP6-ETest Status: (. = Pass, F = Fail, U = Untested)
CPU Subsystem Tests ...seeprom: Pass
1
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential4500 Bootcamp 57
CPUMemory
Packet Processor
Forwarding Engine
Module 6
3
Traffic: L2 Loopback Test
Switching Sub-SystemPacket Memory Test
Traffic: L3 Loopback ...Test Results: Pass
Traffic: L2 Loopback ...Test Results: Pass
Switching Subsystem Memory ...Packet Memory Test Results: Pass
Module 5 Passed
2
3
4
Agenda
• High Availability - Brief Intro / Evolution
• Catalyst 4500 System Resiliency – Redundancy Components
• Supervisor Uplinks / Shared Backplane Mode
• RPR and SSO Mode
• NSF – Non Stop Forwarding
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential4500 Bootcamp 58
• NSF – Non Stop Forwarding
• ISSU – In Service Software Upgrade
• Isolated CPU Queues / CoPP – Control Plane Policing
• FHRP – VRRP / HSRP / GLBP
• GOLD – Generic Online Diagnostics
• EEM – Embedded Event Manager
Embedded Event ManagerEmbedded Event Manager provides a means to automate the operational management in real time. EEM monitors for specific events on the switch and can invoke predefined actions to correct, take remedial action and report an event to network operations…
IOS Policy Director
EEM Tcl Policy
Tcl Shell
EEM Applet Policy
IOS Subsystems
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential4500 Bootcamp 59
Application
CLI
Configuration
Counters
Environment
I/F Counters
IOS Watchdog
OIR
SNMP
RF
Syslog
Timer
IOS Event Detectors
IOS Embedded Event Manager Server
Embedded Event Manager
IOS Embedded Event Manager Server
IOS Policy Director
EEM Tcl Policy
Tcl Shell
EEM Applet Policy IOS Subsystems
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential4500 Bootcamp 60
Application
CLI
Configuration
Counters
Environments
I/F Counters
IOS Watchdog
OIR
SNMP
RF
Syslog
Timer
IOS Event Detectors
IOS Embedded Event Manager Server
Embedded Event Manager – Applet Example
� Monitor syslog for “line protocol down” event on uplink
� After 10th iteration of event…
Issue debug command, and save output to bootflash
Take port out of service
Supervisor6-E# show running-config | begin event manager applet
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential4500 Bootcamp 61
Supervisor6-E# show running-config | begin event manager applet
event manager applet UplinkFlap49
event syslog occurs 10 pattern "%LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet1/49, changed state to down"
action 1.0 cli command "enable"
action 1.1 cli command "show platform software interface ten1/49 all | redirect bootflash:flap-debug.txt"
action 2.0 cli command "config t"
action 2.1 cli command "interface ten1/49"
action 2.2 cli command "shut"
action 2.3 cli command "end"
%EC-5-UNBUNDLE: Interface Te1/49 left the port-channel Po1
%LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet1/49, changed state to down
%SYS-5-CONFIG_I: Configured from console by on vty0(EEM:UplinkFlap49)
%LINK-5-CHANGED: Interface TenGigabitEthernet1/49, changed state to administratively down
Embedded Event Manager – Applet Example
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential4500 Bootcamp 62
administratively down
Supervisor6-E# dir bootflash:flap-debug.txt
Directory of bootflash:/flap-debug.txt
2 -rw- 3435 Mar 17 2008 21:26:38 +00:00 flap-debug.txt
61341696 bytes total (48284856 bytes free)
Supervisor6-E# more bootflash:flap-debug.txt
< snip >
View the results!
EEM helps with the Green Initiative
Example of how an EEM script can be used to improve
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential4500 Bootcamp 63
used to improve power usage…
References
In Service Software Upgrade (ISSU)http://www.cisco.com/en/US/docs/switches/lan/catalyst4500/12.2/50sg/configuration/guide/issu.html
Supervisor Redundancy Using RPR and SSOhttp://www.cisco.com/en/US/docs/switches/lan/catalyst4500/12.2/50sg/configuration/guide/RPR.html
Cisco NSF with SSO Supervisor Redundancy
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential4500 Bootcamp 64
Cisco NSF with SSO Supervisor Redundancyhttp://www.cisco.com/en/US/docs/switches/lan/catalyst4500/12.2/50sg/configuration/guide/NSFwSSO.html
Control Plane Policing (CoPP)http://www.cisco.com/en/US/docs/switches/lan/catalyst4500/12.2/50sg/configuration/guide/cntl_pln.html