Data Center Network Infrastructure Test Engineer’s Journal Cisco · PDF fileTest...

Data Center Network Infrastructure Test Engineer’s Journal Cisco Nexus OSPF Interoperability with Brocade VCS Fabric

Solutions Lab Test Engineer’s Journal 1

Contents Overview ....................................................................................................................................................... 2

Key Contributors ........................................................................................................................................... 2

Test Configuration ......................................................................................................................................... 2

Test Cases ...................................................................................................................................................... 3

Test Case 1: OSPF Interoperability Stress & Scalability Testing ........................................................... 3

Test Case 2: Nexus 7K Supervisor Failover with OSPF ......................................................................... 7

Test Case 3: Nexus 7K Supervisor Failback with OSPF ......................................................................... 8

Test Case 4: Nexus 7000 and IOS OSPF Loop Prevention .................................................................... 8

Test Case 5: NX-OS OSPF Manual Restart in VDC that is part of our VCS L3 Fabric w/OSPF ............... 9

Test Case 6: VCS 3.0 - N7K OSPF Static Route Test ............................................................................ 10

Appendix A – Notes about OSPF and NX-OS ............................................................................................... 12

Appendix B – NX-OS OSPFv2 Verification Commands ................................................................................ 13

Appendix C – Things you should know about NX-OS and VRF’s ................................................................. 14

Appendix D – NX-OS VRF Verification Commands ...................................................................................... 15

Appendix E - Example of configuring STC for OSPF route injection. ........................................................... 16

Appendix F – Non-Stop Forwarding and Graceful Restarts with NX-OS OSPF ............................................ 27

OSPFv2 Stateless Restart ........................................................................................................................ 27

OSPFv2 Graceful Restart on a Switchover .............................................................................................. 27

OSPFv2 Graceful Restart on an OSFPv2 Process Failure ......................................................................... 27

High Availability and Graceful Restart .................................................................................................... 27

Appendix G – VCS 3.0 Fabric and Cisco Nexus 7K OSPF Interoperability Topology .................................... 29

Test bed setup ......................................................................................................................................... 29



Overview This document is the test engineer’s journal with notes complied during testing. Please see the companion Validation Test Summary: Cisco Nexus OSPF Interoperability with Brocade VCS Fabric document for additional details about this test.

References

Data Center Infrastructure-Validation Test: Cisco Nexus OSPF Interoperability

with Brocade VCS Fabric

Key Contributors The content in this guide was provided by the following key contributors.

Test Engineer: Mike O’Conner, Strategic Solution Lab

Test Configuration The topology used for this test is shown in the figure below.

Validation Test Configuration

http://community.brocade.com/docs/DOC-3131

http://community.brocade.com/docs/DOC-3131



Note: There is a connection to Juniper OSPF shown, but this was not included in this test.

Test Cases

Test Case 1: OSPF Interoperability Stress & Scalability Testing

DUT Nexus 7K VCS

Purpose The purpose of this test is to show that OSPF interoperability between the VDX and N7K. OSPF routes injected into the network via the test equipment should be seen on both the VDX and the N7K.

Test Procedure We going to start with a fairly simple STC setup with a tester port connect to the VCS L3 OSPF side and one tester port connected to the N7K OSPF on the 60.60.60.x network. We’ll start with just injecting 50 additional routes into our OSPF interop network and the verification is that from both the VCS and Nexus 7K we should see our additional OSPF routes. For this initial stress test, we’ll let the Spirent test Center run in “continuous” mode and we can do the checks while this is going on as shown below.

Step 1: VDX OSPF Route Injection Verification VDX6720-75# show ip route Total number of IP routes: 60 Type Codes - B:BGP D:Connected I:ISIS O:OSPF R:RIP S:Static; Cost - Dist/Metric BGP Codes - i:iBGP e:eBGP ISIS Codes - L1:Level-1 L2:Level-2 OSPF Codes - i:Inter Area 1:External Type 1 2:External Type 2 s:Sham Link Destination Gateway Port Cost Type Uptime 1 0.0.0.0/0 10.18.233.1 mgmt 1 1/1 S 22h45m 2 1.0.0.0/24 60.60.60.20 Te 1/0/12 110/2 O 2m14s 3 1.0.1.0/24 60.60.60.20 Te 1/0/12 110/2 O 2m14s 4 1.0.2.0/24 60.60.60.20 Te 1/0/12 110/2 O 2m14s 5 1.0.3.0/24 60.60.60.20 Te 1/0/12 110/2 O 2m14s 6 1.0.4.0/24 60.60.60.20 Te 1/0/12 110/3 O 2m14s 7 1.0.5.0/24 60.60.60.20 Te 1/0/12 110/3 O 2m14s 8 1.0.6.0/24 60.60.60.20 Te 1/0/12 110/3 O 2m14s 9 1.0.7.0/24 60.60.60.20 Te 1/0/12 110/3 O 2m14s



10 1.0.8.0/24 60.60.60.20 Te 1/0/12 110/3 O 2m14s 11 1.0.9.0/24 60.60.60.20 Te 1/0/12 110/3 O 2m14s 12 1.0.10.0/24 60.60.60.20 Te 1/0/12 110/3 O 2m14s 13 1.0.11.0/24 60.60.60.20 Te 1/0/12 110/3 O 2m14s 14 1.0.12.0/24 60.60.60.20 Te 1/0/12 110/3 O 2m14s 15 1.0.13.0/24 60.60.60.20 Te 1/0/12 110/3 O 2m14s 16 1.0.14.0/24 60.60.60.20 Te 1/0/12 110/3 O 2m14s 17 1.0.15.0/24 60.60.60.20 Te 1/0/12 110/3 O 2m14s 18 1.0.16.0/24 60.60.60.20 Te 1/0/12 110/4 O 2m14s 19 1.0.17.0/24 60.60.60.20 Te 1/0/12 110/4 O 2m14s 20 1.0.18.0/24 60.60.60.20 Te 1/0/12 110/4 O 2m14s 21 1.0.19.0/24 60.60.60.20 Te 1/0/12 110/4 O 2m14s 22 1.0.20.0/24 60.60.60.20 Te 1/0/12 110/4 O 2m14s 23 1.0.21.0/24 60.60.60.20 Te 1/0/12 110/4 O 2m14s 24 1.0.22.0/24 60.60.60.20 Te 1/0/12 110/4 O 2m14s 25 1.0.23.0/24 60.60.60.20 Te 1/0/12 110/4 O 2m14s 26 1.0.24.0/24 60.60.60.20 Te 1/0/12 110/4 O 2m14s 27 1.0.25.0/24 40.40.40.2 Te 1/0/3 110/6 O 1m58s 28 1.0.26.0/24 40.40.40.2 Te 1/0/3 110/6 O 1m58s 29 1.0.27.0/24 40.40.40.2 Te 1/0/3 110/6 O 1m58s 30 1.0.28.0/24 40.40.40.2 Te 1/0/3 110/6 O 1m58s 31 1.0.29.0/24 40.40.40.2 Te 1/0/3 110/7 O 1m58s 32 1.0.30.0/24 40.40.40.2 Te 1/0/3 110/7 O 1m58s 33 1.0.31.0/24 40.40.40.2 Te 1/0/3 110/7 O 1m58s 34 1.0.32.0/24 40.40.40.2 Te 1/0/3 110/7 O 1m58s 35 1.0.33.0/24 40.40.40.2 Te 1/0/3 110/7 O 1m58s 36 1.0.34.0/24 40.40.40.2 Te 1/0/3 110/7 O 1m58s 37 1.0.35.0/24 40.40.40.2 Te 1/0/3 110/7 O 1m58s 38 1.0.36.0/24 40.40.40.2 Te 1/0/3 110/7 O 1m58s 39 1.0.37.0/24 40.40.40.2 Te 1/0/3 110/7 O 1m58s 40 1.0.38.0/24 40.40.40.2 Te 1/0/3 110/7 O 1m58s 41 1.0.39.0/24 40.40.40.2 Te 1/0/3 110/7 O 1m58s 42 1.0.40.0/24 40.40.40.2 Te 1/0/3 110/7 O 1m58s 43 1.0.41.0/24 40.40.40.2 Te 1/0/3 110/8 O 1m58s 44 1.0.42.0/24 40.40.40.2 Te 1/0/3 110/8 O 1m58s 45 1.0.43.0/24 40.40.40.2 Te 1/0/3 110/8 O 1m58s 46 1.0.44.0/24 40.40.40.2 Te 1/0/3 110/8 O 1m58s 47 1.0.45.0/24 40.40.40.2 Te 1/0/3 110/8 O 1m58s 48 1.0.46.0/24 40.40.40.2 Te 1/0/3 110/8 O 1m58s 49 1.0.47.0/24 40.40.40.2 Te 1/0/3 110/8 O 1m58s 50 1.0.48.0/24 40.40.40.2 Te 1/0/3 110/8 O 1m58s 51 1.0.49.0/24 40.40.40.2 Te 1/0/3 110/8 O 1m58s 52 1.1.1.0/24 DIRECT Ve 100 0/0 D 22h45m 53 2.2.2.0/24 1.1.1.200 Ve 100 110/2 O 22h44m 54 10.10.10.20/32 DIRECT Lo 1 0/0 D 22h45m 55 10.18.233.0/24 DIRECT mgmt 1 0/0 D 22h46m



56 20.20.20.0/24 1.1.1.200 Ve 100 110/2 O 22h44m 57 30.30.30.0/24 1.1.1.200 Ve 100 110/2 O 22h44m 58 40.40.40.0/24 DIRECT Te 1/0/3 0/0 D 22h45m 59 50.50.50.0/24 DIRECT Te 1/0/4 0/0 D 1h15m 60 60.60.60.0/24 DIRECT Te 1/0/12 0/0 D 13m43s 60.60.60.10 is the STC configure for OSPF and emulating lots of OSPF routes into our network.

Step 2: Nexus 7K OSPF Route Injection Verification Nexus7K-Nexus7K-2# sh ip route IP Route Table for VRF "default" '*' denotes best ucast next-hop '**' denotes best mcast next-hop '[x/y]' denotes [preference/metric] '%<string>' in via output denotes VRF <string> 1.0.0.0/24, ubest/mbest: 1/0 *via 40.40.40.1, Eth1/18, [110/6], 00:04:04, ospf-1, intra 1.0.1.0/24, ubest/mbest: 1/0 *via 40.40.40.1, Eth1/18, [110/6], 00:04:04, ospf-1, intra 1.0.2.0/24, ubest/mbest: 1/0 *via 40.40.40.1, Eth1/18, [110/6], 00:04:04, ospf-1, intra 1.0.3.0/24, ubest/mbest: 1/0 *via 40.40.40.1, Eth1/18, [110/6], 00:04:04, ospf-1, intra 1.0.4.0/24, ubest/mbest: 1/0 *via 40.40.40.1, Eth1/18, [110/7], 00:04:04, ospf-1, intra 1.0.5.0/24, ubest/mbest: 1/0 *via 40.40.40.1, Eth1/18, [110/7], 00:04:04, ospf-1, intra 1.0.6.0/24, ubest/mbest: 1/0 *via 40.40.40.1, Eth1/18, [110/7], 00:04:04, ospf-1, intra 1.0.7.0/24, ubest/mbest: 1/0 *via 40.40.40.1, Eth1/18, [110/7], 00:04:04, ospf-1, intra 1.0.8.0/24, ubest/mbest: 1/0 *via 40.40.40.1, Eth1/18, [110/7], 00:04:04, ospf-1, intra 1.0.9.0/24, ubest/mbest: 1/0 *via 40.40.40.1, Eth1/18, [110/7], 00:04:04, ospf-1, intra 1.0.10.0/24, ubest/mbest: 1/0 *via 40.40.40.1, Eth1/18, [110/7], 00:04:04, ospf-1, intra 1.0.11.0/24, ubest/mbest: 1/0 *via 40.40.40.1, Eth1/18, [110/7], 00:04:04, ospf-1, intra 1.0.12.0/24, ubest/mbest: 1/0 *via 40.40.40.1, Eth1/18, [110/7], 00:04:04, ospf-1, intra 1.0.13.0/24, ubest/mbest: 1/0 *via 40.40.40.1, Eth1/18, [110/7], 00:04:04, ospf-1, intra 1.0.14.0/24, ubest/mbest: 1/0 *via 40.40.40.1, Eth1/18, [110/7], 00:04:04, ospf-1, intra 1.0.15.0/24, ubest/mbest: 1/0 *via 40.40.40.1, Eth1/18, [110/7], 00:04:04, ospf-1, intra 1.0.16.0/24, ubest/mbest: 1/0 *via 40.40.40.1, Eth1/18, [110/8], 00:04:04, ospf-1, intra 1.0.17.0/24, ubest/mbest: 1/0 *via 40.40.40.1, Eth1/18, [110/8], 00:04:04, ospf-1, intra 1.0.18.0/24, ubest/mbest: 1/0 *via 40.40.40.1, Eth1/18, [110/8], 00:04:04, ospf-1, intra 1.0.19.0/24, ubest/mbest: 1/0 *via 40.40.40.1, Eth1/18, [110/8], 00:04:04, ospf-1, intra 1.0.20.0/24, ubest/mbest: 1/0 *via 40.40.40.1, Eth1/18, [110/8], 00:04:04, ospf-1, intra 1.0.21.0/24, ubest/mbest: 1/0 *via 40.40.40.1, Eth1/18, [110/8], 00:04:04, ospf-1, intra



1.0.22.0/24, ubest/mbest: 1/0 *via 40.40.40.1, Eth1/18, [110/8], 00:04:04, ospf-1, intra 1.0.23.0/24, ubest/mbest: 1/0 *via 40.40.40.1, Eth1/18, [110/8], 00:04:04, ospf-1, intra 1.0.24.0/24, ubest/mbest: 1/0 *via 40.40.40.1, Eth1/18, [110/8], 00:04:04, ospf-1, intra 1.0.25.0/24, ubest/mbest: 1/0 *via 60.60.60.21, Eth1/17, [110/5], 00:03:57, ospf-1, intra 1.0.26.0/24, ubest/mbest: 1/0 *via 60.60.60.21, Eth1/17, [110/5], 00:03:57, ospf-1, intra 1.0.27.0/24, ubest/mbest: 1/0 *via 60.60.60.21, Eth1/17, [110/5], 00:03:57, ospf-1, intra 1.0.28.0/24, ubest/mbest: 1/0 *via 60.60.60.21, Eth1/17, [110/5], 00:03:57, ospf-1, intra 1.0.29.0/24, ubest/mbest: 1/0 *via 60.60.60.21, Eth1/17, [110/6], 00:03:57, ospf-1, intra 1.0.30.0/24, ubest/mbest: 1/0 *via 60.60.60.21, Eth1/17, [110/6], 00:03:57, ospf-1, intra 1.0.31.0/24, ubest/mbest: 1/0 *via 60.60.60.21, Eth1/17, [110/6], 00:03:57, ospf-1, intra 1.0.32.0/24, ubest/mbest: 1/0 *via 60.60.60.21, Eth1/17, [110/6], 00:03:57, ospf-1, intra 1.0.33.0/24, ubest/mbest: 1/0 *via 60.60.60.21, Eth1/17, [110/6], 00:03:57, ospf-1, intra 1.0.34.0/24, ubest/mbest: 1/0 *via 60.60.60.21, Eth1/17, [110/6], 00:03:57, ospf-1, intra 1.0.35.0/24, ubest/mbest: 1/0 *via 60.60.60.21, Eth1/17, [110/6], 00:03:57, ospf-1, intra 1.0.36.0/24, ubest/mbest: 1/0 *via 60.60.60.21, Eth1/17, [110/6], 00:03:57, ospf-1, intra 1.0.37.0/24, ubest/mbest: 1/0 *via 60.60.60.21, Eth1/17, [110/6], 00:03:57, ospf-1, intra 1.0.38.0/24, ubest/mbest: 1/0 *via 60.60.60.21, Eth1/17, [110/6], 00:03:57, ospf-1, intra 1.0.39.0/24, ubest/mbest: 1/0 *via 60.60.60.21, Eth1/17, [110/6], 00:03:57, ospf-1, intra 1.0.40.0/24, ubest/mbest: 1/0 *via 60.60.60.21, Eth1/17, [110/6], 00:03:57, ospf-1, intra 1.0.41.0/24, ubest/mbest: 1/0 *via 60.60.60.21, Eth1/17, [110/7], 00:03:57, ospf-1, intra 1.0.42.0/24, ubest/mbest: 1/0 *via 60.60.60.21, Eth1/17, [110/7], 00:03:57, ospf-1, intra 1.0.43.0/24, ubest/mbest: 1/0 *via 60.60.60.21, Eth1/17, [110/7], 00:03:57, ospf-1, intra 1.0.44.0/24, ubest/mbest: 1/0 *via 60.60.60.21, Eth1/17, [110/7], 00:03:57, ospf-1, intra 1.0.45.0/24, ubest/mbest: 1/0 *via 60.60.60.21, Eth1/17, [110/7], 00:03:57, ospf-1, intra 1.0.46.0/24, ubest/mbest: 1/0 *via 60.60.60.21, Eth1/17, [110/7], 00:03:57, ospf-1, intra 1.0.47.0/24, ubest/mbest: 1/0 *via 60.60.60.21, Eth1/17, [110/7], 00:03:57, ospf-1, intra 1.0.48.0/24, ubest/mbest: 1/0 *via 60.60.60.21, Eth1/17, [110/7], 00:03:57, ospf-1, intra 1.0.49.0/24, ubest/mbest: 1/0 *via 60.60.60.21, Eth1/17, [110/7], 00:03:57, ospf-1, intra 1.1.1.0/24, ubest/mbest: 1/0, attached *via 1.1.1.2, Lo0, [0/0], 5d03h, direct 1.1.1.2/32, ubest/mbest: 1/0, attached *via 1.1.1.2, Lo0, [0/0], 5d03h, local 2.2.2.0/24, ubest/mbest: 1/0 *via 20.20.20.1, Eth1/27, [110/5], 23:07:51, ospf-1, intra 10.10.10.0/24, ubest/mbest: 1/0, attached *via 10.10.10.1, Po1, [0/0], 6d00h, direct 10.10.10.1/32, ubest/mbest: 1/0, attached *via 10.10.10.1, Po1, [0/0], 6d00h, local 20.20.20.0/24, ubest/mbest: 1/0, attached *via 20.20.20.2, Eth1/27, [0/0], 23:08:47, direct 20.20.20.2/32, ubest/mbest: 1/0, attached *via 20.20.20.2, Eth1/27, [0/0], 23:08:47, local 30.30.30.0/24, ubest/mbest: 1/0 *via 20.20.20.1, Eth1/27, [110/5], 23:07:51, ospf-1, intra 40.40.40.0/24, ubest/mbest: 1/0, attached *via 40.40.40.2, Eth1/18, [0/0], 22:47:21, direct 40.40.40.2/32, ubest/mbest: 1/0, attached *via 40.40.40.2, Eth1/18, [0/0], 22:47:21, local 50.50.50.0/24, ubest/mbest: 1/0 *via 40.40.40.1, Eth1/18, [110/5], 01:17:46, ospf-1, intra 60.60.60.0/24, ubest/mbest: 1/0, attached *via 60.60.60.11, Eth1/17, [0/0], 00:04:54, direct 60.60.60.11/32, ubest/mbest: 1/0, attached *via 60.60.60.11, Eth1/17, [0/0], 00:04:54, local 60.60.60.21 is our STC tester port configure for OSPF



Expected Result The routers establish full adjacency status. All routes are seen on both sides of the network.

Actual Result Verified as expected.

Test Case 2: Nexus 7K Supervisor Failover with OSPF

NOTE: This will require Test 12 to be running but not mandatory.

DUT Nexus 7K VCS

Purpose The purpose of this test is to if the OSPF network will remain stable when the primary controller on the Nexus 7K fails and the backup controller takes over.

Test Procedure While Test 12 is running, thru one of your two console connections in the N7K that has all of our OSPF routes being injected into. We will do a “system switchover” and see what the affect is to our OSPF network and routes.

Step 1: Nexus 7K system switchover You need to make sure you have console on BOTH supervisors and you will soon see messages going across both. On the console that was the standby, log in and check your OSPF with the “show ip ospf nei” and the “sh ip route” as well as the STC that’s generating the xtra OSPF routes. Note: if your OSPF is on a VDC on this N7K chassis, you will have to run the “switchto vdc VDC_NAME” and run the OSPF commands to verify your OSPF. Verify that your OSPF neighbors are up and running with the “show ip ospf nei” command and “sh ip route” command. Once this has been checked out, from the N7K with all the xtra OSPF routes, from the active supervisor, run the follow to do the supervisor failover: Nexus7K# system switchover



Expected Result The output of the CLIs “show ip ospf nei” and the “sh ip route” are the same before and after the switchover.

Actual Result Verified as expected (?)

Test Case 3: Nexus 7K Supervisor Failback with OSPF

Note: This will require Test 12 to be running but not mandatory.

DUT Nexus 7K VCS

Purpose This is a continuation of the previous test. The purpose is to see if the OSPF network remains stable if the currently active controller on the Nexus 7K is switched back to the previously active controller.

Test Procedure This is just like in the previous test. Step 1: [Nexus 7K system switchover] This is as in the previous test. Force a switchover to the backup controller. Nexus7K# system switchover

Expected Result The output of the CLIs “show ip ospf nei” and the “sh ip route” are the same before and after the switchover.

Actual Result Met the expected results for OSPF.

Test Case 4: Nexus 7000 and IOS OSPF Loop Prevention

DUT Nexus 7K VCS

Purpose By default, VDX and Nexus 7K use different criteria for path selection. The purpose of this test is to see if loops can be prevented.



By default NX-OS runs OSPFv2 based on rfc 2328 right out of the box. With the IOS though, it runs OSPF based on rfc 1583! Why is this significant? The path selection criteria for external or summary routes have changed. IOS has always treated rfc1583 as the default behavior for path selection, and as support for rfc2328 was added, IOS added a toggle to prefer this new method of path selection, but continued to keep rfc1583 behavior set by default. With NX-OS however, it was launched with rfc2328 as the default behavior, and then to conform with RFC specs, added a command to make it optionally rfc1583 compliant.

Test Procedure Step 1: [show ip route] With NOS 3.x and OSPF, were are rfc 2328 compliant right out of the box and is backwards compatible by default with OSPF based on rfc 1583. So there’s no need to turn any knobs on our side. But as a sanity check and depending on your topology, the “show ip route a.b.c.d” command can be your best friend to find possible loops. Below is a url from Cisco’s support forum that explains this potential loop issue further. Where this could be a very real issue is between a NX-OS and IOS device. https://supportforums.cisco.com/community/netpro/network-infrastructure/switching/blog/2011/01/28/nexus-7000-and-ios-ospf-loop-prevention-and-interoperability

Expected Result There are no loops.

Actual Result No loops.

Test Case 5: NX-OS OSPF Manual Restart in VDC that is part of our VCS L3 Fabric w/OSPF

DUT Nexus 7K VCS

Purpose The Nexus 7K can operate with multiple VDCs. If OSPF process is restarted in one VDC, it should not affect any OSPF process in any other VDC.

Test Procedure Have a console open on both supervisors before beginning. Verify that all your OSPF is up and running. Then from one of the VDC’s you will issue the follow command to restart OSPF process 1 that we had assigned when we first configured OSPF.

https://supportforums.cisco.com/community/netpro/network-infrastructure/switching/blog/2011/01/28/nexus-7000-and-ios-ospf-loop-prevention-and-interoperability





Step 1: [restart ospf 1] While OSPF is restarting in this VDC, observe if you see any messages come across the console. Also check all your OSPF neighbors in the other VDC’s with OSPF up to your VCS OSPF Fabric. You can also do this on a N7K without any VDC’s as well. See Appendix F for more information concerning Non-Stop Forwarding and Graceful Restarts with NX-OS OSPF.

Expected Result OSPF processes in other VDCs are not affected.

Actual Result Met the expected results for VDC.

Test Case 6: VCS 3.0 - N7K OSPF Static Route Test

DUT Nexus 7K VCS

Purpose The purpose of this test is to check whether a static route off the VCS cloud is reachable from the Nexus 7000 via the OSPF network.

Test Procedure There's a host represented by a VE int for the 80 network hanging off 6720-3 that we need to reach from N7K. The 80.x.x.x is NOT represented anywhere else in our OSPF interop network or VCS 3.0 Fabric! See Appendix G for OSPF VCS Fabric topology. Step 1: [Ping] Do a test ping to make sure we can't reach our host on the 80 network first: Nexus7K(config)# ping 80.80.80.100 PING 80.80.80.100 (80.80.80.100): 56 data bytes ping: sendto 80.80.80.100 64 chars, No route to host Request 0 timed out ping: sendto 80.80.80.100 64 chars, No route to host Request 1 timed out ping: sendto 80.80.80.100 64 chars, No route to host Request 2 timed out ping: sendto 80.80.80.100 64 chars, No route to host Request 3 timed out ping: sendto 80.80.80.100 64 chars, No route to host Request 4 timed out Step 2: Create our (2) static routes with one on N7K going out our ospf 50 network.



Nexus7K(config)#ip route 80.80.80.0/24 50.50.50.1 Step 3: Create the second static route on this VDX going out our OSPF ve int. VDX6720-1(config-rbridge-id-1)# ip route 80.80.80.0/24 ve 10 2012/06/06-15:21:37, [RTM-1021], 6283, DCE, INFO, VDX6720-1, Static route update success: Static Route added for CIDR prefix 80.80.80.0/24 Step 4: Ping Nexus7K# ping 80.80.80.100 PING 80.80.80.100 (80.80.80.100): 56 data bytes 64 bytes from 80.80.80.100: icmp_seq=0 ttl=62 time=13.909 ms 64 bytes from 80.80.80.100: icmp_seq=1 ttl=62 time=4.217 ms 64 bytes from 80.80.80.100: icmp_seq=2 ttl=62 time=4.222 ms 64 bytes from 80.80.80.100: icmp_seq=3 ttl=62 time=3.324 ms 64 bytes from 80.80.80.100: icmp_seq=4 ttl=62 time=4.191 ms --- 80.80.80.100 ping statistics --- 5 packets transmitted, 5 packets received, 0.00% packet loss round-trip min/avg/max = 3.324/5.972/13.909 ms

Expected Result The new static route is reachable from the Nexus 7000.

Actual Result Met the expected results for static routes.



Appendix A – Notes about OSPF and NX-OS

The following list provides some additional facts about Cisco NX-OS that should be helpful

when designing, configuring, and maintaining an OSPF network.

1. Four OSPF instances can be configured per virtual device context (VDC). 2. If you remove the feature ospf command, all relevant OSPF configuration information is also

removed. 3. The shutdown command under the OSPF process can be used to disable OSPF while retaining

the configuration. Similar functionality can also be applied per interface with the ip ospf shutdown command.

4. The show running-config ospf command displays the current OSPF configuration. 5. An OSPF instance can be restarted with the restart ospf <instance> Exec command. 6. Graceful Restart (RFC 3623) is enabled by default. 7. You cannot configure multiple OSPF instances on the same interface. 8. An interface can support multi-area adjacencies using the multi-area option with the ip router

ospf interface command. 9. Secondary IP addresses are advertised by default, but can be suppressed per interface with the

ip router ospf <instance> area <#> secondaries none interface command. 10. By default all loopback IP address subnet masks are advertised in an LSA as a /32. The loopback

interface command ip ospf advertise-subnet can be configured to advertise the primary IP address subnet mask. (This command does not apply to secondary IP addresses. They will still be advertised as a /32.)

11. OSPF supports Bidirectional Forwarding Detection (BFD), which can be configured to reduce network convergence time to less than 1 second.

12. When you rollover an OSPF authentication key in a combined Cisco NX-OS/Cisco IOS network, you should configure both keys on the Cisco NX-OS router to ensure that there is sufficient overlap between the old key and the new key for a smooth transition to the new key. You should configure the new key as a valid accept key on all the NX-OS and IOS routers before the new key becomes a valid generation key in the keychain. During the overlap period, Cisco NX-OS transmits the new OSPF key and accepts OSPF authenticated packets from both the old key and the new key.

13. Maximum prefix thresholds (warning and withdraw) can be configured for redistributed routes using the redistribute maximum-prefix routing instance command.

http://tools.ietf.org/html/rfc3623



Appendix B – NX-OS OSPFv2 Verification Commands



Appendix C – Things you should know about NX-OS and VRF’s 1. When you assign a VRF instance to an interface with an IP address previously configured,

the interface IP address is automatically removed. 2. Static routes or dynamic routing protocols can be configured for routing in a VRF instance

(BGP, EIGRP, ISIS, OSPF, static routes, and RIPv2). 3. IP troubleshooting tools such as ping and traceroute are VRF aware and require the name of

a specific VRF instance if testing in the default VRF instance is not desired. 4. The routing-context vrf command can be executed in EXEC mode to change the routing

context to a non-default VRF instance. For example, typing routing-context vrf management changes the routing context, so all VRF related commands are executed in the management VRF as opposed to the default VRF.

5. Network management–related services such as authentication, authorization and accounting (AAA), Call Home, DNS, FTP, HTTP, NetFlow, NTP, PING, RADIUS, SCP, sFTP, SNMP, SSH, Syslog, TACACS+, TELNET, TFTP, Traceroute, and XML are VRF aware.

6. The global configuration write erase boot command will erase the management VRF instance configuration. The write erase command without the boot option will not.



Appendix D – NX-OS VRF Verification Commands The following table gives some useful NX-OS show commands for verifying and troubleshooting NX-OS VRF instances.



Appendix E - Example of configuring STC for OSPF route injection. Below is a basic OSPF Route Injection using the Spirent Test Center v3.51. Step 1

Click Next Step 2



Put a check mark in the “Enable” box for OSPFv2, the Generate Routes box will then be auto checked. Click Next. Step 3

Make sure the radio button for IPv4 is selected and then click Next. Step 4



For Device role, make sure Router is selected from the pull down. Then in the Router section put a check mark in the “Use IPv4 address as router ID”. Check mark the “Enable RFC 4814 MAC addresses”. In the IPv4 section, out in the starting IP address you want to use, the gateway along with any changes to the “Step” which is used to create the additional devices. We’ll leave the ToS/DiffServ box at the default. Make sure you also put a check mark in the “Launch Traffic Wizard” box in the lower right hand corner. Click Next. Step 5



We’ll keep the default here; notice the “Enable BFD” and “Enable Graceful Restart” boxes. These are additional tests that can also be done individually or separately. Click Next…. Step 6

Click Next Step 7

Click Next Step 8



This is where you decide on the total number of routers to be simulated and for this test were going to go with 50 simulated routers and leave the rest of the settings at their defaults. Click Next. Step 9

We’ll use the defaults here, notice the “Area Type” pull down, this is another area where we can tests with different OSPF area types. Click Next…. Step 10



We go with the default settings here and click Next. Step 11





Click Close



Step 14

Select our ports and click Next. Step 15

Select port 5 as our source and port 7 as the destination, click the Add button, orientation is unidirectional and click Next.



Step 16

We’ll go with the “Random and the mix and max of 128 and 256 for our purpose. Click Next. Step 17

Click Next



Step 18

We’ll go with the defaults, you can set the load as it relates to the amount of traffic you want to generate. Click Next. Step 19

Click Next



Step 20

Click the Apply button at the top and now you’re ready to simulate OSPF route injection into your test network.



Appendix F – Non-Stop Forwarding and Graceful Restarts with NX-OS OSPF

OSPFv2 Stateless Restart

If a NX-OS system that runs OSPFv2 experiences a cold reboot, the network stops forwarding traffic to the system and removes the system from the network topology. In this scenario, OSPFv2 experiences a stateless restart and removes all neighbor adjacencies on the local system. NX-OS applies the startup configuration and OSPFv2 rediscovers the neighbors and establishes the adjacencies again.

OSPFv2 Graceful Restart on a Switchover

After a switchover, NX-OS applies the running configuration, and the OSPFv2 process non-disruptively reestablishes its adjacencies and informs the neighbors that it is operational again when forwarding traffic. This occurs non-disruptively because the routing table and the hardware are fully populated before the restart, and forwarding can occur regardless of the OSPF process actions. When the system returns from the graceful restart state, the neighbor devices help to rebuild the link state database without disrupting forwarding on the data plane.

OSPFv2 Graceful Restart on an OSFPv2 Process Failure

OSPFv2 automatically restarts if the process experiences problems. After the restart, OSPFv2 initiates a graceful restart so that the platform is not taken out of the network topology. If you manually restart OSPF, it performs a graceful restart, which is similar to a stateful switchover. The running configuration is applied in both cases. The graceful restart allows OSPFv2 to remain in the data forwarding path through the process restart.

High Availability and Graceful Restart

NX-OS supports high availability. If a NX-OS system experiences a cold reboot, the network

stops forwarding traffic to the system and removes the system from the network topology. In this

scenario, OSPFv2 experiences a stateless restart and removes all neighbor adjacencies on the

local system. Cisco NX-OS applies the startup configuration and OSPFv2 rediscovers the

neighbors and establishes the adjacencies again.

A platform with two supervisors that run NX-OS can experience a stateful supervisor

switchover. Before the switchover occurs, OSPFv2 initiates a graceful restart by announcing that

OSPFv2 will be unavailable for some time. During a switchover, the network continues to

forward traffic and keeps the system in the network topology.

After a switchover, the NX-OS applies the running configuration, and OSPFv2 informs the

neighbors that it is operational again. The neighbors help to reestablish adjacencies.

OSPFv2 automatically restarts if the process experiences problems. After the restart, OSPFv2

initiates a graceful restart so that the platform is not taken out of the network topology. If you



manually restart OSPF, it performs a graceful restart, which is similar to a stateful switchover.

The running configuration is applied in both cases.

A graceful restart, or nonstop forwarding (NSF), allows OSPFv2 to remain in the data

forwarding path through a process restart. When OSPFv2 needs to restart, it first sends a link-

local opaque (type 9) LSA, called a grace LSA. This restarting OSPFv2 platform is called NSF

capable.

The grace LSA includes a grace period, which is a specified time that the neighbor OSPFv2

interfaces hold onto the LSAs from the restarting OSPFv2 interface. (Typically, OSPFv2 tears

down the adjacency and discards all LSAs from a down or restarting OSPFv2 interface.) The

participating neighbors, which are called NSF helpers, keep all LSAs that originate from the

restarting OSPFv2 interface as if the interface were still adjacent.

When the restarting OSPFv2 interface is operational again, it rediscovers its neighbors,

establishes adjacency, and starts sending its LSA updates again. At this point, the NSF helpers

recognize that graceful restart has finished.

Note: This section was pulled from the NX-OS Configuration Guide.



Appendix G – VCS 3.0 Fabric and Cisco Nexus 7K OSPF Interoperability Topology

Test bed setup Phase 1 – Build 2-node VCS Fabric with OSPF running between the (2) VDX’s. This will serve as the basis to connect up to our Nexus 7K.

1. Setting Up OSPF and VRRP and VE between our (2) VDX’s Note: OSPF is configured under the rbridge-id VDX6720-75 interface vlan 100 rbridge-id 1 ip route 0.0.0.0/0 10.18.233.1 router ospf area 0 ! interface Loopback 1 no shutdown



ip address 10.10.10.20/32 ! protocol vrrp chassis virtual-ip 10.18.233.75/24 interface Ve 100 ip ospf area 0 ip mtu 1500 VDX inserted this and the next line. ip proxy-arp ip address 1.1.1.210/24 no shutdown vrrp-group 100 virtual-ip 1.1.1.230 enable preempt-mode

Pass: Fail:

Test Engineer: Date:

Comments:

VDX6710-27 interface Vlan 100 shutdown ! rbridge-id 2 ip route 0.0.0.0/0 10.18.233.1 router ospf area 0 ! interface Loopback 1 no shutdown ip address 10.10.10.10/32 ! protocol vrrp chassis virtual-ip 10.18.233.27/24 interface Ve 100 ip ospf area 0 ip mtu 1500 ip proxy-arp ip address 1.1.1.200/24



no shutdown vrrp-group 100 virtual-ip 1.1.1.230 enable preempt-mode

Pass: Fail:


Comments:

2. Verifying our VCS 3.0 Fabric is running OSPF, VRRP and VE OSPF Check VDX6720-75# sh ip ospf nei Port Address Pri State Neigh Address Neigh ID Ev Opt Cnt Ve 100 1.1.1.210 1 FULL/BDR 1.1.1.200 10.10.10.10 5 2 0 VDX6710-27# sh ip ospf nei Port Address Pri State Neigh Address Neigh ID Ev Opt Cnt Ve 100 1.1.1.200 1 FULL/DR 1.1.1.210 10.10.10.20 6 2 0

Pass: Fail:


Comments:

VRRP Check VDX6720-75# show vrrp summary Total number of VRRP session(s) : 1



VRID Session Interface State Description ==== ======= ========= ===== =========== 100 VRRP Ve 100 Master VDX6710-27# show vrrp summary Total number of VRRP session(s) : 1 VRID Session Interface State Description ==== ======= ========= ===== =========== 100 VRRP Ve 100 Backup

Pass: Fail:


Comments:

VE Check VDX6720-75# ping 1.1.1.200 Type Control-c to abort PING 1.1.1.200 (1.1.1.200): 56 data bytes 64 bytes from 1.1.1.200: icmp_seq=0 ttl=64 time=2.746 ms 64 bytes from 1.1.1.200: icmp_seq=1 ttl=64 time=3.246 ms 64 bytes from 1.1.1.200: icmp_seq=2 ttl=64 time=3.639 ms 64 bytes from 1.1.1.200: icmp_seq=3 ttl=64 time=3.271 ms 64 bytes from 1.1.1.200: icmp_seq=4 ttl=64 time=2.573 ms --- 1.1.1.200 ping statistics --- 5 packets transmitted, 5 packets received, 0% packet loss round-trip min/avg/max/stddev = 2.573/3.095/3.639/0.386 ms

Pass: Fail:


Comments:

3. Creating a vlAG between our VCS Fabric and the PC running on N7K-4



VDX6720-75 interface TenGigabitEthernet 1/0/1 fabric isl enable fabric trunk enable channel-group 100 mode active type standard lacp timeout long This is automatically inserted by NOS ! interface TenGigabitEthernet 1/0/2 fabric isl enable fabric trunk enable channel-group 100 mode active type standard lacp timeout long no shutdown interface Port-channel 100 vlag ignore-split VDX inserted this automatically switchport switchport mode access switchport access vlan 1 no shutdown VDX6710-27 interface TenGigabitEthernet 2/0/50 fabric isl enable fabric trunk enable channel-group 100 mode active type standard lacp timeout long no shutdown ! interface TenGigabitEthernet 2/0/51 fabric isl enable fabric trunk enable channel-group 100 mode active type standard lacp timeout long no shutdown interface Port-channel 100 vlag ignore-split VDX inserted this automatically switchport switchport mode access switchport access vlan 1 no shutdown Nexus 7K-4 Port-Channel Configuration



N7K Port-Channel Configuration We’ll configure all (4) ports at once that will be part of our port-channel that will be connected to our VCS Fabric. Nexus7K-4(config)# feature lacp Nexus7K-4(config)# interface port-channel 200 Nexus7K-4(config-if)# no shutdown Nexus7K-4(config)# int ethernet 8/29 - 32 Nexus7K-4(config-if-range)# channel-group 200 mode active

Pass: Fail:


Comments:

4. VCS vLAG and Cisco NX-OS Port-Channel Verification

VCS vLAG Verification VDX6720-75# show port-channel summary LACP Aggregator: Po 100 (vLAG) you want to see this! Aggregator type: Standard Ignore-split is disabled Member rbridges: rbridge-id: 1 (2) rbridge-id: 2 (2) Admin Key: 0100 - Oper Key 0100 Member ports on rbridge-id 1: Link: Te 1/0/1 (0x118008000) sync: 1 * Link: Te 1/0/2 (0x118010001) sync: 1 VDX6710-27# show port-channel summary LACP Aggregator: Po 100 (vLAG) you want to see this! Aggregator type: Standard Ignore-split is disabled Member rbridges: rbridge-id: 1 (2) rbridge-id: 2 (2) Admin Key: 0100 - Oper Key 0100 Member ports on rbridge-id 2: Link: Te 2/0/50 (0x21819000F) sync: 1



Link: Te 2/0/51 (0x218198010) sync: 1

Pass: Fail:


Comments:

Nexus7K-4 (PC200 is what is connected to our VCS Fabric)

Note: Next to po200 we see (SU), from the flags this is telling you that this pc is switched and up which is what we want. Also next to each of the ports in our pc, we see each port with the letter “P” next to it which indicates that the port is up and a member in our pc.

Pass: Fail:


Comments:

Phase 2 – Creating the OSPF neighbor’s between two of our N7K’s and our VCS 3.0 Fabric already running OSPF between them in an east-west setup.

5. Configure (2) ports on each of the two VDX’s that will connect up to the (2) N7K’s with OSPF in Area 0.

VDX-1 VDX6710-27(config)# interface TenGigabitEthernet 2/0/52 VDX6710-27(conf-if-te-2/0/52)# shut



VDX6710-27(conf-if-te-2/0/52)# no switchport VDX6710-27(conf-if-te-2/0/52)# ip address 20.20.20.1/24 VDX6710-27(conf-if-te-2/0/52)# ip ospf area 0 VDX6710-27(conf-if-te-2/0/52)# int ten 2/0/53 VDX6710-27(conf-if-te-2/0/53)# shut VDX6710-27(conf-if-te-2/0/53)# no switchport VDX6710-27(conf-if-te-2/0/53)# ip address 30.30.30.1/24 VDX6710-27(conf-if-te-2/0/53)# ip ospf area 0 VDX-2 VDX6720-75(config)# int ten 1/0/3 VDX6720-75(conf-if-te-1/0/3)# shut VDX6720-75(conf-if-te-1/0/3)# no switchport VDX6720-75(conf-if-te-1/0/3)# ip address 40.40.40.1/24 VDX6720-75(conf-if-te-1/0/3)# ip ospf area 0 VDX6720-75(conf-if-te-1/0/3)# int ten 1/0/4 VDX6720-75(conf-if-te-1/0/4)# shut VDX6720-75(conf-if-te-1/0/4)# no switchport VDX6720-75(conf-if-te-1/0/4)# ip address 50.50.50.1/24 VDX6720-75(conf-if-te-1/0/4)# ip ospf area 0 Note: Some of the NOS 3.0 messages to the console were removed for clarity. Note: See Appendix G for OSPF Interoperability topology

Pass: Fail:


Comments:

6. Configure (2) ports on each on two of the N7K’s that will connect up to the (2) VDX’s with

OSPF in Area 0. N7K-1 Nexus7K(config)# router ospf 1 Nexus7K(config-router)# router-id 1.1.1.1 Nexus7K(config) eth 1/10 Nexus7K(config-if)#shut Nexus7K(config-if)# no switchport Nexus7K(config-if)# ip address 30.30.30.2 255.255.255.0 Nexus7K(config-if)# ip router ospf 1 area 0



Nexus7K(config)# router ospf 1 Nexus7K(config-router)# int eth 1/2 Nexus7K(config-if)# ip address 50.50.50.2 255.255.255.0 Nexus7K(config-if)# ip router ospf 1 area 0 Nexus7K(config-if)# no shut

Pass: Fail:


Comments:

N7K-2 Nexus7K-Nexus7K-2(config)# router ospf 1 Nexus7K-Nexus7K-2(config-router)# router-id 1.1.1.2 Nexus7K-Nexus7K-2(config-router)# int eth 1/27 Nexus7K-Nexus7K-2(config-if)# shut Nexus7K-Nexus7K-2(config-if)# no switchport Nexus7K-Nexus7K-2(config-if)# ip address 20.20.20.2 255.255.255.0 Nexus7K-Nexus7K-2(config-if)# ip router ospf 1 area 0 Nexus7K-Nexus7K-2(config-if)# no shut Nexus7K-Nexus7K-2(config)# router ospf 1 Nexus7K-Nexus7K-2(config-router)# int eth 1/18 Nexus7K-Nexus7K-2(config-if)# no switchport Nexus7K-Nexus7K-2(config-if)# ip address 40.40.40.2 255.255.255.0 Nexus7K-Nexus7K-2(config-if)# ip router ospf 1 area 0 Nexus7K-Nexus7K-2(config-if)# no shut

Pass: Fail:


Comments:

7. OSPF Verification

Nexus 7K-VCS OSPF



Nexus7K-Nexus7K-2# sh ip ospf nei OSPF Process ID 1 VRF default Total number of neighbors: 2 Neighbor ID Pri State Up Time Address Interface 10.10.10.20 1 FULL/DR 00:00:13 40.40.40.1 Eth1/18 10.10.10.10 1 FULL/DR 00:43:03 20.20.20.1 Eth1/27 Nexus7K# sh ip ospf nei OSPF Process ID 1 VRF default Total number of neighbors: 2 Neighbor ID Pri State Up Time Address Interface 10.10.10.20 1 FULL/DR 00:00:12 50.50.50.1 Eth1/2 10.10.10.10 1 FULL/DR 00:47:42 30.30.30.1 Eth1/10

Pass: Fail:


Comments:

VCS-Nexus7K OSPF VDX6710-27# sh ip ospf nei Port Address Pri State Neigh Address Neigh ID Ev Opt Cnt Ve 100 1.1.1.200 1 FULL/DR 1.1.1.210 10.10.10.20 4 2 0 Gi 2/0/1 2.2.2.10 128 FULL/DR 2.2.2.20 2.2.2.20 5 2 0 Te 2/0/53 30.30.30.1 1 FULL/BDR 30.30.30.2 1.1.1.1 5 2 0 Te 2/0/52 20.20.20.1 1 FULL/BDR 20.20.20.2 1.1.1.2 2 2 0 Note: Gi 2/0/1 is an OSPF neighbor into a Juniper based OSPF cloud. Ve100 is our OSPF neighbor with our adjacent VDX in our VCS Fabric. VDX6720-75# sh ip ospf nei Port Address Pri State Neigh Address Neigh ID Ev Opt Cnt Ve 100 1.1.1.210 1 FULL/BDR 1.1.1.200 10.10.10.10 6 2 0 Te 1/0/3 40.40.40.1 1 FULL/BDR 40.40.40.2 1.1.1.2 5 2 0 Te 1/0/4 50.50.50.1 1 FULL/BDR 50.50.50.2 1.1.1.1 5 2 0

Pass: Fail:




Comments:

8. Nexus N7K OSPF Route Checking

Note: On each Nexus 7K we need to see some number of “intra area” routes via the “show ip route” command done on each of the two N7K’s. N7K-2 Nexus7K-Nexus7K-2# sh ip route IP Route Table for VRF "default" '*' denotes best ucast next-hop '**' denotes best mcast next-hop '[x/y]' denotes [preference/metric] '%<string>' in via output denotes VRF <string> 1.1.1.0/24, ubest/mbest: 1/0, attached *via 1.1.1.2, Lo0, [0/0], 3d22h, direct 1.1.1.2/32, ubest/mbest: 1/0, attached *via 1.1.1.2, Lo0, [0/0], 3d22h, local 2.2.2.0/24, ubest/mbest: 1/0 *via 20.20.20.1, Eth1/27, [110/5], 3d22h, ospf-1, intra 10.10.10.0/24, ubest/mbest: 1/0, attached *via 10.10.10.1, Po1, [0/0], 4d19h, direct 10.10.10.1/32, ubest/mbest: 1/0, attached *via 10.10.10.1, Po1, [0/0], 4d19h, local 20.20.20.0/24, ubest/mbest: 1/0, attached *via 20.20.20.2, Eth1/27, [0/0], 3d22h, direct 20.20.20.2/32, ubest/mbest: 1/0, attached *via 20.20.20.2, Eth1/27, [0/0], 3d22h, local 30.30.30.0/24, ubest/mbest: 1/0 *via 20.20.20.1, Eth1/27, [110/5], 3d22h, ospf-1, intra 40.40.40.0/24, ubest/mbest: 1/0, attached *via 40.40.40.2, Eth1/18, [0/0], 01:42:06, direct 40.40.40.2/32, ubest/mbest: 1/0, attached *via 40.40.40.2, Eth1/18, [0/0], 01:42:06, local 50.50.50.0/24, ubest/mbest: 1/0 *via 40.40.40.1, Eth1/18, [110/5], 01:41:19, ospf-1, intra N7K-1 Nexus7K# sh ip route



IP Route Table for VRF "default" '*' denotes best ucast next-hop '**' denotes best mcast next-hop '[x/y]' denotes [preference/metric] '%<string>' in via output denotes VRF <string> 1.1.1.0/24, ubest/mbest: 1/0, attached *via 1.1.1.1, Lo0, [0/0], 3d22h, direct 1.1.1.1/32, ubest/mbest: 1/0, attached *via 1.1.1.1, Lo0, [0/0], 3d22h, local 2.2.2.0/24, ubest/mbest: 1/0 *via 30.30.30.1, Eth1/10, [110/5], 3d22h, ospf-1, intra 10.10.10.0/24, ubest/mbest: 1/0, attached *via 10.10.10.2, Po1, [0/0], 4d19h, direct 10.10.10.2/32, ubest/mbest: 1/0, attached *via 10.10.10.2, Po1, [0/0], 4d19h, local 20.20.20.0/24, ubest/mbest: 1/0 *via 30.30.30.1, Eth1/10, [110/5], 3d22h, ospf-1, intra 30.30.30.0/24, ubest/mbest: 1/0, attached *via 30.30.30.2, Eth1/10, [0/0], 3d22h, direct 30.30.30.2/32, ubest/mbest: 1/0, attached *via 30.30.30.2, Eth1/10, [0/0], 3d22h, local 40.40.40.0/24, ubest/mbest: 1/0 *via 50.50.50.1, Eth1/2, [110/5], 01:41:10, ospf-1, intra 50.50.50.0/24, ubest/mbest: 1/0, attached *via 50.50.50.2, Eth1/2, [0/0], 01:41:57, direct 50.50.50.2/32, ubest/mbest: 1/0, attached *via 50.50.50.2, Eth1/2, [0/0], 01:41:57, local

Pass: Fail:


Comments:

9. VCS OSPF Route Checking

VDX6710-27# show ip route Total number of IP routes: 9 Type Codes - B:BGP D:Connected I:ISIS O:OSPF R:RIP S:Static; Cost - Dist/Metric BGP Codes - i:iBGP e:eBGP ISIS Codes - L1:Level-1 L2:Level-2 OSPF Codes - i:Inter Area 1:External Type 1 2:External Type 2 s:Sham Link Destination Gateway Port Cost Type Uptime



1 0.0.0.0/0 10.18.233.1 mgmt 1 1/1 S 14d0h 2 1.1.1.0/24 DIRECT Ve 100 0/0 D 1h47m 3 2.2.2.0/24 DIRECT Gi 2/0/1 0/0 D 14d0h 4 10.10.10.10/32 DIRECT Lo 1 0/0 D 14d0h 5 10.18.233.0/24 DIRECT mgmt 1 0/0 D 14d0h 6 20.20.20.0/24 DIRECT Te 2/0/52 0/0 D 3d22h 7 30.30.30.0/24 DIRECT Te 2/0/53 0/0 D 3d22h 8 40.40.40.0/24 1.1.1.210 Ve 100 110/2 O 1h46m OSPF 9 50.50.50.0/24 1.1.1.210 Ve 100 110/2 O 1h46m <- OSPF VDX6720-75# show ip route Total number of IP routes: 10 Type Codes - B:BGP D:Connected I:ISIS O:OSPF R:RIP S:Static; Cost - Dist/Metric BGP Codes - i:iBGP e:eBGP ISIS Codes - L1:Level-1 L2:Level-2 OSPF Codes - i:Inter Area 1:External Type 1 2:External Type 2 s:Sham Link Destination Gateway Port Cost Type Uptime 1 0.0.0.0/0 10.18.233.1 mgmt 1 1/1 S 1h49m 2 1.1.1.0/24 DIRECT Ve 100 0/0 D 1h49m 3 1.1.1.230/32 DIRECT Ve 100 0/0 D 1h49m 4 2.2.2.0/24 1.1.1.200 Ve 100 110/2 O 1h48m <- OSPF 5 10.10.10.20/32 DIRECT Lo 1 0/0 D 1h49m 6 10.18.233.0/24 DIRECT mgmt 1 0/0 D 1h49m 7 20.20.20.0/24 1.1.1.200 Ve 100 110/2 O 1h48m <- OSPF 8 30.30.30.0/24 1.1.1.200 Ve 100 110/2 O 1h48m <- OSPF 9 40.40.40.0/24 DIRECT Te 1/0/3 0/0 D 1h49m 10 50.50.50.0/24 DIRECT Te 1/0/4 0/0 D 1h49m

Pass: Fail:


Comments:

10. Nexus 7K w/VRF up to our VCS Fabric with OSPF

We will configure VRF or Layer-3 Virtualization (NX-OS speak) on our 30 network to provide some isolation much like you would do in a DC/SP environment for example. Create our NX-OS VRF and we’ll call it vrf-finance Nexus7K(config)# vrf context vrf-finance Enable VRF under our OSPF Process



Nexus7K(config)# router ospf 1 vrf vrf-finance Enable VRF on our 30 network interface that is running OSPF up to our VCS 3.0 Fabric. Nexus7K(config-if)# vrf member vrf-finance % Deleted all L3 config on interface Ethernet1/10 <---DONT FORGET TO ADD THE IP and OSPF INFO BACK IN. Right now YOUR OSPF Neighbor is DOWN! VRF OSPF Verification Nexus7K(config-router-vrf)# sh ip ospf nei OSPF Process ID 1 VRF default Total number of neighbors: 1 Neighbor ID Pri State Up Time Address Interface 10.10.10.20 1 FULL/DR 20:10:46 50.50.50.1 Eth1/2 Our 30 network is NO longer visible from the default VRF as we expected. Note: The default VRF like the default VDC cannot be deleted. Nexus7K(config-router-vrf)# sh ip ospf nei vrf vrf-finance OSPF Process ID 1 VRF vrf-finance Total number of neighbors: 1 Neighbor ID Pri State Up Time Address Interface 10.10.10.10 1 FULL/DR 00:06:04 30.30.30.1 Eth1/10 The above output is what we are looking for! Let's see how our VDX in our OSPF VCS Fabric looks with our VRF OSPF Neighbor. VDX6710-27# sh ip ospf nei Port Address Pri State Neigh Address Neigh ID Ev Opt Cnt Ve 100 1.1.1.200 1 FULL/BDR 1.1.1.210 10.10.10.20 4 2 0 Gi 2/0/1 2.2.2.10 128 FULL/DR 2.2.2.20 2.2.2.20 6 2 0 Te 2/0/52 20.20.20.1 1 FULL/DR 20.20.20.2 1.1.1.2 5 2 1 Te 2/0/53 30.30.30.1 1 FULL/BDR 30.30.30.2 30.30.30.2 4 2 0 It still has the 30 network up and running. Nexus VRF Verification



Nexus7K# sh vrf vrf-finance VRF-Name VRF-ID State Reason vrf-finance 3 Up --



See Appendix D for more information on NX-OS VRF commands.

11. Rebooting the Nexus 7K’s Default VDC w/OSPF Note: Rebooting the “default” VDC will reboot the entire physical Nexus 7K chassis. After the rebooting of your N7K and you’re logged in. You will see the following still taking place on your Nexus 7K with multiple VDC’s configured as shown below. Use the “reload vdc” command from the default VDC to reload it. It’s always a good idea to have console access to your N7K’s.



Note: vdc_id 1 is the “default” VDC and cannot be deleted.



Now that all of our VDC’s are now active, soon our OSPF neighbor’s will be formed with our VCS OSPF L3 Fabric. Once the VDC’s are all active run the OSPF verification commands as described earlier in this document. Note: The time for your N7K chassis to reboot and come back online with everything running can be affected by the likes of the number of line cards, # of VDC’s and stuff like OSPF etc.

Pass: Fail:


Comments:

12. Rebooting a non-default VDC configured with OSPF

Rebooting a vdc that is NOT the default vdc takes about 30-40 seconds and does NOT reboot the physical N7K chassis. Depending on the configuration of your N7K, this time could be affected one way or the other. There are (2) ways to “reboot” a non default VDC: Option 1: Login into the “default” VDC and run the following command. Nexus7K# reload vdc ? <CR> Nexus7K-2 VDC number 2 Nexus7K-3 VDC number 3 Nexus7K-4 VDC number 4 Note: The names Nexus7K-2 thru 4 are the names we gave to our VDC’s Option 2: Log into the non-default VDC you want to “reboot” and run the following command. Nexus7K-Nexus7K-2# reload vdc

Pass: Fail:


Comments:



13. Upgrading your Nexus 7K NX-OS Configured with OSPF In a perfect world, you would want to go for an ISSU (In-Service Software Upgrade). But this will depend on how your N7K that you want to upgrade is configured. The quickest way to find out if an ISSU is doable is to run the following command. From the default VDC run the following command as an example: Nexus7K# show install all impact kickstart n7000-s1-kickstart.6.0.1.bin system n7000-s1-dk9.6.0.1.bin



In our case the impact would be non-disruptive and not required because we already had upgraded before we started this project.

Pass: Fail:


Comments:

Note: As a general BP, always refer to the NX-OS upgrade and downgrade guide for the version you’re upgrading to. This will include scenarios like single and dual sup upgrades etc. Below is the URL for NX-OS Upgrade/Downgrade Guide: http://www.cisco.com/en/US/docs/switches/datacenter/sw/6_x/nx-os/upgrade/guide/b_Cisco_Nexus_7000_Series_NX-OS_Software_Upgrade_and_Downgrade_Guide_Release_6.x.html#con_295882 Note: Also read the corresponding NX-OS release notes as well! This will provide information like what hardware is supported along with feature support information.

14. Upgrading your Nexus 7K Non-Default VDC(s) NX-OS with OSPF

http://www.cisco.com/en/US/docs/switches/datacenter/sw/6_x/nx-os/upgrade/guide/b_Cisco_Nexus_7000_Series_NX-OS_Software_Upgrade_and_Downgrade_Guide_Release_6.x.html#con_295882





In a perfect world, you would want to go for an ISSU (In-Service Software Upgrade) on your non-default VDC. But this will depend on how your N7K that you want to upgrade is configured. The quickest way to find out if an ISSU is doable is to run the following command. From the non-default VDC run the following command as an example: Nexus7K# show install all impact kickstart n7000-s1-kickstart.6.0.1.bin system n7000-s1-dk9.6.0.1.bin

Pass: Fail:


Comments: This command will fail immediately due to the fact that you cannot upgrade your NX-OS on a non-default VDC. It is left up to the tester as to whether this is considered a failed test or not.

15. Upgrading your VCS 3.0 L3 Fabric running OSPF within a VCS 3.0 Fabric and Up To Our Nexus

7K OSPF Network. Part 1 We’ll use our Figure 1 diagram as the basis for upgrading our VCS L3 Fabric running OSPF between them. Before upgrading the first VDX, run the verification commands described earlier under Phase 1 Test 2 before beginning the NOS upgrade. Also verify your VCS Fabric is operational as well with commands like “show vcs, show fabric isl and show fabric islports”.

Pass: Fail:


Comments:

Part 2 Perform the actual NOS upgrade on one of the (2) VDX’s in our VCS 3.0 Fabric. Below is one way of doing the NOS upgrade and were going from NOS 3.0 build 14 to build 18 in this example. Note: Your OSPF neighbors that involve this VDX will be taking down when the VDX is rebooted. firmware download ftp host 1.2.3.4 file release.plist directory /AA/BB/CC/DD/nos3.0.0/nos3.0.0_bld18 password releaseuser user releaseuser You will see a message before the actual upgrade that will include the following line:



This command will cause a cold/disruptive reboot and will require that existing telnet, secure telnet or SSH sessions be restarted. You will be asked if you want to continue, type “y” and hit the return button. When the upgrade is completed and the VDX has rebooted, verify that the new version of NOS is running with the “show version” command. Now verify that your VCS Fabric is up and operational on BOTH VDX’s with commands like “show vcs, show fabric isl and show fabric islports”.

Pass: Fail:


Comments:

Part 3 Verify that OSPF is running on this VDX that was just upgraded to the latest NOS 3.0 build with the “show ip ospf nei” command. Also verify on the N7K’s configured with OSPF to our VCS L3 Fabric with the “show ip ospf nei” command.

Pass: Fail:


Comments:

Part 4 We’ll use our Figure 1 diagram as the basis for upgrading our VCS L3 Fabric running OSPF between them. Before upgrading the second VDX, run the verification commands described earlier under Phase 1 Test 2 before beginning the NOS upgrade. Also verify your VCS Fabric is operational as well with commands like “show vcs, show fabric isl and show fabric islports”.

Pass: Fail:


Comments:



Part 5 Perform the actual NOS upgrade on the (2) VDX’s in our VCS 3.0 Fabric. Below is one way of doing the NOS upgrade and were going from NOS 3.0 build 14 to build 18 in this example. firmware download ftp host 1.2.3.4 file release.plist directory /AA/BB/CC/DD/nos3.0.0/nos3.0.0_bld18 password releaseuser user releaseuser You will see a message before the actual upgrade that will include the following line: This command will cause a cold/disruptive reboot and will require that existing telnet, secure telnet or SSH sessions be restarted. You will be asked if you want to continue, type “y” and hit the return button. When the upgrade is completed and the VDX has rebooted, verify that the new version of NOS is running using the “show version” command. Now verify that your VCS Fabric is up and operational on BOTH VDX’s with commands like “show vcs, show fabric isl and show fabric islports”.

Pass: Fail:


Comments:

Part 6 Verify that OSPF is running on this VDX that was just upgraded to the latest NOS 3.0 build with the “show ip ospf nei” command. Also verify on the N7K’s configured with OSPF to our VCS L3 Fabric with the “show ip ospf nei” command

Pass: Fail:


Comments:

Note: As part of a Best Practice, a NOS upgrades should be done during a scheduled maintenance period, especially since we now have OSPF running in our VCS 3.0 Fabric as well as the possibility of running OSPF between our VCS 3.0 Fabric and with other OSPF capable devices.

Data Center Network Infrastructure Test Engineer’s Journal Cisco · PDF fileTest...

Documents

Transcript of Data Center Network Infrastructure Test Engineer’s Journal Cisco · PDF fileTest...