OSPF Troubleshooting

8/12/2019 OSPF Troubleshooting

1/93

BRKRST-3310

Troubleshooting OSPF


2/93

2010 Cisco and/or its affiliates. All rights reserved. Cisco ConfidentialPresentation_ID 2

Agenda

! LSA Overview! Troubleshooting Commands! Common Issues! OSPF for NX-OS and IOS-XR! Troubleshooting Enhancements


3/93

LSA OverviewDetails of Each LSA


4/93


LSA Type Review

Type LSA

1 Router

2 Network

3 Summary Network

4 Summary ASBR

5 External

6 Group Membership

7 NSSA

8 External Attributes

911 Opaque


5/93


R3#show ip ospf database router 3.3.3.3Router Link States (Area 1)

LS age: 0 Always 0 at origination

Options: (No TOS-capability)LS type: Router Links This is a router LSALink State ID: 3.3.3.3 Router ID of R3Advertising Router: 3.3.3.3 Router ID of R3

LS Seq Number: 80000001 Initial Seq NumberChecksum: 0x146BLength: 48Area Border Router bit B = 1

Number of links: 2 Link Connected to: a Transit Network

(Link ID) Designated Router address): 192.1.1.4 (Link Data) Router Interface address): 192.1.1.3

Number of TOS metrics: 0TOS 0 Metric: 1

Link Connected to: a Stub Network (Link ID) Network/subnet number: 192.1.4.0

(Link Data) Network Mask: 255.255.255.0 Number of TOS metrics = 0

TOS 0 Metric: 2

Router LSA of R3 for Area 1

192.1.1.3

Area 0

DR

R4192.1.1.4

192.1.4.0/24

18.10.0.5/30


6/93


Router LSA of R3 for Area 0 (Cont.)

192.1.1.3

Area 0

DR

R4192.1.1.4

192.1.4.0/24

18.10.0.5/30 6.6.6.6

LS age: 0 Always 0 at originationOptions: (No TOS-capability)LS type: Router Links This is a router LSA

Link State ID: 3.3.3.3 Router ID of R3Advertising Router: 3.3.3.3 Router ID of R3

LS Seq Number: 80000001 Initial Seq NumberChecksum: 0x146B

Length: 48Area Border Router bit B = 1Number of links: 2

Link Connected to: another Router (point-to-point)

(Link ID) Neighboring Router ID: 6.6.6.6(Link Data) Router Interface address: 18.10.0.5

Number of TOS metrics: 0TOS 0 Metric: 8

Link Connected to: a Stub Network (Link ID) Network/subnet number: 18.10.0.4 (Link Data) Network Mask: 255.255.255.252 Number of TOS metrics = 0

TOS 0 Metric: 8

Router Link States (Area 0)


7/93


Router Link Types Summary

Type Description Link ID Link Data

1Point-to-Point

NumberedNeighbors

RIDInterface IP

Address

1Point-to-PointUnnumbered

NeighborsRID

MIB-II IfindexValue

2 TransitIP Addressof the DR

Interface IPAddress

3 Stub IP NetworkNumber

Subnet Mask

4 Virtual LinkNeighbors

RIDInterface IP

Address


8/93


R3#show ip ospf database network 192.1.1.4Network Link States (Area 1)

LS age: 948Options: (No TOS Capability)LS type: Network LinksLink State ID: 192.1.1.4 (intf address of Designated Router)Advertising Router: 4.4.4.4 RID of DR

LS Seq Number: 80000001 Initial Seq NumberChecksum: 0x9A84Length: 40Network Mask: /24

Attached Router: 4.4.4.4Attached Router: 3.3.3.3Attached Router: 2.2.2.2Attached Router: 1.1.1.1

Network LSA for 192.1.1.0

192.1.1.3

Area 0

192.1.4.0/24

18.10.0.4/32

6.6.6.6

192.1.1.41.1.1.1

2.2.2.2

/24

1

2

DR

R4

4.4.4.4


9/93


Area 0

R4#show ip ospf database summary 192.1.2.0Summary Net Link States (Area 0)

LS age: 1514Options: (No TOS Capability)LS type: Summary Links (Networks)Link State ID: 192.1.2.0 (Summary Network Number)Advertising Router: 4.4.4.4 RID of ABR

LS Seq Number: 80000001 Initial Seq NumberChecksum: 0xFC68Length: 28Network Mask: /24TOS 0 Metric: 4

Type 3 Details

192.1.1.3

192.1.4.0/24

18.10.0.4

6.6.6.6

192.1.1.41.1.1.1

2.2.2.2

/24

1

2

192.1.2.0/24

192.1.2.0/24metric 4

O IA

DR

R43


10/93


R4#show ip ospf database asbr-summary 7.7.7.7Summary ASB Link States (Area 0)

LS age: 1548Options: (No TOS Capability)LS type: Summary Links (AS Boundary Router)Link State ID: 7.7.7.7 (As Boundary Router address)Advertising Router: 4.4.4.4 RID of ABR

LS Seq Number: 80000001 Initial Seq NumberChecksum: 0x93E2Length: 28Network Mask: /0

TOS: 0 Metric: 16

Type 4 Details

1.1.1.1

Area 0

192.1.4.0/24

192.1.2.0/24

DR

R4

R7

RID ASBR

7.7.7.7

R3 2 18.10.0.4

6.6.6.6192.1.1.3

192.1.1.4

2.2.2.2

/24

1

8

3 140.10.0.0External Route

Type 4 Summary


11/93


Type 5 Details

1.1.1.1

Area 0

192.1.4.0/24

192.1.2.0/24

DR

R4

R7

RID ASBR

7.7.7.7

R3 2 18.10.0.4

6.6.6.6192.1.1.3

192.1.1.4

2.2.2.2

/24

1

8

3 140.10.0.0External Route

4

External Type 5

R4#show ip ospf database external 140.10.0.0LS age: 156Options: (No TOS Capability)LS type: AS External LinkLink State ID: 140.10.0.0 (External Network Number)Advertising Router: 7.7.7.7 Router ID of R7LS Seq Number: 80000001 Initial Seq Number

Checksum: 0x93E2Length: 36Network Mask: /16

Metric Type: 2 bit E = 1 -> O E2 (Default)Metric: 20Forwarding address: 0.0.0.0


12/93


Forwarding addressMust be known via intra or inter area route

SBROSPF

Network

router ospf 1

redistribute bgp 1 subnets

BGP Gateway

Router

OSPF is running

on this interface


13/93


Forwarding Address (Type 5)

! OSPF is enabled on the ASBRs next hopinterface AND

! The ASBRs next hop interface is non-passiveto OSPF AND

! The ASBRs next hop interface network-typeis not p2p or p2mp AND

! The ASBRs next hop interface address fallsinto OSPF network range

When Will It Set to Non-Zero?


14/93


R7#show ip ospf database nssa-external 140.10.0.0

LS age: 156Options: (No TOS Capability, Type 7/5 translation)LS type: AS External LinkLink State ID: 140.10.0.0 (External Network Number)Advertising Router: 7.7.7.7 Router ID of R7(ASBR)LS Seq Number: 80000001 Initial Seq Number

Checksum: 0x01A6Length: 36Network Mask: /16

Metric Type: 2Metric: 20Forwarding address: 0.0.0.0

1.1.1.1

Area 0

192.1.4.0/24

192.1.2.0/24

DRR4

R3 2 18.10.0.4

6.6.6.6192.1.1.3

192.1.1.4

2.2.2.2

/241

8

3

140.10.0.0

External Route

4

NSSA

RID ASBR

7.7.7.7R7

Type 7

Type 7 Details


15/93


Forwarding Address (Type 7)

! OSPF is enabled on the ASBRs next hop interface AND! The ASBRs next hop interface is non-passive to OSPF AND! The ASBRs next hop interface network-type is not p2p

or p2mp AND

! The ASBRs next hop interface address falls into OSPFnetwork range

! If the above are true then we set the forwarding address to thenext hop address, otherwise

If the next hop interface is passive or non multi-access, use our ownIP address on the next hop interface else

Use one of the loopbacks (if its up and OSPF is enabled) in the areawere announcing to

If no loopbacks in that area, use the address of the first interface

in that area

When Will It Set to Non-Zero?


16/93

Troubleshooting Commands

What to look for in the output?


17/93


R3#show ip ospfRouting Process "ospf 1" with ID3.3.3.3and Domain ID 0.0.0.1Supports only single TOS(TOS0) routes

Supports opaque LSAIt is an area border router

SPF schedule delay 5 secs, Hold time between two SPFs 10 secs

Minimum LSA interval 5 secs. Minimum LSA arrival 1 secsNumber of external LSA 1. Checksum Sum 0x3B57

Number of opaque AS LSA 0. Checksum Sum 0x0Number of DCbitless external and opaque AS LSA 0

Number of DoNotAge external and opaque AS LSA 0

Number of areasin this router is 2. 2 normal 0 stub 0 nssaExternal flood list length 0

Area BACKBONE(0)Number of interfacesin this area is 2

Area has no authentication

SPF algorithm executed 2773 times

Area ranges areNumber of LSA 17. Checksum Sum 0x686B5Number of opaque link LSA 0. Checksum Sum 0x0

Number of DCbitless LSA 0

Number of indication LSA 0Number of DoNotAge LSA 9

Flood list length 0

Show IP OSPF


18/93


R3#show ip ospf database

OSPF Router with ID (3.3.3.3) (Process ID 1)


Link ID ADV Router Age Seq# Checksum Link count3.3.3.3 3.3.3.3 106 0x80000009 0xC3F1 3

. . .Summary Net Link States (Area 0)

Link ID ADV Router Age Seq# Checksum18.10.0.0 7.7.7.7 3 (DNA) 0x80000008 0x3DC2

18.10.0.0 8.8.8.8 1396 0x80000004 0x27D8. . .


Link ID ADV Router Age Seq# Checksum Link count

1.1.1.1 1.1.1.1 671 0x80000016 0xE6CD 2. . .

Show IP OSPF Database


19/93


Show IP OSPF DatabaseDatabase-Summary

R3#show ip ospf database database-summary


Area 0 database summary

LSA Type Count Delete MaxageRouter 6 0 0

Network 4 0 0Summary Net 10 0 0

Summary ASBR 0 0 0

Type-7 Ext 0 0 0Opaque Link 0 0 0

Opaque Area 0 0 0Subtotal 20 0 0

Area 1 database summaryLSA Type Count Delete Maxage

Router 4 0 0Network 1 0 0

Summary Net 10 0 0

Summary ASBR 4 0 0. . .


20/93


R3#show ip ospf neighbor

Neighbor IDPri State Dead Time Address Interface

1.1.1.1 1 FULL/DROTHER 00:00:33 192.1.1.1 FastEthernet0/0

2.2.2.2 1 FULL/DROTHER 00:00:32 192.1.1.2 FastEthernet0/0

4.4.4.4 1 FULL/DR 00:00:39 192.1.1.4 FastEthernet0/06.6.6.6 1 FULL/ - 00:00:38 18.10.0.6 Serial0/0

R3#

Show IP OSPF Neighbor

192.1.1.3

DR

R4

192.1.4.0/24

18.10.0.4

6.6.6.6

192.1.1.41.1.1.1

2.2.2.2

/241

2

4.4.4.4

Area 0


21/93


R3#show ip ospf neighbor detailNeighbor 1.1.1.1, interface address 192.1.1.1

In the area 1via interface FastEthernet0/0

Neighbor priority is 1, State is 2WAY, 2 state changes

DR is 192.1.1.4 BDR is 192.1.1.2

Options is 0x2

Dead timer due in 00:00:39

Neighbor is up for 00:06:30Index 0/0, retransmission queue length 0, number of retransmission 0

First 0x0(0)/0x0(0) Next 0x0(0)/0x0(0)

Last retransmission scan length is 0, maximum is 0

Last retransmission scan time is 0 msec, maximum is 0 msec

Neighbor 2.2.2.2, interface address 192.1.1.2

In the area 1 via interface FastEthernet0/0

Neighbor priority is 1, State is FULL, 6 state changes

DR is 192.1.1.4 BDR is 192.1.1.2Options is 0x42

Dead timer due in 00:00:38

Neighbor is up for 00:06:31

Index 2/2, retransmission queue length 0, number of retransmission 0

First 0x0(0)/0x0(0) Next 0x0(0)/0x0(0)


Last retransmission scan time is 0 msec, maximum is 0 msec

Show IP OSPF Neighbor Detail


22/93


R3#show ip ospf interfaceFastEthernet0/0 is up, line protocol is up

Internet Address 192.1.1.3/24, Area 1

Process ID 1, Router ID 3.3.3.3, Network Type BROADCAST, Cost: 1

Transmit Delay is 1 sec, State DROTHER, Priority 1

Designated Router (ID) 4.4.4.4, Interface address 192.1.1.4

Backup Designated router (ID) 2.2.2.2, Interface address 192.1.1.2

Timer intervals configured, Hello 10, Dead 40, Wait 40, Retransmit 5

Hello due in 00:00:03

Index 1/1, flood queue length 0

Next 0x0(0)/0x0(0)

Last flood scan length is 0, maximum is 5

Last flood scan time is 0 msec, maximum is 0 msecNeighbor Count is 3, Adjacent neighbor count is 2

Adjacent with neighbor 2.2.2.2 (Backup Designated Router)

Adjacent with neighbor 4.4.4.4 (Designated Router)

Suppress hello for 0 neighbor(s)

Show IP OSPF Interface


23/93


R3#show ip ospf virtual-linksVirtual Link OSPF_VL0 to router 4.4.4.4 is up

Run as demand circuit

DoNotAge LSA allowed.

Transit area 1, via interface FastEthernet0/0, Cost of using 1

Transmit Delay is 1 sec, State POINT_TO_POINT,


Hello due in 00:00:09

Adjacency State FULL(Hello suppressed)

Index 1/3, retransmission queue length 0, number of retransmission 1

First 0x0(0)/0x0(0) Next 0x0(0)/0x0(0)


Last retransmission scan time is 0 msec, maximum is 0 msecR3#

Show IP OSPF Virtual-Links


24/93


Show IP OSPF Stat

R3#sh ip ospf stat

Area 0: SPF algorithm executed 42 times


SPF calculation time

Delta T Intra D-Intra Summ D-Summ Ext D-Ext Total Reason

00:22:00 0 0 0 0 0 0 0 R, N, SN,00:21:44 0 0 4 0 0 0 4 R, SN, X

00:21:34 0 0 4 0 0 0 4 R, SN, X

00:21:24 0 0 0 4 0 0 4 R, SN, X

00:21:14 0 0 0 0 0 0 0 R,

00:21:04 0 0 0 0 0 0 0 R, N, SN,

00:20:54 0 0 0 0 0 0 0 X

00:20:44 0 0 4 0 0 0 4 R, SN, X

00:20:34 0 0 0 0 0 0 0 X00:00:17 4 0 0 0 0 0 4 R, N, SN, SA, X

. . .

R=Router LSA;N=NetworkLSA;SN=Summary Network LSA;SA=Summary

ASBR LSA;X=External LSA


25/93


Show IP OSPF Stat

R1#show ip ospf stat detail

OSPF process ID 100------------------------------------------


SPF 1 executed 1w0d ago, SPF type FullSPF calculation time (in msec):SPT Intra D-Intr Summ D-Summ Ext7 D-Ext7 Total0 0 0 0 0 0 0 0LSIDs processed R:1 N:0 Stub:1 SN:0 SA:0 X7:0Change record R,LSIDs changed 1Last 10 LSIDs:4.4.4.4(R)

Summary OSPF SPF statistic

SPF calculation timeDelta T Intra D-Intra Summ D-Summ Ext D-Ext Total Reason1w0d 0 0 0 0 0 0 0 R,


26/93


R3#show ip ospf border-routersOSPF Process 1 internal Routing Table

Codes: i - Intra-area route, I - Inter-area route

i 4.4.4.4 [1] via 192.1.1.4, FastEthernet0/0, ABR, Area 0, SPF 42

i 4.4.4.4 [1] via 192.1.1.4, FastEthernet0/0, ABR, Area 1, SPF 38

i 8.8.8.8 [10] via 18.10.0.6, Serial0/0, ABR/ASBR, Area 0, SPF 42

i 7.7.7.7 [17] via 192.1.1.4, FastEthernet0/0, ABR/ASBR, Area 0, SPF 42

Show IP OSPF Border


27/93


Other Show Commands

R3#show ip ospf database self-originate




3.3.3.3 3.3.3.3 1520 0x80000015 0xABFD 2

Summary Net Link States (Area 0)

Link ID ADV Router Age Seq# Checksum

192.1.1.0 3.3.3.3 1520 0x80000006 0x4E1A

192.1.2.0 3.3.3.3 1521 0x80000006 0x6103

. . .



3.3.3.3 3.3.3.3 1536 0x80000028 0x612D 2


28/93


Other Show Commands (Cont.)

R3#show ip ospf database adv-router 7.7.7.7




7.7.7.7 7.7.7.7 871(DNA) 0x8000000D 0x8FE2 2

Summary Net Link States (Area 0)

Link ID ADV Router Age Seq# Checksum

20.10.0.0 7.7.7.7 871 (DNA) 0x8000000A 0x39C4

Type-5 AS External Link StatesLink ID ADV Router Age Seq# Checksum Tag

140.100.0.0 7.7.7.7 1944 0x80000004 0x3759 0


29/93

Common IssuesCommon things that can go wrong


30/93


Common Issues

!Adjacency is not coming up! OSPF neighbor stuck in? State! Information is in the DB but not in the RT! SPF running constantly! NSSA ABR not translating Type 7 LSA


31/93


Adjacency Is Not Coming Up

! Show IP OSPF neighbor! Show IP OSPF interface! Debug IP OSPF adjacency

Useful Commands for This Problem


32/93



R3#

R3#show ip ospf interface serial 2

Serial2 is down, line protocol isdown

Internet Address 18.10.0.3/30, Area 0Process ID 1, Router ID 3.3.3.3, Network Type POINT_TO_POINT, Cost: 64

Transmit Delay is 1 sec, State DOWN,



! Layer 2 is down


33/93



! OSPF not enabled on the interface

!

! Tip: Check for the wrong network statement;re-enter the network statement


R3#


Serial2 is up, line protocol is up

OSPF not enabled on this interface

In 12.0:


R3#


34/93


R3#show ip ospf neighborR3#

R3#show ip ospf interface e0Ethernet0 is up, line protocol is upInternet Address 192.1.1.3/24, Area 1Process ID 1, Router ID 3.3.3.3, Network Type BROADCAST, Cost: 10Transmit Delay is 1 sec, State DR, Priority 1Designated Router (ID) 192.1.1.4, Interface address 192.1.1.3

No backup designated router on this networkTimer intervals configured, Hello 10, Dead 40, Wait 40, Retransmit 5No Hellos (Passive interface)


! Interface is defined as passive


35/93



! Mismatched subnet maskR3#show ip ospf neighbor

R3#

R3#debug ip ospf adj

OSPF adjacency events debugging is on

R3#OSPF: Mismatched hello parametersfrom 192.1.1.4

Dead R 40 C 40, Hello R 10 C 10 Mask R 255.255.255.192 C 255.255.255.0


36/93



! Mismatched hello/dead interval

!! Tip: Default is 10 second on LAN


R3#



R3#OSPF: Mismatched hello parameters from 192.1.1.4

Dead R 40 C 40, Hello R 15 C 10Mask R 255.255.255.0 C 255.255.255.0

R4(config-if)#interface ethernet 0

R4(config-if)#no ip ospf hello-interval 15


37/93



! Mismatched authentication key

! Tip: Watch for the space at the end of theauthentication key


R3#



R3#OSPF: Rcv pkt from 192.1.1.4, Ethernet0 : Mismatch Authentication Key-

Clear Text


38/93


OSPF: Rcv pkt from 192.1.1.4, Ethernet0, area 0.0.0.1

mismatch area 0.0.0.2in the headerScenario 2 - Neighbor is in Area 0, but we are not:

%OSPF-4-ERRRCV: Received invalid packet: mismatch area ID, from backbone

area must be virtual-link but not found from 192.1.1.4, Ethernet0

R3#show ip ospfneighbor

R3#

Scenario 1 - Neighbor is in Area 2, but we are in Area 1:




! Mismatched area ID


39/93



! Mismatched transit/stub/NSSA option

OSPF: Hello from 18.10.0.2 with mismatched Stub/Transit areaoption bit


R7#

R7#debug ip ospf adjOSPF adjacency events debugging is on

7.7.7.7 8.8.8.8

R8R7

Area 2Area 2

NSSA

18.10.0.218.10.0.1


40/93


Options

Normal area:OSPF: Send DBD to 141.108.97.1 on Serial0 seq 0xBC4 opt 0x2flag 0x3 len 492

E bit is 1, Allow externals, option: 0x2(HEX) = 00000010(Bin)

Stub area:OSPF: Send DBD to 141.108.97.1 on Serial0 seq 0x1866 opt 0x0flag 0x3 len 372

E bit is 0, no external allowed, options: 0x0 = 00000000

NSSA:OSPF: Send DBD to 141.108.97.1 on Serial0 seq 0x118 opt 0x8flag 0x3 len 372

N/P bit is on, options: 0x8 = 00001000

DC:OSPF: Send DBD to 141.108.97.1 on Serial0 seq 0x1A1E opt 0x20flag 0x3 len 392

DC bit is negotiated, options: 0x20 = 00100000

O DC EA N/P MC E* *


41/93


OSPF Neighbor Stuck In? State

! Show IP OSPF neighbor! Debug IP OSPF adjacency



42/93


OSPF Adjacency States

3.3.3.3 6.6.6.6

R3 R6

ATTEMPT

INIT

2-WAY

EXSTART

EXCHANGE

LOADING

FULL


43/93


RID =6.6.6.6

N = 3.3.3.3

Hello

Stuck In ATTEMPT

3.3.3.3 6.6.6.6

RID =3.3.3.3Hello

R3 R6

NBMA


44/93


Stuck In ATTEMPT

! Our hellos are getting lost in NBMA cloud! Neighbor hellos are getting lost in NBMA cloud! We received neighbors hello but reject it for

some reason

! Misconfigured neighbor statement! Broken Unicast

Reasons


45/93


RID =2.2.2.2

Hello

Stuck In INIT

2.2.2.2

RID =1.1.1.1

Hello

R2

RID =1.1.1.1

Hello

N =2.2.2.2

RID =2.2.2.2

Hello

1.1.1.1

R1


46/93


Stuck In INIT

! One side is blocking the hello packet with access-list

! One side is translating (NAT) OSPF hello! One side multicast capability is broken (Layer 2)! Dialer map or Frame Relay map is missing keyword

broadcast

! LLS capability issue when Cisco IOSis upgraded

Reasons


47/93


Stuck In 2-WAY

1.1.1.1 2.2.2.2

RID =1.1.1.1 P=0

Hello

RID =1.1.1.1 P=0

Hello

N =2.2.2.2

R2R1

RID =2.2.2.2 P=0

Hello

N =1.1.1.1


48/93


Stuck In 2-WAY

! This is normal in broadcast network types! This is to reduce the amount of flooding on the wire! Problem can happen if all the routers are

configured with priority equal to 0

! Only low-end routers should be configured withpriority 0 so they dont participate in DR election

Reasons


49/93


Stuck In EXSTART/EXCHANGE

3.3.3.3 6.6.6.6

RID =3.3.3.3

HelloR6R3

RID =6.6.6.6

Hello

N =3.3.3.3

MTU = 1500 flag = 0x7

DBD

Seq = 1E55

MTU = 1500 flag = 0x7

DBD

Seq = 22AB


50/93



! Useful in debugging, defines I, M and MS bitsOSPF: Send DBD to 141.108.97.1 on Serial0 seq 0xBC4 opt 0x2 flag 0x3len 492

Flag 0x7--> 111 means I(Initial) = 1, M = 1(More), MS = 1(Master)

Flag 0x6 --> 110 not possible



Flag 0x3 --> 011 means master has more data to send

Flag 0x2 --> 010 means slave has more data to send

Flag 0x1 --> 001 means master has no more data left to send

Flag 0x0 --> 000 means slave has no more data left to send

0 0 0 0 0 I M MS


51/93



! MTU mismatchEXCHANGENote: If Cisco IOS is < 12.0.3 neighbor will show stuck in EXCHANGE

! Neighbor RID is same as oursEXSTARTNote: If Cisco IOS is > 12.0.7, it displays msg: %OSPF-3-DUP_RTRID

and OSPF neighbor list will be empty

! Unicast is brokenEXCHANGE1. Wrong VC/DLCi mapping in frame/ATM environment in highly

redundant network

2. MTU problem, cant ping across with more than certainlength packet

3. Access-list blocking unicast; after two-way OSPF send unicastpacket except p2p links

4. NAT is translating unicast packet

Reasons


52/93


Stuck In LOADING

3.3.3.3 6.6.6.6

R6R3

LS Type

LS Req

Link State ID

Advertising Router

# LSAs

LS Update

. . .

LSAs


53/93


Stuck In LOADING

! LS request is being made and neighbor is sendingbad packet or memory corruption exists

a. Do show IP OSPF badto see bad LSA

b. Show log will showOSPF-4-BADLSATYPE message

! LS request is being made and neighbor is ignoringthe request

! MTU mismatch problem (RFC 1583 and 2178compatibility issue) CSCee23634. OSPF shoulddetect if the neighbor MTU is smaller than ours.Fixed in more recent IOS

Reasons


54/93


Information Is In the DB But

Not In the RT

! Show IP OSPF interface ! Show IP OSPF database ! Where x can be router, network, summary, summary-

asbr, external, NSSA



55/93


Mismatched Network Types

Area 03.3.3.3 6.6.6.6

18.10.0.4/30


56/93


18.10.0.4/30

Area 03.3.3.36.6.6.6

Mismatched Network Types (Cont.)

R3#show ip ospf database router 3.3.3.3

. . .

Link ID = 6.6.6.6 Router id of the neighbor

Link Data = 18.10.0.5 IP interface address

Type = 1 This is a point-to-point link

# TOS metrics = 0metric = 8

. . .


. . .

Link ID = 18.10.0.6 IP address of the DR

Link Data = 18.10.0.6 Interface address

Type = 2 This is a transit link# TOS metrics = 0

metric = 8


57/93


Point-to-Point Numberedand Unnumbered Links

Area 03.3.3.3 6.6.6.6

18.10.0.4/30

192.1.4.1


Neighbor ID Pri State Dead Time Address Interface

6.6.6.6 1 FULL/ - 00:00:30 18.0.0.6 Serial0

R3#show interface serial0

Serial0 is up, line protocol is upHardware is HD64570

Interface is unnumbered.Using address of Ethernet1 (192.1.4.1)

R6#show interface serial0

Serial0 is up, line protocol is upHardware is HD64570

Internet address is 18.10.0.6/30


58/93


18.10.0.4/30

Area 03.3.3.3

6.6.6.6

Point-to-Point Numberedand Unnumbered Links (Cont.)

192.1.4.1


. . .


Link Data = 0.0.0.5 MIBII IfIndex Value



. . .


. . .


Link Data = 18.10.0.6 IP interface address

Type = 1 This is a point-to-point link# TOS metrics = 0

metric = 8


59/93


Different Mask or IP Subneton P2P Links

Area 03.3.3.3 6.6.6.6

19.10.0.5/30 18.10.0.6/30

R3#show interface serial 0


Hardware is HD64570

Internet address is 19.10.0.5/24

R6#show interface serial 0


Hardware is HD64570Internet address is 18.10.0.6/30


Neighbor ID Pri State Dead Time Address Interface

6.6.6.6 1 FULL/ - 00:00:30 18.0.0.6 Serial0


60/93


Area 03.3.3.3 6.6.6.6

Different Mask or IP Subneton P2P Links (Cont.)


. . .





. . .


. . .



Type = 1 This is a point-to-point link# TOS metrics = 0

metric = 8


61/93


Address Flipped on Dual Links

3.3.3.36.6.6.6

18.10.0.20/30

Area 0

18.10.0.4/30


Neighbor ID Pri State Dead Time Address Interface6.6.6.6 1 FULL/ - 00:00:30 18.10.0.6 Serial0

6.6.6.6 1 FULL/ - 00:00:33 18.10.0.22 Serial1

R3#show interface serial 0Serial0 is up, line protocol is up

Hardware is HD64570Internet address is 18.10.0.21/30


62/93


ABR2ABR1 R2R1 Area 1

Area 0 Area 0

Discontiguous Backbone

! R1 and R2 are not be able to see each other! Summary LSA for inter-area routes must

not be generated into the backbone

! The solution is to create virtual link betweenABR1 and ABR2

SummaryLSA

SummaryLSA

Area 0


63/93


SPF Running Constantly

! Show IP OSPF stat! Show IP OSPF database! Show IP OSPF database database-sum



64/93



! LSA Flaps Due To:Duplicate RID/IP address

Constant link flapping in an area

Reasons


65/93

2010 Cisco and/or its affiliates. All rights reserved.Cisco ConfidentialPresentation_ID

65


R3#sh ip ospf stat



SPF calculation time

Delta T Intra D-Intra Summ D-Summ Ext D-Ext Total Reason

00:22:00 0 0 0 0 0 0 0 R, N, SN,00:21:44 0 0 4 0 0 0 4 R, SN, X

00:21:34 0 0 4 0 0 0 4 R, SN, X

00:21:24 0 0 0 4 0 0 4 R, SN, X

00:21:14 0 0 0 0 0 0 0 R,

00:21:04 0 0 0 0 0 0 0 R, N, SN,

00:20:54 0 0 0 0 0 0 0 X

00:20:44 0 0 4 0 0 0 4 R, SN, X

00:20:34 0 0 0 0 0 0 0 X

00:00:17 4 0 0 0 0 0 4 R, N, SN, SA, X

. . .

R=Router LSA;N=NetworkLSA;SN=Summary Network LSA;SA=Summary

ASBR LSA;X=External LSA


66/93

2010 Cisco and/or its affiliates. All rights reserved. Cisco ConfidentialPresentation_ID

66

Show IP OSPF Stat

R1#show ip ospf stat detail

OSPF process ID 100------------------------------------------


SPF 1 executed 1w0d ago, SPF type FullSPF calculation time (in msec):SPT Intra D-Intr Summ D-Summ Ext7 D-Ext7 Total0 0 0 0 0 0 0 0LSIDs processed R:1 N:0 Stub:1 SN:0 SA:0 X7:0Change record R,LSIDs changed 1Last 10 LSIDs:4.4.4.4(R)

Summary OSPF SPF statistic

SPF calculation timeDelta T Intra D-Intra Summ D-Summ Ext D-Ext Total Reason1w0d 0 0 0 0 0 0 0 R,


67/93


67

R3#deb ip ospf mon

OSPF: Schedule SPF in area 1

Change in LS ID 1.1.1.1, LSA type R,OSPF: schedule SPF: spf_time 0ms wait_interval 861421816s

OSPF: Begin SPF at 0x33585480ms, process time 752ms

spf_time 0ms, wait_interval 861421816s

OSPF: End SPF at 0x33585488ms, Total elapsed time 8ms

Intra: 4ms, Inter: 0ms, External: 0ms



68/93


68


R3#show ip ospf database



Link ID ADV Router Age Seq# Checksum Link count3.3.3.3 3.3.3.3 106 0x80000009 0xC3F1 3

. . .Summary Net Link States (Area 0)

Link ID ADV Router Age Seq# Checksum18.10.0.0 7.7.7.7 3 (DNA) 0x80000008 0x3DC2

18.10.0.0 8.8.8.8 1396 0x80000004 0x27D8. . .


Link ID ADV Router Age Seq# Checksum Link count1.1.1.1 1.1.1.1 2 0x80000016 0xE6CD 2. . .


69/93


69

Area 0

RID: 7.7.7.7

Type 75 Conversion

NSSA ABR Not Translating Type 7 LSA

! Only NSSA ABR with the highest RIDdoes the conversion

No Type 7/5 Translation

Area 1

NSSAType 7

R8 RID: 8.8.8.8

NSSA

ABR

NSSAASBR


70/93


70

Area 0

NSSAA

BR

Area 1

NSSA

R8 RID: 8.8.8.8

NSSAASBR

Type 7

NSSA ABR Not Translating Type 7 LSA

! Only NSSA ABR with the highest RIDdoes the conversion

Type 75 Conversion

RID: 7.7.7.7

NSSA

ABR


71/93

OSPF for NX-OS and IOS-XR


72/93

Cisco NX-OSOSPF on the Nexus Platforms


73/93


73

!You must first enable the ospf feature in NX-OSbefore OSPF commands can be entered:

Nexus(config)# feature ospf


74/93


74

Quick view of ospf config

Nexus# sh run ospf

!Command: show running-config ospf

!Time: Thu Mar 3 15:21:53 2011

version 5.1(2)

feature ospf

router ospf 100

network 2.2.2.0/24 area 0.0.0.2

network 9.9.9.0/24 area 0.0.0.0


75/93


75

show ip route command has new info

Nexus# sh ip route

IP Route Table for VRF "default'*' denotes best ucast next-hop'**' denotes best mcast next-hop

'[x/y]' denotes [preference/metric]

1.1.1.1/32, ubest/mbest: 1/0

*via 9.9.9.2, Eth1/1, [110/41], 08:03:05, ospf-100,inter

5.5.5.5/32, ubest/mbest: 1/0

*via 9.9.9.2, Eth1/1, [110/41], 05:43:23, ospf-100,intra


76/93


76

OSPF event-history feature

Nexus(config)# router ospf 100

Nexus(config-router)# ip ospf event-history ?

adjacency Adjacency formation logs

cli Cli logsevent Internal event logsflooding LSA flooding logsha HA and GR logslsa LSA generation and database logs

redistribution Redistribution logsspf SPF calculation logs


77/93


77

OSPF event-history feature cont.

Nexus# sh ip ospf event-history ?

adjacency Adjacency formation logscli Cli logsevent Internal event logsflooding LSA flooding logsha HA and GR logslsa LSA generation and database logsmsgs IPC logsredistribution Redistribution logsspf SPF calculation logsstatistics Show the state and size of the buffers


78/93

Cisco IOS-XROSPF on the IOS-XR Platforms


79/93


79

Cisco IOS-XR Comparison to IOS OSPF

! IOS-XR introduces a hierarchical CLI for ease ofconfiguration

! IOS-XR uses CLI inheritance to simplifyconfiguration

! Interface specific parameters are now configuredunder the area portion of the OSPF configuration

! Show command CLI no longer uses the IP prefix.Example: show IP OSPF neighbor becomes showOSPF neighbor


80/93


80

IOS-XR Configuration

RP/0/RP0/CPU0:ios#show run router ospf 1

router ospf 1

log adjacency changes detail

router-id 1.1.1.1

network point-to-point

auto-cost reference-bandwidth 100000

area 0.0.0.0

interface Loopback0

!

interface GigabitEthernet0/3/0/2

cost 10000


81/93


81

IOS-XR Show Commands!

IOS-XR show commands are very similar to IOSRP/0/RP0/CPU0:ios#show ospf database ?

adv-router Advertising Router link states

asbr-summary ASBR summary link states

database-summary Summary of database

external External link states

network Network link states

nssa-external NSSA External link states

opaque-area Opaque Area link states

opaque-as Opaque AS link states

opaque-link Opaque Link-Local link states

router Router link states

self-originate Self-originated link states

summary Network summary link states


82/93


IOS-XR Troubleshooting Using Trace! IOS-XR incorporates a built-in debug called traceRP/0/RP0/CPU0:ios#show ospf trace

Trace Name Size Count Description

1. adj 8192 2963 adjacency

2. adj_cycle 8192 72484 dbd/flood events/pkts

3. config 2048 204 config events

4. errors 8192 1017 errors

5. events 4096 300 mda/rtrid/bfd/vrf

6. ha 8192 309 startup/HA/NSF

7. hello 2048 2852464 hello events/pkts

8. idb 8192 862 interface

9. pkt 2048 1453478 I/O packets

10. rib 8192 1927 rib batching

11. spf 8192 8848 spf/topology

12. spf_cycle 8192 12310 spf/topology detail

13. te 4096 3418 mpls-te

14. test 1024 0 testing info

15. mq 256 15510 message queue info


83/93


IOS-XR Troubleshooting Show Tech

RP/0/RP0/CPU0:ios#show tech routing ?

bfd Output show commands for BFD debugging

bgp Output show commands for BGP debugging

isis Show IS-IS-related diagnostics for Tech-support

ospf OSPF show tech-support output

ospfv3 Output show commands for OSPFv3 debugging

rpl RPL show tech-support output

! Output is extensive. Use the file keyword to save dataon local media.


84/93

Troubleshooting Enhancements

Things Done at Cisco to Make Your Lives Better "


85/93


Interface Scoped Debugging

! Enhancement in limiting the OSPF debug output to just a selectionof interfaces

! Example below will generate debug output for only two interfacesspecified below

debug condition interface Ethernet 0/0

debug condition interface Ethernet 1/0

debug ip ospf hello

debug ip ospf adjacency

Available in: 12.4(4)T, 12.2(30)S, 12.0(32)S

debug ip ospf flood

Further enhancement to allow an access-list filter, and a detail option

(for verbose output)Available in: 12.4(4)T, 12.2(30)S, 12.0(32)S

debug ip ospf packets

Available in: 12.4(10), 12.2(33)SXH, 12.2(33)SRB

Ne


86/93


Interface Brief Keyword

! Show ip ospf interface brief! Show ipv6 ospf interface brief

r100#sh ipv6 ospf interface briefInterface PID Area Intf ID Cost State Nbrs F/CLo2147483644 65535 4294967294 21 65535 LOOP 0/0Lo2147483647 65535 255.255.255.255 22 1 LOOP 0/0VL1 6 0 25 65535 DOWN 0/0VL0 6 0 24 62 P2P 1/1Se3/0 6 0 14 64 DOWN 0/0Lo1 6 0 20 1 LOOP 0/0Se2/0 6 6 10 62 P2P 1/1Tu0 1000 0 19 11111 DOWN 0/0


87/93


Interface-Based OSPF Enable

! OSPF has traditionallysupported enabling the protocolon interfacesby a network command

! On unnumbered interfaces,OSPF automatically assumesthe same areaas the correspondingnumbered interface

! What if the unnumberedinterface is desired to bein a different area?

router ospf 1network 1.1.1.0 255.255.255.0 area 0

e0ip addr 1.1.1.1

se2/0ip unnumbered e0


88/93


Interface-Based OSPF Enable

! Allow an interface specific commandto enable OSPF in a different area

! Interface scoped command hashigher precedence

! This allows to preserveaddress space

! A new tool which can be usedinstead of network command

! By default secondary addresses areannounced; can be turned off withan option

ip ospf 1 area 1 secondaries none

! Available starting in:12.3(11)T, 12.0(29)S

router ospf 1network 1.1.1.0 255.255.255.0 area 0

e0ip addr 1.1.1.1

se2/0ip unnumbered e0ip ospf 1 area 1


89/93


Summary

! Overview of OSPF LSAs! Different troubleshooting commands and what to

look for in those commands

! Common issues in OSPF networks; e.g., adjacencyproblems, information in the database but not in RT,SPF problems and NSSA problems and how tocorrect those problems

! OSPF on Other Platforms! OSPF Troubleshooting Enhancements including

debugs and interface based enhancements

What Have We Learned?


90/93


More Information

! White Papers! Web and Mailers! Cisco Press

CCO OSPF Page:http://www.cisco.com/en/US/tech/tk365/tk480/tsd_technology_support_sub-protocol_home.html

RTFB = Read the Fine Book

C l t Y O li S i


91/93


Complete Your Online SessionEvaluation

! Give us your feedback andyou could win fabulous prizes.Winners announced daily

! Receive 20 Cisco PreferredAccess points for each session

evaluation you complete

! Complete your session evaluationonline now (open a browserthrough our wireless network toaccess our portal) or visit one ofthe Internet stations throughout

the Convention CenterDont forget to activate your Cisco Live

and Networkers Virtual account for access

to all session materials, communities, andon-demand and live activities throughout

the year. Activate your account at any internet

station or visit www.ciscolivevirtual.com.


92/93


Thank you.


93/93

OSPF Troubleshooting

Documents

Transcript of OSPF Troubleshooting