OTV in 1000v CSR

OTV and LISP on the CSR 1000v

OTV and LISP are two interesting new data center technologies that are worth examining when

you are studying for a Cisco Data Center certification, such as CCNP or CCIE Data Center.

Unfortunately, not everybody can afford a couple of Nexus 7000s to play with. As an instructor

for Fast Lane I regularly have access to Nexus based labs, but I still thought that it would be nice

to have a lab setup of my own to experiment with. Fortunately, there is now a very nice way to

get some hands-on experience with these protocols through the Cisco Cloud Services Router

(CSR) 1000v, which I blogged about earlier.

The CSR 1000v is based on the same IOS XE code that runs on the ASR 1000, which supports

both OTV and LISP. So I decided to try to build a lab to test VM mobility using OTV and LISP

in my home lab using a number of CSR 1000v instances.

Note: The CSR runs IOS-XE, not NX-OS, and as a result it uses a different command set to

configure OTV and LISP. In that sense, it cannot replace practicing with actual Nexus gear for

the purposes of exam preparation. However, it does allow you to examine the underlying

structures and mechanisms of the technologies, and it allows you to get an idea of the common

configuration elements in an OTV or LISP configuration.

The Basic Lab Setup

I decide to implement the following topology:

http://www.layerzero.nl/blog/2013/02/csr-1000v-first-impressions/

http://www.cisco.com/en/US/docs/ios-xml/ios/wan_otv/configuration/xe-3s/asr1000/wan-otv-xe-3s-asr1000-book.html

http://www.cisco.com/en/US/docs/ios-xml/ios/iproute_lisp/configuration/xe-3s/asr1000/irl-xe-3s-asr1000-book.html

I create two separate VLANs on my vSWITCH which I intend to bridge together using OTV

between the routers “dc1-otv” and “dc2-otv”. I create two VMs, one in each VLAN, to which I

assign IP addresses from the same IP subnet 192.168.200.0/24. VM “VM-1″ gets IP address

192.168.200.1 and “VM-2″ gets 192.168.200.2. The routers “dc1-xtr”, “dc2-xtr”, and “branch-

xtr” will be configured as LISP xTRs later.

I put the following basic configuration on router dc1-otv:

!

hostname dc1-otv

!

enable secret cisco

!

no ip domain lookup

!

interface GigabitEthernet1

ip address 10.200.200.1 255.255.255.0

no shutdown

!

http://www.layerzero.nl/blog/wp-content/uploads/2013/05/lisp-otv-diagram.jpg

router ospf 1

router-id 10.200.200.1

network 10.200.200.1 0.0.0.0 area 0

!

line vty 0 4

exec-timeout 0 0

password cisco

login

!

And I put a similar configuration on router dc2-otv:

!

hostname dc2-otv

!

enable secret cisco

!

no ip domain lookup

!


ip address 10.200.200.2 255.255.255.0

no shutdown

!

router ospf 1

router-id 10.200.200.2

network 10.200.200.2 0.0.0.0 area 0

!

line vty 0 4

exec-timeout 0 0

password cisco

login

!

Configuring OTV

The next thing I do is preparing the OTV join interface for multicast operation. I enable multicast

routing, set the IGMP version to 3 and enable PIM in passive mode on the OTV join interface:

!

ip multicast-routing distributed

!


ip pim passive

ip igmp version 3

!

Note: Unlike the Nexus 7000, the CSR requires multicast routing to be enabled in order to

enable the IGMP functionality that is required for OTV. On the Nexus 7000 it is not necessary to

enable multicast routing and PIM. Simply setting the IGMP version to 3 is sufficient on that

platform.

Next I configure the OTV site ID and create the Overlay interface on router dc1-otv with the

following parameters:

otv site-identifier 0001.0001.0001

!

interface Overlay1

otv control-group 239.37.37.37

otv data-group 232.37.37.0/24

otv join-interface GigabitEthernet1

no shutdown

Of course I configure router dc2-otv in a similar manner:

otv site-identifier 0002.0002.0002

!

interface Overlay1

otv control-group 239.37.37.37

otv data-group 232.37.37.0/24

otv join-interface GigabitEthernet1

no shutdown

I verify the OTV configuration and confirm that the adjacency between the two routers has been

established:

dc1-otv#show otv overlay 1

Overlay Interface Overlay1

VPN name : None

VPN ID : 1

State : UP

AED Capable : No, site interface not up

IPv4 control group : 239.37.37.37

Mcast data group range(s): 232.37.37.0/24

Join interface(s) : GigabitEthernet1

Join IPv4 address : 10.200.200.1

Tunnel interface(s) : Tunnel0

Encapsulation format : GRE/IPv4

Site Bridge-Domain : None

Capability : Multicast-reachable

Is Adjacency Server : No

Adj Server Configured : No

Prim/Sec Adj Svr(s) : None

dc1-otv#show otv adjacency

Overlay 1 Adjacency Database

Hostname System-ID Dest Addr Up Time State

dc2-otv 001e.bd03.a200 10.200.200.2 00:00:47 UP

One thing that is worth noticing, is that the OTV devices are not marked as “AED capable” yet.

This is caused by the fact that the OTV site VLAN is not configured and operational at this

point. The site VLAN configuration is done slightly differently on the CSR compared to the

Nexus 7000. The CSR is not a switch, and therefore does not support direct configuration of

VLANs. Instead of a site VLAN, a site bridge-group is configured. The bridge group represents

the broadcast domain and can be linked to interfaces and VLAN tags using so-called “service

instances”. To setup the site VLAN I use the following commands on router dc1-otv:

!

otv site bridge-domain 2001

!


no shutdown

service instance 2001 ethernet

encapsulation dot1q 2001

bridge-domain 2001

And similarly, I configure the following on router dc2-otv:

!

otv site bridge-domain 2002

!


no shutdown



bridge-domain 2002

These commmands essentially create a bridged domain on the router, which is then associated

with interface GigabitEthernet2 for frames that carry an 802.1Q VLAN tag of 2001. For more

information about Ethernet Service Instances refer to Configuring Ethernet Virtual Connections

on the Cisco ASR 1000 Series Router in the ASR 1000 configuration guide.

At this point the OTV overlay has become fully operational on both sides:

dc1-otv#show otv overlay 1

Overlay Interface Overlay1

VPN name : None

VPN ID : 1

State : UP

AED Capable : Yes

IPv4 control group : 239.37.37.37

Mcast data group range(s): 232.37.37.0/24

Join interface(s) : GigabitEthernet1

Join IPv4 address : 10.200.200.1

Tunnel interface(s) : Tunnel0

Encapsulation format : GRE/IPv4

Site Bridge-Domain : 2001

Capability : Multicast-reachable

Is Adjacency Server : No

Adj Server Configured : No

Prim/Sec Adj Svr(s) : None

In this lab the site bridge-domain configuration is pretty meaningless, because there is only a

single OTV edge device per site. Therefore, I just attached the bridge-group to an arbitrary

interface and VLAN tag, to simply ensure that the overlay interface would become operational.

In reality, you should take care that the VLAN selected for the site bridge-domain is properly

extended between OTV edge devices within the site, but not carried across the overlay.

http://www.cisco.com/en/US/docs/ios-xml/ios/cether/configuration/xe-3s/asr1000/ce-ether-vc-infra-xe.html

http://www.cisco.com/en/US/docs/ios-xml/ios/cether/configuration/xe-3s/asr1000/ce-ether-vc-infra-xe.html

The final piece in this configuration is to actually extend some VLANs across the OTV overlay.

Again, this is done through a bridge-group and corresponding service instance configurations. I

add the following configuration to router dc1-otv:

!



encapsulation untagged

rewrite ingress tag push dot1q 200 symmetric

bridge-domain 200

!

interface Overlay1



bridge-domain 200

!

And I add a similar configuration on dc2-otv:

!



encapsulation untagged

rewrite ingress tag push dot1q 200 symmetric

bridge-domain 200

!

interface Overlay1



bridge-domain 200

!

This configuration is a little peculiar, which has to do with the specifics of my lab setup. The

intention is to create a single VLAN 200 stretched across the two DC sites. However, in my lab

this is all setup on a common virtual infrastructure. To still create two separate “VLAN 200″

instances I essentially created two VMware port-groups and associated VLANS (VLAN 201 and

VLAN 202). CSR dc1-otv and VM-1 are both connected to VLAN 201. Similarly, CSR dc2-otv

and VM-2 are connected to VLAN 202 (see diagram). As a result, the “VLAN 200″ frames

arrive as untagged frames on the internal interfaces of CSR dc1-otv and dc2-otv. These frames

then need to be bridged across the cloud as VLAN 200 frames. In a more realistic scenario the

frames would already arrive with VLAN 200 tags on the internal interfaces and the rewrite

commands would be unnecessary.

With these final steps the OTV configuration is finished and I can put it to the test. So I ping

from VM-1 in DC-1 (192.168.200.1) to VM-2 in DC-2 (192.168.200.2):

The ping succeeds, confirming that OTV is operational.

Update: Brantley Richbourg and Brandon Farmer pointed out in the comments that this ping

fails if you don’t have your VMware vSwitch set to accept promiscuous mode. Initially, I didn’t

notice this behavior, because I already had my vSwitch set to accept promiscuous mode for

different reasons. I retested the lab with promiscuous mode set to “reject” and confirmed that this

stops the ping from working. The explanation for this behavior is that the frames from VM-1 to

VM-2 have the VM-2 MAC address as the destination MAC address, which is not registered to

the CSR virtual NIC. For unicast MAC frames, the vSwitch normally only sends frames with a

particular destination MAC address to the VM that is associated with this MAC address (if it is

local) or to an uplink (if it is remote). Therefore, the VM-1 to VM-2 frame is not sent to the CSR

VM, so the CSR never sees the frame. As a result, the frame cannot be forwarded across the

overlay. When promiscuous mode is set to “accept” on the vSwitch, the CSR receives all traffic

on VLAN 201, allowing it to forward the VM-1 to VM-2 traffic across the overlay. So, if this

http://www.layerzero.nl/blog/wp-content/uploads/2013/05/ping-across-overlay.jpg

ping fails in your lab, make sure that you have your vSwitch set to accept promiscuous

mode! Thanks to Brantley and Brandon for pointing out this potential issue with the lab setup!

Now, let’s verify the MAC address entries and ARP entries on the OTV edge devices:

dc1-otv#show otv route

Codes: BD - Bridge-Domain, AD - Admin-Distance,

SI - Service Instance, * - Backup Route

OTV Unicast MAC Routing Table for Overlay1

Inst VLAN BD MAC Address AD Owner Next Hops(s)

----------------------------------------------------------

0 200 200 000c.296a.a4ad 40 BD Eng Gi2:SI201

0 200 200 000c.297c.e283 50 ISIS dc2-otv

2 unicast routes displayed in Overlay1

----------------------------------------------------------

2 Total Unicast Routes Displayed

dc1-otv#show otv arp-nd-cache

Overlay1 ARP/ND L3->L2 Address Mapping Cache

BD MAC Layer-3 Address Age (HH:MM:SS) Local/Remote

200 000c.297c.e283 192.168.200.2 00:00:33 Remote

Also, the internal OTV IS-IS database can be examined to confirm that the MAC addresses are

advertised by the OTV edge devices:

dc1-otv#show otv isis database detail

Tag Overlay1:

IS-IS Level-1 Link State Database:

LSPID LSP Seq Num LSP Checksum LSP Holdtime ATT/P/OL

dc2-otv.00-00 0x0000000C 0xBED7 933 0/0/0

Area Address: 00

NLPID: 0xCC 0x8E

Hostname: dc2-otv

Metric: 10 IS-Extended dc1-otv.01

Layer 2 MAC Reachability: topoid 0, vlan 200, confidence 1

000c.297c.e283

dc1-otv.00-00 * 0x0000000C 0xE51A 959 0/0/0

Area Address: 00

NLPID: 0xCC 0x8E

Hostname: dc1-otv



000c.296a.a4ad

dc1-otv.01-00 * 0x0000000A 0x2916 753 0/0/0



Although it is usually not necessary to dive into the IS-IS database that is used by the OTV

control plane, it is nice to be able to take a peek under the hood.

So now that we have a working OTV setup that extends VLAN 200 and the corresponding

subnet 192.168.200.0/24 across the two DC sites, it is time to add LISP to optimize the inbound

routing for mobile VMs.

Preparing for LISP

To start, I add two additional routers to the network, which will act as a LISP ingress tunnel

router (ITR) and egress tunnel router (ETR) for their respective sites: router dc1-xtr and router

dc2-xtr. I connect these routers to the IP core and enable HSRP on the interface that faces the

stretched VLAN 200. This results in the following basic configurations:

!

hostname dc1-xtr

!

enable secret cisco

!

no ip domain lookup

!


ip address 10.200.200.3 255.255.255.0

no shutdown

!


ip address 192.168.200.252 255.255.255.0

standby 200 ip 192.168.200.254

no shutdown

!

router ospf 1

router-id 10.200.200.3

network 10.200.200.3 0.0.0.0 area 0

!

line vty 0 4

exec-timeout 0 0

password cisco

login

!

and

!

hostname dc2-xtr

!

enable secret cisco

!

no ip domain lookup

!


ip address 10.200.200.4 255.255.255.0

no shutdown

!


ip address 192.168.200.253 255.255.255.0

standby 200 ip 192.168.200.254

no shutdown

!

router ospf 1

router-id 10.200.200.4

network 10.200.200.4 0.0.0.0 area 0

!

line vty 0 4

exec-timeout 0 0

password cisco

login

!

While doing my verifications I notice something interesting in the behavior of HSRP:

dc1-xtr#show standby brief

P indicates configured to preempt.

|

Interface Grp Pri P State Active Standby Virtual IP

Gi2 200 100 Active local unknown

192.168.200.254

dc2-xtr#show standby brief

P indicates configured to preempt.

|

Interface Grp Pri P State Active Standby Virtual IP

Gi2 200 100 Active local unknown

192.168.200.254

Both routers dc1-xtr and dc2-xtr consider themselves to be the active router and do not list a

standby router. Is OTV not properly bridging the traffic of these routers across the overlay? Let’s

try a quick ping to see if these routers have connectivity across the extended VLAN 200:

dc1-xtr#ping 192.168.200.253

Type escape sequence to abort.

Sending 5, 100-byte ICMP Echos to 192.168.200.253, timeout is 2 seconds:

..!!!

Success rate is 60 percent (3/5), round-trip min/avg/max = 1/1/2 ms

So OTV seems to properly bridge the packets between routers dc1-xtr and dc2-xtr across the

overlay. Let’s have a closer look on the OTV edge devices:

dc1-otv#show otv detail | include FHRP

FHRP Filtering Enabled : Yes

dc1-otv#show otv route

Codes: BD - Bridge-Domain, AD - Admin-Distance,

SI - Service Instance, * - Backup Route

OTV Unicast MAC Routing Table for Overlay1

Inst VLAN BD MAC Address AD Owner Next Hops(s)

----------------------------------------------------------

0 200 200 0000.0c07.acc8 40 BD Eng Gi2:SI201

*0 200 200 0000.0c07.acc8 50 ISIS dc2-otv

0 200 200 000c.290b.cab0 50 ISIS dc2-otv

0 200 200 000c.295d.29e5 40 BD Eng Gi2:SI201

0 200 200 000c.296a.a4ad 40 BD Eng Gi2:SI201

0 200 200 000c.297c.e283 50 ISIS dc2-otv

6 unicast routes displayed in Overlay1

----------------------------------------------------------

6 Total Unicast Routes Displayed

dc1-otv#sh otv isis database detail

Tag Overlay1:

IS-IS Level-1 Link State Database:

LSPID LSP Seq Num LSP Checksum LSP Holdtime ATT/P/OL

dc2-otv.00-00 0x00000025 0xF13B 1178 0/0/0

Area Address: 00

NLPID: 0xCC 0x8E

Hostname: dc2-otv



0000.0c07.acc8 000c.290b.cab0 000c.297c.e283

dc1-otv.00-00 * 0x00000025 0x5EE0 1180 0/0/0

Area Address: 00

NLPID: 0xCC 0x8E

Hostname: dc1-otv



000c.295d.29e5 000c.296a.a4ad

dc1-otv.01-00 * 0x0000000D 0x2319 659 0/0/0



As it turns out, OTV on the CSR has FHRP filtering built-in and enabled by default. This means

that it is not necessary to configure customized access-lists to filter HSRP hellos across the

overlay. Interestingly enough, it does seem to advertise the HSRP MAC address through OTV

IS-IS. When you configure manual FHRP filtering on a Nexus 7000 you would usually suppress

these advertisements as well as the actual HSRP packets. It looks like this behavior on the CSR

could lead to continual MAC updates for the HSRP MAC address, which in turn could affect

control plane stability. This may be a point worth investigating further if you are considering

deploying CSR/ASR-based OTV in production. On the other hand, I really like the fact that the

CSR has FHRP filtering straight out of the box and that it is controllable through a simple

command (otv filter-fhrp), rather than a cumbersome access-list configuration.

Configuring LISP

Now that we have verified the basic setup we can start on the actual LISP configuration. I start

by configuring my LISP map-server and map-resolver. I want to make these vital functions

redundant and I want to separate these functions from the LISP tunnel routers (xTR) to keep the

configurations a bit cleaner. Although it is definitely possible to put the map-server/map-resolver

functions on a router that also acts as a LISP xTR, I think this might make the configurations a

bit harder to understand. Also, I intend to take some LISP sniffer traces on this lab later and

having separate IP addresses for the different functions will make these traces more easily

readable.

Of course I could have added two more routers to my lab, but since the OTV routers are separate

from the LISP xTRs anyway, I decide to make these routers perform the role of LISP map-

servers and map-resolvers. First, I add the LISP map-server function to routers dc1-otv and dc2-

otv:

router lisp

site DC1-DC2

authentication-key DC1-DC2-S3cr3t

eid-prefix 192.168.200.0/24 accept-more-specifics

!

ipv4 map-server

I added the accept-more-specifics keyword to the EID prefix to allow registration of

individual mobile /32 routes later.

Next, I set up the map-resolver function on routers dc1-otv and dc2-otv:

interface Loopback37

description LISP map-resolver anycast address

ip address 10.37.37.37 255.255.255.255

!

router ospf 1

network 10.37.37.37 0.0.0.0 area 0

!

router lisp

ipv4 map-resolver

To make the LISP resolver function redundant I add an anycast address (10.37.37.37) to both

dc1-otv and dc2-otv. This address will be configured as the LISP map-resolver address on the

LISP xTRs.

Note: When configuring anycast IP addresses on routers, you should take proper care that these

addresses never get selected as a router ID for any routing protocol. This is why I specifically

configured the OSPF router ID on these routers using the router-id command.

Now that the LISP map-server and map-resolver have been set up, I configure the routers dc1-xtr

and dc2-xtr as LISP xTRs for the 192.168.200.0/24 EID space that is associated with the OTV

extended VLAN 200. On router dc1-xtr I add the following commands:

router lisp

locator-set DC1

10.200.200.3 priority 10 weight 50

!

database-mapping 192.168.200.0/24 locator-set DC1

!

ipv4 itr map-resolver 10.37.37.37

ipv4 itr

ipv4 etr map-server 10.200.200.1 key DC1-DC2-S3cr3t


ipv4 etr

And on router dc2-xtr I add the following commands:

router lisp

locator-set DC2

10.200.200.4 priority 10 weight 50

!


!


ipv4 itr



ipv4 etr

The only real difference between these configurations is the locator IP address, which is

10.200.200.3 for DC1 and 10.200.200.4 for DC2.

Next, I verify that the EID prefix 192.168.200.0/24 has been registered on the map-servers:

dc1-otv#show lisp site detail

LISP Site Registration Information

Site name: DC1-DC2

Allowed configured locators: any

Allowed EID-prefixes:

EID-prefix: 192.168.200.0/24

First registered: 00:04:05

Routing table tag: 0

Origin: Configuration, accepting more specifics

Merge active: No

Proxy reply: No

TTL: 1d00h

State: complete

Registration errors:

Authentication failures: 0

Allowed locators mismatch: 0

ETR 10.200.200.3, last registered 00:00:09, no proxy-reply, map-notify

TTL 1d00h, no merge, hash-function sha1, nonce

0x24CDDA34-0x0CF777A1

state complete, no security-capability

xTR-ID 0x78ED0ACF-0x8B46F5F7-0xF896252E-0xEC47696E

site-ID unspecified

Locator Local State Pri/Wgt

10.200.200.3 yes up 10/50



0xEB7D2794-0x64B6413D


xTR-ID 0x58ABCEE0-0x5DF04443-0xBB0C7B58-0x4AEB3FD8

site-ID unspecified


10.200.200.4 yes up 10/50

To test the LISP functionality, I need to have a third site that I can run connectivity tests from.

This is the role of the branch router in the lab topology. I add the following basic configuration

to that router:

!

hostname branch-xtr

!

enable secret cisco

!

no ip domain lookup

!

ip dhcp pool BRANCH-LAN

network 172.16.37.0 255.255.255.0

default-router 172.16.37.254

!


ip address 10.200.200.5 255.255.255.0

no shutdown

!


ip address 172.16.37.254 255.255.255.0

no shutdown

!

router ospf 1

router-id 10.200.200.5

network 10.200.200.5 0.0.0.0 area 0

!

line vty 0 4

exec-timeout 0 0

password cisco

login

!

Because this router is part of the LISP enabled network and the prefix 172.16.37.0/24 is part of

the EID space, we need to create a corresponding LISP site configuration for this site on the

LISP map-servers. So I add the following commands to routers dc1-otv and dc2-otv:

router lisp

site BRANCH

authentication-key Br@nch-S3cr3t

eid-prefix 172.16.37.0/24

Now that the map-servers have been set up, we can add the LISP xTR configuration on the

branch router. I add the following to router branch-xtr:

router lisp

database-mapping 172.16.37.0/24 10.200.200.5 priority 10 weight 100


ipv4 itr

ipv4 etr map-server 10.200.200.1 key Br@nch-S3cr3t

ipv4 etr map-server 10.200.200.2 key Br@nch-S3cr3t

ipv4 etr

And again, I confirm that the EID prefix has been properly registered on the map-servers:

dc1-otv#show lisp site name BRANCH

Site name: BRANCH

Allowed configured locators: any

Allowed EID-prefixes:

EID-prefix: 172.16.37.0/24

First registered: 00:00:11

Routing table tag: 0

Origin: Configuration

Merge active: No

Proxy reply: No

TTL: 1d00h

State: complete

Registration errors:

Authentication failures: 0

Allowed locators mismatch: 0



0x4A43B6D4-0xF8540179


xTR-ID 0xE8323230-0xFABD8623-0x2448E48B-0x2B80C3A0

site-ID unspecified


10.200.200.5 yes up 10/100

So now it is time to put the LISP configurations to the test and ping from VM-3 to VM-1 and

VM-2. The pings succeed as expected:

However, at this point we have only implemented a straightforward LISP setup without

introducing the VM mobility concept. The branch router does not know where to forward

packets for individual VM IP addresses. It only knows how to reach the EID prefix

192.168.200.0/24 of VLAN 200. When I actually perform a traceroute to VM-1 and VM-2 from

VM-3 I see that the traffic to both VMs is routed through DC-1:

http://www.layerzero.nl/blog/wp-content/uploads/2013/05/ping-across-lisp.jpg

http://www.layerzero.nl/blog/wp-content/uploads/2013/05/trace-across-lisp.jpg

This means that traffic from VM-3 to VM-2 actually needs to go across the OTV interconnect

between DC1 and DC2 in order to reach VM-2. Clearly, this is sub-optimal and this is where

LISP mobility comes in. By registering individual /32 EID prefixes for each VM, the location of

the VMs can be tracked in the LISP enabled network and the traffic flow can be optimized. So I

add the following commands to router dc1-xtr:

router lisp

dynamic-eid MOBILE-VMS


map-server 10.200.200.1 key DC1-DC2-S3cr3t


map-notify-group 224.0.0.100

!


lisp mobility MOBILE-VMS

lisp extended-subnet-mode

And I add similar commands on dc2-xtr:

router lisp

dynamic-eid MOBILE-VMS




map-notify-group 224.0.0.100

!


lisp mobility MOBILE-VMS

lisp extended-subnet-mode

For the mobile VMs I selected a sub-prefix of the overall 192.168.200.0/24 prefix that belongs to

the twin-DC site. Of course, I could also have used the complete prefix for mobility. In that case

the regular LISP database mapping for that prefix should be removed.

Note: There is one other thing that is specifically worth noting about this configuration: I

configured the link-local multicast group 224.0.0.100 as the LISP map-notify group. This is not a

best practice and I should have used a regular ASM multicast group (for example 239.1.1.1).

However, when I first configured LISP using a random 239.0.0.0/8 ASM group I was

experiencing all sorts of problems. My VMs were reporting duplicate IP addresses and the

traceroutes from VM-3 were inconsistent. After a couple of hours of troubleshooting, I finally

figured out that this was caused by the fact that the LISP map-notify multicast group wasn’t

properly forwarded across OTV between routers dc1-xtr and dc2-xtr. I tried to tackle this

problem in various ways, including converting my OTV configuration to a unicast setup with

adjacency servers, but to no avail. In the end I started suspecting IGMP snooping (or the lack

thereof) in the whole chain of vSwitches, bridge-domains, and OTV to be the cause of the

problem. To test this hypothesis I decided to change the multicast group to a link-local group,

because those groups should always be flooded within a VLAN regardless of IGMP snooping. It

is a bit of a hack, and clearly it isn’t a real solution for the underlying multicast problem. But at

least implementing this workaround allowed me to further concentrate on LISP rather than an

obscure multicast issue. I am hoping that this is a VMware vSwitch problem, rather than an OTV

multicast problem, but further testing is needed to pinpoint the issue.

So with the proper LISP configuration in place we can now test the traceroutes from VM-3 to

VM-1 and VM-2 again and see if the path was optimized:

The two different RLOC addresses for the VMs can also be verified in the LISP map-cache on

the branch router branch-xtr:

branch-xtr#show ip lisp map-cache

LISP IPv4 Mapping Cache for EID-table default (IID 0), 3 entries

0.0.0.0/0, uptime: 00:47:51, expires: never, via static send map-request

Negative cache entry, action: send-map-request

192.168.200.1/32, uptime: 00:45:47, expires: 23:54:47, via map-reply,

complete

Locator Uptime State Pri/Wgt

10.200.200.3 00:05:12 up 10/50


complete


10.200.200.4 00:35:25 up 10/50

As a final test I move VM-1 from DC-1 to DC-2. Due to the way the lab is set up this is not a

vMotion, but simply a change of port-group from VLAN 201 to VLAN 202. When I trace again

after the move I confirm that the traceroute now goes through router dc2-xtr (10.200.200.4).

During the move I also ran a continuous ping from VM-3 to VM-1 and I only lost a couple of

packets. Of course, the LISP map-cache on router branch-xtr also reflects the change in RLOC

IP address:

http://www.layerzero.nl/blog/wp-content/uploads/2013/05/trace-across-mobile-lisp.jpg

branch-xtr#show ip lisp map-cache 192.168.200.1

LISP IPv4 Mapping Cache for EID-table default (IID 0), 3 entries


complete

Sources: map-reply

State: complete, last modified: 00:00:43, map-source: 10.200.200.4

Active, Packets out: 195 (~ 00:01:56 ago)


10.200.200.4 00:00:43 up 10/50

Last up-down state change: 00:00:43, state change count: 1

Last route reachability change: never, state change count: 0

Last priority / weight change: never/never

RLOC-probing loc-status algorithm:

Last RLOC-probe sent: never

At this point, we have a working configuration combining OTV with LISP mobility, which is a

good base for further experimentation with these protocols. Despite the multicast problems that I

experienced, I feel that a virtual lab setup with a couple of CSR 1000v’s is a nice addition to the

toolbox for testing advanced routing and data center technologies like LISP and OTV.

OTV in 1000v CSR

Documents

Transcript of OTV in 1000v CSR