Leveraging Netscaler 10 Clusters to Improve Application Performance

Citrix Virtual Classroom

Leveraging NetScaler clusters to improve

application performance

Self-paced exercise guide


Table of Contents

Table of Contents .............................................................................................................................................. 2

Lab Overview ..................................................................................................................................................... 3

Exercise 1: Migrating an HA pair to a 3-node cluster .................................................................................. 6

Exercise 2: Initial cluster setup ...................................................................................................................... 12

Exercise 3: Cluster load balancing configuration ........................................................................................ 23

Exercise 4 (Optional): Packet capture analysis in a cluster system .......................................................... 37

Exercise 5 (Optional): ECMP configuration ............................................................................................... 52


Lab Overview

Hands-on Training Module

This training module has the following details:

Objective This lab manual provides hands-on training on the new clustering feature included in NetScaler 10. It covers the most common clustering deployment types as well as step-by-step configuration on how to setup a NetScaler cluster.

Upon completion of this lab document, a 3-node cluster environment should be fully operational.

Audience Primary: Event attendees

Lab Environment Details

Machine Details

XenServer Hosts virtual machines

AD.training.lab Domain Controller – DNS server

Apache_MySQL_1 (GENTOO_1) Apache, PHP, MySQL server



EXT-Win_7 Remote network Windows 7 workstation

Win7Client Internal network Windows 7 workstation

NS10_node1 Virtual instance of a NetScaler appliance.

NS10_node2 Virtual instance of a NetScaler appliance.

NS10_node3 Virtual instance of a NetScaler appliance

vyatta-RTR Provides connectivity between internal and remote networks


Lab Topology Diagram

Remote Network

Internal Router

Public Network

EXT-Win_7

Internal network

Win7Client

NS10_node1

NS10_node3

NS10_node2

192.168.10.X/24

DNS server

172.16.2.0/24

Apache2

Apache3

Apache1

172.16.1.0/24

.200 .12

.110

.120

.15

.130

.1 .1

.14

.13

.2

Required Lab Credentials

Below are the login credentials required to connect to the workshop system and complete the lab

exercises.

Machine IP Address Username Password

AD.training.lab 192.168.10.11 TRAINING\Administrator Citrix123

Apache_MySQL_1 (GENTOO_1)

192.168.10.13 root Citrix123

Apache_MySQL_2

(GENTOO_2)

192.168.10.14 root Citrix123

Apache_MySQL_3

(GENTOO_3)

192.168.10.15 root Citrix123

EXT-Win_7 172.16.2.2 TRAINING\Administrator Citrix123

Win7Client 192.168.10.200 TRAINING\Administrator Citrix123


NS10_node0 NSIP: 192.168.10.110 SNIP: 192.168.10.91

nsroot nsroot

NS10_node1 NSIP: 192.168.10.120 SNIP: 192.168.10.92

nsroot nsroot

NS10_node3 NSIP: 192.168.10.130 SNIP: 192.168.10.93

nsroot nsroot

vyatta-RTR eth0: 172.16.2.1

eth1: 172.16.1.1

vyatta Vyatta

How to log into the lab environment

The self-paced lab environment is hosted on a cloud-based Citrix XenServer. Connecting to your server

from the portal page is as easy as 1-2-3.

Step-by-step login instructions

Step Action

1. Once logged in at the self-paced portal, click the Start lab button to launch a connection to published

XenCenter.

2. When XenCenter loads, right-click the XenCenter node and select Add…

3. On the Add New Server screen enter the XenServer IP address provided on the portal and in the

Password field enter the password provided on the portal. The user name will always be root.


Exercise 1: Migrating an HA pair to a 3-node

cluster

Overview

Clustering is the centerpiece of NetScaler 10. It is part of the TriScale architecture to deliver a service

delivery fabric to fundamentally change the ADC space. By allowing all nodes in the cluster to efficiently

process traffic as well as providing high availability, it provides the foundation to deploy the necessary

infrastructure to meet any kind of performance and scalability requirements.

With the existing infrastructure, active-active or HA deployments have shortcomings directly addressed by

clustering:

HA Active/Active

Scalability (Only one appliance processing traffic)

Configuration management (Independent configuration between appliances)

Resource utilization. (One resource always idle)

Heavy dependency on upstream device configuration.

Not all L4-L7 features work across all nodes (max. clients, persistence, session reuse, etc..)

With the new architecture offered by NetScaler 10, requirements for dynamic capacity, fault tolerance,

high scalability and performance are met. The first step is to plan the migration of existing HA

deployments.

In this exercise, we will prepare an existing HA pair to become ready for a 3-node cluster deployment.


Step by step guidance

The lab environment required for this exercise is as follows:

1. NetScaler VPX appliance: NS10_node1 (Existing HA node)

2. NetScaler VPX appliance: NS10_node2 (Existing HA node)

3. NetScaler VPX appliance: NS10_ node3 (Independent node)

An HA pair has been pre-configured with a simple load balancing configuration:

enable ns feature WL SP LB

enable ns mode FR L3 Edge USNIP PMTUD

add ns ip 192.168.10.91 255.255.255.0 -vServer DISABLED

add HA node 1 192.168.10.110 / add HA node 1 192.168.10.120

add server GENTOO_1 192.168.10.13

add server GENTOO_2 192.168.10.14

add service GENTOO_1_HTTP_TCP_80 GENTOO_1 HTTP 80

add service GENTOO_2_HTTP_TCP_80 GENTOO_2 HTTP 80

add lb vserver HTTP_vserver_cluster HTTP 192.168.10.18 80

bind lb vserver HTTP_vserver_cluster GENTOO_1_HTTP_TCP_80

bind lb vserver HTTP_vserver_cluster GENTOO_2_HTTP_TCP_80

Our goal is to migrate to a cluster setup without losing any configuration.


Estimated time to complete this lab: 15 minutes.

Step Action

1. After XenCenter loads the

available VMs, proceed with the

next step. Please make sure only

the following VMs are powered

on:

AD.training.lab

Apache_MySQL_1

Apache_MySQL_2

Apache_MySQL_3

EXT-WIN_7

NS10_node1

NS10_node2

NS10_node3

Vyatta-RTR

Win7Client

2. In order to migrate to a cluster

configuration, we need to break

the existing HA pair. However,

before removing the HA

configuration, a few precautions

need to be completed.

Login to the primary node

(NS10_node1) and verify the

HA status.

NS10_node1> show ha node

1) Node ID: 0

IP: 192.168.10.110 (NS10_node1)

Node State: UP

Master State: Primary

...

2) Node ID: 1

IP: 192.168.10.120

Node State: UP

Master State: Secondary

...

Done

NS10_node1>

3. While in the primary node,

backup the configuration.

# shell

# cp /nsconfig/ns.conf

/nsconfig/ns.conf_beforeCluster


Step Action

4. As part of this exercise, we will

verify that HA disassociation

does not disrupt traffic. For this

purpose we will use a simple

utility to constantly send

HTTP requests to the virtual

server already configured. On

Apache_MySQL_3

(GENTOO_3) use “wbox” to

send 5 GET requests per

second to the load balancing

virtual server. Leave this utility

running.

gentoo_3 ~ # wbox 192.168.10.18 clients 5

WBOX 192.168.10.18 (192.168.10.18) port 80

0. 200 OK 319 bytes 3 ms

0. 200 OK 319 bytes 3 ms

0. 200 OK 319 bytes 4 ms

0. 200 OK 319 bytes 3 ms

0. 200 OK 319 bytes 4 ms

1. 200 OK (318) bytes 2 ms

1. 200 OK (318) bytes 2 ms

1. 200 OK (318) bytes 2 ms

1. 200 OK (318) bytes 2 ms

1. 200 OK (318) bytes 2 ms

5. Next, set the primary node to

STAYPRIMARY and disable

command propagation and

synchronization in order to

avoid any configuration loss or

IP conflicts.

# exit

NS10_node1> set ha node -haStatus STAYPRIMARY

-haprop DISABLED -haSync DISABLED

Done

NS10_node1>

6. Logon to the secondary unit

and clear the configuration to

avoid IP conflicts when we

break the HA pair.

NS10_node2> clear ns config full

Are you sure you want to clear the

configuration(Y/N)? [N]:y

Done

NS10_node2>

7. Logon to the primary and

remove the secondary node

from the HA configuration.

On Apache_MySQL_3, inspect

the output of the probes. Was

any request lost?

Verify the HA status on both

units.

NS10_node1> rm ha node 1

Done

NS10_node1>


1) Node ID: 0

IP: 192.168.10.110 (NS10_node1)

Node State: STAYPRIMARY


...

NS10_node1>


1) Node ID: 0

IP: 192.168.10.120 (NS10_node2)

Node State: UP


...

NS10_node2>

http://hping.org/wbox/



Step Action

8. As a precaution, restore the

original HA settings (Since

clustering does not use HA

configuration, this step might

not be required; however, for

completion, we are reverting

these settings to defaults).

NS10_node1> set ha node -haProp ENABLED -

haStatus ENABLED -haSync ENABLED

Done

NS10_node1>

9. At this stage, node 1 should have continued to serve traffic. We are ready to begin the initial

cluster configuration. Continue with exercise 2.


Summary

Key

Takeaways

The key takeaways for this exercise are:

A high availability pair can be migrated to a cluster setup. This involves

breaking the HA pair setup and clearing the configuration on the secondary.

No loss of connectivity should occur during the HA dissociation.

Unfortunately, upon creating the cluster instance, the configuration will be

cleared. We must backup the configuration before migrating to a cluster

setup to avoid any loss.

NOTES A cluster setup is only functional if at minimum (n/2) +1, where n is the number of cluster nodes, are able to serve traffic. If that formula were applied to a two-node cluster, the cluster would fail if one node went down (n/2 +1=2). A two-node cluster is and exception to the rule and it is fully functional even if only one node is able to serve traffic. Creating a two node cluster is the same as creating any other cluster. You must add one node as the configuration coordinator and the other node as an additional cluster node.


Exercise 2: Initial cluster setup

Overview

A NetScaler cluster uses multiple mechanisms to efficiently distribute traffic across all nodes in the cluster.

This can be achieved by using the following available methods:

ECMP: Equal Cost Multipath Routing

CLAG: Cluster Link Aggregation Channels

Link Set

Since we have a virtual lab infrastructure, in this exercise, we will perform the necessary configuration to

bring up a cluster using a link set deployment which does not require switch or router configuration for

the cluster to be fully operational.



1. NetScaler VPX appliance: NS10 _node1

2. NetScaler VPX appliance: NS10_node2


4. Internal Windows 7 Workstation: Win7Client


Step Action

4. All three VPX appliances

are already configured with

a NSIP and default route.

Additionally, node1 will

contain existing

configuration from the

previous HA setup.

First, login to each

appliance and verify IP

connectivity to each other.

> ping 192.168.10.110

> ping 192.168.10.120

> ping 192.168.10.130


Step Action

5. A cluster topology

introduces the concept of

“communication planes”.

There are four logical

traffic flows in a cluster

system that identify the

relevant information that is

passed between client,

servers, and nodes in the

cluster:

1. Client data plane

2. Control plane

3. Server data plane

4. Back plane

Our topology (link set)

does not require separation

of such planes as they all

can co-exist in the shared

medium. However, for

illustration purposes, in our

environment a new virtual

interface was configured to

simulate the backplane.

In XenCenter, verify that

all 3 VPX appliances are

connected to such logical

segment.

Continue with the next

exercise.


Step Action

6. The next step is to add the

first node to the cluster.

This node will act as the

configuration coordinator

(CCO).

Go back to node 1

(NS10_node1) and define

the cluster instance.

NS10_node1> add cluster instance 1

7. Add node 1 to the cluster

instance. In order to define

spotted SNIPs to each

node later in this exercise,

set the state to passive.

Make sure you select

interface 1/2 as the

backplane interface.

NS10_node1> add cluster node 1 192.168.10.110 -

state PASSIVE -backplane 1/2

8. Enable the cluster instance.

After executing this

command, the

configuration needs to be

saved and the appliance

has to be rebooted.

Note: After enabling the

cluster instance, the

configuration is cleared

internally using the

“clear ns config”

command. We will use the

backup to restore the

entities lost.

NS10_node1> enable cluster instance 1

Warning: The configuration must be saved and

the system rebooted for these settings to take

effect

NS10_node1> save ns config

NS10_node1> reboot –warm

9. When node 1 comes back

online, add the cluster IP

NS10_node1> add ns ip 192.168.10.140

255.255.255.255 -type CLIP


Step Action

10. Verify the cluster instance

and node.

NS10_node1> show cluster instance

1) Cluster ID: 1

Dead Interval: 3 secs

Hello Interval: 200 msecs

Preemption: DISABLED

Propagation: ENABLED

Cluster Status: ENABLED(admin),

ENABLED(operational), UP

Member Nodes:

Node ID Node IP Health Admin

State Operational State

------- ------- ------ --------

--- -----------------

1) 1 192.168.10.110* UP PASSIVE

INACTIVE(CCO)

Done

NS10_node1> show cluster node

1) Node ID: 1

IP: 192.168.10.110*

Backplane: 1/1/2

Health: UP

Admin State: PASSIVE

Operational State: INACTIVE(CCO)

Sync State: ENABLED

Done

NS10_node1>


Step Action

11. From the Internal Windows 7 workstation (Win7Client), SSH into the cluster IP (CLIP) using

PUTTY (in our example, 192.168.10.140, currently owned by node1) and add node 2 to the

cluster.

IMPORTANT:

Make sure the cluster commands are performed from the cluster IP; otherwise, they will

not be replicated to other nodes and issues will arise. Even when logging to the NSIP of

the Cluster Coordinator (CCO), the commands will not replicate correctly.

Specify the state as PASSIVE and the backplane interface for node 2.

NOTE: Cluster interface names are identified by a 3-tuple:

NodeID / ControlledID / InterfaceID

NS10_node1> add cluster node 2 192.168.10.120 -state PASSIVE -

backplane 2/1/2

Done

NS10_node1>

12. While logged in via SSH to

the cluster IP (CLIP),

perform the same step for

node 3.

NS10_node1> add cluster node 3 192.168.10.130 -

state PASSIVE -backplane 3/1/2

Done

NS10_node1>


Step Action

13. On the same SSH session,

verify the cluster node

status using the “show

cluster node”

command

Note that node 2 and node

3 should report the status

as unknown since we have

not completed the required

configuration on these

nodes.


1) Node ID: 1

IP: 192.168.10.110*

Backplane: 1/1/2

Health: UP



Sync State: ENABLED

2) Node ID: 2(Adding)

IP: 192.168.10.120

Backplane: 2/1/2

Health: UNKNOWN


Operational State: UNKNOWN

Sync State: UNKNOWN

3) Node ID: 3(Adding)

IP: 192.168.10.130

Backplane: 3/1/2

Health: UNKNOWN


Operational State: UNKNOWN

Sync State: UNKNOWN

DoneNS10_node1>

14. While logged in to the

cluster IP (CLIP), save the

configuration.


15. Next, login to node 2 and

join the cluster. Save the

configuration and reboot.

NS10_node2> join cluster -clip 192.168.10.140

-password nsroot



effect


Done

NS10_node2> reboot -warm

16. Perform the same step on

node 3.

NS10_node3> join cluster -clip 192.168.10.140 -

password nsroot



effect


Done



Step Action

17. After node 2 and node 3

are online, login to the

cluster IP (CLIP) via SSH

using the Windows 7

workstation (Win7Client).

Display the cluster status.

Since all the nodes joined

the cluster in a PASSIVE

state. The Operational

State for all nodes should

be INACTIVE.

Inspect this by issuing the

“show cluster node”

command from the CCO.


1) Node ID: 1

IP: 192.168.10.110*

Backplane: 1/1/2

Health: UP



Sync State: ENABLED

2) Node ID: 2

IP: 192.168.10.120

Backplane: 2/1/2

Health: UP


Operational State: INACTIVE

Sync State: ENABLED

3) Node ID: 3

IP: 192.168.10.130

Backplane: 3/1/2

Health: UP


Operational State: INACTIVE

Sync State: SUCCESS

Done

NS10_node1>

18. Go back to the cluster IP

session (CLIP) and add

spotted SNIPs for each of

the nodes. Make sure you

select the -ownerNode

parameter.

NS10_node1> add ns ip 192.168.10.91

255.255.255.0 -type SNIP -ownerNode 1

Done

NS10_node1> add ns ip 192.168.10.92


Done

NS10_node1> add ns ip 192.168.10.93


Done

NS10_node1>


Step Action

19. From the cluster IP

(CLIP) SSH session,

display the cluster system

IPs.

NOTE: Every node in the

cluster should show all the

IPs in the cluster system.

However, all

configurations should be

performed using the

Cluster IP.

NS10_node1> show ip

20. From the same SSH

session set the node state

on all nodes to ACTIVE.

NS10_node1> set cluster node 1 -state ACTIVE

Done


Done


Done

NS10_node1>

21. Verify that all nodes are

ACTIVE.


1) Node ID: 1

IP: 192.168.10.110*

Backplane: 1/1/2

Health: UP

Admin State: ACTIVE

Operational State: ACTIVE(CCO)

Sync State: ENABLED

2) Node ID: 2

IP: 192.168.10.120

Backplane: 2/1/2

Health: UP

Admin State: ACTIVE

Operational State: ACTIVE

Sync State: ENABLED

3) Node ID: 3

IP: 192.168.10.130

Backplane: 3/1/2

Health: UP

Admin State: ACTIVE

Operational State: ACTIVE

Sync State: SUCCESS

DoneNS10_node1>


Step Action

22. OPTIONAL: If any of

the nodes appear as NOT

UP and they are currently

out-of-sync, proceed to

remove and re-add it to the

cluster.

Login to the problematic

node’s NSIP (in our

example node 3) and

remove the cluster

instance.

Then proceed to re-join

the node to the cluster to

force a full

synchronization.


1) Node ID: 1

IP: 192.168.10.110*

State: ACTIVE

Backplane: 1/1/2

Health: UP

Master State: ACTIVE(CCO)

Sync State: ENABLED

2) Node ID: 2

IP: 192.168.10.120*

State: ACTIVE

Backplane: 2/1/2

Health: UP

Master State: ACTIVE

Sync State: SUCCESS

3) Node ID: 3

IP: 192.168.10.130

State: ACTIVE

Backplane: 3/1/2

Health: NOT UP

Reason(s):

Service state is being

synchronized with the cluster

Master State: INACTIVE

Sync State: ENABLED

Done

NS10_node1>

NS10_node3> rm cluster instance 1

Done

NS10_node3> join cluster -clip 192.168.10.140 -

password nsroot

Warning: The configuration must be saved and the

system rebooted for these settings to take effect


Done


Are you sure you want to restart NetScaler (Y/N)?

[N]:y

NS10_node3>


Step Action

23. From the cluster IP

(CLIP) SSH session, verify

that all IPs defined so far

are ACTIVE

24. Next, we need to define the distribution method for the cluster system. Since this lab

environment is completely virtualized, we will use the “link set” deployment type as this does

not require any router or switch configuration.

First, while logged on to the cluster IP (CLIP), create the link set definition.

Then, bind all the interfaces connected to the link set.

In our example, the VPX appliances will have interface n/1/1 connected to the upstream virtual

switch (where n is the node id). We will select these interfaces in our link set configuration.

NS10_node1> add linkset LS/1

Done

NS10_node1> bind linkset LS/1 -ifnum 1/1/1

Done


Done


Done

NS10_node1>

25. Verify that the link set

binding is correct.

NS10_node1> show linkset LS/1

1) Id: LS/1

Interfaces: 1/1/1 2/1/1 3/1/1

Done

NS10_node1>

26. While logged on to the

cluster IP (CLIP), save the

cluster configuration


27. The cluster is now fully operational.


Summary

Key

Takeaways


New logical and physical design requirements must be considered before

deploying a cluster system. Communication traffic flows between the nodes

in the cluster, clients, and servers have to be carefully planned in order to

avoid issues during implementation.

New entity types are available in a cluster system. Spotted entities are only

active on a single node. Striped entities are available in all nodes participating

in the cluster.

NOTES One example of a spotted entity is the NSIP of each system. Additionally, in our

previous configuration, we defined SNIPs as spotted entities and each node in the

cluster will use this IPs to contact backend servers. Spotted SNIPs are

recommended by Citrix in order to avoid ARP flux issues.

After enabling the cluster instance and joining a node to the cluster setup, the

configuration is lost and all entities have to be redefined. It is very important the

configuration is backed up before proceeding to add a node to the cluster.


Exercise 3: Cluster load balancing

configuration

Overview

In a cluster system, most of the core NetScaler features are supported. Virtual server entities are

configured from the configuration coordinator which is the system that owns the cluster IP. The election

of the configuration coordinator is done automatically by the Quorum Service Provider (cluster) protocol

upon cluster setup. If the configuration coordinator suffers a failure, the QSP protocol will elect a new

CCO.

In this exercise we will configure a load balancing virtual server striped entity. We will explore traffic flow

between members of the cluster and force a CCO failure while serving traffic to illustrate how the system

behaves.






4. Apache_MySQL_1 (GENTOO_1)


6. Internal Windows 7 Workstation: (Win7Client)


Step Action

1. From the Windows 7

workstation (Win7Client),

login to the cluster IP

(CLIP) and enable the load

balancing feature.

NS10_node1> enable ns feature LB

NS10_node1> enable ns mode USNIP


Step Action

2. While connected to the

cluster IP (CLIP),

configure HTTP and

MYSQL server and

services for servers 1 and 2

(Apache_MySQL_1 /

Apache_MySQL_2)

Note: HTTP virtual

servers and services should

be redefined as they are lost

after joining the initial node

to the cluster. Use the

initial configuration to

restore these settings.

NS10_node1> add server GENTOO_1 192.168.10.13

Done

NS10_node1> add server GENTOO_2 192.168.10.14

Done

NS10_node1> add service GENTOO_1_HTTP_TCP_80

GENTOO_1 HTTP 80

Done

NS10_node1> add service GENTOO_2_HTTP_TCP_80

GENTOO_2 HTTP 80

Done

NS10_node1> add service GENTOO_1_MYSQL_TCP_3306

GENTOO_1 MYSQL 3306

Done

NS10_node1> add service GENTOO_2_MYSQL_TCP_3306

GENTOO_2 MYSQL 3306

Done

3. From the same SSH session

to the cluster IP (CLIP),

create a new virtual server

for HTTP and MYSQL.

Note: Make sure the

configuration commands

are executed while

connected to the cluster IP

(CLIP). Even when

connecting to the CCO

NSIP, the commands are

not replicated to all

members in the cluster. If

by mistake, entities are

created locally, remove

them and reattempt to add

them using the cluster IP

(CLIP).

NS10_node1> add lb vserver HTTP_vserver_cluster

HTTP 192.168.10.18 80

Done

NS10_node1> bind lb vserver HTTP_vserver_cluster

GENTOO_1_HTTP_TCP_80

Done

NS10_node1> bind lb vserver HTTP_vserver_cluster

GENTOO_2_HTTP_TCP_80

Done

NS10_node1>

NS10_node1> add lb vserver MYSQL_vserver_cluster

MYSQL 192.168.10.18 3306

Done

NS10_node1> bind lb vserver

MYSQL_vserver_cluster GENTOO_1_MYSQL_TCP_3306

Done

NS10_node1> bind lb vserver

MYSQL_vserver_cluster GENTOO_2_MYSQL_TCP_3306

Done

NS10_node1>


Step Action

4. Continue on the SSH

session to the cluster IP

(CLIP). Verify that virtual

server definitions and

bindings were replicated to

other nodes. Login to each

node’s NSIP and execute a

“show lb vserver”

5. The cluster system uses a

distributed monitoring

infrastructure. One node in

the cluster is elected as the

monitoring owner and it is

responsible to notify the

CCO of the service health

status. In turn, the CCO

notifies the rest of the

nodes in the cluster.

To verify which node is the

monitoring owner, inspect

the output of the “show service <service_name>”

command from the SSH

session to the cluster IP

(CLIP).

The output from your

environment could be

different as a new

monitoring owner can be

selected.


Step Action

6. Now, we will trace the traffic path when generating a GET request from the Windows 7

workstation (Win7Client).

For simplicity, bind a simple PING monitor. The purpose of this exercise is to avoid cluttering the

TCPDUMP output with probe traffic.

Important: Make sure you perform all configuration operations from the SSH session to

the cluster IP (CLIP).

NS10_node1> bind service GENTOO_1_HTTP_TCP_80 -monitorName PING

Done

NS10_node1> bind service GENTOO_2_HTTP_TCP_80 -monitorName PING

Done

7. Go back to each node SSH

session and exit to the shell.

Then, start a new

TCPDUMP on each node

using the capture filter

provided.

# shell

# nstcpdump.sh -ne arp | grep "192.168.10.18"

8. From the Windows


open the browser and

navigate to the VIP

previously configured

(192.168.10.18). Observe

the TCPDUMP output on

each node.


Step Action

9. When configuring a “Link Set”, the cluster defines an ARP owner for every striped IP configured.

One of the connected nodes is responsible for replying to ARP requests for a VIP. Upon receiving

a request, the ARP owner returns its own MAC address and receives the initial traffic flow.

Inspect the TCPDUMP running on each of the nodes to find out who is the ARP owner.

In our example, NS10_node1 (CCO) is the ARP owner.

-----------------

10. Verify who owns MAC

address given the in the

ARP reply.

From the cluster IP (CLIP)

SSH session, look at the

output of the “show

interface” command.

Which interface owns it?

11. Next, we will verify the traffic flow between client, server, and nodes in the cluster.

From each of the SSH sessions to each node’s NSIP, stop the previous capture and start a new

TCPDUMP dump.

This time, we will use a more specific filter to capture packets destined to the VIP or any of the

backend servers on port 80 as well as ARP traffic.

Additionally, using the –e flag, the output will display the corresponding source and destination

MAC addresses.

# shell

# nstcpdump.sh -ne "((host 192.168.10.18 or host 192.168.10.13 or

host 192.168.10.14) and tcp port 80) or arp"


Step Action

12. From the Win7Client

workstation, refresh the

page.


Step Action

13. Observe the TCPDUMP output on each node. Which node receives the initial flow? Which node

contacts the backend server? In a link set configuration, the traffic will flow from the

CLIENT_IP to the VIP through the MAC owner. Then, the cluster will select the node that will

process the request (flow processor) and forwards the request to this node using the backplane

interface. Next, the flow processor contacts the backend server and fetches the response. Through

the same backplane interface, this is returned to the ARP owner who in turn sends it back to the

CLIENT_IP.

Note: During your testing, the traffic flow might be different depending on which node is selected

as the flow receiver/processor. Inspect the source/destination MAC addresses to understand how

traffic is flowing in your lab environment.

Node1:

Node2:

Node3:


Step Action

14. For node-to-node communication, the backplane interface is utilized and a shared MAC address is

used to identify steered traffic.

This MAC has the following format:

02:00:6F:<cluster_id>:<node_id>:<flags>

In the previous TCPDUMP output used in step 11, inspect the source and destination MAC

addresses used between nodes.

15. OPTIONAL: To further illustrate the traffic flow, we will send sequential GET requests to the

VIP and observe the behavior.

From Apache_MySQL_3 (GENTOO_3), start the wbox utility to send 1 request per second to

the VIP.

gentoo_3 ~ # wbox 192.168.10.18

WBOX 192.168.10.18 (192.168.10.18) port 80

0. 200 OK 318 bytes 2 ms

1. 200 OK 318 bytes 2 ms

From the Windows 7 workstation (Win7Client), open an SSH session to each node’s NSIP and

start a TCPDUMP using the same capture filter provided below (same as in step 14).

# nstcpdump.sh -ne "((host 192.168.10.18 or host 192.168.10.13 or host

192.168.10.14) and tcp port 80) or arp"

Observe the traffic flow for the incoming requests.

1. Is the ARP owner always the same? Why?

2. Is the flow processor always the same?



Step Action

16. Now, we will simulate a

node failure by rebooting

one of the nodes in the

cluster.

While leaving the wbox

utility running to generate

constant HTTP probes

(refer to the previous step),

reboot one of the nodes in

the cluster.


Are you sure you want to restart NetScaler

(Y/N)? [N]:y

Done

NS10_node3>

17. When the node selected for

reboot goes offline, a small

delay might be perceived

from the client while

connections are redirected

to other nodes.

Inspect the probe output.

Was there any HTTP

request lost during your

testing?

gentoo_3 ~ # wbox 192.168.10.18

...

65. 200 OK 318 bytes 2 ms

66. 200 OK 318 bytes 3 ms

67. 200 OK 318 bytes 3 ms

68. 200 OK 318 bytes 2 ms

69. 200 OK 318 bytes 3 ms

70. 200 OK 318 bytes 3 ms

71. 200 OK 318 bytes 3295 ms

72. 200 OK 318 bytes 2 ms

73. 200 OK 318 bytes 2 ms

74. 200 OK 318 bytes 1 ms

75. 200 OK 318 bytes 1 ms

18. After node 3 comes back online, it rejoins the cluster automatically and connections start to flow

through this node once again.

From the Windows 7 workstation (Win7Client), go back to the SSH session to the cluster IP

(CLIP) and verify the output of the “show cluster node”.

What is the status of the node rebooted? Did it re-join the cluster?


Step Action

19. From the same workstation, go back to the SSH session to node3. Stop any trace running and start

a new TCPDUMP to inspect traffic flow. Use the command provided:

# nstcpdump.sh -ne "((host 192.168.10.18 or host 192.168.10.13 or host

192.168.10.14) and tcp port 80) or arp"

Is there any traffic flowing through this node?

See the example below:


Step Action

20. Now, to simulate a

Configuration Coordinator

node failure, we will reboot

the CCO node while

sending HTTP probes.

From the SSH session to

the cluster IP (CLIP),

reboot the CCO while

running the wbox utility

from server 3

(GENTOO_3).

The cluster system will

immediately start the CCO

election process and select

a new configuration

coordinator (CCO). A small

delay might be perceived

from the client while this is

in progress.

Note: Since our cluster

deployment uses a link set,

one of the nodes is the

ARP owner. On step 12 of

this particular example, we

found out that node1, (the

rebooted node) was the

ARP owner. Therefore, a

new node is selected for

this function.

From the CCO node execute:

NS10_node1> reboot –warm

Are you sure you want to restart NetScaler

(Y/N)? [N]:y

Done

NS10_node1>

From Server 3 (GENTOO_3), observe the wbox

output:

395. 200 OK 318 bytes 2 ms

396. 200 OK 318 bytes 2 ms

397. 200 OK 318 bytes 3 ms

398. 200 OK 318 bytes 3 ms

399. 200 OK 318 bytes 2 ms

400. 200 OK 318 bytes 7012 ms

401. 200 OK 318 bytes 2 ms

402. 200 OK (319) bytes 4 ms

403. 200 OK (318) bytes 2 ms

404. 200 OK 318 bytes 3 ms

405. 200 OK 318 bytes 2 ms

406. 200 OK 318 bytes 3 ms

407. 200 OK 318 bytes 2 ms

408. 200 OK 318 bytes 2 ms


Step Action



reconnect to cluster IP

while the previous node is

rebooting.

Execute the “show cluster

node” command.

Was the connection

successful? Was a new

CCO elected?

22. So far, our HTTP vserver had no persistency settings and the LEASTCONNECTIONS method

was used by default. Now, we will explore other load balancing algorithms and how the persistence

state is replicated in a cluster system.

From the SSH session to the cluster IP (CLIP), change the LB method to Round Robin and select

source IP persistence.

NS10_node2> set lb vserver HTTP_vserver_cluster -lbMethod ROUNDROBIN

-persistenceType SOURCEIP

Done

NS10_node2>


Step Action

23. Go back to the SSH session

to Apache_MySQL_3

(GENTOO_3). Using our

probing tool, send HTTP

requests to the VIP in

order to generate constant

traffic.

Note: If you followed the

previous steps, the wbox

utility should be running.

Inspect this is the case.

gentoo_3 ~ # wbox 192.168.10.18

WBOX 192.168.10.18 (192.168.10.18) port 80

0. 200 OK 318 bytes 1 ms

1. 200 OK 318 bytes 1 ms

24. Go back to the SSH session to the cluster IP (CLIP) on the Windows 7 workstation (Win7Client).

Inspect the persistence table by executing the “show persistentSessions” command.

> show persistentSessions

Perform the same command on the other two nodes. Are the persistence entries the same on all

nodes in the cluster?

25. From the SSH session to the cluster IP (CLIP), look at the timeout for the persistence entry

created. Is the timeout value the same across all nodes?

26. This concludes with our load balancing configuration. Next, we will look into the new packet

capture options available in a cluster system.


Summary

Key

Takeaways


A NetScaler cluster system provides a scalable and fault tolerant solution. If

any of the components fail, another member of the cluster will handle the

load or roles assigned.

Load balancing persistence is accomplished by shared hashed tables across

all cluster nodes. Even though traffic does not flow through the same node

for every incoming flow, the same destination server is selected.

NOTES NetScaler clustering uses an extension of the existing persistence algorithms

available in nCore technology. This extension applies to independent systems that

process traffic independently in the same fashion as packet engines.


Exercise 4: Packet capture analysis in a

cluster system

Overview

In a cluster system, traffic flows can be complex depending on the deployment type and environment. In

order to help troubleshoot networking issues, additional markings are included in captured packets when

traces are taken in native format. This feature was already available in previous releases; however, with the

introduction of clustering, additional markings are needed to aid during the packet capture analysis. In this

exercise, we will capture a trace in native format and examine such markings.

Keep in mind that a special version of a protocol analyzer is required as it needs to understand the native

NetScaler format. You must download the latest WireShark version which already includes such changes.

For this exercise, the required application version was already preloaded to the workstation. No additional

download is required.









7. Windows 7 Workstation: (Win7Client)


Step Action


Step Action

1. From the Windows 7 workstation (Win7Client), login to the cluster IP (CLIP) GUI using any browser

installed on the workstation.

2. Navigate to Configuration -> System -> Diagnostics -> Technical Support Tools -> Start new trace


Step Action

3. Verify no TCPDUMP sessions are running individually on each node. Stop them if necessary. Then,

proceed to start a new trace with the following settings:

Packet size = 0

Trace file format = nstrace

Leave other settings default. Click on start.


Step Action

4. From the same workstation (Win7Client), open a different browser and navigate to the HTTP virtual

server in order to generate some traffic to be captured.

Since there is a web application pre-installed on all servers bound to the VIP (Apache_MySQL_1 and

Apache_MySQL_2), use the URL below to access it:

http://192.168.10.18/phpmyadmin

Login as:

U: root

P: Citrix123



Step Action

5. After login in to the web application, go back to the NetScaler trace utility and stop the trace. After

stopping the running trace, click on the download button to open the file browser:

6. Whenever a trace is captured, a new directory will be created containing the trace files for each node.

Select the corresponding folder with the correct timestamp and click on the zip utility to compress the

archive and then hit select.


Step Action

7. Download the compressed archive to the administrator’s account desktop:

8. Close the trace utility. Proceed to extract the

contents of the compressed archive to the

workstation’s desktop.

9. Inspect the contents of the new directory. You should see a subfolder for every node in the cluster.

Open any of the trace files and filter by HTTP.


Step Action

10. Expand any of the frames

and inspect the NetScaler

Packet Trace layer. Note the

additional markings

included in each packet:

Some of these are:

Source Node

Destination Node

Cluster Flags

11. In our example, frame 3267 was received by node 1 and steered to node 2 for further processing. Look

into the source and destination MAC addresses to verify this. From the same workstation (INT-Win_7),

open an SSH session and login to the cluster IP (CLIP) and compare to the output of the “show

interface” command:


Step Action

12. As you can see, the source

MAC address is the virtual

MAC used by all nodes in

the cluster.

In order to get the correct

node to process traffic, the

packet must be “steered”

to the corresponding owner

node (in this case node2).

The receiving node (Flow

Receiver) forwards the

packet to the destination

node (Flow Processor) to

serve the request.


Step Action

13. The elegance of the cluster architecture relies around an active/active configuration. This means that

any node in the traffic path can receive traffic. This complicates things a bit as each trace will have

chunks of the relevant traffic for a particular request and it will be very hard to troubleshoot networking

related issues while having multiple trace files.

To solve this problem, NetScaler 10 offers a new command line utility to merge all trace files into a

single capture.

The nstracemerge.sh utility is available from the shell:

14. Let’s merge the previously captured trace files. From the Windows 7 workstation (Win7Client), login to

the cluster IP (CLIP) using SSH. Access the shell prompt and navigate to the directory where the trace

was captured:

/var/nstrace/DDMonthYYYY_hh_mm_ss

NOTE: The folder is labeled using a current timestamp format. See example below.

> shell

# cd /var/nstrace/19Sep2012_03_16_59/


Step Action

15. From the SSH session, invoke the nstracemerge.sh utility with the default parameters. This will use

the current directory as source and destination for the input/output files. The result will be a single

trace file named nstrace0.cap with the contents of all individual traces:

root@NS10_node2# pwd

/var/nstrace/17Apr2012_21_55_47

root@NS10_node2# ls -alth

drwxr-xr-x 5 root wheel 512B Apr 18 01:38 .

drwxr-xr-x 4 root wheel 512B Apr 17 22:42 ..

drwxr-xr-x 2 root wheel 512B Apr 17 21:56 node2



root@NS10_node2# nstracemerge.sh

nstrace merge

source directory : .

destination directory : .

output file name : nstrace

current working directory [.] :/var/nstrace/17Apr2012_21_55_47

Files in source directory:

./node1/nstrace1_node1.cap



root@NS10_node2# ls -alth

total 2378

drwxr-xr-x 5 root wheel 512B Apr 18 01:46 .

-rwx------ 1 root wheel 2.3M Apr 18 01:46 nstrace0.cap

drwxr-xr-x 4 root wheel 512B Apr 17 22:42 ..




root@NS10_node2#


Step Action

16. From the Windows 7 workstation (Win7Client), use WinSCP to copy the merged trace file to the

desktop:

17. From the Windows 7 workstation (Win7Client), open the file. Verify the size. Is the trace larger that the

individual pieces? Does it contain more packets?

18. In WireShark, specify a display filter for the VIP AND HTTP traffic. Use the filter provided below:

ip.addr==192.168.10.18 && http


Step Action

19. Continue on WireShark and select the first packet. In the packet details, expand the “NetScaler Packet

Trace” sub-node. Start from the first packet displayed and continue to highlight the next. Observe the

Source / Destination node fields as well as the Flow Processor / Flow Receiver flags.

Are all packets processed by the same node?


Step Action

20. Additional display filters have been introduced in the new WireShark release. Now, we have the ability

to filter packets based on cluster flags, source/destination node, etc…

In order to find out which packets were received/processed/steered by a certain cluster node, we can

leverage some of the following filters:

nstrace.snode

nstrace.dnode

nstrace.flags

nstrace.flags.rssh

nstrace.flags.srss

nstrace.flags.dfd

nstrace.flags.fr

nstrace.flags.fp

For example, to find the packets sent by the client and received by node1, we can use the following

filter:

ip.addr==192.168.10.18 && ip.addr==192.168.10.233 && nstrace.snode==-1 &&

nstrace.dnode==1

In WireShark, apply the filter provided to the merged trace. Were there any packets received by node1?


Step Action

21. Change the destination node filter to either node 2 or 3. Was any packet received by either of these

nodes?

Why / Why not?

NOTE: Since we are using a linkset configuration, a node is selected to be the ARP owner. This

means that all incoming packets for a single VIP will be received by this node. If you select a different

destination node in the filter, only 1 of the 3 nodes in the cluster should receive incoming packets.

22. This concludes this exercise. Next we will explore other traffic distribution methods by extending our

lab environment to a remote network and configure ECMP.


Summary

Key

Takeaways


Traffic flowing through a cluster system can add additional complexity when

troubleshooting any network issue. NetScaler 10 introduces new options

when capturing network traces in a cluster system to identify the nodes

processing traffic; thus simplifying the analysis process.

These additional packet markings are only available when capturing a trace in

native format and a special version of a protocol analyzer is required to

analyze such captures. Citrix has submitted the changes to the Wireshark

organization and the latest WireShark build contains the necessary changes

to parse such captures.

An additional utility is available in NetScaler 10 to merge all trace files

generated by each node. This simplifies the analysis process.

NOTES Administrators familiar with NetScaler traces should find the new options familiar as

they extend the existing tracing capabilities of a NetScaler appliance.


Exercise 5: ECMP configuration

Overview

In a previous exercise, we used a linkset configuration as the traffic distribution mechanism. A linkset

configuration is easily implemented as it does not require any changes on the upstream device making it

suitable for deployment environments where administrative control of all network devices is not possible.

This distribution method has one caveat:

Traffic destined to a single VIP will only pass through a single node selected as the MAC address owner,

potentially creating a bottleneck if a large amount of traffic surges to the such IP.

In addition to linksets, other traffic distribution methods are available to mitigate this limitation:

- ECMP: Equal Cost Multipath Routing

- CLAG: Cluster Link Aggregation

In this lab, we will implement an ECMP configuration. For this we will use OSPF as the dynamic routing

protocol to advertise the VIP host routes to an intermediate router in order to provide access to a remote

network to the virtual servers hosted by the cluster. The following diagram will be used during this setup:

Remote Network

Internal Router

Public Network

EXT-Win_7

Internal network

INT-Win_7

NS10_node1

NS10_node3

NS10_node2

192.168.10.X/24

DNS server

172.16.2.0/24

Apache2

Apache3

Apache1

172.16.1.0/24

.11 .12

.110

.120

.15

.130

.1 .1

.14

.13

.2









6. Internal Windows 7 workstation: Win7Client

7. Remote network Windows 7 workstation: EXT-Win_7

8. Vyatta Internal Router: vyatta-RTR


Step Action

1. The 3-node cluster created in a previous exercise should be fully operational. However, it is still

uses a LinkSet configuration.

Since we are going to be configuring a fully connected ECMP setup, from the Windows 7

workstation (Win7Client), establish an SSH session to the cluster IP (CLIP) and remove the

LinkSet definition.

NS10_node2> rm linkset LS/1

Done


Step Action

2. To start with our ECMP

configuration, we must

establish connectivity to the

remote network from the

NetScaler cluster going

through the internal Vyatta

router.

In the next step, we will

create the necessary

interfaces to establish

connectivity. Check the

diagram provided to get

familiar with the logical

topology.

After you have reviewed the

topology diagram, continue

to the next step.

Remote Network

Internal Router

EXT-Win_7

NS10_node1

NS10_node3

NS10_node2

172.16.2.0/24

172.16.1.0/24

.101

.102

.103

.1

.1

.2


Step Action

3. Using XenCenter, add a new interface to Node 1, 2, 3. Bind this interface to the To_172-16-1-0

network on each node.


Step Action

4. All three NetScaler cluster

nodes should have the

following Networks attached.

The new network bound will

provide us the necessary

connectivity to the remote

172.16.2.0/24 subnet.



SSH into the cluster IP

(CLIP) and enable OSPF.

NS10_node2> enable ns feature OSPFRouting

Done

NS10_node2>


Step Action

6. Next, we will create a new

subnet IPs on the new

network space

(172.16.1.0/24). We will turn

on dynamic routing on such

IP addresses in order to “peer

up” with the Vyatta router to

form an OSPF adjacency.

From the SSH session to the

cluster IP (CLIP), create 3

different SNIPs assigned to

each node. Make sure you turn

on the dynamic routing

option.

Use the following IPs:

172.16.2.11 – node1

172.16.2.12 – node2

172.16.2.13 – node3

NS10_node2> add ns ip 172.16.1.11 255.255.255.0

-ownerNode 1 -dynamicRouting ENABLED

NS10_node2> add ns ip 172.16.1.12 255.255.255.0


NS10_node2> add ns ip 172.16.1.13 255.255.255.0



Step Action

7. From each node SSH session (to the NSIP), return to the NetScaler CLI and verify connectivity

with the vyatta router eth1 interface (172.16.1.1) using PING.

NOTE: Since we manipulated the network interfaces on the NetScaler VPX instances while it was

running, it is possible that ICMP replies are not received. If this is the case, please reboot each of

the cluster nodes as well as the Vyatta router VM. Retry the connectivity test.

8. From the Windows 7 workstation (Win7Client), return to the SSH session to the cluster IP (CLIP)

and proceed to enable the dynamic routing options on the VIP (created in a previous exercise) to

inject the host route into OSPF.

Additionally, set the “Vserver Route Health Injection level” to ONE_VSERVER to ensure the

route is taken off the routing table if any virtual servers on this VIP go down.

NS10_node2> set ns ip 192.168.10.18 -hostRoute ENABLED -vserverRHILev

ONE_VSERVER


Step Action

9. Next, we will configure the

intermediate router to run

OSPF and setup an adjacency

with the NetScalers in the

cluster.

Login to the vyatta router

console (vyatta-RTR) using

the default username and

password (vyatta/vyatta).

Switch to configuraton mode

and start the OSPF instance

on area 0.

Proceed to add all the new

subnet IPs (created on the

previous step) as OSPF

neighbors with the same

priority and polling interval.

Enter configuration mode:

vyatta@vyatta:~$ configure

Once in configuration mode, enable OSPF and

configure the NetScaler nodes as neighbors:

vyatta@vyatta# set protocols ospf area 0

network 172.16.1.0/24

vyatta@vyatta# set protocols ospf neighbor

172.16.1.11 poll-interval 60


172.16.1.11 priority 0




172.16.1.12 priority 0




172.16.1.13 priority 0

10. Commit and save the

configuration for the changes

to be active.

vyatta@vyatta# commit

vyatta@vyatta# save


Step Action

11. The next step is to configure

the OSPF instance on the

NetScaler appliances.

NetScaler uses the ZebOS

protocol suite to manage the

dynamic routing protocol

configuration.

From the Windows 7

workstation (Win7Client), go

back to the SSH session to

cluster IP (CLIP) and access

the ZebOS shell.

NS10_node2> vtysh

12. Access the ZebOS

configuration mode, and start

the OSPF instance. Add the

network adjacent to the vyatta

router on area 0.

NS10_node2# conf t

NS10_node2(config)# router ospf

NS10_node2(config-router)# network 172.16.1.0

0.0.0.255 area 0

13. From the ZebOS shell, display the routing table to ensure the host route is being marked as a kernel

route. The VIP/32 route should be displayed.

NS10_node2(config-router)#exit

NS10_node2(config)#exit

NS10_node2#show ip route

http://www.ipinfusion.com/products/advanced/products_advanced.html

http://www.ipinfusion.com/products/advanced/products_advanced.html


Step Action

14. Next, show the OSPF neighbors to ensure the proper adjacencies were established. The vyatta

router as well as the other two NetScalers in the cluster should show up as neighbors.

NS10_node2#show ip ospf neighbor

15. Verify the same adjacencies on the vyatta router. From the vyatta router console, display the OSPF

neighbors.

NOTE: In order for the show commands to work correctly, make sure you exit out of

configuration mode.

vyatta@vyatta# exit

exit

vyatta@vyatta:~$ show ip ospf neighbor


Step Action

16. We are now ready to

redistribute the host routes

into OSPF.

The idea is for the NetScaler

to advertise these routes in

order for the Vyatta router to

include them in its routing

table. This way, hosts on the

remote network will be able to

access the striped VIP through

the advertised route.

From the Windows 7


back to the cluster IP (CLIP)

SSH session. The ZebOS shell

should be available.

Redistribute the kernel routes

into the OSPF instance.

node0#conf t

node0(config)#router ospf

node0(config-router)#redistribute kernel

node0(config-router)#exit

node0(config)#exit

node0#exit

NS10_node1>

17. Go back to the vyatta router console and display the routing table.

Was the host route injected?

Are there multiple routes?

Do they have the same cost?

Does it show the proper next hop IP?

vyatta@vyatta:~$ show ip route


Step Action


workstation at the remote

network (EXT-Win_7),

attempt to PING or access the

virtual server (192.168.10.18)

via HTTP.

Does the VIP respond with

ICMP replies? Why not?

19. At this point, the vyatta router has a specific route(s) to reach the virtual server and traffic is being

forwarded to one of the nodes in the cluster; however, the NetScaler cluster does not know how to

route back to the remote network.

Remote Network

Internal Router

EXT-Win_7

NS10_node1

NS10_node3

NS10_node2

172.16.2.0/24

172.16.1.0/24

.101

.102

.103

.1 .1

.2

HTTP vserver192.168.10.18

MySQL vserver192.168.10.18

ICMP echo request

Destination unreachable


Step Action

20. From the internal network windows workstation (Win7Client), go back to the cluster (CLIP) SSH

session and display the routing table.

Is there a route to the remote network (172.16.2.X/24)? Why not?

NS10_node2> show route

21. To get the correct return route into the NetScaler cluster routing table, we have two options.

Install a static route or use OSPF and have the router to redistribute its routes.

Since there is an OSPF adjacency already established, we can configure the vyatta router to

redistribute the connected routes.

Login to the vyatta router console, and execute the command provided below for the router to

advertise its routes:

vyatta@vyatta$ configure

vyatta@vyatta# set protocols ospf redistribute connected

vyatta@vyatta# commit

vyatta@vyatta# save

vyatta@vyatta# exit

22. Go back to the cluster IP (CLIP) SSH session and inspect the NetScaler cluster routing table. Why

is the route still NOT available?


Step Action

23. In order to include any advertised routes via OSPF into the NetScaler routing table, we need to

instruct the underlying routing layer (ZebOS) to make these routes available to the NetScaler kernel.

Review the diagram below and then continue with the next step.

Internal Router

ZebOS routing table

NS10_node2#show ip route

Codes: K - kernel, C - connected, S - static, R - RIP, B - BGP

O - OSPF, IA - OSPF inter area

N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2

E1 - OSPF external type 1, E2 - OSPF external type 2

i - IS-IS, L1 - IS-IS level-1, L2 - IS-IS level-2

ia - IS-IS inter area, I - Intranet

* - candidate default

C 127.0.0.0/8 is directly connected, lo0

C 172.16.1.0/24 is directly connected, vlan0

O E2 172.16.2.0/24 [110/20] via 172.16.1.1, vlan0, 00:05:32


K 192.168.10.18/32 via 0.0.0.0


NetScaler Routing Table


Network Netmask Gateway/OwnedIP State Type

------- ------- --------------- ----- ----

1) 192.168.10.0 255.255.255.0 192.168.10.120 UP DIRECT

2) 172.16.1.0 255.255.255.0 172.16.1.11 UP DIRECT

3) 0.0.0.0 0.0.0.0 192.168.10.1 UP STATIC

4) 127.0.0.0 255.0.0.0 127.0.0.1 UP PERMANENT

NS10_node1

NS10_node3

NS10_node2

Vyatta Router Routing Table


Codes: K - kernel route, C - connected, S - static, R - RIP, O - OSPF,

I - ISIS, B - BGP, > - selected route, * - FIB route

C>* 127.0.0.0/8 is directly connected, lo

O 172.16.1.0/24 [110/10] is directly connected, eth1, 02:44:30

C>* 172.16.1.0/24 is directly connected, eth1

C>* 172.16.2.0/24 is directly connected, eth0

O>* 192.168.10.18/32 [110/20] via 172.16.1.11, eth1, 02:32:04

* via 172.16.1.13, eth1, 02:32:04

* via 172.16.1.12, eth1, 02:32:04

ADVE

RTIS

E RO

UTE

S

INST

ALL

RO

UTE

S



back to the cluster IP (CLIP)

SSH session and access the

ZebOS shell.

Enter configuration mode and

execute the command provided.

NS10_node1> vtysh

node0#conf t

node0(config)#ns route-install ospf

node0(config)#exit

node0#exit


Step Action

25. From the same SSH session and while on the NetScaler CLI, display the routing table. The OSPF

route should be installed.


26. From the remote network

Windows 7 workstation (EXT-

Win_7), attempt to PING the

VIP.

Did you receive an ECHO

reply?

27. Attempt to access the VIP over

HTTP. On the same

workstation, open a browser and

navigate to the VIP. Can you

access the default site?


Step Action

28. Next, we will verify that all machines in the cluster are receiving traffic for the striped VIP. From

the Windows 7 workstation (Win7Client), SSH into the NetScaler IP of each node, and start a

TCPDUMP to capture packets from the remote workstation’s IP and destined to TCP port 80.

Use the filter provided.

> shell

# nstcpdump.sh host 172.16.2.2 and tcp port 80

29. From the remote Windows 7

workstation (EXT-Win_7), open

a browser and connect to the

VIP.

30. Go back to the TCPDUMP started on step 27. Are multiple nodes receiving traffic?


Step Action

31. From the remote network Windows 7

workstation and login to the

“phpmyadmin” web application hosted

through the VIP.


U: root

P: Citrix123

Was the login successful?

32. Go back to the SSH sessions running on the Internal Windows 7 workstation (Win7Client), are

multiple nodes receiving traffic?



Step Action

33. Since we have defined source IP persistence on the HTTP virtual server configuration, all the

requests from the same workstation should be directed to the same backend server.

From (Win7Client), go back to the cluster IP (CLIP) SSH session and display the statistics on the

HTTP virtual server with the command provided below. Record the number of hits to each of the

services bound.

NS10_node2> stat lb vserver HTTP_vserver_cluster

34. On the remote network Windows 7 workstation (EXT-Win_7), navigate through the webapp to

generate some traffic.


Step Action

35. Go back to the internal Windows 7 workstation (Win7Client). From the cluster IP (CLIP) SSH

session, display the virtual server statistics. Compare the number of hits to each of the services

bound.

Are both services receiving traffic?


Step Action

36. To simulate a node failure, from XenCenter,

force shutdown one of the nodes in the cluster.

In our example, we will use node 3.

37. Go back to the vyatta router console. Display the routing table using the “show ip route”

command.

NOTE: Remember to exit out of configuration mode as the “show ip route” command would

not work. (The prompt should display a $ sign as this indicates we are in operational and not in

configuration mode).

Was the correct host route for the node selected removed?



Step Action

38. Go back to the remote network Windows 7 workstation (EXT-Win_7) and attempt to access the

HTTP VIP.


Can you still browse to it?

Since the vyatta router has two more host routes to the VIP, connectivity to the other two nodes is

still possible and the VIP should continue serving traffic. In this example, it is the upstream device

the one responsible for distributing traffic to each node in the cluster based on the L3 information

contained in the routing table.

39. This is the end of the exercise.

40. IMPORTANT! This is the last exercise for this lab. Please go back to the Self-paced Learning

Lab site and do the following before you go:

Provide us feedback by selecting the “Take web survey…” link.

After completing the survey, click the “Logout” link towards the bottom right to close out

your lab session.



Summary

Key

Takeaways


A NetScaler cluster can use multiple distribution mechanisms to distribute

traffic across all active nodes in the cluster. ECMP provides the best

distribution as each node will receive traffic directly from the upstream

device. State sharing between all nodes in the cluster is crucial for the cluster

to support additional features (persistence, session reuse, etc…)

Load balancing persistence is accomplished by shared hashed tables across

all cluster nodes. Even though traffic does not flow through the same node

for every incoming flow, the same destination server is selected.

NOTES An Equal Cost Multipath Routing configuration is dependent upon the upstream

device configuration. It relies on dynamic routing protocols and support on other

network devices to distribute traffic to all cluster nodes homogeneously.

Please complete this survey

We value your feedback! Please take a moment to let us know about your self-paced

lab experience by completing a brief survey on the web portal before you logout.


Revision History

Revision Change Description Updated By Date

1.0 Original Version WW Technical Readiness October 2012

About Citrix

Citrix Systems, Inc. designs, develops and markets technology solutions that enable information technology (IT)

services. The Enterprise division and the Online Services division constitute its two segments. Its revenues are

derived from sales of Enterprise division products, which include its Desktop Solutions, Datacenter and Cloud

Solutions, Cloud-based Data Solutions and related technical services and from its Online Services division's Web

collaboration, remote access and support services. It markets and licenses its products directly to enterprise

customers, over the Web, and through systems integrators (Sis) in addition to indirectly through value-added

resellers (VARs), value-added distributors (VADs) and original equipment manufacturers (OEMs). In July 2012, the

Company acquired Bytemobile, provider of data and video optimization solutions for mobile network operators.

http://www.citrix.com

© 2012 Citrix Systems, Inc. All rights reserved.

http://www.citrix.com/

Leveraging Netscaler 10 Clusters to Improve Application Performance

Documents

Transcript of Leveraging Netscaler 10 Clusters to Improve Application Performance