CSE 473S (Fall 2009)The Open Network Lab

(Part 1)Ken Wong

Applied Research LaboratoryComputer Science and Engineering Department

http://www.arl.wustl.edu/~kenwk @ l l d

(Part 1)

kenw@arl.wustl.eduhttp://www.onl.wustl.edu (ONL)

National Science Foundation ANI-023826,CNS-0551651, REL-0632580

Testbed Organization

Internet TestbedController

Userwith GUI endhosts Configuration

Switch

●●●

YOUonlusrlogins login host

Switch

Routers

2 - Ken Wong– 10/14/2009

Network Configuration

Queue Length

Bandwidth Usage

Network Processor Routers (NPRs)

6 Pairs of NPRs

3 - Ken Wong– 10/14/2009

100+ endhosts

Why Build ONL?Hands-on experience with real hardware»Make education more concrete reinforce concepts»Achieve through virtualization

N d t t t f ti f th nNeed to protect user from actions of other usersnNeed to mimic propagation delay and link capacity

Support experiment/observation approach»Extensive monitoring facility»Observe effects of config. changes on network traffic

n insights and greater understanding through observations

4 - Ken Wong– 10/14/2009

»Easy-to-use Remote Laboratory Interface (RLI)

Access to advanced router features»Gbps links, filters, packet scheduling»Plugins: Program insertion along packet data path

Get An Account

onl.arl.wustl.edu,onl.wustl.edu,

www.onl.wustl.eduwww.onl.wustl.edu

tutorial

5 - Ken Wong– 10/14/2009

get an account

•Group: onluser•24-hour confirmation•1 password for web & hosts

Properties of ONL AccountsSame password for Web login and host loginHost account restrictions»Can only SSH to onl.arl.wustl.edu from remote host

nActually end up on onlusr, the ONL user host

»Firewall blocks all connections from within ONLnCan’t push from ONL host to remote hostnCan’t pull to ONL host from remote host nBut can push to ONL host from remote host (e.g., scp)nBut can pull from ONL host from remote hostN il

6 - Ken Wong– 10/14/2009

nNo email

»Can only access onlusr host and hosts assigned to your experiment

Password-free SSH between ONL hosts» See “NPR Tutorial Summary Information Password-Free SSH”

Getting StartedSee sidebar at www.onl.wustl.eduFollow instructions in “Getting Started” linkRead NPR Tutorial»Packet Processing»The Remote Laboratory Interface»Filters, Queues and Bandwidth

Other useful tutorial sections»Examples The Remote Laboratory Interface

7 - Ken Wong– 10/14/2009

»Examples Filters, Queues and Bandwidth

SSH Tunnel ConfigurationBuild before each experimental session»Allows your RLI to communicate with ONL daemon»Needed to make reservation or run experimentCookbook for 3 approachesCookbook for 3 approaches»Unix or Unix-like (e.g., cygwin)

nssh -L 7070:onlsrv:7070 onl.arl.wustl.edun I use this (defined as an alias) from Linux

»MS Windows PuTTY (GUI-based)n I use this from my Microsoft XP laptop

And also the SSH agent pageant (optional)

8 - Ken Wong– 10/14/2009

nSee Tutorial for URL (free software)»Microsoft Windows SSH client tool from SSH CorpFollow instructions on ONL web pages»Don’t agonize over it See TA after 2-3 tries»If problems, send email to your grader/consultant

PuTTY SSH Tunnel Configurationexternal host name =

onl.arl.wustl.edu

9 - Ken Wong– 10/14/2009

local port 7070remote host onlsrvremote port 7070

session name = rli

Under the SSH Tunnel Hood

your host(l lh )

controlsvirtualization

NPR

RLI ONLdaemon

port 7070

port 7070

po t 22

onlsrv

(localhost)virtualization

Experiment

10 - Ken Wong– 10/14/2009

ssh sshdport 22

Firewallonl.arl.wustl.edu

onlusr

Configuring a Virtual IP NetworkTopology» NPRs, hosts, links» Used for resource reservation

Route Tables

Queue Tables» per port» port rate» datagram queuesRoute Tables

» per port» Initially empty at all ports» default RT: 1 port or all ports» IP pkt forwarding

n get pkt 1 hop closer to destination

» uses Longest Prefix Match

» datagram queuesn QIDs 0-63

» reserved queuesn QIDs 64-8,191

» queue parametersn threshhold: queue sizen quantum: scheduling weight

Plugin Tables

11 - Ken Wong– 10/14/2009

g

Filter Tables» per port» forward to port/queue» forward to plugin, port/queue

Plugin Tables» per NPR» standard plugins

n e.g., delay

» user-defined plugins

The NPR Data Path

12 - Ken Wong– 10/14/2009

The Network Processor Router (NPR)

TC TC ● ● ●

Local Memory8 threadcontexts

Microengine (ME)

Each NPR is

GPRs

ME

ME

ME● ● ●

ALU

5 Ports

Each NPR isone IXP 2800with 16 MEs

13 - Ken Wong– 10/14/2009

MIMP

SRAM DRAM

input output

PCI,GigE

● ● ● ● ● ●

XscaleCP

NPR Packet Processing (1)Route TableFilters

2 MEs

ParseLookup& Copy

Mux

Plugins

In OutRx

Tx

HdrFmt

QueueManager

port 0

port 4

port 0

port 4

3 1 11 22

5

●●●

●●●

14 - Ken Wong– 10/14/2009

Implements for each output port:Port Rate and Bandwidth Sharing5 x 8,128 Reserved Queues (64-8191)5 x 64 Datagram Queues (0-63)

Fast Path: normalpkt path

drop pkt

NPR Packet Processing (2)

XscaleCP

Queues for each portPkt Scheduler (5 ports)

exception processing(IP options, ARP, 0 TTL)

ParseLookup& Copy

Mux

In OutRx

Tx

HdrFmt

buffer pkt

QueueManager

port 0

port 4

port 0

port 4

3 1 11 22

5

●●●

●●●

15 - Ken Wong– 10/14/2009

Classify pktCopy pkt

Plugins

custom processing8 threads/ME

Ethernet encap

transmitethernet frame

NPR Packet Processing (3)Rx: Receive Pkt»Put pkt in DRAM; Send meta-packetMux: Multiplex traffic (inputs, CP, plugins)PLC: Parse, Lookup and Copy»Implements Route Table and Filter Table lookup»Uses TCAM (Ternary Content-Addressable Memory)QM: Queue Manager»Pkt scheduler for each of 5 output ports

n Implements port rate concept using token bucket

16 - Ken Wong– 10/14/2009

n Implements bandwidth sharing concept using Weighted Deficit Round Robin (WDRR) algorithm

Hdr Format»Create ethernet frameTx: Transmit Ethernet Frame

Experiment 1

17 - Ken Wong– 10/14/2009

Topics2 NPRs, 4 hosts, 2 links between NPRsExperiment files»473-1-f09.exp»473-1-filters-f09.exp (filters at ports 1.0 and 1.4)

Unix commands»/sbin/ifconfig network interface properties»/bin/netstat network interface statistics»/bin/ping ICMP echo/reply»/usr/local/bin/iperf UDP/TCP traffic generator

18 - Ken Wong– 10/14/2009

»/usr/local/bin/iperf UDP/TCP traffic generator

Route tables and port rate (capacity)

Experiment 1 (Final Version)

rcvrsndr

NPR 2, port 1 receivesTraffic at ALMOST 12 Mbps

NPR.1 NPR.21210

sndr

UDP trafficrcvr

1

19 - Ken Wong– 10/14/2009

R

R R

1Gbps

12Mbps 1

2

1

4

0

40

Mbps

10Mbps

Gbps

2 n1p0 starts n1p4 starts

Steps in Running an ExperimentDefine maximal network resources using RLI» Required: Add NPR routers, Hosts, Links

Save configuration» File Save as

Build SSH tunnel to onl wustl eduBuild SSH tunnel to onl.wustl.eduMake a reservation» File Make reservation» Can be an advanced reservation

When reservation time arrives:» Build SSH tunnel to onl.wustl.edu» Run RLI and open configuration file» Ask for network to be built: File Commit

20 - Ken Wong– 10/14/2009

» Ask for network to be built: File Commit» Define more network parameters (Commit) and monitoring» Wait for commit to finish

Log into onl.wustl.edu using ssh» Prompt will show you have logged into onlusr» You can only access your ONL nodes from onlusr host» Use Linux commands to run traffic generators

RLI MenusMenu File …»Open, Commit (binds virtual network to actual

network components)Port Route Table EditPort Route Table Edit»Add/Delete Route, Generate Local Default RoutesPort Queue Table»Port Rate, Edit Add/Delete Queue, Threshold,

QuantumPort Filter Table Edit

dd/ d / l l

21 - Ken Wong– 10/14/2009

»Add/Edit/Delete FilterMenu Monitoring Add Monitoring Display

Send ping pkts from n1p0 to n2p1

rcvrsndr

ping traffic

22 - Ken Wong– 10/14/2009

stats

RTT

send 3 pktsto n2p1

Send UDP pkts from n1p0 to n2p1

sent at 20 Mbps

server got 11.4 Mbps

23 - Ken Wong– 10/14/2009

ifconfig and netstat commands

eth0 control interface10.0.0.90

eth1 data interface192.168.1.16

/bi / t t t

24 - Ken Wong– 10/14/2009

/bin/netstat

Route Tablesall RTs start empty

add/deleteentries

25 - Ken Wong– 10/14/2009

output port

to NPR 2

affects every port, every NPR

Queue Tables and Port Rate

Default port rate = 1 Gbps

26 - Ken Wong– 10/14/2009

p pRequested 12 Mbps»actual max port rate after commit = 11.611 Mbps

• 683 Kbps granularity• 11.611 Mbps = 17 x 0.683 Mbps

Bandwidth Monitoring (1)

27 - Ken Wong– 10/14/2009

Bandwidth Monitoring (2)

select legend label tomodify chart propertiesmodify chart properties

28 - Ken Wong– 10/14/2009

change Y-axis label

Experiment 1 Shell ScriptSee ~kenw/bin/run-473

Puts $n1p0, etc intoUnix environment

start 2 UDP receivers

20 sec

10 Mbps

29 - Ken Wong– 10/14/2009

start 1st UDP sender

start 2nd UDP sender

wait 6 seconds

1 MB bufferUDPrun as client; send to n2p2

Experiment 1 Shell ScriptSee ~kenw/bin/run-473

Puts $n1p0, etc intoUnix environment

start 2 UDP receivers

20 sec

10 Mbps

30 - Ken Wong– 10/14/2009

start 1st UDP sender

start 2nd UDP sender

wait 6 seconds

1 MB bufferUDPrun as client; send to n2p2

Experiment 1 (Final Version)

rcvrsndr

NPR 2, port 1 receivesTraffic at ALMOST 12 Mbps

NPR.1 NPR.21210

sndr

UDP trafficrcvr

1

31 - Ken Wong– 10/14/2009

R

R R

1Gbps

12Mbps 1

2

1

4

0

40

Mbps

10Mbps

Gbps

2 n1p0 starts n1p4 starts

Experiment 2

32 - Ken Wong– 10/14/2009

Using Reserved Queue 64

33 - Ken Wong– 10/14/2009

add reserved queue64 (600,000 bytes)

Monitoring Queue Length

34 - Ken Wong– 10/14/2009

Experiment 2 Displays

35 - Ken Wong– 10/14/2009

36 - Ken Wong– 10/14/2009

Delaying Packets

37 - Ken Wong– 10/14/2009

Delaying Packets From n2p1

standard plugins~onl/npr/plugins/

rcvr

sndr

5 possible plugin MEsb d 0 4NPR 2 NPR 1

•Add a delay plugin D•Add a filter F

sndrUDP traffic

38 - Ken Wong– 10/14/2009

microengine 0

numbered 0-4NPR.2 NPR.1

R

F

F

FQ300

13

2

4

1

34D

Add The Filter

NPR.2 NPR.1

39 - Ken Wong– 10/14/2009

R

F

F

FQ300

13

2

4

1

34D

50 msec Delay (Default)

40 - Ken Wong– 10/14/2009

Send Control Message to Plugin

other messages:•=delay return delay (msec)•=counts return npkts, maxinq, ndrops•reset reset global counters

41 - Ken Wong– 10/14/2009