Networking Virtualization Using FPGAs Russell Tessier, Deepak Unnikrishnan, Dong Yin, and Lixin Gao...
-
Upload
magnus-hunt -
Category
Documents
-
view
216 -
download
0
Transcript of Networking Virtualization Using FPGAs Russell Tessier, Deepak Unnikrishnan, Dong Yin, and Lixin Gao...
Networking Virtualization Using FPGAs
Russell Tessier, Deepak Unnikrishnan, Dong Yin, and Lixin Gao
Reconfigurable Computing GroupDepartment of Electrical and Computer Engineering
University of Massachusetts, Amherst, USA
2
Outline
Network virtualization
Advantage of FPGA for virtualization
Architecture of FPGA-based virtualization system
Partially-reconfigurable implementation
Results
3
Virtual Networking Internet growing to include new applications and services
Cloud computing, Data center networking
Challenges Lack of innovation at network core – Fixed internet routers Coupled infrastructure-service provider model
Solution Network virtualization Router hardware shared
across multiple networks
4
Network Virtualization
Many logical networks over a physical infrastructure
Virtual nodes
Shared network resources among multiple virtual networks
Reduces costs
Independent routing policies
FA
C E
DB
C E
F
DB
A
B D
CE
A F
5
Virtual Router Independent routing policies
for each virtual router
Key challenges• Isolation• Performance• Flexibility• Scalability
Forwarding Table
Routing Control
Virtual router B
Forwarding Table
Routing Control
Physical routerVirtual router A
DEMUX MUX
6
Traditional Network Virtualization Techniques Software
Full virtualization Container virtualization Limitations
Limited performance (~100Mbps) Limited isolation
ASIC Supercharging PlanetLab Platform1
Juniper E series Possible Limitations
Flexibility Scalability
HW
VM VM
Full virtualization
HW
OSinstance
OSinstance
Container virtualization
1 “Supercharging PlanetLab – A High Performance, Multi-Application, Overlay Network Platform”, J. Turner et al., SIGCOMM 2007
7
Virtualization using FPGAs
A novel network virtualization substrate which
• Uses FPGA to implement high performance virtual routers
• Introduces scalability through virtual routers in host software
• Exploits reconfiguration to customize hardware virtual routers
FPGA
VirtualRouter 1
VirtualRouter 2
VirtualRouter 3
VirtualRouter 4
8
System Overview
SRAM
SRAM
PCI
1GEthernet
I/F
SoftwareVirtual Router
SoftwareVirtualRouter
SoftwareVirtualRouter
Kernel driverSoftware bridge
Linux
NetFPGA
PHY
SDRAM
SDRAMHW VRouter
HW VRouter
NIC NIC
9
Architecture
HardwareVirtual
Router 1
VIP TYPE ID0 HW 01 HW 12 SW 03 SW 1
User space
Kernel space
Host Workstation
NetFPGA
MAC RX Q
CPU RX Q
MAC RX Q
CPU RX Q
MAC RX Q
CPU RX Q
MAC RX Q
CPU RX Q
InputArbiter
DynamicDesignSelect
HardwareVirtual
Router 2
CPUTransceiver
OutputQueue
MAC TX Q
CPU TX Q
MAC TX Q
CPU TX Q
MAC TX Q
CPU TX Q
MAC TX Q
CPU TX Q
NetFPGAPCI I/F
NetFPGAdriver
Bridge
PC NIC
Software Virtual
Router 1
SoftwareVirtual
Router 2
Bridge
NetFPGAdriver
NetFPGAPCI I/F
Kernel space
PC NIC
CONTROL
PHY MAC
PHY MAC
PHY MAC
PHY MAC
PHY MAC
PHY MAC
PHY MAC
PHY MAC
PCI BUS
To Switch
FromSwitch
10
Scalable Network Virtualization Approaches for implementing scalable virtual routers
• Single receiver approach• All packets routed through NetFPGA hardware
• Multi-receiver approach• Use a switch to separate packets to NetFPGA and software
11
Virtual Router Customization Multiple virtual routers share the substrate
Individual virtual routers may need customization
Challenge
• Minimize the impact on traffic in shared hardware virtual routers during modification
12
Multi Receiver Approach
HardwareVirtual
Router 1
VIP TYPE ID0 HW 01 HW 12 SW 03 SW 1
User spaceKernel spaceHost OS
NetFPGA
MAC RX Q
CPU RX Q
MAC RX Q
CPU RX Q
MAC RX Q
CPU RX Q
MAC RX Q
CPU RX Q
InputArbiter
DynamicDesignSelect
Hardware Virtual
Router 2
CPUTransceiver
OutputQueue
MAC TX Q
CPU TX Q
MAC TX Q
CPU TX Q
MAC TX Q
CPU TX Q
MAC TX Q
CPU TX Q
NetFPGAPCI I/F
NetFPGAdriver
Bridge
PC NIC
SoftwareVirtual
Router 1
SoftwareVirtual
Router 2
Bridge
NetFPGAdriver
NetFPGAPCI I/F
Kernel space
PC NIC
CONTROL
PHY MAC
PHY MAC
PHY MAC
PHY MAC
PHY MAC
PHY MAC
PHY MAC
PHY MAC
Switch Switch
PCI BUS
So
urc
e
De
stin
atio
n
13
HardwareVirtual
Router 1
VIP TYPE ID0 HW 01 HW 12 SW 03 SW 1
User spaceKernel spaceHost OS
NetFPGA
MAC RX Q
CPU RX Q
MAC RX Q
CPU RX Q
MAC RX Q
CPU RX Q
MAC RX Q
CPU RX Q
InputArbiter
DynamicDesignSelect
Hardware Virtual
Router 2
CPUTransceiver
OutputQueue
MAC TX Q
CPU TX Q
MAC TX Q
CPU TX Q
MAC TX Q
CPU TX Q
MAC TX Q
CPU TX Q
NetFPGAPCI I/F
NetFPGAdriver
Bridge
PC NIC
SoftwareVirtual
Router 1
SoftwareVirtual
Router 2
Bridge
NetFPGAdriver
NetFPGAPCI I/F
Kernel space
PC NIC
CONTROL
PHY MAC
PHY MAC
PHY MAC
PHY MAC
PHY MAC
PHY MAC
PHY MAC
PHY MAC
Switch Switch
PCI BUS
So
urc
e
De
stin
atio
n
Dynamic Reconfiguration
AddressRemap
Sw VirtualRouter
New HW VirtualRouter
Reconfigure FPGA
14
HardwareVirtual
Router 1
VIP TYPE ID0 HW 01 HW 12 SW 03 SW 1
User space
Kernel space
Host Workstation
NetFPGA
MAC RX Q
CPU RX Q
MAC RX Q
CPU RX Q
MAC RX Q
CPU RX Q
MAC RX Q
CPU RX Q
InputArbiter
DynamicDesignSelect
HardwareVirtual
Router 2
CPUTransceiver
OutputQueue
MAC TX Q
CPU TX Q
MAC TX Q
CPU TX Q
MAC TX Q
CPU TX Q
MAC TX Q
CPU TX Q
NetFPGAPCI I/F
NetFPGAdriver
Bridge
PC NIC
Software Virtual
Router 1
SoftwareVirtual
Router 2
Bridge
NetFPGAdriver
NetFPGAPCI I/F
Kernel space
PC NIC
CONTROL
PHY MAC
PHY MAC
PHY MAC
PHY MAC
PHY MAC
PHY MAC
PHY MAC
PHY MAC
PCI BUS
To Switch
FromSwitch
Single receiver approach
15
Partial Reconfiguration Use partial reconfiguration to independently configure
virtual routers
16
Partial Reconfiguration Up to 2 partially reconfigurable virtual routers in Virtex 2
Up to 20 partially reconfigurable virtual routers in Virtex 5
17
Experimental Approach Metrics
• Throughput• Latency
Packet generation• NetFPGA packet generator• iPerf
Ping utility used for latency measurements
Software virtual routers run on 3Ghz AMD X2 6000+ processor, 2GB RAM, Intel E1000 GbE in PCIe slot
SourceNetFPGA Pktgen/
iPerf
VirtualRouter
SinkNetFPGA Pktcap/
iPerf
1Gbps 1Gbps
18
Dynamic Reconfiguration Overhead
Full FPGA reconfiguration• Source pings destination through a single h/w virtual router• Migrate traffic to software• Reconfigure FPGA• Migrate traffic to hardware• 12 seconds reconfiguration overhead
Partial FPGA reconfiguration• 0.6 seconds reconfiguration• Remaining virtual routers in FPGA remain active
19
Throughput of Single Hardware Virtual Router Hardware virtual router 1 to 2 order better than the software
virtual router Consistent forwarding rates vs packet size
1
10
100
1000
64 128 512 1024 1470Packet size (bytes)
Thro
ughp
ut (M
bps)
Hardwarevirtual router
Softwarevirtual router
20
Full FPGA Reconfiguration Two virtual routers (A, B) initially in FPGA During reconfiguration router A migrated to software, the
other eliminated After reconfiguration two virtual routers (A, B’) again in FPGA
ReducedThroughput
21
Partial FPGA Reconfiguration A remains in hardware and operates at full speed 20X speedup in reconfiguration down time due to partial
reconfiguration SustainedThroughput
22
Scalability - Average Throughput of Virtual Routers Aggregate throughput of 1Gbps up to 4 h/w virtual routers
Adding software virtual routers causes drop in aggregate throughput
1
10
100
1000
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
Virtual routers
Thro
ughp
ut (M
bps)
Avg hardware andsoftware
Software only
23
Scalability – Average Latency of Virtual Routers Latency of h/w virtual router substantially better than
software virtual router
0
0.2
0.4
0.6
0.8
1
1.2
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
Virtual routers
Late
ncy
(ms)
Software only
Avg hardware andsoftware
24
Throughput Effect of Reconfiguration Frequency Partial reconfiguration beneficial during frequent updates
25
Average throughput for combined hardware/software Rigid placement constraints limit #hardware virtual routers Larger FPGAs sustain high throughputs for more virtual routers
26
Altera DE4
27
Throughput Measurements on DE-4 Reference Router Consistent packet forwarding rates of 1Gbps for all packet sizes
(64-1024) bytes. Packet throughput matches NetFPGA reference router in all cases. Packet buffering in on-chip SRAM (300Kbit).
28
Resource Utilization
Altera DE4 NetFPGA
Used Available % Used Available %
Combinational LUTs
24,727 182,400 14% 19,783 47,232 41%
Logic Registers
34,218 182,400 19% 14,682 47232 31%
Memory bits
1,448,612
14,625,792
9% 24 232 10.3%
IOs 118 888 13% 356 692 51%
Logic Utilization 14% LUTs 19% Logic registers 9% Memory bits 13% IOs
29
Reconfiguration Time
Partial Reconfiguration
Full Reconfiguration
20X Faster virtual router updates with partial reconfiguration
30
Resource Usage and Power Consumption Virtex 2 can support
• 5 h/w virtual routers (without support for software routers) • 4 h/w virtual routers (with support for software routers)• 2 partially reconfigurable hardware routers
Virtex 5 can support up to 32 virtual routers
A single h/w virtual router consumes • 156mW static power
• 688mW dynamic power
Clock gating saves 10% power
31
Dynamic Virtual Network Allocation Assign virtual networks to software/hardware virtual
routers based on bandwidth-resource requirements 15-20% more successful network upgrades with dynamic
allocation
32
Conclusion FPGAs are ideal for a variety of networking applications
• Partial reconfiguration growing in importance
Virtual networking is an emerging networking area• Combined FPGA and software system gives high performance and
scalability• A novel heterogeneous network virtualization substrate using
FPGAs
NetFPGA platform used to implement up to five virtual routers• Two order of magnitude throughput increase versus software.
Partial FPGA reconfiguration reduces the number of implemented routers but reduces reconfiguration time to 0.6s• Allocation algorithm migrates highest throughput routers to
hardware