Evaluating the Networking Performance of Linux-based
Home Router Platforms for Multimedia Services
Ingo Kofler, Robert Kuschnig, Hermann Hellwagner
Institute for Information TechnologyAlpen-Adria-Universitat Klagenfurt, Austria
WoMAN’11, Barcelona, July 2011
Ingo Kofler Linux-based Home Router Platform Evaluation
Motivation
Popularity of Linux-based network devices
No real-time OS but sufficient for home network deployments
Low development costs
Mature platform and code base
Availability of services and applications
Open platform for running additional multimedia services
What performance can be expected in terms of...
computing performance
memory throughput
application-layer networking throughput
... and where are the bottlenecks?
Ingo Kofler Linux-based Home Router Platform Evaluation
Examples of Multimedia ServicesIn-Network Adaptation of Scalable Video Content
Media-Aware Network Element (MANE) for H.264/SVC
Adapt scalable video directly on network device
Adaptation must be signaling-aware and stateful
Realized as proxy running as process in user space
Proxy adapts video bit rate according to wireless link conditions
Previous work → ACM NOSSDAV’08, ACM NOSSDAV’11
Ingo Kofler Linux-based Home Router Platform Evaluation
Examples of Multimedia ServicesServing Multimedia Content in the Home Network
Deployment as always-on home server
Storage attached via USB port
Serving videos, music and pictures to clients in the network
Open source server implementations available
Support for USB-based DVB devices for serving live content (DVB-T/S/C)
Ingo Kofler Linux-based Home Router Platform Evaluation
Examples of Multimedia ServicesSharing Social Content from Home
Using server at own premises to serve social content
To overcome privacy and copyright concerns in social networks
Recent work by Marcon et al. @ ACM NOSSDAV’11
Private content is stored on USB storage and served via HTTP
Proof-of-concept study indicates feasibility
Ingo Kofler Linux-based Home Router Platform Evaluation
Home Router PlatformsSoftware
openWrt Linux
Popular Linux distribution for embedded systems
Replacement for stock firmware
Supports a variety of network devices/models
Provides SDK for cross-compiling software
Large set of available software packages
Common software platform for comparison
Most recent release: Backfire 10.03 based on Linux 2.6.32 kernel
Deployed on representative router platforms
Running performance and network benchmarks
Ingo Kofler Linux-based Home Router Platform Evaluation
Home Router PlatformsHardware Specifications
Vendor/Model Linksys TP-Link UBNTWRT54GL WR1043ND RS Pro
System-on-a-chip BCM5352EL AR9132 AR7161CPU MIPS32 4Kc MIPS 24Kc MIPS 24KcCPU clock 200 MHz 400 MHz 720 MHzInst./Data-Cache 16 KB / 8 KB 64 KB / 32 KB 64 KB / 32 KBSDRAM size/clock 16 MB / 100 MHz 32 MB / 400 MHz 128 MB / 360 MHzFlash capacity 4 MB 8 MB 16 MB
Networking 100 Mbps 1 Gbps 1 GbpsWireless 802.11b/g 802.11b/g/n 802.11a/b/g/nMisc - USB USB, 3x mini-PCI
Ingo Kofler Linux-based Home Router Platform Evaluation
Platform EvaluationProcessing and memory performance
Determine processing performance and memory throughput
Comparison of results to modern system (Core i7 M620)
CoreMark benchmark
Focus on CPU performance using synthetic workloadList operation, matrix multiplication, state machine and CRCAlgorithms executed in a loopMetric: Number of loop iterations per second
STREAM copy
Part of the STREAM benchmark suiteCopies array content element-by-elementMetric: Megabytes per second
Ingo Kofler Linux-based Home Router Platform Evaluation
Platform EvaluationProcessing and memory performance
Vendor/Model Clock CoreMark CoreMark STREAM copy[MHz] absolute normalized [MB/sec]
WRT54GL 200 333 1.67 59TLWR 400 701 1.75 231RSPRO 720 1263 1.75 381Intel Core 2 Duo E8400 3000 8,651 2.88 4,055Intel Core i7 M620 2667 9,457 3.55 6,830
CoreMark: 7.5 to 28 times slower than modern desktop CPU (single core)
STREAM copy: 18 to 116 times slower
Ingo Kofler Linux-based Home Router Platform Evaluation
Platform EvaluationNetworking performance
Focus on application-layer performance
Evaluation using different traffic scenarios (UDP/TCP, incoming/outgoing)
Investigated impact of different parameters (block size, socket buffer)
Results obtained using iperf and netperf
Experiments performed with firewall rules disabled
Ingo Kofler Linux-based Home Router Platform Evaluation
Platform EvaluationOutgoing and Incoming TCP
0 50000 100000 150000 200000 250000
010
020
030
040
050
0
Block size [Byte]
Thr
ough
put [
Mbp
s]
●●
●
●
●
●●
● ●
●
WRT54TLWRRSPRO
0 50000 100000 150000 200000 250000
010
020
030
040
050
0
Block size [Byte]
Thr
ough
put [
Mbp
s]
●
●●●
● ● ● ● ●
●
WRT54TLWRRSPRO
Ingo Kofler Linux-based Home Router Platform Evaluation
Platform EvaluationOutgoing and Incoming UDP
400 600 800 1000 1200 1400
010
020
030
040
050
0
Block size [Byte]
Thr
ough
put [
Mbp
s]
●
●
●
●
●●
●
●
●
WRT54TLWRRSPRO
0 50 100 150 200 250 300
050
100
150
200
250
300
Transmit rate [Mbps]
Thr
ough
put [
Mbp
s]
●●
●●
●●●●●
●●●
●
●●
●
WRT54TLWRRSPRO
Ingo Kofler Linux-based Home Router Platform Evaluation
Platform EvaluationObservations
Choice of block size significantly influences throughput
Cannot be observed on typical PC-based systems
High overhead per system call
Conclusion: avoid small packet sizes (UDP) and block sizes (TCP)
Application-layer throughput far below nominal link speed
WRT54 only achieves 29 Mbps outgoing UDP throughput
418 Mbps outgoing UDP throughput on RSPRO
Identify performance bottleneck using ftrace framework
Debugging and tracing facility in Linux kernel
Useful to determine function execution time
Execution traces of UDP sendto system call
Ingo Kofler Linux-based Home Router Platform Evaluation
Platform EvaluationObservations
Choice of block size significantly influences throughput
Cannot be observed on typical PC-based systems
High overhead per system call
Conclusion: avoid small packet sizes (UDP) and block sizes (TCP)
Application-layer throughput far below nominal link speed
WRT54 only achieves 29 Mbps outgoing UDP throughput
418 Mbps outgoing UDP throughput on RSPRO
Identify performance bottleneck using ftrace framework
Debugging and tracing facility in Linux kernel
Useful to determine function execution time
Execution traces of UDP sendto system call
Ingo Kofler Linux-based Home Router Platform Evaluation
Platform EvaluationImpact of netfilter
UDP sendto causes around 200 function calls in kernel
Nearly half of the functions caused by the netfilter component
Re-run evaluations using kernel without netfilter
0 50000 100000 150000 200000 250000
010
020
030
040
050
060
0
Block size [Byte]
Thr
ough
put [
Mbp
s]
●●
●
●
●
●●
● ●
●
●●
●●
●● ● ●
●
●
WRT54WRT54 w/o netfilterTLWRTLWR w/o netfilterRSPRORSPRO w/o netfilter
400 600 800 1000 1200 1400
020
040
060
080
0
Block size [Byte]
Thr
ough
put [
Mbp
s]
●
●
●●
●●
●●
●
●
●
●
● ●●
●
●
●
WRT54WRT54 w/o netfilterTLWRTLWR w/o netfilterRSPRORSPRO w/o netfilter
Ingo Kofler Linux-based Home Router Platform Evaluation
Platform EvaluationImpact of netfilter
UDP sendto causes around 200 function calls in kernel
Nearly half of the functions caused by the netfilter component
Re-run evaluations using kernel without netfilter
0 50000 100000 150000 200000 250000
010
020
030
040
050
060
0
Block size [Byte]
Thr
ough
put [
Mbp
s]
●●
●
●
●
●●
● ●
●
●●
●●
●● ● ●
●
●
WRT54WRT54 w/o netfilterTLWRTLWR w/o netfilterRSPRORSPRO w/o netfilter
400 600 800 1000 1200 1400
020
040
060
080
0
Block size [Byte]
Thr
ough
put [
Mbp
s]
●
●
●●
●●
●●
●
●
●
●
● ●●
●
●
●
WRT54WRT54 w/o netfilterTLWRTLWR w/o netfilterRSPRORSPRO w/o netfilter
Ingo Kofler Linux-based Home Router Platform Evaluation
Platform Evaluationftrace Results
Impact of netfilter
1/3 of execution time spent for netfilter code
Throughput increases up to 50 percent
netfilter used for many (often relevant) features like NAT, firewall, . . .
Further investigations
Measuring time-consuming functions of UDP sendto syscall
Aggregate function to certain blocks/task of networking stack
Evaluations performed with disabled netfilter code
Ingo Kofler Linux-based Home Router Platform Evaluation
Platform Evaluationftrace Results
Impact of netfilter
1/3 of execution time spent for netfilter code
Throughput increases up to 50 percent
netfilter used for many (often relevant) features like NAT, firewall, . . .
Further investigations
Measuring time-consuming functions of UDP sendto syscall
Aggregate function to certain blocks/task of networking stack
Evaluations performed with disabled netfilter code
Ingo Kofler Linux-based Home Router Platform Evaluation
Platform EvaluationDissecting the UDP sendto syscall
400 1450
Socket APIBuffer HandlingUDPIPBridge TxHardware Tx
UDP payload size [Byte]
Dur
atio
n [u
secs
]
05
1015
20
Significant part required
for buffer handling (buffer
allocation, copy from user
to kernel space)
Checksum calculation
done in software
Potential for
improvements by
offloading
Ingo Kofler Linux-based Home Router Platform Evaluation
Conclusions
Evaluation of Linux-based router platforms
Interesting platform for multimedia services
Deployment as Media-Aware Network Element
Modest application layer performance
Far below nominal link speed
Use large packet sizes (UDP) and large buffers (TCP)
Disable netfilter component if not really required
No hardware offloading, potential for improvements
Ingo Kofler Linux-based Home Router Platform Evaluation
Top Related