Measurement-Based Analysis of the Performance of several...
Transcript of Measurement-Based Analysis of the Performance of several...
MeasurementMeasurement--Based Analysis of theBased Analysis of thePerformance of several Wireless Performance of several Wireless
TechnologiesTechnologies
R. Cosmaa, A. Cabellos-Apariciob, J. Domenech-Benllochc, J. Gimenez-Guzmanc, J. Martinez-Bausetc, M. Cristiana, A. Fuentetajad, A. Lópezb, J. Domingo-Pascualb, J. Quemadad
a. Technical University of Cluj-Napoca, Romaniab. Universitat Politècnica de Catalunya, Barcelona, Spainc. Universidad Politécnica de Valencia, Spaind. Universidad Politécnica de Madrid, Spain
LANMAN 2008 3-6 Sep 2008, Cluj-Napoca, Romania 1
*Research carried out within CEPS (Spanish-funded research project)
AgendaAgenda
I. IntroductionII. Layer 3 ParametersIII. Measurement MethodologyIV. Experimental ResultsV. Conclusions and Future Work
LANMAN 2008 3-6 Sep 2008, Cluj-Napoca, Romania 2
I. IntroductionI. IntroductionEvents with major implications for the Internet:◦ The advent of broadband connections (DSL,
Cable Modems, Wireless Access)◦ The rapid evolution of Wireless TechnologiesThe current paradigm shift:◦ High Bandwidth◦ Low Delay◦ Extended Area of Service◦ Mobility
LANMAN 2008 3-6 Sep 2008, Cluj-Napoca, Romania 3
Present SolutionsPresent SolutionsIEEE 802.11 (WiFi)◦ Reduced costs◦ Maximum theoretical debits of 54 Mbps (.g)◦ Short-range coverage / low mobilityUMTS / HSDPA◦ 3G Technology: Mobility issues have been solved◦ Standards that enhance the Data RatesIEEE 802.16 (WiMAX)◦ New Infrastructure◦ Mobility support in 802.16e◦ Economical benefits
LANMAN 2008 3-6 Sep 2008, Cluj-Napoca, Romania 4
II. Layer 3 ParametersII. Layer 3 ParametersStandard metrics for the performance of Internet data delivery services:◦ One-way Delay (OWD)◦ Interpacket Delay Variation (IPDV)◦ Packet Loss Ratio (PLR)Relevance:◦ Performance of Transport Layer protocols◦ Performance of Interactive/Real-time applications◦ Insights on the dynamics and effectiveness of Layer 2
protocols/implementations.
LANMAN 2008 3-6 Sep 2008, Cluj-Napoca, Romania 5
III. Measurement MethodologyIII. Measurement Methodology1. Deployment and configuration of the test-bed
scenario2. Clock Synchronization3. Generating the Probe Traffic4. Capture of Test Traffic at the relevant nodes5. Extraction of Layer 3 Metrics from the capture
files
LANMAN 2008 3-6 Sep 2008, Cluj-Napoca, Romania 6
Testbed ScenarioTestbed ScenarioUMTS
Provider
RedIris
Internet
NTP + Control
GPS AntennaStratum 1NTP Server
Capture Point
WiMAX BS
WiFi BS
Flowserver
802.3
802.16
802.11Fast Ethernet
UMTS192.168.0.137
84.88.40.26
84.88.40.28
10.1.24.2
138.4.24.109
147.83.130.52
LANMAN 2008 3-6 Sep 2008, Cluj-Napoca, Romania 7
Clock Synchronization Clock Synchronization -- NTPNTPThe Network Time Protocol◦ Hierarchy-based Architecture (Stratum Levels)◦ Time source differentiation algorithms◦ Information combining (agreement) algorithmsNTP Dedicated Network◦ Separate Network Interfaces◦ Stratum 1 Servers◦ GPS-based Time Source◦ High accuracy
LANMAN 2008 3-6 Sep 2008, Cluj-Napoca, Romania 8
Test traffic Test traffic -- IsabelIsabelWhy did we use a videoconferencing application?◦ Simultaneous traffic generation for all the clients◦ UDP advantages◦ Different nature of Audio and Video flowsIsabel:◦ High modularity◦ Appropriate logical topology
LANMAN 2008 3-6 Sep 2008, Cluj-Napoca, Romania 9
Test traffic Test traffic -- IsabelIsabel
Each client sends its data to the flowserver.Flow multiplexing takes place at the flowserver.
Access Network
Interactive Terminals
Flowserver
Physical Topology Logical Topology
LANMAN 2008 3-6 Sep 2008, Cluj-Napoca, Romania 10
Traffic Capture Traffic Capture -- WiresharkWiresharkRequirements:◦ Timestamping◦ Capture file format
Wireshark:◦ Platform-independent operation◦ Advanced GUI◦ PCAP file format
LANMAN 2008 3-6 Sep 2008, Cluj-Napoca, Romania 11
Layer 3 Metrics ExtractionLayer 3 Metrics ExtractionImplementation:◦ Perl programming language◦ Net::Pcap wrapperFunctionality:◦ Synchronization checking◦ RTP flows listing◦ Packet matching algorithm for metric
extraction
LANMAN 2008 3-6 Sep 2008, Cluj-Napoca, Romania 12
IV. Experimental ResultsIV. Experimental ResultsStructure:◦ Statistical descriptors of central tendency and
variability: min, max, mean, median, standard deviation
◦ Histogram and CDF plots for OWD/IPDV◦ Variation of metrics with respect to packet sizeObjectives:◦ Empirical evaluation of link performances◦ Influence of Layers 1 & 2◦ Link behavior when subject to CBR/VBR traffic
LANMAN 2008 3-6 Sep 2008, Cluj-Napoca, Romania 13
Test traffic overviewTest traffic overview
Link/SSRC Flow Packets Flow size (Mbytes) Throughput(kbps)
Ethernet (2800) Up
Audio 32261 9.04 78.15
Video 22805 16.38 142.59
UMTS (2799) Up
Audio - - -
Video 19117 13.29 115.62
WiFi (2798) Up
Audio 32864 9.21 78.39
Video 20470 14.37 122.37
WiMAX (2797) Up
Audio 22429 6.28 54.26
Video 21005 15.08 129.64
LANMAN 2008 3-6 Sep 2008, Cluj-Napoca, Romania 14
Results Results -- WiFiWiFiStatistical parameters
Metric (ms) Max Mean Median Min Dev PLR
Video Up
OWD 20.373 4.677 1.003 0.550 7.128 0.034%
IPDV 1.443 0.400 0.064 -3.416 2.143
Audio Up
OWD 3.320 1.081 0.587 0.400 1.777 0%
IPDV 0.982 0 0.015 -2.039 1.488
Video Down
OWD 2.849 1.278 0.895 0.516 2.072 0.879%
IPDV 0.686 0 0.059 -1.773 1.487
Audio Down
OWD 1.824 0.895 0.602 0.511 1.616 0.38%
IPDV 0.607 0 0.018 -1.571 1.406
Maximum OWD values due to ARQ mechanismWell-centered IPDV distributionsUplink / Downlink asymmetry due to channel access mechanism
LANMAN 2008 3-6 Sep 2008, Cluj-Napoca, Romania 15
Results Results –– WiFi UplinkWiFi Uplink
Metrics variation with packet size:◦ OWD: unaffected◦ IPDV: increased values for packet sizes > 400
bytes
200 400 600 800
2468
1012
Packet Length [bytes]
OW
D [m
sec]
200 400 600 800
2468
101214
Packet Length [bytes]
IPD
V [m
sec]
2 4 6 8 100
0.2
0.4
OWD [msec]
Rel
ativ
e Fr
eque
ncy
-1 -0.5 0 0.5 10
0.02
0.04
0.06
0.08
0.1
IPDV [msec]
2 4 6 8 100
0.5
1
Cum
ulat
ive
Pro
babi
lity
-1 -0.5 0 0.5 10
0.2
0.4
0.6
0.8
1
LANMAN 2008 3-6 Sep 2008, Cluj-Napoca, Romania 16
Results Results -- WiMAXWiMAXStatistical parameters
Asymmetries: ◦ Uplink / Downlink – different power amplifier gains for
MS and BS◦ Audio (CBR) / Video (VBR)High Packet Loss Ratios:◦ Experimental MS equipment at Valencia
Metric (ms) Max Mean Median Min Dev PLR
Video Up
OWD 275.118 150.672 134.998 54.056 77.686 10.621%
IPDV 49.587 0 8.933 -80.051 35.005
Audio Up
OWD 244.949 107.593 81.300 42.960 67.957 3.237%
IPDV 39.185 400 µs 4.616 -62.378 27.819
Video Down
OWD 158.730 78.567 67.192 55.251 34.513 6.54%
IPDV 10.160 -0.300 0.570 -22.082 9.540
Audio Down
OWD 104.967 68.058 62.121 54.322 23.135 0.857%
IPDV 8.691 0 0.275 -16.799 8.514
LANMAN 2008 3-6 Sep 2008, Cluj-Napoca, Romania 17
Results Results –– WiMAX UplinkWiMAX Uplink
Optimal packet length: ~500 bytes.Discrete nature of IPDV distribution:◦ Frame size: 5ms◦ TDD◦ ARQ mechanism
Multiple retransmissions
200 400 600 800
100
150
200
Packet Length [bytes]
OW
D [m
sec]
200 400 600 800
20
40
60
80
Packet Length [bytes]IP
DV
[mse
c]
100 150 200 2500
0.01
0.02
OWD [msec]
Rel
ativ
e Fr
eque
ncy
-80 -60 -40 -20 0 20 400
0.01
0.02
IPDV [msec]
100 150 200 2500
0.5
1
Cum
ulat
ive
Pro
babi
lity
-80 -60 -40 -20 0 20 400
0.5
1
LANMAN 2008 3-6 Sep 2008, Cluj-Napoca, Romania 18
Results Results –– WiMAX DownlinkWiMAX Downlink
IPDV unstable with small packet sizesDiscrete nature of IPDVLess frame retransmissions than Uplink
200 400 600 80060
70
80
90
Packet Length [bytes]
OW
D [m
sec]
200 400 600 800
2
4
6
8
Packet Length [bytes]IP
DV
[mse
c]
60 80 100 120 1400
0.05
0.1
OWD [msec]
Rel
ativ
e Fr
eque
ncy
-20 -10 0 100
0.01
0.02
IPDV [msec]
60 80 100 120 1400
0.5
1
Cum
ulat
ive
Pro
babi
lity
-20 -10 0 100
0.5
1
Results Results –– UMTS/HSDPAUMTS/HSDPA
Special setup imposed by the maximum uplinkbandwidthHigh maximum values due to congestion in the provider’s networkIPDV distributions better than WiMAXEfficient ARQ mechanisms
Statistical parametersMetric (ms) Max Mean Median Min Dev PLR
Video Up
OWD 258.251 153.877 95.385 75.903 103.481 0.250%
IPDV 27.792 -0.258 1.058 -66.573 30.228
Video Down
OWD 98.176 74.2441 67.500 50.047 43.524 0.242%
IPDV 11.751 0 1.825 -22.244 15.349
LANMAN 2008 3-6 Sep 2008, Cluj-Napoca, Romania 20
Results Results –– UMTS UplinkUMTS Uplink
Metric variation with packet size:◦ OWD stable◦ IPDV less stable,
increasing
Discrete IPDV nature:◦ Link temporization at
RLC layer◦ Loss recovery in integer
number of TTIs
100 150 200 2500
0.05
0.1
OWD [msec]
Rel
ativ
e Fr
eque
ncy
-60 -40 -20 0 200
0.02
0.04
IPDV [msec]
200 400 600 800
200
400
600
800
1000
Packet Length [bytes]
OW
D [m
sec]
200 400 600 800
20406080
100120
Packet Length [bytes]IP
DV
[mse
c]
100 150 200 2500
0.5
1
Cum
ulat
ive
Pro
babi
lity
-60 -40 -20 0 200
0.5
1
LANMAN 2008 3-6 Sep 2008, Cluj-Napoca, Romania 21
Results Results –– UMTS DownlinkUMTS Downlink
Lower OWD valuesIPDV and OWD stable with packet sizeHSDPA effects on IPDV discretization:◦ TTI limited to one radio
frame◦ Reduced radio frame
duration duration
200 400 600 8006080
100120140160180
Packet Length [bytes]
OW
D [m
sec]
200 400 600 800
20
40
60
80
100
Packet Length [bytes]
IPD
V [m
sec]
60 70 80 900
0.005
0.01
0.015
0.02
0.025
OWD [msec]
Rel
ativ
e Fr
eque
ncy
60 70 80 900
0.2
0.4
0.6
0.8
1
Cum
ulat
ive
Pro
babi
lity
-10 -5 0 5 100
0.02
0.04
IPDV [msec]-10 -5 0 5 10
0
0.5
1
LANMAN 2008 3-6 Sep 2008, Cluj-Napoca, Romania 22
V. ConclusionsV. ConclusionsSimple methodology for extracting Layer 3 Metrics for heterogeneous networksWiFi:◦ Good stability of metrics in respect to packet size◦ Extreme values caused by ARQ mechanism◦ Uplink / Downlink asymmetry due to the radio channel
disputing algorithmWiMAX:◦ High PLR – experimental equipment◦ Uplink / Downlink asymmetry◦ IPDV discretization due to ARQ mechanismsUMTS / HSDPA:◦ Efficient retransmission schemes◦ HSDPA effects on Downlink metricsBetter values for metrics associated with CBR (Audio) traffic
LANMAN 2008 3-6 Sep 2008, Cluj-Napoca, Romania 23
V. ConclusionsV. ConclusionsWiFi WiMAX UMTS
OWDIPDVPLRThroughputRange/Mobility
Best
Medium
Worst
LANMAN 2008 3-6 Sep 2008, Cluj-Napoca, Romania 24
V. Future WorkV. Future WorkDevelopment of a system for fingerprinting wireless links:◦ Based on the nature of OWD & IPDV
distributions◦ Network Layer could infer the access link w/o
employing cross-layering techniquesExtending the Perl application:◦ Real-time mode◦ Capture and processing of traffic other than
RTP over UDP
LANMAN 2008 3-6 Sep 2008, Cluj-Napoca, Romania 25