Fundamentals of Statistical Signal Processing: Estimation ...
End-to-End Network Performance Estimation Using Signal ComplexitySlides
-
Upload
marat-zhanikeev -
Category
Technology
-
view
207 -
download
2
description
Transcript of End-to-End Network Performance Estimation Using Signal ComplexitySlides
.
What is the Signal?
• BLACK BOX: end to end (e2e) network, parts of network, data centers,network/data services, applications, etc.◦ in this paper specifically the e2e network, even more specifically availablebandwidth (AB)
• INPUT: active probing, basically trains of packets thrown at e2e paths
• OUTPUT: the same packet trains on the other side
• OBJECTIVE: to measure/understand/model the black box 02
02 myself+1 "Modeling Network Performance of End Hosts" IEICE Trans. vol.E95-D, no.7 (2012)
M.Zhanikeev -- [email protected] -- E2E Network Performance Estimation Using Signal Complexity -- http://tinyurl.com/kyutech131113 --- 2/19...
2/19
.
What is the Problem?
• it is really really hard 06 11
• multiple tools and methods, all come up withmismatched results [07]
• many design tricks, little math [08] [13]
• too much specificity, no unified/universal tool
06 M.Jain+1 "End-to-End Available Bandwidth: Dynamics..." IEEE/ACM Trans., vol.11 (2003)
11 C.Dovrolis+2 "What do packet dispersion techniques measure?" INFOCOM, vol.2 (2001)
M.Zhanikeev -- [email protected] -- E2E Network Performance Estimation Using Signal Complexity -- http://tinyurl.com/kyutech131113 --- 3/19...
3/19
.
The Problem: Specifics (AB)
• this paper talks specifically about available bandwidth (AB)◦ free capacity on some end-to-end path◦ between 6 and 30 hops in real networks
• it is a realtime technology!
• we cannot stress the network too much◦ example: if you want to find out how long some guy can hold his/her breath under
water, it is wrong to drown the person to find out◦ this paper will call that the Brutal Method
M.Zhanikeev -- [email protected] -- E2E Network Performance Estimation Using Signal Complexity -- http://tinyurl.com/kyutech131113 --- 4/19...
4/19
.
The Problem: Classification 1• obvious diagonal grouping
• IGI 07 andPathChirp 08 are the two comparison targets
◦ NOTE: so far all three are in the same cell
Low Precision High Precision
Short trains IGI, PathChirpThis Method
Long Trains iPerfPathLoad
07 N.Hu+1 "Evaluation and Characterization of AB ... Techniques" IEEE JSAC, vol.21, no.6 (2003)
08 V.Ribeiro+4 "pathChirp: Efficient Available Bandwidth Estimation for Paths" PAM Workshop (2003)
M.Zhanikeev -- [email protected] -- E2E Network Performance Estimation Using Signal Complexity -- http://tinyurl.com/kyutech131113 --- 5/19...
5/19
.
The Problem: Classification 2• this is where the proposed method finds its distinction• the basic idea: simple trains, complex analysis
◦ this paper uses Permutation Entropy (PE) 09
Simple Design Intricate Design
Short trains This Method IGI, PathChirp
Long Trains iPerfPathLoad
09 C.Bandt+1 "Permutation Entropy: A Natural Complexity Measure for Time Series" Physical Review Letters, Vol.88(17) (2002)
M.Zhanikeev -- [email protected] -- E2E Network Performance Estimation Using Signal Complexity -- http://tinyurl.com/kyutech131113 --- 6/19...
6/19
.
Things that Don't Work (1)
• this isPathChirp 08
• idea: exponentially shrinkingtrains will self-load (temporarily)
• the upper plot is the expected trend
1 2 3 45
6
7
8
Time/packet sequence
Arri
val g
ap -
Dep
artu
re g
ap
Breaking point
Dep
artu
re g
ap 12
34 5 6 7 8
Exponentialcurve
08 V.Ribeiro+4 "pathChirp: Efficient Available Bandwidth Estimation for Paths" PAM Workshop (2003)
M.Zhanikeev -- [email protected] -- E2E Network Performance Estimation Using Signal Complexity -- http://tinyurl.com/kyutech131113 --- 7/19...
7/19
.
Things that Don't Work (2)
• this is PathChirp inreal life
0 20 40 60 80 100Probe packet/time sequence
-0.2
0.2
0.6
1
Mixe
d n
orm
alize
d v
alue
s
back-to-back (arrival)pathchirp (arrival)
pathchirp (departure)pathchirp (diff)
01 myself "e2eprobe: source code of probing methods" https://github.com/maratishe/e2eprobe (2013)
M.Zhanikeev -- [email protected] -- E2E Network Performance Estimation Using Signal Complexity -- http://tinyurl.com/kyutech131113 --- 8/19...
8/19
.
Other Things that Don't Work
• any kind of groping by making small adjustment to packet space, like in IGI 07• different combinations of packet size in the same probe
• basic packet pair (just two packets) cannot measure AB, either 11
• ... good comparative study at 07
• ... source code for IGI, PathChirp and the proposal at 01
07 N.Hu+1 "Evaluation and Characterization of AB ... Techniques" IEEE JSAC, vol.21, no.6 (2003)
11 C.Dovrolis+2 "What do packet dispersion techniques measure?" INFOCOM, vol.2 (2001)
07 N.Hu+1 "Evaluation and Characterization of AB ... Techniques" IEEE JSAC, vol.21, no.6 (2003)
01 myself "e2eprobe: source code of probing methods" https://github.com/maratishe/e2eprobe (2013)
M.Zhanikeev -- [email protected] -- E2E Network Performance Estimation Using Signal Complexity -- http://tinyurl.com/kyutech131113 --- 9/19...
9/19
.
The Solution: Observations
.The most frequent pattern is .....
.
... that packets experience extreme states of being affected by thenetwork
• see the PathChirp in action a couple of slides ago
• the pattern is found for most train designs, single packets, packet pairs, etc.
.The Solution therefore is.....
.
...to analyze data as a combination of binary states (zeros and ones,basically)
M.Zhanikeev -- [email protected] -- E2E Network Performance Estimation Using Signal Complexity -- http://tinyurl.com/kyutech131113 --- 10/19...
10/19
.
The Solution: PE (algorithm)• arrival packet gaps
{x(i), i = 1, 2, ..
}• embed into m-dimensional space:Xi = [x(i), x(i+ d), ..., x(i+ (m− 1)d)]◦ for each i, there are multiple is◦ warping is allowed to maintain same size in all vectors◦ d is delay, the paper uses d = 1, so only neighbors -- supported by practice 02
• each Xi turned into the vector [0, 1, ...,m− 1] where value is the other, i.e.0th, 1th in Xi.
• m! permutations for each vector, each becomes a symbol• signal is then simply m-d space using in total i symbols for each permutation
• result: normalized entropy of the set of symbols◦ in this paper the best m (smallest entropy) is found using 10
02 myself+1 "Modeling Network Performance of End Hosts" IEICE Trans. vol.E95-D, no.7 (2012)
09 C.Bandt+1 "Permutation Entropy: A Natural Complexity Measure for Time Series" Physical Review Letters, Vol.88(17) (2002)
10 A.Brandmaier "Permutation Distribution Clustering and Structural Equation Model Trees" PhD Diss. (2012)
M.Zhanikeev -- [email protected] -- E2E Network Performance Estimation Using Signal Complexity -- http://tinyurl.com/kyutech131113 --- 11/19...
11/19
.
The Solution: Actual Methods
• static method (BC: bottleneck capacity) (r: tuning parameter = 0.8)
AB = BC− rBC(1− PE). (1)
• adaptive method: reference frame via calibration (max/min AB andPE values)
AB = ABmax −(ABmax − ABmin)(PE− PEmin)
(PEmax − PEmin). (2)
M.Zhanikeev -- [email protected] -- E2E Network Performance Estimation Using Signal Complexity -- http://tinyurl.com/kyutech131113 --- 12/19...
12/19
.
Experiments: Probing Network
Probing point 1
Probingtarget
labanother lab
another location in the city
Trafficdump
Probing point 2
Probing point 3
P2P1
P3
M.Zhanikeev -- [email protected] -- E2E Network Performance Estimation Using Signal Complexity -- http://tinyurl.com/kyutech131113 --- 13/19...
13/19
.
Experiments: Methods
• using software in 01 (IGI,PathChirp, PE)
• IGI and PathChirp are noisy,so catch when diagonal lineis crossed (simple heuristic)
• PE methods (0.8 and adaptive)
are from the previous slide --no probe design
40 50 60 70 80 90 100Sending rate (Mbps)
40
50
60
70
80
90
100
Rece
ivin
g ra
te (M
bps)
pathchirpigi
Detection point
01 myself "e2eprobe: source code of probing methods" https://github.com/maratishe/e2eprobe (2013)
M.Zhanikeev -- [email protected] -- E2E Network Performance Estimation Using Signal Complexity -- http://tinyurl.com/kyutech131113 --- 14/19...
14/19
.
Experiments: One Run• real probing in the wild
• prefer to run all the methods roughly at thesame time
• PE and Brute AB are based onback-to-back probe
Pick randompsize,
probesize
Run the probing
client
Storedata on
server side
Probingstream
To the next run
1 Back-to-back probe2 Pathchirp probe3 IGI probe * 5 gaps Feedback
var psize; // random [100, 1000]var probesize; // random [50, 350]
// open TCP client to remote IP and portvar client = new Client( rip, rport);
// run back-to-back probe// … and wait for RX throughputrunBackToBackTX( psize, probesize); var thru = client->waitForStatus();
// pathchirp probevar low = 0.5 * thru;var high = 5 * thru;var alpha = pickAlpha( low, high);runPathChirpTX( psize, low, high, alpha);
// IGI probevar low = 0.5 * thru;var high = 1.5 * thru;var step = ( high - low) / 5;for ( var rate = low; rate <= high; rate += step) {
runIgiTX( psize, probesize, rate);}
M.Zhanikeev -- [email protected] -- E2E Network Performance Estimation Using Signal Complexity -- http://tinyurl.com/kyutech131113 --- 15/19...
15/19
.
Results (1)• result: 0.8 is better on average, but adaptive is the best!• performance varies with network distance
30 50 70 90Brute AB
30
50
70
90
Estim
ated
AB pathchirp
igiigi
adaptive0.8 0.8
P120 40 60 80
Brute AB
20
40
60
80
100
Estim
ated
AB
pathchirpigi
adaptive
0.8 0.8
P210 30 50 70 90
Brute AB
1030507090
Estim
ated
AB
pathchirpigi
igiadaptive
adaptive0.80.8
P3
56 70 84 98AB
0102030
40
Erro
r
pathchirpigi
igi
adaptive0.8 0.8
30 50 70 90AB
0
20
4050
Erro
r
pathchirpigi
igi
adaptive
adaptive
0.8
0.8
30 50 70 90AB
40
60
Erro
r
pathchirp
pathchirp
igi
igi
adaptive adaptive
0.8
0.8
M.Zhanikeev -- [email protected] -- E2E Network Performance Estimation Using Signal Complexity -- http://tinyurl.com/kyutech131113 --- 16/19...
16/19
.
Results (2) : CCF Analysis
P vs
Brute AB
Brute AB vs
Brute PE
Brute AB vs
Pathchirp AB
Brute AB vs
IGI AB
-0.6
-0.2
0.2
0.6
CC
F
-0.3
-0.58
0.40.53
M.Zhanikeev -- [email protected] -- E2E Network Performance Estimation Using Signal Complexity -- http://tinyurl.com/kyutech131113 --- 17/19...
17/19
.
Wrapup
• PE helps by
1. using dumb probes with simple designs2. reducing analysis to complexity of binary response
• the method can be used for any metric, not just AB◦ jitter◦ tomography (sensing of topology)◦ load or utilization of a system◦ etc....
• will cover all these in future work ...
M.Zhanikeev -- [email protected] -- E2E Network Performance Estimation Using Signal Complexity -- http://tinyurl.com/kyutech131113 --- 18/19...
18/19
.
That’s all, thank you ...
M.Zhanikeev -- [email protected] -- E2E Network Performance Estimation Using Signal Complexity -- http://tinyurl.com/kyutech131113 --- 19/19...
19/19
.
[01] myself (2013)e2eprobe: source code of probing methodshttps://github.com/maratishe/e2eprobe
[02] myself+1 (2012)Modeling Network Performance of End HostsIEICE Trans. vol.E95-D, no.7
[03] 1+myself+1 (2006)Rate-Based and Gap-Based Available Bandwidth Estimation...Springer LNCS, vol.4238
[04] 1+myself+1 (2006)ABshoot: A Reliable and Efficient Scheme for End-to-End AB...IEEE TENCON
[05] 1+M.Zhanikeev (2009)Active Network Measurement: Theory, Methods, and ToolsITU Japan
M.Zhanikeev -- [email protected] -- E2E Network Performance Estimation Using Signal Complexity -- http://tinyurl.com/kyutech131113 --- 19/19...
19/19
.
[06] M.Jain+1 (2003)End-to-End Available Bandwidth: Dynamics...IEEE/ACM Trans., vol.11
[07] N.Hu+1 (2003)Evaluation and Characterization of AB ... TechniquesIEEE JSAC, vol.21, no.6
[08] V.Ribeiro+4 (2003)pathChirp: Efficient Available Bandwidth Estimation for PathsPAM Workshop
[09] C.Bandt+1 (2002)Permutation Entropy: A Natural Complexity Measure for Time SeriesPhysical Review Letters, Vol.88(17)
[10] A.Brandmaier (2012)Permutation Distribution Clustering and Structural Equation Model TreesPhD Diss.
M.Zhanikeev -- [email protected] -- E2E Network Performance Estimation Using Signal Complexity -- http://tinyurl.com/kyutech131113 --- 19/19...
19/19
.
[11] C.Dovrolis+2 (2001)What do packet dispersion techniques measure?INFOCOM, vol.2
[13] J.Strauss+2 (20s03)A Measurement Study of Available Bandwidth Estimation Tools3rd ACM SIGCOMM
M.Zhanikeev -- [email protected] -- E2E Network Performance Estimation Using Signal Complexity -- http://tinyurl.com/kyutech131113 --- 19/19...
19/19