Methodology for Evaluating Encapsulated Beneficial Uses of ...
PingER: Methodology, Uses & Results
description
Transcript of PingER: Methodology, Uses & Results
1
PingER: Methodology, Uses & Results
Prepared by Les Cottrell, SLAC, for the Extending the Reach of Advanced Networking: Special International Workshop
Arlington, VA., April 22, 2004www.slac.stanford.edu/grp/scs/net/talk03/i2-method-apr04.ppt
Partially funded by DOE/MICS Field Work Proposal on Internet End-to-end Performance Monitoring (IEPM), also
supported by IUPAP
2
Outline• What is PingER
• World Internet performance trends
• Regions and Digital Divide
• Examples of use
• Challenges
• Summary of state of world Internet performance
3
Methodology
• Use ubiquitous ping
• Each 30 minutes from monitoring site to target : – 1 ping to prime caches– by default send11x100Byte pkts followed by
10x1000Byte pkts• Low network impact good for developing world
• Record loss & RTT, (+ reorders, duplicates)
• Derive throughput, jitter, unreachability …
4
Architecture
• Hierarchical vs. full mesh
WWWWWW
ArchiveArchive
MonitoringMonitoringMonitoringMonitoring MonitoringMonitoring
RemoteRemote
RemoteRemoteRemoteRemote
RemoteRemote
FNAL
Reports & Data
CacheMonitoringMonitoring
SLAC Ping
HTTP
ArchiveArchive
1 monitor hostremote host pair
5
PingER Benefits
• Provides quantitative historical (> 9yrs) and near real-time information– Aggregate by regions, affiliations etc.– How bad is performance to various
regions, rank countries?– Trends: who is catching up, falling behind,
is progress being made?– Compare vs. economic, development
indicators etc.• Use for trouble shooting setting
expectations, identify needed upgrades, choosing a provider, presenting to policy makers, funding bodies
Monitoring site vs. Remote sites screen shot
• Aimed at: end-user (net-admin & sophisticated user), planners • Measures analyzes & reports round-trip times, losses, availability,
throughput ...– Uses ubiquitous ping, no special host, or software to install/configure at remote
sites, no passwords or credentials needed– Low impact on network << 100bits/s, important for many DD sites– Covers 100+ countries (> 90% of Internet connected population)
6
Regions Monitored
• Recent added NIIT PK as monitoring site• White = no host monitored in country• Colors indicate regions• Also have affinity groups (VOs), e.g. AMPATH, Silk
Road, CMS, XIWT and can select multiple groups
7
World Trends• Increase in sites with Good (<1%) loss
• 25% increase in sites monitored– Big focus on Africa 4=>19 countries– Silk Road
Loss quality ratings seen from SLAC
0
50
100
150
200
250
300
Apr
-00
Jul-0
0
Oct
-00
Jan-
01
Apr
-01
Jul-0
1
Oct
-01
Jan-
02
Apr
-02
Jul-0
2
Oct
-02
Jan-
03
Apr
-03
Jul-0
3
Oct
-03
Nu
mb
er o
f si
tes
Dreadful >12%V. poor >=5% & <12%Poor >=2.5% & < 5%Acceptable >=1% & < 2.5%Good <1%
Ping blocking
50%
60%
WSISICTP
8
TrendsS.E. Europe, Russia: catching upLatin Am., Mid East, China: keeping upIndia, Africa: falling behind Derived throughput~MSS/(RTT*sqrt(loss))
9
Current State – Aug ‘03 thruput ~ MSS / (RTT * sqrt(loss))
• Within region performance better– E.g. Ca|EDU|GOV-NA, Hu-SE Eu, Eu-Eu, Jp-E Asia, Au-Au, Ru-Ru|
Baltics• Africa, Caucasus, Central & S. Asia all bad
Bad < 200kbits/s < DSL Poor > 200, < 500kbits/s
Acceptable > 500kbits/s, < 1000kbits/sGood > 1000kbits/s
10
Examples of Use• Need for constant upgrades• Upgrades• Filtering• Pakistan
11
Usage Examples
• Selecting ISPs for DSL/Cable services for home users– Monitor accessibility of routers etc. from site– Long term and changes
• Trouble shooting– Identifying problem reported is probably network related– Identify when it started and if still happening or fixed– Look for patterns:
• Step functions• Periodic behavior, e.g. due to congestion• Multiple sites with simultaneous problems, e.g. common problem link/router …
– Provide quantitative information to ISPs
Identify need to upgrade and effects
• BW increase by factor 300• Multiple sites track• Xmas & summer holiday
12
Russia
• E.g. Upgrade to KEK-BINP link from 128kbps to 512kbps, May ’02: improved from few % loss to ~0.1% loss
• Russian losses improved by factor 5 in last 2 years, due to multiple upgrades
13
Usage Examples
Median Packet Loss Seen From nbi.dk
0
5
10
15
20
25
30
35
40
45
50
11/1
/98
11/8
/98
11/1
5/98
11/2
2/98
11/2
9/98
12/6
/98
12/1
3/98
12/2
0/98
12/2
7/98
1/3/
99
1/10
/99
1/17
/99
1/24
/99
% 1
00 B
yte
Pac
ket
Lo
ss D
uri
ng
Day
.
Ten-155 became Ten-155 became operational on operational on December 11.December 11.
Smurf Filtersmurf Filtersinstalled oninstalled onNORDUnet’sNORDUnet’sUS connection.US connection.
To North America
To Western Europe
Packet Loss between DESY and FNAL in February and March 2000.
0
2
4
6
8
10
12
1 3 5 7 9 11 13 15 17 19 21 23 25 27 1 3 5 7 9 11 13 15 17 19
Day of the Month
Da
ily
Pa
ck
et
Lo
ss
(%
)
DFN closes Perryman POP and looses direct peering with ESnet
Peering re-established via Dante at 60 Hudson
February March
Peering problems, took long time identify/fix
Upgrades & ping filtering
14
Pakistan Conclusions• Big performance differences to sites, depend on ISP
(at least 3 ISPs seen for Pakistan A&R sites)• To NIIT:
– Get about 300Kbps, possibly 380Kbps at best – Verified bottleneck appeared to be in Pakistan – Requested upgrade to 1Mbps, and verified got it– There is often congestion (packet loss & extended RTTs)
during busy periods each weekday – Video will probably be sensitive to packet loss, so it may
depend on the time of day– H.323 (typically needs 384Kbps + 64Kbps), would appear to
be marginal at best at any time.
• No peering Pakistan between NIIT and NSC
15
Challenges• Effort:
– Negligible for remote hosts– Monitoring host: < 1 day to install and configure, occasional updates to remote
host tables and problem response – Archive host: 20% FTE, code stable, could do with upgrade, contact monitoring
sites whose data is inaccessible– Analysis: your decision, usually for long term details download & use Excel– Trouble-shooting:
• usually re-active, user reports, then look at PingER data• have played with automating alerts, data will/is available via web services
• Ping blocking– Complete block easy to ID, then contact site to try and by-pass, can be
frustrating for 3rd world– Partial blocks trickier, compare with synack
• Funding– “Unfortunately, network management research has historically been very under-
funded, because it is difficult to get funding bodies to recognize this as legitimate networking research.” Sally Floyd, IAB Concerns & Recommendations Regarding Internet Research & Evolution.
– http://www.ietf.org/internet-drafts/draft-iab-research-funding-00.txt
16
Summary
• Performance from U.S. & Europe is improving all over
• Performance to developed countries are orders of magnitude better than to developing countries
• Poorer regions 5-10 years behind• Poorest regions Africa, Caucasus, Central & S.
Asia• Some regions are:
– catching up (SE Europe, Russia), – keeping up (Latin America, Mid East, China), – falling further behind (e.g. India, Africa)
17
More Information• PingER:
– www-iepm.slac.stanford.edu/pinger/
• MonaLisa– monalisa.cacr.caltech.edu/
• GGF/NMWG– www-didc.lbl.gov/NMWG/
• ICFA/SCIC Network Monitoring report, Jan03– www.slac.stanford.edu/xorg/icfa/icfa-net-paper-dec02
• Monitoring the Digital Divide, CHEP03 paper– arxiv.org/ftp/physics/papers/0305/0305016.pdf
• Human Development Index– www.undp.org/hdr2003/pdf/hdr03_backmatter_2.pdf
• Network Readiness Index– www.weforum.org/site/homepublic.nsf/Content/Initiatives+subhome
18
Extra Slides
19
VisualizationKeep it simple, enable user to do their
own by making data available • Tables
– Time series (www-iepm.slac.stanford.edu/cgi-
wrap/pingtable.pl): • select metric (loss, RTT etc.), time ticks,
packet size, aggregations from/to, etc.
• Color code numbers, provide sort, drill down to graphs, download data (TSV), statistical summaries
– Monitoring site vs. Remote sites (www-
iepm.slac.stanford.edu/cgi-wrap/table.pl):• Select metric, region aggregations
• Drill down to time series, download data
• Graphs– Select source(s)/destination(s), metric,
time window, SQL selects, graph type
20
Publish information
#!/usr/bin/perl use SOAP::Lite; my $characteristic = SOAP::Lite -> service(‘http://www-iepm.slac.stanford.edu/tools/soap/wsdl/profile_06.wsdl') -> pathDelayOneWay("tt81.ripe.net:tt28.ripe.net”); print $characteristic->{NetworkTestTool}->{toolName},"\n"; print $characteristic->{NetworkPathDelayStatistics}->{value},"\n";
• www.slac.stanford.edu/cgi-wrap/pingtable.pl => tabular reports, also download data
• Data accessible from MonaLisa• Implementing web services
access prototype– Includes: PingER, IEPM-BE, RIPE-tt,
I2 E2Epi OWAMP– Use GGF/NMWG schema/profile,
e.g.• path.delay.roundTrip
21
Loss to world from US
2001 Dec-2003
In 2001 <20% of In 2001 <20% of the world’s the world’s population had population had Good or Good or Acceptable Loss Acceptable Loss performanceperformance
Loss RateLoss Rate< 0.1 to 1 %< 0.1 to 1 % 1 to 2.5 %1 to 2.5 % 2.5 to 5 %2.5 to 5 % 5 to 12 %5 to 12 % > 12 %> 12 %
BUT by December 2003BUT by December 2003It had improved to 77%It had improved to 77%
22
Rate LimitingRTT Loss
2 hosts at same site see sudden step-like increase in loss from < 1% to 20-30% at similar time
boromir.nask.waw.pl
gollum.nask.pl
www.pol34.pl Another host in Poland sees no problems, i.e. helps to have another nearby host
RT
T Los
s
Similar effects for Greek (uoa.gr), Bulgarian (acad.bg), Kazakhstan (president.kz), Moldovan (asm.md) and Turkish (metud.edu.tr) sites
If no step function or nearby host may not notice, so also compare synack vs ping
Can ping routers along path to see where onset occurs
At any given time, about 5% of monitored hosts are doing this, most in developing countries. Recently (August 2003) seen an increase in ping rate limiting
RTT Loss
boromir.nask.waw.pl
gollum.nask.pl
www.pol34.pl
RT
T Los
s
23
Rate Limiting Moldova
cni.md
lises.asm.md
RTT Loss
Moldova Bulgaria
24
VariabilityRegion Countries #
AfricaGhana, Namibia, Nigeria, Namibia, S. Africa, Uganda 6
C AsiaKazakhstan, Kyrghzstan, Mongolia, Tajikistan, Turkmenistan, Uzbekistan 9
S AsiaBangladesh, India, Indonesia, Malaysia, Pakistan, Thailand, (Vietnam) 16
M East Egypt, Iran, Israel, Jordan, Saudi Arabia, Turkey 10
Caucasus Armenia, Azerbaijan, Georgia 5
S AmericaArgentina, Brazil, Columbia, Chile, Peru, Uruguay, Venezuela 13
China China including Hong Kong 5
Russia Russia 5
C America Costa Rica, Cuba, Guatemala, Mexico 4
SE Europe(Albania), Bulgaria, Croatia, Macedonia, Moldova, Romania, Serbia/Montenegro, Slovenia 13
25
Africa• Hosts in: Ife-Ife/Nigeria, Accra/Ghana,
Kampala/Uganda, Windhoek/Namibia, UCT/ZA, Johannesburg/ZA, Musselbay/ZA
• Carriers:– GH uses UUNET/Satworks, NA uses
UUNET/xantic, NG uses TELIANET/NewSkies, UG uses Level(3)/globalconnex
– ZA varies from site to site: UUNET/ALTERNET, C&W Telecom S. Africa, CAIS telcom S. Africa
• UG, NA, NG, GH use satellites (> 600ms)• ZA uses landlines
26
Africa RTT• Monitored from N.
America & Europe– Depends on remote site
(not monitoring site)– Satellite for all except S.
Africa– Ghana problems
27
West Africa• Ghana very poor performance
– Sudden increase on August 18th– Not rate limiting according to synack– Sometimes get down to a few %– Route ESnet-UUNET/ALTER.NET– Losses appear on
last 2 hops in Ghana
• Nigeria better– Route via
TELIANET/newskies
28
Africa Derived Throughput
• S. Africa (UCT) best, followed by Uganda, Nigeria and Ghana
• Throughput to Nigeria site == home DSL/cable• Throughput to Ghana site === modem dialup
29
30
Human Development Index (HDI ) RankSource: UN
0
20
40
60
80
100
120
Uni
ted
Sta
tes
Cze
chN
ethe
rland
sF
inla
ndH
unga
ryA
ustr
iaS
pain
Sw
itzer
land
Slo
vaki
aD
enm
ark
Italy
Cro
atia
Irel
and
Pol
and
Icel
and
Gre
ece
Bel
gium
Tur
key
Slo
veni
aIr
an, I
slam
icLi
thua
nia
Est
onia
Rom
ania
Geo
rgia
Ukr
aine
Mol
dova
,A
lban
ia
HD
I
GDP per capitaSource: UN
0
5000
10000
15000
20000
25000
30000
35000
40000
Un
ited
Sta
tes
Cze
ch
Ne
the
rla
nd
s
Fin
lan
d
Hu
ng
ary
Au
stri
a
Sp
ain
Sw
itze
rla
nd
Slo
vaki
a
De
nm
ark
Ita
ly
Cro
atia
Ire
lan
d
Po
lan
dIc
ela
nd
Gre
ece
Be
lgiu
m
Tu
rke
y
Slo
ven
ia
Ira
n,
Isla
mic
Lith
ua
nia
Est
on
ia
Ro
ma
nia
Ge
org
ia
Ukr
ain
e
Mo
ldo
va,
Alb
an
ia
GD
P p
er C
apit
a (P
PP
US
$)
NREN Core Network Size (Mbps-km)
10M
1M
100K
10K
1K
100
2000
2001
Leading
Advanced
In transition
Source: From slide prepared by Harvey Newman, presented by David Source: From slide prepared by Harvey Newman, presented by David Williams ICFA/SCIC talk on Serenate report. Data from the TERENA Williams ICFA/SCIC talk on Serenate report. Data from the TERENA CompendiumCompendium
Lagging
Derived throughput~MSS/(RTT*sqrt(loss))
Europe
Netherlands
Turkey
Belgium
31
Loss Comparisons with Development (UNDP)
Even weaker with education & literacy
Weak correlation with Human Development or GDP
32
Digital Access Index• DAI (from ITU 2002)
includes:– Availability of
infrastructure– Affordability of access– Education level– Quality of ICT services– Internet usage
Top DAI countries Good positive correlation between
throughput and DAICare needed with shorter RTTs
34
Network Readiness
• NRI from Center for International Development, Harvard U. http://www.cid.harvard.edu/cr/pdf/gitrr2002_ch02.pdf
• Using derived throughput ~ MSS / (RTT * sqrt(loss))– Fit to exponential is better
Internet for all focusA
&R
focus
NRI Top 14Finland 5.92US 5.79Singapore 5.74Sweden 5.58Iceland 5.51Canada 5.44UK 5.35Denmark 5.33Taiwan 5.31Germany 5.29Netherlands 5.28Israel 5.22Switzerland 5.18Korea 5.10
35
Technology Achievement Index (TAI)
• TAI captures how well a country is creating and diffusing technology and building a human skills base.
• TAI from UNDP hdr.undp.org/reports/global/2001/en/pdf/techindex.pdf TAI top 12Finland 0.744US 0.733Sweden 0.703Japan 0.698Korea Rep. of 0.666Netherlands 0.630UK 0.606Canada 0.589Australia 0.587Singapore 0.585Germany 0.583Norway 0.579
US & Canada off-scale
36
Futures• Get >= 2 hosts/country• Better/quicker detection of rate limiting• Have 4 students at GATech rewriting parts of
PingER to improve (reduce effort required for) day to day management and improve portability
• Submitting a proposal to IDRC for monitoring Africa and adding a measurement host in Nigeria
• Need better automated tools to produce graphs like in this presentation.
37
• 35+ monitoring sites in 13 countries– Plan to add ICTP Trieste if funded – Other projects used toolkit, e.g. XIWT, PPCNG/EDG, IAEA …
• SLAC with help from FNAL• Digital Divide collaboration (MOU) with ICTP, Trieste
– eJDS– We are requesting an IDRC grant for eJDS and PingER
• Need funding for coming year (DoE funding ended):– Tasks:
• (0.5 FTE) ongoing maintain data collection, explain needs, reopen connections, open firewall blocks, find replacement hosts, make limited special analyses, prepare & make presentations, respond to questions
• (+ 0.5 FTE) extend the code for new environment (more countries, more data collections), fix known non-critical bugs, improve visualization, automate some of reports generated by hand today, find new country site contacts, add route histories and visualization, automate alarms, detect rate limiting earlier, update web site for better navigation, add more DD monitoring sites/countries, improve code portability, understand regions better
• ICFA: show importance to policy makers, funding agencies, identify sympathetic contacts at agencies, get support
Collaborations & Funding