LHC Tier 2 Networking BOF Joe Metzger [email protected] Joint Techs Vancouver 2005.
ESnet Abilene 3+3 Measurements Presented at the Joint Techs Meeting in Columbus July 19 th 2004 Joe...
-
Upload
julius-lucas -
Category
Documents
-
view
217 -
download
2
Transcript of ESnet Abilene 3+3 Measurements Presented at the Joint Techs Meeting in Columbus July 19 th 2004 Joe...
ESnet Abilene 3+3 ESnet Abilene 3+3 MeasurementsMeasurements
Presented at the Joint Techs Meeting in Presented at the Joint Techs Meeting in ColumbusColumbus
July 19July 19thth 2004 2004
Joe MetzgerJoe Metzger
ESnet Network EngineerESnet Network Engineer
[email protected]@es.net
CollaboratorsCollaborators
Chin Guok, Bill Johnston & Kevin Oberman@ Chin Guok, Bill Johnston & Kevin Oberman@ ESnetESnet
Chintan Desai & John Moore @ NCSU Chintan Desai & John Moore @ NCSU Darryl Wohlt & Phil Demar @ FERMIDarryl Wohlt & Phil Demar @ FERMI Jeff Boote, Eric Boyd & Guy Almes @ Jeff Boote, Eric Boyd & Guy Almes @
Internet2Internet2 Jin Guojun @ LBL Jin Guojun @ LBL Kevin Walsh @ SDSC Kevin Walsh @ SDSC Prasad Calyam @ OSU / OARnet Prasad Calyam @ OSU / OARnet
3+3 Measurements3+3 Measurements
3 ESnet Sites3 ESnet Sites LBLLBL FERMIFERMI BNLBNL
3 Abilene 3 Abilene ParticipantsParticipants SDSCSDSC NCSUNCSU OSUOSU
Abilene and ESnet have systems in place to measure our portions of the net. We were not measuring performance across our interconnections.
Why?Why?
We want to ensure that the We want to ensure that the ESnet/Abilene cross connects are ESnet/Abilene cross connects are serving the needs of users in the serving the needs of users in the science community who are science community who are accessing DOE facilities and accessing DOE facilities and resources from universities or resources from universities or accessing university facilities from accessing university facilities from DOE labs.DOE labs.
Existing Measurement Existing Measurement Collections were not meeting Collections were not meeting
our needs.our needs. ESnet and Abilene monitor traffic, errors and ESnet and Abilene monitor traffic, errors and
discards on all of our respective links including discards on all of our respective links including interconnection points.interconnection points.
This monitoring shows interconnections are This monitoring shows interconnections are lightly loaded and error free.lightly loaded and error free.
Other measurement systems we looked at do Other measurement systems we looked at do not contain the mix of ESnet and Abilene sites not contain the mix of ESnet and Abilene sites we are looking for.we are looking for.
Why Start with Latency Why Start with Latency Testing?Testing?
Low ImpactLow Impact Sensitive to network eventsSensitive to network events
What is OWAMPWhat is OWAMP
One Way Measurement ProtocolOne Way Measurement Protocol http://www.ietf.org/internet-drafts/draft-ietf-ippm-owdp-http://www.ietf.org/internet-drafts/draft-ietf-ippm-owdp-
08.txt08.txt
A suite of toolsA suite of tools http://e2epi.internet2.edu/owamp/http://e2epi.internet2.edu/owamp/
Data VisualizationData Visualization
What is the best way to display latency What is the best way to display latency data?data? It is difficult to identify trends in numeric It is difficult to identify trends in numeric
tables.tables. What is interesting or meaningful?What is interesting or meaningful?
Mean?Mean? No.No.
Median and 95Median and 95thth percentiles? percentiles? Maybe…Maybe…
Distribution?Distribution? Yes!Yes!
SmokePing by Tobias SmokePing by Tobias OetikerOetiker
http://www.smokeping.orghttp://www.smokeping.org Shows distribution of latency Shows distribution of latency
measurements.measurements.
Our VisualizationsOur Visualizations
Extended SmokePing graphic design to include Extended SmokePing graphic design to include multiple data sets on one graph.multiple data sets on one graph. SmokePing uses 20 shades of gray and plots one data SmokePing uses 20 shades of gray and plots one data
set on a graph.set on a graph. We are using a different color for each data set on a We are using a different color for each data set on a
graph. graph. We are using different saturations of the colors to show We are using different saturations of the colors to show
the distribution of results.the distribution of results. Show NTP error estimates.Show NTP error estimates. Graphs implemented as RRD templates to leverage Graphs implemented as RRD templates to leverage
existing ESnet statistics tools & infrastructure.existing ESnet statistics tools & infrastructure. Do not show loss information at this time.Do not show loss information at this time.
Red: FERMI to LBLRed: FERMI to LBLBlue: LBL to FERMIBlue: LBL to FERMI
Green NTP Error Estimates
A couple packets experienced queuing delays
Clock Event
Measurement ServersMeasurement Servers
LBL and NCSU were ready in late LBL and NCSU were ready in late AprilApril
FNAL and SDSC in MayFNAL and SDSC in May OSU in JuneOSU in June BNL in JulyBNL in July
Interesting ObservationsInteresting Observations
NTP Error Estimate QualityNTP Error Estimate Quality NCSU Metro DWDM RerouteNCSU Metro DWDM Reroute Queuing caused by bandwidth Queuing caused by bandwidth
testingtesting Asymetric RoutingAsymetric Routing
NCSU Metro DWDM RerouteNCSU Metro DWDM RerouteAdds about 350 Micro SecondsAdds about 350 Micro Seconds
Fiber Re-Route
Bandwidth Tests Can Cause Bandwidth Tests Can Cause Queuing on Bottleneck LinksQueuing on Bottleneck Links
Test Traffic Rerouted Tuesday Morning
Large Data Transfers
Asymetric Routing betweenAsymetric Routing betweenFERMI and SDSCFERMI and SDSC
(LBL to CENIC link Maintenance)(LBL to CENIC link Maintenance)
Future DirectionFuture Direction
Utilize a generalized, interoperable Utilize a generalized, interoperable measurement collection and measurement collection and archiving system instead of current archiving system instead of current ad-hoc scripts.ad-hoc scripts.
Look carefully at implementing Look carefully at implementing bandwidth testing, perhaps using bandwidth testing, perhaps using Scavenger QOS.Scavenger QOS.
ConclusionsConclusions
The ESnet/Abilene interconnections The ESnet/Abilene interconnections are not the bottlenecks on the tested are not the bottlenecks on the tested paths.paths.
Latency data can show interesting Latency data can show interesting queuing effects that may not be queuing effects that may not be obvious in other measurements.obvious in other measurements.
A single user with a $5K box can A single user with a $5K box can congest many current access links.congest many current access links.