Performance Debugging Measurements Overview
description
Transcript of Performance Debugging Measurements Overview
2004-Mar-15 2Performance Debugging Measurements
Really
Measurement project collaboration in the US that I am aware of
Two recent events•Performance (Debugging) Measurement Architecture Workshop (December)
•Ongoing work by US Federal R&E nets + Abilene to cooperate (Joint Engineering Team) – workshop upcoming in April
2004-Mar-15 3Performance Debugging Measurements
More Specifically
Working with DANTE, TF-NGN, UCL, GGF, NLANR (DAST on Advisor; MNA on AMP/PMA), and knowing a bunch of more basic research projects… widen the conversation on how we can work more cooperatively?
Is there an overall plan/architecture?
What’s common, what’s missing?
2004-Mar-15 4Performance Debugging Measurements
P(d)MA 2003
Performance (Debugging) Measurement Architecture Workshop 2003
1.75 day workshop last December
Bring together a bunch of NSF-funded projects
Including a related one: end-to-end middleware diagnostics (start with authorization and authentication, but can expand)
And some network-specific [BB, Campus]
2004-Mar-15 5Performance Debugging Measurements
Not a new idea
“Challenge: evolve the field of Internet measurement from a set of disjoint, independent activities of network researchers, operators, and users toward an integrated service of the network that can support diagnosis of problems in real-time.” – kc claffy, ed. “Measurement Roadmap” from the NREN/NASA workshop in 2000 on gigabit networking
2004-Mar-15 6Performance Debugging Measurements
Our leverage
Projects specifically about the integration
R&E backbone networks are interested in supporting
A few big demanding users (e.g. HENP) with diverse endpoints
2004-Mar-15 7Performance Debugging Measurements
Existing Architectures
Grid Measurement
piPEs
TF-NGN Performance Monitoring Group
Internet2 Middleware End-to-End Diagnostics
2004-Mar-15 8Performance Debugging Measurements
Internet2 Middleware E2E
Federated Security – Shibboleth; Authenticate locally, Authorize globally
Based on trust of institutional authentication
Pass minimum amount of information necessary
If failure• What piece of the distributed infrastructure is to blame?
• Want to debug without releasing private information
Diagnostic backplane with common event record
2004-Mar-15 9Performance Debugging Measurements
Current Projects
CalNGI / NPACI• Accessible to researchers
NIMI• Authentication focus• Experiments
SLAC• pinger, iepm-bw• netflow {local}
NLANR DAST: Advisor
NLANR MNA• AMP, PMA {oc192 soon}
piPEs, as implemented
TeraGrid (SAMI)
Esnet Perf. Monitring
LBL: NTAF, SCNM
IU: tools for campuses• Flowscan, gcube, L2TP
Mathis: NPAD
2004-Mar-15 10Performance Debugging Measurements
Backbones
Abilene Observatory, piPEs in Abilene• Ongoing measurements, openly accessable; on-demand with piPEs for throughput (bwctl) or latency (owamp)
TeraGrid• Ongoing measurements
ESnet• On-demand for ESnet sites
(Ongoing work on GEANT)
2004-Mar-15 11Performance Debugging Measurements
Wide-ranging Projects
pinger, iepm-bw•Ongoing active
AMP•Ongoing active
NIMI•Active, but focussed on experiments
(pipEs)•Ongoing active
2004-Mar-15 12Performance Debugging Measurements
Drawing conclusions
Advisor (based on iperf, Web100)• Intelligence at ends
Rich Carlson’s NDT• Intelligence at middle (packet streams, Web100 at Server)
piPEs provide data to others, Rich Carlson is working on the piPEs analysis component
2004-Mar-15 13Performance Debugging Measurements
New tools
Most of the projects that integrate use existing tools
•Ping, tracroute, iperf (or similar)•SNMP (some)
Some new tools arriving•Capacity discovery, lower-overhead throughput testing
•Web100-style host instrumentation
2004-Mar-15 14Performance Debugging Measurements
Network Path & Application Diagnostic
NPAD – New tool(set) under development at PSC, Matt Mathis
•Capitalize on Web100, but look for flaws in link layers
–Tweak TCP on short runs, detect problems that would show up on long runs [drops, reordering]: hits network harder than UDP tests, but doesn’t drive into congestion
–Find bugs in campus infrastructure
•Way to bench-test applications via packet reflection
2004-Mar-15 15Performance Debugging Measurements
Plug: Abilene Observatory
Publish measurement data• Some we collect for operations• Some we collect for research
The ability for research projects to add their equipment, or run on our platform [limited]
• Peer reviewed• Why? Passive, collocation makes analysis easier
• AMP, PMA, Planetlab [prefer run own code here]
2004-Mar-15 16Performance Debugging Measurements
Measurement Data
One way latency, jitter, loss [owamp]• IPv4 and IPv6
Regular TCP/UDP throughput tests – ~1 Gbps• IPv4 and IPv6; On-demand available [bwctl]
SNMP (NOC) [octets, packets, errors; collected frequently]
• NOC working on SNMP proxy
“Netflow” (ITEC Ohio) [anon. by 0-ing last 11 bits]
Multicast beacon with historical data
Routing data (BGP & IGP) [IGP under development]• Looking at Zebra + mods, Japanese routing research is
driver
2004-Mar-15 17Performance Debugging Measurements
“Visual Backbone”
XML access of 1/hr snapshots• Lots of show commands• Show config, isis, bgp, interface, …• Multicast stuff• http://loadrunner.uits.iu.edu/~gcbrowni/Abilene/ • MonaLisa uses this data! http://vinci.cacr.caltech.edu:8080/
Router proxy: ask questions of router: safely, rate controlled
2004-Mar-15 18Performance Debugging Measurements
My view of results
People want to work together, but they also want autonomy (don’t constrain possible solutions by architecture)
Want ways to share results, enable cross-framework testing (but not forced)
Tools folks would like to use existing infrastructures
2004-Mar-15 19Performance Debugging Measurements
My view of results
Common Themes•Discovery (domains, boxes along path)• Interfaces (requests, response)•Authorization, Authentication•Calibration, Meta-data• Infra. Operational issues•Algebra for metrics•How know physical topology [ not common, but big discussion!]
DISPLAY / ANALYSIS
TOOLS
Too
l Ce
rtifi
catio
n
Fed
era
tio
n R
ule
s
Fed
era
tio
n R
ule
s
Framework
Tool Wrapper Library: + CMD + Output + Logs(publish to encouragecommonality
History Real-Time On Demand Interrupt
Discovery Test R/R AAASchedulingResult
R/RTest
Availability
Database Cache Requestor
Scheduler On Demand Continuous Passive
Wrappers
Wrappers
Policy(MDI)
Schedule(PMC)
Invoke(PMP)
Optional:
Who are you?What do you want to do?
2004-Mar-15 21Performance Debugging Measurements
Reference for Workshop
http://e2epi.internet2.edu/WK03/
But… we are still working on a report.
2004-Mar-15 22Performance Debugging Measurements
Summary
Taste of what’s going on (US-centrically)
R&E networks are planning or implementing measurement frameworks
Looking to interoperate
New tools for segments, correlation on horizion
You are an important customer, friendly partner
2004-Mar-15 23Performance Debugging Measurements
Summary
Lots of details (and hard problems!) left
Interoperable federated(?) authentication
Finding the right points
An algebra for measurements
What can we do for “lightpaths” (in part, build measurements/reporting into layer 2 and expose them) (in part, endpoint cooperation)
2004-Mar-15 24Performance Debugging Measurements
Acknowledgements
NSF• This material is based in part on work supported by the National
Science Foundation under Grant No. ANI-0314723. Any opinions, findings and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation (NSF).
San Diego Supercomputer Center
CAIDA
Internet2, especially the End-to-End Performance Initiative
All the participants!
www.internet2.edu
2004-Mar-15 26Performance Debugging Measurements
Abilene URLs
http://abilene.internet2.edu/observatory • Pointers to all measurements/sites/projects
http://www.abilene.iu.edu/ • NOC home page. Weathermap, Proxy, SNMP measurements
http://netflow.internet2.edu/weekly/ • Summarized flow data
http://www.itec.oar.net/abilene-netflow/• “Raw” – matrices; (Anon) feeds available on request