Technical Challenges
description
Transcript of Technical Challenges
DoE SciDAC high-performance networking research project: INCITE INCITE.rice.edu 2004
Technical Challenges
INCITE R. Baraniuk, E. Knightly, R. Nowak, R. Riedi (Rice), L. Cottrell, J. Navratil (SLAC), W. Feng, M. Gardner (LANL)
INCITE:InterNet Control and
Inference Tools at the Edge
Impact and Connections
Edge-based Traffic Processing and Service Inference for High-Performance Networks
1
2
3
• Poor understanding of origins of complex network dynamics• Lack of adequate modeling techniques for network dynamics• Internal network inaccessible • Low impact, large scale monitoring • Application-driven traffic modulation • High-speed measurements
Objectives: Improve throughput over the Internet for DoE high performance projects
Thrust 1: Traffic analysis and modelingThrust 2: Path and tomographic inferenceThrust 3: Data collection tools
(PingER, MAGNeT, +)
Approach: Active and passive network probing Statistical model based inference
PingER/ABwE (SLAC)8• Many scientists are unable to participate in science due to poor Internet connectivity
•e.g. 10-20% of HENP collaborators are from developing nations• To understand need simple, low cost, performance measurements to and within developing regions providing:
The graphs show Abing monitoring data via MonALISA
Bandwidth
Tools: MAGNeT & TICKET (LANL) MAGNeT:
Monitor for Application-Generated Network Traffic
9
TICKET: Traffic Information-Collecting Kernel with Exact Timing Current solutions to network packet capture (e.g., tcpdump) are too slow or too expensive Monitor and record traffic at gigabit-per-second (Gb/s) speeds and nanosecond granularity
Network Tomography (Rice, Wisconsin)4
Chirp: packet train with increasing rate
When probe rate exceeds available bandwidth, queuing delay increases
Monitor traffic immediately after being generated by the application throughout the protocol stack to see how traffic gets modulated. Is TCP/IP the obstacle to high performance?
• planning, setting expectations, policy setting
• PingER meets these needs• < 100bits/s, uses ubiquitous ping• covers > 100 countries (>90% of world’s Internet connected population)
Pinger deploymentBlue=monitoring siteRed=remote site
ABwE tool: abing Characteristics• Interactive (1 – 2 second response)• Low network impact (20 packets/host/direction) • Simple & robust: just need simple responder installing• Provides measurements in both directions• Provides capacity & available bandwidth• Agrees with more intense/complex methods• Used in MonALISA, IEPM-BW & PlanetLab
pathChirp: Efficient Available Bandwidth and Tight Link Estimation (Rice)5
Available bandwidth estimates decrease in proportion to the introduced cross-traffic
Canonical Subproblems: Two senders/receivers problem characterizes network tomography problem in general
1-by-2 Component
2-by-1 Component
?From edge-based traffic measurements (loss/delay/arrival order), infer internal topology, link level loss rates, queuing delays1 1
1
2
3
4
5
6
1
2 3
4
Common Branch Point: Arrival order usually the same
Different Branch Points: arrival order varies depending on delays, offset
Arrival order fixed at joining point
ROC Curve
1000 probesLoss OnlyArrival Order OnlyArrival Order and Loss
Rice LAN
Arrival Order Based Topology ID
Impact: Optimize performance of demanding applications (remote visualization, high- capacity data transfers) New understanding of the complex dynamics of large-scale, high-speed networks New edge-based tools to characterize and map network performance as a function of space, time, resource, application, protocol, and service Highly efficient methods for monitoring in distributed computing systems.
Connections: Rice/SLAC/LANL synergy• Particle Physics Data Grid Collaboratory Pilot (Newman, Cottrell, Mount). • SciDAC Center for Supernova Research (Warren) • Scientific Workspaces of the Future (ANL, UIC, LANL, BU, Brown, NCSA).
Globus• Teragrid• Transpac at Indiana U.• European GridLab Project• San Diego Supercomputing Center• Telcordia• IEPM-BW• Internet2• ns-2 Simulator
UIUCRice tight link
SLACRice tight link
Reduce available bandwidth on Gigabit testbed using cross-traffic generator
Locating tightlinks on twopaths sharing4 common links
TCP Low-Priority (Rice)6Goal: Utilize excessive bandwidth in a non-intrusive fashionApplications: bulk data transfer, P2P file sharing
• TCP alone 745.5 Kb/s
• TCP plus 739.5 Kb/sTCP-LP 109.5 Kb/
• TCP-LP is invisible to TCP
High-speed TCP-LP•TCP-LP + HSTCP [Floyd03]
•Linux-2.4.22-web100 implementation
Alpha-Beta Traffic Model (Rice)7
Mean
99%= +
betaalphabytespertimeplots
• Cause of burstiness in traffic?• Alpha: cause bursts, large transfers, high rate, low RTT,
few connections• Beta: not-bursty, low rate, high RTT, most connections,
possess long-range-dependence
• Key: both application and network properties important for traffic modeling