Post on 09-Dec-2014
description
April 10, 2023
E2E piPEs / AMI / OWAMP StatusE2E piPEs / AMI / OWAMP Status
Eric Boyd and Jeff Boote
April 10, 2003
Eric Boyd and Jeff Boote
April 10, 2003
04/10/23 2
Agenda
• Motivation
• piPEs / AMI Overview
• piPEs / AMI Deployment Plan
• OWAMP Release
• Demo
04/10/23 3
Context for E2E Performance
Even with high bandwidth network links, the Internet2 community often does not see expected performance.
04/10/23 4
The Wizard Gap
04/10/23 5
A Problem
Applications Developer
System Administrator
LAN Administrator
CampusNetworking
Gigapop Gigapop
Backbone
CampusNetworking
LAN Administrator
System Administrator
Applications Developer
How do you solvea problem along a path?
Hey, this is not working right!
The computerIs working OK
Talk to the other guys
Everything isAOK
No othercomplaints
The network is lightly loaded
All the lights are green
We don’t see anything wrong
Looks fine
Others are getting in ok
Not our problem
04/10/23 6
Good Work Already Going On
Many projects and research is underway by others investigating performance on portions of the end-to-end path.
E2Epi seeks to bring these efforts together to look at the end-to-end path as a complete system.
04/10/23 7
Can You Go Direct to the Problem?
How can you tell where is the problem?
Need a tool to tell you:• Where the problem is.• The type of problem• Who to contact to get it fixed
Terry Gray, University of Washington“We Need a Finger Pointing Tool”
04/10/23 8
Gray Finger Pointing Tool
Applications Developer
System Administrator
LAN Administrator
CampusNetworking
Gigapop Gigapop
Backbone
CampusNetworking
LAN Administrator
System Administrator
Applications Developer
Locate the ProblemGray Finger Pointing ToolTerry GrayUniversity of Washington
Finger Pointing Tool Time!
04/10/23 9
Gray Finger Pointing Tool
Applications Developer
System Administrator
LAN Administrator
CampusNetworking
Gigapop Gigapop
Backbone
CampusNetworking
LAN Administrator
System Administrator
Applications Developer
Finger Pointing Tool Time
You ARE the Weakest Link!
OK, I’ll fix it
04/10/23 10
Agenda
• Motivation
• piPEs / AMI Overview
• piPEs / AMI Deployment Plan
• OWAMP Release
• Demo
04/10/23 11
BNI AMI
• Project: BNI Abilene Measurement Infrastructure
• Goal: Instrument Next Generation Abilene network with extensive performance measurement capabilities
• Approach: Collaborative project among BNI, Engineering, and E2E.
04/10/23 12
E2E piPES
• Project: End-to-End Performance Initiative Performance Environment System (E2E piPES)
• Goal: To allow end-users and network operators to determine performance capabilities, locate problems, and contact the right person to get a problem resolved.
• Approach: Collaborative project combining the best work of many organizations.
04/10/23 13
The Measurement System
PMP
PMP
PMP
PMP at each Campus Border
Extend the System to Campuses
AbilenePMP
Campus X
PMP
Campus Y
04/10/23 14
The Measurement System
PMP
PMP
PMP
PMP = Performance Measurement Point (at each Abilene Node)
The New Abilene will have measurement devices as part of its structure
Abilene
04/10/23 15
Performance Reporting
PMP
PMP
PMP
Perform regular tests to provide web based Performance Reports across Abilene and for Campus-Abilene paths
AbilenePMP
Campus X
PMP
Campus YCampus-Abilene CapabilityCampus W – 155 MbpsCampus X – 612 MbpsCampus Y – 1000 MbpsCampus Z – 2 Mbps
Abilene CapabilitySNVA/DNVR – 10 GbpsDNVR/IPLS – 10 GbpsIPLS/NYCM – 10 Gbps
04/10/23 16
Reaching the End-User
PMP
PMP
PMP
Provide a Testing Tool for the End-Users Computer to test across campus and beyond
AbilenePMP
Campus X
PMP
Campus Y
Host A Host B
The tool reports performance of the path to a specified destination by using active tests (like to the campus border) or using stored test results from regularly scheduled tests.
It also performs tests on the host system to check host tuning parameters
04/10/23 17
Adding Path Information
PMP
PMP
PMP
Add PMP along paths to provide additional detail
AbilenePMP
Campus X
PMP
Campus Y
Host A Host B
Regular tests are performed between adjacent PMP and stored in a Data Repository.This information is used by the End-User tool to reduce the number or active tests needed.
PMP
PMP
RegionalNetwork J
PMP
PMP
RegionalNetwork M
Gigapop S
Gigapop T
04/10/23 18
Who You Gonna Call?
PMP
PMP
PMP
E2E piPES will provide contact information when asegment in the path appears to have a problem
AbilenePMP
Campus X
PMP
Campus Y
Host A Host B
The test results obtained by the End-User will be passed on to the contact person
PMP
PMP
RegionalNetwork J
PMP
PMP
RegionalNetwork M
Gigapop S
Gigapop T
piPES Test AnalysisLooks like a problemIn Gigapop T. Pass these test result to:Joe@gigapopT.net
04/10/23 19
E2E piPES Architecture
End UserInterface
Network AdministratorInterface
PMPX
PMPY
PMPZ
CulpritDB
“Turn Yourself In”Culprit Interface
ID Self
CheckShibboleth
QueryCulprit
IDCulprit
Report “Raw” Results
Specify end points &application family
Request TestResult
Check Policy
Schedule Test
Scheduler
Performance MeasurementDB (could be local to PMP)
TestingEngine
AnalysisEngine
DispatchSchedule
Send RawResult/Refusal
Specify Test &End Nodes
Request On-Demand Test
TestArbiter
CheckShibboleth
SendConclusion
ResultArbiter Check
ShibbolethAck/Reject
ResultReady
04/10/23 20
Testing / Analysis Engine
Problem: Encode Matt Z.’s Brain
Open Questions:
•What are measurement results are “acceptable” for a given application (family)?
•What tools generate those results?
•How do you handle incomplete data?
•How do you rank multiple result-generators?
•What is the iterative decision tree to understand E2E problem?
04/10/23 21
Measurement Schema
Measurement Types?
e.g. BW, Latency, Loss, Jitter
Measurement Units?
e.g. Seconds vs. Microseconds
Map Tools to Measurements?
e.g. OWAMP -> 1-way Latency
Measurement Metadata?
Database Table Design?
04/10/23 22
Access, Authentication, Authorization
Roles for Access• Standard End-User?• Near Neighbor (Test Buddy)?• NOC Staff / Network Engineer?
Shibboleth for Implementation• Each campus decides who in each role• Others trust campus designation
04/10/23 23
Other R&D Issues
Design system to scale (eliminate centralization)?
How to “discover” PMPs?
How to insert On-Demand tests into Regularly-Scheduled test set
Balance centralization and distributed database requirements
04/10/23 24
Agenda
• Motivation
• piPEs / AMI Overview
• piPEs / AMI Deployment Plan
• OWAMP Release
• Demo
04/10/23 25
E2E piPES Architecture v1.0
End UserInterface
Network AdministratorInterface
PMPX
PMPY
PMPZ
CulpritDB
“Turn Yourself In”Culprit Interface
ID Self
CheckShibboleth
QueryCulprit
IDCulprit
Report “Raw” Results
Specify end points &application family
Request TestResult
Check Policy
Schedule Test
Scheduler
Performance MeasurementDB (could be local to PMP)
TestingEngine
AnalysisEngine
DispatchSchedule
Send RawResult/Refusal
Specify Test &End Nodes
Request On-Demand Test
TestArbiter
CheckShibboleth
SendConclusion
ResultArbiter Check
ShibbolethAck/Reject
ResultReady
04/10/23 26
piPEs / AMI Rollout
AMIPMP
AMIPMP
AMIPMP
OWAMPPERF
DB
Visualization Package
OWAMP OWAMP
OWAMP
04/10/23 27
piPEs / AMI Rollout
AMIPMP
AMIPMP
AMIPMP
OWAMPPERF
DB
Visualization Package
OWAMP OWAMP
OWAMP
Web Service
04/10/23 28
piPEs / AMI Rollout
AMIPMP
AMIPMP
AMIPMP
OWAMPPERF
DB
Visualization Package
OWAMPIPERF
TracerouteSNMP
OWAMPIPERF
TracerouteSNMP
OWAMP, IPERF, Traceroute, SNMP
Web Service
TraceroutePERF
DB
IPERFPERF
DB
SNMPPERF
DB
04/10/23 29
piPEs / AMI Rollout
AMIPMP
AMIPMP
AMIPMP
OWAMPPERF
DB
Visualization Package
OWAMPIPERF
TracerouteSNMP
OWAMPIPERF
TracerouteSNMP
OWAMP, IPERF, Traceroute, SNMP
Web Service
TraceroutePERF
DB
IPERFPERF
DB
SNMPPERF
DB
CampusPMP
CampusPMP
04/10/23 30
piPEs / AMI Rollout
AMIPMP
AMIPMP
AMIPMP
OWAMPPERF
DB
Visualization Package
OWAMPIPERF
TracerouteSNMP
OWAMPIPERF
TracerouteSNMP
OWAMP, IPERF, Traceroute, SNMP
Web Service
TraceroutePERF
DB
IPERFPERF
DB
SNMPPERF
DB
CampusPMP
CampusPMP
Scheduler
Web Service
Configuration Package
04/10/23 31
piPEs / AMI Development & Testing Environment
nmsXServer
Server
nmsXServer
Development
TestingTests Tests
New Code
Base Production Code
MeasurementNode
New Production Code
MasterMeasurementNode
MeasurementNode
MeasurementNode
MeasurementNode
AbileneMeasurementNodes
. . . . . . . . . . . . . . . . . . . . . .
Base/New Production Code
Tests Tests Tests
Server
Test Results
DatabaseServer
PerformanceVisualization Tool
MeasurementNode
Router
Switch nms1
nms3 nms2nms4
CDMA
04/10/23 32
piPEs / AMI Internal Resources
Eric Boyd – piPEs development, project coordination
Jeff Boote – OWAMP, piPEs development
Prasad Calyam – piPEs development
Chris Heermann – AMI
Matt Zekauskas - AMI
Susan Evett - Documentation
Russ Hobby – Campus Deployment
George Brett – Schema, “Grid Service”
04/10/23 33
piPEs External Resources
Yee-Ting Li – UCL (50%)
Paul Mealor – UCL (up to 100%)
PhD Student – UCL (contingent on grant)
Warren Matthews – SLAC
Jim Ferguson et al – NLANR / DAST
Many early adopter volunteers
04/10/23 34
Agenda
• Motivation
• piPEs / AMI Overview
• piPEs / AMI Deployment Plan
• OWAMP Release
• Demo
04/10/23 35
Latency Tests
Problem: Existing platforms are not interoperable (SURVEYOR,RIPE,…)
Solution: Standards
04/10/23 36
OWAMP
One-Way-Active-Measurement-Protocol• Possible due to growing availability of good time sources
• Wide deployment of “open” servers would allow measurement of one-way delay to become as commonplace as measurement of RTT using ICMP tools such as ping.
• Current Draft: draft-ietf-ippm-owdp-05.txt–Shalunov,Teitelbaum,Zekauskas
• Sample implementation under developmentBeta Release: http://owamp.internet2.edu/
04/10/23 37
OWAMP Implementation
Basically:NTP system call interface
Multiple processes for recv/send loops
Written as an API to allow one-off implementations
04/10/23 38
OWAMP Error Estimates
Rate = 1 packets/sec
Min/Median = 59/64 usec
0.04
0.06
0.08
0.1
0.12
0.14
0.16
0.18
0.2
0 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95
TO: 1/sec95% = 0.114
FROM: 1/sec95% = 0.106
04/10/23 39
OWAMP Error Estimates
Rate = 10 packets/sec
Min/Median = 57/63 usec
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100
TO: 10/sec95% = 0.080
FROM: 10/sec95% = 0.076
04/10/23 40
OWAMP Error Estimates
Rate = 100 packets/secMin/Median = 54/73 usec
0
0.5
1
1.5
2
2.5
3
3.5
0 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100
TO: 100/sec95% = 0.098
FROM: 100/sec95% = 0.094
04/10/23 41
OWAMP Error Estimates
Rate = 1000 packets/secMin/Median = 53/176 usec
0
0.5
1
1.5
2
2.5
3
3.5
4
4.5
0 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100
TO: 1000/sec95% = 0.294
FROM: 1000/sec95% = 0.279
04/10/23 42
OWAMP Errors
Preliminary Findings:• Min error estimates look to be in the 55-60 usec range.
• Serialization Delay: ~5usec x 2• Get Timestamp: ~15usec x 2• Additional error is:
–Time from userland “send” to 1st byte hits the wire–Time from kernel has packet to userland “recv” returns–Potentially recv process data processing before calling
“recv”
04/10/23 43
OWAMP implementation status
Sample implementation
http://owamp.internet2.edu/
• Beta Release:–No “policy”–No authentication/encryption–FreeBSD only
04/10/23 44
Abilene OWAMP deployment
2 overlapping full meshes (IPv4 & IPv6)• 11 measurement nodes = 220 ongoing tests• UDP singletons• Rate: 10 packets/second*• Packetsize: (32 byte payload)*• Results are continuously streamed back to “Measurement Portal” for long-term archive and data dissemination (Near real-time)
04/10/23 45
Agenda
• Motivation
• piPEs / AMI Overview
• piPEs / AMI Deployment Plan
• OWAMP Release
• Demo
04/10/23 46
piPEs / AMI / OWAMP Demo
http://owamp.internet2.edu
when available
04/10/23 47
More Internet2 Information
On the Web• www.internet2.edu• e2epi.internet2.edu
Email• Info-e2epi@internet2.edu
04/10/23 48
Questions?
www.internet2.edu