Synthetic and Real World Web Performance Monitoring
-
Upload
verizon-digital-media-services -
Category
Technology
-
view
6.367 -
download
0
description
Transcript of Synthetic and Real World Web Performance Monitoring
![Page 1: Synthetic and Real World Web Performance Monitoring](https://reader035.fdocuments.in/reader035/viewer/2022062513/5565a6b7d8b42a083a8b4b84/html5/thumbnails/1.jpg)
EdgeCastPerformance Engineering Group
Third Party MonitoringUse Cases
Nathan Moore, Web Performance [email protected]
![Page 2: Synthetic and Real World Web Performance Monitoring](https://reader035.fdocuments.in/reader035/viewer/2022062513/5565a6b7d8b42a083a8b4b84/html5/thumbnails/2.jpg)
2850 OCEAN PARK BLVD, SUITE 110, SANTA MONICA CA
2
Analyze performance trends and come up with ideas to make the CDN faster
© 2013 EDGECAST NETWORKS
Web Performance Group
![Page 3: Synthetic and Real World Web Performance Monitoring](https://reader035.fdocuments.in/reader035/viewer/2022062513/5565a6b7d8b42a083a8b4b84/html5/thumbnails/3.jpg)
2850 OCEAN PARK BLVD, SUITE 110, SANTA MONICA CA
3
• Network Stack• Routing• Kernel• Application
© 2013 EDGECAST NETWORKS
Performance Optimizations
TCP Optimizations
![Page 4: Synthetic and Real World Web Performance Monitoring](https://reader035.fdocuments.in/reader035/viewer/2022062513/5565a6b7d8b42a083a8b4b84/html5/thumbnails/4.jpg)
4
• We monitor as much as we can to find these:
© 2013 EDGECAST NETWORKS2850 OCEAN PARK BLVD, SUITE 110, SANTA
MONICA CA
Performance Optimizations
Needles in haystacks
![Page 5: Synthetic and Real World Web Performance Monitoring](https://reader035.fdocuments.in/reader035/viewer/2022062513/5565a6b7d8b42a083a8b4b84/html5/thumbnails/5.jpg)
2850 OCEAN PARK BLVD, SUITE 110, SANTA MONICA CA
5
You can’t manage what you don’t measure
-- Deming-- Drucker
-- Traditional
© 2013 EDGECAST NETWORKS
Why monitor?
![Page 6: Synthetic and Real World Web Performance Monitoring](https://reader035.fdocuments.in/reader035/viewer/2022062513/5565a6b7d8b42a083a8b4b84/html5/thumbnails/6.jpg)
2850 OCEAN PARK BLVD, SUITE 110, SANTA MONICA CA
6
• Performance– Latency– Bandwidth
• Availability– Uptime
• Rate of Change of Internet– Snapshot of Internet as a whole
© 2013 EDGECAST NETWORKS
What can you monitor with Third Party Applications?
![Page 7: Synthetic and Real World Web Performance Monitoring](https://reader035.fdocuments.in/reader035/viewer/2022062513/5565a6b7d8b42a083a8b4b84/html5/thumbnails/7.jpg)
2850 OCEAN PARK BLVD, SUITE 110, SANTA MONICA CA
7
Web performance drives KPIs
© 2013 EDGECAST NETWORKS
Why care?
![Page 8: Synthetic and Real World Web Performance Monitoring](https://reader035.fdocuments.in/reader035/viewer/2022062513/5565a6b7d8b42a083a8b4b84/html5/thumbnails/8.jpg)
2850 OCEAN PARK BLVD, SUITE 110, SANTA MONICA CA
8
KPIs:
• Fast websites drive Sales and Engagement• Reliable websites drive Sales and Engagement
© 2013 EDGECAST NETWORKS
Why care?
![Page 9: Synthetic and Real World Web Performance Monitoring](https://reader035.fdocuments.in/reader035/viewer/2022062513/5565a6b7d8b42a083a8b4b84/html5/thumbnails/9.jpg)
2850 OCEAN PARK BLVD, SUITE 110, SANTA MONICA CA
9© 2013 EDGECAST NETWORKS
Not just a made-up statistic
http://sites.google.com/site/glinden/Home/StanfordDataMining.2006-11-28.ppt?attredirects=0
![Page 10: Synthetic and Real World Web Performance Monitoring](https://reader035.fdocuments.in/reader035/viewer/2022062513/5565a6b7d8b42a083a8b4b84/html5/thumbnails/10.jpg)
2850 OCEAN PARK BLVD, SUITE 110, SANTA MONICA CA
10
What you can measure, you can optimize
-- Ilya Grigorik
© 2013 EDGECAST NETWORKS
Why monitor?
![Page 11: Synthetic and Real World Web Performance Monitoring](https://reader035.fdocuments.in/reader035/viewer/2022062513/5565a6b7d8b42a083a8b4b84/html5/thumbnails/11.jpg)
2850 OCEAN PARK BLVD, SUITE 110, SANTA MONICA CA
11© 2013 EDGECAST NETWORKS
Simplified Internet Traffic Flow
![Page 12: Synthetic and Real World Web Performance Monitoring](https://reader035.fdocuments.in/reader035/viewer/2022062513/5565a6b7d8b42a083a8b4b84/html5/thumbnails/12.jpg)
2850 OCEAN PARK BLVD, SUITE 110, SANTA MONICA CA
12© 2013 EDGECAST NETWORKS
Categorizing the World
![Page 13: Synthetic and Real World Web Performance Monitoring](https://reader035.fdocuments.in/reader035/viewer/2022062513/5565a6b7d8b42a083a8b4b84/html5/thumbnails/13.jpg)
2850 OCEAN PARK BLVD, SUITE 110, SANTA MONICA CA
13© 2013 EDGECAST NETWORKS
Monitoring the World
![Page 14: Synthetic and Real World Web Performance Monitoring](https://reader035.fdocuments.in/reader035/viewer/2022062513/5565a6b7d8b42a083a8b4b84/html5/thumbnails/14.jpg)
2850 OCEAN PARK BLVD, SUITE 110, SANTA MONICA CA
14
Synthetic monitoring is needed to dynamically monitor the External
Internet
© 2013 EDGECAST NETWORKS
What are Third Party Monitors for
![Page 15: Synthetic and Real World Web Performance Monitoring](https://reader035.fdocuments.in/reader035/viewer/2022062513/5565a6b7d8b42a083a8b4b84/html5/thumbnails/15.jpg)
2850 OCEAN PARK BLVD, SUITE 110, SANTA MONICA CA
15© 2013 EDGECAST NETWORKS
Monitoring Hooks for Synthetic Monitoring
![Page 16: Synthetic and Real World Web Performance Monitoring](https://reader035.fdocuments.in/reader035/viewer/2022062513/5565a6b7d8b42a083a8b4b84/html5/thumbnails/16.jpg)
2850 OCEAN PARK BLVD, SUITE 110, SANTA MONICA CA
16© 2013 EDGECAST NETWORKS
Synthetic Monitoring
Synthetic monitoring is clean
• Small number of nodes• Well defined network• Highly reliable links between nodes and networks• Shows health of the backbone of the Internet
![Page 17: Synthetic and Real World Web Performance Monitoring](https://reader035.fdocuments.in/reader035/viewer/2022062513/5565a6b7d8b42a083a8b4b84/html5/thumbnails/17.jpg)
2850 OCEAN PARK BLVD, SUITE 110, SANTA MONICA CA
17© 2013 EDGECAST NETWORKS
State of the Internet
![Page 18: Synthetic and Real World Web Performance Monitoring](https://reader035.fdocuments.in/reader035/viewer/2022062513/5565a6b7d8b42a083a8b4b84/html5/thumbnails/18.jpg)
2850 OCEAN PARK BLVD, SUITE 110, SANTA MONICA CA
18
Real User Monitoring (RUM) is needed to
measure what your end users really see
© 2013 EDGECAST NETWORKS
What are Third Party Monitors For
![Page 19: Synthetic and Real World Web Performance Monitoring](https://reader035.fdocuments.in/reader035/viewer/2022062513/5565a6b7d8b42a083a8b4b84/html5/thumbnails/19.jpg)
2850 OCEAN PARK BLVD, SUITE 110, SANTA MONICA CA
19© 2013 EDGECAST NETWORKS
From Synthetic to Real User Metrics (RUM)
Reporting Server
CDN 1 CDN 2 CDN 3
![Page 20: Synthetic and Real World Web Performance Monitoring](https://reader035.fdocuments.in/reader035/viewer/2022062513/5565a6b7d8b42a083a8b4b84/html5/thumbnails/20.jpg)
2850 OCEAN PARK BLVD, SUITE 110, SANTA MONICA CA
20© 2013 EDGECAST NETWORKS
Real User Data / RUM
![Page 21: Synthetic and Real World Web Performance Monitoring](https://reader035.fdocuments.in/reader035/viewer/2022062513/5565a6b7d8b42a083a8b4b84/html5/thumbnails/21.jpg)
2850 OCEAN PARK BLVD, SUITE 110, SANTA MONICA CA
21© 2013 EDGECAST NETWORKS
Real User Metrics (RUM)
RUM is messy. Rum is the real observed data generated by real users on the real Internet on real web browsers
Combinatorics:• Hundreds of Millions of TCP connections• Tens of Millions of end users • Thousands of ISPs• Tens of backbone providers• One of your web servers
• Data Distribution issues:• Power law distributions => averages lose explanatory power• Quantiles and Cumulative Density Functions (CDF) gain importance• Aggregating data and mining for correlations becomes more difficult
![Page 22: Synthetic and Real World Web Performance Monitoring](https://reader035.fdocuments.in/reader035/viewer/2022062513/5565a6b7d8b42a083a8b4b84/html5/thumbnails/22.jpg)
2850 OCEAN PARK BLVD, SUITE 110, SANTA MONICA CA
22© 2013 EDGECAST NETWORKS
Real User Data / RUM benefits
Data aggregated across country and displayed across world
![Page 23: Synthetic and Real World Web Performance Monitoring](https://reader035.fdocuments.in/reader035/viewer/2022062513/5565a6b7d8b42a083a8b4b84/html5/thumbnails/23.jpg)
2850 OCEAN PARK BLVD, SUITE 110, SANTA MONICA CA
23© 2013 EDGECAST NETWORKS
Use Case 1:What if you only have synthetic monitors?
New platform launch:
EdgeCastTransact
![Page 24: Synthetic and Real World Web Performance Monitoring](https://reader035.fdocuments.in/reader035/viewer/2022062513/5565a6b7d8b42a083a8b4b84/html5/thumbnails/24.jpg)
2850 OCEAN PARK BLVD, SUITE 110, SANTA MONICA CA
24© 2013 EDGECAST NETWORKS
Use Case 1:Launch of Transact
Directive from CEO:
Performance must match the existing matured network!
![Page 25: Synthetic and Real World Web Performance Monitoring](https://reader035.fdocuments.in/reader035/viewer/2022062513/5565a6b7d8b42a083a8b4b84/html5/thumbnails/25.jpg)
2850 OCEAN PARK BLVD, SUITE 110, SANTA MONICA CA
25© 2013 EDGECAST NETWORKS
Use Case 1:Launch of Transact
Transact is a brand new product from EdgeCast, designed to accelerate e-commerce performance on the web.
Featuring full PCI compliance, it has its own dedicated and highly reliable network, its own dedicated servers, and a fully integrated security layer.
The challenge is to deploy a brand new network, and make it perform like an existing, fully matured network.
Third party monitors are the key to accomplish this, and to know when we’ve accomplished it.
![Page 26: Synthetic and Real World Web Performance Monitoring](https://reader035.fdocuments.in/reader035/viewer/2022062513/5565a6b7d8b42a083a8b4b84/html5/thumbnails/26.jpg)
2850 OCEAN PARK BLVD, SUITE 110, SANTA MONICA CA
26© 2013 EDGECAST NETWORKS
Use Case 1: Launch of TransactEstablished monitors across the major third parties to observe actual latencies worldwide
![Page 27: Synthetic and Real World Web Performance Monitoring](https://reader035.fdocuments.in/reader035/viewer/2022062513/5565a6b7d8b42a083a8b4b84/html5/thumbnails/27.jpg)
2850 OCEAN PARK BLVD, SUITE 110, SANTA MONICA CA
27© 2013 EDGECAST NETWORKS
Use Case 1:Launch of Transact
Compared performance between Transact and EdgeCast ADN at all nodesUsed the delta to determine where issues were
![Page 28: Synthetic and Real World Web Performance Monitoring](https://reader035.fdocuments.in/reader035/viewer/2022062513/5565a6b7d8b42a083a8b4b84/html5/thumbnails/28.jpg)
2850 OCEAN PARK BLVD, SUITE 110, SANTA MONICA CA
28© 2013 EDGECAST NETWORKS
Use Case 1:Launch of Transact
Iterated: As issues were resolved, updated with new monitoring data
Repeat process!
![Page 29: Synthetic and Real World Web Performance Monitoring](https://reader035.fdocuments.in/reader035/viewer/2022062513/5565a6b7d8b42a083a8b4b84/html5/thumbnails/29.jpg)
2850 OCEAN PARK BLVD, SUITE 110, SANTA MONICA CA
29© 2013 EDGECAST NETWORKS
Use Case 1:Launch of Transact
Result: Worldwide average performance within 1.5ms
EdgeCast ADN Transact
![Page 30: Synthetic and Real World Web Performance Monitoring](https://reader035.fdocuments.in/reader035/viewer/2022062513/5565a6b7d8b42a083a8b4b84/html5/thumbnails/30.jpg)
2850 OCEAN PARK BLVD, SUITE 110, SANTA MONICA CA
30
Third party monitoring allows A/B comparison
© 2013 EDGECAST NETWORKS
What are Third Party Monitors For
![Page 31: Synthetic and Real World Web Performance Monitoring](https://reader035.fdocuments.in/reader035/viewer/2022062513/5565a6b7d8b42a083a8b4b84/html5/thumbnails/31.jpg)
2850 OCEAN PARK BLVD, SUITE 110, SANTA MONICA CA
31© 2013 EDGECAST NETWORKS
Use Case 2: RIPE/ATLAS
Using RIPE/ATLAS to evaluate the network for IPv6 launchfor World IPv6 Day, 2012
![Page 32: Synthetic and Real World Web Performance Monitoring](https://reader035.fdocuments.in/reader035/viewer/2022062513/5565a6b7d8b42a083a8b4b84/html5/thumbnails/32.jpg)
2850 OCEAN PARK BLVD, SUITE 110, SANTA MONICA CA
32© 2013 EDGECAST NETWORKS
Use Case 2: RIPE/ATLAS
For World IPv6 Day, we deployed IPv6 across our entire network.
The Challenge:
How does the IPv6 Internet perform? We used RUM data provided by RIPE/ATLAS to find out.
![Page 33: Synthetic and Real World Web Performance Monitoring](https://reader035.fdocuments.in/reader035/viewer/2022062513/5565a6b7d8b42a083a8b4b84/html5/thumbnails/33.jpg)
2850 OCEAN PARK BLVD, SUITE 110, SANTA MONICA CA
33© 2013 EDGECAST NETWORKS
Use Case 2: RIPE/ATLAS
http://atlas.ripe.net/about
RIPE Atlas is a global network of probes that measure Internet connectivity and reachability, providing an unprecedented understanding of the state of the Internet in real time.
http://www.ripe.net
The RIPE NCC is one of five Regional Internet Registries (RIRs) providing Internet resource allocations, registration services and coordination activities that support the operation of the Internet globally.
![Page 34: Synthetic and Real World Web Performance Monitoring](https://reader035.fdocuments.in/reader035/viewer/2022062513/5565a6b7d8b42a083a8b4b84/html5/thumbnails/34.jpg)
2850 OCEAN PARK BLVD, SUITE 110, SANTA MONICA CA
34© 2013 EDGECAST NETWORKS
Use Case 2: RIPE/ATLAS
JSON based API Ability to run multiple tests
![Page 35: Synthetic and Real World Web Performance Monitoring](https://reader035.fdocuments.in/reader035/viewer/2022062513/5565a6b7d8b42a083a8b4b84/html5/thumbnails/35.jpg)
2850 OCEAN PARK BLVD, SUITE 110, SANTA MONICA CA
35© 2013 EDGECAST NETWORKS
Use Case 2: RIPE/ATLAS
We launched a beacon dedicated to IPv6 measurements which tested:
• IPv4 only• IPv6 only• Dual IPv4 & IPv6
To reduce failures, the beacon also checked connectivity to ipv6.google.com
![Page 36: Synthetic and Real World Web Performance Monitoring](https://reader035.fdocuments.in/reader035/viewer/2022062513/5565a6b7d8b42a083a8b4b84/html5/thumbnails/36.jpg)
2850 OCEAN PARK BLVD, SUITE 110, SANTA MONICA CA
36© 2013 EDGECAST NETWORKS
Use Case 2: RIPE/ATLAS
What did we learn from Atlas about our IPv6 performance and availability?
![Page 37: Synthetic and Real World Web Performance Monitoring](https://reader035.fdocuments.in/reader035/viewer/2022062513/5565a6b7d8b42a083a8b4b84/html5/thumbnails/37.jpg)
2850 OCEAN PARK BLVD, SUITE 110, SANTA MONICA CA
37© 2013 EDGECAST NETWORKS
Use Case 2: RIPE/ATLAS
IPv6 Reach Analysis
AS path analysis AS path of failed traces
![Page 38: Synthetic and Real World Web Performance Monitoring](https://reader035.fdocuments.in/reader035/viewer/2022062513/5565a6b7d8b42a083a8b4b84/html5/thumbnails/38.jpg)
2850 OCEAN PARK BLVD, SUITE 110, SANTA MONICA CA
38© 2013 EDGECAST NETWORKS
Use Case 2: RIPE/ATLASVisualization of Country-of-Origin traffic to EdgeCast POP
Ex: Ukraine is served by both POP7 and POP1, and POP7 has higher latency
![Page 39: Synthetic and Real World Web Performance Monitoring](https://reader035.fdocuments.in/reader035/viewer/2022062513/5565a6b7d8b42a083a8b4b84/html5/thumbnails/39.jpg)
2850 OCEAN PARK BLVD, SUITE 110, SANTA MONICA CA
39© 2013 EDGECAST NETWORKS
Use Case 2: RIPE/ATLAS
Availability predictions before launch
![Page 40: Synthetic and Real World Web Performance Monitoring](https://reader035.fdocuments.in/reader035/viewer/2022062513/5565a6b7d8b42a083a8b4b84/html5/thumbnails/40.jpg)
2850 OCEAN PARK BLVD, SUITE 110, SANTA MONICA CA
40© 2013 EDGECAST NETWORKS
Use Case 2: RIPE/ATLASPerformance improvements during pre-launchConclusions: IPv6 still has room for improvement
![Page 41: Synthetic and Real World Web Performance Monitoring](https://reader035.fdocuments.in/reader035/viewer/2022062513/5565a6b7d8b42a083a8b4b84/html5/thumbnails/41.jpg)
2850 OCEAN PARK BLVD, SUITE 110, SANTA MONICA CA
41© 2013 EDGECAST NETWORKS
Use Case 3: Analytics within Third Party Monitors
Using Third Party Monitors to answer real-world questions
Often, we’re asked questions by different groups of the company.
Sales, for instance, always wants to prove that we’re the fastest worldwide.
But what to do when some regions are faster than others?
![Page 42: Synthetic and Real World Web Performance Monitoring](https://reader035.fdocuments.in/reader035/viewer/2022062513/5565a6b7d8b42a083a8b4b84/html5/thumbnails/42.jpg)
2850 OCEAN PARK BLVD, SUITE 110, SANTA MONICA CA
42© 2013 EDGECAST NETWORKS
Use Case 3: Analytics within Third Party Monitors
Sales wanted to know why a prospective client was performing really well in Asia, but not nearly as well in Europe.
AsiaEdgeCast Mystery Competitor X
EuropeEdgeCast Mystery Competitor X
![Page 43: Synthetic and Real World Web Performance Monitoring](https://reader035.fdocuments.in/reader035/viewer/2022062513/5565a6b7d8b42a083a8b4b84/html5/thumbnails/43.jpg)
2850 OCEAN PARK BLVD, SUITE 110, SANTA MONICA CA
43© 2013 EDGECAST NETWORKS
Use Case 3: Analytics within Third Party Monitors
There was a lot of variance within countries
EdgeCast Mystery Competitor X
![Page 44: Synthetic and Real World Web Performance Monitoring](https://reader035.fdocuments.in/reader035/viewer/2022062513/5565a6b7d8b42a083a8b4b84/html5/thumbnails/44.jpg)
2850 OCEAN PARK BLVD, SUITE 110, SANTA MONICA CA
44© 2013 EDGECAST NETWORKS
Use Case 3: Analytics within Third Party Monitors
The scatterplot shows some odd signs of horizontal banding.
EdgeCast Mystery Competitor X
![Page 45: Synthetic and Real World Web Performance Monitoring](https://reader035.fdocuments.in/reader035/viewer/2022062513/5565a6b7d8b42a083a8b4b84/html5/thumbnails/45.jpg)
2850 OCEAN PARK BLVD, SUITE 110, SANTA MONICA CA
45© 2013 EDGECAST NETWORKS
Use Case 3: Analytics within Third Party Monitors
Good clue: Looking at the total downloaded bytes instead of response time, the banding becomes really obvious, and wildly different in size
EdgeCast Mystery Competitor X
![Page 46: Synthetic and Real World Web Performance Monitoring](https://reader035.fdocuments.in/reader035/viewer/2022062513/5565a6b7d8b42a083a8b4b84/html5/thumbnails/46.jpg)
2850 OCEAN PARK BLVD, SUITE 110, SANTA MONICA CA
46© 2013 EDGECAST NETWORKS
Use Case 3: Analytics within Third Party Monitors
A histogram shows the magnitude of this difference
![Page 47: Synthetic and Real World Web Performance Monitoring](https://reader035.fdocuments.in/reader035/viewer/2022062513/5565a6b7d8b42a083a8b4b84/html5/thumbnails/47.jpg)
2850 OCEAN PARK BLVD, SUITE 110, SANTA MONICA CA
47© 2013 EDGECAST NETWORKS
Use Case 3: Analytics within Third Party Monitors
Conclusion:
Client was returning different data depending on the geoIP lookup of the end user. So end users in different countries were deliberately being returned different objects.
However, as caches, both EdgeCast and Mystery Competitor X were caching the index.html page, so whatever was on that page was cached and returned on every request. New data was only returned once every day, when the caching rule expired, and a fresh request to origin was made.
The extreme difference in size was largely a function of random chance- we got lucky to return smaller objects during this sample time.
![Page 48: Synthetic and Real World Web Performance Monitoring](https://reader035.fdocuments.in/reader035/viewer/2022062513/5565a6b7d8b42a083a8b4b84/html5/thumbnails/48.jpg)
2850 OCEAN PARK BLVD, SUITE 110, SANTA MONICA CA
48© 2013 EDGECAST NETWORKS
Questions?