Web Performance Wars - Total Performance Consulting
37
-
Upload
totalperformance -
Category
Business
-
view
405 -
download
1
Transcript of Web Performance Wars - Total Performance Consulting
2
3
Have you ever had a performance related emergency?
4
5
TIME IS MONEY
6
TIME IS MONEY
7
GREATER CONSEQUENCES
War Stories
8
A Park’s Perf War
9
10
PARK RESERVATION PERF WAR
11
WORKLOAD
Business Goals
Requirements Current Usage
HistoricalUsage
Projections
!Success
12
PARK RESERVATION WORKLOAD
HISTORICAL RESERVATIONS COUNTS
Peak Reservations / hr(Opening day for 9 parks) 3509
Peak Reservations on opening day 6950
Peak Reservations / hr(Opening day for 5 other parks) 3400
Peak Reservations on 2nd busiest day 7900
13
WHAT DATA WAS ANALYZED• Perfmon, PAL, and Yslow• Key transaction response times• Resource utilization• Validation of proper application calls
14
BUSINESS LESSONS LEARNED• Performance Considerations• Solid Workload Model• Peak busy min… High traffic days
15
TECHNICAL LESSONS LEARNED• Not what it seems to be• Always suspect the DB• Disk I/O• Front End Performance• Validation of proper application calls
A Retailer’s Perf War
16
17
OVERVIEW• Large Scale Test• Quick Ramp to Full Load• Need to Come from Multiple GEOs
18
BATTLE WOUNDS
19
RETAIL LESSONS• Speak to CDN provider prior to any load test• Ensure at least a 15 minutes ramp-up to full load• Testing off hours• Ensure Load distribution is across as many available
regions as possible• No more than 300 Mb/s traffic should be generated
from one geo area + ISP combination
A Bank’s Perf War
20
21
THE CHALLENGE• 1000+ Branches • New software deployment
22
BANKING BATTLEGROUND• IBM AIX Environment• IBM HTTP Server ( Apache), WebSphere
Application Server, Oracle 10• Web client front end (Javascript)• F5 Load Balancers
23
BANKING STRATEGY• Examine how traffic was being
distributed to the servers • Suspected problem with app traffic load
balancing
24
BANKING VICTORY• Investigated load balancer algorithms • Made the change – problem solved!
Software performance testingand its best practices.
25
26
MONITOR ALL RELEVANT COMPONENTS
27
HOW DOES THE DATA FROM THE TOOLS HELP?
28
ALSO• Make use of collection and analysis tools
readily available for the platform• Examine performance results data against
platform thresholds• Document resource utilization issues• Establish origins of adverse performance
events
29
GET THE LOW HANGING FRUIT
Trend Micro Virus ScanOverall Counter Instance Statistics
30
GET THE LOW HANGING FRUIT
CONDITION\PROCESS(*)\IO DATAOPERATIONS/SEC
MIN AVG MAX HOURLY TREND
STD DEVIATIO
N
10% OF OUTLIERS REMOVED
20%OUTLIERS REMOVED
30% OF OUTLIERS REMOVED
More than 1000 data IO operations (network,
disk, or device IO) per second
SQL1/NTRTScan 0 99 1,985 29 404 1 0 0
31
SOFTWARE PERFORMANCE ANTI-PATTERNS
Traffic didn’t crash the Obamacare site alone. Bad coding did too.
Excessive DB Hits OrganicInheritance
Insufficient DBExpertise
No Active SystemMonitoring for
Continuous Improvement
Non-technicalPMs
The fastest path to a slowdown requires no action
32
ANTI-PATTERNS CONT’D
Excessive ObjectCreation One Lane Bridge
UnmeasureCache
Behavior
33
ANTI-PATTERNS EXAMPLETo prioritize the issues for resolution, one possible ordering is as follows:
1. SQLServer: SQL Errors Errors/sec
2. SQLServer: Access Methods Workfiles Created/sec
3. SQLServer: Locks Lock Timeouts/sec
4. SQLServer: Latches Latch Waits/sec
5. SQLServer: Deprecated Features Usage
34
DOCUMENT CHANGES BETWEEN DELIVERED BUILDS
35
DON’T IGNORE FRONT-END PERFORMANCE• An under performing front end can
negate all downstream optimizations to the end user
• Modernize legacy code• Simplify functionality• Handle exceptions• Sometimes problem(s) exist on
client platform and not application• 80 % of total response time is spent
on the front end
FRONT END ISSUES
Fewer HTTP Requests
Using a CDN
Some content not HTTP compressed
Image optimization opportunities
Minify JS and CSS
Large ASP.NET ViewState
Questions?
36
Amit PatelTotal Performance Consulting
@TotalPerformwww.totalperform.com
37
Thank You.
