Techniques for Agile Performance Testing
-
Upload
techwellpresentations -
Category
Technology
-
view
188 -
download
4
description
Transcript of Techniques for Agile Performance Testing
Session
Presented by:
Jim Hirschauer
Brought to you by:
340 Corporate Way, Suite Orange Park, FL 32073 888‐2
W14 Concurrent4/9/2014 3:15 PM
“Techniques for Agile Performance Testing”
AppDynamics
300,68‐8770 ∙ 904‐278‐0524 ∙ [email protected] ∙ www.sqe.com
Jim Hirschauer AppDynamics
Technology evangelist for AppDynamics Jim Hirschauer has an extensive background in highly available, business critical, large enterprise IT operations environments. Jim has been interested in application performance testing and monitoring since his days as a systems administrator in a retail bank. His passion for performance analysis led him down a path to design, implement, and manage the cloud computing monitoring architecture for a top-ten investment bank. During his tenure at the bank, Jim created new processes and procedures that reduced the time spent in the test and QA cycle while simultaneously increasing overall code release quality and dramatically improving end user experience.
2/18/2014
1
PERFORMANCE TESTINGCRASH COURSE
Jim Hirschauer@HirschOnAPM
Lockheed Martin Aeronautical Systems
Duke Energy
Wachovia
Wells Fargo
AppDynamics
2/18/2014
2
Wh dWhy does performance
matter?matter?
Mi ft f d th t BiMicrosoft found that Bingsearches that were 2 seconds
slower resulted in a 4.3% drop in revenue per userdrop in revenue per user
2/18/2014
3
Wh M ill h d 2 2When Mozilla shaved 2.2seconds off their landing page, Firefox downloads
increased 15.4%increased 15.4%
Making Barack Obama’s website 60%
faster increased donation conversions
by 14%
2/18/2014
4
Amazon and WalmartAmazon and Walmartincreased revenue 1% for
every 100ms of improvementimprovement
2/18/2014
5
Performance directly impacts the bottom
linee
2/18/2014
6
T l f th t dTools of the trade for performance
testingtesting
2/18/2014
7
Understand your baseline performance
Static
vs
Hello World
vsvs
App
2/18/2014
8
Apache Bench
ab -c 10 -t 10 -k http://dustinwhittle.com/
2/18/2014
9
Server Software: nginxServer Hostname: dustinwhittle.comServer Port: 80
Document Path: /Document Length: 6642 bytes
Concurrency Level: 10Time taken for tests: 10.042 secondsC l 286Complete requests: 286Failed requests: 0Write errors: 0Keep-Alive requests: 0Total transferred: 2080364 bytesHTML transferred: 1899612 bytesRequests per second: 28.48 [#/sec] (mean)Time per request: 351.133 [ms] (mean)Time per request: 35.113 [ms] (mean, across all concurrent requests)Transfer rate: 202.30 [Kbytes/sec] received
Connection Times (ms)Connection Times (ms)min mean[+/-sd] median max
Connect: 9 20 14.2 17 148Processing: 117 325 47.8 323 574Waiting: 112 317 46.3 314 561Total: 140 346 46.0 341 589
Percentage of the requests served within a certain time (ms)50% 34166% 35675% 36680% 372
Siege
2/18/2014
10
siege -c 10 -b -t 10S http://dustinwhittle.com/
** SIEGE 2.72** Preparing 10 concurrent users for battle.The server is now under siege...Lifting the server siege... done.
Transactions: 263 hitsAvailability: 100.00 %Elapsed time: 9.36 secsData transferred: 0.35 MBResponse time: 0.35 secsTransaction rate: 28.10 trans/secThroughput: 0.04 MB/secConcurrency: 9.82ySuccessful transactions: 263Failed transactions: 0Longest transaction: 0.54Shortest transaction: 0.19
2/18/2014
11
Crawl the entire app to discover all urls
sproxy -o ./urls.txt
2/18/2014
12
SPROXY v1.02 listening on port 9001...appending HTTP requests to: ./urls.txt...default connection timeout: 120 seconds
wget -r -o verbose txt -l 0 -t 1 --spider -w 1 -ewget -r -o verbose.txt -l 0 -t 1 --spider -w 1 -e robots=on
-e "http_proxy = http://127.0.0.1:9001""http://dustinwhittle.com/"
sort -u -o urls.txt urls.txt
2/18/2014
13
Benchmark traffic across all unique
urls with siegeu s t s ege
siege -v -c 50 -i -t 3M -f urls.txt -d 10
2/18/2014
14
Apache JMeter
2/18/2014
15
M lti M h i iMulti-Mechanize is an open source framework for performance and load
testingtesting
2/18/2014
16
pip install multi-mechanize
multimech-newproject demo
2/18/2014
17
import requestsimport requests
class Transaction(object):def run(self):
r = requests.get('http://dustinwhittle.com/')d()r.raw.read()
class Transaction(object):def run(self):
br = mechanize.Browser()br.set_handle_robots(False)
start_timer = time.time()resp = br.open('http://www.dustinwhittle.com/')resp.read()latency = time.time() - start_timerself custom timers['homepage'] = latencyself.custom_timers[ homepage ] latency
start_timer = time.time()resp = br.open('http://www.dustinwhittle.com/blog')resp.read()latency = time time() start timer
2/18/2014
18
[global]run_time = 10
5rampup = 5results_ts_interval = 1progress_bar = onconsole_logging = offxml_report = on
[user_group-1]threads = 1script = demo.py
multimech-run demo
2/18/2014
19
2/18/2014
20
What about when you need more than
one machine?o e ac e
Who lives in the cloud?
2/18/2014
21
Bees with Machine Guns
A utility for arming (creating)A utility for arming (creating) many bees (micro EC2
instances)to attack (load test)
t t ( b li ti )targets (web applications)
2/18/2014
22
pip install beeswithmachineguns
# ~/.boto
[Credentials]
aws_access_key_id=xxxaws_secret_access_key=xxx
[Boto]
ec2_region_name = us-west-2ec2_region_endpoint = ec2.us-west-2.amazonaws.com
2/18/2014
23
bees up -s 2 -g default -z us-west-2b -i ami-bc05898c -k appdynamics-dustinwhittle-aws-us-west-2 -l ec2-
user
Connecting to the hiveConnecting to the hive.Attempting to call up 2 bees.Waiting for bees to load their machine guns.......Bee i-3828400c is ready for the attack.Bee i-3928400d is ready for the attack.The swarm has assembled 2 bees.
2/18/2014
24
bees report
Read 2 bees from the roster.Bee i-3828400c: running @ 54.212.22.176Bee i-3928400d: running @ 50.112.6.191
2/18/2014
25
bees attack -n 1000 -c 50 -u http://dustinwhittle.com/
Read 2 bees from the roster.Connecting to the hive.Assembling bees.Each of 2 bees will fire 50000 rounds, 125 at a time.Stinging URL so it will be cached for the attack.Organizing the swarm.Bee 0 is joining the swarm.Bee 1 is joining the swarm.Bee 0 is firing his machine gun. Bang bang!Bee 1 is firing his machine gun. Bang bang!Bee 1 is out of ammo.Bee 0 is out of ammo.Offensive complete.
Complete requests: 100000Requests per second: 1067.110000 [#/sec] (mean)Time per request: 278.348000 [ms] (mean)50% response time: 47.500000 [ms] (mean)90% response time: 114.000000 [ms] (mean)
Mission Assessment: Target crushed bee offensive.The swarm is awaiting new orders.
2/18/2014
26
bees down
What about the client side?
2/18/2014
27
Google PageSpeed
Google PageSpeed Insights
2/18/2014
28
2/18/2014
29
Google PageSpeed API
curl "https://www.googleapis.com/pagespeedonline/v1/runPagespeed?url=http://
dustinwhittle.com/&key=xxx"
2/18/2014
30
WBench
gem install wbench
2/18/2014
31
wbench http://dustinwhittle.com/
2/18/2014
32
Automate client-side performance testing
with Gruntt G u t
Use Bower (forUse Bower (for dependencies),
Grunt (for automation),and Yeoman (for bootstrapping)
2/18/2014
33
2/18/2014
34
H lHow many people understand exactly how
fast their site runs in production?production?
2/18/2014
35
2/18/2014
36
2/18/2014
37
2/18/2014
38
2/18/2014
39
http://webpagetest.org/
2/18/2014
40
2/18/2014
41
L d t tiLoad testing services from the
cloudcloud
2/18/2014
42
2/18/2014
43
I t t t t dIntegrate automated performance testing into
continuous integration for server-side and client-sideserver side and client side
2/18/2014
44
U d t d thUnderstand the performance implications of
every deployment and package upgradepackage upgrade
Monitor end user experience from end to end in productionto e d p oduct o
2/18/2014
45
Questions?