Application Performance Testing: A Simplified Universal Approach
-
Upload
techwellpresentations -
Category
Technology
-
view
181 -
download
1
description
Transcript of Application Performance Testing: A Simplified Universal Approach
![Page 1: Application Performance Testing: A Simplified Universal Approach](https://reader034.fdocuments.in/reader034/viewer/2022052504/54918fb8b479599d2d8b5337/html5/thumbnails/1.jpg)
MD AM Tutorial
9/30/2013 8:30:00 AM
"Application Performance
Testing: A Simplified Universal
Approach"
Presented by:
Scott Barber
PerfTestPlus, Inc.
Brought to you by:
340 Corporate Way, Suite 300, Orange Park, FL 32073
888-268-8770 ∙ 904-278-0524 ∙ [email protected] ∙ www.sqe.com
![Page 2: Application Performance Testing: A Simplified Universal Approach](https://reader034.fdocuments.in/reader034/viewer/2022052504/54918fb8b479599d2d8b5337/html5/thumbnails/2.jpg)
Scott Barber
PerfTestPlus, Inc.
CTO of PerfTestPlus and recognized worldwide as a thought-leader in software system
performance and testing, Scott Barber describes himself as a “tester, author, speaker, disrupter,
and dad.” Scott is the author of Web Load Testing for Dummies and co-/contributing author of
Performance Testing Guidance for Web Applications, Beautiful Testing, and How to Reduce the
Cost of Testing.
![Page 3: Application Performance Testing: A Simplified Universal Approach](https://reader034.fdocuments.in/reader034/viewer/2022052504/54918fb8b479599d2d8b5337/html5/thumbnails/3.jpg)
8/27/2013
1
Managing Application Performance:
A Simplified Universal Approach
1
By:
Scott BarberChief Performance Evangelist
SmartBear Software, Inc.
Created for:
Managing Application Performance:A Simplified Universal Approach
![Page 4: Application Performance Testing: A Simplified Universal Approach](https://reader034.fdocuments.in/reader034/viewer/2022052504/54918fb8b479599d2d8b5337/html5/thumbnails/4.jpg)
8/27/2013
2
Chief Performance EvangelistSmartBear Software, Inc. [email protected]
@sbarber
Co-Founder: Workshop On Performance and Reliabilitywww.performance-workshop.org
Co-Author:Author: Contributing Author:
Books: www.perftestplus.com/pubs
About me: about.me/scott.barber
System or application characteristics related to:
Speed: - responsiveness- user experience
Scalability: - capacity- load- volume
Stability: - consistency- reliability- stress
What is Performance?
![Page 5: Application Performance Testing: A Simplified Universal Approach](https://reader034.fdocuments.in/reader034/viewer/2022052504/54918fb8b479599d2d8b5337/html5/thumbnails/5.jpg)
8/27/2013
3
What is a Performant System?
A system optimized to balance:
- Cost
- Time to Market
- Capacity
…while remaining focused on:
Quality of Service to System Users
What is Performance Testing?
Testing designed to:
Inform software system optimization by balancing:
- Cost
- Time to Market
- Capacity
while remaining focused on the quality of service to system users.
![Page 6: Application Performance Testing: A Simplified Universal Approach](https://reader034.fdocuments.in/reader034/viewer/2022052504/54918fb8b479599d2d8b5337/html5/thumbnails/6.jpg)
8/27/2013
4
Performance vs. Load Testing?
The Performance Lifecycle is:
Conception to Headstone
Not
Cradle to Grave
![Page 7: Application Performance Testing: A Simplified Universal Approach](https://reader034.fdocuments.in/reader034/viewer/2022052504/54918fb8b479599d2d8b5337/html5/thumbnails/7.jpg)
8/27/2013
5
© PerfTestPlus, Inc.
Prevent Poor Performance…
… don’t just react when it happens.
Who is Responsible?
![Page 8: Application Performance Testing: A Simplified Universal Approach](https://reader034.fdocuments.in/reader034/viewer/2022052504/54918fb8b479599d2d8b5337/html5/thumbnails/8.jpg)
8/27/2013
6
© PerfTestPlus, Inc.
Where does it fit in the SDLC?
![Page 9: Application Performance Testing: A Simplified Universal Approach](https://reader034.fdocuments.in/reader034/viewer/2022052504/54918fb8b479599d2d8b5337/html5/thumbnails/9.jpg)
8/27/2013
7
Delivering Performant Systems;Commonly, Y2K -> Today
SPE (Software Performance Engineering)� Math-centric Architect/Design method
+ Profiling� Context-free code optimization
+ Load Testing� Demonstrate (too late) poor performance characteristics
+ Capacity Planning� Math-centric guessing to determine hardware needs
+ APM (Application Performance Management)� Proactively react to production issues
=> Inefficient & Reactionary Delivery & Maintenance
Delivering Performant Systems;Moving Forward
Minimize Need for Complex Mathematics� Few teams have mathematicians w/ advanced degrees
Disconnect Accuracy from Empirical Production Data� Because we can’t depend on what we don’t have yet
Remove Socio-Political Barriers� Collaboration improves information sharing & efficiency
Keep/Adapt most Valuable Practices� Many good concepts have been poorly implemented
Distribute Valuable Practices Throughout Life Cycle� Many valuable practices have suffered from stove-piping
I’m testing a model for the entire ‘Performance Journey’
![Page 10: Application Performance Testing: A Simplified Universal Approach](https://reader034.fdocuments.in/reader034/viewer/2022052504/54918fb8b479599d2d8b5337/html5/thumbnails/10.jpg)
8/27/2013
8
Performant Application DeliveryPerformant Application DeliveryPerformant Application DeliveryPerformant Application DeliveryPerformant Application DeliveryPerformant Application DeliveryPerformant Application DeliveryPerformant Application Delivery
© PerfTestPlus, Inc.
“DevOps” Perf Team Individual
• Code Profiling
• Requirements Revision
• Design Validation
• Budgets
• Units & Components
• Monitoring
• Tuning
• Load/Stress Testing
• Environment Config
• Capacity Planning
• Concurrency Checks
• Situation Analysis
• Snapshots
• Comparisons
• Niche Concerns
• Mini-Field Studies
(UAT)
• “What-if…”
Key Points• Entire Lifecycle
• Entire Team Focused
• Individual, Role, & Team Accountability & Responsibility
• Continuous & Collaborative
© PerfTestPlus, Inc.
![Page 11: Application Performance Testing: A Simplified Universal Approach](https://reader034.fdocuments.in/reader034/viewer/2022052504/54918fb8b479599d2d8b5337/html5/thumbnails/11.jpg)
8/27/2013
9
© PerfTestPlus, Inc.
Preventing Poor Performance with
a little work…
every day…
from every one.
© PerfTestPlus, Inc.
Units StoriesTiersResourcesGoals
AssessCompareInvestigateAcceptAnswer
Dev & Prod TimesResourcesSizesFrequenciesDashboard!
ProactiveMicro & Macro Establish GoalsUpdate Targets
![Page 12: Application Performance Testing: A Simplified Universal Approach](https://reader034.fdocuments.in/reader034/viewer/2022052504/54918fb8b479599d2d8b5337/html5/thumbnails/12.jpg)
8/27/2013
10
© PerfTestPlus, Inc.
Unit-Level Testing Tools(‘cause folks always ask)
FireBenchmarks; Performance testing addin for NUnit
JUnitPerf; a collection of JUnit test decorators for performance
Firefox Performance Tester's Pack
HTTPerf
![Page 13: Application Performance Testing: A Simplified Universal Approach](https://reader034.fdocuments.in/reader034/viewer/2022052504/54918fb8b479599d2d8b5337/html5/thumbnails/13.jpg)
8/27/2013
11
© PerfTestPlus, Inc.
An Ounce of Prevention…
![Page 14: Application Performance Testing: A Simplified Universal Approach](https://reader034.fdocuments.in/reader034/viewer/2022052504/54918fb8b479599d2d8b5337/html5/thumbnails/14.jpg)
8/27/2013
12
© PerfTestPlus, Inc.
© PerfTestPlus, Inc.
![Page 15: Application Performance Testing: A Simplified Universal Approach](https://reader034.fdocuments.in/reader034/viewer/2022052504/54918fb8b479599d2d8b5337/html5/thumbnails/15.jpg)
8/27/2013
13
Load Tests
Stress Tests
Endurance Tests
Infrastructure/Configuration Verification
Load Variability Analysis
Background Load for Acceptance Tests…
What is it?
Or any other deliberate approximation of
Production Conditions
© PerfTestPlus, Inc.
![Page 16: Application Performance Testing: A Simplified Universal Approach](https://reader034.fdocuments.in/reader034/viewer/2022052504/54918fb8b479599d2d8b5337/html5/thumbnails/16.jpg)
8/27/2013
14
RPT is:
Inspired by Rapid Software Testing
Consistent with Rapid Software Testing themes
Sanctioned by James Bach, Michael Bolton & the RST
instructors to as a specific implementation of the
Rapid Testing Methodology
For more information about RST, visit:
http://www.satisfice.com/info_rst.shtml
“What have we got?What do we want?
How do we get there…?”--Bob Barber (Scott’s dad)
… as quickly, simply, and
cheaply as possible?
Evolved from:
![Page 17: Application Performance Testing: A Simplified Universal Approach](https://reader034.fdocuments.in/reader034/viewer/2022052504/54918fb8b479599d2d8b5337/html5/thumbnails/17.jpg)
8/27/2013
15
…a common man’s way of expressing the problem solving approach that classical engineers employ.
• Given: “What have we got?”
• Find: “What do we want?”
• Solve: “How do we get there?”
An approach to respond to a specific
performance-related question after 4 or fewer
hours of team effort with 1 or more of:
A) The answer
B) A partial answer
• To determine the value of additional effort
• The level of effort to provide the answer
C) Better questions to address the underlying
concern
What is it?
![Page 18: Application Performance Testing: A Simplified Universal Approach](https://reader034.fdocuments.in/reader034/viewer/2022052504/54918fb8b479599d2d8b5337/html5/thumbnails/18.jpg)
8/27/2013
16
1. Receive Question• Clarify the question• Understand the driver(s) behind the question
2. Generate Test Coverage Outline (TCO) (~20 min)• Simplest path to (partial) answer(s)• Comprehensive path to (partial) answer(s)
3. Transform TCO into Rapid Strategy (~20 min)• Only tasks that fit in time box• Stick to tasks requiring available resources
4. Execute Strategy (~2.5 hrs)• Snapshots are your friends• Anecdotal is sufficient
5. Consolidate/Analyze Data (~30 min)• Identify patterns• Confirm patterns (time permitting)
6. Report Results (~20 min)• Answer(s) / Time or Effort to get answer(s)• Follow-on questions of interest
~4 hr
Conceptual Approach
© PerfTestPlus, Inc.
© PerfTestPlus, Inc.
![Page 19: Application Performance Testing: A Simplified Universal Approach](https://reader034.fdocuments.in/reader034/viewer/2022052504/54918fb8b479599d2d8b5337/html5/thumbnails/19.jpg)
8/27/2013
17
© PerfTestPlus, Inc.
The Bottom Line
© PerfTestPlus, Inc.
![Page 20: Application Performance Testing: A Simplified Universal Approach](https://reader034.fdocuments.in/reader034/viewer/2022052504/54918fb8b479599d2d8b5337/html5/thumbnails/20.jpg)
8/27/2013
18
Addendum
Questions?
![Page 21: Application Performance Testing: A Simplified Universal Approach](https://reader034.fdocuments.in/reader034/viewer/2022052504/54918fb8b479599d2d8b5337/html5/thumbnails/21.jpg)
8/27/2013
19
Contact Info
about.me/scott.barber
Chief Performance Evangelist
SmartBear Softwear, Inc.
E-mail:
Blog:
scott-barber.blogspot.com
Web Site:
SmartBear.com
Twitter:
@sbarber