The secret art of agile performance testing
-
Upload
alois-reitbauer -
Category
Technology
-
view
645 -
download
2
Transcript of The secret art of agile performance testing
![Page 1: The secret art of agile performance testing](https://reader035.fdocuments.in/reader035/viewer/2022062707/55838dfad8b42a8e0c8b500d/html5/thumbnails/1.jpg)
The Secret Art of Agile Performance TestingAlois Reitbauer, dynaTrace Software
![Page 2: The secret art of agile performance testing](https://reader035.fdocuments.in/reader035/viewer/2022062707/55838dfad8b42a8e0c8b500d/html5/thumbnails/2.jpg)
The Classical Approach
![Page 3: The secret art of agile performance testing](https://reader035.fdocuments.in/reader035/viewer/2022062707/55838dfad8b42a8e0c8b500d/html5/thumbnails/3.jpg)
Waterfalls are pretty
![Page 4: The secret art of agile performance testing](https://reader035.fdocuments.in/reader035/viewer/2022062707/55838dfad8b42a8e0c8b500d/html5/thumbnails/4.jpg)
But might get scary
![Page 5: The secret art of agile performance testing](https://reader035.fdocuments.in/reader035/viewer/2022062707/55838dfad8b42a8e0c8b500d/html5/thumbnails/5.jpg)
Dream
Comfort Zone
Reality
Iteration
Iteration
Iteration
Test
Test
![Page 6: The secret art of agile performance testing](https://reader035.fdocuments.in/reader035/viewer/2022062707/55838dfad8b42a8e0c8b500d/html5/thumbnails/6.jpg)
Dream
Comfort Zone
Reality
Iteration
Iteration
Iteration
Test
Test
Iteration
Iteration
Iteration
![Page 7: The secret art of agile performance testing](https://reader035.fdocuments.in/reader035/viewer/2022062707/55838dfad8b42a8e0c8b500d/html5/thumbnails/7.jpg)
NFRs and Maslow
![Page 8: The secret art of agile performance testing](https://reader035.fdocuments.in/reader035/viewer/2022062707/55838dfad8b42a8e0c8b500d/html5/thumbnails/8.jpg)
The Focus Problem
![Page 9: The secret art of agile performance testing](https://reader035.fdocuments.in/reader035/viewer/2022062707/55838dfad8b42a8e0c8b500d/html5/thumbnails/9.jpg)
Methodology
![Page 10: The secret art of agile performance testing](https://reader035.fdocuments.in/reader035/viewer/2022062707/55838dfad8b42a8e0c8b500d/html5/thumbnails/10.jpg)
The Agile Approach
![Page 11: The secret art of agile performance testing](https://reader035.fdocuments.in/reader035/viewer/2022062707/55838dfad8b42a8e0c8b500d/html5/thumbnails/11.jpg)
Defining our Strategy
Start Early
Test Continuously
Break in Pieces
![Page 12: The secret art of agile performance testing](https://reader035.fdocuments.in/reader035/viewer/2022062707/55838dfad8b42a8e0c8b500d/html5/thumbnails/12.jpg)
Creating FocusRi
sk
Impact
AutomateRegular TestingSkip
![Page 13: The secret art of agile performance testing](https://reader035.fdocuments.in/reader035/viewer/2022062707/55838dfad8b42a8e0c8b500d/html5/thumbnails/13.jpg)
Frequency vs. Granularity
Frequency
Gra
nu
lari
ty
JUnit-basedTests (2x day)
Total SystemTests
Long-running Stabiltiy Tests(2 w duration)
![Page 14: The secret art of agile performance testing](https://reader035.fdocuments.in/reader035/viewer/2022062707/55838dfad8b42a8e0c8b500d/html5/thumbnails/14.jpg)
Granularity
Comparability
Complexity
Quality
![Page 15: The secret art of agile performance testing](https://reader035.fdocuments.in/reader035/viewer/2022062707/55838dfad8b42a8e0c8b500d/html5/thumbnails/15.jpg)
Define KPIs
Throughput
Response Time
Memory Consumption
Other KPIs …
![Page 16: The secret art of agile performance testing](https://reader035.fdocuments.in/reader035/viewer/2022062707/55838dfad8b42a8e0c8b500d/html5/thumbnails/16.jpg)
MonitorGain Visibility
ResolveReduce Resolution
Time
PreventProactively Avoid
Problems
Follow a Strategy
![Page 17: The secret art of agile performance testing](https://reader035.fdocuments.in/reader035/viewer/2022062707/55838dfad8b42a8e0c8b500d/html5/thumbnails/17.jpg)
Avoid Re-Runs
• What could happen?• Which information do you
want?• What describes your system?• What is different from the last
run?
![Page 18: The secret art of agile performance testing](https://reader035.fdocuments.in/reader035/viewer/2022062707/55838dfad8b42a8e0c8b500d/html5/thumbnails/18.jpg)
Don´t Forget the Real World
![Page 19: The secret art of agile performance testing](https://reader035.fdocuments.in/reader035/viewer/2022062707/55838dfad8b42a8e0c8b500d/html5/thumbnails/19.jpg)
Lessons learned
![Page 20: The secret art of agile performance testing](https://reader035.fdocuments.in/reader035/viewer/2022062707/55838dfad8b42a8e0c8b500d/html5/thumbnails/20.jpg)
Profiling is not enough
Good for finding problems
Result comparison hard
Only valid until next check-in
Too much work
![Page 21: The secret art of agile performance testing](https://reader035.fdocuments.in/reader035/viewer/2022062707/55838dfad8b42a8e0c8b500d/html5/thumbnails/21.jpg)
Measurements
![Page 22: The secret art of agile performance testing](https://reader035.fdocuments.in/reader035/viewer/2022062707/55838dfad8b42a8e0c8b500d/html5/thumbnails/22.jpg)
Stability of Tests
![Page 23: The secret art of agile performance testing](https://reader035.fdocuments.in/reader035/viewer/2022062707/55838dfad8b42a8e0c8b500d/html5/thumbnails/23.jpg)
Use Dedicated Hardware
Comparability
Stability
Efficiency
![Page 24: The secret art of agile performance testing](https://reader035.fdocuments.in/reader035/viewer/2022062707/55838dfad8b42a8e0c8b500d/html5/thumbnails/24.jpg)
Trends in Unstable Tests
![Page 25: The secret art of agile performance testing](https://reader035.fdocuments.in/reader035/viewer/2022062707/55838dfad8b42a8e0c8b500d/html5/thumbnails/25.jpg)
Create Instability
.. adding some volatility increases the likelyness to discover problems …“
![Page 26: The secret art of agile performance testing](https://reader035.fdocuments.in/reader035/viewer/2022062707/55838dfad8b42a8e0c8b500d/html5/thumbnails/26.jpg)
Testing scalability
Small Dump Operations Big Dump Operations
![Page 27: The secret art of agile performance testing](https://reader035.fdocuments.in/reader035/viewer/2022062707/55838dfad8b42a8e0c8b500d/html5/thumbnails/27.jpg)
Understand your measurements
Response Time only Response Time and GC
![Page 28: The secret art of agile performance testing](https://reader035.fdocuments.in/reader035/viewer/2022062707/55838dfad8b42a8e0c8b500d/html5/thumbnails/28.jpg)
Motivate your team
![Page 29: The secret art of agile performance testing](https://reader035.fdocuments.in/reader035/viewer/2022062707/55838dfad8b42a8e0c8b500d/html5/thumbnails/29.jpg)
How to make developers write tests
#1 Heroism#2 Boomerang#3 The other guy#4 Bug me not#5 Feedback#6 Code vs. Wine#7 Newb vs. Noob
![Page 30: The secret art of agile performance testing](https://reader035.fdocuments.in/reader035/viewer/2022062707/55838dfad8b42a8e0c8b500d/html5/thumbnails/30.jpg)
Test Case Complexity
FirstStart dynaTrace infrastructureWhen readyStart n WebSphere instances on servers … When readyStart Loadtest against WebSphere serversAfter loadtest startExecute test case
![Page 31: The secret art of agile performance testing](https://reader035.fdocuments.in/reader035/viewer/2022062707/55838dfad8b42a8e0c8b500d/html5/thumbnails/31.jpg)
Making complex things easy
@DtdRemoteSud( host = "lab2", name = "WAS7.0", startupPriority = 1, postStartClosure = WaitForWebSphereSudIsUp.class ) private SudInterface webSphereSud;
![Page 32: The secret art of agile performance testing](https://reader035.fdocuments.in/reader035/viewer/2022062707/55838dfad8b42a8e0c8b500d/html5/thumbnails/32.jpg)
Finding the responsible code
Version Control History Lookup
![Page 33: The secret art of agile performance testing](https://reader035.fdocuments.in/reader035/viewer/2022062707/55838dfad8b42a8e0c8b500d/html5/thumbnails/33.jpg)
E-Mail Notification
… better humans
![Page 34: The secret art of agile performance testing](https://reader035.fdocuments.in/reader035/viewer/2022062707/55838dfad8b42a8e0c8b500d/html5/thumbnails/34.jpg)
Wrap Up1. Say „Yes“2. Focus3. Define KPIs4. Redefine Done5. Remove Barriers6. Automate7. Monitor8. Resolve9. Prevent