Continuous Integration - A Performance Engineer's Tale
-
Upload
stephen-thair -
Category
Technology
-
view
1.797 -
download
0
description
Transcript of Continuous Integration - A Performance Engineer's Tale
![Page 1: Continuous Integration - A Performance Engineer's Tale](https://reader033.fdocuments.in/reader033/viewer/2022061305/549a3c88b47959604d8b58ce/html5/thumbnails/1.jpg)
CONFIDENTIAL and not for reproduction without prior written consent. © of the Sporting Exchange Limited.
London Web Meetup – Dec 2011Continuous Integration – A Performance Engineers Tale
![Page 2: Continuous Integration - A Performance Engineer's Tale](https://reader033.fdocuments.in/reader033/viewer/2022061305/549a3c88b47959604d8b58ce/html5/thumbnails/2.jpg)
CONFIDENTIAL and not for reproduction without prior written consent. © of the Sporting Exchange Limited.
2
“WE WILL DO CONTINUOUS INTEGRATION TESTING”1128th commandment - The Management
![Page 3: Continuous Integration - A Performance Engineer's Tale](https://reader033.fdocuments.in/reader033/viewer/2022061305/549a3c88b47959604d8b58ce/html5/thumbnails/3.jpg)
CONFIDENTIAL and not for reproduction without prior written consent. © of the Sporting Exchange Limited.
INITIAL THOUGHTS
•Automate all of our Performance Tests
•Add tests to the Build process
3
CI
![Page 4: Continuous Integration - A Performance Engineer's Tale](https://reader033.fdocuments.in/reader033/viewer/2022061305/549a3c88b47959604d8b58ce/html5/thumbnails/4.jpg)
CONFIDENTIAL and not for reproduction without prior written consent. © of the Sporting Exchange Limited.
WHY IT WONT WORK
•Performance testing needs a functional application
•The length of time needed to gather significant amount of data points
•Performance testing requires analysis of results to find meaningful conclusions
4
![Page 5: Continuous Integration - A Performance Engineer's Tale](https://reader033.fdocuments.in/reader033/viewer/2022061305/549a3c88b47959604d8b58ce/html5/thumbnails/5.jpg)
CONFIDENTIAL and not for reproduction without prior written consent. © of the Sporting Exchange Limited.
MORE INFORMATION
5
we'll receive immediate
feedback
continuous integration (CI) implements continuous processes of applying quality
control
For a build to be self-testing the failure of a
test should cause the build to fail.
![Page 6: Continuous Integration - A Performance Engineer's Tale](https://reader033.fdocuments.in/reader033/viewer/2022061305/549a3c88b47959604d8b58ce/html5/thumbnails/6.jpg)
CONFIDENTIAL and not for reproduction without prior written consent. © of the Sporting Exchange Limited.
GOALS FOR PERFORMANCE
•Measure Performance as part of every build
•Compare between builds (regression)•Alert if performance deviates from normal
•Fail the build if performance does not meet SLA’s
•Automatic process
6
![Page 7: Continuous Integration - A Performance Engineer's Tale](https://reader033.fdocuments.in/reader033/viewer/2022061305/549a3c88b47959604d8b58ce/html5/thumbnails/7.jpg)
CONFIDENTIAL and not for reproduction without prior written consent. © of the Sporting Exchange Limited.
7
Build
Delivery Team
![Page 8: Continuous Integration - A Performance Engineer's Tale](https://reader033.fdocuments.in/reader033/viewer/2022061305/549a3c88b47959604d8b58ce/html5/thumbnails/8.jpg)
CONFIDENTIAL and not for reproduction without prior written consent. © of the Sporting Exchange Limited.
8
Build
Environment
Delivery Team
Deploy
Ready for
Test?
![Page 9: Continuous Integration - A Performance Engineer's Tale](https://reader033.fdocuments.in/reader033/viewer/2022061305/549a3c88b47959604d8b58ce/html5/thumbnails/9.jpg)
CONFIDENTIAL and not for reproduction without prior written consent. © of the Sporting Exchange Limited.
9
Build
Environment
Performance Measurement
s
Perf Trends
Delivery Team
Deploy
Ready for
Test?
![Page 10: Continuous Integration - A Performance Engineer's Tale](https://reader033.fdocuments.in/reader033/viewer/2022061305/549a3c88b47959604d8b58ce/html5/thumbnails/10.jpg)
CONFIDENTIAL and not for reproduction without prior written consent. © of the Sporting Exchange Limited.
10
Build
Environment
Performance Measurement
s
Perf Trends
Delivery Team
Alert
Deploy
Fail Fail to meet
SLA’s?
Deviation from
normal?
Ready for
Test?
![Page 11: Continuous Integration - A Performance Engineer's Tale](https://reader033.fdocuments.in/reader033/viewer/2022061305/549a3c88b47959604d8b58ce/html5/thumbnails/11.jpg)
CONFIDENTIAL and not for reproduction without prior written consent. © of the Sporting Exchange Limited.
11
Build
Environment
ContinuousLoad
Injection
Production logs
Performance Measurement
s
Perf Trends
Delivery Team
Usage Profile
Alert
Deploy
Fail Fail to meet
SLA’s?
Deviation from
normal?
Ready for
Test?
![Page 12: Continuous Integration - A Performance Engineer's Tale](https://reader033.fdocuments.in/reader033/viewer/2022061305/549a3c88b47959604d8b58ce/html5/thumbnails/12.jpg)
CONFIDENTIAL and not for reproduction without prior written consent. © of the Sporting Exchange Limited.
12
Build
Environment
ContinuousLoad
Injection
Production logs
End User Response
Perf Trends
Delivery Team
Usage Profile
Alert
Deploy
Service/API response
Measurements
System monitors
Fail Fail to meet
SLA’s?
Deviation from
normal?
Ready for
Test?
![Page 13: Continuous Integration - A Performance Engineer's Tale](https://reader033.fdocuments.in/reader033/viewer/2022061305/549a3c88b47959604d8b58ce/html5/thumbnails/13.jpg)
CONFIDENTIAL and not for reproduction without prior written consent. © of the Sporting Exchange Limited.
13
Build
Environment
ContinuousLoad
Injection
Production logs
End User Response
Perf Trends
Perf Details
Delivery Team
Usage Profile
For Each Data Point
Alert
Deploy
Service/API response
Measurements
System monitors
Fail Fail to meet
SLA’s?
Deviation from
normal?
Ready for
Test?
![Page 14: Continuous Integration - A Performance Engineer's Tale](https://reader033.fdocuments.in/reader033/viewer/2022061305/549a3c88b47959604d8b58ce/html5/thumbnails/14.jpg)
CONFIDENTIAL and not for reproduction without prior written consent. © of the Sporting Exchange Limited.
14
“THE POINT OF TESTING IS TO FLUSH OUT, UNDER CONTROLLED CONDITIONS, ANY PROBLEM THAT THE SYSTEM WILL HAVE IN PRODUCTION. A SIGNIFICANT PART OF THIS IS THE ENVIRONMENT WITHIN WHICH THE PRODUCTION SYSTEM WILL RUN. IF YOU TEST IN A DIFFERENT ENVIRONMENT, EVERY DIFFERENCE RESULTS IN A RISK THAT WHAT HAPPENS UNDER TEST WON’T HAPPEN IN PRODUCTION”
MARTIN FOWLER
![Page 15: Continuous Integration - A Performance Engineer's Tale](https://reader033.fdocuments.in/reader033/viewer/2022061305/549a3c88b47959604d8b58ce/html5/thumbnails/15.jpg)
CONFIDENTIAL and not for reproduction without prior written consent. © of the Sporting Exchange Limited.
15
Build
Production Environment
End User Response
Perf Trends
Perf Details
Delivery Team
For Each Data Point
Alert
Deploy
Service/API response
Measurements
System monitors
Fail Fail to meet
SLA’s?
Deviation from
normal?
Ready for
Test?
![Page 16: Continuous Integration - A Performance Engineer's Tale](https://reader033.fdocuments.in/reader033/viewer/2022061305/549a3c88b47959604d8b58ce/html5/thumbnails/16.jpg)
CONFIDENTIAL and not for reproduction without prior written consent. © of the Sporting Exchange Limited.
16
Build
Environment
ContinuousLoad
Injection
Production logs
End User Response
Perf Trends
Perf Details
Delivery Team
Usage Profile
For Each Data Point
Alert
Deploy
Service/API response
Measurements
System monitors
Fail Fail to meet
SLA’s?
Deviation from
normal?
Ready for
Test?
![Page 17: Continuous Integration - A Performance Engineer's Tale](https://reader033.fdocuments.in/reader033/viewer/2022061305/549a3c88b47959604d8b58ce/html5/thumbnails/17.jpg)
CONFIDENTIAL and not for reproduction without prior written consent. © of the Sporting Exchange Limited.
17
Build
Environment
ContinuousLoad
Injection
Production logs
End User Response
Perf Trends
Perf Details
Delivery Team
Usage Profile
For Each Data Point
Alert
Deploy
Service/API response
Measurements
System monitors
Fail Fail to meet
SLA’s?
Deviation from
normal?
Ready for
Test?
LoadRunner
![Page 18: Continuous Integration - A Performance Engineer's Tale](https://reader033.fdocuments.in/reader033/viewer/2022061305/549a3c88b47959604d8b58ce/html5/thumbnails/18.jpg)
CONFIDENTIAL and not for reproduction without prior written consent. © of the Sporting Exchange Limited.
18
Build
Environment
ContinuousLoad
Injection
Production logs
End User Response
Perf Trends
Perf Details
Delivery Team
Usage Profile
For Each Data Point
Alert
Deploy
Service/API response
Measurements
System monitors
Fail Fail to meet
SLA’s?
Deviation from
normal?
Ready for
Test?
Keynote
![Page 19: Continuous Integration - A Performance Engineer's Tale](https://reader033.fdocuments.in/reader033/viewer/2022061305/549a3c88b47959604d8b58ce/html5/thumbnails/19.jpg)
CONFIDENTIAL and not for reproduction without prior written consent. © of the Sporting Exchange Limited.
19
Build
Environment
ContinuousLoad
Injection
Production logs
End User Response
Perf Trends
Perf Details
Delivery Team
Usage Profile
For Each Data Point
Alert
Deploy
Service/API response
Measurements
System monitors
Fail Fail to meet
SLA’s?
Deviation from
normal?
Ready for
Test?
Dynatrace Ajax Premium
![Page 20: Continuous Integration - A Performance Engineer's Tale](https://reader033.fdocuments.in/reader033/viewer/2022061305/549a3c88b47959604d8b58ce/html5/thumbnails/20.jpg)
CONFIDENTIAL and not for reproduction without prior written consent. © of the Sporting Exchange Limited.
ADVANTAGES
•Allows standard performance regression testing to be fully automated, freeing up time for more complex performance work
•Separating load injection from performance measurements enables performance to be included in build cycle
•Provides immediate feedback to delivery teams20
![Page 21: Continuous Integration - A Performance Engineer's Tale](https://reader033.fdocuments.in/reader033/viewer/2022061305/549a3c88b47959604d8b58ce/html5/thumbnails/21.jpg)
CONFIDENTIAL and not for reproduction without prior written consent. © of the Sporting Exchange Limited.
NOT IN SCOPE
•Capacity Testing•Soak Testing•Failure Testing•Stress Testing
21
Still Plenty of Traditional work to do
![Page 22: Continuous Integration - A Performance Engineer's Tale](https://reader033.fdocuments.in/reader033/viewer/2022061305/549a3c88b47959604d8b58ce/html5/thumbnails/22.jpg)
CONFIDENTIAL and not for reproduction without prior written consent. © of the Sporting Exchange Limited.
THINGS I LEARNT
•Keep an open mind to new stuff
22
![Page 23: Continuous Integration - A Performance Engineer's Tale](https://reader033.fdocuments.in/reader033/viewer/2022061305/549a3c88b47959604d8b58ce/html5/thumbnails/23.jpg)
CONFIDENTIAL and not for reproduction without prior written consent. © of the Sporting Exchange Limited.
THINGS I LEARNT
•Keep an open mind to new stuff•Give it more than 2 minutes of thought
23
![Page 24: Continuous Integration - A Performance Engineer's Tale](https://reader033.fdocuments.in/reader033/viewer/2022061305/549a3c88b47959604d8b58ce/html5/thumbnails/24.jpg)
CONFIDENTIAL and not for reproduction without prior written consent. © of the Sporting Exchange Limited.
THINGS I LEARNT
•Keep an open mind to new stuff•Give it more than 2 minutes of thought•Get past the reasons it won’t work
24
![Page 25: Continuous Integration - A Performance Engineer's Tale](https://reader033.fdocuments.in/reader033/viewer/2022061305/549a3c88b47959604d8b58ce/html5/thumbnails/25.jpg)
CONFIDENTIAL and not for reproduction without prior written consent. © of the Sporting Exchange Limited.
THINGS I LEARNT
•Keep an open mind to new stuff•Give it more than 2 minutes of thought•Get past the reasons it won’t work•Use Google
25
![Page 26: Continuous Integration - A Performance Engineer's Tale](https://reader033.fdocuments.in/reader033/viewer/2022061305/549a3c88b47959604d8b58ce/html5/thumbnails/26.jpg)
CONFIDENTIAL and not for reproduction without prior written consent. © of the Sporting Exchange Limited.
THINGS I LEARNT
•Keep an open mind to new stuff•Give it more than 2 minutes of thought•Get past the reasons it won’t work•Use Google•Design for the ideal, compromise later
26
![Page 27: Continuous Integration - A Performance Engineer's Tale](https://reader033.fdocuments.in/reader033/viewer/2022061305/549a3c88b47959604d8b58ce/html5/thumbnails/27.jpg)
CONFIDENTIAL and not for reproduction without prior written consent. © of the Sporting Exchange Limited.
THANK YOU