Agile Load Testing in the Real World
-
Upload
lee-barnes -
Category
Technology
-
view
129 -
download
3
description
Transcript of Agile Load Testing in the Real World
© Utopia Solutions
Agile Load Testing in the Real World
Lee Barnes, CTO Utopia Solutions @USI_LeeBarnes
© Utopia Solutions 2
Key Questions
What needs to change?
How do we make those changes?
Why don’t traditional performance testing models fit nicely with agile?
© Utopia Solutions 3
Case Study
Background
• Large global manufacturer • Performance Testing CoE in operation for ~10 yrs • Seen as a huge value to the enterprise • Began moving toward agile a few years ago…
and the luster of the CoE began to dull…
© Utopia Solutions 4
Traditional Performance Testing Model
Center of Excellence
Project 1
Project 2 Project N
Provides Consistency in… • Resources / Skills • Processes • Deliverable Quality • Technology
Request Test
Project Team
Center of Excellence
Perform Discovery
Develop & Execute
Deliver Results Evaluate Results
Info / Environment
© Utopia Solutions 5
• One size fits all • Strict entry criteria • Performed late in SDLC • One-way communication of
results
Service Characteristics
CoE in the Waterfall Years
Plan Design Build Test Deploy
Performance Testing
Too late!
© Utopia Solutions 6
Agile Development
Analyze
Develop
Integrate
Acceptance Test
Deploy
Where does performance testing fit?
© Utopia Solutions 7
Early Attempts on Agile Projects
Sprint 1 Sprint 2 Sprint 3 … Deploy
Agile
Performance Testing
Same Process…
Same Risk!
© Utopia Solutions 8
Early Attempts on Agile Projects
Sprint 1 Sprint 2 Sprint 3 … Deploy
Agile
Performance Testing
Performance Testing
Performance Testing
Performance Testing
CoE Approach • Rigid • 4 week test cycle
Agile Approach • Flexible • 2 week sprints
+ =
© Utopia Solutions
Back to the drawing board…
© Utopia Solutions 10
Early Cycle / Ongoing
Performance Analysis
Establish Objectives
Ultimate Goal…
Late Cycle Performance
Testing Shift Left
© Utopia Solutions 11
Identify Gaps
What’s stopping us from analyzing performance continuously?
• Performance objectives weren’t part of user stories
• Agile teams lacked performance engineering expertise
• Performance testing knowledge and tools were locked in the CoE
• APM tools / skills were non-existent
© Utopia Solutions 12
Disclaimers
It was not easy.
The changes discussed on the next few slides were implemented over a two year duration.
It was not painless.
© Utopia Solutions 13
Performance Testing in an Agile Environment
Build performance objectives in to user stories
Make performance everyone’s job
Create a role to coordinate performance activities
CoE performance test architects become mentors
Begin developing a performance focused culture
© Utopia Solutions 14
Performance Testing in an Agile Environment
Map business and end user requirements onto system components
Create both single and multi user tests
Benchmark in development / QA environment
Look for trends in measurements vs. absolutes
Use service virtualization to stub out 3rd party or yet to be developed components
Execute unit performance tests as part of continuous integration
Implement Unit / Component Performance Testing
© Utopia Solutions 15
Waterfall Agile
Responsibility for Performance
Performance CoE
Role of CoE End of cycle PT
Role of development
Throw system over wall Wait for results
Tools Traditional load test tools
Environment Dedicated / production-like Deliverable Boiler plate results delivered
just prior to release
Waterfall Agile
Responsibility for Performance
Performance CoE Everyone
Role of CoE End of cycle PT Provide mentorship to agile team Pre-deployment system PT
Role of development
Throw system over wall Wait for results
Continuous focus on performance
Tools Traditional load test tools Traditional load test tools APM tools Service virtualization Continuous integration
Environment Dedicated / production-like Development / desktop Deliverable Boiler plate results delivered
just prior to release Continuous feed of actionable data
Performance Testing in an Agile Environment
Transformation Summary
© Utopia Solutions 16
Summary
Key takeaways… • Agile performance testing is not more
/ quicker performance tests • Success requires an shift in people,
process and technology
© Utopia Solutions 17
Questions…
…. and Answers!
Direct future questions to: Lee Barnes Founder and CTO Utopia Solutions, Inc. Email: [email protected] Twitter: twitter.com/USI_LeeBarnes LinkedIn: linkedin.com/in/leebarnes Blog: www.RaiseYourMQ.com
?