Automated Testing in a Microservices Architectureeuropeantestingconference.eu/slides18/Bache.pdf ·...
Transcript of Automated Testing in a Microservices Architectureeuropeantestingconference.eu/slides18/Bache.pdf ·...
![Page 1: Automated Testing in a Microservices Architectureeuropeantestingconference.eu/slides18/Bache.pdf · Microservices architecture. ... for reference Event Logs to debug the test. Independent](https://reader031.fdocuments.in/reader031/viewer/2022022013/5b26b3de7f8b9a231b8b4fd2/html5/thumbnails/1.jpg)
www.praqma.com
![Page 2: Automated Testing in a Microservices Architectureeuropeantestingconference.eu/slides18/Bache.pdf · Microservices architecture. ... for reference Event Logs to debug the test. Independent](https://reader031.fdocuments.in/reader031/viewer/2022022013/5b26b3de7f8b9a231b8b4fd2/html5/thumbnails/2.jpg)
www.praqma.com
Techniques for AutomationTesting Microservices
European Testing Conference 2018Emily Bache@emilybache
![Page 3: Automated Testing in a Microservices Architectureeuropeantestingconference.eu/slides18/Bache.pdf · Microservices architecture. ... for reference Event Logs to debug the test. Independent](https://reader031.fdocuments.in/reader031/viewer/2022022013/5b26b3de7f8b9a231b8b4fd2/html5/thumbnails/3.jpg)
www.praqma.com
Emily Bache
Practice Lead for Test Automation
Consultant
Author of
“The Coding Dojo Handbook”
@emilybache
![Page 4: Automated Testing in a Microservices Architectureeuropeantestingconference.eu/slides18/Bache.pdf · Microservices architecture. ... for reference Event Logs to debug the test. Independent](https://reader031.fdocuments.in/reader031/viewer/2022022013/5b26b3de7f8b9a231b8b4fd2/html5/thumbnails/4.jpg)
Invoice Issuer
Invoice Recipient
Pagero Online Cloud Service
![Page 5: Automated Testing in a Microservices Architectureeuropeantestingconference.eu/slides18/Bache.pdf · Microservices architecture. ... for reference Event Logs to debug the test. Independent](https://reader031.fdocuments.in/reader031/viewer/2022022013/5b26b3de7f8b9a231b8b4fd2/html5/thumbnails/5.jpg)
www.praqma.com
Some numbers…
• We were around 45 developers, in 7 teams
• We had around 60 microservices
• Our monolith was in production for nearly 10 years before we started with microservices, 3-4 years ago
![Page 6: Automated Testing in a Microservices Architectureeuropeantestingconference.eu/slides18/Bache.pdf · Microservices architecture. ... for reference Event Logs to debug the test. Independent](https://reader031.fdocuments.in/reader031/viewer/2022022013/5b26b3de7f8b9a231b8b4fd2/html5/thumbnails/6.jpg)
www.praqma.com
Pagero Online’s architecture
![Page 7: Automated Testing in a Microservices Architectureeuropeantestingconference.eu/slides18/Bache.pdf · Microservices architecture. ... for reference Event Logs to debug the test. Independent](https://reader031.fdocuments.in/reader031/viewer/2022022013/5b26b3de7f8b9a231b8b4fd2/html5/thumbnails/7.jpg)
www.praqma.com
A Microservice
• Provides one business capability
• Independently Deployable
• Lightweight API
• Not too big: fits in my head Docker is used to containerise the service
![Page 8: Automated Testing in a Microservices Architectureeuropeantestingconference.eu/slides18/Bache.pdf · Microservices architecture. ... for reference Event Logs to debug the test. Independent](https://reader031.fdocuments.in/reader031/viewer/2022022013/5b26b3de7f8b9a231b8b4fd2/html5/thumbnails/8.jpg)
www.praqma.com
In my experience
Testing a Microservices Architecture
![Page 9: Automated Testing in a Microservices Architectureeuropeantestingconference.eu/slides18/Bache.pdf · Microservices architecture. ... for reference Event Logs to debug the test. Independent](https://reader031.fdocuments.in/reader031/viewer/2022022013/5b26b3de7f8b9a231b8b4fd2/html5/thumbnails/9.jpg)
www.praqma.com
Test Pyramid: Mike Cohn
End-To-End tests exercise the whole system
Unit tests are for individual functions
End-to-End
“Service Tests”
Unit
https://www.mountaingoatsoftware.com/blog/the-forgotten-layer-of-the-test-automation-pyramid
![Page 10: Automated Testing in a Microservices Architectureeuropeantestingconference.eu/slides18/Bache.pdf · Microservices architecture. ... for reference Event Logs to debug the test. Independent](https://reader031.fdocuments.in/reader031/viewer/2022022013/5b26b3de7f8b9a231b8b4fd2/html5/thumbnails/10.jpg)
www.praqma.com
Testing Strategy for the Monolith
Unit tests and end-to-end GUI tests still valid for Microservices architecture
![Page 11: Automated Testing in a Microservices Architectureeuropeantestingconference.eu/slides18/Bache.pdf · Microservices architecture. ... for reference Event Logs to debug the test. Independent](https://reader031.fdocuments.in/reader031/viewer/2022022013/5b26b3de7f8b9a231b8b4fd2/html5/thumbnails/11.jpg)
www.praqma.com
Testing Strategy
Testing a monolith Testing a bunch of microservices
![Page 12: Automated Testing in a Microservices Architectureeuropeantestingconference.eu/slides18/Bache.pdf · Microservices architecture. ... for reference Event Logs to debug the test. Independent](https://reader031.fdocuments.in/reader031/viewer/2022022013/5b26b3de7f8b9a231b8b4fd2/html5/thumbnails/12.jpg)
www.praqma.com
Testing a Microservice
![Page 13: Automated Testing in a Microservices Architectureeuropeantestingconference.eu/slides18/Bache.pdf · Microservices architecture. ... for reference Event Logs to debug the test. Independent](https://reader031.fdocuments.in/reader031/viewer/2022022013/5b26b3de7f8b9a231b8b4fd2/html5/thumbnails/13.jpg)
www.praqma.com
Microservice Tests
• Deploy your service on localhost (and the database if it has one)
• Test uses the public API to trigger functionality
• Mock a response to any calls made to other services or queues
Test Case
µSUT
µSDBcollaborator µS
![Page 14: Automated Testing in a Microservices Architectureeuropeantestingconference.eu/slides18/Bache.pdf · Microservices architecture. ... for reference Event Logs to debug the test. Independent](https://reader031.fdocuments.in/reader031/viewer/2022022013/5b26b3de7f8b9a231b8b4fd2/html5/thumbnails/14.jpg)
www.praqma.com
Faster execution
• In-process testing: Deploy your service with an in-memory database and in-memory api
• ‘internal api’ to set up test data and query internals
Test Case
µSUT
µSDBcollaborator µS
Deploy in Same Process
Internal api
![Page 15: Automated Testing in a Microservices Architectureeuropeantestingconference.eu/slides18/Bache.pdf · Microservices architecture. ... for reference Event Logs to debug the test. Independent](https://reader031.fdocuments.in/reader031/viewer/2022022013/5b26b3de7f8b9a231b8b4fd2/html5/thumbnails/15.jpg)
www.praqma.com
More production-like
• Out-of-process testing: Deploy your service in one process, and the test case in another
• Security needed for ‘internal api’ so only test/non-production code can access
• Easier to performance test
Test Case
µSUT
µSDBcollaborator µS
Deploy in Different Processes
![Page 16: Automated Testing in a Microservices Architectureeuropeantestingconference.eu/slides18/Bache.pdf · Microservices architecture. ... for reference Event Logs to debug the test. Independent](https://reader031.fdocuments.in/reader031/viewer/2022022013/5b26b3de7f8b9a231b8b4fd2/html5/thumbnails/16.jpg)
www.praqma.com
Business-Facing Feature Tests
Test a whole scenario across several microservices
![Page 17: Automated Testing in a Microservices Architectureeuropeantestingconference.eu/slides18/Bache.pdf · Microservices architecture. ... for reference Event Logs to debug the test. Independent](https://reader031.fdocuments.in/reader031/viewer/2022022013/5b26b3de7f8b9a231b8b4fd2/html5/thumbnails/17.jpg)
www.praqma.com
Testing a whole Feature
Test Case
This is where it starts getting really complicated…
![Page 18: Automated Testing in a Microservices Architectureeuropeantestingconference.eu/slides18/Bache.pdf · Microservices architecture. ... for reference Event Logs to debug the test. Independent](https://reader031.fdocuments.in/reader031/viewer/2022022013/5b26b3de7f8b9a231b8b4fd2/html5/thumbnails/18.jpg)
www.praqma.com
Example End-to-End Test Strategy
• Test crucial workflows, not every detail
• Use APIs instead of the GUI for most tests
• Techniques:
• Selective Deployment
• Approval Testing
• Event Monitoring
Keep testing costs as low as possible
![Page 19: Automated Testing in a Microservices Architectureeuropeantestingconference.eu/slides18/Bache.pdf · Microservices architecture. ... for reference Event Logs to debug the test. Independent](https://reader031.fdocuments.in/reader031/viewer/2022022013/5b26b3de7f8b9a231b8b4fd2/html5/thumbnails/19.jpg)
www.praqma.com
Service-layer tests in a monolith
• Test cases are for different business-facing workflows
• Will exercise different components within the monolith
Test Case A
Monolith
Database
Test Case B
![Page 20: Automated Testing in a Microservices Architectureeuropeantestingconference.eu/slides18/Bache.pdf · Microservices architecture. ... for reference Event Logs to debug the test. Independent](https://reader031.fdocuments.in/reader031/viewer/2022022013/5b26b3de7f8b9a231b8b4fd2/html5/thumbnails/20.jpg)
www.praqma.com
Feature tests in microservices
Test Case A
Microservices
Test Case B
• Test cases are for different business-facing workflows
• Will exercise different groups of microservices
• Need not deploy everything for every test case
![Page 21: Automated Testing in a Microservices Architectureeuropeantestingconference.eu/slides18/Bache.pdf · Microservices architecture. ... for reference Event Logs to debug the test. Independent](https://reader031.fdocuments.in/reader031/viewer/2022022013/5b26b3de7f8b9a231b8b4fd2/html5/thumbnails/21.jpg)
www.praqma.com
Selective Deployment
Test Case A
• Need not deploy everything for every test case
• But: all test cases needed a few essential infrastructure services.
![Page 22: Automated Testing in a Microservices Architectureeuropeantestingconference.eu/slides18/Bache.pdf · Microservices architecture. ... for reference Event Logs to debug the test. Independent](https://reader031.fdocuments.in/reader031/viewer/2022022013/5b26b3de7f8b9a231b8b4fd2/html5/thumbnails/22.jpg)
Invoice Issuer
Invoice Recipient
Pagero Online Cloud Service
![Page 23: Automated Testing in a Microservices Architectureeuropeantestingconference.eu/slides18/Bache.pdf · Microservices architecture. ... for reference Event Logs to debug the test. Independent](https://reader031.fdocuments.in/reader031/viewer/2022022013/5b26b3de7f8b9a231b8b4fd2/html5/thumbnails/23.jpg)
www.praqma.com
Parts of a test case
(Almost) all the tests involve these elements
Invoice Issuer
Invoice Recipient
Document Recipient PresentationTest Workflow
Arrange Act Assert
![Page 24: Automated Testing in a Microservices Architectureeuropeantestingconference.eu/slides18/Bache.pdf · Microservices architecture. ... for reference Event Logs to debug the test. Independent](https://reader031.fdocuments.in/reader031/viewer/2022022013/5b26b3de7f8b9a231b8b4fd2/html5/thumbnails/24.jpg)
www.praqma.com
Approval Testing specifies the Assert: compare against an ‘Approved’ result
Invoice Issuer
Invoice Recipient
Document Recipient PresentationTest Workflow
Arrange Act Assert
Approval Testing
![Page 25: Automated Testing in a Microservices Architectureeuropeantestingconference.eu/slides18/Bache.pdf · Microservices architecture. ... for reference Event Logs to debug the test. Independent](https://reader031.fdocuments.in/reader031/viewer/2022022013/5b26b3de7f8b9a231b8b4fd2/html5/thumbnails/25.jpg)
www.praqma.com
• Find the important outputs. Convert them to plain text.
• Use textual diff to decide if the actual output matches the approved version
Recipient Presentation
pdf-to-text utility
Recipient Presentation in plain text
The “Approved” Result as Text
![Page 26: Automated Testing in a Microservices Architectureeuropeantestingconference.eu/slides18/Bache.pdf · Microservices architecture. ... for reference Event Logs to debug the test. Independent](https://reader031.fdocuments.in/reader031/viewer/2022022013/5b26b3de7f8b9a231b8b4fd2/html5/thumbnails/26.jpg)
www.praqma.com
• Minimise test creation work • Minimise test maintenance work • Maximise serendipity - find bugs you didn’t anticipate
Invoice Issuer
Invoice Recipient
Document
Input data varies by test case
Test Workflow
common to many tests
Approved Output
to determine pass/fail
Recipient Presentation
for reference
Elements of a Test Case
![Page 27: Automated Testing in a Microservices Architectureeuropeantestingconference.eu/slides18/Bache.pdf · Microservices architecture. ... for reference Event Logs to debug the test. Independent](https://reader031.fdocuments.in/reader031/viewer/2022022013/5b26b3de7f8b9a231b8b4fd2/html5/thumbnails/27.jpg)
www.praqma.com
Debugging a failing test?
Test Case
Which part broke?
![Page 28: Automated Testing in a Microservices Architectureeuropeantestingconference.eu/slides18/Bache.pdf · Microservices architecture. ... for reference Event Logs to debug the test. Independent](https://reader031.fdocuments.in/reader031/viewer/2022022013/5b26b3de7f8b9a231b8b4fd2/html5/thumbnails/28.jpg)
www.praqma.com
Correlation ID
Pick out all events with ‘123’
![Page 29: Automated Testing in a Microservices Architectureeuropeantestingconference.eu/slides18/Bache.pdf · Microservices architecture. ... for reference Event Logs to debug the test. Independent](https://reader031.fdocuments.in/reader031/viewer/2022022013/5b26b3de7f8b9a231b8b4fd2/html5/thumbnails/29.jpg)
www.praqma.com
Event log recorded in a test case
Helps you to debug what happened when the test fails
![Page 30: Automated Testing in a Microservices Architectureeuropeantestingconference.eu/slides18/Bache.pdf · Microservices architecture. ... for reference Event Logs to debug the test. Independent](https://reader031.fdocuments.in/reader031/viewer/2022022013/5b26b3de7f8b9a231b8b4fd2/html5/thumbnails/30.jpg)
www.praqma.com
Test Case Elements
• Recorded traffic is stored with the approved output
Invoice Issuer
Invoice Recipient
Document
Input data varies by test case
Test Workflow
common to many tests
Approved Output
to determine pass/fail
Recipient Presentation
for reference
Event Logs
to debug the test
![Page 31: Automated Testing in a Microservices Architectureeuropeantestingconference.eu/slides18/Bache.pdf · Microservices architecture. ... for reference Event Logs to debug the test. Independent](https://reader031.fdocuments.in/reader031/viewer/2022022013/5b26b3de7f8b9a231b8b4fd2/html5/thumbnails/31.jpg)
www.praqma.com
Independent Microservices & Teams
Organizing Testing efforts
![Page 32: Automated Testing in a Microservices Architectureeuropeantestingconference.eu/slides18/Bache.pdf · Microservices architecture. ... for reference Event Logs to debug the test. Independent](https://reader031.fdocuments.in/reader031/viewer/2022022013/5b26b3de7f8b9a231b8b4fd2/html5/thumbnails/32.jpg)
www.praqma.com
Testing in the pipeline
unit test
systemtest
manualtest
production
monitoring
Pair Programming,
TDD
version control
microservice test
test larger pieces of code together
more production-like environment
Test costs increase
![Page 33: Automated Testing in a Microservices Architectureeuropeantestingconference.eu/slides18/Bache.pdf · Microservices architecture. ... for reference Event Logs to debug the test. Independent](https://reader031.fdocuments.in/reader031/viewer/2022022013/5b26b3de7f8b9a231b8b4fd2/html5/thumbnails/33.jpg)
Multi-team development
Infrastructure & ArchitectureComponent A
Component B
Components E&FComponents C&D
![Page 34: Automated Testing in a Microservices Architectureeuropeantestingconference.eu/slides18/Bache.pdf · Microservices architecture. ... for reference Event Logs to debug the test. Independent](https://reader031.fdocuments.in/reader031/viewer/2022022013/5b26b3de7f8b9a231b8b4fd2/html5/thumbnails/34.jpg)
www.praqma.com
Testing in the pipeline
unit test
systemtest
manualtest
production
monitoring
Pair Programming,
TDD
version control
component test
![Page 35: Automated Testing in a Microservices Architectureeuropeantestingconference.eu/slides18/Bache.pdf · Microservices architecture. ... for reference Event Logs to debug the test. Independent](https://reader031.fdocuments.in/reader031/viewer/2022022013/5b26b3de7f8b9a231b8b4fd2/html5/thumbnails/35.jpg)
Team B
Team A
Multi-Team CD
unit test
systemtest
manualtest production
monitoring
version control
micro-service
test
These tests are often broken!
![Page 36: Automated Testing in a Microservices Architectureeuropeantestingconference.eu/slides18/Bache.pdf · Microservices architecture. ... for reference Event Logs to debug the test. Independent](https://reader031.fdocuments.in/reader031/viewer/2022022013/5b26b3de7f8b9a231b8b4fd2/html5/thumbnails/36.jpg)
Team A
micro-service
test
Team B
Team pipelines
unit test
systemtest
manualtest production
monitoring
version control
team-system
test
One environment shared by all teams
![Page 37: Automated Testing in a Microservices Architectureeuropeantestingconference.eu/slides18/Bache.pdf · Microservices architecture. ... for reference Event Logs to debug the test. Independent](https://reader031.fdocuments.in/reader031/viewer/2022022013/5b26b3de7f8b9a231b8b4fd2/html5/thumbnails/37.jpg)
Team Amicro-service
test
Team B
Teams Deploy Independently
unit test
team-manual
test
incremental roll-out to productionversion
control
team-system
test
monitoring &manual testing
![Page 38: Automated Testing in a Microservices Architectureeuropeantestingconference.eu/slides18/Bache.pdf · Microservices architecture. ... for reference Event Logs to debug the test. Independent](https://reader031.fdocuments.in/reader031/viewer/2022022013/5b26b3de7f8b9a231b8b4fd2/html5/thumbnails/38.jpg)
Summary
Testing in a Microservices Architecture
@emilybache
team-system
test
monitoring
System Tests
![Page 39: Automated Testing in a Microservices Architectureeuropeantestingconference.eu/slides18/Bache.pdf · Microservices architecture. ... for reference Event Logs to debug the test. Independent](https://reader031.fdocuments.in/reader031/viewer/2022022013/5b26b3de7f8b9a231b8b4fd2/html5/thumbnails/39.jpg)
Summary
Testing in a Microservices Architecture
@emilybache
team-system
test
monitoring
System Tests
![Page 40: Automated Testing in a Microservices Architectureeuropeantestingconference.eu/slides18/Bache.pdf · Microservices architecture. ... for reference Event Logs to debug the test. Independent](https://reader031.fdocuments.in/reader031/viewer/2022022013/5b26b3de7f8b9a231b8b4fd2/html5/thumbnails/40.jpg)
www.praqma.com
The Continuous Delivery and DevOps Company