Tests for Every Branch Using CircleCI and Sauce Labs to Continuously Test CS Curriculum at Code.org
Continuously testing govt.nz Using Specification by ...Continuously testing govt.nz Using...
Transcript of Continuously testing govt.nz Using Specification by ...Continuously testing govt.nz Using...
![Page 1: Continuously testing govt.nz Using Specification by ...Continuously testing govt.nz Using Specification by Example, BeHat and CircleCI Amanda Baker, Developer/Tester, Department of](https://reader030.fdocuments.in/reader030/viewer/2022040613/5f07ec047e708231d41f6e23/html5/thumbnails/1.jpg)
Continuously testing govt.nzUsing Specification by Example, BeHat and CircleCI
Amanda Baker, Developer/Tester, Department of Internal Affairs GIS Allen Geer, Principal Consultant, Assurity Consulting Ltd
![Page 2: Continuously testing govt.nz Using Specification by ...Continuously testing govt.nz Using Specification by Example, BeHat and CircleCI Amanda Baker, Developer/Tester, Department of](https://reader030.fdocuments.in/reader030/viewer/2022040613/5f07ec047e708231d41f6e23/html5/thumbnails/2.jpg)
Government Information Services
![Page 3: Continuously testing govt.nz Using Specification by ...Continuously testing govt.nz Using Specification by Example, BeHat and CircleCI Amanda Baker, Developer/Tester, Department of](https://reader030.fdocuments.in/reader030/viewer/2022040613/5f07ec047e708231d41f6e23/html5/thumbnails/3.jpg)
Technology
Common Web Platform for Infrastructure
Silverstripe 3 /PHP
![Page 4: Continuously testing govt.nz Using Specification by ...Continuously testing govt.nz Using Specification by Example, BeHat and CircleCI Amanda Baker, Developer/Tester, Department of](https://reader030.fdocuments.in/reader030/viewer/2022040613/5f07ec047e708231d41f6e23/html5/thumbnails/4.jpg)
SilverStripe out of date
Customisation
Upgrades high risk
Our challenges
![Page 5: Continuously testing govt.nz Using Specification by ...Continuously testing govt.nz Using Specification by Example, BeHat and CircleCI Amanda Baker, Developer/Tester, Department of](https://reader030.fdocuments.in/reader030/viewer/2022040613/5f07ec047e708231d41f6e23/html5/thumbnails/5.jpg)
No automated testing
No test metrics
Manual testing for upgrades
Our challenges
![Page 6: Continuously testing govt.nz Using Specification by ...Continuously testing govt.nz Using Specification by Example, BeHat and CircleCI Amanda Baker, Developer/Tester, Department of](https://reader030.fdocuments.in/reader030/viewer/2022040613/5f07ec047e708231d41f6e23/html5/thumbnails/6.jpg)
Initial workflow
● Micro-waterfall Agile● Non-Standard Developer Environments
○ (With PHP Stack this is important)● Manual Handoffs to Testing
![Page 7: Continuously testing govt.nz Using Specification by ...Continuously testing govt.nz Using Specification by Example, BeHat and CircleCI Amanda Baker, Developer/Tester, Department of](https://reader030.fdocuments.in/reader030/viewer/2022040613/5f07ec047e708231d41f6e23/html5/thumbnails/7.jpg)
Micro Waterfall Antipattern
TesterDeveloperProduct Owner
Here’s a User Story Work on It
Im Done With The User Story Now You Test It
I found a defect, Let’s go through this futile exercise again where we hand back off to each other ad infinitum
Testing Found this defect, lets have a conversation about this feature without them to further confuse the matter.
![Page 8: Continuously testing govt.nz Using Specification by ...Continuously testing govt.nz Using Specification by Example, BeHat and CircleCI Amanda Baker, Developer/Tester, Department of](https://reader030.fdocuments.in/reader030/viewer/2022040613/5f07ec047e708231d41f6e23/html5/thumbnails/8.jpg)
Specification by Example - 3 “Amigos”
Tester
Developer
Product Owner
New feature
Gherkin examples
Automated, executable test cases!
+
=
![Page 9: Continuously testing govt.nz Using Specification by ...Continuously testing govt.nz Using Specification by Example, BeHat and CircleCI Amanda Baker, Developer/Tester, Department of](https://reader030.fdocuments.in/reader030/viewer/2022040613/5f07ec047e708231d41f6e23/html5/thumbnails/9.jpg)
Continuous Integration - Testing
We want to execute and report upon a suite of unit and behaviour tests
![Page 10: Continuously testing govt.nz Using Specification by ...Continuously testing govt.nz Using Specification by Example, BeHat and CircleCI Amanda Baker, Developer/Tester, Department of](https://reader030.fdocuments.in/reader030/viewer/2022040613/5f07ec047e708231d41f6e23/html5/thumbnails/10.jpg)
CI Workflow
Developer
Commits Code
Checks out commit
Executes unit tests Executes BDD tests
Reports results back to
Create Pull Request
![Page 11: Continuously testing govt.nz Using Specification by ...Continuously testing govt.nz Using Specification by Example, BeHat and CircleCI Amanda Baker, Developer/Tester, Department of](https://reader030.fdocuments.in/reader030/viewer/2022040613/5f07ec047e708231d41f6e23/html5/thumbnails/11.jpg)
Visualise
• Stocktake
• Total of over 300 features (Gazette almost half)
• Tricky date / time driven features
• Used ‘Spec By Example’ to describe each feature
1
![Page 12: Continuously testing govt.nz Using Specification by ...Continuously testing govt.nz Using Specification by Example, BeHat and CircleCI Amanda Baker, Developer/Tester, Department of](https://reader030.fdocuments.in/reader030/viewer/2022040613/5f07ec047e708231d41f6e23/html5/thumbnails/12.jpg)
• Couldn’t do everything
• Determine high risk features and focus on those first
• How important is this feature to the product?
• If it breaks will the site still work?
Prioritise
2
![Page 13: Continuously testing govt.nz Using Specification by ...Continuously testing govt.nz Using Specification by Example, BeHat and CircleCI Amanda Baker, Developer/Tester, Department of](https://reader030.fdocuments.in/reader030/viewer/2022040613/5f07ec047e708231d41f6e23/html5/thumbnails/13.jpg)
![Page 14: Continuously testing govt.nz Using Specification by ...Continuously testing govt.nz Using Specification by Example, BeHat and CircleCI Amanda Baker, Developer/Tester, Department of](https://reader030.fdocuments.in/reader030/viewer/2022040613/5f07ec047e708231d41f6e23/html5/thumbnails/14.jpg)
• Wrote all features test in the Gherkin syntax
• Which then was living documentation for the team
Given there is a home page with the content
“Welcome to DIA”
When I go to “/”
Then I should see “Welcome to DIA”
Gherkinise
3
![Page 15: Continuously testing govt.nz Using Specification by ...Continuously testing govt.nz Using Specification by Example, BeHat and CircleCI Amanda Baker, Developer/Tester, Department of](https://reader030.fdocuments.in/reader030/viewer/2022040613/5f07ec047e708231d41f6e23/html5/thumbnails/15.jpg)
Change of mentality to Test Driven Development
![Page 16: Continuously testing govt.nz Using Specification by ...Continuously testing govt.nz Using Specification by Example, BeHat and CircleCI Amanda Baker, Developer/Tester, Department of](https://reader030.fdocuments.in/reader030/viewer/2022040613/5f07ec047e708231d41f6e23/html5/thumbnails/16.jpg)
Build Quality into development process
![Page 17: Continuously testing govt.nz Using Specification by ...Continuously testing govt.nz Using Specification by Example, BeHat and CircleCI Amanda Baker, Developer/Tester, Department of](https://reader030.fdocuments.in/reader030/viewer/2022040613/5f07ec047e708231d41f6e23/html5/thumbnails/17.jpg)
Visualisation and Metrics
Culture of Improvement
![Page 18: Continuously testing govt.nz Using Specification by ...Continuously testing govt.nz Using Specification by Example, BeHat and CircleCI Amanda Baker, Developer/Tester, Department of](https://reader030.fdocuments.in/reader030/viewer/2022040613/5f07ec047e708231d41f6e23/html5/thumbnails/18.jpg)
Path to automation isn’t binary
![Page 19: Continuously testing govt.nz Using Specification by ...Continuously testing govt.nz Using Specification by Example, BeHat and CircleCI Amanda Baker, Developer/Tester, Department of](https://reader030.fdocuments.in/reader030/viewer/2022040613/5f07ec047e708231d41f6e23/html5/thumbnails/19.jpg)
Involve everyone in the team
![Page 20: Continuously testing govt.nz Using Specification by ...Continuously testing govt.nz Using Specification by Example, BeHat and CircleCI Amanda Baker, Developer/Tester, Department of](https://reader030.fdocuments.in/reader030/viewer/2022040613/5f07ec047e708231d41f6e23/html5/thumbnails/20.jpg)
Automation only works if humanised