Testing Practices in Playtech Estonia - ut › ... › 2015_fall › uploads › Main ›...
Transcript of Testing Practices in Playtech Estonia - ut › ... › 2015_fall › uploads › Main ›...
![Page 1: Testing Practices in Playtech Estonia - ut › ... › 2015_fall › uploads › Main › SE2015-lectu… · Testing Practices in Playtech Estonia Kerli Rungi Quality Assurance Manager](https://reader033.fdocuments.in/reader033/viewer/2022042322/5f0c72347e708231d43570f1/html5/thumbnails/1.jpg)
Testing Practices
in Playtech Estonia
Kerli Rungi
Quality Assurance Manager
Playtech Estonia, Casino Unit
![Page 2: Testing Practices in Playtech Estonia - ut › ... › 2015_fall › uploads › Main › SE2015-lectu… · Testing Practices in Playtech Estonia Kerli Rungi Quality Assurance Manager](https://reader033.fdocuments.in/reader033/viewer/2022042322/5f0c72347e708231d43570f1/html5/thumbnails/2.jpg)
Agenda
• Playtech – Organization & Products
• General QA Concept
• General Testing Practices
• Supportive QA Processes
• Testing Challenges in Playtech
• Common Misconceptions about Testing
![Page 3: Testing Practices in Playtech Estonia - ut › ... › 2015_fall › uploads › Main › SE2015-lectu… · Testing Practices in Playtech Estonia Kerli Rungi Quality Assurance Manager](https://reader033.fdocuments.in/reader033/viewer/2022042322/5f0c72347e708231d43570f1/html5/thumbnails/3.jpg)
Playtech as an Organization
• World’s largest online gaming software supplier
• Founded in 1999 (Estonia, Israel)
• ~5000 employees in 13 countries (666 in Estonia)
• Cooperation with entertainment corporations (e.g.
Marvel, HBO, MGM) and governmental regulators (e.g.
Italy, Finland, Spain)
• Traded on London Stock Exchange Main Market
![Page 4: Testing Practices in Playtech Estonia - ut › ... › 2015_fall › uploads › Main › SE2015-lectu… · Testing Practices in Playtech Estonia Kerli Rungi Quality Assurance Manager](https://reader033.fdocuments.in/reader033/viewer/2022042322/5f0c72347e708231d43570f1/html5/thumbnails/4.jpg)
Playtech’s Products
• End-to-end solution provider
• Wide range of fully integrated products – Casino, Poker,
Bingo, Live, TV, Sports, IMS, …
• Web, Download and Mobile platform
• Integrations with 3rd party systems
• More than 100 different product components
![Page 5: Testing Practices in Playtech Estonia - ut › ... › 2015_fall › uploads › Main › SE2015-lectu… · Testing Practices in Playtech Estonia Kerli Rungi Quality Assurance Manager](https://reader033.fdocuments.in/reader033/viewer/2022042322/5f0c72347e708231d43570f1/html5/thumbnails/5.jpg)
![Page 6: Testing Practices in Playtech Estonia - ut › ... › 2015_fall › uploads › Main › SE2015-lectu… · Testing Practices in Playtech Estonia Kerli Rungi Quality Assurance Manager](https://reader033.fdocuments.in/reader033/viewer/2022042322/5f0c72347e708231d43570f1/html5/thumbnails/6.jpg)
![Page 7: Testing Practices in Playtech Estonia - ut › ... › 2015_fall › uploads › Main › SE2015-lectu… · Testing Practices in Playtech Estonia Kerli Rungi Quality Assurance Manager](https://reader033.fdocuments.in/reader033/viewer/2022042322/5f0c72347e708231d43570f1/html5/thumbnails/7.jpg)
General QA Concept
• Quality assurance is a proactive management
approach to assure product quality and stability by
improving processes, work culture, methods etc.
• Quality control is a set of reactive means to measure
version quality for management decision regarding
deployment.
![Page 8: Testing Practices in Playtech Estonia - ut › ... › 2015_fall › uploads › Main › SE2015-lectu… · Testing Practices in Playtech Estonia Kerli Rungi Quality Assurance Manager](https://reader033.fdocuments.in/reader033/viewer/2022042322/5f0c72347e708231d43570f1/html5/thumbnails/8.jpg)
General QA Concept (cont.)
![Page 9: Testing Practices in Playtech Estonia - ut › ... › 2015_fall › uploads › Main › SE2015-lectu… · Testing Practices in Playtech Estonia Kerli Rungi Quality Assurance Manager](https://reader033.fdocuments.in/reader033/viewer/2022042322/5f0c72347e708231d43570f1/html5/thumbnails/9.jpg)
General QA Concept (cont.)
• Measure, learn and improve mentality
• Processes, work methods and tools
• Internal knowledge sharing
• Wider testing scope and coverage
• Lessons learned
![Page 10: Testing Practices in Playtech Estonia - ut › ... › 2015_fall › uploads › Main › SE2015-lectu… · Testing Practices in Playtech Estonia Kerli Rungi Quality Assurance Manager](https://reader033.fdocuments.in/reader033/viewer/2022042322/5f0c72347e708231d43570f1/html5/thumbnails/10.jpg)
Who is Responsible for Quality?
![Page 11: Testing Practices in Playtech Estonia - ut › ... › 2015_fall › uploads › Main › SE2015-lectu… · Testing Practices in Playtech Estonia Kerli Rungi Quality Assurance Manager](https://reader033.fdocuments.in/reader033/viewer/2022042322/5f0c72347e708231d43570f1/html5/thumbnails/11.jpg)
General Testing Activities
• Development Oriented
• Project Specific
• Release Closure
• Automated Testing
![Page 12: Testing Practices in Playtech Estonia - ut › ... › 2015_fall › uploads › Main › SE2015-lectu… · Testing Practices in Playtech Estonia Kerli Rungi Quality Assurance Manager](https://reader033.fdocuments.in/reader033/viewer/2022042322/5f0c72347e708231d43570f1/html5/thumbnails/12.jpg)
Development Oriented Activities
• Design & other documentation review/approval
• Unit testing
• Code review
• Developer’s alpha testing
• Continuous Integration
– Nightly builds & autotests
– Post-compilation autotests
![Page 13: Testing Practices in Playtech Estonia - ut › ... › 2015_fall › uploads › Main › SE2015-lectu… · Testing Practices in Playtech Estonia Kerli Rungi Quality Assurance Manager](https://reader033.fdocuments.in/reader033/viewer/2022042322/5f0c72347e708231d43570f1/html5/thumbnails/13.jpg)
Project Specific Activities
• Test planning & Test design
• Define test objectives (consider
risks, best practices)
• Determine prerequisites
• Define detailed test cases
• Artifacts
• Test plan – What to test?
• Test design – How to test?
![Page 14: Testing Practices in Playtech Estonia - ut › ... › 2015_fall › uploads › Main › SE2015-lectu… · Testing Practices in Playtech Estonia Kerli Rungi Quality Assurance Manager](https://reader033.fdocuments.in/reader033/viewer/2022042322/5f0c72347e708231d43570f1/html5/thumbnails/14.jpg)
Project Specific Activities (cont.)
• Testing - Iterative manual testing to
verify that the project meets its design
specifications and other functional
and non-functional requirements
• Functional testing
• Behavior, game logic, game rules
• Game Payout
• Compliance to regulations
![Page 15: Testing Practices in Playtech Estonia - ut › ... › 2015_fall › uploads › Main › SE2015-lectu… · Testing Practices in Playtech Estonia Kerli Rungi Quality Assurance Manager](https://reader033.fdocuments.in/reader033/viewer/2022042322/5f0c72347e708231d43570f1/html5/thumbnails/15.jpg)
Project Specific Activities (cont.)
• Non-functional testing
• Usability
• Localization/translations
• Performance
• Failover & recovery
• Security
• Exploratory testing
![Page 16: Testing Practices in Playtech Estonia - ut › ... › 2015_fall › uploads › Main › SE2015-lectu… · Testing Practices in Playtech Estonia Kerli Rungi Quality Assurance Manager](https://reader033.fdocuments.in/reader033/viewer/2022042322/5f0c72347e708231d43570f1/html5/thumbnails/16.jpg)
Project Specific Activities (cont.)
• Payout Assurance – payout testing activities embedded
to the whole game lifecycle
• Payout simulation
• Detailed math sheetGame Design
• Payout unit testsDevelopment
• 100 Million game rounds
• Real server logic executionPayout tests
• 10 Million game rounds
• Load & Performance Non functional
tests
• Math team
• Review all tests resultsGame approval
![Page 17: Testing Practices in Playtech Estonia - ut › ... › 2015_fall › uploads › Main › SE2015-lectu… · Testing Practices in Playtech Estonia Kerli Rungi Quality Assurance Manager](https://reader033.fdocuments.in/reader033/viewer/2022042322/5f0c72347e708231d43570f1/html5/thumbnails/17.jpg)
Release Closure Activities
• Regression testing
• Performance testing
• Client compilation, installation & upgrade testing
• Compatibility testing
• Middleware, browser & platform based
• Between different product versions
• Staging – cross product integration testing
• Rollback path testing
• Post-upload testing
![Page 18: Testing Practices in Playtech Estonia - ut › ... › 2015_fall › uploads › Main › SE2015-lectu… · Testing Practices in Playtech Estonia Kerli Rungi Quality Assurance Manager](https://reader033.fdocuments.in/reader033/viewer/2022042322/5f0c72347e708231d43570f1/html5/thumbnails/18.jpg)
Automated Testing
• In-house built java-based FW
• Dedicated team
• Mandatory step in each project
• Performance testing
• Regression test cases
• Staging test cases
• Continuous Integration activities
![Page 19: Testing Practices in Playtech Estonia - ut › ... › 2015_fall › uploads › Main › SE2015-lectu… · Testing Practices in Playtech Estonia Kerli Rungi Quality Assurance Manager](https://reader033.fdocuments.in/reader033/viewer/2022042322/5f0c72347e708231d43570f1/html5/thumbnails/19.jpg)
Other Supportive QA Processes
• Documentation review & approval
• Risk management
• Project KPI monitoring
• Project closure meetings & lessons learned
• Production defects RCA
• Company-wide masstests
• …
![Page 20: Testing Practices in Playtech Estonia - ut › ... › 2015_fall › uploads › Main › SE2015-lectu… · Testing Practices in Playtech Estonia Kerli Rungi Quality Assurance Manager](https://reader033.fdocuments.in/reader033/viewer/2022042322/5f0c72347e708231d43570f1/html5/thumbnails/20.jpg)
Testing Challenges in Playtech
• Big & complex systems
– Knowledge sharing
– Integrations with 3rd parties (mock ups)
– Many different technologies & tools
– Instability of testing environments
– Difficult to troubleshoot & localize problems
– High test maintenance cost
• Effective/optimal compatibility testing – mobile devices,
browsers, operating systems.
![Page 21: Testing Practices in Playtech Estonia - ut › ... › 2015_fall › uploads › Main › SE2015-lectu… · Testing Practices in Playtech Estonia Kerli Rungi Quality Assurance Manager](https://reader033.fdocuments.in/reader033/viewer/2022042322/5f0c72347e708231d43570f1/html5/thumbnails/21.jpg)
Testing Challenges in Playtech (cont.)
• High expectations towards quality
• Communication, communication, communication!
![Page 22: Testing Practices in Playtech Estonia - ut › ... › 2015_fall › uploads › Main › SE2015-lectu… · Testing Practices in Playtech Estonia Kerli Rungi Quality Assurance Manager](https://reader033.fdocuments.in/reader033/viewer/2022042322/5f0c72347e708231d43570f1/html5/thumbnails/22.jpg)
Misconceptions about Testing
• Testing is simple – it’s just GUI-checking
• Testers don’t know and don’t have to know anything
about programming
• Tester’s responsibility is to localize and fix defects
• Tester’s ensure the quality