[Thao Vo] Deadly Traps of Automation Testing
-
Upload
ho-chi-minh-city-software-testing-club -
Category
Software
-
view
381 -
download
1
description
Transcript of [Thao Vo] Deadly Traps of Automation Testing
Deadly Traps of Test Automation
STC JULY 2014
THAO VO
A landscape
2
1. Automation idol
2. UI addiction
3. Over misusing on commercial tools
4. Poor collaboration
5. Lazy maintenance
6. Intensive ROI
Source: http://www.lifehack.org/articles/productivity/how-to-manage-common-productivity-traps-for-improved-productivity.html
3
Trap #1: Adore Automation
Source: http://www.theguardian.com/media/shortcuts/2014/mar/16/could-robots-be-journalist-of-future
Trap #1: Adore Automation
Symptoms • Depending on automation for all testing
activities
• Absence of code reviews
• Absence of exploratory testing
• Absence of UX testing
• All tests are built by developers
4
Source: http://www.calgolftech.com/1-problem-with-buying-golf-clubs/
Trap #1: Adore Automation
5
Treatments Prevent comparison between
manual and automation
Use automation in conjunction with manual techniques
Use automation to provide a baseline – a sanity check
Source: http://cureforhives.org/chronic-idiopathic-urticaria-treatments
6
Trap #2: Love UI so much
A Pyramid
7
GUI
Tests
Acceptance Tests
Integration Tests
Unit Tests
Trap #2: Love UI so much
Symptoms • Limited technical skills
• Cannot understand the application or system architecture
• Huge proportion of tests are being run via the UI
• No collaboration with developers
• Difficult to setup test context
• Test results are slow & fragile 8
Source: http://www.bostonsearchgroup.com/blog/ceo-search-break-through-medical-devices-for-wound-closure/
Trap #2: Love UI so much
Treatments • Limit the investment in UI
automation
• Invest more at lowest possible level with clear test purposes
• Collaborate with developers
• Make sure automation gives a fast feedback
9
10
Trap #3: Over misusing on commercial testing tools
Source: http://www.fastcoexist.com/1679718/your-bad-eating-habits-are-contagious
Trap #3: Over misusing on commercial testing tools
Symptoms
11
A commercial tool forms the basis a testing strategy
Only certain teams or individuals can access the tool or run tests
Developers and testers have not been consulted in tool selections
Source: http://www.sfgate.com/news/article/Apple-exploring-cars-medical-devices-to-reignite-5239850.php
Trap #3: Over misusing on commercial testing tools
Treatments • Use open source tools wherever
possible
• Use tools that can be extended or integrated into existing development tool chain
• Use tools that match specified test intent
• Use clear matrix tool evaluation
• Keep educating!!!
12
Source: http://stugalactic.co.uk/student-shopping-medicine.aspx
13
Trap #4: Too pride to collaborate when creating
tests
Source: http://www.jonathankettleborough.com/tag/collaboration-2/
Trap #4: Poor collaboration
Symptoms • Automated testers are isolated
from team: duplicated test scripts, too much tests but inefficiency
• Poor automation design
• Maintainability or compatibility issues
• No definition of quality: unclear, not match the desired system quality
14
Source: http://thaineuro.org/category/pharma-and-healthcare/healthcare/
Trap #4: Poor collaboration
Treatments • Collaborate to create good tests
and prevent duplication
• Ensure good technical practices and design
• Collaborate to improve quality definition and verification
• Limit investment in UI based automated tests
15
Source: http://medicine1.info/medicine/
16
Trap #5: Too lazy maintenance
Source: http://www.tastyhuman.com/the-illusion-of-8-hour-workday-are-you-hard-working-or-just-lazy/
Trap #5: Too lazy maintenance
Symptoms • Test suite has not been run
recently – state is unknown
• Test suite requires manual intervention
• Duplication within automation code
• Small changes trigger a cascade of failures
17
Source: http://orangedocfamilymedicine.com/
Trap #5: Too lazy maintenance
Treatments • Automated tests should be
executed using a CI environment
• Ensure tests are always executable – even if the system is not being actively developed
• Ensure collaboration between developers and testers
18
Source: http://www.psmag.com/blogs/the-101/the-spectrum-of-color-response-take-your-medicine-51195/
19
Trap #6: Cut costs through automation
Source: http://insuranceknock.com/questions/155/saving-money-with-2-phrases
Trap #6: Cut costs through automation
Symptoms • Using a predicted ROI to
reduce testing budget
• Hidden investments are not counted
20
Source: http://bloomintegrativehealth.com/news-updates/
Trap #6: Cut costs through automation
Treatments • Clear objectives and reasons
for automation
• Cost-benefit analysis should include ongoing maintenance
• Quality is the key
21
Source: https://today.duke.edu/2013/03/operationmedicinedrop
Summary
22
23
Q&A
References
• [1]. Testing Experience – published No. #21, #23 - http://www.testingexperience.com/
• [2]. 7 Deadly Sins of Agile Software Test Automation – Adrian Smith http://www.slideshare.net/adrianlsmith/7-deadly-sins-of-software-test-automation
• [3]. Manual Test cannot be automated http://www.satisfice.com/blog/archives/58
• [4]. Testing Club Vietnam 2014 Trends in SW Testing – Paul Holland
24
Profile
• Thao Vo – QA Architect at KMS Technology Inc. (http://www.kms-technology.com/); Board member of HCMC Software Testing Club (http://www.hcmc-stc.org/).
• Email: [email protected] or [email protected] or [email protected]
• LinkedIn: http://vn.linkedin.com/pub/thao-van-vo/24/31a/b35
25
© 2014 HCMC Software Testing Club
THANK YOU