End of the Road For GUI Automation – What Next?

Dec 2015

Shrini Kulkarnishrinik@gmail.com

http://shrinik.blogspot.com

Best kept Secrets about GUI Automation

• Very expensive to develop and maintain

• Prerequisites for success are very demanding and often impossible to meet

• GUI tests are fundamentally flaky

• Growing post run effort of analysis of results with size of automation suite

Key Considerations for a Good Test

• Fast• Reliable / Trust worthy• Consistent• Developed as parallel

as possible to product code

• Integrates with Dev IDE• Easy to troubleshoot

How these secrets are kept ?• Industry lobby

– GUI automation so Seductive and intimidating– Let us reduce cost of testing by automating– You can achieve faster time to market by Automation– Comfort at distressed state

• Skill levels in Testing– Manual tester stereotype– Automation tester is super tester

• Propaganda in terms of Risk, Ease of use, Low cost– Test at level of user interaction– Regulatory needs require screenshots and evidence of testing– Testing at GUI level truly validates all up/downstream systems are wired correctly

• Lack of Rigorous scrutiny of claims and critical thinking– Initiate run at night – get the results in the morning– Automation for Non Technical users

Conflicting motivations

• Encourage “test last” and involve non technical testers

• Create and encourage distance from developer - independent testing

• Difficult to “sell” manual testing service without “tools”

• How can testing be fast and cheap without “automagic” ?

• Non GUI automation is difficult to demo and not a cash cow

Industry Trends …

• GUI Code is almost generated code

• Mostly on web/Mobile• Browser Technology

converging to fewer standards

• Agile• DevOps• eCommerce• Open source tools

• Era of innovators and Disruptors

• Start-ups• SMAC

Is pressure to overlook limitations and challenges of GUI automation so powerful?

Is GUI automation delivering as promised?

Few Clarifications !!!

• GUI Testing != GUI Automation• Ux/User Experience assessment != GUI

Automation• GUI Automation is not the only form of

automation

Enter Agile Pyramid

Let’s compare unit/API/GUI tests

Unit Tests• Focus on testing logic• All Data and dependencies

are Mocked out• Run straight from Dev IDE

and in mem• Do not require any

deployment• Do not touch file-system,

network or any out side resource

• Several thousands can be run in minutes

• Failed test is almost sure a bug or an unintended change

API Tests• Focus on testing of

connected logic – integration between components

• Need application deployment• Do not require GUI to be

available• Susceptible to failures due to

data and dependencies not set up as needed

• Do not suffer from brittleness of GUI tests

• Failed test is STILL sure a bug or an unintended change

GUI Tests

???

What next ?

Move from ….

Unit/Small Tests API Tests GUI Tests

To …

Unit/Small Tests API Tests GUI Tests

And

• Do more deeper and broader Testing• Do not fight with developers• Do not hold bugs dear to you• Use tools as aids to testing not replacement• Focus on skill development as tester• Do not afraid to code or look into code• Destroy Manual tester stereotype

Testing Karma …

Testing is more like works of …

Take PRIDE in your work as testers !!!!

• Automation is not an option – not discretionary• Traditional ROI models of waterfall model do not apply• Every one in the team participates to build automation• Happens all along with the project (sprint/scrum

cycles)• Without automation – you can not run an agile project• A failing test (automated) – breaks the build• Right from provisioning of environments to production

deployment – automation shows up its head

Bonus Slide --- Agile Automation

@shrinik

shrinik@gmail.com

shrinik.blogspot.com