AutomatedBrowser Testing

Darren Hickling September 2014

Overview

3 / 22Automated Browser Testing Darren Hickling

In a Nutshell

• Automate browsers to create repeatable integration and regression tests

• Those tests can be run in parallel on different browsers on different hardware to more efficiently ensure consistent behaviour

4 / 22Automated Browser Testing Darren Hickling

The Tech

• Selenium emerged as the obvious choice• Installed and managed via NuGet for C#• Supports all major browsers, including mobile

• Uses the same API for each• Makes direct calls to the browser where possible

• To provide e.g. file downloading• Easy to learn and very powerful

Examples

6 / 22Automated Browser Testing Darren Hickling

Browser Switching

Download a browser driver from NuGet and load it

using OpenQA.Selenium.Chrome;using OpenQA.Selenium.IE;

// Setup...RemoteWebDriver driver = new InternetExplorerDriver();

// Later...Driver = new ChromeDriver();

7 / 22Automated Browser Testing Darren Hickling

Element Selection

Select and filter elements by their tags and attributes

var tableContentRows = WebDriver .FindElements(By.TagName("table")) .FirstOrDefault(e => e.GetAttribute("media") == "screen") .FindElements(By.TagName("tr")) .Skip(1);

8 / 22Automated Browser Testing Darren Hickling

Actions

Queue and perform a set of actions fluently

new Actions(WebDriver) .ClickAndHold(firstListItem) .MoveToElement(secondListItem) .MoveByOffset(0, 5) .Release() .Perform();

Impressions

10 / 22Automated Browser Testing Darren Hickling

Patterns Rock

• Two useful patterns have already emerged• Use a Page to interact with the elements of a page

• Fill forms• Click buttons

• Build a Navigator to perform a user journey• Take the name a problem domain• Chain many Page interactions

11 / 22Automated Browser Testing Darren Hickling

Problems

• Complex element selection can be convoluted• Finding a parent element, then filtering its

descendants• Using XPath• jQuery selector plugins are also available!• However, you might be masking the fact that this is

unnecessarily difficult?• No other complaints so far

12 / 22Automated Browser Testing Darren Hickling

Act as Documentation

• Watching a full integration test suite in action is surprisingly magical and enlightening!

• Make a change here, watch it impact that thing there, understand the domain more

• Great for new engineers and those unfamiliar with a product/concept

13 / 22Automated Browser Testing Darren Hickling

Theme and Resize Testing

• Selenium can take a screenshot of the browser• Pick specific screens, screenshot them and compare to

the originals• Resize the browser and screenshot at the different

sizes for responsive testing• Store a hotspot overlay of the failed comparison to

quickly identify the problem• No drawing red ovals in Paint!

14 / 22Automated Browser Testing Darren Hickling

Project Workflow Changes

• QA write/evolve test plans from:• High-level project goals• Diagrams/wireframes• Experience!

• SE create automated tests from the plans• Code-reviewed with their BI logic and unit tests

• QA review:• By manually following their test plans

• Checking recordings via supported cloud VM service

15 / 22Automated Browser Testing Darren Hickling

Bug Workflow Changes

Triage an issue

QA test and release

Add or update unit tests

Fix the problem!

Ensure browser test now pass

Replicate with browser tests

Code review all changes

16 / 22Automated Browser Testing Darren Hickling

Cross-Browser Testing

• Browsers limited to those available on the box used for testing

• At a minimum, use installed browsers on the build server

• Will slow down other builds, though• Not parallelised, either• VMs seem a better and recommended option• Kick off the move to Azure now? Or use…

17 / 22Automated Browser Testing Darren Hickling

Sauce Labs

• Provide 359 device/OS/browser platforms via VMs• Used by EventBrite, Yelp, Mozilla and others• Free to evaluate at a very small scale• Automated tests capped per subscription• Unlimited manual tests• C# sample project on Github• Integrates with TeamCity and Bamboo

18 / 22Automated Browser Testing Darren Hickling

Test Dashboard

19 / 22Automated Browser Testing Darren Hickling

Profile the Test Server

• Run a profiler on the box that the tests hit• Pick from one of four contenders, such as ANTS• Provide consistent, historical profile trends

• Have our changes affected performance?• Automated performance issue alerts

• Including advertising huge improvements!• Could mostly prevent (lucky?) devs with a profiler

license from needing to help others

In Conclusion

21 / 22Automated Browser Testing Darren Hickling

Selling Points

• Confidently assert the product functions effectively on many platforms

• Resolved issues are less likely to reappear• Worth the investment for this alone?

• Identify and fix performance problems before they reach clients

• Advertise significant performance gains

22 / 22Automated Browser Testing Darren Hickling

More Selling Points

• Reduce the time taken for engineers to understand a issue/product/concept

• Reduce the likelihood that large changes will adversely affect existing functionality

• Clients do/will expect this level of testing