How To Use Selenium, Successfully

by Dave Haeffner, @TourDeDave

http://www.wpclipart.com/geography/features/chasm.png.html

http://en.wikipedia.org/wiki/Optimal_solutions_for_Rubik's_Cube

Write business valuable tests that are reusable, maintainable and resilient across all relevant browsers.

Then package and scale them for you & your team.

Selenium Overview

• What it is — the Reader’s Digest version

• What it is and is not good at

• IDE vs. Local vs. Remote

• Slow, brittle, and hard to maintain?

Step 1 Define a Test Strategy

Test Strategy1. How does your business make money?

2. What features of your application are being used?

3. What browsers are your users using?

4. What things have broken in the app before?

Outcome: What to test and which browsers to care about

Step 2 Pick a Programming

Language

Programming Language

• Same language as the app?

• Who will own it?

• Build a framework or use an existing one?

• http://bit.ly/seleniumframeworks

Step 3 Use Selenium fundamentals

Selenium Fundamentals

• Mimics human action

• Uses a few common actions

• Works with “locators”

Locators tell Selenium which HTML element to interact with

Common Actions

• get();

• findElement();

• click(); //or submit();

• sendKeys();

• isDisplayed();

Locator Strategies• Class

• CSS selectors

• ID

• Link Text

• Partial Link Text

• Tag Name

• XPath

Good locators are: • unique • descriptive • unlikely to change

That rules a few of these out

Locator Strategies• Class

• CSS selectors

• ID

• Link Text

• Partial Link Text

• Tag Name

• XPath

Good locators are: • unique • descriptive • unlikely to change

That rules a few of these out

Locator Strategies• Class

• CSS selectors

• ID

• Link Text

• Partial Link Text

• Tag Name

• XPath

Good locators are: • unique • descriptive • unlikely to change

That rules a few of these out

Start with IDs and Classes

Locator Strategies• Class

• CSS selectors

• ID

• Link Text

• Partial Link Text

• Tag Name

• XPath

Good locators are: • unique • descriptive • unlikely to change

That rules a few of these out

Start with IDs and Classes

Use CSS or XPath (with care)

Locator Strategies• Class

• CSS selectors

• ID

• Link Text

• Partial Link Text

• Tag Name

• XPath

CSS vs XPath http://bit.ly/seleniumbenchmarks http://bit.ly/cssxpathexamples

Finding Quality Locators• Inspect the page

• Verify your selection

• e.g., FirePath or FireFinder

• http://bit.ly/verifyinglocators

• Learn through gaming

• http://bit.ly/locatorgame

• Conversation

Step 4 Write your first test

Good Test Anatomy

• Write for BDD or xUnit test framework

• Test one thing (atomic)

• Each test can be run independently (autonomous)

• Anyone can understand what it is doing

• Group similar tests together

A Login Example

1. Visit the login form

2. Find the login form’s username field and input text

3. Find the login form’s password field and input text

4. Find the submit button and click it

1. or, find the form and submit it

http://the-internet.herokuapp.com/login

Now to find an assertion

1. Login

2. Inspect the page

3. Find a locator

4. Verify it

5. Add it to the test

Exception Handling• org.openqa.selenium.NoSuchElementException:

Unable to locate element: {"method":"css selector","selector":".flash.error"}

• Most common ones you’ll run into: NoSuchElement and StaleElementReferenceError

• A list of all WebDriver exceptions: http://bit.ly/se-exceptions-java

Exception Handling cont’d

http://bit.ly/se-exceptions-howto

Step 5 Write reusable and

maintainable test code

Page Objects

Application Under Test

Test 1 Test 2 Test 3 Test 4 Test 5Test 1 Test 2 Test 3 Test 4 Test 5

Need to update EVERY test :-(

Application Under Test

Page Object(s)

Test 1 Test 2 Test 3 Test 4 Test 5

Need to update JUST the page object :-D

Let’s look at a page object for login

And here’s what the test looks like when using it

Page object helpers: http://bit.ly/po-html-elements http://bit.ly/po-page-factory

Base Page Object

Selenium Commands

Page Object 1

Page Object 2

Page Object 3

Page Object 4

Page Object 5

Base Page Object

Page Object 1

Page Object 2

Page Object 3

Page Object 4

Page Object 5

Selenium Commands

• Global reuse • More readable • Insulates you from

Selenium API changes http://bit.ly/se-upgrade

Let’s take a look at a Base Page Object

And here it is implemented

How everything fits together

Test TestTest

Page Object

Page Object

Base Page

Object

Tests use page objects

Page objects inherits the base page object

The base page object wraps your Selenium commands

Step 6 Make your tests resilient

Waiting

Thread.sleep(); Implicit wait Explicit waits

Thread.sleep(); Implicit wait Explicit waits

Thread.sleep(); Implicit wait Explicit waits

http://bit.ly/se-waiting

Explicit Waits

• Specify an amount of time, and an action

• Selenium will try repeatedly until either:

• The action is completed, or

• The amount of time specified has been reached (and throw a timeout exception)

In the Base page object

In the DynamicLoading page object

Browser Timing Considerations

Step 7 Prep for use

Test Harness• Simple organizational structure

• Central setup and teardown

• Configurable at run-time (with sensible defaults)

• Reporting & Logging

• Parallelization

• Test Grouping

Folder structure

Central setup/teardownMore on JUnit Rules: http://bit.ly/junit-rules

Simple config with defaults

Reporting & Logging

• Machine readablee.g., JUnit XML

• Human readable e.g., screenshots, failure message, stack trace

Fantastic Test Report Tool http://bit.ly/se-reporter (Allure Framework)

Parallelization• In code

• Through your test runner

• Through your Continuous Integration (CI) server

#protip Enforce random order execution of tests http://bit.ly/junit-random-order

Recommended approach: http://bit.ly/mvn-surefire

Test Grouping• Metadata (a.k.a. Categories)

• Enables “test packs”

• Some category ideas

• wip

• shallow

• deep

• story number

More info: bit.ly/junit-categories

Step 8 Add in cross-browser

execution

Locallyhttp://bit.ly/se-chromedriver http://bit.ly/se-firefoxdriver http://bit.ly/se-iedriver http://bit.ly/se-operadriver (12.16) http://bit.ly/se-safaridriver

Chrome

Grid

Grid Hub

Browser

Tests

All done with the Selenium Standalone Server Just requires additional runtime flags

Grid Node

Grid Node

Grid Node

Browser

Browser

GridHub

Node(s)

Grid

More on Selenium Grid http://bit.ly/se-grid-docs http://bit.ly/se-grid-post http://bit.ly/se-grid-extras http://bit.ly/se-grid-scaler

Sauce Labs

Sauce Labs BrowserTests

Sauce Labs

Additional Considerations - Test name - Pass/Fail status - Secure tunnel

More on Sauce: http://bit.ly/sauce-platforms http://bit.ly/sauce-post http://bit.ly/sauce-tutorial-java

Step 9 Build an automated

feedback loop

Feedback loops• The goal: Find failures early and often

• Done with continuous integration and notifications

• Notifications e.g., remote: Email, chat, SMSin-person: audio/visual, public shaming

Code Committed

Unit/Integ. (pass?)

Deploy to autom. test

server (success?)

Run automated

tests (pass?)

Deploy to next env.

yes

yes

yes

Notify team if no

Code Promotion

Bonus points: stop the line

Simple CI configuration1. Create a Job

2. Pull In Your Test Code

3. Set up Build Triggers

4. Configure Build steps

5. Configure Test Reports

6. Set up Notifications

7. Run Tests & View The Results

8. High-five your neighbor

Step 10 Find information on

your own

http://bit.ly/se-info-slides

http://bit.ly/se-info-video

Steps to solve the puzzle1. Define a Test Strategy

2. Pick a programming language

3. Use Selenium Fundamentals

4. Write Your First Test

5. Write re-usable and maintainable test code

6. Make your tests resilient

7. Package your tests into a framework

8. Add in cross-browser execution

9. Build an automated feedback loop

10. Find information on your own

Write business valuable tests that are reusable, maintainable and resilient across all relevant browsers.

Then package them and scale them for you & your team.

–Dave Haeffner

“You may think your puzzle is unique. But really, everyone is

trying to solve the same puzzle. Yours is just configured

differently — and it’s solvable”

https://seleniumguidebook.com/

Selenium Tips & Tricks

by Dave Haeffner @TourDeDave

http://ElementalSelenium.com

etc.

Headless w/ XvfbTip 38

Headless w/ GhostDriverTip 38

Step 2 - Start PhantomJS w/ WebDriver flag

Step 1 - Download PhantomJShttp://phantomjs.org/download.html

Step 3 - Connect Your Test to PhantomJS using Selenium Remote

NOTE: You can also connect PhantomJS to a Selenium Grid

HTTP Status CodesTip 17

Configuration

• Use a proxy server to capture the traffic from your Selenium test(s)

• Find the status code for the action you’re interested in (e.g., visiting a URL)

• Assert that the status code is what you expect

Selenium Browser Proxy Server

Application Under Test

BlacklistingTip 66

Configuration

• Use a proxy server to manipulate the traffic from your Selenium test(s)

• Identify third-party resources that are slow to load (which could negatively impact your tests) and blacklist them (e.g., make it so they don’t load)

Load TestingTip 68

Configuration

• Use a proxy server to capture the traffic from your Selenium test(s)

• Convert the HTTP Archive into a JMeter JMX file

• Run the new JMX file with JMeter to enact load on your application (modify as needed)

Code adapted from ruby-jmeter example from flood.io

Broken Image CheckingTip 67

Option 1 - Proxy Server

Option 2 - HTTP Library

Option 3 - JavaScript

Option 1: Proxy Server

Option 2: HTTP Library

Option 3: JavaScript

Forgot PasswordTip 43

Configuration• Use Selenium to trigger the forgot password

workflow (which sends an e-mail to a Gmail account) and keep the session active

• Retrieve the e-mail and it’s contents (e.g., a URL) through the Gmail API

• Launch the URL from the e-mail using the Selenium session

A/B TestingTip 12

Example Explanation

• There are 3 states available in the example on the-internet. Each state has different header text.

• The control has the text ‘A/B Test Control’

• The variation has the text ‘A/B Test Variation 1’

• Outside of the split test, the text says ‘No A/B Test’

Configuration

• You can easily opt-out of A/B tests by forging a cookie or appending a query to the URL

• This way you get a known state of the page which isn’t likely to change without your knowledge

Downloading a FileTips 2, 8, & 15

Two approaches• Configure Selenium to download to local disk, and

delete the file when done

• Use an HTTP library, perform a HEAD request, and check the headers for the correct content type & length.

A HEAD request with an HTTP library is an order of magnitude faster than downloading with Selenium since you’re just checking the headers. And, it can be used in

tandem with Selenium.

With Selenium

With an HTTP library

With an HTTP library (for secure files)

Highlight ElementsTip 65

Growl NotificationsTip 55

See the video at http://bit.ly/se-growl-video

Visual Testing

Some Write-ups

http://bit.ly/se-visual-1 Getting Started

http://bit.ly/se-visual-2 False Positives part 1

http://bit.ly/se-visual-3 False Positives part 2

http://bit.ly/se-visual-4 Add Visual Testing To Your Existing Tests

http://bit.ly/se-visual-5 Add Visual Testing To Your BDD Tests

A Robot On Every Desk ™

Tapsterbot

http://bit.ly/tapster-build

http://bitly.com/tapster-buy

Build Your Own

Buy One Fully Assembled

Get in touch

@TourDeDave

dhaeffner@gmail.com

DaveHaeffner.com