Test Automation for Packaged Systems: Yes, You Can!

W15 Test Automation

5/1/2013 3:00:00 PM

Test Automation for Packaged

Systems: Yes, You Can!

Presented by:

Chris Bushell

ThoughtWorks

Brought to you by:

340 Corporate Way, Suite 300, Orange Park, FL 32073

888-268-8770 ∙ 904-278-0524 ∙ [email protected] ∙ www.sqe.com

Chris Bushell

A lead consultant at ThoughtWorks, Chris Bushell has a wide variety of roles including software developer, architect, test automation specialist, agile coach, and trainer. Chris has more than ten years of industry experience working in finance, retail, telecoms, and mobile in the U.K., Australia, and the U.S. Passionate about software development process and technology, he is an advocate of using open source software for building simple solutions to complex problems. Chris is an active member of the local software development community wherever he's based. Contact Chris on Twitter @chrisbushelloz, by email at [email protected], or converse with him face-to-face.

Test Automa+on For Packaged Systems:Yes You Can!Chris BushellLead Consultant, ThoughtWorks

What I do

• Developer/Tech Lead/Architect/Agile coach

• Custom so8ware development

• Focus on building the right thing

• Focus on high quality

• Test driven

How I do it – TDD/BDD

How I do it – Revision Control

How I do it – Con+nuous Integra+on

How I do it – Open Source

JUnit• Light-‐weight, powerful

• Extensible

• Excellent community

• Examples: xUnit, Selenium/WebDriver, RSpec, Concordion, Cucumber etc.

November 2011

In the beginning....

The COTS Con!

Highly extensible software “configured” in powerful

development environments, comparable in complexity, and

susceptibility to defects as anything in the custom software

development world

The land that +me forgot

Complete lack of test automa+on

“Computers are designed to do simple repeKKve tasks. The second you have humans doing repeKKve tasks, all the computers get together late at night and laugh at you”

“Every Kme you do something for the third Kme, automate it. You’ll be doing it a million Kmes”

-‐ Neal Ford

Surely someone must be doing this already?

• Commercial tools exist, but tend to be a poor fit for Agile environments-‐ TesKng viewed as a luxury (an add-‐on)

-‐ Available to few ($$$$ licensing)-‐ Discourages collecKve ownership

-‐ Difficult to use good engineering pracKces

-‐ Difficult to integrate with ConKnuous IntegraKon process (lack of scriptability/headless mode)

Lessons learned -‐ what worked and what didn’t

Oracle Siebel CRM

But first, browser automa+on basics

• HTML + DOM

• Selectors-‐ Id, class, xpath

• AutomaKon API-‐ e.g. Selenium, WebDriver, Sahi, WaKr etc.

Our biggest problem

• AcKveX

• <object>

• Impervious to tradiKonal automaKon toolset

All is not lost

• Not all parts of the applicaKon used AcKveX

• Some controls were regular HTML (albeit machine generated)-‐ WebDriver

• Perhaps we could use some kind of hybrid approach to test automaKon?-‐ WebDriver + X, Y or Z?

Introducing Sikuli -‐ automa+on using images

An example using Google

package com.tw.xconf;

import org.sikuli.script.FindFailed;import org.sikuli.script.Screen;import org.sikuli.script.Settings;

public class Demo {

public static void main(String[] args) { Screen screen = new Screen(); try { screen.click("images/criteria.png"); screen.type("ThoughtWorks"); screen.click("images/search.png"); } catch (FindFailed e) { e.printStackTrace(); } }}

Drawbacks of Sikuli

• Difficult to manage large numbers of images

• Can be confused when different regions of the screen look the same

• Tests easily broken by small changes in the UI

• ComparaKvely slow to execute

The story so far....

• Ability to automate the UI

-‐ WebDriver for the regular HTML

-‐ Sikuli for the AcKveX controls

The last hurdle

• How do we extract values from the page so we can perform asserKons on them?

Tried but failed

• OpKcal Character RecogniKon (OCR)-‐ Tesseract

• White

• AutoIt

• Win32-‐Autogui

Plan B

• Oracle ApplicaKon TesKng Suite (OATS)

-‐ Record + playback

-‐ Proprietary test automaKon API

Test automa+on is soYware development

• Code duplicaKon is bad, resulKng in fragile test suites

-‐ Small changes can break many tests which requires significant maintenance

-‐ Frequently see test automaKon abandoned as a result of this

• Record + playback tools make it difficult to introduce abstracKons to remove duplicaKon

Oracle Applica+on Tes+ng Suite

• A process of learning through experimentaKon

• Generates Java code under the covers (for both Seibel and HTML automaKon)-‐ Liberate the API from tooling

• Creates executable JWG files as test arKfacts-‐ Understand the format of these files so we can create them programmaKcally

Gains

• Ability to more easily use good programming pracKces (abstracKons, code refactoring etc)

• Ability to integrate test arKfacts with ConKnuous IntegraKon process

• A maintainable test suite for an Agile team

Remedy +cket management system

Great news!

• A more tradiKonal webapp (albeit with many quirks)

• AutomaKon possible with familiar toolset (WebDriver, in our case)

Example quirk #1: windows

Solu+on: model the windows a stack

• WindowManagingWebDriver

• Allowed us to keep tests focused

• Also responsible for maximizing browser windows (quirk 1.1)

Example quirk #2: mouse ac+ons alone not enough

public void navigateToRelationshipsTab() { findElement(RELATIONSHIPS).sendKeys(Keys.ENTER); findElement(RELATIONSHIPS).click(); }

The importance of abstrac+ons

• Page model

• Domain objects e.g. Incident

• Builders for domain objects

• Page level gekers/sekers e.g. IncidentSeker, IncidentGeker

• Workflows to model sequences of interacKons

Data management

Solu+on: APIs (again)

• AcKon Request System (ARS) API allows creaKon of state e.g. Incidents programmaKcally

• Simple key/value interface

• Gave us the ability to replace the use of some of our Workflows with faster running API based Workflows

• Objects created by API based workflows could be looked up in the browser

The story so far....

• Ability to automate the UI using WebDriver

• AbstracKons in place for robust tesKng

• Ability to setup state using fast API

• Growing regression test suite

Reverse Con+nuous Integra+on

Summary

• COTS test automaKon is possible!-‐ Be prepared to find challenges/quirks along the way

• ConKnuous IntegraKon is possible

• APIs are your friends -‐ seek them out

• AbstracKons are crucial, especially with machine generated code and environmental differences

Ques+ons

Email: [email protected]`er: @chrisbushelloz

Github: www.github.com/cbushellMy talks: www.speakerdeck.com/cbushellLinkedIn: www.linkedin.com/in/chrisbushell

315 Montgomery Street, 16th Floor San Francisco, California 94104T: +1 415 273 1389 C: +1 415 568 8186 F: +1 415 986 2964E: [email protected]




















Chris BushellLead Consultant




















Credits

• Ross Macbeth, J3474 : Old night safe, Belfast, licensed under CreaKve Commons, hkp://www.geograph.ie/photo/2261863

• Alan Chia, A pile of Lego blocks, of assorted colours and sizes, licensed under CreaKve Commons, hkp://en.wikipedia.org/wiki/File:Lego_Color_Bricks.jpg

Test Automation for Packaged Systems: Yes, You Can!

Technology

Transcript of Test Automation for Packaged Systems: Yes, You Can!