Boise Code Camp 2010John Sonmez

http://simpleprogrammer.com

Automated regression testing is hard Fragile tests Difficult to write tests Tools are expensive Recording is not flexible enough Tests failures don’t indicate code problem

#1 Problem with automated tests is the tests break easily.

Screen changes cause multiple tests to break.

Functionality or text changes cause tests to break.

Why? Multiple tests are dependant on parts of the user interface.

Recording is easy at first, until you need variables or reuse.

Scripting the tests requires knowledge of HTML, CSS, Javascript and a high level programming language such as C# or Java.

QA or business person thinks in terms of domain knowledge, not button clicks and HTML divs.

Most of the record tools have license costs from several hundred to several thousand dollars.

Cost prohibitive, so only certain people in organization get license.

Support contracts. Upgrade costs. “Automation Experts” trained in the

vendor specific tool are expensive.

Recording seems to work at first, but then the scenarios get more complicated.

When the UI changes scripts have to be re-recorded.

Recording limits the kinds of testing you can do. Test result interactions Check against external data source

False failures reduce the confidence in the tests.

Start treating test code like real code. Instead of building “test scripts”, build a

“testing application.” Apply the same tools of good software

craftmanship to test code craftmanship. Bridge the gap between domain

knowledge and test writing.

User: Add a person to the case. Tester:

1. Click button with id=“loginPage123”. 2. Set focus userName field. Add username to

field. 3. Set focus passWord field. Add password to field. 3. Click button with id=“login”. 4. Check HTML title for text containing

“successful”. 5. Click div in top navigation with id of addPerson. 6. blah blah blah… blah blah blah

User: Add a person to the case. Tester:

1. Case.AddPerson(“Joe Smith”); 2. AssertTrue(Case.isPersonOnCase(“Joe

Smith”));

Abstraction… Business language becomes testing language.

Internal Domain Specific Language (DSL) A language to write tests which uses the

same or similar vocabulary as the business or user.

Framework built specific to the application, not a general purpose testing framework.

Framework developed by developers and testers collaboratively. User of framework is test automation resources.

Domain Specific Language According to Wikipedia: A programming

language or specification language dedicated to a particular problem domain, a particular problem representation technique, and/or a particular solution technique.

Basically a custom programming language designed for a specific problem or set of problems.

A DSL that is a set of APIs still in the native language used to approximate a full DSL.

Writing a language is hard. Writing an API is easier. Objected oriented programming techniques

already lead us in this direction. We use internal DSLs in English all the time.

Company acronyms and language. Programmer speak Slang

Remember the gap? Bridge the gap by building an application

specific layer that exposes a DSL for writing tests.

Hide the details about web pages, HTML, CSS, Javascript from the tests.

Easy to use, fluent design.

The web browser provides us a layer we can use to abstract away the notion of parsing HTML, CSS and Javascript.

Without the browser, we would have to write code to open HTTP connections and GET web pages.

We would have to parse CSS, and Javascript, to verify functionality.

Web testing framework interacts with the browser for us.

Operates on the DOM the browser provides.

Without the Web Testing Framework, we would have to write code to manipulate the browser, and change it’s state through COM or some other interface.

We build this on top of the Web Testing Framework.

Abstracts away all concepts having to do with browsers and HTML.

Uses the Web Testing Framework to translate business language into web functionality.

Contains all our changing things about the application.

Without this layer our code would contain hard coded ids of elements and would have to deal with page layouts.

Uses only the application framework. Uses the internal DSL. Simple Written as close to business language as

possible.

A good starting place is to start writing the test.

Pretend like you don’t know anything about the web, but you do know about the application.

Think “fluent.” Think easy to use.

Web based store. Write a test to create a new account and

buy bananas. Account.Create(“Joe Smith”, “123 Main Street,

Boise ID 83713”, “joe@joesmith.com”); Category.Fruits.GoTo(); Cart.AddItem(Items.BANANAS, 1); Cart.CheckOut(PaymentType.CreditCard);

Simple, business logic based syntax.

Similar to Test Driven Development when unit testing.

Build the business objects and methods needed to make the test syntax possible.

Don’t worry about all the variations, only add what is needed for the test you are working on.

Here the identifiers are hard coded in the business objects, NOT in the tests.

Account.Create Make an account class with a static method

called create, that takes in a name, address, and email.

In the method use the web testing framework to navigate to the account creation page and enter the information into the fields.

After the information is entered create the new account.

Default non-important selections. (You can make them specific later if needed.)

Actual test layer. Think about how to achieve the same

amount with fewer lines of code. The simpler and fewer lines of code to

write the tests, the easier to write the tests.

Eliminate variables that are not important to the test.

Eliminate any steps that cannot vary.

Do we are about the account name, address, etc? Account.Create(“Joe Smith”, “123 Main Street,

Boise ID 83713”, “joe@joesmith.com”); becomes Account.Create(People.Base_Case);

Do we need to explicitly go to the fruits page to add a banana? Category.Fruits.GoTo(); Cart.AddItem(Items.BANANAS, 1); becomes Cart.AddItem(Items.BANANAS, 1);

Created for Dept of Health and Welfare IBES System to automate web testing.

Creates test cases in very few lines of code.

Able to write a test cases which goes through 30+ screens in a few minutes.

Test frameworks Watij (http://watij.com/) Watin (http://watin.sourceforge.net/) Watir (http://watir.com/)

My Blog, posts on automation and Watij. http://simpleprogrammer.com

Contact me I can help you build your framework John.Sonmez@gmail.com Twitter jsonmez