Automated Website Testing

matt.hooper@vc3.com

About Me

• Product Architect at VC3– VC3 Academic Suite– Etc…

• Find me at:– matt.hooper@vc3.com– mhoop.wordpress.com

WatiN (“What-in”)

• watin.sourceforge.net• .NET assembly (watir is Ruby version)• API for controlling web browser, and DOM.– Ajax support

WatiNWeb

Browser.exe Web site

WatiN Functionality

• Search for elements by ID, etc– Regex’s too

• Interact with DOM to simulate user– button clicking– typing

WatiN Demo

A Web App

• TestView is 6 years old• Periodic team changes• Continuous enhancements• Customer hosted (intranet)• Size:– 15+ Visual Studio projects– 600+ db tables– 600+ .aspx files– 3000+ .cs files (app-specific)– And code shared across all VC3 apps

Testing Needs

• More obvious:– Enhancements, bug fixes

• Less obvious:– What did we break? (regression tests)

Goal: Improve quality and delivery speed

How WatiN fits in

WatiN

Browser

exerciseand verify

Test Runner

run andmanage

setup preconditions, verify results

Page1Page1

Tests

Web Site

How WatiN fits in

WatiN

Browser

exerciseand verify

Database

Visual Studio

run andmanage

setup preconditions, verify results

Page1Page1

Tests

Web Site

Why full stack testing?

Testing against the UI isnatural and

end-user oriented

Testing Approaches

UsersFull stack

tests

Real World

UI

Model

Data

Unittestsvs

Developer Tendencies

1. Code2. Test– find data to test in db– use the UI to perform actions– verify results

3. Fix4. Repeat until satisfied

Developer Tendencies

1. Code2. Test– find data to test in db– use the UI to perform actions– verify results

3. Fix4. Repeat until satisfied

Automate!

UI Automation Challenges

Interface can be:• Complex• Dynamic• Fluid over lifetime

Cleaner Tests

SignInWithGoodCredentials SignInPageUserNamePassword

SignInButtonNoSignInWithBadCredentials

WatiN

• Encapsulate ID’s etc in page model classes• Makes tests more readable (example)

Design goals for tests

• Easy to write (and read)• Isolated from UI brittleness• Start from a known state– no side effects ideally

• Test a specific case/scenario• Consider overlap among tests

Our Results

• Added a layer on WatiN to simplify tests

Our Results

• Incrementally adding tests• No side effects is key• Useful for long efforts• Slow to run large number of tests• Tests have helped with regression tests• Response from developers– Mostly positive– Writing good tests is hard!

Your Website?

• More complex monitoring

• Regression tests– CMS upgrades– Server migrations

Other Links

• Email testing– nDumbster: http://ndumbster.sourceforge.net