Web Test Automation Framework with Open Source Tools powered by Google WebDriver

Kapil Bhalla & Nikhil Bhandari

Intuit Inc. (Bangalore)

Understand Present Scenario

Look at Pain Points

Framework

Q&A Discuss

What we will be doing?

Agenda Understand Present Scenario

ü Present Scenario

ü Product

ü Development

ü Customer

ü Agreement

ü What can we Do ?

Present Scenario - Product

Decade Old Product

Interleaved with multiple technologies

Focus on new development technologies and framework

Increasing team size with newer people

Flex

Java Script

HTML

DOJO

AJAX

SOAP

Web Services

REST

Present Scenario – Development

Agile Development

Shorter Release Cycles

Long Testing Cycles…growing continuously with every release.

No Automation Yet !!

Need for Speed

Present Scenario – Customer Expectation

Should work on my current browser !

Should work on Operating System I use!

No Compromise on Quality !

Reliability is a must for us!

Even higher expectation from Customer Support

Google Chrome

Internet Explorer 7

Internet Explorer 8

Internet Explorer 6

FireFox 3

FireFox 3.7

Windows XP SP2Windows XP SP3

Windows Vista

Windows 7

10 / 10

We are in agreement !

What Can we Do ?

Automate

We know That!

Then What ??

Agenda Pain Points

ü Pain Points in Automation

ü Agreement

Pain PointsAbsence of Single “Silver Bullet” tool.

Poor Ease of automating test cases. (The coding way)

Learning Curve for new automation libraries, frameworks.

Diversified domain / functional knowledge among team members.

Enabling multiple contributors to automation projects.

Expected Low maintenance in long run .

Always expensive to switch to a different automation tool/framework.

We are in agreement !

Agenda Framework

ü Overviewü Onboardingü Architectureü What did we achieve

Framework an OverviewObject Oriented way to test a Web Application

It is a mechanism to enable

• Structured testing effort

• Multiple team members to contribute in automation

• Leverage work done by others

Separation of concerns

• Test code

• Web Page Objects (Page modeling code)

• Suite Formations

Loose Test-Tool Coupling

On demand Test Launch

Framework On-boarding : Page ObjectA Page Object is Object representation of a Web Page in Java Programming World.

All elements seen on the web page can be interacted with using the respective methods in the page

object. Page object uses information in Web Element Repository to interact with an element.

Framework On-boarding : Web Element Repository

Refers to XML based collection of meta information about elements seen on

the Web Page. Framework uses this information to identify the Web Element

and then interact with it.

<element key="SAVE_BUTTON" findBy="ID" id="save" tag="input" name="“ xPath=""/>

Here this button is identified by using the ID and the id is "save".

Framework On-boarding - Tests

Test Cases : Test cases are assertions/verifications made on compilation of

workflows using one or more real web pages. Interaction with the web pages is

done using Page Objects. Example- Hit the login page, assert that the page

loaded completely and it is login page indeed.

Test Suites : These are collection of 1 or more test cases.

Framework On-boarding - PackagingIt is an attempt to enable multiple contributors to contribute to the automation

project aligned with the team structure. An product area based segregation

helps both in division of work/impact areas and gives liberty to the area owner

to experiment with different ways of automating his/her area.

Packages aligned with common functionality provided by framework, like

utilities, logging features, framework code, etc.. should be clearly demarcated.

Framework On-boarding …

Reporting : The users can write messages to logs using the APIs exposed by the

framework. These messages are displayed in the final reports generated by the TESTNG

report engine.

Browser : Browser is a place where most of the noticeable action happens.

Test Engine :Test Engine takes in test cases and test suites and execute them.

Framework On-boarding - WebDriverWeb Driver : Page Objects indirectly interact with the set of Web Driver libraries

to hook on to the Web Elements and to interact with them.

Some typical scenarios are finding an element, sending text to element, clicking

a button, reading value of an element, and so on..

User can use the libraries directly or through the APIs provided by the

framework (Recommended way is Framework APIs )

Framework Architecture

CONFIDENTIAL

Framework What did we achieve

• Ease of use - Its only Simple Java APIs

• Easy to extend

• Reuse code and domain knowledge

• Track execution flow.

• Flexible and Powerful

• Straightforward integration with other tools/libraries (Html Unit, Selenium)

Framework is …

aggregation of various Open Source Powers enabling us to do:

§ UI testing on Multiple Browsers and Operating Systems

§ Business Logic layer testing using APIs

§ Profiling data (like page load time, first/last request, http timing, status, etc)

§ Integration with continuous integration build system with different Test Suits classes

§ Integration Virtual Test Execution Lab and Test Case Management System.

Discussion

References

Google WebDriver Project homepage

http://code.google.com/p/webdriver/

Google Test Automation Conference (GTAC) 2007 Presentation on WebDriver

http://www.youtube.com/watch?v=tGu1ud7hk5I

Latest Revision of this presentation slides

http://docs.google.com/leaf?id=0B_NzEJt8br0aNzgzYjE5OWItMjJiZC00YjhmLWEyY2YtOWE0YmY4ZjEzNjk5&hl=en

Contact Info

Kapil Bhalla

bhalla.kapil.10@gmail.com

http://in.linkedin.com/pub/kapil-bhalla/7/a47/a21

Nikhil Bhandari

nikhil.bhandari@gmail.com

http://in.linkedin.com/in/nikhilbhandari

Why WebDriver?• Multi-platform support (Mac & Win)

• Multi-browser support (IE, FF, Safari, Chrome)

• Faster adoption of new technologies (Win7, new browsers, etc.)

• Free. The test developers may have fewer hassles obtaining licenses

• Tests are written in Java programming language

• Connections to browsers are optimized for each browser (IE uses COM, FF uses

a plug-in, Safari uses AppleScript). Selenium executes javascript inside the

browser, which causes some testing complications and a risk of a "single

source error“

• API is object oriented and "cleaner" than Selenium

• Manipulates browsers directly (instead of hacking JavaScript code into the site)

• Selenium and WebDriver merging! - Best of both worlds

• Eclipse IDE

Browser Drivers

Decision Criteria – Web DriverDeveloper Productivity  

Object/API discoverability Good: Using tools like FireBug, IE Developer, Or source code directly  (hit objects using Text, partial text, id - best way usually, xpath, etc...)

Training/Consulting available Active Web Driver Community, Internal Users

Ease of use Simple Java Programming concepts, Normal Web Elements awareness - Buttons, Anchor, Tables …

Language knowledge in team Java.  Beginners 1 to 2 days are good to get going

Vendor responsiveness Its GOOGLE Open Source Community - Google Web Driver. Issues get fixed and tracked.  Have seen post replys from simon stewart WD creator.

Cross Browser/Platform testing  IE 7/8 YES Both FF 3.0/3.5 YESSafari 4.0 Mac Should workTool/Language Features  Program Control (loops, decisions, etc.) ALL Available, you can do almost anything

Ability to handle popups Can handle pop ups that are certain, remaining need to be figured out.

Incremental Cost One Time effort to build up Specific Infrastructure. (Tool Independent) Then equivalent to : writing simple Java program.

License availability FREE (Open Source)Other Tools Integration  Selenium, TestNG, Java Robots, Fitnesse YES, Any tool with which Java can be easily integrated

Decision Criteria …

Criteria Selenium WebDriver Watir Sahi

Browser/Platform Support

All All Some (immature Safari support)

All

Development/Community

Great Good Decent Small, not well known

Infrequent releases

Ease of use

Requires server Concept of Page Object, simple API

Ruby learning curve

manual proxy, no frames + multiple domains

Hacky multiple-domain support

Language SupportMany! Java Ruby, Java

comingJavaScript

Modal Dialogs Coming? Coming… Limited Limited