NoVaTAIG April 7, 2010
description
Transcript of NoVaTAIG April 7, 2010
NoVaTAIG
April 7, 2010Automated In-Browser Testing with
Cucumber and Watir
Rudy Regner Mike Perz Erik Scheirer Rasik Pandey
Rudy Regner• Introductory Notes
– We’re all on linkedin.com.– Send us questions regarding any details presented
tonight• Presentation Goals and Objectives
– Mike Perz: Test Automation Framework– Erik Scheirer: End-to-End SDLC Architect– Rus Pandey: BDD Tools Expert
Our Situation
• Writing regression tests for an app in production– Only required to support IE
• Rails v1.2.3– Precludes using Webrat latest version– ?? Could possibly have used earlier version ??
• why regression tests for earmarks?– Bug made it through manual testing – Encapsulate critical functionality in automated
regression test scripts to decrease likelihood of this failure occurring again.
Mike Perz• How accomplished the automation framework• Challenges encountered
Our Learning Curve• Started with Watir
– Easy to get up and running• Added Cucumber (RSpec) • Replaced Watir with Webrat and Selenium
– Sought flexibility gain through re-use of Webrat to drive Selenium, Mechanize or straight Rails
• Productivity dropped with Selenium– Watir can manipulate the DOM as objects. (you can iterate
through the cells of an html table)– With Selenium you would have to find each cell individually.– Script debugging difficult without irb
• You can attach to a browser window* (with Watir)• Converted existing Selenium code to Watir
Cucumber Feature snippet
associated Step Definition
Scenario Outline example (similar to fitnesse tables)
Cucumber Test Summary Output (sample)
Once we switched back to Watir
• At first our tests were dependent on each other– They needed to be run in a particular order. – We used Rake to accomplish this.
• Open 4 browsers with 4 different users (Watir::IE.new_process vs. Watir::IE.new)
– Now we can attach Watir to whichever browser session we need
• Watir::IE.attach
– We use SQLite to keep track of which user is associated with which browser session
Interesting side benefits to writing automated tests
• Writing automated tests helped/made us understand the app better.
• Found application bugs that were missed earlier
Other useful tools
• Watircuke• IE Developer Toolbar• Firebug• FireWatir recorder:
www.itest2.com/downloads
Erik Scheirer• Spreading the Developer Joy• Benefits of a single executable Cucumber
artifact end-to-end
Status - Your Organization
• What Rules? Confusion? Sanity?• Even if its sane, it probably can be more so.• Some groups have managed a high degree
of sanity.• But what most organizations are
experiencing is ...
Chaos
• little Trust between groups + infighting• bad communication• no ‘Process’ (or its limping, bleeding, out
the door)• no time to implement anything new,
constantly fighting fires instead
This approach
• provides an end-to-end process via architecture, convention, and tools
• is easy to implement• is easy to customize to exacting technical
needs and group culture
This approach
• builds better quality• builds more trust between technical and
non-technical stakeholders• utilizes resources more effectively
Ingredients
• CTBE - catch that bug early!• Unit tests, feature and black-box testing,
via one single artifact• in this case an Agile approach (scrum)• artifact re-use creates self-reinforcing
system
One Artifact, multiple uses
• developer can create cucumbers as part of T/BDD, and also to capture ‘drive-by requirements’
• QA can create ‘cukes’ for regression tests• Cukes can be fed from dev. to QA or vice
versa• Cukes can be used by Continuous
Integration, and in Production monitoring
Next, the Process from multiple perspectives:
Stakeholder interaction,design and architecture,
includes non-technical
Iterations
testing & daily work The overall lifecycle
The daily grind
Time
Building Trust - spreading the joy
Conclusion•There is a way to achieve more success, better utilization of
existing resources, and get more sleep; using a simple, adaptable architecture coupled with process, convention,
and tools.
• Did we mention ‘metrics’ and how these can be used to CTBE in this manner, too?
• Its also all open-source, so its customizable to a great degree.
• Thank you for listening.
Rus Pandey• How BDD would work• Other possible tools
Ideal BDD Process:
• 1. Describe behaviour in plain text• 2. Write a step definition in cucumber• 3. Run and watch it fail• 4. Write code to make the step pass• 5. Run again and see the step pass• 6. Repeat 2-5 for all your functionality
Performance:
• Since it takes 3 hours to run a test suite we should investigate using spork and --drb and testjour to distribute the test load if possible to speed up test runs.
Test Web apps in any language: Drive a full or headless browser using one of these
• *Webrat – Ruby acceptance testing for web applications • *Capybara – Acceptance testing framework with a webrat-like API and support for multiple
backends, including RackTest, Selenium, Celerity and Culerity • *Steam – Drives a fast headless browser with Javascript support. Support for the normal
webrat step definitions, see Setting up Steam • *Testing PHP app using Webrat • *WebDriver – Drives IE, Firefox, Chrome • *Watir – Drives IE (Windows only), see: Setting up FireWatir • *FireWatir – Drives Firefox on Windows, OSX and GNU/Linux • *SafariWatir – Drives Safari (OS X only) • *ChromeWatir – Drives Google Chrome • *Celerity – Drives a fast headless browser with Javascript support. Examples here • *Culerity – For when you can’t run your app under jRuby • *Selenium – Runs any browser (any OS), see: Setting up Selenium • *Mechanize – Runs a headless browser (any OS)
Examples of Cucumber being used in the real world across
various languages/platforms to test all sorts of things.• http://wiki.github.com/aslakhellesoy/cucumb
er/tutorials-and-related-blog-posts