Agile Test Management with Open Source Testing
-
Upload
appvance-inc -
Category
Technology
-
view
2.015 -
download
3
description
Transcript of Agile Test Management with Open Source Testing
AGILE PERFORMANCE TESTING
Todd Bradfute, [email protected] Cohen, +01 408-871-0122, [email protected]
August 2, 2011
Managing Software Projects With Agile Methodology
Open Source Software (OST) Adoption
Open Source Test Automation
About PushToTest‣Our Mission: We Test, For A More Reliable World
‣Make It Easier To Move From Manual To Automated Testing
‣Support Advanced Testing, including Agile, Load, Integration
‣Open Source Testing (OST) For Everyone
2
Open Source Test Automation
Intellectual Property of PushToTest. (c) 2009 All rights reserved.
Your Team‣Frank Cohen, CEO and Founder, PushToTest
‣Todd Bradfute, Professional Services, [email protected]
Open Source Test Automation
Slides on Twitter‣http://twitter.com/fcohen
‣Please retweet
‣Please follow me
4
Next let’s talk about
Open Source Test Automation
Agile Open Source Testing
‣Test Driven Development
‣Roles and Fungability
‣Testing and Test Management
5
Open Source Test Automation
Agile Is Seldom Achieved‣Test-First development
‣Resource interchangeability (aka “fungibility”)
‣Commercial tools lock test tools to “testers”
‣OST lets everyone on the team be more involved.
Open Source Test Automation
Overview‣Test Driven Development
‣Resource Fungibility
‣OST as the Lingua Franca
‣Certification using OST
‣Demonstrable Results
‣Detailed Results
‣Provable Results
Open Source Test Automation
Test Driven Development‣Write the test before implementing the feature
‣To expand lookupZip(String addr) to return zip+4
• First: write test that expects #####-#### as output
• Second: Run it and watch it fail because output is #####
• Third: Change implementation
• Fourth: Run it and watch it pass
‣You’re finished adding the feature when this (and all other) tests pass
‣OST enables TDD
Open Source Test Automation
Resource Fungibility‣Any person can work any task
‣Most Agile teams don’t even try
‣The prize is extreme velocity and flexibility!
Open Source Test Automation
OST - The Lingua Franca‣Cross testing ... i.e. person A’s code is tested by person B
‣A and B are both members of the agile team
‣Both A and B may be developers, but not testing their own code
‣Using OST the team selects a standard tool set
‣On the current Best Buy project the primary OST is soapUI
Open Source Test Automation
Certification Using OST‣At Best Buy the organization has a “certification” process
‣Starts with the sub-teams own test artifacts
‣Reviews plans and execution output
‣Responsible for repurposing tests as concurrency/load/performance tests
Open Source Test Automation
Demonstrable Tests‣Reports are saved to the database
‣The Agile process is enabled by using Rally
‣Process flow:
‣User Story -> Test Case -> Test Card -> Test Results -> Test Output
Open Source Test Automation
Rally/TestMaker integration
Open Source Test Automation
Easy Control For EverybodyPushToTest TestCards - Best Buy
New TestCard
Assign To Rally
List TestCards
My TestCards
3rd Party Connector
TSH Mock Services
Address Lookup
Verizon Registration
fcohen signed-in, help, sign-out
TestMaker Enterprise 6.0.1
Licensed for 50 virtual users
and 10 service monitors
Welcome To TestMaker TestCards
3rd Party Connector View ResultsDeleteAssociate Rally User StoryCreate Copy
Open Source Test Automation
Detailed Results‣Seeing a pass is comforting, but how do you know it really did?
‣Use drill-down to see details on a passing case
‣Drill down from a fail to the actual error messages
‣Anyone on the team can review any pass/fail results
Open Source Test Automation
Transaction Report
Open Source Test Automation
Drilled Down Detail‣Input:
‣http://server1.bestbuy.com/TSH/v1/offers HTTP/1.1
‣…
‣{ "variable1": "value1", "variable2": "value2" …}
‣Output:
‣HTTP/1.1 500 Internal Server Error
‣...
‣<response>
‣ <error>Unrecognised resource: offers</error>
Open Source Test Automation
Provable Results‣Test results should be 100% reliable, but what’s the safety net?
‣Run it again, Sam.
‣Rally -> Test Case -> TestCard -> Execute
Next let’s talk about
Open Source Test Automation
Agile Performance Testing
‣Test Orchestration
‣Repurposing Methodology
‣Cloud and Grid Deployment
19
Open Source Test Automation
Why An OST Alternative?‣HP QuickTest Professional does a great job for testing a small part of
the Internet
‣Functional Testing (Smoke, Regression, Integration)
‣Windows using ActiveX
‣Flash, Flex
‣MS Internet Explorer
‣Web 2.0 with Coding Effort
‣Best Money Can Buy
20
Open Source Test Automation
OST For Everywhere Else‣Open Source Testing Tools
‣TestMaker Object Designer
‣Selenium
‣Sahi
‣Unit Tests (Java, Ruby, Python, PHP)
‣Record in WebKit based browsers (Chrome, Safari, Opera), IE, Firefox
‣Agile Testing, Agile Performance Testing and Production Monitoring
‣Rich Internet Applications (RIA,) Mobile, Pads
‣OST For Everyone (Developers, Testers, Business Managers)
21
Open Source Test Automation
OST Architecture
22
Application Under TestWeb, RIA, BPM, SOA
PTTMonitorCPU, Net, Memory
Test Injector
BundlesMySQL, HtmlUnit
ScriptRunnersSahi, Selenium, SoapUI, Java
Data Production Librariescsv files, rdbms queries
Test Operation Console
RepositoryMySQL RDBMS
Results AnalysisBusiness Intelligence Tool
DesignerRecord/Playback
Grid and CloudTest Deployment
Continuous Integration Test Management
Open Source Test Automation
Where To Go From Here‣Watch A Screencast and Tutorial
‣http://www.pushtotest.com/screencastcentral
‣Attend An Open Source Test Workshop
‣http://workshop.pushtotest.com
‣Ask For A Proposal on Licenses, Training, Support, Consulting
‣Call +01 408 871 0122, [email protected]
23