Global Chemical Manufacturer Reduces the Cost and Risk of Managing Change Across SAP with Worksoft
The Future of Test Automation Worksoft, Inc. © 2003 All Rights Reserved.
-
Upload
shannon-bradford -
Category
Documents
-
view
213 -
download
0
Transcript of The Future of Test Automation Worksoft, Inc. © 2003 All Rights Reserved.
![Page 1: The Future of Test Automation Worksoft, Inc. © 2003 All Rights Reserved.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d965503460f94a7ee2b/html5/thumbnails/1.jpg)
The Future of Test AutomationThe Future of Test Automation
Worksoft, Inc.© 2003 All Rights Reserved
![Page 2: The Future of Test Automation Worksoft, Inc. © 2003 All Rights Reserved.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d965503460f94a7ee2b/html5/thumbnails/2.jpg)
Agenda
Business Case for Test Automation– Why automate?
Test Automation Challenges– Common mistakes
Software Development Trends– Risk is increasing
Test Automation Trends– Technology is evolving
Future of Test Automation– Quality maturity
![Page 3: The Future of Test Automation Worksoft, Inc. © 2003 All Rights Reserved.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d965503460f94a7ee2b/html5/thumbnails/3.jpg)
Business Case forTest Automation
![Page 4: The Future of Test Automation Worksoft, Inc. © 2003 All Rights Reserved.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d965503460f94a7ee2b/html5/thumbnails/4.jpg)
Why Automate?
Save timeLeverage resourcesReduce costsIncrease coverage
![Page 5: The Future of Test Automation Worksoft, Inc. © 2003 All Rights Reserved.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d965503460f94a7ee2b/html5/thumbnails/5.jpg)
Time
Repeatable tests– Multiple builds– Multiple versions
Unattended execution– Time to design more tests– Time to do manual testing
Concurrent testing– Write tests during requirements– Develop tests during coding
![Page 6: The Future of Test Automation Worksoft, Inc. © 2003 All Rights Reserved.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d965503460f94a7ee2b/html5/thumbnails/6.jpg)
Resources
Reproducible tests– Audit trail of test results– Consistency from one tester to another
Higher volumes– Acquire test data from production– Generate data programmatically
Reduced rework– Catch errors early– Fewer emergencies, patches
![Page 7: The Future of Test Automation Worksoft, Inc. © 2003 All Rights Reserved.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d965503460f94a7ee2b/html5/thumbnails/7.jpg)
Costs
Reduce failures in production– Loss of revenues, market share– Loss of customers, confidence– Loss of user productivity
Reduce support, rework costs– Costs to recover from errors– Costs to remedy defects– Costs to retest, redeploy
![Page 8: The Future of Test Automation Worksoft, Inc. © 2003 All Rights Reserved.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d965503460f94a7ee2b/html5/thumbnails/8.jpg)
Coverage
Functionality increases over time– Inventory of existing features is cumulative– New features are added– Fixes must be verified– Ad hoc tests are needed
Do the math– Cycle times are shorter– Developers outnumber testers– A 10% software change requires 100% testing– A single release will have multiple builds
Features
Tests
Time#
Risk
![Page 9: The Future of Test Automation Worksoft, Inc. © 2003 All Rights Reserved.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d965503460f94a7ee2b/html5/thumbnails/9.jpg)
Test Automation Challenges
![Page 10: The Future of Test Automation Worksoft, Inc. © 2003 All Rights Reserved.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d965503460f94a7ee2b/html5/thumbnails/10.jpg)
Common Mistakes
You don’t have timeYou don’t have resourcesYou don’t understand costsYou can’t measure coverage
![Page 11: The Future of Test Automation Worksoft, Inc. © 2003 All Rights Reserved.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d965503460f94a7ee2b/html5/thumbnails/11.jpg)
Time is Limited
Shorter cycle times– Schedules are market-driven, not effort-based
– Original schedule is always compressed
– Poor quality means longer cycles
Test automation takes time– It’s a development project
– 5X to 10X manual effort
– Ongoing maintenance effort
![Page 12: The Future of Test Automation Worksoft, Inc. © 2003 All Rights Reserved.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d965503460f94a7ee2b/html5/thumbnails/12.jpg)
Resources are Limited
Recording tests takes manual effort– Tests have to be performed manually– You can’t start until software is ready
Scripting takes skill– Tools are programming languages– Automation is a development project
Implementation takes resources– Design, document tests– Develop framework, scripts– Execute, evaluate results
![Page 13: The Future of Test Automation Worksoft, Inc. © 2003 All Rights Reserved.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d965503460f94a7ee2b/html5/thumbnails/13.jpg)
Costs Aren’t Fully Apparent
Cost of acquisition– Educate, evaluate, negotiate, license
Cost to implement– Planning, training, design, development,
documentation
– Requires development resources
Cost of ownership– Continuous maintenance, support
– Change, version control
– Training for new personnel
![Page 14: The Future of Test Automation Worksoft, Inc. © 2003 All Rights Reserved.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d965503460f94a7ee2b/html5/thumbnails/14.jpg)
Coverage is Difficult to Measure
Requirements are missing, obsolete– Test cases are ad hoc– Test knowledge is organic
Regression tests are undefined– Existing functionality not captured– Past defects are forgotten
New features are informal– Poor or missing specifications– Casual or inconsistent hand-off
![Page 15: The Future of Test Automation Worksoft, Inc. © 2003 All Rights Reserved.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d965503460f94a7ee2b/html5/thumbnails/15.jpg)
Software Development Trends
![Page 16: The Future of Test Automation Worksoft, Inc. © 2003 All Rights Reserved.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d965503460f94a7ee2b/html5/thumbnails/16.jpg)
Risks are Increasing
Increased exposureDecreased cycle timesGreater functionalityMore complexity
![Page 17: The Future of Test Automation Worksoft, Inc. © 2003 All Rights Reserved.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d965503460f94a7ee2b/html5/thumbnails/17.jpg)
Increased Exposure
Business rules exposed to users– Rules engines enable analyst involvement– More changes made faster
Applications exposed to customers– Web access becomes customer interface– Direct impact on credibility
Applications exposed to suppliers– Web services open applications– Direct impact on production
![Page 18: The Future of Test Automation Worksoft, Inc. © 2003 All Rights Reserved.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d965503460f94a7ee2b/html5/thumbnails/18.jpg)
Decreased Cycle Times
Market-driven schedules– Software is competitive weapon– Customer, competition drive demands– Supply chain integration drives dates
Accelerated development techniques– Agile, extreme programming practices– Short interval, incremental deliveries– Cycles times of weeks or months
![Page 19: The Future of Test Automation Worksoft, Inc. © 2003 All Rights Reserved.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d965503460f94a7ee2b/html5/thumbnails/19.jpg)
Greater Functionality
Component-based development– Incorporate new functionality quickly– Less development, more testing
Integrated applications– Massive ERP, CRM systems– Easy to configure, difficult to test
Integrated enterprises– Supply chain integration– Standards for development, none for test
![Page 20: The Future of Test Automation Worksoft, Inc. © 2003 All Rights Reserved.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d965503460f94a7ee2b/html5/thumbnails/20.jpg)
Test Automation Trends
![Page 21: The Future of Test Automation Worksoft, Inc. © 2003 All Rights Reserved.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d965503460f94a7ee2b/html5/thumbnails/21.jpg)
Technology is Evolving
Record/ReplayRecord/Replay/ScriptingData-Driven/Keyword Driven
FrameworkClass/Action Framework
![Page 22: The Future of Test Automation Worksoft, Inc. © 2003 All Rights Reserved.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d965503460f94a7ee2b/html5/thumbnails/22.jpg)
Capture/Replay
Test cases executed manually and recorded– No leverage– No concurrent test development
Scripts are unreliable– Sensitive to timing, changes– No logic for error handling
Scripts are not maintainable– Unstructured, undocumented– Hard coded data means huge volumes
![Page 23: The Future of Test Automation Worksoft, Inc. © 2003 All Rights Reserved.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d965503460f94a7ee2b/html5/thumbnails/23.jpg)
Example Script
# Add Accountset_window ("Account Manager", 15);edit_set ("txtName", “Worksoft");obj_type ("txtName","<kTab>");edit_set ("txtAddress", “123 Main Street");obj_type ("txtAddress","<kTab>");edit_set ("txtCity", “Dallaslist_select_item ("cbState", "Texas"); # Item Number 42;edit_set ("Edit", "Texas");edit_set ("txtZip", “99999");obj_type ("txtZip","<kTab>");button_press ("Save_2");
![Page 24: The Future of Test Automation Worksoft, Inc. © 2003 All Rights Reserved.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d965503460f94a7ee2b/html5/thumbnails/24.jpg)
Record/Replay/Scripting
Test cases executed manually, recorded then modified– No concurrent test development
– Requires programming skills
Scripts are complex– Writing programs to test programs
– Code added for synchronization, logic, error handling
– Code added for external data
Scripts are difficult to maintain– Script code can exceed application code
– Turnover means obsolescence
![Page 25: The Future of Test Automation Worksoft, Inc. © 2003 All Rights Reserved.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d965503460f94a7ee2b/html5/thumbnails/25.jpg)
Example Script# Add Account
set_window ("Account Manager", 15);if ( obj_exists “txtName != E_OK ){
gsStepStatus = "edit_Input: Edit object does not exists";return;}status = edit_set ( sAccountName );
}if ( status != E_OK ){
gsStepStatus = "edit_Input: Value could not be input";return;
}type("<kTab>");
![Page 26: The Future of Test Automation Worksoft, Inc. © 2003 All Rights Reserved.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d965503460f94a7ee2b/html5/thumbnails/26.jpg)
Data-Driven/Keyword Driven Framework
Keywords recorded or generated, then modified– Script language specific– Requires programming skills
Scripts are complex, high volume– Complete framework must be developed– May be hundreds of action/keyword variations
Scripts are difficult to maintain– Added or modified for new functionality– Multiple changes for application UI changes
![Page 27: The Future of Test Automation Worksoft, Inc. © 2003 All Rights Reserved.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d965503460f94a7ee2b/html5/thumbnails/27.jpg)
Keyword Framework Structure
Proprietary Tool
Business Processes
Execution ScriptsData Files
Configuration Managed Assets
Keyword Functions
Calls to BPs
Object Map
![Page 28: The Future of Test Automation Worksoft, Inc. © 2003 All Rights Reserved.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d965503460f94a7ee2b/html5/thumbnails/28.jpg)
Example Test Case
Begin testcase Cust-AUD
Name Address City State Zip
Add Account Worksoft 123 Main Dallas Texas 77777
Update Account Acme AC 456 Elm Phoenix Arizona 88888
Delete Account Bay Brick 789 Shore San Jose California 99999
![Page 29: The Future of Test Automation Worksoft, Inc. © 2003 All Rights Reserved.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d965503460f94a7ee2b/html5/thumbnails/29.jpg)
Class/Action Framework Document then execute
– Pre-written libraries, customer-extensible– Proven, supported application
Point and click development– Enforced structure, conventions– Add, manage data easily
Powerful framework– Synchronization, error handling, recovery– Standard and shared processes
Relational database– Automated impact analysis, maintenance– Multi-user, centralized maintenance, administration– All test assets can be housed in a database—no flat files
Open architecture– Single interface across multiple platforms, tools
![Page 30: The Future of Test Automation Worksoft, Inc. © 2003 All Rights Reserved.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d965503460f94a7ee2b/html5/thumbnails/30.jpg)
Class/Action Framework Structure
Business Processes
Data (in DB)
Configuration Managed Assets
Proprietary Tool
Action FunctionsObject
Map
Business Processes and data are decoupled from tool
![Page 31: The Future of Test Automation Worksoft, Inc. © 2003 All Rights Reserved.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d965503460f94a7ee2b/html5/thumbnails/31.jpg)
Application Map
![Page 32: The Future of Test Automation Worksoft, Inc. © 2003 All Rights Reserved.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d965503460f94a7ee2b/html5/thumbnails/32.jpg)
Define Process
![Page 33: The Future of Test Automation Worksoft, Inc. © 2003 All Rights Reserved.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d965503460f94a7ee2b/html5/thumbnails/33.jpg)
Add Application
![Page 34: The Future of Test Automation Worksoft, Inc. © 2003 All Rights Reserved.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d965503460f94a7ee2b/html5/thumbnails/34.jpg)
Add Window
![Page 35: The Future of Test Automation Worksoft, Inc. © 2003 All Rights Reserved.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d965503460f94a7ee2b/html5/thumbnails/35.jpg)
Select Object
![Page 36: The Future of Test Automation Worksoft, Inc. © 2003 All Rights Reserved.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d965503460f94a7ee2b/html5/thumbnails/36.jpg)
Select Action
![Page 37: The Future of Test Automation Worksoft, Inc. © 2003 All Rights Reserved.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d965503460f94a7ee2b/html5/thumbnails/37.jpg)
Input Value
Literal data
Variable data
![Page 38: The Future of Test Automation Worksoft, Inc. © 2003 All Rights Reserved.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d965503460f94a7ee2b/html5/thumbnails/38.jpg)
Generic Functions
public function edit_Input ( sObjectId, sObjectName, sWindowId, sWindowName ){
if ( obj_exists ( sObjectId ) != E_OK ) {gsStepStatus = "edit_Input: Edit object does not exists";return;
} if ( GetStringArgument ( "Value", gsExpectedValue, sDefaultValue ) == FAILED )
{gsStepStatus = "edit_Input: Action parameter (Value) could not be retrieved";return;
}status = edit_set ( sObjectId, gsExpectedValue );
if ( status != E_OK ){
gsStepStatus = "edit_Input: Value could not be input";return;
}type("<kTab>");gbActionStatus = PASSED;return;
}
![Page 39: The Future of Test Automation Worksoft, Inc. © 2003 All Rights Reserved.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d965503460f94a7ee2b/html5/thumbnails/39.jpg)
Specify Result on Pass, Fail
![Page 40: The Future of Test Automation Worksoft, Inc. © 2003 All Rights Reserved.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d965503460f94a7ee2b/html5/thumbnails/40.jpg)
Specify Log Result
![Page 41: The Future of Test Automation Worksoft, Inc. © 2003 All Rights Reserved.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d965503460f94a7ee2b/html5/thumbnails/41.jpg)
Add Data
![Page 42: The Future of Test Automation Worksoft, Inc. © 2003 All Rights Reserved.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d965503460f94a7ee2b/html5/thumbnails/42.jpg)
Automated Documentation
Process / Window Step Description*Account Master
Press New CommandButtonInput Worksoft into Account Name TextBoxInput 123 Main into Account Address TextBoxInput Dallas into Account City TextBoxSelect Item Texas from Account State ListBoxPress Save CommandButton
*MessageBoxVerify Text is Equal To “Account Added”Press OK CommandButton
![Page 43: The Future of Test Automation Worksoft, Inc. © 2003 All Rights Reserved.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d965503460f94a7ee2b/html5/thumbnails/43.jpg)
Automated Impact Analysis
![Page 44: The Future of Test Automation Worksoft, Inc. © 2003 All Rights Reserved.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d965503460f94a7ee2b/html5/thumbnails/44.jpg)
Automated Maintenance
![Page 45: The Future of Test Automation Worksoft, Inc. © 2003 All Rights Reserved.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d965503460f94a7ee2b/html5/thumbnails/45.jpg)
Extensive Reporting
![Page 46: The Future of Test Automation Worksoft, Inc. © 2003 All Rights Reserved.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d965503460f94a7ee2b/html5/thumbnails/46.jpg)
Quality Maturity
Proprietary languages are being retired– Shift to native languages
Testability is becoming accepted– Shift to cooperative development
Automation is becoming automated– Tools are maturing
Increased management awareness– Risks are higher
![Page 47: The Future of Test Automation Worksoft, Inc. © 2003 All Rights Reserved.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d965503460f94a7ee2b/html5/thumbnails/47.jpg)
Shift to Native Languages
Proprietary languages too costly– Skilled resources are scarce– 10X development costs– Deployment license costs– Not all tools support all platforms
Major vendors in transition– Mercury: WinRunner to QuickTest Pro/VBScript– Compuware: QA Run to Test Partner/VBA– Rational: Robot VB to Java
![Page 48: The Future of Test Automation Worksoft, Inc. © 2003 All Rights Reserved.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d965503460f94a7ee2b/html5/thumbnails/48.jpg)
Shift to Cooperative Development
Development practices improving– Coding standards, conventions– Object properties, methods exposed– Unique, persistent, meaningful names
Development support enhanced– Code instrumentation more likely– Standards compliance more rigorous– Test APIs more common
![Page 49: The Future of Test Automation Worksoft, Inc. © 2003 All Rights Reserved.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d965503460f94a7ee2b/html5/thumbnails/49.jpg)
Tools are Maturing
Next generation automation products– Certify– Unified TestPro, ABT, TestFrame
Automated unit testing – JUnit, NUnit– .TEST– AUTT
Testing designed into architectures– Test standards for Web services
![Page 50: The Future of Test Automation Worksoft, Inc. © 2003 All Rights Reserved.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d965503460f94a7ee2b/html5/thumbnails/50.jpg)
Risks are Higher
Sarbanes Oxley raises stakes– CXO-level responsibility for financial accuracy– Duty imposed for internal controls– Potential civil, criminal liability
Customer, exterprise integration raises exposure– Defects are visible to outside world– Errors can impact entire supply chain
![Page 51: The Future of Test Automation Worksoft, Inc. © 2003 All Rights Reserved.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d965503460f94a7ee2b/html5/thumbnails/51.jpg)
Build/Buy the Right Framework
Less development, more testing– Introduce development to testability– Focus test engineers on changes and exceptions– Empower business analysts to automate
Reduce cost of implementation, ownership– Less training, faster deployment– Less maintenance, support– Lower tool license costs
Introduce enterprise standards– Enable seamless end to end execution– Provide portability across tools, platforms– Leverage resources across applications