St Mark Missionary Baptist Church Announcements - 2010 Feb 28
Mark meninger-feb-2010
-
Upload
kitchener-waterloo-agile -
Category
Documents
-
view
840 -
download
0
Transcript of Mark meninger-feb-2010
![Page 1: Mark meninger-feb-2010](https://reader034.fdocuments.in/reader034/viewer/2022052505/554d32fcb4c905ab268b54ee/html5/thumbnails/1.jpg)
An Examination of Test Automation Use Cases(and the factors behind the examination)
January 19, 2010
1
Mark Meninger
![Page 2: Mark meninger-feb-2010](https://reader034.fdocuments.in/reader034/viewer/2022052505/554d32fcb4c905ab268b54ee/html5/thumbnails/2.jpg)
Confidential McAfee Internal Use Only
An Examination of Test Automation Use Cases
Contents
• The Speaker
• Presentation Objectives
Objectives and Perspective
• Objectives of test automation
• GUI vs non-GUI
• A look at 2 products
Test Automation Philosophy
Test Automation Use Cases
How does this all fit in?
January 20, 2010
2
![Page 3: Mark meninger-feb-2010](https://reader034.fdocuments.in/reader034/viewer/2022052505/554d32fcb4c905ab268b54ee/html5/thumbnails/3.jpg)
Confidential McAfee Internal Use Only
January 20, 2010
3
Objectives and Perspective
![Page 4: Mark meninger-feb-2010](https://reader034.fdocuments.in/reader034/viewer/2022052505/554d32fcb4c905ab268b54ee/html5/thumbnails/4.jpg)
Confidential McAfee Internal Use Only
January 20, 2010
4
The Speaker ([email protected])
Objectives and Perspective
• Current role @ McAfee: Driving test automation on Consumer side
• Functional GUI automation (support for L10n) across 3 sites
• Functional non-GUI automation (development focused)
• Automated Performance testing
• Previously @ RIM
• As test automation manager for end-user BlackBerry testing
• RIM Test Automation Conference Chair
• A student of test automation and testing
• Worked (and work) with some very capable individuals
• Made some good mistakes
• Learned from my mistakes
• Little experience with Agile
• Presentations @ StarEast, QAI, KWSQA
• My Blog (as of this Friday): http://automatedtestingblog.blogspot.com
![Page 5: Mark meninger-feb-2010](https://reader034.fdocuments.in/reader034/viewer/2022052505/554d32fcb4c905ab268b54ee/html5/thumbnails/5.jpg)
Confidential McAfee Internal Use Only
January 20, 2010
5
Presentation Objectives
Objectives and Perspective
1. Talk about my learnings and philosophy of test automation
2. Hear your perspective on what I’m talking about
3. Evaluate test automation use cases/examples (group-
exercise)
a) Interface complexity
b) Speed of test automation
c) Speed of execution
d) Derived value
e) How this fits into the testing cycle
4. In the context of Agile, I would…
![Page 6: Mark meninger-feb-2010](https://reader034.fdocuments.in/reader034/viewer/2022052505/554d32fcb4c905ab268b54ee/html5/thumbnails/6.jpg)
Confidential McAfee Internal Use Only
January 20, 2010
6
Mark’s Test Automation Philosophy
Test Automation Philosophy
![Page 7: Mark meninger-feb-2010](https://reader034.fdocuments.in/reader034/viewer/2022052505/554d32fcb4c905ab268b54ee/html5/thumbnails/7.jpg)
Confidential McAfee Internal Use Only
January 20, 2010
7
Manual Testing - A path in the trees
Test Automation Philosophy
• Strategy/Plan (resourcing, infrastructure, approach)
• Need to learn the SUT:
• User-stories
• Use Cases
• Tools
• Technologies
• Testing types & cycles (regression, defect, smoke)
• Sprints
• Apply common techniques
• Scripted test cases (with different methodologies),
• Exploratory testing
• Then testing can begin
• I’ve seen most testing organizations do this well to varying degrees
![Page 8: Mark meninger-feb-2010](https://reader034.fdocuments.in/reader034/viewer/2022052505/554d32fcb4c905ab268b54ee/html5/thumbnails/8.jpg)
Confidential McAfee Internal Use Only
January 20, 2010
8
Automated testing – Find a path in the forest
Test Automation Philosophy
• Test automation is less of a cookie-cutter approach than manual testing
• Common tasks in test automation
• Interface, framework, integration
• Very specific considerations:
• The technology of the system under test (SUT)
• The technologies used to test the SUT
• Impact on the development/testing schedule
• These details impact the success of the
implementation, the derived value, the
time to delivery, availability etc
• This makes each test automation
implementation unique and challenging
• Not all orgs do test automation well
![Page 9: Mark meninger-feb-2010](https://reader034.fdocuments.in/reader034/viewer/2022052505/554d32fcb4c905ab268b54ee/html5/thumbnails/9.jpg)
Confidential McAfee Internal Use Only
January 20, 2010
9
Evolution of the Approach to Test Automation
Test Automation Philosophy
In the beginning:
Here is a test automation tool… now automate! (seen it)
![Page 10: Mark meninger-feb-2010](https://reader034.fdocuments.in/reader034/viewer/2022052505/554d32fcb4c905ab268b54ee/html5/thumbnails/10.jpg)
Confidential McAfee Internal Use Only
January 20, 2010
10
Evolution of the Approach to Test Automation
Test Automation Philosophy
And Then:
Evaluate tools, choose one and automate!
(done it)
![Page 11: Mark meninger-feb-2010](https://reader034.fdocuments.in/reader034/viewer/2022052505/554d32fcb4c905ab268b54ee/html5/thumbnails/11.jpg)
Confidential McAfee Internal Use Only
January 20, 2010
11
Evolution of the Approach to Test Automation
Test Automation Philosophy
After much learning (now do it!) :Examine my test automation requirements
a) Look at
• Objectives
• SUT technology, roadmap
• Available budget, resources
• Available tool technologies, capabilities and constraints
• SDLC methodology (Agile, Waterfall etc.)
• Development culture
• Testing culture
• Relationships
b) Put together my business case
c) Start with a careful plan
![Page 12: Mark meninger-feb-2010](https://reader034.fdocuments.in/reader034/viewer/2022052505/554d32fcb4c905ab268b54ee/html5/thumbnails/12.jpg)
Confidential McAfee Internal Use Only
January 20, 2010
12
Objectives of test automation
1. Executed successfully
relatively early within the
development/testing cycle
2. Reduction in manual testing
time
Test Automation Philosophy
�Test functionality sooner
rather than later in the cycle
�Interface used is available
and stable
�Automated execution as soon
as a build is released:
� Evenings, Weekends
�Automated execution run in
parallel
� Execution across platforms and
test suites
� Limited by availability of
infrastructure
![Page 13: Mark meninger-feb-2010](https://reader034.fdocuments.in/reader034/viewer/2022052505/554d32fcb4c905ab268b54ee/html5/thumbnails/13.jpg)
Confidential McAfee Internal Use Only
January 20, 2010
13
Test Automation Objectives
3. Provides reliable results on
1st run
4. The solution is scalable &
maintainable
Test Automation Philosophy
�Tests can be executed repeatedly
with minimal false negatives
�Regular testing of interface and
framework essential
�Changing interface not breaking
tests
�We can increase coverage of
manual test cases without excessive
framework growth
�Grow test case numbers with
confidence:
� 10’s -> 100’s
� 100’s -> 1000’s
![Page 14: Mark meninger-feb-2010](https://reader034.fdocuments.in/reader034/viewer/2022052505/554d32fcb4c905ab268b54ee/html5/thumbnails/14.jpg)
Confidential McAfee Internal Use Only
January 20, 2010
14
Test Automation Objectives
5. Agile Environment
Test Automation Philosophy
�Out-of-the-box, light-weight test
automation supports planned sprints
�Test automation team structured,
organized to support agile testing
requirements
�Solution, approach is flexible and
expandable
![Page 15: Mark meninger-feb-2010](https://reader034.fdocuments.in/reader034/viewer/2022052505/554d32fcb4c905ab268b54ee/html5/thumbnails/15.jpg)
Confidential McAfee Internal Use Only
January 20, 2010
15
GUI vs non-GUI Technology Considerations
Test Automation Philosophy
GUINon-GUI(some API)
![Page 16: Mark meninger-feb-2010](https://reader034.fdocuments.in/reader034/viewer/2022052505/554d32fcb4c905ab268b54ee/html5/thumbnails/16.jpg)
Confidential McAfee Internal Use Only
January 20, 2010
16
Test Automation – The Big Question
Test Automation Philosophy
Re
UI-Logic Interface
UI
And Another
The Basement
Another Abstraction
Do we start here?
Or here?
Or here?
Where?Where?Where?Where?
SoftwareAbstractions
First Business Logic Layer
![Page 17: Mark meninger-feb-2010](https://reader034.fdocuments.in/reader034/viewer/2022052505/554d32fcb4c905ab268b54ee/html5/thumbnails/17.jpg)
Confidential McAfee Internal Use Only
January 20, 2010
17
Test Automation - GUI Fat Client (Generalities)
Test Automation Philosophy
• Interface
• Typically complex - more points of failure
• Changes more frequent
• Increased maintenance
• Automation - dependent upon being stable
• Technology
• Vendor tools provide better support
• Scripting technology usually has limited support (i.e. VBScript)
• Performance
• Management for unresponsive GUI
• GUI adds performance hit each time accessed
• Framework can add considerable overhead
GUI - Fat Client• Skill-set
• Requires solid technical skill-set with good design and programming abilities
• Deeper test automation skill-set and experience required
• More expensive
• Localization
• Running identical functional tests across localized builds
• Language independence support is essential to support L10n automation
![Page 18: Mark meninger-feb-2010](https://reader034.fdocuments.in/reader034/viewer/2022052505/554d32fcb4c905ab268b54ee/html5/thumbnails/18.jpg)
Confidential McAfee Internal Use Only
January 20, 2010
18
Test Automation – GUI Web (Generalities)
Test Automation Philosophy
• Interface
• Typically simpler – fewer complexities to manage
• Changes frequently over time
• Automation – dependent upon being stable
• Technology
• Good open source (Selenium, Watir)
• Performance
• Fewer if no performance issues
• Good available open source frameworks
• Skill-set
• Better use of ‘scripters’ rather than developers
• Less investment required in more experienced test automation skill-sets
• Less expensive
• Localization
• Language independent testing can be supported
GUI - Web
![Page 19: Mark meninger-feb-2010](https://reader034.fdocuments.in/reader034/viewer/2022052505/554d32fcb4c905ab268b54ee/html5/thumbnails/19.jpg)
Confidential McAfee Internal Use Only
January 20, 2010
19
Test Automation – non-GUI (Generalities)
Test Automation Philosophy
• Interface
• Stable and reliable earlier in the dev/testing cycle
• Less changes over time
• Fewer points of failure
• High-level testing can be supported
• Usually needs to be modified for increased testability over time (fits Agile well IMO)
• Technology
• Choose scripting technology with robust library support
• Integrate into development cycle
Non-GUI (API) Automation
• Performance
• No GUI impact - much faster
• Choose integrated framework with little overhead
• Skill-set
• Requires knowledge of underlying business logic, application architecture and design
• More expensive
• Localization
• Issues around language independence (localized strings integrated into GUI abstractions)
![Page 20: Mark meninger-feb-2010](https://reader034.fdocuments.in/reader034/viewer/2022052505/554d32fcb4c905ab268b54ee/html5/thumbnails/20.jpg)
Confidential McAfee Internal Use Only
January 20, 2010
20
GUI Automation
Test Automation Philosophy
• Automation Implication
• Coverage
• Good top-to-bottom testing solution
• Broader
• # of automated test cases dependent upon GUI comlexity
• Development (GUI complexity)
• Higher maintenance costs
• More effort to write tests
• Longer development times
• Execution
• Dependent upon stable GUI
• Slower execution times
• More expensive (resources, equipment and license cost)
• Value-add later in dev/testing cycle for products with major GUI changes
GUI
![Page 21: Mark meninger-feb-2010](https://reader034.fdocuments.in/reader034/viewer/2022052505/554d32fcb4c905ab268b54ee/html5/thumbnails/21.jpg)
Confidential McAfee Internal Use Only
January 20, 2010
21
Non-GUI Automation
Test Automation Philosophy
• Automation Implication
• Coverage
• Not top-to-bottom
• Deeper testing
• Larger # of automated test cases
• Development
• Easier to write tests
• Shorter development times
• Lower maintenance costs
• Execution
• Ready when the API are ready
• Test much sooner in the dev/test
• Test results faster; test more often
• Gate before release to testing
• Cheaper (resources, no licenses)
• Add quality paradigm to development organization (code for testability)
• Value-add sooner in dev/testing cycle
Non-GUI Automation
![Page 22: Mark meninger-feb-2010](https://reader034.fdocuments.in/reader034/viewer/2022052505/554d32fcb4c905ab268b54ee/html5/thumbnails/22.jpg)
Confidential McAfee Internal Use Only
January 20, 201022
RTQA Build Release Cycle
API Automation
Quality AssuranceStarts
Automated
API Testing
Manual
Testing
TestsPass?
Automated
API Testing
Automated GUI
Testing & Manual
Testing
TestsPass?
!!!??
Manual TestingGUI Automation
Start End
No No
YesYes
![Page 23: Mark meninger-feb-2010](https://reader034.fdocuments.in/reader034/viewer/2022052505/554d32fcb4c905ab268b54ee/html5/thumbnails/23.jpg)
Confidential McAfee Internal Use Only
January 20, 2010
23
Automated Testing Coverage
Cost
Complex GUI - Cost for Coverage
• Complex GUI applications will incur an increasing cost to
automate larger #’s of test cases
• Cost: Effort, Infrastructure, Dollars, Time for Execution
Automation Ceiling
Costs become prohibitive
Test Automation Philosophy
GUIGUIGUIGUI
![Page 24: Mark meninger-feb-2010](https://reader034.fdocuments.in/reader034/viewer/2022052505/554d32fcb4c905ab268b54ee/html5/thumbnails/24.jpg)
Confidential McAfee Internal Use Only
January 20, 2010
24
Automated Testing Coverage
Cost
Automation Philosophy Review – Cost for Coverage
• Non-GUI Automation has lower cost for coverage
• Cost: Effort, Infrastructure, Dollars, Time for Execution
Automation Ceiling
Less expensive
Test Automation Philosophy
GUIGUIGUIGUI
NonNonNonNon----GUIGUIGUIGUI
![Page 25: Mark meninger-feb-2010](https://reader034.fdocuments.in/reader034/viewer/2022052505/554d32fcb4c905ab268b54ee/html5/thumbnails/25.jpg)
Confidential McAfee Internal Use Only
r
Consumer Test Automation Framework (CTAF)/QTP/MAGI
MAGI FrameworkServices
Core Functions
CTAF Extensions
HP Quick Test Pro
GUI HTMLControl
ID HTMLControl
ID HTMLControl
ID
CTAF External LibrariesGUI/Table
LanguageDependentAssert
VBScript Test Script
CTAF Internal Libraries
Unit Test
Helper
GlobalMPF MVS
Lots of librariesto build(and support)
Complex Framework(yet very helpful)
Our ownextensions
![Page 26: Mark meninger-feb-2010](https://reader034.fdocuments.in/reader034/viewer/2022052505/554d32fcb4c905ab268b54ee/html5/thumbnails/26.jpg)
Confidential McAfee Internal Use Only
Python py.test/nose(?) Test Automation Framework
Python
Libraries
COM InterfaceImplement IDispatch
High-Level Class
High-Level Class
High-Level Class
Python Test Script
CTAF API Python
Libraries
Example API Test Automation – Using Python
Python hasnumerousrich and diverse libraries
Framework exists which nicely integrates with Python
![Page 27: Mark meninger-feb-2010](https://reader034.fdocuments.in/reader034/viewer/2022052505/554d32fcb4c905ab268b54ee/html5/thumbnails/27.jpg)
Confidential McAfee Internal Use Only
January 20, 2010
27
Test Automation Use Cases
A look at some interfaces
• Google Search
• McAfee Security Center (2010 release)
Disclaimer
• These thoughts are my own and are not necessary correct
• Part of the process would be for me to understand the
technology and determine the best approach
(Finding a path through the forest I find myself in)
![Page 28: Mark meninger-feb-2010](https://reader034.fdocuments.in/reader034/viewer/2022052505/554d32fcb4c905ab268b54ee/html5/thumbnails/28.jpg)
Confidential McAfee Internal Use Only
Test Automation Use Cases
Points to Evaluate
• What approach would you take?
• What are the risks of doing it this way?
• What are the costs/investment requirements of doing this? (be
specific)
• What are the gains of doing this?
• Why would you do this?
• How would you integrate this into a testing cycle?
• Why would you integrate this into the testing cycle this way?
• How could this approach be fit into an Agile/Lean
methodology?
• What type of buy-in would you need to achieve this?
• What relationships would you need to establish to be
successful?
January 20, 2010
28
![Page 29: Mark meninger-feb-2010](https://reader034.fdocuments.in/reader034/viewer/2022052505/554d32fcb4c905ab268b54ee/html5/thumbnails/29.jpg)
Confidential McAfee Internal Use Only
Test Automation Use Cases
What I will focus on
• Interface complexity
• Speed of test automation
• Speed of execution
• Derived value
• How this fits into the testing cycle
January 20, 2010
29
![Page 30: Mark meninger-feb-2010](https://reader034.fdocuments.in/reader034/viewer/2022052505/554d32fcb4c905ab268b54ee/html5/thumbnails/30.jpg)
Confidential McAfee Internal Use Only
January 20, 2010
30
![Page 31: Mark meninger-feb-2010](https://reader034.fdocuments.in/reader034/viewer/2022052505/554d32fcb4c905ab268b54ee/html5/thumbnails/31.jpg)
Confidential McAfee Internal Use Only
Test Automation Use Cases – Google Search
•Google Search
– From: http://en.wikipedia.org/wiki/Google_search
– PageRank Logic
– synonyms, weather forecasts, time zones, stock quotes,
maps, earthquake data, movie showtimes, airports, home
listings, and sports scores
– prices, temperatures, money/unit conversions ("10.5 cm in
inches"), calculations ( 3*4+sqrt(6)-pi/2 ), package tracking,
patents, area codes,[6] and rudimentary language
translation of displayed pages
– ‘I’m feeling lucky’
– Privatization logic (Switzerland)
– Ajax code
– ….
January 20, 2010
31
![Page 32: Mark meninger-feb-2010](https://reader034.fdocuments.in/reader034/viewer/2022052505/554d32fcb4c905ab268b54ee/html5/thumbnails/32.jpg)
Confidential McAfee Internal Use Only
Test Automation Use Cases – Google Search
Our job to automate Search logic • Core functionality
• Page rank is an algorithm that evaluates an index using
supposedly over 200 factors
January 20, 2010
32
![Page 33: Mark meninger-feb-2010](https://reader034.fdocuments.in/reader034/viewer/2022052505/554d32fcb4c905ab268b54ee/html5/thumbnails/33.jpg)
Confidential McAfee Internal Use Only
Test Automation Use Cases – Google Search
•What I’d do– Back-end (non-gui) automation
– Focus just on the engine and it’s data
– Evaluate the probability and weighting of each factor
– Generate a list of results and probably would fit into
some level of confidence of accuracy
– Rendering of data could be easily verified manually
January 20, 2010
33
![Page 34: Mark meninger-feb-2010](https://reader034.fdocuments.in/reader034/viewer/2022052505/554d32fcb4c905ab268b54ee/html5/thumbnails/34.jpg)
Confidential McAfee Internal Use Only
Test Automation Use Cases – Google Search
•How• Work with developers to build a testing engine that could handle
‘plug-ins’ of new testing factors
• A common testing pattern would be for each factor
• Determine how each factor would return results on its own or in
interaction with another factor
• Integrate this all into the testing engine
• Establish a common mechanism for executing tests, gathering
expected results and comparing actual results
• Utilize the common testing pattern
• Utilize the same pattern for evaluation in the testing engine
• Drive a common testing interface across those adding new ranking
functionality to the google search engine
January 20, 2010
34
![Page 35: Mark meninger-feb-2010](https://reader034.fdocuments.in/reader034/viewer/2022052505/554d32fcb4c905ab268b54ee/html5/thumbnails/35.jpg)
Confidential McAfee Internal Use Only
Test Automation Use Cases
Interface complexity
• Interface would be simple – an API
• The combination of algorithms (factors) would make this
complex
Speed of test automation delivery
• Fast
– Could start writing tests for each factor
– Could start building in some relationships for each factor
• Slower
– Integration of everything together
– This would also include the test engine
January 20, 2010
35
![Page 36: Mark meninger-feb-2010](https://reader034.fdocuments.in/reader034/viewer/2022052505/554d32fcb4c905ab268b54ee/html5/thumbnails/36.jpg)
Confidential McAfee Internal Use Only
Test Automation Use Cases
Speed of execution
• Very fast
• Could run very often
Derived value
• Substantial
• Could fully automate the algorithm
• If the integration of factors together could be done
successfully, this would be substantial
How this fits into the testing cycle
• Immediately
• Write code, test code
• No throw-over to SV&V/QA
January 20, 2010
36
![Page 37: Mark meninger-feb-2010](https://reader034.fdocuments.in/reader034/viewer/2022052505/554d32fcb4c905ab268b54ee/html5/thumbnails/37.jpg)
Confidential McAfee Internal Use Only
January 20, 2010
37
![Page 38: Mark meninger-feb-2010](https://reader034.fdocuments.in/reader034/viewer/2022052505/554d32fcb4c905ab268b54ee/html5/thumbnails/38.jpg)
Confidential McAfee Internal Use Only
January 20, 2010
38
Cialis - erectile dysfunction drug
Radical Prostatectomy - is an operation to
remove the prostate gland and some of
the tissue around it Very fast
![Page 39: Mark meninger-feb-2010](https://reader034.fdocuments.in/reader034/viewer/2022052505/554d32fcb4c905ab268b54ee/html5/thumbnails/39.jpg)
Confidential McAfee Internal Use Only
January 20, 2010
39
![Page 40: Mark meninger-feb-2010](https://reader034.fdocuments.in/reader034/viewer/2022052505/554d32fcb4c905ab268b54ee/html5/thumbnails/40.jpg)
Confidential McAfee Internal Use Only
Test Automation Use Cases – McAfee Security
Center
•McAfee Consumer Product (2010)
– Security Center
– Firewall, AntiVirus, AntiSpam, Parental Controls…
– Focus on AntiVirus
– Verify GUI and Functional behaviour
January 20, 2010
40
![Page 41: Mark meninger-feb-2010](https://reader034.fdocuments.in/reader034/viewer/2022052505/554d32fcb4c905ab268b54ee/html5/thumbnails/41.jpg)
Confidential McAfee Internal Use Only
Test Automation Use Cases
• McAfee Consumer Product (2010)
– GUI – thick GUI
– Verify GUI and Functional behaviour
January 20, 2010
41
![Page 42: Mark meninger-feb-2010](https://reader034.fdocuments.in/reader034/viewer/2022052505/554d32fcb4c905ab268b54ee/html5/thumbnails/42.jpg)
Confidential McAfee Internal Use Only
Test Automation Use Cases
• How
• Vendor tool
January 20, 2010
42
![Page 43: Mark meninger-feb-2010](https://reader034.fdocuments.in/reader034/viewer/2022052505/554d32fcb4c905ab268b54ee/html5/thumbnails/43.jpg)
Confidential McAfee Internal Use Only
Test Automation Use Cases
• McAfee Consumer Product (2010)
– GUI – thick GUI
– Verify GUI and Functional behaviour
January 20, 2010
43
![Page 44: Mark meninger-feb-2010](https://reader034.fdocuments.in/reader034/viewer/2022052505/554d32fcb4c905ab268b54ee/html5/thumbnails/44.jpg)
Confidential McAfee Internal Use Only
Test Automation Use Cases
• McAfee Consumer Product (2010)
– GUI – thick GUI
– Verify GUI and Functional behaviour
January 20, 2010
44
![Page 45: Mark meninger-feb-2010](https://reader034.fdocuments.in/reader034/viewer/2022052505/554d32fcb4c905ab268b54ee/html5/thumbnails/45.jpg)
Confidential McAfee Internal Use Only
Test Automation Use Cases – McAfee Security
Center
•What I’d do & How– Get a good GUI automation tool
– Find a manageable way to integrate with the GUI DOM
• Re-usability
• Maintainability
– First go: limit my automation to easiest test cases
– Find a good framework to integrate with my GUI
automation tool
• Automate the automation
– Aim would be reduce manual testing with an eye to
reduce maintenance
January 20, 2010
45
![Page 46: Mark meninger-feb-2010](https://reader034.fdocuments.in/reader034/viewer/2022052505/554d32fcb4c905ab268b54ee/html5/thumbnails/46.jpg)
Confidential McAfee Internal Use Only
Test Automation Use Cases – McAfee Security
Center
Interface complexity
• Interface is very complex
– Lots of objects to manage (lots of points of failure)
– How we work with the interface is complex under the
covers (constrained by tool)
Speed of test automation delivery
• Slow!!
– Most off-the-shelf have limited library support
– Have common set of libraries across the products
– Need to build libraries for each product at the GUI & functional
level
– Integration into the framework could take even longer
January 20, 2010
46
![Page 47: Mark meninger-feb-2010](https://reader034.fdocuments.in/reader034/viewer/2022052505/554d32fcb4c905ab268b54ee/html5/thumbnails/47.jpg)
Confidential McAfee Internal Use Only
Test Automation Use Cases – McAfee Security
Center
Speed of execution
• Slow!
• Fat client GUIs are usually slower
• Hooks from application driving the GUI adds overhead
• Integrating a framework that drives all this adds additional
time
Derived value
• Some!
• New GUI; some changes – later in the cycle
• Same GUI; little changes – earlier in the cycle
• Need more infrastructure to support
January 20, 2010
47
![Page 48: Mark meninger-feb-2010](https://reader034.fdocuments.in/reader034/viewer/2022052505/554d32fcb4c905ab268b54ee/html5/thumbnails/48.jpg)
Confidential McAfee Internal Use Only
Test Automation Use Cases – McAfee Security
Center
How this fits into the testing cycle
• If no GUI changes - > Right Away
• GUI changes - > Later
• Testing cycles will take longer
• Slower to execute
January 20, 2010
48
![Page 49: Mark meninger-feb-2010](https://reader034.fdocuments.in/reader034/viewer/2022052505/554d32fcb4c905ab268b54ee/html5/thumbnails/49.jpg)
Confidential McAfee Internal Use Only
January 20, 2010
49
Test Automation – The Big Question
Test Automation Use Cases – McAfee Security Center
Re
UI-Logic Interface
UI
And Another
The Basement
First Business Logic Layer
Another Abstraction
Do we start here?
Or here?
SoftwareAbstractions
![Page 50: Mark meninger-feb-2010](https://reader034.fdocuments.in/reader034/viewer/2022052505/554d32fcb4c905ab268b54ee/html5/thumbnails/50.jpg)
Confidential McAfee Internal Use Only
Test Automation Use Cases – McAfee Security
Center
•What I’d do & How– Examine the layers below the GUI
– Can I use any of them to automate testing?
– Work to drive this idea within the organization
– Most likely some developer has thought the same thing
– That is my foothold and I build on that
January 20, 2010
50
![Page 51: Mark meninger-feb-2010](https://reader034.fdocuments.in/reader034/viewer/2022052505/554d32fcb4c905ab268b54ee/html5/thumbnails/51.jpg)
Confidential McAfee Internal Use Only
Test Automation Use Cases – McAfee Security
Center
Interface complexity
• Interface is less complex
– New technology learning curve
– Easier to call some API than manage GUI objects
– Less change; more stability
Speed of test automation delivery
– Fast!!
– Available frameworks (don’t need to build your own) (i.e. py.test)
– Access to rich scripting environments & libraries (i.e. Python) – don’t
need to build your own
– Less points of failure to manage
January 20, 2010
51
![Page 52: Mark meninger-feb-2010](https://reader034.fdocuments.in/reader034/viewer/2022052505/554d32fcb4c905ab268b54ee/html5/thumbnails/52.jpg)
Confidential McAfee Internal Use Only
Test Automation Use Cases – McAfee Security
Center
Speed of execution
• Fast!! No GUI overhead
• Integrated framework will also add little overhead (i.e.
TestNG, Py.test)
Derived value
• Higher value
How this fits into the testing cycle
• Almost always test earlier in the cycle
• Test more frequently
• Integrate into the development cycle rather than QA
– Quality moving upstream
January 20, 2010
52
![Page 53: Mark meninger-feb-2010](https://reader034.fdocuments.in/reader034/viewer/2022052505/554d32fcb4c905ab268b54ee/html5/thumbnails/53.jpg)
Confidential McAfee Internal Use Only
January 20, 2010
53
![Page 54: Mark meninger-feb-2010](https://reader034.fdocuments.in/reader034/viewer/2022052505/554d32fcb4c905ab268b54ee/html5/thumbnails/54.jpg)
Confidential McAfee Internal Use Only
January 20, 2010
54
How does this all fit in?
Objectives and Perspective
• Have your test automation specialist begin to develop a methodology that
will fit your agile development cycle
• Tell her/him what your requirements are and ask for a solution that will
meet this
• Optimize your test automation execution for Agile!
• This will most likely be an out-of-the-box approach to test automation
• All pay-offs should be well understood:
• Light-weight, quick to execute, easy to develop, not-too-deep solution
• Heavier, complex, longer-to execute, harder to develop, deeper test
automation solution
![Page 55: Mark meninger-feb-2010](https://reader034.fdocuments.in/reader034/viewer/2022052505/554d32fcb4c905ab268b54ee/html5/thumbnails/55.jpg)
Confidential McAfee Internal Use Only
January 20, 2010
55
How does this all fit in?
Objectives and Perspective
• They should have a good understanding of the available technologies to
use and what the trade-offs are for each
• What solution will best meet the above requirements?
• Let that test automation specialist know what’s needed and why. This will
hopefully inspire them to build the solution that meets these needs
• Integrate automated testing into your testing cycles
• Fit the automated testing for a given sprint/release
• Each automated testing approach will have a given set of benefits and
costs
• Choose the automated testing items that make most sense
![Page 56: Mark meninger-feb-2010](https://reader034.fdocuments.in/reader034/viewer/2022052505/554d32fcb4c905ab268b54ee/html5/thumbnails/56.jpg)
Confidential McAfee Internal Use Only
Consumer Applications QATest Automation Strategy – Detailed Review
January 20, 2010
56
As of this Friday
http://automatedtestingblog.blogspot.com