Wso2 test automation framework internal training
-
Upload
dharshana-warusavitharana -
Category
Documents
-
view
113 -
download
4
description
Transcript of Wso2 test automation framework internal training
![Page 1: Wso2 test automation framework internal training](https://reader034.fdocuments.in/reader034/viewer/2022042814/54c5e66a4a7959905e8b4578/html5/thumbnails/1.jpg)
WSO2 Test Automation Framework
Approach and Adoption
By
Dharshana WarusavitharanaSenior Software Engineer, Test Automation
![Page 2: Wso2 test automation framework internal training](https://reader034.fdocuments.in/reader034/viewer/2022042814/54c5e66a4a7959905e8b4578/html5/thumbnails/2.jpg)
What will be Automated at WSO2● Platform scenarios.
● Integration scenarios.
● UI scenarios.
● Real world use cases.
● Performance and Security scenarios.
● Patches provided.
![Page 3: Wso2 test automation framework internal training](https://reader034.fdocuments.in/reader034/viewer/2022042814/54c5e66a4a7959905e8b4578/html5/thumbnails/3.jpg)
Basic Understanding - Unit Test ● Focused on testing behavior of a
particular class, module or method.
● Smallest part of the application.
● Fast (less than 0.1 second per test)
● No external implementation
dependencies
(filesystem, database, web services, etc.). All dependencies are
faked for the test context.
● Can easily be parallelized, since each test is atomic
![Page 4: Wso2 test automation framework internal training](https://reader034.fdocuments.in/reader034/viewer/2022042814/54c5e66a4a7959905e8b4578/html5/thumbnails/4.jpg)
Integration Test● Focused on verifying the integration
of one or more components together.
● May have external dependencies.
● A test that takes longer than a unit
Test should
Integration Tests - with in WSO2 ● Executes on product pack created by build.
● Not depending on other product environments.
● Independent.
● Lightweight.
● Covers only scenarios related to features on individual
product.
![Page 5: Wso2 test automation framework internal training](https://reader034.fdocuments.in/reader034/viewer/2022042814/54c5e66a4a7959905e8b4578/html5/thumbnails/5.jpg)
Overview of Automation Framework
Test Automation Framework is to make automation
● Eazy
● Organized
● Relevant
● Optimized
![Page 6: Wso2 test automation framework internal training](https://reader034.fdocuments.in/reader034/viewer/2022042814/54c5e66a4a7959905e8b4578/html5/thumbnails/6.jpg)
Technology Outline
Test Framework Build Systems
Code Coverage
Tooling SupportExecution Environment
![Page 7: Wso2 test automation framework internal training](https://reader034.fdocuments.in/reader034/viewer/2022042814/54c5e66a4a7959905e8b4578/html5/thumbnails/7.jpg)
Flexibilities Provided By Automation Framework
● Manage several products inside single test.
● Manage users in several product domains.
● Deployment of artifacts for different products under different
users.
● Automatic authentication.
● Configured admin service clients as test oriented API.
● Retrieving test environment configurations easily.
● Running same test in both local and Stratos environments.
● Reporting TestNg and surefire reports for all your tests.
● Tooling support of Selenium and Jmeter.
![Page 8: Wso2 test automation framework internal training](https://reader034.fdocuments.in/reader034/viewer/2022042814/54c5e66a4a7959905e8b4578/html5/thumbnails/8.jpg)
Architecture Overview
● Depends on TestNg Listeners.
● Platform wide single framework
● Context providing interfaces.
● Admin Service based Automation API.
● Selenium page object repository and element mapper.
● Automation framework Utils.
● Coverage and Reporting.
![Page 9: Wso2 test automation framework internal training](https://reader034.fdocuments.in/reader034/viewer/2022042814/54c5e66a4a7959905e8b4578/html5/thumbnails/9.jpg)
Why TestNG
● Link JUnit but not JUnit
● Annotation based Architecture
● Test Grouping
● Extensibility features
○ Listeners
○ Bean Shell
○ Annotation Transformers
● Easiness of managing Test cases.
● Failed test execution (Straight and easy)
![Page 10: Wso2 test automation framework internal training](https://reader034.fdocuments.in/reader034/viewer/2022042814/54c5e66a4a7959905e8b4578/html5/thumbnails/10.jpg)
Automation Framework Overall Architecture
user.csv / tenant.csv
automation.properties
instrument.txt
Result
Automation Framework CoreTest
Dashboard
MavenTest Managers
Test Suite
Surefire
TestNg Automation Framework API
Automation Framework Utills
User Populator
Coverage(emma
Server manager
Custom Reporter
Context Provider
![Page 11: Wso2 test automation framework internal training](https://reader034.fdocuments.in/reader034/viewer/2022042814/54c5e66a4a7959905e8b4578/html5/thumbnails/11.jpg)
Framework Core
Core
PlatformExecutionManager
PlatformSuiteManager
PlatformTestManager
PlatformAnnotationTransferManager
PlatformInvokedMethodManager
PlatformReportManager
Environment Context
Framework Annotations
Artifact Management
User Management
Server ManagementCoverage
Custom Error Reporting
![Page 12: Wso2 test automation framework internal training](https://reader034.fdocuments.in/reader034/viewer/2022042814/54c5e66a4a7959905e8b4578/html5/thumbnails/12.jpg)
Execution Template - TestNg
Execution Started@BeforeSuite
@AfterSuite
Execution Closed
@BeforeTest
@AfterTest
@BeforeClass
@BeforeGroups
@BeforeMethod
@AfterMethod
@AfterGroups
@AfterClass
Reference:- http://www.ibm.com/developerworks/java/library/j-testng/
![Page 13: Wso2 test automation framework internal training](https://reader034.fdocuments.in/reader034/viewer/2022042814/54c5e66a4a7959905e8b4578/html5/thumbnails/13.jpg)
TestNg Listeners● Listeners are interfaces allows to modify TestNG's
behavior.
● Listeners are binned to a testNg execution.
● Listeners implement the interface org.testng.ITestListener○ IExecutionListener
■ Triggers at start and end of any execution.■ ISuiteListener.
○ ITestListner■ Triggers at suite start and end.■ Triggers at Test start, Finish, Failure, Skip and partial failure.
● Reporters implement the interface org.testng.IReporter ○ Notified when all the suites have been run by TestNG. ○ The IReporter instance receives a summary of entire test run
![Page 14: Wso2 test automation framework internal training](https://reader034.fdocuments.in/reader034/viewer/2022042814/54c5e66a4a7959905e8b4578/html5/thumbnails/14.jpg)
Usage of Listeners
● PlatformExecutionManager (IExecutionListener)○ onExecutionStart()
■ Emma instrumentations
■ Server start
○ onExecutionFinish()
■ Server Shutdown
■ Emma report generation
● PlatformSuiteManager (ISuiteListener)○ Set environment properties ex- Key Store Paths
○ Populate Users.
![Page 15: Wso2 test automation framework internal training](https://reader034.fdocuments.in/reader034/viewer/2022042814/54c5e66a4a7959905e8b4578/html5/thumbnails/15.jpg)
Usage of Listeners contd..
● PlatformReportManager (IReporter)○ Generate TestNg Report
○ Generate surefire report
○ Export data for Dashboard
● PlatformInvokedMethodManager (InvokedMethodListener)○ Artifact deployment in platform scenarios.
● PlatformAnnotationTransferManager (IAnnotationTransformer)○ Annotation-Platform-user type based test selection.
![Page 16: Wso2 test automation framework internal training](https://reader034.fdocuments.in/reader034/viewer/2022042814/54c5e66a4a7959905e8b4578/html5/thumbnails/16.jpg)
Context providing interfaces
● Execution Based context - automation.properties
● Platform Based Context - automation.properties
● User Based Context - user.csv / tenant.csv
● Coverage Based Context - instrumentation.txt
![Page 17: Wso2 test automation framework internal training](https://reader034.fdocuments.in/reader034/viewer/2022042814/54c5e66a4a7959905e8b4578/html5/thumbnails/17.jpg)
Automation Properties
stratos.test=false#execution.environment = integration|platform|stratosexecution.environment=integration#execution.mode = user|tenant|allexecution.mode=userport.enable=truecarbon.web.context.enable=falseservice.deployment.delay=30000cluster.enable=trueproduct.list=ESBbuilder.enable=truecoverage.enable=true
● To configure Test framework
for the environment and
configure package details
● Contains all information about The environment.
![Page 18: Wso2 test automation framework internal training](https://reader034.fdocuments.in/reader034/viewer/2022042814/54c5e66a4a7959905e8b4578/html5/thumbnails/18.jpg)
Server Management
● Needs to configure the server and start the server with in the
test case.
● Most of the cases it might be ○ Single serve with single instance
○ Single server with multiple instance
○ Multiple product servers in a clustered environment.
● In this module framework is managing○ Extract and configure the server.
○ Add offsets and deploy custom modules needs for the test.
![Page 19: Wso2 test automation framework internal training](https://reader034.fdocuments.in/reader034/viewer/2022042814/54c5e66a4a7959905e8b4578/html5/thumbnails/19.jpg)
Test Execution Modes● Handled by a custom annotation introduced at test class and method level
●
●
●
● Annotations are based on the execution environment and the user type of the test
● Tests are skipped at the "PlatformAnnotationTransferManager"
● integration_all, integration_user, integration_tenant, platform_all, platform_user,
platform_tenant, stratos, all
@SetEnvironment(executionEnvironments = {ExecutionEnvironment.integration_all})
public void createGroup() throws Exception
{ }
#execution.environment = integration|platform|stratosexecution.environment=integration#execution.mode = user|tenant|allexecution.mode=user
![Page 20: Wso2 test automation framework internal training](https://reader034.fdocuments.in/reader034/viewer/2022042814/54c5e66a4a7959905e8b4578/html5/thumbnails/20.jpg)
Automation Framework API
● Test Friendly api wrapping service stubs.
○ Depends on service stubs.
○ Provides unified approach for perform verify and assert
of each admin service.
● Encapsulates the complexity of changing all available tests
in a case of a stub change.
● Maintain Page object classes for Selenium Automation.
● Using UI maps to store locators required for test scripts.
● Updates with the relevant release version.
![Page 21: Wso2 test automation framework internal training](https://reader034.fdocuments.in/reader034/viewer/2022042814/54c5e66a4a7959905e8b4578/html5/thumbnails/21.jpg)
Automation Framework Utils● Includes all utility classes that can play supportive role inside a
test.
○ Ex :-
■ Axis2 Client.
■ Wire message monitor.
■ Custom server startup scenarios (Axis2, Tomcat,Qpid).
■ Concurrency test scenarios.
● Provides common methodology for all development teams to
maintain a set of supportive classes without changing the Core.
● Reusability of utility methods in integration and platform tests.
![Page 22: Wso2 test automation framework internal training](https://reader034.fdocuments.in/reader034/viewer/2022042814/54c5e66a4a7959905e8b4578/html5/thumbnails/22.jpg)
Reporting
● Includes TestNg reports ,
SureFire report, And Emma
Coverage report
● Directories containing separate
results for each suite ( Directory
name is as same as the suite
name).
● Surefire reports.
● index.html (Dashboard for
executed test).
|-- reports | |-- BPSStructuredSuite | | |-- classes.html | | |-- FlowClient.html | | |-- FlowClient.properties | | |-- FlowClient.xml | | |-- ForEachClient.html | | |-- ForEachClient.properties | | |-- ForEachClient.xml | |-- index.html | |-- junitreports | | |-- TEST-org.wso2.automation.common.test.bps.mgts.BpelStructAct_FlowClient.xml | | |-- TEST-org.wso2.automation.common.test.bps.mgtst.BpelStructAct_forEachClient.xml
![Page 23: Wso2 test automation framework internal training](https://reader034.fdocuments.in/reader034/viewer/2022042814/54c5e66a4a7959905e8b4578/html5/thumbnails/23.jpg)
Platform Automated Test Suite● Distribution contain product integration and platform test jars.
● ANT based test jar executor - using TestNg ant task
● Ability to execute tests on different environments.
● Provide options to run test suites, packages and individual
classes.
● TestNg reports.
● Generate mail with test results.
![Page 24: Wso2 test automation framework internal training](https://reader034.fdocuments.in/reader034/viewer/2022042814/54c5e66a4a7959905e8b4578/html5/thumbnails/24.jpg)
Project Structure of Automated Tests
![Page 25: Wso2 test automation framework internal training](https://reader034.fdocuments.in/reader034/viewer/2022042814/54c5e66a4a7959905e8b4578/html5/thumbnails/25.jpg)
Bit on Automation (Tips and Tricks)
Referance - “Pragmatic Unit Testing in Java with JUnit” -Andrew Hunt and David Thomas
● Tip#1: One Assertion Per Test
● Tip #2: Write a Test, Then Fix it
● Tip #3: Keep Tests Independent
● Tip #4: Use Good Design in Code & Tests
● Tip #5: Test for Basic Correctness
● Tip #6: You have made a mess So clean it.
![Page 26: Wso2 test automation framework internal training](https://reader034.fdocuments.in/reader034/viewer/2022042814/54c5e66a4a7959905e8b4578/html5/thumbnails/26.jpg)
Questions?