Automated tests for inter-application data flows - iT … · Automated tests for inter-application...
Transcript of Automated tests for inter-application data flows - iT … · Automated tests for inter-application...
![Page 1: Automated tests for inter-application data flows - iT … · Automated tests for inter-application data flows September 2017 . Overview 2 1 Problems and requirements 2 Solution ...](https://reader031.fdocuments.in/reader031/viewer/2022022515/5afe36927f8b9a814d8eaa6c/html5/thumbnails/1.jpg)
© CGI Group Inc.
iT-Toolbox Testing Module Automated tests for inter-application data flows
September 2017
![Page 2: Automated tests for inter-application data flows - iT … · Automated tests for inter-application data flows September 2017 . Overview 2 1 Problems and requirements 2 Solution ...](https://reader031.fdocuments.in/reader031/viewer/2022022515/5afe36927f8b9a814d8eaa6c/html5/thumbnails/2.jpg)
Overview
2
Problems and requirements 1
Solution 2
Concepts 3
Usage 4
Roadmap 5
![Page 3: Automated tests for inter-application data flows - iT … · Automated tests for inter-application data flows September 2017 . Overview 2 1 Problems and requirements 2 Solution ...](https://reader031.fdocuments.in/reader031/viewer/2022022515/5afe36927f8b9a814d8eaa6c/html5/thumbnails/3.jpg)
Problems and requirements
What test-related difficulties are encountered on exchange platforms?
Flawless quality for critical data exchanges
• Major business impact in case of error (billing, logistics, …)
• Conformance / regression tests required before deployment
Complexity when testing exchanges
• Manual execution is slow, risky, costly
• Deep understanding of the platform and flows is required
Multiple and heterogenous technologies
• Transport and data formats, ESB, application-specific, …
• Dependency with an ever-changing application environment
Lack of tools dedicated to data flow test automation
3
![Page 4: Automated tests for inter-application data flows - iT … · Automated tests for inter-application data flows September 2017 . Overview 2 1 Problems and requirements 2 Solution ...](https://reader031.fdocuments.in/reader031/viewer/2022022515/5afe36927f8b9a814d8eaa6c/html5/thumbnails/4.jpg)
Problems and requirements
What are the main tooling requirements?
Ensuring the absence of regression for all kinds of data flows
• During technical migrations (e.g. new exchange platform)
• During application upgrades impacting data flows
• During framework updates
• During canonical data format updates
Building a realistic test case repository
• Centralized storage for test cases (avoiding mail exchanges, local files, …)
• Keeping test cases up-to-date
Automating tests in a controlled environment
• Testing often, easily, quickly to increase quality and reactivity
• Independence from external systems when running exchange tests
• Reducing the cost of tests (configuration and execution)
4
![Page 5: Automated tests for inter-application data flows - iT … · Automated tests for inter-application data flows September 2017 . Overview 2 1 Problems and requirements 2 Solution ...](https://reader031.fdocuments.in/reader031/viewer/2022022515/5afe36927f8b9a814d8eaa6c/html5/thumbnails/5.jpg)
Solution
iT-Toolbox Testing Module: a tool to automate data flow tests
Black-box testing
• Injection of input data to launch the data flow
• Collection of actual data from the output
Regression checks
• Comparison of actual and expected data
• Support for data format specifics
• Precise problem identification (line, field name, difference, …)
• Report generation
Web interface to define and execute tests with a REST API
Based on popular open-source solutions (Maven, JUnit, …) and compatible with continuous integration servers (Jenkins, …)
5
![Page 6: Automated tests for inter-application data flows - iT … · Automated tests for inter-application data flows September 2017 . Overview 2 1 Problems and requirements 2 Solution ...](https://reader031.fdocuments.in/reader031/viewer/2022022515/5afe36927f8b9a814d8eaa6c/html5/thumbnails/6.jpg)
Concepts – Overview
Hierarchical test repository
• Test project: container for test campaigns (e.g. business process)
• Test campaign: container for test suites (e.g. data exchange)
• Test suite: container for test cases (e.g. flow or half-flow)
• Test case: comparison of actual/expected data (e.g. nominal, errors)
Connectors concept
• Launch flows by injecting data and collecting output
• Many transports supported (FTP, HTTP, SOAP, JMS, JDBC, etc.)
Formats concept
• Define how actual/expected data comparison is executed
• Individual fields can be named or ignored during comparison
• Many formats supported (delimited/fixed flat file, XML, JSON, etc.)
6
![Page 7: Automated tests for inter-application data flows - iT … · Automated tests for inter-application data flows September 2017 . Overview 2 1 Problems and requirements 2 Solution ...](https://reader031.fdocuments.in/reader031/viewer/2022022515/5afe36927f8b9a814d8eaa6c/html5/thumbnails/7.jpg)
Concepts – Hierarchical repository
3 levels of configuration: project, campaign and test suite
• Modular configuration of connectors / formats
• Parameters can be inherited for increased reuse
• Parameters can reference properties (key/values)
7
Project Campaign
Test suite
Connectors / Formats / Properties
Connectors / Formats / Properties
Connectors / Formats / Properties
Test case Test case Test case
![Page 8: Automated tests for inter-application data flows - iT … · Automated tests for inter-application data flows September 2017 . Overview 2 1 Problems and requirements 2 Solution ...](https://reader031.fdocuments.in/reader031/viewer/2022022515/5afe36927f8b9a814d8eaa6c/html5/thumbnails/8.jpg)
Concepts – Test cases
Test case execution steps:
1. Setting up the environment
2. Reading input data
3. Launching the flow
4. Collecting output data
5. Comparing actual / expected data
6. Generating difference report
7. Cleaning up the environment
Test types:
• Comparison (default)
• Error (message / exception)
• Performance (time limit)
8
Test case
Pre-action
Actual data Expected data
Format
Input data
Post-action
WS
wM
JMS
FTP
BDD Connectors
Differences
![Page 9: Automated tests for inter-application data flows - iT … · Automated tests for inter-application data flows September 2017 . Overview 2 1 Problems and requirements 2 Solution ...](https://reader031.fdocuments.in/reader031/viewer/2022022515/5afe36927f8b9a814d8eaa6c/html5/thumbnails/9.jpg)
Usage – Overview
9
Definition
• Configuration of connectors and formats
• Creation of test campaigns, suites and cases
Execution
• Criteria-based definition of the test perimeter
• Launching tests
Reports
• Results analysis
• Relevant errors
• History of previous results
![Page 10: Automated tests for inter-application data flows - iT … · Automated tests for inter-application data flows September 2017 . Overview 2 1 Problems and requirements 2 Solution ...](https://reader031.fdocuments.in/reader031/viewer/2022022515/5afe36927f8b9a814d8eaa6c/html5/thumbnails/10.jpg)
Usage – Definition
Web interface for test definition
• Project / campaign / suite / test case hierarchy
• Properties / connectors / formats configuration
• Test data management (upload / view / download)
10
![Page 11: Automated tests for inter-application data flows - iT … · Automated tests for inter-application data flows September 2017 . Overview 2 1 Problems and requirements 2 Solution ...](https://reader031.fdocuments.in/reader031/viewer/2022022515/5afe36927f8b9a814d8eaa6c/html5/thumbnails/11.jpg)
Usage – Execution
Execution from the test repository web interface
• Selection of campaigns / suites / tests by criteria
• On-demand execution without leaving the interface
• Reports generation and history
Execution by a continuous integration server (e.g. Jenkins)
• Project downloaded and executed by CI server
• Reports generation and history
• On-demand, scheduled or triggered execution
• Notification (e.g. by mail) when a test failure occurs
Execution in standalone mode
• For use by developers / testers
• Local execution by script
• Reports generation
• Native integration with IDE such as Eclipse
11
![Page 12: Automated tests for inter-application data flows - iT … · Automated tests for inter-application data flows September 2017 . Overview 2 1 Problems and requirements 2 Solution ...](https://reader031.fdocuments.in/reader031/viewer/2022022515/5afe36927f8b9a814d8eaa6c/html5/thumbnails/12.jpg)
Usage – Reports
Identification of relevant errors
• Name and path of the difference
• Detection of moved blocks between actual / expected data
• Option to ignore extra data or specific fields
12
![Page 13: Automated tests for inter-application data flows - iT … · Automated tests for inter-application data flows September 2017 . Overview 2 1 Problems and requirements 2 Solution ...](https://reader031.fdocuments.in/reader031/viewer/2022022515/5afe36927f8b9a814d8eaa6c/html5/thumbnails/13.jpg)
Roadmap
Development of new features
• Productivity and user-friendliness improvements
• Fine-grained permission management in the web interface
• Easy search and browsing of test case data
• Deeper integration with Jenkins CI server
• New connectors and comparators according to user needs
Integration of a capture / mock module
• Creation of tests cases from captures
• Activation of mocks before executing tests
• webMethods-specific plugin in the works
13
![Page 14: Automated tests for inter-application data flows - iT … · Automated tests for inter-application data flows September 2017 . Overview 2 1 Problems and requirements 2 Solution ...](https://reader031.fdocuments.in/reader031/viewer/2022022515/5afe36927f8b9a814d8eaa6c/html5/thumbnails/14.jpg)
Roadmap
Discover more SOA solutions by CGI
• Web site: https://it-toolbox.fr
• Email: [email protected]
14
![Page 15: Automated tests for inter-application data flows - iT … · Automated tests for inter-application data flows September 2017 . Overview 2 1 Problems and requirements 2 Solution ...](https://reader031.fdocuments.in/reader031/viewer/2022022515/5afe36927f8b9a814d8eaa6c/html5/thumbnails/15.jpg)
Our platforms
15
![Page 16: Automated tests for inter-application data flows - iT … · Automated tests for inter-application data flows September 2017 . Overview 2 1 Problems and requirements 2 Solution ...](https://reader031.fdocuments.in/reader031/viewer/2022022515/5afe36927f8b9a814d8eaa6c/html5/thumbnails/16.jpg)
Thank you for your attention
![Page 17: Automated tests for inter-application data flows - iT … · Automated tests for inter-application data flows September 2017 . Overview 2 1 Problems and requirements 2 Solution ...](https://reader031.fdocuments.in/reader031/viewer/2022022515/5afe36927f8b9a814d8eaa6c/html5/thumbnails/17.jpg)
Our commitment to you We approach every engagement with one
objective in mind: to help clients succeed