Scheduling system for test automation frameworkScheduling system for test automation framework...
Transcript of Scheduling system for test automation frameworkScheduling system for test automation framework...
![Page 1: Scheduling system for test automation frameworkScheduling system for test automation framework Citation for published version (APA): Wahyudi, D., & Technische Universiteit Eindhoven](https://reader033.fdocuments.in/reader033/viewer/2022051923/6011324fdb00301ad8544bee/html5/thumbnails/1.jpg)
Scheduling system for test automation framework
Citation for published version (APA):Wahyudi, D., & Technische Universiteit Eindhoven (TUE). Stan Ackermans Instituut. Software Technology (ST)(2014). Scheduling system for test automation framework. Technische Universiteit Eindhoven.
Document status and date:Published: 01/10/2014
Document Version:Publisher’s PDF, also known as Version of Record (includes final page, issue and volume numbers)
Please check the document version of this publication:
• A submitted manuscript is the version of the article upon submission and before peer-review. There can beimportant differences between the submitted version and the official published version of record. Peopleinterested in the research are advised to contact the author for the final version of the publication, or visit theDOI to the publisher's website.• The final author version and the galley proof are versions of the publication after peer review.• The final published version features the final layout of the paper including the volume, issue and pagenumbers.Link to publication
General rightsCopyright and moral rights for the publications made accessible in the public portal are retained by the authors and/or other copyright ownersand it is a condition of accessing publications that users recognise and abide by the legal requirements associated with these rights.
• Users may download and print one copy of any publication from the public portal for the purpose of private study or research. • You may not further distribute the material or use it for any profit-making activity or commercial gain • You may freely distribute the URL identifying the publication in the public portal.
If the publication is distributed under the terms of Article 25fa of the Dutch Copyright Act, indicated by the “Taverne” license above, pleasefollow below link for the End User Agreement:www.tue.nl/taverne
Take down policyIf you believe that this document breaches copyright please contact us at:[email protected] details and we will investigate your claim.
Download date: 27. Jan. 2021
![Page 2: Scheduling system for test automation frameworkScheduling system for test automation framework Citation for published version (APA): Wahyudi, D., & Technische Universiteit Eindhoven](https://reader033.fdocuments.in/reader033/viewer/2022051923/6011324fdb00301ad8544bee/html5/thumbnails/2.jpg)
Scheduling System for
Test Automation Framework
Djohan Wahyudi September 2014
![Page 3: Scheduling system for test automation frameworkScheduling system for test automation framework Citation for published version (APA): Wahyudi, D., & Technische Universiteit Eindhoven](https://reader033.fdocuments.in/reader033/viewer/2022051923/6011324fdb00301ad8544bee/html5/thumbnails/3.jpg)
Scheduling System for Test Automation Framework
Djohan Wahyudi September 2014
![Page 4: Scheduling system for test automation frameworkScheduling system for test automation framework Citation for published version (APA): Wahyudi, D., & Technische Universiteit Eindhoven](https://reader033.fdocuments.in/reader033/viewer/2022051923/6011324fdb00301ad8544bee/html5/thumbnails/4.jpg)
![Page 5: Scheduling system for test automation frameworkScheduling system for test automation framework Citation for published version (APA): Wahyudi, D., & Technische Universiteit Eindhoven](https://reader033.fdocuments.in/reader033/viewer/2022051923/6011324fdb00301ad8544bee/html5/thumbnails/5.jpg)
Scheduling System for Test Automation Framework
Eindhoven University of Technology Stan Ackermans Institute / Software Technology
Partners
Philips Healthcare Eindhoven University of Technology Steering Group Alex Visser
Johan Lukkien Wil van den Berk
Date September 2014
![Page 6: Scheduling system for test automation frameworkScheduling system for test automation framework Citation for published version (APA): Wahyudi, D., & Technische Universiteit Eindhoven](https://reader033.fdocuments.in/reader033/viewer/2022051923/6011324fdb00301ad8544bee/html5/thumbnails/6.jpg)
Contact Address
Eindhoven University of Technology Department of Mathematics and Computer Science MF 7.090, P.O. Box 513, NL-5600 MB, Eindhoven, The Netherlands +31402474334
Published by Eindhoven University of Technology
Stan Ackermans Institute Printed by Eindhoven University of Technology
UniversiteitsDrukkerij ISBN 978‐90‐444‐1313‐7
Abstract These comments should be the abstract issued for the ISBN-request. Keywords
These keywords should be at least the keywords as issued for the ISBN-request
Preferred reference
Djohan Wahyudi, Automatic System Preconditioning and Testing Environment. Eindhoven University of Technology, SAI Technical Report, September, 2014. (978‐90‐444‐1313‐7)
Partnership This project was supported by Eindhoven University of Technology and Philips Healthcare. Disclaimer Endorsement
Reference herein to any specific commercial products, process, or service by trade name, trademark, manufacturer, or otherwise, does not necessarily constitute or imply its endorse-ment, recommendation, or favoring by the Eindhoven University of Technology or Philips Healthcare. The views and opinions of authors expressed herein do not necessarily state or reflect those of the Eindhoven University of Technology or Philips Healthcare, and shall not be used for advertising or product endorsement purposes.
Disclaimer Liability
While every effort will be made to ensure that the information contained within this report is accurate and up to date, Eindhoven University of Technology makes no warranty, represen-tation or undertaking whether expressed or implied, nor does it assume any legal liability, whether direct or indirect, or responsibility for the accuracy, completeness, or usefulness of any information.
Trademarks Product and company names mentioned herein may be trademarks and/or service marks of
their respective owners. We use these names without any particular endorsement or with the intent to infringe the copyright of the respective owners.
Copyright Copyright © 2014. Eindhoven University of Technology. All rights reserved. No part of the material protected by this copyright notice may be reproduced, modified, or
redistributed in any form or by any means, electronic or mechanical, including photocopy-ing, recording, or by any information storage or retrieval system, without the prior written permission of the Eindhoven University of Technology and Philips Healthcare.
![Page 7: Scheduling system for test automation frameworkScheduling system for test automation framework Citation for published version (APA): Wahyudi, D., & Technische Universiteit Eindhoven](https://reader033.fdocuments.in/reader033/viewer/2022051923/6011324fdb00301ad8544bee/html5/thumbnails/7.jpg)
Foreword Philips Healthcare produces equipment for treating various illnesses. The specialty of the business unit Interventional Xray (iXR) is equipment for treating cardiovascular diseases, such as visualizing the placement of a stent after a heart attack. To guaran-tee the safety of the patient and the physicians and nursing staff the equipment must be thoroughly tested. Because the equipment and the real estate (e.g. lead cages) needed for these tests are very expensive, Philips aims to use the equipment for test-ing purposes as efficiently as possible. In the past, we took steps to automate most of the testing. What remained was the manual and labor-intensive task of the installation and configuration of the system prior to executing the tests. This resulted in test systems being allocated to one pro-ject for weeks or months and required more test systems than strictly necessary. The automation of the installation and configuration gives iXR the opportunity to increase the breadth of testing, to decrease the time to market, and to decrease the number of test systems. The entire project entails the scheduling and reporting of installation tasks, and the installation and configuration of test environments. Djohan created the scheduling and reporting part of the project. He investigated the use of Common off the Shelf (COTS) tools to do the job, as well as existing tools that are used within Philips. He developed the missing pieces and integrated these with the existing tools. With his contribution, we are now able to compile a set of tasks that are required to install and configure a test environment and to run a series of tests at a given time. These tasks are then executed without further user interaction at the given time, 24 hours a day and seven days a week. Alex Visser September 2014
![Page 8: Scheduling system for test automation frameworkScheduling system for test automation framework Citation for published version (APA): Wahyudi, D., & Technische Universiteit Eindhoven](https://reader033.fdocuments.in/reader033/viewer/2022051923/6011324fdb00301ad8544bee/html5/thumbnails/8.jpg)
![Page 9: Scheduling system for test automation frameworkScheduling system for test automation framework Citation for published version (APA): Wahyudi, D., & Technische Universiteit Eindhoven](https://reader033.fdocuments.in/reader033/viewer/2022051923/6011324fdb00301ad8544bee/html5/thumbnails/9.jpg)
Preface This report is the final deliverable of my graduation project in the Software Tech-nology program. The project timeline was nine months and it was located in Philips Healthcare, Best. The project title is Scheduling System for Test Automa-tion Framework (SSTAF). The target audience for this report is a technical audience with basic knowledge of software design, software testing, and UML notation. The document covers the tools to be used for SSTAF Project. Furthermore, it covers the design and the implementation of the Scheduling System. The introduction of the project context is explained in Chapter 1. The stakeholder analysis is made in Chapter 2. The system requirements are defined in Chapter 3. The problem and the possible solutions are analyzed in Chapter 4. Feasibility analysis is described in Chapter 5. Design work is explained in Chapter 6 and 7. The implementation of the system is explained in Chapter 8. Those interested in the verification and validation should read Chapter 9. The project management is described in Chapter 10. The results and conclusions are discussed in Chapter 11. Djohan Wahyudi September 2014
![Page 10: Scheduling system for test automation frameworkScheduling system for test automation framework Citation for published version (APA): Wahyudi, D., & Technische Universiteit Eindhoven](https://reader033.fdocuments.in/reader033/viewer/2022051923/6011324fdb00301ad8544bee/html5/thumbnails/10.jpg)
![Page 11: Scheduling system for test automation frameworkScheduling system for test automation framework Citation for published version (APA): Wahyudi, D., & Technische Universiteit Eindhoven](https://reader033.fdocuments.in/reader033/viewer/2022051923/6011324fdb00301ad8544bee/html5/thumbnails/11.jpg)
Acknowledgements I would like to express a great appreciation to my company supervisor, Alex Visser for his continuous support, guidance and feedback throughout the project. His experience and knowledge has provided me with valuable feedback for my project and my development. I am grateful to my university supervisor, Johan Lukkien for his valuable feedback that kept pushing me hard for my improvement and development. I would like to thank my customer in Philips that has provided me with help and good cooperation during the project: Wil van den Berk, Marcel van Veggel, Luuk van der Veer, Saco Meeuwig, and Arjan van Osch. I would like to thank Martijn Drabbels and Marc Urlings for their technical guidance and help during the pro-ject implementation. Many thanks I give to Henk Meulendijks, Peer Bruin and Melina Bonin for their view and knowledge about Continuous Integration tools. I would like to give thanks also to Thomas de Laet, Ben van Mierloo, Remco Hijmans, and Anja van der Bolt-Jacobs for their view and knowledge in Test En-vironment tool. I would like to thank Judith B. Strother and Darko Jovanovic for their help to revise my document. I would like to extend my thanks to several OOTI alumni in Philips that help me with discussion and guidance: Paul Robert Marcu, Zlatka Manevska, Athanasios Papakostopoulos, Ivana Kostadinovska, Marco Verstege, Chetan Nair, and Marcin Grodek. I would like to give thanks to the program director of PDEng Software Technolo-gy, Ad Aerts, for his valuable support and help during my entire Software Tech-nology program. I would like to thank all the coaches for their lectures that help me to improve myself during last two years of my studies. Special thanks to the management assistant, Maggy de Wert for her great support and care with so much love and enthusiasm. I would like to thank all my colleagues for their feed-back and support throughout the program. I will not forget my great experience working with them in the OOTI program. Finally, I would like to offer my thanks to my family, especially to my parents, my brother and sister, and my friends for their love, support, and encouragement. Djohan Wahyudi September 2014
![Page 12: Scheduling system for test automation frameworkScheduling system for test automation framework Citation for published version (APA): Wahyudi, D., & Technische Universiteit Eindhoven](https://reader033.fdocuments.in/reader033/viewer/2022051923/6011324fdb00301ad8544bee/html5/thumbnails/12.jpg)
![Page 13: Scheduling system for test automation frameworkScheduling system for test automation framework Citation for published version (APA): Wahyudi, D., & Technische Universiteit Eindhoven](https://reader033.fdocuments.in/reader033/viewer/2022051923/6011324fdb00301ad8544bee/html5/thumbnails/13.jpg)
Executive Summary An Interventional X-ray (iXR) system provides real time X-ray imaging with high image clarity and low X-ray dose. After several years of development, the iXR System has become complex. In order to verify the correct operation of the system, the system integration and test group performs extensive testing on a va-riety of test environments. The preparation of these test environments is a time-consuming and labor-intensive activity. The system integration and test group wants to automate this activity. This project is created and defined to solve this problem. The project title is “Scheduler System for Test Automation Framework”. The project goal is to automate the System Integration and Test in Allura Xper Sys-tem. The project aims to provide a scheduler system that will run the precondi-tioning activities (installation, configuration, customization, calibration) and test activities automatically. The common off the shelf scheduler and the related Philips tools are explored in the start of the project. The scheduler system consists of a custom scheduler tool and several related Philips tools. The scheduler system has the scheduler website as the user interface. Therefore, it provides an easy user access. The scheduler system integrates with TAF, an existing Test Automation tool, for the execution of preconditioning and test activities. The scheduler system pro-vides interfaces to TAF execution. After the execution, the results are collected automatically. The complete system is tested and verified. The architecture, de-sign, and implementation of this project can be found in this document.
![Page 14: Scheduling system for test automation frameworkScheduling system for test automation framework Citation for published version (APA): Wahyudi, D., & Technische Universiteit Eindhoven](https://reader033.fdocuments.in/reader033/viewer/2022051923/6011324fdb00301ad8544bee/html5/thumbnails/14.jpg)
![Page 15: Scheduling system for test automation frameworkScheduling system for test automation framework Citation for published version (APA): Wahyudi, D., & Technische Universiteit Eindhoven](https://reader033.fdocuments.in/reader033/viewer/2022051923/6011324fdb00301ad8544bee/html5/thumbnails/15.jpg)
Table of Contents
Foreword .................................................................................................... i
Preface ...................................................................................................... iii
Acknowledgements ................................................................................... v
Executive Summary ................................................................................ vii
Table of Contents ..................................................................................... ix
List of Figures ........................................................................................ xiii
List of Tables ........................................................................................... xv
1. Introduction ....................................................................................... 1
1.1 Context ......................................................................................... 1
1.2 Current Testing Environment ....................................................... 2 1.2.1. System under Test (SUT) .......................................................... 2 1.2.2. Current Test Setup ..................................................................... 2 1.2.3. SUT Related Activities .............................................................. 3 1.2.4. SUT Usage ................................................................................ 3
1.3 Problem Description .................................................................... 4
1.4 Outline .......................................................................................... 4
2. Stakeholder Analysis ......................................................................... 7
2.1 Introduction .................................................................................. 7
2.2 Stakeholders’ Concerns ............................................................... 7
2.3 Stakeholders’ Engagement Actions .............................................. 7
2.4 Customer’ Interview and Meeting ................................................ 8
3. System Requirements ........................................................................ 9
3.1 Main Requirements ...................................................................... 9
3.2 Requirements ................................................................................ 9 3.2.1. Activity ...................................................................................... 9 3.2.2. Activity Block ......................................................................... 10 3.2.3. Schedule .................................................................................. 10 3.2.4. Side PC .................................................................................... 10 3.2.5. Reporting ................................................................................. 11
3.3 Requirement Priority .................................................................. 11
4. Problem Analysis ............................................................................. 13
4.1 Problem Examination ................................................................. 13
4.2 Domain Model ............................................................................ 13
4.3 Mapping of COTS Tools to Domain Model ............................... 14
![Page 16: Scheduling system for test automation frameworkScheduling system for test automation framework Citation for published version (APA): Wahyudi, D., & Technische Universiteit Eindhoven](https://reader033.fdocuments.in/reader033/viewer/2022051923/6011324fdb00301ad8544bee/html5/thumbnails/16.jpg)
4.4 Mapping of Related Philips tools to Domain Model .................. 15
4.5 Problem Analysis Result ............................................................. 16
5. Feasibility Analysis .......................................................................... 19
5.1 Test Environment (TE) Project ................................................... 19
5.2 Test Automation Framework (TAF) ........................................... 19
5.3 Step Manager ............................................................................. 19
5.4 QlikView ..................................................................................... 19
5.5 Windows Task Scheduler (WTS) ................................................. 19
5.6 Feasibility Analysis Result ......................................................... 20
6. System Architecture ........................................................................ 21
6.1 Context Diagram ........................................................................ 21
6.2 Use Case ..................................................................................... 21
6.3 Preliminary System Architecture ................................................ 22 6.3.1. Relationship between Schedule, Activity Block and Activity . 22 6.3.2. Preliminary Class Diagram ...................................................... 22 6.3.3. Preliminary Activity Diagram ................................................. 23 6.3.4. Preliminary Deployment Diagram ........................................... 24
6.4 Full System Architecture ............................................................ 24 6.4.1. Logical View ........................................................................... 24 6.4.2. Process View ........................................................................... 27 6.4.3. Deployment View .................................................................... 28
7. System Design .................................................................................. 31
7.1 Main Design Decisions .............................................................. 31 7.1.1. Fat or Thin Client Architecture ................................................ 31 7.1.2. Smart Server or Smart Agent ................................................... 31 7.1.3. Interface between Scheduler System Components .................. 32 7.1.4. Database or XML File ............................................................. 33
7.2 Components Design .................................................................... 33 7.2.1. Mapping of Class Diagram to Deployment Diagram .............. 33 7.2.2. Detail Sequence Diagram ........................................................ 35 7.2.3. State Machine Diagram ........................................................... 36 7.2.4. Components Design Summary ................................................ 37
8. Implementation ................................................................................ 39
8.1 Data Components ....................................................................... 39 8.1.1. Database ................................................................................... 39 8.1.2. XML Storage ........................................................................... 40
8.2 Tool Components ........................................................................ 41 8.2.1. Scheduler Server ...................................................................... 41 8.2.2. Scheduler Agent ...................................................................... 44 8.2.3. Step Manager ........................................................................... 46
9. Verification & Validation ............................................................... 49
9.1 Verification ................................................................................. 49
![Page 17: Scheduling system for test automation frameworkScheduling system for test automation framework Citation for published version (APA): Wahyudi, D., & Technische Universiteit Eindhoven](https://reader033.fdocuments.in/reader033/viewer/2022051923/6011324fdb00301ad8544bee/html5/thumbnails/17.jpg)
9.2 Validation ................................................................................... 49
10. Project Management .................................................................... 51
10.1 Way of working ....................................................................... 51
10.2 Project Schedule ..................................................................... 51
10.3 Risk Management ................................................................... 52
11. Conclusion .................................................................................... 53
11.1 Results ..................................................................................... 53
11.2 Lessons Learned ..................................................................... 53
11.3 Future Works .......................................................................... 54
Appendix A. COTS Selection ................................................................. 55
Job Scheduler ........................................................................................ 55
Continuous Integration Tool ................................................................. 56
Tool Comparison................................................................................... 58
Appendix B. Design Evolution ............................................................... 61
Initial Design......................................................................................... 61
Alternative Design with Server and Agent ............................................ 62
Alternative Design with TE (Test Environment) ................................... 63
Alternative Design with Web Service without TE ................................. 64
Appendix C. Scheduler System Swim Lane Diagram ......................... 65
Appendix D. XML Files ......................................................................... 67
SCHEDULER AGENT FILES ............................................................... 67
STEP MANAGER FILES ...................................................................... 68
TAF FILES ............................................................................................ 69
Glossary ................................................................................................... 71
Bibliography ............................................................................................ 73
References ............................................................................................. 73
About the Author .................................................................................... 75
![Page 18: Scheduling system for test automation frameworkScheduling system for test automation framework Citation for published version (APA): Wahyudi, D., & Technische Universiteit Eindhoven](https://reader033.fdocuments.in/reader033/viewer/2022051923/6011324fdb00301ad8544bee/html5/thumbnails/18.jpg)
![Page 19: Scheduling system for test automation frameworkScheduling system for test automation framework Citation for published version (APA): Wahyudi, D., & Technische Universiteit Eindhoven](https://reader033.fdocuments.in/reader033/viewer/2022051923/6011324fdb00301ad8544bee/html5/thumbnails/19.jpg)
List of Figures Figure 1 – Allura Xper System ....................................................................... 1 Figure 2 – Current Test Setup ......................................................................... 2 Figure 3 – SUT Related Activities ................................................................... 3 Figure 4 – Scheduler Gantt Chart .................................................................. 11 Figure 5 – Domain Model ............................................................................ 14 Figure 6 – Mapping of COTS Tools to Domain Model ................................... 14 Figure 7 – Mapping of Philips Tools to Domain Model ................................... 15 Figure 8 – Basic Plan for TE Extension ......................................................... 16 Figure 9 – Expected Test Setup ..................................................................... 17 Figure 10 – Windows Task Scheduler Prototype ............................................ 20 Figure 11 – Final Mapping of Philips Tools to Domain Model ........................ 20 Figure 12 – Context Diagram ....................................................................... 21 Figure 13 – Use Case Diagram ..................................................................... 21 Figure 14 – Relationship of Schedule, Activity Block and Activity .................. 22 Figure 15 – Preliminary Class Diagram ......................................................... 23 Figure 16 – Preliminary Activity Diagram ..................................................... 23 Figure 17 – Preliminary Deployment Diagram ............................................... 24 Figure 18 – Class Diagram ........................................................................... 25 Figure 19 – Sequence Diagram ..................................................................... 26 Figure 20 – Activity Diagram ....................................................................... 27 Figure 21 – Deployment Diagram ................................................................. 29 Figure 22 – Mapping Class Diagram to Deployment Diagram ......................... 34 Figure 23 – Data Input Sequence Diagram ..................................................... 35 Figure 24 – Scheduler Execution Sequence Diagram ...................................... 36 Figure 36 – Scheduler Agent State Diagram ................................................... 36 Figure 37 – Step Manager State Diagram ....................................................... 37 Figure 25 – Database Diagram ...................................................................... 39 Figure 26 – XML Storage Structure .............................................................. 40 Figure 27 – Model View Controller ............................................................... 41 Figure 28 – Scheduler Server User Interface .................................................. 42 Figure 29 – Scheduler Server FileWatcher ..................................................... 43 Figure 30 – Scheduler Agent User Interface ................................................... 44 Figure 31 – Scheduler Agent Time Trigger .................................................... 45 Figure 32 – Scheduler Agent File Watcher ..................................................... 46 Figure 33 – Step Manager Sequence Diagram ................................................ 47 Figure 34 – Project Gantt Chart .................................................................... 51 Figure 35 – Swim Lane Diagram .................................................................. 65
![Page 20: Scheduling system for test automation frameworkScheduling system for test automation framework Citation for published version (APA): Wahyudi, D., & Technische Universiteit Eindhoven](https://reader033.fdocuments.in/reader033/viewer/2022051923/6011324fdb00301ad8544bee/html5/thumbnails/20.jpg)
![Page 21: Scheduling system for test automation frameworkScheduling system for test automation framework Citation for published version (APA): Wahyudi, D., & Technische Universiteit Eindhoven](https://reader033.fdocuments.in/reader033/viewer/2022051923/6011324fdb00301ad8544bee/html5/thumbnails/21.jpg)
List of Tables Table 1 – Stakeholders ................................................................................... 7 Table 2 – Stakeholders’ Concern .................................................................... 7 Table 3 – Stakeholders’ Engagement Action .................................................... 7 Table 4 – Customers’ Wish List ...................................................................... 8 Table 5 – Requirement Priority ..................................................................... 11 Table 6 – Translation for Class Diagram ........................................................ 25 Table 7 – Class Diagram Element ................................................................. 26 Table 8 – Activity Diagram Element ............................................................. 28 Table 9 – Fat or Thin Client Architecture....................................................... 31 Table 10 – Smart Server or Smart Agent........................................................ 31 Table 11 – Interface/Communication ............................................................. 32 Table 12 – Database or XML File ................................................................. 33 Table 13 – Class Diagram to Deployment Diagram ........................................ 33 Table 30 – Step Manager State ..................................................................... 37 Table 31 – Step Manager State ..................................................................... 37 Table 14 – XML Storage Folder Structure ..................................................... 40 Table 15 – MVC Components ...................................................................... 42 Table 16 – Scheduler Command Result ......................................................... 43 Table 17 – Scheduler Command ................................................................... 45 Table 18 – Step Manager Command Parameters ............................................. 47 Table 19 – Step Manager File Interfaces ........................................................ 47 Table 20 – Design Status .............................................................................. 49 Table 21 – Requirement Status ..................................................................... 50 Table 22 – Issue and Risk ............................................................................. 52 Table 23 – Domain against functionality........................................................ 55 Table 24 – Job Scheduler First Round Selection ............................................. 55 Table 25 – Job Scheduler Second Round Selection ......................................... 56 Table 26 – Continuous Integration First Round Selection ................................ 56 Table 27 – Continuous Integration Second Round Selection ............................ 58 Table 28 – Tool Selection ............................................................................. 58 Table 29 – Swim Lane Element .................................................................... 65
![Page 22: Scheduling system for test automation frameworkScheduling system for test automation framework Citation for published version (APA): Wahyudi, D., & Technische Universiteit Eindhoven](https://reader033.fdocuments.in/reader033/viewer/2022051923/6011324fdb00301ad8544bee/html5/thumbnails/22.jpg)
![Page 23: Scheduling system for test automation frameworkScheduling system for test automation framework Citation for published version (APA): Wahyudi, D., & Technische Universiteit Eindhoven](https://reader033.fdocuments.in/reader033/viewer/2022051923/6011324fdb00301ad8544bee/html5/thumbnails/23.jpg)
1.Introduction Philips Healthcare delivers advanced solutions for healthcare professionals. Interventional X-ray (iXR) department is responsible for the development of X-ray systems used for diagnostics and interventional treatment of cardiac and vascular diseases. The X-ray system is called Allura Xper System. The development of this type of systems is performed in multidisciplinary teams where hardware, mechanics, and software come together. In order to verify the correct operation of the system, the system integration and test group performs extensive testing in a variety of test environments. The preparation of these test environments is a time-consuming and labor-intensive activity. The Sched-uler System for Test Automation Project is created to automate this activity.
1.1 Context An Interventional X-ray system provides real-time X-ray imaging with high image clarity and low X-ray dose. To provide these functionalities, the most advanced tech-nology is used. As a result, the product is very complex, with many different hard-ware configurations that need to be tested. The testing itself is equally complex and time-consuming. Furthermore, the installation of those hardware configurations is currently a manual task. This problem leads to the creation and the definition of this project. The customer of the project is the “System integration and test group” in iXR de-partment of Philips Healthcare. The “System integration and test group” has the task to integrate units into a complete system and to test the completed system. A repeating, time-consuming activity in this process is preconditioning of a System under Test (SUT). System under Test is an Allura Xper System used in the testing environment. The Allura Xper System is shown in Figure 1
Figure 1 – Allura Xper System Preconditioning of a SUT consists of PDC (Product Development Code) installation, system configuration, system customization, and system calibration. The purpose of
![Page 24: Scheduling system for test automation frameworkScheduling system for test automation framework Citation for published version (APA): Wahyudi, D., & Technische Universiteit Eindhoven](https://reader033.fdocuments.in/reader033/viewer/2022051923/6011324fdb00301ad8544bee/html5/thumbnails/24.jpg)
the Scheduling System for Test Automation Framework (SSTAF) is to automate these activities. These activities should be triggered at any moment, so that a SUT can be used 24 hours a day, 7 days a week. The SSTAF consists of the scheduler tool and interfaces to other tools (like TAF). The process of preconditioning with a new PDC and testing is repeated (during the project) until the right product quality is ob-tained, from the start of integration until the end of validation.
1.2 Current Testing Environment This subchapter describes the current testing environment and the usage of SUT. The current testing environment is evaluated to understand the problem that needs to be solved.
1.2.1. System under Test (SUT) System under test (SUT) is built at the Philips site for different purposes: develop-ment, functional test, integration test, and system verification and validation. SUTs come in different shapes and sizes to accommodate for specific needs and pur-poses. For software development, small systems with only computers are sufficient (small and cheap). For final system tests, entire systems including the mechatronics and X-ray are needed (big and expensive). SUTs are grouped by their type and configuration. They are then allocated to a pro-ject. The type of SUT that is required by a project depends on new features that are introduced by the project.
1.2.2. Current Test Setup The current test setup is shown in Figure 2:
Side PC is a PC that is located near the SUT and it is used to execute the test script
TAF is a test tool to run a test script. Test Environment (TE) is a tool to plan a manual test. Reporting (QlikView) is a reporting tool used by Test Environment (TE).
Figure 2 – Current Test Setup
Planner/ Tester
Test Enviroment (TE)
SUTs
SidePCs
Reporting (QlikView)
N M
N M
1 N
1 1
![Page 25: Scheduling system for test automation frameworkScheduling system for test automation framework Citation for published version (APA): Wahyudi, D., & Technische Universiteit Eindhoven](https://reader033.fdocuments.in/reader033/viewer/2022051923/6011324fdb00301ad8544bee/html5/thumbnails/25.jpg)
1.2.3. SUT Related Activities
Figure 3 – SUT Related Activities Figure 3 shows the activities related to SUT. The explanation of all the activities can be found below: o Preconditioning Activities
Installation It installs the product development code (PDC) to the Allura System. It might involve the OS Installation.
System Configuration The system configuration imports the available configuration for the particu-lar SUT.
System Customization The system customization imports the customization setting generated from the customer request.
System Calibration The calibration makes sure that the X-Ray images are shown correctly.
o Test Activities Smoke Test.
A short test that is executed to verify a new PDC is installed correctly. Performance Test
A test that not only verifies whether a function is available or not, but also gives a quantitative judgment on how well a function is performed (an im-portant part is the responsiveness measurement).
Reliability Test A test that is executed repeatedly to verify the amount of time the system functions without failure.
Manual Test A test performed manually by a test engineer.
1.2.4. SUT Usage A SUT can be assigned to a project and to testers of a project. At regular intervals, reports of the SUT usage are made. A report is made in a tool called Test Environ-ment. Test Environment is a planner tool for manual test. The Test Environment tool shows the manual test schedule using reporting tool called QlikView. Currently, the tests can be planned only in regular tester working hours (between 07:00 and 19:00).
![Page 26: Scheduling system for test automation frameworkScheduling system for test automation framework Citation for published version (APA): Wahyudi, D., & Technische Universiteit Eindhoven](https://reader033.fdocuments.in/reader033/viewer/2022051923/6011324fdb00301ad8544bee/html5/thumbnails/26.jpg)
The night and weekend tests are planned and triggered manually. Some testers made a small application to trigger these tests automatically using Windows Task Sched-uler. The night and weekend test normally covers one reliability test. The problem is only one test can be done in this time. A tester has difficulty to run two or more tests during a night or a weekend. This leads to an inefficient usage of the test environ-ment.
1.3 Problem Description The project needs to improve the current way of testing. There are several problems recognized:
Many different types and configurations of SUT A tester needs to collect a lot of information about SUT type, configuration and PDC Installation. This is a time consuming activity. There is also a risk that the information is not up to date resulting in the incorrect input for the test.
The characteristics of preconditioning activities o Time Consuming o Labor Intensive o Repeating
The whole installation, customization, configuration and calibration process is done manually, requiring considerable human effort and resulting in hu-man errors. Currently, the unattended installation is not possible.
Ineffective usage of SUT The Allura system is not used optimally since the scheduling of more than one test outside the working hours (night and weekend) requires considera-ble effort and is generally avoided by testers.
After recognizing the problem, several goals that need to be achieved, are defined:
To reduce human effort in preconditioning and test activities To reduce human errors To increase the SUT utilization in Testing Environment
1.4 Outline To understand the document well, its outline in relation to the project is explained below: Chapter 1 contains the project introduction. Chapter 2 contains the stakeholder analysis. It identifies the stakeholders involved in the project. Chapter 3 contains the project requirements. The project requirements are derived from interviews and meetings with the customers. Chapter 4 contains the problem and domain analysis. The available tools that can be used for the project are analyzed. Chapter 5 contains the feasibility analysis. It includes the issues and risks for each tool that is used in the project. Chapter 6 contains the description of the system architecture. The 4+1 architectural view model used in the project is documented here. Chapter 7 contains the description of the system design. The design decisions are documented here. Chapter 8 contains the description of each system component implementation.
![Page 27: Scheduling system for test automation frameworkScheduling system for test automation framework Citation for published version (APA): Wahyudi, D., & Technische Universiteit Eindhoven](https://reader033.fdocuments.in/reader033/viewer/2022051923/6011324fdb00301ad8544bee/html5/thumbnails/27.jpg)
Chapter 9 contains the verification and validation of the project. Chapter 10 contains the description of the project management. Chapter 11 contains the results and conclusions. ■
![Page 28: Scheduling system for test automation frameworkScheduling system for test automation framework Citation for published version (APA): Wahyudi, D., & Technische Universiteit Eindhoven](https://reader033.fdocuments.in/reader033/viewer/2022051923/6011324fdb00301ad8544bee/html5/thumbnails/28.jpg)
![Page 29: Scheduling system for test automation frameworkScheduling system for test automation framework Citation for published version (APA): Wahyudi, D., & Technische Universiteit Eindhoven](https://reader033.fdocuments.in/reader033/viewer/2022051923/6011324fdb00301ad8544bee/html5/thumbnails/29.jpg)
2.Stakeholder Analysis Bridge – In this chapter, the stakeholder analysis is described. The stakeholders in-volved were identified and their concerns were analyzed. Interviews and meetings were held with the customers to understand the problem. The wish list is generated from the information collected.
2.1 Introduction Several stakeholders were identified. The listed stakeholders and their roles in this project can be found below: Table 1 – Stakeholders Role Name University Supervisor Johan Lukkien Company Supervisor Alex Visser Customer/User System Integration and Test Group
Wil van den Berk Marcel Van Veggel Luuk van der Veer Arjan van Osch Saco Meeuwig
Module/Tool Owner Test Tool (TAF) Martijn Drabbels
Marc Urlings Testing Environment (TE) Thomas de Laet
Remco Hijmans Ben van Mierloo
Reporting Tool (QlikView) Anja van der Bolt-Jacobs
2.2 Stakeholders’ Concerns Several stakeholders identified were analyzed based on their role. Their role and con-cern can be seen in the table below: Table 2 – Stakeholders’ Concern Role Concerns University Supervisor
Supervise and guide the trainee to make a successful project. Evaluate the trainee during the project.
Company Supervisor
Supervise and guide the trainee in the project. Direct the trainee to the right design and implementation aim-
ing for the high quality final product. Evaluate the trainee during the project.
Customer/User To have a final product that can help them to solve their prob-lem
Module/Tool Owner
To have the tool used without any changes (Since it might affect some other Philips organization that uses the tool)
2.3 Stakeholders’ Engagement Actions Several stakeholders identified were analyzed based on their engagement action. Table 3 – Stakeholders’ Engagement Action Role Engagement Actions University Project’s Guidance
![Page 30: Scheduling system for test automation frameworkScheduling system for test automation framework Citation for published version (APA): Wahyudi, D., & Technische Universiteit Eindhoven](https://reader033.fdocuments.in/reader033/viewer/2022051923/6011324fdb00301ad8544bee/html5/thumbnails/30.jpg)
Supervisor Steering Group Meeting Document Revision
Company Supervisor
Project’s Guidance Steering Group Meeting Document Revision Demo feedback
Customer/User Requirement Analysis Demo feedback Document Revision Validation
Module/Tool Owner
Help in the Tool Usage
2.4 Customer’ Interview and Meeting Interview and meeting is the strategy being used to get the customer requests. A rapid interview and meeting with the customers was the strategy in the first period of the project. Unclear information was confirmed and clarified in the subsequent meetings. Communication in the later stage was organized in weekly meetings with the cus-tomers. The wish list was created after the information from the customers was col-lected. The wish list created can be seen in Table 4. Table 4 – Customers’ Wish List No Wish List Priority 1 Scheduling of Activities High 2 Running of Activities (precondition and test) Automatically High 3 Warning before the start of automatic activity (Safety) High 4 Stop the Running Activity High 5 Skip the Running Activity Medium 7 Suspend the Running Activity High 8 Adjust the time for the schedule when the schedule is changed Medium 9 Grouping of Activity (Activity Block) High 10 Estimation Time filled by user High 11 Estimation Time updated by the last running activity Low 12 Access Control Low 13 Connection to Qlikview Low 14 Centralized and Easy Access from anywhere High 15 Chose the Activity (File) to execute High 16 Directory Structure of File Store Low 17 Reporting and Email Medium 18 Gantt Chart Overview Medium 19 Handle TAF UI Low 20 Integration with TE (Test Environment) Medium 21 Template for Activity Block Medium Beside the wish list, the other information was also discussed with the customers. This information confirmed what the customers did not require. The listed functional-ity below is out of the scope of the project:
Priority in Activity – no priority is required in the task. Dependency between activities – no dependency needed, however simple
dependency might be needed at the later stage. Execution from the build server – no trigger needed from new code deploy-
ment. This will be done manually. ■
![Page 31: Scheduling system for test automation frameworkScheduling system for test automation framework Citation for published version (APA): Wahyudi, D., & Technische Universiteit Eindhoven](https://reader033.fdocuments.in/reader033/viewer/2022051923/6011324fdb00301ad8544bee/html5/thumbnails/31.jpg)
3.System Requirements Abstract – In this chapter, the requirements are defined. The requirements are com-piled from the wish list created in the previous chapter and finalized in this Chapter. The system requirements are presented below.
3.1 Main Requirements After the problem description in the previous chapter, functionalities to address the problem are defined in this subchapter. The functionalities that the project needs to address are described below:
Scheduling Scheduling is the basic functionality needed for the usage of a SUT. Several SUTs were used for different activities in different times. SSTAF needs the scheduler functionality to organize different activities and different tasks to run.
Running Task / Activity Automatically The Tasks / Activities that need to be run in a SUT fall into two categories: Preconditioning Activity and Test Activity. This can be handled by TAF functionality.
Controlling Access The SUTs are valuable resources for the company. Access control function-ality is needed to regulate the usage of the SUTs.
Reporting Reporting is needed to give the users the overview information. The sched-ule and information for every SUT needs to be viewed by everyone. Fur-thermore, a report should show the result of the preconditioning and test ac-tivity.
3.2 Requirements The system requirements are made according to Philips standard so that each re-quirement has a tag allowing it to be verified and validated later on.
3.2.1. Activity An activity is a task or program that runs in the Side PC. In this project scope, one activity is related to one preconditioning or test script. The requirements related to the activity are listed below:
SSTAF.Activity SSTAF must be able to select a script and assign it to an activity.
SSTAF.Activity.Execution The SSTAF must execute the activity in Side PC at the scheduled time.
SSTAF.Activity.FileStore The SSTAF must retrieve a list of scripts from the file storage.
SSTAF.Activity.Result The SSTAF should log the result of an activity.
SSTAF.Activity.EstimatedTime SSTAF should store the estimated time for an activity.
SSTAF.Activity.Stop A user must be able to stop an activity. The user must be able to start the selected activity after the stop.
![Page 32: Scheduling system for test automation frameworkScheduling system for test automation framework Citation for published version (APA): Wahyudi, D., & Technische Universiteit Eindhoven](https://reader033.fdocuments.in/reader033/viewer/2022051923/6011324fdb00301ad8544bee/html5/thumbnails/32.jpg)
3.2.2. Activity Block An activity block is a group of activities that need to run sequentially. The require-ments related to an activity block are listed below:
SSTAF.ActivityBlock SSTAF must be able to group one or more activities into ActivityBlock.
SSTAF.ActivityBlock.Popup SSTAF must inform the user that an automatic Activity Block will run. The user must have the ability to suspend or stop this action.
SSTAF.ActivityBlock.Stop A user must be able to stop an ActivityBlock. After stop, the user should be able to start the selected activity block.
SSTAF.ActivityBlock.EstimatedTime SSTAF should store and show the estimated time for ActivityBlock.
SSTAF.ActivityBlock.Result The SSTAF should log the result of ActivityBlock.
SSTAF.ActivityBlock.Template SSTAF should be able to reuse the ActivityBlock in the schedule.
3.2.3. Schedule A schedule in this project scope is a list of times at which activity blocks are intended to take place.
SSTAF.Schedule SSTAF must be able to schedule an activity block.
SSTAF.Schedule.Access SSTAF should be accessible remotely.
SSTAF.Schedule.ActivityBlock.Parameter SSTAF should be able to change activity block parameters when the block is sched-uled.
SSTAF.Schedule.OverlappingSchedule It should not be allowed to have two schedules overlapping on the same SUT.
SSTAF.Schedule.OverlappingSchedule.Warning If there are overlapping schedules, the system should give warning to the user. The user could force the system to accept the schedule whereby the system would auto-matically adjust the schedule to avoid overlapping.
SSTAF.Schedule.AdjustSchedule SSTAF should adjust the schedule automatically. When there is overlapping of schedules, the system should move the last added and the subsequent schedules for-ward.
SSTAF.Schedule.AdjustScheduleConstraint An activity containing the “Repeat-until” option must be executed at least once.
SSTAF.Schedule.ResponseTime The system should give a user a feedback within 1 min (response time)
3.2.4. Side PC Side PC is a normal PC that has a direct connection to SUT. The requirements related to a Side PC are listed below:
SSTAF.SidePC SSTAF must be able to execute a schedule in the Side PC.
SSTAF.SidePC.IsAlive
![Page 33: Scheduling system for test automation frameworkScheduling system for test automation framework Citation for published version (APA): Wahyudi, D., & Technische Universiteit Eindhoven](https://reader033.fdocuments.in/reader033/viewer/2022051923/6011324fdb00301ad8544bee/html5/thumbnails/33.jpg)
SSTAF maintains a connection for each Side PC. If the Side PC is not connected for more than 1 hour, the status is “Unknown”.
3.2.5. Reporting A user needs the reporting functionality (this functionality is provided by the report-ing tool).
SSTAF.Reporting.Result SSTAF should log the result of the schedule. The result is listed in the Figure 4.
SSTAF.Reporting.GanttChart SSTAF should show the schedule as Gantt chart.
= passed = skippedSUT = failed = stoppedSUT 1 = in progress = interruptedSUT 2 = not applicable = scheduledSUT 3
=> current timeTime ==>
Planning project X7:00 8:00 9:00 10:00 11:00 12:00 13:00 14:00 15:00
Figure 4 – Scheduler Gantt Chart
3.3 Requirement Priority Table 5 lists the priorities of the requirements. Table 5 – Requirement Priority Priority Requirement 1 SSTAF.Activity
SSTAF.Activity.Execution SSTAF.ActivityBlock SSTAF.Schedule
2 SSTAF.Activity.FileStore SSTAF.Activity.EstimatedTime SSTAF.ActivityBlock.EstimatedTime SSTAF.Schedule.Access
3 SSTAF.ActivityBlock.Template SSTAF.Schedule.ActivityBlock.Parameter SSTAF.Activity.TAFParameter
4 SSTAF.Activity.Result SSTAF.ActivityBlock.Result SSTAF.ActivityBlock.Popup
5 SSTAF.Activity.Stop SSTAF.ActivityBlock.Stop
6 SSTAF.Schedule.OverlappingSchedule SSTAF.Schedule.OverlappingSchedule.Warning SSTAF.Schedule.AdjustSchedule SSTAF.Schedule.AdjustScheduleConstraint
7 SSTAF.Schedule.ResponseTime SSTAF.SidePC.IsAlive SSTAF.Reporting.Ganttchart
■
![Page 34: Scheduling system for test automation frameworkScheduling system for test automation framework Citation for published version (APA): Wahyudi, D., & Technische Universiteit Eindhoven](https://reader033.fdocuments.in/reader033/viewer/2022051923/6011324fdb00301ad8544bee/html5/thumbnails/34.jpg)
![Page 35: Scheduling system for test automation frameworkScheduling system for test automation framework Citation for published version (APA): Wahyudi, D., & Technische Universiteit Eindhoven](https://reader033.fdocuments.in/reader033/viewer/2022051923/6011324fdb00301ad8544bee/html5/thumbnails/35.jpg)
4.Problem Analysis Bridge – In this chapter, the problem is analyzed to define the direction of the project. Several Common off the Shelf (COTS) tools are investigated. Several pro-jects and tools already used in Philips are also investigated.
4.1 Problem Examination In this subchapter, the problems are examined to find the solutions. The problems recognized in the current test environment are listed below: Many different types and configurations of SUT
In order to avoid the mistakes in the manual process of collecting the data (TAF parameter), each time a test needs to be executed this data should be readily available from a central storage where it is regularly maintained.
The Time Consuming, Labor Intensive, and Repeating character of precondition-ing activities. In order to reduce the human effort, the preconditioning activities need to be ex-ecuted automatically. Currently, the unattended installation is not possible. The iXR software installer should be made ready for this purpose. Smart Suite ver-sion of iXR software is being developed to accommodate this new feature. The automatic installation is still an ongoing project that needs to be integrated with SSTAF later.
Ineffective usage of SUT In order to increase the usage of SUT, multiple unattended scheduling of tests, especially outside the working hours (night and weekend) is needed. For this to work the scheduling functionality is needed. The SUT usage would also need to be regulated (who can use the SUT at a particular time). To provide this feature, access control functionality would be needed.
The problems above are addressed well by the main requirements defined in the pre-vious chapter:
Scheduling of activities Running of activities automatically Access Control Reporting
In the next subchapter, the possible solutions to fulfill these main requirements are analyzed.
4.2 Domain Model Preliminary domain model that was made to fulfill the main requirements can be seen in Figure 5. Short description to explain the domain model can be found below (Bold – domain model): Access Control to an Allura System is provided by assignment of a Tester to a Pro-ject. The Scheduler assigns a Schedule of Activities to the Allura System. The Time Trigger should trigger the Executor to execute the Activity automatically. After the execution, the Report should show the Schedule Result and the Activity Result.
![Page 36: Scheduling system for test automation frameworkScheduling system for test automation framework Citation for published version (APA): Wahyudi, D., & Technische Universiteit Eindhoven](https://reader033.fdocuments.in/reader033/viewer/2022051923/6011324fdb00301ad8544bee/html5/thumbnails/36.jpg)
Figure 5 – Domain Model
4.3 Mapping of COTS Tools to Domain Model After completing the domain model with the necessary components, we try to map the model to common off the shelf tools. There are three types of tools related to the domain model above:
Planner / Calendar, e.g. Microsoft Project, Google Calendar The tool provides access control and scheduling of activity
Job Scheduler, e.g. Windows Task Scheduler The tool provides scheduling and running of activity automatically
Continuous Integration, e.g. Electric Commander, Jenkins The tool provides almost all functionality required except the reporting
The mapping of the COTS tools to Domain Model can be seen in the Figure 6.
Figure 6 – Mapping of COTS Tools to Domain Model
Scheduling of Activities
Controlling Access
Running of Activities
Automatically
Reporting
Use
Register
Trigger
Assign
Job Scheduler
Calendar / Planner
Continuous Integration
![Page 37: Scheduling system for test automation frameworkScheduling system for test automation framework Citation for published version (APA): Wahyudi, D., & Technische Universiteit Eindhoven](https://reader033.fdocuments.in/reader033/viewer/2022051923/6011324fdb00301ad8544bee/html5/thumbnails/37.jpg)
The details of the COTS Selection can be found in Appendix A. COTS Selection. None of the COTS tools met the reporting requirement. COTS tool selection process points out Jenkins as the result. It is open source software. It has many plugins and has different customization and it is easy to extend. However, it also might take more time to customize and extend Jenkins to have the desired functionality.
4.4 Mapping of Related Philips tools to Domain Model Several tools available in Philips (e.g. TAF and Step Manager) were found suitable to be used in the project. The possibilities for use of these tools are discussed further below. Furthermore, the Test Environment tool (yet to be realized) is also considered. The Test Environment (TE) tool is a planning system for manual tests. The mapping between the related Philips tools against the domain model can be seen in Figure 7. The related Philips tools are described below: Test Automation Framework (TAF)
TAF is the custom test framework being used in iXR for testing purposes. TAF has been used for several years, so the tool is already stable. The test script that needs to run is written in TAF format. The TAF tool is used for executing the test script.
Step Manager Step Manager is the custom tool being used in the factory to execute several ac-tivities in sequence automatically. Step Manager is already stable. The use of Step Manager is recommended for TAF execution functionality.
Test Environment (TE) The TE project covers the development of planning and reporting systems for manual tests. The project delivers the tool called Test Environment (TE). The tool fulfills many of the reporting, access control, and UI requirements (required for SSTAF). Most of the TE and SSTAF functionalities are common except the scheduling and task execution. For this reason, SSTAF could be built as an ex-tension of TE. The design of this extended TE model can be found in Appendix B. Design Evolution
QlikView QlikView is the COTS reporting tool that is widely used in Philips. The man-agement prefers to have QlikView reports, so the use of it is recommended. TE tool also uses QlikView.
Figure 7 – Mapping of Philips Tools to Domain Model
Step Manager
TAF QlikView
Test Environment
![Page 38: Scheduling system for test automation frameworkScheduling system for test automation framework Citation for published version (APA): Wahyudi, D., & Technische Universiteit Eindhoven](https://reader033.fdocuments.in/reader033/viewer/2022051923/6011324fdb00301ad8544bee/html5/thumbnails/38.jpg)
4.5 Problem Analysis Result The analysis of the COTS tools (Appendix A. COTS Selection) points out Jenkins as a strong candidate. Furthermore, Jenkins is free and it is used by Philips organiza-tion. However, it is difficult to integrate Jenkins with TE tool. The integration with TE tool is necessary and is treated as a design constraint so Jenkins is not considered anymore. Figure 7 shows that many Philips tools can be assembled and reused for the project purpose. Based on Figure 7 several decisions were made:
TAF The execution of TAF is necessary.
QlikView The reporting with QlikView is recommended, although it can be automati-cally integrated through TE.
Step Manager The use of Step Manager is recommended for TAF execution.
Test Environment The integration with TE is necessary since TE has many overlapping func-tionalities with SSTAF. SSTAF project will be made ready for this integra-tion. The design of SSTAF will be in-sync with TE so the integration in the future can be done with minimal effort. The picture below provides a rough sketch of the integration. The integration point of TE and SSTAF will be the database.
Figure 8 – Basic Plan for TE Extension
TE
SSTAF
![Page 39: Scheduling system for test automation frameworkScheduling system for test automation framework Citation for published version (APA): Wahyudi, D., & Technische Universiteit Eindhoven](https://reader033.fdocuments.in/reader033/viewer/2022051923/6011324fdb00301ad8544bee/html5/thumbnails/39.jpg)
Figure 7 shows that Time-trigger functionality is not covered by the tools. Windows Task Scheduler (WTS) can be used to fill the Time-trigger functionality. The extra advantage is that WTS is free and included in the existing support contracts. Based on the Figure 7, the proposed test setup is given in Figure 9.
Figure 9 – Expected Test Setup ■
Test Enviroment (TE)
SUTs
SidePCs
Reporting (QlikView)
Planner/ Tester
N 1
1 1
N M
![Page 40: Scheduling system for test automation frameworkScheduling system for test automation framework Citation for published version (APA): Wahyudi, D., & Technische Universiteit Eindhoven](https://reader033.fdocuments.in/reader033/viewer/2022051923/6011324fdb00301ad8544bee/html5/thumbnails/40.jpg)
![Page 41: Scheduling system for test automation frameworkScheduling system for test automation framework Citation for published version (APA): Wahyudi, D., & Technische Universiteit Eindhoven](https://reader033.fdocuments.in/reader033/viewer/2022051923/6011324fdb00301ad8544bee/html5/thumbnails/41.jpg)
5.Feasibility Analysis Abstract – In this chapter, the feasibility analysis is described. Several issues and risks related to the tools are described in this chapter. The possibility of using the Windows Task Scheduler (WTS) is also investigated.
5.1 Test Environment (TE) Project Test Environment Project is a running project that has not yet been released. After the start of SSTAF project, some concerns were raised that it might influence or even endanger certain aspects of TE Project. Therefore, there were objections to allowing the SSTAF project to be included in the TE project. The issue was raised to a higher level of management who decided to separate the development of these tools without any interaction in the first releases. It was also decided that SSTAF and TE integra-tion would be necessary, although not in this project timeline. The access control functionality is removed from the requirements of SSTAF since TE project has ac-commodated this.
5.2 Test Automation Framework (TAF) Test Automation Framework is a stable tool. The tool is used by many organizations in Philips Healthcare. Because of this, it would be difficult to make changes in TAF in the case they are required. The test activities are currently executed by TAF. How-ever, the execution of preconditioning activities in TAF is still not supported. Cur-rently, Factory Tooling project is developing the automatic installation tool to be used in the factory. Step Manager is the base for that tool. SSTAF will use Step Man-ager to provide easy integration with factory tooling (to perform automatic installa-tion). The risk of using Step Manager is explained further in the next subchapter.
5.3 Step Manager Step Manager is a tool being used in the factory to execute workflow steps in the installation, configuration and test of a product. The tool has already been used for almost 10 years. Step Manager executes TAF as one of the workflow steps. This functionality can be used in SSTAF project. However, currently there is no commu-nication to and from Step Manager. Changes to accommodate this functionality should be done in Step Manager and that could be a risk to the project.
5.4 QlikView QlikView is a reporting tool. It can show the report in the chart, table or graph format easily. It is used in many organizations of Philips as it meets the management expec-tations. However, the user has stated that QlikView integration is a low priority re-quirement. Therefore, QlikView will be out of the scope to this project. The connec-tion to QlikView will be made after the essential requirements are met.
5.5 Windows Task Scheduler (WTS) Windows Task Scheduler is a COTS tool considered to accommodate the scheduling functionality. It is able to schedule a task at a scheduled time. One prototype applica-tion was built to show this functionality. The prototype was built in the server to schedule a task/activity in the Side PC. To schedule this task the Side PC username and password are needed. This is considered a big risk since the customer is not com-fortable with putting the username or password of every Side PC on the server. Be-cause of this, it was decided to make a timer application to handle the scheduler func-tionality. By having its own code, it is possible to customize and control the sched-ule.
![Page 42: Scheduling system for test automation frameworkScheduling system for test automation framework Citation for published version (APA): Wahyudi, D., & Technische Universiteit Eindhoven](https://reader033.fdocuments.in/reader033/viewer/2022051923/6011324fdb00301ad8544bee/html5/thumbnails/42.jpg)
Figure 10 – Windows Task Scheduler Prototype
5.6 Feasibility Analysis Result The feasibility study has big influence on the project. The final direction and the tool used in the project can be seen in Figure 11. Philips tools (TAF, Step Manager, QlikView) will be used. However, the scheduler (dark blue color) needs to be devel-oped. The Scheduler will need to integrate with TE in the future.
Figure 11 – Final Mapping of Philips Tools to Domain Model ■
QlikView
Future TE Integration
Step Manager
TAF
Scheduler
Server
SidePCs
1 N Windows Task Scheduler Windows Task
Scheduler SidePC UserName +Password Trigger &
Execute
![Page 43: Scheduling system for test automation frameworkScheduling system for test automation framework Citation for published version (APA): Wahyudi, D., & Technische Universiteit Eindhoven](https://reader033.fdocuments.in/reader033/viewer/2022051923/6011324fdb00301ad8544bee/html5/thumbnails/43.jpg)
6.System Architecture Abstract – In this chapter, the system architecture is described using the 4 + 1 archi-tectural view model. Firstly, the context diagram is shown describing the boundaries of the system. Then the use case follows. Preliminary system architecture is made in order to define the system further. Then the preliminary architecture is further devel-oped into the full system architecture.
6.1 Context Diagram The context diagram shows the boundaries of the system. Step Manager is not in-cluded in the context diagram since it is considered the part of the system. SUT is not included in the context diagram since the access to SUT is provided by TAF.
Scheduling System for Test
Automation Framework (SSTAF)
Reporting
User / Tester
Test Automation Framework
(TAF)
CRUD Schedule, Activity Block, Activity
Log and Report
Run with Parameter
Figure 12 – Context Diagram
6.2 Use Case Figure 13 shows the use case diagram of Scheduler System.
Figure 13 – Use Case Diagram
![Page 44: Scheduling system for test automation frameworkScheduling system for test automation framework Citation for published version (APA): Wahyudi, D., & Technische Universiteit Eindhoven](https://reader033.fdocuments.in/reader033/viewer/2022051923/6011324fdb00301ad8544bee/html5/thumbnails/44.jpg)
Activity Block is the unit used in the Use Case Diagram. An activity block is a group of activities that need to be executed sequentially. Further explanation of Activity Block can be seen in the following subchapter.
6.3 Preliminary System Architecture From the Feasibility Analysis, we know that we need to develop the Scheduler Sys-tem. This subchapter describes the preliminary design to define the system further.
6.3.1. Relationship between Schedule, Activity Block and Activity A schedule is a list of times at which activity blocks are intended to take place. An activity block is a group of activities that need to be executed sequentially. An activi-ty is a preconditioning or test activity. The relationship between schedule, activity block and activity is shown in Figure 14.
Figure 14 – Relationship of Schedule, Activity Block and Activity
6.3.2. Preliminary Class Diagram The preliminary class diagram is based on the domain model. The Access Control is removed from the model. “Allura System” is changed into “Side PC of Allura Sys-tem” since TAF execution is in the Side PC. Activity Block (with the executor and the result) is added to the model. The preliminary class diagram is shown in Figure 15. The colors show the mapping between the classes and the tools.
Schedule
Activity Block
Activity
Activity
Activity
Activity Block
Activity
Activity
Schedule of Activity Block(s)
when they need to run
Activity Block Activities that need to run
in sequence
Activity Preconditioning / Test
Activity
![Page 45: Scheduling system for test automation frameworkScheduling system for test automation framework Citation for published version (APA): Wahyudi, D., & Technische Universiteit Eindhoven](https://reader033.fdocuments.in/reader033/viewer/2022051923/6011324fdb00301ad8544bee/html5/thumbnails/45.jpg)
Figure 15 – Preliminary Class Diagram
6.3.3. Preliminary Activity Diagram Activity Diagram is made to described the process of the scheduler system. Figure 16 shows the activity diagram for the Scheduler System.
Figure 16 – Preliminary Activity Diagram
Schedule
Trigger
Execute
Execute
Register
Trigger
Execute
Step Manager
TAF
Scheduler
![Page 46: Scheduling system for test automation frameworkScheduling system for test automation framework Citation for published version (APA): Wahyudi, D., & Technische Universiteit Eindhoven](https://reader033.fdocuments.in/reader033/viewer/2022051923/6011324fdb00301ad8544bee/html5/thumbnails/46.jpg)
The activity diagram shows three major stages: Data Input
Data input is the stage of getting the user input for activity, activity block and schedule. This process needs to take place in the Server since a central data repository is needed.
Schedule Execution The schedule execution needs to take place in the Side PC since this is where TAF is located.
Reporting The reporting is the stage of getting the schedule results. The process needs to take place in the Server since the results should be stored in a central data repository.
6.3.4. Preliminary Deployment Diagram From the previous subchapter we understand that the system should have two differ-ent deployment computers:
1. Server It needs to handle data input and reporting. The server should be connected to User PC to handle data input.
2. Side PC It needs to handle the Schedule Execution. The Side PC should get the data from the server for the Schedule Execution.
To make it more clear the preliminary deployment diagram is shown in Figure 17.
Server Side PC
Allura XPer System
TAF
User PC
Figure 17 – Preliminary Deployment Diagram
6.4 Full System Architecture In this subchapter, the preliminary system architecture is further developed into the complete system architecture.
6.4.1. Logical View
Class Diagram The class diagram (Figure 18) is developed from the preliminary class diagram shown in Figure 15. This process comprises the assignment of concrete components (the existing ones and the ones to be developed) to the components identified in the preliminary phase and it can be seen in Table 6.
1 N
1 N
1 1
![Page 47: Scheduling system for test automation frameworkScheduling system for test automation framework Citation for published version (APA): Wahyudi, D., & Technische Universiteit Eindhoven](https://reader033.fdocuments.in/reader033/viewer/2022051923/6011324fdb00301ad8544bee/html5/thumbnails/47.jpg)
Table 6 – Translation for Class Diagram Preliminary Class Diagram Scheduler Scheduler Server Time Trigger Scheduler Agent Activity Block Executor Step Manager Executor TAF Side PC of Allura System Side PC Schedule Schedule Schedule Result Schedule Result Activity Block Step Manager Recipe Activity Block Result Step Manager Result Activity TAF Configuration Activity Result TAF Result - Activity Block (Template) - Activity (Template) The class diagram that shows the classes of the scheduler system can be seen in Fig-ure 18. Activity Block (Template) and Activity (Template) is added to the model. This addition allows the user to reuse the Activity Block and Activity already defined before.
Figure 18 – Class Diagram
![Page 48: Scheduling system for test automation frameworkScheduling system for test automation framework Citation for published version (APA): Wahyudi, D., & Technische Universiteit Eindhoven](https://reader033.fdocuments.in/reader033/viewer/2022051923/6011324fdb00301ad8544bee/html5/thumbnails/48.jpg)
The short description of the classes in Figure 18 can be found in Table 7 Table 7 – Class Diagram Element Classes Description SidePC The SidePC class defines the SidePC attributes. Scheduler Server The Scheduler Server class provides the user input functionality Command The command class contains the communication data between Sched-
uler Server and Scheduler Agent. Scheduler Agent The Scheduler Agent class provides the schedule execution in the Side
PC Schedule The schedule class contains the schedule information. It also contains
the activity block information. Schedule Result The schedule result class contains the result of a schedule execution.
One schedule can have several results if the schedule is executed several times.
Activity Block (Template)
The activity block class contains the sequence of activities and their information.
Activity (Template) The activity class contains the Activity information (TAF Configura-tion)
Step Manager The Step Manager class represents the Step Manager functionality Step Manager Recipe
The Step Manager Recipe class represents the input file of Step Manag-er
Step Manager Result
The Step Manager Result class represents the output file of Step Man-ager
TAF The TAF class represents the TAF functionality TAF Configuration The TAF Configuration class represents the input file of TAF TAF Result The TAF Result represents the output file of TAF
Sequence Diagram The sequence diagram is made to show the interaction between components. The sequence diagram in Figure 19 shows the sequence from schedule creation by tester until activity execution by TAF.
Figure 19 – Sequence Diagram
![Page 49: Scheduling system for test automation frameworkScheduling system for test automation framework Citation for published version (APA): Wahyudi, D., & Technische Universiteit Eindhoven](https://reader033.fdocuments.in/reader033/viewer/2022051923/6011324fdb00301ad8544bee/html5/thumbnails/49.jpg)
6.4.2. Process View The Activity Diagram (Figure 20) is developed from the activity diagram shown in Figure 16. Another version of the process view as swim lane diagram can be seen in Appendix C.
CRUD Activity
CRUD Activity Block
CRUD Schedule
Schedule TimeTrigger
Generate and Copy File
Schedule Execution
Step Manager Execution
TAF Execution
Update Result
Save Data
User Input
Create TAF Test File
Save Data
Put Server Data
Reporting
Create Command
Get Server Data
ExecuteCommand
Figure 20 – Activity Diagram
Test Step for TAF
Activity Activity Block
Schedule
Command
All Data
All Data
Command
ScheduleStep Manager Recipe Test Step
for TAF
Schedule Result Step
Manager Result TAF Result
![Page 50: Scheduling system for test automation frameworkScheduling system for test automation framework Citation for published version (APA): Wahyudi, D., & Technische Universiteit Eindhoven](https://reader033.fdocuments.in/reader033/viewer/2022051923/6011324fdb00301ad8544bee/html5/thumbnails/50.jpg)
Each of the elements shown in the Activity diagram is described in short detail in Table 8. Table 8 – Activity Diagram Element Activity Diagram Element
Description
User Input The user performs data input activity on the system. Create TAF Test File
Create TAF test file using the TAF software in the user PC
CRUD Activity Create, Read, Update, or Delete of the Activity CRUD Activity Block
Create, Read, Update, or Delete of the Activity Block
CRUD Schedule
Create, Read, Update, or Delete of the Schedule
Save Data Save data to database Generate and Copy File
Generate XML file and copy TAF data
Get Server Data Get data from the server to the Side PC Execute Command Execute command data Schedule Time Trigger
Register the schedule data in the time trigger
Schedule Execution The execution of schedule Step Manager Exe-cution
Process of running Step Manager
TAF Execution Process of running TAF Put Server Data Put data from the Side PC to the server Update Result Update results to database Save Data Save data to database Reporting Show the results in reporting Activity Diagram Data Object
Description
Test Step for TAF
The TAF configuration file
Activity The activity data Activity Block The activity block data Schedule The schedule data Command The command data Step Manager Rec-ipe
The step manager data
Command Result The command result data Schedule Result The schedule result data Step Manager Re-sult
Step Manager result data
TAF Result TAF result data
6.4.3. Deployment View The deployment diagram shows where each component is deployed. There are four deployment machines: The Server
A Server connected to Philips Network. It contains several components: Scheduler Server
It provides the user input. Database
The Scheduler Server stores the scheduler data in the database. XML Storage
![Page 51: Scheduling system for test automation frameworkScheduling system for test automation framework Citation for published version (APA): Wahyudi, D., & Technische Universiteit Eindhoven](https://reader033.fdocuments.in/reader033/viewer/2022051923/6011324fdb00301ad8544bee/html5/thumbnails/51.jpg)
The Scheduler Server needs to store and organize the XML files for Side PC. The use of XML files is necessary since TAF and Step Man-ager use XML files as their input. Detail reasons are explained in the next chapter.
Reporting The reporting accesses the data from database and XML Storage.
The Side PC A computer connected to a particular SUT. It contains several components:
The scheduler Agent The scheduler agent is the centralized control of the system. The sched-uler agent handles the scheduling functionality in the side PC.
Step Manager It handles the execution of the Activity Block.
TAF It handles the execution of Activity (TAF File).
The User PC A computer used to access the scheduler system. It contains several components:
Browser It provides the data input to the Server in the User PC.
TAF It is used to create an activity (TAF File).
Allura Xper System The Allura Xper System used for the test or preconditioning activities. This sys-tem is also known by System under Test (SUT).
Server Side PC
Allura XPer System
Scheduler Server
TAF
Step Manager
Scheduler Agent
User PC
Browser
Reporting
TAF
Database
XML Storage
Figure 21 – Deployment Diagram
■
![Page 52: Scheduling system for test automation frameworkScheduling system for test automation framework Citation for published version (APA): Wahyudi, D., & Technische Universiteit Eindhoven](https://reader033.fdocuments.in/reader033/viewer/2022051923/6011324fdb00301ad8544bee/html5/thumbnails/52.jpg)
![Page 53: Scheduling system for test automation frameworkScheduling system for test automation framework Citation for published version (APA): Wahyudi, D., & Technische Universiteit Eindhoven](https://reader033.fdocuments.in/reader033/viewer/2022051923/6011324fdb00301ad8544bee/html5/thumbnails/53.jpg)
7.System Design Abstract – In this chapter, main design decisions are documented. The design deci-sions explain the decision taken in System Architecture (Chapter 6). Further designs of the scheduler system components are also documented in this chapter.
7.1 Main Design Decisions This subchapter explains the reasoning behind the decision making process.
7.1.1. Fat or Thin Client Architecture Regarding the global architecture of the system shown in Figure 21, design decisions are made for Server – User architecture (comprising Server – User PC). The defini-tion of Fat Client and Thin Client taken from [9] is quoted below. “Fat-client computing refers to a multi-tier client server paradigm where (in the sim-plest model) the client part of the application (i.e., those programs used by the end-user) executes on the desktop PC and the server part of the application (i.e., those programs used to drive the relational database) resides on a single server together with the application code. Thin-client computing refers to multi-tier client server paradigm where the client (end-user) programs display in a browser (such as Internet Explorer or Netscape) but the execution of that user code takes place on a central web server, not at the desktop PC “. The advantages and disadvantages of the design options are explained in Table 9. Table 9 – Fat or Thin Client Architecture Option Advantages Disadvantages Fat Client Architecture
The system has more control on the client side because the client is an application.
The system needs installation of the client applications. When it is updated then all the appli-cations need to be updated and installed again.
Thin Client Architecture
The system uses web browser as a client, so no installation is required.
The system has no control of the client because the client only uses web browser.
Considering these advantages and disadvantages, Thin Client Architecture was cho-sen for Server – User architecture. The user does not need to install the client applica-tion to schedule the task. He just uses the browser to give input to the scheduler web-site. Another reason is the TE Integration design constraint.
7.1.2. Smart Server or Smart Agent For the Server – Agent architecture a choice must be made where to put the ‘brain’. A component with the ‘brain’ should handle the schedule execution. The Server component that might handle the schedule execution is the Scheduler Server. The Side PC component that might handle the schedule execution is the Scheduler Agent. A decision to choose whether the scheduler server or scheduler agent should be smart is made in this subchapter. The advantages and the disadvantages of both options are listed below: Table 10 – Smart Server or Smart Agent Option Advantages Disadvantages
![Page 54: Scheduling system for test automation frameworkScheduling system for test automation framework Citation for published version (APA): Wahyudi, D., & Technische Universiteit Eindhoven](https://reader033.fdocuments.in/reader033/viewer/2022051923/6011324fdb00301ad8544bee/html5/thumbnails/54.jpg)
Smart Server – Dumb Agent
The server has centralized control and the server control everything.
1. The schedule execution will need network connection.
2. There is a network delay for the execution.
Dumb Server – Smart Agent
1. The agent able to work by itself without server con-nection.
2. The agent has more con-trol because the schedule execution is performed on the same computer.
3. There is no network delay for the schedule execu-tion.
The agent needs to update the server about the schedule exe-cution status. There is a net-work delay involved, but the schedule execution will not have a delay.
After analyzing the advantages and disadvantages, the decision was made to put the ‘brain’ in the Scheduler Agent. The smart agent should be able to work independent-ly even without server connection. If the network is disconnected or if the server is unavailable, the smart agent is still able to operate.
7.1.3. Interface between Scheduler System Components There are three types of interface investigated to be used between the components (Scheduler Server – Scheduler Agent – Step Manager – TAF). The analysis of the different types of interface can be found below: Table 11 – Interface/Communication Option Advantages Disadvantages File Interface
1. File Interface can go behind the firewall; it on-ly needs to map the net-work drive.
2. File Interface will inte-grate well with Philips tools (TAF and Step Manager) since it is us-ing XML as input.
File Interface depends on sta-bility of the network connec-tion.
TCP/IP Interface TCP/IP Interface has direct communication and full con-trol of the execution time and the response
TCP/IP Interface cannot go behind firewall if the port is not open.
Command Line Interface
1. Command Line Interface has direct communica-tion and full control of the execution time and the response.
2. Basic interface for win-dows applications
Command Line Interface can be only executed on the same computer.
Based on the analysis above, the decision is explained below:
Scheduler Server – Scheduler Agent: File Interface was chosen since XML File is used as input for Step Manager and TAF. Another reason is to avoid the firewall problem.
Scheduler Agent – Step Manager: Command Line Interface was chosen since it is a basic and easy interface to be used in windows applications.
Step Manager – TAF: Command Line Interface is used since TAF already has the command line interface.
![Page 55: Scheduling system for test automation frameworkScheduling system for test automation framework Citation for published version (APA): Wahyudi, D., & Technische Universiteit Eindhoven](https://reader033.fdocuments.in/reader033/viewer/2022051923/6011324fdb00301ad8544bee/html5/thumbnails/55.jpg)
7.1.4. Database or XML File The question is what data storage should be used in Scheduler System. The strong candidates are database and XML file. The analysis is described below: Table 12 – Database or XML File Option Advantages Disadvantages Database 1. Database is common, fast
and organized data stor-age.
2. Database has remote ac-cess.
3. Database is easier for web application.
Database needs installation and maintenance.
XML File
1. XML File is a light form of data storage.
2. XML File is easy to move around
1. More effort is needed to access the information in-side the XML File.
2. More effort is needed to organize the XML File.
From the analysis above, they both appear as strong candidates to be used in Sched-uler System. It is decided to use both in the project. Further reasoning can be seen below:
Database is used as Server application data storage o Integration with the Test Environment in the future will be easier to
implement. XML File is used as Side PC application data storage
o Philips tools already use XML files for their input (TAF and Step Manager).
o Database installation in each Side PC is not necessary. After the decision to use XML File as Side PC application data storage, centralized XML Storage is needed to accommodate the data input in the server. The XML File will be generated by scheduler server and copied to the local folder by Scheduler Agent.
7.2 Components Design In this subchapter, design of each of the system components is described. The design continues by mapping the components from the class diagram to the deployment dia-gram. Sequence Diagram for the Scheduler System also described into more detail in this subchapter.
7.2.1. Mapping of Class Diagram to Deployment Diagram Mapping of the components from the class diagram (Figure 18) to the deployment diagram (Figure 21) is explained in Table 13.
Table 13 – Class Diagram to Deployment Diagram Class Diagram Deployment Dia-
gram Description
Scheduler Server Scheduler Server in Server
This class represents Scheduler Server and its functionali-ty.
Scheduler Agent Scheduler Agent in Side PC
This class represents Scheduler Agent and its functionality.
Step Manager Step Manager in Side PC
This class represents Step Manager and its functionality.
TAF TAF in Side PC This class represents TAF and its functionality. Command Database in Server, The class represents Command data. This data is stored in
![Page 56: Scheduling system for test automation frameworkScheduling system for test automation framework Citation for published version (APA): Wahyudi, D., & Technische Universiteit Eindhoven](https://reader033.fdocuments.in/reader033/viewer/2022051923/6011324fdb00301ad8544bee/html5/thumbnails/56.jpg)
XML Storage in Server and Side PC
the database and subsequently in a XML file. This data is the input for Scheduler Agent.
Activity Block (Tem-plate)
Database in Server The class represents Activity Block data. This data is stored in the database
Activity (Template) Database in Server The class represents Activity data. This data is stored in the database
Side PC Database in Server; XML Storage in Server
This class represents Side PC data. This data is stored in the database. This data is realized as folder in Server XML Storage.
Schedule Database in Server; XML Storage in Server and Side PC
The class represents Schedule data. This data is stored in the database and subsequently in a XML file. One XML file is created for each side PC. This data is input for Scheduler Agent.
Schedule Result Database in Server; XML Storage in Server and Side PC
The class represents Schedule Result data. The data is gen-erated from the schedule execution and placed in a XML file. Subsequently the data is placed in the database.
Step Manager Recipe XML Storage in Server and Side PC
The class represents the input of Step Manager.
Step Manager Result XML Storage in Server and Side PC
The class represents the output of Step Manager.
TAF Configuration XML Storage in Server and Side PC
The class represents the input of TAF.
TAF Result XML Storage in Server and Side PC
The class represents the output of TAF.
The mapping for the class diagram is shown in Deployment Diagram in Figure 22
Server Side PC
Allura XPer System
Scheduler Server
TAF
Step Manager
Scheduler Agent
User PC
Browser TAF
Database
Server XML Storage
Agent XML Storage
The Database contains : SidePC,Command,Schedule, ScheduleResult, ActivityBlockTemplate, ActivityTemplate
The XML Storage contains :SidePC Folder, Command, Schedule, Schedule Result, Step Manager Recipe, Step Manager ResultTAF Configuration, TAF Result
The Agent XML Storage is copied fromServer XML Storage, It contains:Command, Schedule, ScheduleResult, Step Manager Recipe, Step Manager Result,TAF Configuration,TAF Result
The result is copied back to server and deleted after it finished copied
Figure 22 – Mapping Class Diagram to Deployment Diagram
![Page 57: Scheduling system for test automation frameworkScheduling system for test automation framework Citation for published version (APA): Wahyudi, D., & Technische Universiteit Eindhoven](https://reader033.fdocuments.in/reader033/viewer/2022051923/6011324fdb00301ad8544bee/html5/thumbnails/57.jpg)
Figure 22 and Table 13 show that the data classes (Command, Schedule, Schedule Result, Step Manager Recipe, Step Manager Result, TAF Configuration and TAF Result) are shared by Server and Side PC in the XML Storage. Scheduler Server is using the database as its data storage and the data is generated into XML File in Server XML Storage. Scheduler Agent gets the XML File from the Server XML Storage and copies it to the Agent XML Storage in the Side PC. The implementation of these components is shown in the next chapter.
7.2.2. Detail Sequence Diagram The Scheduler System is designed further from the sequence diagram in Figure 19. User Interrupt is added in the schedule execution. The sequence diagram is made from the use case in Figure 13. The sequence diagram is separated into two different stages: Data Input and Schedule Execution.
Data Input Figure 23 shows the data input sequence diagram. The tester’s browser is used to access Scheduler Server. Scheduler Server provides data input functionality (for da-tabase and XML Storage). Scheduler Agent copies XML files from Server XML Storage to Agent XML Storage.
Figure 23 – Data Input Sequence Diagram
Schedule Execution & User Interrupt Figure 23 shows the sequence diagram of schedule execution. User Interrupt is also included in the sequence diagram. User Interrupt is generated as command XML. Scheduler Agent provides schedule execution and command execution. Step Manag-er provides Activity Block execution. TAF provides Activity execution. Scheduler Server provides input for User Interrupt and updates to database.
![Page 58: Scheduling system for test automation frameworkScheduling system for test automation framework Citation for published version (APA): Wahyudi, D., & Technische Universiteit Eindhoven](https://reader033.fdocuments.in/reader033/viewer/2022051923/6011324fdb00301ad8544bee/html5/thumbnails/58.jpg)
Scheduler Server Scheduler AgentBrowser of Tester Database Server XML Storage
TimeTrigger
Step Manager
TAF
Run(StepManagerRecipe)
Run(Test.taf)
Finished
UpdateScheduleResult
SaveData
CheckResult
Generate(Command.xml)
Command = DeleteResult
Execute(Command.xml)
ExecuteCommand(DeleteResult)
Stop
Generate(Command.xml)
Command = Stop
Execute(Command.xml)
ExecuteCommand(Stop)
StopCopy(CommandResult.xml)
StopCopy(CommandResult.xml)
SaveData
Agent XML Storage
GetScheduleData
ScheduleData
Finished
Put(ScheduleResult)
Copy(ScheduleResult)
Put(ScheduleResult) will copyall the schedule result to the SidePC
Figure 24 – Scheduler Execution Sequence Diagram
7.2.3. State Machine Diagram In this subchapter, the state machine of Scheduler and Step Manager are described.
Scheduler Agent State Machine The Scheduler Agent state machine diagram is shown in Figure 25. The Scheduler Server gets the state by sending command to Scheduler Agent.
Figure 25 – Scheduler Agent State Diagram The explanation of each state can be found in Table 14
![Page 59: Scheduling system for test automation frameworkScheduling system for test automation framework Citation for published version (APA): Wahyudi, D., & Technische Universiteit Eindhoven](https://reader033.fdocuments.in/reader033/viewer/2022051923/6011324fdb00301ad8544bee/html5/thumbnails/59.jpg)
Table 14 – Step Manager State State Description Idle The scheduler agent is running but there is
no schedule being executed at that mo-ment.
Running Step Manager The scheduler is executing Step Manager. TimeOut The scheduler is executing Step Manager,
and it has exceeded the estimated execu-tion time.
Stopped The scheduler is stopped (by the user). Confirmation The scheduler is showing the confirmation
window to the user
Step Manager State Machine Several states of Step Manager are shown in Figure 26.
Not Active Idle
Execute
Start Step / TAF.Run,TAF.StartTest
Running Single
Running Batch
Start Batch / TAF.Run,TAF.StartTest
when: Finished()
when: Finished()
Exit
Stop / TAF.Stop,TAF.Exit
Stop / TAF.Stop,TAF.Exit
Figure 26 – Step Manager State Diagram The explanation of each state can be found in Table 15. Table 15 – Step Manager State State Description Not Active There is no running instance of Step Man-
ager Idle Step Manager is running but there are no
steps being executed Running Single Step Manager is running only one step.
After the step finishes Step Manager re-turns to Idle
Running Batch Step Manager is running a sequence of steps
7.2.4. Components Design Summary After designing the sequence diagram of scheduler system, we move further to de-velop and implement the components. At this point, three main components need to be developed: Scheduler Server, Scheduler Agent and Step Manager. The data stor-age (Database and XML Storage) is used by these components and need to be devel-
![Page 60: Scheduling system for test automation frameworkScheduling system for test automation framework Citation for published version (APA): Wahyudi, D., & Technische Universiteit Eindhoven](https://reader033.fdocuments.in/reader033/viewer/2022051923/6011324fdb00301ad8544bee/html5/thumbnails/60.jpg)
oped and integrated inside them. The Step Manager modification is needed since currently there is no way to communicate with Step Manager (Command Line Inter-face needs to be added). The next chapter will describe the implementation of the components. Below is a short description of the implementation of each component: Scheduler Server:
o It provides data input as web application o It stores the data in the database and generated in the XML Storage o It updates the result of schedule execution to the database
Scheduler Agent: o It provides schedule execution (Control of Step Manager Execution) o It gets the data from the server o It puts the result of the schedule execution to the server
Step Manager o It provides the Activity Block execution (Control of TAF Execution) o It gets the Step Manager recipe as input o It gives the result of activity block execution as output.
■
![Page 61: Scheduling system for test automation frameworkScheduling system for test automation framework Citation for published version (APA): Wahyudi, D., & Technische Universiteit Eindhoven](https://reader033.fdocuments.in/reader033/viewer/2022051923/6011324fdb00301ad8544bee/html5/thumbnails/61.jpg)
8.Implementation Abstract – In this chapter, the implementation of the scheduler system components is described. These are divided in two groups: data and tool components. Data compo-nents consist of Database and XML Storage (Server and Agent). Tool components consist of Scheduler Server, Scheduler Agent, and Step Manager.
8.1 Data Components In this subchapter, we further describe the development of Database and XML Stor-age.
8.1.1. Database Database is used as data storage for server application. The database should contain the data classes listed:
Side PC – List of available test systems Command – Command Data from Scheduler Server to Scheduler Agent Schedule – Contains the schedule time and the link to activity block data Schedule Result – Contains the result of schedule execution Activity Block Template – Contains the sequence of activities Activity Template – Contains the link to TAF File
The database is shown in the Database Diagram in Figure 27. The database is made with the aim of integrating it with TE in the future. Scheduler System tables (Green color) will have a field to accommodate the integration with TE tables (Blue Color). Only Scheduler System tables are developed during this project.
Figure 27 – Database Diagram Legend: PK – Primary Key FK – Foreign Key Bold - Required
TE Scheduler System
![Page 62: Scheduling system for test automation frameworkScheduling system for test automation framework Citation for published version (APA): Wahyudi, D., & Technische Universiteit Eindhoven](https://reader033.fdocuments.in/reader033/viewer/2022051923/6011324fdb00301ad8544bee/html5/thumbnails/62.jpg)
8.1.2. XML Storage The XML Storage (Server and Agent) is needed for storing the data for Side PC. The Server XML storage is a shared folder in the server. The Agent XML Storage (in the Side PC) is the copy of the “Server/<SidePC>” folder from the Server XML Storage (see Figure 28). For every different SUT configuration, a different TAF file is needed since the SUT configuration is stored inside the TAF configuration. Therefore, the XML Storage is designed to have a subfolder for each Side PC. The details of the folder structure of Server and Agent XML Storage are listed below (bold – folder):
Template
<SidePC> TAF
<SmokeTest.taf> <ReliabilityTest.taf>
Server <SidePC>
Schedule.xml Command.xml <ScheduleId>
Recipe-<ScheduleId>.xml TAF
TAF Config <SmokeTest.taf> <ReliabilityTest.taf>
Result ScheduleResult.xml CommandResult.xml <ScheduleId>
Result-<ScheduleId>.xml TAF Result
<SmokeTest>–<DateTime> <ReliabilityTest>–<DateTime>
Figure 28 – XML Storage Structure
The explanation of each file and folder from the folder structure above can be seen in Table 16
Table 16 – XML Storage Folder Structure Type Folder / File Description Folder Template Folder in Server to store the template data Folder <SidePC> Subfolder of Template folder – it contains the data related
to a SidePC. The folder name is the Side PC name Folder TAF Subfolder of Template/<SidePC> folder to store the
TAF data File <SmokeTest.taf>
<ReliabilityTest.taf> TAF configuration files – they are used as input for TAF to execute an Activity. The file name is determined by the user
Folder Server Folder in Server to store the data for Side PC Folder <SidePC> Subfolder of Server folder – it contains the data related to
a Side PC. The folder name is the Side PC name File Schedule.xml The file contains the schedule data. The file is generated by
the Scheduler Server File Command.xml The file contains the command data. The file is generated by
the Scheduler Server Folder <ScheduleId>
Subfolder of Server/<SidePC> folder – – it contains the data related to a schedule. The folder name is the Id of the schedule from the database
File Recipe-<ScheduleId>.xml
The Step Manager Recipe File – it contains the activity block data
Folder TAF Subfolder of Server/<SidePC>/ <ScheduleId> folder used to organize TAF data
Folder TAF Config Subfolder of Server/<SidePC>/ <Schedu-leId>/TAF folder that contains TAF configuration files
Server XML Storage
Agent XML Storage (Side PC)
![Page 63: Scheduling system for test automation frameworkScheduling system for test automation framework Citation for published version (APA): Wahyudi, D., & Technische Universiteit Eindhoven](https://reader033.fdocuments.in/reader033/viewer/2022051923/6011324fdb00301ad8544bee/html5/thumbnails/63.jpg)
File <SmokeTest.taf> <ReliabilityTest.taf>
TAF configuration files – they are used as input for TAF to execute an Activity
Folder Result Subfolder of Server/<SidePC> folder – it contains the results of schedule execution. The Scheduler Agent puts the results in this folder.
File ScheduleResult.xml The file contains the results of schedule execution. The file is generated by the Scheduler Agent
File CommandResult.xml The file contains the results of command execution. The file is generated by the Scheduler Agent
Folder <ScheduleId>
Subfolder of Server/<SidePC>/Result folder – it contains the result data related to a schedule. The folder name is the Id of the schedule from the database
File Result-<ScheduleId>.xml
The Step Manager Result File – it contains the result data of activity block execution
Folder TAF Result The folder contains the result data of TAF execution. Scheduler Agent collects the result data in this folder.
Folder <SmokeTest>–<DateTime> <ReliabilityTest>–<DateTime>
The folders contain the result of TAF execution
8.2 Tool Components In this subchapter, further development for Scheduler Server, Scheduler Agent, and Step Manager are described. TAF is not explained in this chapter since no changes in TAF is required for this project.
8.2.1. Scheduler Server The implementation of Scheduler Server is described in this subchapter. The imple-mentation consists of website implementation as user input and File Watcher imple-mentation as result updates.
User Input The Scheduler Server is a web application. The website is built using the MVC (model-view-controller) pattern. The Scheduler Server uses Entity Framework to generate the model of database. The diagram of the MVC pattern can be found in Figure 29
Figure 29 – Model View Controller The explanation of the MVC pattern can be found below:
Controller It handles the user request from the webpage. It gathers the model, manipu-lates the model and shows the model in the view.
Model It is implemented using Entity Framework. The generation of the model is done automatically and synchronized with the database. Changes in the da-tabase are handled by generating the model again through Entity Framework (by running the custom tool).
View
Model
View
Controller Request
Browser (User)
Response
Manipulates
Updates
![Page 64: Scheduling system for test automation frameworkScheduling system for test automation framework Citation for published version (APA): Wahyudi, D., & Technische Universiteit Eindhoven](https://reader033.fdocuments.in/reader033/viewer/2022051923/6011324fdb00301ad8544bee/html5/thumbnails/64.jpg)
It is implemented by Razor / cshtml. The View handles the display format of the model.
The Scheduler Server website is shown in Figure 30
Figure 30 – Scheduler Server User Interface The overview of menus seen in the Figure 30 is described in more detail in Table 17. Each of the menus has its own class of MVC (Model – View – Controller). Table 17 – MVC Components MVC Components Components Responsibility SidePC To handle the SidePC CRUD in the database
To handle actions like e.g. start and stop Schedule To handle the Schedule CRUD in the database
To handle the generation of Schedule.xml To handle the generation of StepManagerRecipe.xml To update the ScheduleResult in the database from
ScheduleResult.xml Command (Hidden) To handle the communication with Side PC
To handle the generation of Command.xml To update the CommandResult in the database from
CommandResult.xml Activity Block To handle the ActivityBlock CRUD in the database
(Activity Block and Activity Block Detail) Activity To handle the Activity CRUD in the database
To handle TAF File Upload and Download
File Watcher File watcher functionality is needed to monitor the result of schedule execution. Scheduler Agent copies the result (a XML file) to the server. This file copy event triggers the File Watcher. The File Watcher executes the updateResult function, which parses the copied file and puts the result in the database.
![Page 65: Scheduling system for test automation frameworkScheduling system for test automation framework Citation for published version (APA): Wahyudi, D., & Technische Universiteit Eindhoven](https://reader033.fdocuments.in/reader033/viewer/2022051923/6011324fdb00301ad8544bee/html5/thumbnails/65.jpg)
The sequence diagram of File Watcher (Figure 31) is developed from the sequence diagram in Figure 24. Figure 31 shows the command result update and the schedule result update.
Scheduler Server Scheduler AgentDatabase Server XML Storage
FileWatcher(ScheduleResult.xml)
SaveData
CheckResult
Generate(Command.xml)
Command = DeleteResult
FileWatcherCommand.xml)
Update Result
Get(ScheduleResult.xml)
ScheduleResult.xml
Put(CommandResult.xml)
FileWatcher(CommandResult.xml)
Get(CommandResult.xml)
CommandResult.xml
Update Result
SaveData
Local XML Storage
Put(ScheduleData)
Copy(ScheduleResult.xml)
Copy(StepManagerResult.xml)
Copy(TAFResult)
Schedule Data Contain Schedule Result, Step Manager Result and TAF Result
copy(CommandResult.xml)
Figure 31 – Scheduler Server FileWatcher Two files need to be watched: CommandResult.xml and ScheduleResult.xml. CommandResult.xml It contains the result of command execution. Several commands with their possible result can be found in the Table 18.
Table 18 – Scheduler Command Result Command Result NewSchedule Success / Failed Stop Success / Failed Start Success / Failed GetState Several possible value:
Idle RunningStepManager TimeOut Stopped
![Page 66: Scheduling system for test automation frameworkScheduling system for test automation framework Citation for published version (APA): Wahyudi, D., & Technische Universiteit Eindhoven](https://reader033.fdocuments.in/reader033/viewer/2022051923/6011324fdb00301ad8544bee/html5/thumbnails/66.jpg)
Confirmation PutResult Success / Failed RemoveResult Success / Failed
ScheduleResult.xml It contains the result of schedule execution. The possible result values are listed be-low:
0 – Success 1 – Failed 2 – In Progress 3 – Not Applicable 4 – Skipped 5 – Stopped 6 – Interrupted / Suspended 7 – Scheduled
8.2.2. Scheduler Agent In this chapter, Scheduler Agent implementation is documented. The implementation consists of Time Trigger implementation as schedule execution and File Watcher implementation as XML Storage monitoring. The user interface of Scheduler Agent is a small menu in the task bar (shown in Fig-ure 32).
Figure 32 – Scheduler Agent User Interface
Time Trigger The time trigger functionality is realized with timer. The scheduler can only execute one activity block at a particular time. Because of this, the scheduler will only keep track of the current activity block. After it finishes, the next activity block is sched-uled and executed. The sequence diagram of time trigger functionality is shown in Figure 33.
Scheduler Agent
![Page 67: Scheduling system for test automation frameworkScheduling system for test automation framework Citation for published version (APA): Wahyudi, D., & Technische Universiteit Eindhoven](https://reader033.fdocuments.in/reader033/viewer/2022051923/6011324fdb00301ad8544bee/html5/thumbnails/67.jpg)
Figure 33 – Scheduler Agent Time Trigger
File Watcher File Watcher is needed to trigger the Scheduler Command. File Watcher monitors the Command.xml that is created by Scheduler Server. When file creation event happens File Watcher triggers the Scheduler Agent to execute the Command.xml. After the execution, the CommandResult.xml is copied to the server and the Command.xml is deleted. The commands and their description are listed in Table 19. Table 19 – Scheduler Command Command Parameter Description NewSchedule ScheduleId To indicate there is a new schedule Stop To instruct the SidePC to stop Start ScheduleId
(Optional) To instruct the SidePC to start the schedule
GetState To get the state of the scheduler in the Side PC PutResult ScheduleId To instruct the SidePC to put the result to the
server RemoveResult ScheduleId To instruct the SidePC to remove the result since
the server already has the result The sequence diagram of File Watcher is shown in Figure 34. The sequence diagram shows the NewSchedule command as an example.
![Page 68: Scheduling system for test automation frameworkScheduling system for test automation framework Citation for published version (APA): Wahyudi, D., & Technische Universiteit Eindhoven](https://reader033.fdocuments.in/reader033/viewer/2022051923/6011324fdb00301ad8544bee/html5/thumbnails/68.jpg)
Figure 34 – Scheduler Agent File Watcher
8.2.3. Step Manager Step Manager needs to be modified to support communication with Scheduler Agent. Command line interface was chosen to provide the command input to Step Manager. This subchapter describes the implementation of command line interface in Step Manager. The ‘Required’ condition added in Step Manager is also described in this subchapter.
Command Line Interface Command line parameter is used to provide command input to Step Manager. In the following example, we shall demonstrate this and show how to issue a ‘stop’ com-mand. Step Manager is started with the following command:
StepManager.exe –auto This creates one running instance of Step Manager. In order to stop it we can issue the following command:
StepManager.exe -stop This command should stop the running Step Manager, but this is not possible since this command executes the second Step Manager instance. To solve this problem, a way needs to be found to send the ‘stop’ command to the first instance of Step Man-ager. To realize this, TCP/IP connection is added to Step Manager. It allows the sec-ond instance of Step Manager to communicate with the first instance. The sequence diagram of this mechanism can be seen in Figure 35.
![Page 69: Scheduling system for test automation frameworkScheduling system for test automation framework Citation for published version (APA): Wahyudi, D., & Technische Universiteit Eindhoven](https://reader033.fdocuments.in/reader033/viewer/2022051923/6011324fdb00301ad8544bee/html5/thumbnails/69.jpg)
Figure 35 – Step Manager Sequence Diagram The Step Manager Command parameters are listed below. The blue color shows that a command needs to be developed.
Table 20 – Step Manager Command Parameters Command Parameters Description Allow
Combination -auto To start the execution of steps au-
tomatically (without this com-mand, Step Manager will not start executing the steps)
Yes
-recipeFile=<SMRecipe.xml> To set the recipe file Yes -resultFile=<SMResult.xml> To set the result file Yes -getState To get Step Manager state No -closeAfter To close Step Manager after exe-
cuting the steps Yes
-stop To stop the execution No -exit To exit Step Manager No
Required Condition There are two XML files related to Step Manager: Recipe.xml and Result.xml. Table 21 shows the file interface for Step Manager. The ‘Required’ condition field was added to the recipe. This field is an indication whether Step Manager can continue to the next step or not when the current step fails. The example of these files can be found in the Appendix D. Table 21 – Step Manager File Interfaces Files Description Recipe.xml The recipe file provides the input for Step Manager Result.xml The result file stores the output from Step Manager ■
![Page 70: Scheduling system for test automation frameworkScheduling system for test automation framework Citation for published version (APA): Wahyudi, D., & Technische Universiteit Eindhoven](https://reader033.fdocuments.in/reader033/viewer/2022051923/6011324fdb00301ad8544bee/html5/thumbnails/70.jpg)
![Page 71: Scheduling system for test automation frameworkScheduling system for test automation framework Citation for published version (APA): Wahyudi, D., & Technische Universiteit Eindhoven](https://reader033.fdocuments.in/reader033/viewer/2022051923/6011324fdb00301ad8544bee/html5/thumbnails/71.jpg)
9.Verification & Validation Abstract – In this chapter, the verification and validation phase is documented. The system design is verified and the system requirements are validated.
9.1 Verification The system design is verified by comparing the implementation against the design. The author performs the verification. Table 22 shows the table for the design verifi-cation. Table 22 – Design Status Design Status TAF – Execution Pass TAF – Stop Pass TAF – Result Pass Step Manager – Execution Pass Step Manager - TCP/IP Connection (First and Second Application) Pass Step Manager - TCP/IP Connection – Command Line Arguments Pass Step Manager - Command Line – recipeFile Pass Step Manager - Command Line – resultFile Pass Step Manager - Command Line – getState Pass Step Manager - Command Line – closeAfter Pass Step Manager - Command Line – stop Pass Step Manager - Command Line – start Pass Step Manager - Command Line – close Pass Step Manager - RecipeFile – StopCommand Pass Step Manager - RecipeFile – Required Pass Step Manager – Result Pass Scheduler Agent – Read schedule.xml Pass Scheduler Agent – Scheduling Execution Pass Scheduler Agent – Write scheduleResult.xml Pass Scheduler Agent – File Watcher Pass Scheduler Agent – Get File Pass Scheduler Agent – Command Pass Scheduler Agent – Put File Pass Database – Design Pass File Store – Design Pass Scheduler Server – Schedule – Database CRUD Pass Scheduler Server – Activity Block Template – Database CRUD Pass Scheduler Server – Activity Template – Database CRUD Pass Scheduler Server – TAF – File Upload Pass Scheduler Server – Step Manager – XML CRUD Pass Scheduler Server – Schedule – XML CRUD Pass Scheduler Server – Command Pass Scheduler Server – Get Result Pass Scheduler Server – Layout and other User Interface Pass
9.2 Validation The implementation is compared against the requirements. The user performs the validation. Table 23 shows the table for requirement validation.
![Page 72: Scheduling system for test automation frameworkScheduling system for test automation framework Citation for published version (APA): Wahyudi, D., & Technische Universiteit Eindhoven](https://reader033.fdocuments.in/reader033/viewer/2022051923/6011324fdb00301ad8544bee/html5/thumbnails/72.jpg)
Table 23 – Requirement Status Requirement Status SSTAF.Activity Pass SSTAF.Activity.Execution Pass SSTAF.Activity.FileStore Pass SSTAF.Activity.Result Pass SSTAF.Activity.EstimatedTime Pass SSTAF.Activity.Stop Pass SSTAF.ActivityBlock Pass SSTAF.ActivityBlock.Popup Pass SSTAF.ActivityBlock.Stop Pass SSTAF.ActivityBlock.EstimatedTime Pass SSTAF.ActivityBlock.Result Pass SSTAF.ActivityBlock.Template Pass SSTAF.Schedule Pass SSTAF.Schedule.Access Pass SSTAF.Schedule.ActivityBlock.Parameter Pass SSTAF.Schedule.OverlappingSchedule Pass SSTAF.Schedule.OverlappingSchedule.Warning Pass SSTAF.Schedule.AdjustSchedule Pass SSTAF.Schedule.AdjustScheduleConstraint Pass SSTAF.Schedule.ResponseTime Pass SSTAF.SidePC.IsAlive Pass SSTAF.Reporting.Result Pass SSTAF.Reporting.GanttChart Not done ■
![Page 73: Scheduling system for test automation frameworkScheduling system for test automation framework Citation for published version (APA): Wahyudi, D., & Technische Universiteit Eindhoven](https://reader033.fdocuments.in/reader033/viewer/2022051923/6011324fdb00301ad8544bee/html5/thumbnails/73.jpg)
10. Project Management Abstract – In this chapter, the project management tasks are documented. The way of working, project schedule, and risk management are described.
10.1 Way of working The project was managed using agile software development method. The idea with this method is to develop a system through repeated cycles (iterations) in smaller portions of time (incremental). This way of working decomposes the system func-tionality into smaller chunks. A chunk of functionality is delivered per iteration. Up-dates on the documentation were also made per iteration. There are three big phases of the project: inception, development and test. Each phase has its own deliverables. In the inception phase, the deliverables are the document and presentation. Rapid meetings with the users were held in this phase. Later, weekly meetings were held. The meetings were held to define the requirements and the scope of the project. Re-quirement Document and Design Document were made as the deliverable of the first stage. Research about the common off the shelf tools and their implementation was also done in this phase. In the development phase, the deliverable is the software system. Two weekly soft-ware demos with the customer were held in this phase. The demos had two goals:
To update the user with the status To get the feedback about the project direction
After getting the customer feedback, the software was changed and the document was updated based on the changes. In the testing phase, the deliverables are the testing documents. Tests have been done with the users to determine whether the system functions correctly. Further details of the breakdown of the iteration can be seen in the Figure 36.
10.2 Project Schedule The project schedule was changed several times since there was new information that changed the project direction. The final version of the project Gantt chart can be found in Figure 36.
Figure 36 – Project Gantt Chart
![Page 74: Scheduling system for test automation frameworkScheduling system for test automation framework Citation for published version (APA): Wahyudi, D., & Technische Universiteit Eindhoven](https://reader033.fdocuments.in/reader033/viewer/2022051923/6011324fdb00301ad8544bee/html5/thumbnails/74.jpg)
Legend:
Task
Documentation
Presentation and Meeting
Evaluation & Feedback
10.3 Risk Management Several issues and risks are investigated. Table 24 lists several risks that might affect the project. Table 24 – Issue and Risk Issues or Risks Likelihood Critical Mitigation Strategy Customer do not have time High Medium Plan early Customer changes his mind Medium High Record every decision Organization problem High Medium Management Awareness Cannot use the SUT Low Low Plan early Cannot solve the technical problem
Low High Get Expert Advice
Network problem Medium Medium Help from the IT system management
Communication Problem Medium High Frequent update and communication Misinterpretation Medium High
Expectation difference Medium High Integration necessary with TE
Low Medium Consider the integration in the Design
Get more requirements in the later stage
Medium High Frequent Demo
Not enough time Medium High Plan & Requirement man-agement
New tool to be used / inte-grate
Medium High Requirement management
New information that en-dangered the project
Medium High
Supervisor do not satisfied with the performance
Medium High Frequent update and communication
Project Cancellation Medium High - Demo Failed Medium High Prepare and Revise all the
step Presentation Failed Medium High Prepare, get comment and
feedback, Practice before the presentation
■
![Page 75: Scheduling system for test automation frameworkScheduling system for test automation framework Citation for published version (APA): Wahyudi, D., & Technische Universiteit Eindhoven](https://reader033.fdocuments.in/reader033/viewer/2022051923/6011324fdb00301ad8544bee/html5/thumbnails/75.jpg)
11. Conclusion Abstract – In this chapter, the conclusions of the SSTAF project are described: the results, the lessons learned, and the future works.
11.1 Results The scheduler system for Test Automation Framework has gone through all the phas-es of development cycle. The requirements were defined with the user in the starting phase of the project. Then, several common off the shelf tools and related Philips tools were explored in the investigation phase. After the selections, several architec-ture and design decisions were made in the design phase. Then, the implementation of the system was done, followed by verification and validation phase. The project management was done in Agile with iterations. Periodically the results were presented to the customer to provide updates and obtain feedback. This was done to ensure that the project was on the track. Many architecture decisions were made in this project. In every architecture decision, the advantages and the disad-vantages were investigated and the decision was taken based on the analysis. The design decisions taken in this project provide several advantages listed below:
By using TAF o The Scheduler System can be used by other Philips organization
that use TAF as their testing framework By using Step Manager
o The Scheduler System reuse the Factory Tooling that make the au-tomatic installation possible
By having Activity Block (Template) o The tester can reuse the sequence of activities
By having Smart Agent o The scheduler system is more reliable and it can operate without
server connection o The schedule execution will have no network delay
By having Website and Database o The system is easily accessible by User. o The future integration with Test Environment (TE) can be easily
realized Several conclusions related to this project are listed below:
Scheduler Server was created to handle the user input, using web technolo-gy.
Database was designed and used to store the schedule data for the server. XML Storage was designed and used to store the schedule data for the Side
PC. Scheduler Agent was created to handle the time trigger and step manager
control. Step Manager was modified to handle the command and the “required” con-
dition. Connection to TAF was made to handle the execution of TAF Script.
11.2 Lessons Learned The SSTAF project had both technical and non-technical challenges. As an important technical challenge, SSTAF had to accommodate different available applications. Complex applications were involved and it was needed to understand how they worked. The strategy was chosen to reuse the available applications as much as pos-sible instead of developing the new ones. As an example of non-technical challenges,
![Page 76: Scheduling system for test automation frameworkScheduling system for test automation framework Citation for published version (APA): Wahyudi, D., & Technische Universiteit Eindhoven](https://reader033.fdocuments.in/reader033/viewer/2022051923/6011324fdb00301ad8544bee/html5/thumbnails/76.jpg)
SSTAF faced some organizational problems that had an impact on the project. How-ever, the project was continued with the same initiative and goal. Regarding the technical challenge, it was learned that it was necessary to allocate enough time to understand the available tool in depth. Most of the tools used in Philips had been used for a long time and they were stable. Only small changes could have been done at a time since the knowledge of the tools was quite limited. Regarding the non-technical challenge, it was learned that the organizational issues could influence the project greatly. It was learned that these organizational issues needed to be solved in the early stage of the project. This was necessary to ensure the continuation of the project. Another non-technical challenge was the communication. The project related infor-mation needed to be given to the stakeholders and their feedback needed to be gath-ered and synchronized. This was not always an easy task to do. Following the meet-ings with stakeholders, the minutes were handed out by email but that turned out to be insufficient. So it became necessary to update the stakeholders by short direct communication. The communication with the customers as one of the most important stakeholders is the key to make the project successful. As the summary, it was essential to understand and detect the risks early. Investiga-tion and feasibility studies were necessary as a mitigation strategy to remove or at least reduce the detected risks. However, even though it was important to overcome the risks, the crucial tasks with lower risk were given higher priority. This prioritiza-tion and risk analysis helped the author to handle the project successfully.
11.3 Future Works The TE future integration is necessary. The Scheduler System has been prepared for the integration. The integration will be through the database. The web application of the Scheduler System can be reused and integrated in TE. The automatic installation of the new Allura System is currently still in development. It is a part of Factory Tooling project. Future integration with Factory Tooling is nec-essary in order to reach the automatic installation. Design of Factory Tooling is aligned with SSTAF project. With this in mind, the integration in the future can be done easily. ■
![Page 77: Scheduling system for test automation frameworkScheduling system for test automation framework Citation for published version (APA): Wahyudi, D., & Technische Universiteit Eindhoven](https://reader033.fdocuments.in/reader033/viewer/2022051923/6011324fdb00301ad8544bee/html5/thumbnails/77.jpg)
Appendix A. COTS Selection The comparison between the types of COTS tools against the functionalities can be seen below. Table 25 – Domain against functionality Tool \ Functionality Scheduling RunningTask
Automatically Controlling Access
Reporting
Calendar Yes No Yes Yes Job Scheduler Yes Yes Partial Partial Continuous Integration Yes Yes Partial Partial For the analysis of COTS tools that can serve the purpose, two types of tools are con-sidered. They are the job scheduler and the continuous integration. The calendar is not included on the short list because the calendar is not intended to execute a task/activity. List of all the job schedulers and continuous integration products is found in [7] and [8]. The selection was made in several stages based on the infor-mation available.
Job Scheduler To select the scheduler, several criteria were inspected in order to match the project requirements. In the first round, three criteria were used. Note that the criteria match the features defined in [7].
Windows support. The majority of servers within iXR run under the Windows Operating Sys-tem. Support and maintenance of a product is much easier when it runs on the same OS.
Low cost (No ERP support). The lack of ERP support is used as indication of low cost. Investigation of the price is done after this condition is met.
Support for Role Based Security (RBS). RBS is considered the same as access control functionality.
Table 26 – Job Scheduler First Round Selection
Job Scheduler Windows support Low Cost (No ERP) Role Based Secu-
rity ActiveBatch Yes No Yes
AutoSys Yes No Unknown
Batchman Unknown No Unknown
Control-M Yes No Yes
Dollar Universe Yes No Yes
JAMS Scheduler Yes No Yes
JobScheduler Yes No No
OpCon Yes No Yes
Skybot Scheduler Yes No Yes
Sked No No No
Tivoli Workload Sched-uler
Yes No Yes
Grid Engine Unknown Yes Open Source Unknown
Hinemos Yes Yes Unknown Yes
JobServer Yes Yes USD $2,995.00 - Yes
![Page 78: Scheduling system for test automation frameworkScheduling system for test automation framework Citation for published version (APA): Wahyudi, D., & Technische Universiteit Eindhoven](https://reader033.fdocuments.in/reader033/viewer/2022051923/6011324fdb00301ad8544bee/html5/thumbnails/78.jpg)
Up to 100 Jobs
Juice Box Yes Yes Free Partial
Platform LSF Yes Yes Unknown Unknown
ProActive Parallel Suite Yes Yes Open Source Yes
Schedulix Yes Yes Open Source No
VisualCron Yes Yes $1,999.00 with 1 year support
Unknown
Cisco Tidal Enterprise Scheduler
Yes Yes Unknown SAP, PeopleSoft
After the first round, there were still too many job schedulers to consider in more depth. Therefore, other criteria were introduced:
Intended use Is the product built for the same or similar task? A big deviation of the in-tended use means that the product will have too many capabilities that we do not need or lacking essentials that we do need.
After the second round, only two tools were considered for the selection: Juice Box and ProActive Parallel Suite.
Continuous Integration Tool To select the scheduler, several criteria of the continuous integration tool from [8] were inspected to match the project requirements.
Supported Platform It must be Windows or Cross-platform for the same reasons as described for the Job Server.
Development Language iXR department has many C, C++, C# and batch programmers. For ease of maintenance, the selected tool should support any of these languages. Philips does not have many people for other language skills, so Java and Py-thon are out.
Table 28 – Continuous Integration First Round Selection
Scheduler \ Criteria Platform Development lan-guage
AnthillPro Yes Yes
Apache Continuum Yes No
Apache Gump No No
AppVeyor CI No Yes
AutoDE No Yes
Table 27 – Job Scheduler Second Round Selection Job Scheduler Intended Use
Grid Engine It is meant for grid computing computer cluster software system. No need for cluster
Hinemos It manages several servers, computers, and other devices, as if it were only managing one single computer. No need for this func-tionality
Juice Box It is a simple job scheduler but not able to extend
Platform LSF It is for distributed HPC environments. No need for High Per-formance Computing
ProActive Parallel Suite It uses XML in the scheduler. It focuses on Parallel processing for Grid and Cloud Computing.
Cisco Tidal Enterprise Scheduler It focuses on big data
![Page 79: Scheduling system for test automation frameworkScheduling system for test automation framework Citation for published version (APA): Wahyudi, D., & Technische Universiteit Eindhoven](https://reader033.fdocuments.in/reader033/viewer/2022051923/6011324fdb00301ad8544bee/html5/thumbnails/79.jpg)
Automated Build Studio Yes No
Bamboo Maybe Yes
Build Forge Yes Yes
BuildBot No Yes
BuildMaster Yes Yes
CABIE No No
CircleCI No No
node.ci No No
Codeship No No
CruiseControl Yes Yes
CruiseControl.NET Yes Yes
Draco.NET Yes No
Drone.io No Yes
EasyCIS Yes No
ElectricCommander Yes Yes
Go Yes Yes
Integrity No Yes
Jenkins/Hudson Maybe Yes
Koality Yes Yes
Lordui Yes Yes
LuntBuild Maybe Yes
QuickBuild Yes Yes
Semaphore No Yes
Shippable No No
Strider No Yes
Team Foundation Server Yes Yes
TeamCity Maybe Yes
Tinderbox Yes No
Travis-CI No Yes
After the first round, there are still too many continuous integration tools to consider for detailed analysis. The following criteria were used for further selection:
Scheduler support It was investigated from the description in [8] whether the tool has a job scheduler. The keyword search is the “nightly build”. If the tool has this fea-ture, then it can be concluded that it has scheduler or time trigger functional-ity.
Expected Cost The one-time fee should not exceed $2000 and/or the yearly fee not more than $1000/year.
![Page 80: Scheduling system for test automation frameworkScheduling system for test automation framework Citation for published version (APA): Wahyudi, D., & Technische Universiteit Eindhoven](https://reader033.fdocuments.in/reader033/viewer/2022051923/6011324fdb00301ad8544bee/html5/thumbnails/80.jpg)
Tool Comparison The next criteria to be considered was the usage in the Philips organization, timeline presentation, and user interface extensibility. The criteria of user interface extensibil-ity were added after the demo with a system test in mobile environment.
Usage in the organization If the tool is already used within the organization this is a plus (+), because we already have people that are familiar with the product, zero (0) if it is not.
Timeline presentation The tool scores minus (-) if the tool does not support timeline presentation or if it takes a lot of effort to add that. It scores zero (0) if the product can be extended easily with additional code. It scores plus (+) if it supports timeline representation available.
UI Extensibility The product scores minus (-) if it cannot, or with great effort, be extended to support additional UI screens, zero (0) if moderate effort is needed, plus (+) if it is easy to extend.
Table 29 – Continuous Integration Second Round Selection
Continuous Integration Scheduler Expected Cost
AnthillPro Yes
Pricing starts at $999 USD for small teams
Bamboo Needs some trick 10 Agents $250 USD/month
Build Forge Yes
User License USD $823.00. Can be integrated with ClearCase and ClearQuest.
BuildMaster Yes Up to 10 users $1999 per year
CruiseControl Difficult open source, BSD-style license
CruiseControl.NET Needs some trick open source, BSD-style license
ElectricCommander Yes Already bought, no extra cost
Go Unknown Up to 100 Process - $5,000.00
Jenkins/Hudson Yes open source, already used
Koality No Unknown
Lordui No Unknown
LuntBuild Yes Open Source
QuickBuild Unknown
Commercial License 3600 USD per physical site
Team Foundation Server Done by Script
One device client access for Visual Studio Team Foundation Server 2013 $499.00
TeamCity Yes Up to 100 build agents $1,999
Table 30 – Tool Selection
Scheduler Tool Used in the organization Timeline
Presentation UI Extensibility Overall
Score Juice Box 0 - - - -
ProActive Parallel Suite 0 0 0 0
CruiseControl or Cruise-Control.NET
0 0 0 0
LuntBuild 0 0 0 0
Build Forge 0 0 - -
ElectricCommander + 0 0 +
Jenkins/Hudson + 0 + ++
![Page 81: Scheduling system for test automation frameworkScheduling system for test automation framework Citation for published version (APA): Wahyudi, D., & Technische Universiteit Eindhoven](https://reader033.fdocuments.in/reader033/viewer/2022051923/6011324fdb00301ad8544bee/html5/thumbnails/81.jpg)
COTS tool selection produces Jenkins as the result. It is an open source project. It has many plugins and has different customization and it is easy to extend. However, it also might take more time to customize and extend Jenkins to have the desired func-tionality. ■
![Page 82: Scheduling system for test automation frameworkScheduling system for test automation framework Citation for published version (APA): Wahyudi, D., & Technische Universiteit Eindhoven](https://reader033.fdocuments.in/reader033/viewer/2022051923/6011324fdb00301ad8544bee/html5/thumbnails/82.jpg)
![Page 83: Scheduling system for test automation frameworkScheduling system for test automation framework Citation for published version (APA): Wahyudi, D., & Technische Universiteit Eindhoven](https://reader033.fdocuments.in/reader033/viewer/2022051923/6011324fdb00301ad8544bee/html5/thumbnails/83.jpg)
Appendix B. Design Evolution There are many changes of architecture during the project lifetime. New information is collected and it causes some changes in the architecture. The evolution of the de-sign can be seen below.
Initial Design Description: This design was made with the assumption that TAF should be the only connection to the SUT. This architecture makes a clear functionality separation between TAF and the scheduler. Reason not to use: It needs time trigger functionality in the Side PC
![Page 84: Scheduling system for test automation frameworkScheduling system for test automation framework Citation for published version (APA): Wahyudi, D., & Technische Universiteit Eindhoven](https://reader033.fdocuments.in/reader033/viewer/2022051923/6011324fdb00301ad8544bee/html5/thumbnails/84.jpg)
Alternative Design with Server and Agent Description: Client server architecture is needed since the scheduler needs time trigger function-ality on the side PC. The scheduler agent should receive the schedule and execute TAF. Reason not to use: The server components need to be modular.
![Page 85: Scheduling system for test automation frameworkScheduling system for test automation framework Citation for published version (APA): Wahyudi, D., & Technische Universiteit Eindhoven](https://reader033.fdocuments.in/reader033/viewer/2022051923/6011324fdb00301ad8544bee/html5/thumbnails/85.jpg)
Alternative Design with TE (Test Environment) Description: Since there is a planner tool being developed to schedule manual tests, it is quite clear that both manual and automatic tests should be integrated. The overview of the architecture can be seen in the diagram. Reason not to use: The management decided to build ASPTE and TE separately. TE Integration is not going to be done in this project timeline.
![Page 86: Scheduling system for test automation frameworkScheduling system for test automation framework Citation for published version (APA): Wahyudi, D., & Technische Universiteit Eindhoven](https://reader033.fdocuments.in/reader033/viewer/2022051923/6011324fdb00301ad8544bee/html5/thumbnails/86.jpg)
Alternative Design with Web Service without TE Description: Web Service was made to supply the scheduler agent with the necessary data. Reason not to use: File interface through file sharing is considered a better option than Web Service since Web Service is not built for “push” mechanism.
■
![Page 87: Scheduling system for test automation frameworkScheduling system for test automation framework Citation for published version (APA): Wahyudi, D., & Technische Universiteit Eindhoven](https://reader033.fdocuments.in/reader033/viewer/2022051923/6011324fdb00301ad8544bee/html5/thumbnails/87.jpg)
Appendix C. Scheduler System Swim Lane Diagram
The flow or swim lane diagram is made to show how components interact with each other. The Swim Lane Diagram (Figure 20) is developed from the activity diagram shown in Figure 16.
Figure 37 – Swim Lane Diagram Each of the elements shown in the Swim Lane diagram is described in short detail in Table 31 Table 31 – Swim Lane Element Swim Lane Element
Component Description
![Page 88: Scheduling system for test automation frameworkScheduling system for test automation framework Citation for published version (APA): Wahyudi, D., & Technische Universiteit Eindhoven](https://reader033.fdocuments.in/reader033/viewer/2022051923/6011324fdb00301ad8544bee/html5/thumbnails/88.jpg)
Start TAF in UserPC Starting point of the system Create TAF Test File
TAF in UserPC Create TAF Test File using the TAF software in the user PC
Test Step for TAF
TAF in UserPC, TAF in SidePC
The TAF configuration file (XML)
CRUD Activity Browser in UserPC Create, Read, Update, or Delete the Activity Choose Test Step for TAF
Browser in UserPC The upload process of TAF configuration file to the server
CRUD Activity Block
Browser in UserPC Create, Read, Update, or Delete of the Activity Block
CRUD Schedule
Browser in UserPC Create, Read, Update, or Delete the Schedule
Save Data Scheduler Server Save data to database Generate File Scheduler Server Generate XML File Database Scheduler Server The database storage Command XML Storage,
Scheduler Agent in SidePC
The command data (XML)
Schedule XML Storage, Scheduler Agent in SidePC
The schedule data (XML)
Step Manager Recipe
XML Storage, Step Manager in SidePC
The step manager data (XML)
Uploaded Test Step for TAF
XML Storage The TAF configuration file (XML)
Get Server Data Scheduler Agent in SidePC
Get data from the server to the SidePC
Execute Com-mand
Scheduler Agent in SidePC
Execute command data
Schedule Time Trigger
Scheduler Agent in SidePC
Register the schedule data in the time trigger
Check Step Manager
Scheduler Agent in SidePC
Check Step Manager state
Execute Step Manager
Scheduler Agent in SidePC
Initiate the execution of Step Manager
Step Manager Execution
Step Manager in SidePC
Process of running Step Manager
TAF Execution TAF in SidePC Process of running TAF Command Re-sult
Scheduler Agent in SidePC, XML Storage
The command result data
Schedule Result Scheduler Agent in SidePC, XML Storage
The schedule result data
Step Manager Result
Step Manager in SidePC, XML Storage
Step Manager result data
TAF Result TAF in SidePC, XML Storage
TAF result data
Put Server Data Scheduler Agent in SidePC
Put data from the SidePC to the server
Update Result Scheduler Server Update results to Database Show Result in Report
Scheduler Server Show the results in Reporting
End Scheduler Server End point of the system ■
![Page 89: Scheduling system for test automation frameworkScheduling system for test automation framework Citation for published version (APA): Wahyudi, D., & Technische Universiteit Eindhoven](https://reader033.fdocuments.in/reader033/viewer/2022051923/6011324fdb00301ad8544bee/html5/thumbnails/89.jpg)
Appendix D. XML Files There are many xml files are needed for SSTAF. The detailed files can be seen be-low.
SCHEDULER AGENT FILES
Schedule.XML The example of the Schedule.xml is shown below. One Schedule.xml only made for scheduling one SUT.
<Scheduler> <MachineName>TestMachine1</MachineName> <IPAddress>localhost</IPAddress> <Schedule> <Name>Test1</Name> <Person>Djohan Wahyudi</Person> <RecipeFile>MCIEStepManagerRecipe.xml</RecipeFile> <ResultFile>MCIEStepManagerResult.xml</ResultFile> <PlannedDateTime>2014-05-21T16:15:00</PlannedDateTime> </Schedule> <Schedule> <Name>Test2</Name> <Person>Djohan Wahyudi</Person> <RecipeFile>MCIEStepManagerRecipe1.xml</RecipeFile> <ResultFile>MCIEStepManagerResult1.xml</ResultFile> <PlannedDateTime>2014-05-21T16:16:00</PlannedDateTime> </Schedule> … </Scheduler>
ScheduleResult.XML The example of the scheduleResult.xml is shown below. The blue color shows the difference between the Schedule.xml and the ScheduleResult.xml.
<Scheduler> <MachineName>TestMachine1</MachineName> <IPAddress>localhost</IPAddress> <LastPerformedSchedule>1</LastPerformedSchedule> <Result>Successful</Result> <State>Idle</State> <Schedule> <Name>Test1</Name> <Person>Djohan Wahyudi</Person> <RecipeFile>MCIEStepManagerRecipe.xml</RecipeFile> <ResultFile>MCIEStepManagerResult.xml</ResultFile> <PlannedDateTime>2014-05-21T16:15:00+02:00</PlannedDateTime> <Result>Successful</Result> <StepManager> <StartDateTime>2014-05-21T16:37:36.6000096+02:00</StartDateTime> <StopDateTime>2014-05-21T16:39:38.8774344+02:00</StopDateTime> <Result>Successful</Result> </StepManager> <StepManager> <StartDateTime>2014-05-21T17:39:36.6000096+02:00</StartDateTime> <StopDateTime>2014-05-21T17:41:46.8774344+02:00</StopDateTime> <Result>Successful</Result> </StepManager> </Schedule> <Schedule> <Name>Test2</Name> <Person>Djohan Wahyudi</Person> <RecipeFile>MCIEStepManagerRecipe1.xml</RecipeFile> <ResultFile>MCIEStepManagerResult1.xml</ResultFile> <PlannedDateTime>2014-05-21T16:16:00</PlannedDateTime> </Schedule> … </Scheduler>
![Page 90: Scheduling system for test automation frameworkScheduling system for test automation framework Citation for published version (APA): Wahyudi, D., & Technische Universiteit Eindhoven](https://reader033.fdocuments.in/reader033/viewer/2022051923/6011324fdb00301ad8544bee/html5/thumbnails/90.jpg)
Command.XML and CommandResult.XML The example of the Command.xml and CommandResult.xml is shown below. The ExecutedTime and the Result field in the CommandResult.xml are filled af-ter the execution (It was shown by the blue color).
<SidePC> <Name>SUT1</Name> <IP>192.121.121.121</IP> <Command> <Id>b7a9011b-3064-49ef-a0bd-b79aecabe90c</Id> <Command>NewSchedule</Command> <Parameter>9fef45ae-a07e-4080-ae5c-f879b0b52573</Parameter> <Time>2014-07-28T10:55:03.3628112+02:00</Time> <ExecutedTime /> <Result>0</Result> </Command> <Command> <Id>2fc405b0-e01a-438f-966f-e67c8107ed82</Id> <Command>NewSchedule</Command> <Parameter>ca21fec1-99c1-4676-bf6a-cb30dd11bdf4</Parameter> <Time>2014-07-28T10:52:24.27</Time> <ExecutedTime /> <Result>0</Result> </Command> .. </SidePC>
STEP MANAGER FILES
Recipe.XML Below is the example of recipe.xml. The blue color shows the addition needed to be done to recipe.xml. Required default value is true so the step will stop when the test failed. TAF in the recipe is just an argument for the batch file.
<Recipe> <Name>M-Cabinet</Name> <Step> <Name>Automatic step 1</Name> <Action> <Name>Automatic step 1</Name> <Flow>Automatic</Flow> <ExecutableName>z:\stepmanager\tools\runtafbatch.cmd</ExecutableName> <Arguments>PrepareHostPC.taf 0040_PrepareHostPC</Arguments> <TimeToExecute>100000</TimeToExecute> <HelpPage>empty.html</HelpPage> <Repetitions>1</Repetitions> <Required>false</Required> </Action> </Step> <Step> <Name>Manual step 1</Name> <Action> <Name>Manual step 1</Name> <Flow>Manual</Flow> <HelpPage>empty.html</HelpPage> </Action> </Step> … </Recipe>
Result.XML The example of result.xml is shown below. The blue color shows the difference between recipe.xml and result.xml. There is no change required in the result.xml.
<Recipe> <Name>M-Cabinet</Name> <LastPerformedStep>6</LastPerformedStep> <LastPerformedAction>0</LastPerformedAction> <State>1</State>
![Page 91: Scheduling system for test automation frameworkScheduling system for test automation framework Citation for published version (APA): Wahyudi, D., & Technische Universiteit Eindhoven](https://reader033.fdocuments.in/reader033/viewer/2022051923/6011324fdb00301ad8544bee/html5/thumbnails/91.jpg)
<Result>Successful</Result> <Step> <Name>Automatic step 1</Name> <Result>Successful</Result> <Action> <Name>Automatic step 1</Name> <Flow>Automatic</Flow> <ExecutableName>sleep.exe</ExecutableName> <Arguments>3</Arguments> <TimeToExecute>100000</TimeToExecute> <Result>Successful</Result> <StartDateTime>2014-05-14T16:04:30.9174141+02:00</StartDateTime> <StopDateTime>2014-05-14T16:04:34.0834141+02:00</StopDateTime> <HelpPage>empty.html</HelpPage> <Repetitions>1</Repetitions> <Required>false</Required> </Action> <Action> <Name>Automatic step 1</Name> <Flow>Automatic</Flow> <ExecutableName>sleep.exe</ExecutableName> <Arguments>3</Arguments> <TimeToExecute>100000</TimeToExecute> <Result>Successful</Result> <StartDateTime>2014-05-14T17:00:16.2426033+02:00</StartDateTime> <StopDateTime>2014-05-14T17:00:19.4718654+02:00</StopDateTime> <HelpPage>empty.html</HelpPage> <Repetitions>1</Repetitions> <Required>false</Required> </Action> </Step> … </Recipe>
TAF FILES File.taf The example of taf configuration file is shown below. <?xml version="1.0"?> <TafSetup xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema"> <TafConfigurationStorage> <WriteableTafConfigurationItems> <ConfigurationItems> <ConfigurationItem> <Value>\\130.144.240.124\ATT\TAF 4\Data Directories\TAFExamples\Verified Releas-es\Version 1.0.0</Value> <ReadOnly>false</ReadOnly> <Name>DataDirectory</Name> <Type>String</Type> </ConfigurationItem> <ConfigurationItem> <Value>\\130.144.240.124\ATT\TAF 4\TAF Keyword Libraries\Verified Releases\Version 8.0.0</Value> <ReadOnly>false</ReadOnly> <Name>TafLibrary</Name> <Type>String</Type> </ConfigurationItem> <ConfigurationItem> … </ConfigurationItem> </WriteableTafConfigurationItems> </TafConfigurationStorage> <ExternalsConfigurationStorage> <ConfigurationItems /> </ExternalsConfigurationStorage> <VariableValues> <Item> <Key> <string>ExampleVariable1</string> </Key> <Value> <string>Value 1</string> </Value>
![Page 92: Scheduling system for test automation frameworkScheduling system for test automation framework Citation for published version (APA): Wahyudi, D., & Technische Universiteit Eindhoven](https://reader033.fdocuments.in/reader033/viewer/2022051923/6011324fdb00301ad8544bee/html5/thumbnails/92.jpg)
</Item> <Item> … </Item> </VariableValues> </TafSetup>
![Page 93: Scheduling system for test automation frameworkScheduling system for test automation framework Citation for published version (APA): Wahyudi, D., & Technische Universiteit Eindhoven](https://reader033.fdocuments.in/reader033/viewer/2022051923/6011324fdb00301ad8544bee/html5/thumbnails/93.jpg)
Glossary SSTAF Scheduling System for Test Automation Framework. In Philips, the
name of the project is ASPTE (Automatic System Preconditioning and Testing Environment).
TAF Test Automation Framework. TAF is a framework used in Philips Healthcare to run automatic test.
SUT System under Test. System under test (SUT) is an Alura Xpern System that is built at the Philips site for different purposes: development, functional test, integration test, and system verification and validation.
PDC Product Development Code WTS Windows Task Scheduler COTS Common off the shelf ERP Enterprise Resource Planning RBS Role Based Security CRUD Create, Read, Update and Delete. The four use patterns of objects and
data. MVC Model View Controller. A design pattern for implementing user inter-
faces. MVVM Model View View-Model. Another design pattern for implementing
user interfaces. Side PC Side PC is a normal PC that has a direct connection to SUT. Stub A stub is a piece of code used to replace or simulate the behavior of
existing code or hardware. Activity An activity is a task or program that runs in the Side PC. In this project
scope, one activity is related to one TAF Script. Activity Block An activity block is a group of activity that needed to be run sequen-
tially. Schedule A schedule is a list of times at which activity blocks are intended to
take place. QlikView QlikView is a reporting tool.
![Page 94: Scheduling system for test automation frameworkScheduling system for test automation framework Citation for published version (APA): Wahyudi, D., & Technische Universiteit Eindhoven](https://reader033.fdocuments.in/reader033/viewer/2022051923/6011324fdb00301ad8544bee/html5/thumbnails/94.jpg)
![Page 95: Scheduling system for test automation frameworkScheduling system for test automation framework Citation for published version (APA): Wahyudi, D., & Technische Universiteit Eindhoven](https://reader033.fdocuments.in/reader033/viewer/2022051923/6011324fdb00301ad8544bee/html5/thumbnails/95.jpg)
Bibliography References [1]. Wil van de Berk, Marcel van Veggel, Arjan van Osch, and Luuk van der Veer, Philips Internal Document:Use Case Description - Automatic System Precondition-ing and Testing Environment, 2014 [2]. Philips Medical Systems Nederland B.V, Philips Internal Document: TAF 4 User Manual, 2014 [3]. Djohan Wahyudi, Philips Internal Document: COTS Scheduler Selection for ASPTE, 2014 [4]. Djohan Wahyudi, Philips Internal Document: User Requirement Specification - Automatic System Preconditioning and Testing Environment - Scheduler, 2014 [5]. Djohan Wahyudi, Philips Internal Document: Design Specification - Automatic System Preconditioning and Testing Environment - Scheduler, 2014 [6]. Djohan Wahyudi, Philips Internal Document: Verification and Validation - Au-tomatic System Preconditioning and Testing Environment - Scheduler, 2014 [7]. List of job scheduler software. Wikipedia. [Online]. Available: http:// http://en.wikipedia.org/wiki/List_of_job_scheduler_software. [8]. Comparison of continuous integration software. Wikipedia. [Online]. Available: http://en.wikipedia.org/wiki/Comparison_of_continuous_integration_software. [9]. Frank McKenna, Thin-Client VS Fat-Client Computing, 2012. [10]. QlikView Tool. QlikView. [Online]. Available: http://www.qlik.com/us [11]. Tutorial and Examples. MSDN. [Online]. Available: http://msdn.microsoft.com [12]. Tutorial and Examples. CodeProject. [Online]. Available: http://www.codeproject.com [13]. Technical Question and Answer. StackOverFlow. [Online]. Available: http://stackoverflow.com
![Page 96: Scheduling system for test automation frameworkScheduling system for test automation framework Citation for published version (APA): Wahyudi, D., & Technische Universiteit Eindhoven](https://reader033.fdocuments.in/reader033/viewer/2022051923/6011324fdb00301ad8544bee/html5/thumbnails/96.jpg)
![Page 97: Scheduling system for test automation frameworkScheduling system for test automation framework Citation for published version (APA): Wahyudi, D., & Technische Universiteit Eindhoven](https://reader033.fdocuments.in/reader033/viewer/2022051923/6011324fdb00301ad8544bee/html5/thumbnails/97.jpg)
About the Author
Djohan Wahyudi graduated from Sekolah Tinggi Teknik Surabaya, Indonesia, with a degree in In-formation Systems and Computer Engineering (2006). During his bachelor studies, he developed several software systems for different companies in Indonesia as a freelancer. After his graduation, he was recruited by Philip Morris International and worked as an IT Consultant. He also worked in a software company called Artindo. He continued his master studies under an Erasmus Mundus scholarship fully funded by the European Commission. After finishing his studies he was re-cruited by a web-based company in the UK and was responsible for developing and maintaining several websites. He has a passion for Software Engineering and he wants to continue gaining experience and expertise in this field. In September 2012, he joined two years PDEng program at Eindhoven University of Technology. In the first year, he studied the technical and non-technical courses, and worked for different projects from different companies such as BOSCH, Sioux, TNO, and CERN. During his second year, he was working in Philips Healthcare on Automatic System Preconditioning and Test Environment Project what resulted in this report.
![Page 98: Scheduling system for test automation frameworkScheduling system for test automation framework Citation for published version (APA): Wahyudi, D., & Technische Universiteit Eindhoven](https://reader033.fdocuments.in/reader033/viewer/2022051923/6011324fdb00301ad8544bee/html5/thumbnails/98.jpg)