Scheduling system for test automation frameworkScheduling system for test automation framework...

Scheduling system for test automation framework

Citation for published version (APA):Wahyudi, D., & Technische Universiteit Eindhoven (TUE). Stan Ackermans Instituut. Software Technology (ST)(2014). Scheduling system for test automation framework. Technische Universiteit Eindhoven.

Document status and date:Published: 01/10/2014

Document Version:Publisher’s PDF, also known as Version of Record (includes final page, issue and volume numbers)

Please check the document version of this publication:

• A submitted manuscript is the version of the article upon submission and before peer-review. There can beimportant differences between the submitted version and the official published version of record. Peopleinterested in the research are advised to contact the author for the final version of the publication, or visit theDOI to the publisher's website.• The final author version and the galley proof are versions of the publication after peer review.• The final published version features the final layout of the paper including the volume, issue and pagenumbers.Link to publication

General rightsCopyright and moral rights for the publications made accessible in the public portal are retained by the authors and/or other copyright ownersand it is a condition of accessing publications that users recognise and abide by the legal requirements associated with these rights.

• Users may download and print one copy of any publication from the public portal for the purpose of private study or research. • You may not further distribute the material or use it for any profit-making activity or commercial gain • You may freely distribute the URL identifying the publication in the public portal.

If the publication is distributed under the terms of Article 25fa of the Dutch Copyright Act, indicated by the “Taverne” license above, pleasefollow below link for the End User Agreement:www.tue.nl/taverne

Take down policyIf you believe that this document breaches copyright please contact us at:[email protected] details and we will investigate your claim.

Download date: 27. Jan. 2021

https://research.tue.nl/en/publications/scheduling-system-for-test-automation-framework(f80e5f54-23b2-4ded-8afb-3669df11c509).html

Scheduling System for

Test Automation Framework

Djohan Wahyudi September 2014

Scheduling System for Test Automation Framework

Djohan Wahyudi September 2014

Scheduling System for Test Automation Framework

Eindhoven University of Technology Stan Ackermans Institute / Software Technology

Partners

Philips Healthcare Eindhoven University of Technology Steering Group Alex Visser

Johan Lukkien Wil van den Berk

Date September 2014

Contact Address

Eindhoven University of Technology Department of Mathematics and Computer Science MF 7.090, P.O. Box 513, NL-5600 MB, Eindhoven, The Netherlands +31402474334

Published by Eindhoven University of Technology

Stan Ackermans Institute Printed by Eindhoven University of Technology

UniversiteitsDrukkerij ISBN 978‐90‐444‐1313‐7

Abstract These comments should be the abstract issued for the ISBN-request. Keywords

These keywords should be at least the keywords as issued for the ISBN-request

Preferred reference

Djohan Wahyudi, Automatic System Preconditioning and Testing Environment. Eindhoven University of Technology, SAI Technical Report, September, 2014. (978‐90‐444‐1313‐7)

Partnership This project was supported by Eindhoven University of Technology and Philips Healthcare. Disclaimer Endorsement

Reference herein to any specific commercial products, process, or service by trade name, trademark, manufacturer, or otherwise, does not necessarily constitute or imply its endorse-ment, recommendation, or favoring by the Eindhoven University of Technology or Philips Healthcare. The views and opinions of authors expressed herein do not necessarily state or reflect those of the Eindhoven University of Technology or Philips Healthcare, and shall not be used for advertising or product endorsement purposes.

Disclaimer Liability

While every effort will be made to ensure that the information contained within this report is accurate and up to date, Eindhoven University of Technology makes no warranty, represen-tation or undertaking whether expressed or implied, nor does it assume any legal liability, whether direct or indirect, or responsibility for the accuracy, completeness, or usefulness of any information.

Trademarks Product and company names mentioned herein may be trademarks and/or service marks of

their respective owners. We use these names without any particular endorsement or with the intent to infringe the copyright of the respective owners.

Copyright Copyright © 2014. Eindhoven University of Technology. All rights reserved. No part of the material protected by this copyright notice may be reproduced, modified, or

redistributed in any form or by any means, electronic or mechanical, including photocopy-ing, recording, or by any information storage or retrieval system, without the prior written permission of the Eindhoven University of Technology and Philips Healthcare.

Foreword Philips Healthcare produces equipment for treating various illnesses. The specialty of the business unit Interventional Xray (iXR) is equipment for treating cardiovascular diseases, such as visualizing the placement of a stent after a heart attack. To guaran-tee the safety of the patient and the physicians and nursing staff the equipment must be thoroughly tested. Because the equipment and the real estate (e.g. lead cages) needed for these tests are very expensive, Philips aims to use the equipment for test-ing purposes as efficiently as possible. In the past, we took steps to automate most of the testing. What remained was the manual and labor-intensive task of the installation and configuration of the system prior to executing the tests. This resulted in test systems being allocated to one pro-ject for weeks or months and required more test systems than strictly necessary. The automation of the installation and configuration gives iXR the opportunity to increase the breadth of testing, to decrease the time to market, and to decrease the number of test systems. The entire project entails the scheduling and reporting of installation tasks, and the installation and configuration of test environments. Djohan created the scheduling and reporting part of the project. He investigated the use of Common off the Shelf (COTS) tools to do the job, as well as existing tools that are used within Philips. He developed the missing pieces and integrated these with the existing tools. With his contribution, we are now able to compile a set of tasks that are required to install and configure a test environment and to run a series of tests at a given time. These tasks are then executed without further user interaction at the given time, 24 hours a day and seven days a week. Alex Visser September 2014

Preface This report is the final deliverable of my graduation project in the Software Tech-nology program. The project timeline was nine months and it was located in Philips Healthcare, Best. The project title is Scheduling System for Test Automa-tion Framework (SSTAF). The target audience for this report is a technical audience with basic knowledge of software design, software testing, and UML notation. The document covers the tools to be used for SSTAF Project. Furthermore, it covers the design and the implementation of the Scheduling System. The introduction of the project context is explained in Chapter 1. The stakeholder analysis is made in Chapter 2. The system requirements are defined in Chapter 3. The problem and the possible solutions are analyzed in Chapter 4. Feasibility analysis is described in Chapter 5. Design work is explained in Chapter 6 and 7. The implementation of the system is explained in Chapter 8. Those interested in the verification and validation should read Chapter 9. The project management is described in Chapter 10. The results and conclusions are discussed in Chapter 11. Djohan Wahyudi September 2014

Acknowledgements I would like to express a great appreciation to my company supervisor, Alex Visser for his continuous support, guidance and feedback throughout the project. His experience and knowledge has provided me with valuable feedback for my project and my development. I am grateful to my university supervisor, Johan Lukkien for his valuable feedback that kept pushing me hard for my improvement and development. I would like to thank my customer in Philips that has provided me with help and good cooperation during the project: Wil van den Berk, Marcel van Veggel, Luuk van der Veer, Saco Meeuwig, and Arjan van Osch. I would like to thank Martijn Drabbels and Marc Urlings for their technical guidance and help during the pro-ject implementation. Many thanks I give to Henk Meulendijks, Peer Bruin and Melina Bonin for their view and knowledge about Continuous Integration tools. I would like to give thanks also to Thomas de Laet, Ben van Mierloo, Remco Hijmans, and Anja van der Bolt-Jacobs for their view and knowledge in Test En-vironment tool. I would like to thank Judith B. Strother and Darko Jovanovic for their help to revise my document. I would like to extend my thanks to several OOTI alumni in Philips that help me with discussion and guidance: Paul Robert Marcu, Zlatka Manevska, Athanasios Papakostopoulos, Ivana Kostadinovska, Marco Verstege, Chetan Nair, and Marcin Grodek. I would like to give thanks to the program director of PDEng Software Technolo-gy, Ad Aerts, for his valuable support and help during my entire Software Tech-nology program. I would like to thank all the coaches for their lectures that help me to improve myself during last two years of my studies. Special thanks to the management assistant, Maggy de Wert for her great support and care with so much love and enthusiasm. I would like to thank all my colleagues for their feed-back and support throughout the program. I will not forget my great experience working with them in the OOTI program. Finally, I would like to offer my thanks to my family, especially to my parents, my brother and sister, and my friends for their love, support, and encouragement. Djohan Wahyudi September 2014

Executive Summary An Interventional X-ray (iXR) system provides real time X-ray imaging with high image clarity and low X-ray dose. After several years of development, the iXR System has become complex. In order to verify the correct operation of the system, the system integration and test group performs extensive testing on a va-riety of test environments. The preparation of these test environments is a time-consuming and labor-intensive activity. The system integration and test group wants to automate this activity. This project is created and defined to solve this problem. The project title is “Scheduler System for Test Automation Framework”. The project goal is to automate the System Integration and Test in Allura Xper Sys-tem. The project aims to provide a scheduler system that will run the precondi-tioning activities (installation, configuration, customization, calibration) and test activities automatically. The common off the shelf scheduler and the related Philips tools are explored in the start of the project. The scheduler system consists of a custom scheduler tool and several related Philips tools. The scheduler system has the scheduler website as the user interface. Therefore, it provides an easy user access. The scheduler system integrates with TAF, an existing Test Automation tool, for the execution of preconditioning and test activities. The scheduler system pro-vides interfaces to TAF execution. After the execution, the results are collected automatically. The complete system is tested and verified. The architecture, de-sign, and implementation of this project can be found in this document.

Table of Contents

Foreword .................................................................................................... i

Preface ...................................................................................................... iii

Acknowledgements ................................................................................... v

Executive Summary ................................................................................ vii

Table of Contents ..................................................................................... ix

List of Figures ........................................................................................ xiii

List of Tables ........................................................................................... xv

1. Introduction ....................................................................................... 1

1.1 Context ......................................................................................... 1

1.2 Current Testing Environment ....................................................... 2 1.2.1. System under Test (SUT) .......................................................... 2 1.2.2. Current Test Setup ..................................................................... 2 1.2.3. SUT Related Activities .............................................................. 3 1.2.4. SUT Usage ................................................................................ 3

1.3 Problem Description .................................................................... 4

1.4 Outline .......................................................................................... 4

2. Stakeholder Analysis ......................................................................... 7

2.1 Introduction .................................................................................. 7

2.2 Stakeholders’ Concerns ............................................................... 7

2.3 Stakeholders’ Engagement Actions .............................................. 7

2.4 Customer’ Interview and Meeting ................................................ 8

3. System Requirements ........................................................................ 9

3.1 Main Requirements ...................................................................... 9

3.2 Requirements ................................................................................ 9 3.2.1. Activity ...................................................................................... 9 3.2.2. Activity Block ......................................................................... 10 3.2.3. Schedule .................................................................................. 10 3.2.4. Side PC .................................................................................... 10 3.2.5. Reporting ................................................................................. 11

3.3 Requirement Priority .................................................................. 11

4. Problem Analysis ............................................................................. 13

4.1 Problem Examination ................................................................. 13

4.2 Domain Model ............................................................................ 13

4.3 Mapping of COTS Tools to Domain Model ............................... 14

4.4 Mapping of Related Philips tools to Domain Model .................. 15

4.5 Problem Analysis Result ............................................................. 16

5. Feasibility Analysis .......................................................................... 19

5.1 Test Environment (TE) Project ................................................... 19

5.2 Test Automation Framework (TAF) ........................................... 19

5.3 Step Manager ............................................................................. 19

5.4 QlikView ..................................................................................... 19

5.5 Windows Task Scheduler (WTS) ................................................. 19

5.6 Feasibility Analysis Result ......................................................... 20

6. System Architecture ........................................................................ 21

6.1 Context Diagram ........................................................................ 21

6.2 Use Case ..................................................................................... 21

6.3 Preliminary System Architecture ................................................ 22 6.3.1. Relationship between Schedule, Activity Block and Activity . 22 6.3.2. Preliminary Class Diagram ...................................................... 22 6.3.3. Preliminary Activity Diagram ................................................. 23 6.3.4. Preliminary Deployment Diagram ........................................... 24

6.4 Full System Architecture ............................................................ 24 6.4.1. Logical View ........................................................................... 24 6.4.2. Process View ........................................................................... 27 6.4.3. Deployment View .................................................................... 28

7. System Design .................................................................................. 31

7.1 Main Design Decisions .............................................................. 31 7.1.1. Fat or Thin Client Architecture ................................................ 31 7.1.2. Smart Server or Smart Agent ................................................... 31 7.1.3. Interface between Scheduler System Components .................. 32 7.1.4. Database or XML File ............................................................. 33

7.2 Components Design .................................................................... 33 7.2.1. Mapping of Class Diagram to Deployment Diagram .............. 33 7.2.2. Detail Sequence Diagram ........................................................ 35 7.2.3. State Machine Diagram ........................................................... 36 7.2.4. Components Design Summary ................................................ 37

8. Implementation ................................................................................ 39

8.1 Data Components ....................................................................... 39 8.1.1. Database ................................................................................... 39 8.1.2. XML Storage ........................................................................... 40

8.2 Tool Components ........................................................................ 41 8.2.1. Scheduler Server ...................................................................... 41 8.2.2. Scheduler Agent ...................................................................... 44 8.2.3. Step Manager ........................................................................... 46

9. Verification & Validation ............................................................... 49

9.1 Verification ................................................................................. 49

9.2 Validation ................................................................................... 49

10. Project Management .................................................................... 51

10.1 Way of working ....................................................................... 51

10.2 Project Schedule ..................................................................... 51

10.3 Risk Management ................................................................... 52

11. Conclusion .................................................................................... 53

11.1 Results ..................................................................................... 53

11.2 Lessons Learned ..................................................................... 53

11.3 Future Works .......................................................................... 54

Appendix A. COTS Selection ................................................................. 55

Job Scheduler ........................................................................................ 55

Continuous Integration Tool ................................................................. 56

Tool Comparison................................................................................... 58

Appendix B. Design Evolution ............................................................... 61

Initial Design......................................................................................... 61

Alternative Design with Server and Agent ............................................ 62

Alternative Design with TE (Test Environment) ................................... 63

Alternative Design with Web Service without TE ................................. 64

Appendix C. Scheduler System Swim Lane Diagram ......................... 65

Appendix D. XML Files ......................................................................... 67

SCHEDULER AGENT FILES ............................................................... 67

STEP MANAGER FILES ...................................................................... 68

TAF FILES ............................................................................................ 69

Glossary ................................................................................................... 71

Bibliography ............................................................................................ 73

References ............................................................................................. 73

About the Author .................................................................................... 75

List of Figures Figure 1 – Allura Xper System ....................................................................... 1 Figure 2 – Current Test Setup ......................................................................... 2 Figure 3 – SUT Related Activities ................................................................... 3 Figure 4 – Scheduler Gantt Chart .................................................................. 11 Figure 5 – Domain Model ............................................................................ 14 Figure 6 – Mapping of COTS Tools to Domain Model ................................... 14 Figure 7 – Mapping of Philips Tools to Domain Model ................................... 15 Figure 8 – Basic Plan for TE Extension ......................................................... 16 Figure 9 – Expected Test Setup ..................................................................... 17 Figure 10 – Windows Task Scheduler Prototype ............................................ 20 Figure 11 – Final Mapping of Philips Tools to Domain Model ........................ 20 Figure 12 – Context Diagram ....................................................................... 21 Figure 13 – Use Case Diagram ..................................................................... 21 Figure 14 – Relationship of Schedule, Activity Block and Activity .................. 22 Figure 15 – Preliminary Class Diagram ......................................................... 23 Figure 16 – Preliminary Activity Diagram ..................................................... 23 Figure 17 – Preliminary Deployment Diagram ............................................... 24 Figure 18 – Class Diagram ........................................................................... 25 Figure 19 – Sequence Diagram ..................................................................... 26 Figure 20 – Activity Diagram ....................................................................... 27 Figure 21 – Deployment Diagram ................................................................. 29 Figure 22 – Mapping Class Diagram to Deployment Diagram ......................... 34 Figure 23 – Data Input Sequence Diagram ..................................................... 35 Figure 24 – Scheduler Execution Sequence Diagram ...................................... 36 Figure 36 – Scheduler Agent State Diagram ................................................... 36 Figure 37 – Step Manager State Diagram ....................................................... 37 Figure 25 – Database Diagram ...................................................................... 39 Figure 26 – XML Storage Structure .............................................................. 40 Figure 27 – Model View Controller ............................................................... 41 Figure 28 – Scheduler Server User Interface .................................................. 42 Figure 29 – Scheduler Server FileWatcher ..................................................... 43 Figure 30 – Scheduler Agent User Interface ................................................... 44 Figure 31 – Scheduler Agent Time Trigger .................................................... 45 Figure 32 – Scheduler Agent File Watcher ..................................................... 46 Figure 33 – Step Manager Sequence Diagram ................................................ 47 Figure 34 – Project Gantt Chart .................................................................... 51 Figure 35 – Swim Lane Diagram .................................................................. 65

List of Tables Table 1 – Stakeholders ................................................................................... 7 Table 2 – Stakeholders’ Concern .................................................................... 7 Table 3 – Stakeholders’ Engagement Action .................................................... 7 Table 4 – Customers’ Wish List ...................................................................... 8 Table 5 – Requirement Priority ..................................................................... 11 Table 6 – Translation for Class Diagram ........................................................ 25 Table 7 – Class Diagram Element ................................................................. 26 Table 8 – Activity Diagram Element ............................................................. 28 Table 9 – Fat or Thin Client Architecture....................................................... 31 Table 10 – Smart Server or Smart Agent........................................................ 31 Table 11 – Interface/Communication ............................................................. 32 Table 12 – Database or XML File ................................................................. 33 Table 13 – Class Diagram to Deployment Diagram ........................................ 33 Table 30 – Step Manager State ..................................................................... 37 Table 31 – Step Manager State ..................................................................... 37 Table 14 – XML Storage Folder Structure ..................................................... 40 Table 15 – MVC Components ...................................................................... 42 Table 16 – Scheduler Command Result ......................................................... 43 Table 17 – Scheduler Command ................................................................... 45 Table 18 – Step Manager Command Parameters ............................................. 47 Table 19 – Step Manager File Interfaces ........................................................ 47 Table 20 – Design Status .............................................................................. 49 Table 21 – Requirement Status ..................................................................... 50 Table 22 – Issue and Risk ............................................................................. 52 Table 23 – Domain against functionality........................................................ 55 Table 24 – Job Scheduler First Round Selection ............................................. 55 Table 25 – Job Scheduler Second Round Selection ......................................... 56 Table 26 – Continuous Integration First Round Selection ................................ 56 Table 27 – Continuous Integration Second Round Selection ............................ 58 Table 28 – Tool Selection ............................................................................. 58 Table 29 – Swim Lane Element .................................................................... 65

1.Introduction Philips Healthcare delivers advanced solutions for healthcare professionals. Interventional X-ray (iXR) department is responsible for the development of X-ray systems used for diagnostics and interventional treatment of cardiac and vascular diseases. The X-ray system is called Allura Xper System. The development of this type of systems is performed in multidisciplinary teams where hardware, mechanics, and software come together. In order to verify the correct operation of the system, the system integration and test group performs extensive testing in a variety of test environments. The preparation of these test environments is a time-consuming and labor-intensive activity. The Sched-uler System for Test Automation Project is created to automate this activity.

1.1 Context An Interventional X-ray system provides real-time X-ray imaging with high image clarity and low X-ray dose. To provide these functionalities, the most advanced tech-nology is used. As a result, the product is very complex, with many different hard-ware configurations that need to be tested. The testing itself is equally complex and time-consuming. Furthermore, the installation of those hardware configurations is currently a manual task. This problem leads to the creation and the definition of this project. The customer of the project is the “System integration and test group” in iXR de-partment of Philips Healthcare. The “System integration and test group” has the task to integrate units into a complete system and to test the completed system. A repeating, time-consuming activity in this process is preconditioning of a System under Test (SUT). System under Test is an Allura Xper System used in the testing environment. The Allura Xper System is shown in Figure 1

Figure 1 – Allura Xper System Preconditioning of a SUT consists of PDC (Product Development Code) installation, system configuration, system customization, and system calibration. The purpose of

the Scheduling System for Test Automation Framework (SSTAF) is to automate these activities. These activities should be triggered at any moment, so that a SUT can be used 24 hours a day, 7 days a week. The SSTAF consists of the scheduler tool and interfaces to other tools (like TAF). The process of preconditioning with a new PDC and testing is repeated (during the project) until the right product quality is ob-tained, from the start of integration until the end of validation.

1.2 Current Testing Environment This subchapter describes the current testing environment and the usage of SUT. The current testing environment is evaluated to understand the problem that needs to be solved.

1.2.1. System under Test (SUT) System under test (SUT) is built at the Philips site for different purposes: develop-ment, functional test, integration test, and system verification and validation. SUTs come in different shapes and sizes to accommodate for specific needs and pur-poses. For software development, small systems with only computers are sufficient (small and cheap). For final system tests, entire systems including the mechatronics and X-ray are needed (big and expensive). SUTs are grouped by their type and configuration. They are then allocated to a pro-ject. The type of SUT that is required by a project depends on new features that are introduced by the project.

1.2.2. Current Test Setup The current test setup is shown in Figure 2:

Side PC is a PC that is located near the SUT and it is used to execute the test script

TAF is a test tool to run a test script. Test Environment (TE) is a tool to plan a manual test. Reporting (QlikView) is a reporting tool used by Test Environment (TE).

Figure 2 – Current Test Setup

Planner/ Tester

Test Enviroment (TE)

SUTs

SidePCs

Reporting (QlikView)

N M

N M

1 N

1 1

1.2.3. SUT Related Activities

Figure 3 – SUT Related Activities Figure 3 shows the activities related to SUT. The explanation of all the activities can be found below: o Preconditioning Activities

Installation It installs the product development code (PDC) to the Allura System. It might involve the OS Installation.

System Configuration The system configuration imports the available configuration for the particu-lar SUT.

System Customization The system customization imports the customization setting generated from the customer request.

System Calibration The calibration makes sure that the X-Ray images are shown correctly.

o Test Activities Smoke Test.

A short test that is executed to verify a new PDC is installed correctly. Performance Test

A test that not only verifies whether a function is available or not, but also gives a quantitative judgment on how well a function is performed (an im-portant part is the responsiveness measurement).

Reliability Test A test that is executed repeatedly to verify the amount of time the system functions without failure.

Manual Test A test performed manually by a test engineer.

1.2.4. SUT Usage A SUT can be assigned to a project and to testers of a project. At regular intervals, reports of the SUT usage are made. A report is made in a tool called Test Environ-ment. Test Environment is a planner tool for manual test. The Test Environment tool shows the manual test schedule using reporting tool called QlikView. Currently, the tests can be planned only in regular tester working hours (between 07:00 and 19:00).

The night and weekend tests are planned and triggered manually. Some testers made a small application to trigger these tests automatically using Windows Task Sched-uler. The night and weekend test normally covers one reliability test. The problem is only one test can be done in this time. A tester has difficulty to run two or more tests during a night or a weekend. This leads to an inefficient usage of the test environ-ment.

1.3 Problem Description The project needs to improve the current way of testing. There are several problems recognized:

Many different types and configurations of SUT A tester needs to collect a lot of information about SUT type, configuration and PDC Installation. This is a time consuming activity. There is also a risk that the information is not up to date resulting in the incorrect input for the test.

The characteristics of preconditioning activities o Time Consuming o Labor Intensive o Repeating

The whole installation, customization, configuration and calibration process is done manually, requiring considerable human effort and resulting in hu-man errors. Currently, the unattended installation is not possible.

Ineffective usage of SUT The Allura system is not used optimally since the scheduling of more than one test outside the working hours (night and weekend) requires considera-ble effort and is generally avoided by testers.

After recognizing the problem, several goals that need to be achieved, are defined:

To reduce human effort in preconditioning and test activities To reduce human errors To increase the SUT utilization in Testing Environment

1.4 Outline To understand the document well, its outline in relation to the project is explained below: Chapter 1 contains the project introduction. Chapter 2 contains the stakeholder analysis. It identifies the stakeholders involved in the project. Chapter 3 contains the project requirements. The project requirements are derived from interviews and meetings with the customers. Chapter 4 contains the problem and domain analysis. The available tools that can be used for the project are analyzed. Chapter 5 contains the feasibility analysis. It includes the issues and risks for each tool that is used in the project. Chapter 6 contains the description of the system architecture. The 4+1 architectural view model used in the project is documented here. Chapter 7 contains the description of the system design. The design decisions are documented here. Chapter 8 contains the description of each system component implementation.

Chapter 9 contains the verification and validation of the project. Chapter 10 contains the description of the project management. Chapter 11 contains the results and conclusions. ■

2.Stakeholder Analysis Bridge – In this chapter, the stakeholder analysis is described. The stakeholders in-volved were identified and their concerns were analyzed. Interviews and meetings were held with the customers to understand the problem. The wish list is generated from the information collected.

2.1 Introduction Several stakeholders were identified. The listed stakeholders and their roles in this project can be found below: Table 1 – Stakeholders Role Name University Supervisor Johan Lukkien Company Supervisor Alex Visser Customer/User System Integration and Test Group

Wil van den Berk Marcel Van Veggel Luuk van der Veer Arjan van Osch Saco Meeuwig

Module/Tool Owner Test Tool (TAF) Martijn Drabbels

Marc Urlings Testing Environment (TE) Thomas de Laet

Remco Hijmans Ben van Mierloo

Reporting Tool (QlikView) Anja van der Bolt-Jacobs

2.2 Stakeholders’ Concerns Several stakeholders identified were analyzed based on their role. Their role and con-cern can be seen in the table below: Table 2 – Stakeholders’ Concern Role Concerns University Supervisor

Supervise and guide the trainee to make a successful project. Evaluate the trainee during the project.

Company Supervisor

Supervise and guide the trainee in the project. Direct the trainee to the right design and implementation aim-

ing for the high quality final product. Evaluate the trainee during the project.

Customer/User To have a final product that can help them to solve their prob-lem

Module/Tool Owner

To have the tool used without any changes (Since it might affect some other Philips organization that uses the tool)

2.3 Stakeholders’ Engagement Actions Several stakeholders identified were analyzed based on their engagement action. Table 3 – Stakeholders’ Engagement Action Role Engagement Actions University Project’s Guidance

Supervisor Steering Group Meeting Document Revision

Company Supervisor

Project’s Guidance Steering Group Meeting Document Revision Demo feedback

Customer/User Requirement Analysis Demo feedback Document Revision Validation

Module/Tool Owner

Help in the Tool Usage

2.4 Customer’ Interview and Meeting Interview and meeting is the strategy being used to get the customer requests. A rapid interview and meeting with the customers was the strategy in the first period of the project. Unclear information was confirmed and clarified in the subsequent meetings. Communication in the later stage was organized in weekly meetings with the cus-tomers. The wish list was created after the information from the customers was col-lected. The wish list created can be seen in Table 4. Table 4 – Customers’ Wish List No Wish List Priority 1 Scheduling of Activities High 2 Running of Activities (precondition and test) Automatically High 3 Warning before the start of automatic activity (Safety) High 4 Stop the Running Activity High 5 Skip the Running Activity Medium 7 Suspend the Running Activity High 8 Adjust the time for the schedule when the schedule is changed Medium 9 Grouping of Activity (Activity Block) High 10 Estimation Time filled by user High 11 Estimation Time updated by the last running activity Low 12 Access Control Low 13 Connection to Qlikview Low 14 Centralized and Easy Access from anywhere High 15 Chose the Activity (File) to execute High 16 Directory Structure of File Store Low 17 Reporting and Email Medium 18 Gantt Chart Overview Medium 19 Handle TAF UI Low 20 Integration with TE (Test Environment) Medium 21 Template for Activity Block Medium Beside the wish list, the other information was also discussed with the customers. This information confirmed what the customers did not require. The listed functional-ity below is out of the scope of the project:

Priority in Activity – no priority is required in the task. Dependency between activities – no dependency needed, however simple

dependency might be needed at the later stage. Execution from the build server – no trigger needed from new code deploy-

ment. This will be done manually. ■

3.System Requirements Abstract – In this chapter, the requirements are defined. The requirements are com-piled from the wish list created in the previous chapter and finalized in this Chapter. The system requirements are presented below.

3.1 Main Requirements After the problem description in the previous chapter, functionalities to address the problem are defined in this subchapter. The functionalities that the project needs to address are described below:

Scheduling Scheduling is the basic functionality needed for the usage of a SUT. Several SUTs were used for different activities in different times. SSTAF needs the scheduler functionality to organize different activities and different tasks to run.

Running Task / Activity Automatically The Tasks / Activities that need to be run in a SUT fall into two categories: Preconditioning Activity and Test Activity. This can be handled by TAF functionality.

Controlling Access The SUTs are valuable resources for the company. Access control function-ality is needed to regulate the usage of the SUTs.

Reporting Reporting is needed to give the users the overview information. The sched-ule and information for every SUT needs to be viewed by everyone. Fur-thermore, a report should show the result of the preconditioning and test ac-tivity.

3.2 Requirements The system requirements are made according to Philips standard so that each re-quirement has a tag allowing it to be verified and validated later on.

3.2.1. Activity An activity is a task or program that runs in the Side PC. In this project scope, one activity is related to one preconditioning or test script. The requirements related to the activity are listed below:

SSTAF.Activity SSTAF must be able to select a script and assign it to an activity.

SSTAF.Activity.Execution The SSTAF must execute the activity in Side PC at the scheduled time.

SSTAF.Activity.FileStore The SSTAF must retrieve a list of scripts from the file storage.

SSTAF.Activity.Result The SSTAF should log the result of an activity.

SSTAF.Activity.EstimatedTime SSTAF should store the estimated time for an activity.

SSTAF.Activity.Stop A user must be able to stop an activity. The user must be able to start the selected activity after the stop.

3.2.2. Activity Block An activity block is a group of activities that need to run sequentially. The require-ments related to an activity block are listed below:

SSTAF.ActivityBlock SSTAF must be able to group one or more activities into ActivityBlock.

SSTAF.ActivityBlock.Popup SSTAF must inform the user that an automatic Activity Block will run. The user must have the ability to suspend or stop this action.

SSTAF.ActivityBlock.Stop A user must be able to stop an ActivityBlock. After stop, the user should be able to start the selected activity block.

SSTAF.ActivityBlock.EstimatedTime SSTAF should store and show the estimated time for ActivityBlock.

SSTAF.ActivityBlock.Result The SSTAF should log the result of ActivityBlock.

SSTAF.ActivityBlock.Template SSTAF should be able to reuse the ActivityBlock in the schedule.

3.2.3. Schedule A schedule in this project scope is a list of times at which activity blocks are intended to take place.

SSTAF.Schedule SSTAF must be able to schedule an activity block.

SSTAF.Schedule.Access SSTAF should be accessible remotely.

SSTAF.Schedule.ActivityBlock.Parameter SSTAF should be able to change activity block parameters when the block is sched-uled.

SSTAF.Schedule.OverlappingSchedule It should not be allowed to have two schedules overlapping on the same SUT.

SSTAF.Schedule.OverlappingSchedule.Warning If there are overlapping schedules, the system should give warning to the user. The user could force the system to accept the schedule whereby the system would auto-matically adjust the schedule to avoid overlapping.

SSTAF.Schedule.AdjustSchedule SSTAF should adjust the schedule automatically. When there is overlapping of schedules, the system should move the last added and the subsequent schedules for-ward.

SSTAF.Schedule.AdjustScheduleConstraint An activity containing the “Repeat-until” option must be executed at least once.

SSTAF.Schedule.ResponseTime The system should give a user a feedback within 1 min (response time)

3.2.4. Side PC Side PC is a normal PC that has a direct connection to SUT. The requirements related to a Side PC are listed below:

SSTAF.SidePC SSTAF must be able to execute a schedule in the Side PC.

SSTAF.SidePC.IsAlive

SSTAF maintains a connection for each Side PC. If the Side PC is not connected for more than 1 hour, the status is “Unknown”.

3.2.5. Reporting A user needs the reporting functionality (this functionality is provided by the report-ing tool).

SSTAF.Reporting.Result SSTAF should log the result of the schedule. The result is listed in the Figure 4.

SSTAF.Reporting.GanttChart SSTAF should show the schedule as Gantt chart.

= passed = skippedSUT = failed = stoppedSUT 1 = in progress = interruptedSUT 2 = not applicable = scheduledSUT 3

=> current timeTime ==>

Planning project X7:00 8:00 9:00 10:00 11:00 12:00 13:00 14:00 15:00

Figure 4 – Scheduler Gantt Chart

3.3 Requirement Priority Table 5 lists the priorities of the requirements. Table 5 – Requirement Priority Priority Requirement 1 SSTAF.Activity

SSTAF.Activity.Execution SSTAF.ActivityBlock SSTAF.Schedule

2 SSTAF.Activity.FileStore SSTAF.Activity.EstimatedTime SSTAF.ActivityBlock.EstimatedTime SSTAF.Schedule.Access

3 SSTAF.ActivityBlock.Template SSTAF.Schedule.ActivityBlock.Parameter SSTAF.Activity.TAFParameter

4 SSTAF.Activity.Result SSTAF.ActivityBlock.Result SSTAF.ActivityBlock.Popup

5 SSTAF.Activity.Stop SSTAF.ActivityBlock.Stop

6 SSTAF.Schedule.OverlappingSchedule SSTAF.Schedule.OverlappingSchedule.Warning SSTAF.Schedule.AdjustSchedule SSTAF.Schedule.AdjustScheduleConstraint

7 SSTAF.Schedule.ResponseTime SSTAF.SidePC.IsAlive SSTAF.Reporting.Ganttchart

■

4.Problem Analysis Bridge – In this chapter, the problem is analyzed to define the direction of the project. Several Common off the Shelf (COTS) tools are investigated. Several pro-jects and tools already used in Philips are also investigated.

4.1 Problem Examination In this subchapter, the problems are examined to find the solutions. The problems recognized in the current test environment are listed below: Many different types and configurations of SUT

In order to avoid the mistakes in the manual process of collecting the data (TAF parameter), each time a test needs to be executed this data should be readily available from a central storage where it is regularly maintained.

The Time Consuming, Labor Intensive, and Repeating character of precondition-ing activities. In order to reduce the human effort, the preconditioning activities need to be ex-ecuted automatically. Currently, the unattended installation is not possible. The iXR software installer should be made ready for this purpose. Smart Suite ver-sion of iXR software is being developed to accommodate this new feature. The automatic installation is still an ongoing project that needs to be integrated with SSTAF later.

Ineffective usage of SUT In order to increase the usage of SUT, multiple unattended scheduling of tests, especially outside the working hours (night and weekend) is needed. For this to work the scheduling functionality is needed. The SUT usage would also need to be regulated (who can use the SUT at a particular time). To provide this feature, access control functionality would be needed.

The problems above are addressed well by the main requirements defined in the pre-vious chapter:

Scheduling of activities Running of activities automatically Access Control Reporting

In the next subchapter, the possible solutions to fulfill these main requirements are analyzed.

4.2 Domain Model Preliminary domain model that was made to fulfill the main requirements can be seen in Figure 5. Short description to explain the domain model can be found below (Bold – domain model): Access Control to an Allura System is provided by assignment of a Tester to a Pro-ject. The Scheduler assigns a Schedule of Activities to the Allura System. The Time Trigger should trigger the Executor to execute the Activity automatically. After the execution, the Report should show the Schedule Result and the Activity Result.

Figure 5 – Domain Model

4.3 Mapping of COTS Tools to Domain Model After completing the domain model with the necessary components, we try to map the model to common off the shelf tools. There are three types of tools related to the domain model above:

Planner / Calendar, e.g. Microsoft Project, Google Calendar The tool provides access control and scheduling of activity

Job Scheduler, e.g. Windows Task Scheduler The tool provides scheduling and running of activity automatically

Continuous Integration, e.g. Electric Commander, Jenkins The tool provides almost all functionality required except the reporting

The mapping of the COTS tools to Domain Model can be seen in the Figure 6.

Figure 6 – Mapping of COTS Tools to Domain Model

Scheduling of Activities

Controlling Access

Running of Activities

Automatically

Reporting

Use

Register

Trigger

Assign

Job Scheduler

Calendar / Planner

Continuous Integration

The details of the COTS Selection can be found in Appendix A. COTS Selection. None of the COTS tools met the reporting requirement. COTS tool selection process points out Jenkins as the result. It is open source software. It has many plugins and has different customization and it is easy to extend. However, it also might take more time to customize and extend Jenkins to have the desired functionality.

4.4 Mapping of Related Philips tools to Domain Model Several tools available in Philips (e.g. TAF and Step Manager) were found suitable to be used in the project. The possibilities for use of these tools are discussed further below. Furthermore, the Test Environment tool (yet to be realized) is also considered. The Test Environment (TE) tool is a planning system for manual tests. The mapping between the related Philips tools against the domain model can be seen in Figure 7. The related Philips tools are described below: Test Automation Framework (TAF)

TAF is the custom test framework being used in iXR for testing purposes. TAF has been used for several years, so the tool is already stable. The test script that needs to run is written in TAF format. The TAF tool is used for executing the test script.

Step Manager Step Manager is the custom tool being used in the factory to execute several ac-tivities in sequence automatically. Step Manager is already stable. The use of Step Manager is recommended for TAF execution functionality.

Test Environment (TE) The TE project covers the development of planning and reporting systems for manual tests. The project delivers the tool called Test Environment (TE). The tool fulfills many of the reporting, access control, and UI requirements (required for SSTAF). Most of the TE and SSTAF functionalities are common except the scheduling and task execution. For this reason, SSTAF could be built as an ex-tension of TE. The design of this extended TE model can be found in Appendix B. Design Evolution

QlikView QlikView is the COTS reporting tool that is widely used in Philips. The man-agement prefers to have QlikView reports, so the use of it is recommended. TE tool also uses QlikView.

Figure 7 – Mapping of Philips Tools to Domain Model

Step Manager

TAF QlikView

Test Environment

4.5 Problem Analysis Result The analysis of the COTS tools (Appendix A. COTS Selection) points out Jenkins as a strong candidate. Furthermore, Jenkins is free and it is used by Philips organiza-tion. However, it is difficult to integrate Jenkins with TE tool. The integration with TE tool is necessary and is treated as a design constraint so Jenkins is not considered anymore. Figure 7 shows that many Philips tools can be assembled and reused for the project purpose. Based on Figure 7 several decisions were made:

TAF The execution of TAF is necessary.

QlikView The reporting with QlikView is recommended, although it can be automati-cally integrated through TE.

Step Manager The use of Step Manager is recommended for TAF execution.

Test Environment The integration with TE is necessary since TE has many overlapping func-tionalities with SSTAF. SSTAF project will be made ready for this integra-tion. The design of SSTAF will be in-sync with TE so the integration in the future can be done with minimal effort. The picture below provides a rough sketch of the integration. The integration point of TE and SSTAF will be the database.

Figure 8 – Basic Plan for TE Extension

TE

SSTAF

Figure 7 shows that Time-trigger functionality is not covered by the tools. Windows Task Scheduler (WTS) can be used to fill the Time-trigger functionality. The extra advantage is that WTS is free and included in the existing support contracts. Based on the Figure 7, the proposed test setup is given in Figure 9.

Figure 9 – Expected Test Setup ■

Test Enviroment (TE)

SUTs

SidePCs

Reporting (QlikView)

Planner/ Tester

N 1

1 1

N M

5.Feasibility Analysis Abstract – In this chapter, the feasibility analysis is described. Several issues and risks related to the tools are described in this chapter. The possibility of using the Windows Task Scheduler (WTS) is also investigated.

5.1 Test Environment (TE) Project Test Environment Project is a running project that has not yet been released. After the start of SSTAF project, some concerns were raised that it might influence or even endanger certain aspects of TE Project. Therefore, there were objections to allowing the SSTAF project to be included in the TE project. The issue was raised to a higher level of management who decided to separate the development of these tools without any interaction in the first releases. It was also decided that SSTAF and TE integra-tion would be necessary, although not in this project timeline. The access control functionality is removed from the requirements of SSTAF since TE project has ac-commodated this.

5.2 Test Automation Framework (TAF) Test Automation Framework is a stable tool. The tool is used by many organizations in Philips Healthcare. Because of this, it would be difficult to make changes in TAF in the case they are required. The test activities are currently executed by TAF. How-ever, the execution of preconditioning activities in TAF is still not supported. Cur-rently, Factory Tooling project is developing the automatic installation tool to be used in the factory. Step Manager is the base for that tool. SSTAF will use Step Man-ager to provide easy integration with factory tooling (to perform automatic installa-tion). The risk of using Step Manager is explained further in the next subchapter.

5.3 Step Manager Step Manager is a tool being used in the factory to execute workflow steps in the installation, configuration and test of a product. The tool has already been used for almost 10 years. Step Manager executes TAF as one of the workflow steps. This functionality can be used in SSTAF project. However, currently there is no commu-nication to and from Step Manager. Changes to accommodate this functionality should be done in Step Manager and that could be a risk to the project.

5.4 QlikView QlikView is a reporting tool. It can show the report in the chart, table or graph format easily. It is used in many organizations of Philips as it meets the management expec-tations. However, the user has stated that QlikView integration is a low priority re-quirement. Therefore, QlikView will be out of the scope to this project. The connec-tion to QlikView will be made after the essential requirements are met.

5.5 Windows Task Scheduler (WTS) Windows Task Scheduler is a COTS tool considered to accommodate the scheduling functionality. It is able to schedule a task at a scheduled time. One prototype applica-tion was built to show this functionality. The prototype was built in the server to schedule a task/activity in the Side PC. To schedule this task the Side PC username and password are needed. This is considered a big risk since the customer is not com-fortable with putting the username or password of every Side PC on the server. Be-cause of this, it was decided to make a timer application to handle the scheduler func-tionality. By having its own code, it is possible to customize and control the sched-ule.

Figure 10 – Windows Task Scheduler Prototype

5.6 Feasibility Analysis Result The feasibility study has big influence on the project. The final direction and the tool used in the project can be seen in Figure 11. Philips tools (TAF, Step Manager, QlikView) will be used. However, the scheduler (dark blue color) needs to be devel-oped. The Scheduler will need to integrate with TE in the future.

Figure 11 – Final Mapping of Philips Tools to Domain Model ■

QlikView

Future TE Integration

Step Manager

TAF

Scheduler

Server

SidePCs

1 N Windows Task Scheduler Windows Task

Scheduler SidePC UserName +Password Trigger &

Execute

6.System Architecture Abstract – In this chapter, the system architecture is described using the 4 + 1 archi-tectural view model. Firstly, the context diagram is shown describing the boundaries of the system. Then the use case follows. Preliminary system architecture is made in order to define the system further. Then the preliminary architecture is further devel-oped into the full system architecture.

6.1 Context Diagram The context diagram shows the boundaries of the system. Step Manager is not in-cluded in the context diagram since it is considered the part of the system. SUT is not included in the context diagram since the access to SUT is provided by TAF.

Scheduling System for Test

Automation Framework (SSTAF)

Reporting

User / Tester

Test Automation Framework

(TAF)

CRUD Schedule, Activity Block, Activity

Log and Report

Run with Parameter

Figure 12 – Context Diagram

6.2 Use Case Figure 13 shows the use case diagram of Scheduler System.

Figure 13 – Use Case Diagram

Activity Block is the unit used in the Use Case Diagram. An activity block is a group of activities that need to be executed sequentially. Further explanation of Activity Block can be seen in the following subchapter.

6.3 Preliminary System Architecture From the Feasibility Analysis, we know that we need to develop the Scheduler Sys-tem. This subchapter describes the preliminary design to define the system further.

6.3.1. Relationship between Schedule, Activity Block and Activity A schedule is a list of times at which activity blocks are intended to take place. An activity block is a group of activities that need to be executed sequentially. An activi-ty is a preconditioning or test activity. The relationship between schedule, activity block and activity is shown in Figure 14.

Figure 14 – Relationship of Schedule, Activity Block and Activity

6.3.2. Preliminary Class Diagram The preliminary class diagram is based on the domain model. The Access Control is removed from the model. “Allura System” is changed into “Side PC of Allura Sys-tem” since TAF execution is in the Side PC. Activity Block (with the executor and the result) is added to the model. The preliminary class diagram is shown in Figure 15. The colors show the mapping between the classes and the tools.

Schedule

Activity Block

Activity

Activity

Activity

Activity Block

Activity

Activity

Schedule of Activity Block(s)

when they need to run

Activity Block Activities that need to run

in sequence

Activity Preconditioning / Test

Activity

Figure 15 – Preliminary Class Diagram

6.3.3. Preliminary Activity Diagram Activity Diagram is made to described the process of the scheduler system. Figure 16 shows the activity diagram for the Scheduler System.

Figure 16 – Preliminary Activity Diagram

Schedule

Trigger

Execute

Execute

Register

Trigger

Execute

Step Manager

TAF

Scheduler

The activity diagram shows three major stages: Data Input

Data input is the stage of getting the user input for activity, activity block and schedule. This process needs to take place in the Server since a central data repository is needed.

Schedule Execution The schedule execution needs to take place in the Side PC since this is where TAF is located.

Reporting The reporting is the stage of getting the schedule results. The process needs to take place in the Server since the results should be stored in a central data repository.

6.3.4. Preliminary Deployment Diagram From the previous subchapter we understand that the system should have two differ-ent deployment computers:

1. Server It needs to handle data input and reporting. The server should be connected to User PC to handle data input.

2. Side PC It needs to handle the Schedule Execution. The Side PC should get the data from the server for the Schedule Execution.

To make it more clear the preliminary deployment diagram is shown in Figure 17.

Server Side PC

Allura XPer System

TAF

User PC

Figure 17 – Preliminary Deployment Diagram

6.4 Full System Architecture In this subchapter, the preliminary system architecture is further developed into the complete system architecture.

6.4.1. Logical View

Class Diagram The class diagram (Figure 18) is developed from the preliminary class diagram shown in Figure 15. This process comprises the assignment of concrete components (the existing ones and the ones to be developed) to the components identified in the preliminary phase and it can be seen in Table 6.

1 N

1 N

1 1

Table 6 – Translation for Class Diagram Preliminary Class Diagram Scheduler Scheduler Server Time Trigger Scheduler Agent Activity Block Executor Step Manager Executor TAF Side PC of Allura System Side PC Schedule Schedule Schedule Result Schedule Result Activity Block Step Manager Recipe Activity Block Result Step Manager Result Activity TAF Configuration Activity Result TAF Result - Activity Block (Template) - Activity (Template) The class diagram that shows the classes of the scheduler system can be seen in Fig-ure 18. Activity Block (Template) and Activity (Template) is added to the model. This addition allows the user to reuse the Activity Block and Activity already defined before.

Figure 18 – Class Diagram

The short description of the classes in Figure 18 can be found in Table 7 Table 7 – Class Diagram Element Classes Description SidePC The SidePC class defines the SidePC attributes. Scheduler Server The Scheduler Server class provides the user input functionality Command The command class contains the communication data between Sched-

uler Server and Scheduler Agent. Scheduler Agent The Scheduler Agent class provides the schedule execution in the Side

PC Schedule The schedule class contains the schedule information. It also contains

the activity block information. Schedule Result The schedule result class contains the result of a schedule execution.

One schedule can have several results if the schedule is executed several times.

Activity Block (Template)

The activity block class contains the sequence of activities and their information.

Activity (Template) The activity class contains the Activity information (TAF Configura-tion)

Step Manager The Step Manager class represents the Step Manager functionality Step Manager Recipe

The Step Manager Recipe class represents the input file of Step Manag-er

Step Manager Result

The Step Manager Result class represents the output file of Step Man-ager

TAF The TAF class represents the TAF functionality TAF Configuration The TAF Configuration class represents the input file of TAF TAF Result The TAF Result represents the output file of TAF

Sequence Diagram The sequence diagram is made to show the interaction between components. The sequence diagram in Figure 19 shows the sequence from schedule creation by tester until activity execution by TAF.

Figure 19 – Sequence Diagram

6.4.2. Process View The Activity Diagram (Figure 20) is developed from the activity diagram shown in Figure 16. Another version of the process view as swim lane diagram can be seen in Appendix C.

CRUD Activity

CRUD Activity Block

CRUD Schedule

Schedule TimeTrigger

Generate and Copy File

Schedule Execution

Step Manager Execution

TAF Execution

Update Result

Save Data

User Input

Create TAF Test File

Save Data

Put Server Data

Reporting

Create Command

Get Server Data

ExecuteCommand

Figure 20 – Activity Diagram

Test Step for TAF

Activity Activity Block

Schedule

Command

All Data

All Data

Command

ScheduleStep Manager Recipe Test Step

for TAF

Schedule Result Step

Manager Result TAF Result

Each of the elements shown in the Activity diagram is described in short detail in Table 8. Table 8 – Activity Diagram Element Activity Diagram Element

Description

User Input The user performs data input activity on the system. Create TAF Test File

Create TAF test file using the TAF software in the user PC

CRUD Activity Create, Read, Update, or Delete of the Activity CRUD Activity Block

Create, Read, Update, or Delete of the Activity Block

CRUD Schedule

Create, Read, Update, or Delete of the Schedule

Save Data Save data to database Generate and Copy File

Generate XML file and copy TAF data

Get Server Data Get data from the server to the Side PC Execute Command Execute command data Schedule Time Trigger

Register the schedule data in the time trigger

Schedule Execution The execution of schedule Step Manager Exe-cution

Process of running Step Manager

TAF Execution Process of running TAF Put Server Data Put data from the Side PC to the server Update Result Update results to database Save Data Save data to database Reporting Show the results in reporting Activity Diagram Data Object

Description

Test Step for TAF

The TAF configuration file

Activity The activity data Activity Block The activity block data Schedule The schedule data Command The command data Step Manager Rec-ipe

The step manager data

Command Result The command result data Schedule Result The schedule result data Step Manager Re-sult

Step Manager result data

TAF Result TAF result data

6.4.3. Deployment View The deployment diagram shows where each component is deployed. There are four deployment machines: The Server

A Server connected to Philips Network. It contains several components: Scheduler Server

It provides the user input. Database

The Scheduler Server stores the scheduler data in the database. XML Storage

The Scheduler Server needs to store and organize the XML files for Side PC. The use of XML files is necessary since TAF and Step Man-ager use XML files as their input. Detail reasons are explained in the next chapter.

Reporting The reporting accesses the data from database and XML Storage.

The Side PC A computer connected to a particular SUT. It contains several components:

The scheduler Agent The scheduler agent is the centralized control of the system. The sched-uler agent handles the scheduling functionality in the side PC.

Step Manager It handles the execution of the Activity Block.

TAF It handles the execution of Activity (TAF File).

The User PC A computer used to access the scheduler system. It contains several components:

Browser It provides the data input to the Server in the User PC.

TAF It is used to create an activity (TAF File).

Allura Xper System The Allura Xper System used for the test or preconditioning activities. This sys-tem is also known by System under Test (SUT).

Server Side PC

Allura XPer System

Scheduler Server

TAF

Step Manager

Scheduler Agent

User PC

Browser

Reporting

TAF

Database

XML Storage

Figure 21 – Deployment Diagram

■

7.System Design Abstract – In this chapter, main design decisions are documented. The design deci-sions explain the decision taken in System Architecture (Chapter 6). Further designs of the scheduler system components are also documented in this chapter.

7.1 Main Design Decisions This subchapter explains the reasoning behind the decision making process.

7.1.1. Fat or Thin Client Architecture Regarding the global architecture of the system shown in Figure 21, design decisions are made for Server – User architecture (comprising Server – User PC). The defini-tion of Fat Client and Thin Client taken from [9] is quoted below. “Fat-client computing refers to a multi-tier client server paradigm where (in the sim-plest model) the client part of the application (i.e., those programs used by the end-user) executes on the desktop PC and the server part of the application (i.e., those programs used to drive the relational database) resides on a single server together with the application code. Thin-client computing refers to multi-tier client server paradigm where the client (end-user) programs display in a browser (such as Internet Explorer or Netscape) but the execution of that user code takes place on a central web server, not at the desktop PC “. The advantages and disadvantages of the design options are explained in Table 9. Table 9 – Fat or Thin Client Architecture Option Advantages Disadvantages Fat Client Architecture

The system has more control on the client side because the client is an application.

The system needs installation of the client applications. When it is updated then all the appli-cations need to be updated and installed again.

Thin Client Architecture

The system uses web browser as a client, so no installation is required.

The system has no control of the client because the client only uses web browser.

Considering these advantages and disadvantages, Thin Client Architecture was cho-sen for Server – User architecture. The user does not need to install the client applica-tion to schedule the task. He just uses the browser to give input to the scheduler web-site. Another reason is the TE Integration design constraint.

7.1.2. Smart Server or Smart Agent For the Server – Agent architecture a choice must be made where to put the ‘brain’. A component with the ‘brain’ should handle the schedule execution. The Server component that might handle the schedule execution is the Scheduler Server. The Side PC component that might handle the schedule execution is the Scheduler Agent. A decision to choose whether the scheduler server or scheduler agent should be smart is made in this subchapter. The advantages and the disadvantages of both options are listed below: Table 10 – Smart Server or Smart Agent Option Advantages Disadvantages

Smart Server – Dumb Agent

The server has centralized control and the server control everything.

1. The schedule execution will need network connection.

2. There is a network delay for the execution.

Dumb Server – Smart Agent

1. The agent able to work by itself without server con-nection.

2. The agent has more con-trol because the schedule execution is performed on the same computer.

3. There is no network delay for the schedule execu-tion.

The agent needs to update the server about the schedule exe-cution status. There is a net-work delay involved, but the schedule execution will not have a delay.

After analyzing the advantages and disadvantages, the decision was made to put the ‘brain’ in the Scheduler Agent. The smart agent should be able to work independent-ly even without server connection. If the network is disconnected or if the server is unavailable, the smart agent is still able to operate.

7.1.3. Interface between Scheduler System Components There are three types of interface investigated to be used between the components (Scheduler Server – Scheduler Agent – Step Manager – TAF). The analysis of the different types of interface can be found below: Table 11 – Interface/Communication Option Advantages Disadvantages File Interface

1. File Interface can go behind the firewall; it on-ly needs to map the net-work drive.

2. File Interface will inte-grate well with Philips tools (TAF and Step Manager) since it is us-ing XML as input.

File Interface depends on sta-bility of the network connec-tion.

TCP/IP Interface TCP/IP Interface has direct communication and full con-trol of the execution time and the response

TCP/IP Interface cannot go behind firewall if the port is not open.

Command Line Interface

1. Command Line Interface has direct communica-tion and full control of the execution time and the response.

2. Basic interface for win-dows applications

Command Line Interface can be only executed on the same computer.

Based on the analysis above, the decision is explained below:

Scheduler Server – Scheduler Agent: File Interface was chosen since XML File is used as input for Step Manager and TAF. Another reason is to avoid the firewall problem.

Scheduler Agent – Step Manager: Command Line Interface was chosen since it is a basic and easy interface to be used in windows applications.

Step Manager – TAF: Command Line Interface is used since TAF already has the command line interface.

7.1.4. Database or XML File The question is what data storage should be used in Scheduler System. The strong candidates are database and XML file. The analysis is described below: Table 12 – Database or XML File Option Advantages Disadvantages Database 1. Database is common, fast

and organized data stor-age.

2. Database has remote ac-cess.

3. Database is easier for web application.

Database needs installation and maintenance.

XML File

1. XML File is a light form of data storage.

2. XML File is easy to move around

1. More effort is needed to access the information in-side the XML File.

2. More effort is needed to organize the XML File.

From the analysis above, they both appear as strong candidates to be used in Sched-uler System. It is decided to use both in the project. Further reasoning can be seen below:

Database is used as Server application data storage o Integration with the Test Environment in the future will be easier to

implement. XML File is used as Side PC application data storage

o Philips tools already use XML files for their input (TAF and Step Manager).

o Database installation in each Side PC is not necessary. After the decision to use XML File as Side PC application data storage, centralized XML Storage is needed to accommodate the data input in the server. The XML File will be generated by scheduler server and copied to the local folder by Scheduler Agent.

7.2 Components Design In this subchapter, design of each of the system components is described. The design continues by mapping the components from the class diagram to the deployment dia-gram. Sequence Diagram for the Scheduler System also described into more detail in this subchapter.

7.2.1. Mapping of Class Diagram to Deployment Diagram Mapping of the components from the class diagram (Figure 18) to the deployment diagram (Figure 21) is explained in Table 13.

Table 13 – Class Diagram to Deployment Diagram Class Diagram Deployment Dia-

gram Description

Scheduler Server Scheduler Server in Server

This class represents Scheduler Server and its functionali-ty.

Scheduler Agent Scheduler Agent in Side PC

This class represents Scheduler Agent and its functionality.

Step Manager Step Manager in Side PC

This class represents Step Manager and its functionality.

TAF TAF in Side PC This class represents TAF and its functionality. Command Database in Server, The class represents Command data. This data is stored in

XML Storage in Server and Side PC

the database and subsequently in a XML file. This data is the input for Scheduler Agent.

Activity Block (Tem-plate)

Database in Server The class represents Activity Block data. This data is stored in the database

Activity (Template) Database in Server The class represents Activity data. This data is stored in the database

Side PC Database in Server; XML Storage in Server

This class represents Side PC data. This data is stored in the database. This data is realized as folder in Server XML Storage.

Schedule Database in Server; XML Storage in Server and Side PC

The class represents Schedule data. This data is stored in the database and subsequently in a XML file. One XML file is created for each side PC. This data is input for Scheduler Agent.

Schedule Result Database in Server; XML Storage in Server and Side PC

The class represents Schedule Result data. The data is gen-erated from the schedule execution and placed in a XML file. Subsequently the data is placed in the database.

Step Manager Recipe XML Storage in Server and Side PC

The class represents the input of Step Manager.

Step Manager Result XML Storage in Server and Side PC

The class represents the output of Step Manager.

TAF Configuration XML Storage in Server and Side PC

The class represents the input of TAF.

TAF Result XML Storage in Server and Side PC

The class represents the output of TAF.

The mapping for the class diagram is shown in Deployment Diagram in Figure 22

Server Side PC

Allura XPer System

Scheduler Server

TAF

Step Manager

Scheduler Agent

User PC

Browser TAF

Database

Server XML Storage

Agent XML Storage

The Database contains : SidePC,Command,Schedule, ScheduleResult, ActivityBlockTemplate, ActivityTemplate

The XML Storage contains :SidePC Folder, Command, Schedule, Schedule Result, Step Manager Recipe, Step Manager ResultTAF Configuration, TAF Result

The Agent XML Storage is copied fromServer XML Storage, It contains:Command, Schedule, ScheduleResult, Step Manager Recipe, Step Manager Result,TAF Configuration,TAF Result

The result is copied back to server and deleted after it finished copied

Figure 22 – Mapping Class Diagram to Deployment Diagram

Figure 22 and Table 13 show that the data classes (Command, Schedule, Schedule Result, Step Manager Recipe, Step Manager Result, TAF Configuration and TAF Result) are shared by Server and Side PC in the XML Storage. Scheduler Server is using the database as its data storage and the data is generated into XML File in Server XML Storage. Scheduler Agent gets the XML File from the Server XML Storage and copies it to the Agent XML Storage in the Side PC. The implementation of these components is shown in the next chapter.

7.2.2. Detail Sequence Diagram The Scheduler System is designed further from the sequence diagram in Figure 19. User Interrupt is added in the schedule execution. The sequence diagram is made from the use case in Figure 13. The sequence diagram is separated into two different stages: Data Input and Schedule Execution.

Data Input Figure 23 shows the data input sequence diagram. The tester’s browser is used to access Scheduler Server. Scheduler Server provides data input functionality (for da-tabase and XML Storage). Scheduler Agent copies XML files from Server XML Storage to Agent XML Storage.

Figure 23 – Data Input Sequence Diagram

Schedule Execution & User Interrupt Figure 23 shows the sequence diagram of schedule execution. User Interrupt is also included in the sequence diagram. User Interrupt is generated as command XML. Scheduler Agent provides schedule execution and command execution. Step Manag-er provides Activity Block execution. TAF provides Activity execution. Scheduler Server provides input for User Interrupt and updates to database.

Scheduler Server Scheduler AgentBrowser of Tester Database Server XML Storage

TimeTrigger

Step Manager

TAF

Run(StepManagerRecipe)

Run(Test.taf)

Finished

UpdateScheduleResult

SaveData

CheckResult

Generate(Command.xml)

Command = DeleteResult

Execute(Command.xml)

ExecuteCommand(DeleteResult)

Stop


Command = Stop

Execute(Command.xml)

ExecuteCommand(Stop)

StopCopy(CommandResult.xml)

StopCopy(CommandResult.xml)

SaveData

Agent XML Storage

GetScheduleData

ScheduleData

Finished

Put(ScheduleResult)

Copy(ScheduleResult)

Put(ScheduleResult) will copyall the schedule result to the SidePC

Figure 24 – Scheduler Execution Sequence Diagram

7.2.3. State Machine Diagram In this subchapter, the state machine of Scheduler and Step Manager are described.

Scheduler Agent State Machine The Scheduler Agent state machine diagram is shown in Figure 25. The Scheduler Server gets the state by sending command to Scheduler Agent.

Figure 25 – Scheduler Agent State Diagram The explanation of each state can be found in Table 14

Table 14 – Step Manager State State Description Idle The scheduler agent is running but there is

no schedule being executed at that mo-ment.

Running Step Manager The scheduler is executing Step Manager. TimeOut The scheduler is executing Step Manager,

and it has exceeded the estimated execu-tion time.

Stopped The scheduler is stopped (by the user). Confirmation The scheduler is showing the confirmation

window to the user

Step Manager State Machine Several states of Step Manager are shown in Figure 26.

Not Active Idle

Execute

Start Step / TAF.Run,TAF.StartTest

Running Single

Running Batch

Start Batch / TAF.Run,TAF.StartTest

when: Finished()

when: Finished()

Exit

Stop / TAF.Stop,TAF.Exit

Stop / TAF.Stop,TAF.Exit

Figure 26 – Step Manager State Diagram The explanation of each state can be found in Table 15. Table 15 – Step Manager State State Description Not Active There is no running instance of Step Man-

ager Idle Step Manager is running but there are no

steps being executed Running Single Step Manager is running only one step.

After the step finishes Step Manager re-turns to Idle

Running Batch Step Manager is running a sequence of steps

7.2.4. Components Design Summary After designing the sequence diagram of scheduler system, we move further to de-velop and implement the components. At this point, three main components need to be developed: Scheduler Server, Scheduler Agent and Step Manager. The data stor-age (Database and XML Storage) is used by these components and need to be devel-

oped and integrated inside them. The Step Manager modification is needed since currently there is no way to communicate with Step Manager (Command Line Inter-face needs to be added). The next chapter will describe the implementation of the components. Below is a short description of the implementation of each component: Scheduler Server:

o It provides data input as web application o It stores the data in the database and generated in the XML Storage o It updates the result of schedule execution to the database

Scheduler Agent: o It provides schedule execution (Control of Step Manager Execution) o It gets the data from the server o It puts the result of the schedule execution to the server

Step Manager o It provides the Activity Block execution (Control of TAF Execution) o It gets the Step Manager recipe as input o It gives the result of activity block execution as output.

■

8.Implementation Abstract – In this chapter, the implementation of the scheduler system components is described. These are divided in two groups: data and tool components. Data compo-nents consist of Database and XML Storage (Server and Agent). Tool components consist of Scheduler Server, Scheduler Agent, and Step Manager.

8.1 Data Components In this subchapter, we further describe the development of Database and XML Stor-age.

8.1.1. Database Database is used as data storage for server application. The database should contain the data classes listed:

Side PC – List of available test systems Command – Command Data from Scheduler Server to Scheduler Agent Schedule – Contains the schedule time and the link to activity block data Schedule Result – Contains the result of schedule execution Activity Block Template – Contains the sequence of activities Activity Template – Contains the link to TAF File

The database is shown in the Database Diagram in Figure 27. The database is made with the aim of integrating it with TE in the future. Scheduler System tables (Green color) will have a field to accommodate the integration with TE tables (Blue Color). Only Scheduler System tables are developed during this project.

Figure 27 – Database Diagram Legend: PK – Primary Key FK – Foreign Key Bold - Required

TE Scheduler System

8.1.2. XML Storage The XML Storage (Server and Agent) is needed for storing the data for Side PC. The Server XML storage is a shared folder in the server. The Agent XML Storage (in the Side PC) is the copy of the “Server/<SidePC>” folder from the Server XML Storage (see Figure 28). For every different SUT configuration, a different TAF file is needed since the SUT configuration is stored inside the TAF configuration. Therefore, the XML Storage is designed to have a subfolder for each Side PC. The details of the folder structure of Server and Agent XML Storage are listed below (bold – folder):

Template

<SidePC> TAF

<SmokeTest.taf> <ReliabilityTest.taf>

Server <SidePC>

Schedule.xml Command.xml <ScheduleId>

Recipe-<ScheduleId>.xml TAF

TAF Config <SmokeTest.taf> <ReliabilityTest.taf>

Result ScheduleResult.xml CommandResult.xml <ScheduleId>

Result-<ScheduleId>.xml TAF Result

<SmokeTest>–<DateTime> <ReliabilityTest>–<DateTime>

Figure 28 – XML Storage Structure

The explanation of each file and folder from the folder structure above can be seen in Table 16

Table 16 – XML Storage Folder Structure Type Folder / File Description Folder Template Folder in Server to store the template data Folder <SidePC> Subfolder of Template folder – it contains the data related

to a SidePC. The folder name is the Side PC name Folder TAF Subfolder of Template/<SidePC> folder to store the

TAF data File <SmokeTest.taf>

<ReliabilityTest.taf> TAF configuration files – they are used as input for TAF to execute an Activity. The file name is determined by the user

Folder Server Folder in Server to store the data for Side PC Folder <SidePC> Subfolder of Server folder – it contains the data related to

a Side PC. The folder name is the Side PC name File Schedule.xml The file contains the schedule data. The file is generated by

the Scheduler Server File Command.xml The file contains the command data. The file is generated by

the Scheduler Server Folder <ScheduleId>

Subfolder of Server/<SidePC> folder – – it contains the data related to a schedule. The folder name is the Id of the schedule from the database

File Recipe-<ScheduleId>.xml

The Step Manager Recipe File – it contains the activity block data

Folder TAF Subfolder of Server/<SidePC>/ <ScheduleId> folder used to organize TAF data

Folder TAF Config Subfolder of Server/<SidePC>/ <Schedu-leId>/TAF folder that contains TAF configuration files

Server XML Storage

Agent XML Storage (Side PC)

File <SmokeTest.taf> <ReliabilityTest.taf>

TAF configuration files – they are used as input for TAF to execute an Activity

Folder Result Subfolder of Server/<SidePC> folder – it contains the results of schedule execution. The Scheduler Agent puts the results in this folder.

File ScheduleResult.xml The file contains the results of schedule execution. The file is generated by the Scheduler Agent

File CommandResult.xml The file contains the results of command execution. The file is generated by the Scheduler Agent

Folder <ScheduleId>

Subfolder of Server/<SidePC>/Result folder – it contains the result data related to a schedule. The folder name is the Id of the schedule from the database

File Result-<ScheduleId>.xml

The Step Manager Result File – it contains the result data of activity block execution

Folder TAF Result The folder contains the result data of TAF execution. Scheduler Agent collects the result data in this folder.

Folder <SmokeTest>–<DateTime> <ReliabilityTest>–<DateTime>

The folders contain the result of TAF execution

8.2 Tool Components In this subchapter, further development for Scheduler Server, Scheduler Agent, and Step Manager are described. TAF is not explained in this chapter since no changes in TAF is required for this project.

8.2.1. Scheduler Server The implementation of Scheduler Server is described in this subchapter. The imple-mentation consists of website implementation as user input and File Watcher imple-mentation as result updates.

User Input The Scheduler Server is a web application. The website is built using the MVC (model-view-controller) pattern. The Scheduler Server uses Entity Framework to generate the model of database. The diagram of the MVC pattern can be found in Figure 29

Figure 29 – Model View Controller The explanation of the MVC pattern can be found below:

Controller It handles the user request from the webpage. It gathers the model, manipu-lates the model and shows the model in the view.

Model It is implemented using Entity Framework. The generation of the model is done automatically and synchronized with the database. Changes in the da-tabase are handled by generating the model again through Entity Framework (by running the custom tool).

View

Model

View

Controller Request

Browser (User)

Response

Manipulates

Updates

It is implemented by Razor / cshtml. The View handles the display format of the model.

The Scheduler Server website is shown in Figure 30

Figure 30 – Scheduler Server User Interface The overview of menus seen in the Figure 30 is described in more detail in Table 17. Each of the menus has its own class of MVC (Model – View – Controller). Table 17 – MVC Components MVC Components Components Responsibility SidePC To handle the SidePC CRUD in the database

To handle actions like e.g. start and stop Schedule To handle the Schedule CRUD in the database

To handle the generation of Schedule.xml To handle the generation of StepManagerRecipe.xml To update the ScheduleResult in the database from

ScheduleResult.xml Command (Hidden) To handle the communication with Side PC

To handle the generation of Command.xml To update the CommandResult in the database from

CommandResult.xml Activity Block To handle the ActivityBlock CRUD in the database

(Activity Block and Activity Block Detail) Activity To handle the Activity CRUD in the database

To handle TAF File Upload and Download

File Watcher File watcher functionality is needed to monitor the result of schedule execution. Scheduler Agent copies the result (a XML file) to the server. This file copy event triggers the File Watcher. The File Watcher executes the updateResult function, which parses the copied file and puts the result in the database.

The sequence diagram of File Watcher (Figure 31) is developed from the sequence diagram in Figure 24. Figure 31 shows the command result update and the schedule result update.

Scheduler Server Scheduler AgentDatabase Server XML Storage

FileWatcher(ScheduleResult.xml)

SaveData

CheckResult


Command = DeleteResult

FileWatcherCommand.xml)

Update Result

Get(ScheduleResult.xml)

ScheduleResult.xml

Put(CommandResult.xml)

FileWatcher(CommandResult.xml)

Get(CommandResult.xml)

CommandResult.xml

Update Result

SaveData

Local XML Storage

Put(ScheduleData)

Copy(ScheduleResult.xml)

Copy(StepManagerResult.xml)

Copy(TAFResult)

Schedule Data Contain Schedule Result, Step Manager Result and TAF Result

copy(CommandResult.xml)

Figure 31 – Scheduler Server FileWatcher Two files need to be watched: CommandResult.xml and ScheduleResult.xml. CommandResult.xml It contains the result of command execution. Several commands with their possible result can be found in the Table 18.

Table 18 – Scheduler Command Result Command Result NewSchedule Success / Failed Stop Success / Failed Start Success / Failed GetState Several possible value:

Idle RunningStepManager TimeOut Stopped

Confirmation PutResult Success / Failed RemoveResult Success / Failed

ScheduleResult.xml It contains the result of schedule execution. The possible result values are listed be-low:

0 – Success 1 – Failed 2 – In Progress 3 – Not Applicable 4 – Skipped 5 – Stopped 6 – Interrupted / Suspended 7 – Scheduled

8.2.2. Scheduler Agent In this chapter, Scheduler Agent implementation is documented. The implementation consists of Time Trigger implementation as schedule execution and File Watcher implementation as XML Storage monitoring. The user interface of Scheduler Agent is a small menu in the task bar (shown in Fig-ure 32).

Figure 32 – Scheduler Agent User Interface

Time Trigger The time trigger functionality is realized with timer. The scheduler can only execute one activity block at a particular time. Because of this, the scheduler will only keep track of the current activity block. After it finishes, the next activity block is sched-uled and executed. The sequence diagram of time trigger functionality is shown in Figure 33.

Scheduler Agent

Figure 33 – Scheduler Agent Time Trigger

File Watcher File Watcher is needed to trigger the Scheduler Command. File Watcher monitors the Command.xml that is created by Scheduler Server. When file creation event happens File Watcher triggers the Scheduler Agent to execute the Command.xml. After the execution, the CommandResult.xml is copied to the server and the Command.xml is deleted. The commands and their description are listed in Table 19. Table 19 – Scheduler Command Command Parameter Description NewSchedule ScheduleId To indicate there is a new schedule Stop To instruct the SidePC to stop Start ScheduleId

(Optional) To instruct the SidePC to start the schedule

GetState To get the state of the scheduler in the Side PC PutResult ScheduleId To instruct the SidePC to put the result to the

server RemoveResult ScheduleId To instruct the SidePC to remove the result since

the server already has the result The sequence diagram of File Watcher is shown in Figure 34. The sequence diagram shows the NewSchedule command as an example.

Figure 34 – Scheduler Agent File Watcher

8.2.3. Step Manager Step Manager needs to be modified to support communication with Scheduler Agent. Command line interface was chosen to provide the command input to Step Manager. This subchapter describes the implementation of command line interface in Step Manager. The ‘Required’ condition added in Step Manager is also described in this subchapter.

Command Line Interface Command line parameter is used to provide command input to Step Manager. In the following example, we shall demonstrate this and show how to issue a ‘stop’ com-mand. Step Manager is started with the following command:

StepManager.exe –auto This creates one running instance of Step Manager. In order to stop it we can issue the following command:

StepManager.exe -stop This command should stop the running Step Manager, but this is not possible since this command executes the second Step Manager instance. To solve this problem, a way needs to be found to send the ‘stop’ command to the first instance of Step Man-ager. To realize this, TCP/IP connection is added to Step Manager. It allows the sec-ond instance of Step Manager to communicate with the first instance. The sequence diagram of this mechanism can be seen in Figure 35.

Figure 35 – Step Manager Sequence Diagram The Step Manager Command parameters are listed below. The blue color shows that a command needs to be developed.

Table 20 – Step Manager Command Parameters Command Parameters Description Allow

Combination -auto To start the execution of steps au-

tomatically (without this com-mand, Step Manager will not start executing the steps)

Yes

-recipeFile=<SMRecipe.xml> To set the recipe file Yes -resultFile=<SMResult.xml> To set the result file Yes -getState To get Step Manager state No -closeAfter To close Step Manager after exe-

cuting the steps Yes

-stop To stop the execution No -exit To exit Step Manager No

Required Condition There are two XML files related to Step Manager: Recipe.xml and Result.xml. Table 21 shows the file interface for Step Manager. The ‘Required’ condition field was added to the recipe. This field is an indication whether Step Manager can continue to the next step or not when the current step fails. The example of these files can be found in the Appendix D. Table 21 – Step Manager File Interfaces Files Description Recipe.xml The recipe file provides the input for Step Manager Result.xml The result file stores the output from Step Manager ■

9.Verification & Validation Abstract – In this chapter, the verification and validation phase is documented. The system design is verified and the system requirements are validated.

9.1 Verification The system design is verified by comparing the implementation against the design. The author performs the verification. Table 22 shows the table for the design verifi-cation. Table 22 – Design Status Design Status TAF – Execution Pass TAF – Stop Pass TAF – Result Pass Step Manager – Execution Pass Step Manager - TCP/IP Connection (First and Second Application) Pass Step Manager - TCP/IP Connection – Command Line Arguments Pass Step Manager - Command Line – recipeFile Pass Step Manager - Command Line – resultFile Pass Step Manager - Command Line – getState Pass Step Manager - Command Line – closeAfter Pass Step Manager - Command Line – stop Pass Step Manager - Command Line – start Pass Step Manager - Command Line – close Pass Step Manager - RecipeFile – StopCommand Pass Step Manager - RecipeFile – Required Pass Step Manager – Result Pass Scheduler Agent – Read schedule.xml Pass Scheduler Agent – Scheduling Execution Pass Scheduler Agent – Write scheduleResult.xml Pass Scheduler Agent – File Watcher Pass Scheduler Agent – Get File Pass Scheduler Agent – Command Pass Scheduler Agent – Put File Pass Database – Design Pass File Store – Design Pass Scheduler Server – Schedule – Database CRUD Pass Scheduler Server – Activity Block Template – Database CRUD Pass Scheduler Server – Activity Template – Database CRUD Pass Scheduler Server – TAF – File Upload Pass Scheduler Server – Step Manager – XML CRUD Pass Scheduler Server – Schedule – XML CRUD Pass Scheduler Server – Command Pass Scheduler Server – Get Result Pass Scheduler Server – Layout and other User Interface Pass

9.2 Validation The implementation is compared against the requirements. The user performs the validation. Table 23 shows the table for requirement validation.

Table 23 – Requirement Status Requirement Status SSTAF.Activity Pass SSTAF.Activity.Execution Pass SSTAF.Activity.FileStore Pass SSTAF.Activity.Result Pass SSTAF.Activity.EstimatedTime Pass SSTAF.Activity.Stop Pass SSTAF.ActivityBlock Pass SSTAF.ActivityBlock.Popup Pass SSTAF.ActivityBlock.Stop Pass SSTAF.ActivityBlock.EstimatedTime Pass SSTAF.ActivityBlock.Result Pass SSTAF.ActivityBlock.Template Pass SSTAF.Schedule Pass SSTAF.Schedule.Access Pass SSTAF.Schedule.ActivityBlock.Parameter Pass SSTAF.Schedule.OverlappingSchedule Pass SSTAF.Schedule.OverlappingSchedule.Warning Pass SSTAF.Schedule.AdjustSchedule Pass SSTAF.Schedule.AdjustScheduleConstraint Pass SSTAF.Schedule.ResponseTime Pass SSTAF.SidePC.IsAlive Pass SSTAF.Reporting.Result Pass SSTAF.Reporting.GanttChart Not done ■

10. Project Management Abstract – In this chapter, the project management tasks are documented. The way of working, project schedule, and risk management are described.

10.1 Way of working The project was managed using agile software development method. The idea with this method is to develop a system through repeated cycles (iterations) in smaller portions of time (incremental). This way of working decomposes the system func-tionality into smaller chunks. A chunk of functionality is delivered per iteration. Up-dates on the documentation were also made per iteration. There are three big phases of the project: inception, development and test. Each phase has its own deliverables. In the inception phase, the deliverables are the document and presentation. Rapid meetings with the users were held in this phase. Later, weekly meetings were held. The meetings were held to define the requirements and the scope of the project. Re-quirement Document and Design Document were made as the deliverable of the first stage. Research about the common off the shelf tools and their implementation was also done in this phase. In the development phase, the deliverable is the software system. Two weekly soft-ware demos with the customer were held in this phase. The demos had two goals:

To update the user with the status To get the feedback about the project direction

After getting the customer feedback, the software was changed and the document was updated based on the changes. In the testing phase, the deliverables are the testing documents. Tests have been done with the users to determine whether the system functions correctly. Further details of the breakdown of the iteration can be seen in the Figure 36.

10.2 Project Schedule The project schedule was changed several times since there was new information that changed the project direction. The final version of the project Gantt chart can be found in Figure 36.

Figure 36 – Project Gantt Chart

Legend:

Task

Documentation

Presentation and Meeting

Evaluation & Feedback

10.3 Risk Management Several issues and risks are investigated. Table 24 lists several risks that might affect the project. Table 24 – Issue and Risk Issues or Risks Likelihood Critical Mitigation Strategy Customer do not have time High Medium Plan early Customer changes his mind Medium High Record every decision Organization problem High Medium Management Awareness Cannot use the SUT Low Low Plan early Cannot solve the technical problem

Low High Get Expert Advice

Network problem Medium Medium Help from the IT system management

Communication Problem Medium High Frequent update and communication Misinterpretation Medium High

Expectation difference Medium High Integration necessary with TE

Low Medium Consider the integration in the Design

Get more requirements in the later stage

Medium High Frequent Demo

Not enough time Medium High Plan & Requirement man-agement

New tool to be used / inte-grate

Medium High Requirement management

New information that en-dangered the project

Medium High

Supervisor do not satisfied with the performance

Medium High Frequent update and communication

Project Cancellation Medium High - Demo Failed Medium High Prepare and Revise all the

step Presentation Failed Medium High Prepare, get comment and

feedback, Practice before the presentation

■

11. Conclusion Abstract – In this chapter, the conclusions of the SSTAF project are described: the results, the lessons learned, and the future works.

11.1 Results The scheduler system for Test Automation Framework has gone through all the phas-es of development cycle. The requirements were defined with the user in the starting phase of the project. Then, several common off the shelf tools and related Philips tools were explored in the investigation phase. After the selections, several architec-ture and design decisions were made in the design phase. Then, the implementation of the system was done, followed by verification and validation phase. The project management was done in Agile with iterations. Periodically the results were presented to the customer to provide updates and obtain feedback. This was done to ensure that the project was on the track. Many architecture decisions were made in this project. In every architecture decision, the advantages and the disad-vantages were investigated and the decision was taken based on the analysis. The design decisions taken in this project provide several advantages listed below:

By using TAF o The Scheduler System can be used by other Philips organization

that use TAF as their testing framework By using Step Manager

o The Scheduler System reuse the Factory Tooling that make the au-tomatic installation possible

By having Activity Block (Template) o The tester can reuse the sequence of activities

By having Smart Agent o The scheduler system is more reliable and it can operate without

server connection o The schedule execution will have no network delay

By having Website and Database o The system is easily accessible by User. o The future integration with Test Environment (TE) can be easily

realized Several conclusions related to this project are listed below:

Scheduler Server was created to handle the user input, using web technolo-gy.

Database was designed and used to store the schedule data for the server. XML Storage was designed and used to store the schedule data for the Side

PC. Scheduler Agent was created to handle the time trigger and step manager

control. Step Manager was modified to handle the command and the “required” con-

dition. Connection to TAF was made to handle the execution of TAF Script.

11.2 Lessons Learned The SSTAF project had both technical and non-technical challenges. As an important technical challenge, SSTAF had to accommodate different available applications. Complex applications were involved and it was needed to understand how they worked. The strategy was chosen to reuse the available applications as much as pos-sible instead of developing the new ones. As an example of non-technical challenges,

SSTAF faced some organizational problems that had an impact on the project. How-ever, the project was continued with the same initiative and goal. Regarding the technical challenge, it was learned that it was necessary to allocate enough time to understand the available tool in depth. Most of the tools used in Philips had been used for a long time and they were stable. Only small changes could have been done at a time since the knowledge of the tools was quite limited. Regarding the non-technical challenge, it was learned that the organizational issues could influence the project greatly. It was learned that these organizational issues needed to be solved in the early stage of the project. This was necessary to ensure the continuation of the project. Another non-technical challenge was the communication. The project related infor-mation needed to be given to the stakeholders and their feedback needed to be gath-ered and synchronized. This was not always an easy task to do. Following the meet-ings with stakeholders, the minutes were handed out by email but that turned out to be insufficient. So it became necessary to update the stakeholders by short direct communication. The communication with the customers as one of the most important stakeholders is the key to make the project successful. As the summary, it was essential to understand and detect the risks early. Investiga-tion and feasibility studies were necessary as a mitigation strategy to remove or at least reduce the detected risks. However, even though it was important to overcome the risks, the crucial tasks with lower risk were given higher priority. This prioritiza-tion and risk analysis helped the author to handle the project successfully.

11.3 Future Works The TE future integration is necessary. The Scheduler System has been prepared for the integration. The integration will be through the database. The web application of the Scheduler System can be reused and integrated in TE. The automatic installation of the new Allura System is currently still in development. It is a part of Factory Tooling project. Future integration with Factory Tooling is nec-essary in order to reach the automatic installation. Design of Factory Tooling is aligned with SSTAF project. With this in mind, the integration in the future can be done easily. ■

Appendix A. COTS Selection The comparison between the types of COTS tools against the functionalities can be seen below. Table 25 – Domain against functionality Tool \ Functionality Scheduling RunningTask

Automatically Controlling Access

Reporting

Calendar Yes No Yes Yes Job Scheduler Yes Yes Partial Partial Continuous Integration Yes Yes Partial Partial For the analysis of COTS tools that can serve the purpose, two types of tools are con-sidered. They are the job scheduler and the continuous integration. The calendar is not included on the short list because the calendar is not intended to execute a task/activity. List of all the job schedulers and continuous integration products is found in [7] and [8]. The selection was made in several stages based on the infor-mation available.

Job Scheduler To select the scheduler, several criteria were inspected in order to match the project requirements. In the first round, three criteria were used. Note that the criteria match the features defined in [7].

Windows support. The majority of servers within iXR run under the Windows Operating Sys-tem. Support and maintenance of a product is much easier when it runs on the same OS.

Low cost (No ERP support). The lack of ERP support is used as indication of low cost. Investigation of the price is done after this condition is met.

Support for Role Based Security (RBS). RBS is considered the same as access control functionality.

Table 26 – Job Scheduler First Round Selection

Job Scheduler Windows support Low Cost (No ERP) Role Based Secu-

rity ActiveBatch Yes No Yes

AutoSys Yes No Unknown

Batchman Unknown No Unknown

Control-M Yes No Yes

Dollar Universe Yes No Yes

JAMS Scheduler Yes No Yes

JobScheduler Yes No No

OpCon Yes No Yes

Skybot Scheduler Yes No Yes

Sked No No No

Tivoli Workload Sched-uler

Yes No Yes

Grid Engine Unknown Yes Open Source Unknown

Hinemos Yes Yes Unknown Yes

JobServer Yes Yes USD $2,995.00 - Yes

Up to 100 Jobs

Juice Box Yes Yes Free Partial

Platform LSF Yes Yes Unknown Unknown

ProActive Parallel Suite Yes Yes Open Source Yes

Schedulix Yes Yes Open Source No

VisualCron Yes Yes $1,999.00 with 1 year support

Unknown

Cisco Tidal Enterprise Scheduler

Yes Yes Unknown SAP, PeopleSoft

After the first round, there were still too many job schedulers to consider in more depth. Therefore, other criteria were introduced:

Intended use Is the product built for the same or similar task? A big deviation of the in-tended use means that the product will have too many capabilities that we do not need or lacking essentials that we do need.

After the second round, only two tools were considered for the selection: Juice Box and ProActive Parallel Suite.

Continuous Integration Tool To select the scheduler, several criteria of the continuous integration tool from [8] were inspected to match the project requirements.

Supported Platform It must be Windows or Cross-platform for the same reasons as described for the Job Server.

Development Language iXR department has many C, C++, C# and batch programmers. For ease of maintenance, the selected tool should support any of these languages. Philips does not have many people for other language skills, so Java and Py-thon are out.

Table 28 – Continuous Integration First Round Selection

Scheduler \ Criteria Platform Development lan-guage

AnthillPro Yes Yes

Apache Continuum Yes No

Apache Gump No No

AppVeyor CI No Yes

AutoDE No Yes

Table 27 – Job Scheduler Second Round Selection Job Scheduler Intended Use

Grid Engine It is meant for grid computing computer cluster software system. No need for cluster

Hinemos It manages several servers, computers, and other devices, as if it were only managing one single computer. No need for this func-tionality

Juice Box It is a simple job scheduler but not able to extend

Platform LSF It is for distributed HPC environments. No need for High Per-formance Computing

ProActive Parallel Suite It uses XML in the scheduler. It focuses on Parallel processing for Grid and Cloud Computing.

Cisco Tidal Enterprise Scheduler It focuses on big data

Automated Build Studio Yes No

Bamboo Maybe Yes

Build Forge Yes Yes

BuildBot No Yes

BuildMaster Yes Yes

CABIE No No

CircleCI No No

node.ci No No

Codeship No No

CruiseControl Yes Yes

CruiseControl.NET Yes Yes

Draco.NET Yes No

Drone.io No Yes

EasyCIS Yes No

ElectricCommander Yes Yes

Go Yes Yes

Integrity No Yes

Jenkins/Hudson Maybe Yes

Koality Yes Yes

Lordui Yes Yes

LuntBuild Maybe Yes

QuickBuild Yes Yes

Semaphore No Yes

Shippable No No

Strider No Yes

Team Foundation Server Yes Yes

TeamCity Maybe Yes

Tinderbox Yes No

Travis-CI No Yes

After the first round, there are still too many continuous integration tools to consider for detailed analysis. The following criteria were used for further selection:

Scheduler support It was investigated from the description in [8] whether the tool has a job scheduler. The keyword search is the “nightly build”. If the tool has this fea-ture, then it can be concluded that it has scheduler or time trigger functional-ity.

Expected Cost The one-time fee should not exceed $2000 and/or the yearly fee not more than $1000/year.

Tool Comparison The next criteria to be considered was the usage in the Philips organization, timeline presentation, and user interface extensibility. The criteria of user interface extensibil-ity were added after the demo with a system test in mobile environment.

Usage in the organization If the tool is already used within the organization this is a plus (+), because we already have people that are familiar with the product, zero (0) if it is not.

Timeline presentation The tool scores minus (-) if the tool does not support timeline presentation or if it takes a lot of effort to add that. It scores zero (0) if the product can be extended easily with additional code. It scores plus (+) if it supports timeline representation available.

UI Extensibility The product scores minus (-) if it cannot, or with great effort, be extended to support additional UI screens, zero (0) if moderate effort is needed, plus (+) if it is easy to extend.

Table 29 – Continuous Integration Second Round Selection

Continuous Integration Scheduler Expected Cost

AnthillPro Yes

Pricing starts at $999 USD for small teams

Bamboo Needs some trick 10 Agents $250 USD/month

Build Forge Yes

User License USD $823.00. Can be integrated with ClearCase and ClearQuest.

BuildMaster Yes Up to 10 users $1999 per year

CruiseControl Difficult open source, BSD-style license

CruiseControl.NET Needs some trick open source, BSD-style license

ElectricCommander Yes Already bought, no extra cost

Go Unknown Up to 100 Process - $5,000.00

Jenkins/Hudson Yes open source, already used

Koality No Unknown

Lordui No Unknown

LuntBuild Yes Open Source

QuickBuild Unknown

Commercial License 3600 USD per physical site

Team Foundation Server Done by Script

One device client access for Visual Studio Team Foundation Server 2013 $499.00

TeamCity Yes Up to 100 build agents $1,999

Table 30 – Tool Selection

Scheduler Tool Used in the organization Timeline

Presentation UI Extensibility Overall

Score Juice Box 0 - - - -

ProActive Parallel Suite 0 0 0 0

CruiseControl or Cruise-Control.NET

0 0 0 0

LuntBuild 0 0 0 0

Build Forge 0 0 - -

ElectricCommander + 0 0 +

Jenkins/Hudson + 0 + ++

COTS tool selection produces Jenkins as the result. It is an open source project. It has many plugins and has different customization and it is easy to extend. However, it also might take more time to customize and extend Jenkins to have the desired func-tionality. ■

Appendix B. Design Evolution There are many changes of architecture during the project lifetime. New information is collected and it causes some changes in the architecture. The evolution of the de-sign can be seen below.

Initial Design Description: This design was made with the assumption that TAF should be the only connection to the SUT. This architecture makes a clear functionality separation between TAF and the scheduler. Reason not to use: It needs time trigger functionality in the Side PC

Alternative Design with Server and Agent Description: Client server architecture is needed since the scheduler needs time trigger function-ality on the side PC. The scheduler agent should receive the schedule and execute TAF. Reason not to use: The server components need to be modular.

Alternative Design with TE (Test Environment) Description: Since there is a planner tool being developed to schedule manual tests, it is quite clear that both manual and automatic tests should be integrated. The overview of the architecture can be seen in the diagram. Reason not to use: The management decided to build ASPTE and TE separately. TE Integration is not going to be done in this project timeline.

Alternative Design with Web Service without TE Description: Web Service was made to supply the scheduler agent with the necessary data. Reason not to use: File interface through file sharing is considered a better option than Web Service since Web Service is not built for “push” mechanism.

■

Appendix C. Scheduler System Swim Lane Diagram

The flow or swim lane diagram is made to show how components interact with each other. The Swim Lane Diagram (Figure 20) is developed from the activity diagram shown in Figure 16.

Figure 37 – Swim Lane Diagram Each of the elements shown in the Swim Lane diagram is described in short detail in Table 31 Table 31 – Swim Lane Element Swim Lane Element

Component Description

Start TAF in UserPC Starting point of the system Create TAF Test File

TAF in UserPC Create TAF Test File using the TAF software in the user PC

Test Step for TAF

TAF in UserPC, TAF in SidePC

The TAF configuration file (XML)

CRUD Activity Browser in UserPC Create, Read, Update, or Delete the Activity Choose Test Step for TAF

Browser in UserPC The upload process of TAF configuration file to the server

CRUD Activity Block

Browser in UserPC Create, Read, Update, or Delete of the Activity Block

CRUD Schedule

Browser in UserPC Create, Read, Update, or Delete the Schedule

Save Data Scheduler Server Save data to database Generate File Scheduler Server Generate XML File Database Scheduler Server The database storage Command XML Storage,

Scheduler Agent in SidePC

The command data (XML)

Schedule XML Storage, Scheduler Agent in SidePC

The schedule data (XML)

Step Manager Recipe

XML Storage, Step Manager in SidePC

The step manager data (XML)

Uploaded Test Step for TAF

XML Storage The TAF configuration file (XML)

Get Server Data Scheduler Agent in SidePC

Get data from the server to the SidePC

Execute Com-mand


Execute command data

Schedule Time Trigger


Register the schedule data in the time trigger

Check Step Manager


Check Step Manager state

Execute Step Manager


Initiate the execution of Step Manager

Step Manager Execution

Step Manager in SidePC

Process of running Step Manager

TAF Execution TAF in SidePC Process of running TAF Command Re-sult

Scheduler Agent in SidePC, XML Storage

The command result data

Schedule Result Scheduler Agent in SidePC, XML Storage

The schedule result data

Step Manager Result

Step Manager in SidePC, XML Storage

Step Manager result data

TAF Result TAF in SidePC, XML Storage

TAF result data

Put Server Data Scheduler Agent in SidePC

Put data from the SidePC to the server

Update Result Scheduler Server Update results to Database Show Result in Report

Scheduler Server Show the results in Reporting

End Scheduler Server End point of the system ■

Appendix D. XML Files There are many xml files are needed for SSTAF. The detailed files can be seen be-low.

SCHEDULER AGENT FILES

Schedule.XML The example of the Schedule.xml is shown below. One Schedule.xml only made for scheduling one SUT.

<Scheduler> <MachineName>TestMachine1</MachineName> <IPAddress>localhost</IPAddress> <Schedule> <Name>Test1</Name> <Person>Djohan Wahyudi</Person> <RecipeFile>MCIEStepManagerRecipe.xml</RecipeFile> <ResultFile>MCIEStepManagerResult.xml</ResultFile> <PlannedDateTime>2014-05-21T16:15:00</PlannedDateTime> </Schedule> <Schedule> <Name>Test2</Name> <Person>Djohan Wahyudi</Person> <RecipeFile>MCIEStepManagerRecipe1.xml</RecipeFile> <ResultFile>MCIEStepManagerResult1.xml</ResultFile> <PlannedDateTime>2014-05-21T16:16:00</PlannedDateTime> </Schedule> … </Scheduler>

ScheduleResult.XML The example of the scheduleResult.xml is shown below. The blue color shows the difference between the Schedule.xml and the ScheduleResult.xml.

<Scheduler> <MachineName>TestMachine1</MachineName> <IPAddress>localhost</IPAddress> <LastPerformedSchedule>1</LastPerformedSchedule> <Result>Successful</Result> <State>Idle</State> <Schedule> <Name>Test1</Name> <Person>Djohan Wahyudi</Person> <RecipeFile>MCIEStepManagerRecipe.xml</RecipeFile> <ResultFile>MCIEStepManagerResult.xml</ResultFile> <PlannedDateTime>2014-05-21T16:15:00+02:00</PlannedDateTime> <Result>Successful</Result> <StepManager> <StartDateTime>2014-05-21T16:37:36.6000096+02:00</StartDateTime> <StopDateTime>2014-05-21T16:39:38.8774344+02:00</StopDateTime> <Result>Successful</Result> </StepManager> <StepManager> <StartDateTime>2014-05-21T17:39:36.6000096+02:00</StartDateTime> <StopDateTime>2014-05-21T17:41:46.8774344+02:00</StopDateTime> <Result>Successful</Result> </StepManager> </Schedule> <Schedule> <Name>Test2</Name> <Person>Djohan Wahyudi</Person> <RecipeFile>MCIEStepManagerRecipe1.xml</RecipeFile> <ResultFile>MCIEStepManagerResult1.xml</ResultFile> <PlannedDateTime>2014-05-21T16:16:00</PlannedDateTime> </Schedule> … </Scheduler>

Command.XML and CommandResult.XML The example of the Command.xml and CommandResult.xml is shown below. The ExecutedTime and the Result field in the CommandResult.xml are filled af-ter the execution (It was shown by the blue color).

<SidePC> <Name>SUT1</Name> <IP>192.121.121.121</IP> <Command> <Id>b7a9011b-3064-49ef-a0bd-b79aecabe90c</Id> <Command>NewSchedule</Command> <Parameter>9fef45ae-a07e-4080-ae5c-f879b0b52573</Parameter> <Time>2014-07-28T10:55:03.3628112+02:00</Time> <ExecutedTime /> <Result>0</Result> </Command> <Command> <Id>2fc405b0-e01a-438f-966f-e67c8107ed82</Id> <Command>NewSchedule</Command> <Parameter>ca21fec1-99c1-4676-bf6a-cb30dd11bdf4</Parameter> <Time>2014-07-28T10:52:24.27</Time> <ExecutedTime /> <Result>0</Result> </Command> .. </SidePC>

STEP MANAGER FILES

Recipe.XML Below is the example of recipe.xml. The blue color shows the addition needed to be done to recipe.xml. Required default value is true so the step will stop when the test failed. TAF in the recipe is just an argument for the batch file.

<Recipe> <Name>M-Cabinet</Name> <Step> <Name>Automatic step 1</Name> <Action> <Name>Automatic step 1</Name> <Flow>Automatic</Flow> <ExecutableName>z:\stepmanager\tools\runtafbatch.cmd</ExecutableName> <Arguments>PrepareHostPC.taf 0040_PrepareHostPC</Arguments> <TimeToExecute>100000</TimeToExecute> <HelpPage>empty.html</HelpPage> <Repetitions>1</Repetitions> <Required>false</Required> </Action> </Step> <Step> <Name>Manual step 1</Name> <Action> <Name>Manual step 1</Name> <Flow>Manual</Flow> <HelpPage>empty.html</HelpPage> </Action> </Step> … </Recipe>

Result.XML The example of result.xml is shown below. The blue color shows the difference between recipe.xml and result.xml. There is no change required in the result.xml.

<Recipe> <Name>M-Cabinet</Name> <LastPerformedStep>6</LastPerformedStep> <LastPerformedAction>0</LastPerformedAction> <State>1</State>

<Result>Successful</Result> <Step> <Name>Automatic step 1</Name> <Result>Successful</Result> <Action> <Name>Automatic step 1</Name> <Flow>Automatic</Flow> <ExecutableName>sleep.exe</ExecutableName> <Arguments>3</Arguments> <TimeToExecute>100000</TimeToExecute> <Result>Successful</Result> <StartDateTime>2014-05-14T16:04:30.9174141+02:00</StartDateTime> <StopDateTime>2014-05-14T16:04:34.0834141+02:00</StopDateTime> <HelpPage>empty.html</HelpPage> <Repetitions>1</Repetitions> <Required>false</Required> </Action> <Action> <Name>Automatic step 1</Name> <Flow>Automatic</Flow> <ExecutableName>sleep.exe</ExecutableName> <Arguments>3</Arguments> <TimeToExecute>100000</TimeToExecute> <Result>Successful</Result> <StartDateTime>2014-05-14T17:00:16.2426033+02:00</StartDateTime> <StopDateTime>2014-05-14T17:00:19.4718654+02:00</StopDateTime> <HelpPage>empty.html</HelpPage> <Repetitions>1</Repetitions> <Required>false</Required> </Action> </Step> … </Recipe>

TAF FILES File.taf The example of taf configuration file is shown below. <?xml version="1.0"?> <TafSetup xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema"> <TafConfigurationStorage> <WriteableTafConfigurationItems> <ConfigurationItems> <ConfigurationItem> <Value>\\130.144.240.124\ATT\TAF 4\Data Directories\TAFExamples\Verified Releas-es\Version 1.0.0</Value> <ReadOnly>false</ReadOnly> <Name>DataDirectory</Name> <Type>String</Type> </ConfigurationItem> <ConfigurationItem> <Value>\\130.144.240.124\ATT\TAF 4\TAF Keyword Libraries\Verified Releases\Version 8.0.0</Value> <ReadOnly>false</ReadOnly> <Name>TafLibrary</Name> <Type>String</Type> </ConfigurationItem> <ConfigurationItem> … </ConfigurationItem> </WriteableTafConfigurationItems> </TafConfigurationStorage> <ExternalsConfigurationStorage> <ConfigurationItems /> </ExternalsConfigurationStorage> <VariableValues> <Item> <Key> <string>ExampleVariable1</string> </Key> <Value> <string>Value 1</string> </Value>

</Item> <Item> … </Item> </VariableValues> </TafSetup>

Glossary SSTAF Scheduling System for Test Automation Framework. In Philips, the

name of the project is ASPTE (Automatic System Preconditioning and Testing Environment).

TAF Test Automation Framework. TAF is a framework used in Philips Healthcare to run automatic test.

SUT System under Test. System under test (SUT) is an Alura Xpern System that is built at the Philips site for different purposes: development, functional test, integration test, and system verification and validation.

PDC Product Development Code WTS Windows Task Scheduler COTS Common off the shelf ERP Enterprise Resource Planning RBS Role Based Security CRUD Create, Read, Update and Delete. The four use patterns of objects and

data. MVC Model View Controller. A design pattern for implementing user inter-

faces. MVVM Model View View-Model. Another design pattern for implementing

user interfaces. Side PC Side PC is a normal PC that has a direct connection to SUT. Stub A stub is a piece of code used to replace or simulate the behavior of

existing code or hardware. Activity An activity is a task or program that runs in the Side PC. In this project

scope, one activity is related to one TAF Script. Activity Block An activity block is a group of activity that needed to be run sequen-

tially. Schedule A schedule is a list of times at which activity blocks are intended to

take place. QlikView QlikView is a reporting tool.

Bibliography References [1]. Wil van de Berk, Marcel van Veggel, Arjan van Osch, and Luuk van der Veer, Philips Internal Document:Use Case Description - Automatic System Precondition-ing and Testing Environment, 2014 [2]. Philips Medical Systems Nederland B.V, Philips Internal Document: TAF 4 User Manual, 2014 [3]. Djohan Wahyudi, Philips Internal Document: COTS Scheduler Selection for ASPTE, 2014 [4]. Djohan Wahyudi, Philips Internal Document: User Requirement Specification - Automatic System Preconditioning and Testing Environment - Scheduler, 2014 [5]. Djohan Wahyudi, Philips Internal Document: Design Specification - Automatic System Preconditioning and Testing Environment - Scheduler, 2014 [6]. Djohan Wahyudi, Philips Internal Document: Verification and Validation - Au-tomatic System Preconditioning and Testing Environment - Scheduler, 2014 [7]. List of job scheduler software. Wikipedia. [Online]. Available: http:// http://en.wikipedia.org/wiki/List_of_job_scheduler_software. [8]. Comparison of continuous integration software. Wikipedia. [Online]. Available: http://en.wikipedia.org/wiki/Comparison_of_continuous_integration_software. [9]. Frank McKenna, Thin-Client VS Fat-Client Computing, 2012. [10]. QlikView Tool. QlikView. [Online]. Available: http://www.qlik.com/us [11]. Tutorial and Examples. MSDN. [Online]. Available: http://msdn.microsoft.com [12]. Tutorial and Examples. CodeProject. [Online]. Available: http://www.codeproject.com [13]. Technical Question and Answer. StackOverFlow. [Online]. Available: http://stackoverflow.com

About the Author

Djohan Wahyudi graduated from Sekolah Tinggi Teknik Surabaya, Indonesia, with a degree in In-formation Systems and Computer Engineering (2006). During his bachelor studies, he developed several software systems for different companies in Indonesia as a freelancer. After his graduation, he was recruited by Philip Morris International and worked as an IT Consultant. He also worked in a software company called Artindo. He continued his master studies under an Erasmus Mundus scholarship fully funded by the European Commission. After finishing his studies he was re-cruited by a web-based company in the UK and was responsible for developing and maintaining several websites. He has a passion for Software Engineering and he wants to continue gaining experience and expertise in this field. In September 2012, he joined two years PDEng program at Eindhoven University of Technology. In the first year, he studied the technical and non-technical courses, and worked for different projects from different companies such as BOSCH, Sioux, TNO, and CERN. During his second year, he was working in Philips Healthcare on Automatic System Preconditioning and Test Environment Project what resulted in this report.

Scheduling system for test automation frameworkScheduling system for test automation framework...

Documents

Transcript of Scheduling system for test automation frameworkScheduling system for test automation framework...