Automated Testing of HVDC Control & Protection Systems1233259/FULLTEXT01.pdf Abstract Automated...

80
IT 18 031 Examensarbete 30 hp July 2018 Automated Testing of HVDC Control & Protection Systems A Study on Automated Regression Testing Arnav Jain Institutionen för informationsteknologi Department of Information Technology

Transcript of Automated Testing of HVDC Control & Protection Systems1233259/FULLTEXT01.pdf Abstract Automated...

Page 1: Automated Testing of HVDC Control & Protection Systems1233259/FULLTEXT01.pdf Abstract Automated Testing of HVDC Control & Protection Systems Arnav Jain Testing is an important activity

IT 18 031

Examensarbete 30 hp July 2018

Automated Testing of HVDCControl & Protection SystemsA Study on Automated Regression Testing

Arnav Jain

Institutionen för informationsteknologiDepartment of Information Technology

Page 2: Automated Testing of HVDC Control & Protection Systems1233259/FULLTEXT01.pdf Abstract Automated Testing of HVDC Control & Protection Systems Arnav Jain Testing is an important activity

This page has intentionally been left blank.

Page 3: Automated Testing of HVDC Control & Protection Systems1233259/FULLTEXT01.pdf Abstract Automated Testing of HVDC Control & Protection Systems Arnav Jain Testing is an important activity

Teknisk- naturvetenskaplig fakultet UTH-enheten

Besöksadress: Ångströmlaboratoriet Lägerhyddsvägen 1 Hus 4, Plan 0

Postadress: Box 536 751 21 Uppsala

Telefon: 018 – 471 30 03

Telefax: 018 – 471 30 00

Hemsida: http://www.teknat.uu.se/student

Abstract

Automated Testing of HVDC Control & Protection Systems

Arnav Jain

Testing is an important activity when developing a system. Testing requires resources in terms of time, labour and money. By correctly automating the tests, the development time may either be shortened or there will be a possibility to run more tests. ABB in Ludvika has developed MACH, a control & protection system for HVDC (high power electrical transmission over long distances) applications. During development of the control & protection system for each HVDC project, which are all unique, the system is today tested manually, which takes considerable time.

This thesis project studies the possibility of automating parts of the MACH system tests, by investigating current testing procedures, the control & protection system itself, and how a test tool may interact with the system. Using this information a test framework, aimed towards test automation, was created, and a simple test execution tool was developed. A new test case, a combination of multiple smaller test cases, ran on the system using the test execution tool.

The outcome proves the proof of concept of automating parts of the system tests. The economics and the scope of the automated testing however, is dependent on how automation is implemented.

Handledare: Raheleh Mahdivand-Avilagh Ämnesgranskare: Bengt Jonsson Examinator: Justin Pearson

IT 18 031Tryckt av: Reprocentralen ITC

Page 4: Automated Testing of HVDC Control & Protection Systems1233259/FULLTEXT01.pdf Abstract Automated Testing of HVDC Control & Protection Systems Arnav Jain Testing is an important activity

This page has intentionally been left blank.

Page 5: Automated Testing of HVDC Control & Protection Systems1233259/FULLTEXT01.pdf Abstract Automated Testing of HVDC Control & Protection Systems Arnav Jain Testing is an important activity

Automated Testing of HVDC Control & Protection Systems 5

Popular Scientific Summary in Swedish

När ett system ska utvecklas är det viktigt att det även testas, vilket kostarbåde arbetstid och pengar. För att försöka minska ner på de resurser som krävsför att utföra tester, kan testning automatiseras. Med automatisering menas attett datorprogram utför idag de uppgifter som en människa normalt sett utför.Automatisering kan leda till att fler tester hinner utföras under en bestämdperiod eller minska ner den tid som behövs för att utföra samma tester.

På ABB i Ludvika utvecklas system för elektrisk kraftöverföring med hjälp avHVDC (högspänd likström). Kraftstationerna som sammankopplar HVDC sys-temet med det vanliga elnätet har ett sofistikerat kontrollsystem som utvecklatspå ABB (MACH). MACH-systemet kontrollerar viktiga funktioner på HVDCkraftstationen, till exempel hur mycket kraft som överförs, men även andra funk-tioner på stationen såsom ventilation och reservkraft. Då varje HVDC-projektär unikt, behöver MACH-systemet anpassas för varje individuellt projekt. Un-der utvecklingen av MACH-systemet utförs det tester kontinuerligt, av vilkaalla idag utförs manuellt.

Syftet med detta examensarbete har varit att studera vilka möjligheter som finnsför att automatisera testningen av MACH-systemet. Detta för att möjliggöramer frekvent testning jämfört med idag.

Metoden har varit att undersöka de tester som utförs idag för att identifiera vilkatyper av tester som utförs, och även identifiera vilka gränssnitt som finns motMACH-systemet för ett datorprogram. Därefter, genom att använda materialetsom tagits fram via undersökningen, utveckla en struktur för hur automatiskatester bör utformas, samt skapa en program-prototyp som kan utföra testerautomatiskt.

Efter litteraturstudie och konsultering med anställda på ABB utvecklades sedanen struktur för testautomatisering som utgick ifrån att gränssnittet skall varaenkel för en testare att förstå sig på och använda. Efter att ett lämpligt sättatt ansluta sig mot MACH-systemet funnits, utvecklades ett datorprogram somkunde tolka enkla instruktioner för tester och sedan utföra dessa instruktioner.

För att utvärdera det utvecklade programmet och strukturen utfördes ett demon-strationstest där de olika funktionaliteterna i programmet testades på realistiskatestfall. Dessa visade att det finns möjligheter för att införa automatisk test-ning på MACH-systemet. Lönsamheten däremot beror på hur den automatiskatestningen implementeras.

Page 6: Automated Testing of HVDC Control & Protection Systems1233259/FULLTEXT01.pdf Abstract Automated Testing of HVDC Control & Protection Systems Arnav Jain Testing is an important activity

This page has intentionally been left blank.

Page 7: Automated Testing of HVDC Control & Protection Systems1233259/FULLTEXT01.pdf Abstract Automated Testing of HVDC Control & Protection Systems Arnav Jain Testing is an important activity

Automated Testing of HVDC Control & Protection Systems 7

Executive Summary

Testing requires resources in terms of time, labour and money. By correctlyautomating tests it may be possible to run regressions tests on large parts ofthe system more frequently, compared to manual testing. Frequent regressiontests are desirable, since they may help to find unexpected defects in the systemin an earlier stage of the development process.

This thesis project studies the possibility of automating parts of testing of theABB’s MACH Control & Protection system, starting by investigating threedifferent areas:

1. Today’s testing procedure: review what actions that is included in currenttest process to see what is needed in the framework for automated testing.

2. Control & protection system itself: reviewing architecture of the MACH-system and the software that is used.

3. Interfaces that could be used for a test tool to interact with the system.

A literature study in automated testing was made to find a good practice of howan automated testing framework may be structured. Based on the informationfrom the investigation, a framework aimed towards automated testing for MACHwas created. Using the framework, a simple text execution tool was developed,which uses HiBug (an existing protocol at ABB) to interface with the system.

An evaluation was made by creating a demonstrative test case that included acombination a few realistic test cases into one test. The different functionalitiesfrom the framework were implemented in test files and run by the tool. Thedemo showed that there are possibilities to automate the execution of testson the MACH-system. The feasibility of the automated testing however, isdependent on how automation is implemented.

The proposed framework and tool proves a concept of how a test executing toolcan be implemented in practice. There is is still areas left to study in order toreach a complete automated testing procedure e.g. tests including operations onRTDS, automatic test result verification, coordination with a test managementtool.

Page 8: Automated Testing of HVDC Control & Protection Systems1233259/FULLTEXT01.pdf Abstract Automated Testing of HVDC Control & Protection Systems Arnav Jain Testing is an important activity

This page has intentionally been left blank.

Page 9: Automated Testing of HVDC Control & Protection Systems1233259/FULLTEXT01.pdf Abstract Automated Testing of HVDC Control & Protection Systems Arnav Jain Testing is an important activity

Automated Testing of HVDC Control & Protection Systems 9

Foreword

This thesis work has been both captivating and educational. We would like tothank to following persons for their contributions and involvement in this thesisproject.

• Raheleh Mahdivand-Avilagh, Archit Singh, & Linlin Wang; who super-vised this project at ABB and offered both guidance and ideas all through-out the project.

• Aditya Deb, Ted Persson, and other colleagues at ABB’s TK department;for repeatedly answering our unreasonable questions.

• Bengt Jonsson, professor at Department of Information Technology; andJuan de Santiago, senior lecturer at Department of Engineering Sciences;our reviewers at Uppsala University for their invaluable feedback duringthe writing of the thesis.

Lastly we would like to thank everyone at ABB who, during our time at theoffice, made it a good working environment.

Page 10: Automated Testing of HVDC Control & Protection Systems1233259/FULLTEXT01.pdf Abstract Automated Testing of HVDC Control & Protection Systems Arnav Jain Testing is an important activity

This page has intentionally been left blank.

Page 11: Automated Testing of HVDC Control & Protection Systems1233259/FULLTEXT01.pdf Abstract Automated Testing of HVDC Control & Protection Systems Arnav Jain Testing is an important activity

Automated Testing of HVDC Control & Protection Systems 11

Table of Contents

Abstract 3

Popular Scientific Summary in Swedish 5

Executive Summary 7

Foreword 9

Abbreviations and Definitions 18

1 Introduction 19

1.1 Statement of Originality . . . . . . . . . . . . . . . . . . . . . . . 19

1.2 Setting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

1.3 Project Purpose and Goal . . . . . . . . . . . . . . . . . . . . . . 20

1.4 Scope . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

1.5 Delimitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

2 Background 22

2.1 Systems Development Life Cycle . . . . . . . . . . . . . . . . . . 22

2.1.1 Requirements . . . . . . . . . . . . . . . . . . . . . . . . . 22

2.1.2 Design Phase . . . . . . . . . . . . . . . . . . . . . . . . . 23

2.1.3 Implementation Phase . . . . . . . . . . . . . . . . . . . . 23

2.1.4 Testing Phase . . . . . . . . . . . . . . . . . . . . . . . . . 23

2.1.5 Delivery Phase . . . . . . . . . . . . . . . . . . . . . . . . 24

2.2 Software Testing Process . . . . . . . . . . . . . . . . . . . . . . . 24

2.2.1 Test Activities . . . . . . . . . . . . . . . . . . . . . . . . 24

2.2.2 Verification and Validation . . . . . . . . . . . . . . . . . 25

2.2.3 Automated Testing . . . . . . . . . . . . . . . . . . . . . . 25

2.2.4 Regression Testing . . . . . . . . . . . . . . . . . . . . . . 26

Page 12: Automated Testing of HVDC Control & Protection Systems1233259/FULLTEXT01.pdf Abstract Automated Testing of HVDC Control & Protection Systems Arnav Jain Testing is an important activity

Automated Testing of HVDC Control & Protection Systems 12

2.2.5 Different Types of Regression Testing and Verification Tools 26

2.3 HVDC Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

2.3.1 HVDC compared to HVAC Systems . . . . . . . . . . . . 27

2.3.2 HVDC Converter Station . . . . . . . . . . . . . . . . . . 27

2.4 MACH . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

2.4.1 Main Computers . . . . . . . . . . . . . . . . . . . . . . . 28

2.4.2 I/O System . . . . . . . . . . . . . . . . . . . . . . . . . . 30

2.4.3 HMI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

2.5 Control & Protection Systems Development . . . . . . . . . . . . 33

2.5.1 HiDraw Studio Software . . . . . . . . . . . . . . . . . . . 34

2.5.2 Factory System Test, FST . . . . . . . . . . . . . . . . . . 40

3 Design & Implementation 41

3.1 Assessing Inspection and Test Instructions (ITI) . . . . . . . . . 41

3.1.1 Test Case 1 - Ready For Conditions . . . . . . . . . . . . 41

3.1.2 Test Case 2 - Tap Changer Energised Control . . . . . . . 42

3.1.3 Test Case 3 - Operational Test: Start-Up . . . . . . . . . 42

3.2 Concept Demo Test Case . . . . . . . . . . . . . . . . . . . . . . 43

3.3 Communication Interfaces . . . . . . . . . . . . . . . . . . . . . . 44

3.3.1 Record & Playback . . . . . . . . . . . . . . . . . . . . . . 44

3.3.2 SuiteLink . . . . . . . . . . . . . . . . . . . . . . . . . . . 45

3.3.3 HiBug & Windows Communication Foundation . . . . . . 45

3.4 Testing Framework . . . . . . . . . . . . . . . . . . . . . . . . . . 47

3.4.1 Reference State, Prepare and Restore Scripts . . . . . . . 48

3.4.2 Test Case File . . . . . . . . . . . . . . . . . . . . . . . . 48

3.4.3 Configuration File . . . . . . . . . . . . . . . . . . . . . . 48

3.4.4 Action File . . . . . . . . . . . . . . . . . . . . . . . . . . 49

3.5 Creating Script Files in HiDraw . . . . . . . . . . . . . . . . . . . 51

Page 13: Automated Testing of HVDC Control & Protection Systems1233259/FULLTEXT01.pdf Abstract Automated Testing of HVDC Control & Protection Systems Arnav Jain Testing is an important activity

Automated Testing of HVDC Control & Protection Systems 13

3.5.1 Creating a HiDraw Symbol . . . . . . . . . . . . . . . . . 51

3.5.2 Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . 52

3.6 Automation Tool . . . . . . . . . . . . . . . . . . . . . . . . . . . 53

3.6.1 Main Program . . . . . . . . . . . . . . . . . . . . . . . . 53

3.6.2 Parsing files . . . . . . . . . . . . . . . . . . . . . . . . . . 53

3.6.3 Running Actions . . . . . . . . . . . . . . . . . . . . . . . 55

3.6.4 Analog and Digital Orders . . . . . . . . . . . . . . . . . . 58

3.6.5 Collecting Signals Connected to Orders . . . . . . . . . . 59

3.7 Error Handling . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62

3.8 Concept Demo Implementation . . . . . . . . . . . . . . . . . . . 63

3.8.1 Creating Files . . . . . . . . . . . . . . . . . . . . . . . . . 63

3.8.2 Execution . . . . . . . . . . . . . . . . . . . . . . . . . . . 63

4 Evaluation 64

4.1 Assessment of Inspection and Test Instructions . . . . . . . . . . 64

4.1.1 Test Case 1 . . . . . . . . . . . . . . . . . . . . . . . . . . 64

4.1.2 Test Case 2 . . . . . . . . . . . . . . . . . . . . . . . . . . 65

4.1.3 Test Case 3 . . . . . . . . . . . . . . . . . . . . . . . . . . 65

4.2 Concept Demo outcome . . . . . . . . . . . . . . . . . . . . . . . 65

4.3 Adapting Scripts to Another Project . . . . . . . . . . . . . . . . 66

4.4 System Snapshot . . . . . . . . . . . . . . . . . . . . . . . . . . . 67

4.5 Error Handling . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68

4.6 Possible Applications . . . . . . . . . . . . . . . . . . . . . . . . . 69

4.7 Feasibility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69

5 Conclusions 70

6 Future Work 72

6.1 Testing Framework . . . . . . . . . . . . . . . . . . . . . . . . . . 72

Page 14: Automated Testing of HVDC Control & Protection Systems1233259/FULLTEXT01.pdf Abstract Automated Testing of HVDC Control & Protection Systems Arnav Jain Testing is an important activity

Automated Testing of HVDC Control & Protection Systems 14

6.2 Automation Tool . . . . . . . . . . . . . . . . . . . . . . . . . . . 72

6.3 Verification and Presentation of Results . . . . . . . . . . . . . . 72

7 References 73

8 Appendix: Example of a test suite 76

8.1 Test Case File examples . . . . . . . . . . . . . . . . . . . . . . . 76

8.1.1 Test Case 1 Example . . . . . . . . . . . . . . . . . . . . . 76

8.1.2 Test Case 2 Example . . . . . . . . . . . . . . . . . . . . . 76

8.2 Configuration File Example . . . . . . . . . . . . . . . . . . . . . 77

8.3 Action File Example . . . . . . . . . . . . . . . . . . . . . . . . . 78

8.3.1 Preparatory Actions: Station 1 . . . . . . . . . . . . . . . 78

8.3.2 Preparatory Actions: Station 2 . . . . . . . . . . . . . . . 78

8.3.3 Test Actions: Power Transfer Station 1 to Station 2 . . . 78

8.3.4 Test Actions: Power Transfer Station 2 to Station 1 . . . 79

8.3.5 Test Actions: Stop Power Transfer . . . . . . . . . . . . . 79

8.3.6 Restore Actions: Station 1 . . . . . . . . . . . . . . . . . . 79

8.3.7 Restore Actions: Station 2 . . . . . . . . . . . . . . . . . . 80

Page 15: Automated Testing of HVDC Control & Protection Systems1233259/FULLTEXT01.pdf Abstract Automated Testing of HVDC Control & Protection Systems Arnav Jain Testing is an important activity

Automated Testing of HVDC Control & Protection Systems 15

List of Figures

1 The V-model development life cycle . . . . . . . . . . . . . . . . 22

2 A typical MACH System setup [7] . . . . . . . . . . . . . . . . . 29

3 A typical popup window for order execution showing a voltagesetting [7] . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

4 A screenshot of the HMI showing the SLD and operator actions [7] 32

5 Illustration of C&P phases of an HVDC project [26] . . . . . . . 33

6 Illustration of C&P V-model structure [26] . . . . . . . . . . . . 33

7 A signal with three connection points (an excerpt from Figure 11) 35

8 Two different types of symbol blocks in HiDraw, a boolean NOTand a boolean AND . . . . . . . . . . . . . . . . . . . . . . . . . . 35

9 An analog order . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

10 A digital order . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

11 A typical HiDraw drawing showing inputs and outputs for a dig-ital order . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

12 On the right side, the cubicles that contains the MACH maincomputers and the I/O systems are shown. . . . . . . . . . . . . 40

13 Demonstration test case visualised . . . . . . . . . . . . . . . . . 43

14 Testing structure overview . . . . . . . . . . . . . . . . . . . . . . 47

15 Adding sequence of actions to a test file in HiDraw . . . . . . . . 52

16 An overview over the flow of the automation tool . . . . . . . . . 54

17 Overview of the flow for parsing actions files . . . . . . . . . . . . 56

18 Overview of the flow for running actions . . . . . . . . . . . . . . 57

Page 16: Automated Testing of HVDC Control & Protection Systems1233259/FULLTEXT01.pdf Abstract Automated Testing of HVDC Control & Protection Systems Arnav Jain Testing is an important activity

Automated Testing of HVDC Control & Protection Systems 16

List of Tables

1 Input and output signals of an analog order . . . . . . . . . . . . 37

2 Input and output signals of a digital order . . . . . . . . . . . . 38

3 Status indication of a digital order . . . . . . . . . . . . . . . . . 59

4 A simple feasibility calculation for running tests automatically. . 69

Page 17: Automated Testing of HVDC Control & Protection Systems1233259/FULLTEXT01.pdf Abstract Automated Testing of HVDC Control & Protection Systems Arnav Jain Testing is an important activity

Automated Testing of HVDC Control & Protection Systems 17

List of Code Listings

1 Example of a test file . . . . . . . . . . . . . . . . . . . . . . . . . 48

2 Example of a configuration file . . . . . . . . . . . . . . . . . . . 49

3 Examples of actions . . . . . . . . . . . . . . . . . . . . . . . . . 50

4 The structure of a configuration file is keyword, system name,and finally data. . . . . . . . . . . . . . . . . . . . . . . . . . . . 55

5 Excerpt from the source file generated by HiDraw, from the draw-ing in Figure 11 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60

6 The input file to the command utility . . . . . . . . . . . . . . . 61

7 Example of output from the command utility . . . . . . . . . . . 61

8 A example of a log file from the system . . . . . . . . . . . . . . 67

9 Test case file; Kriegers.tst . . . . . . . . . . . . . . . . . . . . . . 76

10 Test case file; Kriegers2.tst . . . . . . . . . . . . . . . . . . . . . 76

11 Configuration file; Nordlink.conf . . . . . . . . . . . . . . . . . . 77

12 Preparatory actions file, Kriegers.Prep.PC1.STBY.OPN.act . . . 78

13 Preparatory actions file, Kriegers.Prep.PC2.STBY.OPN.act . . . 78

14 Test 1 actions file, Kriegers.Run.PowerTransfer.S1S2.act . . . . . 78

15 Test 1 actions file, Kriegers.Run.PowerTransfer.S2S1.act . . . . . 79

16 Test 1 actions file, Kriegers.Run.PowerTransfer.Stop.act . . . . . 79

17 Restoration actions file, Kriegers.Restore.PC1.OPN.STBY.act . . 79

18 Restoration actions file, Kriegers.Restore.PC2.OPN.STBY.act . . 80

Page 18: Automated Testing of HVDC Control & Protection Systems1233259/FULLTEXT01.pdf Abstract Automated Testing of HVDC Control & Protection Systems Arnav Jain Testing is an important activity

Automated Testing of HVDC Control & Protection Systems 18

Abbreviations and Definitions

Abbreviation ExplanationAC Alternating CurrentAO Analog OrderBoolean 0 Boolean falseBoolean 1 Boolean trueC&P Control & ProtectionDC Direct CurrentDO Digital OrderFST Factory System TestGTO Gate Turn-off ThyristorsGUI Graphical User InterfaceHMI Human Machine Interface, i.e. User InterfaceHVAC High Voltage1 Alternating CurrentHVDC High Voltage Direct CurrentIDE Integrated Development EnvironmentIGBT Insulated Gate Bi-polar TransistorsIP Internet ProtocolITI Inspection and Test InstructionsLCC Line Commutated ConverterLAN Local Area NetworkMACH Modular Advanced Control for HVDCRFE Ready For EnergisationRTDS Real Time Digital SimulatorSCM Station Control & MonitoringSLD Single Line DiagramTFR Transient Fault RecorderTCC Tap Changer Control (for transformer)TCP Transmission Control ProtocolVSC Voltage Source ConverterWCF Windows Communication Foundation

1Typically, ≥ 115kV

Page 19: Automated Testing of HVDC Control & Protection Systems1233259/FULLTEXT01.pdf Abstract Automated Testing of HVDC Control & Protection Systems Arnav Jain Testing is an important activity

Automated Testing of HVDC Control & Protection Systems 19

1 Introduction

1.1 Statement of Originality

This thesis was written by two persons studying different programs at UppsalaUniversity. Hampus Halvarsson’s main focus has been the testing framework.Arnav Jain’s main focus has been the communication interfaces and the au-tomation tool. The report is published in two copies with following identifiers

• Arnav: Sections 3.3, 3.6

• Hampus: Section 3.4

• The remaining sections are combined work

1.2 Setting

Almost every type of development today requires many hours of testing beforea product is ready for production release, this testing may translate to verifica-tion and validation of intended behaviour in different scenarios [1]. The sametesting is required again when the product is changed, either for correction ofemerging unintended behaviour or for further development, as this may causenew unintended behaviour of previously correct behaviour [2].

Testing of complicated systems may be very costly or resource intensive, interms of time and/or labour [3], as a small change in one part of the systemmay have completely unexpected consequences in another part of the system [2]which requires re-testing of the whole system.

ABB, a multinational corporation [4], while expanding in many different fields,has stayed true to its origin, mainly electric power transmission over long dis-tances [5], for the longest distances using HVDC (High Voltage Direct Current)transmission [6][7]. There are many different systems, parts, and componentsinvolved in electric power transmission and any unforeseen and unintended be-haviour may cause major impact on the electrical grid, which is a key part ofthe modern infrastructure (although this may not always be visible) [8]. Thus,system development in this field requires rigorous testing prior to field deploy-ment.

HVDC Converter stations (one part of the HVDC transmission system) are spe-cialised electrical substations [9] that either convert AC to DC (rectifiers [10])or DC to AC (inverters [10]). The Control & Protection (C&P) system de-veloped by ABB for controlling these converter stations is known as MACH,Modular Advanced Control for HVDC [7][11][12]. Designing MACH system forany HVDC system is unique and very much tailored for the HVDC project.

During development for every HVDC project, the MACH system undergoesmultiple rounds of testing and verification [11], to ensure proper functional-

Page 20: Automated Testing of HVDC Control & Protection Systems1233259/FULLTEXT01.pdf Abstract Automated Testing of HVDC Control & Protection Systems Arnav Jain Testing is an important activity

Automated Testing of HVDC Control & Protection Systems 20

ity of the system. One of these rounds is known as Factory Systems Test,FST [7][11][13]. During the FST, the system is tested in accordance with ITI,Inspection and Test Instructions, which defines the testing procedure. When abug or system defect is found, said defect is rectified and the rectified system isthen re-validated. Due to the complexity and interconnectivity of the system, acorrection made in one part of the system may affect other parts of the system,sometimes with unintended consequences. Today these consequences may notbe discovered until later in the development/testing cycle, as the whole systemsis not re-tested continuously but rather only the worked/corrected parts arere-tested.

1.3 Project Purpose and Goal

The purpose of this thesis project is to evaluate possibilities for optimising theverification process, during the FST phase, through introduction of automatedregression testing to enable more frequent re-testing of the whole system.

The goal is to review the current test and verification process focusing on thedifferent types of test cases run, the systems currently in use to find possibleinterfaces against the MACH system. By using this data, create a prototypefor a platform that may be used for implementing automated regression testing.Focus for this project will be on the test execution part.

1.4 Scope

The scope of this thesis project will be:

• Assess a number of test cases from the current ITI, Inspection and TestInstructions, to determine if the human inputs in the test cases may beautomated.

– Select three or four test cases able to be automated to create a proofof concept.

– Suggest actions to enable automation of the test cases judged unableto be automated at this stage.

• Study the current systems setup and investigate possible interfaces toMACH main computers for automation.

• Sketch on a test framework which may be used for automated testing.

• Develop a test execution tool based on the framework which can commu-nicate with the MACH main computers and run test cases.

• Investigate the feasibility of automation; taking resources spent on devel-opment and potential reduction in resources for testing.

Page 21: Automated Testing of HVDC Control & Protection Systems1233259/FULLTEXT01.pdf Abstract Automated Testing of HVDC Control & Protection Systems Arnav Jain Testing is an important activity

Automated Testing of HVDC Control & Protection Systems 21

1.5 Delimitations

• The number of cases in a test suite is dependent on the scope of eachHVDC project but may require several hundred independent cases. Whilethe end goal of the company is to have all possible test cases run auto-matically in a (possibly) pre-determined order, this thesis project (in caseautomation is possible) is limited to to create a proof of concept whichfocuses on two or three different types of cases and analysis of them.

• The test cases may be split into two different categories [7], control testingand protection testing. Test cases in this project will be selected from thecontrol category. Time did not permit to analyse protection cases.

• After verifying the test outcome, the results need to be presented in amanner which can easily be interpreted by the end user (i.e. the operatorreviewing the results). The goal for the company is to have a test reportgenerated at the end of the automatic test cycle for the operator to reviewthe results from all test cases in the test run.This report is not included in the scope of this degree project. If theopportunity arises during the project in terms of resources, the reportmay be included then. In such case the specification for the report will bespecified at that point in time.

Page 22: Automated Testing of HVDC Control & Protection Systems1233259/FULLTEXT01.pdf Abstract Automated Testing of HVDC Control & Protection Systems Arnav Jain Testing is an important activity

Automated Testing of HVDC Control & Protection Systems 22

2 Background

2.1 Systems Development Life Cycle

Today, there are many different ways to manage the SDLC, System DevelopmentLife Cycle [14][15]. The very first model (or rather lack of) utilised is referredto as Code-and-Fix model [14], but some of the more common ones are theWaterfall model, Incremental model [15] and V-model [3].

Regardless of which SDLC model is chosen for development of a system, thereare some steps which are common to most models. These steps include [14][15][16]:

• Requirements

• Design / Specification

• Build / Develop / Implement / Manufacture / Code

• Integration / Verification / Testing / Quality Assurance

• Acceptance / Delivery / Maintenance

2.1.1 Requirements

When developing a new system or modifying an existing system, the developerneeds to know what the system is supposed to do, this is known as the systemrequirements [14][17]. Without a clear specification of the system requirements,the developed system will never meet the requirements and subsequently all theeffort and time spent on developing will most probably be wasted [14].

Requirements Specifications

SystemArchitecture

Detailed Design

Implementation

Unit Testing

Integration Testing

System Testing

Figure 1: The V-model development life cycle

Page 23: Automated Testing of HVDC Control & Protection Systems1233259/FULLTEXT01.pdf Abstract Automated Testing of HVDC Control & Protection Systems Arnav Jain Testing is an important activity

Automated Testing of HVDC Control & Protection Systems 23

2.1.2 Design Phase

Designing a system specifies how to develop a system which is capable of per-forming the activities specified in the requirements. Design is the process oftransforming a problem into a solution. [14]

The design of the system can be divided into two different types, the first type isthe architecture. The architecture describes interfaces of the system; how doesthe system look or how does the users interact with the system. Designing thearchitecture of a system is also sometimes referred to as the style. [17]

The second type is the detailed design. Detailed design describes the algorithmsand the procedures for implementing the style of architecture, i.e. the behindthe scenes functionality and workings of the system. [14]

2.1.3 Implementation Phase

The implementation phase is often the heart of the systems development [17].During implementation, the design created in the design phase is realised intoa product, physical or virtual.

2.1.4 Testing Phase

Testing is defined in A Dictionary of Computer Science [18] as:

Any activity that checks by means of actual execution whether a sys-tem or component behaves in the desired manner. This is achievedby one or more test runs in which the system is supplied with in-put data, known under these circumstances as test data, and thesystem’s responses are recorded for analysis.Tests can be categorized according to the conditions under whichthey are performed and the purposes they serve. Module testing(or unit testing) is performed on individual components in isola-tion. At the time that components are brought together to formcomplete subsystems or systems, integration testing is performedin order to check that the components operate together correctly. In-tegration testing typically pays particular attention to the interfacesbetween components. By contrast, system testing normally treatsthe complete system as a ‘black box’ and investigates its behaviourwithout concern for individual components or internal interfaces.Acceptance testing is normally under the control of the procurersof the system, and is designed to ensure that the system is suitablefor operational use.

Rectifying found errors during testing is also often covered during the testingphase.

Page 24: Automated Testing of HVDC Control & Protection Systems1233259/FULLTEXT01.pdf Abstract Automated Testing of HVDC Control & Protection Systems Arnav Jain Testing is an important activity

Automated Testing of HVDC Control & Protection Systems 24

2.1.5 Delivery Phase

When development of the system is completed, access to the system may begiven to administrators, operators and end users. The process of delivery alsorequires planning and can be done in multiple ways. [14]

2.2 Software Testing Process

2.2.1 Test Activities

There are a number of different activities in testing, and how they are carriedout differs. The core activities and the order that they are carried out are oftensimilar to the described activities written below. [3]

The first thing to do is determine what should be tested i.e. the identifyingthe conditions that can be verified by a test. There will be a lot of differentconditions that can be tested for a system, such as the software functionality,security, performance etc. [3]

When test conditions are identified, test cases may be designed. Based onthe test objective (reason or purpose of the test), a set of tests are chosenrelated to that objective. A test case should contain the specified input values,prerequisites, expected outcome and other relevant information that is neededto perform the case. If the test outcome is not specified in advance, there is anoption to use the first correct outcome as a reference for future testing. Usageof this option requires the tester to have adequate knowledge of the software sothat the reference outcome is correctly assessed. [3]

The next step of the process is to build the test scripts, i.e. to implement thetest design into scripts. All the data, instructions, test inputs and outcomesshould be in a form that could be interpreted by the test execution tool used(if there is one). All the preconditions must be implemented in the cases, sincesome of the tests may require a database to be initialized or some hardware isready to be used in order to work as intended. [3]

Using the test cases, the tests may now run. If the tests are run manually,the tester may need to follow a procedure where all the actions and inputs arelisted. If there is a tool for automated execution of tests, the tester would needto choose which tests that should run and then start the tool. [3]

Comparing the actual outcomes of the test, verification, is the last step in thetesting process. The outcomes need to be checked to see if the software haspassed the test. There are two different approaches for comparison: dynamiccomparison, where results are compared during the test execution; and post-execution comparison, where results are compared after the test execution iscomplete. [3]

Page 25: Automated Testing of HVDC Control & Protection Systems1233259/FULLTEXT01.pdf Abstract Automated Testing of HVDC Control & Protection Systems Arnav Jain Testing is an important activity

Automated Testing of HVDC Control & Protection Systems 25

2.2.2 Verification and Validation

There are various interpretations of the terms verification and validation [3][14].To illustrate our interpretation of these terms we will use a simple example.Suppose that a small bank wants to develop a simple accounting system wherethe bank can create accounts and carry out transactions (e.g. deposit, with-drawal) on said accounts. The development of the system is divided into twophases, the first phase being dedicated for the deposit functionality and thesecond phase being dedicated for the withdrawal functionality.ññ

Verification is the process of checking whether the system meets all design re-quirements [14] and produces the correct outcome [3]. In our example abovewith the bank, it is translated to the following:

• By end of development phase one, verification is performed to check thata deposit can be performed correctly.

• By end of development phase two, verification is performed to check thata withdrawal can be performed correctly.

Validation is the process of checking whether the design requirements meetsthe intention of the product, i.e. is the system being built fulfilling its inten-tion [3][14]? The system may pass verification by meeting the set design require-ments, but fail validation because the design requirements set are not corrector incomplete.

2.2.3 Automated Testing

Automated testing, compared to manual testing, can substantially decrease theeffort required to perform tests, or increase the amount of tests that can beperformed during a limited time. This may lead to significant savings in costsand/or time for the company [3].

Boring and repetitive tasks are often good candidates for automation [3]. Othersuitable candidates for automation are re-runs of already existing tests, regres-sion testing. One major benefit with automation is that tests may run more of-ten, with considerable consistency and repeatability [3]. However, non-repetitivetests may not be suitable for automation as the automation process requiresmore resources than could be benefited from it [19].

There are some pitfalls to avoid when implementing automated testing. Auto-mated tests will not fix any deficiencies that exists in cases and procedures thatthe automated testing should be based upon. If tests are poorly organized andwith inadequate documentation, the focus should rather be on improving thetesting procedure instead of on automation. [3]

There are also limitations with automated testing. Manual testing is not fullyeliminated, as sometimes not all steps in a test can be automated. It is hard

Page 26: Automated Testing of HVDC Control & Protection Systems1233259/FULLTEXT01.pdf Abstract Automated Testing of HVDC Control & Protection Systems Arnav Jain Testing is an important activity

Automated Testing of HVDC Control & Protection Systems 26

to fully automate a test if it requires human interaction during the test, suchas connecting/disconnecting equipment. If there are major changes during thedevelopment so that new test scripts are required, or if test cases are only useda few times, automation is not likely to be cost effective. [3]

2.2.4 Regression Testing

Regression testing is when existing test cases are re-run when a part of thesystem is modified [20]. Purpose of regression testing is to discover if the mod-ification has affected other parts of the system, and caused new errors (causinga regression of that component) [20]. In large systems it is not always pos-sible to run all the different regression test cases in one go since it may betoo time consuming, thus there are different techniques to handle the situatione.g. test selection and test prioritization [21]. These techniques use differentmethods to ensure that the most important test cases fit within the collectionof tests [21][22].

Regression tests, like many other tests, are often run manually, but unlike othertests, are good candidates for automation [3]. Take the simple bank program(from Section 2.2.2) as an example. When verifying the new functionality ofsecond phase, the system should be regression tested to ensure that the function-ality from first phase remains unchanged. This regression testing of depositingmoney should be performed after each and every change performed in the sys-tem, and as such this test may be automated as the input to the system andthe expected output from the system is known.

2.2.5 Different Types of Regression Testing and Verification Tools

When running an automatic regression test, in some scenarios a single tool maybe performing all of the actions required for configuration, running the test andverifying the test results. In other cases these different actions may be dividedinto multiple separate tools.

The test execution tool is often the heart of automated regression testing. Oftenthe number of test cases to be run are large, and running these manually withouta execution tool is either quite challenging or difficult [23].

The comparator is not a mandatory tool, but is a often part of automated re-gression testing. Performing one or multiple comparisons between the expectedoutcome and the actual outcome is the process of verifying a test case [3]. Thecomparator tool helps reducing the time and effort required to identify testfailures [23].

When having multiple teams or having multiple versions of the system, trackingall different versions manually may be challenging. A configuration managementtool is also not a mandatory tool, but may help tracking system versions andthe associated test files. [23]

Page 27: Automated Testing of HVDC Control & Protection Systems1233259/FULLTEXT01.pdf Abstract Automated Testing of HVDC Control & Protection Systems Arnav Jain Testing is an important activity

Automated Testing of HVDC Control & Protection Systems 27

A test management tool is also not a mandatory tool, but a useful one. A testmanagement tool may help keeping track of tests executed and their results,any faults identified, and actions taken or planned as a result of said faults. [23]

2.3 HVDC Systems

2.3.1 HVDC compared to HVAC Systems

The traditional usage of HVDC is for bulk transfer of electrical power, but thetechnology has other benefited applications such as long distance transmissionand enabling connections between asynchronous AC networks. There are manyfactors that differ when comparing an HVAC transmission system with HVDCtransmission system. An AC transmission system often uses three conductorswith AC, alternative current, where as DC transmission system may use oneconductor or two conductors.

A DC transmission has lower losses compared to an AC transmission, since thereis no skin effect present in DC transmissions. An AC conductor has a physicallength restrictions due to the reactive properties, but HVDC has no such limit.HVDC converters have inherent functions for active power control, which makesHVDC more flexible. Another important benefit is that an HVDC cable can beused in long distance underwater or underground links, whereas an AC-cablewould get too high capacitance. [24]

A cost comparison between HVAC and HVDC system for bulk power transmis-sion shows that HVDC is cheaper in distances above 30-40 km for undergroundand sub-marine cables. For overhead cables, the HVDC technology has lowercosts in systems with a distance above approximately 400 km. Some of thedrawbacks with HVDC, compared to AC, is the expensive cost of the convert-ers, creation of harmonics that need filtering and difficulty in creating DC gridswith different voltage levels. [24]

2.3.2 HVDC Converter Station

A converter is the component in the HVDC system that transforms AC to DCand vice versa. The converter at the station that transmits power acts as arectifier, transforming three-phase AC into DC. On the receiving side the con-verter at the station acts as an inverter, transforming DC into three phase AC.Today, there are two different types of HVDC, the traditional line commutatedconverter (LCC), and the voltage source converter (VSC), both types havingtheir benefits and applications. The biggest difference between the two tech-niques is the power electronic components that are used as switching devices inthe valves, where LCC uses Thyristors, VSC are using insulated gate bipolartransistors (IGBT) or gate turn-off thyristors (GTO). There are many differenttopologies for HVDC operating configuration depending on earthing, amount ofconnected converters, symmetry/asymmetry etc. [24]

Page 28: Automated Testing of HVDC Control & Protection Systems1233259/FULLTEXT01.pdf Abstract Automated Testing of HVDC Control & Protection Systems Arnav Jain Testing is an important activity

Automated Testing of HVDC Control & Protection Systems 28

2.4 MACH

MACH, or Modular Advanced Control for HVDC, is a Control & Protectionsystem developed by ABB [12] which is fully computerised [7][11]. The soft-ware or applications running on the MACH system are developed using HiDraw(Section 2.5.1.1), a graphical block programming language, and analysed withhelp of HiBug (Section 2.5.1.2), a graphical debugger tool [7][11]. The MACHsystem consists of Station Control & Monitoring (SCM) servers, Operator Work-stations (OWS), Engineering Workstations (EWS), Control & Protection maincomputers and I/O system units, typically connected in a configuration shownin Figure 2 [7].

Operators are able to monitor and control the MACH system using an HMI,which runs on the OWS. The HMI communicates or interfaces with the maincomputers, which are the the central components of the MACH system. TheControl & Protection logics developed in HiDraw runs on the main computer.Each main computer is in turn connected to I/O systems. In a live environment(i.e. after delivery, on site), the I/O system is connected to and communicateswith physical equipment, e.g. transformers, breakers, disconnecters, instrumenttransformers. During testing in the factory, before delivery, the I/O is systemconnected to and communicates with simulated virtual equipment modelled in aReal-Time Digital Simulator, RTDS (more information about the , RTDS, andcan be found in Section 2.5.2).

All system parts involved with the operation of the HVDC transmission linkand its power transfer (e.g. control system, protection system, SCM LAN) aredesigned with redundancy in mind, i.e. the operation of the transmission systemshould not be interrupted by a single point of failure. Subject systems aretherefore duplicated and run in parallel. [7][12]. Among other, for the maincomputer this means there are two active computers for each type of system, anA system and a B system, see Figure 2. The current active system is designatedactive, and the inactive is designated standby.

A number of different standardized buses and protocols are used for communica-tion, these are visualised in Figure 2. The communication between the servers,workstations, and the main computers on the SCM LAN is based on the IEEE802.3 standard (Ethernet). The communication between the main computersand the I/O system, using the Field Bus System is carried out through Ether-CAT which is an Ethernet based bus. The communication between the I/Osystems and main circuit equipment (e.g. disconnecters, breakers) is carried outthrough CAN (Control Area Network, ISO 11898). [7]

2.4.1 Main Computers

The main computers are the brains of the MACH system. On these, the Control& Protection applications developed in HiDraw runs. Each main computer isconnected to the SCM LAN for communication with SCM servers and otherworkstations. Each main computer is connected with I/O systems through the

Page 29: Automated Testing of HVDC Control & Protection Systems1233259/FULLTEXT01.pdf Abstract Automated Testing of HVDC Control & Protection Systems Arnav Jain Testing is an important activity

Automated Testing of HVDC Control & Protection Systems 29

Field Bus System. As the main computers are designed with redundancy inmind, for each system there are two main computers.

Main Computer A Main Computer B

OperatorWorkstation

RTDS

LAN(Ethernet) Engineering

Workstation 

(CAN)

I/O Systems A I/O Systems B

Physical Equipment

FST Environment Site Environment

MACH System

(EtherCAT) (EtherCAT) 

Field Bus System A Field Bus System B

Figure 2: A typical MACH System setup [7]

For control systems, running on the computers, there is only one active systemat any time, the other non-active system is running as a hot standby, ready togo active at any time. The switch command for a standby system to becomethe active system may be initiated automatically or manually. The automaticswitchover, for the standby system to take control, is performed in case a faultis detected in the active system. Also, a switchover to a faulty system is notpossible. [7][12]

For protection systems, also running on main computers, both systems are activeconcurrently. If any one of the systems detects a fault, a protective action willbe triggered by that system. [7][12]

Page 30: Automated Testing of HVDC Control & Protection Systems1233259/FULLTEXT01.pdf Abstract Automated Testing of HVDC Control & Protection Systems Arnav Jain Testing is an important activity

Automated Testing of HVDC Control & Protection Systems 30

2.4.2 I/O System

I/O systems is the interface connecting the main computers to the HVDC facil-ity. The system communicates with the main circuit equipment throughout thefacility using analog and digital electrical signals. The system communicateswith the main computers using optical signal. [7]

2.4.3 HMI

Operators are able to monitor and control the HVDC transmission system inreal-time using the HMI, running on the OWS, which communicates with themain computers through the local area network (SCM LAN, see Figure 2) [25]using the SuitLink protocol. The HMI also shows integrated diagnostic infor-mation about the HVDC converter station such as list of alarms, events, andfaults; as well as disturbance recordings [25].

The HMI contains tabs to navigate between different pages on the HMI. Eachpage shows different types of information and real-time status of the HVDCconverter station, e.g. the Single Line Diagram, SLD, page visualises the con-verter station using a single line diagram, and shows real-time information infields on the diagram. To change a setting of the system, the operator moves tothe relevant page and selects the relevant symbol. When selecting a symbol foraction, a new window pops up for confirmation before the requested change isordered (see Figure 3). Then, the command is sent to the relevant active maincomputer where an analog or digital order (see Section 2.5.1.1) is executed.

Figure 3: A typical popup window for order execution showing a voltagesetting [7]

In Figure 4, an example of a typical HMI SLD page is shown. The page in thisexample is divided into two sections. The top sections contains the main SLD,i.e. how the main circuit equipment is organised, whereas the bottom sectioncontains informational fields showing real-time status of the converter station

Page 31: Automated Testing of HVDC Control & Protection Systems1233259/FULLTEXT01.pdf Abstract Automated Testing of HVDC Control & Protection Systems Arnav Jain Testing is an important activity

Automated Testing of HVDC Control & Protection Systems 31

and buttons which represents various other kinds of actions, that the operatorcan perform. The squares in the SLD represent switching devices (e.g. breakers,disconnecters) that can be operated (opened or closed) by the operator. Whenthe operator selects a square representing the switching device, a popup window(similar to Figure 3) is shown for confirmation of action. Similarly to the squaresin the top section, a popup windows as in Figure 3 is shown when the operatorpresses on any of the buttons in the bottom section of this page. The operatorthen has to confirm the action in the popup window.

Let us observe the process for when the operator initiates a command (e.g. in-crease power transfer from 20 MW to 50 MW) on the HMI.

1. The operator initiates and confirms the command on the HMI.

2. The command is sent over SCM LAN to active main computer.

3. The main computer executes the relevant order (Section 2.5.1.1).

4. The output signals from the executed order are then sent to the relevantI/O system (Section 2.4.2) over the Field Bus System.

5. The I/O system then converts the command signal to an output signal,analog or digital as required.

6. The output signal is sent either to the RTDS or to physical equipmentdepending on whether the system is under test or running live.

Page 32: Automated Testing of HVDC Control & Protection Systems1233259/FULLTEXT01.pdf Abstract Automated Testing of HVDC Control & Protection Systems Arnav Jain Testing is an important activity

Automated Testing of HVDC Control & Protection Systems 32

Figure 4: A screenshot of the HMI showing the SLD and operator actions [7]

Page 33: Automated Testing of HVDC Control & Protection Systems1233259/FULLTEXT01.pdf Abstract Automated Testing of HVDC Control & Protection Systems Arnav Jain Testing is an important activity

Automated Testing of HVDC Control & Protection Systems 33

2.5 Control & Protection Systems Development

A powerful control & protection system is required to operate an HVDC systemin a reliable, flexible, and efficient way [7]. To enable this, ABB implements alinear development model (Figure 5) during the development of the C&P sys-tems for each HVDC project, which is based on the V-Model (Figure 6) [26].The left hand side creates and refines the design requirements of the C&P systemfor the specific HVDC project. The right hand side compares the implementa-tion against the design requirements specified during the earlier left hand sidephases.

Basic Conceptual

Design

Software Application

Design

MS3 MS4 MS5MS1 MS6 MS9MS7 MS8

TenderWork

InitialReview &

Setup

VerificationFST2B & 3

System Integration(Installation, FST1 & 2A)

C&PClose Out

Commissioning

MS2

DetailedSW Design

DetailedHW Design

Figure 5: Illustration of C&P phases of an HVDC project [26]

Contract&

Base Design

Implemented & TestedUnit

Verified System;

Inspection & Test Record

Inspection& Test

Instruction

Integrated& Tested

Functionality

Inspection& TestPlan

FunctionDescription

SystemDesignSpec.

Figure 6: Illustration of C&P V-model structure [26]

Page 34: Automated Testing of HVDC Control & Protection Systems1233259/FULLTEXT01.pdf Abstract Automated Testing of HVDC Control & Protection Systems Arnav Jain Testing is an important activity

Automated Testing of HVDC Control & Protection Systems 34

2.5.1 HiDraw Studio Software

HiDraw Studio is an integrated development environment (IDE) for develop-ment using HiDraw. HiDraw Studio manages libraries such as HiDraw draw-ings and symbol libraries, and includes HiDraw GUI and debugger HiBug [27].HiDraw is mainly used for development of various applications required for Con-trol & Protection functions of an HVDC project.

HiDraw Studio and its components are developed internally by ABB, primarilyfor the MACH system, although most of the functionality can be used for otherplatforms and applications as well. [28]

2.5.1.1 HiDraw

HiDraw is (ABB’s inhouse) graphical block programming language, based ona pick-place-connect method. The user creates drawings of the logic by plac-ing symbols (function blocks) and connecting symbols with so called signals.HiDraw can then generate source code in virtually any type of language e.g. C,FORTRAN. Using the drawings, HiDraw can not only generate source code,but also compile the generated source code to an executable file that can runon the target system. [28]

Each application created in HiDraw consists of multiple drawings, and alwaysincludes a main drawing which specifies metadata of the application and otherdrawings [28]. Drawings may cross-refer to signals from other drawings in thesame application, but also from drawings in other applications [28]. For examplethe signals TCP_EQUAL and DEBLOCK_IND in Figure 11 are signals from otherdrawings in the TCC application.

A simplified workflow for designing an application for the MACHmain computerin HiDraw is as followed:

1. The C&P system is designed, using HiDraw Studio, in drawings usingsignals, and other symbol blocks. See Figure 8 for example of symbol twodifferent symbol blocks and Figure 11 for example of a drawing.

2. From the drawing, HiDraw Studio generates source code in the C pro-gramming language or the C++ programming language.

3. Using the source code generated, an executable binary is compiled usingthe inbuilt compiler in HiDraw Studio. The target platform for this binaryis set when designing the system in step 1.

4. The binary file is sent to the target computer where the application runs.

Signals are analogous to variables in a procedural programming language, whichmay be of boolean, integer, or float type. Each individual signal has a name,known as signal ID, which is unique in the application. As there may be mul-tiple applications where identical signal IDs are present, each unique signal is

Page 35: Automated Testing of HVDC Control & Protection Systems1233259/FULLTEXT01.pdf Abstract Automated Testing of HVDC Control & Protection Systems Arnav Jain Testing is an important activity

Automated Testing of HVDC Control & Protection Systems 35

therefore identified by a combination of computer name, application name, andsignal ID. All signals must be initialised, i.e. they must have a value even if saidvalue is just the default value.

Each signal has a minimum of two connection points, but may have multipleconnection points, where each connection point is connected to a symbol. Oneconnection point (usually the visually leftmost connection point) writes thesignal value, whereas rest of the connection points read the signal value.

Permit Info (Open)Permit Info (Close)

0

Figure 7: A signal with three connection points (an excerpt from Figure 11)

The signal in Figure 7 (excerpt from Figure 11) is connected to an Integersymbol on the left side which sets the value to 0, while both of the connectionpoints on right side are connected to the Digital Order symbol inputs PermitInfo (Open) and Permit Info (Close). The signal PERMIT_MAN in Figure 11

is connected on the left side to a BOOLEAN symbol which sets the value to TRUE,and to the right side to the Digital Order symbol input Permit (Open).

1

1

&

Figure 8: Two different types of symbol blocks in HiDraw, a boolean NOT anda boolean AND

Symbols are function blocks in HiDraw that read signal values, assign signalvalues or with help of logic manipulates signal values. Signals are in a drawingconnected to symbols with the use of connection points. Each symbol has atleast 1 connection point where a signal is connected (be it a input signal oroutput signal). Some symbols only perform assignments, e.g. setting a signalvalue to 0 such as the left symbol in Figure 7. Other symbol may performboolean operations such as boolean NOT or boolean AND, as shown in Figure 8.Other symbols are more complex with multiple input signals and multiple outputsignals.

Each order has, similarly to signals, a name which is unique in the application.

Page 36: Automated Testing of HVDC Control & Protection Systems1233259/FULLTEXT01.pdf Abstract Automated Testing of HVDC Control & Protection Systems Arnav Jain Testing is an important activity

Automated Testing of HVDC Control & Protection Systems 36

For example, the order name in the digital order in Figure 10 is DO_GCC_ON andin Figure 11 is DO_TCCMANCTRL. Each unique order is identified, due to the samepossible duplication as signals, by a combination of computer name, applicationname, and order name.

When designing the system, a parameter for execution time is set, usuallyaround a few milliseconds. This is how long one iteration of the system looptakes. During each iteration of the loop, most symbols are evaluated, i.e. theoutput signals of a symbol are assigned (sometimes new) values. Signal as-signments (such as the one performed by the left symbol in Figure 7) andboolean operations (such as the ones performed by the AND and NOT symbolsas in Figure 8) are performed every iteration. For example, if the drawing inFigure 11 has an execution time of 5 ms, the signals TCC_MAN_ORD (PUBLIC) (as-signed by a boolean OR symbol), TCC_MANUAL_CONTROL (PUBLIC) (assigned byboolean NOT symbol), and TCC_MAN_IND_DURING_OPERATION (PUBLIC) (assignedby a boolean AND symbol) are all evaluated every 5 ms during each iteration ofthe loop. Also, if an operator manually changes a signal value, which is an out-put from a symbol (such as TCC_MAN_ORD (PUBLIC)), using HiBug (see Section2.5.1.2), that signal value will be overwritten at next evaluation of the symbol(the boolean OR symbol).

Analog orders (Figure 9) and digital orders (Figure 10) are two symbols of themore complex type with multiple input and output signals. The main functionof these symbols is to provide an interface for the HMI to change configurationor initiate actions in the MACH system. Consequently the output signals of theorder symbols are not evaluated during each iteration of the loop, but ratherwhen the main computer receives a command from the HMI are the outputsignals for the order symbols evaluated. For example, signals LOG_EVENT_MAN(PUBLIC) and CONSOLE_MAN (PUBLIC) (in Figure 11) are only update when thedigital order DO_TCCMANCTRL is executed (command from HMI is received).

Analog Order

ID Name: AO_BPO_RAMPExtended Interlock text: 0

Permit

Actual Ctrl LocMaxMin

SetpointEnter

Actual valuePermit status

UserConsole

Log/EventTag/Untag

Tag from other

Figure 9: An analog order

Analog orders are symbols that write an analog value (i.e. integer value or afloat value) received from the HMI to a signal. This analog value could forexample be a power order reference (i.e. how much power should be transferredbetween stations), or in the case of Figure 9, setting the power order ramp speed(i.e. how quickly the transmitted power changes, first derivative of the powerorder). An analog order has seven input signals and six output signals, whichare defined in Table 1.

Page 37: Automated Testing of HVDC Control & Protection Systems1233259/FULLTEXT01.pdf Abstract Automated Testing of HVDC Control & Protection Systems Arnav Jain Testing is an important activity

Automated Testing of HVDC Control & Protection Systems 37

Table 1: Input and output signals of an analog order

Input Signals Output SignalsPermit SetpointPermit Status EnterActual Value Tag/UntagActual Control Location Log/EventMax ConsoleMin UserTag from other

Before any of the output signals may be changed, some checks are performed,by evaluating the input signals. First check is to verify that the order is allowedto change the value (as changing value at inappropriate times may cause un-expected outcomes) by ensuring the Permit signal is set to a boolean 1. Nextcheck is to ensure that the new value to be set is within allowable limits, thelimits are read from the Max and Min input signals, the new value should begreater or equal to minimum allowed value and less or equal to maximum al-lowed value. The current real-time value of the output signal can be read fromthe input signal Actual Value. The signal Actual Control Location identifieswhich workstation (OWS) is interfacing with the order.

When the order is executed i.e. the input signals and the output signals areevaluated, the new analog value is written to the output signal Setpoint. ThisSetpoint signal is only a buffer for the new value, the buffered value is writtento the actual output signal when the output signal Enter is toggled to a boolean1 and then back to a boolean 0. The output signals Log/Event and Consolecontains metadata about the execution of the order and are used for loggingpurposes.

The signals Permit Status, Tag from other, Tag/Untag, and User are legacysignals and not used today.

Digital Order

ID Name: DO_GCC_ONExtended Interlock text: 0

Permit (Open)

Permit (Close)

Permit Info (Open)

Permit Info (Close)

Actual Ctrl Loc

Ind (Open)

Ind (Close)

Order (Open)

Order (Close)

User

Console

Log/Event

Tag/Untag

Tag from other

Figure 10: A digital order

Page 38: Automated Testing of HVDC Control & Protection Systems1233259/FULLTEXT01.pdf Abstract Automated Testing of HVDC Control & Protection Systems Arnav Jain Testing is an important activity

Automated Testing of HVDC Control & Protection Systems 38

Digital orders are symbols that write boolean values to signals and are often usedto trigger individual actions or sequence of actions in the MACH system. Onesuch action (for which the logic is shown in Figure 11) is toggling the operationof the TCC equipment between manual mode or automatic mode. Each orderhas eight input signals and six output signals. The input signals and the outputsignals of an digital order are defined in Table 2:

Table 2: Input and output signals of a digital order

Input Signals Output SignalsPermit (Open) Order (Open)Permit (Close) Order (Close)Permit Info (Open) Tag/UntagPermit Info (Close) Log/EventActual Control Location ConsoleIndication (Open) UserIndication (Close)Tag From Other

Similarly to analog orders, before any of the output signals of the digital orderare changed, some checks are performed by evaluating the input signals to theorder. A boolean 1 on the input signal Permit (Open) means that the order isallowed to be set or opened, whereas a boolean 1 on the input signal Permit (Close) means that the order is allowed to be reset or closed. The signal ActualControl Location identifies which workstation (OWS) is interfacing with the

order. The input signal Indication (Open) and the input signal Indication(Close) indicates whether the order is currently opened or closed.

When the digital order is set/opened, the output signal Order (Open) is brieflytoggled to boolean 1 then back to boolean 0. Similarly, when the digital orderis reset/closed, the output signal Order (Close) is briefly toggled to boolean 1then back to boolean 0. The output signals Log/Event and Console containsmetadata about the execution of the order and are used for logging purposes.

The signals Permit Info (Open), Permit Info (Close) Tag from other, Tag/Untag, and User are legacy signals and not used today.

2.5.1.2 HiBug

Together with HiDraw a graphical debugger tool, HiBug, is included [11]. HiBugenables the user to connect to a MACH main computer and view the currentreal-time value of each signal in any one of the currently open drawings [28].The real-time value of a signal can not only be viewed, but also changed in real-time [11]. HiBug is a protocol based on Windows Communication Foundation,WCF, for communication between HiDraw and the MACH main computer. Formore information information about the protocol, see Section 3.3.3.

Page 39: Automated Testing of HVDC Control & Protection Systems1233259/FULLTEXT01.pdf Abstract Automated Testing of HVDC Control & Protection Systems Arnav Jain Testing is an important activity

Automated Testing of HVDC Control & Protection Systems 39

Digi

tal O

rder

ID N

ame:

DO

_TC

CM

ANC

TRL

Exte

nded

Inte

rlock

text

:0

Perm

it (O

pen)

Perm

it (C

lose

)

Perm

it In

fo (O

pen)

Perm

it In

fo (C

lose

)

Actu

al C

trl L

oc

Ind

(Ope

n)In

d (C

lose

)

Ord

er (O

pen)

Ord

er (C

lose

)

User

Con

sole

Log/

Even

tTa

g/Un

tag

Tag

from

oth

er

EX

T

TRU

E

EX

T

EX

T

&E

XT

q-1

1

1D

elay 10

SE

C

t Tim

e

EX

T

FALS

E

0

EX

T

1&

Forc

e St

ate

S R

Q

On

New

Sta

te

1>

PER

MIT

_MAN

TCC

_MAN

_OR

D (P

UBL

IC)

LOG

_EVE

NT_

MAN

(PU

BLIC

)

CO

NSO

LE_M

AN (P

UBL

IC)

TCC

_MAN

_FO

SYS

UPD

ATE_

FOSY

S

DEB

LOC

K_IN

DTC

C_M

AN_I

ND

_DU

RIN

G_O

PER

ATIO

N (P

UBL

IC)

CTR

L_LO

C

TCC

_AU

TO_C

ON

TRO

L (P

UBL

IC)

TCP_

EQU

AL

RES

YNC

HED

_TW

ICE_

SET_

TCC

_MAN

TCC

_MAN

UAL

_CO

NTR

OL

(PU

BLIC

)

2156 56 56

15 14 1455

143635

2041

5355

58

Figure 11: A typical HiDraw drawing showing inputs and outputs for a digitalorder

Page 40: Automated Testing of HVDC Control & Protection Systems1233259/FULLTEXT01.pdf Abstract Automated Testing of HVDC Control & Protection Systems Arnav Jain Testing is an important activity

Automated Testing of HVDC Control & Protection Systems 40

The main use case for the HiBug tool is to find the current real-time valueof a signal, and which symbol is causing that specific value. Using the cross-reference functionality of the graphical block view, the user can trace the signaland symbols through multiple drawings and between applications to find outwhich signals and symbols affect a specific signal. [28]

2.5.2 Factory System Test, FST

FST refers to the systems test performed at ABB’s factory before the MACHsystem is delivered to the site. The setup, during FST, consists of two sys-tems [11]. The first system is the MACH system that contains all the logicand also performs the logging/collecting of data, and which will be delivered tosite for installation after successful completion of FST; the second system is anHVDC Simulator [7][13] known as RTDS (Real Time Digital Simulator), thatsimulates the electrical grid (AC and DC sides), the HVDC converter itself, aswell as the surrounding electrical components present at the converter stationthat the MACH system will have access to (when deployed) [29][30]. The datatransfer between the two systems is performed through the analog and digitalsignal I/O systems connected to the RTDS [30]. Both systems are transmittersand receivers for the signals.

The test cases that are run during FST may be categorised in two different cat-egories; control cases and protection cases. Control cases are initiated by givingcommands to the MACH system to carry out its intended control functions [7]through the HMI (i.e. a terminal) which then sends commands to the RTDS,monitors and verifies that the outcome is as intended. Protection cases are initi-ated by inducing an electrical disturbance (e.g. three-phase fault or single-phasefault [13]) in the RTDS and the MACH system reacts to subject disturbance todetermine if any action is required and then carries out any required actions.Generally, this means control tests are triggered in the HMI and MACH controlsystem, while protection tests are triggered in the RTDS.

Figure 12: On the right side, the cubicles that contains the MACH maincomputers and the I/O systems are shown.

Page 41: Automated Testing of HVDC Control & Protection Systems1233259/FULLTEXT01.pdf Abstract Automated Testing of HVDC Control & Protection Systems Arnav Jain Testing is an important activity

Automated Testing of HVDC Control & Protection Systems 41

3 Design & Implementation

3.1 Assessing Inspection and Test Instructions (ITI)

The Control & Protection system for each HVDC project is unique, however,there are also many common factors, both in terms of hardware and software.Thus, successful development from previous projects may be reused in newprojects, instead of starting from the very beginning every time. This commonstarting point for each new project is known as Base Design. The base designat ABB is updated on regular basis. [11]

The ITI for current Base Design [31] include many test cases common to mul-tiple, if not most, projects, thus the ITI for Base Design was chosen for thisassessment. From this base document, three tests were chosen for further in-vestigation to analyse how the tests are performed today. The actions thatare performed manually today by the operator are the actions that must beconsidered in the testing framework.

3.1.1 Test Case 1 - Ready For Conditions

In order to provide a safe and reliable switching within the converter stations theMACH system contains interlocking and maneuvering sequences. Some actionsrequire certain conditions to be fulfilled, prior to the action being performed,otherwise the action will be blocked (i.e. not allowed to run) [32]. The inter-locking conditions are tested in the control tests. One category of interlockingconditions is Ready for Conditions tests, which tests the interlocking for chang-ing between different operational states [31], which can be many in one HVDCproject.

Ready for energisation (RFE) is one of the tests included in the ready for con-ditions category. The test verifies that only when all pre-requisites are fulfilled,only then is the system allowed to energise. The precondition for the test isfor the station to be de-energised and with the RTDS in operation [31]. Thetesting procedure contains:

1. Fulfilling the conditions for RFE: There are certain conditions that allneed to be fulfilled in order to obtain the RFE status. Each conditionis a status indicator for different parts of the system, such as auxiliarypower system is running, or that there is no protective action ongoing.The operator will have to go to the appropriate page in the HMI (referto Section 2.4.3 for tabs/pages) that contains the conditions and visuallycheck them. Then, the operator changes to each page that contains theactions that is necessary for fulfilling the conditions [31].

2. When all the conditions are fulfilled, each condition should be removed.one by one, to manually verify that RFE is lost for each condition removed.This is performed at the HMI pages that controls each condition [31].

Page 42: Automated Testing of HVDC Control & Protection Systems1233259/FULLTEXT01.pdf Abstract Automated Testing of HVDC Control & Protection Systems Arnav Jain Testing is an important activity

Automated Testing of HVDC Control & Protection Systems 42

3. Last step is to test whether the converter can be energised when RFE isobtained [31]. This action is commanded on the HMI.

In the steps above, step one may be possible to automate by reading signal valuesfrom relevant MACH main computers. Step two and three may be possible toautomate by changing signal values and by running appropriate analog anddigital orders.

The conditions for RFE are different between HVDC projects and are declaredin the project specific documents [31]. The following documentation needs tobe collected by the operator for this test:

1. The events log from HMI [31].

2. Screenshot of the drawing (Single Line Drawing, SLD) of the HMI wherestatus of RFE may be observed [31], similarly to Figure 4.

From the required documentation above, item one may be challenging to auto-mate, whereas item two will not be possible without having access to an HMI.Documentation item two may however be replaced by logging a momentarysnapshot of the system by saving the signal values required.

3.1.2 Test Case 2 - Tap Changer Energised Control

The tap changer control (TCC) application controls the tap changer of thetransformer in order to ensure correct relationship between the AC and DCvoltages by stepping up and down the tap changer of the transformer [7]. Thetest case verifies that the tap changer keeps the DC voltages within acceptedrange even when the AC grid voltage is changed [31]. The AC grid voltage iscontrolled by the RTDS, so the operator has to perform the voltage changes onthat system in order to complete this test case.

3.1.3 Test Case 3 - Operational Test: Start-Up

This test ensures that the start-up of power transmission is operating accordingto the HVDC project specific reference document [31]. The precondition of thistest case is that the station is de-energised and both station are connected onthe DC-side. Starting in that state, the operator commands the converter tothe state of power transmission, which includes following steps:

1. Fulfilling the conditions for RFE for both stations: (as in test case 1) [31].

2. Energizing stations by starting a energisation sequence through HMI. Thisresults in the AC-breakers are closed and the bus voltage reaches the setlimits. [31]

Page 43: Automated Testing of HVDC Control & Protection Systems1233259/FULLTEXT01.pdf Abstract Automated Testing of HVDC Control & Protection Systems Arnav Jain Testing is an important activity

Automated Testing of HVDC Control & Protection Systems 43

3. Change to the appropriate page on HMI to verify the stations being readyfor operation, followed by taking actions to fulfil the missing conditions ifthere are any. [31]

4. Deblocking/starting the converter, setting the station into operationalstate. [31]

The conditions for each step in this test are different between projects and aredeclared in the project specific documents [31]. The following documentationneeds to be collected by the operator for this test:

1. List of events from HMI. [31]

2. Screenshot of the HMI that shows condition of each state [31]

3. Triggered Transient Fault Recorder files (TFR). [31]

3.2 Concept Demo Test Case

A realistic demo test case was created in consultation with the employees atABB. The case tests the power transmission between two stations, includingconfiguration of the required prerequisites. The idea was to automate this testcase, using a framework and a tool that was to be developed. The demo shouldconfirm that the tool works as intended and has the functions required to runsimilar test cases. All the different actions that the tool should execute (see Sec-tion 3.4.4) were included in the test case to verify that they worked as intended.The proposed demo included the following activities:

400

-200

200

[MW]

Set Digital Order

Test 2Test 1

Set Digital Order

Set Analog Order

Set AnalogOrder

Log

Prepare

Restore

Test 1 Complete

Test Complete

Set Digital Order

Log

Figure 13: Demonstration test case visualised

Page 44: Automated Testing of HVDC Control & Protection Systems1233259/FULLTEXT01.pdf Abstract Automated Testing of HVDC Control & Protection Systems Arnav Jain Testing is an important activity

Automated Testing of HVDC Control & Protection Systems 44

• Preparation

– Start in a known reference state– Perform appropriate action to reach a state where power transmission

is allowed.

• Test Case

– Initiate power transmission from Station A to Station B with a spec-ified power order (in MW) and ramp speed (i.e. the rate of change,in MW/min).

– Wait until power ramp completes and the current power transmissionlevel is same as the set level.

– Reverse the power direction and initiate power transmission fromStation B to Station A.

• Restoration

– Stop power transmission– Restore the system into same state as the beginning of the test.

3.3 Communication Interfaces

When creating an automation test tool, the tool needs to interface with thesystem being tested. Most types of interfaces can be can be categorised eitheras graphical user interfaces (GUI) or command line interfaces (CLI). GUI areoften more user friendly when a human is supposed to work with the system.CLI on the other hand are (often) less user friendly for humans, but are bettersuited for a programmatical approach.

For this thesis project, there are several alternatives for how an automation toolmay interface with the MACH main computers. Some alternatives use CLI,but may require usage of proprietary protocols, others use GUI. Three differentalternatives were analysed; one GUI approach which is known as Record &Playback, two independent CLI approaches which uses the SuiteLink protocoland the HiBug protocol respectively.

3.3.1 Record & Playback

A record & playback tool records the actions performed on the GUI by anoperator during a test run and saves the actions in a script [3]. The script canlater be played and the actions are executed in the same order as when theywere recorded [33]. Record & replay tools are an easy way of creating throwawaytest cases [34], however, if the GUI is modified there is a risk that the playbackwill not work since the layout is not longer the expected one [33]. Consequently,there is a need to edit or re-record the test cases in order to make them workagain, which is neither easy nor cost effective [3].

Page 45: Automated Testing of HVDC Control & Protection Systems1233259/FULLTEXT01.pdf Abstract Automated Testing of HVDC Control & Protection Systems Arnav Jain Testing is an important activity

Automated Testing of HVDC Control & Protection Systems 45

Record & playback was considered to have potential for usage as the interfaceagainst the MACH main computer. However, since there is diversity in thegraphical user interfaces between different HVDC projects it would be necessaryto make a new test tool for every project. If the testing scripts are created inthe early stages of FST, there is a risk for the scripts becoming obsolete whensomething in the HMI is changed during the course of testing. With these factsunder consideration, record & replay was only considered as a last solution asan interface in this thesis project.

3.3.2 SuiteLink

The HMI interface used in the SCM system is built using Wonderware InTouch,which communicates with the MACH main computers using a proprietary pro-tocol known as SuiteLink. The SuiteLink protocol communicates over the SCMLAN network using a TCP/IP based protocol [35]. The main advantage of us-ing SuiteLink is the possibility of executing symbols (such as analog or digitalorder). As no detailed description nor any libraries for SuiteLink were availableduring the thesis project, usage of this protocol would have required a depen-dency on a 3rd party vendor, Wonderware. Hence, we decided to look for otheralternatives, while keeping SuiteLink as a backup alternative.

3.3.3 HiBug & Windows Communication Foundation

As stated above in Section 2.5.1.2, HiBug is a graphical debugger which canbe used to debug HiDraw code running on the main computers. HiBug usedin the MACH platform uses a proprietary protocol based on WCF (WindowsCommunication Foundation) for communication between HiDraw and the maincomputers. The WCF communication is performed using a secure binary TCPprotocol. [36]

The HiBug monitor service running on the main computer may be used bothfor reading the real-time values of the signals in the running code as well aswriting/changing subject values. HiBug can also use a serial connection forcommunication with the main computers [36], but this option was not consideredin this thesis project as the communication today and in future will be overTCP/IP network (SCM LAN). The main disadvantage of using HiBug is thatit just reads and writes to signals, and can not execute symbols in HiDraw inthe same way HMI does (compare SuiteLink in Section 3.3.2).

3.3.3.1 FSTSettings

FSTSettings is a tool created inhouse at ABB, to quickly change values of signalsin the MACH system. Often there is some equipment (such as auxiliary poweror air condition) which is not simulated during FST, but which is a requirementfor the converter station to function. If systems are not sending the correctsignals, the rest of the converter may initiate protective actions. To circumvent

Page 46: Automated Testing of HVDC Control & Protection Systems1233259/FULLTEXT01.pdf Abstract Automated Testing of HVDC Control & Protection Systems Arnav Jain Testing is an important activity

Automated Testing of HVDC Control & Protection Systems 46

this, FSTSettings is used to manually set the missing signal values.

FSTSettings uses HiBug (and WCF) for communication with the MACH maincomputers. The input to the FSTSettings application is a text file which defineswhich signals to read or to write, being able to combine signals over differentmain computers and applications in each run. In the beginning of this thesisproject, before other options were discovered, FSTSettings was used for commu-nication and sending commands to the main computer. The advantage of usingFSTSettings is that the utility is known by the testers and is actively used. Theutility is actively maintained by ABB and there is also ongoing development.The disadvantage is that each command sent to the main computer creates lotsof overhead, both in terms of network traffic as well as execution time.

3.3.3.2 WCF Interface

HiBug uses a proprietary protocol based on WCF (Windows CommunicationFoundation) for communication between HiDraw and the main computers. Thisprotocol is developed inhouse, thus there exists documentation, a C# dotnetlibrary (known as IApplicationMonitor2), as well as usage examples for theprotocol (and library). [36]

The basics of the protocol requires knowledge of three different objects to suc-cessfully read values, and four object to write values:

1. Computer hostname or IP addressA resolvable hostname or the IP address is required to reach the computerover a TCP/IP network.

2. Application NameThe name of the application on the computer that has the signal.

3. Signal NameThe name of the signal that is to be read.

4. New valueThe new value, required only if writing a new value to the signal.

Out of the three different approaches available (record & replay, SuitLink, andHiBug), one utilised GUI and the two others utilised CLI. The record & replaywas not discarded, but considered as a last option, if neither of the other alterna-tives were suitable. The SuiteLink protocol was not considered as the protocol isproprietary and no documentation was available. Lastly for the HiBug protocolthere existed both tools, documentation, and libraries at ABB. This combinedwith HiBug being supported natively by the MACH system as a core component(for debugging), HiBug with the IApplicationMonitor2 library stands out asbest option.

Page 47: Automated Testing of HVDC Control & Protection Systems1233259/FULLTEXT01.pdf Abstract Automated Testing of HVDC Control & Protection Systems Arnav Jain Testing is an important activity

Automated Testing of HVDC Control & Protection Systems 47

3.4 Testing Framework

The testing structure is a part of the design of the developed test automationsuite. It is important to have a well engineered design structure since it providesthe basic framework for how the scripts are made [14]. The technique of creatingmaintainable testing scripts is similar to the technique of creating maintainablesoftware programs [3], which is desirable in this type of project. There are someprinciples to take into account for how a good script should be [3]:

• Functional: Perform one single task.

• Structured: Makes it easier to maintain, read and understand.

• Understandable: Makes it easier to maintain.

• Annotated: To give guidance to the user and in maintenance.

• Documented: To give assistance when reusing and in maintenance.

With listed principles under consideration a file structure was created, see Fig-ure 14. The different file types in the structure were based on three types ofscripts, which are explained below.

PrepareAction 1

Action 2

Action n

Actions 1Action 1

Action 2

Action n

Actions nAction 1

Action 2

Action n

RestoreAction 1

Action 2

Action n

ConfigComputer A

Computer B

C:\MyFiles

Test file

Configuration file

Prepare actions

Test actions 1⋮Test actions n

Restore actions

Figure 14: Testing structure overview

Page 48: Automated Testing of HVDC Control & Protection Systems1233259/FULLTEXT01.pdf Abstract Automated Testing of HVDC Control & Protection Systems Arnav Jain Testing is an important activity

Automated Testing of HVDC Control & Protection Systems 48

3.4.1 Reference State, Prepare and Restore Scripts

The control system can be configured in numerous ways. In order to have atest structure with the ability of reusing test-scripts there was a need to have areference state with known system configuration that could be used as a startingpoint for the tests. Then, by using prepare scripts (action files), the system canbe configured into the configuration that comes as a prerequisite for the specifictest case. The prepare script can be saved and used in other test cases with thesame prerequisites. As a last part in the test case a restore script that turns thesystem back into the reference state should be implemented so that the nexttest case starts in a known state.

3.4.2 Test Case File

A test case file is a plain text file with the file extension of.tst, that definesthe structure of one test case. It is a list containing filenames of the files thatis needed to run one ore more tests. The filenames in the list are ordered inthe way they should be interpreted by the automation tool, starting with aconfiguration file and is followed by one or more action files. One example of atest case file may be found in Listing 1, more examples of test case files may befound in appendix (Section 8.1).

Listing 1: Example of a test file1 ;Simple demo for concept2 # Configuration3 Krieger.conf4 # Prepare5 Krieger.Prep.act6 # Run7 Krieger.Run.act8 # Restore9 Krieger.Restore.act

3.4.3 Configuration File

The configuration file is made in plain text format with the file extension of .cfg,in order to make the contents of the file easy for a person to read, create, and editthe script, without the requirement of special software. The configuration filecontains all computer host names (or ip addresses) and the paths to associatedHiDraw drawings needed for the test execution tool to run the test case. Thereis typically one drawing for each computer. The configuration file is uniquefor each HVDC project since computer hostnames and other functions changesbetween projects. There is only one configuration file for each HVDC projectand is the first file to be read and parsed starting the automation tool. Oneexample of a configuration file may be found in Listing 2, another example maybe found in appendix (Section 8.2).

Page 49: Automated Testing of HVDC Control & Protection Systems1233259/FULLTEXT01.pdf Abstract Automated Testing of HVDC Control & Protection Systems Arnav Jain Testing is an important activity

Automated Testing of HVDC Control & Protection Systems 49

Listing 2: Example of a configuration file1 # Computer2 ## PC13 KF−PC1A4 KF−PC1B56 # Drawing7 ## PC18 D:\Kriegers_Flak\AppSoftware\U2_Wx.1_SC\MC1\MC1.hwsx910 # Active11 ## PC112 PC1|SSL!Active

3.4.4 Action File

An action file contains information about the different actions required to per-form a test. Action files are also in a plain text file format where each linecontains the actions to be carried out and the required data/arguments, eachseparated by whitespace. There are three different types of actions: commandaction, delaying action, and logging action. The basis for these action types wereidentified in Section 3.1, as required actions to run the test cases automatically.

In order to reduce complexity of the action files a technique called shared scriptsis implemented. This means that action sequences that are used in more thanone test case can be reused [3], one example is preparatory actions that are usedin more than one test case or when one test case is a part of another test caseetc.

One example of an action file may be found in Listing 3. This example showsone action file that contains four different actions:

1. The analog order BCP|BAPC!AO_P_RAMP represents the ramp speed, i.e. howquickly the amount of powered being transferred changes. This analogorder is set to 800 MW/min.

2. The analog order BCP|BAPC!AO_P_REF represents the power order, i.e. howmuch power should be transferred. This analog order is set to 0 MW,i.e. stop power transfer. The rate of change to reach this new value is setduring step one above.

3. The signal BCP|BAPC!P_RAMP_IN_PROG_HMI represents the signal that keepstrack of active power ramps, i.e. when the value is one (1), there a changein power transfer ongoing; when the value is zero (0), the power transferstable. This delay action puts the tool in a waiting state, until the signalvalue reaches the value 0. When the value reaches 0, the tool continueswith next action.

Page 50: Automated Testing of HVDC Control & Protection Systems1233259/FULLTEXT01.pdf Abstract Automated Testing of HVDC Control & Protection Systems Arnav Jain Testing is an important activity

Automated Testing of HVDC Control & Protection Systems 50

4. The digital order BCP|BSTSQ!O_TO_BLOCK initiates an action to block theHVDC converter, i.e. removing the load from the converter. Blocking ofa converter requires performing multiple actions in sequence, which arepre-programmed part of the MACH system. Setting this digital order tothe value one (open) is a trigger action to perform the pre-programmedsequence.

Further examples of action files may be found in the appendix (Section 8.3).

Listing 3: Examples of actions1 ;;Stop power transfer2 ;1 Set ramp speed3 Write AnalogOrder BCP|BAPC!AO_P_RAMP 8004 ;2 Set power order5 Write AnalogOrder BCP|BAPC!AO_P_REF 06 ;3 Wait until ramp complete7 Delay Signal BCP|BAPC!P_RAMP_IN_PROG_HMI 08 ;4 Block the converter9 Write DigitalOrder BCP|BSTSQ!O_TO_BLOCK 1

3.4.4.1 Command Actions

To initiate actions or change a setting in the MACH system, an operator gives acommand using the HMI. Similarly, the automation tool can also give commandsto the MACH system. A command action changes either an analog order, adigital order, or a signal value. An analog order change any integer or floatvalue (such as ramp speed shown in Listing 3. A digital order may be set toopen or close. Lastly, a signal can be set to either a boolean value, integer valueor a float value depending on signal type.

Keyword for a command action is Write.

3.4.4.2 Delaying Actions

The delaying action puts the tool into a waiting state until either:

1. permission to change an analog order is given, i.e. the MACH systemreaches a state where changing the analog value is allowed (see Section2.5.1.1).

2. permission to open or close a digital order is given, i.e. the MACH systemreaches a state where operating the digital order is allowed (see Section2.5.1.1.

3. a signal reaches a predetermined value.

4. a predetermined time, specified either in seconds or minutes.

Page 51: Automated Testing of HVDC Control & Protection Systems1233259/FULLTEXT01.pdf Abstract Automated Testing of HVDC Control & Protection Systems Arnav Jain Testing is an important activity

Automated Testing of HVDC Control & Protection Systems 51

Keyword for a delaying action is Delay.

3.4.4.3 Logging Actions

To enable troubleshooting and manual verification, the current state of MACHmain computer may be saved, which is by logging individual signals value orby logging values of all signals pertaining to analog and digital orders. Loggingcan be performed on an individual signal or order basis, all signals and ordersin a computer or all orders in the current system.

There are five different types of logging actions. When logging an individualsignal, only the value of the signal is saved to file. When saving either anindividual analog or digital order, the values of input and output signals of thatspecific orders are saved to file. When saving a computer all individual signalsand order noted in the script files for that specific computer will be saved tofile. Logging the system is similar to logging a computer, values of all signalsand orders mentioned in the (prepare, test, and restore) script files are saved tofile.

Keyword for a logging action is Log. An example of a log from the automationtool may be found in Listing 8 below.

3.5 Creating Script Files in HiDraw

The script files in the test framework are plain text files that can be created andedited using a text editor, but it may be discouraging for a non-programmer touse a text editor for creating and editing scripts. Employees at ABB suggestedto should study whether HiDraw could help testers to create and maintain testscripts. The underlying idea is that testers at ABB are already familiar withthe HiDraw software, and would rather use a known user interface instead ofhaving to learning a new one. The pick-place-connect method in HiDraw couldbe a good way of supplying a user friendly interface. In order to supply thisfunctionality new HiDraw symbols were required.

3.5.1 Creating a HiDraw Symbol

HiDraw has a built in symbol library (a collection of symbols for use in HiDrawdrawings) which works as a direct interface between the control system andthe station control & monitoring system [37]. HiDraw is available to generatecode [37] which was the functionality that could be used when creating scripts.The first step in the study was to gather information and the necessary toolsto create source code for a new symbol which could be added to the HiDrawsymbol library. An employee at ABB provided an empty symbol template. Thetemplate did not contain any functionality but, included predefined source codewhich is necessary for generating code through HiDraw.

Page 52: Automated Testing of HVDC Control & Protection Systems1233259/FULLTEXT01.pdf Abstract Automated Testing of HVDC Control & Protection Systems Arnav Jain Testing is an important activity

Automated Testing of HVDC Control & Protection Systems 52

Using the symbol template, a graphical action symbol was created in HiDraw.One symbol represents one action. This action symbol has one input connector,one output connector, and four parameters/arguments (see Figure 15). Theinput and output connectors are used for connecting multiple symbols, andthus defining the sequence in which the actions should be run. In Figure 15 theleft action symbol is succeeded by the right action symbol.

The parameters in the action symbol have the following alternatives/format:

1. Action type: Write, Delay or log.

2. Order type: Signal, DigitalOrder or AnalogOrder.

3. Destination: COMPUTER|APPLICATION!SignalId.

4. Value: null, bool or float value.

My Symbol

ActionType WRITE

OrderType AnalogOrder

Destination BCP|BACVC!O_UAC_RAMP

Value 2

Input 1 Output 1

My Symbol

ActionType WRITE

OrderType AnalogOrder

Destination BCP|BACVC!O_UAC_RAMP

Value 3

Input 1 Output 1

Task1_94

Figure 15: Adding sequence of actions to a test file in HiDraw

Using the framework that ABB uses for symbol programming a function waswritten to the symbol source code that will write the values of the parametersinto a text file when code is generated through HiDraw. Then by connecting thesymbols using pick-place-connect method, a text file can be made when codegeneration from HiDraw is run.

3.5.2 Evaluation

One main disadvantage of HiDraw is the lack of support of GUI (such as ra-dio buttons or combo box) for configuring the test symbol. This means eachparameter of each action must be written as text in the test symbol. Anotherchallenge is to read and modify existing test files, as HiDraw does not have aparser for reading and converting test files to an HiDraw drawing similar toFigure 15. Due to these two shortcomings, HiDraw was deemed unsuitable formodification of test files, then consequently also creation of test files.

Page 53: Automated Testing of HVDC Control & Protection Systems1233259/FULLTEXT01.pdf Abstract Automated Testing of HVDC Control & Protection Systems Arnav Jain Testing is an important activity

Automated Testing of HVDC Control & Protection Systems 53

3.6 Automation Tool

The automation tool implements the testing framework (see Section 3.4) andconnects to the MACH main computers. The tool, developed using C# andobject orientation programming, performs the following actions: read and parsetest files, configuration files, and action files; configure the tool for the cur-rent test; and runs actions specified (in the action files). The protocol HiBug(see Section 2.5.1.2) is used to communicate with the MACH main comput-ers. The main reason for choosing C# as the programming language is theIApplicationMonitor2 library, which is a library written using .NET frame-work. Also other software such as HiDraw (Section 2.5.1.1) and related toolssuch as FSTSettings (Section 3.3.3.1) are also developed using C#. Using thesame language for this tool as other tools at ABB enables easier maintenanceof the tool after this thesis project concludes.

3.6.1 Main Program

When launching the tool, command line arguments may be passed to the tool.One argument is mandatory, this argument specifies which test file to run. Thetool starts by reading the test file (see Listing 1 and Section 8.1 for examples oftest files). From the test file, following information is parsed: path to configu-ration files, and paths to the different categories of action files.

After the information from the test file is parsed, the tool reads and parsesthe information in the configuration files. From the configuration file, followinginformation is parsed: systemnames, hostnames or ip addresses to the maincomputers for each system, path to the drawing for each system, and the signalto show which system is active (to manage redundancy, see Section 2.4.1).

Next step is to read the actions files specified in the test file. From the action filesall actions to be performed are parsed, together with some syntax verification.Each action file is parsed in order specified in the same order specified in thetest file, thus the order sequence of action files and of each individual action issignificant.

At this point all preparation relating to reading the test scripts is completed.The next step is to run the actions specified in the action files. If either thetest case file is missing preparatory action files, or if the action files themselvesare empty, preparatory actions are skipped. Same logic apply for test actionfiles and for restoration action files, if either files are missing or empty, no testrespective restoration actions are performed.

3.6.2 Parsing files

When parsing files, the first character in each line is checked (after trimming anyleading whitespace character). If that first character is a semicolon ;, the lineis interpreted as a comment and not parsed. In test file and in configuration

Page 54: Automated Testing of HVDC Control & Protection Systems1233259/FULLTEXT01.pdf Abstract Automated Testing of HVDC Control & Protection Systems Arnav Jain Testing is an important activity

Automated Testing of HVDC Control & Protection Systems 54

files any line starting with the character # indicates keywords on that line.Also, similarly to a commented line, any empty line is skipped. To avoid issuesarising from minor typographical errors, all of the lines are parsed without casesensitivity and the data is stored (in memory) in upper case.

Read input arguments

No

Yes

Test fileexists? 

Display Error

Parse  test file

No

1+ Exists

Configuration  file(s) exist? 

ParseConfiguration

file(s) 

Exit tool

Parse  action files

Tool started

Run actions

Figure 16: An overview over the flow of the automation tool

As mentioned above, the test files contain paths to configuration files and to

Page 55: Automated Testing of HVDC Control & Protection Systems1233259/FULLTEXT01.pdf Abstract Automated Testing of HVDC Control & Protection Systems Arnav Jain Testing is an important activity

Automated Testing of HVDC Control & Protection Systems 55

action files. The path to configuration files is preceded by a line containing thekeyword # Configuration. The action files are divided into three categories:preparatory actions, test actions, and restoration actions; the line containingthe actions are preceded by a line containing the keywords # Prepare, # Run,and # Restore respectively.

Similarly for configuration files the keywords for different data types are #Computer, # Drawing, # Active. In the configuration file after specifying thekeyword, but before specifying hostname, drawing path, or active signal the,the parser expects the system name for which the hostname, drawing or signalis defined, see Listing 4.

Listing 4: The structure of a configuration file is keyword, system name, andfinally data.

1 # Computer2 ## PC13 KF−PC1A45 # Drawing6 ## PC17 D:\Kriegers_Flak\AppSoftware\C12_Wx.1_PC1\MC1\MC1.hwsx

When parsing the action file some minor syntax validation is performed, suchas each action having correct type and number of arguments.

3.6.3 Running Actions

When a test is started, the system should be in the pre-defined initial state. Eachtest case has preconditions that must be met, running preparatory actions meetsthese preconditions. Then the actions for the actual test case are executed.Before concluding the test case, the system needs to be restored to the initialstate, which is the responsibility of the restoration actions.

As mentioned above in Section 3.4.4 there are three types of actions, commandaction, delaying action, and logging action. When any action is executed thereis a sequence of events occurring.

1. Detection of action type, i.e. if the current action is a command, delaying,or logging action.

2. Relevant active computers are detected, i.e. is system A or system B cur-rently active?

1. For command actions, execute order or change signal value2. For delaying actions, wait until the either specified time has elapsed

or until conditional requirement in the action has fulfilled.3. For logging actions, save the state of system to file.

Page 56: Automated Testing of HVDC Control & Protection Systems1233259/FULLTEXT01.pdf Abstract Automated Testing of HVDC Control & Protection Systems Arnav Jain Testing is an important activity

Automated Testing of HVDC Control & Protection Systems 56

Parse Actions

1+ Exists

No

Preparatory Action filesexist?

Parse  preparatoryaction file(s)

1+ Exists

No

Test case action filesexist?

Parse test caseaction file(s)

1+ Exists

No

Restorationaction filesexist?

Parse  restorationaction file(s)

Validate actions

Parse complete

Figure 17: Overview of the flow for parsing actions files

Page 57: Automated Testing of HVDC Control & Protection Systems1233259/FULLTEXT01.pdf Abstract Automated Testing of HVDC Control & Protection Systems Arnav Jain Testing is an important activity

Automated Testing of HVDC Control & Protection Systems 57

Run actions

Yes

No

Preparatoryactions exist? 

Yes

No

Test case  actions  exist? 

Yes

No

Restorationactions exist? 

Run next action

Yes

No

Further  actions  exist?

Run next action

Yes

No

Further  actions  exist?

Run next action

Yes

No

Further  actions  exist?

End actions

Figure 18: Overview of the flow for running actions

Page 58: Automated Testing of HVDC Control & Protection Systems1233259/FULLTEXT01.pdf Abstract Automated Testing of HVDC Control & Protection Systems Arnav Jain Testing is an important activity

Automated Testing of HVDC Control & Protection Systems 58

3.6.3.1 Logic for Changing Signals

Each time a new value is written to a signal, a verification is performed toensure the actual new value matches the intended new value. When writinga float or integer value, a tolerance of ±0.01 is allowed, whereas for booleanvalues, no tolerance is allowed as the new value may be either true or false.If the verification fails, the tool will repeatedly attempt to re-write the newintended value to the signal, until the verification passes. There exists a smallprobability that this may result in an endless loop, this behaviour has yet notbeen encountered, and no logic has been implemented to avoid this.

3.6.3.2 Logic for Reading Permission Signals

Operating analog and digital orders require verifying the permission signals tooperate. To ensure that a potential incorrect reading does not throw an error,some logic has been implemented. For each permission required, the signal isread. If the signal does not grant permission, the signal is read once again withthe same verification. If this permit check fails 5 times, permission to operatethe order is rejected.

3.6.4 Analog and Digital Orders

Analog and digital order symbols (see Figure 9 and Figure 10 respectively) areused initiate actions and change settings in the MACH main computers. Eachorder has a number of inputs signals and output signals connected; some ofwhich are control signals, while other are information signals. Unfortunately,interfacing the order symbol itself is not possible using the HiBug protocol,however interfacing the input and output signals of the symbol is possible. Theconsequence of this is that the tool must implement the same logic that thesymbol contains, such as verifying permission to change order or new valuesbeing within allowable limits.

3.6.4.1 Control Logic for Analog Order

An analog order controls a float or integer value. The current value of thisorder can be read from the input signal Actual Value (which can be found asthe third signal from top in Figure 9). Before a new value can be set, someconditions must be satisfied:

• Firstly, the input signal Permit must have a value of a boolean 1.

• Secondly, the new value must be within allowable limits, i.e. less than thevalue of the input signal Max and greater than the float of the input signalMin.

Once the above conditions are satisfied, the new value can be set by writing the

Page 59: Automated Testing of HVDC Control & Protection Systems1233259/FULLTEXT01.pdf Abstract Automated Testing of HVDC Control & Protection Systems Arnav Jain Testing is an important activity

Automated Testing of HVDC Control & Protection Systems 59

new value to the output signal Setpoint followed by briefly toggling the outputsignal Enter; first to a boolean 1, then to a boolean 0.

3.6.4.2 Control Logic for Digital Order

A digital order usually controls a switch to on/off, but can sometimes be usedas a trigger for a specific action. The current status of the order can be readfrom the boolean input signals Ind (Open) and Ind (Close) (the 6th and 7thinput signals (left side) from the top in Figure 10) and comparing with Table 3.To change the state of a digital order, some conditions must be satisfied:

• To set the digital order to open, the input signal Permit (Open) musthave a value of boolean 1.

• To set the digital order to close, the input signal Permit (Close) musthave a value of boolean 1.

Once the permit condition is fulfilled, the digital order can be opened or closedby briefly setting the output signals Order (Open) and Order (Close) respec-tively to a boolean 1 followed by a boolean 0.

Table 3: Status indication of a digital order

Ind (Open) Ind (Close) Status1 0 Open0 1 Closed0 0 Closed1 1 Invalid

3.6.5 Collecting Signals Connected to Orders

By using the HiBug protocol, we are not able to interface with order symbolsdirectly, but are rather interfacing with input and output signals for the orders.To ensure an easy-to-use syntax for the action files, the instructions in theaction files only references the unique order name, e.g. Write AnalogOrderBCP|BAPC!AO_P_REF 0 where BCP|BAPC!AO_P_REF represents the unique nameof the digital order. Thus, the challenge is to associate the order name with theinput and output signals connected to the order. The signal names are availablein the HiDraw drawing as can be seen in Figure 11, but requiring the user toenter the unique signal ID for each order to be used in a test case is neither userfriendly or practical. A way to cross reference the signals with the order wasrequired.

Page 60: Automated Testing of HVDC Control & Protection Systems1233259/FULLTEXT01.pdf Abstract Automated Testing of HVDC Control & Protection Systems Arnav Jain Testing is an important activity

Automated Testing of HVDC Control & Protection Systems 60

3.6.5.1 Searching Generated Source Files

As mentioned in Section 2.5.1.1, HiDraw can generate source files from theHiDraw drawings. The source files that are generated targeting the MACHmain computers are C/C++ files, thus they are in plain text format and easilysearchable. A review of the source files show that for each analog and digitalorder, the names of the signals is specified in the source files as variables. Ex-cerpt of one such source file, generated by HiDraw from the drawing showing inFigure 11, is shown in Listing 5.

The first implementation for collecting signals, the tool searched all of the sourcefiles for the correct order name. When a source file with the order name wasfound, the tool searched for the appropriate signal IDs in the file. The advantageof this methods is the speed of finding the sought after signals, the disadvantagethough is that the search is in no way efficient, and is very dependent on theformat of the source code i.e. if the format changes in a newer version of HiDraw,the search may fail.

Listing 5: Excerpt from the source file generated by HiDraw, from the drawingin Figure 11

1 /* Declaration for DIGITAL_ORDER */2 struct TCC_DO_TCCMANCTRL_HiDraw2InTouchStruct {3 unsigned Interlock : 1;4 unsigned ConsolID : 8;5 unsigned Open : 1;6 unsigned Close : 1;7 unsigned Occupied : 1;8 unsigned TagUnTag : 1;9 unsigned Selected : 1;10 unsigned ControlLocation : 1;11 unsigned ExtendedInterlockText : 3;12 unsigned Spare : 13;13 unsigned Toggle : 1;14 } *TCC_DO_TCCMANCTRL_HiDraw2InTouch;1516 [...]1718 LOCAL bool TCC_MAN_AUTO_CTRL_32;19 LOCAL bool TCC_MAN_AUTO_CTRL_44;20 LOCAL unsigned long TCC_MAN_AUTO_CTRL_ORD_ZZTMP50;21 PUBLIC unsigned long TCC_CONSOLE_MAN;22 PUBLIC bool TCC_LOG_EVENT_MAN;23 LOCAL bool TCC_MAN_AUTO_CTRL_ORD_ZZTMP51;2425 [...]2627 // Update the actual status.28 TCC_DO_TCCMANCTRL_HiDraw2InTouch−>Open = TCC_MAN_AUTO_CTRL_1766;29 TCC_DO_TCCMANCTRL_HiDraw2InTouch−>Close = TCC_MAN_AUTO_CTRL_1888;30

Page 61: Automated Testing of HVDC Control & Protection Systems1233259/FULLTEXT01.pdf Abstract Automated Testing of HVDC Control & Protection Systems Arnav Jain Testing is an important activity

Automated Testing of HVDC Control & Protection Systems 61

31 if( TCC_MAN_AUTO_CTRL_1766 && ( ! TCC_MAN_AUTO_CTRL_1888 ) ) {32 TCC_DO_TCCMANCTRL_HiDraw2InTouch−>Interlock = TCC_TCP_EQUAL;33 TCC_DO_TCCMANCTRL_InterlockStatus = TCC_MAN_AUTO_CTRL_21;34 } else if( ( ! TCC_MAN_AUTO_CTRL_1766 ) && TCC_MAN_AUTO_CTRL_1888 )

{35 TCC_DO_TCCMANCTRL_HiDraw2InTouch−>Interlock = TCC_PERMIT_MAN;36 TCC_DO_TCCMANCTRL_InterlockStatus = TCC_MAN_AUTO_CTRL_21;37 }

3.6.5.2 Searching HiDraw Drawings

Completely unrelated to this thesis project, but during the same time, a newcommand utility was being developed as part of an upcoming HiDraw release.This utility has the functionality to programmatically extract information aboutsymbols from a HiDraw project, when using the /FindSymbolInfo command lineswitch and specifying a xml file (shown in Listing 6) which contains informa-tion about which input and output signals are being sought after. The outputfrom the utility is also a xml file (shown in Listing 7 ) which can be easily beinterpreted by the automation tool.

The main disadvantage when using this utility is the execution time. Generationof the output that contains all digital and analog orders in the MACH maincomputer takes significant longer time compared to searching source files. It isexpected that this disadvantage decreases with the number of orders required tointerface against, when running a test, increases. The main advantage of usingthis utility is that the utility is actively being developed and maintained, thusany change in the format of the source files or structure of the HiDraw files,should not affect the output xml file.

Listing 6: The input file to the command utility1 <?xml version="1.0" encoding="UTF−8"?>2 <Find>3 <SymbolInfo Name="DIGITAL_ORDER" ParameterNo="1" InputNo="

1;2;6;7" OutputNo="1;2"/>4 <SymbolInfo Name="ANALOG_ORDER" ParameterNo="1" InputNo="

1;4;5;6" OutputNo="1;2"/>5 </Find>

Listing 7: Example of output from the command utility1 <?xml version="1.0" encoding="UTF−8"?>2 <FindResult xmlns:xsi="http://www.w3.org/2001/XMLSchema−instance"

xmlns:xsd="http://www.w3.org/2001/XMLSchema">3 <SymbolResult>4 <SymbolName>DIGITAL_ORDER</SymbolName>5 <SymbolVersion>2</SymbolVersion>6 <ApplicationName>TCC</ApplicationName>7 <Path>C:\Vault\Nordlink\S1_Ertsmyra\AppSoftware\

Page 62: Automated Testing of HVDC Control & Protection Systems1233259/FULLTEXT01.pdf Abstract Automated Testing of HVDC Control & Protection Systems Arnav Jain Testing is an important activity

Automated Testing of HVDC Control & Protection Systems 62

S1PxAK1Ax_PCP\MC1\Main\Control\TCC\MAN_AUTO_CTRL.HGF</Path>

8 <Parameter No="1" Name="ID Name:" Value="O_MAN_AUTO" />9 <Input No="1" Name="Permit (Open)" Signal="MAN_AUTO_CTRL_80

" DataType="Bool [1 bit]" />10 <Input No="2" Name="Permit (Close)" Signal="

MAN_AUTO_CTRL_81" DataType="Bool [1 bit]" />11 <Input No="6" Name="Ind (Open)" Signal="MAN_AUTO_CTRL_1766"

DataType="Bool [1 bit]" />12 <Input No="7" Name="Ind (Close)" Signal="MAN_AUTO_CTRL_1888

" DataType="Bool [1 bit]" />13 <Output No="1" Name="Order (Open)" Signal="MAN_AUTO_ORD"

DataType="Bool [1 bit]" />14 <Output No="2" Name="Order (Close)" Signal="

TCC_AUTO_ORD_TOSYS" DataType="Bool [1 bit]" />15 </SymbolResult>16 </FindResult>

3.7 Error Handling

One of the more challenging tasks when planning the automation tool was todefine how errors should be handled. The main challenge is that errors, bytheir nature, occur unexpectedly, i.e. the when errors occur the system is putinto an unknown state. A human operator is inherently smart and capableof adapting dynamically to the situation whereas an automation tool can onlyact in accordance with predetermined logic. If one test case fails to executeautomatically due to an error, there is high probably that next test case willalso fail due to system not being in a known state [3].

One way for an automatic test execution tool to handle errors (or rather notto) is to throw an error, and stop the execution of current case and remainingcases. Another way is to wait for an operator who may attend to, and reset thesystem to a known state, and then continue executing remaining test cases. Athird way is to reset the system to a known state, either by restarting the wholesystem or part of the system where the error occurred. This last option, whenautomatically resetting to a known state, is the most useful option in terms ofautomatic test execution. To enable this, the tool needs instructions on resetthe system.

The MACH C&P system is complex, with multiple subsystems involved. If onesubsystem encounters an error, either that system must be restored to a knownstate or the whole system must be reset. There are two types of errors that mayoccur, either an action planned to be carried out is not allowed to be carriedout or the system stuck is waiting for an event (e.g. signal value) which neverhappens. When either of these occur, a reset sequence needs to be initiated forthe system to achieve a known state.

Page 63: Automated Testing of HVDC Control & Protection Systems1233259/FULLTEXT01.pdf Abstract Automated Testing of HVDC Control & Protection Systems Arnav Jain Testing is an important activity

Automated Testing of HVDC Control & Protection Systems 63

3.8 Concept Demo Implementation

3.8.1 Creating Files

The demo test case (see Section 3.2) was run manually in order to detect andcollect the names for the orders and signals which were needed to create theaction files. The manual test procedure is performed on the HMI where it alsois possible to identify which orders in HiDraw that corresponds to the HMIaction.

The reference state, where the tests starts, is dependent on the HVDC project ortest sequence and was in this case defined as converter standby. This referencestate is dependent on project or test sequence. First of all, a configuration filefor the project was created by listing required computer hostnames and thepath to their associated HiDraw drawing. Then, a preparatory action file wascreated, containing digital orders and delays, which commands the system intopower transmission mode, for the action files see Section 8.3.1 for station 1and 8.3.2 for station 2. An action file was then created to transfer power innormal direction (station one to station two), see Section 8.3.3 for the actionfile.

In a second test case we added one more action file which tested power trans-mission in reverse direction, see Section 8.3.4 for actions file. The file was addedto a new test case file right after the first test action file, see Section 8.1.2 forthe test file.

3.8.2 Execution

To execute the demo, all the files were collected in the =same directory asthe executable automation framework file. The converter control system wasmanually set to the reference standby starting point where the test was executedfrom. The demo progress was monitored using the HMI.

Page 64: Automated Testing of HVDC Control & Protection Systems1233259/FULLTEXT01.pdf Abstract Automated Testing of HVDC Control & Protection Systems Arnav Jain Testing is an important activity

Automated Testing of HVDC Control & Protection Systems 64

4 Evaluation

4.1 Assessment of Inspection and Test Instructions

To confirm the functionality of the proposed testing framework, the selectedcases in Section 3.1 were used as a base for testing framework, and examplesare given for how they may be automated.

4.1.1 Test Case 1

The ready for condition, RFE, test case can be run automatically. Suppose thatthe system has three different sub-conditions, that all needs to be fulfilled, inorder to achieve the condition RFE. The test verifies that the system loses theRFE condition when any of the three sub-conditions are removed. The sub-conditions needs to be removed one at a time, by performing the actions thatremoves each respective sub-condition. Changing signals (or orders) is requiredto remove a condition. The procedure starts in a known standby state that isconfigured manually before the test is started, then the test case scripts containsfollowing actions:

1. Perform preparatory actions.

2. Log the RFE signal, plus signals and orders of the sub-conditions.

3. Remove the first condition by writing an action.

4. Log the RFE signal and signals and orders of the sub-conditions.

5. Return the first condition by restoring it to original state.

6. Repeat actions 2–5 above for all sub-conditions.

7. Energise the system by executing the energise sequence order.

8. Log the system.

9. De-energise the system and return to the start-up state.

All the actions in the list above are possible to automate using the testingframework. The outcome of the test is saved in a log file that tells if RFE isfulfilled and the configuration at each step. It is important that all signals andorders that are relevant to this test case are logged every time to be certain thatthe system is configured as expected during the whole test. When running thistest manually, the operator can visually verify the configuration on the HMIduring the whole test (i.e. dynamic comparison). If running this test manually,the log file can be used for verification of the test outcome using a post-executioncomparator.

Page 65: Automated Testing of HVDC Control & Protection Systems1233259/FULLTEXT01.pdf Abstract Automated Testing of HVDC Control & Protection Systems Arnav Jain Testing is an important activity

Automated Testing of HVDC Control & Protection Systems 65

4.1.2 Test Case 2

In the tap changer control test, a varying AC-grid voltage is required. Thevoltage of the AC-grid is controlled by the RTDS and could therefore not beautomated using the proposed automation tool.

The RTDS is running on another system and it was not in the scope of thisthesis project to study how that system could be implemented in our testingframework. One of the problem is that there is no communication between thecontrol & protection system and the RTDS that could be used. Further studieson communications with the RTDS system (and the RTDS system itself) isneeded to investigate if it is possible to automate test cases involving the RTDS.

This case shows that test cases including operations at the RTDS during testis unavailable at this point. There might be other test cases that can not beautomated, since they need an operation on another system that can not bereached by the HiBug protocol.

4.1.3 Test Case 3

The start-up operational test case was included as prepare actions in the demotest case (Listing 12). HMI screenshots are needed in the real test case todocument and verify that the conditions were fulfilled for each step. Since it isnot possible to take screenshots, a log of the system provided an alternative wayfor this documentation and verification. The produced log file can be adaptedso that only the signals of interest are analysed.

This is another test case that can be automated. During manual test, theoperator is able to dynamically verify that the test performs as expected. In aregression test, the operator will need to look at the log file to verify the testafter its completion. The person who writes the test script should know whichsignals are important to log for verification.

4.2 Concept Demo outcome

The interface against the MACH main computers worked mostly as intended.The actions (defined in the action files) were executed in a timely manner with-out unreasonable delays and without long execution times. Some defects wereencountered.

During the first trials, the permissions required to change orders (analog orderor digital) were denied, even though the actual permission signal was showinga boolean 1. Although the root cause was not discovered, the problem resultedin a reworked order permission logic, described in Section 3.6.3.2. With thereworked logic, the tool worked as intended. During the first trials, the per-missions required to change orders (analog order or digital) were denied, eventhough the actual permission signal was showing a boolean 1. Although the root

Page 66: Automated Testing of HVDC Control & Protection Systems1233259/FULLTEXT01.pdf Abstract Automated Testing of HVDC Control & Protection Systems Arnav Jain Testing is an important activity

Automated Testing of HVDC Control & Protection Systems 66

cause was not discovered, the problem resulted in a reworked order permissionlogic, described in Section 3.6.3.2. With the reworked logic, the tool worked asintended.

Another problem that occurred during the first trials was that saving a log ofall signal in the system took unreasonably long time. When verifying a test asnapshot of the current system state is sometimes required.

When testing manually, the current system state is saved by taking a screenshotof the HMI, which is not possible with the automation tool. Instead, a log ofall signals in the system, was proposed as the snapshot. During the first trials,logging all available signals in the system took unreasonably long time (morethan 15 minutes). If it takes long time between the first and last signal to belogged, it is not a snapshot of the system. There is a risk that the system statechanges during the logging procedure. To overcome this problem, the systemlog was redefined to only log orders and signals included in the action files. Thenew system log did the logging under one minute and the output log file wascontained less data, making it more manageable to read the file manually.

The automation tool took some minutes to read all the configurations beforethe actual testing started. This was not really a defect but it is an additionalstep compared to manual testing. The extra time that it takes comes when thetool processes the configuration file. The extra step only occurs the first timethe tool is run for that HVDC project or when the configuration file is changed.

The test case, the tool, and test scripts were demonstrated to the potentialend users. The reception was positive with multiple suggestions of usages, spe-cific for the individual end users. In addition there are suggestions for furtherdevelopment of the automation tool.

4.3 Adapting Scripts to Another Project

A focus for this thesis project has been that the testing framework should beeasy to maintain and work with. This means creating, maintaining and under-standing scripts, as well as adapt scripts to different HVDC projects should notbe difficult. HVDC Projects differs not on only in the technical functionality,but also in the architecture of the control system and the naming of computers,applications, orders and signals.

The demonstration test cases from Section 3.2 was initially trialled with oneHVDC project, (namely Nordlink, between Norway and Germany) and its labwith all systems. However, during the trials, due to issues not related to thisthesis project, the lab unexpectedly became unavailable. To further progressdevelopment and trials, another HVDC project (namely Kriegers Flak, betweenDenmark and Germany) and its lab was made available to this thesis project.This resulted in a requirement for adapting the same test case for another HVDCproject.

A new configuration file was created with the computer hostnames and HiDraw

Page 67: Automated Testing of HVDC Control & Protection Systems1233259/FULLTEXT01.pdf Abstract Automated Testing of HVDC Control & Protection Systems Arnav Jain Testing is an important activity

Automated Testing of HVDC Control & Protection Systems 67

drawing paths for the new HVDC project. Next, the action files needed to beadapted to the new project, this adaption is performed by finding the signals,analog order, and digital orders in the new project which corresponded with theold project. By running the test case manually using the HMI, it was possibleto gather the necessary signal and order names to adapt the action scripts tothe new project.

Due to the structure of the developed framework it is quite straight forwardto set up an environment to create new test cases or adapt cases from otherprojects. Familiarisation of the new project and adapting the whole test casetook just approximately 2 hours to complete.

Although the task was quite unexpected, it gave us a good insight in the adapt-ability of automation tool. Actually changing the signal and order names wasnot difficult, only tedious. The rather difficult part was finding which order andsignal in the new system corresponds to the order and signal in the old system.The duration of performing the task was shorter than anticipated, consideringthat not only did the names of signals and order differ, but also the architectureand scope of the projects. On the other hand we adapted only one single testcase, adapting multiple test cases combined with more familiarity with the newproject that we had, should enable a drastic reduction in the required time.

4.4 System Snapshot

Today, each test case often has a pre-action and a post-action to take screenshot,for documentation and verification purposes. Using the automation tool, theHMI is not a requirement, although the HMI may be used to oversee the systemduring the test run. As there is no communication between the OWS and theautomation tool, the tool may not take snapshots as per current procedure,therefore another solution by logging sought after values was implemented.

The logging actions enable the test tool to save current real-time values ofeither individual signals or orders, signals and orders on a specific computer, orsignals and orders in the system (multiple computers). This log file can be usedboth in manual verification and in automatic verification using a comparator.During this thesis project no comparator was implemented, however this log fileis suitable for verification using a post-execution comparator (see Section 2.2.5).

Listing 8: A example of a log file from the system1 2018−05−18 09:14:46.456887z System2 2018−05−18 09:14:46.456887z Signal

\\KF−PC1A\Main|SEQ!CONV_RFE 03 2018−05−18 09:14:46.831289z Signal

\\KF−PC2A\Main|SEQ!CONV_RFE 04 2018−05−18 09:14:48.219698z AnalogOrder

PC2|APC!AO_P_REF5 2018−05−18 09:14:48.219698z Signal

\\KF−PC2A\Main|APC!P_ORD 0

Page 68: Automated Testing of HVDC Control & Protection Systems1233259/FULLTEXT01.pdf Abstract Automated Testing of HVDC Control & Protection Systems Arnav Jain Testing is an important activity

Automated Testing of HVDC Control & Protection Systems 68

6 2018−05−18 09:14:48.219698z Signal\\KF−PC2A\Main|APC!PREF_132 0

7 2018−05−18 09:14:48.219698z Signal\\KF−PC2A\Main|APC!P_HMI_179 0

8 2018−05−18 09:14:48.219698z Signal\\KF−PC2A\Main|APC!P_HMI_181 0

9 2018−05−18 09:14:48.219698z Signal\\KF−PC2A\Main|APC!P_REF_MAX_HMI 410

10 2018−05−18 09:14:48.219698z Signal\\KF−PC2A\Main|APC!P_REF_MIN_HMI −410

11 2018−05−18 09:14:50.247711z DigitalOrderPC1|SSQ!O_STATE_STBY

12 2018−05−18 09:14:50.247711z Signal\\KF−PC1A\Main|SSQ!STATE_STANDBY_ORD 0

13 2018−05−18 09:14:50.247711z Signal\\KF−PC1A\Main|SSQ!HMI_DIRECT_STATE_ORD_165 0

14 2018−05−18 09:14:50.247711z Signal\\KF−PC1A\Main|SSQ!HMI_DIRECT_STATE_ORD_162 0

15 2018−05−18 09:14:50.247711z Signal\\KF−PC1A\Main|SSQ!STN_STDBY_ACTIVE 1

16 2018−05−18 09:14:50.247711z Signal\\KF−PC1A\Main|SSQ!HMI_DIRECT_STATE_ORD_166 0

4.5 Error Handling

The target for this thesis project with regards to error handling was to im-plement the third option (automatically reset the system to a known state, seeSection 3.7). Due to the complexity of the MACH C&P system, no suitable wayto reset the system was discovered. As a result of this, no recovery sequencewas implemented in the automation tool.

Ideally, for the situations when the tool encounters an issue, the tool shouldrun a reset sequence (similar to a normal action sequence), which resets thesystem including all stations, and then perform the actions required to bringthe stations back to the initial state. A possible place to define this reset scriptcould be in the test script (.tst) file, the advantage of this placement would bespecific recovery scripts for each test case, the disadvantage is that there maybe many reset scripts to manage. Another possible place is to define the resetscript in the configuration file for the HVDC project, the main advantage is asingle reset script for resetting, although this would require the reset to dealwith all possible situation that may occur. If the C&P system was developedwith automated testing in mind, resetting of the system could be similar totriggering an protective action by restarting the whole system.

Page 69: Automated Testing of HVDC Control & Protection Systems1233259/FULLTEXT01.pdf Abstract Automated Testing of HVDC Control & Protection Systems Arnav Jain Testing is an important activity

Automated Testing of HVDC Control & Protection Systems 69

4.6 Possible Applications

The initial idea behind a tool for automated test execution was to apply thetool to run full test cases during FST (systems test in accordance with the v-model in Figure 1). However, separating the actions of each tests into differentscripts enables execution of not only the full test case, but also independentlyexecute sequences or individual actions. This enables usage of the tool also inintegration testing and unit testing during development, such as running (nontest case) sequences or setting up the system environment by changing signalvalues (similarly to FSTSettings).

4.7 Feasibility

Depending on how many test runs are performed during an HVDC project, itmay be highly economical to implement automatic testing as can be seen fromthe simplified calculations presented in Table 4 below.

Facts assumed when calculating costs:

• Labour costs 500 SEK per person per hour.

• 8 hours per work day, i.e. 4 000 SEK.

• 40 hours per work week, i.e. 20 000 SEK.

• 178 hours per work month, i.e. 89 000 SEK.

• Test cases will need to be designed regardless if the tests are ran automat-ically or manually.

• Creating the automation tool is estimated to take 2 persons 2 months.

• Creating the test scripts is estimated to take 1 persons 2 weeks.

• Running all tests manually takes 1 person 2 weeks labour, whereas run-ning automatically using takes one person 8 hours including verificationof results.

• It is assumed full tests are run manually 4 times during an HVDC project,whereas with an automation tool tests may be ran at least 20 times.

Table 4: A simple feasibility calculation for running tests automatically.

Manual AutomaticDesigning Test Cases No difference No differenceDeveloping automation tool 356 000Creating test scripts 40 000Running test 40 000 4 000Number of runs per project 4 20Total testing cost 160 000 476 000Cost per test run 40 000 23 800

Page 70: Automated Testing of HVDC Control & Protection Systems1233259/FULLTEXT01.pdf Abstract Automated Testing of HVDC Control & Protection Systems Arnav Jain Testing is an important activity

Automated Testing of HVDC Control & Protection Systems 70

5 Conclusions

During this thesis project selected test cases were reviewed, and most test caseswere found possible to automate. To help with automation a testing frameworkwas created to identify how automated testing can be structured. To show thatthis testing framework was sound, an automated test execution tool was cre-ated, which proved a proof of concept demonstrated by automatically runninga specified test case. To verify the outcome of a test case, an additional com-parator is required. Although this “proof of concept” has been proven to bepositive, for a full implementation further study with regards to protection testcases and verification is required.

A review was performed on selected control test cases to find possible candi-dates to run automatically. Although not all test cases are possible to runautomatically within the scope of this thesis project, most of the control casesare possible. One new test case, which combined multiple individual control testcases, was created for demonstrative purpose of the automation tool prototype.

A testing framework to run tests automatically on the MACH C&P systemwas created. This framework contains a structure for how the files used inautomated testing, i.e. which types of files are required and their content. Thetesting framework in its current state enables automation of control test cases,but not protection test cases. Based on this framework, an automation toolwas developed to automatically execute test cases. The demonstrative testcase was applied to this tool, mainly to demonstrate that the test execution ispossible to automate, but also to show the possibilities with the test framework.Verifying results require a comparator to compare the actual outcome with theexpected outcome. The testing framework takes the requirement of a post-execution comparator into account, but no comparator was implemented in thetest execution tool to demonstrate this ability.

Available communication interfaces between the automation tool and the MACHmain computers were evaluated for use in the automation tool. From the avail-able options the HiBug communication protocol using IApplicationMonitor2library was chosen.

The main usage area as initially intended for the testing framework and the testexecution tool is to run tests during the systems test performed at the ABBfactory. The framework and tool may also be used for unit and integrationtesting.

As the C&P system in current version was not developed with automatic testingin mind, it is today quite challenging to manage any error that occurs duringtest runs. For implementation of automatic testing, either the C&P systemrequires adapting for resetting, or the automation tool need conditional logic toadapt to different situations.

Whether automated testing is economical depends on how feasibility is cal-culated. In terms of explicit benefits and costs, automation is an expensiveinvestment, and requires many runs before cost per test run is same as manual

Page 71: Automated Testing of HVDC Control & Protection Systems1233259/FULLTEXT01.pdf Abstract Automated Testing of HVDC Control & Protection Systems Arnav Jain Testing is an important activity

Automated Testing of HVDC Control & Protection Systems 71

testing. However with implicit benefits such as shorter development times andimplicit costs as more complexity are difficult to calculate.

Page 72: Automated Testing of HVDC Control & Protection Systems1233259/FULLTEXT01.pdf Abstract Automated Testing of HVDC Control & Protection Systems Arnav Jain Testing is an important activity

Automated Testing of HVDC Control & Protection Systems 72

6 Future Work

This thesis project has given insight in test automation by presenting a conceptof testing framework and a simple test execution tool. This is however only onepart of possible automated testing as part of the FST.

6.1 Testing Framework

The test framework that was developed in this thesis project has been focused inthe control part of the system. If protection tests are to be implemented, actionsstructure will be needed for protection test cases. Since protection cases involvesoperation on the RTDS, a study may be required to analyse the possibilities andchallenges with coordination and communication of the two systems (MACHand RTDS)

The scripts are currently created in a simple text editor, which may not bethe most user friendly tool. Other alternatives should be investigated, such asgraphical tools or more intelligent text editors, tailored for test script creation.

As the HVDC project grows, usage of test automation tool is beneficial. A studymay be required to analyse how the automated tests can be implemented usingtest management tools. The amount of tests today are already extensive, andintroducing automated testing will likely result in further more tests. Managinga large amount of tests without a test management tool might be arduous.

6.2 Automation Tool

The functionality of the automation tool may be extended to include loops andconditional statements. There is also a need for investigation of whether theprotection test cases, those requiring operations on the RTDS, can be imple-mented through this tool or with another tool in combination with this tool.The tool should also be tested with more different system to confirm that thetool works on different MACH system that have differences in configurationcompared with the two HVDC projects trialled during this thesis project.

6.3 Verification and Presentation of Results

The execution tool in current state does not support any automatic verification.If the goal is to have a report to be automatically written after the test setshas been run, presenting the results of the test, some kind of comparator isneeded. For control testing both dynamic and post-execution comparators maybe used. Also a report needs to be created which can present the test executionand verification results.

Page 73: Automated Testing of HVDC Control & Protection Systems1233259/FULLTEXT01.pdf Abstract Automated Testing of HVDC Control & Protection Systems Arnav Jain Testing is an important activity

Automated Testing of HVDC Control & Protection Systems 73

7 References

[1] J. Pan, “Software testing.” Available at https://users.ece.cmu.edu/~koopman/des_s99/sw_testing/ (2017-11-20), 1999.

[2] R. V. Hooda, “An automation of software testing: A foundation for thefuture,” International Journal of Latest Research in Science and Technology,vol. 1, no. 2, pp. 152–154, 2012.

[3] M. Fewster and D. Graham, Software test automation. Addison-WesleyProfessional, 1999.

[4] ABB, “Kort om abb.” Available at http://new.abb.com/se/om-abb/kort(2017-11-24).

[5] ABB, “Power grids.” Available at http://new.abb.com/se/om-abb/verksamhet/power-grids (2017-11-24).

[6] ABB, “HVDC.” Available at http://new.abb.com/systems/hvdc/why-hvdc(2018-03-16).

[7] ABB, “HVDC Light It’s time to connect,” ABB Technical Report, Dec. 2012.

[8] K. Lindholm, “Elnätet – distribution - Energiföretagen Sverige.” Availableat https://www.energiforetagen.se/sa-fungerar-det/el/distribution/(2017-11-24), 08-Feb-2017.

[9] D. Jovcic and K. Ahmed, High voltage direct current transmission: Convert-ers, systems and DC grids. Wiley, 2015.

[10] S. Sharma, Power electronics. Laxmi Publications Pvt Limited, 2008.

[11] T. Karlsson, M. Hyttinen, L. Carlsson, and H. Björklund, “Modern con-trol and protection system for HVDC.” Available at http://new.abb.com/se/temasidor/polhemspriset/mach-kontrollsystem (2018-02-09).

[12] L. Bertoni, M. F. Araújo, W. A. Da Silva, and L. F. A. F. Nobre, “MACH2- Modular Advanced Control 2nd edition,” 2004, pp. 884–889.

[13] M. Hyttinen and K. Bentzen, “Operating experiences with a voltage sourceconverter (HVDC Light) on the gas platform Troll A.” Available at http://search-ext.abb.com/library/Download.aspx?DocumentID=9AKK101130D3007&LanguageCode=en&DocumentPartId=&Action=Launch (2018-02-09).

[14] P. K. J. Mohapatra, Software engineering: A lifecycle approach. New Delhi:New Age International, 2010.

[15] V. Massey and K. Satao, “Comparing various SDLC models and the newproposed model on the basis of available methodology,” International Journal,vol. 2, no. 4, 2012.

Page 74: Automated Testing of HVDC Control & Protection Systems1233259/FULLTEXT01.pdf Abstract Automated Testing of HVDC Control & Protection Systems Arnav Jain Testing is an important activity

Automated Testing of HVDC Control & Protection Systems 74

[16] R. S. Carson, “4.2.1 Can Systems Engineering be Agile? DevelopmentLifecycles for Systems, Hardware, and Software,” INCOSE International Sym-posium, vol. 23, no. 1, pp. 16–28, 2013.

[17] J. Dooley, Software development and professional practice. New York,N.Y.;New York; Apress.

[18] A. Butterfield and G. E. Ngondi, A dictionary of computer science, Seventh.Oxford: Oxford University Press, 2016.

[19] J. Kasurinen, O. Taipale, and K. Smolander, “Software test automation inpractice: Empirical observations,” Advances in Software Engineering, vol. 2010,2010.

[20] G. J. Myers, T. Badgett, T. M. Thomas, and C. Sandler, The art of softwaretesting, 2nd ed. Hoboken, N.J: Wiley, 2004.

[21] G. Rothermel, R. H. Untch, C. Chu, and M. J. Harrold, “Prioritizing testcases for regression testing,” IEEE Transactions on Software Engineering, vol.27, no. 10, pp. 929–948, Oct. 2001.

[22] M. Harrold, R. Gupta, and M. Soffa, “A methodology for controlling the sizeof a test suite,” ACM Transactions on Software Engineering and Methodology(TOSEM), vol. 2, no. 3, pp. 270–285, 1993.

[23] A. K. Onoma, W.-T. Tsai, M. Poonawala, and H. Suganuma, “Regressiontesting in an industrial environment,” Commun. ACM, vol. 41, no. 5, pp.81–86, May 1998.

[24] Z. Melhem, Electricity transmission, distribution and storage systems, vol.Number 38. Oxford: Woodhead Publishing, 2013.

[25] M. P. Bahrman, J. G. Johansson, and B. A. Nilsson, “Voltage source con-verter transmission technologies: The right fit for the application,” 2003, vol.3, pp. 1840–1847 Vol. 3.

[26] ABB, “Control and Protection Delivery Control,” ABB Technical Report,Jun. 2016.

[27] H. Lerneman, “RO hantering på ABB, TPE.” Karlstad University; Depart-ment of Physics; Electrical Engineering, 2014.

[28] ABB, HiDraw studio, 9.11 Beta 2. ABB AB, 2018.

[29] T. Magg, M. Manchen, E. Krige, E. Kandjii, R. Palsson, and J. Wasbor,“Caprivi link hvdc interconnector: Comparison between energized system test-ing and real-time simulator testing,” in Cigré session, 2012, pp. 26–31.

[30] U. Åström, V. Lescale, D. Menzies, M. Weimin, and L. Zehong, “Thexiangjiaba-shanghai 800 kV uhvdc project, status and special aspects,” in Powersystem technology (powercon), 2010 international conference on, 2010, pp. 1–6.

Page 75: Automated Testing of HVDC Control & Protection Systems1233259/FULLTEXT01.pdf Abstract Automated Testing of HVDC Control & Protection Systems Arnav Jain Testing is an important activity

Automated Testing of HVDC Control & Protection Systems 75

[31] ABB, “HVDC control system-Inspection and Test Instrucitons,” ABB Tech-nical Report, Sep. 2017.

[32] ABB, “Mode sequences,” ABB Technical Report, Mar. 2017.

[33] A. M. Memon and M. L. Soffa, “Regression testing of guis,” SIGSOFTSoftw. Eng. Notes, vol. 28, no. 5, pp. 118–127, Sep. 2003.

[34] B. Pettichord, “Success with test automation,” in Proceedings of the ninthinternational, 1996.

[35] Invensys Systems, Wonderware® FactorySuite™ InTouch™ user’s guide,Version A. Invensys Systems, Inc., 2002.

[36] ABB, “Interfaces between MACH system software and HiDraw,” ABB Tech-nical Report, Jul. 2015.

[37] O. Laursen, H. Björklund, and G. Stein, “Modern man-machine inter-face for HVDC systems.” Available at https://library.e.abb.com/public/4877596fd88ecf4ac1256fda004aead9/abb_mmi.pdf (2018-05-16).

Page 76: Automated Testing of HVDC Control & Protection Systems1233259/FULLTEXT01.pdf Abstract Automated Testing of HVDC Control & Protection Systems Arnav Jain Testing is an important activity

Automated Testing of HVDC Control & Protection Systems 76

8 Appendix: Example of a test suite

8.1 Test Case File examples

8.1.1 Test Case 1 Example

This test case is part of demonstration test case (Section 3.2) and starts withboth stations in standby mode. First station 1 is set to operational mode, thenstation 2 is set to operational mode. When both stations are operating, poweris transferred from station 1 to station 2, the power ramp is performed withtwo different ramp speeds. When the power ramp is completed, power transferis stopped. Finally, station 2 is set to standby followed by station being set tostandby.

Listing 9: Test case file; Kriegers.tst1 ;Simple demo of concept23 # Configuration4 Kriegers.conf56 # Prepare7 Kriegers.Prep.PC1.STBY.OPN.act8 Kriegers.Prep.PC2.STBY.OPN.act910 # Run11 Kriegers.Run.PowerTransfer.S1S2.act12 Kriegers.Run.PowerTransfer.Stop.act13 Kriegers.Log.act1415 # Restore16 Kriegers.Restore.PC2.OPN.STBY.act17 Kriegers.Restore.PC1.OPN.STBY.act

8.1.2 Test Case 2 Example

This test case is an extension to test case 1 example above and is the fulldemonstration test case (Section 3.2). The preparation sequence is same asthe sequence in test case 1 example. After the power transfer is accomplishedin normal direction, another power transfer is initiated, in reverse direction,i.e. from station 2 to station 1. Finally, both stations are set to standby, sameas in the test case 1 example.

Listing 10: Test case file; Kriegers2.tst1 ;Simple demo of concept23 # Configuration

Page 77: Automated Testing of HVDC Control & Protection Systems1233259/FULLTEXT01.pdf Abstract Automated Testing of HVDC Control & Protection Systems Arnav Jain Testing is an important activity

Automated Testing of HVDC Control & Protection Systems 77

4 Kriegers.conf56 # Prepare7 Kriegers.Prep.PC1.STBY.OPN.act8 Kriegers.Prep.PC2.STBY.OPN.act910 # Run11 Kriegers.Run.PowerTransfer.S1S2.act12 Kriegers.Run.PowerTransfer.Stop.act13 Kriegers.Run.PowerTransfer.S2S1.act14 Kriegers.Run.PowerTransfer.Stop.act15 Kriegers.Log.act1617 # Restore18 Kriegers.Restore.PC2.OPN.STBY.act19 Kriegers.Restore.PC1.OPN.STBY.act

8.2 Configuration File Example

Listing 11: Configuration file; Nordlink.conf1 # Computer2 ## PC13 KF−PC1A4 KF−PC1B5 ## PC26 KF−PC2A7 KF−PC2B89 # Drawing10 ## PC111 D:\Kriegers_Flak\AppSoftware\C12_Wx.1_PC1\MC1\MC1.hwsx12 ## PC213 D:\Kriegers_Flak\AppSoftware\EC50_Wx.1_PC2\MC1\MC1.hwsx1415 # Active16 ## PC117 PC1|SSL!Active18 ## PC219 PC2|SSL!Active

Page 78: Automated Testing of HVDC Control & Protection Systems1233259/FULLTEXT01.pdf Abstract Automated Testing of HVDC Control & Protection Systems Arnav Jain Testing is an important activity

Automated Testing of HVDC Control & Protection Systems 78

8.3 Action File Example

In the listings below, are the action files used in the demonstration test casediscussed in Section 3.2

8.3.1 Preparatory Actions: Station 1

Listing 12: Preparatory actions file, Kriegers.Prep.PC1.STBY.OPN.act1 ; Start at standby for PC123 Log System45 Delay Signal PC1|SEQ!CONV_RFE 16 Write DigitalOrder PC1|SSQ!O_STATE_BLKD 17 Delay Signal PC1|SEQ!RFO 18 Write DigitalOrder PC1|SSQ!O_STATE_NORM_OPN 1910 Delay Seconds 5

8.3.2 Preparatory Actions: Station 2

Listing 13: Preparatory actions file, Kriegers.Prep.PC2.STBY.OPN.act1 ; Start at standby for PC223 Log System45 Delay Signal PC2|SEQ!CONV_RFE 16 Write DigitalOrder PC2|SSQ!O_STATE_BLKD 17 Delay Signal PC2|SEQ!RFO 18 Write DigitalOrder PC2|SSQ!O_STATE_NORM_OPN 1910 Delay Seconds 5

8.3.3 Test Actions: Power Transfer Station 1 to Station 2

Listing 14: Test 1 actions file, Kriegers.Run.PowerTransfer.S1S2.act1 Delay AnalogOrder PC2|APC!AO_P_RAMP2 Write AnalogOrder PC2|APC!AO_P_RAMP 1003 Delay AnalogOrder PC2|APC!AO_P_REF4 Write AnalogOrder PC2|APC!AO_P_REF 3056 Delay Signal PC2|APC!P_MEAS 207 Write AnalogOrder PC2|APC!AO_P_RAMP 40

Page 79: Automated Testing of HVDC Control & Protection Systems1233259/FULLTEXT01.pdf Abstract Automated Testing of HVDC Control & Protection Systems Arnav Jain Testing is an important activity

Automated Testing of HVDC Control & Protection Systems 79

89 Delay Signal PC2|APC!P_RAMP_IN_PROG 01011 Log System12 Delay Seconds 5

8.3.4 Test Actions: Power Transfer Station 2 to Station 1

Listing 15: Test 1 actions file, Kriegers.Run.PowerTransfer.S2S1.act1 Delay AnalogOrder PC2|APC!AO_P_RAMP2 Write AnalogOrder PC2|APC!AO_P_RAMP 603 Delay AnalogOrder PC2|APC!AO_P_REF4 Write AnalogOrder PC2|APC!AO_P_REF −3056 Delay Signal PC2|APC!P_MEAS −207 Write AnalogOrder PC2|APC!AO_P_RAMP 20089 Delay Signal PC2|APC!P_RAMP_IN_PROG 01011 Log System12 Delay Seconds 5

8.3.5 Test Actions: Stop Power Transfer

Listing 16: Test 1 actions file, Kriegers.Run.PowerTransfer.Stop.act1 Write AnalogOrder PC2|APC!AO_P_RAMP 2002 Write AnalogOrder PC2|APC!AO_P_REF 03 Delay Signal PC2|APC!P_RAMP_IN_PROG 04 Write AnalogOrder PC2|APC!AO_P_RAMP 40

8.3.6 Restore Actions: Station 1

Listing 17: Restoration actions file, Kriegers.Restore.PC1.OPN.STBY.act1 ;Move station to standby mode2 Delay Signal PC1|SEQ!RFB 13 Write DigitalOrder PC1|SSQ!O_STATE_BLKD 14 Delay DigitalOrder PC1|SSQ!O_STATE_STBY 15 Write DigitalOrder PC1|SSQ!O_STATE_STBY 167 Delay Signal PC1|SSQ!STN_STDBY_ACTIVE 189 log system

Page 80: Automated Testing of HVDC Control & Protection Systems1233259/FULLTEXT01.pdf Abstract Automated Testing of HVDC Control & Protection Systems Arnav Jain Testing is an important activity

Automated Testing of HVDC Control & Protection Systems 80

8.3.7 Restore Actions: Station 2

Listing 18: Restoration actions file, Kriegers.Restore.PC2.OPN.STBY.act1 ;Move station to standby mode2 Delay Signal PC2|SEQ!RFB 13 Write DigitalOrder PC2|SSQ!O_STATE_BLKD 14 Delay DigitalOrder PC2|SSQ!O_STATE_STBY 15 Write DigitalOrder PC2|SSQ!O_STATE_STBY 167 Delay Signal PC2|SSQ!STN_STDBY_ACTIVE 189 log system