IN THE FIELD OF TECHNOLOGYDEGREE PROJECT VEHICLE ENGINEERINGAND THE MAIN FIELD OF STUDYINDUSTRIAL MANAGEMENT,SECOND CYCLE, 30 CREDITS

, STOCKHOLM SWEDEN 2018

Managing Validation in a Safety Critical System Regarding Automation of Air Traffic Control

ANDRÉS DE FREITAS MARTINEZ

NURDIN MOHAMED

KTH ROYAL INSTITUTE OF TECHNOLOGYSCHOOL OF INDUSTRIAL ENGINEERING AND MANAGEMENT

This page is intentionally left blank.

Managing Validation in a Safety Critical System Regarding

Automation of Air Traffic Control

Nurdin Mohamed

Andrés De Freitas Martinez

Master of Science Thesis

TRITA-ITM-EX 2018:632

KTH Industrial Engineering and Management

Industrial Management

SE-100 44 STOCKHOLM

Master of Science Thesis TRITA-ITM-EX 2018:632

Managing Validation in a Safety Critical System Regarding

Automation of Air Traffic Control

Andrés De Freitas Martinez

Nurdin Mohamed

Approved Examiner

Pernilla Ulfvengren

Supervisor

Matthew Stogsdill

Commissioner

European Performance Management Systems

Committee

Contact person

Peter Griffiths

Abstract

The aviation industry is under increasing pressure to reduce cost and manage the increased number

of passengers. One area under pressure is the Air Traffic Control. The Air Traffic Control will in

a foreseeable future manage the introduction of drones also known as Unmanned Aerial Vehicles

by integrating them into civil airspace with manned aircraft. Drones are lacking consensus from

authorities with regards to standards due to their rapid expansion. Given their size, shape and speed,

they can also pose threats to manned aircrafts and there is a need to address them in an Air Traffic

Management system interoperating with manned aircrafts. The purpose in this study is to identify

what considerations to make when automating complex system elements with respect to safety.

Safety involves all the different stakeholders in the air transportation system, which is a Safety

critical System. Furthermore, the aim is also to identify areas in which European Operational

Concept Validation Methodology (E-OCVM) can be complemented with. Standard E-OCVM is

missing specific assessment criteria with regards to safety and how it can interact with other

standards. The approach is thereby to use various standards with focus on Systems Engineering to

complement E-OCVM since it is lacking with regards to how it is used to validate Air Traffic

Control systems. To capture the complexity of automating elements of an industry involving many

stakeholders, a qualitative analysis was conducted in this project, using a System Engineering

approach with four standards A-SLP, A-RLP, A-DAS and A-SAS. A-SLP and A-RLP are two

general standards while A-DAS and A-SAS are focusing on the contexts of aircrafts and software

development. Empirical data was gathered by semi-structured interviews of seven experts within

the relevant areas in the field. From the review of the four standards, it was found that they can for

instance complement E-OCVM in how software errors can lead to a failure condition among other

ways. The main identified considerations faced with an integration of drones into civil airspace, is

to manage the human interaction with the introduced Air Traffic Management systems. More

specifically, the human element must be involved from the training phase in the development of

systems in a Safety Critical System to minimize risk. Furthermore, redundancies that are built into

the system has to, not only be able to put the system into a safe state, but also be carefully analyzed

in how they interact with other systems to avoid misjudgement for the Air Traffic Controllers.

Lastly, to obtain specific details on how interoperability could occur using standards, the standards

used in this study refer to usage of other documents and standards. Standards specifically tailored

for the operational context of drones would facilitate further testing and implementation of their

integration into civil airspace. Given that different standards were used to complement the E-

OCVM standard, a set of unified standards are required that are proportional with the type of

drones, the type of operations and in the environment that they are operating in. This will be needed

to fulfill the European vision of safe integration of drones and needs thereby to be carried out in a

global manner, thus also share experience with other actors to advance the new technology

adaptation.

Keywords: Air Traffic Control (ATC), Air Traffic Controller (ATCo), Unmanned Aerial Vehicles

(UAV), Drones, Safety, Validation, System integration, Quality Assurance, Mixed operation,

Interoperability, Training, Standards, System Engineering.

Acknowledgements

We would like to express our appreciation to our supervisor Matthew Stogsdill for the inspirational

feedback and cheerfulness to help us stay motivated throughout the project. Furthermore, we would

also like to thank Pernilla Ulfvengren for helping us in the initial phase and for introducing us to

our company supervisor Peter Griffiths, and additionally for providing us with helpful feedback.

We would further like to express our gratitude to our supervisor at the European Performance

Management Systems Committee, Peter Griffiths. He pointed us in a favorable direction with

regards to important stakeholders and highly intelligent people in the aviation industry. Lastly, we

would like to thank several stakeholders that were our interview candidates, namely Fredrik

Asplund, Paul Kennedy, Bengt-Göran Sundqvist, Marc Baumgartner, Eric Kroese and Marek

Bekier.

Thank you!

Abbreviations 8

List of Figures 9

1.1 Problematization 16

1.3 Delimitations 17

1.4 Expected Contribution 18

2 Literature Review 20

2.1 Systems Thinking 20

2.2.1 Validation 22

2.3 Safety Critical System 24

2.4 Automation, UAV and ATM 25

2.4.1 UAV 26

2.4.2 ATM, ATC and ATCo 27

2.5 Standards 32

2.5.1 E-OCVM - European Operational Concept Validation Methodology: E-OCVM Version

3.0 Volume I 33

2.5.2 A-SLP - Systems and Software Engineering - System Life-cycle Processes:

ISO/IEC/IEEE 15288 33

2.5.3 A-RLP - Systems and Software engineering - Life-Cycle Processes - Risk Management:

ISO/IEC 16085 33

2.5.4 A-DAS - Aerospace Recommended Practice: SAE Aerospace ARP4754A 34

2.5.5 A-SAS - Software Considerations in Airborne System and Equipment Certification:

RTCA DO-178C 34

3 Method 36

3.1.1 Choice of Research Design & Pre Study 36

3.1.2 Literature Study 38

3.1.3 Interviews 39

3.1.4 Standards Review 44

3.1.5 Method Process 46

3.1.6 Theory on Method Criticism 49

4 Results & Analysis 51

4.1 Standard Review using Key Terms 51

4.1.1 Safety 51

4.1.2 Validation 52

4.1.3 System integration 53

4.1.4 Quality Assurance 55

4.2 Interviews 58

4.2.1 Safety 58

4.2.2 Training Phase 60

4.2.3 Future of System element Design 63

5 Discussion and conclusions 67

5.1 Discussion on Sustainability 70

5.2 Scrutiny of Method 70

5.2.1 Validity 71

5.2.2 Generalizability and Reliability 71

5.3 Conclusion 72

5.4 Further Research 73

7 Appendix 74

7.1 Appendix A 74

7.2 Appendix B 75

7.3 Appendix C 76

8 References 79

This page is intentionally left blank.

Abbreviations

ANSP Air Navigation Service Provider

ATM Air Traffic Management

ATC Air Traffic Control

ATCo Air Traffic Controller

IAA Irish Aviation Authority

ACR Aviation Capacity Resources

UAV Unmanned Aerial Vehicles

SOI System Of Interest

SoS System of System

ScS Safety critical System

CPS Cyber Physical System

E-OCVM Standard: European Operational Concept

Validation Methodology

A-SLP Standard ISO 15288: System Life-cycle

Processes

A-RLP Standard ISO 16085: Risk Management for

Life-cycle Processes

A-DAS Standard ARP4754A: Guidelines for

Development of Civil Aircrafts and Systems

A-SAS Standard DO-178C: Software Considerations

in Airborne Systems

List of Figures

Figure 1. Illustration of how SoS, SOI and system elements can be viewed in the context of ATC

(Systems Engineering Handbook, 2006).

Figure 2. A figure on the V-model (Asplund, 2014).

Figure 3. An illustration of controlled and uncontrolled airspace where the grey shaded area is

controlled airspace and the white is uncontrolled airspace in proximity to an airport (Eurocontrol,

2013).

Figure A. Illustration of how standard E-OCVM’s table of contents was analyzed (E-OCVM,

2010).

Figure B. Illustration of how standard A-SLP’s table of contents were analyzed separately by the

two authors (Systems and software engineering - System life cycle processes, 2015).

Figure C. An illustration of how intelligence is divided with regards to AI in different categories

(Russell & Norvig, 2010).

List of Tables

Table 1. A table on the nomenclature used to ease referencing to standards.

Table 2. A table demonstrating segregated airspace & non-segregated airspace.

Table 3. An illustration of different approaches to automation adapted from HALA! ( 2010).

Table 4. An illustration of the levels of automation adapted from HALA! (2010).

Table 5. Illustration of which methods were used to answer each research question.

Table 6. A chart on the procedure of analyzing the literature review.

Table 7a. A table on the interviewees and their respective roles.

Table 7b. Continuation of Table 7a

Table 8a. A table on the interview questions after pre-study.

Table 8b. Continuation of Table 8a.

Table 9. A table on the specific key terms used in the project to facilitate review of standards.

Table 10a. A chart on the procedure of analyzing the standards.

Table 10b. Continuation of Table 10a.

Table 11a. Illustration of how standard E-OCVM’s table of contents were analyzed (this is only

an extract from the original picture, for more detailed information see Appendix A) adapted from

E-OCVM (2010).

Table 11b. Continuation of Table 11a.

Table 12a. Illustration of how standard A-SLP’s table of contents was analyzed separately by the

two authors on each side of the table (this is only an extract from the original picture, for more

detailed information see Appendix B) adapted from Systems and software engineering - System

life cycle processes (2015).

Table 12b. Continuation of Table 12a.

1 Introduction The aviation industry is one of the most important transportation modes for a country’s

accessibility in the global market and has similarly as in other industries kept a high pace to meet

sustainability requirements (Wittmer and Bieger, 2018). The aviation industry consists of various

elements in a value chain such as airport operations, yield management for airlines and aircraft

maintenance. Moreover, there is also an important element which manages the intermediary

connection between an airport and an aircraft, namely Air Traffic Management (ATM). More

specifically, ATM refers to the overall management of air traffic while Air Traffic Control (ATC)

is a part which controls the movement of aircrafts in the airspace or at airports. The tower can be

staffed by one or more Air Traffic Controllers (ATCo), which are the ones providing the service

to the aircrafts. The increased pressure from technology transformation and the entrance to the

digitization and automation era is forcing the aviation industry to change (Baumgartner, 2017).

With the exception of ATM, almost all previously mentioned elements such as airport operations

and aircraft maintenance have been optimized while ATM soon is reaching their limits in terms of

capacity and costs (IATA, 2016).

Unmanned Aerial Vehicles (UAV) or drones are rapidly entering the markets (Finger et al., 2016)

which is a vehicle with the responsible pilot on the ground. Given the drones current size, shape

and speed, they pose threats to commercial aircrafts and are currently flying below height of

commercial aircrafts where both parts lack sophisticated detect and avoid systems for each other

(Cohn et al., 2017). Currently, tests are being conducted to integrate UAVs to the current system

of managing manned aircrafts. One of these tests, a European cooperation, managed by Saab under

the framework of the European Defense Agency, is the “MIDCAS Projects”. Their objective is to

integrate Remote Piloted Aircraft System (RPAS) or drones into the civil airspace and to function

alongside the manned aviation (Saab Corporate, 2015). Besides drones, there are also current

advancements in providing a platform for integrating information from actors in the proximity of

an airport called SWIM. SWIM has the aim to provide real time information sharing between actors

such as airline operations center, airport, ANSP (Air Navigation Service Provider) and vehicles at

the airport (SESAR SWIM Factsheet, 2016). Previously, the information received from similar

actors were less organised and inflexible which with the increase in capacity demand, attention to

environmental pressure and overall economic impact puts pressure on seamless information

exchange and access (SESAR SWIM Factsheet, 2016).

Interoperability is a notion to mirror the considerations to be made when drones are to be integrated

into the current civil airspace. Given the drone’s capabilities, they can pose threats to aircrafts and

there is a need to address them in an ATM system. Interoperability will also be required since ATM

is built upon a radar-based system which primarily is beholden to a World War II era system (Oster

and Emeritus, 2015), which assumes that similar procedures as today will be used in the future

(Griffiths, 2018). The first steps in providing automation in the aircraft industry is to incrementally

introduce incremental automation tools, this will continue until the entire system is (or could be)

automated (Tay and Becker, 2018). Until this level of full automation is achieved, cooperation

between the human and machine will be even more important as the human operator is still vital

to ensure safety and performance (Pacaux M. P. et. al, 2011).

One of the ATCo’s main priorities is to provide safe separation between aircraft which with regards

to future implementation of automation at least needs to be as safe as today. A Safety critical

System (ScS) such as ATC is a system that is sensitive with regards to safety and whose failure

could cause severe damage such as loss of lives (Knight, 2002; Sommerville, 2011). Along with

this, there is a fast pace of technological change and the time to market new products has

significantly decreased which entails a lag of engineering techniques coping with the new

technology (Leveson, 2004). Thereby, introducing new technology leads to an uncertainty within

the system to understand all potential risks and behaviors before commercial use. Additionally,

automation is starting to make higher level of decisions, making the integration between the

automated system and the human more important than ever. Accordingly, this creates new types

of system risk which has to be addressed in the different contexts they occur in to avoid accidents.

By conducting a validation and verification on technology advancements one can reassure both for

the stakeholders and for the public that the conceptual ScSs are safe (Asplund, 2014). The system

developments often also consider easing adaption to stakeholders to facilitate an extensive product

introduction. More specifically, validation is defined as “the process by which the fitness-for-

purpose of a new system element or operational concept being developed is established” (MAEVA,

2004). Verification is defined as the approach of adjusting system elements and other details if

faults or defects are detected to make sure that the individual system is built correctly. Moreover,

the terms validation and verification are tools which allow areas such as safety and reliability

among others to be structured and transparent (E-OCVM, 2010).

The ATC is viewed as the System Of Interest (SOI) in this study. The SOI is currently facing the

challenge of combining drones into the civil airspace, an addition that will further complicate an

already complex system (as depicted in Figure 1). A SOI is defined as “a system whose life-cycle

is under construction” (Systems Engineering Handbook, 2006). An implementation of an

enhancement further requires assurance with regards to quality of the services and products

provided in the SOI. The ATC needs to adapt as the drones are introduced, in order for the benefits

of drones to be fully utilized safely (Jiang et al., 2016). A paper published by The European

Aviation Safety Agency (EASA) specifies that introducing drones into existing airspace has to

occur safely and in a proportional manner, which includes congestion management, route planning,

weather and wind avoidance (Jiang et al., 2016). Moreover, quality assurance does further need to

be considered as several system elements operate in a system of systems. System of Systems (SoS)

is defined as an interoperating collection of systems elements that are producing results not

achievable by the individual systems alone. Each SoS involves several system elements with

different life-cycle phases, which results in a variety of technology maturity levels within SoS. A

system element or sub-system is defined as a member of several elements that establishes a system.

(Systems Engineering Handbook, 2006).

Interoperability which is viewed as a SoS creates several challenges as a consequence that each

system element functions individually and has its own life-cycle. One system element might be

being designed while another system element is being deployed. An interoperating SoS are

complex, since more system elements can be continuously added in a non-linearly way.

Incompatible additions could therefore create challenges in the gathering of data from the system

elements. The borders between one system element and another is often unclear if not properly

defined. Figure 1 below demonstrates an example of an airport transport system with its

corresponding system elements; in this depiction the cross system criticality of Global Positioning

System (GPS) to air, land and sea navigation is shown. Thus, while GPS is integral for many

aviation operations it cannot be changed to fit only the needs of the air transport system but must

also consider many other actors and their requirements.

Figure 1. Illustration of how SoS, SOI and system elements can be viewed in the context of ATC

(Systems Engineering Handbook, 2006).

The introduction of UAVs will change the training context for ATCo as it is important to address

changes in the form of increasing objects in the terminal/approach airspace. A complemented

simulation platform would be required in order for ATCo to maintain the required skill levels.

Barzanty (2018) argues that the role of the ATCo will have to be adjusted to monitor the operations

of an automated system regarding failures. Additionally, the current training is mainly based on

performance indicators and could focus more on how attention should be allocated in case of a

malfunction (Barzanty, 2018). Furthermore, automated tools already exists, such as 4D trajectory

management, which coordinates the optimal paths for flights which permits less dependence on

ATCo, in order to use optimal flight paths to the destination (ICAO, 2012).

To capture the complexity of automating elements of an industry involving many stakeholders, a

qualitative analysis was conducted in this project. Based on a conducted pre-study involving

interviewing experts within development of drones and Cyber Physical System (CPS), it was

decided that four standards are of relevance to study the standard E-OCVM, namely ISO 15288

which describes system life-cycle processes, ISO 16085 deals with risk management for life cycle

processes, ARP4754A is a recommended practice with guidelines for development of civil aircraft

and systems and lastly DO-178C which manages software considerations in airborne systems and

equipment certification. They are illustrated in Table 1 below with their abbreviations used from

now on in this report where the A-standards are aimed to complement E-OCVM. E-OCVM is

chosen as a foundation for analysis in the thesis since it is a standard used for managing

developments in ATM contexts and further provides structure and transparency when conducting

validation processes. However, standard E-OCVM is missing specific assessment criteria with

regards to safety and how it can interact with other standards (Scholte et al., 2009). Additionally,

Peter Griffiths (2018) argued that the E-OCVM is lacking in regards to conceptual prototypes, for

example the model needs updating to take consideration of software techniques. Therefore, four

contemporary standards used within system’s engineering contexts have been examined in order

to complement standard E-OCVM to enhance it as a validation tool.

There are various stakeholders involved in developing systems regarding ATC with a variety of

objectives (Schaar and Sherry, 2010). For instance, there are airlines and airports which have a

profound impact on ATC operations as the ATCo manages the communication with the airlines or

aircrafts in a given airspace and is often situated at an airport. Therefore, one has to be conscious

with regards to safety when conducting changes to systems such as providing an extensive

automated system. This thesis will focus on ATC rather than airlines and airports but describe them

whenever distinction between these actors are valuable for the comprehensiveness of the report.

Air Navigation Service Providers (ANSP) which can be viewed as private or public entities

providing air navigation services in a region or country, will also be considered as they are

responsible for the procedures and policies used by the ATCo’s. The relationship between ATCo

and ANSP is that ATM is a service provided by ANSPs in which ATC is a part. ANSPs exist in a

variety of ownership forms, ranging from governmental departments and state-owned companies,

to privately held organizations. This thesis includes interviews with two ANSPs who helped to

frame the problem.

Table 1. A table on the nomenclature used to ease referencing to standards.

Standard Abbreviation in this

report

Description of standard

E-OCVM E-OCVM European Operational Concept

Validation Methodology

ISO 15288 A-SLP System Life-cycle Processes

ISO 16085 A-RLP Risk Management for Life-cycle

Processes

ARP4754A A-DAS Guidelines for Development of Civil

Aircrafts and Systems

DO-178C A-SAS Software Considerations in Airborne

Systems

Commissioner

This master thesis was performed in collaboration with European Performance Management

Systems Committee (EPMSC) based in the UK. EPMSC is a company overseeing interactive

techniques for various types of changes where the aim is to manage risk and assess performance

tools based on the challenges of a changing world. EPMSC was originally contracted for 6 years

to do the European Performance System for Air traffic Management. The company supervisor was

Peter Griffiths who was the chairman of the Performance Review Body of the European Union

from 2010 to 2016 and the former Director of General Civil Aviation UK. EPMSC’s mission is to

automate the aviation industry in areas such as ATC by taking incrementally small steps such as

automating small drones into civil airspace and subsequently larger UAVs into the same airspace.

The final stage of which is to automate large passenger UAVs into ATC. In addition, EPMSC

works closely with different aviation authorities in an iterative process, by sending them prototypes

and receiving feedback. The industry problem also lies in the soft managerial and public factors,

ensuring to the public and stakeholders that the technology is safe for large-scale implementation

(Griffiths, 2018).

1.1 Problematization

Beyond integrating UAVs into civil airspace, there is a complexity involving customer interaction

posing a challenge of introducing new developments into a ScS (Finger et al., 2016). Additionally,

given that the previous SoS is regarded as safe, an issue faced by the new system element combined

into the previous SoS is to preserve the safety levels to allow further operations in a larger scale.

Given that many of the system elements are in a development phase, standards will be of

importance to ensure that the developing system elements are achieving specific safety assessment

and certification requirements. The problem lies in using the right standards that are intended to

give rise to the development of an automated complex system element that is not currently existing.

Additionally, drones can be represented by a wide range of aircraft that vary in size and complexity,

it will thereby also be important that the standards developed are proportional with the type of

environment they will operate in (Sesarju, 2018).

One way to validate systems is to use various standards when it comes to conceptual systems. An

attempt to supplement the validation of conceptual systems can be made by using the standard E-

OCVM, but this standard is lacking as argued by Scholte (2009). Scholte discusses that E-OCVM

restricts validation and the overall interaction with other documents is not covered, specifically it

is mentioned that E-OCVM can not validate diverse and contradicting requirements with various

validation views. Scholte’s approach of improving E-OCVM is by making sure effective

communication is established between developers and validation teams, where important aspects

are operational concept versions of maturity. However, a different approach is to include a

combination of relevant standards that can complement E-OCVM. Therefore, a comparison

between various standards has to be made in order to complete and supplement the standard E-

OCVM. The use of several standards are necessary because according to Maeva (2004) (an earlier

version of standard E-OCVM), no real defined standardized framework for conducting validation

exercise has been made, secondly, the identification of gaps and avoidance of overlaps in the

validation activities conducted by several European projects need improvements, and thirdly, it

lacks promotion of synergy between validation activities conducted at national levels (MAEVA,

2004). Ultimately, a comparison between technology and standard levels is necessary in this ScS.

To summarize, one has to ease adaptation of new products to stakeholders which can be managed

if new systems are validated extensively. Furthermore, validation of systems can be conducted by

using standards and in this case supplement E-OCVM to enhance it as a validation tool. It has

further been mandatory to apply the standard E-OCVM in collaborative ATM R&D projects of the

European Commission and Eurocontrol since 2005 (E-OCVM, 2010). Standard E-OCVM is a

commonly used standard in ATC contexts but is lacking with regards to safety and deployability

with other frameworks (Scholte et al., 2009). Therefore, a comparison between various standards

are to be made in order to complement and enhance E-OCVM as a validation method.

1.2 Purpose & Research Questions

The purpose in this study is to identify what considerations to make when automating complex

system elements involving different stakeholders in a ScS. Furthermore, the aim is also to identify

areas in which E-OCVM can be complemented by using standards since standard E-OCVM is

lacking with regards to how it is used to validate ATC systems.

Given our problem formulation, we have formulated the following research questions (RQ):

Main RQ: How can a conceptualized system be evaluated to ensure that it meets or exceeds the

current system safety performance?

RQ1: What are the primary concerns of stakeholders’ in this specific ScS (ATC) in terms of

merging automated new systems into the existing system?

RQ2: What are the predictions for future system element design according to stakeholders in

regards to a ScS (ATC)?

RQ3: How can a currently mandated standard E-OCVM be supplemented by already available

knowledge about other complex systems?

1.3 Delimitations

The supplement of E-OCVM is not meant to include every detail of the chosen standards but rather

focus on analyzing key terms. A pre-study gave the crucial information to what these key terms

were. The reason for using key terms were to facilitate the analysis of standards with regards to the

limited time frame of the project.

To facilitate the analysis of the standards, four key terms are used namely safety, validation, system

integration and quality assurance. These terms were chosen as they were thought to cover the most

areas of the standards which were chosen based on a conducted pre-study (which will be described

in the method section).

One of the aims with the study is to analyze how standard E-OCVM can be complemented to

enhance its validation of ATC using validation as a foundation. However, the paper is excluding

the verification aspects because of the complexity of verification in terms of involving detailed

system functionalities and therefore considerations are made primarily to the validation

requirements. The project is not focusing on implementing a physical solution to the problem, but

rather focusing on the opportunities and threats an implementation of the new system can create

considering safety.

Finally, the study will focus on terminal/approach areas which more specifically involves areas

around airports since they are considered to be the most congested areas.

The thesis will primarily focus on non-segregated airspace which today consists of controlled

airspace where airliners fly and uncontrolled airspace where UAVs fly which is described in the

following Table 2. However, segregated airspace has been touched upon in certain areas in this

study to add value to the understanding of non-segregated airspace. In addition, the notion of

interoperability used in this thesis refers to UAVs being integrated into controlled airspace.

Table 2. A table demonstrating segregated airspace & non-segregated airspace.

Segregated airspace Non-segregated airspace

Controlled airspace Controlled airspace Uncontrolled airspace

Manned Aircraft & UAV

(Military)

Manned Aircraft UAV

1.4 Expected Contribution

With this research, we aim to contribute to the academic literature by analyzing several concerns

stakeholders have within ATC to obtain a more extensively automated system for ATC. An

automated system for ATC is needed to reduce cost and manage the increased amount of

passengers. Another area of concern is the multiple commercial opportunities provided by UAVs

which go beyond photography and surveillance to possibly operate similarly to a large passenger

aircraft. However, given the drones current size, shape and speed, they pose threats to commercial

aircrafts and are currently flying below the height of commercial aircrafts where both parts lack

sophisticated detect and avoid systems for each other (Cohn et al., 2017). In addition, there is

currently no interoperability where ATC can communicate and track drones which requires

enforcement of rules by aviation authorities (Sesarju, 2018). However, there is a lack of notion on

how these challenges can be faced in both theory and industry, nor how they can be used to create

an opportunity. Moreover, since UAVs is an emerging technology, there is also a lack of standards

which would facilitate obstacle removal in areas such as safety and reliability but also how it can

interoperate with other products and services. Therefore considerations with regards to future

system element design related to the stakeholders opinions are conducted along with their

implications.

1.5 Layout of Thesis

Chapter 2 will present the theories and literature review, including information about the standards

used based on the purpose and research questions of the study. Chapter 3 will describe how the

study has been executed with the choice and purpose of the research design as well as methods for

data gathering. The results of which will subsequently be described in Chapter 4, provides the

information retained from each interviewee, in addition to the information collected from the

standards that will be based on a chosen set of key terms. In the same chapter, the findings from

empirical material is also compared among each other and to the literature review and argued for

with regards to the research questions. Chapter 5 includes the scrutiny of method, discussion and

conclusions by presenting the most important aspects that acknowledge the purpose and research

questions along with interesting topics that potentially could be further research material.

2 Literature Review

To fulfill the purpose of the study, existing theories and literature relating to the context of the

study are addressed and explained in this section.

2.1 Systems Thinking

System thinking, can be defined as the method to which problems are solved through a System

Engineering approach or operational research. For example, deconstructing problems and issues

into simple understandable pieces and then reconstructing the pieces to understand the holistic

problem (Adams et al., 2014). When all aspects of system thinking are specifically assembled

based on a scientific foundation, this is what is known as System Theory. Furthermore, what

Systems Theory implies is that it describes real-world systems. System theory is a collection of

propositions that all have the one common goal, to provide consensus within the systems (Adams

et al., 2014). Systems Engineering is a proper choice to examine the problem of this study since it

gives a framework which allows for the integration of many different actors’ perspectives. For the

ATM to function, each of the actors must be able to work and communicate effectively even though

they each have different perspectives.

Interoperability is a specific term used to provide consensus within systems. More specifically, it

depends on the compatibility of both larger and smaller systems involving different ranges of

complexity to function as a single entity (Systems Engineering Handbook, 2006). Given that many

systems that are existing were built based on a historical preference, components of a technical

system can be rather difficult to replace due to existing barriers such as high transaction costs to

pass on or to create an enhancement of a system (Driscoll, 2014). Therefore, it is often preferred

to complement older system elements with newer which makes interoperability among the

complex system elements important to achieve (Systems Engineering Handbook, 2006). Similarly,

System of Systems (SoS) is defined as an interoperating collection of systems elements that are

producing results not achievable by the individual systems alone. The challenges SoS can create

during development are that the systems have capabilities of being operational without the other

systems, because these can have different life cycles, creating boundaries such as older systems

limiting the overall performance of the SoS (Systems Engineering Handbook, 2006). To put this

in a context, the implementation of including UAVs into the existing operation of manned aircraft,

even further adds to the complexity of the SoS. By adding system elements, the complexity can

increase because of conflicting or missing interface and can further worsen data exchanges across

the SoS. The UAVs giving rise to complexity can be alleviated by providing proof that the system

will operate safely under normal conditions and by using specific validation procedures.

2.2 System Engineering Approach

System Engineering is a combination of several disciplines and it enables the understanding of

successful systems. It takes into consideration both technical and business needs from the client

and stakeholder. The life spans from the concept to the retirement of the system. The System

Engineering disciplines also assist with the collaboration among all parties involved to manage a

modern system (Systems and software engineering - System life cycle processes, 2015).

To shed light upon a detailed strategy in which safety is one of the main priorities, a life cycle

model consisting of different stages is applied to capture the complexity of system development.

More specifically, the life cycle model is comprised of different stages such as presentation of a

concept, development, production, utilization, support and retirement (Systems Engineering

Handbook, 2006). Each step has a certain purpose to fulfill which initially is to identify

stakeholders and their requirements. Then the system is developed while verifying components

and refining system requirements. To further facilitate the development phase, one often prioritizes

the most important stakeholder requirements to obtain a simple prototype and subsequently

consider other requirements when enhancing the product. Depending on how well the development

is carried out, the production of the system is subsequently initiated in which a test and redesign

similarly as in the development phase are conducted. Subsequently, the product is operated in the

utilization stage where there often are product modifications throughout the introduction to

enhance system capabilities. Lastly, there are support and retirement stages with the purpose of

providing maintenance, logistics and other support services to facilitate operation of the product.

Whereas, the retirement stage is focusing on how to provide capabilities of system removal during

the end of the life cycle.

In Figure 2, the V-model is described which aims to holistically illustrate the activities in the

lifecycle stages from a system’s engineering perspective and further highlights the importance of

continuous verification and validation during the different life cycle stages (Systems Engineering

Handbook, 2006). More specifically, it is necessary to act on verifying the system requirements

during the initial stages and simultaneously validating the quality with stakeholders to assess risks

and opportunities. The V-model can be further viewed from a horizontal and a vertical perspective.

Iterations made along the horizontal axis describes how far in time and the maturity of the project

while upward iterations involve stakeholders to validate the ongoing activities. On the contrary,

downward vertical iterations activities comprises risk management investigations along with

measures taken to ensure an acceptable finished product. Based on Figure 2 below, the verification

is part of the iterative processes in the system design and implementation while validation

comprises primarily the initial and final stage of the V-model. As described in the delimitation in

section 1.5, the study is primarily excluding the verification aspects and focusing on the validation

requirements.

Figure 2. A figure on the V-model (Asplund, 2014)

2.2.1 Validation

Validation aims to control whether an item or a product has been built to fulfill its purpose which

in the previous figure involves the initial requirement collection and the final checks of whether

the system fulfills the initial requirements (Honour, 2018). The development of a product has in

general many inputs such as suppliers’, stakeholders’ and acquirers’ requirements while

simultaneously balancing with the capabilities of the designers in terms of their preconditions.

Furthermore, this creates a situation where the expected output of the product can show to not

fulfill its intended purpose (Honour, 2018). In addition, to overcome this situation, the products

often have to be redesigned during the product development along with the review of standards

towards the fulfillment of the validation aspects (Systems Engineering Handbook, 2006). The

translation of the stakeholders desires into system requirements is also complex which makes the

process more difficult. Upon the final completion of the product, the product is tested to ensure the

final system performs as the stakeholders desire.

Challenges in validation testing

Beyond managing complex requirements from stakeholders in the process of validating, other

challenges exist such as conducting a complete testing when deploying safe autonomous vehicles

into existing traffic. This challenge is important to address given that interoperability among UAVs

and manned aircraft will be conducted in a similar way. The infeasibility of testing an operation

with a large number of vehicles to ensure safety is due to safety concerns towards the public but

also due to the repetition of the tests to achieve statistical significance (Koopman and Wagner,

2016). Moreover, in the context of conducting validations on aircraft, the environment of

simulation is important to consider to imitate aircraft performance since there is an infeasibility

with regards to costs of conducting an aircraft validation (Aerospace Recommended Practice: SAE

Aerospace ARP4754A, 2010). Beglerovic et al. (2018) argue that testing and proving on public

ground can be very expensive, time consuming and hard to reproduce. On the contrary, simulations

offer high reproducibility with regards to effort but the challenge lies in selecting proper scenarios

along with parameter variations which will cover a set of variations sufficient to properly model

the system in question (to a reasonable degree). In addition, the difficulties further lies in not being

able to perform a full factorial testing comprising 10x tests which contributes to a lack of certainty

about the assumptions one can make in a real scenario. Koopman and Wagner (2016) argue that

based on the impracticality of deploying vehicles managing all scenarios, a change to the current

developer practices has to be made. A suggestion according to the same source is to use a phased

development which entails using a method whereas few scenarios as possible are tested in a

simulation before combining various scenarios more extensively.

Other existing challenges in the context of autonomous vehicles testing is the shift of human

intervention such as lack of control input. These situations where an ability to take corrective

measures is limited requires a more advanced back-up in the autonomous system. In addition, this

adds significant complexity to deal with all of the possible scenarios. Koopman and Wagner (2016)

argue that regardless of these challenges in an autonomous vehicle, a common denominator is to

detect when functions are not working properly and accordingly, this is viewed as an important

first step to bring the system to a safe state.

Validating Existing and Conceptual ATC Systems

According to MAEVA (2004), validation within ATM context, is defined as “the process through

which an ATM concept goes during its life cycle in order to ensure that it addresses the ATM

problem for which it was designed and that it achieves its stated aims” (MAEVA, 2004). Validating

from the existing to conceptual ATC as SOI, apart from fulfilling its initial aim, the system requires

to meet what the standard requires and thereby complete the validation exercises that exist within

this standard. Validation of conceptual tools can follow similar approach as conducted by the two

the examples below:

1) In the paper Validation of the OPTAIN-SA tool for Continuous Descent Operations by

Lorenzo et al. (2018), they perform a validation exercise on a new ATM tool called

OPTAIN-SA, the tool assists ATCo with their everyday work spreading the usage of an

operation of descending, which helps aircraft descend in a particular fashion for fuel saving.

The validation exercises they performed was firstly, a fast and real time simulation, using

only the OPTAIN-SA tool. Secondly, it was conducted in a real time flight demonstration,

through Barcelona area control center (ACC) to Palma terminal control area (TMA).

Thirdly, comparing data from vertical and longitudinal separation based on both

surveillance data collection (old way) and using OPTAIN-SA data analysis (new way).

(Lorenzo et al., 2018).

2) According to Manfredi (2018), they validate their Collision Avoidance System (CAS) by

dividing it into three categories, safety, operational and acceptability metrics. In this case

the safety metrics (e.g risk ratio comparing safety with and without the tool) measure the

capacity of CAS to prevent near midair collision. The operation metrics measure the

disturbance of the avoidance of aircraft movement in an airspace filled with different actors

including ATC. Acceptability metrics is a measurement that demonstrates how the fidelity

is rated of the remote pilot to the system. The three different metrics represent how much

a remote pilot can trust the system element to represent a real pilot. (Manfredi et al., 2018)

Beyond validation and its usage in conceptual systems, verification aims to confirm system

requirements in detail with regards to system elements which shows that the system has been built

right (Systems Engineering Handbook, 2006). In contrast, validation aims to answer if the system

is fulfilling its intended purpose after the product has been built. Verification is similarly as

validation further used as a process in the V-model where the process confirms whether all the

elements in a SOI perform their intended functions and meet their performance requirements.

Given that both validation and verification are a necessity in system development, they give rise

to different issues in terms of perceived risks, safety and criticality of the element under

consideration. Accordingly, verification has been excluded from the scope but is however

described whenever distinction between the two terms is valuable for the comprehensiveness of

the report.

2.3 Safety Critical System

Safety and risks are terms used in several different contexts but the definition also varies in relation

to the context they are used in. For example in economics, risks can have positive aspects whereas

in the context of aviation risk is often connected to unwanted outcomes from hazardous events.

Furthermore, one general definition of risk is “the probability for an unwanted event to potentially

cause harm” (Westergård, 2016). Raussand (2011) argues that safety is “a state where the risk has

been reduced to a level that is as low as reasonably practicable and where the remaining risk is

generally accepted”. Furthermore, within the context of aviation, ICAO has a similar definition

which is “the state in which the possibility of harm to persons or of property damage is reduced to,

and maintained at or below, an acceptable level through a continuing process of hazard

identification and safety risk management” (Safety Management Manual, 2018). The relation

between safety and risks is further used in systems engineering through identification of risks

inherent in a design in which risk mitigation measures are suggested as the design progresses.

During the design process, hazards are usually tracked and identified so a decision can be taken

with decision makers to continue the process if the hazards are below a specified level (System

Safety Engineering, 2018).

Leveson (2004) argues that many of the flaws with regards to safety in systems are due to

dysfunctional interactions among system components rather than failure of an individual

component. An example is the loss of the spacecraft Mars Polar Lander and the rocket Ariane 5

which fully satisfied their individual system requirements but lacked the understanding of the

components behaviors on the system as a whole (Leveson, 2004). The same source also argues that

the intention of including a redundancy to protect against individual component errors should

involve careful consideration as how these affect the whole system with regards to system risk

since it can even exacerbate the situation by adding complexity. Rasmussen (1997) argues that the

role of humans in accidents will depend on the contexts in which human action takes place, and

that the context will dictate what is the most effective approach to maintain safety.

2.3.1 Maintain a Safety Critical System Safe

The simplest way when introducing safety in regards to aviation, is to maintain the system element

as it is, or better to not even lift the plane from the ground. This way of reaching the safety goal is

not feasible because then no aircraft would ever be utilized. Regulations or standards seeks to

change behaviors of being too safe which can hinder deployment in order to produce a desired

outcome which in this case is to fly the aircraft as safely as possible (Coglianese, 2012). However,

with the usage of safety standards it is important to emphasize the contexts that they are used in

since a consideration used in one specific standard can violate the attempts of another (Asplund,

2014). Furthermore it can also be complex to measure the effects due to the involvement of a

complex chain of interactions, interventions and impacts. Asplund (2014) further argues that

standards should be viewed as best practices to provide high level and infrequent feedback rather

than precise measures with specific assessments.

2.4 Automation, UAV and ATM

Automation is defined in relation to a technology where a process is executed without human

interaction (Grover M.P., 2010). Asplund (2014) defines automation as “the automatically

controlled operation of an apparatus, a process, or a system by mechanical or electronic devices

that take the place of human organs of observation, decision, and effort”. More specifically,

automation is using control systems in a variety of applications, removing human labouring (in this

context, activities that are either standardized or demanding for the body) by the use of previous

collected data (Rifkin J., 1995). Automation is used for repetitive tasks and exist in a variety of

different sectors such as product realization and manufacturing. It has been a way for the industry

to meet the competition with the low income countries in the repetitive tasks like in China and

India (Frohm et al., 2008). The term has grown to involve high degree of cognitive level which has

led to a change when designing automation products like how the machine will cooperate with the

human. (Frohm et al., 2008)

One can identify a “mental model” described in Asplunds doctoral thesis (2014) in which the

fidelity is endured by the constraints during the development process. Regarding the context of

automation, trust from stakeholders can occur in a similar fashion, treating the integration of the

process by each level of automation. This bottom-up approach puts the standards constraints from

the initial levels of automation, which will help create trust from stakeholder from initiation.

Furthermore one way to view automation is through Cyber Physical Systems (CPS) according to

Asplund (2014), in which he defines it as “an integration of computational and physical processes,

distinguished from traditional embedded systems by the new emphasis on networking

computational entities”. In other words, CPS is the combination of physical and computational

processes. One has to bare in mind that lack of support during automation could lead to software

defects through too much reliance on automation in the context of unfinished or wrongly used

automation tool (Asplund, 2014). An example of a system element that is heavily linked to

automation is UAVs.

UAVs and ATM are two extensive fields and are therefore divided into the following two

subsections to facilitate the understanding of these areas.

2.4.1 UAV

UAVs have been increasing in numbers and are rapidly entering the markets across many nations

and continents. The drones’ commercial success is based on advancements in several different

areas such as infrastructure maintenance, aerial photography and agriculture management

(Futurism, n.d.). A common denominator for these areas is the capabilities of drones to aid people

in quickly assessing information when for instance monitoring or inspecting an infrastructure

without physical presence (Rao et al., 2016). More specifically, they have the capabilities to carry

transmitters, multiple sensors and imaging equipment. Furthermore, drones can rely on several

sophisticated technologies many of which are still under development such as detect-avoid

systems, increased battery performance to fly longer distances and identification of their location

where GPS signals are limited (Cohn et al., 2017). Within the area of logistics and distribution, a

drone’s application is being explored as it has the potential to more efficiently deliver packages to

people with less direct (and expensive) human input. However, beyond the benefits of the drones,

given their size, shape and speed, they can pose threats to aircrafts and there is a need to address

the security aspects of the drones before integrating them into airspace. Although, UAVs are new

with regards to integration into civil airspace, it has been successfully used in the military in a

separate airspace for many years due to its capabilities. For instance, they have been used since

1930’s for target practice during military operations as well as subsequently functioning as

surveillance during the Vietnam war (DeGarmo, 2004). But during these times the drones were

limited to relatively basic maneuvers and only operated in designated airspaces at predetermined

times; thus communication and ATM integration was not needed.

A major advantage and interest of using UAVs over large regular aircrafts is that it could save the

air transportation industry 35 billion dollars each year, and additionally cut passenger ticket price

by 10% without the human pilots onboard (Josephs, 2017; Collison, 2017). According to Jiang et

al. (2016), all flights are scheduled to avoid violation proximity in the airspace to avoid collisions

en route, which helps to reduce the workload of the ATC. Jiang et al. (2016) argue that the main

key driver for increasing the capacity of airspace is to reduce the workload of ATC. An Unmanned

aircraft Traffic Management (UTM) system element will help with reducing workload of the ATC.

The basic and paramount ideas from regular large-scale ATC will be the same, with differences

needed for UAVs, such as the method in control, the function and the operational constraint (Jiang

et al., 2016). The interoperability of UAVs with normal aircrafts would require that the terminal

airspace also is considered, by implementing a similar UTM system element.

2.4.2 ATM, ATC and ATCo

ATM comprises several areas such as Air Traffic control (ATC), Air Traffic Flow Management

(ATFM) and Aeronautical Information Services (AIS) (Eurocontrol, n.d.). ATC is a functioning

element which manages the intermediary connection between an airport and pilots. Specifically,

they provide active support to pilots to ensure aircrafts are safely separated in the sky as well as on

the ground. ATFM manages the activity conducted before a flight takes place which comprises

sending a flight plan to a central repository where it is analyzed. The notion is to not allow too

many flights at once within certain parts of airspace and to reduce the Air Traffic Controllers

(ATCo) workload (Eurocontrol, n.d.). Given the flight plan, the ATFM can compute where an

aircraft will be at any given moment so controllers safely can cope with the flight. However, this

is based on a plan and changes are often made during the flight by ATC due to for instance weather

conditions, separation requirements, and other delays (Deener, 2017).

AIS is responsible for the collection and dissemination of aeronautical information that is crucial

for users of the airspace. Information such as safety, navigation, technical and administration such

as legal questions (Eurocontrol, n.d). According to the same source the primary task of an ATCo

is to make sure that the airborne aircraft avoid collision and manage the flow of traffic in their

sector. Each physical ATC tower consist of one or several ATCo (Granberg, 2016). The airspace

is further divided into a grid, that consist of several small sectors, and each ATCo is responsible

for their own sector with an arbitrary (and changing) number of aircraft. The ATCo gives

instructions to the existing aircraft that are flying in the controllers airspace, and their instructions

are based on the feedback provided by the flight plan, surveillance sensors and by the feedback

that is received by the pilot of the aircrafts (Granberg, 2016).

There are two types of unsegregated airspaces, the controlled and uncontrolled airspace (see Figure

3). Figure 3 emphasizes protection of Instrument Flight Rules (IFR) which addresses that flights

outside the designated boundaries is not safe, therefore flying near the margins is not permitted

(US Department of Aviation, 2012). The same figure also shows controlled airspace in grey and

uncontrolled airspace in white. The two above quadrants demonstrates a side view of aircraft

landing (red arrow) and taking-off (purple arrow). The top left quadrant shows the correct way,

and top right the incorrect way. The top right quadrant is not permitted, shown in two dotted circles,

because the aircraft flies marginally close to uncontrolled airspace which is prohibited according

to IFR. The bottom left quadrant demonstrates the permitted path and the bottom right quadrant

shows the unpermitted path, in both cases the aircraft is on the ground. The dotted circles in the

bottom right quadrant demonstrates an IFR violation.

Figure 3. An illustration of controlled and uncontrolled airspace where the grey shaded area is

controlled airspace and the white is uncontrolled airspace in proximity to an airport (Eurocontrol,

nd).

ATC and ATFM have been the solution for solving the congestion problems, but skies are getting

more crowded (Honeywell, 2018). Vaaben et al. (2015) states that in 2010 24% of all flights in

Europe and 18% of all flights in the US were delayed more than 15 minutes and thus experienced

a disruption, this aggravates the congestion in major airports (EUROCONTROL Performance

Review Commission & FAA Air Traffic Organization System Operations Services, 2010). This

was due to technical issues, weather, crew absence and congestion problems. With the introduction

of innovative vehicles such as UAVs, an increase in demand for the controlled airspace will be

created and the integration of UAVs will be further compounded due to the need for traditional

ATC and infrastructure (Mueller E., Kopardekar P., 2017). More demand of airspace puts an extra

burden on the ATCo, assuming that the current (and older) navigation and communication systems

are still being used.

Some proposed solutions are that ANSPs can take extra charge for infrastructure use at rush hours,

when congestion occurs. A differentiation in cost for different volumes, meaning that for rush

hours, airliners would pay more than when it is not rush hour (Granberg et al., 2016). By simply

adding more ATC towers one would dramatically increase the cost which in general is aimed to be

decreased. Another proposed solution is introducing remote piloted towers, or so called Remote

Operated Towers (ROT) concept where each center contains several remote tower modules, and is

controlled by one ATCo. The Remote Tower Centre is a favorable implementation with current

systems elements used within ATC, as it is cost effective and is cheaper to maintain, according to

Granberg et al. (2016). Granberg et al. (2016) argue that the ROT concept will lower the cost of

ATCo on duty, by splitting the time on duty between airports. One problem with Granbergs study

is that simulations have not yet been conducted. Granberg’s model might be useful as an assistant

tool between smaller airports. At a larger airport, where the ATCo is working full time providing

service and managing aircraft, the ATCo will presumably not have time to remotely direct other

external aircraft at another airport.

The implementation of UAVs will pose a challenge when it comes to the training of ATCo, due to

the increase of objects in the terminal airspace. Today basic training of ATCo comprises of a basic

theoretical training that is fundamental in order to work as an ATCo. This is followed by a

simulation training to assist and mimic the work of ATCo and to develop the necessary skills in

the basic training of ATCo which takes up to 16 weeks (Skyguide Solutions, 2017).

Given that the current airspace is getting more congested as passenger numbers are rising, the

emergence of commercial UAV market further poses challenges to the aviation system. De Garmo

(2004) argues that to integrate UAVs into civil airspace, they will have to interact with various

systems of systems (SoS) such as having transponders and positioning reporting devices to address

the safety issues towards manned aircraft. More specifically, beyond having positioning reporting

devices etc. to work effectively in conformance with ATC, it is needed to have modifications in

the current existing manned ATC and aircrafts due to the capabilities of the drones.

In the military, drones are not normally allowed to enter civil airspace, in order to do so a special

authorization is required according to ICAO (article 3, 1944). This is further repeated in article 8

ICAO (1944) which implies that pilotless military aircraft need special authorisation in the

operation on civil airspace as well. In relation to the articles in ICAO, Bernauw (2015) argues that

a pilot less aircraft would qualify as an aircraft since many of the capabilities in a drone are not

fundamentally different from those in manned aviation.

Eurocontrol is currently testing integration of drones into controlled airspace in which several

challenges have been highlighted such as a delay in radio message transmissions between the

remote pilot and the UAVs (Domecq and Guillermet, 2018). The time lags further affect the

transmission between remote pilot simulator and the ATCo. According to the same source, given

the size and speed of the drones, they are significantly more impacted than civil aircrafts with

regards to strong winds which sometimes can lead them to a complete stop (relative to the ground).

Another Eurocontrol project is the SWIM concept (System Wide Information Management) which

aims to provide real time information sharing between actors such as airline operations center,

airport, ANSP and vehicles at the airport (SESAR SWIM Factsheet, 2016). Previously, the

information received from similar stakeholders were less organized and inflexible which with the

increase in capacity demand, attention to environmental pressure and overall economic impact puts

pressure on seamless information exchange and access. DeGarmo (2004) argues that UAVs will

need reliant and accurate information for navigational guidance, thrust control and flight path

optimization which ideally is to be aligned with the data being processed, distributed and

communicated by ATC for manned flight. Furthermore, DeGarmo evaluates the possibility of

drones being sufficiently integrated with manned aircraft through SWIM since the attribute of

SWIM is to enable common data standards and a dynamic data exchange. Similarly, Peña et al.

(2008) argue that an implementation of SWIM would facilitate for the integration of UAVs in

ATM, which is the network centric concept provided by SWIM which potentially could facilitate

accurate drone data acquirement. Furthermore, Peña et al. (2008) further argue the possibilities of

drones acquiring information from areas with a higher uncertainty with regards to weather

conditions which can eventually in areas close to an airport help to enhance weather information

during for instance bad weather.

Air Traffic Management Reaction to Outside Forces, 4D Trajectory.

The 4D trajectory is according to the International Civil Aviation Organization (ICAO) (2012), a

four-dimensional or business trajectory that is being created by Single European Sky ATM

Research (SESAR). ANSPs and ATCo are coordinating with airspace users the optimum trajectory

for the flight taking place, in four dimensions, meaning space (3D) and time, from the day the

planning of the flight commences to the day the flights takes place. The 4D-trajectory takes into

account airport capacity and possible airspace constraints (ICAO, 2012). 4D trajectory reduces

delays on ground and in the air (Iovanella et al., 2011). Predicting key performances areas will

depend on 4D trajectory, such as minimization of departure variability, arrival punctuality and

flight duration. Critics against 4D trajectory are that 4D-trajectory needs to be implemented all

over Europe, otherwise variation of aircraft utilizing and not using 4D-trajectory will emerge.

Thus, an interoperating environment will prompt interruptions and delay all other aircrafts, as a

result of the difficulties of 4D trajectory in an interoperational context because of the volatile delay

times of worst equipped aircrafts (or non-4D trajectory users) (Iovanella et al., 2011).

As more technologies are successfully challenging this standardized industry, more disruptive

technology will be developed such as automated passenger UAVs. The belief of the ‘International

Federation of Air Traffic Controllers’ Associations’ (IFATCA) (Baumgartner, 2017) is that the

second technology revolution is emerging and a push for restructuring of ATM. Outside forces,

from Google, Amazon, Facebook, Apple (GAFA), Microsoft, NASA and other major players in

the telecommunication industry are in the process with experimentation of autonomous solutions

for UAVs. The standardized solution and the operational processes have the possibility to

transform the ATM, and even replace the current ATM entirely (Baumgartner, 2017). According

to Baumgartner’s article (2017), he argues that it is difficult to predict what the future changes will

look like. As some European controllers have grasped the concepts of virtualization and cloud-

based services, some core activities are estimated to be outsourced like flight data processing.

Future challenges will be imagining future problems, and our own scoped thinking will limit the

thinking processes according to Baumgartner. A few existing examples of disruptive technology

in ATM are ROTs and cloud-based services. Cloud-based services are methods for providing air

traffic control services through regular and standardized platforms from a virtual independent

location environment, using principles of shared allocation of computing processing power,

storage and services (Baumgartner, 2017).

In the paper by HALA! (2010), they highlight one key element that the goal of automation in ATM

is not to replace humans but to improve the overall system performance. It should not be human

versus machines, see Table 3, but automation should be seen as human-machine coordination as a

team. The expected benefits of an incremental level of automation are an increase in efficiency

regarding ATM functions, to handle growing traffic demand. The continued advancement of

information and communication technology has forced the development of automation in control

system and ATM. A continuing issue regarding automation is the function allocation, such as

whether the machine or the human is better at performing a task in a safe and efficient manner.

(HALA!, 2010)

Table 3. An illustration of different approaches to automation adapted from HALA! ( 2010).

Automation is About...

Human vs Machine

(Replacement)

Human-Machine Coordination

(Team)

The existing airspace is separated into sectors and a ATCo is responsible for its own airspace

sector, with a certain dimension. In each sector, the ATCo has a limit of the number of aircraft for

which can be managed. When traffic escalates, then current methods of handling high density

traffic (by increasing the amount of ATC sectors, thus decreasing the sector dimension) becomes

infeasible to cope with the increased air traffic. Additionally, there is an inability for the airports

to expand due to new requirements in regards to economical, environmental and safety issues.

Considering the European ATM system, the airports are regarded as the biggest bottlenecks in

relation to capacity and flow of traffic. Despite the bottleneck problem in European airspace, it is

one of the busiest in the world with over 33 000 flights on busy days. (HALA!, 2010) A way to

solve this is to increase the automation within SoS. “An advanced level of automation for different

ATM functions is required for a more efficient system to cope with a growing traffic demand”

(HALA!, 2010). An incremental approach of automation in the SOI is required for implementing

an automation process and Table 4 below demonstrates how different automations levels could be

portrayed. Table 4 below demonstrates a model in a 10-point scale, originally created by

Parasuraman (2000), where higher levels represents higher automation of computer over human

action. For example, level 2 provides several options for the human to make a decision and the

computer is not allowed to execute anything. At level 4, the computer provides one alternative that

the human can decide to execute or not. At level 6, the computer provides the human a limited time

for a veto before continuing its decision. (Parasuraman et al., 2000); (HALA!, 2010)

Table 4. An illustration of the levels of automation adapted from HALA! (2010).

Level 10. The computer decides everything and acts autonomously, ignoring the human

Level 9. Informs the human only if the computer decides to

Level 8. Informs the human only if asked

Level 7. Executes automatically, then necessarily informs the human

Level 6. Allows the human a restricted time to veto before automatic execution

Level 5. Executes the suggestion if the human approves

Level 4. Suggests one alternative

Level 3. Narrows the selection down to a few

Level 2. The computer offers a complete set of decision/action alternatives

Level 1. The computer offers no assistance: the human must take all decisions and actions

2.5 Standards

In general, when a new complex system element is implemented, there have to be measures in

order for it to be considered safe by insuring that these complicated systems are all managed in a

uniform manner. This is done by making sure that all systems follow a set of rules in detail, which

are specifically described in standards (Coglianese, 2012). The standards are also important given

that new system elements often are a part of a wider entity such as a SoS, creating challenges such

as the system elements having different life cycles. By using standards, the challenges can easier

be encountered making it one of the keys to obtain interoperability (Systems Engineering

Handbook, 2006).

E-OCVM is a standard used for managing developments in ATM contexts and further provides

structure and transparency when conducting validation processes. However, since the standard is

missing extensive safety considerations as well as how it could interact with other standards with

regards to different validation perspectives (Scholte et al., 2009), a number of standards have been

chosen to complement E-OCVM which are more specifically described in the following sections:

Based on a pre-study conducted in the initial phase of the project, it was decided that the best

representation was to use standards comprising the Systems Engineering (SE) criteria but also the

context of aircrafts. More specifically, all standards are touching upon SE, where A-SLP and A-

RLP are general in terms of not specifying a particular context, whereas A-DAS includes the

context of aircraft functions and A-SAS focuses on software considerations.

In the following sections, E-OCVM is described along with descriptions of the four other A-

standards:

2.5.1 E-OCVM - European Operational Concept Validation Methodology: E-OCVM

Version 3.0 Volume I

E-OCVM provides transparency and structure in developing ATM, also assessing progress from

early phases of development towards implementation. The objective with the framework is to

obtain a coherent approach and facilitate comparisons across validation projects and activities

while giving freedom to specify practical planning and execution of individual projects. Since 2005

it is mandatory to apply the E-OCVM in collaborative ATM R&D projects of Eurocontrol and the

European Commission. (E-OCVM, 2010)

Validation with E-OCVM is concerned both with the identification of the operational needs of the

ATM stakeholders and the establishment of appropriate solutions (the operational concept). It

follows an iterative process to ensure that the needs are properly understood, the solution is well

adapted (the right system is being developed) and adequate supporting evidence has been gathered

(E-OCVM, 2010).

2.5.2 A-SLP - Systems and Software Engineering - System Life-cycle Processes:

ISO/IEC/IEEE 15288

Standard A-SLP is a standard that comprises the increasing complexity of man-made systems

which has given new opportunities for enterprises that develop and use systems but also their

respective challenges. The standard specifically describes the challenges that exist in all aspects in

the life-cycle process of a System Engineering process (Systems and software engineering - System

life cycle processes, 2015). This standard was created with the intention to provide a mutual

framework of a system within different life-cycles, embracing a System Engineering approach.

Furthermore, also to provide a simplification in communication between stakeholders. The

limitations of A-SLP are, firstly, the standard does not emphasize a specific system or technique.

The method is not defined in this standard and the users of the standard are responsible for the

method tailored to what is going to be reviewed (Systems and software engineering - System life

cycle processes, 2015).

2.5.3 A-RLP - Systems and Software engineering - Life-Cycle Processes - Risk

Management: ISO/IEC 16085

Standard A-RLP aims to provide stakeholders such as suppliers, developers and managers with a

continuous process when managing risk (Systems and software engineering — Life cycle

processes — Risk management, 2006). More specifically, the standard helps to define a process

for risk management throughout the life-cycle of the product. However, detailed risk management

measures and techniques have been excluded to instead emphasize process initiation and

sustainment (Systems and software engineering — Life cycle processes — Risk management,

2006).

2.5.4 A-DAS - Aerospace Recommended Practice: SAE Aerospace ARP4754A

Standard A-DAS describes the development of aircraft systems also taking into account aircraft

functions and operating environments (Aerospace Recommended Practice: SAE Aerospace

ARP4754A, 2010). The standard further addresses the development cycle for aircraft and systems

that implement aircraft functions. However, the standard does not cover the electronic hardware

development nor specific coverage of detailed software or safety assessment processes (Aerospace

Recommended Practice: SAE Aerospace ARP4754A, 2010). The purpose of the standard is to

direct and complement system elements which support aircraft-level functions with the potential

to influence the safety of the aircraft. Furthermore, vast amounts of the elements are developed by

groups, organizations and individuals which requires structured development and discipline to

ensure operational requirements and safety can be obtained and sustained.

2.5.5 A-SAS - Software Considerations in Airborne System and Equipment Certification:

RTCA DO-178C

Standard A-SAS was developed after the rise of software use in aviation systems and equipment.

Furthermore, the standard’s purpose is to provide a guidance when developing software for

aviation systems and equipment in regards to software life-cycles and how to reach the objectives

for those life-cycles (Software Considerations in Airborne System and Equipment Certification:

RTCA DO-178C, 2011). A system life-cycle process requirements of the system are obtained from

the operational needs, specifically from the safety related aspects. The safety assessment process

is what determines and maps the failure conditions. The whole reason for why these safety-related

requirements are made are to make sure that the system is immune to the defined failure conditions.

The requirements are both in software and hardware and exist to remove, detect and avoid fault.

These system conditions are functional and operational requirements, interface requirements,

safety-related requirements, security requirements, maintenance requirements and certification

requirements. The failure conditions are categorized into five different categories according to the

standard A-SAS (Software Considerations in Airborne System and Equipment Certification:

RTCA DO-178C, 2011), namely ‘Catastrophic’, ‘Hazardous’, ‘Major’, ‘Minor’ and ‘No Safety

Effect’.

The standards used in this project touches upon the safety theme in different ways. Based on an

analysis of the standards (which is described more profoundly in method), the standards A-DAS

and A-SAS includes extensive content with regards to safety in comparison with standards A-SLP

and A-RLP. For instance, A-SAS considers safety in the context of software used in airborne

systems where an emphasis is put on providing a guidance on how to create activities for safety

assessment rather than a set of activities for safety procedures. Whereas, standard A-DAS provides

a safety program plan which includes examples on how to create specific safety activities besides

defining the scope of a safety program plan. However, there are similarities between these two

standards when it comes to the definition of failure conditions. A-SAS mentions that a software

error might be latent and can therefore not immediately create a failure condition as well as

describing that the sequence of events that leads from a software error to a failure condition can be

complex.

3 Method

The purpose of the study is to identify what considerations to make when automating complex

system elements involving different stakeholders in a ScS. To capture the complexity of

interoperability, a qualitative analysis using interviews provides a deeper understanding of the area

while concurrently preserving ambiguity (Blomkvist and Hallin, 2015). The many actors within

the aviation system is problematic when conducting an analysis, as many areas require significant

technical depth that cannot easily be obtained without substantial operational experience.

Therefore, this thesis uses a case study approach in order to elicit the required details and weave

together a coherent picture of the requirements.

3.1.1 Choice of Research Design & Pre Study

Given the complexity of automatization influencing several different stakeholders along with

managing interoperability, a case study was chosen. Yin (2003) defines a case study as “an

empirical inquiry that investigates a contemporary phenomenon in depth and within its real-life

context, especially when the boundaries between phenomenon and context are not evident”. The

case study helps to provide an opportunity to obtain an in-depth understanding among the

stakeholders’ views. This will help to beyond understand, also assess the importance of

stakeholders’ requirements and their interrelation among each other when understanding how

future systems can be designed while preserving safety along with interoperability.

A pre-study was conducted to gather information about general important aspects within the project

frame. Moreover, the three initial interviews were conducted with experts within CPS, UAVs and

a safety regulator, see Table 7 in section 3.1.3. In the interviews obtainment of information with

regards to several areas such as the preconditions about ATC have been conducted. Moreover, the

other interviews yielded specific information about considerations one has to make with regards

to validation and safety. As described in the literature study, validation of conceptual tools could

follow a similar approach as the examples from Lorenzo et al. (2018) and Manfredi et al. (2018),

which inspired a foundation for supplementing E-OCVM.

The pre-study resulted in a decision to focus on two general standards A-SLP and A-RLP along

with the two standards A-DAS and A-SAS focusing on the contexts of aircrafts and software

development. A-SLP and A-RLP were chosen since they are two general standards that comprise

system life-cycle processes and risk management for life cycle processes. Thus, reflects the ATC

since it can be viewed as a ScS containing various system elements with different life-cycles. On

the contrary, A -DAS and A-SAS were selected since they have a focus on specific contexts such

as civil aircrafts and software development which was considered to be appropriate based on the

assumption that UAVs have to adapt to the performance of existing systems. Accordingly, a

balance between general guidelines and their practical execution was aimed for (two general

standards and two using a particular context) with the disadvantage of the general ones being

difficult to put in a context while A-DAS and A-SAS are limited to their specific contexts.

Considerations were further made for another standard called ISO/IEC 12207 which is used to

create a framework for software life cycle processes involving a set of processes to facilitate

communication among stakeholders (Ieeexplore, 2018). However, given that the standard states

that there are similarities with A-SLP such as having same process purposes and process outcomes,

it was decided to not include it. Another reason for not using ISO/IEC 12207 was that it argues

that the usage of the standard depends on the SOI, specifically because the standard does not

consider that the SOI in this thesis involves ATC along with the context of civil manned aircrafts.

Additionally, standard A-SAS was chosen instead as it considers software in regards to civil

aircrafts and thereby a specific context in comparison with ISO/IEC 12207.

The ‘ScS’ discussion materialized from the pre-study interview with the first interviewee, Fredrik

Asplund, a PostDoc active in the field of Cyber-Physical Systems safety. According to Asplund,

standards define the best practice required to show that proper care has been adhered to in a legal

sense. In relation to this, problems with standards according to Paul Kennedy (safety regulator at

IAA) is that they do not change unless there is an accident.

Another area that emerged during the pre-study was validation in which Bengt-Göran Sundqvist

(an aeronautical engineer in detect and avoid systems at Saab) described can be achieved through

simulations. He further argued that “When you test a detect and avoid system you cannot test direct

collisions or failure analysis of engines, as it is too dangerous which is why you need to do it in

the simulations”. Bengt further mentioned that their MIDCAS project would function in an

interoperational environment in a non-segregated airspace, sharing the same airspace as regular

passenger aircraft.

The pre-study also yielded that a qualitative study was being aimed for. A quantitative study such

as experiments and surveys would instead require current data about several cases from primary

data (e.g. usage of surveys in which results are converted into numbers) or secondary data (data

obtained from various publications, registers and official statistics etc.) according to Blomkvist

and Hallin (2015). More specifically, quantitative data can help to provide a good overview of the

phenomena but since there is a limitation of access to data along with the importance of measuring

all factors if conducted, it was decided to instead use a qualitative case study as the choice of

research design.

The following table describes the methods used to answer the specific research questions. From

the broad overview of the industry and problem assessed in the pre-study, to the more refined and

specific inquiries made with the literature review and expert interviews, each sub research question

was answered which in turn allows for a discussion on the main research question to be conducted

in this section. As an example, the table below can be read as follows, the pre-study and literature

review gave rise to the interview questions (the method) and subsequently lead to the primary and

secondary sources (contributed to) which answered RQ1 etc.

Table 5. Illustration of which methods were used to answer each research question.

Research

Questions

Method Contributed to Solution

RQ1 Pre-study +

Literature

Review

= Interview

Questions

Primary and

secondary source

= Answered RQ1

RQ2 Pre-study +

Literature

Review

= Interview

Questions

Primary and

secondary source

= Answered RQ2

RQ3 Pre-study +

Literature

Review

= Interview

Questions

- Standards

to be used

- Primary

secondary

source

= Comparison &

complement of

standards being

used

= Supplement of

E-OCVM &

Answered RQ3

3.1.2 Literature Study

The main point of the literature review was to build a theoretical framework, that could be applied

to collect empirical data and then be analyzed and evaluated. Empirics were gathered from

interviews, experts, scientific articles, standards using the key terms, company websites, KTHB

database Primo, and recommendations from interviews and highly knowledgeable researchers in

the field. The key terms used for the analysis of the standards, namely safety, validation, system

integration and quality assurance were also used as a foundation for the overall literature review

to facilitate obtainment of patterns within the context of the study.

The literature review had several important key roles throughout the project. The objective was to

increase knowledge in key aspects as well as theories directly related to the project, such as System

Engineering, automation and safety management. The following phase of managing empirics was

more directed towards comparisons, and looking for correlation and causality in areas such as

review of standards.

The method of reviewing research articles were as follows:

1. Review abstract

2. Study introduction

3. Review results and discussion

4. Detailed study of conclusion

From the method of reading research articles, an overview was obtained and gave insights on the

key arguments. From there a decision was made if the information gained from the articles was

relevant enough and would provide value to the thesis, see Table 6 below.

Table 6. A chart on the procedure of analyzing the literature review.

Steps Literature Means Goals:

1 Review of System

Thinking

Experts in the field

& scientific articles

Comprehend difficulties of SoS

2 Review of ScS Experts in the field

& scientific articles

Understand different layers of safety

specifically in the aviation industry

and contemporary systems

3 Review of System

Engineering

Approach

Standards, scientific

articles & websites

Use system engineering structure to

model SOI (ATC)

4 Review UAV and

ATM

Experts in the field,

scientific articles &

websites

Provide a recommendation on how

future system element design is

influenced to identify the

considerations to make when

automating complex system elements

3.1.3 Interviews

This thesis conducted interviews with two ANSPs (among other stakeholders); the Irish Aviation

Authority (IAA) and the Aviation Capacity Resources (ACR) group since both are an influence in

determining what technology advancements will be adopted within the ATM. Additionally,

interviews were conducted with a consultant within civil aviation and several current and former

CEOs where some had previous experience as ATCo. These interviewees were chosen as they

provided the required background information for this thesis to be able to explore what interactions

are needed in order to safety include UAVs into the current aviation system.

The interviews conducted are presented below in Table 7 in which the three initial interviews were

part of the pre-study:

Table 7a. A table on the interviewees and their respective roles.

Nr Name (type of

conversation)

Role & Company Date &

Duration

Country

1

Fredrik Asplund

(Telephone)

PostDoc within the

Safety of Cyber-Physical

systems at Rolls-Royce

16/3-2018

55 min

Sweden & UK

2 Paul Kennedy

(Telephone)

Safety Regulator at Irish

Aviation Authority (IAA)

21/3-2018

45 min

Ireland

3 Bengt-Göran Sundqvist

(Telephone)

Aeronautical Engineer in

Flight Control Systems in

Detect and Avoid

Systems at Saab AB

Chairman of MIDCAS

project in Saab AB

27/3-2018

65 min

Sweden

4 Eric Kroese

(Video call on Skype)

Consultant for Civil

Aviation companies.

Former Chairman & CEO

for Luchtverkeersleiding

(LVNL). The agency in

charge of ATC in the

Netherlands

17/4-2018

50 min

The

Netherlands

Table 7b. Continuation of Table 7a.

Nr Name (type of

conversation)

Role & Company Date &

Duration

Country

5 Marc Baumgartner

(Telephone)

Air Traffic Controller for

Skyguide in Geneva,

Switzerland. Former

President & CEO for the

International Federation

of Air Traffic

Controllers’ Association

(IFATCA).

14/5-2018

40 min

Switzerland

6 Marek Bekier

(Telephone)

Vice president of

Aviation Capacity

Research (ACR) AB

13/6-/2018

25 min

Switzerland

7 Peter Griffiths

(Telephone & Skype)

Director of GTS

Robotics Designated

Activity Company.

Former Chairman of

Performance Review

Body of the European

Union 2010 to 2016.

Former Director of

General Civil Aviation

UK

Continuous

contact

during study

UK

During the pre-study, the interviews were less structured to provide open answers to the interview

questions to exploit the interviewees respective technical backgrounds. For example, besides

Sundqvist being asked what he thinks were the benefits and risk regarding the implementation of

an automated ATC, specific questions were asked regarding Saabs MIDCAS project, which is

Saab’s drone interoperability initiative. Based on the interviewees extensive knowledge, the

purpose of the open style questions was to gain information on what was the most important aspects

according to the interview candidates.

After the pre-study, more detailed questions were asked to all three following interview candidates,

and indirectly adapted based on their expertise. Some interview candidates asked for the interview

question beforehand, specifically Sundqvist and Bekier, which gave them time to formulate their

answer in the best possible manner. Griffiths was a continuous contact during the study rather than

a specific interview candidate but was a part of information acquisition and framing the problem.

A sample of the interview questions are presented in the table below. Given that semi-structured

interviews were conducted, the questions were more or less adapted to the specific interviewee.

The interview questions were created in accordance with the literature review and the method using

inspiration from Yin (2003) and Blomkvist and Hallin (2015).

Table 8a. A table on the interview questions after pre-study.

# Questions

Q1 What are the different segments of ATC training?

What are the most difficult areas to teach and learn in

ATC training?

Q2 What do you see as the major consequences (good or

bad) to the digitalization of ATC?

Q3 What kind of services and opportunities can

digitalization bring?

Q4 One of the challenges that will encounter ATM in the

future is the reliance on automation while maintaining

safety. What key parts of human/system integration and

what types of measures do you feel is needed to ensure

that such a connection is maintained at a high enough

level to ensure safety?

Table 8b. Continuation of Table 8a.

# Questions

Q5 What is your opinion on the conceptual standard E-

OCVM (an updated MAEVA version)?

Q6 What is your ideal system from first day of class until you

have a fully trained controller? What type of advances do

you view as the most needed to maintain safety in highly

congested airspaces?

Q7 Is there anything we have forgotten to ask you that you

feel is important to mention?

Along with a literature review to gather data, the study was supported with interviews to facilitate

the opportunities to discover unexpected dimensions of the phenomena as discussed by Blomkvist

and Hallin (2015). More specifically, the interviews were conducted with employees representing

different stakeholders such as Irish Aviation Authority (IAA), Aviation Capacity Resources (ACR)

AB1, Rolls Royce, Saab and two former CEO in the ATC industry combined with secondary

sources.

To capture the complexity of automation in ATC, the interviews were aimed to be conducted in a

semi-structured manner. In addition, the previously conducted literature review helped to shape

question areas in advance while not being too specific to encourage the interviewee to develop

their trail of thoughts. As described by Collis and Hussey (2014), an open question helps to create

longer and more developed answers to for instance understand the respondent’s point of view on

the matter. Lastly, the interviews were ended with asking the interviewees questions aiming to let

them express if there was anything of relevance they wanted to add to the discussion.

The interview method was based on information from secondary resources and data collected from

primary resources as well as previous research and standards used by stakeholders. The secondary

sources were used to support gathered data and in combination with the pre-study used to form

interview questions for the interviews. Furthermore, according to Blomkvist and Hallin (2015) it

is important to have source criticism, meaning the evaluation of the empirical source reliability and

that the facts and statements are credible.

The disadvantage with interviews is the lack of supporting interview answers with sources. A

counteract is to use triangulation which is to support interview results with secondary sources

1 AB is an abbreviation for “aktiebolag” and is equivalent to a corporation in the UK or US.

(Gibbert, Ruigrok, & Wicki, 2008). This can further help the reliability of the interview results to

increase and also facilitate comparison among the various interview results. Beyond comparing

interview results, triangulation helps to increase the reliability of the research design and further

facilitates a reiteration of the study. In addition, the interviews were also recorded along with notes

to extract a more robust and comprehensive interpretation.

3.1.4 Standards Review

The process of achieving technical and operational procedures which are uniform towards specific

criteria, methods and practices can be accomplished by using standards. Standards can be more

specifically used to ensure safety and is approved by several organisations and stakeholders.

To fully cover the complexity of interoperability also viewed as a SoS to be integrated with newer

system elements of UAVs, standards are used to describe several aspects such as risk management,

development of aircraft systems and life cycle processes which influences a corresponding system.

Additionally, a conceptual validation standard given by Eurocontrol (standard E-OCVM) is used

to compare the four standards A-SLP, A-RLP, A-DAS and A-SAS. The aim is to do cross

comparisons by comparing different sections with each other and thereby address key issues with

the conceptual standard.

The chosen method of analyzing the standards was to first analyze the conceptual standard E-

OCVM to identify gaps which subsequently were to be complemented by the A-standards. On the

contrary, it is preferable to review the A-standards first before identifying possible gaps in the

conceptual standard E-OCVM, but the time limit of the study yielded to go with the former method.

More specifically, relevant areas of the standards were reviewed and further highlighted with

regards to what was initially thought to be the important areas in the table of contents. Furthermore,

this was conducted along with a deeper analysis of the selected areas which was assumed to cover

the time limit of the project.

The method of reviewing the standards were as follows:

1. Review the table of contents and highlight what we think is important.

2. Study the scope and purpose.

3. Review the highlighted areas in the table of contents.

4. Subsequently make a consideration on whether what we thought would be important still

is important or not.

To facilitate the analysis of standards, the following key terms were selected based on the purpose

and research questions proposed in this thesis, see Table 9.

Table 9. A table on the specific key terms used in the project to facilitate review of standards.

Key Terms

“Safety”, “Validation”, “System Integration”, “Quality Assurance”

The review of the table of contents was conducted based upon the relevance to the key terms as

well as identifying other areas that might be interesting depending on the context. As an example,

on several occasions areas in the table contents were selected due to the relevance to system of

systems such as life-cycle phases of systems beyond the key terms. Additionally, other areas in the

table contents that were included was limitations of the specific standards along with their

conformance to other standards and documents. For specific details see section 3.1.5.

In Table 10, a detailed step-by-step guide on how the analysis of the standards were conducted is

presented. In addition, cross comparisons among the A-standards were also carried out based on

the verdicts on each key term.

Table 10a. A chart on the procedure of analyzing the standards.

Steps Standards Means Goals

1 Review of gaps in E-

OCVM

All key terms Identify gaps in E-OCVM

2 Review of A-SLP, A-

RLP, A-DAS, A-

SAS

All key terms Gain a holistic overview of the A-

standards (as well as their purpose

and table of contents)

3 A-SLP, A-RLP, A-

DAS, A-SAS

Specific key term:

Safety

Identify how the A-standards can

complement E-OCVM with regards

to the key term safety

Table 10b, Continuation of Table 10a.

Steps Standards Means Goals

4 A-SLP, A-RLP, A-

DAS, A-SAS

Specific key term:

Validation

Identify how the A-standards can

complement E-OCVM with regards

to the key term validation

5 A-SLP, A-RLP, A-

DAS, A-SAS

Specific key term:

System Integration

Identify how the A-standards can

complement E-OCVM with regards

to the key term system integration

6 A-SLP, A-RLP, A-

DAS, A-SAS

Specific key term:

Quality Assurance

Identify how the A-standards can

complement E-OCVM with regards

to the key term quality assurance

7 Supplement of E-

OCVM

Based on each key

term

Based on the results from the

previous key terms, compile how E-

OCVM can be supplemented to

enhance it as a validation method

3.1.5 Method Process

Table 11 describes an example of how the standard’s table of contents were highlighted in the

conceptual standard E-OCVM. In addition, the yellow2 areas in the table are sections viewed as

more important than others and the green where adjustments had to be made based on a second or

third review of the standards. In this Table 11, only an extract of the relevant sections are brought

up, for a more detailed picture, see Appendix A.

2 If printed in black and white, then the yellow color is the lighter shade and the green color is the

darker shade.

Table 11a. Illustration of how standard E-OCVM’s table of contents were analyzed (this is only

an extract from the original picture, for more detailed information see Appendix A) adapted from

E-OCVM (2010).

Table of Contents

1 Introduction 3

1.1 Scope 3

1.2 Intended Audience 3

1.3 Structure of the E-OCVM Version 3.0 4

2 Role of Operational Concept Validation in

ATM System Development

5

2.1 Operational Concept Validation 5

2.2 Assumptions on the Role of Validation in

ATM System Development

5

2.3 ATM Concept Lifecycle Phases 6

2.5 Managing Process through the Lifecycle 7

3 Organising Validation in Large-Scale

Concept Development

9

4 Risks And Challenges To Validation 10

5 Principles Of The E-OCVM 13

5.7 Balancing “Generic” and “Local”

Assessment

14

Table 11b, continuation of Table 11a.

Table of Contents

6 The E-OCVM - A Process Of Several Parts 15

6.1 Concept Lifecycle Model & Maturity

Assessment

15

7 Documenting The Validation Process 23

7.1 Validation Strategy: Organising the Work

of Validation

23

For the A-standards that were used to complement E-OCVM, a similar procedure was used which

was based on the key terms safety, validation, system integration and quality assurance. In Table

12, the important areas have been highlighted in standard A-SLP and then compiled based on

subsequent reviews. In the same table, only an extract of the relevant sections are brought up, for

a more detailed picture, see Appendix B.

Table 12a. Illustration of how standard A-SLP’s table of contents was analyzed separately by the

two authors on each side of the table (this is only an extract from the original picture, for more

detailed information see Appendix B) adapted from Systems and software engineering - System

life cycle processes (2015).

Table of Contents Table of Contents

Introduction Introduction

1 Overview 1 Overview

2. Conformance 1.1 Scope

5. Key Concepts and application of this

International Standard

1.2 Purpose

5.2 System Concepts 1.3 Field of application

5.4 Life Cycle Concepts 1.4 Limitations

Table 12b, Continuation of Table 12a.

Table of Contents Table of Contents

6 System life cycle processes 2 Conformance

6.2.1 Life cycle model management process 2.1 Intended usage

6.2.5 Quality management process 3 Normative references

4 Terms, definitions and abbreviated terms

4.1 Terms and definitions

4.2 Abbreviated terms

5 Key concepts and application of this

International Standard

5.1 Introduction

5.2 System Concepts

5.4 Life cycle concepts

3.1.6 Theory on Method Criticism

Blomkvist and Hallin (2015) recognized that to keep a high validity, reliability and generalizability

on the research, source criticism is essential. Therefore, the reliability, validity and generalizability

of the sources that will be conducted in this study aims to follow a criteria much similar to the one

presented in the Blomkvist and Hallin’s book. The criteria specifically addresses areas such as

authenticity, proximity, tendency and representativity. A common denominator for these areas are

the importance of the searchability of the sources used, whether the information is up-to-date as

well as the representativity of the material to represent the phenomenon which is under

investigation.

Validity aims to describe the establishment of appropriate measures for the intended concepts of

the study (Creswell, 2009). It further refers to the accuracy of the findings and depending on the

specific research design chosen addresses terms such as credibility, authenticity and

trustworthiness. Yin (2003) uses a similar relation where the terms are further divided into concepts

such as construct validity and external validity. Construct validity refers to the establishment of

correct operational measures for the chosen concept while external validity is attributed to the

generalization of the study’s findings.

Reliability refers to the study’s ability to be reiterated and thus generate similar results as in the

original study. More specifically, it emphasises the consistency of the research approach to

minimize errors and biases in the study. When conducting interviews, a high reliability can be

achieved by having as low ambiguity as possible in the interpretation of the empirical data as well

as focusing on demanding impartiality and mutual respect regarding those who do the

interpretation (Blomkvist and Hallin, 2015).

Generalizability can be assessed in various ways to increase the scientific value of a study but a

common denominator is a systematic approach with regards to the choice of case, analysis method

and data gathering method (Blomkvist & Hallin, 2015). More specifically, it can also be described

as to what extent the study’s findings can be extended to other cases. However, there is discussion

on how the generalizability aspect can be applied to for instance multiple case studies since the

characteristics of the cases can have a considerable variation.

4 Results & Analysis

Given the method described in the previous section, the results and analysis are initially presented

with the standard review using key terms followed by an analysis related to the interview results

using selected themes.

4.1 Standard Review using Key Terms

General

The standard E-OCVM is suggested to support and help prepare different scales of validation

activities. According to the standard, it is implied that the same standard cannot be followed like a

recipe but must instead be intelligently applied to develop an adapted validation process.

Furthermore, the standard is intended to be a part of a larger system development/engineering

process involving requirements management, verification and concept refinement (E-OCVM,

2010).

The following sections identifies potential areas to complement the standard E-OCVM in

enhancing the validation of ATC with regards to the four key terms, namely safety, validation,

system integration and quality assurance. For each key term a description of the content in E-

OCVM is first described before presenting how the A-standards can provide complements.

4.1.1 Safety

Given the key term safety, the conceptual standard E-OCVM is describing a case approach to meet

the priorities, expectations and concerns of the stakeholders. It further involves the critical aspects

regarding safety, environment and human factors to reflect decision-making priorities of the

stakeholders.

The standard specifically describes the importance of considering safety at the beginning of a

concept. One of the reasons to consider safety early in the process is to facilitate error search and

to allow stakeholders to receive information about concept evaluation regarding delivering the

desired level of safety.

The output of a safety case is also emphasized in each R&D case since it describes the potential of

a concept to meet defined safety goals according to the standard. However, if there would be any

concerns about concepts not being safe enough, it is important to clarify the explanation to the

concept developers and decision makers on why it is not safe enough according to the standard.

Based on what is described about safety in the conceptual standard, the following A-standards can

provide complements in the following ways:

● Standard A-DAS has safety details described in which some areas are similar to the one for

A-SAS regarding for instance failure conditions.

● Standard A-SAS considers safety on a general basis besides failure conditions. The only

part it has in common with A-DAS is failure conditions whereas A-DAS also has examples

on safety program plans one can pursue.

● Standard A-SLP uses the key term safety throughout the whole standard. Although, safety

is important in this standard, it does not include how it can be conducted in a specific way

and it further refers to another standard 61508 (see p.56 in standard A-SLP).

● Standard A-RLP is describing safety and risk management on a holistic level but there is

no specific safety measurements or techniques to use in the standard other than that it helps

to create a process in which the organization can manage risk.

According to these results, there are two standards which are being elaborated on further, namely

A-DAS and A-SAS for the key term safety. The standard A-SAS provides information regarding

software errors when it comes to safety and how it leads to a failure condition. More specifically,

the standard can contribute to E-OCVM with the argument that a software error can be latent and

therefore not immediately create a failure condition. Moreover, in a real operation, the sequences

of events leading from a software error to a failure condition may also be complex. The standard

further emphasizes the importance of understanding that the likelihood of a software containing an

error cannot be quantified in the same way as for random hardware failures. On the contrary, A-

DAS gives descriptions on safety assessments including failure conditions but also details on how

to use a safety program plan. There are several activities that can be included within the plan such

as identifying requirements for the specific aircraft system element. This is conducted to ensure

safety design and analysis responsibility for the input requirements. Secondly, another area that

might be covered in the plan is the identification of applicable safety standards as well as describing

the safety activities and deliverables. The level of detail to consider in a safety program plan is

further dependent on the degree of integration and the complexity of a system implementation. The

E-OCVM can further be complemented with the responsibility regarding the safety assessment

often being split among the organizations for each specific process task and is updated throughout

the development program.

4.1.2 Validation

The purpose of validation according to E-OCVM is to make sure that projects fulfill their function

by making sure that all parties have a common understanding of the shared principles and practices.

E-OCVM has a general view on validation from four perspectives, namely concept life cycle

model, maturity assessment and life cycle transition, structured planning framework and case based

approach (for more information review standard E-OCVM). Moreover, the standard has extensive

suggestions on how to document the validation processes.

Based on what is described about validation in the conceptual standard, the following standards

can provide complements in the following ways:

● Standard A-SLP describes validation specifically with regards to their different life cycles.

Moreover, the standard focuses on the translation from stakeholder needs into system

requirements and has a specific assessment plan for it.

● Standard A-DAS describes validation specifically taking into account concepts such as

correctness and completeness regarding the requirements along with a validation plan. The

standard focuses a lot on how to validate the requirements.

● Standard A-RLP is referring to another standard for specific validation activities (namely

IEEE Std 1012-1998).

● Standard A-SAS has no specific validation aspects but rather focuses more on life cycles

for the systems.

According to these results, there are two standards which are being elaborated further namely A-

SLP and A-DAS for the key term validation. A comparison between A-DAS and A-SLP shows

that A-DAS focuses on the requirements by introducing terms such as completeness and

correctness of assumptions along with a validation plan which is lacking in the E-OCVM.

Furthermore, the validation plan includes several methods to support validation such as

traceability, analysis, modeling, testing and similarity checks which are applied in various ways

depending on the development assurance level (DAL). On the contrary, A-SLP only briefly brings

up the identification of constraints to the system and incorporation of them into the system

requirements but lacks further details on how to define the requirements. However, A-SLP has a

previous chapter dedicated to stakeholder requirements which focuses on the translation from

stakeholder needs to definition of requirements (Systems and software engineering - System life

cycle processes, 2015). This part is extensively describing how to translate and define the

requirements from stakeholder needs while A-DAS requires that they already are developed

through the terms completeness and correctness. The terms further entail posing questions on how

to assess them rather than having specific activities on how to define them.

4.1.3 System integration

System integration is corresponding to interoperability in this study. Based on the limited results

when reviewing the standards with regards to the key term “interoperability”, system integration

was chosen as a corresponding key term.

E-OCVM touches upon system integration but there are no specific ways on how to assess them,

however other standards do have such specification and can be used to refine the E-OCVM:

● Standard A-SLP (p.68) provides an extensive process for system integration and how to

prepare as well as manage the results from the integration.

● Standard A-DAS considers system integration and how to replace an item or system with

another on existing aircraft (p.31 and p.84-85).

● Standard A-RLP has few links to system integration.

● Standard A-SAS has several links to system integration. However, the specific areas are

related to verification which is not included in the project scope.

According to these results, there are two standards which are being elaborated further, namely A-

SLP and A-DAS for the key term system integration. Based on the key terms of system integration,

the standard A-SLP describes a holistic plan on how to assess system integration involving areas

such as preparation, performance and management of results. Furthermore, there are considerations

regarding application of system life cycle processes to a system of system (p.102 in A-SLP). In

comparison with A-DAS, the standard emphasizes that there should be specific means to show that

intrasystem requirements have been fulfilled. Furthermore, one has to ensure that all system

elements operate correctly individually and together in the context of an aircraft. To facilitate

systems operations are conducted correctly, identified deficiencies should be referred back to

appropriate development or integral activity such as capture of requirements, implementation or

allocation of validation.

Additionally, an important aspect to take into consideration is the environment of simulation of an

aircraft, because of the nature of performing validation regarding system integration due to the

costs. Therefore, it is preferred to use other cost-effective measures involving simulations and

laboratory work to imitate the on aircraft integration.

A-DAS further states some considerations for modifying aircraft, system elements or items when

introducing a new aircraft level function or replacement of an item or system element with another

on an existing aircraft. A common denominator is that functional hazard assessment should address

failure conditions and hazards for the system elements and identify safety objectives for items and

system elements to be modified etc. This is further used as a basis for the proposed modifications

along with an implementation strategy including considerations for an impact analysis. The

standard further refers to a standard ARP4761 (describes guidelines and methods for performing

safety assessment on civil aircrafts) for a detailed safety assessment process regarding system

development.

4.1.4 Quality Assurance

E-OCVM mentions the key terms quality and assurance of the validation methods only in a few

sentences throughout the whole conceptual standards, which indicates that it can be complemented

by the other A-standards in the following ways:

● A-SLP discusses the processes of quality management and quality assurance.

● In A-DAS, there are areas such as development assurance and process assurance aiming to

enable safety throughout the whole development phase in an aircraft.

● A-SAS discusses activities and objectives of software quality assurance process and further

includes a software quality assurance plan. The objective is to provide confidence that

software life cycle processes conforms to their requirements and that detected deficiencies

are evaluated, tracked and resolved to further conform to certification requirements.

● Standard A-RLP has few links to quality assurance.

According to these results, there are three standards which are elaborated on further, namely A-

SLP, A-DAS and A-SAS for the key term quality assurance. In standard A-SLP, quality assurance

is further defined as the process focused on providing confidence that quality requirements from

both the organization and customers will be fulfilled whereas quality management entails the

coordinated activities to direct and control an organization with regard to quality (Systems and

software engineering - System life cycle processes, 2015). More specifically, the standard

describes tasks to help planning, assessment and performance of quality management, but also

refer to other supplementary standards for detailed information regarding for instance customer

satisfaction and performance improvements.

A-DAS provides guidelines on how to develop requirements in the development process.

Development process is defined as a process which establishes a level of confidence that

development errors that can cause or contribute to failure conditions have been minimized with an

appropriate level of rigor.

The standard further highlights a concern regarding efficiency and coverage of techniques used to

evaluate safety aspects for complex systems elements and interrelated functions such as usage of

electronic and software based techniques. Furthermore, there is a concern with analysis and design

techniques which are traditionally applied to deterministic risks or to non-complex system

elements not being able to adequately and safely cover for more complex system elements. Thus,

the standard highlights that other assurance techniques such as development assurance utilizing a

combination of process assurance and validation may be better suited to these more complex

systems.

A-SAS discusses the objectives within the context of software assurance and the considerations to

make in a quality assurance plan. The standard further puts emphasis on increasing confidence in

the system elements by ensuring detected deficiencies are tracked, evaluated and resolved. A-DAS

focuses on giving guidelines in the development process to avoid creation of development errors

causing or contributing to aircraft failure conditions. The standard further highlights as previously

mentioned that there are concerns with analysis and design techniques traditionally applied to non-

complex systems not being able to safely cover for more complex systems. Specific guidelines

starting from aircraft level through to item level, are assigned in functional development assurance

levels (FDAL) and are given in A-DAS. For item development assurance level (IDAL), namely

for software and electronic hardware items, the objectives for accomplishment are given in

standards DO-178C/ED-12C (standard A-SAS) and DO-254/ED-80 (safety management for

airborne electronic hardware). Moreover, A-DAS further describes activities regarding process

assurance to fulfill regulatory compliance such as project plan reviews and evidence of

conformance with the project plan. In standard A-SLP, quality assurance is further defined as the

process focused on providing confidence that quality requirements from both the organization and

customers will be fulfilled whereas quality management entails the coordinated activities to direct

and control an organization with regard to quality (Systems and software engineering - System life

cycle processes, 2015). More specifically, the standard describes tasks to help planning, assessment

and performance of quality management, but also refers to other supplementary standards for

detailed information regarding for instance customer satisfaction and performance improvements.

Summary on complementary standards

Based on the comparisons among the A-standards, the conceptual standard E-OCVM can be

complemented in the following areas based upon the key terms used during the analysis of the

standards.

Given the key term safety, E-OCVM can be complemented by standards A-SAS and A-DAS in

areas such as highlighting the importance of acknowledging the sequence of events leading to a

failure condition can be complex. On the contrary, A-DAS contributes by providing an extensive

safety program plan including several activities such as identifying requirements at an early stage

for a specific aircraft system to function, to ensure safety is able to be maintained in that condition.

Given the key term validation, A-DAS can complement E-OCVM in the requirement development

by using terms such as completeness and correctness. Moreover, one can also use the validation

plan given in the same standard to support validation in areas such as traceability, analysis,

modeling and testing. On the other hand, A-SLP can complement E-OCVM on how to translate

stakeholder needs to the definition of requirements which is the previous step before checking

correctness and completeness.

Based on the results with relevance to system integration, E-OCVM can be complemented by

standard A-SLP with regards to a specific assessment plan for integrating different life-cycle

processes to a system of system. In addition, A-DAS stresses that there should be specific means

to show how intrasystem requirements have been fulfilled. When conducting a validation in this

area, it is further important to consider the environment of simulation. As this standard comprises

the context of an aircraft, cost-effective measures are emphasized to imitate the on aircraft

integration.

Given the key term quality assurance, A-SAS and A-DAS provide guidelines with regards to

objectives within areas of software assurance as well as in avoiding creation of development errors

leading to a failure condition. In A-DAS, there are concerns with analysis and design techniques

which are traditionally applied to deterministic risks or to non-complex system elements not being

able to safely cover for more complex system elements. Thus, the standard highlights that a specific

process is needed to assure techniques, such as development assurance utilizing a combination of

process assurance and validation, are used to better suit to these more complex systems.

However, the given standard A-DAS states that the operational context associated with air traffic,

ATC and passengers in areas such as traffic density and performance limitations should further be

considered. This is based on the primary owner’s requirements of the system elements often being

difficult to agree with and the standard further suggest that other documents or standards may act

on behalf on these requirements to facilitate assumptions about the operational context. Several

other standards such as A-SLP refer to other documents in areas such as identifying stakeholder

needs and quality management. Given that ATC also is a ScS which is sensitive with regards to

system risk since a failure could potentially cause severe damage, quality assurance will be more

important. Two of the reasons are that there is a fast pace of technological change in the industry

and new technology can lead to an uncertainty of understanding all potential risks and behaviors

with the system elements.

4.2 Interviews

Given the method, the interview results along with three different themes are outlined to fulfill the

purpose of the study in the following section.

4.2.1 Safety

Safety within aviation systems is crucial, as a consequence of that errors in the aviation industry

can result in major casualties. Previously, a standard within aviation industry was created only if a

hazard occurred, a fatality had to occur according to Griffiths and Asplund. Bekier argues that a

change for a procedure in the aviation industry is only acceptable if it increases safety and capacity

or reduces cost. In addition, if a project aims to fulfill only two of the three parameters, it is still

considered as a solid case according to Bekier. Regarding the safety question, Baumgartner argued

that safety is so important that it is not even mentioned, he compared it with breathing for a human

being. Baumgartner argued, to keep aviation industry safe during the automation then you will

have to secure certain data. Meaning, the way flight planning occurs e.g. getting access to an

airspace or airport, are aspects the airlines are unwilling to share. When information is not easily

shared then it is very difficult to create one standard which fits everybody within automation.

Asplund brought up the possibilities and threats of Cyber Physical Systems (CPS), which is further

viewed to belong to the automation sections of this thesis. One of many advantages with CPS are

the capabilities to control and monitor different functions in a system. According to him, a security

risk related to CPS is that they can potentially be hacked. In addition, he mentioned that for instance

an attack on the infrastructure in Europe could be a potential target for attacks but also the common

privacy aspects connected to laws etc. By conducting updates on autonomous systems with the aim

of making it more efficient, can pose challenges as to creating unsafe situations if a detail in the

system is not correctly updated. This is also viewed as a problem according to Paul Kennedy,

because a requirement for the commercial autonomous product is that the SOI can operate equally

as good as if there would be a human in the SOI. A solution argued by Kennedy is that you would

do a trial and show gradually that the SOI meets the requirements, and then move it up to the next

phase to get more confidence in the system element based upon its exposure to real training. The

commercial product must at the end of the day show evidentially that the system element is not

going to make the whole ATM system less safe. Sundqvist elaborated on this by mentioning that

there should be a separated airspace, a segregated controlled airspace just for automation process

as agreed by Baumgartner. He says that there will be autonomous forms of drones, and ATC would

be managing these on their own limited segregated airspace with automated tool and automated

procedures. Sundqvist mentioned that an essential precondition is that the ATC should never have

direct control of the aircraft, to ensure that the communication between the ATC and the drone is

not hijacked in anyway. Marek Bekier argues that a countermeasure towards hijacking data, which

can extensively damage a country’s infrastructure, is that it is very important to invest more in

cyber-security as digitalization increases and make sure that the data will never be corrupted or

understand when it is.

Sundqvist further elaborated that you setup a dedicated data-link between the ground and the drone

(for communication between the ATC and the drone), to ensure that the pilot is the only one that

commands the aircraft. “There is one pilot in command of every flight in the world” according to

Sundqvist which implies that if the pilot rests during an operation, and something were to occur

during that time then the pilot would still be responsible. Regarding safety, Kroese concluded that

there are two limitations in the automation process in the aircraft industry. Firstly, that there will

always be a pilot responsible for the actions of the aircraft. Secondly, the ATC will never command

the aircraft. The command has to instead be managed through a dedicated pilot accepting the

maneuver.

Several results from the interviews indicate that an automated reliable system element in the

context of ATC requires careful consideration on how the human element in the loop would be

shifted in the case of automation. Bekier argues that the interaction between the human and

machine is complex but also very crucial to manage the consequences they bring. He further

brought up a case where there is a lack of understanding of how an ATCo reacts when executing a

task in a stressful situation which can lead to a loss of the comprehensive picture in the situation

during training. Kennedy argued that a change to the human element in the existing loop while

ensuring new or modified equipment have flexibility is important since it otherwise requires an

extensive analysis of system functionalities and their interaction with humans to find a

technological replacement (for the human). Other major consequences for the human element to

the automation of ATC is the human centricity according to Eric Kroese. The current ATCo job is

centered with a lot of manual work and an automation would change the job from manual to more

strategic and monitoring kind of job.

An integration of the human into the whole SOI will have to be achieved in order to produce high

enough safety levels in the aspect of automation until full level of automation is achieved (Pacaux

M. P. et. al, 2011). The challenge according to Baumgartner is that the human should not be singled

out, to make sure the human can be integrated in the SOI, into the design of the system and into

the evolution of the system, as argued by HALA! (2010), if automation of decision making is made

in a high level in risky situations, then it is recommended that the human has some degree of human

action (see Table 4). Baumgartner further elaborates that when a system element fails, the blame

is often put on the human since the system in most cases is not designed in a human friendly

manner. Baumgartner believes that when designing an automated system element, the human has

to be part of the system, in combination with standards. By using standards, the challenges such as

system elements having different life cycles can easier be encountered making it one of the keys

to obtain interoperability (Systems Engineering Handbook, 2006). Coglianese (2012) argued that

standards seeks to change behaviours to produce a desired outcome which is to fly the aircraft as

safely as possible, and Asplund (2014) acknowledged that standards should be viewed as best

practices to provide high level and infrequent feedback.

A reliable automated system element further requires redundancy according to Kennedy. In

addition, he mentioned the importance of having redundancy in the system as a challenge to

validate the system. Sundqvist emphasized that redundancies are required when conducting tests

on a newer system to make sure that even though the system fails, is still safe to fly. Furthermore,

the environment in which the system is operating has to be analyzed under strict circumstances to

ensure the well-functioning of the system. Asplund argued in relation to this that a lot of safety

redundancies which considerably facilitates the system to operate in a safe state can also hinder

the deployability of the system. Leveson (2004) argues that designing systems with more

redundancy in terms of protection towards individual component failure may even increase risk

since it increases the complexity of understanding the system. There is further an example of flight

testing a NASA experimental aircraft using two computer based systems in which one contained

more extensive redundancy towards software errors. Eventually, the simpler based control system

performed better which indicates that although the intention is to increase safety, it is important to

address the behaviors of the systems when adding system functionalities to an existing system.

4.2.2 Training Phase

To be an ATCo, a long process of training is required, from theory tests to simulations in a safe

environment. Initial training starts with practical knowledge, which can be gained by anybody

through online courses, regardless of nationality according to Baumgartner. This is basically the

whole framework of rules and regulation of ICAO, specifically how the rules are being maintained

and implemented as clarified by Eric Kroese. The knowledge framework is what the training

individual needs as background in order to understand the role of the ATCo and what the different

organizations are. Kroese further explained that countries normally don’t waste time with

simulations and developing competencies until the training of the individual has passed the theory

tests. It is crucial to know the theory by heart, as argued by Kroese, the ATCo will have limited

time to search for information when in the operational field. After 3-6 months, basic training starts,

meaning that the ATCo develops competencies in a simulator environment, essentially playing a

video game. In the simulation environment, the pressure is increased on the ATCo to see their

capabilities and limits. In the end of the basic training course a European training license will be

gained, to work for an entitled European service provider, such as ACR. The European service

provider will provide the unit training, which is the next module after the basic training. In unit

training specifications of a specific airport is learnt, firstly in a simulator and in the job, which

means working in the tower and having a coach monitoring and constantly aiding the ATCo,

making sure that correct decisions are being made. Kroese emphasized that there are a lot of

knowledge being exchanged between the various countries and member states of ICAO in order to

see what the best practice in training is, but there is no standardized training secret or syllable that

is sold worldwide, in contrast to pilot training.

In an automated ScS one important element to address is the training of the ATCo. A consideration

to take into account according to Kroese is addressing the 3D thinking which is a crucial goal of

the simulation environment in which aircrafts are flying at high speeds crossing courses while

simultaneously climbing and descending. People must have a certain capability to multitask

meaning to be able to sort out several separation issues between airplanes simultaneously. This is

one of the more difficult phases of today’s training for ATCo trainees since it currently helps to

create a certain mental logical order. Bekier agreed that this phase is more complicated as a

consequences of that training of controllers is normally regulated. Automation would require that

the mental logical order needs to be reconfigured due to a possible forthcoming change of

introducing UAV into controlled airspace which requires a modification to the previous mental

logical order. As Leveson (2004) argued, these new designs to the systems have to be emphasized

to avoid operator error. In addition, in a case where the newer systems fail, there should also be a

possibility to allow the ATCo to go back to the previous system without ATCo feeling “rusty” and

manage to operate it safely. Moreover, offering a possibility to have a redundancy in terms of using

the previous systems elements also requires full attention of how these systems element interrelate

to avoid unnecessary complexity.

As automation questions come into consideration, Bekier emphasises that the interaction between

the human and machine is complex but also very crucial to manage the consequences they bring.

As an example, he described human factor training in the aviation industry such as crew resource

management in the airborne part and team resource management in the ATC side. He believes that

these types of trainings are critical to understand how they react, what their weaknesses are but

also their strengths. On the contrary, Kennedy argued for that in the training environment the

requirements for safety is lowered, due to emphasis of demonstrating the commercial product.

Kroese agreed with Bekier in terms of the training aspects of ATCo would need to be reworked

during the automation. Kroese repeated that it is essential that people in their mind can build up a

picture of the actual situation in the air in three dimensions and also in the dimension of speed

because the dimension of speed determines the ability to predict what that picture will be in the

future, as a redundancy if the automation system element would fail. Kroese believes that to make

judgements requires awareness with regards to the limitations a human has such as knowing what

their weaknesses and strengths are beyond the limitations for the machine. Therefore, the human

factor should have a prominent position in training. Bekier further elaborated on this argument, in

an ideal system the training should be more individualized because people have strengths and

weaknesses in different areas. However, there occurs failure in understanding how the person

executing the task is actually reacting in a real stressful situation, triggering a situation where one

could lose the holistic overview of the task. Moreover, it is interesting how a person potentially

recovers from a similar situation which is difficult to exercise and put into training, especially in

unit training according to him. Bekier argues that one possibly could realistically simulate unusual

scenarios because today an ATC simulator can arguably be viewed as artificial. Furthermore, he

thinks that this area of training can be more addressed with the increasing quality of simulators.

Training ATCo with unusual scenarios could be a way to train ATCo with new system element

designs.

The challenges with obtaining a reliable automated system in other industries such as in vehicle

testing has yielded an approach aiming at having a phased development. A phased development

refers to limiting the initial stakeholder requirements used in testing to facilitate a validation

(Koopman and Wagner, 2016). Similarly, Kennedy argued the importance of increasing

confidence in a prototype before demonstrating the product to stakeholders such as IAA. In the

context of autonomous vehicle testing, this notion is built upon increasing confidence at an early

stage of product development and testing which further eases the scope to be widened to manage

the challenge of combining many scenarios in terms of complex requirements. This further

facilitates in the case of having for instance the driver out of loop when conducting autonomous

vehicle testing since the driver cannot provide control inputs to the vehicle during operation.

Therefore, a fully autonomous vehicle ought to have significant added complexity to manage many

of the possible combinations of scenarios without a driver. Asplund argued in a similar manner

that an automated process could be implemented safely by carrying out the testing with fewer

parameters and thereby develop the product in a strict environment. He further brought up the

example of rolling out different types of automated fleets into an area where the aim is to see how

they are interacting in the given environment. By firstly rolling out a certain type of fleet in the

environment to test and obtain data would facilitate the analysis before combining all the fleets

into environment according to him. Based on the same principles in the context of automating

UAVs into controlled airspace or managing the change of mental logical order, it would for

instance be arguably important to gradually increase complexity of managing redundancy of the

systems in a smaller environment, after having fully understood other areas such as the human

interaction (human error) with the systems in a limited environment as well. In the context of

training ATCo, there were concerns about existing training phase not being able to consider the

reaction of a person executing a task in a stressful situation, triggering a situation where one could

potentially lose the holistic overview of the task. Bekier mentions a way to utilize the opportunity

of simulation is by increasing the quality, by providing more unusual scenarios with the new

system element design, to better correspond to reality in terms of considering the reaction of the

ATCo.

This further aligns with the findings in the standard A-SAS in the context of software errors which

highlights the importance of understanding that software errors can be latent and therefore not

immediately create a failure level at aircraft level which indicates a need for clear directions with

regards to safely enabling a more advanced automated system. In addition, standard A-DAS argues

that a safety program should be involved for an appropriate management of safety assessment

processes which depending on the complexity of integration and system implementation requires

corresponding level of detail.

4.2.3 Future of System element Design

One of the primary aspects in a future system element design is that the new regulation will address

the issues of the current way of creating standards, which are that accidents need to occur in order

for a standard to be created, as confirmed by Asplund and Griffiths. As of now, validation of

conceptual system is possible, as explained in the literature review with the examples of CAS and

OPTAIN-SA. Saab AB, conducted validation through simulations according to Sundqvist. He

mentioned that a detect and avoid system test cannot test direct collisions or failure analysis of

engines because it is too dangerous, and therefore has to be validated through simulations.

Sundqvist argued that a future system element can only be trusted if it is certified, which is achieved

only after the system element has been utilized in the real environment. Therefore, simulations will

have a more important role in providing accurate representation of reality which however

according to Koopman and Wagner (2016) has its own challenges. The simulations often offer

high reproducibility with regards to effort but the challenge lies in selecting proper scenarios along

with parameter variations which will cover a set of variations sufficient to properly model the

system in question (to a reasonable degree). A suggestion according to the same source is to use a

phased development which entails using a method whereas few scenarios as possible are tested in

a simulation before combining various scenarios more extensively.

Additionally, Kroese argued for the future of a system element such as the ATCo’s work, will be

redefined from a purely tactical style of work to a more strategic and monitoring style of work. He

elaborated that present description of the responsibilities and authorities will need to be redesigned

to reflect the new situations of responsibilities of the ATCo’s as there is a new technological shift

in the aviation industry. Kroese and Baumgartner agreed that computers can manage traffic better

than humans, but the primary goal will always be to contain system risk according to Griffiths. As

mentioned before, if the future automation of ATC fails, an ATCo needs to still be able to manage

to provide service for the aircraft from a radio as a redundancy. The system risk aspect is according

to Kroese not clear yet, as future system elements are evolving, and therefore requires further clear

directives.

Furthermore, Baumgartner argues that the standardized way of conducting automation in the

aviation industry is not clear. He argues that the autonomous UAVs will be segregated, meaning

that they will have their own airspace separated from the civil controlled airspace. The

opportunities when implementing an automated ATC according to Baumgartner is that a new

airspace will have to be created, an automated airspace involving a drone airspace with automated

tools and with automated procedures (Baumgartner, 2018). A stepwise approach would be required

with incrementally small steps during automation according to Baumgartner (2018). As agreed by

airlines & IATA (2017), “It’s clear that UTM system capabilities will be implemented

incrementally over the next few years”. The initial process in the stepwise approach will at least

start in a non-segregated interoperational environment as argued by Lorenzo et al. (2018) and the

MIDCAS project by Saab Corporate (2015) in the terminal/approach airspace. The interoperational

environment creates the complexity of SoS, due to the emergences of different life cycles of each

system element. Specifically, the differences in life-cycles will create limiting boundaries for the

new system element and affect the overall performance of the SoS (Systems Engineering

Handbook, 2006). An example is the current issues the 4D trajectory system is facing, as discussed

below.

Eric Kroese argued that there are tools that can copy the work of a controller and be more accurate,

for example the 4D trajectory management, which entails space (3D) and time. Iovanella et al.

(2011) argue that “4D trajectory management will be effective and will significantly enhance the

ATM system overall predictability, only if the adoption of 4D technologies will be widespread all

over Europe”. Griffiths (2018) argues that the problem with 4D trajectory will only benefit the five

biggest and most congested airports in Europe and therefore it would not be effective for the other

airports to invest in the required 4D trajectory infrastructure. Additionally, the airports in need of

4D trajectory have built secondary and tertiary airports to relieve the burden which counteracts the

intention of 4D trajectory. Frankfurt main airport is in competition with the secondary airport

Frankfurt Hahn, Paris’ main airport Charles de Gaulle airport has competition with Paris Orly

airport, Amsterdam Schiphol has soon competition with regards to the soon finished Lelystad

airport close to Amsterdam, Heathrow has competition with Gatwick airport and Rome’s main

airport Leonardo Da Vinci has competition with Ciampino airport. This is due to the high cost and

need for coordination between all airports in order for 4D trajectory to work. Another issue

according to Iovanella et al. (2011) is the mix traffic situation, which involves aircrafts using 4D

trajectory and those that do not, which has to be managed. Although, by slowly introducing an

automated system element, in an interoperational environment, and coordinating this system

element with the current aviation and additionally validating it in a similar fashion as the CAS

system by Manfredi et al. (2018) then the problems faced by the 4D trajectory technology would

be facilitated.

To fulfill interoperability between UAVs and manned aircrafts, it is important that there is an

ability to provide accurate data on position, thrust control and flight path for the UAVs. This will

further require ATC to manage the additional workload of managing UAVs in terminal/approach

areas. DeGarmo (2004) argues that SWIM can be an important factor in integrating UAVs and

manned aircraft since the foundation of SWIM is built upon common data standards and a dynamic

data exchange. Similarly, Peña et al. (2008) argue that an implementation of SWIM would facilitate

for the integration of UAVs in ATM using a similar argument as DeGarmo (2004) which is the

network centric concept provided by SWIM which potentially could facilitate accurate drone data

acquirement. There are further possibilities of drones acquiring information from areas with a

higher uncertainty with regards to weather conditions which eventually in areas close to an airport

can even help to enhance weather information during for instance bad weather.

As supporting argument for interoperability, external aviation companies are collecting and

transforming data in the aviation industry, for example voice and radar picture are transformed into

internet protocols, which will enable automation, as argued in section 2.3.2 Automation. This is

creating a new set of standards when it comes to data. Baumgartner (2018) argues that foreigners

to aviation companies will enter the market, ‘New Kids the Block (NKB, i.e. Google, Apple,

Facebook, Microsoft and telecommunications industries), and when they are able to set a standard

then that is when harmonizations can occur as confirmed by Airlines & IATA (2017). The reason

for this being that the ATC is not attractive for mass manufacturing industry because the ATC

industry is monopolistic. Specifically, the problem is those who are able to provide the IT

knowledge, the software knowledge, the way of creating a new standard, they will face the same

problem as the manufacturing industry. This leads to a group of people who are not interested to

produce more automation when it comes to ATC. When NKB start becoming active, then they will

not seek partnerships with aviation companies who already have the infrastructure, but will create

their own infrastructure. Microsoft are already active in the drone industry as they have Airmap.

Airmap is a Microsoft software that lets the user provide safe drone operations (Airmap, 2017).

There will be a necessary collection of data due to the increase in UAVs in non-segregated airspace,

as discussed in 2.4.1 and remote controlled aircraft in controlled non-segregated airspace projects

from Saab, such as MIDCAS, will help to provide safe and efficiently automated ATC solutions.

Baker (2018) argues automated tools assist ATCo with getting accurate information, aid in

increasing visibility at airports and improvements in communication with pilots. Automating these

aspects assist ATCo with their primary task which is to separate aircraft (Baker, 2018).

The main task for an ATCo is to organize the flow of traffic, that is separation of aircraft. As argued

by the International Federation of Air Traffic Control Association (IFATCA) and Baumgartner

initially, one area that is to be automated or digitalized are the housekeeping tasks, which is

communication with aircraft, providing maneuvering assistance, changing frequency, route

clearance and climb clearance. Some housekeeping task have been digitized with datalink, which

is a form of connecting one location to another in order to transmit and receive information

(techopedia, nd). Baumgartner argues that with increasing air traffic it is unreasonable to capture

substantial amounts of aircrafts at any given moment for an ATCo, which requires many assistance

tools to separate the traffic even before it reaches the ATCo’s sector. Lastly, Baumgartner believed

that in a future element design, a blockchain approach to exchange secure classified information is

a solution, in contrast to current way of exchanging information. (Baumgartner, 2018).

Automation

In addition, there are none or few parts among the chosen standards that are considering

automation. This indicates that more standards and documents are needed to cover these areas

since the key terms do not involve automation. The key terms did not include automation due to

the low result of the key term in the standards. Furthermore, it is therefore reasonable to pay

attention to how the human role in the system changes and the overall interaction between humans

and machines with regards to automation. As argued by several of the interviewees one can always

enhance the methods of understanding the relation between humans and machines.

Asplund (2014) argued that usage of safety standards makes it important to emphasize the contexts

that they are used in since a consideration used in one specific standard can violate the attempts of

another. Based on the key term system integration there were concerns about not being able to

fully find correspondence to the interoperability between UAVs and civil airspace due to the lack

of distinctness in the word. However, the two standards that gave an adequate result are partly

having a foundation in a general context while the other standard is used within the contexts of

aircrafts. More specifically, a UAV will have to comply with many regulations used by aircrafts

in a general context but simultaneously has to adapt certain functionalities based on the capabilities

of the drones which the other standard can contribute with. Another concern that has to be

highlighted is the standards’ ability to be used in a operational context since the target is to provide

considerations with regards to the interoperability between UAVs and civil airspace. The purpose

of the standard A-DAS is to describe the development of aircraft systems also considering aircraft

functions and operating environments (Aerospace Recommended Practice: SAE Aerospace

ARP4754A, 2010). A-DAS states that an emphasis on the operational context associated with other

elements such as ATC, traffic density and performance should be considered by suggesting usage

of other documents and standards. The other documents and standards could further facilitate how

the future system will be established and more importantly controlled. There will preferably be

testings of the new emerging standards to clarify whether they are able to meet the requirements

to go forward in the process. Once these standards are controlled to yield the intention of

integrating drones into civil airspace, the operational testings of the drones can be added into the

ATM.

5 Discussion and conclusions

In the discussion and conclusions, the implications of the results are elaborated on in relation to

the purpose and research questions. Additionally, a discussion on sustainability is included along

with criticism on the used method.

RQ1: What are the primary concerns of stakeholders’ in this specific ScS (ATC) in terms of

merging automated new systems into the existing system?

There are several concerns of ATC stakeholders’ in terms of merging automated system aspects

into the current system. The most important area is the considerations one has to make with regards

to how the human element is shifted in the existing loop in the case of automating a system.

Humans have to be kept in the loop to keep the system as safe as possible, and further carefully

analyze how the systems can be made to optimize the end user experience. Additionally,

redundancies would be required, meaning that if a system element fails, the operation would be

able to still function and put the system into a safe state. The redundancies should also be designed

with carefulness as they will interact with other system components with different life-cycles. This

can otherwise generate a risk and potentially lead to human errors for the end users such as ATCo.

An automated aviation system would have to address the current 3D thinking in the training

environment of ATCo in order for the ATCo to retain a certain mental logical order while

introducing drones into civil airspace. This is also needed as a redundancy to go back to the

previous system if a system element fails to function.

Given that safety is of utmost importance in the aviation industry along with zero tolerance of

fatalities, simulations of scenarios in a testing environment will have a greater role in providing

reassurance both for the stakeholders and for the public that the conceptual ScSs are safe. By

conducting a phased development one can increase confidence in a prototype and demonstrating it

to a stakeholder and the public using few parameters in a strict environment. This can further help

with the integration of drones into civil airspace and become a foundation for subsequent

automation. Current training of ATCo does not take into consideration the reaction of the

individual, which is an important key performance area. Given the safety concern, prospective

simulations will have to acknowledge these issues by providing more realistic scenarios of training

for ATCo and further capture ATCo’s reactions.

RQ2: What are the predictions for future system element design according to stakeholders in

regards to a ScS (ATC)?

The most important predictions for future system element design according to stakeholders in

regards to a ScS are firstly, that there is no clear standardized way of conducting automation in the

aviation industry. In the final stage, several of the interviewees believe that UAVs will be

segregated to fully exploit the benefits of automation but a stepwise approach is crucial with

incrementally small steps and therefore the initial process could commence in a non-segregated

interoperational environment.

A current assistance tool that exist for ATCo is the 4D trajectory, that mimic the task of an ATCo

and provides better predictability of traffic along with reduced fuel costs and emissions. Griffiths

(2018) & Iovanella (2011) argue that 4D trajectory management will be effective and will

significantly enhance the ATM system predictability, if the adoption of 4D technologies are

widespread all over Europe. However, this requires that airports are congested to obtain the full

potential of 4D trajectory which is counteracted by many cities building secondary airports.

As one of the concerns were how interoperability between UAVs and civil airspace could occur,

the study has yielded that it is important that there is an ability to provide accurate data on position,

speed and flight path for the UAVs not only for ATC but also for manned aircraft in its proximity.

This will further require ATC to manage the additional workload of managing UAVs in

terminal/approach areas. SWIM can be an important factor in integrating UAVs and manned

aircraft since the foundation of SWIM is built upon common data standards and a dynamic data

exchange. An implementation of SWIM could facilitate for the integration of UAVs in ATM using

the network centric concept provided by SWIM which potentially could ease accurate drone data

acquirement.

RQ3: How can a currently mandated standard E-OCVM be supplemented by already available

knowledge about other complex systems?

Given the key term safety, E-OCVM can be complemented by standards A-DAS and A-SAS in

areas such as how software errors can lead to a failure condition. As regards the key term

validation, A-DAS can complement E-OCVM by utilizing the given validation plan while A-SLP

can complement E-OCVM on how to translate stakeholder needs to definition of requirement.

Based on the results with relevance to system integration, E-OCVM can be complemented by

standard A-SLP with regards to a specific assessment plan for integrating different life-cycle

processes to a system of system. Standard A-DAS further emphasizes that there should be specific

means to show how intra-system requirements have been fulfilled and the importance of

considering the environment of simulation. Given the key term quality assurance, A-DAS and A-

SAS provides guidelines with regards to objectives within areas of software assurance as well as

in avoiding creation of development errors leading to a failure condition.

Based on the key term safety with regards to standard A-DAS, it is evident that it is aimed for

current civil aircraft and systems which drones will have to fulfill since they will have to act in

accordance with manned aircraft as argued by Bernauw (2015). However, it would be beneficial

to use standards adapted for the context of UAVs to obtain specific details on how interoperability

could occur. There are several standards under development such as ISO/TC 20/SC 163 and

3 For more information review: https://www.iso.org/committee/5336224.html (Accessed 15 June. 2018).

standard UL 30304 which considers operating environments, design and safety management as

well as electrical system of UAVs. Despite the four A-standards being balanced in terms of two

standards used in a general context and two standards used specifically for the context of software

development and civil aircrafts, only a standard specifically tailored for UAVs would facilitate the

acquirement of specific details on how interoperability could happen. Given that different

standards were used to complement the E-OCVM standard, it would also be beneficial if standard

and automation creators used a single and unified set of understandings and standards to facilitate

UAVs integration into civil airspace.

The key terms chosen do not emphasize how integration with regards to drones can be obtained

and should therefore be used as a guidance towards what is important to address when an

integration is conducted. To give more specific details on how an integration could be conducted,

additional standards specifically tailored for drones would be beneficial. The analysis of the A-

standards also yielded that usage of other additional standards and documents would be beneficial,

especially in the area of safety since the context of the used standards are arguably not general.

Based on the key term safety, the standards A-DAS and A-SAS are based upon the contexts of

safety with regards to civil aircraft systems and software. An integration of drones with regards to

civil airspace might therefore require additional standards comprising more general contexts and

drones.

To clarify, the purpose with complementing the standard E-OCVM was to enhance its purpose of

being a widely used validation plan rather than conducting a clear validation on ATC. Verification

is contrary to validation aiming to confirm system requirements with regards to system elements

which shows that the system has been built right (Systems Engineering Handbook, 2006). In

contrast, validation aims to answer if the system is fulfilling its intended purpose after the product

has been built. Given that both validation and verification are a necessity in system development,

they give rise to different issues in terms of perceived risks, safety and criticality of the element

under consideration. Accordingly, verification has been excluded from the scope but has however

been described whenever distinction between the two terms were valuable for the

comprehensiveness of the report.

4 For more information review:

https://industries.ul.com/energy/e-mobility/personal-e-transportation/drones-or-unmanned-aerial-vehicles-uav

(Accessed 15 June. 2018)

5.1 Discussion on Sustainability

There are three pillars when discussing implications on sustainability namely, economic, social

and environmental sustainability (Gibson, 2006). Sustainability was not the main topic in this

thesis, but some indirect effects on sustainability exist. For example, changes in the aviation

industry such as the 4D trajectory management system do have direct effect on the environment

beyond increased predictability which is shorter and more efficient flight paths that reduce the CO2

emissions in the atmosphere.

Social sustainability can be viewed as issues that are important for society, customers and

workforce (Chopra & Meindl, 2008). Accordingly, in the context of this thesis it can be viewed as

how people will be affected by the more extensive automation processes in the industry. As a result

of this study, humans will play a vital role in the automation loop but necessarily not with the same

tasks as of today. Furthermore, there are occurring more complex relationships between humans

and automations as humans are moving into positions of higher-level decision making while the

automation is implementing the decisions. Accordingly, this creates new types of jobs which

however also involves system risk which has to be addressed in the different contexts they occur

in to avoid accidents.

Economic sustainability is one of the reasons that these changes have to occur since ATM is under

pressure to reduce cost and manage the increased number of passengers. Beyond these two aspects,

drones are to be integrated into civil airspace which puts more tension on sustainability given the

drone’s capabilities as they can pose threats to aircrafts. The integration will arguably have a cost

that the ATM system will incur for the benefit of the security for manned aircrafts. Nonetheless,

one shall not forget the multiple commercial opportunities provided by UAVs which go beyond

photography and surveillance to possibly operate similarly to a large passenger aircraft. This could

facilitate for many airlines given the low margins that airlines have by not having pilots onboard

to steer the aircraft which could reduce costs.

5.2 Scrutiny of Method

The analysis of the standards comprised four key terms which yielded various results. By only

using four key terms, there is a possibility that very important areas in the standards have not been

acknowledged which could have been counter measured by using more key terms. The method

used also entailed that standard A-RLP was less used than the other standards. A different set of

key terms would enable different parts of the standard A-RLP to become more relevant. For

example, if a key term such as “life cycle” was used then A-RLP would have brought more results.

Furthermore, the disadvantage with using key terms is that words tend to have many synonyms

which can lead to loss of information. In this case, system integration was the only key term that

was checked in detail for synonyms (interoperability and mixed operation). Due to the time limit,

other areas of the standards were not able to be analyzed but it is reasonable to believe that they

would have generated additional key terms to investigate.

5.2.1 Validity

As described in the method, validity can be divided into construct validity and external validity.

Construct validity was obtained by conducting triangulation on the data gathered, acknowledging

contextual factors and by explaining the data collection methods as advised by Gibbert et al.

(2008). Several of the areas with regards to data gathering methods such as interviews were

triangulated with primary or secondary literature sources. An example of contextual factors

influencing the data gathered is the word “system” which depending on the context could have

been used in various ways. A countermeasure was to clarify how the term was used in theory and

detach it from how it can be used in a practical manner which influenced the utility of it in the

study. In contrast, given that the problem in the industry is not well defined due to the lack of

consensus in regards to usage of drones, it can be argued that other operational measures such as

using other standards could have been applied. However, several of the standards used were

suggested from the interviewees in the pre-study and further reviews resulted in them being an

appropriate path.

External validity was obtained by interviewing stakeholders with different objectives within the

industry and conduct cross comparisons as a way to triangulate the obtained results. However, the

limitation of the study is that it considers few stakeholders and therefore it is rather difficult to

generalize it to a greater population. The stakeholders were extensively knowledgeable in their

respective fields which could offset the limited amount of interviews conducted. The keywords

and standards chosen were limited in this report due to the timeframe of this project and is thereby

a weakness in this study. Additionally, if more standards and keywords were reviewed, it would

have possibly given a different outcome. The majority of the stakeholders in the pre-study

suggested the same standards independently which after the pre-study turned out to be an

appropriate path, hence a strength. Lastly, data has occasionally been received by our supervisors

and may hence be biased towards their view on the subject.

5.2.2 Generalizability and Reliability

Generalizability can be assessed in several ways but a common denominator is a systematic

approach. A systematic approach was obtained in this study by asking the interviewees similar

questions in a semi-structured manner depending on the context (see method section for a sample

of the interview questions). As the purpose of the study was to identify considerations to make

when automating complex system elements involving different stakeholders in a ScS, the aim with

the interview questions were to let the interviewees hint about general issues that has to be

considered in a future ATC system which helps to increase the generalizability of the study.

However, there are specific areas of the interviews which could have been strengthened such as

having a specific interview question about the interviewees views on the drones integration into

civil airspace. In several of the interviews, the integration of drones into civil airspace were touched

upon especially during the introduction as it was a general topic when describing the study to the

interviewees and the topic re emerged throughout the interview in interview questions Q3, Q4 and

Q5 (see interview questions in method). However, a specific question could have increased the

reliability of the study and accordingly its generalizability to similar studies.

In regards to the nature of the obtained keywords, replicating this study following our method

could bring forth a different set of keywords since the interviews were conducted in a semi-

structured manner. Additionally, reviewing the standards based on the key terms also entailed a

subjective opinion about what is important rather than a clear and unbiased direction on how to

review the standards. For this reason, several reviews of the standards were conducted

independently to minimize the ambiguity towards this uncertainty. Given that standards within the

contexts of civil aircrafts and software were used along with two general standards within Systems

Engineering, a standard comprising general automation guidelines could have enhanced the

outcome of the results. More specifically, there were assumptions that the standards used in this

study would have included automation aspects but they had few connections to automation.

Furthermore, the data collection aimed to increase generalizability by specifying the research

process as how literature were reviewed to give an understanding of how empirical data was

analyzed. However, given that the study was solely qualitative in nature makes it more difficult to

replicate the results compared to a quantitative study.

5.3 Conclusion

The most important aspect that one has to consider is how the work tasks shift when the automation

process is achieved. Additionally, the human has to be kept in the loop when the shift occurs to

maintain safety within the ATC in conjunction with the UAVs. However, if the ATC or the UAV

fails in regards to an automation process then redundancies need to be set. The redundancies have

to beyond setting the system back to a safe state also be carefully analyzed in how they interact

with other system components to avoid misjudgement for the ATCo. These areas have to also be

addressed at an early stage for the ATCo, preferably in the training phase since ATCo have a

certain mental logical order which can be difficult to change.

There are no standardized ways of how the automation will emanate, therefore there are several

options permitted. One constraint is that for the automation of UAVs and ATC, a separate airspace

will need to exist to utilize the automation potential. Furthermore, there exist current innovative

tools such as 4D trajectory management system (not for UAVs yet) and will be effective if the

adoption of 4D technologies are widespread all over Europe. Therefore, the mix of old technology

with the new technology will be the starting point for the automation of ATC and UAVs in the

same terminal/approach airspace.

Lastly, the conceptual standard that is used to prove that the conceptual systems are safe need to

be complemented by other existing standards such as those specifically tailored for drones. This

would further facilitate how interoperability could happen as several of the standards refer to usage

of other standards and documents. Given that different standards were used to complement E-

OCVM, a set of unified standards are required that are proportional with the type of drones, the

type of operations and in the environment that they are operating in. This will be needed to fulfill

the European vision of safe integration of drones and needs thereby to be carried out in a global

manner, and accordingly also share experience with other actors to advance the new technology

adaptation. This will require beyond standard harmonization also adaptation to the training phases

of ATCo’s.

5.4 Further Research

Based on the findings in the report, there are several interesting research topics that arise which

have been excluded from this study’s scope. For instance, given that an implementation of drones

will occur into civil airspace, it would be of interest to further develop and evaluate how an

implementation can be conducted along with a larger scale implementation.

Another interesting area is how ATCo would cope with the integration of UAVs into civil airspace

in terms of workload, other psychological aspects and more specifically how the training phase of

ATCo accordingly should adapt.

Due to the delimitations of this project, verification was not taken into consideration, and for this

reason a future research could consider the specificity of verification in regards to a ScS in the

aviation industry.

As the study has had a primary focus on terminal/approach areas which are the most congested

areas, an integration of drones into civil airspace with regards to higher altitudes would be

interesting since it would give foundation for a full integration of UAVs into civil airspace. This

can further be combined with a study on how blockchain and AI can be involved with regards to

the increased concern of safety in terms of for instance cyber security and how it accordingly can

be implemented.

A future study could be based on the need for the 4D trajectory, the requirements of it and how 4D

trajectory is chosen to be implemented in order for drones to be fully autonomous without human

interaction.

Other interesting aspects in regards to AI are how AI will be integrated in other areas in the aviation

industry such as in AI assistance, smart logistics and facial recognition (Sennaar, 2018). In

Appendix C, we have elaborated on what AI entails in relation to the humans who create these

systems with regards to the importance of safety in the aviation industry.

7 Appendix

7.1 Appendix A

The figure below is the accurate illustration of how E-OCVM’s table of contents was analyzed.

Furthermore, the accurate figure was not used due to concerns with the resolution as well as a

considerable amount of information in the original figure.

Figure A. Illustration of how standard E-OCVM’s table of contents was analyzed.

7.2 Appendix B

The figure below is the accurate illustration of how standard A-SLP’s table of contents was

analyzed. Furthermore, the accurate figure was not used due to concerns with the resolution as well

as a considerable amount of information in the original figure.

Figure B. Illustrations on how standard A-SLP’s table of contents were analyzed separately by

the two authors.

7.3 Appendix C

The AI perspective was viewed to be an important part of automating complex system involving

different stakeholders in the ATC. However, based on the time frame of the project and the notion

of using a stepwise approach of automation in the study, it was decided that AI should be out of

scope but yet an important part of future research.

Robotics and automation emerged in heavy or repetitive human labour tasks such as in assembly

lines, for example part placements welding, painting etc. In many tasks robots have been more cost

effective than humans. Additionally, vehicle autonomous straddle carriers outperformed skilled

human drivers when transporting containers from containerships to trucks on loading docks. Some

automated tasks require a cognitive ability, for example driving a car in a crowded street or playing

games such as Chess, and it is in these areas where AI is useful. (Russell & Norvig, 2010) AI has

its foundation in learning symbolic representations of concepts from humans (Mitchell, 1997). In

Figure C, intelligence is divided into different categories, where the definitions on the top rows are

more thought and reasoning processes in contrast to behaviour which is the focus on the bottom

row. The definition on the left column measures success rates in regards to human performance

and the right column measures rationality (Russell & Norvig, 2010).

Figure C. An illustration of how intelligence is divided with regards to AI in different categories

(Russell & Norvig, 2010).

AI, in more specific terms refers to pattern finding analysis. The user inserts the necessary data

and the machine replicates the cognitive ability of a human to find patterns and eventually be able

to create a machine that performs as good as humans. AI is being used in the aviation industry,

according to Sennaar (2018) in three different areas, such as in AI assistance, smart logistics and

facial recognition. In AI assistance, the AI aids in answering customer request and questions.

Additionally, addresses voice command input from customers. In smart logistics, currently AI

algorithms are being used to facilitate automation in airline operations. Lastly, facial recognition

is currently being used to easier match customer luggage. The AI algorithms will be adapted to the

issues reported by pilots and other actors in the aviation industry such as having obstacles on the

runway during takeoff and landing. (Sennaar, 2018)

According to Vasiloglou (2018), AI will disrupt the aviation industry in a similar way as Airbnb

and Uber managed to disrupt their respective market within the policies and regulations. Current

complaints of aircrafts are the unseen obstacles from the cockpit on the runway when taking off

and landing which are not documented in any database, which is beneficial information for the AI.

This can be easily solved by increasing the eyes in the cockpit with innovative technology such as

Amazon Deeplens or a GoPro camera. AI can augment data sources which could disrupt the

monopolized hold that the aviation industry has on the current data. (Vasiloglou, 2018)

Baumgartner (2018) believed that outside forces from other industries are already disrupting the

aviation industry, from major companies like GAFA, Microsoft, NASA and other players in the

telecommunication industries. What these companies have in common is they all have access to

extensive amount of data, which will facilitate the AI process (Russell & Norvig, 2010).

Given that safety is of utmost importance in the aviation industry, an important area is how the

transfer of knowledge will be conducted between machines and humans. Humans are viewed to

learn about risk by practice and experience where the development already beginning in the infancy

years when learning how to crawl, walk and talk (Adams, 2002). This type of risk management is

then progressively enhanced when learning how to cross the street, ride a bicycle and handle hot

things to manage risk as a balancing act. The risk is a balancing act between being in safety and

being in danger where the potential rewards of an act is balanced against the potential

consequences. In addition, since it is a balancing act it further creates an uncertainty which always

will be a disadvantage for the human whenever things go wrong (Adams, 2002). An example is

that many of the flaws with regards to safety in systems are due to dysfunctional interactions among

system components rather than failure in the individual components which are man-made systems.

Regardless if the individual system requirements fully satisfied their requirements, they could still

give rise to a failure in the whole system due to a lack of understanding of how the components

behaviors affect the system as a whole (Leveson, 2004). One of the reasons is, as argued by Adams

(2002) that the uncertainty is created by the human since we are not being able to fully manage our

risk perceptions. As machines are created by humans, especially in the context of AI, it will entail

that machines will still not be able to fully manage this phenomenon since they are learning from

the humans. In addition, the AI has been used in a wide range of different industries where one is

usage of surgical robots during surgery of patients assisted by a surgeon. The robot control system

receives the commands from the surgeon and then translates it to precisely engineered movements

inside the patient’s body (Varshney and Alemzadeh, 2016). However, given the large variability

in operating environments, and behaviors of the surgeons along with incidental failures on the

instruments used, there have been reports of safety incidents negatively impacting patients.

Similarly, a self-driving car (in auto-pilot) mode collided with a truck after failing to apply brakes

leading to the death of the truck driver (Lowy, 2016). This happened despite over 130 million miles

of testing the automated driving system due to the extremely rare circumstance of the height of the

truck, its white colour under the bright sky combined with the positioning of the cars across the

road (Varshney and Alemzadeh, 2016).

Anything made by a human will make mistakes and there will be fatalities in the automation of

aviation industry unless there are well developed standards that can avoid them from happening.

As argued by several interviewees in this study, standards are often created after accidents and are

thereby are a profound element in providing an enhanced technology. Given the importance of

safety in the aviation industry in which fatalities are not permitted according to Griffiths (2018)

when developing technology in the industry, it will therefore be important to carefully understand

the limitations of future deployments.

8 References

Adams, J. (2002). Risk. [ebook] London and New York: Taylor & Francis Group.

Available at: http://www.john-adams.co.uk/wp-content/uploads/2017/01/RISK-BOOK.pdf

[Accessed 21 Aug. 2018].

Adams K. M., Hester P. T., Bradley J. M., Meyers T. J. & Keating C. B. (2014). System Theory as

the Foundation for Understanding Systems. [Research Article]. National Centers for System of

Systems Engineering. Old Dominion University. Norfolk.

Airlines & IATA. (2017). Drones: A new player on the aviation’s radar. [online].

Available at:

https://airlines.iata.org/analysis/constructive-technology

[Accessed 19 august 2018]

Airmap. (2017). The AirMap UTM Dashboard. [online].

Available at: https://www.airmap.com/utm-dashboard/

[Accessed 21 July 2018].

Aerospace Recommended Practice: SAE Aerospace ARP4754A. (2010). SAE Aerospace.

Asplund F. (2014) Risks Related to the Use of Software Tools when Developing Cyber-Physical

Systems: A Critical Perspective on the Future of Developing Complex, Safety-Critical Systems.

Doctor Thesis. KTH.

Available at: https://www.diva-portal.org/smash/get/divA-RLP:751097/FULLTEXT02.pdf

[Accessed 22 August 2018]

Baumgartner M. (2017). DIGITATMISATION: Is a radical reform of the technology pillar

needed? Or is it to late? Technology. Article. SESAR Joint Undertaking

Baker J. (2018). The role of automation in air traffic control. Airport-Technology. [online].

Available at: https://www.airport-technology.com/features/automation-air-traffic-control/

[Accessed 22 August 2018]

Barzantny C. (2018). Training Operational Monitoring in Future ATCOs Using Eye Tracking.

[Research Article]. German Aerospace Center. Hamburg, Germany.

Available at:

http://delivery.acm.org/10.1145/3210000/3207412/a79-

barzantny.pdf?ip=130.229.166.205&id=3207412&acc=ACTIVE%20SERVICE&key=74F76877

61D7AE37%2EE53E9A92DC589BF3%2E4D4702B0C3E38B35%2E4D4702B0C3E38B35&__

acm__=1536765648_7251aaebcf69e99558cee8a80408233f

[Accessed 12 September 2018].

Beglerovic, H. Metzner, S. and Horn, M. (2018). Challenges for the validation and testing of

Automated Driving Functions. [ebook] p.10.

Available at:

https://www.researchgate.net/profile/Halil_Beglerovic/publication/319404248_Challenges_for_t

he_Validation_and_Testing_of_Automated_Driving_Functions/links/59c0e4ff0f7e9b21a8261c8f

/Challenges-for-the-Validation-and-Testing-of-Automated-Driving-Functions.pdf [Accessed 24

July 2018].

Bernauw, K. (2015). DRONES: THE EMERGING ERA OF UNMANNED CIVIL AVIATION.

Ghent, Belgium.

Blomkvist, P. and Hallin, A., 2015, Method for engineering students. Lund: Studentlitteratur AB.

Collison P. (2017). Aircraft passenger wary of pilotless planes - even if they lead to lower fares.

[online] The Guardian.

Available at:

https://www.theguardian.com/business/2017/aug/07/air-passengers-pilotless-planes-fares-ubs

[Accessed 18 august 2018].

Coglianese, C. (2012). Measuring Regulatory Performance. 1st ed. [ebook] OECD. Available at:

https://www.oecd.org/gov/regulatory-policy/1_coglianese%20web.pdf [Accessed 28 May 2018].

Cohn, P., Green, A., Langstaff, M. and Roller, M. (2017). Commercial drones are here: The future

of unmanned aerial systems. [ebook] Sydney: McKinsey&Company.

Available at:

https://eu-smartcities.eu/sites/default/files/2018-01/commercial-drones-are-here-the-future-of-

unmanned-aerial-systems.pdf [Accessed 6 Aug. 2018].

Collis, J. and Hussey, R. (2013), “Business Research”, Nature, Vol. 142, pp. 410–411.

Creswell, J. (2009). Research Design. 4th ed.

Deener, S. (2017). Technique: The Life of a Flight Plan - AOPA. [online] Aopa.org.

Available at:

https://www.aopa.org/news-and-media/all-news/2017/july/flight-training-magazine/technique-

flight-plan [Accessed 22 May 2018].

DeGarmo, M. (2004). Issues Concerning Integration of Unmanned Aerial Vehicles in Civil

Airspace. [online] Mitre.org.

Available at:

https://www.mitre.org/sites/default/files/pdf/04_1232.pdf [Accessed 22 May 2018].

Driscoll, P. (2014). Breaking Carbon Lock-In: Path Dependencies in Large-Scale Transportation

Infrastructure Projects. [online] tandfonline.

Available at:

https://www.tandfonline.com/doi/pdf/10.1080/02697459.2014.929847?needAccess=true[Accesse

d 18 Oct. 2018].

Software Considerations in Airborne System and Equipment Certification: RTCA DO-178C.

(2011). RTCA.

Domecq, J. and Guillermet, F. (2018). THE CHALLENGE OF UAS TRAFFIC MANAGEMENT.

Eurocontrol.

Eurocockpit. (2018). RPAS. [Online]

Available at:

https://www.eurocockpit.be/expertise/rpashttps://www.eurocockpit.be/expertise/rpas [Accessed

28 May 2018].

Eurocontrol. (n.d.). What is air traffic management? | Eurocontrol. [online].

Available at:

http://www.eurocontrol.int/articles/what-air-traffic-management [Accessed 22 Apr. 2018].

Eurocontrol. (n.d.). What Does An Air Traffic Controller Actually Do? [online].

Available at:

https://atco.eurocontrol.int/#about [Accessed 28 May 2018].

EUROCONTROL Performance Review Commission & FAA Air Traffic Organization System

Operations Services. (2010). U.S./Europe Comparison of ATM-Related Operational Performance.

[pdf] Eurocontrol & FAA. Availaible at:

https://www.faa.gov/air_traffic/publications/media/us_eu_comparison_2010.pdf

[Accessed 24 September 2018]

Eurocontrol. (2013). Airspace Volumes & Sectorisation: Module 13 - Activity 9: European

Airspace Concept Workshops for PBN Implementation. [pdf]. ICAO.

E-OCVM.(2010). 3rd ed. [ebook] Eurocontrol.

Available at:

https://www.eurocontrol.int/sites/default/files/publication/files/e-ocvm3-vol-1-022010.pdf

[Accessed 19 Mar. 2018].

Frohm J., Lindström V., Winroth M., Stahre J. (2008). Levels of automation in manufacturing.

[pdf]. ResearchGate.

Available at: http://publications.lib.chalmers.se/records/fulltext/76667/local_76667.pdf

Futurism. (n.d.). The Top 12 Benefits of Drones: Emergency Response, Animal Protection, and

More. [online]

Available at: https://futurism.com/images/benefitsofdrones/ [Accessed 22 May 2018].

Finger M., Bert, N. and Kupfer, D. (2016). Disruptive Technologies in Air Traffic Management.

[ebook] Florence: Florence School of Regulation.

Available at:

http://cadmus.eui.eu/bitstream/handle/1814/44404/FSR_Transport_Observer_2016_03.pdf?seque

nce=1&isAllowed=y

[Accessed 14 Feb. 2018].

Gibbert M., Ruigrok, W., & Wicki, B. (2008). Research notes and commentary: What passes as a

rigorous case study? Strategic management journal, 1465-1474.

Gibson R. B. (2006). Sustainability-based assessment criteria and associated frameworks for

evaluations and decisions: theory, practice and implications for the Mackenzie Gas Project

Review. [pdf] Mackenzie Gas Project Joint Review Panel. Available at:

http://reviewboard.ca/upload/project_document/1218741818_Gibson%20Report.pdf

[Accessed 27 September 2018]

Granberg T. A., Axelsson P., Petersson J., Polishchuk T., Polishchuk V., Schmidt C. (2016).

Configuration and Planning of the Remote Tower Modules in a Remoter Tower Center. [pdf].

Linköping University, Sweden.

Griffiths, P (2018, February 19; Mars 12 & 15; April 13; May 3 & 14; June 7; August 8 & 15).

Interview.

Grover M.P. (2010). Fundamentals of Modern Manufacturing: Materials, Processes and Systems.

Lehigh University. Wiley. Fourth Edition.

Honour, E. (2018). Verification and Validation Issues in System of Systems. [ebook] Spring Hill.

Available at: http://arxiv.org/pdf/1311.3626.pdf [Accessed 15 Mar. 2018].

Honeywell (2018). Air Traffic Management. [www]

Available at:

https://aerospace.honeywell.com/en/pages/air-traffic-management [Accessed 24 May 2018]

IATA, 2016. IATA Forecasts Passenger Demand to Double Over 20 Years. [Online]

Available at: http://www.iata.org/pressroom/pr/Pages/2016-10-18-02.aspx

[Accessed 15 August 2018].

ICAO. (2012). [pdf]. ICAO.

Available at: https://www.icao.int/Meetings/anconf12/WorkingPapers/ANConfWP56.5.2.EN.pdf

[Accessed 20 August 2018]

ICAO. (1944). [ebook]. ICAO.

Available at: https://www.icao.int/publications/Documents/7300_orig.pdf

[Accessed 17 Aug. 2018].

Ieeexplore. (2018). 12207-2017 - ISO/IEC/IEEE International Standard - Systems and software

engineering -- Software life cycle processes - IEEE Standard. [online] Available at:

https://ieeexplore.ieee.org/document/8100771 [Accessed 11 Oct. 2018].

Iovanella A, Scoppola B., Pozzi S. (2011). The impact of 4D trajectories on arrival delays in mixed

traffic scenarios. SESAR WPE. Long Term Innovative Research. [article] Rome. Available at:

https://www.sesarju.eu/sites/default/files/documents/sid/2011/SID%202011-12.pdf [Accessed 17

August 2018]

Jiang T., Geller J., Daiheng N., Collura J. (2016). Unmanned Aircraft System traffic management:

Concept of operation and system architecture. [article]. ScienceDirect. International Journal of

Transportation Science and Technology.

Available at:

https://www.sciencedirect.com/science/article/pii/S2046043016300260

[Accessed 18 August 2018]

Josephs L. (2017). Your plane could fly itself by 2025...if you’re cool with that. [Online]. Quartz.

Available at:

https://qz.com/1047825/your-airplane-could-fly-itself-by-2025-if-youre-cool-with-that/

[Accessed 18 August 2018]

Kay, A. (2005). A CRITIQUE OF THE USE OF PATH DEPENDENCY IN POLICY STUDIES.

[ebook] Wiley.

Available at:

https://onlinelibrary.wiley.com/doi/pdf/10.1111/j.0033-3298.2005.00462.x [Accessed 24 May

2018].

Knight, J. (2002). Safety Critical Systems: Challenges and Directions. [ebook] Charlottesville,

VA: University of Virginia.

Available at:

http://users.encs.concordia.ca/~ymzhang/courses/reliability/ICSE02Knight.pdf [Accessed 20 May

2018].

Koopman, P. and Wagner, M. (2016). Challenges in Autonomous Vehicle Testing and Validation.

[ebook] SAE World Congress, p.10.

Available at:

https://users.ece.cmu.edu/~koopman/pubs/koopman16_sae_autonomous_validation.pdf

[Accessed 16 Mar. 2018].

Leveson, N. (2004). A New Accident Model for Engineering Safer Systems. Cambridge,

Massachusetts: Massachusetts Institute of Technology.

Lowy, J. (2016). Driver killed in self-driving car accident for first time. [online] PBS NewsHour.

Available at:

https://www.pbs.org/newshour/nation/driver-killed-in-self-driving-car-accident-for-first-time

[Accessed 21 Sep. 2018].

Lorenzo M. A. P, Fumero D. A., Lubrani P., Díaz M. V. (2018). Validation of the OPTAIN-SA tool

for Continuous Descent Operations. ICRAT 2018

Manfredi G., Jestin Y. (2018). An Introduction to Fast Time Simulations for RPAS Collision

Avoidance System Evaluation. ICRAT 2018

Mitchell, T. (1997). Machine Learning. McGraw-Hill Science/Engineering/Math.

Mueller E., Kopardekar P. (2017). Enabling Airspace Integration for High-Density On-Demand

Mobility Operations. NASA. Denver, Colorado.

Available at:

https://utm.arc.nasa.gov/docs/2017-Mueller_Aviation_ATIO.pdf [Accessed 8 Augusti 2018]

Oster C. and Emeritus P. (2015). 1 Cited Problems with the Current Air Traffic Control System

and Concerns about Changing the Organizational Structure. [ebook] Indiana University: Indiana

University.

Available at:

http://onlinepubs.trb.org/onlinepubs/sp/Cited_Problems_Concerns_6.10.2015.pdf [Accessed 5

Aug. 2018].

Pacaux M. P., Debernard S., Godin A., Rajaonah B., Anceaux F., Vanderhaegen F. (2011). Levels

of automation and human-machine cooperation: Application to human-robot interaction. The

International Federation of Automatic Control. Milano.

Parasuraman R., Sheridan T. B., Wickens C. D. (2000).

Peña, N., Scarlatti, D. and Ollero, A. (2008). UAVs Integration in the SWIM Based Architecture

for ATM. [ebook] Springer Science.

Available at:

https://link.springer.com/content/pdf/10.1007%2F978-1-4020-9137-7.pdf [Accessed 20 Aug.

2018].

Rao, B., Gopi, A. and Maione, R. (2016). The Societal impact of commercial drones.

ResearchGate.

Rasmussen, J., 1997. Risk Management in a Dynamic Society: A Modelling Problem, Safety

Science, vol. 27, No. 2/3, Elsevier Science Ltd., pages 183–213.

Rausand, Marvin (2011). Risk Assessment - Theory, Methods and Applications. John Wiley &

Sons Inc.

Rifkin J. (1995). The End Of Work: The Decline of the Global Labor Force and the Dawn of the

Post-Market Era. [Book] G.P. Putnam’s Sons. New York.

Russell S. & Norvig P. (2010). Artificial Intelligence: A Modern Approach. [Book]. Third Edition.

Pearson Education. Upper Saddle River, New Jersey.

Saab Corporate. (2015). Important Progress has Been Achieved Within The MIDCAS Project.

[online]

Available at:

https://saabgroup.com/media/stories/stories-listing/2015-05/midcas-project/ [Accessed 3 August

2018]

Safety Management Manual (SMM). (2018). 3rd ed. [ebook] International Civil Aviation

Organization.

Available at:

https://www.icao.int/safety/SafetyManagement/Documents/Doc.9859.3rd%20Edition.alltext.en.p

df [Accessed 7 Aug. 2018].

Schaar, D. and Sherry, L. (2010). Analysis of Airport Stakeholders. Research Gate.

Scholte, J., Blom, H., van den Bos, J. and Jansen, R. (2009). Management of ATM performance in

operational concept development and validation: a case study. Amsterdam: ResearchGate.

Available at:

https://www.researchgate.net/publication/255686132_Management_of_ATM_performance_in_o

perational_concept_development_and_validation_a_case_study [Accessed 21 August 2018]

Sennaar K. (2018). How the 4 Largest Airlines Use Artificial Intelligence. [online] techemergence.

Available at: https://www.techemergence.com/airlines-use-artificial-intelligence/

[Accessed 20 September 2018]

SESAR SWIM Factsheet. (2016). [ebook] Brussels: Eurocontrol.

Available at: https://www.eurocontrol.int/download/publication/node-field_download-5138-0

[Accessed 17 Feb. 2018].

Sesarju. (2018). European ATM Master Plan: Roadmap for the safe integration of drones into all

classes of airspace. [online]

Available at:

https://www.sesarju.eu/sites/default/files/documents/reports/European%20ATM%20Master%20

Plan%20Drone%20roadmap.pdf [Accessed 18 Oct. 2018].

Skyguide Solution. (2017). ATCo Basic Training. [pdf]

Available at:

https://www.skyguide.ch/wp-content/uploads/2017/08/ATCO-Basic-Training.pdf [Accessed 28

May 2018]

Sommerville I. (2011). Software Engineering. [book]. Addison-Wesley. Ninth edition.

Available at:

https://edisciplinas.usp.br/pluginfile.php/2150022/mod_resource/content/1/1429431793.203Soft

ware%20Engineering%20by%20Somerville.pdf [Accessed 27 August 2018]

Sunil Chopra and Peter Meindl (2008), Supply Chain Management , 6th edition.

Systems and software engineering — Life cycle processes — Risk management. (2006). 2nd ed.

[ebook] ISO/IEC.

Available at:

https://ieeexplore-ieee-org.focus.lib.kth.se/stamp/stamp.jsp?tp=&arnumber=4042193 [Accessed

6 Mar. 2018].

Systems and software engineering - System life cycle processes. (2015). 1st ed. ISO/IEC/IEEE.

Systems Engineering Handbook. (2006). 3rd ed. INCOSE.

System Safety Engineering. (2018). System Safety Engineering. [online]

Available at:

https://www.systemsafetyengineering.com/system-safety-engineering.html [Accessed 11 May

2018].

Tay, G. and Becker, A. (2018). Automation in Commercial Aviation 2030+. [ebook] Munich and

Hamburg. Available at:

https://www.lls.mw.tum.de/fileadmin/w00bdw/www/Vorlesungen/Handout_Abschlusspraesentat

ion_WS1617.pdf [Accessed 7 Aug. 2018].

Techopedia. (nd). Data Link. [online] Available at:

https://www.techopedia.com/definition/6749/data-link [Accessed 15 Augusti 2018].

US Department of Aviation. (2012). Instrument Flying Handbook. Federal Administration. [pdf].

Available at:

https://www.faa.gov/regulations_policies/handbooks_manuals/aviation/media/FAA-H-8083-

15B.pdf

[Accessed 19 September 2018]

Vaaben B. & Larsen J. (2015). Mitigation of airspace congestion impact on airline networks. [pdf].

Science Direct. Available at:

https://www.sciencedirect.com/science/article/pii/S0969699715000459 [Accessed 28 May 2018]

Varshney, K. and Alemzadeh, H. (2016). On the Safety of Machine Learning: Cyber-Physical

Systems, Decision Sciences, and Data Products.

Vasiloglou N. (2018). Opinion: Disrupting Aviation With Artificial Intelligence. [online]

InsideMRO. Available at:

https://www.mro-network.com/emerging-technology/opinion-disrupting-aviation-artificial-

intelligence

[Accessed 20 September 2018]

Westergård, Morten Jarvis (2016). Degree Project in Mechanical Engineering Second Cycle. KTH.

Stockholm.

Yin, R. (2003). Case Study Research. 2nd ed. [ebook]

Available at:

http://www.madeira-edu.pt/LinkClick.aspx?fileticket=Fgm4GJWVTRs%3D&tabid=3004

[Accessed 15 Mar. 2018]

www.kth.se