EUROCONTROL...Ron Slootbeek (LvNL) Ops expert Peter Roben (LvNL) Ops expert Victor van Kempen...

© European Organisation for the Safety of Air Navigation EUROCONTROL 2007 This document is published by EUROCONTROL in the interest of the exchange of information. It may be copied in whole or in part

providing that the copyright notice and disclaimer are included. The information contained in this document may not be modified without prior written permission from EUROCONTROL.

EUROCONTROL makes no warranty, either implied or express, for the information contained in this document, neither does it assume any legal liability or responsibility for the accuracy, completeness or usefulness of this information.

EUROPEAN ORGANISATION FOR THE SAFETY OF AIR NAVIGATION

EUROCONTROL

EUROCONTROL EXPERIMENTAL CENTRE

AMRUFRA REAL-TIME SIMULATION VALIDATION REPORT

EEC Report No. 409

Project: AMRUFRA

Public Issued: October 2008

REPORT DOCUMENTATION PAGE

Reference: EEC Report No. 409

Security Classification: Unclassified

Originator: EEC - (Air Traffic Control)

Originator (Corporate Author) Name/Location: EUROCONTROL Experimental Centre Centre de Bois des Bordes B.P.15 F - 91222 Brétigny-sur-Orge CEDEX France Telephone: +33 (0)1 69 88 75 00 Internet : www.eurocontrol.int

Sponsor: EUROCONTROL

Sponsor (Contract Authority) Name/Location: EUROCONTROL Agency 96, Rue de la Fusée B - 1130 Brussels BELGIUM Telephone: +32 2 729 90 11 Internet : www.eurocontrol.int

TITLE: AMRUFRA REAL-TIME SIMULATION VALIDATION REPORT

Author

Renée Schuen-Medwed Angelo Scorciarino Michel Geissel

Date 03/2008

Pages xx + 140

Figures 40

Tables 8

Annexes 13

References 14

Project AMRUFRA

Task no. sponsor -

Period

2007-2008 Distribution Statement: (a) Controlled by: (b) Distribution: Public Restricted Confidential (c) Copy to NTIS: YES / NO

Descriptors (keywords): Real Time Simulation, New Airspace & Structure organisation, MUAC, DFS, LvNL

Abstract:

Page intentionally left blank

AMRUFRA Real-time Simulation Validation Report EUROCONTROL

AMRUFRA project – EEC Report No. 409 v

EDITION HISTORY

Edition No. Effective date or status Author(s) Reason

0.1 20/06/2008 Renée Schuen-Medwed First raw draft

0.2 18/07/2008 Renée Schuen-Medwed First advanced draft delivery (internal)

0.3 19/08/2008 Renée Schuen-Medwed

Angelo Scorciarino

Michel Geissel

Implementation of comments

Comments from Herbert Koppe

0.4 27/08/2008 Michel Geissel Document editing

Implementation of Herbert Koppe's new comments


Angelo Scorciarino

Michel Geissel

New document elements

MUAC Deco & Ruhr analysis

Comments from Herbert Koppe

Appendix


Angelo Scorciarino

Michel Geissel

General document review:

- English native speaker

- AMRUFRA Working Group comments 19/09

1.1 30/09/2008 Michel Geissel Final comments / document references

FINAL VERSION

1.2 03/10/2008 Michel Geissel Last Herbert Koppe comments

EUROCONTROL AMRUFRA Real-time Simulation Validation Report

vi AMRUFRA project – EEC Report No. 409



AMRUFRA project – EEC Report No. 409 vii

DISTRIBUTION LIST

EEC Internal Distribution List

Pierre Andribet Core Business Manager

Jean Paul Zabka Short-Term Support Impl. Manager

Eric Hoffman ATC Research Area Manager

Bernard Brunner Deputy ATC Research Area Manager

José Seixo Real-time Simulation Facility Manager

Michel Geissel Simulation Project Manager

Angelo Scorciarino Simulation Operational Leader

Marie-Pierre Balloy Simulator Technical Coordinator

Renée Schuen-Medwed Validation Expert

Sophie Gillet STC Deputy

Adrian Gizdavu Operational Expert

Pieter Slingerland Operational Expert

Marie Christine Leduc Simulation Data Preparation

Pierrick Pasutto HMI Project Leader

Mickael Dubreuil STERIA HMI Expert

EEC External Distribution List

Jean-Marie Leboutte (MUAC) Maastricht UAC AOM

Herbert Koppe (MUAC) Project Manager

Nick Schede (MUAC) Ops expert

Rene Coffa (MUAC) Ops expert

Stephen Mohammed (MUAC) Ops expert

Herbert Funk (DFS) Project Leader

Ephraim Volker (DFS) Ops expert

Werner Huck (DFS) Ops expert

Roland Knospe (DFS) Ops expert

Thomas Straub (DFS) Ops expert

Ed Hanenberg (LvNL) Project Leader

Niels Van Zuylen (LvNL) Ops expert

Ron Slootbeek (LvNL) Ops expert

Peter Roben (LvNL) Ops expert

Victor van Kempen (RNLAF) Military ops expert

Jeroen Kroese (RNLAF) Military ops expert


viii AMRUFRA project – EEC Report No. 409



AMRUFRA project – EEC Report No. 409 ix

APPROVAL

Part People Responsibility Visa

Jean Paul Zabka Short-Term Support Implementation Manager

Eric Hoffman ATC Research Area Manager

EEC EUROCONTROL Approvals

Michel Geissel Simulation Project Manager

Jean-Marie Leboutte MUAC/UAC AOM

Herbert Koppe MUAC/Project Manager

Herbert Funk DFS Langen/Project Leader

Ed Hanenberg LvNL/Project Leader

Victor van Kempen RNLAF/Project Leader

Client Working Group Approvals

Roland Knospe Maastricht DFS LIPPE RADAR /Project Leader


x AMRUFRA project – EEC Report No. 409



AMRUFRA project – EEC Report No. 409 xi

EXECUTIVE SUMMARY

A real-time experiment was conducted at the EEC, Brétigny, from 7 to 25 April 2008, to validate the benefits of the new proposed routes and airspace structure designed to cope with the increase in traffic demand from 2009 to 2013, especially within the airspace between the two major airports of Amsterdam and Frankfurt.

The simulated airspace was included in the FABEC initiative and covered the area controlled by LvNL/Amsterdam (2E and 2I sectors), DFS/Langen (NOR, MGB and TAU sectors) and Maastricht UAC (Deco and Ruhr sectors), including both civil and military operations (DFS Maastricht, hereafter referred as Lippe Radar and Dutch Military). The Project was called AMRUFRA (Amsterdam-Ruhr-Frankfurt Interface). The aim of the AMRUFRA RTS was to assess the airspace solutions proposed to overcome delays and overload created by the forecast traffic increase with the current route structure. The solutions for DFS Langen, hereafter referred as DFS, were (1) to change the sector volume and shape, (2) to change the location of the Frankfurt arrival and departure routes, and (3), as a consequence, to spread the Düsseldorf routes to the south. For LvNL it was proposed to (1) expand the airspace to the south and (2) segregate the Amsterdam departure stream by creating a new route. While DFS Langen and LvNL were the initiating stakeholders, the Dutch and German military ATS provider, responsible for Operational Air Traffic and MUAC were the stakeholders affected by the changes to the route network. Lippe Radar and Dutch Mil were not directly measured, but their feedback was collected.

The objectives of the Simulation were to:

1. Assess the impact of the new route structure and airspace structure on capacity.

2. Assess the impact of the new route structure and airspace structure on acceptability.

3. Assess the impact of the new route structure and airspace structure on safety.

4. Assess the impact of the new route structure and airspace structure on the route length.

5. Assess the feasibility of the new route and airspace structure with a traffic increase.

To meet the objectives, the AMRUFRA Project validation was organised over a three-week period as follows:

• Over the two first weeks, the new organisation (ORG1) was compared with the reference organisation (ORG0).

• At the end of the second week, and to ensure maximum benefits from the simulation, the AMRUFRA Working Group decided to optimise the airspace design (due to internal DFS problem). A number of improvements were implemented to resolve civil-military issues relating to the Volkel area and in the TRA12A, and with coordination procedures between 2E and MGB.

• In the third week, exercises were run with a traffic increase of up to 20% (ORG2). Two demonstration exercises were also introduced to help civil and military discussions relating to air tanker issues in the Ruhr sector (JOJO track).

• To further improve this new airspace structure, the AMRUFRA Working Group decided to validate new solutions to resolve other operational issues concerning military descents to ETAD/ETAR, convergence between Frankfurt arrivals and Amsterdam departures, and NOR sector overload. Three exercises were dedicated to testing the associated solutions.


xii AMRUFRA project – EEC Report No. 409

The results and findings of the AMRUFRA RTS were based on the controllers' subjective opinions expressed during the simulation in post-exercise briefings and questionnaires, on the analysis of subjective data relating to workload and situational awareness, and on analyses of objective data recorded during the simulation exercises.

The new route and airspace structure was not acceptable to DFS controllers. The feedback from the controllers was explained and confirmed by a significant increase in objective task load and subjective workload within the new route and airspace structure. This impact was partly attributable to the redesigned NOR sector, which resulted in more traffic to be handled but also in an increase in conflict points and therefore complexity. The military activity and the increased frequency load were also mentioned as safety concerns. The splitting of the NOR sector into DKAR and NOR in organisation 2, which was planned to accommodate the traffic increase, was not beneficial, since the resized sectors resulted in time pressure on the controller in order to solve the conflicts. Furthermore, the new sector design did not reduce coordination, since the majority of the traffic was common to both sectors, DKAR and NOR.

For LvNL, the expected benefits were partly confirmed. The new route and airspace structure was acceptable to the controllers, since the routes split "naturally" separated the traffic flows. However, controllers mentioned that the new route structure raised new safety concerns linked to the Volkel area. No safety concerns regarding the new Amsterdam departure routes were highlighted. The Volkel area was changed in organisation 2 (week 3), and this made the new route and airspace structure acceptable to the controllers. Moreover, it was stated that it would not be feasible to work with the increased amount of traffic within the current route structure.

The majority of MUAC controllers rated the new route and airspace structure as acceptable. A key argument in favour of accepting the design was the de-confliction of the Frankfurt arrival route and the Amsterdam departure route. Additionally, military window 1 had a less negative impact in the new route structure, which required fewer heading instructions from the controllers. Furthermore, the new route and airspace structure was acceptable even with the traffic increase (ORG2). The descriptive data indicated a tendency towards shorter and therefore more efficient routes for the DDH and the Ruhr sector in organisation 1 compared to organisation 0.

The simulated route and airspace structure was originally unacceptable for the military users. The optimalisation of the Airspace Design at the end of the week two led to better handling of military traffic and therefore to the approval of the military units.

Finally, safety concerns were eliminated and improvements were simulated at the end of the simulation study as organisation 4. Nevertheless, further studies are needed, especially in order for DFS to validate these possible improvements.


AMRUFRA project – EEC Report No. 409 xiii

ACKNOWLEDGEMENTS

The AMRUFRA Real-time Simulation Project Manager would like to thank all the controllers, expert group members and engineers, and the Language-Service, technical and administrative staff who contributed to the simulation.

All the participants displayed a high level of professionalism and enthusiasm. Their efforts made the simulation and this report possible.

The AMRUFRA Project Manager would like to thank MUAC, DFS, LvNL and the military representatives (Lippe Radar and Dutch Military) for making their controllers available for the training and for the simulation activities of the AMRUFRA RTS simulation.

The authors of the present report would also like to thank the staff of the CRDS for their support, and the pilot staff who worked with us during this simulation.

The AMRUFRA RTS Validation Report is going to be published on the EEC website:

http://www.eurocontrol.int/eec/public/standard_page/eec_reports_2008.html


xiv AMRUFRA project – EEC Report No. 409



AMRUFRA project – EEC Report No. 409 xv

TABLE OF CONTENTS

EDITION HISTORY ............................................................................................................ V

DISTRIBUTION LIST........................................................................................................ VII

APPROVAL....................................................................................................................... IX

EXECUTIVE SUMMARY................................................................................................... XI

ACKNOWLEDGEMENTS ............................................................................................... XIII

1. INTRODUCTION...........................................................................................................1 1.1 SCOPE AND STRUCTURE OF THIS DOCUMENT ...................................................... 1 1.2 PURPOSE OF THIS DOCUMENT................................................................................. 1 1.3 CONTEXT ...................................................................................................................... 2 1.4 OPERATIONAL BACKGROUND ................................................................................... 2

2 EXPECTED EXPERIMENT OUTCOMES, OBJECTIVES AND HYPOTHESES ..........5 2.1 DESCRIPTION OF EXPECTED EXPERIMENT OUTCOMES ...................................... 5 2.2 DESCRIPTION OF EXPERIMENT OBJECTIVES ......................................................... 5 2.3 DESCRIPTION OF EXPERIMENT HYPOTHESES....................................................... 6

3 CHOICE OF METRICS AND MEASUREMENTS .........................................................9 3.1 SITUATIONAL AWARENESS........................................................................................ 9 3.2 CAPACITY ..................................................................................................................... 9

3.2.1 Workload ...........................................................................................................9 3.2.2 Task Load........................................................................................................10

3.3 SAFETY ....................................................................................................................... 10 3.4 ROUTE LENGTH ......................................................................................................... 10 3.5 CONTROLLER ACCEPTABILITY................................................................................ 11

4 METHODS AND TECHNIQUES SELECTED .............................................................12 4.1 DATA RECORDINGS .................................................................................................. 12 4.2 QUESTIONNAIRES ..................................................................................................... 12 4.3 DEBRIEFINGS ............................................................................................................. 12

5 SIMULATION ENVIRONMENT...................................................................................13 5.1 ORGANISATION SPECIFICATIONS........................................................................... 13

5.1.1 ORG0 and ORG0A..........................................................................................13 5.1.2 ORG1 and ORG1A..........................................................................................13 5.1.3 ORG2 and ORG2A..........................................................................................16

5.2 EXPERIMENTAL VARIABLES AND DESIGN ............................................................. 17 5.2.1 Traffic...............................................................................................................18 5.2.2 Controller Role (EC/PC) ..................................................................................18 5.2.3 Organisations ..................................................................................................19

6 CONDUCT OF VALIDATION EXERCISE RUNS........................................................20 6.1 EXPERIMENT PREPARATION ................................................................................... 21

6.1.1 Training and Acceptance Test.........................................................................21


xvi AMRUFRA project – EEC Report No. 409

6.1.2 Simulation Plan................................................................................................22 6.2 CHANGES IN THE SIMULATION PROGRAMME AND SCHEDULE.......................... 23 6.3 PARTICIPANTS ........................................................................................................... 25 6.4 STATISTICAL ANALYSIS ............................................................................................ 26

7 DFS – EXPERIMENT RESULTS ................................................................................27 7.1 ASSESS THE IMPACT OF THE NEW ROUTE STRUCTURE AND AIRSPACE

STRUCTURE ON CAPACITY...................................................................................... 27 7.1.1 Task Load........................................................................................................28 7.1.2 Workload .........................................................................................................32

7.2 ASSESS THE IMPACT OF THE NEW ROUTE STRUCTURE AND AIRSPACE STRUCTURE ON ACCEPTABILITY............................................................................ 34

7.3 DFS RESULTS FOR OBJECTIVE 2 ............................................................................ 35 7.3.1 Short-term Conflict Alerts (STCAs)..................................................................35 7.3.2 Situational Awareness .....................................................................................35 7.3.3 Controller Feedback ........................................................................................36

7.4 ASSESS THE FEASIBILITY OF THE NEW ROUTE AND AIRSPACE STRUCTURE WITH A TRAFFIC INCREASE (ORGANISATION 2) ................................................... 37

8 LVNL – EXPERIMENT RESULTS ..............................................................................41 8.1 ASSESS THE IMPACT OF THE NEW ROUTE STRUCTURE AND AIRSPACE

STRUCTURE ON CAPACITY...................................................................................... 41 8.1.1 Task load .........................................................................................................42


8.3 ASSESS THE IMPACT OF THE NEW ROUTE STRUCTURE AND AIRSPACE STRUCTURE ON SAFETY.......................................................................................... 44 8.3.1 Short-term Conflict Alerts (STCAs)..................................................................44 8.3.2 Controller Feedback ........................................................................................45

8.4 ASSESS THE FEASIBILITY OF THE NEW ROUTE AND AIRSPACE STRUCTURE WITH A TRAFFIC INCREASE (ORGANISATION 2) ................................................... 46

9 MUAC – EXPERIMENT RESULTS.............................................................................49 9.1 ASSESS THE IMPACT OF THE NEW ROUTE STRUCTURE AND AIRSPACE

STRUCTURE ON CAPACITY...................................................................................... 49 9.1.1 Task load .........................................................................................................50 9.1.2 Workload .........................................................................................................51


9.3 ASSESS THE IMPACT OF THE NEW ROUTE STRUCTURE AND AIRSPACE STRUCTURE ON SAFETY.......................................................................................... 54 9.3.1 Short-term Conflict Alerts (STCAs)..................................................................54 9.3.2 Controller Feedback ........................................................................................54

9.4 ASSESS THE FEASIBILITY OF THE NEW ROUTE AND AIRSPACE STRUCTURE WITH A TRAFFIC INCREASE (ORGANISATION 2) ................................................... 55 9.4.1 Controller Feedback ........................................................................................55

10 ASSESS THE IMPACT OF THE NEW ROUTE STRUCTURE AND AIRSPACE STRUCTURE ON THE ROUTE LENGTH...................................................................57

11 FEEDBACK FROM THE MILITARY UNITS ...............................................................61


AMRUFRA project – EEC Report No. 409 xvii

12 CONCLUSIONS..........................................................................................................63 12.1 DFS .............................................................................................................................. 63 12.2 LVNL ............................................................................................................................ 63 12.3 MUAC........................................................................................................................... 64 12.4 MILITARY UNITS ......................................................................................................... 65 12.5 GENERAL CONCLUSION ........................................................................................... 65

13 RECOMMENDATIONS ...............................................................................................67 13.1 SIMULATED IMPROVEMENTS................................................................................... 67 13.2 FURTHER RECOMMENDATIONS.............................................................................. 68

LIST OF APPENDICES

APPENDIX ........................................................................................................................69 APPENDIX A.1 PLANNED EXERCISE SCHEDULE............................................................... 71 APPENDIX A.2 GRAPHS TRA 12A ACTIVE VERSUS TRA 12A INACTIVE.......................... 72 APPENDIX A.3 STATISTICAL VALUES OF THE ANOVAS ................................................... 76 APPENDIX A.4 SASHA QUESTIONNAIRE............................................................................. 77 APPENDIX A.5 NASA-TLX QUESTIONNAIRE ....................................................................... 79 APPENDIX A.6 DFS QUESTIONNAIRES ............................................................................... 81 APPENDIX A.7 LVNL QUESTIONNAIRES ............................................................................. 83 APPENDIX A.8 MUAC QUESTIONNAIRES............................................................................ 85 APPENDIX A.9 ASSESSMENT OF TTA CONCEPT USING TASK ANALYSES.................... 88 APPENDIX A.10 ORGANISATION QUESTIONNAIRES : LVNL FEEDBACK RESULTS....... 94 APPENDIX A.11 ORGANISATION QUESTIONNAIRES : MUAC FEEDBACK RESULTS ... 101 APPENDIX A.12 OBJECTIVES, VALIDATION METHODS, TOOLS AND HYPOTHESES .. 110 APPENDIX A.13 SIMULATION ORGANISATION OPERATIONAL UPDATES .................... 112

LIST OF FIGURES

Figure 5-1: ORG0(A) and ORG1(A), showing the Frankfurt departure route..................................................14 Figure 5-2: ORG0(A) and ORG1(A), showing the Frankfurt arrival route .......................................................15 Figure 5-3: ORG0(A) and ORG1(A), showing the Amsterdam departure route..............................................16 Figure 7-1: DFS – telephone calls by the PC ..................................................................................................28 Figure 7-2: DFS – time spent on frequency.....................................................................................................29 Figure 7-3: Radio contacts ..............................................................................................................................29 Figure 7-4: DFS – instructions.........................................................................................................................30 Figure 7-5: Number of aircraft on frequency ...................................................................................................31 Figure 7-6: EC workload (ISA).........................................................................................................................32 Figure 7-7: PC workload (ISA).........................................................................................................................32 Figure 7-8: NASA overall EC...........................................................................................................................33 Figure 7-9: NASA overall PC...........................................................................................................................33 Figure 7-10: Acceptability part I.......................................................................................................................34 Figure 7-11 Acceptability part II.......................................................................................................................34 Figure 7-12: Situational awareness – DFS......................................................................................................36 Figure 7-13: Safety concerns – DFS ...............................................................................................................36


xviii AMRUFRA project – EEC Report No. 409

Figure 7-14: Acceptability of organisation 2 ....................................................................................................38 Figure 7-15: Acceptability of the NOR split......................................................................................................38 Figure 7-16: Feedback on the splitting of the NOR sector ..............................................................................38 Figure 7-17: Feedback on problems created ..................................................................................................38 Figure 7-18: Number of telephone calls in the DKAR sector...........................................................................39 Figure 7-19: Number of telephone calls in the NOR sector..............................................................................39 Figure 7-20: Number of telephone calls in the MGB sector .............................................................................39 Figure 8-1: Heading instructions......................................................................................................................42 Figure 8-2: Acceptability of the eastbound route split .....................................................................................43 Figure 8-3 Acceptability of the new sector shape............................................................................................43 Figure 8-4: Number of STCAs (sector 2E) ......................................................................................................44 Figure 8-5: LvNL safety concerns....................................................................................................................45 Figure 8-6: Acceptability of route split with traffic increase .............................................................................46 Figure 8-7: Acceptability of new sector shape with traffic increase.................................................................46 Figure 8-8: Problems created by the traffic increase.......................................................................................46 Figure 8-9: Acceptability of new RENDI area..................................................................................................46 Figure 9-1: Heading instructions......................................................................................................................50 Figure 9-2: First-order interaction between traffic and organisation for direct instructions .............................51 Figure 9-3: First-order interaction between traffic and organisation: NASA-TLX overall score for the PC ....52 Figure 9-4: Acceptability part I .........................................................................................................................53 Figure 9-5: Acceptability part II ........................................................................................................................53 Figure 9-6: MUAC Hannover sectors: safety concerns ...................................................................................54 Figure 9-7: MUAC Deco sectors: safety concerns ..........................................................................................54 Figure 9-8: Validity of the benefits with the traffic increase for the Deco sectors ...........................................55 Figure 9-9: Problems in organisation 2 for the Deco sectors ..........................................................................55 Figure 9-10: Validity of the benefit with the traffic increase for the Hannover sectors ....................................56 Figure 9-11: Problems in organisation 2 for the Hannover sectors.................................................................56 Figure 10-1: Distance flown in the 2E sector in organisation 0 .......................................................................57 Figure 10-2: Distance flown in the 2E sector in organisation 1 .......................................................................57 Figure 10-3: Distance flown in the 2I sector in organisation 0.........................................................................57 Figure 10-4: Distance flown in the 2I sector in organisation 1.........................................................................57 Figure 10-5: Distance flown in the DDL sector in organisation 0 ....................................................................58 Figure 10-6: Distance flown in the DDL sector in organisation 1 ....................................................................58 Figure 10-7: Distance flown in the DDH sector in organisation 0....................................................................58 Figure 10-8: Distance flown in the DDH sector in organisation 1....................................................................58 Figure 10-9: Distance flown in the DDH sector in organisation 0.....................................................................58 Figure 10-10: Distance flown in the DDH sector in organisation 1...................................................................58 Figure 12-1: Workload (ISA) first-order interaction between TRA 12A active/inactive and the traffic sample72 Figure 12-2: Telephone calls made by the PC when TRA 12A was active and when TRA 12A was inactive73 Figure 12-3: 12-4 Number of frequency contacts; first-order interaction between TRA A active/inactive

and the traffic sample ................................................................................................................73 Figure 12-5: Time spent on frequency: effect of TRA 12A active versus TRA 12A inactive...........................74 Figure 12-6: PC Situational awareness when TRA 12A was active and when TRA 12A was inactive ..........74 Figure 12-7: NASA-TLX frustration – EC: first-order interaction between TRA 12A active/inactive

and the traffic sample .................................................................................................................75


AMRUFRA project – EEC Report No. 409 xix

LIST OF TABLES

Table 10-1: Length of the routes in the old route structure compared with the new route structure...............59 Table 13-1: Improvements in the route structure/ORG2 .................................................................................67 Table 13-2: Improvements in the route structure/ORG 4 ................................................................................68 Table 13-3: Simulation week 1 schedule.........................................................................................................71 Table 13-4: Simulation week 2 schedule.........................................................................................................71 Table 13-5: Simulation week 3 schedule.........................................................................................................72 Table 13-6: Statistical values...........................................................................................................................76 Table 13-7: Validation objectives and tools...................................................................................................110


xx AMRUFRA project – EEC Report No. 409



AMRUFRA project – EEC Report No. 409 xxi

REFERENCES

[1] DFS, Entwicklung Luftraum CC Langen; Niederlassung Mitte CC/F-M" Version 1.0, DFS Deutsche Flugsicherung GMbH, Center Langen, 2007.

[2] Koninklijke Luchtmacht, Ministry of Defence, Netherlands (2007), Proposal document for extension of the shaded area in the EHAA FIR for implementing a new route network in the Amsterdam-Ruhr-Frankfurt Interface Project, Version 1.

[3] EUROCONTROL (2007), AMRUFRA Project Management Plan, EUROCONTROL Experimental Centre, Brétigny.

[4] EUROCONTROL (2007), E-OCVM: European Operational Concept Validation Methodology, Version 2, EUROCONTROL Headquarters, Brussels.

[5] EUROCONTROL (2005), Single European Sky (SES) Regulations; EUROCONTROL Final Report on European Commission's Mandate : to Support the Establishment of Functional Airspace Blocks (FABs), EUROCONTROL Headquarters, Brussels.

[6] EUROCONTROL (2007);AMRUFRA Simulation Operational specification Document V0.1, EUROCONTROL Experimental Centre, Brétigny.

[7] EUROCONTROL (2007), AMRUFRA Simulation Technical Specification Document, V0.1, EUROCONTROL Experimental Centre, Brétigny.

[8] Endsley, M.R (1987b), SAGAT: A methodology for the measurement of situation awareness (NOR DOC 87-83), Hawthorne, CA: Northrop Corp.

[9] Endsley, M.R (1988b), "Situation Awareness Global Assessment Technique (SAGAT)", in Proceedings of the National Aerospace and Electronics Conference (pp. 789-795), New York: IEEE.

[10] EUROCONTROL, Situational Awareness for SHAPE Questionnaire, http://www.eurocontrol.int/humanfactors/gallery/content/public/docs/SHAPE_questionnaires/SASHA.pdf

[11] Hart, S.G. and Staveland, L.E. (1988), "Development of NASA-TLX: Results of Empirical and Theoretical Research", in P.A. Hancock, and N. Meshkati (eds.), Human Mental Workload, pp. 143-154, New York NY: Plenum.

[12] EUROCONTROL (2008), AMRUFRA Validation Strategy, EEC, Brétigny.

[13] EUROCONTROL(2008), AMRUFRA Metrics and Recordings Requirements Documents.

[14]. SPSS (2006), SPSS Advanced Models™ 15.0 Manual, Chicago, IL: SPSS.


xxii AMRUFRA project – EEC Report No. 409



AMRUFRA project – EEC Report No. 409 xxiii

ACRONYMS

ACC Area control centre

AMRUFRA Amsterdam-Ruhr-Frankfurt

ANOVA Analyses of variance

ANSP Air navigation service provider

ASD Airspace design

ATCO Air traffic control officer

DDH DECO Delta High

DDL DECO Delta Low

DFH DECO Flevo High

DFS Deutsche Flugsicherung GmbH [German Air Traffic Services]

DKAR Köln Arrival West

DM Dutch Military

DMEAN Dynamic Management of European Airspace

DZH DECO Zeeland High

EC Executive controller

EEC EUROCONTROL Experimental Centre

E-OCVM European Operational Concept Validation Methodology

FAB Functional airspace block

GLM General linear model

HRH Hannover Ruhr High

HRL Hannover Ruhr Low

LR Lippe Radar, DFS Maastricht

LV Luftverteidigung (air defence)

LvNL Luchtverkeersleiding Nederland [Air Traffic Control the Netherlands]

MIL ATCC Military air traffic control centre

MGB Mönchengladbach sector

MUAC Maastricht Upper Area Control Centre

NOR Nörvenich sector

PC Planning controller

RFL Requested flight level

RR Ruhr sector

SES Single European Sky

SESAR Single European Sky ATM Research Programme

TAU Taunus sector

TRA Temporary reserved area

XFL Exit flight level


xxiv AMRUFRA project – EEC Report No. 409



AMRUFRA project – EEC Report No. 409 1

1. INTRODUCTION

1.1 SCOPE AND STRUCTURE OF THIS DOCUMENT

This Validation Report corresponds to step 4.3 of the European Operational Concept Validation Methodology (E-OCVM) [4]. It describes the results of a real-time simulation (RTS) performed by and for three different air navigation service providers (ANSP) at the EUROCONTROL Experimental Centre (EEC) in Brétigny.

The document is structured as follows:

• Section 1: provides an introduction describing the structure and purpose of this document, and the context of and operational background to the simulation conducted.

• Section 2: presents the expected outcomes of the experiments as well as the objectives and hypotheses.

• Section 3: describes the metrics and measures selected for use in this real-time simulation.

• Section 4: presents the methods and techniques selected.

• Section 5: outlines the simulation environment.

• Section 6: describes how the validation exercise runs were conducted.

• Section 7: describes the experiment results for DFS.

• Section 8: describes the experiment results for LvNL.

• Section 9: describes the experiment results for MUAC.

• Section 10: describes common results relating to efficiency and the environment.

• Section 11: outlines the feedback from Lippe Radar and Dutch Mil.

• Section 12: presents the conclusions.

• Section 13: outlines the recommendations.

1.2 PURPOSE OF THIS DOCUMENT

This document presents the final report on the results of the AMRUFRA (Amsterdam-Ruhr-Frankfurt Interface) real-time simulation (RTS). The AMRUFRA Project addresses the needs of the three air navigation service providers (ANSPs) in coping with the increase in civil traffic demand from 2009 to 2013 (LvNL (Luchtverkeersleiding Nederland), DFS (Deutsche Flugsicherung) and MUAC (Maastricht Upper Area Control Centre)). Changes in the airspace of individual ANSPs, allowing increase of traffic handling, could have an influence on neighbouring sectors, States and ANSPs. Amsterdam (LvNL) and Langen (DFS) are planning changes in their airspace and route structures, which will have an influence on Maastricht (MUAC).The goal of the AMRUFRA RTS was to validate the interface between the three ANSPs with the integrated route and airspace changes, and to give the military units an opportunity to see if the military needs are fulfilled. The AMRUFRA (Amsterdam-Ruhr-Frankfurt Interface) simulation took place from 7 to 25 April 2008 at the EEC in Brétigny. The outcome of the simulation should support the decision-makers in their next steps regarding the implementation of the proposed changes.


2 AMRUFRA project – EEC Report No. 409

1.3 CONTEXT

The FABEC (Belgocontrol, DFS (Germany), DSNA (France), MUAC (Maastricht), LAA (Luxembourg), LvNL (the Netherlands), skyguide (Switzerland) and their military partners) is set to become one pillar of the Single European Sky (SES). The feasibility study for FABEC is currently in progress and will demonstrate the following benefits of the FABEC [5]:

• a mechanism to cope with the forecast traffic increase;

• optimised use of airspace, considering traffic flows;

• optimised use of technical and human resources;

• cost-efficient service;

• improved civil-military cooperation. To reach these goals, the three ANSPs must harmonise and coordinate their operations. Additionally, the procedures must facilitate the most efficient usage for civil and military users. This concept will:

• be supported by EUROCONTROL's Dynamic Management of the European Airspace Network (DMEAN) programme;

• fulfil the specific requirements of the core area within the framework of SESAR up to 2020;

• harmonise the safety competence of all ANSPs and improve the already high level of safety;

• focus on the interface with other countries and FABs, and, with the help of a systematic technical roadmap, introduce stepwise common functions, interoperable technical systems and common services.

The AMRUFRA Project is not a direct part of the Functional Airspace Block Europe Central (FABEC) programme, but the outcome of the Project constitutes an input into FABEC, since the AMRUFRA Project will contribute to FABEC as a baseline scenario to allow performance measurement of FABEC.

1.4 OPERATIONAL BACKGROUND

The scope of the AMRUFRA Project is focused on the identification of an improved airspace structure to deal with the future growth of air traffic, especially within the airspace between the two major airports of Amsterdam (EHAM) and Frankfurt (EDDF). In order to achieve a short- to medium-term solution, a new airspace structure has been designed by all participating ANSPs to cope with future traffic. Involved in the Project are the ANSPs EUROCONTROL (MUAC), LvNL and DFS, and the military units DM (Dutch Military) and LR (Lippe Radar). The following section outlines the problems of and the proposed changes to the airspace structure in the AMRUFRA Project for each stakeholder. DFS and LvNL constitute the two stakeholders initiating the changes in the route network.



Langen (DFS)

The airspace for which Langen ACC is responsible faces the two problems set out below.

1. Existing problem

In March 2006 the capacity of Düsseldorf Airport was increased. This caused often a four minute departure sequence in the morning rush hour (6.30-8.30 local time). This restriction on departing traffic has resulted in significant delays, and controllers have increasingly needed to perform tactical control to de-conflict the traffic and coordination with the adjacent sectors. This has resulted in high task load and workload.

In order to overcome this problem, the following solutions are proposed:

• Change the location of the Frankfurt arrival and departure routes.

• Spread the Düsseldorf routes to the south-east as a consequence.

This change should locate the conflict points in one sector, which would require less coordination and facilitate strategic control and easier planning.

2. Anticipated problem

In 2011, the capacity of Frankfurt Airport will be increased as a result of a newly built runway. At the moment, it is not known if the proposed change to the Frankfurt arrival and departure routes is also a suitable solution for coping with this additional traffic increase.

Amsterdam (LvNL)

In the current route structure, all departure flights use the same ATS route. Outbound flights from Amsterdam and the general traffic increase since 2002 mean that this route is saturated. With the expected future traffic increase, this will lead to constraints regarding traffic departing from Amsterdam (delays), as the current route structure does not offer the required space for the increased need of vectoring and rerouting the aircraft. This will lead to complex traffic handling, high workload and trade-offs in efficiency.

The following solutions are proposed:

• expansion of airspace;

• segregation of the traffic streams. The newly created route is expected to accommodate approximately 55-70% of Amsterdam sector 2 outbound flights.

The inbound sector might be affected positively by the new route and the expansion of the airspace, since the traffic is currently rerouted close to and sometimes into the inbound sector. The implementation of the new route will require fewer tactical interactions by ATCOs than in the current situation, in which ATCOs must reroute aircraft individually by giving headings to ensure separation.

While DFS and LvNL are the initiating stakeholders, Dutch Military/Lippe Radar and MUAC are the stakeholders affected by the changes to the route network.



MIL ATCC (Dutch Military and Lippe Radar)

The proposed new route network in the Amsterdam-Ruhr-Frankfurt Interface Project has a major impact on the existing airspace structure between Amsterdam CTA East and TRA12(A). Current legislative and coordination procedures for implementing the new proposed route network resulting from the Amsterdam-Ruhr-Frankfurt Interface Project are insufficient. Without changing the current airspace structure, the new proposal based on the Project cannot be implemented. The proposed solution involves changing the military exercise area; the military area is therefore simulated in order to be sure that military activities can be conducted in accordance with needs. [2]

MUAC

The proposed route changes in LvNL and DFS have an impact on Maastricht. The new organisation might solve the conflict points in the Ruhr sector but could shift the conflict area to the Deco sector, e.g. Frankfurt arrivals and departures currently intersect in the Ruhr sector, often leading to complex situations. Under the new organisation, these flows are well segregated. In the Deco sector, however, the changes have a different impact. Today, where Frankfurt departures cross with Luton and Stansted arrivals, the routes are separated vertically, with the Luton and Stansted flights on a higher level. However, when the crossing point of the routes is shifted (in the Deco sector), these flights have already started to descend to a lower flight level, which results in more complexity caused by more vertical crossings.

Additionally, owing to the splitting of Amsterdam departures and arrivals in the new organisation, the route for Frankfurt departures has been shifted further to the south. Depending on the status of TRA12A, the splitting of Frankfurt departures overflying London TMA on the way to destinations beyond may be further away from the airport. This might lead to congestion downstream, as Frankfurt departures can only be split in the Deco sector (BREDA) if TRA12A is active.

It is expected that the workload in the Maastricht Ruhr sector will decrease as a result of the proposed changes, but in the Deco sectors the changes might lead to an increase in workload.

The aim of the real-time simulation was to validate the balance between the benefits and trade-offs of the proposed solutions. These solutions were validated in the AMRUFRA RTS, not only focusing on increased capacity but also bearing in mind the efficiency of the solutions for airspace users, the environmental impact, and safety issues, in order to ensure that the changes were acceptable and that there was no decrease in the safety level.



2 EXPECTED EXPERIMENT OUTCOMES, OBJECTIVES AND HYPOTHESES

2.1 DESCRIPTION OF EXPECTED EXPERIMENT OUTCOMES

Regarding the operational background above, each of the three ANSPs expected different outcomes from the experiment. While LvNL expected the route splitting and the airspace expansion to yield a 12% capacity gain and reduce delays in the area, DFS expected the change of route structure to allow it to manage the traffic without a decrease in capacity and with reduced delays in both the Düsseldorf and Frankfurt areas. Maastricht UAC, whose area of responsibility extends above both Langen and LvNL, was confronted with the requests for change from its partners in the airspace beneath it. Although MUAC envisaged a marginal decrease in the current workload in the Ruhr sector as a result of the new airspace design, and a slight increase in workload in the Deco sector, it agreed to proceed with the project in order to allow benefits for the European network.

Lastly, the military airspace users expected the necessary arrangements for their requirements to be accommodated and training to remain possible.

2.2 DESCRIPTION OF EXPERIMENT OBJECTIVES

The main aim of the AMRUFRA RTS was to validate the benefits of the proposed new route and airspace structure. As stated in section 1.4, increases in airport and airspace capacity, resulting inter alia from new runways, were expected in the near future for the Amsterdam-Ruhr-Frankfurt airspace interface. The new route and airspace structure was designed to cope with this traffic increase. These changes were assessed against the current route and airspace structure. The simulation validated not only the benefits expected from the changes in capacity, but also the benefits in terms of efficiency for the airspace users and efficiency from the air traffic controller's perspective. The air traffic controllers' assessment of the acceptability of the new route and airspace changes indicated that the changes had resulted in greater efficiency for them. The implemented changes were also assessed in terms of their impact on safety, in order to be in line with the requirement not to decrease the level of safety. The following project validation objectives were formulated:






These project validation objectives were broken down into exercise validation objectives. The exercise validation objectives formulated measurable elements. Table 2-1 outlines the exercise validation objectives related to the project validation objectives.



Table 2-1: Project validation objectives linked to exercise validation objectives part I

Project validation objectives

Exercise validation objectives

Investigate the effect of the route and airspace changes on the controllers' traffic-handling capability.

Investigate the effect of the route and airspace changes on R/T usage.

Investigate the effect of the route and airspace changes on telephone coordination.


Investigate the effect of the route and airspace changes on intra-centre SYSCO usage (where applicable).


Investigate the controllers' perceptions of the acceptability of the changed airspace and routes.

Investigate the effect of the route and airspace changes on the number of STCAs and losses of separation.


Investigate the effect of the route and airspace changes on the level of situational awareness.


Investigate the effect of the route and airspace changes on route length.


Investigate the acceptability of the new route and airspace structure with the traffic increase.

2.3 DESCRIPTION OF EXPERIMENT HYPOTHESES

The hypotheses were formulated with reference to the problem description and the analyses of the proposed solution. The Amsterdam departure route was split into two traffic flows which made it possible to accommodate more aircraft. Furthermore, the new route structure de-conflicted the traffic. The additional Amsterdam route therefore resulted in fewer potential conflicts, allowing the ATCO to perform less tactical conflict resolution. Because of the shifted Amsterdam route, the Frankfurt arrival and departure routes were relocated. The new airspace shape and route structure in DFS airspace enabled conflicts to be resolved with the involvement of fewer sectors. The NOR and MGB sectors were reshaped in such a way that the Düsseldorf departure sector handed over airspace to them. This meant that the NOR and MGB sectors needed to perform less coordination. Additional silent transfers facilitated by letters of agreement (LoAs) were designed to ease the ATCOs' work. These facts were supposed to decrease the controllers' workload and therefore to increase handling capability, i.e. capacity. This allowed the benefits and hypotheses set out below to be formulated.

Expected Benefits

It was expected that the new route (here referred to as ORG1(A)) would allow controllers in LvNL airspace to give fewer tactical control instructions to aircraft; these controllers would also, therefore, perform less R/T usage compared to the baseline (referred to as ORG0). With regard to the changes in DFS airspace, less coordination would be performed in ORG1(A).

This made it possible to formulate the following hypotheses for LvNL and DFS:



Hypothesis 1: TASK LOAD

Controllers will need to perform fewer tasks with the new route and airspace structure in ORG1(A).

The term "tasks" refers to the interventions set out below.

Tactical interventions:

• Heading instructions.

• Flight level change instructions.

• Communication tasks.

• R/T communication.

• Telephone coordination/communication.

The thought that led to Hypothesis No. 1, that the new route and airspace structure would lead to a reduction in the task load, made it possible to speculate on the controllers' perceived workload and to formulate the following hypothesis:

Hypothesis 2: WORKLOAD

Controllers will perceive less workload in ORG1(A) than in ORG0(A).

The new route was designed in order to increase capacity and keep or increase the level of safety. If an acceptable safety level was not ensured, the new route structure would not be acceptable.

Hypothesis 3: STCAs

The new route and airspace structure will allow more space to manoeuvre the aircraft, which will lead to fewer conflicts. Fewer STCAs will be recorded.

It was expected that the reductions in the occurrence of critical events, as mentioned in hypothesis 3, and any benefits in controller task load and workload, as mentioned in hypotheses 1 and 2, would allow to speculate about an improvement in controllers' situational awareness.

Hypothesis 4: Situational Awareness

Controllers will experience better situational awareness in ORG1(A) than in ORG0(A).



Another aim of the new route structure was to comply with the airspace users' need to follow more often their planned routes.

Hypothesis 5: Route Length

The new route structure will result in aircraft flying shorter routes in ORG1 than in ORG0.

The benefits of the proposed solutions for DFS and LvNL could have resulted in a trade-off for MUAC. This was because the new route structure could have solved the conflict points in the Ruhr sector but shifted the conflict area to the Deco sector. In organisation 0(A), flights arriving at Luton or Stansted were vertically separated from Frankfurt departures. In organisation 1(A), the flows were shifted in such a way that the crossing point was located in the Deco sector, where those flows were no longer separated vertically but crossed each other.

Furthermore, the crossing point was precisely on the boundary between the DZH and DFH sectors, which could have required additional coordination, increasing the overall workload. Additionally, the Frankfurt departure route was shifted further to the south. These changes could have led to benefits for the Ruhr sector, which was expected to be offloaded with these changes, and a trade-off for the Deco sector, since the traffic increase in the Deco sector was expected to create more workload.



3 CHOICE OF METRICS AND MEASUREMENTS

The relation between the project validation objectives and the respective exercise validation objective are outlined in Appendix A.12. Table 13-7. This table also depicts the hypotheses, measurements and tools used to capture the data needed to achieve the objectives.

3.1 SITUATIONAL AWARENESS

Situational awareness can be defined as "the perception of the elements in the environment within a volume of time and space, the comprehension of their meaning, and the projection of their status in the near future" [8] [9].

The specific tool used in this simulation was the Situational Awareness for SHAPE (SASHA) questionnaire. [10].

The SASHA questionnaire is a self-rating questionnaire for measuring situational awareness. It was developed by EUROCONTROL within the framework of the Solutions for Human-Automation Partnerships in European ATM (SHAPE) project. It uses questions that focus on key elements of SA, identified by controllers themselves. The ratings are made on a seven-point rating scale.

The SASHA questions were included in the post-exercise questionnaire (see Appendix A.4. SASHA Questionnaire ).

3.2 CAPACITY

In the current ATC environment the limiting factor for sector capacity is primarily executive controller workload. Workload is understood as a subjective quality reflecting the individual controller's personal perception of the task load imposed on him/her by a particular air traffic situation. Subjective workload ratings are therefore determined to some extent by a component that cannot be derived solely using indicators of task load. Hereinafter all subjective measurements are referred to as "workload measurements", and all objectively recordable measurements of controller activity occurring in response to the tasks imposed on the controller will be referred to as "task-load measurements".

3.2.1 Workload

In contrast to task load, which reflects objective task demands, workload is influenced by the controller's internalized standards of performance, ability, and experience. The specific measurement tools used were instantaneous self-assessment (ISA) and the NASA Task-load Index (NASA-TLX) [11].

Instantaneous Self-assessment (ISA)

The instantaneous self-assessment (ISA) technique is based on the use of a specific device, the ISA box. It requires the participants to rate their subjectively experienced level of workload on a scale from 1 to 5. Measurements are taken every two minutes during a simulation run with a flashing red light indicating when an input has to be made. The level of workload is assessed by pressing one of the five numbered buttons, which range from "very low" (1) to "very high" (5).

With the ISA box, the controllers not only rate their level of workload but also their spare capacity under the different experimental conditions: the higher the perceived workload, the lower the information-processing resources left available.



NASA Task-load Index (NASA-TLX) questionnaire [11]

The NASA-TLX questionnaire is a subjective workload assessment tool developed by NASA. On the basis of the premise that perceived workload is a combination of six factors, it derives an overall workload score from a multi-dimensional rating scale.

The NASA-TLX scale was included in the post-exercise questionnaire (see Annex A.10. NASA-TLX Questionnaire).

3.2.2 Task Load

With reference to the tasks that each controller had to perform and that were influenced by the changed route and airspace structure, the following tasks were recorded:

Instructions to the pilot

The radio instructions given by the controller to the pilot were recorded on the pilot platform. Every input that the pilot made into the system was recorded. These data were recorded in order to be able to draw conclusions about the impact on the ATCO's task load, and especially on his/her communication tasks, of the efficiencies resulting from the route and airspace changes.

Telephone and SYSCO coordination

Telephone coordination was recorded to assess the impact of the route and airspace changes on the ATCO's task load and especially on his/her communication tasks. Only the duration of each call could be recorded, not the content. Every HMI input performed after communication on the frequency was recorded (MUAC only).

3.3 SAFETY

Safety occurrences were measured in terms of the number of short-term conflict alerts [6] and losses of separation (horizontal separation of less than 5 NM and vertical separation of less than 1,000 ft). In the post-exercise questionnaires, controllers were able to express their safety concerns.

3.4 ROUTE LENGTH

In the context of the AMRUFRA RTS, both the difference of the planned new and the planned old routes and the flown flight path through the sector were recorded.



3.5 CONTROLLER ACCEPTABILITY

The controllers' expert judgment on the acceptability of the new route and airspace structure following their experience of applying it was of paramount importance.

Controller acceptability was assessed by gathering the controllers' opinion through a series of questions:

• Are these newly introduced route and airspace changes perceived as useful?

• Are these changes acceptable?

• Do these changes create other problems?



4 METHODS AND TECHNIQUES SELECTED

4.1 DATA RECORDINGS

The recorded data concerned controller and pilot inputs, R/T and telephone communications, and aircraft navigation data. In addition, subjective data were collected through controller feedback and comments, and questionnaire ratings.

All inputs from the controllers (if applicable) and the pilots on the HMI were recorded by the system. Moreover, every communication with the pilot or with another sector via AUDIOLAN was recorded. A detailed description of the recorded data is provided in the recording and metrics requirement document [13].

4.2 QUESTIONNAIRES

The following types of questionnaire were used in the AMRUFRA RTS:

• Pre-simulation questionnaires.

• A set of post-exercise questionnaires.

• Post-organisation questionnaires.

The pre-simulation questionnaire captured feedback on the training and biographical information on the controllers participating in the simulation.

The post-exercise questionnaire was distributed to the controllers at the conclusion of each exercise. The set of post-exercise questionnaires contained a situational awareness questionnaire (SASHA) and a workload questionnaire (NASA-TLX).

A post-organisation questionnaire was distributed at the end of each organisation. This questionnaire served to collect controller feedback on the new route and airspace structure.

4.3 DEBRIEFINGS

Every second day (from the second day of the simulation onwards), debriefings were conducted after the exercises to collect the feedback of the controller regarding the acceptability of the new route and airspace structure.



5 SIMULATION ENVIRONMENT

A technical description of the simulation environment can be found in the technical specifications [7]. In this document, the operational characteristics of the simulation are described.

5.1 ORGANISATION SPECIFICATIONS

To be able to assess the changes resulting from the proposed solutions mentioned in Chapter 1.4, various organisations were modelled on the basis of the current route network.

5.1.1 ORG0 and ORG0A

Organisations 0 and organisation 0A functioned as a baseline. These organisations contained the current airspace and route structure, with the above-mentioned problems.

ORG0: airspace and route structure as it is designed and operated today, with Military Area TRA12 up to FL 285. This organisation depicted the baseline.

ORG0A: airspace and route structure as it is designed and operated today, with Military Area TRA12 and TRA12A up to FL 660. This organisation depicted baseline 0A.

ORG0 and ORG0A differed only in the volume of the TRA level band.


Organisation 1 and organisation 1A depicted the proposed solutions.

ORG1: this organisation simulated the modification to the route network and changes to the LvNL (splitting of routes) and Frankfurt (reshaped NOR, MGB, TAU) sectors, and, as a result, the change in the Maastricht sectors. In ORG1, TRA12 (up to FL 285) was active.

ORG1A: this organisation simulated the modification to the route network and changes to the LvNL (splitting of routes) and Frankfurt (reshaped NOR, MGB, TAU) sectors, and the resulting change in the Maastricht sectors. In ORG1A TRA12 and TRA12A (up to FL 660) were active.

ORG1 and ORG1A differed in the volume of the TRA level band.

One aspect of the change from organisation 0 to organisation 1 for the DFS sectors was the shape of the NOR and MGB sectors. The benefit expected from the changes in sector shape was that the conflict areas would be handled by one sector, thus saving on coordination, which intensifies workload. Airspace from the Düsseldorf departure sector was therefore moved to the NOR sector. The result expected from this change was less coordination and even silent coordination with the help of LoAs (letters of agreement) between TAU/NOR, NOR/HMM and NOR/Ruhr. With the new route structure, fewer coordination partners were present, since at the higher levels the NOR and MGB sectors were connected directly to the HAMM sector. Figure 5-1 depicts how the Frankfurt departure route differed in ORG0(A) and ORG1(A).



Figure 5-1: ORG0(A) and ORG1(A), showing the Frankfurt departure route

The orange routes illustrate the old route structure while the green routes indicate the new routes. The dotted line indicates a route through the military area. This route was therefore not always available (CDR). The green line shows the new route structure.



Figure 5-2: ORG0(A) and ORG1(A), showing the Frankfurt arrival route

The same colour coding applies as in the previous graph, with the addition of the blue lines indicating the changes mentioned in the previous graph. The orange line indicates the current route structure, and the green line indicates the new Frankfurt arrival route.

For the LvNL sectors, the change from organisation 0 to organisation 1 comprised the splitting of the Amsterdam departure route, as illustrated in Figure 5-3.



Figure 5-3: ORG0(A) and ORG1(A), showing the Amsterdam departure route

The blue lines indicate the changes discussed above. The orange line illustrates the current Amsterdam departure route, while the green line shows the additional Amsterdam departure route implemented in organisation 1(A). It should be noted that the northern Amsterdam departure route via SONEB was still valid and the southern route had been shifted further to the south via NAPSI, thus creating more space for the outbound flow.

In summary, the change in ORG1(A) for LvNL involved the Amsterdam departure route being split into two segregated routes and additional airspace being gained. This change was expected to result in less complexity. The new route structure for MUAC allowed more space between the different inbound and outbound flows, and was therefore intended to reduce complexity in the Ruhr sector.


To be able to assess the feasibility of the new route structure with the forecast traffic increase, ORG2 and ORG2A were created.

ORG2: this organisation simulated the modification to the route network as in organisation 1, but with a traffic increase of up to 20%. This sector increase required the splitting of the NOR (Nörvenich) sector into NOR and DKAR (Köln arrival). The TAU (Taunus) sector was simulated as a feed sector. The Ruhr and Deco sectors were also split. The Ruhr sector became HRH and HRL, while DDH became DZH (Zeeland High) and FLH (Flevo High). TRA12 (up to FL 285) was active.

ORG2A: this organisation simulated the modification to the route network as in organisation 1, but with a with traffic increase. This increase required the splitting of the NOR sector into NOR and



DKAR. The TAU sector was simulated as a feed sector. The Ruhr and Deco sectors were also split. RHR became HRH and HRL, while DDH became DZH and DFH. TRA12 and TRA12A (up to FL 660) were active.

ORG2 and ORG2A differed in the volume of the TRA level band.

For LvNL, no route or airspace changes took place with the increased traffic sample in organisation 2(A).

Organisation 2 was not comparable with any other organisation, and was therefore the subject of a feasibility study.

During the simulation changes were implemented to improve organisation 2. These changes were documented under the name “organisation 3”. In the last week further modifications led to the creation of a new organisation, referred to as “organisation 4”. Organisation 4 is described in detail in chapter 13.

Table 5-1 gives an overview of the organisations, showing their respective characteristics. Table 5-1: Organisation overview

Traffic Military activity Baseline New route and airspace structure

Today TRA12 ORG0 ORG1

Today TRA12A ORG0A ORG1A

TRA12 ORG2 Today +

TRA12A ORG2A

Unplanned Modifications ORG3 + ORG4

5.2 EXPERIMENTAL VARIABLES AND DESIGN

The following variables were manipulated throughout the simulation exercise:

• Traffic (traffic 1 and traffic 2).

• Controller role (EC/PC).

• Organisation (ORG0/0A; ORG1/1A; ORG2/2A).



5.2.1 Traffic

The traffic samples used for the simulation were created from the real traffic on 26 June 2007. This 24-hour traffic was recovered by SAAM (system for traffic assignment & analysis at macroscopic level) from CFMU radar data, cut from 07:30 to 10:00, sent to the AMRUFRA representatives for approval and then exported to IPAS (integrated data preparation and analysis system) as ORG0. The same portion of traffic was rerouted around TRA12A by SAAM and exported to IPAS as ORG0A.

Later, the same portion of traffic was rerouted on the new AMRUFRA agreed route structure by SAAM and exported to IPAS as ORG1. The traffic from ORG1 was rerouted around TRA12A by SAAM and exported to IPAS as ORG1A.

The traffic used for ORG1 and ORG1A was increased by 20% by SAAM and imported to IPAS as ORG2A and ORG2. To accommodate all traffic characteristics, each of the above samples was divided into two traffic samples, one where the simulated measured hour was from 07:40 to 08:40, and another from 09:05 to 10:05.

Therefore the variable "traffic sample" had two different levels:

Traffic 1 = 07:40-08:40 (7 o'clock).

Traffic 2 = 09:05-10:05 (9 o'clock).

One of the differences in the traffic characteristics was that in traffic sample 2 more Frankfurt departure flights were simulated. This traffic sample also contained more flights, especially in the MUAC sectors.

The military traffic samples were prepared in parallel, and inserted in each simulated traffic sample. The military traffic was simulated in a way all types of profiles and conflict-points/traffic solutions with civil traffic were to be found.

5.2.2 Controller Role (EC/PC)

The measured controller team consisted of the executive controller and the planning controller. There were thus two levels of the variable "controller role":

• Planning controller (PC).

• Executive controller (EC).

Feed controllers were not measured, and were thus not considered as experimental variables.



5.2.3 Organisations

Table 5-2 outlines the simulated sectors in each organisation in detail, with the expected benefits and trade-offs.

Table 5-2: Organisation description

Organisation name Description

ORG0 (Reference) Present airspace and route structure with TRA up to FL 285.

ORG0A (Reference) Present airspace and route structure with TRA12 (up to FL 285) + TRA12A (FL 285/660).

ORG1 (Future) New airspace shape with new routes and TRA12 (to FL 285).

ORG1A (Future) New airspace shape with new routes and TRA12 (to FL 285) + new TRA12A (FL 285/660).

ORG2 (Future + increase of traffic) ORG1 with split sectors in DFS airspace and MUAC airspace with TRA12 (to FL 285) plus traffic increased by up to 20%.

ORG2A (Future + increase of traffic)

ORG1 with split sectors in DFS airspace and MUAC airspace with TRA12 (to FL 285) + new TRA12A (FL 285/660) plus traffic increased by up to 20%.

A detailed description of the changes to the route and airspace structure can be found in the operational and technical specifications. [6, 7].

To sum up the difference between the organisations, the (A) version of each organisation simulated a different military area. Organisation 1(A) simulated the new route and airspace structure, taking organisation 0(A) as the baseline. Organisation 2(A) simulated an increased traffic sample, and split sectors were therefore introduced. These split sectors were partly different to the sectors in the other organisations and also different controllers participated. Therefore the results of this organisation cannot be compared to any other organisation. Table 5-3 outlines the different variables within each organisation, and which approach was used for comparison.

Table 5-3: Variables within the organisations, and analytical approaches

Traffic Military activity Baseline New route and airspace structure

Analytical approach

Today TRA12 ORG0 ORG1 Direct comparison between ORG0 and ORG1 (3-factorial general linear model (GLM)1)

Today TRA12A ORG0A ORG1A Direct comparison between ORG0 and ORG1 (3-factorial GLM1)

TRA12 ORG2 Feasibility study Today +

TRA12A ORG2A Feasibility study

In contrast to the comparison of the organisations and the resulting statistical analyses, the feasibility study was carried out via the collection of mainly qualitative data such as questionnaires and debriefings.



6 CONDUCT OF VALIDATION EXERCISE RUNS

6.1 EXPERIMENT PREPARATION

6.1.1 Training and Acceptance Test

The objectives of the training days were to:

• familiarise controllers with the new route and airspace structure;

• familiarise controllers with the simulation environment;

• familiarise the controllers with the increased traffic sample.

A stepwise training approach was adopted to familiarise controllers with the simulation environment and the new route and airspace structure.

Since not all controllers could participate in the training days at the simulation centre, they were briefed before the training days at their home centres about the airspace changes, procedures and data-collection methods that would be applied during the simulation.

During the training days at the simulation centre, the controllers also worked on the exercises in order to validate the traffic samples. Therefore all the various exercises were simulated during the training week.

Training plan Table 6-1: Training plan

Monday Tuesday Wednesday Thursday Friday

9:00-10:20 ORG0A tr1 ORG1 tr2 ORG2A tr1

Break

10:40-12:00 ORG1A tr2 ORG1A tr1 ORG2 tr1

Lunch break

13:00-14:20 Briefing ORG0A tr2 ORG1 tr1

Break

14:30-15:30

ORG0 tr1 ORG0A tr2

ORG2 tr2

Key: ORG = organisation; tr1 = traffic 1 (time of day: 07:40-08:40); tr2 = traffic 2 (time of day: 09:05-10:05).



6.1.2 Simulation Plan

On the first day of the first week, the simulation briefing was followed by two different refresher exercises, which were not measured. On the afternoon of the first day, a measured exercise was planned. This plan was valid for all three Mondays of the simulation.

For the rest of the first two weeks, three exercises per day were measured, followed by a planned briefing in the afternoon. The possibility nevertheless existed to use the slot planned for the briefing for an additional measured exercise if necessary.

Each exercise ran for 1 hour 20 minutes, 60 minutes of which were measured.

The planned order of the exercises had to respect the controller rotation needs described in detail in Appendix A.1. For the organisations with the different variables valid for LvNL, it was calculated that 12 exercises were needed, while for MUAC and DFS 16 exercises were needed to fulfil a complete experimental design.

The LvNL controllers had to work at three positions (EC1/EC2/PC), with two different organisations (ORG0 and ORG1) and two different traffic samples (1 and 2). It should be noted that the military activity of TRA12A had no impact on the LvNL airspace; no distinction was therefore made between ORG0 and ORG0A or between ORG1 and ORG1A. It was thus calculated that 12 exercises were needed for LvNL.

The MUAC and DFS controllers had to work at two positions (EC/PC) with two different organisations, each taking account of two different levels of military activity (ORG0, 0A, 1, 1A, 2 and 2A) and two different traffic samples (1 and 2). It was thus calculated that 16 exercises were needed for MUAC and DFS.

This meant that, since not all the controllers were available in all three weeks, one week would need to accommodate 16 exercises. This was not feasible owing to time constraints. A maximum of 12 exercises could be accommodated in a week. A high priority was therefore given to the organisations and traffic samples, while the controller roles had a low priority. This made it possible to simulate the eight preferred conditions (four organisations with two traffic samples) with a stable controller position. Four exercises were therefore available for swapping MUAC controllers. This allowed the feed controllers, who were scheduled to work at the measured positions in week three, to gain experience. Additionally, these four exercises were needed to obtain a complete data set from the LvNL controllers. In the second week, the inverse configuration for DFS (i.e. the first week's EC controller would work at the PC position in the second week) was simulated under the preferred eight conditions (four organisations with two traffic samples). The four exercises were planned for the same purpose as mentioned above. However, certain needs raised during the simulation made a modification of the route structure and procedures necessary. For these changes, the "additional" eight exercises were used. This meant that the planned schedule could not be used, and a modified version was used during the simulation. The planned exercise schedule can be found in Appendix A.1. describe the simulation schedule actually used.

In week three, additional sectors with different controllers were simulated. Therefore, no direct comparison can be made with weeks one and two. In week three, eight conditions needed to be fulfilled: the controllers worked as scheduled at the two working positions (EC/PC), with two organisations (ORG2 and 2A) and two traffic samples (1 and 2). In week three, it was possible to fulfil the "controller working positions" condition.

The JOJO track exercises in the middle of the third week were simulated to demonstrate the impact of this area on the new route and airspace structure to military representatives.



6.2 CHANGES IN THE SIMULATION PROGRAMME AND SCHEDULE

Owing to client needs, the simulation schedule was changed and adapted.

The AMRUFRA real-time simulation was conducted over three weeks in April 2008 at the EEC, Brétigny. The training days took place from 17 to 20 March 2008. The simulation weeks took place from 7 to 25 April 2008.

The basics of the schedule as set out in 6.1.2 remained valid. In week two, certain exercises were changed. These changed exercises are indicated in bold letters.

Table 6-2: Simulation week 1 schedule

Week 1 Monday Tuesday Wednesday Thursday Friday

9:00-10:20 Briefing ORG1 tr1 (2) ORG1 tr2 (5) ORG0 tr1 ORG0 tr2

Break

10:40-12:00 Refresher

ORG0 tr1 ORG1A tr2 (3) ORG0A tr1 (6) ORG1 tr1 ORG1 tr2

Lunch break


ORG1A tr2 ORG0 tr1 (4) ORG0 tr2 (7) ORG1A tr1 (8) Debriefing

Break

14:30-15:30 ORG0A tr2 (1) Debriefing Debriefing Debriefing



9:00-10:20 Briefing ORG1 tr1 (10) ORG1 tr2 (13) ORG 3 tr1 ORG 3A tr1

Break

10:40-12:00 Refresher ORG0 tr1

ORG1A tr2 (11) ORG0A tr1 (14) ORG 3 tr1 ORG 3 tr2

Lunch break

13:00-14:20 Refresher ORG1A tr2 ORG0 tr1 (12) ORG0 tr2 (15)

Working groups Debriefing

Break

14:30-15:30 ORG0A tr2 (9) Debriefing Debriefing Working groups

Break

15:45-17:05 ORG1A tr1 (16)

Key: ORG0 = organisation 0; ORG1 = organisation 1; ORG0A=organisation 0A, ORG1A= organisation 1A, tr1 = traffic 1 (time of day: 07:40-08:40); tr2 = traffic 2 (time of day: 09:05-10:05); (1)-(16) indicates the run number; ORG 3 = organisation 1 with changes (internal improvements to test the new ORG2 and the future ORG 4).



The last exercise was originally planned for Thursday, but as the need for more exercises became obvious the exercise was shifted to Monday, as the refresher exercises were shortened and did not last 1 hour 20 minutes each. On Thursday and Friday, four exercises (ORG 3/internal testing) were simulated with implemented changes regarding the NAPSI delegated area. The lowest vertical limit of the area was changed from FL 205 to FL 195.

A working group session was held in the afternoon to work on further changes for week three. The exercises that involved these unplanned changes were not comparable with and were therefore not included in the analyses. In week three, further changes were implemented, affecting mainly the MUAC sectors. These exercises are indicated with bold letters.



9:00-10:20 Briefing ORG2A tr1 (2)

JOJO track (ORG2 tr2)

ORG2A tr2 (7) (RH not

measured) ORG 4 tr2

Break


ORG2 tr1 100% ORG2 tr2 (3)

JOJO track (ORG2 tr2) ORG2 tr2 (8) ORG 4 tr2

Lunch break


ORG2A tr2 100% ORG2A tr2 (4) ORG2 tr1 (6)

ORG 4 tr2 (training)

Break

14:30-15:30 ORG2 tr1 (1) ORG2A tr1 (5) Debriefing ORG 4 tr2

Key: ORG2 = organisation 2 with 20% increased traffic ; ORG2A= organisation 2A with 20% increased traffic ORG 4 = organisation 1+ changes; tr1 = traffic 1 (time of day: 07:40-08:40); tr2 = traffic 2 (time of day: 09:05-10:05); (1)-(8) indicates the run number. The exercises with the changes (ORG 4) will be dealt with in individual qualitative analyses, and the results are reported in section 12 as simulated improvements.

The JOJO track exercises were not included in the analyses.



6.3 PARTICIPANTS

Over the 3 weeks, a total of 34 controllers participated at the positions measured. The participants comprised 9 LvNL controllers (8 male,1 female), 9 DFS controllers (9 male) and 12 MUAC controllers (11 male, 1 female).

Table 6-5: Distribution of LvNL controller age and experience

LvNL Age Seniority

Years 25-35 36-45 46-55 <10 11-20 21-30

Number of ATCOs 3 5 1 2 6 1

The ages of the LvNL controllers ranged from 26 to 54 (M = 36,778; SD = 7,758), and their experience as licensed controllers from 3 to 33 years (Table 6-5 hows how the controllers were distributed among the categories). Two of the nine controllers participated in two weeks, while the other eight controllers participated in just one week of the simulation.

Table 6-6: Distribution of DFS controller age and experience

DFS Age Seniority

Years 25-35 36-45 46-55 <10 11-20 21-30

Number of ATCOs 8 0 1 8 0 1

The ages of the DFS controllers ranged from 28 to 48 (M=32.44; SD=6.207), and their experience as licensed controllers at the position measured ranged from 2 to 25 years (Table 6-6 hows how the controllers were distributed among the categories). The same five controllers participated in week 1 as in week 2. One controller participated in week 1 and week 3; additionally, three new controllers participated in week 3.

Table 6-7: Distribution of MUAC controller age and experience

MUAC Age Seniority

Years 25-35 36-45 46-55 <10 11-20 21-30

Number of ATCOs* 6 5 0 9 2 0

*One controller did not provide personal data.

The ages of the MUAC controllers ranged from 25 to 43 (M=34.181; SD=5.896) and their experience as licensed controllers at the position measured ranged from 0 to 12 years. Table 6-7 shows how the controllers were distributed among the categories). In week one and two the same controllers worked at the position measured, while in week 3 an additional four new controllers participated.



6.4 STATISTICAL ANALYSIS

This section presents the results of the two-week simulation study. All usage data (R/T, system inputs, etc.) are based on a measured exercise time of 60 minutes. Wherever possible, data collected using the various indicators and metrics were analysed with an analysis of variance (ANOVA1) involving the four two-level experimental variables :

• organisations (ORG1 versus baseline ORG0),

• military area (TRA A active versus TRA A inactive),

• traffic (traffic 1 [7 o'clock sample] versus traffic 2 [9 o'clock sample]), and

• controller role (EC, PC).

In this report, reference is made only to results containing the organisation factor. This means that if a difference between the two traffic samples is observed, but has no influence on the organisations, it is not reported, e.g. a difference only in workload between TRA A active and TRA A inactive is simply reported in Appendix A.2 and not further interpreted, since it was not part of the objectives.

These analyses were conducted with the GLM (general linear model) repeated-measures procedure of the Statistical Package for the Social Sciences (SPSSTM) [14]. Separate ANOVAs were computed for each dependent variable. All presented results indicating a significant difference were confirmed via a statistical test. The statistical values are presented in Appendix A.3.

For most of the controller feedback ratings, for instance, relating to usability questions collected during the individual interview, simple frequency distributions across the range of possible answers were computed and either verbally reported or graphically displayed. The results are reported in subsections structured on the basis of the high-level objectives set out in section 2, separated for each ANSP.

For the LvNL data only, the organisation and traffic variables were taken into account, since the activity of the TRA A military area did not influence these sectors; moreover, only the data relating to sector 2E are reported in this document, since this was the sector in which most of the impact was anticipated and since, owing to the changes mentioned in section 6.2, not all the planned data could be collected. For LvNL, the paired t-test was used to compare the means of the dependent variables under the different conditions (organisation and traffic).

1 The purpose of an analysis of variance (ANOVA) is to test differences in means (for groups or variables) for statistical

significance. This is accomplished by analysing the variance, that is, by partitioning the total variance into the component that is due to random error and the components that are due to systematic differences between the means of the experimentally controlled conditions. These latter variance components are then tested for statistical significance, and, if significant, the null hypothesis of no differences between means is rejected, and the alternative hypothesis that the means (in the population) are different from each other is accepted. In an ANOVA, significance testing is based on a comparison of the variance due to between-condition (or group) variability (called the mean square effect or MSeffect) and the within-condition variability (called the mean square error or MSerror). This comparison takes the form of a ratio (the F-ratio) which is high when MSeffect is high or MSerror is low. For each value of F (and the number of degrees of freedom also needs to be taken into account) a p-value can be derived. The p-value represents a decreasing index of the reliability of a result. Or in other words: the higher the p-value, the less one can believe that the observed relation between variables in the measured sample (here, for MUAC and DFS 6[7] ATCOs, and for LvNL 5 ATCOs [only controllers that worked under all conditions on the 2E sector]) is a reliable indicator of the relation between the respective variables in the population (all ATCOs on the basis of which one wishes to generalise the result). It is common practice to consider a result as being statistically reliable if the p-value is 0.05 or smaller (see [16] for further reading, or the URL http://www.statsoft.com/textbook/stathome.html for a good web-based tutorial).



7 DFS – EXPERIMENT RESULTS

Each of the three ANSPs' results are presented in a separate chapter. The description of the objectives and the measured indicators is repeated for each ANSP, in order to facilitate the separation of the results for each ANSP. Only significant results are presented for the comparison of organisation 0 with organisation 1. Furthermore, only results relating to the factor "organisation" are presented, i.e. differences in relation only to the traffic sample or in relation only to the activity of the military area TRA A (e.g. the results of exercises with TRA12A active were better than exercises with TRA12A inactive, with no effect in terms of the organisation) are not reported, since the objectives of the simulation referred to comparison of the organisations. The results are always reported with reference to the high-level objectives.

7.1 ASSESS THE IMPACT OF THE NEW ROUTE STRUCTURE AND AIRSPACE STRUCTURE ON CAPACITY

As explained in section 3.2 capacity was not measured directly. The focus in AMRUFRA was on controller workload, since this is the key factor constraining sector capacity. Controller workload was expected to change as a function of the task load imposed on the controller, e.g. the number of radio calls to aircraft, and coordination and telephone calls to neighbouring sectors. Whereas controller workload was assessed on the basis of subjective workload ratings, changes in controller task load could be directly and objectively derived from the controllers' behaviour in the course of their interaction with the system. In the following subsections, the results obtained from the various task load measurements are reported first, followed by the results for subjective workload.

Objective 1 was broken down into three experimental objectives:

1. Investigate the effect of the route and airspace changes on the controllers' traffic-handling capability (workload).

2. Investigate the effect of the route and airspace changes on R/T usage (task load).

3. Investigate the effect of the route and airspace changes on telephone coordination (task load).

With regard to these task load measurements, the following hypotheses specified for DFS were examined using the analyses of recorded data:


Controllers will need to perform fewer tasks (telephone coordination) with the new route and airspace structure in ORG1(A).

As explained in 2.1, for DFS this hypothesis referred to coordination tasks, since it was assumed that because the main conflict points were now located within one sector they would need less coordination than in the baseline. The assumed reduction in task load led to the second hypothesis.





7.1.1 Task Load

In section 3, a set of task-load measurements was described that were expected to be influenced by the new route structure and that had the potential to alleviate controller workload in comparison to the baseline (ORG0).

For DFS, it was assumed that in ORG1 less coordination would be necessary, since the new airspace structure comprised, inter alia, the Frankfurt departure route into the NOR sector, which was expected to require less interaction with other sectors. Coordination with other sectors and centres was performed by telephone.

Telephone coordination

The task of telephone coordination can be performed by both controllers, but is mainly performed by the PC. It was hypothesised that the new organisation, with its new route structure, would reduce the amount of telephone coordination, since the routes were organised in such a way as to be shifted into the NOR sector, which would require less coordination with the adjacent sectors.

The results indicated that this hypothesis could not be confirmed. Figure 7-1 illustrates that significantly more telephone calls were made in ORG1 than in ORG0 in all sectors. The statistical analysis was performed over all sectors, but for better illustration the sectors are indicated separately in the graph.

Phone Calls PC

ORG0 ORG1

Mea

n nu

mbe

r of p

hone

cal

ls

0

20

40

60

80

100

MGB NOR TAU

Figure 7-1: DFS – telephone calls by the PC

Most of the telephone calls were made in the NOR sector. An example of the additional telephone calls in organisation 1 was the coordination of exit flight levels mentioned by the controllers. This coordination was caused by the fact that, owing to the amount of vertical traffic, the aircraft did not reach the agreed exit flight level. Especially in the NOR sector, an increase in vertical traffic was experienced. Furthermore, the PC in the NOR sector asked for a release in order to deal with the aircraft as early as possible, so that conflicts could be solved in time. Another example of the increase in telephone coordination was the need for coordination with the military units. The Ramstein inbound route, which formed an additional conflict point, required more telephone coordination because it was a military traffic stream.



R/T communication

Two measures were derived from the R/T air-ground communication for each exercise. First, the cumulative time spent on frequency was recorded. Second, the number of radio calls was counted. Separate ANOVAs were computed for these two measures.

ORG0 ORG1

mea

n of

tim

e sp

ent o

n fre

quen

cy in

sec

onds

0

500

1000

1500

2000

2500

MGB NOR TAU

Figure 7-2: DFS – time spent on frequency

A significant result could be revealed in terms of the time controllers spent talking on the frequency. In organisation 1, controllers spent significantly more time on radio communication than in organisation 0, especially in the NOR sector. These results were in line with the number of radio contacts. More radio contacts were made in organisation 1 than in organisation 0.

ORG0 ORG1

mea

n nu

mbe

r of r

adio

con

tact

0

100

200

300

400

500

600

700

MGB NOR TAU

Figure 7-3: Radio contacts



Instructions to pilots

Instructions given to pilots over the radio frequency were recorded by the system at the pilot positions. Four different types of instructions were recorded: heading instructions, speed instructions, level instructions and direct instructions. Figure 7-4 presents, for each sector separately, a summary of all the cumulated instructions cleared by the EC to the pilot.

All instructions

ORG0 ORG1

mea

n nu

mbe

rs o

f all

inst

ruct

ions

0

20

40

60

80

100

120

140

160

180

200

MGB NOR TAU

Figure 7-4: DFS – instructions

The statistical analysis was performed over all the DFS sectors, and the result revealed significantly more instructions for organisation 1 than for organisation 0. The figures indicate that the difference was greatest for the NOR sector. Another significant effect could be revealed in terms of heading instructions, since more heading instructions were issued in organisation 1 than in organisation 0. Especially for the NOR sector, the heading instructions were necessary in order to vector Frankfurt departure traffic out of the inbound traffic flow.



Number of aircraft on frequency

The results relating to radio communication and telephone coordination, and to instructions given to the pilot, are partly explained by the number of aircraft on the frequency.

Mean number of aircraft on frequency

ORG0 ORG1

mea

n nu

mbe

r of a

ircra

ft

0

10

20

30

40

50

60

MGB NOR TAU

Figure 7-5: Number of aircraft on frequency

Since a higher number of aircraft were on the frequency in organisation 1 than in organisation 0, owing to the new airspace and route structure, more interactions with these aircraft were needed, especially in the NOR sector.



7.1.2 Workload

The controllers' perception of workload was addressed through a combination of workload assessment techniques. With the instantaneous self-assessment (ISA), workload ratings were collected during the exercise (every two minutes), whereas the NASA-TLX questionnaire was completed after each exercise.


For this analysis, ratings given on a five-point scale labelled "very low" (1), "low" (2), "fair" (3), "high" (4), and "very high" (5) were averaged across two-minute intervals to obtain an average rating for each exercise.

The mean ISA ratings for both the EC and the PC revealed a significant difference associated with the organisation factor.

ISA EC

ORG0 ORG1

ISA

mea

n

very low

low

fair

high

very high

MGBNORTAU

ISA PC

ORG0 ORG1

ISA

mea

n

very low

low

fair

high

very high

MGBNORTAU

Figure 7-6: EC workload (ISA) Figure 7-7: PC workload (ISA)

At both positions, EC and PC, controllers experienced more workload in organisation 1 than in organisation 0. Hypothesis 2 cannot, therefore, be confirmed.

NASA Task-load Index (TLX)

Controllers gave their ratings after each exercise on a 20-point scale ranging from low to high (see the form in Appendix A.5. NASA-TLX Questionnaire). An ANOVA was performed on the unweighted NASA-TLX score obtained by averaging the collected ratings over the six subscales (mental demand, physical demand, temporal demand, performance, effort, and frustration).

Additionally, separate ANOVAs were performed for the scores of each subscale. Controllers' mean unweighted NASA-TLX workload score for the EC was higher for ORG1 than for the baseline, ORG0, independently of the activation of the military area TRA A.



NASA overall EC

ORG0 ORG1

NA

SA

mea

n

0

2

4

6

8

10

12

14

16

18

20

MGB NOR TAU

Figure 7-8: NASA overall EC

ANOVAs computed for sub-scores of each of the six NASA-TLX subscales for the EC revealed that for five scales organisation was found to have a significant main effect. These scales were mental demand, temporal demand, performance, effort and frustration. Higher temporal demand, effort and frustration were perceived in organisation 1, while performance decreased in that organisation. Additionally the interaction between organisation and traffic became significant for the effort subscale. The effort perceived by the EC increased more under traffic 2 conditions in organisation 1 than in organisation 0. No further effects were significant.

PC controllers' mean unweighted NASA-TLX workload score was higher for ORG1 than for the baseline, ORG0, independently of the activation of the military area TRA A.

NASA overall PC

ORG0 ORG1

NA

SA

mea

n

0

2

4

6

8

10

12

14

16

18

MGB NOR TAU

Figure 7-9: NASA overall PC

No further effects were significant. ANOVAs computed for sub-scores of each of the six NASA-TLX subscales for the PC revealed that for four scales organisation was found to have a significant main effect. These scales were mental demand, temporal demand and effort. Higher mental demand, temporal demand and effort were perceived in organisation 1 than in organisation 0. The increase in workload was reflected in and can be explained by the above-mentioned results regarding the task load and traffic increase in the NOR sector. The increase in the number of



conflict points in organisation 1 compared to organisation 0 and the vertical flight profile of the traffic required more instructions to the pilot, which resulted in more radio communication by the EC and more telephone coordination by the PC.

DFS results for objective 1

More telephone coordination, radio calls and instructions were needed under the conditions of the new route and airspace structure. The increase in the number of heading instructions in particular was an indicator of the increased demand for conflict resolution, which is reflected in the impact on workload. The impact on task load is partly attributable to the impact on the redesigned NOR sector. This resulted not only in more traffic to be handled but also in an increase in the number of conflict points and therefore complexity, contributing to the measured task load. A significant workload increase from organisation 0 to organisation 1 was therefore perceived by the ATCOs.

7.2 ASSESS THE IMPACT OF THE NEW ROUTE STRUCTURE AND AIRSPACE STRUCTURE ON ACCEPTABILITY

In the debriefings, the controllers were asked if the new route and airspace structure represented in organisation 1 was acceptable.

No specific hypotheses were formulated with regard to these quality aspects of the new route and airspace structure. The questionnaire completed by the controllers at the end of week two, after all the exercises for organisation 0 and organisation 1 had been simulated, contained questions regarding the acceptability of the new route structure in organisation 1 in terms of its impact on workload and its safety impact. While only a small number of results are presented in this chapter, the full and complete feedback from the controllers can be found in Appendix A.11. Organisation Questionnaires : MUAC Feedback Results.

The shift of the Frankfurt departure route to the w est is acceptable

5

1 1

0

1

2

3

4

5

6

stronglydisagree

disagree undecided agree stronglyagree

Num

ber o

f ATC

Os

The new sector shape is acceptable

3 3

1

0

1

2

3

4

5

6

7

stronglydisagree


Num

ber o

f ATC

Os

Figure 7-10: Acceptability part I Figure 7-11 Acceptability part II

The new sector shape was not acceptable to six of the seven controllers, and the shifting of the Frankfurt departure route was not acceptable to any of the seven controllers, especially to those working in the NOR sector. This result is explained to a large extent by the results mentioned in the preceding chapters. The new sector shape and route structure resulted in higher workload and higher task load.


The new route and airspace structure was not acceptable to the controllers. This was because of a perceived increase in workload and the increase in task load in the new organisation.



7.3 DFS RESULTS FOR OBJECTIVE 2

The impact on safety was derived from both the qualitative and the quantitative data records. The following experimental objectives were derived from the high-level objective:

1. Investigate the effect of the route and airspace changes on the number of STCAs and losses of separation.

2. Investigate the effect of the route and airspace changes on the level of situational awareness.

7.3.1 Short-term Conflict Alerts (STCAs)

Hypothesis 3: STCAs


No significant difference was found between the organisations in terms of the number of STCAs.

7.3.2 Situational Awareness

Situational awareness is primarily a human performance rather than a safety concept. However, any negative impact on situational awareness can be regarded as a safety issue.

Hypothesis 4: Situational Awareness


Information on situational awareness was gained by a post-exercise questionnaire (SASHA) that was distributed to the controllers to collect data relating to their perceived situational-awareness level. Controllers were asked to rate their perceived situational awareness with the help of statements such as "I was always ahead of the traffic", and they answered using a scale ranging from 0 (never) to 6 (always). An ANOVA performed on the collected ratings (SASHA scores) revealed that traffic load had a significant main effect: F (1, 5) = 26,687; p<0, 05.

Situational awareness was perceived by the EC to be significantly lower in organisation 1 than in organisation 0.



Situational Awareness EC

ORG0 ORG1

mea

n SA

SH

A s

core

0

1

2

3

4

5

6

MGBNORTAU

Figure 7-12: Situational awareness – DFS

Controllers mentioned in debriefings that they perceived a "team split", i.e. that since the EC was no longer able to document the flight strips the PC could not follow the actions, and vice versa. Under normal circumstances, the EC and the PC write comments on these strips in order to remind themselves or the other team member of clearances given to aircraft and requiring further attention or actions, or tasks needing to be performed on a specific aircraft as a result of telephone coordination. In organisation 1, this form of communication vanished. The EC was no longer aware of the coordination performed by the PC. Furthermore, the EC felt that he/she could no longer follow the various traffic flows properly.

7.3.3 Controller Feedback

The post-organisation questionnaire gave the controllers the opportunity to express their safety concerns regarding the new route and airspace structure.

Does the new route and airspace structure create any problems (safety concerns)?

7

0

1

2

3

4

5

6

7

yes no

Num

ber o

f ATC

Os

Figure 7-13: Safety concerns – DFS



The following safety concerns were mentioned by the controllers:

• Number of conflict points in the NOR sector

• Lack of time for resolution of conflicts

• Lack of time for coordination

• Increased frequency load

• Military traffic

The number of conflict points was perceived as too high by the controllers, who also experienced time pressure, since aircraft entering the sector had to be dealt with immediately in terms of resolution of conflicts. This also required more coordination by the PC with the adjacent sectors, which resulted in more reactive than proactive control of the aircraft.


The controllers perceived less situational awareness in organisation 1 than in organisation 0. The decrease in situational awareness was a result of the increase in workload and sector complexity with the new route structure, which generated a number of conflict points, and the lack of time for resolving those conflicts. Military activity and the increased frequency load were also safety concerns.

7.4 ASSESS THE FEASIBILITY OF THE NEW ROUTE AND AIRSPACE STRUCTURE WITH A TRAFFIC INCREASE (ORGANISATION 2)

As had already been the case during the simulation, procedural problems were raised regarding the Amsterdam departure traffic affecting the MGB sector. Therefore, the following change was implemented for organisation 2:

• The flights were directly handed over to the Ruhr sector without coordination with the MGB sector. Nevertheless, the MGB got the flight information by means of an activation message (ACT) to keep the ATCO in the loop.

Owing to the traffic increase, it was planned to split the NOR sector horizontally into the NOR sector and the DKAR (Köln arrival) sector. Additional modifications, which became necessary after simulating organisation 1, were implemented. While in organisation 1 Amsterdam departure flights were transferred to the MGB sector and then to the Ruhr sector, in organisation 2 these flights were transferred immediately to the Ruhr sector. MGB received the flight information but did not need to communicate either with the aircraft or with the Amsterdam sector. This change was implemented in order to support military activity in German airspace.

A questionnaire was distributed to the participants (number of ATCOs = 6) after organisation 2. The controllers were asked to rate the acceptability and feasibility of the new route and airspace structure with the traffic increase.



Controller Feedback

Five of the six controllers stated that the new route structure was not acceptable with a traffic increase, and the splitting of the NOR sector was not acceptable to three of the six controllers. The controllers saw the need for the splitting of the NOR sector, but the way it was split during the simulation was not acceptable.

The new route structure w ith the traffic increase is acceptable

5

1

0123456

stro

ngly

disa

gree

disa

gree

unde

cide

d

agre

e

stro

ngly

agre

e

Num

ber o

f ATC

Os

The split of the NO sector acceptable

1

2

3

0

1

2

3

4

5

6

stron

gly di

sagree

disag

ree

unde

cided

agree

stron

gly ag

ree

Num

ber o

f ATC

Os

Figure 7-14: Acceptability of organisation 2 Figure 7-15: Acceptability of the NOR split

The split of the NO sector decreases the need for coordination

3

2

1

0

1

2

3

4

5

6

stron

gly di

sagr

ee

disag

ree

unde

cided

agre

e

stron

gly ag

ree

Num

ber o

f ATC

Os

Do the new route and airspace structure plus the traffic increase create any

problems?

5

1

0

1

2

3

4

5

6

yes no

Num

ber o

f ATC

Os

Figure 7-16: Feedback on the splitting of the

NOR sector Figure 7-17 : Feedback on problems created



The splitting of the NOR sector not only failed to bring benefits but also created an additional problem. The sectors became too small to solve problems in time without coordination, since the aircraft arrived late – with reference to the conflict point – on the frequency. Additionally, since the two sectors DKAR and NOR shared the majority of traffic, the need for coordination could not be decreased.

Coordination

The coordination figures retrieved from the telephone calls made by the PC confirmed the feedback from the controllers.

Mean number of phone calls per traffic samplOranisation 2 - position : KA/PLC

A7 A9 X7 X9

Mea

n nu

mbe

r of

pho

ne c

alls

0

15

30

45

60

75

90

105

120

135

Mean number of phone calls per traffic samplOranisation 2 - position : NO/PLC

A7 A9 X7 X9

Mea

n nu

mbe

r of

pho

ne c

alls

0

11

22

33

44

55

66

77

88

99

Figure 7-18: Number of telephone calls in the DKAR sector

Figure 7-19: Number of telephone calls in the NOR sector

Mean number of phone calls per traffic samplOranisation 2 - position : MG/PLC

A7 A9 X7 X9

Mea

n nu

mbe

r of

pho

ne c

alls

0

11

22

33

44

55

66

77

88

99

Figure 7-20: Number of telephone calls in the MGB sector



The graphs above indicate the mean number of telephone calls for coordination in the NOR, MGB and DKAR sectors. An average of between 75 and 90 telephone calls were made by the PC in the DKAR sector, and an average of between 50 and 75 telephone calls were made by the PC in the NOR sector. In the MGB sector an average of between 50 and 65 telephone calls were made.


The new route structure was not acceptable to the controllers because of the increase in traffic. The splitting of the NOR sector, which was supposed to redress the situation, did not bring the expected benefits; more coordination was required for traffic common to these sectors. Additionally, the sector’s vertical extension was reduced by the split, which made conflict resolution more time critical.



8 LVNL – EXPERIMENT RESULTS

The following chapter presents the results for Amsterdam. The data forming the results were derived from the most impacted sector, 2E.

Note: the quantitative calculation for the LvNL sectors was based only on sector 2E. The reason for this limitation was that some of the planned exercises were modified as described in ANNEX Appendix A.1, and therefore not all of the data set was collectable.

Only significant results are presented for the comparison between organisation 0 and organisation 1. Furthermore, only results with the "organisation" factor are presented, i.e. differences relating only to the traffic sample (e.g. the results for exercises with traffic 1 active were better than exercises with traffic 2 inactive, with no effect in terms of the organisation) are not reported, since the objectives of the simulation referred to the comparison of the organisations. The results are always reported with reference to the high-level objectives.


Capacity was not measured directly. The focus in AMRUFRA was on controller workload, as this is the key factor constraining sector capacity. Controller workload was expected to change as a function of the task load imposed on the controller, e.g. the number of radio calls to aircraft, and coordination and telephone calls to neighbouring sectors.

Whereas controller workload was assessed on the basis of subjective workload ratings, changes in controller task load could be directly and objectively derived from the controllers' behaviour in the course of their interaction with the system. In the following subsections, the results obtained from the various task-load measurements are reported, followed by the results for subjective workload.

Workload is understood as a subjective quality reflecting the individual controller's personal perception of the task load imposed on him/her by the particular air traffic situation. Subjective workload ratings are therefore determined to some extent by a component that cannot be derived solely using indicators of task load. Hereinafter all subjective measurements are referred to as "workload measurements", and all objectively recordable measurements of controller activity occurring in response to the tasks imposed on the controller are referred to as "task-load measurements".







With regard to these task-load measurements, the following hypotheses were examined using the analyses of recorded data:


Controllers will need to perform fewer tasks with the new route and airspace structure in ORG1(A).



8.1.1 Task load

In section 3, a set of task-load measurements was described that were expected to be influenced by the new route structure and that had the potential to alleviate controller workload in comparison to the baseline (ORG0).

For LvNL, organisation 1 contained, first, a route change that increased the distance between the Amsterdam departure and arrival routes and, second, a splitting of the departure route, which "naturally" divided the traffic flows. This change allowed the hypothesis that fewer headings would be performed in organisation 1. In the baseline – organisation 0 – the controller had to separate the traffic in a tactical way with heading instructions. With the new route and airspace structure (organisation 1), however, separation was provided by the nature of the new route structure.

traffic 1

ORG0 ORG1

no o

f hea

ding

0

10

20

30

40

Figure 8-1: Heading instructions

Figure 8-1 indicates a significant difference for the seven o'clock traffic sample (traffic 1). In this traffic sample, fewer heading instructions were given to the pilots in organisation 1 than in organisation 0. The fact that the hypothesised benefit of fewer heading instructions could not be observed under all conditions is attributable to the Volkel area and the new airspace structure.



The explanation given by controllers for this result in the debriefings and questionnaires was that the traffic needed to be vectored to avoid the area, and that since the airways were shorter in the new route structure more traffic was affected. The benefits brought by splitting the routes were thus partly offset by the Volkel area.

No other values for workload or task load for LvNL were significant.

LvNL results for objective 1

The hypothesised reduction in heading instructions with the new route structure was partly confirmed. For traffic 1 (7 o'clock traffic with fewer Frankfurt departures), fewer heading instructions were given to the pilots in organisation 1 than in organisation 0. This result could not be confirmed for traffic 2. Furthermore, the hypothesis that controllers would perceive a lower workload in the new organisation was not confirmed. The benefits expected from the splitting of the route were partly offset by the implementation of the Volkel area.


In the debriefings, the controllers were asked whether or not the new route and airspace structure represented by organisation 1 was acceptable.

No specific hypotheses were formulated with regard to these quality aspects of the new route and airspace structure. The questionnaire completed by the controllers (number of ATCOs that filled in the questionnaires = 8) at the end of week two, after all exercises for organisation 0 and organisation 1 had been simulated, contained questions regarding the acceptability of the new route structure in organisation 1 in terms of its impact on workload and its safety impact. While only a small number of results are presented in this chapter, the full and complete feedback from the controllers can be found in Appendix A.11. Organisation Questionnaires : MUAC Feedback Results.

The split of the eastbound routes in Sector 2E is acceptable

12

5

012345678

stron

gly di

sagr

ee

disag

ree

unde

cided

agre

e

stron

gly ag

ree

Num

ber o

f ATC

Os

The new sector shape is acceptable (compared w ith ORG 0)

5

12

0

1

2

3

4

5

6

7

8

stron

gly di

sagr

ee

disag

ree

unde

cided

agre

e

stron

gly ag

ree

Num

ber o

f ATC

Os

Figure 8-2: Acceptability of the eastbound route split

Figure 8-3: Acceptability of the new sector shape



Eight controllers participated in the simulation exercises for organisation 0 and organisation 1, and gave their opinions on the acceptability of the new route structure. Seven controllers rated the splitting of the eastbound routes in sector 2E as acceptable. With this new route structure, the traffic was separated "naturally" and did not need any additional monitoring or action from the controller. In addition, the splitting of the routes provided more airspace for manoeuvring the aircraft.


The new route and airspace structure was acceptable to the majority of the controllers, as the route split "naturally" separated the traffic.

8.3 ASSESS THE IMPACT OF THE NEW ROUTE STRUCTURE AND AIRSPACE STRUCTURE ON SAFETY





Hypothesis 3: STCAs


ORG0 ORG1

mea

n nu

mbe

r of S

TCA

0

2

4

6

Figure 8-4: Number of STCAs (sector 2E)

The hypothesis could not be confirmed since significant results could be revealed. More STCAs were recorded for traffic sample 1 in ORG1.




In the post-exercise questionnaire, controllers were asked to express their safety concerns regarding organisation 1.

Does the new route and airspace structure create any new problems (safety concerns)?

8

0

1

2

3

4

5

6

7

8

yes no

Num

ber o

f ATC

Os

Figure 8-5: LvNL safety concerns

Eight of the nine controllers stated that the new route structure created new problems. The following problems and safety concerns were raised:

• Slow-climbing traffic owing to the shorter routes with the new route structure

• The Volkel area

The main concern mentioned by all participants related to the Volkel area. The activation of this military area would require tactical vectoring of the traffic. The vectoring was necessary as soon as an aircraft was unable to climb over the military area. This had an influence on other traffic streams, and created more workload and more coordination.

No effect on the controllers' situational awareness could be detected.


The controllers stated that the new route structure raised new safety concerns linked to the Volkel area. This was confirmed by a significant difference in the number of STCAs, since more STCAs were recorded in organisation 1. No safety concerns regarding the new route split were mentioned by the controllers.




Since the problems with the Volkel area were already evident during the simulation, the following changes were made to the organisation 2 exercises:

• Change in the RENDI area: the levels between FL 145 and FL 195 were controlled and used by LvNL, while the military unit, Dutch Military (DM), notified LvNL when they had traffic flying in this area. This change resulted in less coordination for the LvNL PC.

• Change in the NAPSI area: the borders were extended to the south, which allowed some extra time for aircraft in this area; less coordination was therefore needed from the controllers.

A questionnaire was distributed to the participants (number of ATCOs = 4) after organisation 2. The controllers were asked to rate the acceptability and feasibility of the new route and airspace structure with the traffic increase.

Controller Feedback

The split of the eastbound route in sector 2E with the traffic increase is acceptable.

2 2

0

1

2

3

4

strong

ly dis

agree

disagree

undecid

ed

agree

strong

ly agree

Num

ber o

f ATC

Os

The new sector shape with the traffic increase is acceptable (compared to ORG 0)

1

2

0

1

2

3

4

strong

ly dis

agree

disagree

unde

cided

agree

strong

ly ag

ree

Num

ber o

f ATC

Os

Figure 8-6: Acceptability of route split with traffic increase

Figure 8-7: Acceptability of new sector shape with traffic increase

Controllers stated in the debriefings that they were busy but that the traffic was still manageable, and that this amount of traffic would not have been manageable in the route structure represented by organisation 0.

Does the traffic increase in ORG 2 create any new problems (Safety concerns)?

4

0

1

2

3

4

yes no

Num

ber o

f ATC

Os

The RENDI area up to 145 in ORG 2 is acceptable

4

01234

stro

ngly

disa

gree

disa

gree

unde

cide

d

agre

e

stro

ngly

agre

eNum

ber o

f ATC

OS

Figure 8-8: Problems created by the traffic increase

Figure 8-9: Acceptability of new RENDI area




The new route and airspace structure with the increase in traffic was considered acceptable by the controllers. It was also stated that this amount of traffic, as simulated in organisation 2, would have been unworkable with the old route and airspace structure (ORG0).

In organisation 2, changes to the Volkel area were implemented, eliminating all the safety concerns mentioned in organisation 1.



9 MUAC – EXPERIMENT RESULTS

The following chapter presents the result for Maastricht UAC. Only significant results will be presented for the comparison of organisation 0 with organisation 1. Furthermore, only results relating to the factor "organisation" are presented, i.e. differences in relation only to the traffic sample or in relation only to the activity of the military area TRA A (e.g. the results for the exercises with TRA12A active were better than for exercises with TRA12A inactive, with no effect in terms of the organisation) are not reported, since the objectives of the simulation referred to comparison of the organisations. The results are always reported with reference to the high-level objectives.


As explained in chapter 2 capacity was not measured directly. The focus in AMRUFRA was on controller workload, since this is the key factor constraining sector capacity. Controller workload was expected to change as a function of the task load imposed on the controller, e.g. the number of radio calls to aircraft, and coordination and telephone calls to neighbouring sectors. Whereas controller workload was assessed on the basis of subjective workload ratings, changes in controller task load could be directly and objectively derived from the controllers' behaviour in the course of their interaction with the system. In the following subsections, the results obtained from the various task-load measurements are reported first, followed by the results for subjective workload. Workload is understood as a subjective quality reflecting the individual controller's personal perception of the task load imposed on him/her by the particular air traffic situation. Subjective workload ratings are therefore determined to some extent by a component that cannot be derived solely using indicators of task load. Hereinafter all subjective measurements are referred to as "workload measurements", and all objectively recordable measurements of controller activity occurring in response to the tasks imposed on the controller are referred to as "task-load measurements".





With regard to these task-load measurements, the following hypotheses were examined using the analyses of recorded data:


Controllers will have to perform fewer tasks with the new route and airspace structure in ORG1(A).





As stated in section 2, no hypotheses were formulated for the MUAC sectors. MUAC's aim was to identify further improvements to the route and airspace structure, and to investigate whether or not the proposed route structure was feasible in MUAC airspace. Nevertheless, the results are presented in the same way as for the other ANSPs. The statistical significance was calculated across the sectors, while the graphs illustrate the values for each sector.

9.1.1 Task load

No significant difference in the number of telephone calls made in each of the two organisations was observed.

SYSCO coordination

No significant difference in terms of SYSCO coordination was found.

R/T communication

No significant main effect in terms of R/T communication was found.

Instructions to pilots

A significant difference was found in the number of heading instructions given. Significantly fewer heading instructions were given in organisation 1 than in organisation 0.

Heading instructions

ORG0 ORG1

mea

n nu

mbe

r of h

eadi

ng in

stru

ctio

ns

0

2

4

6

8

10

12

DH DL RR

Figure 9-1: Heading instructions

The new route structure was designed in such a way as to reduce conflict points, which yielded the benefit that less heading instructions were necessary in organisation 1 than in organisation 0. An example in relation to such conflicts is the activation of the military area's window 1, which had an effect on the Frankfurt departure traffic. With the new route structure, this traffic flow was no longer affected by window 1, and therefore the traffic did not have to be vectored around.



Direct instructions

In addition to the heading instructions, a significant first-order interaction between traffic and organisation was also found. The interaction related to traffic 2, for which significantly fewer direct instructions were given in organisation 1 than in organisation 0.

Direct instructions

traffic 1

ORG0 ORG1

mea

n nu

mbe

r of d

irect

0

10

20

30

40

50

DH DL RR

traffic 2

ORG0 ORG10

10

20

30

40

50

Figure 9-2: First-order interaction between traffic and organisation for direct instructions

Traffic sample 2 was characterised not only by an increase in the number of Frankfurt departure flights but also by an increase in the number of flights in general. The routes in the new route structure were designed in order not only to reduce the number of conflict points but also to cross the MUAC sectors in the most direct way. This reduced the need for direct routing instructions. The benefits were more visible in the Ruhr sector and in the traffic sample with the higher volume of traffic (traffic 2).

9.1.2 Workload

The controllers' perception of workload was addressed through a combination of workload-assessment techniques. With the instantaneous self-assessment (ISA), workload ratings were collected during each exercise (every two minutes), whereas the NASA-TLX questionnaire was completed after each exercise.


For the analysis, ratings given on a five-point scale labelled "very low" (1), "low" (2), "fair" (3), "high" (4), and "very high" (5) were averaged across two-minute intervals to obtain an average rating for each exercise.

The mean ISA rating of the PC revealed a significant first-order interaction between the factors "organisation" and "traffic". The PC experienced higher workload in traffic sample 2 (9 o'clock) than in traffic sample 1 (7 o'clock) under the conditions of organisation 1. However, it should be pointed out that workload was assessed as between "very low" and "fair".



NASA Task-load Index (TLX)

Controllers gave their ratings after each exercise using a 20-point scale ranging from "low" to "high", with no scale labels given at intermediate scale levels (see the form at annex). An ANOVA was performed on the unweighted NASA-TLX score obtained by averaging the collected ratings over the six subscales (mental demand, physical demand, temporal demand, performance, effort, and frustration).

A significant first-order interaction between the factors "traffic" and "organisation" was found for the PC.

NASA overall PC

traffic 1

ORG0 ORG1

NA

SA

mea

n ov

eral

l

0

5

10

15

20DH DL RR

traffic 2

ORG0 ORG10

5

10

15

20

Figure 9-3: First-order interaction between traffic and organisation:

NASA-TLX overall score for the PC

The result indicates that the PCs' workload was greater for traffic sample 2 (9 o'clock) than for traffic sample 1 (7 o'clock) with the new route structure, independently of the activation of the military area TRA A. The same pattern emerged in the NASA-TLX mental demand subscore for the PCs. A higher mental demand was perceived by the PCs in organisation 1 with traffic sample 2 than in organisation 1 with traffic sample 1.

The workload increase with traffic sample 2 resulted from the increase in Frankfurt departure traffic and in the number of flights in general with this traffic sample. The workload increase was seen especially in the Delta low sector.

MUAC results for objective 1

In summary, the task-load recordings show the benefits of organisation 1 under traffic sample 2 conditions, whereas the workload results do not indicate these benefits.

While fewer direct instructions were issued for traffic sample 2 (9 o'clock) than for traffic sample 1 (7 o'clock) with the new route structure, the PCs' workload increased for traffic sample 2. The workload increase can be explained by the fact that traffic sample 2 was characterised by more flights, in particular more Frankfurt departures, and more flights in general. The heading instructions revealed clear benefits from organisation 1, independently of the traffic sample. Fewer headings were issued by the ECs in organisation 1 than in organisation 0.




In the debriefings, the controllers were asked if the new route and airspace structure represented in organisation 1 was acceptable.

No specific hypotheses were formulated with regard to these quality aspects of the new route and airspace structure. The questionnaire completed by the controllers at the end of week two, after all exercises for organisation 0 and organisation 1 had been simulated, contained questions regarding the acceptability of the new route structure in organisation 1 in terms of its impact on workload and its safety impact. While only a small number of results are presented in this chapter, the full and complete feedback from the controllers can be found in Appendix A.11 Organisation Questionnaires : MUAC Feedback Results.

The new route structure is acceptable

1

6

1

0

1

2

3

4

5

6

stron

gly di

sagr

ee

disag

ree

unde

cided

agre

e

stron

gly ag

ree

Num

ber o

f ATC

Os

The new interface with the new sector shapes of the adjacent ANSPs is acceptable

(compared to ORG0)

34

01

234

56

stron

gly di

sagr

ee

disag

ree

unde

cided

agre

e

stron

gly ag

ree

Num

ber o

f ATC

Os

Figure 9-4: Acceptability part I Figure 9-5: Acceptability part II

Seven of the eight controllers rated the new route structure as acceptable, and four of the eight controllers (one ATCO did not give feedback on this item) rated the interface with adjacent ANSPs in the new route structure as acceptable. The reasons for the controllers' feedback were that in the new route structure the Amsterdam departure route and the Frankfurt arrival route were de-conflicted, that the new route structure provided more time to climb outbound traffic and that the Frankfurt departure traffic did not conflict with the military area's window 1. Controllers also stated that the new route and airspace structure made the coordination process with the NOR sector easier. This was because a general release for Frankfurt departure traffic to FL 260 was agreed, while in organisation 0 each flight had to be coordinated for a higher level.


Most of the controllers considered the new route and airspace structure to be acceptable. A major argument put forward in favour of this was the de-confliction of the Frankfurt arrival route and the Amsterdam departure route.

For the Frankfurt departure route, the benefits of the release up to FL 260 were also mentioned.



9.3 ASSESS THE IMPACT OF THE NEW ROUTE STRUCTURE AND AIRSPACE STRUCTURE ON SAFETY





Hypothesis 3: STCAs


No significant difference was found between the organisations in terms of the number of STCAs.


Does the new route and airspace structure create any problems (safety concerns)

(ATCOs from Hannover sectors)?

1

2

0

1

2

3

45

6

7

8

9

yes no

Num

ber o

f ATC

Os


(ATCOs from DECO sectors)?

2

3

0

1

2

3

4

5

6

7

8

9

yes no

Num

ber o

f ATC

Os

Figure 9-6: MUAC Hannover sectors:

safety concerns Figure 9-7: MUAC Deco sectors: safety

concerns

Five of the eight controllers did not anticipate any safety concerns, whereas three controllers expressed the following concerns:

• the turn of the route right in the military area, and

• the crossing over of MIMVA-RAVLO-GORLO traffic.

The turn after the military area did not allow planning for the resolution of potential conflicts in time, and the crossing over of MIMVA-RAVLO-GORLO traffic created a more complex traffic pattern. This complexity required additional effort from the controllers.

No significant difference regarding situational awareness was found.




The controllers raised certain safety concerns in the new organisation. First, a turning route in the military area did not allow proper planning and coordination. Second, there was a cross-over between two routes and therefore an increase in the number of conflict points, which required increased attention on the part of the controller.


As an initial result of the safety concerns arising in organisation 1, a change was implemented in organisation 2 in the MUAC airspace:

• The waypoints BREDA and TOTNA were moved further to the west to move the crossing point out of the military area.

• Amsterdam departure flights were directly coordinated with LvNL rather than with MGB as before.

A questionnaire was distributed to the participants (number of ATCOs = 12) after organisation 2. The controllers were asked to rate their acceptability and the feasibility of the new route and airspace structure with the traffic increase.


Deco sector group (number of ATCOs = 7)

The benefits of the new route structure are still valid with the traffic increase

1

4

2

0123456

stron

gly di

sagree

disag

ree

unde

cided

agree

stron

gly ag

ree

Num

ber o

f ATC

Os

Does the new route and airspace structure create any problems (Safety concerns)?

3

4

0

1

2

3

4

5

6

yes no

Num

ber o

f ATC

Os

Figure 9-8: Validity of the benefits with the traffic increase for the Deco sectors

Figure 9-9: Problems in organisation 2 for the Deco sectors

Six of the seven controllers agreed that the benefits generated by organisation 1 were still valid with the traffic increase. Three of the seven controllers experienced safety concerns in organisation 2.

The safety concerns related to Stansted inbounds, which could not descend until very late owing to the traffic increase. Furthermore, the crossing of route TB6 with routes UZ81 and UZ292 created a safety concern. Additionally, a large amount of coordination between DDL, DZH and DFH was needed, since the conflict between Frankfurt departures and London TMA arrivals was close to the London FIR boundary.



Nevertheless, the general improvements raised by the new route and airspace structure were still valid.

Hannover sector group (Ruhr) (number of ATCOs = 5)


1

4

0

1

2

3

4

5

stron

gly di

sagr

ee

disag

ree

unde

cided

agre

e

stron

gly ag

ree

Num

ber o

f ATC

Os

Does the new route and airspace structure create any problems (Safety

concerns)?

1

4

0

1

2

3

4

5

yes no

Num

ber o

f ATC

Os

Figure 9-10: Validity of the benefit with the

traffic increase for the Hannover sectors Figure 9-11: Problems in organisation 2 for

the Hannover sectors

Five controllers agreed that the benefits generated by organisation 1 were still valid with the traffic increase. Four of the five controllers did not experience safety concerns in organisation 2. The concern mentioned by one controller related to the activation of military area TRA12A, since traffic could not fly via route UZ81 (the Frankfurt departure route) and needed to be rerouted via route UZ291. This was hindered by the limited time available for rerouting via R/T.


The new route and airspace structure was acceptable to the majority of controllers even with the traffic increase in organisation 2. Nevertheless, certain safety concerns were raised, especially in the Deco sectors.



10 ASSESS THE IMPACT OF THE NEW ROUTE STRUCTURE AND AIRSPACE STRUCTURE ON THE ROUTE LENGTH

The route length measurements are presented in a separate chapter and not related to the ANSPs, as parts of chapter are calculated on the bases of the entire airspace.

The assumption was made that the new routes are designed in a more direct way and therefore shorter way through the airspace.

Distance flown per sector-

Mean value per aircraft and per traffic sample-

Sector 2E - Organisation 0

A7 A9 X7 X9

sect

or fl

own

dist

ance

(Nm

)

0

10

20

30

40

50

60

70

80

90

100



Sector 2E - Organisation 1

A7 A9 X7 X9

sect

or fl

own

dist

ance

(Nm

)

0

10

20

30

40

50

60

70

80

90

100

Figure 10-1: Distance flown in the 2E sector in organisation 0

Figure 10-2: Distance flown in the 2E sector in organisation 1

Distance flown per sector

-Mean value per aircraft and per traffic sample

-Sector 2I - Organisation 0

A7 A9 X7 X9

sect

or fl

own

dist

ance

(Nm

)

0

10

20

30

40

50

60

70

80

90

100

110

120

130

140

150



Sector 2I - Organisation 1

A7 A9 X7 X9

sect

or fl

own

dist

ance

(Nm

)

0

10

20

30

40

50

60

70

80

90

100

110

120

130

140

Figure 10-3: Distance flown in the 2I sector in organisation 0

Figure 10-4: Distance flown in the 2I sector in organisation 1





Sector DL - Organisation 0

A7 A9 X7 X9

sect

or fl

own

dist

ance

(Nm

)

0102030405060708090

100110120130140150160170180



Sector DL - Organisation 1

A7 A9 X7 X9

sect

or fl

own

dist

ance

(Nm

)

0102030405060708090

100110120130140150160170180

Figure 10-5: Distance flown in the DDL sector in organisation 0

Figure 10-6: Distance flown in the DDL sector in organisation 1



Sector DH - Organisation 0

A7 A9 X7 X9

sect

or fl

own

dist

ance

(Nm

)

0102030405060708090

100110120130140150160170180190




A7 A9 X7 X9

sect

or fl

own

dist

ance

(Nm

)

0102030405060708090

100110120130140150160170

Figure 10-9: Distance flown in the DDH sector in organisation 0





A7 A9 X7 X9

sect

or fl

own

dist

ance

(Nm

)

0102030405060708090

100110120130140150160170180190




A7 A9 X7 X9

sect

or fl

own

dist

ance

(Nm

)

0102030405060708090

100110120130140150160170





Only the sectors in which there was no change in sector shape between organisation 0 and organisation 1 are displayed. For the DDH and the Ruhr sectors, a tendency towards more efficient and therefore shorter routes in organisation 1 than in organisation 0 was observed, while for the TAU sector the opposite was valid. For the LvNL sectors, no major difference was observed. However, the data must be considered as descriptive only.

The route length was computed in order to compare the current (old) route structure with the new one. No major differences could be highlighted, as can be seen in Table 10-1:

Table 10-1: Length of the routes in the old route structure compared with the new route structure

Route name Distance (NM) % shorter

UZ29 new 65.258 -2.21%

UZ29 old 66.731

UZ29/291/292 new 191.325 3.05%

UZ29/291/292 old 185.658

UZ81 (CDR) new 164.072 -4.99%

UZ728/UP73 (CDR) old 172.697

UT 150 new 97.249 -3.90%

T 150 old 101.193

UZ738 (DISOS - COL) new 132.183 -4.45%

UL620/603/UZ738 old 138.344

UZ738 (DISOS – GMH) new 129.071 1.68%

UL620/603 old 126.936



11 FEEDBACK FROM THE MILITARY UNITS

The military units Lippe Radar (LR) and Dutch Military (DM) were facilitators for the simulation. Although the working positions were not measured quantitatively, the participants were asked for feedback. While organisation 2 was acceptable to both the0 LR and DM units after a number of changes, organisation 1 had a number of shortcomings.

DM

• The Volkel area simulated at the beginning of the simulation did not provide enough space for the needs of the military unit.

• Descend traffic on TB & eastbound was nearly impossible regarding opposite traffic streams.

• It was hard to get traffic down out of Window 1 to fit within the designed Volkel area at first.

LR

• Route GIX IBAGU TL2N was hampered by the new Frankfurt departure route, which necessitated a considerable amount of coordination and vectoring.

• UZ 81 (Frankfurt departure) was too close to the Amsterdam departure route, which did not allow enough space for military traffic.

• ETAR (Ramstein) military arrivals had to descend through the Amsterdam departure traffic flow; since the conflict point was close to the boundary, traffic had to be vectored to descend and re-vectored to the original route afterwards.

• ETAR/ETAD (Rammstein/Spangdahlem) military arrivals had to descend through the Amsterdam departure, Frankfurt departure and Frankfurt arrival traffic flows, necessitating an early descent into Dutch Military airspace.

• EDDF and EHAM departures created problems with ETAD/ETSB (Buechel) arrivals via MAXID.



12 CONCLUSIONS

12.1 DFS

To overcome the delays and restrictions in German airspace caused by a traffic increase, the following solutions were proposed: (1) change the sector volume and shape; (2) change the location of the Frankfurt arrival and departure routes; and, as a consequence, (3) spread the Düsseldorf routes to the south-east. The DFS objective for the AMRUFRA RTS was to assess the impact of the new sector volume and shape, and of the new Frankfurt arrival and departure routes. It was hypothesised that the new sector volume and shape would result in less telephone coordination and therefore less workload for the controller. Furthermore, it was to be assessed whether or not the new airspace design facilitated the forecast traffic increase.

The results of the AMRUFRA RTS indicated that the new route and airspace structures (ORG1), independently of the activation of TRA12A, were not acceptable to the controllers participating in the simulation. This feedback was supported by a significant increase in subjective workload and objective measured task load. Overall, more instructions were issued to the pilots and more telephone coordination was performed in the new organisation (ORG1) than in the reference organisation (ORG0). This result was partly explained by the descriptive data for the sector load, since more aircraft – especially in the NOR sector – were counted in organisation 1 than in organisation 0. It was partly explained by the complex nature of the route structure, which was characterised by vertical movements, and the new traffic flows (e.g. Frankfurt departure) in the NOR sector. The departure traffic had to be vectored out of the inbound flow, which explained the increase in the number of heading instructions, and the vertical movements required more telephone coordination, since new exit flight levels had to be coordinated. Therefore, the hypothesised reduction in coordination and therefore in telephone calls did not materialise. Moreover, this increased complexity as a result of the integration of parts of the TAU sector into the measured NOR sector, thereby increasing the number of conflict points:

• comprised by the Cologne arrival flow from the south,

• with Frankfurt arrivals, and

• with the arrival flows to the military bases.

In ORG2 a traffic increase of up to 20% was simulated. In order to accommodate the traffic increase, the Nörvenich sector was split into the DKAR (Köln Arrival) and NOR (Nörvenich) sectors. However, this split did not bring the expected benefits, since the majority of traffic was common to both sectors, which did not reduce the need for coordination; as the sectors became smaller, the controllers perceived more time pressure in resolving the conflicts. The new route and airspace structure with the traffic increase was therefore not acceptable to the ATCOs.

12.2 LVNL

The general traffic increase in recent years had saturated the Amsterdam departure route, and the forecast traffic increase had led to the assumption that major delays to Amsterdam departure traffic were to be expected. This traffic increase was expected to lead to a workload increase caused by tactical control via heading instructions, especially to aircraft on this Amsterdam departure route. As a solution, it was proposed to (1) expand the airspace to the south and (2) segregate the Amsterdam departure stream by creating a new route. The LvNL objective for the AMRUFRA RTS simulation was to validate these airspace changes. It was hypothesised that the new route and airspace structure would result in less tactical control, i.e. less heading instructions issued to aircraft. Furthermore, it was to be assessed whether or not the new airspace design facilitated the forecast traffic increase.



The hypothesised reduction in heading instructions with the new route structure was partly confirmed. For traffic sample 1, fewer heading instructions were given to pilots in organisation 1 than in organisation 0. This could not be confirmed for traffic sample 2 or for ATCO workload.

Furthermore, hypothesis 2, which postulated that there would be less workload in organisation 1, could not be confirmed. No difference was recorded between the workload in organisation 0 and organisation 1. The benefits expected from the route split were partly offset by the implementation of the Volkel area, since the new route structure imposed a shorter route, and aircraft that were not able to climb above the Volkel area had to be vectored around it. Additionally, the hypothesis that the new organisation would result in more space to manoeuvre aircraft, thus leading inter alia to fewer STCAs, was not confirmed. More STCAs were recorded in organisation 1 than in organisation 0.

Because of the safety concerns raised during the simulation of organisation 1, the Volkel area was changed in organisation 2. The result of these changes was that the new route and airspace structure was acceptable to the participants, since Amsterdam departure traffic was "naturally separated", and the safety concerns were eliminated.

The controllers also mentioned that the traffic increase simulated in organisation 2 would not have been acceptable with the old route structure, whereas it was workable with the new route structure.

12.3 MUAC

The proposed route changes in LvNL and DFS sectors had an impact on Maastricht UAC. It was thought that the new organisation might resolve the conflict points in the Ruhr sector but shift the conflict area to the Deco sector. For example, Frankfurt arrivals and departures currently intersected in the Ruhr sector, often leading to complex situations. In the new organisation these flows were well segregated in the Ruhr sector, but a more complex traffic pattern was created in the Deco sectors. The MUAC objective for the AMRUFRA RTS was to find a balance between the benefits and trade-offs of the proposed solutions.

The majority of controllers rated the new route and airspace structure as acceptable. A key argument for accepting it was the de-confliction of the Frankfurt arrival route and the Amsterdam departure route. Moreover, the military area “window 1” had a less negative impact in the new route structure, which required fewer heading instructions from the controllers. Furthermore, in addition to the fact that the new route structure was defined in a way that decreased conflict points and therefore heading instructions, the routes themselves were designed to take the most direct course through the sectors, which reduced the need for direct instructions. In organisation 1 under the conditions of traffic sample 2 (9 o'clock), fewer direct instructions were issued. However, this benefit was not reflected in the controllers' workload. The PCs experienced more workload in organisation 1 under the conditions of traffic sample 2 than in organisation 0. The increased workload in organisation 1 under traffic sample 2 conditions was attributable to the effects of the traffic increase in traffic sample 2, especially the increase in the number of Frankfurt departures.

The workload increase was perceived especially in the Delta low sector. The expected drawback for the Delta sector could thus be partly confirmed. However, no workload ratings exceeded the mean score of "fair".

The cross-over between two routes and the increase in the number of conflict points were raised by the controllers as safety concerns for the new organisation (ORG1); however, these issues were addressed in the changes and improvements simulated during the last days of the simulations (ORG 4). The activity of TRA12A did not have any influence in dependence of the organisation.



The benefits of organisation 1 were seen by the ATCOs as still being valid with the increase in traffic in organisation 2. However, certain issues gained importance with the traffic increase, especially in the Delta sector:

• Stansted inbound traffic could not be descended until a late stage.

• The cross-over between routes TB6 and UZ81 raised safety concerns.

• There was limited time on the frequency for rerouting instructions.

• A large amount of coordination between DDL, DZH and DFH was needed, since the conflict point between Frankfurt departures and London TMA arrivals was close to the London FIR boundary.

The new route structure's efficiency in terms of route length was observed for the MUAC sectors, especially for the Ruhr sectors. In organisation 1, the tendency of a shorter distance flown by the aircraft in organisation 1 then in organisation 0, was measured.

12.4 MILITARY UNITS

The military units were not directly measured, but the subjective feedback and expert input of the military controllers was collected. The new route and airspace structure simulated in organisation 1 raised certain safety concerns for the military units.

The Volkel area did not provide enough space for Dutch Military's training needs. The new Frankfurt departure route had a major impact on the German military unit, since it required more coordination and vectoring and did not allow enough space for the performance of military training. The new Amsterdam departure route also created problems which required more vectoring of aircraft.

Certain changes were made to the route structure in order to facilitate the training needs of the military unit, which resulted in a satisfactory final outcome.

12.5 GENERAL CONCLUSION

In summary, it can be said that while for two of the three ANSPs the benefits of the new route and airspace structure were confirmed and demonstrated, DFS experienced a major negative impact, which made the new route and airspace structure unfeasible.



13 RECOMMENDATIONS

13.1 SIMULATED IMPROVEMENTS

As stated in the conclusions regarding the planned organisations (ORG1 and 2), the results were not satisfying for all parties. Adjustments (changes to UZ291/81, the NAPSI area and the EHAM departure rules) were agreed on in week 2 of the simulation, optimising the results where possible.

Table 13-1: Improvements in the route structure/ORG2

Improvements applied in ORG2 and retained in ORG 4

Problem Solution

When TRA12A was inactive, traffic flying via route UZ291/292 crossed traffic flying on UZ81 at the same FL at MONIL (crossing point).

Routes UZ81 and UZ291/292 were put on parallel tracks to prevent this traffic from crossing (de-conflictions).

Because of the large turns on UZ291/292 and UZ81, conflicts were hard to predict in the Deco sectors (impact on conflict resolution).

The turning points on the parallel routings (UZ291/81) were moved to the west (outside the lateral limits of TRA12) to allow earlier turns (without prior coordination with DM), hence allowing time to solve potential conflicts with crossing north-south tracks in the Deco sectors.

The crossing point of UZ81/TB6 was on the boundary (conflict resolution).

The crossing point was moved out of the military area (see above).

Slow-climbing EHAM departures cross the MGB sector, resulting in additional coordination/workload.

The NAPSI area has been adjusted to avoid generally the crossing of the MGB sector; hence coordination is in general no longer needed.

These modifications, however, did not allow acceptable airspace design for the DFS. Therefore, the partners agreed to re-design the original ORG1 (associated with a 20% increase) to form a renewed airspace structure, taking into account the benefits of ORG1 and ORG2 for all three ANSPs.

This updated airspace design was marked as ORG 4 and validated during the last 3 exercises of week 3.



Table 13-2: Improvements in the route structure/ORG 4

Improvements applied in ORG 4 only

Problem Solution

Military had problems descending the ETAD/ETAR arrivals.

The northerly EDDF departure route (UZ81) was moved to the south (BOKDA to DIBIR), creating more space for the Lippe Radar (LR) to descend traffic. Furthermore, the southern EHAM departure route (UZ738) was slightly bent to the east to allow more space for LR to vector aircraft.

EDDF arrivals converged with the EHAM departures.

In the EDDF arrival stream a dog-leg was created (COFFA) to allow parallel tracks until a level change was established.

The Brussels arrival route was shifted out of the NOR sector with a new sector sequence, RR-MGB.

Overload of Nörvenich sector

The re-sectorisation of the DFS Langen sectors in ORG1 and 2 was considered unworkable by the controllers. The route structure in ORG 4 was established in such a way as to have no negative impact on the current DFS Langen sectorisation, hence allowing possible implementation in the near future.

Three exercises were carried out in which the new changes in the route structure (ORG 4) were validated. However, as DFS needed significant changes in the airspace and route design, which could not be implemented at such short notice in the simulation, the exercises concentrated on the MUAC and LvNL sectors. The DFS sectors served as feed sectors. By way of general feedback, it can be said that the improvements were experienced as beneficial for the Maastricht (MUAC) sectors, especially for the Hannover sectors and for Amsterdam (LvNL).

13.2 FURTHER RECOMMENDATIONS

Following the changes to the airspace (ORG 4), a number of recommendations were made:

• Deco sector overflights should be routed via DIDAM in order to achieve a better/faster split from EDDF arrivals.

• The space between the Düsseldorf departure and Amsterdam departure routes should be increased.

• The changes resulting from the activation of TRA12A should be investigated.

• The procedures for the separation of military traffic and Amsterdam departures should be improved: optimisation of procedures between GAT and OAT (especially within MUAC) is necessary.

• A parallel routing for Amsterdam departures, Frankfurt arrivals and overflights should be created.

In addition to these explicit recommendations, a further airspace investigation using (fast-time or real-time) simulation tools is recommended so that DFS can identify new solutions. This investigation should involve the adjacent ANSPs and States concerned.

The involvement of other ANSPs will ensure more integrated management of the airspace, in line in particular with the planned development of FABEC.



APPENDIX



Appendix A.1 PLANNED EXERCISE SCHEDULE

The order of the exercises was chosen to avoid masking the differences between the baseline and the changed route and airspace structure, thus between ORG0(A) and ORG1(A). As it was not possible to randomise the conditions across the ATCOs with repeated measures, since all ATCOs were needed for each run, the priority was given to the conditions "ORG" and "traffic". The sequence A-B-B-A-B-A-A-B was used for the order of the exercises. Letter "A" stands for ORG0(A) and letter "B" stands for ORG1(A). At the beginning of the week the ATCOs started with organisation 0(A), while in the second half of the week the roster started with ORG1(A). In the second quarter of the week the initial order was inverted and the same ratio was valid for the third and fourth quarters. In addition, the traffic sample was randomised in accordance with the sequence A-B-A-B-A-B-A-B. Letter "A" stands for traffic 1. and letter "B" stands for traffic 2. This sequence was used to avoid any training effects. The variable "TRA activity" had the lowest priority and it was not possible to balance it optimally.



9:00-10:20 Briefing ORG1 tr1 (2) ORG1 tr2 (5) ORG1(A) tr1 (8) ORG0 tr2 Break

10:40-12:00 Refresher ORG0 tr1 ORG1A tr2 (3) ORG0A tr1 (6) ORG0 tr1 ORG1 tr2

Lunch break

13:00-14:20 Refresher ORG1A tr2 ORG0 tr1 (4) ORG0 tr2 (7) ORG1 tr1 Debriefing

Break 14:30-15:30 ORG0A tr2 (1) Debriefing Debriefing Debriefing


Week 2 Monday Tuesday Wednesday Thursday Friday 9:00-10:20 Briefing ORG1 tr1 (10) ORG1 tr2 (13) ORG1A tr1 (16) ORG0A tr2

Break


ORG1A tr2 (11) ORG0A tr1 (14) ORG0A tr1 ORG1A tr2

Lunch break

13:00-14:20 Refreshing ORG1A tr2 ORG0 tr1 (12) ORG0 tr2 (15) ORG1A tr1 Debriefing

Break 14:30-15:30 ORG0A tr2 (9) Debriefing Debriefing Debriefing

Key: ORG0 = organisation 0; ORG1 = organisation 1; ORG0A= organisation 0A ORG1A= organisation 1A, tr1= traffic 1 (time of day: 07:40-08:40); tr2 = traffic 2 (time of day: 09:05-10:05); (1)-(16) indicates the run number.





9:00-10:20 Briefing ORG2 tr1 (1) ORG2A tr2 (4) ORG2 tr1 (7)

JOJO track (ORG2A tr2)

Break


100% ORG2A tr1 (2) ORG2 tr2 (5) ORG2A tr2 (8) Lunch break

13:00-14:20 Refresher ORG2A tr2

100% ORG2 tr2 (3) ORG2A tr1 (6) JOJO track (ORG2 tr2)

Break 14:30-15:30 ORG2 tr1 Debriefing Debriefing Debriefing

Key: ORG2 = organisation 2; ORG2A= organisation 2A. tr1 = traffic 1 (time of day: 07:40-08:40); tr2= traffic 2 (time of day 09:05-10:05; 100% = without traffic increase, for training purposes. (1)-(8) indicates the run number.

Appendix A.2 GRAPHS TRA 12A ACTIVE VERSUS TRA 12A INACTIVE

The graphs illustrate the significant effect of TRA 12A's being active or inactive. In no case was the factor "organisation" involved. That is why these graphs are reported in the Annex only, since no objectives were posted in connection with the activity of the military area TRA 12A.

• DFS The following graphs illustrate the effect of the activation of the military area TRA 12A on the DFS sectors.

ISA EC

Traffic 1 (7 o'clock) Traffic 2 (9 o'clock)

ISA

mea

n

very low

low

fair

high

very high

with TRA A without TRA A

Figure 13-1: Workload (ISA) first-order interaction between TRA 12A

active/inactive and the traffic sample



A significant difference was observed between traffic sample 1 and traffic sample 2 when TRA 12A was active. A higher EC workload was recorded for traffic sample 2 with TRA 12A active than for traffic sample 1 with TRA12A active.

PC


phon

e ca

lls

0

10

20

30

40

50

60

Figure 13-2: Telephone calls made by the PC when TRA 12A was active and

when TRA 12A was inactive

The PC made significantly fewer telephone calls when TRA 12A was inactive.


no o

f fre

quen

cy c

onta

ct

0

200

400

600

800 with TRA A without TRA A

Figure 13-3: 13-4 Number of frequency contacts; first-order interaction between TRA A

active/inactive and the traffic sample

Significantly fewer frequency contacts were made when the TRA A was active under the conditions of traffic sample 1 than under the conditions of traffic sample 2.



time spent on frequency


seco

nds

0

200

400

600

800

1000

1200

1400

Figure 13-5: Time spent on frequency: effect of TRA 12A active versus TRA 12A inactive

Significantly less time was spent on the frequency when the TRA 12A was active.

• LvNL For LvNL the activation of the TRA 12A was not relevant.

• MUAC The following graphs illustrate the effect of the activation of the military area TRA 12A on the MUAC sectors.

PC


SASH

A m

ean

0

1

2

3

4

5

6

Figure 13-6: PC Situational awareness when TRA 12A was active and when TRA 12A was inactive



Less situational awareness was recorded when TRA 12A was inactive. Nevertheless, all the values are above the mean value of three.

NASA Frustration EC


NAS

A TL

X m

ean

0

5

10

15

20


Figure 13-7: NASA-TLX frustration – EC: first-order interaction between

TRA 12A active/inactive and the traffic sample

Significantly less frustration was observed in traffic sample 1 than in traffic sample 2 when TRA 12A was inactive.



Appendix A.3 STATISTICAL VALUES OF THE ANOVAS

Table 13-6: Statistical values

Graph number Graph title F-value Figure 7-1 DFS telephone calls PC F(1,5) = 10,365 Figure 7-2 DFS – time spent on frequency F(1,5) = 8,543 Figure 7-3 Radio contact F(1,5) = 7,548 Figure 7-4 DFS – instructions F(1,5) = 8,316;

No graph (page 46) Heading instructions F(1,5) = 9,157 Figure 7-5 Number of aircraft on frequency Figure 7-6 EC workload (ISA) F (1, 5) = 11,837 Figure 7-7 PC workload (ISA) F (1, 5) =16,044 Figure 7-8 NASA overall EC F(1,5) = 26,687 No graph EC mental demand F(1,6) = 22,607 No graph EC temporal demand F(1,5) = 26,743 No graph EC performance F(1,5) = 7,938 No graph EC effort F(1,5) = 8,426; p <0.05 No graph EC effort: first-order interaction between

organisation and traffic F(1,5) = 7,353; p<0,05.

No graph EC frustration F(1,5) = 10,8834; p<0.05 Figure 7-9 NASA overall PC F(1,5) = 13,121; p < 0.05 No graph Mental demand F(1,5) = 21,386; p < 0.05), No graph Temporal demand F(1,5) = 12,557; p < 0.05 No graph Effort F(1,5) = 21,063; p <0.05 Figure 8-1 Heading instructions t=4,693; p<0, 05 Figure 8-4 Number of STCAs (sector 2E) t=-7,000; p>0, 05 Figure 9-1 Heading instructions F(1,5) = 10,371 Figure 9-2 First-order interaction between traffic and

organisation for direct instructions F(1, 5) = 212,521

No graph PC ISA first-order interaction between the factors "organisation" and "traffic"

F(1,5)=8,357

Figure 9-3 First-order interaction between traffic and organisation. NASA-TLX overall score for

the PC

F(1,5)=34,883

No graph PC mental demand F (1, 5) =6,854



Appendix A.4 SASHA QUESTIONNAIRE



Appendix A.5 NASA-TLX QUESTIONNAIRE

Participant-ID: Date: Trial:

Role: Condition:

NASA TLX RATING SHEET INSTRUCTIONS: On each scale, place a mark that represents the

magnitude of that factor in the task you just performed.

LOW HIGH MENTAL DEMAND

LOW HIGH PHYSICAL DEMAND

LOW HIGH TEMPORAL DEMAND

PERFECT FAILURE PERFORMANCE

LOW HIGH EFFORT

LOW HIGH FRUSTRATION



NASA-TLX RATING SCALE DEFINITIONS

Title Endpoints Description

MENTAL DEMAND Low/High How much mental activity and perceptual activity was required (e.g. thinking, deciding, calculating, remembering, searching, etc.)? Was the task easy or demanding, simple or complex, exacting or forgiving?

PHYSICAL DEMAND

Low/High How much physical activity was required (e.g. pushing, pulling, turning, controlling, activating, etc.)? Was the task easy or demanding, slow or brisk, slack or strenuous, restful or laborious?

TEMPORAL DEMAND

Low/High How much time pressure did you feel owing to the rate or pace at which the tasks or task elements occurred? Was the pace slow and leisurely or rapid and frantic?

PERFORMANCE Perfect/Failure How successful do you think you were in accomplishing the goals of the task set by the experimenter (or yourself)? How satisfied were you with your performance in accomplishing these goals?

EFFORT Low/High How hard did you have to work (mentally and physically) to accomplish your level of performance?

FRUSTRATION LEVEL

Low/High How insecure, discouraged, irritated, stressed and annoyed versus secure, gratified, content, relaxed and complacent did you feel during the task?

Participant-ID: Date:

Condition: ORG1



Appendix A.6 DFS QUESTIONNAIRES

Your opinion is very important for the evaluation of the effectiveness of the new route structure introduced in ORG1. Consequently, we would ask you to answer the enclosed questions, giving your individual opinion and personal experience of the new route structure.

All the individual data for this experiment, including this questionnaire, will be treated in strict confidence (only ID numbers will be used in the reporting of individual results, and even then only when absolutely necessary). Only the experiment analysis team will see your questionnaire. They will not pass any personal details and opinions to anyone outside the team.

Instructions

The questionnaire contains a number of statements on aspects of the ATC tasks you performed. Please indicate to what extent you agree or disagree with each statement by making a cross in the box that comes closest to your opinion, as shown below.

Please answer all the items quickly in the order that they are given, but do not cross-check your answers with previous items. Please put any comments further explaining your decisions in the free spaces below the items (or overleaf with reference to the relevant question number if necessary). Please work on your own; do not discuss any questions with your colleagues while filling in the questionnaire (you can, of course, discuss them later).

Example: Towers should be built even higher to give the controllers a better view.

Strongly

disagree

Disagree Undecided Agree Strongly

agree

The cross mark means that you "strongly agree" with the idea that towers should be built even higher.



AMRUFRA Post-organisation 1 Questions

DFS

1. The shifting of the Frankfurt departure route to the west is acceptable. 2. The shifting of the Frankfurt arrival route is acceptable. 3. The shifting of the Frankfurt departure route to the west decreases the need for

coordination (compared with ORG0). 4. The shifting of the Frankfurt arrival route to the west decreases the need for

coordination (compared with ORG0). 5. The new sector shape is acceptable. 6. The new sector shape decreases the need for coordination (compared with ORG0). 7. The new route and airspace structure decreases my workload (compared with ORG0). 8. The new route and airspace structure increases my situational awareness (compared

with ORG0). 9. The military activity in ORG1 is acceptable. 10. The military activity in ORG1(A) is acceptable. 11. Does the new route and airspace structure create any problems (safety concerns)?


DFS

12. The new route structure with the traffic increase is acceptable. 13. The split of the NO sector is necessary. 14. The split of the NO sector is acceptable. 15. The split of the NO sector decreases the need for coordination (compared with ORG0

and ORG1). 16. The split of the NO sector decreases my workload (compared with ORG0 and ORG1). 17. The benefits of the new route structure are also visible with the traffic increase. 18. The military activity in ORG2 is acceptable. 19. The military activity in ORG2(A) is acceptable. 20. Do the new route and airspace structure plus the traffic increase create any problems

(safety concerns)?



Appendix A.7 LVNL QUESTIONNAIRES


All the individual data from this experiment, including this questionnaire, will be treated in strict confidence (only ID numbers will be used in the reporting of individual results, and even then only when absolutely necessary). Only the experiment analysis team will see your questionnaire. They will not pass any personal details and opinions to anyone outside the team.

Instructions



Example: Towers should be built even higher to give controllers a better view.

Strongly

disagree


agree





LvNL

21. The splitting of the eastbound routes in sector 2E is acceptable. 22. If EC: the splitting of the Amsterdam departure route/sector change decreases the need

for heading instructions (compared with ORG0). 23. The new sector shape is acceptable (compared with ORG0). 24. The new sector shape decreases the need for coordination with adjacent centres

(compared with ORG0). 25. The new route and airspace structure decreases my workload (compared with ORG0). 26. The new route and airspace structure increases my situational awareness (compared

with ORG0). 27. The military activity in ORG1 is acceptable. 28. Does the new route and airspace structure decrease the need to split the sector? 29. Is the traffic sample realistic? 30. Is the way the traffic was handed over realistic? 31. Does the new route and airspace structure create any new problems (safety concerns)? 32. Does the new route and airspace structure increase any existing problems?


LvNL

33. The split of the eastbound route in sector 2E with the traffic increase is acceptable. 34. The new sector shape with the traffic increase is acceptable (compared with ORG0). 35. The Volkel area in ORG2 is acceptable. 36. Does the traffic increase in ORG2 create any new problems (safety concerns)? 37. Does the traffic increase in ORG2 enlarge existing problems? 38. Does the coordination with Langen increase in ORG2? 39. Does the coordination with MUAC increase in ORG2? 40. Does the coordination with DM increase in ORG2? 41. Is the traffic sample realistic? 42. Is the way the traffic was handed over realistic?



Appendix A.8 MUAC QUESTIONNAIRES


All the individual data from this experiment, including this questionnaire, will be treated in strict confidence (only ID numbers will be used in the reporting of individual results, and even then only when absolutely necessary). Only the experiment analysis team will see your questionnaire. They will not pass any personal details and opinions to anyone outside the team.

Instructions



Example: Towers should be built even higher to give controllers a better view.

Strongly

disagree


agree





MUAC

43. The new route structure is acceptable 44. The new interface with the new sector shapes of the adjacent ANSPs is acceptable

(compared with ORG0). 45. The new route and airspace structure decreases my workload in the Ruhr sector

(compared with ORG0). 46. The new route and airspace structure increases my workload in the Ruhr sector

(compared with ORG0). 47. The new route and airspace structure increases my situational awareness in the Ruhr

sector (compared with ORG0). 48. The new route and airspace structure decreases my situational awareness in the Ruhr

sector (compared with ORG0). 49. The new route and airspace structure decreases my workload in the Deco sector

(compared with ORG0). 50. The new route and airspace structure increases my workload in the Deco sector

(compared with ORG0).

If there is an increase in workload:

50A The increase in workload in the Deco sector (compared with ORG0) is acceptable.

51. The new route and airspace structure increases my situational awareness in the Deco sector (compared with ORG0).

52. The new route and airspace structure decreases my situational awareness in the Deco sector (compared with ORG0).

If there is a decrease in situational awareness: 52A. The decrease in my situational awareness in the Deco sector (compared with ORG0) is acceptable.

53. The military activity in ORG1 is realistic. 54. The military activity in ORG1 is acceptable. 55. The military activity in ORG1(A) is realistic. 56. The military activity in ORG1(A) is acceptable. 57. Does the new route and airspace structure create any problems (safety concerns)?




MUAC

58. I am able to handle the traffic with the new route structure and the traffic increase in "a

safe way". 59. I am able to handle the traffic with the new route structure and the traffic increase in "a

safe way". 60. The splitting of the Deco high sector is necessary. 61. The splitting of the Ruhr sector is necessary. 62. The benefits of the new route structure are still valid with the traffic increase. 63. The military activity in ORG2 (traffic increase) is acceptable. 64. The military activity in ORG2(A) is acceptable. 65. Does the new route and airspace structure create any problems (safety concerns)?

JOJO track

66. Is the JOJO track an increase to your workload?

67. Do you think the JOJO track could stay in the current location?

68. Does the JOJO track have an effect on capacity?

69. Does the JOJO track have an effect on safety?



Appendix A.9 ASSESSMENT OF TTA CONCEPT USING TASK ANALYSES

The shift of the Frankfurt departure route to the west is acceptable

5

12

01234567

stron

gly di

sagr

ee

disag

ree

unde

cided

agre

estr

ongly

agre

e

Num

ber o

f ATC

Os

The shift of the Frankfurt arrival route is acceptable

5

2

01234567

stron

gly di

sagr

ee

disag

ree

unde

cided

agre

estr

ongly

agre

e

Num

ber o

f ATC

Os

4D Trajectory Management Controller Simulation EUROCONTROL


The shift of the Frankfurt departure route to the west is decreasing the need for coordination.(compared with ORG0)

2

4

1

01234567

stron

gly di

sagr

ee

disag

ree

unde

cided

agre

estr

ongly

agre

e

Num

ber o

f ATC

Os

The shift of the Frankfurt arrival route to the west is decreasing the need for coordination.

(compared with ORG0)

1 1

5

0

1

2

3

4

5

6

7

stronglydisagree


Num

ber o

f ATC

Os

The new sector shape is acceptable

3 3

1

0

1

2

3

4

5

6

7

stronglydisagree


Num

ber o

f ATC

Os

The new sector shape decreases the need for coordination. (compared with ORG0)

2

5

0

1

2

3

4

5

6

7

stronglydisagree


Num

ber o

f ATC

Os



The new route and airspace structure decreases my workload.


4

21

01234567

stron

gly di

sagr

ee

disag

ree

unde

cided

agre

estr

ongly

agre

e

Num

ber o

f ATC

Os

The new route and airspace structure increases my situational awareness.


32 2

01234567

stron

gly di

sagr

ee

disag

ree

unde

cided

agre

estr

ongly

agre

e

Num

ber o

f ATC

Os

The new route and airspace structure increases my situational awareness.


1 1 1

4

0

1

2

3

4

5

6

7

stronglydisagree


Num

ber o

f ATC

Os

The military activity in ORG1 A is acceptable

1 1 1

4

0

1

2

3

4

5

6

7

stron

gly di

sagre

e

disag

reeun

decid

ed

agree

stron

gly ag

ree

Num

ber o

f ATC

Os



Does the new route and airspace structure create any problems (safety concerns)?

7

0

1

2

3

4

5

6

7

yes no

Num

ber o

f ATC

Os

The new route structure w ith the traffic increase is acceptable

5

1

0123456

stro

ngly

disa

gree

disa

gree

unde

cide

d

agre

e

stro

ngly

agre

e

Num

ber o

f ATC

Os

The split of the NO sector is necessary

4

1 1

0123456

stron

gly di

sagre

e

disag

ree

unde

cided

agre

estr

ongly

agre

eNum

ber o

f ATC

Os



The split of the NO sector acceptable

1

2

3

0

1

2

3

4

5

6

stron

gly di

sagr

ee

disag

ree

unde

cided

agre

estr

ongly

agre

e

Num

ber o

f ATC

Os

The split of the NO sector decreases the need for coordination

3

2

1

0

1

2

3

4

5

6

stron

gly di

sagr

ee

disag

ree

unde

cided

agre

estr

ongly

agre

e

Num

ber o

f ATC

Os

The split of the NO sector decreases my workload

2

4

0

1

2

3

4

5

6

stron

gly di

sagr

ee

disag

ree

unde

cided

agre

estr

ongly

agre

e

Num

ber o

f ATC

Os

The benefits of the new route structure are visible also w ith the traffic increase

2

3

1

0

1

2

3

4

5

6

stron

gly di

sagr

ee

disag

ree

unde

cided

agre

estr

ongly

agre

e

Num

ber o

f ATC

Os



The military activity in ORG 2 is acceptable

2 2 2

0

1

2

3

4

5

6

stron

gly di

sagr

ee

disag

ree

unde

cided

agre

estr

ongly

agre

e

Num

ber o

f ATC

Os

The military activity in ORG 2A is acceptable

1 1

2 2

0

1

2

3

4

5

6

stron

gly di

sagr

ee

disag

ree

unde

cided

agre

estr

ongly

agre

e

Num

ber o

f ATC

Os

Do the new route and airspace structure plus the traffic increase create any

problems?

5

1

0

1

2

3

4

5

6

yes no

Num

ber o

f ATC

Os



Appendix A.10 ORGANISATION QUESTIONNAIRES : LVNL FEEDBACK RESULTS

The split of the eastbound routes in Sector 2E is acceptable

12

5

012345678

stron

gly di

sagr

ee

disag

ree

unde

cided

agre

estr

ongly

agr

ee

Num

ber o

f ATC

Os

The split of the Amsterdam departure route decreases the need for heading instruction

(compared with ORG 0)

3

4

0

1

2

3

4

5

6

7

8

stronglydisagree


Num

ber o

f ATC

Os



The new sector shape is acceptable (compared w ith ORG 0)

5

12

0

1

2

3

4

5

6

7

8

stron

gly di

sagr

ee

disag

reeun

decid

ed

agre

estr

ongly

agree

Num

ber o

f ATC

Os

The new sector shape decreases the need for the coordination w ith adjacent cnetres

(compared w ith ORG 0)

22

1

3

0

1

2

3

4

5

6

7

8

stron

gly di

sagr

ee

disag

reeun

decid

ed

agre

estr

ongly

agree

Num

ber o

f ATC

Os

The new route and airspace structure decreases my workload (compared with ORG 0)

5

12

0

1

2

3

4

5

6

7

8

stron

gly d

isagr

ee

disag

ree

unde

cided

agre

estr

ongly

agr

ee

Num

ber o

f ATC

Os

The new route and airspace structure increases my situational awareness

(compared with ORG 0)

332

0

1

2

3

4

5

6

7

8

stron

gly d

isagr

ee

disag

ree

unde

cided

agre

estr

ongly

agr

ee

Num

ber o

f ATC

Os




1

5

1 1

0

1

2

3

4

5

6

7

8

stron

gly di

sagr

ee

disag

reeun

decid

ed

agre

estr

ongly

agree

Num

ber o

f ATC

Os

Does the new route and airspace structure decrerase the need to split the sector?

5

2

0

1

2

3

4

5

6

7

8

yes no

Num

ber o

f ATC

Os

Is the traffic sample realistic?

5

2

0

1

2

3

4

5

6

7

8

yes no

Num

ber o

f ATC

Os

Is the way the traffic was handed over realistic?

6

2

0

1

2

3

4

5

6

7

8

yes noN

umbe

r of A

TCO

s



Does the new route and airspace structure create any new problems (safety concerns)?

8

0

1

2

3

4

5

6

7

8

yes no

Num

ber o

f ATC

Os

Does the new route and airspace structure increase any existing problems?

3

5

0

1

2

3

4

5

6

7

8

yes no

Num

ber o

f ATC

Os



The split of the eastbound route in sector 2E with the traffic increase is acceptable.

2 2

0

1

2

3

4

stro

ngly

disag

ree

disa

gree

unde

cided

agre

est

rong

ly ag

ree

Num

ber o

f ATC

Os

The new sector shape with the traffic increase is acceptable (compared to ORG 0)

1

2

0

1

2

3

4

stro

ngly

disag

ree

disa

gree

unde

cided

agre

est

rong

ly ag

ree

Num

ber o

f ATC

Os

The RENDI area up to 145 in ORG 2 is acceptable

4

01234

stro

ngly

disa

gree

disa

gree

unde

cide

d

agre

e

stro

ngly

agre

eNum

ber o

f ATC

OS

The coordination with Langen increases in ORG 2

1

3

01234

stro

ngly

disa

gree

disa

gree

unde

cide

d

agre

e

stro

ngly

agre

eNum

ber o

f ATC

Os



The coordination with MUAC increases in ORG 2

1 1

2

0

1

2

3

4

stro

ngly

disag

ree

disa

gree

unde

cided

agre

e

stro

ngly

agre

e

Num

ber o

f ATC

Os

The coordination with DM increases in ORG 2

1 1 1 1

0

1

2

3

4

stro

ngly

disag

ree

disag

ree

unde

cided

agre

e

stro

ngly

agre

e

Num

ber o

f ATC

Os

Does the traffic increase in ORG 2 create any new problems (Safety concerns)?

4

0

1

2

3

4

yes no

Num

ber o

f ATC

Os

Is the traffic sample realistic?

2 2

0

1

2

3

4

yes no

Num

ber o

f ATC

Os



Is the way the traffic was handed over realistic?

2 2

0

1

2

3

4

yes no

Num

ber o

f ATC

Os

Does the traffic increase in ORG 2 enlarge exisiting problem problems

(Safety concerns)?

1

3

0

1

2

3

4

yes noNum

ber o

f ATC

Os



Appendix A.11 ORGANISATION QUESTIONNAIRES : MUAC FEEDBACK RESULTS

The new route structure is acceptable

1

6

1

0

1

2

3

4

5

6

stron

gly di

sagr

ee

disag

ree

unde

cided

agre

estr

ongly

agre

e

Num

ber o

f ATC

Os

The new interface with the new sector shapes of the adjacent ANSPs is acceptable

(compared to ORG0)

34

01

234

56

stron

gly di

sagr

ee

disag

ree

unde

cided

agre

estr

ongly

agre

e

Num

ber o

f ATC

Os



The new route and airspace structure decreases my workload in the Ruhr sector.


2

1

0

1

2

3

stron

gly di

sagr

ee

disag

ree

unde

cided

agre

estr

ongly

agre

e

Num

ber o

f ATC

Os

The new route and airspace structure increases my workload in the Ruhr sector.


3

0

1

2

3

stron

gly di

sagr

ee

disag

ree

unde

cided

agre

estr

ongly

agre

e

Num

ber o

f ATC

Os

The new route and airspace structure increases my situational awareness in the

Ruhr sector (compared with ORG0)

2

1

0

1

2

3

stronglydisagree


Num

ber o

f ATC

Os

6. The new route and airspace structure decrease my situational awareness in the

Ruhr sector (compared with ORG0)

2

1

0

1

2

3

stronglydisagree


Num

ber o

f ATC

Os



The new route and airspace structure decreases my workload in the Delta sector.


32

01

234

56

stron

gly di

sagr

ee

disag

ree

unde

cided

agre

estr

ongly

agre

e

Num

ber o

f ATC

Os

The new route and airspace structure increases my workload in the Delta sector.


1

4

01

234

56

stron

gly di

sagr

ee

disag

ree

unde

cided

agre

estr

ongly

agre

e

Num

ber o

f ATC

Os

The increase of workload in the Delta sector (compared with ORG0) is acceptable.

2

0

1

2

3

4

5

6

stron

gly di

sagr

ee

disag

ree

unde

cided

agre

estr

ongly

agre

e

Num

ber o

f ATC

Os

The new route and airspace structure increases my situational awareness in the

Delta sector (compared with ORG0)

3

2

0

1

2

3

4

5

6

stronglydisagree


Num

ber o

f ATC

Os



The new route and airspace structure decreases my situational awareness in the

Delta sector (compared with ORG0)

3

1 1

0

1

2

3

4

5

6

stronglydisagree


Num

ber o

f ATC

Os

The decrease of my situational awareness in the Delta sector (compared with ORG0) is

acceptable.

1

2

0

1

2

3

4

5

6

stronglydisagree


Num

ber o

f ATC

Os

The military activity in ORG1 is realistic

2

4

2

0

1

2

3

4

5

6

7

8

9

stronglydisagree


Num

ber o

f ATC

Os

The military activity in ORG1 is acceptable

1

2

1

3

1

0

1

2

3

4

5

6

7

8

9

stronglydisagree


Num

ber o

f ATC

Os



The military activity in ORG1A is realistic

1

4

1

2

0

1

2

3

4

5

6

7

8

9

stronglydisagree


Num

ber o

f ATC

Os

The military activity in ORG1A is acceptable

1

3

2

1 1

0

1

2

3

4

5

6

7

8

9

stronglydisagree


Num

ber o

f ATC

Os




(ATCOs from Hannover sectors)?

1

2

0

1

2

3

45

6

7

8

9

yes no

Num

ber o

f ATC

Os


(ATCOs from DECO sectors)?

2

3

0

1

2

3

4

5

6

7

8

9

yes no

Num

ber o

f ATC

Os

I am able to handle the traffic with the new route structure and the traffic

increase in "a safe way"

2

3

0

1

2

3

4

5

stron

gly d

isagr

ee

disag

ree

unde

cided

agre

estr

ongly

agre

e

Num

ber o

f ATC

Os

The split of the Ruhr sector is necessary

1

4

0

1

2

3

4

5

stron

gly di

sagr

ee

disag

ree

unde

cided

agre

estr

ongly

agre

e

Num

ber o

f ATC

Os




1

4

0

1

2

3

4

5

stron

gly di

sagr

ee

disag

ree

unde

cided

agre

estr

ongly

agre

e

Num

ber o

f ATC

Os


1

2 2

0

1

2

3

4

5

stron

gly di

sagr

ee

disag

ree

unde

cided

agre

estr

ongly

agre

e

Num

ber o

f ATC

Os




1

4

0

1

2

3

4

5

stron

gly di

sagr

ee

disag

ree

unde

cided

agre

estr

ongly

agre

e

Num

ber o

f ATC

Os

Does the new route and airspace structure create any problems (Safety

concerns)?

1

4

0

1

2

3

4

5

yes no

Num

ber o

f ATC

Os

I am able to handle the traffic with the new route structure and the traffic

increase in "a safe way"

1

6

0123456

stron

gly di

sagr

ee

disa

gree

unde

cided

agre

estr

ongly

agre

e

Num

ber o

f ATC

Os

The split of the Delta high sector is necessary

6

1

0123456

stro

ngly

disag

ree

disa

gree

unde

cided

agre

est

rong

ly ag

ree

Num

ber o

f ATC

Os




1

4

2

0123456

stron

gly di

sagr

ee

disag

ree

unde

cided

agre

estr

ongly

agre

e

Num

ber o

f ATC

Os


2 23

0123456

stron

gly di

sagr

ee

disag

ree

unde

cided

agre

est

rongly

agre

e

Num

ber o

f ATC

Os


1 1

3

1

0123456

stron

gly di

sagr

ee

disag

ree

unde

cided

agre

estr

ongly

agre

e

Num

ber o

f ATC

Os

Does the new route and airspace structure create any problems (Safety concerns)?

3

4

0

1

2

3

4

5

6

yes no

Num

ber o

f ATC

Os



Appendix A.12 OBJECTIVES, VALIDATION METHODS, TOOLS AND HYPOTHESES

Table 13-7: Validation objectives and tools

Project Validation Objectives

Experimental Validation Objectives

Indicators Tools/Techniques Hypotheses

Investigate the effect of the route and airspace changes on the controllers' traffic-handling capability.

Workload ratings; pilot activities (ATCO instructions; no. of headings, speed instructions, direct routings, level changes)

Instantaneous self-assessment (ISA) Post-exercise questionnaire (NASA-TLX) System recordings

Investigate the effect of the route and airspace changes on R/T usage.

Number of times R/T is used, and duration of usage

System recordings

Investigate the effect of the route and airspace changes on telephone coordination.

Number and duration of telephone coordination calls

System recordings


Investigate the effect of the route and airspace changes on intra-centre SYSCO usage (where applicable).

Number of SYSCO messages

Hypothesis 1 Controllers will need to perform fewer tasks (e.g. giving headings) with the new structure. Hypothesis 2 Controllers will perceive less workload with the new structure.


Investigate the controllers' perceptions of the acceptability of the changed airspace and routes.

Rating pertaining to acceptance of the new route structure and its usefulness; feedback in debriefings and questionnaire

Post-organisation questionnaire, de-briefings

No Hypothesis was formulated.


Investigate the effect of the route and airspace changes on the number of STCAs and losses of separation

Number of STCAs and losses of separation

System recordings Hypothesis 3




Project Validation Objectives

Experimental Validation Objectives

Indicators Tools/Techniques Hypotheses

Investigate the effect of the route and airspace changes on the level of situational awareness.

Situational awareness rating SASHA questionnaire Hypothesis 4


4. Assess the impact of the new route structure and airspace structure on the environment and efficiency.

Investigate the effect of the route and airspace changes on route length.

Length of the routes System recordings The new route structure will result in aircraft flying shorter and therefore more efficient routes in ORG1 than in ORG0.

5. Assess the feasibility of the new route structure with the traffic increase.

Assess the acceptability of the new route structure with the traffic increase.

Rating pertaining to acceptance of the new route structure and its usefulness; feedback in debriefings and questionnaire

Post-organisation questionnaire, de-briefings



Appendix A.13 SIMULATION ORGANISATION OPERATIONAL UPDATES

Table 13-1: Simulation operational updates

Date for requested updates

Concerned ORG

Details Concerned Exercises

16/04/2008

evening

ORG 4

V 0.1

Source = ORG1 (copy of ORG1)

o Airspace and videomap updates

o TOTNA becomes TOTNI.

o TOTNI coordinates: 51 59N06, 004 41E57.

o BREDA becomes BREDI.

o BREDI coordinates: 51 36N53, 004 31E37.

o Existing ATC constraint for BREDA is applied to BREDI.

o Volkel and NAPSI areas.

o New delivered videomap for MUAC, LvNL, and DFS.

o CWP-HMI updates

o New procedure for EHAM departures: could be transferred from 2E to RR or from 2E to MG.

o For all flights coming from 2E and going to RR, strips will be printed into MG.

33X7: 17/04 morning

33X9: 17/04 morning




Concerned ORG


17/04/2008 evening

ORG 4 V 0.2

Source = OR G4 (V 0.1)


o BODKA takes the DIBIR coordinates.

o DITAM takes the ABAXA coordinates.

o BIBKI takes the BIBOS coordinates.

o New videomap delivery for MUAC: TOTNI, BREDI, BOKDA, DITAM, BIBKI.

o New videomap delivery for DFS: BOKDA, DITAM, BIBKI.

o CWP-HMI updates

o EFL rule added, if ADEP = EDDF, crossing DITAM, from TA to RR, EFL= 260.

o New skip rule: if ADEP = EDDF, NO skipped in the TA-NO-RR sequence.

33X7: 18/04 morning

33X9: 18/04 morning

18/04/2008 afternoon

New ORG2 V 0.1

Source = ORG2


o TOTNA becomes TOTNI.

o TOTNI coordinates: 51 59N06, 004 41E57.

o BREDA becomes BREDI.

o BREDI coordinates: 51 36N53, 004 31E37.

o Existing ATC constraint for BREDA is applied to BREDI.

o Volkel and NAPSI areas.

o New delivered videomap for MUAC, LvNL, and DFS.

o CWP-HMI updates

o New procedure for EHAM departures: could be transferred from 2E to RR or from 2E to MG.

o For all flights coming from 2E going in RR, strip will be printed into MG.

34X7: 21/04 morning (training)

34A9: 21/04 morning (training)

54A7: 21/04 afternoon (lost)

54X7: 21/04 afternoon

54A7: 22/04 morning

54X9: 22/04 morning

54A9: 22/04 afternoon

54A7: 22/04 afternoon (spare)

54A7: 23/04 afternoon

54A9: 24/04 morning

54X9: 24/04 morning




Concerned ORG


24/04/2008 morning

ORG 4 V 0.3

Source = ORG 4 V 0.2


o New points

TOBMI: (51 30N57, 07 08E56) => V3D, V3M (Videomap ORG 3 DFS and MUAC)

MOSID: (51 20N29, 06 43E34) => V3D,V3M

BOSIP: (51 07N53, 07 00E04) => V3D,V3M

MISDO: (51 26N34, 06 47E53) => V3A, V3D, V3M (all videomaps)

DIBIR: (51 16N37, 06 07E28) => V3D,V3M

ABAXA:(50 45N53,07 23E09) => V3D,V3M

DITAM: (50 33N29, 07 23E09)=> V3D,V3M

BIBOS (50 57N20, 07 10E17) => V3A, V3D, V3M

COFFA (51 30N27, 06 57E32) => V3M (videomap ORG 3 MUAC only)

o Remove points

T6438

o Remove BOKDA from the map

o Airways updates

NAPSI-DISOS shall be replaced NAPSI-MISDO (without point on the segment)

o Constraint modifications

o On DIBIR, add constraint at FL 300

o Existing ATC constraint on ABAMI has been applied on TOBMI (A_EBBR_ABAMI).

o Existing ATC constraint on RUDEL has been applied on MOSID (A_EBBR_RUDEL).

33X9: 24/04 afternoon




Concerned ORG


o Flight-path updates

o ABAMI-RUDEL-AGENI (RR-NO-FB) replaced by TOBMI-MOSID-MODRU (RR-MG-FB).

o DITAM-BIBKI-THN (TA-NO-RR) replaced by DITAM-ABAXA-BIBOS-THN (TA-NO-RR).

o DITAM-BIBKI-NEKER-BOKDA (TA-NO-RR) replaced by DITAM-ABAXA-BIBOS-BOSIP-DIBIR.

o RENDI-NAPSI-DISOS-COL replaced by RENDI-NAPSI-MISDO-COL.

o RENDI-NAPSI-DISOS-GMH replaced by RENDI-NAPSI-MISDO-GMH.

o CWP-HMI updates

o Many new implemented EFL/XFL rules.

o New reference point rules added for paper strip.

24/04/2008 evening

ORG 4 V 1.0

Source = ORG 4 V 0.3

o Flight-path updates

o COFFA added between INPAR and LIPMI for ENITO_MTR and R91_ETARU flight paths.

o Increase in A/C numbers (35% more) from the 33X9 traffic sample: the increased traffic sample name = 53X9.

53X9: 25/04 morning

53X9: 25/04 morning

EUROCONTROL...Ron Slootbeek (LvNL) Ops expert Peter Roben (LvNL) Ops expert Victor van Kempen...

Documents

Transcript of EUROCONTROL...Ron Slootbeek (LvNL) Ops expert Peter Roben (LvNL) Ops expert Victor van Kempen...