1 centre dÉtudes de la navigation aérienne COMMON METRICS FRAMEWORK FOR ATM PERFORMANCE ASSESSMENT...

1

centred’Études dela navigationaérienne

COMMON METRICS FRAMEWORK FOR ATM PERFORMANCE

ASSESSMENT AND MONITORING

Almira Williams, CSSI Inc.

2


Background

Conclusions from ATM system performance R&D are often inadequate for policy decisions due to the lack of:

Standards for quantitative analysis and clear terminology definition; Consistency and comparability of outcomes from different analyses; Links between quantitative analysis and corporate objectives.

Performance assessments of the operational and/or future ATM concepts, systems and goals are often incomparable

In 1995, FAA and EUROCONTROL initialized a co-operative R&D effort to address this and other similar problems

Action Plans were developed, each addressing a specific area of interest, e.g. AP1–Airborne Separation Concepts, AP6–ATM Decision Support Tools, etc.

In 2002, AP2–Air Traffic Operational Concepts, and AP5–Validation and Verification Strategy, combined their activities in searching for a more general and systematic approach

Collaborative effort of CSSI, FAA, EuroControl, and CENAin support of the Action Plans 2 and 5

3


Objectives

Develop common ATM performance assessment and monitoring capability for:

periodic evaluation of system performance, operational concept validation and/or comparison, and target setting;

Support an understanding of terminology and performance metrics used by different organizations by:

providing a common metrics taxonomy, organizing the existing ATM metrics to maintain the connection between the

identical metrics, identify metrics that have identical definitions but different names, identify metrics that have identical names but different definitions

Establish a top-down framework for ATM performance assessment and monitoring that roll-ups performance measures to organizational goals and expectations of various ATM stakeholders

Founded on ISO principles for QOS in IT, and ICAO hierarchical structure of the performance based ATM Operational Concept

4


Principal Considerations

Common performance framework must facilitate the following: Evaluation of both overall system and individual aspects of performance; Evaluation of system-wide and regional performance; Evaluation within the real-world or a modeling environment; and Comparison of system performance before and after a change is implemented.

System Resources, Procedures and

Policies

System Performance

Management/Control

Performance Evaluation

Demand

5


0

2

4

6

8

10

Capacity

FlightEfficiency

Safety

#1#2

#3

#4

#5

#6

#7#8

#9

#10

#11

#12

#13

#14

0

2

4

6

8

10Airspace Organization

And Management

AerodromeOperations

Demand And Capacity

Balancing

TrafficSynchronization

ConflictManagement

AirspaceUser

Operations

ATM DeliveryManagement

BeforeAfter

Performance Based ATM Concept

Performance Outcomes* Radar GraphsLevel 1: Political and Socio-Economic Requirements

Level 2: Required Total System Performance

Level 3: Required Total System Performance

Level 4: System Requirements

Level 5: Standards and Specifications

Safety

Cap

aci

ty

metric

indicator

Cos

t-E

ffic

ienc

yE

nvir

onm

ent S

ecur

ity N

atio

nal

Secu

rity

Hum

an

Per

form

.U

nifo

rmi

tyQua

lity

indicator

indicatorindicator

metricmetric

metric

metricmetric

metric metric metric

indicator

ATM Performance Framework* ICAO Concepts of RASP/RTSP

Metric Collection

metric specification



Cla

ssif

icat

ions



Metric Taxonomy * ISO Concept of QOS in IT

Metric Attributes

Performance Metrics and IndicatorsSpecializations, Derivations, etc.

entitiesprocesses/operations

events

6


The Relationship Between the Metrics Taxonomy and the ATM Performance Framework

•

Level 1

Level 2

Level 5

Level 4

Level 3

Safety

ATM SecurityCost Effectiveness

PrimaryCharacteristics

Primary Classifications

Specification

ATM Performance Framework

Metrics Taxonomy

7


Example: Safety Primary Characteristic

Number of ATM induced accidents

Number of accidents due to, among other things, an error in the Airspace Organization and Management

Number of accidents due to, among other things, an error in the Aerodrome Operations

Airspace sectorization errors: Fix position

Fix naming Errors

Percent of Airspace Unrestricted for Civil Aviation Activity

Airway NOTAM delivery Errors

Airway NOTAM delivery Timeliness

FDP errors: Forgotten SUA activation

FDP errors: Fix name errors

Airport NOTAM delivery Errors

FDP errors: Forgotten airport closure

RASP

RTSP

System Requirements

Standards and Specifications

AIS

FDPS

Runway incursion causes: Surface movement management

Political and Socio-Economic Requirements

Economic and Societal Costs of Aviation Accidents

8


0

20

40

60

80

100

Safety

ATM Security

Cost-Effectiveness

Capacity

National Securityand Defense

Environment

Human Involv. &

Commitment

Uniformity

Quality

BeforeAfter

0

2

4

6

8

10

AirspaceOrganization and

Management

AerodromeOperations

Demand and Capacity

Balancing

TrafficSynchronization

ConflictManagement

AirspaceUser

Operations

ATM DeliveryManagement

BeforeAfter

Common Dashboard

•0

•2

•4

•6

•8

•10

Ratio of Airport VMC to IMC Capacity

Airport Capacity Utilization

Airport Capacity UtilizationDuring Peak Hours

Airport ARR + DEP Rates

Actual Capacity Variation vs. Demand Variation

ATFM Overloads

Percent of Flights on Time

Avg. Minutes of Delayper Flight

Avg. Minutes of Delayper Delayed Flight

Actual vs. OptimalFuel Burn

Actual vs. PreferredFlight Distance

ATSP'sPerception of Safety

Aircraft Operator'sPerception of Safety

ATM Conflict Ratio

Capacity

Flight Efficiency

Safety

9


Conclusions

We developed guiding principles for elaborating a comprehensive ATM performance framework that is applicable to all phases of (gate-to-gate) flight and in a uniform manner across international civil aviation community.

Proposed framework is consistent with the International Civil Aviation Organization (ICAO) Required ATM System Performance (RASP) and Required Total System Performance (RTSP) approach

Proposed framework is based on the principal concepts for Quality of Service (QOS) framework developed by the International Organization for Standardization (ISO).

10


Next Steps

Expand and consolidate the performance example- safety,

Extend the ATM Performance Framework to the remaining principal objectives, and

Further efforts on the aggregation of lower level into higher level results using the cause-effect approach.

[email protected]

1 centre dÉtudes de la navigation aérienne COMMON METRICS FRAMEWORK FOR ATM PERFORMANCE ASSESSMENT...

Documents

Transcript of 1 centre dÉtudes de la navigation aérienne COMMON METRICS FRAMEWORK FOR ATM PERFORMANCE ASSESSMENT...