Assessment of the

tolerable level of safety in

Europe: a review of

regulatory approaches

Dr. Arnab Majumdar

Lloyds Register Educational Trust Research Fellow

a.majumdar@imperial.ac.ukEUROCONTROL, 24 October 2008

Outline

• TLS for in-service + design change

• How to regulate?

• Questionnaire + interviews

• Different approaches - fit for purpose

• Some guidelines

Introduction

• Safety - prime concern, expressed in targets

• Confusion for TLS

• Design change TLS outlined in ESSAR 4

• Brooker (2006) critique - noted absence of regulatory failure

considerations

• How is the TLS regulated?

Countries assessed

Country ANSP Regulato r

U K X (2) X (3)

France X (3) X (2)

Germa n y X (2) N C

Sweden X (1) -

Denma r k - X (1)

Portugal X (1) X (3)

Austria X (1) W

Netherlands X (1) X (2)

Italy X (1) X (3)

Irela n d X (2) X

Belgium X (4) -

Switzerla n d - X (1)*

Questionnaire asked

• What data is collected by regulators + ANSPs?

• What definitions exist and how are these understood?

• How is a TLS set for in-service provision and for design change?

- the assessment of the change in an ATM;

- the nature of data used for the calculation;

- the statistical methodology;

- the methods of apportionment of the risk budget;

- use for decision making

• The role of the human and organisation?

• Any other industries considered?

Data collection

• Western Europe - situation is good with improvements (PPR)

• Data is collected by reporting and increasing automation

- Role of safety culture

- Small country issue - reporting

• Regulators completing the picture with airline data

• Development of intelligent databases with multiple sources

- future analytical tools

• Greater in-depth analysis of causal factors

• Move towards “positive” data + normal operations data collection

• Severity schemes - ESSAR2, SSE, Finavia

• Beware! “Negotiating with data”

In-service provision TLS

• Based upon incident data - ESSAR2 categories, SSEs etc.

• Reported incidents - Safety culture crucial

• Regulators also note non-ANSP accident data

• Targets set by rate usually - often compared to similar nations

• Some set Target for units according to ED125 method

• Note the move towards absolute

• Comparison with similar size countries

• But what happens when the in-service TLS is breached?

- a lot of explanations

- discussions with regulator

Design change TLS

• Based upon ESSAR4 requirements

• Need to consider case for equipment, procedures and human

• No specified quantified methodology for regulator

- check ANSP method is appropriate

• A variety of methods in Europe

• Equipment and procedures - understood

• Human performance - problematic

• What about regulatory failure?

• Apportionment issues with multiple providers

- resource effort required to regulate

Typical safety assessmentInputs for the S .A.:

- Concept of Operation- Change Description

- Scope definition

CSSA

FHA

PSSA

SSA – Pre -OPS

Safety

Assessment Plan

Approval by :- Change Leader

- Safety Manager- Belgian Civil Aviation Authorities

Change is “MINOR”

Change is “MAJOR”

Change is considered as “MAJOR”

SSA – OPS monitoring

Safety impact of Change could

be considered as Minor

So what is change?

• ESARR4: “An ATM Service Provider shall ensure that hazard identification aswell as risk assessment and mitigation are systematically conducted for anychanges to those parts of the ATM System and supporting services within hismanagerial control”.

• Often - a change considered as any modification in Operational System.

• Safety cases - expensive time and resources!

• Subjective analysis of the impact of the change

• Certain nations formalise with weighted checklist - EPIS

• Process for minor change - at local units process

• Major change - full safety assessment

• Careful! Cost implications of major vs. minor change mean careful guidelines

are needed + regularly updated

Equipment + Procedures case

• Straightforward enough for ANSP

• Migrate this to the equipment provider

• Go through the MATS manual with controllers

• Can provide separate cases, fault trees etc.

• Safety folders on hardware + software modelling developed

• These are quantitative

The human factor

• Difficult with no accepted model of human performance

• Usually no quantitative human case

- as long as it makes sense then regulator fine!

• Moves to use HERA for human performance modelling

- issues involved this.

• Any lessons from the US nuclear regulator?

Approaches

Apportionment

• By unit using ED 125

- includes complexity

• LVNL Safety criteria - by phase of flight

• Small country with dominant ANSP + AFIS stations:

- minimal capability

- impact on safety of AFIS

• UK CAA studies indicate

- high density controlled airspace => significant loss of separation correlation

- collision risk model for high density airspace

- nothing for low density

Verfication + Validation

• Not always possible for regulator to verify quantifiable model:

- attends session using the model;

- considers documentation.

• Verify during trials by attendance and interviews with users

• Assess operational suitability prior to implementation:

- 30 days controller use

- qualitative human factors case

Decision making

• Nature of decision making - safety vs. commerical

• One set of beliefs - no trade-off analysis please

• Another set - use these models to assess impacts of alternative

strategies

• For those nations with quantified models:

- increasing use for decision making;

• Problem: no formalised method in many cases for decision making

using models

Regulatory aspects

• Nature of the NSA - new roles and ANSP relationship

• Safety cases - how much regulatory expertise is there?

• Workload for regulators - time requirements

• External pressure

• For small nations:

- cooperation with other nations;

- hire external bodies;

• Greater cooperation framework with SESAR

• Note: How can we use once in career experience of major change safety case

creatively?

Other industry experience

• Great desire to understand how other industries deal with safety

targets

• Special focus on human case

• By far nuclear is the biggest example

• Other aviation experience

• Railways and medicine

• Chemical plants

Conclusions

• Safety a priority - now manifesting itself in data collection

+ culture

• Major challenges faced by regulators for design change

• Need for understanding safety case methodology

• Concentrate on administrative aspects also

• Creativity in the future

Thank you

To the safety experts of ANSPs and regulators who gave their valuable time for

this research I am truly grateful.