NOSS Trial May / June 2005

Experience

FormatFormat

• Brief History of Airways NZ

• Safety Culture

• Why we ran the trial

• Preparation for the Trial

• Conduct of the trial

• Management viewpoint

• Observers viewpoint

• Staff viewpoint (as told by an observer)

• Presentation of the Report

• Main findings

• Action Plan to address issues

• Summary of NOSS for Airways

• Questions

CEO Perspective video

Airways Loss of Separation Incidents per 100,000 MovementsAirways Loss of Seperation Incidents per 100,000 Movements

4.2

3.5

2.7 2.6

2.7

2.5

3.43.3

2.3

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

"96/97" "97/98" "98/99" "99/00" "00/01" "01/02" "02/03" "03/04" "04/05"

Main Trunk Managers perspective video

Advantages of NOSS for AirwaysAdvantages of NOSS for Airways

• The use of TEM framework gave far greater structure than our Position Checks

• NOSS would allow a broader view of the organisation as a whole

• We saw potential to adopt TEM in other areas e.g. incident investigation

• The base concept was already well proven through LOSA which was being used by most of our major customers

• The analytical support offered by the UT was significant to a small organisation

• Clear guidelines, processes and forms were being developed by ICAO

• The potential to gain from the experience of other providers

• The potential for benchmarking

Summary of Key FactorsSummary of Key Factors

• High level management support

• Union buy in

• Budget $30,000NZD

• Support from ICAO in the form of draft procedures

• Technical and analytical support from the UT

Staff CommunicationsStaff Communications

• Update on Airways involvement in the ICAO working group and general introduction to the NOSS and LOSA

• Discussion on TEM principles

• General update on the proposed trial, including a union endorsement

• Notice from senior management formally announcing the trail and highlighting their support and expectations

• Reminder that the trial is about to start

Observer QualitiesObserver Qualities

• Professional Credibility

– Not just experience

– Not Checkers

– Some were quite junior

– Integrity

• Analytical

• Relatively open minded

– Many controller can be quite rigid in their thinking

• Enthusiasm and passion

– Wanted people who would promote and sell the process

Video Tim 1

Tim Training Video

Video Tim First Observation

Slide Terminal Suite

DeparturesArrivals Planner

Typical Terminal Radar Suite

Co-OrdSMC Tower

Typical Tower Layout

Tim Staff Perspective video

Tim Data Cleaning

Sa m p le = 6 3 N OSSOb s e r v a t io n s

T hr eat Prev alenc e Index (Percentag e of observ ations w ith one o r more threats) M isma n a g e d

T h re a t /T h re a t C o u n t

T hre at M ismanage ment Index (Percentage of threats linke d or contr ibuting to contro lle r er ror)

T h re a t T yp e s10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 1 0% 20% 30% 40% 50% 60% 70% 80% 90% 100%

All T h re a ts 6 5 / 3 6 2

Oth e r C o n tro lle rs 2 0 / 5 1

W o rk sp a ce / Ma te r ia ls 4 / 4 1

Eq u ip me n t 3 / 2 6

Op e ra tio n a lPe r fo rma n ce

2 / 9

R / T C o mmu n ica tio n 9 / 7 0

Pilo t / Aircra ft 1 1 / 5 6

T ra ffic 2 / 2 4

Ad ve r se W e a th e r 6 / 4 0

Airsp a ce /In fra s tru ctu re D e sig n

7 / 3 0

Airp o r t C o n fig u ra tio n /L a yo u t

1 / 8

Ge o g ra p h ica lEn viro n me n t

0 / 4

ANSP

Thr

eats

Airb

orne

Thr

eats

Ove

rall

1 0 0 % 1 8 %

2 9 %

1 3 %

1 0 %

2 9 %

1 4 %

6 7 %

1 2 %

1 5 %

8 %

2 3 %4 1 %

1 0 %

3 9 %5 7 %

5 4 %

5 7 %

6 4 % 2 0 %

Envir

onm

enta

l

6 %

S a mp le = 6 3 N O S SO b s e r v a tio n s

Error Pr evalence Index (Percentage of obser vations with one or more err ors) M i sm a n a g e d

E r ro r /E rr o r C o u n t

E rro r T yp e s10 % 20 % 30 % 40 % 5 0% 6 0% 7 0% 8 0% 9 0% 10 0%

A l l E r ro r s 4 0 /1 6 5

A l l C o m m u n ic a ti o nE r ro r s

5 /6 6

C o o r d in a t i o n 5 / 2 2

P h r a se o l o g y 0 / 1 4

O th e r C o m mu n ica ti o nE r ro r

0 / 1 6

R e a d b a c ks 0 / 1 4

A l l P r o ce d u ra l E rr o rs 1 8 /5 4

O th e r P ro c e d u ra lE r ro r

1 3 / 2 2

A i rc ra f t T ra n s fe r 1 / 1 7

B r ie f i n g 3 /1 0

F l ig h t P ro g r e ss S t ri p 1 / 5

A l l E q u i p m e n t /A u to m a t io n E rr o rs

7 /1 2

In fo rm a t io n D is p la y 3 / 4

C o m p u te r In p u t 2 / 5

C o m m u n i ca ti o nS ys te m s M a n i p u la ti o n

2 / 3

T r a ff i c H a n d l i n g 7 / 2 9

Co

mm

un

icatio

nP

roce

du

ral

Ove

rall

8 1 %

1 4 %

5 %

2 2 %

2 1 %

2 5 %

6 %

3 0 %

2 2 %

6 %

2 2 %

Eq

uip

men

t / A

uto

ma

tion

5 %

Tra

ffic

3 8 %

5 7 %

5 1 %

1 6 %

Error Management

76% of 165 errors had inconsequential outcomes22% led to an undesired state2% led to additional errorTotal 40 errors mismanaged

Error Management

76% of 165 errors had inconsequential outcomes22% led to an undesired state2% led to additional errorTotal 40 errors mismanaged

Threat Prevalence

100% of observations had at least one threatTotal 363 threats observed

Threat Prevalence

100% of observations had at least one threatTotal 363 threats observed

Threat to Error Linkage

44% of 165 errors were linked to mismanaged threat56% of errors were not

Threat to Error Linkage

44% of 165 errors were linked to mismanaged threat56% of errors were not

Threat Management

82% of 362 threats were effectively managed18% were mismanagedTotal 65 threats mismanaged

Threat Management

82% of 362 threats were effectively managed18% were mismanagedTotal 65 threats mismanaged

Threats - Undesired State Linkage

7% of threats were linked to undesired states63% of 40 undesired states were linked to threats

Threats - Undesired State Linkage

7% of threats were linked to undesired states63% of 40 undesired states were linked to threats

Error Prevalence

81% of observations had at least one error19% were error freeTotal 165 errors were observed

Error Prevalence

81% of observations had at least one error19% were error freeTotal 165 errors were observed Detected Error Outcomes

74% of errors that were detected were inconsequential22% were linked to undesired states4% were linked to additional error

Detected Error Outcomes

74% of errors that were detected were inconsequential22% were linked to undesired states4% were linked to additional error Undesired State

Prevalence and Management

45% of observations had USTotal 40 USs

100% of 40 US were inconsequential

Undesired StatePrevalence and Management

45% of observations had USTotal 40 USs

100% of 40 US were inconsequential

Error Response

13% of 165 errors were detected by the observed controller20% were detected by another party67% went undetected or were ignored

Error Response

13% of 165 errors were detected by the observed controller20% were detected by another party67% went undetected or were ignored

Undetected Error Outcomes

76% of errors that were detected were inconsequential22% were linked to undesired states2% were linked to additional error

Undetected Error Outcomes

76% of errors that were detected were inconsequential22% were linked to undesired states2% were linked to additional error

Total 63 observationsTotal 63 observations

41% Airborne -Readback errors -Failure to respond -Parachuting near BDY

35% ANSP -Distractions,conversation -Incorrect co-ordination

23% Environmental -WX deviations -Low / poor visibility

Overall R/T comms produced the most threats and occurred on 70% of Obs

76% of 165 errors had inconsequential outcomes

22% lead to Undesired states

2% lead to an additional error

40 Undesired states equally spread across the sectors observed

-Lack of separation assurance

-Strip board layout

All 40 were managed

67%of errors went undetected or were ignored82% of all Threats were managed

18% were mismanaged

The most mismanaged threats were those posed by other controllers

81% of observations had at least 1 error

165 errors in Total

Co-ordination errors were the most common

CountermeasuresCountermeasures

• 14% of the observations received a marginal rating for

post-handover support.

• 11% received a marginal rating for workload management.

• Generally late opening of additional positions

• Not restricting traffic flow.

– Interestingly those controllers who received marginal ratings in this area also encountered more threats and had more mismanaged threats and errors than other controllers

Recommendations Recommendations

• 12 were made, 11 have been accepted for action e.g.

– Setting up working group with the airspace users to address Airborne / ground threats e.g. Similar callsigns, visual approaches etc

– Address co-ordination issues through TRM

– Mandate degraded mode training in annual cyclic training

– Review hand-off procedures in some sectors

– Adopt formal HO/TO procedures across all sectors(Checklists)

– Review opening and closing position procedures through cyclic training

– Review traffic complexity issues within specific sectors

– Increase training on ATS systems