Post on 27-Mar-2015
Federal AviationAdministration
Normal Operations Data:Air Traffic Facility Evaluations and NOSS
Presented to: Second ICAO Global Symposium on TEM/NOSS in Air Traffic Control
By: Karen Burcham
Date: February 8, 2007
Safety Evaluations and NOSS2Federal Aviation
AdministrationFebruary 8, 2007
Briefing Objectives
• FAA Safety Data Sources
• Purpose, objectives, and philosophy of Safety Evaluations and Audits
• Internal Evaluation and Audit processes
• NOSS role and status
Safety Evaluations and NOSS3Federal Aviation
AdministrationFebruary 8, 2007
Heinrich’s Triangle and Safety
Lea
din
g In
dic
ato
rs
La
gg
ing
In
dic
ato
rs1Aircraft
Accident
~30Incidents
~300HazardousConditions
~1,000Unreported
“Unsafe Acts”
(OE, OD, NMAC, PD)(OE, OD, NMAC, PD)
(Non-Compliance, Bad(Non-Compliance, Bad Documentation, bad phraseology, etc.)Documentation, bad phraseology, etc.)
(Non-Reported OE’s/OD’s, Missed Readbacks, etc.)(Non-Reported OE’s/OD’s, Missed Readbacks, etc.)
Safety Evaluations and NOSS4Federal Aviation
AdministrationFebruary 8, 2007
Safety Data Reporting RequirementsFAA Orders and Processes Related to Safety Data Reporting
Type of Data Overview References Mandatory Reporting Data
Air traffic incidents
This order mandates that data concerning air traffic incidents is collected and analyzed.
Order 7210.56, Air Traffic Quality Assurance
Aircraft accident/ incident
This order contains reporting requirements regarding safety issues, concerns, incidents, and accidents.
Order 8020.16, Aircraft Accident And Incident Notification, Investigation, and Reporting
System outages This order mandates outage reports and contributes to the daily system performance and incident reporting.
Order 6040.15, National Airspace Performance Reporting System
Significant system events
This order mandates the reporting of significant events and contributes to the daily system performance and incident reporting.
Order 6030.41, Notification Plan for Unscheduled Facility and Service Interruptions and Other Significant Events
Unsatisfactory condition
This order provides Agency employees a means of advising management of unsatisfactory conditions.
Order 1800.6, Unsatisfactory Condition Report
Oceanic Altitude/Nav Errors
This order establishes procedures for processing reports and for collecting system data for analysis.
Order 7110.82. Monitoring of Navigation, Longitudinal Separation, and Altitude Keeping Performance in Oceanic Airspace
Safety Evaluations and NOSS5Federal Aviation
AdministrationFebruary 8, 2007
Safety Data SourcesFAA Orders and Processes Related to Safety Data Reporting
Type of Data Overview Mandatory Reporting Data
National Aviation Information Management System (NAIMS) and Air Traffic Quality Assurance (ATQA)
A database where safety events are collected and categorized.
Runway Safety Directorate within the Safety Service Unit
The Runway Safety Directorate categorizes surface events that include surface operational errors, operational deviations, pilot, vehicle, and pedestrian deviations.
Accident/Incident Reporting Data System (AIDS) The FAA AIDS database contains accident and incident data records for all categories of civil aviation.
NTSB Accident/Incident Database The NTSB accident/incident database is the official repository of aviation accident data and causal factors. In this database, events are categorized as accidents or incidents.
Operational Error/Deviation System (OEDS) Used by FAA to determine if actions of a controller resulted in a loss of separation or an aircraft landing or departing on a closed runway.
Pilot Deviation System (PDS) Used by FAA to determine if actions of a pilot violated regulations.
Hazard Tracking System, FAST Toolset, and the Acquisition Management System
Designed to identify, eliminate, or resolve determined or assigned risk, estimate a likelihood of occurrence, and track hazards throughout the life cycle of a program.
Safety Evaluations and NOSS6Federal Aviation
AdministrationFebruary 8, 2007
FAA Safety Data Sources (cont.)Voluntary Reporting Data
Aviation Safety Reporting System (ASRS) and Aviation Safety Reporting Program (ASRP)
ASRS and ASRP are voluntary programs designed to encourage the identification and reporting of deficiencies and discrepancies in the airspace system.
Advisory Circular 00-46, Aviation Safety Reporting Program
Aviation Safety Action Program (ASAP)
Voluntary reporting of safety issues and events that come to the attention of employees of certain certificate holders. To encourage an employee to voluntarily report safety issues even though it may involve an alleged violation of Title 14 of the Code of Federal Regulations (14 CFR), enforcement-related incentives have been designed into the program.
Order 1110.129, Aviation Safety Action Program Aviation Rulemaking Committee and Advisory Circular 120-66, Aviation Safety Action Program (ASAP)
Automatic Reporting Data Incident and maintenance reporting in Maintenance Management System (MMS)
The MMS contains general maintenance logging, which contributes to the daily system performance and incident reporting.
Order 6000.48, General Maintenance Handbook for Automated Logging
Safety Evaluations and NOSS7Federal Aviation
AdministrationFebruary 8, 2007
.
So We Went Looking For The Big Picture
• Some data only shows the tip of the problem.
• Multiple databases may provide a lot of information but what does it all mean?
• What is needed to identify better precursors?
– Photo courtesy of Claudia VIRLAN, ANSS Directorate, Romanian CAA
Safety Evaluations and NOSS8Federal Aviation
AdministrationFebruary 8, 2007
Sources of Leading Indicators
• Data Collected During Normal Operations—not as the result of an Incident/Accident– Revamped Air Traffic Facility Evaluations/Audit
Process– NOSS
Safety Evaluations and NOSS9Federal Aviation
AdministrationFebruary 8, 2007
Safety Evaluations Serve the Assurance Component of SMS
Supporting Requirements
ATO must continue to meet its performance targets for safety
Reduce operational errors
Reduce runway incursions
To date, ATO collects the number, rate, and severity of these incidents (“lagging” safety indicators)
ATO does not consistently collect data that may identify precursors to these incidents (“leading” safety indicators)
Continual tracking and analysis of lagging and leading indicators will drive toward the identification of mitigation strategies that have a greater impact on the achievement of safety performance targets
ATO must continue to meet its performance targets for safety
Reduce operational errors
Reduce runway incursions
To date, ATO collects the number, rate, and severity of these incidents (“lagging” safety indicators)
ATO does not consistently collect data that may identify precursors to these incidents (“leading” safety indicators)
Continual tracking and analysis of lagging and leading indicators will drive toward the identification of mitigation strategies that have a greater impact on the achievement of safety performance targets
Assurance Component of SMSAssurance Component of SMS
Safety assurance and evaluations1
2 Safety data tracking and analysis
FAASafety Management
System
Policy Architecture
Assurance
Safety Promotion
Safety Evaluations and NOSS10Federal Aviation
AdministrationFebruary 8, 2007
Safety Evaluations Purpose/Mission
• Conduct audits, program assessments, operational assessments, and facilitate evaluations of ATC facilities to promote safety, quality, and efficiency of the ATC system.
• Measure, through the evaluation and audit processes, ATC system compliance with established policies, procedures, and requirements.
• Identify leading indicators of operational errors and deviations (OE/Ds), near mid-air collisions, controlled flight into terrain, and other safety hazards.
• Monitor and report trends of system compliance with established policies, procedures, and requirements.
Safety Evaluations and NOSS11Federal Aviation
AdministrationFebruary 8, 2007
Goals of New Evaluations Process
• Knowledge sharing through sharing of mitigation plans across facilities/service areas
• Ability to compare facility performance on key safety indicators
• Data collection/analysis capability that results in:– Identification of safety hazards – Identification of leading indicators – Communication of Evaluations/Audit findings
throughout the ATO
Safety Evaluations and NOSS12Federal Aviation
AdministrationFebruary 8, 2007
Safety Evaluations Data Sharing
Safety Evaluations
Field Customers-- Service Areas-- Hubs-- Facilities
Headquarters Customers-- OIG / GAO-- Administrator-- COO & Executive Council-- Safety Oversight
Safety Evaluations and NOSS13Federal Aviation
AdministrationFebruary 8, 2007
Facility Internal Evaluations
• Purpose is to identify compliance issues and safety hazards at a facility by evaluating and rating a set of checklist items.
• Conducted by every Facility in the NAS except Flight Service and NFCTs (approximately 520 facilities)
• Completed at least once per year• Due August 1st each year (beginning
August 1, 2006)
Safety Evaluations and NOSS14Federal Aviation
AdministrationFebruary 8, 2007
Safety Audits
• An audit is a method of assessing an air traffic facility’s performance and compliance with FAA directives and procedures.
• Audit is Independent Oversight• Audit is NOT a check of the facility Internal Evaluation• All AT Facilities audited at least every 3 years. Federal
Contract Towers (FCT’s) audited every other year.• Conducted by Safety Employees. Support from Service
Areas/Hubs.• Audit sites determined through application of criteria• Audits are little or no notice to the Facility.
Safety Evaluations and NOSS15Federal Aviation
AdministrationFebruary 8, 2007
Facility Safety Assessment System (FSAS)• Web-based database tool that supports Safety
Evaluations and Audits– Facilitates conduct of facility internal evaluations and
audits by serving as a central source of information on requirements as contained in current directives.
– Serves as a central collection point of the evaluation/audit data as it is completed.
– Serves to facilitate coordination and approval throughout the process.
• Plans to expand FSAS to support other safety processes
Safety Evaluations and NOSS16Federal Aviation
AdministrationFebruary 8, 2007
Where NOSS Fits
• NOSS provides data about normal situations– Identify situations that resulted in safe and efficient operations– Profile effective system performance
• A new technique for the Evaluations office toolbox • More in-depth look at a facility or portion of a
facility than is provided by current audit process• Provides structured look at operations• Assess precursors to operational errors (leading
indicators)• Use results to help target safety improvements
Safety Evaluations and NOSS17Federal Aviation
AdministrationFebruary 8, 2007
FAA NOSS Status
• The ATO-P Human Factors Research and Engineering Group focused first on the science of NOSS
• Conducted a study of NOSS observer inter-rater reliability at WJHTC (8/06) to assess the consistency in how operational experts make the same interpretations of data– Partial training of 4 Evaluations specialists on the
TEM model and NOSS observation approach• Plan to conduct a field trial at an En Route
facility in the April/May timeframe
Safety Evaluations and NOSS18Federal Aviation
AdministrationFebruary 8, 2007
Summary
• FAA ATO-Safety Evaluations focuses on Safety Assurance portion of SMS (leading indicators)
• Most data currently collected on incidents (lagging indicators)
• NOSS could provide a more in-depth tool for collecting leading indicators