1

ATS ATS Evaluation Evaluation ProgramsPrograms

The Federal Aviation Administration

Presented By:

Gary Romero

2

ATS Evaluation ProgramsATS Evaluation Programs

Why have an evaluation? What challenges existed? What were the benefits? How is an evaluation conducted?

3

An Identified NeedAn Identified Need

Prior to 1988, evaluations were accomplished using QA staffs within regional offices.

However, this practice presented several challenges to effective evaluations.

4

ChallengesChallenges

The FAA needed to STANDARDIZE operating practices from facility to facility and region to region.

FAA upper management needed NONPARTISAN, UNBIASED feedback on the performance of all air traffic field facilities.

The FAA needed a way to identify air traffic field facility COMPLIANCE with national policies.

5

Air Traffic Evaluations was created and established in 1988

6

Results and BenefitsResults and Benefits

Since 1988, Air Traffic Evaluations has reported individual facility compliance with national standards.

Since 1999, the evaluation findings have been archived in a database that will provide future managers with important historical data to use when addressing future issues.

Air Traffic Evaluations has maintained its identity as a separate nonpartisan organization, in order to continue to provide unfiltered information from the field to the upper levels of FAA management.

7

Results and BenefitsResults and Benefits

The Sharing of Ideas:

Air Traffic Evaluations has helped the FAA share good ideas, first through word-of-mouth as evaluators traveled from facility to facility and talked about useful programs and effective practices.

Currently:

Information is also shared electronically because each facility has access to the “National Database” where evaluation findings throughout the United States are maintained.

8

How is an Evaluation Conducted?How is an Evaluation Conducted?

9

ATS Evaluation ProgramATS Evaluation Program

ATS Evaluation Process Responses Data Analysis

10

ATS Evaluation ProcessATS Evaluation Process

Evaluation Cycle Standardized checklist Operational monitoring Procedures review Personnel interviews Customer feedback Formal report presentation

11

Standardized ChecklistStandardized Checklist

Each facility measured against the same standard Checklist derived from ATC regulations Checklist are option specific Itemized categorized as operational or

operational support Checklist is extensive—more than 200 items

12

Evaluation CycleEvaluation Cycle

Facility changes in management and traffic levels necessitate evaluations at regular intervals

13

Operational MonitoringOperational Monitoring Evaluators plug-in with controllers Typical monitor times range from 30-150 hours,

depending on facility size Adherence to procedures Safe and efficient operating practices Observe operational oversight Discuss observations at evaluation team meeting

to determine trends

14

Procedures ReviewProcedures Review

Are procedures in compliance with ATC regulations?

Do procedures promote safe and efficient movement of aircraft?

Are procedures current? Are the procedures being used? Do controllers understand procedures?

15

Personnel InterviewsPersonnel Interviews

Can highlight problem areas in a facility Can be validated through research Can be an indicator of employees’

commitment to the ATC mission

16

Customer FeedbackCustomer Feedback

Contact air carrier, military and general aviation users Contact nearby ATC facilities Solicit feedback on quality of service

17

Formal Report PresentationFormal Report Presentation

Verbally present findings to management and controllers Encourage questions to ensure problems are fully

understood Findings are documented in a written report

18

ResponsesResponses

Required for all problems identified in report Ensure action is taken to correct problems

– Corrective Action– Follow-up– Management Control

Problem correction mandated within a predefined time limit

19

Data AnalysisData Analysis

Data collected and retained for every evaluation Trend analysis conducted Preventative measures developed based on trends

20