Air Traffic ControlSoftware System Failure

Case Study – As-is ScenarioPresented by Stanley Dam

ForSE 6361 Advanced Requirement Engineering

10/25/2013

Overview

• Introduction• About The System• Software Glitch• Known Problem and Solution• What Went Wrong?

Introduction

• At about 5:00PM PST on Tuesday, September 14, 2004 air traffic controllers lost voice contact with 400 airplanes they were tracking over the southwestern United States

• Control Center located in Palmdale, CA• Controlled traffics above 13,0000 ft in 460,000 square km of airspace

Introduction (cont.)

• Planes started to head one another• In at lease five cases airplanes came within minimum separation distances• Two airplane accidents almost occurred• Disrupted about 800 flights• Impacted over 30,000 passengers

About The System

• Voice Switching and Control System (VSCS)– Controllers use a touch-screen to select a phone

line to connect to other controllers– Or to select radio frequency to talk to flight crews

• VSCS Control Subsystem Upgrade (VCSU)– Control system for VSCS– Monitors VSCS health status by continually

running built-in test (BIT)• Developed by Harris Corp., Melbourne, Florida

Software Glitch

• Inside the VCSU control system unit is a countdown timer

• VCSU uses that timer as a pulse to send out periodic queries to the VSCS

• Timer starts at 232 (~4 billion ms or ~50 days)• When timer hits zero, system can no longer

time itself, and it shuts down

Known Problem and Solution

• Multiple incidents reported indicating the system shutdown on its own after about 50 days

• The manufacturer, Harris Corp., was aware of the problem but didn’t know how it would impact the system

• After a system reboot, everything seemed to be working fine

• FAA released a maintenance procedure that required a system reboot every 30 days

What Went Wrong?

• The technician failed to perform the reset that must occur every 30 days

• Internal clock within the system subsequently shut down the system

• The backup system also failed within a minute after it was turned on

Questions?