American Institute of Aeronautics and Astronautics1

Human Performance Modeling to

Predict Controller Workload

Ken Leiden* and Jill Kamienski†

Micro Analysis & Design, Boulder, Colorado, 80301

and

Parimal Kopardekar, Ph.D.‡

NASA Ames Research Center, Moffett Field, California, 94035

A human performance model of en route sector teams has been developed to predictcontroller workload. Detailed tasks, procedures, roles, and responsibilities of the radarcontroller and radar associate controller are modeled. The human performance model isintegrated with a model of airspace, aircraft dynamics, and controller decision support toolsto emulate closed-loop behavior of an arrival metering environment.

I. Introductionn the National Airspace System (NAS), en route sector capacity is often times constrained by controller workloadlimitations. Consequently, the FAA and NASA are introducing new air traffic management (ATM) tools and

technologies [e.g., aeronautical data link services (ADLS) §] to improve the productivity of en route controllers.Improved controller productivity should enable sector capacities to be increased and traffic management restrictionsto be reduced. Traditionally, new ATM tools and technologies are evaluated through full fidelity or part-task human-in-the-loop (HITL) simulations in dedicated research facilities. However, the rate at which new NAS concepts arebeing proposed and the significant complexity and cost of large scale distributed air-ground tests suggest that other,more cost effective methods of human factors research are needed. One such method gaining popularity iscomputational human performance modeling. In this paradigm, the humans (e.g., controllers, pilots) are representedas computational entities. These entities are modeled to interact with the ATM system elements just as realcontrollers and pilots would in current operations or simulated future operations. Importantly, the performance andworkload of the human operators can be predicted. In the case of new ATM tools and technologies, humanperformance models can predict increases in controller productivity and the corresponding increase in sectorcapacities. Decisions about NAS investments and upgrades may need to leverage techniques such as humanperformance modeling if the appropriate tradeoff studies are to be conducted in a timely and cost-efficient manner.Furthermore, although human performance modeling is not intended to replace HITL simulations, it certainly can beleveraged to assist researchers in focusing HITL simulations on smaller sets of conditions and scenarios to maximizethe benefit-to-cost of conducting them.

II. BackgroundDuring 2002-2003, a human performance model was developed to support a NASA benefits assessment of two

Distributed Air/Ground Traffic Management (DAG-TM) concepts: En Route Trajectory Negotiation and En RouteFree Maneuvering.1 (Note that only the former concept is discussed here because it is the focus of the current study).En Route Trajectory Negotiation is an extension of today’s operations in that the locus of control remains with the

* Manager, Aviation Systems Modeling, 4949 Pearl East Circle Suite 300, AIAA Member.† Analyst, Aviation Systems Modeling, 4949 Pearl East Circle Suite 300, AIAA Member.‡ Manager, Strategic Airspace Usage Project, Mail Stop 210-15, AIAA Senior Member.§ For this research, ADLS is assumed to be a fully evolved form of controller pilot data link communication(CPDLC), incorporating complex data link messages to support advanced traffic management concepts as needed.

I

AIAA 5th Aviation, Technology, Integration, and Operations Conference (ATIO) <br>26 - 28 September 2005, Arlington, Virginia

AIAA 2005-7378

This material is declared a work of the U.S. Government and is not subject to copyright protection in the United States.

American Institute of Aeronautics and Astronautics2

air traffic service provider (ATSP). Through the integration of advanced flight deck automation, ADLS, andcontroller decision support tools (DSTs), this concept enables the flight crew to exchange automated trajectory andclearance-related data and submit and/or negotiate trajectory change requests with the ATSP. In addition, En RouteTrajectory Negotiation includes three features to increase controller productivity:

1. Basic data exchange, which allows data (e.g., atmospheric conditions, aircraft weight) to be exchangedbetween aircraft and controller DSTs to improve the accuracy and performance of trajectory predictionfunctions.

2. Integration of controller DST capabilities with ADLS to compose ready-to-send data link messages andupdate the DST intent from messages sent.

3. Integration of the controller DST with an aircraft’s flight management system (FMS) via ADLS to facilitatestrategic trajectory-based clearances/instructions that would otherwise be too complex to communicate byvoice or manual CPDLC interactions.

Of these three features, the human performance modeling effort only focused on the last two. The secondfeature, the integration of controller DSTs with data link, results in two significant benefit mechanisms. First,controller productivity is increased since the DST can automatically compose a clearance/advisory into a ready-to-send data link message. This eliminates message composition workload, reduces revision time, and inhibits manualentry errors. Second, when any data link messages are sent, the DST is automatically updated withclearance/advisory information, thereby improving the DST knowledge of intent. This provides substantialimprovements in trajectory prediction accuracy by improving the effectiveness of DST functions (e.g., metering,conflict detection and resolution).

The third feature, integration of the DST and FMS via ADLS, allows clearances/advisories from the DST to beautomatically loaded into the FMS, thereby allowing the flight crew to review the amended trajectory on a graphicaldisplay. This capability reduces controller workload because a sequence of easily executed tactical instructions canbe combined into a single, strategic instruction. Furthermore, the DST/FMS integration enables the controller tosend 4D-based clearances/instructions** that would otherwise be too difficult and/or error-prone for a flight crew toenter manually into the FMS (e.g., a sequence of latitude/longitude waypoints). This trajectory-based applicationshould also lead to more efficient and predictable trajectories.

The 2002-2003 human performance modeling effort emulated these benefit mechanisms in a fast-time modelingframework representing radar (R-side) and radar associate (D-side) controllers in several sectors of a genericextended terminal area environment. Results demonstrated a reduction in the time required to complete commoncontroller tasks such as conflict resolution and time-based metering. This reduction in time enabled the controllers tobe more productive compared to a baseline, meaning they had more time available to handle more aircraft in theirrespective sectors. New sector capacities were established by incrementally increasing traffic levels in the modeluntil controller utilization (a normalized value representing the time required to perform controller tasks andprocedures divided by the elapsed time) returned to baseline values corresponding to currently acceptable capacities(e.g., the typical en route sector capacity is 18 aircraft). The final results from this modeling effort indicated thatcurrent sector capacities could be increased by over 70% by implementing the following tools and technologies:ADLS, reduced vertical separation minimum (RVSM), NASA’s En route Descent Advisor (EDA)††, and the EnRoute Trajectory Negotiation’s benefit mechanisms from above. For the En Route Trajectory Negotiation benefitmechanisms alone, the capacity increase was 11%.

Given the time and funding limitations of the 2002-2003 human performance modeling work, assumptions wererequired about workload predictions and the fidelity of the models to represent the airspace, aircraft, and decisionsupport tool functionality. In the current effort, the human performance model has been refined to address some ofthese assumptions. In particular, five significant improvements have been or will be made to the modelingframework:

1. A hybrid model, previously documented,2 has been developed that integrates the human performance modelwith a high fidelity airspace and aircraft simulation that also represents the functionality of the TrafficManagement Advisor (TMA), a conflict detection and resolution DST, and EDA.

** A 4-dimensional clearance/instruction specifies the position state of the aircraft as a function of time, aparticularly useful method for meeting time-based metering constraints.†† EDA is an R-side DST to provide a conflict-free, scheduled time of arrival meet time capability with trajectory-oriented procedures

American Institute of Aeronautics and Astronautics3

2. Higher fidelity models of workload have been added to the human performance model to represent visual,auditory, cognitive, and psychomotor workload channels for both the R-side and D-side controllers.

3. More detailed modeling of the D-side has been incorporated, particularly with respect to coordinationbetween sector teams.

4. An algorithm will be incorporated to represent controller’s perceived time pressure and the resultingdecision on how to prioritize tasks under high workload and traffic levels. This step requires analysis ofHITL data collected from controller experiments conducted during the summer of 2005 at the FAA WilliamJ. Hughes Technical Center.

5. Validation of the human performance model against FAA Technical Center data set mentioned in #4.

The focus of this paper, corresponding to item 2, is to describe in more detail the workload metrics predicted by thehuman performance model.

III. Multi-tasking and WorkloadIn the hybrid model, controller tasks are initiated under two conditions. The first condition occurs when an

airspace event requiring a controller action or response is encountered through either a visual monitoring scan of thedisplay (e.g., an aircraft approaching a sector boundary, thus needing a handoff) or hearing an auditory event (e.g.,call from pilot). The second condition occurs based on the modeler-specified task decomposition (task 2 necessarilyfollows task 1 – for example, in Figure 0 where the R-side decision to accept the handoff must follow the task ofacquiring situation awareness about the incoming aircraft). Prior to task initiation, activities/tasks are scheduled bythe model based on task priorities, resources required, and resources available. The priorities and resources requiredfor each activity are assigned by the modeler (often times with subject matter expertise input) whereas the resourcesavailable are calculated during run-time to reflect the actual resources being utilized by both the controller and thesystem.

The scheduling mechanism in the hybrid model considers both the priority of the task and the availability ofvisual (V), auditory (A), cognitive (C), and psychomotor (P) channel (VACP) resources based on a scale shown inTable 1.3 The accompanying theory assumes that the VACP channels are independent of each other. The scale foreach channel ranges from 0 (no workload) to 7 (very high workload). The visual and auditory channels refer to thesignal detection required for a task. The cognitive channel consists of the information processing and thepsychomotor channel is the physical action (e.g., speech, moving a trackball) required to accomplish a task.

One advantage to the VACP method is that it quantifies whether multi-tasking is possible in a straightforwardfashion by assuming the workload levels are additive within channels and independent between channels. Forexample, it is common practice for controllers to monitor their displays while simultaneously listening to a pilotreadback because the tasks use different channels. The monitoring task utilizes primarily the visual channel withsmall amounts of the cognitive channel (V=3.7, C=1.0 from Table 1) while listening utilizes the auditory andcognitive channels (A=4.9, C=5.3). If the monitoring task is being performed initially, the listening task can only beinitiated concurrently (i.e., multi-tasking) if each individual channel’s total workload is less than 7, which is the casefor this example since the total amount of channel utilization for both tasks is V = 3.7, A = 4.9, and C = 6.3.Numerical VACP scores above the threshold of 7 are not seen in the hybrid model because the scheduler accountsfor the excessive demand and postpones tasks accordingly based on task priorities. Scheduling is accomplishedthrough a set of queues of current, interrupted, and postponed activities to keep track of the activity order.

American Institute of Aeronautics and Astronautics4

Table 1. VACP Values and Descriptors3

Value Visual Scale Descriptor Value Auditory Scale Descriptor0.0 No visual activity 0.0 No auditory activity1.0 Register/detect image 1.0 Detect/register sound3.7 Discriminate or detect

visual differences2.0 Orient to sound, general

4.0 Inspect/check(discrete inspection)

4.2 Orient to sound, selective

5.0 Visually locate/align(selective orientation)

4.3 Verify auditory feedback(detect occurrence ofanticipated sound)

5.4 Visually track/follow(maintain orientation)

4.9 Interpret semantic content(speech)

5.9 Visually Read (symbol) 6.6 Discriminate soundcharacteristics

7.0 Visually scan/search/monitor(continuous/serial inspection,Multiple conditions)

7.0 Interpret sound patterns(pulse rates, etc.)

Value Cognitive Scale Descriptor Value Psychomotor Scale Descriptor0.0 No cognitive activity 0.0 No psychomotor activity1.0 Automatic (simple association) 1.0 Speech1.2 Alternative selection 2.2 Discrete actuation

(button, toggle, trigger)3.7 Sign/signal recognition 2.6 Continuous adjusting

(flight control, sensor control)4.6 Evaluation/judgment

(consider single aspect)4.6 Manipulative

5.3 Encoding/decoding, recall 5.8 Discrete adjusting(rotary, vertical thumbwheel,lever position)

6.8 Evaluation/judgment(consider several aspects)

6.5 Symbolic production(writing)

7.0 Estimation, calculation, 7.0 Serial discrete manipulation

Figure 0. Representative controller task depicting task sequence for R-side and D-side when receiving ahandoff from an adjacent sector. If the aircraft is in conflict, the D-side must coordinate with the adjacentsector’s D-side

American Institute of Aeronautics and Astronautics5

Conversion (keyboard entries)

In addition to the VACP considerations, multi-tasking is further constrained by system resources. For the currentHPM implementation, this includes the trackball, keyboard, and radio resources. For example, setting the radio flagto true for a given radio frequency while a controller or pilot is speaking prevents other pilots from using thatfrequency. While to a large extent the psychomotor channel encompasses these resources from the human side of theequation, these additional system resources provide further delineation and provide a boolean condition test as towhether a task can be initiated. Thus, for most of the controller task nodes (e.g., the nodes depicted Figure 0) in thetask network representation of the human performance model, the following input data is required:

1. Mean task time (sometimes calculated during run-time based on airspace events)

2. Standard deviation

3. Type of distribution (e.g., normal, gamma – human performance is often represented by a gammadistribution whereas system processes are often represented by normal distributions)

4. Visual resource allocation (0 – 7)

5. Auditory resource allocation (0 - 7)

6. Cognitive resource allocation (0 - 7)

7. Psychomotor resource allocation (0 - 7)

8. Trackball required (true/false)

9. Keyboard required (true/false)

10. Radio required (true/false)

Not surprisingly, the VACP theory of workload is also useful for quantifying workload as an output of themodel. As each task element is executed (e.g., the nodes in Figure 0), the model increments each of the VACPchannels by the amounts identified in Table 1. Upon completion of the task element, the model decrements each ofthe channels by the identical amount. As raw workload metrics, the values representing the VACP channels areextremely noisy. One task element may require high values whereas the next element may require low values. Thus,a running average of the channels is useful in better comprehending the data. In the following four figures, fourhours of traffic is depicted for the Wichita Falls High sector in Fort Worth Center for the R-side controller. A fiveminute running average was calculated for the workload channels. The running average of workload andinstantaneous traffic count are plotted as a function of time. Figure 2 depicts current operations whereas Figure 3depicts a future ATM system consisting of ADLS, EDA, and the En Route Trajectory Negotiation benefitmechanisms. In both figures, there is a strong correlation between workload and traffic count. The cognitive andvisual channels are higher than the auditory and psychomotor channels suggesting that although there isconsiderable speaking and listening being performed in current operations, they do not dominate the overallworkload. As expected, the auditory workload is essentially zero in Figure 3 since ADLS essentially eliminates theneed for voice communication. Furthermore, there is a significant drop in cognitive workload between currentoperations and the future ATM system. Figure 4 and 5 depict a similar comparison between current operations andthe future ATM system, but the four VACP channels of workload are averaged to provide a single measure ofworkload to compare against traffic count. In Figure 4, the strong correlation between workload and traffic count isreadily apparent. Figure 5 depicts a significant reduction in workload for the future ATM system as compared tocurrent operations. Amarillo, Wichita Falls, and Ardmore High sectors (from left to right) are depicted in Figure 6.

American Institute of Aeronautics and Astronautics6

0

1

2

3

4

5

6

60 120 180 240 300

Time (min)

Wo

rklo

adra

tin

g

0

4

8

12

16

20

24

Nu

mb

ero

fA

ircr

aft

inS

ecto

r

Visual Auditory Cognitive Psychomotor Aircraft count

Figure 2. Workload predictions for current operations.

0

1

2

3

4

5

6

60 120 180 240 300

Time (min)

Wo

rklo

adra

tin

g

0

4

8

12

16

20

24

Nu

mb

ero

fA

ircr

aft

inS

ecto

r

Visual Auditory Cognitive Psychomotor Aircraft count

Figure 3. Workload predictions assuming an En Route Trajectory Negotiation paradigm

American Institute of Aeronautics and Astronautics7

0

1

2

3

4

5

6

60 120 180 240 300

Time (min)

Ave

rag

eW

ork

load

Rat

ing

0

4

8

12

16

20

24

Nu

mb

ero

fA

ircr

aft

inS

ecto

r

Workload Aircraft count

Figure 4. Average workload predictions for current operations.

0

1

2

3

4

5

6

60 120 180 240 300

Time (min)

Ave

rag

eW

ork

load

Rat

ing

0

4

8

12

16

20

24

Nu

mb

ero

fA

ircr

aft

inS

ecto

r

Workload Aircraft count

Figure 5. Average workload predictions assuming an En Route Trajectory Negotiation paradigm

American Institute of Aeronautics and Astronautics8

Average workload (Table 2) is a useful parameter for characterizing relative differences between a baselinecondition and a condition that represents new procedures or new DST. Average workload over a time period ofinterest (such as an arrival rush) is expressed by:

∑=

∆=

n

j total

jijiavgi t

tWLWL

1

,,

where i is the VACP channel, j represents each task element executed by the model (e.g., move trackball, read valuefrom display, listen to pilot, etc.), n is the number of task elements executed over the total time duration, WLi,j is theVACP workload of each task element, ∆ti,j is the duration of the task element, and ttotal. is total time duration toaverage across. Note that the WLi,j values are VACP channel inputs assigned by the modeler for every possible taskelement to be performed whereas ∆ti,j is calculated during run-time based on a time distribution and standarddeviation. The four-channel average workload is reduced by 43% for the future ATM system when compared tocurrent operations.

Table 2. Average workload over 4 hour periodVisual Auditory Cognitive Psychomotor Four-channel

averageAverage

traffic countCurrent Operations 2.6 0.64 2.3 0.56 1.5 5.4Future ATM system 2.1 0.003 1.1 0.24 0.85 5.3

Figure 6. Amarillo, Wichita Falls, and Ardmore High sectors (from left to right) in Ft. Worth Center

American Institute of Aeronautics and Astronautics9

IV. ConclusionA hybrid model has been developed that integrates a human performance model of R-side and D-side tasks and

procedures with a high fidelity airspace and aircraft simulation. The airspace simulation also serves to emulate thefunctionality of controller DSTs. Higher fidelity models of workload have been added to the human performancemodel to represent visual, auditory, cognitive, and psychomotor workload channels. More detailed modeling of theD-side has been incorporated, particularly with respect to coordination between sector teams. Workload predictionshave been presented for current operations and a future ATM system consisting of ADLS, EDA, and the En RouteTrajectory Negotiation benefit mechanisms. Four hours of traffic for Fort Worth Center were simulated. A 43%reduction in workload was predicted by the model for the future ATM system assuming same traffic levels. Once themodel is validated (early 2006), new sector capacities can be established by incrementally increasing traffic levels inthe model until controller workload returns to baseline values corresponding to currently acceptable capacities.

AcknowledgmentsThe authors would like to acknowledge NASA’s Advanced Air Transportation Technologies Project as the

sponsor of this work. In addition, the authors would like to thank Steve Green (NASA Ames) for his continuedsupport and Richard Western and Stephane Mondoloni (CSSI) for their efforts in the development of the airspacesimulation.

References

1 Leiden, K., Kopardekar, P., and Green, S., “Controller Workload Analysis Methodology to Predict Increases in AirspaceCapacity”, AIAA 2003-6808, November 2003.

2 Western, R., Mondoloni, S., Leiden, K., Kopardekar, P., and Green, S., “Development of a Fast-Time, Hybrid AirspaceModel”, AIAA 2004-6398, September 2004.

3 McCracken, J.H., & Aldrich, T.B., Analyses of selected LHX mission functions: Implications for operator workload andsystem automation goals (Technical Note ASI479-024-84), Fort Rucker, AL: Army Research Institute, Aviation Research andDevelopment Activity, 1984.