San Jose State University Research Foundation

National Aeronautics and Space Administration

Controller Support Tools forSchedule BasedSchedule-BasedTerminal-Area Operations

Michael Kupfer, Todd Callantine, Lynne Martin, Joey MercerSan Jose State University Research Foundation

Everett PalmerNASA Ames Research Center

9th USA/EUROPE ATM Seminar

www.nasa.gov

June 14 to 17 2011Berlin, Germany

Outline

• Purpose

• Operational Concept

Airspace Operations Laboratory (AOL) at NASA Ames

Operational Concept

• Experimental Design– ApproachApproach

– Method

• Results

• Conclusions

2

The Problem

Low route conformance:vectoring + step-down descents

High route conformance by using g ou e co o a ce by us g4DT support tools: enabling OPDs

3

Control of Arrivals on RNAV OPDs

All aircraft are assumed FMS- and

Time-based scheduling provides runway schedule

ADS-B out-equipped

Assume nominal

En route speed and path assignments to meet meter fix schedule

Assume nominal errors in the range of

approx. 60 s early and 30 s late

Aircraft arrive with spacing errors at the

meter fix

Meter Fix

Flight in LNAV/VNAV mode – largely without controller

intervention. Terminal controllers correct

Meter Fix

Study Focus

4

residual spacing and cope with disturbances and off-nominal

events using 4DT-tools.

Controller Support for Managing OPDs

Video: Overview of Controller Managed Spacing Tools

VIDEO

5

Controller Support for Managing OPDs

"S d

Three Successively More Advanced Toolsets

"Timeline" "Slot Marker""Speed

Advisory"

/ S

Spacing Bracket

Early/Late Indication Early/Late Indication +Slot Marker

Slot Marker +Speed Advisory

Display elements available in all conditions:Timeline ith d ell able req ired spacing brackets

6

• Timeline with dwell-able required spacing brackets• Airspeed next to aircraft target• J-rings, route display, LA spacing (‘splat’)• Terminal proximity alert spacing cones (‘bats’)

ETA STA

Experimental Design

• Independent operations to LAX RWY 25L and RWY 24R• Minimum required wake vortex separation: 3, 4, or 5 nmi

S h d l b ff 15 i ti d 30• Schedule: buffer = 15 s; maximum time advance = 30 s

• Two 1-hour base traffic scenarios- 25 aircraft per runway Histogram of ‘Open-Loop’ Runway Schedule

Errors Measured at Meter Fixesp y(scheduled throughput ~64 aircraft/hr)

• Headwind off runway 8%

10%

12%

f Airc

raft

Errors Measured at Meter Fixes

early lateHeadwind off runway- 10 kts faster, 10 kts slower, and

same as forecast winds

T il t di t b2%

4%

6%

Perc

enta

ge o

f• Two pilot disturbances per run

- Early/late slowdown for a speed restriction

0%-80 -60 -40 -20 0 20 40

Schedule Error Bins [x, x+5 s)

3 Tool Conditions X 3 Wind Conditions X 2 Base Scenarios = 18 Experimental Trials7

Simulation Airspace and Routes

• Continuous RNAVContinuous RNAV routes to LAX24R and LAX25L

• Approx. 2.4 deg d t l t lldescent angle to allow for speed control

• Nominal speed/altitude restrictions- 280 kts at 17,000 ft- 240 kts at 10,000 ft- 210 kts at 7,000 ft- 180 kts at 4,500 ft 8

Route and Profile Conformance

No vectoring; high conformance to OPD profiles (N = 900 aircraft)9

Arrival Throughput and Workload

70Throughput Histogram (15 min ‘windows’)

40

50

60

Airc

raft/

h]

Slot Marker

AdvisorySpeed Advisory

Tool Condition

10

20

30

Thro

ughp

ut [

A Advisory

Timeline

Speed Advisory

00 5 10 15 20 25 30 35 40 45

T

Time Bins [x - 7.5, x + 7.5 min)

Reported workload: always low

10

Spacing Accuracy

100

Histogram of Inter-arrival Spacing Errors Measured at Runway Threshold

80

Pai

rs Speed Advisory

Slot Marker

Tool Condition

40

60

r of A

ircra

ft

Timeline

20Num

ber

0-1 -0.5 0 0.5 1 1.5 2

Spacing Bins [x, x + 0.1 nmi)

No significant differences across tools or winds (α = 0.05)

11

Wake Spacing Violation Instances

• Five wake spacing violations• 1 final controller error• Five wake spacing violations• 1 final controller error 250

300

Aircraft Pairwise Spacing at the Runways(all conditions)

1 final controller error3 (tower) pseudo pilot error 1 software error

1 final controller error3 (tower) pseudo pilot error 1 software error

150

200

250

ency

violation excess spacing

50

100Freq

ue0

-1.0 -0.5 0.0 0.5 1.0 1.5 2.0

Spacing Bins [x, x+0.1 nmi)89

10

violationscontroller

34567 controller

error

0123

-1.0 -0.8 -0.6 -0.4 -0.2 0.0 12

Schedule Conformance

10%

12%

craf

t

Histogram of Runway Schedule Errorsat the Meterfix (Openloop runs)

early late

(All conditions)

4%

6%

8%

rcen

tage

of A

irc

60%

Histogram of Runway Schedule ErrorsAt the Runway Threshold (data runs)

early late

0%

2%

-80 -60 -40 -20 0 20 40

Per

Schedule Error Bins [x, x + 5 s)30%

40%

50%

60%

e of

Airc

raft

early late

0%

10%

20%

30%

Per

cent

age

(All conditions)

mean = -17.7 s

-80 -60 -40 -20 0 20 40

Schedule Error Bins [x, x + 5 s)

(N = 279 (N = 900 mean = -1.2 sstd. dev. = 23.8 s‘Open-loop’ runway schedule errors, measured at meter fixes

(aircraft)

(aircraft)std. dev. = 5.2 s

13

Tool Usage

100]Questionnaire data

Support Tool

70

80

90

timelline

e U

sed

[%]

Timeline

40

50

60 slot marker

advisory

/age

of T

ime Slot Marker

Speed Advisory

0

10

20

30 early/late indicators

Perc

enta Early/Late Indicators

0timeline condition slot marker condition advisory condition

Tool Condition

"Timeline" "Slot Marker" "Speed Advisory"

Reported timeline usage significantly different (α = 0.05)

14

Tool Usability and Usefulness

5Slot Marker

Questionnaire dataVery helpfulVery helpful

3.5

4

4.5

Timeline

ss

2.5

3Speed Advisory

Use

fuln

es

Somewhat helpfulSomewhat helpful

1

1.5

2

Tool

U

Not helpful at allNot helpful at all

0

0.5

1

1 1 5 2 2 5 3 3 5 4 4 5 5

Not helpful at allNot helpful at all

1 1.5 2 2.5 3 3.5 4 4.5 5

Tool Usability 15

Very difficult tool to useVery difficult tool to use O.k. to useO.k. to use Very easy tool to useVery easy tool to use

Tool Usability and Usefulness

5Slot Marker

Questionnaire dataVery helpfulVery helpful

3.5

4

4.5

Timeline

ss

Early/Late Indicator

Airspeed on Slot Marker

2.5

3Speed Advisory

Use

fuln

es

Groundspeed in Data Block

Spacing Brackets on Timeline

Airspeed on Aircraft Target

Spacing Cones

Somewhat helpfulSomewhat helpful

1

1.5

2

Tool

U

J-rings, Route display, LA spacing

Spacing Cones

Not helpful at allNot helpful at all

0

0.5

1

1 1 5 2 2 5 3 3 5 4 4 5 5

Not helpful at allNot helpful at all

1 1.5 2 2.5 3 3.5 4 4.5 5

Tool Usability 16

Very difficult tool to useVery difficult tool to use O.k. to useO.k. to use Very easy tool to useVery easy tool to use

Conclusions

• Successfully mitigated runway schedule errors and avoided spacing violations with basic displayavoided spacing violations with basic display enhancements

• Avoidance of vectoring strategies, enabling OPDs

• Slot markers yielded a slight performance advantage and were most preferred by controllersand were most preferred by controllers

• Speed advisories highlight issues to consider in implementing new controller support tools

17

Future Research & Integration

• CMS4: off-nominal conditions• Continuous tool improvement• SDO support• Field DemonstrationField Demonstration

18

Questions?Questions?

Michael Kupfer, Todd Callantine, Lynne Martin, Joey MercerSan Jose State University Research Foundation

Everett Palmer

Airspace Operations LaboratoryNASA Ames Research CenterBldg N262, Rm. H207

NASA Ames Research Center

19http://humansystems.arc.nasa.gov/groups/AOL/

Moffett Field, CA 94035U.S.A.

Backup Slides

20

Spacing Violations

3

4

Spacing ViolationsFFT2406: "1-7-0 to

LIMMA, then charted speeds"

1

2speeds"

Charted speeds were 180 kts

0Minus None Plus Minus None Plus Minus None Plus

Advisories Slots TL

Run Condition Winds Lead Lead Type Trail Trail

Type Runway Relative Distance

Error Type

12 Timeline Plus FFT2406 Large ASA450 Large LAX25L -0.12 Controller Error

13SlotMarker

None ASA243 Large UAL507 Heavy LAX24R -0.12 Pilot Error

18 Timeline None NWA6104 Large NWA303 B757 LAX25L -0.47Software Error

21

19 Timeline None ASA470 Large ANZ1428 Heavy LAX24R -0.41 Pilot Error

19 Timeline None SKW6102 Large UAL935 Heavy LAX25L -0.73 Pilot Error

Display Example with Tool Descriptions

• Slot markers show each aircraft’s predicted location were it to fly the nominal profile through the forecast wind field and arrive on schedule (dwelling on an

• Speed advisories show a speed to fly until resuming the nominal profile speeds at a waypoint (if no such advisory can be found thewind field and arrive on schedule (dwelling on an

aircraft highlights its slot marker)• Display shows airspeed of both slot marker and

target aircraft

waypoint (if no such advisory can be found, the early/late indication is displayed)

• Cones (‘bats’) may be toggled for all aircraft or specified aircraft

22

Throughput

Runway Throughput

50

60

70

hr]

Throughput(by tools condition)

20

30

40

50

Freq

uenc

y [a

c/h

Slot Marker

Advisory

Timeline

0

10

0 5 10 15 20 25 30 35 40 45

F

Time bins [x-7.5, x+7.5) [min]

Timeline

Throughput(by wind condition)

50

60

70ac

/hr]

( y )

10

20

30

40

Freq

uenc

y [a

Minus-Bias

No-Bias

Plus-Bias

23

0

10

0 5 10 15 20 25 30 35 40 45

Time Bins [x-7.5, x+7.5) [min]

Schedule Conformance

150

200

r [s]

Runway Schedule ConformanceRunways

early late

50

100

150

ched

ule

Erro

r

Advisories

Slots

TL

Advisory

Slot Marker

Timeline

0

S

A R S h d l E

4

6

8

[s]

Average Runway Schedule ErrorBoth runways, all runs

Error bars: +/- 1σ

• Advisory and the slot marker condition: no statistical difference (α=0.05)

-2.03 -2 58

0.96

4

-2

0

2

4

ched

ule

Erro

r [• Timeline condition: significant different compared to other two tool conditions (advisory vs. timeline:t (299)=1.97, p<0.01; slot marker vs. timeline: t (299)=1 97 p<0 01) 2.58

-8

-6

-4

Advisories Slots TL

Sctimeline: t (299)=1.97, p<0.01)

24

Schedule Conformance

160180200

[s]

Runway Schedule ConformanceBoth runways, all runs

early late

6080

100120140

ched

ule

Erro

r s

Minus

None

Plus

Minus-BiasNo-BiasPlus-Bias

02040

-35--31-30--26-25--21-20--16-15--11 -10--6 -5--1 0-4 5-9 10-14 15-19 20-24

Sc

4

6

8

ec]

Average Runway Schedule ErrorBoth runways, all runsSignificant differences between all

wind conditions

T-tests for paired two sample for

Error bars: +/- 1σ

-1.12

1.94

-4

-2

0

2

4

hedu

le E

rror

[seT tests for paired two sample for

means:• minus-bias vs. no-bias:

t (299)=1.97, p<0.01• minus-bias vs plus-bias:

-4.46

-10

-8

-6

Minus None Plus

Schminus bias vs. plus bias:

t (299)=1.97, p<0.01 • no-bias vs. plus- bias:

t(299) = 1.97, p<0.01 Minus-Bias No-Bias Plus-Bias 25

Mean ATWIT workload ratings

2 3 6

VeryHigh

Somewhat High High 

1       2        3       4         5        6         VeryLow

Somewhat Low

Low

Tool condition

26

Tests of mean ATWIT workload ratings

27

* Significantly different at the p < 0.05 level* Significantly different at the p < 0.05 level

With standard error bars

Mean TLX workload ratings

ad ra

ting

6

7

Timeline

Slot Markers

Very HighTool condition

oc workloa

4

5 Advisories

Average

ean po

st‐h

2

3

M 1

Minus‐bias No‐bias Plus‐bias

Very Low

3 wind‐forecast‐error conditions

28

Values plotted with respect to the TLX scale, i.e., 1 to 7 & with standard error barsValues plotted with respect to the TLX scale, i.e., 1 to 7 & with standard error bars

Mean TLX workload ratings

wind‐forecast‐error conditions

The six TLX scalesThe six TLX scalesValues plotted with respect to the TLX scale, i.e., 1 to 7 (with standard error bars)

29

* Significantly different at the P < 0.05 level# Significantly different at the P < 0.01 level* Significantly different at the P < 0.05 level# Significantly different at the P < 0.01 level

Actual and Forecast Winds

• The same forecast wind profile was used for all runs.

• The same forecast wind profile was used for all runs.

Forecast and Actual Wind Profiles• The aircraft’s FMS and air traffic control tools used the same forecast wind profile

• The aircraft’s FMS and air traffic control tools used the same forecast wind profile

25000

30000

Forecast and Actual Wind Profiles

profile.

• Experimental runs used one of three actual wind profiles.

profile.

• Experimental runs used one of three actual wind profiles. 15000

20000Forecast

Actual wind -MINUSA

ltitu

de

Minus-bias

No-bias

• The actual wind profiles are either ~10 knots faster, ~10 knots slower or the same as the forecast at

• The actual wind profiles are either ~10 knots faster, ~10 knots slower or the same as the forecast at

5000

10000Actual wind -PLUS

Plus-bias

same as the forecast at altitudes below 20,000 ft.

• From 1,500 to the ground the three wind profiles are id ti l

same as the forecast at altitudes below 20,000 ft.

• From 1,500 to the ground the three wind profiles are id ti l

00 10 20 30 40 50 60 70 80

Wind speed kts]

30

identical.identical.

Trajectory-based Advisory Tool

220220 JETSA

210

200

d [k

ts] 200 JETSA

190

180ed A

irspe

ed 190 GAATE

170Indi

cate

Charted speeds(a) a/c early: reduce speed(b) a/c delayed: speed up

160

150

JETSA GAATE

( ) y p p(c) a/c delayed: reduce speed + hold

31

140JETSA GAATE

Distance to go

Scenario Characteristics

Scenario A Scenario BAbsolute 

A/ccount

Relative A/c count

Absolute A/c count

Relative A/c count

Scenario A  Scenario B Runway/Route

Absolute A/c count

Relative A/c count

Absolute A/c count

Relative A/c count

LAX24R 25 50.00% 25 50.00% count

Total 50 100% 50 100%B757 5 10% 9 18%Heavy 12 24% 9 18%

LAX24R  25 50.00% 25 50.00%SADDE7 19 38.00% 16 32.00%RIIVR2 6 12.00% 9 18.00%OLDEE1SEAVU2 Heavy 12 24% 9 18%

Large 33 66% 32 64%SEAVU2SHIVE1LEENA4LAX25L  25 50.00% 25 50.00%SADDE7SADDE7RIIVR2 8 16.00% 7 14.00%OLDEE1 3 6.00% 2 4.00%SEAVU2 6 12.00% 7 14.00%SHIVE1 5 10 00% 4 8 00%SHIVE1 5 10.00% 4 8.00%LEENA4 3 6.00% 5 10.00%

32

Number of Clearances per Aircraft

2.64 2.702.18

22.5

33.5

44.5

ces/

airc

raft Mean Standard

DeviationMedian Min. Max.

2.5 1.42 3 0 7

00.5

11.5

2

Advisories Slots TL

# cl

eara

n

Advisory Slot Marker Timeline

• Timeline condition fewer clearances than other tool conditions

• No-bias wind condition fewer clearances than other wind conditions

2.73

1.86

2.92

22.5

33.5

44.5

ces/

airc

raft

other wind conditions• Sector 205 (Feeder South) and final sector 203

(Downe) issued the least clearances

Clearance Frequencies86

00.5

11.5

2

Minus None Plus

# cl

eara

nc

Minus-Bias No-Bias Plus-Bias

Clearance Frequencies

33

Some more speed advisory considerations

Speed advisories:• Match the controller strategies, or ask the controllers to get used to / g , g

accept the algorithm logic?• Advisories might be necessary to help/enforce actions that are not

intuitive/obvious to the controllers but would benefit the overall system

• Also, maybe advisories were not as useful because controllers were easily able to formulate own speeds and mitigate the well-bound y p gschedule errors

• Research effort: speed advisories for time-based schedule information speed advisories in support of Interval Managementp pp g

34