Post on 29-Jan-2016
description
Collaborative Convective Forecast Product “CCFP”
CygnaCom Solutions, Inc. Suite 100 West, 7927 Jones Branch Drive, McLean, VA 22102-3305 (703) 848-0883
Kevin Browne FAA ARW-100Mark Phaneuf CygnaCom SolutionsDenny Nestoros CygnaCom Solutions
October 13, 1999
2
Agenda
• Goal/Purpose• Evaluation methods• Evaluation results• Next steps
3
Goal/Purpose
• GOAL– Improve the decision making process within the CDM
framework and lead to reduction in delays, reroutes and cancellations influenced by convective events
• PURPOSE– A test program to evaluate the CCFP in an operational
setting to determine its usefulness in aiding the decision making process for ATC service providers and airlines with the CDM framework
5
Evaluation Methods
• Production assessment• Quantitative assessment • Qualitative assessment being done by Forecast
Systems Lab (FSL)• Questionnaire
6
Production Assessment
• The objective of this phase of the evaluation was to determine how well the coordination process worked and what procedures were needed to improve it
7
Production Assessment Continued
• Production Assessment– Number of participants– Number of messages– Number of iterations– Number of agreements– Number of agreements by default– Number of disagreements– Length of collaboration– Trends
Participants
15.16
16.71
14.34
14.63
15.31
15.56
16.06
13.00
13.50
14.00
14.50
15.00
15.50
16.00
16.50
17.00
AVG Glbl AvgMorn
Glbl AvgAft
Glbl AvgMay
Glbl AvgJune
Glbl AvgJuly
Glbl AvgAugust
Metric
Nu
mb
er
Messages
19.63
21.07
17.97
18.73
18.98
20.47
19.32
16.00
16.50
17.00
17.50
18.00
18.50
19.00
19.50
20.00
20.50
21.00
21.50
AVG Glbl AvgMorn
Glbl AvgAft
Glbl AvgMay
Glbl AvgJune
Glbl AvgJuly
Glbl AvgAugust
Metric
Nu
mb
er
Iterations
6.46 6.58
5.97
3.60
5.05
8.03
6.65
0.00
1.00
2.00
3.00
4.00
5.00
6.00
7.00
8.00
9.00
AVG Glbl AvgMorn
Glbl AvgAft
Glbl AvgMay
Glbl AvgJune
Glbl AvgJuly
Glbl AvgAugust
Metric
Nu
mb
er
Agreement
5.76
6.16
5.06
4.07
5.43
6.06 6.03
0.00
1.00
2.00
3.00
4.00
5.00
6.00
7.00
AVG Glbl AvgMorn
Glbl AvgAft
Glbl AvgMay
Glbl AvgJune
Glbl AvgJuly
Glbl AvgAugust
Metric
Nu
mb
er
Default Agreement
9.21
9.45
8.93
8.33
9.00
9.24
9.68
7.50
8.00
8.50
9.00
9.50
10.00
AVG Glbl AvgMorn
Glbl AvgAft
Glbl AvgMay
Glbl AvgJune
Glbl AvgJuly
Glbl AvgAugust
Metric
Nu
mb
er
Disagreement
0.52
0.45
0.23
0.67
0.50
0.31
0.11
0.00
0.10
0.20
0.30
0.40
0.50
0.60
0.70
AVG Glbl AvgMorn
Glbl AvgAft
Glbl AvgMay
Glbl AvgJune
Glbl AvgJuly
Glbl AvgAugust
Metric
Nu
mb
er
Minutes of Conversation
49.83
47.58
53.45
43.90
50.02
52.42 52.15
0.00
10.00
20.00
30.00
40.00
50.00
60.00
AVG Glbl AvgMorn
Glbl AvgAft
Glbl AvgMay
Glbl AvgJune
Glbl AvgJuly
Glbl AvgAugust
Metric
Nu
mb
er
15
Quantitative Evaluation Method
• Quantitative Assessment– Historical baseline
• Certain days from 1999 that will show traffic movement and deviations from their filed flight plan during non-CCFP days.
• Tracks time of delay from flight-plan route
– Current Procedures with CCFP• Certain days from 1999 data with similar representation showing
movement and deviations but comparing how it was handled with CCFP using POET’s data mining tools
– This analysis will evaluate the differences between system performance under the current procedures (the baseline) versus the system performance with the CCFP
16
Quantitative Evaluation Method Continued
• Quantitative Assessment Continued– Four specific areas will be evaluated:
• Delays attributable to weather
• Net deviation (in time) from planned arrival time caused by weather
• Number of cancellations
• Number of diversions
17
Quantitative Results
POET Analysis, Actual Flight Time < Planned Flight TimeAirlinesPeriod / Metric
AAL DAL NWA TWA UALTotal Instances
Total 8/99 to 9/99 9 5 1 - 5 208/99 total # / % 4 / 33% 4 / 33% - - 4 / 33% 12 / 60%9/99 total # / % 5 / 63% 1 / 13% 1 / 13% - 1 / 13% 8 / 40%
18
Questionnaire
• There were three objective questions on the questionnaire that were designed to gather information for the operational evaluation:
– (5) Did you use the CCFP for planning purposes?
– (6) The CCFP was useful for planning purposes?
– (7) The CCFP was reflective of weather conditions?
19
Questionnaire Results
• Results from question number (5) Did you use the CCFP for planning purposes?
– 60% used the CCFP for planning purposes
– 23% reported they did not use CCFP for planning purposes
– 3% abstained from answering this question
• Results from question number (6) The CCFP was useful for planning purposes?
– Over 57% of the respondents agreed or strongly agreed
– 33% of respondents had no opinion
– 10% disagreed or strongly disagreed
20
• Results from question (7) The CCFP was reflective of weather conditions?
– 71% indicated that the CCFP was often accurate
– 8% indicated that the CCFP was always accurate
– 14% indicated that the CCFP was seldom accurate
– 2% indicated never accurate
– Approximately 5% abstained from answering
Questionnaire Results, Continued
21
Next Steps
• Complete evaluation report• Identify funding for next year and beyond if the
product becomes operational• Determine the product output and how often it
will be generated• Training
22
Points of Contact
• Kevin Browne FAA ARW-100 kevin.browne@faa.gov (202) 366-1066
• Mark Phaneuf CygnaCom Solutions, Inc. mphaneuf@cygnacom.com (703) 848-0883
• Denny Nestoros CygnaCom Solutions, Inc. dnestoros@cygnacom.com