Project on Unmanned Aircraft in the NAS Final Review Panel Meeting

1

Project on Unmanned Aircraft in the NASFinal Review Panel Meeting

2

Integration of Unmanned Aircraft into the National

Airspace System

A Project Course byCarnegie Mellon University

Dept. of Engineering and Public Policy

Dept. of Social and Decision Sciences

May 1, 2007

3

Expert Review Panel

Tom Curtin, AUVSI Bret Davis, AUVSI Lexa Garrett, America West Airlines Jim Geibel, GAO David Gerlach, FAA Tom Henricks, Aviation Week Ramon Lopez, Aurora Flight Sciences Edmond Menoche, GAO Rene Rey, FAA Melissa Rudinger, Aircraft Owners & Pilots

Assn. James Sizemore, FAA Larry Thomas, GAO Dyke Weatherington, DoD/OSD

4

Purpose of CMU Project Courses in Technology and Policy

Analyze a “real world” policy problem involving technology

Combine diverse information and analytic frameworks to derive policy insights

Learning objectives: Problem decomposition, structuring

and formulation Interdisciplinary problem solving Communication Teamwork

5

Examples of past project courses

Title Year

Safety and the Light Truck Craze: Who Wins? Who Loses? Who Cares?

2000

Environmental Impacts of E-commerce - A case study of book purchasing

2000

Sustaining Pittsburgh's Vital Services When the Power Goes Out

2004

Wireless Communications Systems for Emergency Responders

2004

Hybrids and Diesels in the American Automobile Fleet

2005

The Impact of Spyware 2005

Safety of Implanted Cardiac Devices 2007

6

Contributors to our UAS project

20 undergraduates majoring in: Engineering Social Science Business Administration

3 Ph.D. student managers 3 faculty advisors Expert review panel Other experts

7

Background for this Project

Increasing demand for UA Military (many current uses) Civilian (many potential uses)

Federal Aviation Administration (FAA) is developing a roadmap for integrating UA into the NAS

A few of the issues to be addressed: Safety and reliability Public acceptability Market viability

8

Analysis Areas Economics

How cost-effective are UA compared to alternative means of providing specific services?

Risk, Technology and Standards What are the regulatory implications of

different approaches to “equivalent level of safety?”

Public Awareness and Perceptions Are risks of UA of greater public concern than

risks of manned aircraft? Governance

How can the current system for deliberation and decision-making on UA access be improved?

9

Project Outcomes

~16 person-months of research completed across the four focus areas

Economic model of market viability Risk model of fatality implications of

UA introduction Better understanding of public

awareness & risk perception Actor & “roadblock” analysis yields

insight on deliberative process for UA integration

Regulatory & policy recommendations

1010

Economics

Team Members:Nathan Diorio-TothFeng DengReiko BaughamVictoria MortonBrad Brown

Team Manager:Ryan Kurlinski

1111

Purpose

Assess the market viability of UAS applications using relative cost effectiveness

Assess the effect of various regulatory measures on the market viability of UAS applications

12

Goals

12

Develop UAS cost model Cost components

Airframe Communications Insurance Pilot Etc.

Apply cost model to chosen applications and alternatives to compare cost

Examine sensitivity of overall cost to changes in each cost component

Estimate cost implications of different regulatory measures and technology improvements

13

UAS Applications

Weather Reconnaissance Alternative:

WC-130J Hercules: high-wing, medium range aircraft

Pipeline monitoring Alternative:

Concentric sensors: pressure sensitive sensors

Localized Surveillance Alternative:

Traffic Helicopter: e.g. Bell JetRanger

13

14

Analysis Method

Used triangular distributions to assign probable ranges to each input cost From this, generated a Probability

Density Function Probability Density Function shows

the entire range of possible costs with the associated likelihood of each cost

Allows analysis of the most probable cost advantages

14

15

Importance Analysis

Triangular probability

distributions of all input variables

Economic Model

Contribution of uncertainty in each input to uncertainty in total cost

16

Weather Reconnaissance

Analyzed the use of Aersonde UAS for Weather Reconnaissance vs. the use of the WC-130J Hercules

Aerosonde UAS currently in use for Weather Reconnaissance

Hercules WC-130J currently in use by Keesler Air Force Base

16

17

Pro

bab

ility

De

nsi

ty

Alt Cost-UAS Cost0 100K 200K 300K 400K 500K 600K 700K

0

2u

4u

6u

8u

10u

12u

Results: Weather Reconnaissance

17

($/flight hour)

Probability Density of UAS Cost Advantage

18

Imp

ort

an

ce i

n A

lt C

ost

-UA

S C

ost

Alt Cost-UAS Cost Inputs

Component costsOperational lifetime

ManpowerCom-Link Cost

Cost per gallonInsurance Rate

Hours per yearGallons per Hour

Safety Technology Cost0

0.2

0.4

0.6

0.8

1


18

Importance Analysis of Model Inputs

Mission Hours per Year

Com-Link CostOperational Lifetime

19


Key Results: UAS more cost effective than current

manned alternative Most important inputs in determining

overall cost effectiveness: Mission hours per year Com link cost Operational lifetime

Currently available sense-and-avoid equipment cause significant decrease in cost effectiveness, but does not cause the UAS to be more expensive than the manned alternative

19

20

Pipeline Monitoring

Analyzed the use of the Aero Environment AeroPuma vs. the use of concentric wire sensors ($6+/m)

Note the difference in monitoring style UAS monitors using thermal imaging with

each pass and relays pertinent leak info to docking stations

Concentric sensors constantly monitor pipeline and relay information

21

Results: Pipeline Monitoring

Key Results: UAS cheaper depending on number in

use Important to note difference in

monitoring styles between UAS and concentric sensor

Important inputs: Relay/Docking station cost Number of UASs in use

21

22

Localized Surveillance

Application based on the surveillance of a 1km2 area for a short time (~1-3 hours)

Considered the use of a Cyber Defense Systems CyberBUG vs. the use of a traffic helicopter

For model inputs, considered monitoring a large traffic accident over 2 hours

For policy considerations, analyzed the addition of mandated sense-and-avoid hardware to the UAS

22

23

Results: Localized Surveillance

23

PDF of Cost per Mission for UAS Compared with Manned Alternative

Cost per Mission ($)

Probability Density

2000 4000 6000 8000 10K8000 12K 14K 16K

Note: no meaningful overlap

2424

PDF of Cost per Mission for UAS Compared with Manned Alternative with High-Range

Fixed Cost Variance


Probability Density

2500 5000 7500 17.5K 20K 22.5K

Note: still no meaningful overlap


10K 12.5K 15K

25


25

PDF of Cost per Mission for a Larger UAS Capable of Carrying Sense-and-Avoid Equipment Compared with the Cost of

Manned Alternative

Probability Density

0 10K


20K 30K 40K 50K 60K 70K

Note: Significant overlap indicating that UAS would likely no longer be a viable alternative to manned craft

26


26

Importance of inputs.

Input CostsInput Importance for Cost Per Mission

Missions per Year

Mission Related Costs

Flight Hours Per Mission

27


Key Results: UAS less expensive in almost every case Levelized cost for manned more

sensitive than to utilization hours & discount rate than cost for unmanned

UAS cost effectiveness reduced significantly by requirement for sense-and-avoid hardware

Important inputs: Missions per year Discount rate Flight hours per mission

27

28

Policy Implications

Analyzed the effect of the following policies: Mandated insurance premiums Mandated use of A/N hardware

(Increased fixed cost)

Mandated record-keeping practices (Increased yearly cost)

Mandated airframe materials (Increased fixed cost)

Mandated minimum amount of safety equipment (Increased fixed cost)

Mandated pilot/operator training

28

29

Policy Implications: Results

All policies except mandated sense-and avoid hardware had little effect on the cost advantage of UAS over manned alternative

Required sense-and-avoid hardware greatly affects cost-effectiveness, however Localized Surveillance and Pipeline

Monitoring would no longer be viable as larger, much more expensive UAS would be necessary

29

3030

Risk, Technologies, & Standards

Team Members:Samiah AkhtarJonathan CornellNicole HaywardWill KimNick MisekDoug Robl

Team Manager:Keith Florig

31

Purpose Derive a risk model to explore how risk is related to UAS numbers, dimensions, and flight zones

Research on elements of risk mitigation such as human factors, sense and avoid

Exploration of alternative incident reporting systems

Predator

Source: http://www.fs.fed.us/psw/news/PSW_News/2005_09/ima

ges/uav.gif

32

Technology and Risk Outline

Goals Risk Modeling

Purpose Assumptions & Approach Findings

UAS Risk Mitigation

33

Risk Modeling Purpose

Provide a way of modeling that creates some groundwork for future modeling

Use model to compare relative risk calculations

Pointer to the future, not the answer Points of interest

Mid-air vs. single-craft crash Effect of sense and avoid technology UAS to displace manned aircraft

Source: http://www.maximog.com/images/sublevel/uav_left.jpg

34

Risk Modeling Assumptions Uniform national model Uniform traffic density Uniform ground population density Uniform aircraft per type Appropriate for:

VFR traffic Rural, less populated areas

NOT Appropriate for: Urban settings Airports High traffic densities

35

Risk Model

36

Risk Modeling Approach

UAV Picture Source: http://www.evworld.com/press/spider_lion_uav.jpg

Number of midair collisions:

N = total number of aircraft in defined airspaceρ = aircraft traffic densityD = diameter of plane (wingspan)S = average aircraft speedP(A) = probability of avoidance

(Used for calibration)

yearSDAPN sec/1031)](1[ 62

VFR operations only

37

Risk Modeling: UAs displacing Manned

Small risk from unmanned at lower extrema

Expected Annual Fatalities vs. % of Airspace Unmanned

0

2

4

6

8

10

12

14

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

% of Total Aircraft is Unmanned

Ex

pe

cte

d A

nn

ua

l M

ort

ali

ty

Manned Mid-AirsManned Single-Craft

Unmanned Mid-Airs P(Avoid)=0Unmanned Mid-Airs P(Avoid)=0.5

Unmanned Single-Craft 100 failures/100k flight hrsUnmanned SIngle-Craft 10 failures/100k flight hrs

Unmanned Single-Craft 500 failures/100k flight hrs

Risk from unmanned at low levels less than

decreased risk from manned

Single-craft crashes still present less risk than mid-

airs

38

Expected Annual Mortality vs. % UASs Added to NAS

0

5

10

15

20

25

30

35

40

0% 20% 40% 60% 80% 100% 120% 140% 160% 180% 200%

% UASs Added

Exp

ecte

d A

nn

ual M

ort

ality

Single-Craft 50 failures/100k flight hrsSingle-Craft 200 failures/100k flight hrsMid-Airs P(Avoid)=0Mid-Airs P(Avoid)=0.25Mid-Airs P(Avoid)=0.5Mid-Airs P(Avoid)=0.95Single-Craft From Substituting MannedMid-air from Substituting Manned

Risk Modeling: Mid-Air vs. Single-Craft

At some point, manned

risk surpasses

unmanned risk

Single-Craft generally less risk than mid-

air

At low numbers, sense and avoid has little effect

39

Risk Modeling Conclusions

Mid-air collisions generally have more risk than single-craft crashes

Displacing small to moderate amounts of manned craft represents decrease in risk

Smaller, less reliable UAs can present less risk than larger more reliable manned aircraft

For small numbers of UAs in low traffic densities, sense and avoid has small effect

40


Goals Risk Modeling UAS Risk Mitigation

Human Factors Sense and Avoid

41

Human Factors Implications

Risks - Caused Most Number of Accidents “Sensory Isolation” [McCarley et al]

UAS operator does not receive same sensory cues as manned aircraft operator

Automation Malfunction of automated components

controlled by the UAS operator Operator Hand-Off

Issues with handing off control of vehicle from one operator or crew to another

42

Human Factors Implications

Recommendations Training and Procedures

Up to date training as new technology advancements arise

Ensure that operator has accurate knowledge of automated components within UAS

Multimodal displays Prevent sensory isolation Allow for audio, visual and speech

control Example: simulated cockpit

43

Detect, Sense and Avoid

Risks Market impact of single fatal

collision Lack of standardization among DSA

systemsSense and Avoid Technology

Cost* Size* Weight* Power Usage*

Pros Cons

IFF Transponder $500-3000 1.5m2 6 kg 30 watts, nominal

Low-cost, Transponder requirement

ADS-B Transponder

$2000-6000 2.0m2 8 kg 50 watts, nominal

Compatible with NGATS

High power, transponder

EO/IR Sensors $50,000 - 200,000

4.0m2 5-10 kg 10-20 watts

Detection in IFR and VFR, no metal requirement

Short detection range, high cost

Synthetic Aperture Radar

$10,000 - 50,000

3.0m2 20 kg 80 W All-weather Weight, high power

44

Detect, Sense and Avoid

Recommendations Create regulations specific to

size, weight, application etc Testing Periods Phased Integration

45


Issues Goals Risk Modeling UAS Risk Mitigation

Reporting systems

46

Current Reporting Systems

Two Options NTSB Reporting (as required by FAR) -

Accident NASA ASRS Voluntary Reporting - Incident

Current Implementation NTSB mandates detailed information when:

Flight control system malfunction, Illness of crewmember, Turbine Engine Failure, In-flight fire, Mid-air collision or Damage in excess of $25,000 to other property

ASRS System is anonymous and does not have any reporting requirements

47

Reporting Recommendations Initially mandate reporting of all accidents

and incidents Re-evaluate strategy after testing period

NTSB

FAAUAS manufacturers

- NTSB provides useful information on UAS failures

- UAS responds with improved design and engineering

- NTSB information helps FAA to assess standards

- FAA responds with rules for reporting incidents.

Communication

Triangle

4848

Public Awareness & Perceptions

Team Members:Darian GhorbiJenny KimMark PetersonLaura SeitzPatrick Snyder

Team Manager:Pete Tengtrakul

49

Statement of Purpose

Add the element of public perception to the discussions of UAS in the NAS

Motivation: the fact that there has never been a formal presentation of public perception on the topic

Findings: useful for the creation of regulations and policy implications

50

Objectives

Compare public perceptions of the risks concerning manned and unmanned aircraft

Find demographic groups with certain risk and benefit patterns of UAs

Research implications of opinion of UAs

Create survey to aid in completing objectives

51

Hypotheses Perceived Risk

- Manned < Remotely Piloted < Autonomous Ground vs. Air

- More risk of UAS perceived in air Prior Knowledge vs. Risk Perception

- Prior knowledge, associate less risk Benefit vs. Risk Perception

- Higher benefit, lower risk Education vs. Risk Perception

- Technical education, associate less risk Age vs. Risk Perception

- Older participants more cautious Frequency of Flight

- Those that fly frequently, associate less risk

52

Layout of Survey First Page

Provide information about UAS Autonomous Remotely Piloted

Gauge previous knowledge Source

Last Page Demographics

Gender Age Education Frequency of Flight Voting (identify opinions of those that are

politically engaged) Pilot

53

Layout of Survey

Picture of UAS application

Application

•Traffic Monitoring

•Pipeline Monitoring

•Disaster Relief

•Border Patrol

Description of UAS

•Physical Information

•Current application

Questions

• Quick Response

• Benefit

• Stakeholder

• Public

• Risk

• Ground

• Air

• 7 Point Scale

• 1 - Much Less

• 4 - Same

• 7 - Much More

54

Obtaining Surveys

Method # of Surveys

Mon Valley NGO

79 (56%)

Word of Mouth

62 (44%)

Total 141

Coding: numerical code assignedScreening: data obtained from those under 16 years of age were not counted

55

Statistical Methods

Paired T-tests Across applications

ANOVA Significance of mean

Regression Correlations

Demonstrated the strength of the variables (risk and benefit)

56

Results:Descriptive Statistics

Demographic Variable Mean Scale Previous Knowledge 0.53 1= heard, 0 =never heard

Gender 0.44 1=Male, 0=Female Age 36.04 Years

Education 2.33 1=High School,4= Graduate Register to Vote 0.66 1=Registered,0=not

Frequency of Flight 2.51 1=never, 5=more than 12x Licensed Pilot 0,01 1=Licensed,0=not

Variable Remotely Piloted Autonomous Support .75

(0.45) .58

(0.50) Benefit to Stakeholders 4.65

(1.57) 4.29

(1.77) Benefit to Society 4.14

(1.58) 4.01

(1.66) Risk to people on Ground 4.36

(1.51) 4.81

(1.66) Risk to people in the Air 4.55

(1.39) 4.87

(1.68)

57

Perceived Relative Risks Between Remotely Operated vs. Autonomous

Autonomous applications are viewed to have more risk in comparison to remotely operated UAs.

DisasterBorderTrafficPipeline

0

1

2

3

4

5

6

7

UA Applications

Perc

eiv

ed

Ris

k (

1=

Lo

w,

7=

Hig

h)

Remotely operated UA

Autonomous UA

58

Relative Risks Across Applications

Pipeline TrafficBorder Disaster

0

1

2

3

4

5

6

7

UA Applications

Pe

rce

ive

d R

isk

(1

=L

ow

, 7

=H

igh

)


Autonomous UA

Traffic Monitoring has the highest perceived risk.

59

Relative Benefit Across Applications

Pipeline*Traffic

Border Disaster

0

1

2

3

4

5

6

7

UA Applications

Pe

rce

ive

d B

en

efi

t (1

=L

ow

, 7

=H

igh

)


Autonomous UA

The more risky the public perceived the application, the less benefit they associated with the application.

60

Relative Perceived Risk to People on Ground vs. Air

Disaster*Border*Traffic*Pipeline*

0

1

2

3

4

5

6

7

UA Applications

Perc

eiv

ed

Ris

k (

1=

Lo

w,

7=

Hig

h)

Risk to people on Ground

Risk to people in the Air

There is no difference between risk perceived on ground vs. air. Also, there is no difference between perceived benefit between stakeholders and society.

61

Demographics and Risk

Age N Mean StDev

Under 25 56 4.340 1.541

36-35 28 4.679 1.786

36-45 13 5.192 1.665

46-55 14 4.964 1.216

56-65 21 5.310 1.512

Over 65 9 2.333 1.871

Those over the age of 65 perceived UAs as least risky and least beneficial; the mean value is insignificant.

62

Risk Perception Conclusions

Unmanned aircraft risk > manned Autonomous risk > Remotely piloted No difference:

Risk: Ground vs. Air Benefit: Stakeholders vs. Society

54% heard of UAS Need education programs 78% of those that heard of UAS obtained information

from television The more familiar, the more comfortable

Traffic Monitoring - higher risk Fear of operating around high population density

areas

63

Impact on Policy

Limit flight path/area

Limited population density

Implement education/outreach programs

64

Future Insights

Limitations: Time, Resources, and Budget

Sample National scale-different regions

Future Surveys: Compare UAS to other risky

technologies Size of aircraft Privacy concerns Economics concerns Lengthened

6565

Governance

Team Members:Nora DarcherNorma EspinosaScott FortuneAndrea Fuller

Team Manager:Leonardo Reyes-Gonzalez

66

Purpose

Evaluate current system of governance for UAS integration against principles of good governance

Suggest measures that could improve the governance process

67

Analysis

Principles of Good Governance Rules for FAA governance Historical Technologies Actor Interactions Roadblocks Cost and Benefits for each actor

68

Characteristics of Good Governance

http://www.unescap.org/pdd/prs/ProjectActivities/Ongoing/gg/governance.asp

69

Governance requirements on FAA

OMB rule requires FAA standards adoption procedures to have the following: Openness Balance of interest Due process Appeal process Consensus

70

Historical Analysis

How did the governance system handle the introduction of new technologies?

Supersonic Transport(1960s)

Public Interest NGOs can have a large impact

Automation of Radar System(1970s)

Incremental changes are easier than changing entire system at once

Microwave landing system (1970s)

Intl. adoption of US standard is advantageous to US technology firms

Technology What We Learned

71

Actor Analysis

Objective: Provide a systematic assessment of the

actors involved in integrating UAs into the NAS

Process: Identified key actors, examined their

goals and looked at problem from each actor’s perspective

72

Actor AnalysisResistant

Neutral

For

Hesitant

73

Roadblock Analysis

Objective: prioritize problems inhibiting the integration of UAs

Categories Technological Organizational Infrastructural Public Concern

74

Roadblock Analysis

75

Roadblock Analysis

0

1

2

3

4

5

6# of actors

Complexity

Low High

Airspace Access

Equivalent/Acceptable Level of Safety

Data AcquisitionTransponders

76

Most obvious needs

Defining an equivalent/acceptable level of safety

Allowing UAS operations in scarcely-used airspace to facilitate testing and development for civil and commercial applications.

Potentially large public concern about UAS safety argues for proactive public involvement in deliberations

77

Conclusions

78

Summary of Conclusions

Economics Risk, Technologies, and

Standards Public Awareness and Perception Governance

79

Economics

Some civil UAS applications seem highly competitive with alternatives

Initial policy ought to be tailored to the most commercially viable applications

Cost models show that (i) costs are most sensitive to hours of utilization, (ii) safety equipment has modest cost effect, except for small systems using sense and avoid

Foreign UAS firms may develop an advantage if they gain airspace first

80

Risk, Technologies, and Standards

For some applications in some classes of airspace, unmanned aircraft result in fewer fatalities than manned aircraft used for the same task

Sense and avoid is important only in airspace with significant traffic density

Low risk areas could be used for experimentation and testing without posing a high risk to those on the ground or in other aircraft

A mandatory incident reporting system has potential to greatly improve both airworthiness and human factors reliability

81

Public Awareness and Perception

All UAS applications surveyed were considered more risky and less beneficial than the manned alternative

Traffic monitoring perceived as most risky (likely due to flight over dense population)

About half of participants had heard of UAs

Those more familiar with UAS technology perceive less risk

82

Governance

Integration problem is more complex than many people realize

Incremental approach allows for policy experimentation at low risk (e.g., sparsely populated areas/airspace)

Standards need to be established to provide benchmark and incentive for manufacturing

Attention to public perception and involvement can greatly influence unfolding of UAS issue

83

Thank You for Coming!

Questions?

Project on Unmanned Aircraft in the NAS Final Review Panel Meeting

Documents

Transcript of Project on Unmanned Aircraft in the NAS Final Review Panel Meeting