Futron Corporation 2021 Cunningham Drive, Suite 303 Hampton, Virginia 23666 Phone 757-262-2074 Fax...

Futron Corporation • 2021 Cunningham Drive, Suite 303 • Hampton, Virginia 23666Phone 757-262-2074 • Fax 757-262-2173 • www.futron.com

Results You Can Trust

AssessingAssessingUncertaintyUncertainty

HRA – INCOSE November 8 & 9 2007

• 2 Results You Can Trust

OverviewOverview• Interactive Exercise• Definitions• Why Assess Uncertainty• Interactive Exercise• Types of Uncertainty• Sources of Uncertainty• Assessing (Bounding) Uncertainty• Interactive Exercise• Summary


Interactive ExerciseInteractive Exercise


Definitions: UncertaintyDefinitions: Uncertainty

• Uncertainty: may range from a falling short of certainty to an almost complete lack of conviction or knowledge especially about an outcome or result. Lack of clear definition


Definitions: RiskDefinitions: Risk

• Risk: possibility (chance) of loss or injury (Likelihood, Consequence. Ordered Pair )

• Probability: (1) : the ratio of the number of outcomes in an exhaustive set of equally likely outcomes that produce a given event to the total number of possible outcomes (2) : the chance that a given event will occur (3) A numerical measure of the likelihood of an event relative to a set of alternative events.

• Likelihood: probability

Consequence

54321

1

2

3

4

5

Likelihood

Consequence

54321

1

2

3

4

5

Likelihood

Generic Risk Matrix


Definitions: Confidence IntervalDefinitions: Confidence Interval

• Confidence Interval: a group of continuous or discrete adjacent values that is used to estimate a statistical parameter (as a mean or variance) and that tends to include the true value of the parameter a predetermined proportion of the time if the process of finding the group of values is repeated a number of times

• Used to quantify/communicate amount of certainty

• 90% confidence interval (limit/bounds)

mean


Why Assess Uncertainty?Why Assess Uncertainty?• We manage risks to ensure we meet our objectives

• We have tasks, projects, and programs to accomplish They all have objectives We all make decisions and commit resources to meet these objectives Decision are choices between alternatives and involve uncertainty This uncertainty leads to risk

• Managing uncertainty is a part of our job Involves understanding our decisions Managing the uncertainty in those decisions Identifying the risks to meeting our objectives that come from uncertainty Make better decisions / select better mitigation strategies


• Aleatory Randomness or variability that remains when the system is well known Inherent Variability You can calculate the odds Example: Drawing a queen of diamonds from a standard deck of 52

playing cards on the first draw. Not reducible

Types of Uncertainty: AleatoryTypes of Uncertainty: Aleatory


• Epistemic Stems from limitations of fundamental

knowledge about a phenomenon of interest. Model Uncertainty Can be revealed by divergences in the opinions

of experts who may disagree. Example: Drawing a queen of diamonds from a

deck of playing cards on the first draw when you have no idea how many cards are in the deck

This type of uncertainty is reducible:• Improvements of current knowledge• Improvements of corresponding models• Collection of additional data (test results)

Types of Uncertainty: EpistemicTypes of Uncertainty: Epistemic


Sources of UncertaintySources of Uncertainty

• Insufficient data Measurement errors State not directly observed (inferred) Imperfect understanding of processes Not enough data to be statistically significant

• Natural variability Varies spatially (with space) Varies temporally (with time)

• Model imperfections

• Underestimating uncertainty and the illusion of control - We tend to underestimate future uncertainty because we tend to believe we have more control over events than we really do. We believe we have control to minimize potential problems in our decisions.


Six levels of “sophistication”• Simply establish the existence of uncertainty

(risk)• Identifying a “worst-case” scenario

(unbiased bounds – one tailed or two tailed)• Computing the effects of the quasi-worst

case scenario or plausible upper bound• Computing a maximum-likelihood estimate

of the failure probability of losses• Modeling the probability distribution of

potential outcomes (risk curve) • Modeling explicitly the uncertainties about

the risk, separating epistemic from aleatory

Bounding UncertaintyBounding UncertaintyUniform(-0.17000, 0.20000)

X <= 0.1815095.0%

X <= -0.151505.0%

0

0.5

1

1.5

2

2.5

3

-0.2 -0.15 -0.1 -0.05 0 0.05 0.1 0.15 0.2 0.25

Triang(-.17, 0, .2)X <= 0.13917

95.0%X <= -0.11392

5.0%

0

1

2

3

4

5

6

-0.2 -0.15 -0.1 -0.05 0 0.05 0.1 0.15 0.2 0.25

Normal(0, .113) Trunc(-.4,+inf) Shift=+.015X <= 0.20088

95.0%X <= -0.17066

5.0%

0

0.5

1

1.5

2

2.5

3

3.5

4

-0.5 -0.4 -0.3 -0.2 -0.1 0 0.1 0.2 0.3 0.4


SummarySummary

• It is not easy• We think we know more than we really do• Understand, assess and communicate your own uncertainty• Understand how others assess and communicate their

uncertainty• Focus Mitigation Strategies


BibliographyBibliography

• Ang, Alfred H-S. and Tang, Wilson H. “Probability Concepts in Engineering Planning and Design, Volume I – Basic Principles”

• Ang, Alfred H-S. and Tang, Wilson H. “Probability Concepts in Engineering Planning and Design, Volume II – Decision, Risk and Reliability.” John Wiley, New York 1984

• Roberds, William J. (1990). “Methods for Developing Defensible Subjective Probability Assessments,” Transportation Research Record No. 1288, 183-190.

• CE311S Elementary Probability and Statistics for Civil Engineers class notes, University of Texas – Austin

• Futron Technical University (FTU) Fundamentals of Risk Management course notes

• Decision Analysis Training for ESMD Risk Management Officers, by Dr. Douglas Stanley, Georgia Institute of Technology, National Institute of Aerospace

• The Epistimic Uncertainty Project, Sandia National Laboratories http://www.sandia.gov/epistemic/ • Wikipedia (interactive exercises) http://en.wikipedia.org/wiki/Main_Page• Encyclopedia Britannica Online http://www.britannica.com/

http://www.sandia.gov/epistemic/

http://en.wikipedia.org/wiki/Main_Page

http://www.britannica.com/

Futron Corporation 2021 Cunningham Drive, Suite 303 Hampton, Virginia 23666 Phone 757-262-2074 Fax...

Documents

Transcript of Futron Corporation 2021 Cunningham Drive, Suite 303 Hampton, Virginia 23666 Phone 757-262-2074 Fax...