1

COVER LETTER

This is the final report for the ENMA 605 Capstone Course project. The purpose of this

project is to apply the appropriate concepts, techniques, and knowledge learned throughout

the course of study in order to analyze a complex a problem. The topic of this particular

research project deals with the differences of frequentist and Bayesian approaches to risk

analysis. This project also entails the research done during the fall semester of 2015 as a

graduate assistant for Dr. Unal. The goal of this research paper will be to help me better

understand the concept of risk and uncertainty, which was one topic that I was not an expert

on. So if my understanding of the concept is substantially higher by the end of the project, it

will be considered a success. Following the end of the program, I hope to be able to use what I

have learned not only during this semester for this project, but also what I have learned during

my time in the Master of Engineering Management program.

The information used in this project will mainly be gathered from online sources. The

information will then be analyzed in a way that the advantages and disadvantages of both

frequentist and Bayesian methods will be laid out. Unlike the thesis, the capstone was limited

in time as it was only for one semester. Also a conclusion will be determined depending on the

information gathered about these two methods. These two methods have other applications

other than for risk analysis, but for this particular project, its relationship to uncertainty and risk

will be the most important.

2

Analysis of Frequentist and Bayesian

Approaches to Risk Analysis

Old Dominion University

ENMA 605-Capstone

Karaoz, Can

December 4th, 2015

ckara005@odu.edu

3

EXECUTIVE SUMMARY

Uncertainty can be defined as the lack of knowledge on an outcome or a result. In order

to overcome uncertainty, in many aspects, it requires data. Also in order to overcome these

uncertainties, certain risks must be taken. There are certain things to take into consideration

though about uncertainty and risk. Certain risk can be determined without the use of data

while other risk factors can only be determined through the use of data acquired through

experiments and studies. The purpose of this project is going to be to determine the

advantages and disadvantages of the using the frequentist method or using the Bayesian

method. The goal is to use these methods of risk analysis and concepts from a systems analysis

to analyze how the two different methods stated affect uncertainty of certain systems and

projects in the engineering field of study. The method taken into particular consideration

under the frequentist method was a two dimensional Monte Carlo simulation, whereas, Bayes'

theorem was the basis for the Bayesian method of study. The objectives are simple, to gain a

better understanding on the topics of risk and uncertainty, to identify the advantages and

disadvantages of Bayesian statistics and Frequentist statistics, to analyze the relationship

between managing risk and the engineering field, and to ultimately determine implications on

which methods are better at predicting uncertainty.

Through extensive literary analysis done in a period of two-three months, much

information was found on each method of uncertainty/risk analysis. After careful

consideration, both methods had their own benefits and limitations which are all precisely

described in the body of this report. A quick explanation though provides enough evidence to

4

prove that both methods are independent of one another and that the frequentist method is

more practical but the Bayesian method is more probable to use. The purpose of this report is

to expand my knowledge on the topic of risk and uncertainty so that in the future if I encounter

either variable of study, I can provide the proper type of feedback.

5

TABLE OF CONTENTS

COVER LETTER ................................................................................................................................ 1

EXECUTIVE SUMMARY ................................................................................................................... 3

TABLE OF TABLES ............................................................................................................................ 7

TABLE OF FIGURES .......................................................................................................................... 8

BACKGROUND/INTRODUCTION .................................................................................................... 9

GENERAL FOCUS OF THE PROJECT .............................................................................................. 9

ORGANIZATION FOR THE PROJECT ............................................................................................. 9

IMPORTANCE OF THE ISSUE/PROBLEM RESOLUTION ................................................................ 9

PROJECT DEFINITION .................................................................................................................... 11

DEFINITION OF THE PROJECT PROBLEM/FOCUS ...................................................................... 11

PROJECT SIGNIFICANCE ............................................................................................................. 13

PROJECT APPROACH .................................................................................................................... 17

PROJECT DESIGN OVERVIEW ..................................................................................................... 17

SPECIFIC PROJECT DESIGN ......................................................................................................... 19

PROJECT MANAGMENT ............................................................................................................. 20

PROJECT DESIGN ISSUES............................................................................................................ 22

6

PROJECT RESULTS AND IMPLICATIONS ....................................................................................... 22

INTERPRETATION OF DATA ....................................................................................................... 22

DISCUSSION OF PROJECT DELIVERABLES .................................................................................. 28

RECOMMENDATIONS/PROJECT RESULTS ................................................................................. 29

REFERENCES .................................................................................................................................. 31

STUDENT BIOGRAPHICAL DATA ................................................................................................... 32

7

TABLE OF TABLES

Table 1-Element of the Analytic Strategy and Description of Each Element .............................................. 17

Table 2-Advantages of the Bayesian Approach(Ferson) ............................................................................. 26

Table 3-Disadvantages of the Bayesian Approach(StasticalAnalysisSystem9.2, 2009, Ferson) ................. 27

Table 4-Advantages of Frequentist Approach(Ferson) ............................................................................... 27

Table 5-Disadvantages of the Frequentist Approach(Ferson) .................................................................... 28

8

TABLE OF FIGURES

Figure 1-Pressures on a Program Manager’s Decision Space (Garvey, 2015) ............................................ 14

Figure 2-General Risk Management (Garvey, 2015a) ................................................................................. 15

Figure 3-WBS/Gantt Chart .......................................................................................................................... 21

Figure 4-Network Diagram .......................................................................................................................... 21

Figure 5-Bayes' Rule Representation (Garvey, 2015b) ............................................................................... 23

Figure 6-Frequentist Probability Equation .................................................................................................. 24

9

BACKGROUND/INTRODUCTION

GENERAL FOCUS OF THE PROJECT

As said in the executive summary, uncertainty is known the lack of knowledge on an

outcome or a result but in reality it is actually more than that. According to Funtowicz and

Ravets, uncertainty can be classified as a "situation of inadequate information" which can fall

under three categories: inexactness, unreliability, and border with ignorance (Walker et al.,

2003) . Also they state that new information can also cause uncertainty to either decrease or

increase depending on the amount of information available. This also draws on systems

principles as well. For example the system darkness principle, not everything can be known

about a system. This can be applied to uncertainty as well. Since what is known is part of the

system and everything outside the system hasn't been learned yet, the more knowledge that is

known about a complex processes, the possibility arises that previously known uncertainties

may reveal themselves. Therefore, the more knowledge that is present can conclude that

either understanding of the processes are either limited or more complex than before(Walker

et al., 2003). The main focus of this study will be to analyze different types of risks and

uncertainties in the engineering field of study and to compare the types of risks and

uncertainties with respect to the systems they are associated with.

ORGANIZATION FOR THE PROJECT

With respect to this study, there is not a traditional sense of organizations, or one

particular company. The purpose of this study is to be as thorough as can be within the limited

10

time to complete this project. So another way to look at this is to analyzing the different

methods of risk analysis. When making decisions based on judgment, it mainly depends on

certain approaches taken, whether they be the classical, statistical approach or the combined

classical and Bayesian approach. These methods focus specifically on establishing estimates of

statistical quantities, such as probabilities and failure rates (Apeland et al., 2002). If a group or

organization was to be named for the usefulness or risk analysis and uncertainty estimation,

then, hypothetically, anything could be named. Risk and uncertainty are problems that all

companies deal with in decision making situations. If risk and uncertainty aren't taken into

consideration, it can lead to reprehensible consequences.

IMPORTANCE OF THE ISSUE/PROBLEM RESOLUTION

The importance of understanding risk and uncertainty are a significant part of risk

analysis. Especially when taking into consideration the analysis of data. When using data in

probabilistic risk analysis, failure rates are also very important. The failure rates must be taken

into consideration, otherwise uncertainties can end up being underestimated (Apostolakis,

1982). Analyzing data can also lead to the making a decision between using Bayesian statistics

or frequentist statistics. Frequentist statistics is very appealing because it provides a sense of

objectivity but when "statistically significant" data is available, it fails to provide results when

judgment is just as important as the statistical evidence(Apostolakis, 1982). Another thing to

take into consideration is looking at the difference between probability and frequency. It will

give a better understanding of data analysis when analyzing risk and uncertainties. A

frequency, technically, is a measurable number such as a failure rate, whereas probabilities

11

measure the degrees of belief of whether or not an event is true or false, and they are not

measureable (Apostolakis, 1982). So the importance of knowing the different between these

two types of statistical values can help with the interpretation of risk and uncertainty.

PROJECT DEFINITION

DEFINITION OF THE PROJECT PROBLEM

PURPOSE:

So the purpose behind this project was essentially to better my knowledge on the

concept of risk and uncertainty as that was one of my weak points during my time in the

Engineering Management program here at ODU. Understanding risk is actually an important

concept. My goal in the future is to work on prosthetic devices, including artificial organs and

body parts. There is a certain level of risk associated with these types of devices, not

necessarily with prosthetic devices but artificial internal organs carry many risks associated with

them. Many things need to be taken into consideration before they can be used on humans

(materials, size, compatibility, etc...). So understanding, statistically, what risk is then it can be

prevented. Also, in decision making, risk can determine how engineering systems are

produced, developed, and sustained. In a systems engineering perspective, risk management

can be used to identify, analyze, and adjucate events, so that if they do occur unwanted

impacts could be minimized and the system can then complete its main objective.

12

OBJECTIVES:

1. Achieve better knowledge on the topics of risk and uncertainty

2. Identify the benefits and disadvantages of Bayesian statistics and Frequentist

statistics.

3. Analyzing the relationship between risk management and engineering.

4. Detailing the risk management process and application of risk management

5. Ultimately come to a conclusion on what methods are better at predicting

uncertainty

PROJECT SCOPE:

As stated before the purpose of this study is to determine the advantages and

disadvantages on the uses of data-based risks and non-data-based risks in order to reduce or

prevent uncertainty. This is important because uncertainty is an important aspect in project

management when it comes to making decisions. Of course risk cannot be completely

eliminated and has effect on uncertainty but uncertainty can be reduced so that a better

judgment can be made when it comes to decision making. So the focus of this study will be to

interpret the relationship between these two important variables in decision making and to

compare and conclude, in certain engineering systems, if the relationship can provide better

solutions to the problems associated with those systems. Some limitations associated with this

study include the possibility of skewed or outdated data, time constraints, and also limitation of

readable resources. These limitations might affect but not completely ruin the outcome of the

study. Since there is only approximately three months to conduct the study, time might play a

13

key role in the accuracy of the results. Also the ability to find resources is time restrictive

because not all publications are available right away. Also the possibility of finding skewed or

outdated data is a real possibility and should be taken into consideration. Adding to what was

said before; also information learned during the Risk Analysis class can be used as well. Even

within the time restriction of one semester, an extensive literature review was performed as

part of this project. While not much numerical data was collected, in analysis of plausible

solutions to the objectives, many equations and diagrams have been found to support the

objectives. Since one full semester isn't nearly enough time to complete a whole complex study

compared to someone who would be working on a thesis, the information gathered is still a

worthy amount to complete an informational study. Since the main objective of this project

was to analyze data based and non-data based risk, particularly choosing one specific topic

wouldn't have made the study accurate. Risk needs to be looked at in a general way so that

understanding problems associated with risk can be better understood.

PROJECT SIGNIFICANCE

LOCAL LEVEL IMPACT:

In order to make an impact on the local level while managing risk, engineering systems

need continuous attention. Managing this risk is designed in a way that the system that is

being taken into consideration has the chance to be completed on time, is very cost effective,

and where it also meets safety and performance standards. The importance of this project is

essentially to help in this process. So at a local level, ultimately the goal should be to determine

what the risk is, and then finding ways to determine how to mitigate that risk. Since systems

14

nowadays are more complex, they behave more unpredictably, thus, by looking at the diagram

below it can be seen that managing risk is technically managing the "contention" that exist

among the three dimensions: Performance, Cost, and Schedule:

Figure 1-Pressures on a Program Manager’s Decision Space (Garvey, 2015)

So in terms of risk management at the local level, if risk isn't mitigated this could lead to

problems at the local level. Let's put this in retrospect: An example of how engineering and risk

management affect each other could mean taking into consideration the possible loss of life as

a consequence of not taking risk into consideration. One example of this that actually ended in

tragedy happened in September of 2013. A residential building in the city of Mumbai, in India,

collapsed. Many reasons were cited such as the building being too old, not being built with

correct material, etc..., but the reason that stuck out the most and has relevance to a local level

impact to this study is the fact that an extra floor was built on top of the preexisting

building(Gardiner and Bagri, 2013). This ultimately caused the building to collapse killing 61

15

people. This is a situation where the risks weren't probably taken into consideration and

analyzed. The buildings are already poorly built and then the decision to build a mezzanine

floor on top of the building was a huge mistake which led to catastrophe. So this shows the

importance of analyzing risk.

APPLICATION OF ENGINEERING MANAGEMENT KNOWLEDGE

In order to successfully complete my research and achieve my objectives, a complex

knowledge of different engineering management principles are needed. First of all the basis for

risk management can be seen through the figure below:

Figure 2-General Risk Management (Garvey, 2015a)

1. Risk

Identification

Risk events and their

relationships are defined

2. Risk

Impact

Assessment

Probabilities and

consequences of risk

events are assessed

Consequences may include cost,

schedule, technical performance

impacts, as well as capability or

functionality impacts

3. Risk

Prioritization

Analysis

Decision-analytic rules applied to

rank-order identified risk events

from “most-to-least” critical

Risk

Tracking

4. Risk Mitigation

Planning,

Implementation,

and Progress

Monitoring Risk events assessed as medium or high criticality might go into risk

mitigation planning and implementation; low critical risks might be

tracked/monitored on a watch-list

Reassess existing risk

events and identify new

risk events

Identify

Risks

Assess

Probability &

Consequence

Assess Risk

CriticalityWatch-listed

Risks

Risk Mitigation

16

The figure shows the basic risk management process starting with the identification of the risk

as the first part of the process. The next step is to assess the impact of the risk to determine

the probability and consequences of risk. Then the third step entails prioritization of the risks in

the order of severity. The next step has two directions: risk tracking and risk mitigation. The

risk tracking step is only utilized for risks that are classified as low in the prioritization step,

whereas risk mitigation deals with risk events that are considered to be medium or high during

the prioritization step. Then the process starts all over again by reassessing current risks and

also determining possible new ones. This isn't the only principle of engineering management

that is used when analyzing risk. Understanding statistics is an important aspect of risk

management and mitigation. As stated before, one of the objectives was to identify the

benefits and disadvantages of Bayesian statistics and Frequentist statistics. Also project

management skills were necessary during the planning process of the project.

POTENTIAL EXTENSION OF PROJECT APPROACH OR FINDINGS BEYOND THE LOCAL

APPLICATION:

This project has the possibility to extend beyond the local application. No real testing

was done, more or less; it was a literary analysis research paper. The next step in this process

is to actual use real-time data to perform a real risk analysis. The analysis will actually involve

models, calculations, and simulations. Of course, there will also be more time to achieve this in

the future, as there was a time constraint of one semester. Using the information attained at

the end of the study, I can comfortably say that I can perform a risk analysis in the future.

17

PROJECT APPROACH

PROJECT DESIGN OVERVIEW

A full system analysis wasn't the best option for this particular project even though I

took certain concepts from systems engineering. The reason for this is because there isn't a set

type of risk that is trying to be eliminated; a particular engineering system isn't trying to be

fixed here. The main systems concept though that was used for this particular concept is

detailed in the table below:

Table 1-Element of the Analytic Strategy and Description of Each Element

Element of the Analytic Strategy Description and Components of Each Element

Strategy Formulation The objectives of the study must be laid out

Relationship from problem to the purpose of the study must always be determined

The assumptions for data collection and analysis must be stated up front

Data The data set must be good and should be linked to the analysis

There are many collection requirements o There must be a collection plan for the data.

(Data should not be collected just for the sake of collecting data)

o The method of collection must also be stated(experiments or contextual data that has been researched)

The relationship between the data and the problem should also be determined at the beginning

Same can be said between the data and the objectives of system analysis.

Analysis of Data This is where the different methods and techniques for the treatment of data are put on the table

o First the source of the data is determined or referenced

o Also assumptions and limitations are determined or calculated

18

o The acceptability of the technique is also important, some of the stakeholders might only like to use specific methods of data analysis

The outputs and outcomes from the analytic strategy need to be accounted for.

o This is where the expected products of the analysis are put on the table.

o Also the relationship between the system problem, the objectives of the study, and of course, ultimately, the solution from the data

Interpretation of Data Interpretation is all about taking the meaning out of the quantitative and qualitative data results

Alternative sets of that data can actually help with rating the solutions in order to determine the best one.

Determining the meaning of the data by linking the study objectives and system problem is also very important. Data can help make critical decisions.

Every system study has a context, and a question to consider is to what degree will the system analysis be consistent with that context?

This is essentially the exact strategy that was used for this project. The strategy was to perform

literary analysis gathering information relevant to the objectives of the study. The next step

was to collect data, whether it be quantitative or qualitative, and then to analyze it. The final

step was then to interpret the data to see whether or not the objectives were completed or

not.

19

SPECIFIC PROJECT DESIGN

DATA COLLECTION:

Since the project topic was such a broad topic, data collection involved was through literature

review. As stated before the objectives weren't really based upon data but, more or less,

measured upon the understanding of the concepts. Through careful literature review and

analysis of previously written studies, two of the main objectives can be completed:

determining what statistical methods are better at predicting uncertainty and identifying the

benefits and disadvantages of Bayesian statistics and Frequentist statistics.

PLAN FOR DATA ANALYSIS:

For analysis of the data, involved a basic compare and contrasting system was used. The

data analysis involved analyzing different information regarding uncertainty and using risk to

determine the uncertainty. By looking at different literary pieces and notes from previous

classes, the analysis was performed solely based upon the differences and similarities

presented within the literature. By looking at the two different types of statistics, Frequentist

and Bayesian, much of the information gathered was then synthesized using the most

important data from the literature. Then using the information gathered here, since risk and

uncertainty are so connected to one another, the Bayesian approach and the Frequentist

approach can then be analyzed. This is one of the bases of the objectives which are to

determine the better ways of predicting uncertainty and analyzing risk. Then after all of the

20

analysis, the only thing left to do will be to summarize the findings and come to a conclusion on

which method is better.

RESULTS OF DATA COLLECTION

The information analysis will be considered a success if the two methods of statistics in

risk analysis are clearly defined each with their advantages and disadvantages and if a

conclusion reached on which method is better in predicting uncertainty. Ultimately, the

success of this project will be defined on a personal level. It will be measured on how much my

knowledge of the subject has been extended.

PROJECT MANAGEMENT

The first step associated with this study is the literary analysis. After substantial

research has been done and a detailed understanding of risk and uncertainty has been

obtained, the next step in the process was to map out the approach that was taken to make

this study a success. A work breakdown structure and a network diagram were also created in

order to come up with a clear methodology and timeline on how to proceed. Once the WBS

and the PERT diagrams were been created, the next step was to begin the data collection

process. This was mainly done through an advanced literary search. The next step in the

process was to come up with an analytic strategy to determine the best possible way to

proceed with the data. Using our knowledge of systems and systems analysis, some of the

methodologies learned during our time in the program were presented to help in our analysis.

The milestones that were followed in order to make this project a success is as follows:

21

1. September 15th: Project Proposal Due

2. September 22nd: Completed WBS/PERT diagram

3. October 1st: Extensive Literary Search is complete, begin data collection and analysis

4. November 1st: Analysis should be complete by now, begin write up of the research

paper.

5. December 1st: Paper should be complete

6. December 4th: Project Report is turned in

7. December 10th: Oral Presentation.

8. December 11th: Program Evaluation must be completed

Below both a Gantt chart with the work breakdown can be seen:

Also a network diagram can be visible below as well:

8/24/15 9/15/15

Dur: 17 days Slack: 0 days

8/24/15 9/15/15

9/16/15 9/22/15

Dur: 5 days Slack: 0 days

9/16/15 9/22/15

9/23/15 10/1/15

Dur: 7 days Slack: 0 days

9/23/15 10/1/15

10/2/15 10/30/15

Dur: 21 days Slack: 0 days

10/2/15 10/30/15

11/2/15 12/1/15

Dur: 22 days Slack: 0 days

11/2/15 12/1/15

12/7/15 12/10/15

Dur: 4 days Slack: 1 day

12/8/15 12/11/15

12/7/15 12/11/15

Dur: 5 days Slack: 0 days

12/7/15 12/11/15

Project Proposal DueCreate WBS

diagram/PERT DiagramLiterary search Data Analysis Research Paper Write Up Oral Presentation

Program Evaluation

Figure 3-WBS/Gantt Chart

Figure 4-Network Diagram

22

PROJECT DESIGN ISSUES

The primary design issue was that not much primary data was not used. This project

was more of a study aimed at determining the benefits of methods of risk analysis and methods

of determining uncertainty. So rather than performing calculations, facts and different opinions

of statisticians were taken into consideration throughout the literature and then analyzed to

come up with a personal conclusion on which method is more sufficient in risk and uncertainty

analysis.

PROJECT RESULTS AND IMPLICATIONS

INTERPRETATION OF DATA

Ok so in the analysis, as said before the differences between Bayesian approach and the

Frequentist are the main things being taken into consideration here. So the first thing to

analyze was Bayes' Rule, which evidently, is one of the main concepts to the Bayesian

approach. So the concept behind Bayes' rule is pretty simple. There are two probabilities,

probability A and probability B, each independent from one another. Bayes' rule is used as a

conversion of the probability of B given A has occurred to the probability of A given B is

occurred (Ferson). It ultimately is used to find relationships between probabilities. The

following equation below shows the representation of Bayes' rule:

23

Figure 5-Bayes' Rule Representation (Garvey, 2015b)

Ok now to go deeper into Bayesian statistics and how it relates to risk. So there are essentially three

ways that Bayesian statistics can be used in risk analysis: to take over the assessment and decision

process, it can be used to estimate risk distributions, or it can be use to select or parameterize input

distributions (Ferson). So the within the first way, Bayesians like to use this method to assess and make

decisions rather than use a formal infrastructure because of the unpredictability that risk is associated

with. Being in charge of the decision making and not making decisions solely based on a uniform system

are key to making right decisions in the engineering world. Using the Bayesian method to estimate risk

distributions instead makes distributions and quantities, more or less, a crucial part of the Bayesian

analysis, whereas, the process of decision making instead goes outside the system boundary of the

Bayesian analysis. After the first two, the last possibility involve using the method as a tool for

parameterizing input distributions, or in other words, it makes the analyst have more of a support role

because the risk models and the decision process are completely out of the jurisdiction of the Bayesian

method (Ferson). Now that there is a basic understanding of the main concepts of the Bayesian

approach to risk analysis, next was to analyze the Frequentist approach to risk analysis.

24

One of the main components of the Frequentist is using historical data in order to perform a risk

analysis. This tends to be the preferred approach if such data is available. The Bayesian approach is

generally used in situations where they need an expert's opinion, but the downside to that is that

experts usually have a hard time agreeing with one another. The equation that is generally used in

probabilistic risk assessment for Frequentists can be seen below and states that "the probability of

event A is the proportion of times that A occurs in an infinite sequence of separate tries

(DukeUniversity).

𝑷 𝑨 = 𝐥𝐢𝐦𝒏→∞

# 𝒐𝒇 𝒕𝒊𝒎𝒆𝒔 𝑨 𝒉𝒂𝒑𝒑𝒆𝒏𝒔

𝒏

Figure 6-Frequentist Probability Equation

One reason that Frequentist probabilistic risk analysis is widely preferred compared to Bayesian

probabilitistic analysis is the fact that Frequentist probabilities are easy to justify and are backed up by

some type of historical data, whereas, Bayesian probabilities matter strongly dependent on the

judgment of experts. This dependency on judgment can be a problem because most of the time

judgment can contain bias. If there is some sort of data to use, the Bayesian probabilities can then be

easily computed using Bayes' Theorem in Figure 5 (DukeUniversity). One of the main components to

risk analysis in the Frequentist method involves a two-dimensional Monte Carlo simulation. First, what

is a Monte Carlo simulation? It is a type of method or technique using simulation software that helps

the analyst understand the impacts of risk and uncertainty particularly in financial, project management,

cost, and other important forecasting models. It, essentially, can tell you how likely the resulting

outcomes are going to be, and this can be very useful when trying to make important decisions . This is

one of the main reasons that many experts and analysts prefer the Frequentist method over the

Bayesian method. The Monte Carlo Simulation process involves the obtaining of estimates for the

25

solutions of certain problems through the use of random numbers (Zio, 2013). The method entailed in

Scott Ferson's study involves a two dimensional version of the standard Monte Carlo simulation. It

involves the nesting of one Monte Carlo simulation within another specifically to determine how

variability and uncertainty interact with one another to create risk (Ferson).

Some of the concepts used in Monte Carlo simulations can be used in both Bayesian and

frequentist analyses. The simulation itself, though, is not necessarily used in the Bayesian approach.

The purpose of the two-dimensional Monte Carlo simulation is particularly to distinguish between two

types of uncertainties. One of the objectives stated earlier was to come to a conclusion on what

methods are better at predicting uncertainty. The two dimensional Monte Carlo simulation

helps distinguish between two types of uncertainty: variability and incertitude. So what is

variability and incertitude? Variability refers to the "stochastic fluctuations in a quantity

through time, variation across space, manufacturing difference among components, genetic

phenotypic differences among individuals or similar heterogeneity within some population,"

whereas, incertitude is "the lack of knowledge about a quantity that arises from imperfect

measurements, limited sampling effort, or incomplete scientific understanding about the

underlying processes that govern a quantity" (Dienstfrey and Boisvert). In terms of engineering

these two variables are also known as "aleatory uncertainty" (variability) and "epistemic

uncertainty" (incertitude). The reason behind the wording for these are pretty simple actually,

aleatory details the uncertainty that is associated in certain games, as the word comes from the

word alea (Latin for dice) and epistemic emphasizes the scarcity of knowledge (Ferson). Now

that there is a basic understanding to both methods, the next step is to lay out the advantages

and disadvantages of each specific method.

26

As said before, the debate on the Bayesian method and the frequentist method is still going on

for which one is more useful or which one is more viable. Some of the advantages of the Bayesian

method are the approach's naturalness, its ability to data mine, its advantage at decision making, its

rationality, its explicit use of subjective information, and its ability to work without data. Some

advantages of the frequentist approach, particularly with the two dimensional Monte Carlo method, is

that the method incorporates uncertainty into its mathematical computation of the risks. The outputs

provided by this particular model can be advantageous in directing future data gathering by identifying

variables with high incertitude (Ferson). Below the advantages of the Bayesian analysis can be seen

with a brief explanation of each advantage:

Advantage of the Bayesian Approach

Brief Explanation

Naturalness Bayesians can compute "credibility intervals" which they feel are more natural and easier to work with.

Also it allows the use of probability distributions for both data and parameters within the models.

Data Mining Compared to the frequentist methods, in the Bayesian approach looking at the data before forming a hypothesis is completely ok, whereas, in the frequentist method it is highly frowned upon(Hypothesis should be formulated before looking at data)

Decision Making Since the Bayesian method allows for judgment to help in decision making, analysts and decision makers can construct a set of decisions about the risk assessments.

In hypothesis testing, the frequentist approach only allows a tester to rejecting the null hypothesis.

Rationality It states that different people will have different perspectives and will be more likely to draw different conclusions when data is sparse.

Subjective Information The Bayesian approach allows the use of personal judgments made by experts and analysts, in which the risk analyst has the option of accepting.

Not everything is about the data. The use of the knowledge of the experts can bring something to the table when analyzing risk.

Working without data Now this is one of the more important advantages and ties the objective of this project to the information seen in the literature.

Bayesian methods can produce answers even when there is no sample data available.

Essentially it is stated that the trick is to use the probability distribution that represents uncertainty before sampled data is taken instead of the probability distribution representing uncertainty after data is sampled

Table 2-Advantages of the Bayesian Approach(Ferson)

27

This table points out some of the limitations and disadvantages associated with the Bayesian approach

to risk analysis:

Disadvantage of the Bayesian Approach

Brief Explanation

Prior Selection The Bayesian method doesn't tell you how to select a prior (a probability distribution that represents uncertainty before sampled data is taken).

It could lead to misleading results

Posterior Influence The posterior (probability distribution representing uncertainty after data is sample) distributions can be heavily influenced by the priors.

Could cause problems when trying to convince experts of the findings

Computational Cost The Bayesian method requires lots of models and large number of parameters. Since so much computation is needed, with the use of random numbers, this can cause skewing of the results.

Table 3-Disadvantages of the Bayesian Approach(StasticalAnalysisSystem9.2, 2009, Ferson)

This table points out some of the advantages associated with the Frequentist approach:

Advantage of the Frequentist Approach

Brief Explanation

Objective The method is very objective as it is more data based and not based on opinions of experts and analysts. Some analysts might prefer that.

It allows analysts to make fewer assumptions and be able to be forthright with what they know and what they don't know.

Uncertainty It incorporates more uncertainty into the mathematical calculations of risk.

The outputs also help in directing future data gathering solely for the purpose of identifying variables which have a high level of incertitude

Table 4-Advantages of Frequentist Approach(Ferson)

28

This table points out the disadvantages associated with the Frequentist approach:

Disadvantage of the Frequentist approach

Brief Explanation

Computational Cost The computation complexity involved is associated with having two Monte Carlo simulations that are nested. This isn't too much of a problem anymore though because the statistical software is capable of computing complex simulations in a matter of hours.

Back calculations Back calculations are often difficult and very time consuming due to the trial and error process associated with it but they are necessary in the end.

Ugly Outputs The analyses of metadistributions tend to be often complicated and very confusing even to experts and analysts. Analysts often replace the metadistributions with three-curve displays. This causes the loss of information though making the results less accurate

Incertitude Frequentist often use the two dimensional Monte Carlo to predict uncertainty but it lacks the ability to model incertitude correctly.

Table 5-Disadvantages of the Frequentist Approach(Ferson)

DISCUSSION OF PROJECT DELIVERABLES

The purpose of this report involved five separate objectives: Achieving a better

knowledge on the topics of risk and uncertainty, identifying the advantages and disadvantages

of Bayesian statistics and frequentist (two dimensional Monte Carlo analysis) statistics,

Analyzing the relationship between risk management and engineering, detailing the risk

management process and application of risk management, ultimately come to a conclusion on

what methods are better at predicting uncertainty. These five associated topics were each

covered in detail in different areas of the report. As seen above the advantages and

disadvantages pertaining to certain areas of the two approaches were detailed. The conclusion

that I came up with is that if an analyst has the possibility to run both types analyses then it

would be very useful. The frequentist method seems very practical, but the Bayesian method

29

seems more probable to use. The information gathered in this report, I plan to use in the

future again. If I have to make a decision dealing with risk, both methods can help.

RECOMMENDATIONS/PROJECT RESULTS

LOCAL LEVEL IMPLICATIONS/RECOMMENDATIONS

The local level implications and recommendations generated by this project involve

actually testing out each method thoroughly. Gathering specific data and actually performing

the analysis to calculate risk and uncertainty associated with a systems engineering problem.

These particular methods have the possibility of becoming optimized in the future or even

brand new methods might be created. Now knowing the advantages and disadvantages of

each of the methods, it is easier to expect the unexpected. The main goal though at the end of

the project is to utilize what was learned and be able to apply it to the real world and real world

problems.

LOCAL LEVEL ISSUES IDENTIFIED AS A RESULT OF THE PROJECT

The issues associated as a result of the project involve the difficulty finding a particular

set of data in which both methods could be utilized to perform a thorough risk analysis. This

paper was mainly informational based rather than an experimental project. Also as stated

before, even though the simulation programs have come along so far, they still could take a

long to run. Also one full semester isn't enough to complete a full complex simulation; more

time would be needed to test both methods.

30

PROJECT IMPLICATIONS/ISSUES BEYOND THE LOCAL LEVEL

These techniques are utilized for many things, not just for risk analysis but also for

other statistical problems as well. In the future, if I decide to further my study, it might be a

problem since I won't have access to the information of databases provided to me by the

school. That might cause a hindrance in the future. If risk data is ever collected in the future,

using this study I can decide on what to use in order to calculate my uncertainty and determine

if I can reduce risk or not.

31

REFERENCES

APELAND, S., AVEN, T. & NILSEN, T. 2002. Quantifying Uncertainty Under a Predictive, Epistemic

Approach to Risk Analysi. Reliability Engineering and System Safety, Vol. 75.

APOSTOLAKIS, G. 1982. Data Analysis in Risk Assessments. Nuclear Engineering and Design, Vol. 71, 375-

381.

DIENSTFREY, A. M. & BOISVERT, R. F. Uncertainty Quantification in Scientifici Computing. Boulder, CO,

USA.

DUKEUNIVERSITY Lecture 24. Risk Analysis.

FERSON, S. Bayesian methods in risk assessment

GARDINER, H. & BAGRI, N. T. 2013. Scores Feared Trapped in Collapse of Mumbai Building [Online].

Available: http://www.nytimes.com/2013/09/28/world/asia/scores-feared-trapped-in-collapse-

of-mumbai-building.html?_r=0.

GARVEY, P. 2015a. Chapter 2 Lecture-Risk and Decision Theory in Engineering Management.

GARVEY, P. 2015b. Chapter 3-Foundations of Risk and Decision Theory.

RISKAMP.COM. What is Monte Carlo Simulation [Online]. Available:

https://www.riskamp.com/files/RiskAMP%20-%20Monte%20Carlo%20Simulation.pdf.

STASTICALANALYSISSYSTEM9.2. 2009. Overview of Bayesian Analysis [Online]. Available:

https://www.cpp.edu/~djmoriarty/wed/bayes_handout.pdf.

WALKER, W. E., HARREMOES, P., ROTMANS, J., SLUUS, J. P. V. D., ASSELT, M. B. A. V., JANSSEN, P. &

KRAUS, M. P. K. V. 2003. Defining Uncertainty- A Conceptual Basis for Uncertainty

Managementin Model-Based Decision Support. Vol. 4, pp. 5- 17.

ZIO, E. 2013. The Monte Carlo Simulation Method for System Reliability and Risk Analysis.

32

STUDENT BIOGRAPHICAL DATA

I was born in Virginia Beach, VA, and have lived in the area for my whole life. My

mother and father are both of Turkish decent and have lived in the United States for a very long

time. My father is retired from the printing press business and my mother is currently a

manager at a bridal gallery. My father has been in the United States since 1973 and even

completed high school and university in the United States. I am considered to be the first

generation in my family to be born in the United States and I am very grateful to my parents

who gave me the opportunity to live in this wonderful country.

I attended Lands town High School here in Virginia Beach, which is a school that has a

pre-engineering program, which is what made me want to enter the engineering field of study.

After graduating in 2010, I decided to attend Old Dominion University and enrolled in the

Mechanical Engineering department. I am proud to say that I finished the program in exactly

four years. Without any time to waste, once I finished my Bachelors degree, I decided to

further my educational career and enrolled in the Engineering Management program. I am on

track to graduate this fall of 2015. My goal, after I graduate, is to find a career in the

biomedical engineering field as I am very interested in prosthetic devices.