Risk assessment

Tor Stålhane

NTNU / IDI

What is risk - 1

Risks are characterized by three factors:

• They are concerned with events that may – or may not – happen in the future.

• The events are identifiable but their effect and probability are uncertain.

• The outcome of the events can be influenced by our actions

What is risk - 2

A risk is something that can be a problem in the future. It is defined by two parameters

• The probability - p. What is the probability that the risk will become a problem?

• The consequences - C. What will happen if the risk becomes a problem?

The risk – R – is defined as R = C*p

How large is the risk - 1

In order to find the size of a risk, we need values for p and C.

In some cases we can estimate these values from historical data but in most cases we will have to use expert opinions or other subjective data sources.

It is not always possible – or meaningful – to assign a numerical value to a consequence, e.g. loss of lives.

How large is the risk - 2

Even though assessment is a subjective activity it is not about throwing out any number that you like.

To be useful, an assessment must be• Based on relevant experience.• Anchored in real world data, e.g. “How bad can it

get?”• The result of a documented and agreed-upon

process. Having a process makes it possible to later improve the process based on experiences.

Assessing risk

The quality of an assessment increases when the background info gets more specific.

Don’t ask: “What is the consequence of X?” or “What is the probability of Y?”

It is better to ask: “What is the consequence of X in scenario S?” or “What is the probability of Y in scenario S?”

Assessment and scenarios - 1

If the probability of scenario Si is p(Si), and pi and Ci are the probability and consequence of an accident in scenario Si, we have that:

iii CpSpR )(

The method is critically dependent on the • Quality of the scenario descriptions• Independence of the scenarios

Assessment and scenarios - 2

We can improve our assessments even more if we do not ask for consequences in general but for consequences for one particular asset. Thus, in scenario i we have consequence Cj,i for asset j.

jijii CpSpR ,)(

Assessing C and p

We can assess consequences and probabilities in several ways:

• Textual categories – e.g. High, Medium, Low.• Numerical categories – e.g. values from 1 to 10. • Value intervals.• Statistical distributions.

Textual categories – 1

When using categories, it is important to give a short description as to what each category implies. E.g. it is not enough to say “High consequences”. We must relate it to something already known, e.g.

• Project size

• Company turn-over

• Company profit

Textual categories – 2

Two simple examples:

• Consequences: we will use the category “High” if the consequence will gravely endanger the profitability of the project.

• Probability: we will use the category “Low” if the event can occur but only in extreme cases.

The CORAS consequence table

Consequence values

Category Insignificant Minor Moderate Major Catastrophic

Measuredrelated toincome

0.0 – 0.1% 0.1 – 1.0% 1 – 5% 5 – 10% 10 – 100%

Measuredloss due toimpact onbusiness

No impact onbusiness. Minor delays

Lost profits

Reduce theresources of oneor moredepartmentsLoss of a coupleof customers

Close downdepartments orbusinesssectors

Out ofbusiness

The CORAS frequency tableFrequency values

Category Rare Unlikely Possible LikelyAlmostcertain

Number ofUnwantedincidents perYear

1/100 1/100 – 1/50 1/50 - 1 1 - 12 > 12

Number ofUnwantedincidents perDemand

1/1000 (1/500) 1/50 (1/25) 1/1

Interpretationof number ofdemands

UnwantedincidentneverOccurs

Eachthousandtime thesystem isused

Each fivetimes thesystem isused

Each tenthtime thesystem isused

Everysecondtime thesystem isused

Consequence and probability - 1

Consequence

Probability H M L

H H H M

M H M L

L M L L

Consequence and probability - 2

The multiplication table is used to rank risks. It can not tell us how large they are.

We should only use resources on risk that are above a certain, predefined level.

Numerical categories -1

We can use numbers instead of names. This does not make the assessment more precise but will free us from the need to define a multiplication table in order to identify risks.

In principle we can use any numbers. The best solution is, however, to just assign number to the three aforementioned categories

Numerical categories – 2

The following values are often used in practice, both for consequences, benefits and probabilities:

• 10 – high

• 4 – medium

• 1 – low

Thus, a medium consequence and a low probability will give a risk of 4*1 = 4.

Numerical categories – 3

Consequence

Probability H / 10 M / 3 L / 1

H / 10 H / 100 H / 30 M / 10

M / 3 H / 30 M / 9 L / 3

L / 1 M / 10 L / 3 L / 1

Value intervals

If we have more info available we can give better estimates. Even though we cannot give exact values, we can give our assessments as intervals.

An interval has a start and an end value – denoted a and b. We denote the interval I as I = [a, b]

In our case, the width of the interval is a measure of our uncertainty.

Simple interval arithmetic

As long as all interval limits are positive, we can write:

• I = I1 * I2, I = [a1*a2, b1*b2] • I = I1 + I2, I = [a1 + a2, b1 + b2] • I = I1 - I2, I = [a1 - a2, b1 - b2] • I = I1 / I2, I = [a1*b2, b1/a2]

If we use intervals for consequence (C) and probability (p) we get

R = [C1*p1,C2*p2]

Statistical distributions - 1

We can use statistical distribution for C and p. In this case, the distributions are used to show our uncertainty.

Practical solutions could be:

• Beta distribution for p

• Gamma distribution for C

Statistical distributions - 2

Based on the distributions of p and C, we can compute the distribution of the risk in three ways:

• Mellin transforms

• Monte Carlo simulation

• Approximation methods

We will only look at the third alternative.

Statistical distributions - 3

The following approximation holds:

)()(

)(),...(),()(

),,...,(

1

2

21

21

i

n

i i

n

jin

xVarx

fYVar

xExExEfYE

xindxxxxfY

Risk approximation

Using the expressions from the previous slide we get the following approximations:

)()()()()(

)()()(22 CVarpEpVarCERVar

CEpERE

It is now straight forward to find the expected value and variance for R

Simple risk assessment

In order to a simple risk assessment we need to identify:

• Dangerous events

• Each event’s – consequence – C– probability – p

• Possible barriers – changes or controls

• Person responsible for each risk - Resp.

Simple risk table

Event C p R Barriers Resp

Events

We start by identifying dangerous events. The simple way to do this is to use brainstorming – just sit down and envisage your worst nightmares related to the activities under consideration.

Be realistic – only consider things that you believe can happen.

Barriers

Barriers can be realized through:

• Prevention – we change the system so that the event cannot occur.

• Mitigation – we can– change the system in order to reduce the

event’s probability or consequences.– define activities that will reduce the problems

if the event occurs.

Bar

rier

1 Bar

rier

2 Bar

rier

3 Bar

rier

4 Bar

rier

5 Bar

rier

6

Risk Prob. Event

Prevention barriersPrevent risk from becoming a problem

Handling barriersPrevent event from having bad consequences

Reduction barriersReduce effect of event

Benefits

It is important to bear in mind that:

• We usually expect to gain something through change – new products, new ways to work etc.

• Risks stem from changes.

• Reducing risk is a cost factor

We need to look at the total picture.

The total picture - 1

The total picture of the situation shows the risks and the benefits that stem from a planned change.

This is not a mechanism that can be used to identify the best solution.

It is, however, an important input when we want to make a decision.

The total picture - 2

The total picture shows risks and benefits. Risk can be shown in two ways:

1. Unmitigated risks

2. Mitigated risks – include the effect of risk reduction activities, e.g. barriers. This can be done by

– Modifying the risk assessment– Indicate how the risk will move in the

diagram

Consequences and benefits

B

HReduced number of MMI-related defects

M

L

p L M H

C

LExtra work needed for MMI-specification

M

H

Unmitigated risks

B

HReduced number of MMI-related defects

M

L

p L M H

C

LExtra work needed for MMI-specification

M

HLarge disagreements between designers and MMI experts

Partnership does not work

The mitigation effect

B

HReduced number of MMI-related defects

M

L

p L M H

C

LExtra work needed for MMI-specification

M

HLarge disagreements between designers and MMI experts

Partnership does not work

1

2

Including benefits

B

HReduced number of MMI-related defects

Better MMI for existing products

Better MMI requirements will reduce imp. costs

M

L

p L M H

C

L Extra work needed for MMI-specification

M

HLarge disagreements between designers and MMI experts

Partnership does not work

1

2

C and p as intervals - 1Benefit

Consequence

p

C and p as intervals - 2Benefit

Consequence

p

Mitigation effectCost of mitigationand benefits’ value and probability

Increased value or probability

The tyranny of “either – or”

All too often we are confronted by the statement that we can get only get X if we are willing to suffer Y.

This is the wrong attitude. The right attitude is that we will

1. Do what is needed to get X

2. Perform activities that will remove or reduce the bad effects of Y.

Leverage

Leverage is a prioritizing mechanism:

Leverage = (Benefit – Cost) / Cost

Leverage will prioritize activities with

• Large net benefits

• Small costs

Extended risk table -1

We can use cause – consequence chains or event trees for a risk to identify the best place to insert a barrier.

For each barrier, we need to assess:

• Cost - the cost of implementing it. We will use the scale H = 10, M = 3 and L = 1.

• E – how effective is the barrier? We will use the scale h = 1.0, m = 0.5 and l = 0.2

Extended risk table - 2

Event C p R Barrier Cost E L Resp.

Barrier leverage

Leverage = (C*p*E – Cost) / Cost

The leverage will prioritize barriers which:

• Have low costs – Cost is small

• Have high efficiency – E is large

• Attack important risks – C*p is high

Barrier – example Event Cons

.p R Mitigation E Cost

LResp

Partnership doesnot work – businessconflicts

10 3 30

Do a thorough researchon selected partner’sbusiness goals

0.5 10 0.5

John

Customers do notprioritize projectparticipation 10 3 30

State the conditions andconsequences of customerparticipation in thecontract

1.0 3 9.0

Pete

Some comments on barriers

It is important to remember that:• Each risk will usually need a different barrier – a

barrier that works against one risk can be valueless against another risk.

• It is important to consider the three main barrier strategies:– Prevent the risk from becoming a problem– Control the problem to avoid the consequences– Reduce the consequences

ALARP and GALE

There are two competing principles in the assessment of risk:

• ALARP – As Low As Reasonably Possible- We have done all that is reasonable to prevent problems and dangers.

• GALE – Globally At Least Equivalent. E.g. introducing a new process will not increase the risks compared to what it is today.

ALARP

ALARP requires that we analyze each risk separately and then implement mitigation activities.

A reasonable goal is to reduce each risk until the extra mitigation costs exceed the value of the risk reduction achieved.

All that we have seen up till now fits into an ALARP policy .

GALE

GALE requires us to look at the total risk of a change. In this way we can start by attacking the cheapest risk or the risk with the largest leverage.

The problem with the GALE principle is that we need to perform arithmetic on risks. E.g. we need to decide how many medium risks we need before we have a large risk

ALARP vs. GALE

The one important thing with using the GALE principle is that it forces us to ask “What is the current risk level?”

All too often we act as it the current way of doing things is risk free and all risk stems from changes.

This stance is enforced by the human tendency to underestimate the risk of status quo.

Using GALE

Important points

• GALE is a method for risk analysis. Benefits must be included elsewhere

• We need to look at both our current risk and the risk resulting from the proposed changes.

• Always perform a sensitivity analyses.

Risk – status quo vs. change

In many cases, maybe even in most of them, we do risk assessment because we want to compare two or more alternatives, e.g.:

• Status quo – no changes

• One or more changes - improvements

Event identification

• All significant dangerous events must have been identified.

• There must be a minimal overlap between the dangerous events .

• There must be a maximum of commonality between the dangerous events considered for the status quo and for the system after the proposed changes

The three event sets

The previous rules split the dangerous events into three sets – dangerous events that:

• Apply both to the status quo and to the new system.

• Are unique to the status quo

• Are unique to the new system

GALE and risk assessment - 1

GALE uses the following parameters for risk assessment:

• FE – the event frequency

• PE – the probability that the event will lead to an accident

• S – the severity score of an event

GALE and risk assessment - 2

We can compute individual and accumulated risk indices:

IE = FE + PE + S

IGR = log Sumi(10I)

IE is the risk index for a hazardous event

IGR is the global risk index

The GALE scoring scheme

The scoring scheme of GALE • Focuses on deviations from current

average. This is reasonable, given that it is mainly concerned with comparing status quo to a new situation.

• Must be tailored to each situation. The next slide shows an example from road safety. We need a scheme adapted to SPI.

Road safety - frequency score for event

Frequency classification

Occurrences / year on M42 ATM section FE

Very frequent 10000 Hourly 6

Frequent 1000 A few times a day 5

Probable 100 Every few days 4

Occasional 10 Monthly 3

Remote 1 Annually 2

Improbable 0.1 Every 10 years 1

Incredible 0.01 Every 100 years 0

Frequency score for event Frequency

classificationOccurrences per project FE

Very frequent 200 Every project 6

Frequent 100 Every few projects 5

Probable 40 Every 10th project 4

Occasional 10 Every 100th project 3

Remote 1 A few times in the company’slifetime

2

Improbable 0.2 One or two times during thecompany’s lifetime

1

Incredible 0.01 Once in the company’slifetime

0

Probability score for event

Classification Interpretation PE

Probable It is probable that this event, if it occurs, will cause a problem 3

Occasional The event, if it occurs, will occasionally cause a problem 2

Remote There is a remote chance that this event, if it occurs, will cause a problem

1

Improbable It is improbable that this event, if it occurs, will cause a problem 0

Severity score for event

Severityclassification

Interpretation S

Severe The portion of occurring problems thathave serious consequences is muchlarger than average

2

Average The portion of occurring problems thathave serious consequences is similarto our average

1

Minor The portion of occurring problems thathave serious consequences is muchlower than average

0

Sensitivity analysis

The global risk index is made of many indices. Each index will have a certain degree of uncertainty connected to it.

Usually, a few indices will have a large influence on the result while the rest will have but little influence.

Pareto’s rule applies - we need to identify the few important indices.

Important things to remember - 1The most important things to remember:• Risk assessment is by its nature subjective. • Use group techniques and include all

stakeholders• Use simple techniques so that you do not

exclude one or more stakeholders• Anchor it in experience and available data will,

however, improve the quality• Subjective values like “High” must be anchored

in each company’s reality. One company’s “High” may be another company’s “Low”.

Important things to remember - 2

• Include the effect of choosing status quo in all risk analyses.

• Always include opportunities• Consider the three barrier categories –

prevention, handling and reduction• Rank risks and opportunities according to

their leverage• The results from a risk assessment is just

one of several inputs to a decision