1

Managing RiskTravis McGrath, PhD, PEChief Operations Officer

March 2020

Safety Share… and Risk Share

2

What does “risk” mean to you?

3

What does risk look like for a DOT?

4

Design Risk

5

Design errors and omissions

Environmental Process Risk

6

Procurement Risk

7

1.00 1.02 0.99

1.03 1.04

1.19

1.23

0.95

1

1.05

1.1

1.15

1.2

1.25

2012 2013 2014 2015 2016 2017 2018

INFL

ATIO

N IN

DEX

YEAR

ITD Construction Cost Index

Nominal 3%/yr Inflation

ITD Index - Actual

8

Construction Risk

9

Operations/Maintenance/Resilience Risks

10

O&M/Resilience Risks… from our customers

11

Employee Safety Risks

No employees were harmed in the making of this photo.

But he did win a Daytime Emmy.

Risks from Our Own Risk Management

12

ITD Enterprise Risk Dashboard – Fall 2019

Enterprise Risks – the ‘umbrella’

What other risks do you face in your work?

14

So…

How do we define risk?

How can we manage risk?

15

16

Risk: Potential event or condition that could harm (threat) or improve (opportunity) our ability to achieve goals

Generally described in terms of a likelihood and an impact, either qualitatively or quantitatively

Risk Response: Action or response to improve a risk. Strategies typically include:

• Terminate (Avoid)• Treat (Reduce threats)• Take Advantage of (opportunities)• Transfer• Tolerate (Accept residual risk)

Risk and Risk Response

Approaches for Characterizing Risk

Risk List / Risk Paragraph• Brainstorm a list of risks relative to an assumed ‘base’ case• Can sort the list based on which risks you feel are most significant• In a paragraph, describe what actions you will take to respond to key risks

• Not the recommended approach, but if you absolutely don’t have time for anything else, or it’s a simple project, this is better than nothing

17

Risk Category / Type of Impact

EA required instead of CE Environmental / Time, Cost

Delays getting RR agreement Third Party / Time

Asphalt unit price way higher Procurement / Cost

Have to pave ramps too Construction / Cost

Approaches for Characterizing RiskQualitative• Descriptors like Low, Medium, High or Green, Yellow, Red• Useful for quickly identifying critical risks to guide initial responses• Caution!

– Very approximate/imprecise – don’t base big decisions on qualitative ratings– Probability and impact ratings must be defined for common understanding– Don’t use non-linear or undefined scales (like often done with “1 to 5” scales)

18

VH = Very HighH = HighM = MediumL = Low

Rare Unlikely Possible Likely Very Likely

Very Significant M M H VH VH

Major L M M H VH

Moderate L M M M H

Minor L L L M M

Insignificant L L L L M

Likelihood Rating

Aggr

egat

e Im

pact

(acr

oss a

ll im

pact

type

s)

Approaches for Characterizing RiskQuantitative

Quantify • Likelihood of occurrence (probability or frequency)• Impact of occurrence (e.g., additional $, additional time)

Useful for • Combining different types of impacts into a single measure (“total risk”)• Making more defensible and transparent decisions

19

3.12.7

2.52.3

1.71.11.1

0.80.3

0 1 2 3 4

C5. Uncertain construction sequencingP6. Delays getting railroad agreements

P4. Delays in agency approvalsD12. Permitting issues

R1. Difficult ROW AcquisitionIdentified Minor Risks (aggregate)

Unidentified Risks (aggregate)U1. Difficult utility relocations

D1. EIS Re-evaluation required

Average Delay to Project Schedule (months)

Risk Event

20

Risk Assessment/Management Process

Identify Risks

Establish “Base”

Assess / Analyze Risks and Establish Risk Tolerance

Plan and Implement Risk

Responses

Document, Monitor, and

Update

Purpose: Proactively identify, assess, prioritize, and mitigate risks to the DOT,to better understand and control outcomes

What events could occur and change your planned outcomes?

How can you manage those potential events and their outcomes if they occur?

https://www.fhwa.dot.gov/GoSHRP2/Solutions/Renewal/R09

21

Risk Assessment/Management Process

Identify Risks

Establish “Base”

Assess / Analyze Risks and Establish Risk Tolerance

Plan and Implement Risk

Responses

Document, Monitor, and

Update

RA/RM Process applies at any level• Enterprise Risk Management (ERM)• Program risk management

• Asset Management (TAMP)• Asset Resilience (all-hazards)• Transportation Improvement

Program (TIP/STIP)• Individual project risk management

But specific tools or approach may vary at each level (e.g., qualitative vs. quantitative)

Illustration

Qualitative RA/RM for a simple project

(i.e., don’t need a probability expert to do this)

22

Establish Project Base

• The planned project, based on assumptions used to develop:

– scope of work

– cost estimate

– schedule estimate

– third-party engagement (subs, client, public, etc.)

– logistical plans

• Generally optimistic - even if it wasn’t intended to be - since PMs are generally optimistic about their project’s outcome

• A basis against which risks can be identified and measured

If you don’t first define a Base, you can’t characterize risk

Risk assessment can’t “cover” for poor Base cost and schedule estimates

Project Base - Example

• DOT scope of work: Design and construction inspection for a proposed permanent soil-nail wall, to be built through an existing basement wall next to the roadway. DOT/consultant are proposing an innovative drilling method and nail bar design as a cost-saving alternative to a soldier-pile & tieback system. Consultant is performing the design. The DOT is contracting the driller.

• Key assumptions and conditions– Soil nailing is appropriate for the purpose and site conditions– The innovative drilling method / bar design will perform adequately– The innovative approach can be accepted/permitted quickly– The drilling contractor (from another state) will

perform adequately– The existing geotechnical information is is

sufficient for completing design (i.e., no surprises or misinterpretations)

Identify Risks

• Goal: comprehensive, non-overlapping list of risks (both threats and opportunities) to the DOT

• Suggested risk identification process1. For the current project:

• Identify issues of concern (what keeps you awake at night?)• Consider how Base assumptions might change or be invalid

2. Brainstorm with others what other problems or improvements could occur, considering collective experience on other projects

3. Consult risk checklists to “prompt” anything you might have missed4. Document in risk register – sort and categorize logically

• Guidance– Document all credible possibilities. Think broadly; don’t dismiss prematurely.

Avoid overconfidence. It can happen to your project.– Consider the whole project; most schedule risk is prior to construction!– Several tens of risks is typical without overlapping or artificial separation

Identify Risks• Sources of project risk to DOT

– Uncertainty in conditions (e.g., ground, groundwater, unusual weather, market conditions, force majeure)

– The prime contractor and subcontractors (contract language, unclear expectations, relationship, performance)

– Design and CE&I consultants (unclear scope, not meeting Standard of Care)

– Third parties (e.g., permitting agencies, railroads, utilities, public, legislative)

– The DOT’s own performance – particularly technical and managerial, including over-optimism, poor assumptions, and mistakes. For example:

• Failing to hold consultants and contractors to acceptable quality or agreed-to schedules

• Failing to manage scope or document / approve changes

• Inadequate technical work or review

– Any other potential changes relative to Base assumptions, whether real or perceived by someone else

Identify Risks –Example

Project Risk Register Project Number:Project Manager:

Date:Before (Without) Risk Response

Risk Category Unique Risk Number Risk Title Summary of the Risk to DOT Probability of

OccurrenceConsequence if Risk Occurs

Risk Severity

Category 1 1 0Category 1 2 0Category 1 3 0Category 1 4 0Category 1 5 0Category 2 6 0Category 2 7 0Category 2 8 0Category 3 9 0Category 3 10 0

00

Excerpt from Simple Risk Register Template, page 1 of 2

Identify Risks –ExampleRisk Category Unique Risk

Number Risk Title Summary of the Risk to DOT

Design D1 Additional information needed for wall design

Existing information from others might not be adequate for us to complete our soil-nail design. Could lead to either additional borings or more conservative wall design, at owner's expense.

Construction C1 Groundwater conditions worse than anticipated

Could create excessive pressures behind the wall and therefore unanticipated movement. Could require additional dewatering and/or remedial wall support.

Design D2Soil nailing isn't appropriate for this site, but not discovered until construction

Separate from the issue of innovative technology (see below). The previous designer believed soil nailing won't work. Could require re-design of more robust system or switch back to soldier piles. This re-design would be DOT's risk.

Construction C2 Innovative technology doesn't perform as intended

The innovative drilling method and bar design might not perform as needed (e.g., can't make capacity), requiring additional nails, grout, or a switch to traditional soil-nail technology. Since DOT is proposing this technology in lieu of existing design, client might not be willing to pay for significant changes if technology doesn't work. Hence, risk to DOT.

Construction C3Drilling contractor doesn't bring the required skill

Contractor has a good reputation in another state, but is untried in our area. Could cause quality issues and/or delays.

Construction C4Drilling contractor has significant problems with drilling equipment

Brings the wrong equipment, poorly-maintained equipment, or suffers signficant break-down. Could cause delays and extended overhead costs. DOT would look to Prime to pay these costs, but time risk is DOT's.

Third-Party / Political P1 Difficult City approval process for proposed

approach

The City might be nervous about soil nailing and/or the proposed innovative technology. Could cause delays and additional cost to work through issues. Prime would not be happy.

Third-Party / Political P2 Prime contractor resists the change to soil

nailing

The Prime contractor might have reasons to stick with original design (e.g., already started procurement of soldier piles and tiebacks). Could cause delays and/or switch back to soldier piles.

Construction C5Extended poor weather impacts construction

Especially if construction is delayed into the wet season. Could result in delays and extra costs.

Construction C6Experienced DOT construction staff not available when needed

Multiple construction projects underway. We can't afford to delay this project due to staffing limitations, and we're unlikely to staff this with inexperienced people, so the impact would likely be to other DOT projects.

Construction C7Underestimate the costs to design and staff the CE&I Could lead to project write-offs if we can't get change-orders approved.

Design D3 Errors in Consultant design Could require re-design plus could result in delay claims from the drilling and/or Prime contractors

Assess Risks /Perform Risk Analysis

• Goal: Evaluate risk ‘severity’ to prioritize for mitigationFor many ‘typical’ DOT projects, qualitative RA (using ratings) is appropriate.For complex or risky projects, specialists can help PMs with a quantitative approach.

• Simplified qualitative risk analysis process and guidance– Define the ratings for probability and impact – use linear scales if possible– Assess each risk in terms of the ratings

• Generally define impacts first, then a consistent probability (no unique combinations, so try for a reasonable, matched pair)

• Consider all impact types (e.g., $, time) – use combined total (e.g., equivalent $) if possible

• Use relevant data if available. But most likely you’ll have to draw from your team’s expertise and judgment → subjective assessments

– The goal is unbiased, defensible assessmentsBe honest. Avoid overconfidence and “wishful thinking”

– After all the risks are assessed, prioritize based on risk ratings (e.g., P x C) to “set the stage” for risk-response planning

Assess Risks / Perform Risk Analysis -Example

Qualitative Rating Scale for Probability of Risk Occurrence

Rating Range Average

1 <20% 15%2 20% - 35% 30%3 35% - 55% 45%4 55% - 70% 60%5 > 70% 75%

Qualitative Rating Scale for Consequences if Risk Occurs

Corresponding Cost Impact to

Project

RatingPercent of

Estimated Project Cost Column1

1 <= 2.5%2 5.0%3 7.5%4 10.0%5 >= 12.5%

Corresponding Probability

Note: These are linear scales, which means for example:• A “4” is twice as

much as a “2”• A “risk” P x C of

4 x 2 is the same as 2 x 4

Before (Without) Risk Response

Risk Category Unique Risk Number Risk Title Summary of the Risk to DOT Probability of

OccurrenceConsequence if Risk Occurs

Risk Severity

Design D1Additional information needed for wall design

Existing information from others might not be adequate for us to complete our soil-nail design. Could lead to either additional borings or more conservative wall design, at owner's expense.

4 1 4

Construction C1 Groundwater conditions worse than anticipated

Could create excessive pressures behind the wall and therefore unanticipated movement. Could require additional dewatering and/or remedial wall support.

3 2 6

Design D2Soil nailing isn't appropriate for this site, but not discovered until construction

Separate from the issue of innovative technology (see below). The previous designer believed soil nailing won't work. Could require re-design of more robust system or switch back to soldier piles. This re-design would be DOT's risk.

2 5 10

Construction C2 Innovative technology doesn't perform as intended

The innovative drilling method and bar design might not perform as needed (e.g., can't make capacity), requiring additional nails, grout, or a switch to traditional soil-nail technology. Since DOT is proposing this technology in lieu of existing design, client might not be willing to pay for significant changes if technology doesn't work. Hence, risk to DOT.

3 5 15

Construction C3 Drilling contractor doesn't bring the required skill

Contractor has a good reputation in another state, but is untried in our area. Could cause quality issues and/or delays. 3 3 9

Construction C4Drilling contractor has significant problems with drilling equipment

Brings the wrong equipment, poorly-maintained equipment, or suffers signficant break-down. Could cause delays and extended overhead costs. DOT would look to Prime to pay these costs, but time risk is DOT's.

2 4 8

Third-Party / Polit P1Difficult City approval process for proposed approach

The City might be nervous about soil nailing and/or the proposed innovative technology. Could cause delays and additional cost to work through issues. Prime would not be happy.

4 2 8

Third-Party / Polit P2Prime contractor resists the change to soil nailing

The Prime contractor might have reasons to stick with original design (e.g., already started procurement of soldier piles and tiebacks). Could cause delays and/or switch back to soldier piles.

3 1 3

Construction C5Extended poor weather impacts construction

Especially if construction is delayed into the wet season. Could result in delays and extra costs. 1 2 2

Construction C6Experienced DOT construction staff not available when needed

Multiple construction projects underway. We can't afford to delay this project due to staffing limitations, and we're unlikely to staff this with inexperienced people, so the impact would likely be to other DOT projects.

2 2 4

Construction C7Underestimate the costs to design and staff the CE&I

Could lead to project write-offs if we can't get change-orders approved. 2 3 6

Design D3 Errors in Consultant design Could require re-design plus could result in delay claims from the drilling and/or Prime contractors 2 4 8

Assess Risks / Perform Risk Analysis -Example

Plan Risk Responses (Mitigation)

• Goal: Plan cost-effective responses for at least the most significant risks

• Suggested risk-response planning process and guidance

1. Be sure to address the highest-priority risks (i.e., highest risk ratings, from the sorted risk register)

2. Brainstorm practical, cost-effective responses to manage each risk; e.g.,• Can you modify the design to avoid or reduce/mitigate the risk?• Can you contractually transfer the risk to a party who can better manage it?

3. Evaluate the combined effectiveness of the planned responses for each risk, in terms of the residual (remaining) consequence and/or probability of occurrence. Avoid over-optimism.

4. Select cost-effective responses for implementation (mitigation benefit > cost)• Can DOT tolerate the residual risks? (risk tolerance defined by DOT leadership)• Include a contingency ($, time) to “cover” some of the overall residual risk

5. Record the results in the risk register and re-sort to identify the most significant residual (post-mitigation) risks

Plan Risk Responses (Mitigation) –Example

Excerpt from Simple Risk Register Template, page 2 of 2

Project Risk Register Project Number:Project Manager:

Date:After (With) Risk Response

Risk Category Unique Risk Number Risk Title Summary of the Risk

to DOT Planned Risk ResponseResidual

Probability of Occurrence

Residual Consequence if Risk Occurs

Residual Risk

Severity

Timing for Implementation

Risk Owner and Response

OwnerCategory 1 1 0Category 1 2 0Category 1 3 0Category 1 4 0Category 1 5 0Category 2 6 0Category 2 7 0Category 2 8 0Category 3 9 0Category 3 10 0

00

Plan Risk Responses (Mitigation) –Example

Unmitigated After (With) Risk Response

Unique Risk Number Risk Title Risk

Severity Planned Risk ResponseResidual

Probability of Occurrence

Residual Consequence if Risk Occurs

Residual Risk

Severity

Timing for Implementation

Risk Owner and Response Owner

D1 Additional information needed for wall design 4

Minimal risk to DOT. However, would request additional borings (client cost) in proposal. If not granted, design more conservative wall to account for the increased uncertainty/risk.

4 During proposal preparation PM

C1Groundwater conditions worse than anticipated 6 Design redundant drainage for "worst case" 2 2 4 During design Wall Designer

D2Soil nailing isn't appropriate for this site, but not discovered until construction

10No risk responses identified, other than not proposing soil nailing as an alternative (i.e., which would mean not doing this project).

10

C2 Innovative technology doesn't perform as intended 15

Perform Verification tests prior to production to determine what capacity can be achieved, and modify design if needed prior to construction. Should reduce the likelihood, but not the cost impact to DOT.

2 5 10Document in proposal and write into specifications

PM

C3Drilling contractor doesn't bring the required skill 9 Write specifications to require submittal and approval of

crew experience and permission for crew substitutions. 2 3 6Document in proposal and write into specifications

PM

C4Drilling contractor has significant problems with drilling equipment

8Ensure our contract with the driller places responsibility for equipment downtime (and related consequences) on the contractor.

2 2 4During contract negotiations with driller

PM

P1 Difficult City approval process for proposed approach 8 Meet with City to understand their requirements for

approval, and to establish issue-resolution process. 2 2 4 During design PM

P2Prime contractor resists the change to soil nailing 3 No risk responses identified, although will meet with

team prior to moving forward with proposal. 3 Now PM

C5 Extended poor weather impacts construction 2 No risk responses identified. 2

C6Experienced DOT construction staff not available when needed

4No risk responses identified, although will meet with other DOT PMs to alert them to the issue and encourage an office-wide staffing plan.

4 Now PM

C7 Underestimate the costs to design and staff the CE&I 6

Write the proposal to address uncertainty in construction duration; propose time-and-materials compensation to allow for extended schedule.

1 3 3 During proposal and estimate preparation PM

D3 Errors in Consultant design 8 Ensure adequate internal check and review of design calcs and specifications 1 4 4 During design PM, Wall Designer

Implement, Monitor, and Update• Goal: Control risks by selecting, assigning, and implementing

cost-effective risk responses

• Risk management implementation guidance

– For each risk to be mitigated:

• Assign owner(s) for the risk and the selected risk responses

• Provide the resources and support to implement the planned responses

• Monitor the effectiveness of the risk response

• Update and revise responses as appropriate (e.g., if the risk changes or the response isn’t working as planned)

• If the risk occurs, allocate contingency to cover its residual impact

• If the risk is averted or the project advances beyond the phase in which the risk could occur, “retire” the risk

– Be vigilant for new risks as the project evolves– Remember to keep management up to date, as appropriate

Questions about the Risk Management Process?

36

38

Hot Topic in AASHTO:“Risk and Resilience”

What does“transportation system resilience”

mean to you?

39

40

Example definitions from the pros:

• “… to anticipate… and adapt to changing conditions, and withstand,… and recover rapidly from disruptions” (FHWA)

• “… absorb, recover from, or more successfully adapt to adverse events” (National Research Council)

• From reliability theory: the ability to resist downtime, and to recover quickly if it ‘goes down’

System Resilience

41

Those definitions share two common elements:

Resistance of system to disruption

Recovery of the system if disrupted

System Resilience

42

One perspective:

Natural and human-induced risks affect system resilience, either by threatening or enhancing resilience

How we respond to those risks also affects system resilience

How are Risk and Resilience Related?

What does risk to resilience look like?

43

44

Risk to Resilience during PlanningWill this thing last until we get money to replace it?

Risk to Resilience during Design

45

Uncertainty in design requirements / conditions

46

Risk to Resilience during Construction

47

Risk to Resilience during Construction

48

Risk to Resilience during Operations

Risk to Resilience during Operations

50

Risk to Resilience during Operations

51

Risk to Resilience during Operations

52

Risk to Resilience during Operations

53

Sources of Risk to Resilience

Limited funding

Constraints on how funding is used

Changing political priorities

Uncertainty in design requirements / under-designed

Changing system use

Design errors and omissions

Poor-quality construction

Unsuitable materials

Construction impacts/accidents

Accelerating climate change• Loss of permafrost• Sea-level rise / erosion• Increasing variability in extreme

weather events

User-caused damage

Inadequate maintenance practices

Slow emergency response

Cyber attacks

Uncertainty in ground conditions • General geologic / geotechnical /

hydrogeological• Seismic• Thermal• Geohazards

54

AASHTO Subcommittee on Risk Management

Purpose: Help you benefit from Risk Management

Guidance and implementation assistance Share best practices / Community of Practice Support research; e.g., NCHRP 20-24(129) - scoping study for all-hazards risk

and resilience analysis NCHRP 08-118 – risk assessment techniques for asset

management

Questions?

55

56

Enjoy your stay in Coeur d’Alene!