Risk Analysis in Information Technology Projects

Tennessee Summit ‘09October 20, 2009

Thomas Danford

Chief Information Officer

Tennessee Board of Regents

PRESENTATION BACKGROUND

The examples in this presentation are based upon contract work to analyze two major IT projects to develop go forward options, baseline cost estimates, acquisition cost estimates, and risk analysis of the options being considered by the clients.

Goals, Objectives, and Ground Rules

Discussion of Current Budgetary Climate Overview of Risk Analysis Techniques and

Methodologies Used for major IT Projects The Role of Risk Analysis in Risk Management and

Resource Allocation Decisions No Math/Accounting Lessons or Review! Examples are for Illustrative Purposes Only! Focus on Implementation of New Projects

Why Project Risk Analysis?

Improved information to support decisions regarding project direction, scheduling, and budget

Identify proactive actions that will improve technical solutions, scheduling, and ROI

Develop contingencies for known causes of poor project performance

Identify project metrics for project monitoring and status reporting

Demonstrate due diligence for audit and compliance requirements

Risk Analysis vs. Risk Management

(Risk analysis is broadly defined to include risk assessment, risk categorization, risk communication, risk management, and policy relating to risk. In evaluating large scale IT projects they are typically done independently)

What is Risk Analysis?Risk analysis is the systematic study of uncertainties and risks that could be encountered in business, engineering, public policy, and IT (as well as many other areas).

What Is Risk Management?Active process of assessing, communicating and managing the risks facing an organization to ensure that an organization meets its objectives.

Risk Analysis & Management Process

Project’s Strategic Objectives

Risk Analysis

Risk ReportingThreats and Opportunities

Decision

Risk Management

Residual Risk Reporting

Monitoring

Risk Identification

Qualitative

Risk Estimation

Quantitative

Risk Evaluation

An

alys

isM

an

age

me

nt

Roles in Risk Analysis/Management

(In evaluating large scale IT projects risk analysis is typically part of the project evaluation process)

Risk Analysts – identify risks faced, determine how and when they arise, and estimate the severity of impact of adverse outcomes.

Risk Managers – Mitigate or hedge identified risks.

Primary Methodologies for Risk Analysis

Quantitative & Qualitative Risk Analysis Risk Simulation Models Monte Carlo Analysis

Methodologies not easily adapted to IT Project Risk Analysis

Risk Simulation Models – Useful in situations with "flows" of materials or parts, people, etc. with complex interrelationship through a system with multiple steps (logistics, manufacturing, budgeting)

Monte Carlo Analysis – Useful for modeling where there is such significant uncertainty in many inputs that randomizing variables is viable for analysis (economics, oil production, sales)

Qualitative & Quantitative Risk Analysis

Qualitative Risk Analysis – Used to identify potential risks, as well as assets and resources which are vulnerable to these risks. Includes both internally and externally driven risk elements

Quantitative Risk Analysis – Provides arithmetic assessment of the probability and impact of the identified risks. Quantitative risk analysis is also used to create overall risk scores for the risk elements and project alternatives.

Financial RisksCost of Ownership Project ScopeCost Benefit

ComplexityProvisioningChange Management

Technology Risks

ContractsGovernance

CommunicationEnvironment

Management Risks

Strategic RisksCompetition

RequirementsIndustry Changes

Customer Demand

Life Cycle

Integration

State Appropriations

Products & ServicesRecruitment Re-skillingPolitics

Technology Advances

Maintenance & Upgrades

Many risk elements have both external and internal drivers. Hence, those elements overlap.

Qualitative Risk Elements

Ishikawa’s “Fishbone” Technique

Quantifying Risk

    Impact on Project  

Likelihood Low Medium High

    (10) (50) (100)

High (1.0) Low Medium High    10 X 1.0 = 10 50 X 1.0 = 50 100 X 1.0 = 100

Medium (0.5) Low Medium Medium

    10 X 0.5 = 5 50 X 0.5 = 25 100 X 0.5 = 50

Low (0.1) Low Low Low

    10 X 0.1 = 1 50 X 0.1 = 5 100 X 0.1 = 10

Comparative Risk Analysis

Comparative Risk Analysis

Risk, Cost, & Schedule

Risk Analysis Explicitly Addresses:

Heuristics – Tendency of people to use "rules of thumb", intuition, educated guesses or even common sense, which doesn't serve very well in complex IT, business, and policy decisions.

Cognitive Bias – Tendency to over-weight the most recent adverse event and projecting current good or bad outcomes too far into the future.

Optimism Bias – The demonstrated systematic tendency for people to be overly optimistic about the outcome of planned actions.

Fear, Uncertainty, and Doubt (FUD) – Strategy to influence decision making by disseminating negative (dis)information designed to undermine the credibility of a project.

Determining RiskTips for a Better Analysis

Don’t start with any predetermined conclusions Cross-functional team involvement is essential Heuristics as well as cognitive, optimism, and

pessimism (FUD) bias must be addressed Deal appropriately with risk and uncertainty

Tangible Benefits of Proactive Risk Analysis

Schedule: Improves planning & upstream activities.

Costs: Proactive identification of potential cost drivers.

Quality: Meeting all scope and feature objectives of the project.

Time Quality

Cost

Time Quality

Cost

Summary & a Few Caveats

Business case requires risk analysis Judgment – art as well as science Heuristics, cognitive, optimism, and pessimism

(FUD) bias must be controlled Strategic Misrepresentation Quantitative issues accompany risk (magnitude) Cost and risk should be evaluated together

Additional Resources

The Society for Risk Analysis (SRA) http://www.sra.org/

Risk Management Association http://www.rmahq.org/RMA/

Thanks for joining me today!!