Risk Assessment

Case Study: ATCSoft

The ATCSoft project was launched, and steady progress was made

When the team set out to integrate the ATCSoft application with existing RADAR equipment, however, they hit a snag

The team members could not figure out how to integrate the systems

The RADAR system did not have appropriate physical connections, nor was there an appropriate driver for the interconnection

The project manager had to hire engineering consultants to work out the integration details

This diversion took 2 months and cost a significant amount of money (additional labour, consultancy fees, and business value)

Case Study: PathFinder 2.0

The PathFinder project was started in August Rory Thompson was the star developer

His knowledge of neural nets inspired him to suggest that a neural net implementation of a PathFinder would be a good idea

He wrote up much of the foundation code for the neural network Initial tests showed a definite improvement in the PathFinder

performance, and more realistic, human-like, decisions Despite the large salary increase he was offered, Rory took a

good offer with another firm When some tests showed that the neural net was not fast enough to

make real-time decisions, the team had no immediate answers A bug was found that sent the avatars wandering aimlessly around

the maze in a circuit, when certain rare conditions were present Again, the team had no idea how to approach the problem

What happened?

What happened in both of these case studies?

What happened?

What happened in both of these case studies? In the ATCSoft scenario, a technical problem ended

up stalling development This could easily have become a complete disaster, if it were

not possible to integrate the systems

What happened?



not possible to integrate the systems In the PathFinder scenario, a personnel problem was

at fault The team’s over-reliance on a hero was their downfall Taking the hero out of the equation stalled development

What happened?



not possible to integrate the systems In the PathFinder scenario, a personnel problem was

at fault The team’s over-reliance on a hero was their downfall Taking the hero out of the equation stalled development

In both scenarios, the team neglected their risks It is critical for a project team to understand and plan for risks

Risk Mitigation

In the ATCSoft project, the team should have investigated the integration of various systems at the start of the project

Given adequate time, the integration could have been worked out before it was needed

Risk Mitigation

In the ATCSoft project, the team should have investigated the integration of various systems at the start of the project

Given adequate time, the integration could have been worked out before it was needed

In the PathFinder project, Rory should have been the project’s neural network consultant

He could have thoroughly documented the neural network code as it was developed

He could have had seminars for team members, explaining the concepts of neural networks

Understanding neural networks, the team would have a better chance of carrying on without Rory

60-499 Project Management: Tools & Techniques

Risk Assessment

Risk Assessment

The following describes the risk assessment process:

Once risks are assessed, a project manager should plan for them

1. Identifying risks 1. Identifying risks

2. Estimating a risk’s cost/effects2. Estimating a risk’s cost/effects

3. Estimating a risk’s likelihood 3. Estimating a risk’s likelihood

4. Identifying alternatives4. Identifying alternatives

5. Evaluating/comparing alternatives5. Evaluating/comparing alternatives


Risk Assessment

Risk Identification

Risk Identification

The first step in risk analysis is to identify the project’s risks

Each project has its own set of unique risks

Identifying risks seems like a dark art How do you identify something that could potentially

be hidden until it is too late? However, risk identification can be made easier using

categories of risk This leverages the knowledge of many project managers

who have experienced risks

Categories of Risk

Here is one way to categorize risks: Technical risks (related to using a particular technology)

Performance Reliability Availability Complexity

Project management risks Poor resource allocation Poor planning Poor prioritization

Organizational risks Lack of support or resources Inadequate or inefficient management Interference from other projects & management agendas

p.65

Categories of Risk

Here is one way to categorize risks (continued): Constraint risks

Deadlines Resources

Business risks Marketability Timing Vendor delays Economic conditions

External risks Changing laws and regulations Dependence upon suppliers and contractors

p.65

When do we identify risks?

There are a two major approaches to risk identification Identify risks before the planning phase

Some risks may be difficult to spot when looking at requirements at a high level

Identify risks after the planning phase It is useful to know risks before the planning phase, so that extra

time can be dedicated to their mitigation

A good compromise is to perform risk identification during the planning phase:

After creating the work breakdown structure Before creating the schedule

Common Risks

These risks are related to schedule: Feature creep

New features are frequently added after development has started Requirements gold-plating

Too much documentation Implementation gold-plating

Developers are working on the perfect implementation Inadequate design

Too little attention has been paid to design Overly optimistic schedules

Management pushed schedules down, rather than schedules work their way upward from developers

Poor motivation/weak personnel Developers are working at a less-than-optimal pace

Silver-bullet syndrome A trendy technology was expected to produce the equivalent to 10,000 lines

of code in only 50 lines of code Contractor failure

A contractor lacked expertise/commitment needed to do the job on schedule


Risk Assessment

Estimating Risk Cost & Effects

Estimating Risk Costs & Effects

Estimating the costs & effects of a risk is dependent upon the risk

e.g. A project using a new technology might realize that the technology is inadequate or unreliable Now, the application must be retrofitted to another (trusted)

technology Much of the software may need to be replaced The cost in this case is the cost of developing the obsolete

components In addition, there may be hidden costs due to delays (such

as customer confidence or personnel availability)

Estimating Risk Costs & Effects

Estimating the costs & effects of a risk is dependent upon the risk

e.g. In some projects there is a risk that a key developer will leave the project If the key developer leaves, what will it take to replace her? Given market conditions, you might estimate a replacement

in 2 months Some project deliverables might be delayed by up to that

amount in her absence Also, you may have to consider signing bonuses, relocation

expenses, travel expenses, and other hiring costs It depends on the project whether or not these costs are

considered high

How to estimate cost?

To estimate the cost, imagine a scenario where the event occurs

e.g. Ok, Sarah has left the company. What do we do now? We could promote Gerard to team leader We would need one more developer with technological

expertise Our corporate headhunter estimates hiring will take 10 weeks

e.g. PHP didn’t solve the problem. What now? We could use Ruby on Rails We would need to send our developers to training courses

In-depth RoR training courses are 4 months


Risk Assessment

Estimating Risk Likelihood


Like risk cost, risk likelihood also depends on the risk e.g. The likelihood that a technology will fail can usually be

estimated accurately Tristan: “How likely is it that the technology (e.g. PHP) will fail to

meet our expectations?” Carlos: “PHP has been used for many projects successfully.

There have been many failed PHP projects, but very few cite the technology as the cause of the failure. On the other hand, there have been many very high-profile PHP projects.”

Carlos: “Consider FaceBook, which has similar (but more complex) functionality to our GroupWare application.”

Tristan: “FaceBook uses PHP? That is similar to our project.” Carlos: “I can’t forsee any features we will need that will be

impossible in PHP.”


Like risk cost, risk likelihood also depends on the risk e.g. How likely is it that Sarah Windermere will leave the project?

Her project manager may be aware of her job satisfaction, or her peers However, a face-to-face meeting to assess her job satisfaction has no

substitute Example:

Tristan: “Sarah, we’ve been very happy with your performance on previous projects. You will be an integral part of this upcoming project. Are there any concerns you have about this project or your job?”

Sarah: “Not really. I like the challenges.” Tristan: “If you do have any concerns, I want you to tell me right away. Your

satisfaction and motivation are very important to me.” Sarah: “Actually, I have been having a tough time working with Gerard. He

doesn’t respect my leadership, and tries to rally the team against me.” Tristan: “How does the rest of the team feel about his actions?” Sarah: “Many of them either have confronted him about or have told me

they don’t agree with him.” Tristan: “I’ll have a talk with your team to confirm what you’ve said, but it

sounds like Gerard and I need to have a talk about each of your responsibilities.”

How to estimate likelihood?

The best way to estimate is to ask the people closest to the problem

e.g. For Sarah, the best person to ask would be her (followed by her direct supervisor)

e.g. For PHP technology, anyone who has used PHP for a previous project


Risk Assessment

Identifying Alternatives


e.g. What are the alternatives to PHP? JSP/J2EE ASP.NET Ruby/ROR Perl


e.g. Are there alternatives to Sarah? Gerard Lepalme: Has leadership, but lacks the

technological expertise Helen Driskoll: Knows some of the technology, but is

very inexperienced


Risk Assessment

Evaluating & Comparing Alternatives


e.g. Let us examine the alternatives to PHP: JSP/J2EE

Much of our existing staff is familiar with this technology Using this technology would require a few days of training Architecture should be more robust

ASP.NET We have no in-house personnel trained in this technology Using this technology would require at least one month of training Development of user interfaces, however, should be made easier

Ruby/ROR We have only a few in-house developers trained in this technology Using this technology would require weeks of training Development should be made easier Architecture should be more robust

Perl We have no in-house personnel trained in this technology Using this technology would require at least one month of training


e.g. Let us examine alternatives to Sarah: Gerard Lepalme: Has leadership, but lacks the technological

expertise Gerard is a take charge kind of person He is also a get it done kind of person However, he is not familiar with XML and many other technologies

we plan to use Helen Driskoll: Knows some of the technology, but is very

inexperienced Helen knows XML and a few other technologies we plan to use However, Helen is just starting her career She has difficulty being assertive and taking charge She doesn’t command respect from her colleagues Her development itself is slow

Assigning Probabilities

There are many approaches to assigning probabilities It often doesn’t matter if you only care about the value relative to

other risks However, this is a good rule of thumb:

Very low: 0.05 Low: 0.20 Medium: 0.40 High: 0.60 Very High: 0.80


Risk Assessment

Bringing It All Together

Risk Assessment

Ok, we’ve collected dataWhat is next?

We need to crunch some numbers to determine which option is the best Qualitative assessment

Risks are evaluated and planned for if they have significant likelihood and significant costs

Dynamic risk assessment Risks are given a score for each development phrase The stage scores are totaled to calculate the risk’s score

Risk exposure assessment The risk likelihood is expressed in percentages The risk cost is expressed in weeks These two are multiplied to calculate an expected loss

Risk impact assessment Similar to risk exposure, but percentages are also used for cost

Qualitative Risk Assessment

Low Medium High

Low Ignore Ignore Consider

Medium Ignore Consider Take action

High Consider Take action Take action

Dynamic Risk Assessment

A B C Score

Requirements 1 3 1 5

Design 2 2 2 6

Implementation 3 1 2 6

Testing 3 1 4 8

Integration 2 1 4 7

Score 11 8 13 32

Risk Exposure Assessment

Probability of Loss

CostRisk

Exposure

Technology Failure

5% 20 weeks 1 week

Key Developer Attrition

40% 10 weeks 4 weeks

…etc…

Risk Impact Assessment

Probability of Loss

ImpactRisk

Exposure

Technology Failure

0.05 0.60 0.03

Key Developer Attrition

0.40 0.40 0.16

…etc…

Evaluating Alternatives

When considering alternatives, you can apply the same techniques to all alternatives

e.g. J2EE vs. ASP.NET vs. RoR

Probability of Failure

Cost Risk Exposure

J2EE 0.05 1 week 0.05 weeksASP.NET 0.05 5 weeks 0.25 weeks

RoR 0.05 3 weeks 0.15 weeks

Risk Assessment

Documents

Transcript of Risk Assessment