Applied Information Economics: Kickoff for Risk Return Analysis

HubbardDecision Research

The Applied Information Economics Company

Intro toAIE

Risk/Return AnalysisProcess

Workshops

Applied Information Economics: Kickoff for Risk Return Analysis



Intro toAIE


Workshops

AppliedAppliedInformationInformationEconomicsEconomics

EconomicsEconomics

Decision/Game Decision/Game TheoryTheory

Empirical Empirical Decision TheoryDecision Theory

StatisticsStatistics

Information TheoryInformation TheorySoftwareSoftwareMetricsMetrics

InformationInformationEngineeringEngineering

Modern PortfolioModern PortfolioTheoryTheory

Operations Operations ResearchResearch

What is AIE?Applied Information Economics is the practical application of

scientific and mathematical methods to quantify the value of IT-enabled business investments



Intro toAIE


Workshops

Real Solutions to…

• …the economics of information• …the economics of IT infrastructure• …the economics of risk• …the economics of labor reduction when

headcount is not reduced• Bottom Line: AIE assesses and prioritizes IT

investments based on quantitative and economically rational methods



Intro toAIE


Workshops

What Do the Critics Say?

• “Quantifying the risk and comparing its risk/return with other investments sets AIE apart from other methodologies. It can substantially assist in financially justifying a project -- especially projects that promise significant intangible benefits.” The Gartner Group

• “AIE represents a rigorous, quantitative approach to improving IT investment decision making…..this investment will return multiples by enabling much better decision making. Giga recommends that IT executives learn more about AIE and begin to adopt its tools and methodologies, especially for large IT projects.” Giga Information Group

• “AIE-like methods must become the standard way to make (IT) investment decisions.” Forrester Research, Inc.



Intro toAIE


Workshops

Basic Risk/Return Analysis

3. MakeMeasure-

ments

4. Conduct VIA

5. Conduct Risk/Return

Analysis

6. MakeRecommenda-

tions

1. Describe &Classify

2. Clarify Decision

Model

ProcedureProcedure

AIE

AIE

DeliverablesDeliverablesToolsTools

OrganizationOrganization



Intro toAIE


Workshops

Describe, Classify & Plan

• Purpose: Agree on the specific investment to be analyzed, Agree on the specific question to be answered, Plan the rest of the analysis

3. ConductMeasurements

3. ConductMeasurements

4. Conduct VIA

4. Conduct VIA

5. Conduct Risk & Return

Analysis

5. Conduct Risk & Return

Analysis6. Make

Recommendations

6. MakeRecommendations

1. Classify &Plan

1. Classify &Plan

2. Clarify Intangibles &

CBA

2. Clarify Intangibles &

CBA



Intro toAIE


Workshops

Clarify The Decision Model

3. MakeMeasure-

ments

4. Conduct VIA


Analysis

6. MakeRecommenda-

tions


2. Clarify Decision

Model

• During Clarification we translate the “Intangibles” into measurable units• These are ultimately modeled in a spreadsheet



Intro toAIE


Workshops

Understanding Measurement(The Measurement.com approach)

• Gilb’s Law “Anything can be measured in a way which is superior to not measuring it at all”

• The perceived impossibility of measurement is an illusion caused by not understanding:– the Concept of measurement– the Object of measurement– the Methods of measurement

• See my “Everything is Measurable” article in CIO Magazine (got to “articles” link on www.hubbardresearch.com



Intro toAIE


Workshops

Conduct Measurements

3. MakeMeasure-

ments

4. Conduct VIA


Analysis

6. MakeRecommenda-

tions


2. Clarify Decision

Model

• We use the variety of measurement methods previously discussed

• We usually start with what we know now (i.e. calibrated estimates)

• More elaborate measurements (large controlled experiments or surveys) are only taken if we can show they are economically justified



Intro toAIE


Workshops

When asked to provide a subjective 90% confidence interval, most managers providea range that only has about a 40%-50% chance of being right

When asked to provide a subjective 90% confidence interval, most managers providea range that only has about a 40%-50% chance of being right

Actual 90% Confidence Interval

Calibrated Estimates• Measuring your own

uncertainty about a quantity is a general skill that can be taught with a measurable improvement

• Studies show that most managers are statistically “overconfident” when assessing their own uncertainty

• Training can “calibrate” people so that when they provide a 90% confidence interval, it still has a 90% chance of being right (even though it is subjective)

Perceived 90% Confidence Interval



Intro toAIE


Workshops

• The value of additional information can be calculated for each uncertain variable in the analysis

• Measurement efforts will be more productive by focusing on variables that matter the most (results are often surprising)

• This method is based on the probability of a change in a decision with additional information and the difference in the value of the decision

Calculate the Value of Information

$ $$$

3. MakeMeasure-

ments

4. Conduct VIA


Analysis

6. MakeRecommenda-

tions


2. Clarify Decision

Model



Intro toAIE


Workshops

EVI p r V p r V p r V p r EVi j j ij

z

j j i l j j ij

z

j

z

i

k

( ) max ( | ), ( | ),... ( | ), *, , ,1

12

111

The Decision Theory Formula:

What it means: Information reduces uncertainty Reduced uncertainty improves decisions Improved decisions satisfy business objectives (by definition)

The Economic Value of Information



Intro toAIE


Workshops

Conduct Risk/Return Analysis

Administrative Cost Reduction

Total Project Cost

% Improvement in Customer Retention

5% 10% 15%

10% 20% 30%

$2 million $4 million $6 million

ROI0% 50% 100%

3. MakeMeasure-

ments

4. Conduct VIA


Analysis

6. MakeRecommenda-

tions


2. Clarify Decision

Model



Intro toAIE


Workshops

Make Recommendations

• The recommendations include:– To accept or reject the investment

– Possible modifications to the proposed investment

– Various risk management tactics

• Deliverables include– The written report

– The spreadsheet

– The presentation

3. MakeMeasure-

ments

4. Conduct VIA


Analysis

6. MakeRecommenda-

tions


2. Clarify Decision

Model



Intro toAIE


Workshops

Overview of RRA Analysis

Intangibles“Customer Satisfaction”“Strategic Alignment”“Technology Risk”“Information Quality” etc.

MeasurablesErrors in Decision XChange to Strategic Measure MProductivity in Activity YChance of cancellation, etc.

5% 10% 15%

10% 20% 30%

$2 mill $4 mill $6 mill

Measurements

$

$$$

$$

Value of Info.

Calculate Risk/Return Position"expected" ROI

50% 100% 150% 200% 250%-50% 0%

Probabilityof a negative ROI

Probability of a positive ROI

Organization's investment

limit

Acceptable region of investment

Return

Risk

Classification



Intro toAIE


Workshops

Workshops

• Much of the initial data gathering is from a series of workshops

• We need to schedule 5-6 workshops for the following activities:– Define & Classify the investment– Clarify Decision Model– Measurement (initial)

• Calibration

• Estimation



Intro toAIE


Workshops

Defining the Investment• What is the objective of this investment? (A one-

sentence description of why)

• What costs are unique to this investment?

• What benefits are unique to this investment?

• What are the risks of the investment?

• What “decision dimensions” are important besides just an accept/reject recommendation?

• Is all of the investment optional?

• The decision is analyzed on behalf of which investor?



Intro toAIE


Workshops

The Concept Of Measurement

• Sometimes one believes that a thing is “immeasurable” only because they do not actually understand the concept of measurement

• The “Measurement Theory” definition of measurement: “A measurement is an observation that results in information (reduction of uncertainty) about a quantity.”

• Any “reduction of uncertainty” about a quantity can be of value

?



Intro toAIE


Workshops

Normal Distribution

Uniform Distribution

Lognormal Distribution

Hybrid

Threshold confidence 15% 85%

Ideal Values: Point

Real-world Meas.

Real-world Measurements vs. Ideal Values



Intro toAIE


Workshops

The Object of Measurement

• If a thing seems like and immeasurable “intangible” it may just be ill-defined

• Often, if we can define what we mean by a certain “intangible” we find ways to measure it

?



Intro toAIE


Workshops

The Clarification Chain

AIE assumes that if a benefit or cost is defined unambiguously, then it is

measurable.• If it is “Better” it is different in some

relevant way...• If it is relevantly different then it is

observable...• If it is observable then it is observable in

some amount... • If we can observe it in some amount

then it is measurable.



Intro toAIE


Workshops

• Imagine that you are a scientist capable of making clones of entire companies and that you have a cloned pair of your company

• Change one of the companies so that one has the stated “intangible” and the other does not

• Ask what would you actually observe that would be different between the two companies

The “Thought Experiment”



Intro toAIE


Workshops

The “Intangible” Possible Meanings After Clarification

“Employee Empowerment”

“Information Availability”

“Customer Relationship”

• Less management overhead• Certain decisions are more accurate and faster

• Time and cost of searching is reduced• Certain costly errors are less frequent

• Increased repeat business

Examples of Clarification

• Tools like “The Clarification Chain” are used to identify unit-of-measure variables hidden beneath the “intangible” label

• I offer a challenge that given any intangible, I can clarify it and identify a method of measurement within 15 minutes (I’ve never lost)

Applied Information Economics: Kickoff for Risk Return Analysis

Documents

Transcript of Applied Information Economics: Kickoff for Risk Return Analysis