Proprietary & confidential. © Decision Lens 2010

Decision Making, Hierarchical Structures and Pairwise

ComparisonsFebruary 2010

Concepts

Decision Making and Hierarchical Thinking

Criteria

Pairwise Comparisons

Priorities

2

Analytical Hierarchy Process (AHP)

The Analytic Hierarchy Process enables decision makers to structure decisions hierarchically: the goal of the decision at the top, strategic objectives in the higher levels, evaluation criteria in the middle, and alternative choices at the bottom.

3

Criteria Definition

At the most basic, a criterion is a driving factor that is used in making a decision. In Decision Lens, we use a criteria hierarchy

4

At the highest level, objectives are stated as actions or goals to be accomplished− i.e., Increase Impact to Mission

At the lowest level, drivers or indicators used to evaluate alternatives at a detailed level− i.e., Mission Area 1

• Don’t confuse criteria with strategies, actions, or methods of execution

• The criteria should be the end state, not the path to get there.

– i.e. “Increase Growth” is a good criterion. “International Expansion” is not a criterion, it is a strategy to achieve the objectives.

– “Expand Brand” is a good criterion. “Acquire New Products” is not a criterion, it is a strategy to achieve the objectives.

Module 1: Building the Model

Criteria Modeling Example – Selecting Which New Products to Bring to Market

5Module 1: Building the Model

Best Practice Criteria Concepts

While dependencies may exist, criteria should be as mutually exclusive, collectively exhaustive (MECE) as possible

We normally see at least 2 levels of criteria where a criterion is too broad of a “bucket” but not more than 3 levels is recommended

The ideal number of criteria at the highest level is 7

6

Magic numberof

seven plus or minus two

Human Cognitive Capacity

Purpose of Comparing Criteria

Perform Pairwise Comparisons to develop relative weights on criteria hierarchy

Facilitating discussion during voting to reveal stakeholder viewpoints and positions

Defining consensus and how to use it in support of prioritization

Consistency and Alignment among decision-makers is measured as a result of their comparisons

7Module 2: Compare Criteria

Do Numbers Have an Objective Meaning?

Butter: 1, 2,…, 10 lbs.; 1,2,…, 100 tons

Sheep: 2 sheep (1 big, 1 little)

Temperature: 30 degrees Fahrenheit to New Yorker, Kenyan, Eskimo

Since we deal with Non-Unique Scales such as [lbs., kgs], [yds, meters], [Fahr., Celsius] and such scales cannot be combined, we need the idea of PRIORITY.

PRIORITY becomes an abstract unit valid across all scales.

A priority scale based on preference is the AHP way to standardize non-unique scales in order to combine multiple criteria.

8Module 2: Compare Criteria

This is Today’s Reality: Liter and KilogramThe Basis of Any Scale is Arbitrary

In 1889, the first Conference Generale des Poids et Mesures (CGPM) sanctioned the international prototype of the kilogram, made of platinum-iridium, and declared: This prototype shall henceforth be considered to be the unit of mass. The third CGPM (1901), in a declaration intended to end the ambiguity in popular usage concerning the word weight, confirmed that:

The kilogram is the unit of mass; it is equal to the mass of the international prototype of the kilogram. The International Prototype Kilogram (IPK) kept in Sèvres, near Paris, is the standard of mass. National prototype kilograms are maintained in several countries and periodically checked against the International Prototype.

9Module 2: Compare Criteria

Pairwise Comparison

Participants perform multiple sets of comparisons for each level of hierarchy

For each judgment, participants determine which criterion is more important AND by how much

Judgments are used to form ratios in a matrix; The matrix is used to calculate priorities for the judgment set (eigenvector)

10Module 2: Compare Criteria

Scale for Pairwise Comparison

1 Equal importance

3 Moderate importance of one over another

5 Strong or essential importance

7 Very strong or demonstrated importance

9 Extreme importance

2,4,6,8 Intermediate values

Use Reciprocals for Inverse Comparisons

11Module 2: Compare Criteria

Pairwise Comparison Steps

12

4

Click the Compare Criteria tab

Select the Pairwise Comparison step

Each person makes judgmentsusing keypads. First, evaluate which criteria is more important. Second, evaluate by how much using the Equal to Extreme scale

3

5

The facilitator will click Next Vote to move to the next comparison until all criteria have been compared to each other1

The facilitator will ask the group, “With respect to the goal of buying a car, what is more important, safety or performance?”

2

Use the tree view in the left navigation as an alternative way to select which criteria are being selected

Module 2: Compare Criteria

Best Practice Pairwise Criteria Concepts

Keep in mind:

The facilitator should promote group discussion to inform and provide rationale for voting discrepancies

To make a comparison one holds the smaller one in the pair in mind as the unit and estimates how many times larger the other is

13Module 2: Compare Criteria

Viewing Priority Graph

14

A) Clicking the second sub step Priority GraphB) Clicking Next Vote at the last comparison there are no more votes so it will take you to the next step)C) Or by clicking Next (see previous slides)

A CB

Once all comparisons are made you can view results in 3 ways:

Module 2: Compare Criteria

Priority Graph Options

15

Click Sort Descending to get a highest to lowest graph of the weighted priorities. Note that you will see the weights for what is selected on the criteria tree view on the left navigation panel.

1

Click the Current Properties drop down to pick which data set is shown. You can choose to see results of a specific individual’s judgments or the group.

2

You can select either the Graph or the Tree view. Graph view provides a bar chart like what you see here. Tree view is a blown up version of the tree view that you see on the left navigation pane.

The Alignment is an index shown in percentage to indicate how much consensus the re were around judgments.

4

Click either Global or Local radio button to change how the weights are shown in the tree view. Global shows overall weights of sub factors. Local shows how sub factors are distributed within that node only

5

See next slide 3

Module 2: Compare Criteria

Inconsistency and Alignment

16

Feature Definition Use Helpful Hint

Inconsistency

The consistency of the judgments is tracked using the rigorous math analytics behind the AHP to validate the decision process.

If the group’s (not an individual’s) inconsistency is more than .10 or 10%, certain pairwise comparisions should be revisited to ensure that logical integrity is kept

Individual’s inconsistency will always be higher than the groups. Groups are known to be more accurate and consistent.

Having good consistency does not mean you are closer to the “right answer” in the decision

Alignment

The final priorities in the Priority Graph, tells you the mix of values for the group, and the Alignment Index score then tells you how close that Priority Graph was for all of the individuals as compared to the group average.

The largest score in the Alignment Index is 100, telling you that the final weighted priorities for each individual came out exactly the same as the final weighted priorities for the group. There is no action regardless of the Alignment number.

The smaller the Alignment Index score drops from 100, the farther away are the individuals’ priorities from those of the group, and the less alignment there is.

Module 2: Compare Criteria

Inconsistency Diagram

A is bigger than BB is bigger than CC is bigger than A!?

This is not logical – the inconsistency tracker tracks individual and group logic when judgments are made in pairwise comparisons

Group consistency is best to be within 10%

17

A B C

Module 2: Compare Criteria

Evaluating Inconsistency

18

Click the Compare Criteria tab and the Inconsistency Analysis sub step.

Alternatively, if you are using the Next button at the right of the screen to navigate through tab 2, you will see Inconsistency at the end of each comparison set following the Priority Graph for that comparison set.

1

2

Ensure the goal level of the tree is selected

In this case, Inconsistency is fairly close to zero therefore, no action is necessary. You can click the Go To Comparison button next to those comparisons that are high. You want to decrease overall consistency to 10%. This may require changing one or more judgments.

3

Module 2: Compare Criteria

Critical Facilitation Technique – 3 Part Test

Using facilitation techniques, ensure that participants pass…..

The 3 Part Test:

1.Can I live with the decision made here today?

2.Can I refrain from behaving or speaking negatively to others about the decision?

3.Can I readily support the decision and devote the resources necessary to make the decision successful?

19Module 2: Compare Criteria

Summary

20

Prioritize decision-making criteria

Inputs Any previously developed decision criteria; Decision makers are identified; Any assumptions are understood.

Actions Complete head-to-head (pairwise) comparisons of criteria. Ensure that the group’s logic is not too inconsistent.

Helpful Hints Encourage participants to discuss the rationale behind their voting as a way to get information and insights out in the open. Ensure consistency is less than 10%.

Things to Avoid Do not let any one individual dominate the discussion. Make sure your criteria are defined very specifically and that all participants understand and agree with their meaning. When making judgments, participants are encouraged to explain their position but should not be forced to agree with everyone else.

Outputs Weighted decision criteria model.

Module 2: Compare Criteria