TransparentChoice Training

Changing the world one decision at a time

Agenda

• What is TransparentChoice?• What is it used for?• An introduction to core concepts

WHAT is TCh?

• Analytic Hierarchy Process, augmented by– simple “efficient frontier”– Import– Scales– Reporting and sensitivity analysis– Simple decision optimization

• A set of tools for helping groups of people make better decisions

• Powerful and simple collaborative decision making software

TransparentChoice is used for

• Project prioritization and selection• Procurement / vendor selection• Strategy development• R&D pipeline management• Technology selection• Hiring• … any other multi-faceted, multi-stakeholder

decision

Core Concepts

Tutorials

Overview of a decision

Define and weight criteria

Collect and score

alternatives

Analyse results

Make a decision

Discover your preferences and build consensus

Example: Project Prioritization

Define criteria

Weight criteria

Collect and validate alternatives

Score alternatives

Build consensus

Review results and decide

What is a project?

• Any decision you are making using TransparentChoice is a project

• A project consists of;– A “project name and description”– A set of alternatives (the list of options available to you as

an outcome of your decision)– A set of criteria (the list of factors that you will use to

evaluate your criteria)– Optionally, a set of scales and / or attributes– A team– Results and analysis

Confidential

Alternatives

• Alternatives are just the possible outcomes of your decision. Examples include;

Type of Decision The Alternatives

Portfolio prioritization and selection

The list of projects from which you’re trying to choose.

Procurement / vendor selection

The list of vendors

R&D pipeline management

The list of potential research projects

Hiring The list of candidates

Your decision can only ever be as good

as your best alternative, so make sure you spend time brainstorming some

great, wonderful, creative ideas

Criteria

• Criteria are the factors you use to evaluate the alternatives

• Often, criteria are goals, outcomes or targets you want to affect by your decision

• Criteria are often made up of sub-criteria

Getting your criteria right is key to making a

good decision. Brainstorming is a good

way to generate a strong list of criteria.

Click here for that tutorial.

What is pairwise comparisonPairwise comparison and the analytic hierarchy process (AHP) came out of Wharton business school in the 1970s. It is a way to break down complex decisions into a series of small judgements – this helps eliminate bias and moves us from unreliable “intuition” towards more rational decisions.

The pairwise comparison process itself is remarkably simple. You’re presented with two options, A and B, and you’re asked which you prefer. You indicate the strength of your preference on a 9-point scale (called the Saaty scale, if you’re interested)

Pairwise comparison is extremely useful for capturing subjective decisions about things that cannot be measured. For example, if you’re choosing a machine to diagnose disease, you will probably have to make a trade-off between speed of diagnosis and accuracy. This is an inherently subjective decision. You can’t create a scale that objectively measures the importance of speed and accuracy in diagnosing illness.

Pairwise comparison is valuable because it lets you make trade-offs between multiple competing goals in a rational and transparent way. And it’s all built off the combined fields of cognitive neuroscience and decision science. It is this mechanism of breaking complex decisions into smaller judgements that allows us to get over the “bounded rationality” problem and to dramatically reduce the impact of bias on your decisions.

The most common use of pairwise comparison in TransparentChoice is to “discover” the weighting of the criteria you will use to make decisions.

Scales

• Scales are used to score alternatives against criteria

• They are used to translate an “judgement” into a score. An judgement might be “Risk is high” or “This project is strongly aligned with strategy”

• The scores are then combined with the weighting of criteria to create an overall score for each alternative

What are attributes? What can I use them for?

• Your alternatives can have “attributes”, or properties, that are fixed. – The energy efficiency of a fridge might be based

on lab data– or the lift generated by a particular wing design

might be the output of a model• Unlike scales and pairwise comparisons, you

don’t collect “opinions” on attributes, rather they are entered into the alternative directly

What is a survey?

• A survey is simply a list of questions that is generated by TransaprentChoice

• A survey is used to collect judgements– Judgements about the relative importance of your

criteria or– Scores for your different alternatives

• You can send a survey that covers– Just criteria– Just alternatives– Or both criteria and alternatives

What is consensus building?

TransparentChoice helps improve the quality of your decision but, just as importantly, it allows you to build consensus and buy-in to your decision. As this work by Stanford University shows, confidence in a decision can be as important as the decision quality.

Consensus building in TransparentChoice is possible because we’ve broken a big, complex (and possibly contentious) decision down into small judgements. Each person fills in a survey and then, when comparing answers, you can instantly see where there is consensus and where you need a little… discussion.

The processFill in individual surveys•Can be done separately•Or can be done as a group execise

Create a “Review group” – a group of people whose answers you want to compare

Identify areas of consensus and areas of disagreement

Bring the group together and discuss the differences one by one and resolve them

Sit back and enjoy a well-earned cup of tea

What is a review group?• A review group is just a group

of people whose votes you want to bring together into a singe consensus opinion

• Review groups can be used to build consensus within a team and then between teams

• The example is for a hospital, where the decision team is broken into nurses, doctors and hospital administrators

• Consensus is built from the bottom up

Nursesreview group

Doctors review group

Admins review group

Clinical review group

Final decision

group

Scores• The scores in

TransparentChoice are a measure of the attractiveness, or benefit, of the different alternatives

• To get the overall score for an alternative, you multiply each criterion weight with the individual score for that criterion and then add them all together.

• But there is a critical difference… see the next slide

Weighting Score Weighted ScoreCriterion 1 10% x 0.130 = 0.013 Criterion 2 8% x 0.253 = 0.020 Criterion 3 12% x 0.108 = 0.013 Criterion 4 20% x 0.303 = 0.061 Criterion 5 17% x 0.498 = 0.085 Criterion 6 7% x 0.471 = 0.033 Criterion 7 14% x 0.217 = 0.030 Criterion 8 12% x 0.498 = 0.060

Total: 100% 0.315