Selection Methods

Rev: Nov, 2013

Euiho (David) Suh, Ph.D.

POSTECH Strategic Management of Information and Technology Laboratory(POSMIT: http://posmit.postech.ac.kr)

Dept. of Industrial & Management EngineeringPOSTECH

Contents1 Conjoint Analysis

2 Q-Sort

3 DEA

4 Brainstorming

5 Delphi Method

6 Majority Voting

7 AHP

8 Financial Options vs. Real Options

3

Conjoint Analysis

■ What is Conjoint Analysis?– Statistical technique used in market research to determine how people value different

features that make up an individual product or service

– The objective of conjoint analysis• To Determine what combination of attributes is the most influential on respondents’ utilities

■ How to use Conjoint Analysis

Decide the important attributes and their levels

Construct the product’s profile

Collect the data (e.g., individuals rate each in terms of desirability or rank them)

Estimate the partial value using multiple regression

Apply the result to strategies (e.g., forecast market share of each product based on estimated coefficients)

Step 1

Step 2

Step 3

Step 4

Step 5

A1 A2

B1 B2

A1 A1

B1 B2

A2

B1

A2

B2

4

Conjoint Analysis – Example (1/2)

■ Organizing a Package tour product

■ Step 1: Decide the important attributes and their levels– Extract important factors through survey• Place: Southeast Asia, Australia, Guam• Price: 1.5 million won, 1.8 million won, 2.0 million won• Value added tax: Include, Exclude

■ Step 2: Construct the product’s profile– Make all possible conjoint profiles– Use fractional factorial design to reduce the number of profiles

3 X 3 X 2 X (1/2) = 9 Profiles– Profile 1: Guam, 2.0 million won, included VAT

Profile 2: Southeast Asia, 1.8 million won, included VATProfile 3: Guam, 1.5 million won, included VATProfile 4: Southeast Asia, 2.0 million won, excluded VATProfile 5: Australia, 2.0 million won, included VATProfile 6: Guam, 1.8 million won, excluded VATProfile 7: Australia, 1.8 million won, included VATProfile 8: Australia, 1.5 million won, excluded VATProfile 9: Southeast Asia, 1.5 million won, included VAT

5

Conjoint Analysis – Example (2/2)

■ Step 3: Collect the data– Let individuals rank the profile

■ Step 4: Estimate the partial value using multiple regression– Use statistic package software to estimate importance

of each factors (e.g. SPSS)– The most important attribute: Place– The most preferred place: Guam

■ Step 5: Apply the result to strategies

 idpref1

 pref2

 pref3

 pref4

 pref5

 pref6

 pref7

 pref8

 pref9

 

1 8 3 7 5 1 6 9 2 4

2 6 9 1 3 7 5 8 2 4

3 1 6 3 9 4 2 5 8 7

4 1 8 3 9 4 2 5 6 7

5 8 9 3 7 2 6 5 4 1

6 5 2 7 9 4 8 6 3 1

7 8 6 7 4 2 3 5 9 1

6

Q-Sort (1/2)

■ What is Q-Sort?– Method that offers you systematic and quantitative means for examining human sub-

jectivity– Method that gives you insight into understanding consensus or disagreement on ideas

among your group

– Key factor• 1st. Q-samples

Statements or ideas which are needed to be ranked

statements or ideas which are needed to be ranked

• 2nd. Q-Sorting Filling the blanks of the distribution-box with Q-Samples

• 3rd. Analysis Understanding the preferences for the ideas

Disagree(-4)

Neutral(0) Agree(+4)

7

Q-Sort (2/2)

■ How to use Q-Sort

Design Q-sort model

1. Identify a topic

2. Make Q-samples to represent the spectrum of opinions on a given topic

3. Select response distribution (e.g., -4 ~ +4 or -5 ~ +5)

Execute Q-sort

1.Categorize each statement into three groups (disagreement, neutrality, and agreement)

2.Place statements to the distribution: The higher degree of agreement of statement, the more the statement is placed at right

Analyze the result

1.Analyze correlations between respondents2.Group statements - 1st. Calculate factor loadings about each factor - 2nd. Classify respondents into factors by checking the factor loadings3.Analyze normalized factor score to find significant statements

8

Q-Sort – Example (1/2)

■ Step 1: Design Q-sort model– Topic: Ability ph.D in MIS must have– Q-samples: Table 1– Response distribution

■ Step 2: Execute Q-sort– Categorize each statement into three groups

(disagreement, neutrality, and agreement)– place each statement to

the distribution model(Agree Right, Disagree Left)

– e.g.

-2 -1 0 +1 +2

5

2 8 11

7 3 6 12 14

4 10 1 9 13

-2 -1 0 +1 +2

9

Q-Sort – Example (2/2)

■ Step 3: Analyze the result– Analyze correlations between respondents– Classify respondents by calculating factor loadings– Calculate normalized score for each type of respondents– Identify characteristics of responses from a certain type

(Check the absolute value of the normalized score)

10

DEA

■ What is Data Envelopment Analysis (DEA)?– Linear programming methodology to measure the efficiency of multiple decision-

making units (DMU) when the production process presents a structure of multiple in-puts and outputs

– Nonparametric method in operations research and economics for the estimation of production frontiers

– Advantages of DEA• No need to explicitly specify a mathematical form for the production function• Proven to be useful in uncovering relationships that remain hidden for other methodologies• Capable of handling multiple inputs and outputs• Capable of being used with any input-output measurement• The sources of inefficiency can be analyzed and quantified for every evaluated unit

– How to use DEAUtilize some variable to establish the efficiency frontier

Define the efficiency frontier

A numerical coefficient is given to each firm, defining its relative efficiency

Step 1

Step 2

Step 3

11

DEA – Example

■ A, B, C, D, E: Companies producing product y with raw material x1 and x2

– Efficiency of A and D: 1– Relative efficiency of B, C, E: 0.82, 0.78, 0.75 respectively (Excessive input of x1 and

x2)– B: Benchmarking A– C: Benchmarking A and D with weights 0.64 and 0.36 respectively

Required x2 to makeone y product

Efficiency Frontiers

Required x1 to makeone y product

12

Brainstorming

■ What is brainstorming?– Group creativity technique by which efforts are made to find a conclusion for a specific

problem by gathering a list of ideas spontaneously contributed by its members

– How to use Brainstorming

Find a comfortable meeting environment, and set it up ready for the brainstorming

Appoint one person to record the ideas. These should be noted in a format than everyone can see and refer to. Depending on the approach you want to use, you may want to record ideas on flip charts, whiteboards, or computers with data projectors

If people aren’t already used to working together, consider using an appropriate warm-up exer-cies

Define the problem you want solved clearly, and lay out any criteria to be met. Make it clear that the objective of the meeting is to generate as many ideas as possible

Give people plenty of time on their own at the start of the session to generate as many ideas as possible

Ask people to give their ideas, making sure that you give everyone a fair opportunity to contrib-ute

Step 1

Step 2

Step 3

Step 4

Step 5

Step 6

In a long brainstorming, take plenty of breaks so that people can continue to concentrateStep 7

13

Brainstorming – Example

■ Colab– Computer-supported conference system

by Xerox’s Palo Alto Research Center(PARC)1) “Bordnoter” : an electronic blackboard2) “Cognoter” : brainstorming, organizing, evaluating information3) “Argnoter”: presenting, arguing, and evaluating ideas

– Design principle: information sharing,“WYSIWIS” (What you see is what I see)

■ KJ method– Card-based knowledge-creation technique– Computerization of the KJ method• Prototype example: workstation-based GrIPS by Fujitsu

Divergent thinking phase:pick up data such as key words, phrases, picturesfrom Keyword Associater/Picture Library put into “cards” on the “Shared Board”

Convergent thinking phase:Using “Card Editor”, organize the cards into pieces of information, related into a diagram using “D-ABDUCTOR”

Communicate through “Miniature meeting Room”

KJ method

Colab

14

Delphi Method

■ What is Delphi Method?– Communication technique based on a structured process for collecting and synthesiz-

ing knowledge from a group of experts by means of a series of questionnaires accom-panied by controlled opinion feedback

– Key characteristics• Structuring of information flow• Regular feedback• Anonymity of the participants

■ How to use Delphi Method

Define the problemIdentifying the problem(s) in various forms from a questionnaire to a broad and open question

Give everyone the prob-lem

Recruiting experts to the Delphi group, sending the problem(s) to everyone in the group and asking them to respond

Taking the responses that experts send back and collating these into a single anonymous list or sets of listsCollate the response

Sending the collation back out to everyone with request to score each item on a given scale (typically 1 to 5) and may allow them to add further items or commentsGive everyone the colla-

tionRepeating the rounding until a certain stopping condition meets(Number of iterations, a specific level of agreement)Repeat as necessary

Analyzing the findings and putting plans in place to deal with future risks and opportunities in the projectAct on the findings

15

Delphi Method – Example (1/2)

■ Example– Choosing the next strategy for the company with 5 experts (Stopping Condition: 3

rounds)• 1st round (Questionnaire & Scoring result)

• 2nd round (Questionnaire & Scoring result)

Strategy + CommentsScore (1 is low, 7 is high)

E1 E2 E3 E4 E5

Organizing a task force team 2 6 4 5 7

Promoting a new supervisor for the project 5 3 4 6 2

Proceeding the project as now 3 2 1 4 3

Organize a new department for the project 6 2 1 3 1

Strategy + CommentsScore (1 is low, 7 is high)

E1 E2 E3 E4 E5

Organizing a task force team 4 6 5 6 7

Promoting a new supervisor for the project+ It can be hard to find a right person for the job

3 3 4 2 2

Proceeding the project as now+ It is inefficient

3 2 2 3 3

Organize a new department for the project+ Reorganizing will be needed after the project is over+ It is premature to make a new department+ The future of project is a little gray

2 1 1 2 1

16

Delphi Method – Example (2/2)

■ Example– Choosing the next strategy for the company with 5 experts (Stopping Condition: 3

rounds)• 3rd round (Questionnaire & Scoring result)

The company choose “Organizing a task force team” strategy

Strategy + CommentsScore (1 is low, 7 is high)

E1 E2 E3 E4 E5

Organizing a task force team+ It would be a good chance to make proper people in-volved to the project regardless of their departments

6 6 5 7 7

Promoting a new supervisor for the project+ It can be hard to find a right person for the job+ It can be a threat to the current supervisor and make him upset

2 3 2 2 1

Proceeding the project as now+ It is inefficient+ The project is too big to be proceeded with the current temporary process

3 2 2 1 3

Organize a new department for the project+ Reorganizing will be needed after the project is over+ It is premature to make a new department+ The future of project is a little gray

1 1 1 2 1

17

Majority voting

■ What is Majority Voting?– A decision rule that selects alternatives which have a majority, that is, more than half

the votes

– Advantage• Being able to produce a prompt and clear decision

– Limitation• Disregarding the basic rights of the minority

– Properties of Majority Voting• Fairness

Anonymity» The decision rule treats each voter identically» When using majority rule, it makes no difference who casts a vote; indeed the voter's identity need not even

be known. Neutrality» The decision rule treats each alternative equally

• Decisiveness The decision rule selects a unique winner.

• Monotonicity The decision rule would always, if a voter were to change a preference, select the alternative that the voter

preferred, if that alternative would have won before the change in preference. Similarly, the decision rule would never, if a voter were to change a preference, select a candidate the voter did not prefer, if that alterna-tive would not have won before the change in preference

18

Majority voting – Example

■ Should the university set curfew for students living in the dormitory?

The statement will be rejected,but the opinion of the one who agreed to the statement will be ignored

19

AHP

■ What is AHP(Analytic Hierarchy Process)?– A modeling structure for representing multicriteria (multiple goals, multiple objectives)

problems – with sets of criteria and alternatives (choices) – commonly found in busi-ness environments

– Benefits• Helping capture both subjective and objective evaluation measures• Providing a useful mechanism for checking the consistency of the evaluation measures and al-

ternatives suggested by the team

■ How to use AHP

Defining policy issues and establishing policy-making mem-bers

Constructing the hierarchy layer structure of the problems

Conducting questionnaire surveys and expert preference in-tegration

Establishing pair-wise comparison matrices

Conducting consistency test

Selecting the most optimal option

Step 1

Step 2

Step 3

Step 4

Step 5

Step 6

Goal1.00

Criterion 1

0.75

Criterion 2

0.25

SubC 1

0.15

SubC 2

0.2

SubC 3

0.4

SubC 3

0.1

SubC 3

0.15

Hierarchy layer structure

20

AHP – Example

■ Example

Defining policy issues and estab-lishing policy-making members

Constructing the hierarchy layer structure of the problems

Conducting questionnaire surveys and expert preference integration(1-9 scale preference question-naire)

Establishing pair-wise comparison matrices( if i≠j; if i=j)(Weight: normalization of geo-metric means of each row; ∑Weight = 1)

Conducting consistency test

Selecting the most optimal optionby comparing each alternative’s weighted sum of criteria values

Step 1

Step 2

Step 3

Step 4

Step 5

Step 6

• Issue: Which car is the best car for you?

Choice of a Car

FunctionLevel (FL)

PriceLevel (PL)

SafetyLevel (SL)

DesignLevel (DL)

PL9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8

9SL

FL PL SL DL Weight

FL 1 5 6 9 0.641

PL 1/5 1 2 7 0.198

SL 1/6 1/2 1 5 0.123

DL 1/9 1/7 1/5 1 0.038

• FL>PL, PL>SL, FL>SL → Consis-tency

• FL>PL, PL>SL, SL>FL → Inconsis-tencyFL PL SL DL Score

Car A 80 60 60 90 73.96

Car B 60 90 80 50 68.02

21

Financial Options vs. Real Options

■ What is Financial Options?

■ How to valuate Financial Options

■ What is Real Options?

■ How to valuate Real Options

An instrument that conveys the right,but not the obligation, to engage in a future transaction with fixed price

E.g.) Call option, Put option

Financial Options The right, but not the obligation, to

undertake certain business initia-tives

To reduce uncertainty of future planes

Real Options

Prepare a number of

scenarios

Put plan into ac-tion immediately

One of the sce-

nario happen

Expand

Mathematical model of a financial mar-ket containing certain derivative in-vestment instruments

To calculate “future option value” using stock price, volatility, risk free interest

Black–Scholes

A suite of option pricing tools to quan-tify the embedded strategic value for a range of proposed or existing in-vestment scenarios

Real Options Valuation (ROV)

22

Financial Options – Example

■ Call option– Right to buy certain stocks at the fixed strike price in the future– In-the-money: Future price > Strike price• Use the call option if the difference is bigger than the price of the call option

– At-the-money: Future price = Strike price– Out-of-the-money: Future price < Strike price

■ Put option– Right to sell certain stocks at the fixed strike price

in the future– In the money: Future price < Strike price• Use the put option if the difference is bigger than

the price of the put option

– At-the-money: Future price = Strike price– Out-of-the-money: Future price < Strike price

Use the call optionGive up the put option

Give up the call optionUse the put option

23

Real Option – Example (1/3)

■ Real Option Valuation (ROV) in Mining Industry– Factors: mineral commodity price (major), uncertainty of size of the orebody,

proper timing for exploration, cut-off grade, development of the mine Uncertainty will increase the risk of investment Estimate the project by ROV

– Value of operating mine• V = V (S, Q, t, j, φ)

24

Real Option – Example (2/3)

■ Real Option Valuation (ROV) in Mining Industry (Cont’d)– Free Cash Flow to the Firm (FCFF)• Cash that a company is able to generate after laying out the money required to maintain or ex-

pand its asset base

•

25

Real Option – Example (3/3)

■ Real Option Valuation (ROV) in Mining Industry (Cont’d)– Discounted Cash flow Method (DCF)

•

• Method of valuing a project, company, or asset using the concepts of the time value of money• All future cash flows are estimated and discounted to give their present values (PVs) – the sum

of all future cash flows, both incoming and outgoing, is the net present value (NPV), which is taken as the value or price of the cash flows in question

– ROV

•

• It is more appropriate than DCF to deal with changing price of commodity by considering closing mine permanently, temporary and reopen case

26

■ Alternative Selection Methods

Conjoint Analysis

Q-Sort

DEA

Brainstorming

Delphi Method

Majority Voting

AHP

Financial OptionsReal Options

27

Discounted Cash Flow (DCF)

■ Commonly used quantitative methods include discounted cash flow methods and real options.

■ Discounted Cash Flow (DCF)– Net Present Value (NPV): Expected cash inflows are discounted and compared to out-

lays.

28

Discounted Cash Flow (DCF) Cont’d

■ Internal Rate of Return (IRR): The discount rate that makes the net present value of investment zero.– Calculators and computers perform by trial and error.– Potential for multiple IRR if cash flows vary

■ Strengths and Weaknesses of DCF Methods:– Strengths• Provide concrete financial estimates• Explicitly consider timing of investment and time value of money

– Weaknesses• May be deceptive; only as accurate as original estimates of cash flows.• May fail to capture strategic importance of project

29

Real Options

■ Real Options: Applies stock option model to nonfinancial resource investments. E.g.,with respect to R&D:

■ The cost of the R&D program can be considered the price of a call option.

■ The cost of future investment required to capitalize on the R&D program (such as the cost of commercializing a new technology that is developed) can be considered the exercise price.

■ The returns to the R&D investment are analogous to the value of a stock pur-chased with a call option.

30

The Aggregate Project Planning Framework

■ The Aggregate Project Planning Framework– Managers map their R&D projects according to levels of risk, resource commitment

and timing of cash flows