Contrary Schools of ThoughtContrary Schools of ThoughtWithin Military Decision-Within Military Decision-

making Groupsmaking Groups

Contrary Schools of ThoughtContrary Schools of ThoughtWithin Military Decision-Within Military Decision-

making Groupsmaking Groups

Fred CameronFred CameronOperational Research Advisor to the Operational Research Advisor to the Commander, 1st Canadian DivisionCommander, 1st Canadian Division

Fred CameronFred CameronOperational Research Advisor to the Operational Research Advisor to the Commander, 1st Canadian DivisionCommander, 1st Canadian Division

Agenda

• Introduction

• Methodology

• An Application

• A Second Example

• Observations on Methodology

• Conclusion

Introduction

• ‘Squishy’ problems

• No obvious quantitative analysis

• Seeking problem investigation, not necessarily a solution

• Rule by majority

• Consideration of minority

The Methodology

• Participants develop:–Alternatives, with descriptions–Individual rankings of alternatives

• CDSP* derives the group ranking• Analyst calculates:

–Kendall’s Coefficient of Concordance, W

–Rank Correlations, Tau-b or b

–Distance Metrics–Multidimensional Scaling–Cluster Analysis

*CDSP = Consensus Decision Support Program

Rank Correlation Coefficients

• Kendall’s Tau-b and Spearman’s Rho• Examples for two judges, four objects:

–A B C D and A B C D Tau-b = 1–A B C D and A B D C Tau-b = 0.67–A B C D and B D A C Tau-b = 0.0–A B C D and D C B A Tau-b = -1

• Ties:–A B C D and A (B C) D Tau-b = 0.914

A B C D E F G H I JX 1 4 ½ 2 4 ½ 3 7 ½ 6 9 7 ½ 10Y 2 ½ 1 2 ½ 4 ½ 4 ½ 8 9 6 ½ 10 6 ½Z 2 1 4 ½ 4 ½ 4 ½ 4 ½ 8 8 8 10

Ranks of ten objects by three judges.

X Y ZX 1Y 0.47 1Z 0.66 0.67 1

X Y ZX 0Y 0.53 0Z 0.34 0.33 0

Pairwise rank correlations between judges.

Pairwise distances between judges.

Note the use of Kendall’s convention where tied objects are given the average rank. Judge X has tied B and D after E (3rd place) so they are given rank 4½.

Ranking and Rank Correlations

Past Applications

• Workshops on Characteristics of the Future Environment

• Workshops on Re-organization of the Army

• Questionnaire on Future Reconnaissance Capabilities

Canada’s Future Army

• Management and Objectives• Three Parallel Initiatives:

–Future Environment–Future Force Capabilities–Future Force Concept Matrix

• Workshops on Future Environment–18 Participants in two workshops–Structured brainstorming–List of elements, with descriptors in shorthand–Consensus on critical elements

Workshops on Future Environment

Grouping of concepts from structured brainstorming:

‘Advance of technology’

‘Emergence of the information age’

‘Importance of space’

‘Revolution in military affairs - Non-linear Battle Space’

‘Globalization - coalition operations’

‘Greater global instability’

‘Evolution of the social order’

‘Civil-military relationship - Interdependence of institutions’

‘Resource constraints’

CDSP Consensus Ranking Algorithm

• Choose winner(s) in pairwise majority voting (the Condorcet winner)

• Remove winner(s) and repeat until all objects have been ranked

• Problems:– ties– intransitivity - the voting paradox

CDSP Results of Workshops on Future Environment

1. ‘Advance of technology’

2. ‘Greater global instability’

3. ‘Emergence of the information age’ & ‘Resource constraints’

5. ‘Globalization - coalition operations’

6. ‘Revolution in military affairs - Non-linear Battle Space’

7. ‘Civil-military relationship - Interdependence of institutions’ & ‘Evolution of the social order’

9. ‘Importance of space’

A01 0.9A02 0.9A03 0.9A04 0.7A05 0.7A06 0.6A07 0.5A08 0.5A09 0.5A10 0.5A11 0.5A12 0.4A13 0.2A14 0.2A15 0.2A16 0.1A17 0A18 0

Note:

• Same value of correlation coefficient does not imply same ranking

• Each participant’s nom de guerre (A01 to A18) was assigned based on this ranking

Correlations between Individuals and CDSP

Matrix of Pairwise bA01 A02 A03 A04 A05 A06 A07 A08 A09 A10 A11 A12 A13 A14 A15 A16 A17 A18

A01 1 . . . . . . . . . . . . . . . . .A02 0.7 1 . . . . . . . . . . . . . . . .A03 0.7 0.9 1 . . . . . . . . . . . . . . .A04 0.6 0.8 0.7 1 . . . . . . . . . . . . . .A05 0.7 0.5 0.6 0.5 1 . . . . . . . . . . . . .A06 0.5 0.6 0.6 0.5 0.3 1 . . . . . . . . . . . .A07 0.5 0.5 0.4 0.5 0.4 0.1 1 . . . . . . . . . . .A08 0.6 0.4 0.3 0.5 0.4 0.3 0.3 1 . . . . . . . . . .A09 0.3 0.7 0.6 0.6 0.1 0.5 0.4 0.3 1 . . . . . . . . .A10 0.3 0.6 0.6 0.6 0.4 0.4 0.5 0.1 0.3 1 . . . . . . . .A11 0.5 0.3 0.3 0.4 0.7 0.4 0.3 0.4 0.1 0.3 1 . . . . . . .A12 0.4 0.3 0.3 0.2 0.2 0.3 0.3 0.3 0.1 0.2 0.3 1 . . . . . .A13 0.2 0.2 0.1 0.2 0 0.3 -0.1 0.3 0.2 -0.2 0.1 0.6 1 . . . . .A14 0.1 0.2 0.3 0 0.3 0.1 0.1 -0.3 0 0.2 0.1 0 -0.3 1 . . . .A15 0.1 0.3 0.2 0.1 -0.1 0.5 0.1 0.1 0.4 0.1 0 0.2 0.3 -0.1 1 . . .A16 0.1 0 0.1 -0.2 0.2 0 -0.1 -0.2 -0.1 -0.1 -0.1 -0.3 -0.2 0.6 -0.1 1 . .A17 0.1 -0.1 -0.2 0 0.1 0.1 -0.2 0.5 -0.1 -0.3 0.2 0.1 0.5 -0.3 -0.1 0 1 .A18 0.1 -0.2 -0.1 -0.2 0.2 0.2 -0.3 0.1 -0.3 -0.3 0.3 0.1 0.4 0.2 -0.1 0.4 0.5 1

Multidimensional ScalingConfiguration

By Multidimensional Scaling

A18

A17

A16

A13

A11

A08

A12

A14

A05A07

A10

A09

A04A02A03

A15

GROUP

A06

A01

MDS Algorithm

Derive distance matrix from correlations:dij = 1-tij

For 2-D MDS:Minimize STRESS = ((dij-d´ij)2/dij

2))½

Over all (xi,yi), i=1,…,n (coordinates in 2-space)

where n = number of participants

dij = distance between i and j in multiple dimensions

d´ij = distance between i and j in 2 dimensions,

d´ij = ((xi-xj)2+(yi-yj)2)1/2

Cluster AnalysisCluster Tree

By Single Linkage

A01

A02A03

A04

A05

A06

A07

A08

A09

A10

A11

A12A13

A14

A15

A16

A17

A18

Note:

• Single linkage is also called ‘nearest neighbour’

• Start with the two individuals who are the closest pairwise

• Distance to a new cluster is the distance to the nearest member

Cluster Analysis Algorithm ‘Nearest Neighbour’

• At each stage we may combine any pair of objects (individuals or clusters of individuals). At the first stage, all objects are individuals.

• Then two steps:

–1. Combine the two objects that are closest

–2. Recalculate distances for the new cluster:

If objects s and t were combined to form object j

dij = min(dis,dit), for all remaining objects i

• Stop when only one cluster remains

MDS Map with Clusters

The Problem of Stopping Criteria

• Cluster of all 18

• Clusters of 16 and 2, outlined with light blue lines

• Cluster of 7, outlined with darker blue line

Configuration

By Multidimensional Scaling

A18

A17

A16

A13

A11

A08

A12

A14

A05A07

A10

A09

A04A02A03

A15

GROUP

A06

A01

A Second Example

•Two captains on the Technical Staff Course

•Future reconnaissance capabilities

•Seven main areas of interest

•Several concepts in each

•Workshop ruled out -- cost and logistics

•Results by questionnaire

Future Reconnaissance Doctrine

Options as provided:– Formation of ISTAR Unit(s)– Integration of LOS with NLOS Capabilities– 360° Reconnaissance and Surveillance Capability– Maintenance of Reconnaissance Platoons and Troops– Light Reconnaissance and Dismounted Reconnaissance

Functions– Interoperability with Allies– Joint Operations with Air Force Reconnaissance and

Surveillance Assets– Centralize and Standardize Reconnaissance Training

Reconnaissance Results

1. Integration of LOS with NLOS Capabilities,

2. Interoperability with Allies,

3. (multi-way tie) – Maintenance of Reconnaissance Platoons and Troops, – Formation of ISTAR Unit(s), – Joint Operations with Air Force Reconnaissance and

Surveillance Assets, – Centralize and Standardize Reconnaissance Training, – Light Reconnaissance and Dismounted Reconnaissance

Functions, and – 360° Reconnaissance and Surveillance Capability

Multidimensional ScalingMDS Configuration Map

Question 1: Doctrinal Concepts

A11

A16

A21

A20

A09

A05

A06

A02

A12

A18

GROUP

A14

A13

A19A03

A15

A04

A17

A07

A10

A08

A01

Cluster AnalysisCluster Tree

Question 1: Doctrinal Concepts

A01

A02

A03

A04

A05A06

A07

A08

A09

A10

A11

A12

A13

A14

A15

A16

A17A18

A19

A20

A21

GROUP

Sub-group of 12

Sub-group of 9

MDS Map with ClustersMDS Configuration Map

Question 1: Doctrinal Concepts

A11

A16MINORITY

A21

A20

A09

A05

A06

A02

A12

A18

GROUP

A14

MAJORITY

A13

A19A03

A15

A04

A17

A07

A10

A08

A01

Sub-group of 9Sub-group of 12

Observations on Methodology

• Ease of use (e.g., Excel, Systat, SPSS, SAS, Statistica)

• Use of ranks

• Number of clusters

• Optimization in the MDS procedure

• Clustering algorithms

• Flexible interpretations

Observations on Methodology (cont’d)

• Anonymity

• Workshops or questionnaires

• Dictatorships

• Contamination between participants

• A closure crutch

• More ruthless decision making

Conclusion

• Consensus Ranking Valuable

• Include a Measure of Concordance

• Include Other Diagnostics– Pairwise Rank Correlations– Multidimensional Scaling– Cluster Analysis

Further Information

• Fred Cameron–Operational Research Advisor to Commander 1st Canadian Division

–1-613-992-4584 (Ottawa) or 1-613-541-5010 x 8719 (Kingston)

–FCameron@ncs.dnd.ca

• References:Maurice Kendall and Jean Dickinson Gibbons, Rank Correlation Methods, 5th ed., Oxford University Press, 1990

Joseph B Kruskal and Myron Wish, Multidimensional Scaling, Sage, 1977

John A Hartigan, Clustering Algorithms, Wiley, 1973

Charles Romesburg, Cluster Analysis for Researchers, 1990

Leland Wilkinson, SYSTAT 6.0 for Windows: Statistics, Systat Inc., 1996