MAPPing Dark Networks:A Data Transformation Strategy for

Clandestine Organizations

Luke M. GerdesMinerva Fellow, United States Military Academy

Luke.Gerdes@usma.edu

Hamburg, Germany24 May 2013

The views expressed herein are those of the authors and do not purport to represent the official policy or position of the United States Military Academy, the Department of the Army, the Department of Defense, or the U.S. Government.

A Presentation for the 2013 INSNA Sunbelt Conference

Bottom Line Up Front

• Data transformation plays an important role in determining results

• Topical context/subject matter must be considered when determining a transformation strategy

• Data mis-management can cause sub-optimal performance in influence &/or decapitation campaigns

Basic Questions

• Why transform?– Necessary to

implement standard network measures• Multi-modal measures

(e.g. Faust 1997) not widely accepted• Multi-modal measures

non-intuitive

• How to transform?– Several approaches• Binary folding• One-way sums• Discount by size of

partnership (Newman)• Resource flow-based

approach (Zhou, et al)• Median Additive

Projection Process (MAPP)

Binary Folding

Weighted Folding: A Bad Idea

Weighted Data

One-Way Sums

Newman’s Method of Discounting

Zhou’s Binary Resource Flows

A Weighted Generalization of Zhou

• Data is always undirected

• Exact timing of interaction is unknown

• Maximum possible interactions between two agents equal to larger number of ties to event

• Minimum possible interactions between two agents equal to zero

Dick and Harry’s Participation in Event C

Assumptions About Dark Networks

Median Additive Projection Process

• 27 settings– Agents (25, 50, or 100)– Events (5, 10, 0r 20)– Density (.1, .25, or .33)

• 5 networks/setting– 135 2-mode networks

• 6 transformation processes/ 2-mode network– 810 1-mode networks

• 3 measures of centrality / 1-mode network– Opsahl’s 3rd generation measures (alpha = 0.5)– 2430 comparisons of rank– Spearman’s rho– Bonferroni’s correction for multiple

comparisons (by setting)

Evaluating Differences

Tenth Quarter ThirdBF OWS N BZ WZ MAPP BF OWS N BZ WZ MAPP BF OWS N BZ WZ MAPP

50 x 5

BF 1 0.87 0.77 0.99 0.90 0.87 1 0.97 0.93 0.99 0.98 0.95 1 0.98 0.97 0.99 0.96 0.97

OWS ***** 1 0.95 0.85 0.89 0.82 ***** 1 0.98 0.97 0.97 0.92 ***** 1 0.99 0.97 0.96 0.95N **** ***** 1 0.74 0.81 0.76 ***** ***** 1 0.93 0.93 0.90 ***** ***** 1 0.97 0.97 0.94BZ ***** ***** *** 1 0.90 0.86 ***** ***** ***** 1 0.99 0.94 ***** ***** ***** 1 0.97 0.96WZ ***** ***** **** ***** 1 0.69 ***** ***** ***** ***** 1 0.90 ***** ***** ***** ***** 1 0.91

MAPP ***** ***** ***** ***** *** 1 ***** ***** ***** ***** ***** 1 ***** ***** ***** ***** ***** 1

50 x 10

BF 1 0.97 0.92 0.99 0.95 0.97 1 0.99 0.97 0.99 0.98 0.98 1 0.99 0.98 0.98 0.97 0.98

OWS ***** 1 0.96 0.96 0.95 0.96 ***** 1 0.98 0.98 0.98 0.97 ***** 1 0.99 0.98 0.97 0.98N ***** ***** 1 0.90 0.88 0.91 ***** ***** 1 0.96 0.96 0.96 ***** ***** 1 0.97 0.96 0.97BZ ***** ***** ***** 1 0.96 0.96 ***** ***** ***** 1 0.99 0.97 ***** ***** ***** 1 0.99 0.96WZ ***** ***** ***** ***** 1 0.89 ***** ***** ***** ***** 1 0.95 ***** ***** ***** ***** 1 0.94

MAPP ***** ***** ***** ***** ***** 1 ***** ***** ***** ***** ***** 1 ***** ***** ***** ***** ***** 1

50 x 20

BF 1 0.98 0.94 0.98 0.96 0.98 1 0.99 0.98 0.97 0.95 0.98 1 0.99 0.97 0.91 0.88 0.97

OWS ***** 1 0.96 0.96 0.95 0.98 ***** 1 0.99 0.97 0.95 0.98 ***** 1 0.98 0.91 0.88 0.97N ***** ***** 1 0.90 0.88 0.94 ***** ***** 1 0.94 0.92 0.97 ***** ***** 1 0.87 0.85 0.95BZ ***** ***** ***** 1 0.98 0.96 ***** ***** ***** 1 0.98 0.95 ***** ***** ***** 1 0.94 0.90WZ ***** ***** ***** ***** 1 0.93 ***** ***** ***** ***** 1 0.91 ***** ***** ***** ***** 1 0.85

MAPP ***** ***** ***** ***** ***** 1 ***** ***** ***** ***** ***** 1 ***** ***** ***** ***** ***** 1

Results for 50 Agents: Degree

Each star (*) represents a test that was significant at the 0.05 level

Results for 50 Agents: Closeness

Tenth Quarter ThirdBF OWS N BZ WZ MAPP BF OWS N BZ WZ MAPP BF OWS N BZ WZ MAPP

50 x 5

BF 1 0.59 0.54 0.94 0.64 0.81 1 0.56 0.48 0.80 0.58 0.83 1 0.65 0.60 0.87 0.63 0.91OWS ** 1 0.95 0.56 0.94 0.89 ***** 1 0.94 0.49 0.96 0.87 ***** 1 0.96 0.54 0.96 0.87N * ***** 1 0.55 0.95 0.86 *** ***** 1 0.53 0.96 0.81 **** ***** 1 0.57 0.98 0.83BZ ***** * * 1 0.66 0.76 ***** ** ***** 1 0.59 0.74 ***** *** *** 1 0.62 0.79WZ **** ***** ***** *** 1 0.89 ***** ***** ***** ***** 1 0.86 ***** ***** ***** ***** 1 0.84MAPP **** ***** ***** ***** ***** 1 ***** ***** ***** ***** ***** 1 ***** ***** ***** ***** ***** 1

50 x 10

BF 1 0.71 0.62 0.87 0.69 0.88 1 0.69 0.64 0.91 0.71 0.93 1 0.80 0.77 0.94 0.79 0.96OWS ***** 1 0.94 0.67 0.95 0.88 ***** 1 0.93 0.66 0.95 0.87 ***** 1 0.96 0.78 0.97 0.91N ***** ***** 1 0.67 0.96 0.82 ***** ***** 1 0.71 0.98 0.81 ***** ***** 1 0.80 0.99 0.88BZ ***** ***** ***** 1 0.71 0.84 ***** ***** ***** 1 0.76 0.87 ***** ***** ***** 1 0.82 0.92WZ ***** ***** ***** ***** 1 0.87 ***** ***** ***** ***** 1 0.87 ***** ***** ***** ***** 1 0.89MAPP ***** ***** ***** ***** ***** 1 ***** ***** ***** ***** ***** 1 ***** ***** ***** ***** ***** 1

50 x 20

BF 1 0.73 0.62 0.88 0.74 0.91 1 0.83 0.80 0.95 0.82 0.96 1 0.80 0.79 0.97 0.79 0.96OWS ***** 1 0.88 0.65 0.92 0.89 ***** 1 0.96 0.81 0.97 0.93 ***** 1 0.95 0.79 0.96 0.91N ***** ***** 1 0.68 0.91 0.77 ***** ***** 1 0.83 0.98 0.90 ***** ***** 1 0.82 0.99 0.89BZ ***** ***** ***** 1 0.78 0.77 ***** ***** ***** 1 0.85 0.94 ***** ***** ***** 1 0.82 0.94WZ ***** ***** ***** ***** 1 0.84 ***** ***** ***** ***** 1 0.91 ***** ***** ***** ***** 1 0.89MAPP ***** ***** ***** ***** ***** 1 ***** ***** ***** ***** ***** 1 ***** ***** ***** ***** ***** 1

Each star (*) represents a test that was significant at the 0.05 level

Results for 50 Agents: Betweenness

Tenth Quarter ThirdBF OWS N BZ WZ MAPP BF OWS N BZ WZ MAPP BF OWS N BZ WZ MAPP

50 x 5

BF 1 0.95 0.95 1.00 0.77 0.87 1 0.76 0.75 0.93 0.64 0.81 1 0.65 0.64 0.84 0.51 0.70

OWS ***** 1 1.00 0.95 0.82 0.87 ***** 1 0.99 0.75 0.84 0.89 ***** 1 0.97 0.54 0.67 0.85

N ***** ***** 1 0.95 0.82 0.86 ***** ***** 1 0.74 0.83 0.89 ***** ***** 1 0.56 0.68 0.81

BZ ***** ***** ***** 1 0.77 0.87 ***** ***** ***** 1 0.64 0.78 ***** **** ***** 1 0.48 0.57

WZ ***** ***** ***** ***** 1 0.86 ***** ***** ***** ***** 1 0.83 *** ***** ***** *** 1 0.59

MAPP ***** ***** ***** ***** ***** 1 ***** ***** ***** ***** ***** 1 ***** ***** ***** *** ***** 1

50 x 10

BF 1 0.99 0.97 1.00 0.91 0.93 1 0.78 0.77 0.75 0.71 0.81 1 0.80 0.79 0.78 0.61 0.81

OWS ***** 1 0.99 0.98 0.91 0.95 ***** 1 0.95 0.61 0.80 0.90 ***** 1 0.98 0.65 0.79 0.92

N ***** ***** 1 0.97 0.90 0.93 ***** ***** 1 0.66 0.83 0.88 ***** ***** 1 0.66 0.80 0.90

BZ ***** ***** ***** 1 0.91 0.93 ***** ***** ***** 1 0.65 0.64 ***** ***** ***** 1 0.61 0.65

WZ ***** ***** ***** ***** 1 0.92 ***** ***** ***** ***** 1 0.76 ***** ***** ***** ***** 1 0.74

MAPP ***** ***** ***** ***** ***** 1 ***** ***** ***** ***** ***** 1 ***** ***** ***** ***** ***** 1

50 x 20

BF 1 0.93 0.90 0.94 0.85 0.91 1 0.87 0.83 0.79 0.59 0.88 1 0.81 0.79 0.71 0.54 0.87

OWS ***** 1 0.97 0.88 0.88 0.94 ***** 1 0.96 0.74 0.75 0.91 ***** 1 0.91 0.66 0.68 0.87

N ***** ***** 1 0.90 0.89 0.92 ***** ***** 1 0.77 0.80 0.89 ***** ***** 1 0.72 0.74 0.83

BZ ***** ***** ***** 1 0.85 0.86 ***** ***** ***** 1 0.62 0.73 ***** ***** ***** 1 0.55 0.65

WZ ***** ***** ***** ***** 1 0.89 ***** ***** ***** ***** 1 0.72 ***** ***** ***** ***** 1 0.65

MAPP ***** ***** ***** ***** ***** 1 ***** ***** ***** ***** ***** 1 ***** ***** ***** ***** ***** 1

Each star (*) represents a test that was significant at the 0.05 level

. . . And When Considering ‘Top’ Agents (Degree)

Tenth Quarter ThirdBF OWS N BZ WZ MAPP BF OWS N BZ WZ MAPP BF OWS N BZ WZ MAPP

50 x 5

BF 1 0.83 0.61 0.90 0.56 0.77 1 0.67 0.35 0.96 0.83 0.41 1 0.73 0.25 0.69 0.36 0.75

OWS * 1 0.79 0.73 0.60 0.68 * 1 0.77 0.63 0.66 0.09 * 1 0.60 0.49 0.54 0.47

N ** 1 0.41 0.23 0.69 ** 1 0.31 0.32 -0.12 * * 1 0.58 0.81 0.10

BZ **** 1 0.66 0.67 *** * 1 0.90 0.30 ** * * 1 0.69 0.46

WZ * 1 0.25 * ** * * 1 0.05 * * * * 1 0.06

MAPP * * 1 * ** * * ** 1 1

50 x 10

BF 1 1.00 0.35 0.93 0.78 0.84 1 0.86 0.67 0.77 0.47 0.72 1 0.92 0.69 0.41 0.19 0.83

OWS **** 1 0.37 0.90 0.75 0.86 * 1 0.85 0.60 0.46 0.54 ** 1 0.81 0.34 0.26 0.72

N * * 1 0.30 0.17 0.35 * 1 0.34 0.26 0.35 * * 1 0.47 0.46 0.48

BZ ** ** * 1 0.89 0.71 1 0.84 0.51 1 0.64 0.46

WZ ** ** * *** 1 0.53 * * 1 0.09 1 -0.03

MAPP ** ** ** * * 1 * 1 1

50 x 20

BF 1 0.75 0.24 0.82 0.68 0.77 1 0.87 0.77 0.25 -0.19 0.83 1 0.96 0.87 -0.15 -0.19 0.83

OWS * 1 0.29 0.68 0.72 0.72 * 1 0.85 0.45 0.01 0.76 ** 1 0.91 -0.24 -0.19 0.81

N * ** 1 -0.01 -0.18 0.53 1 0.20 -0.13 0.68 * ** 1 -0.36 -0.22 0.67

BZ *** * * 1 0.86 0.49 1 0.42 0.32 1 0.48 -0.09

WZ ** ** ** ** 1 0.33 1 -0.11 1 -0.24

MAPP ** * * ** * 1 * 1 * * 1

Each star (*) represents a test that was significant at the 0.05 level

• Data transformation is not trivial– Different processes produce different rankings– Different processes select different actors as highly central nodes

• Degree more robust than closeness & betweenness– None of the measurements robust in selection of ‘top’ agents

• No means to determine best performance– Determination would require comparison against 1-mode network built

through direct observation– Selection of method must be rooted in theory– Different methods appropriate to different topics

• MAPP the most appropriate to dark networks

Conclusions on Data Transformation

Policy Implications

• Countering dark networks– Projection processes determine decapitation &

influence strategies

– Conventional wisdom (i.e. binary folding) likely leads to less-effective interventions• Removal of the wrong people• Mis-targeted campaigns to influence ‘key’ opinion-

makers