100 June 2005/Vol. 48, No. 6 COMMUNICATIONS OF THE ACM

COMMUNICATIONS OF THE ACM June 2005/Vol. 48, No. 6 101

A great deal has been written over the last four years regarding how academics have contributed possibletechnological solutions for uncoveringterrorist networks to enhance publicsafety and national security. Both the

public and the Pentagon have realized that knowledge ofthe structure of terrorist networks and how those net-works operate is one of the key factors in winning the so-called “netwar.” Probably the most critical weapons ourintelligence and law enforcement agencies should carryare reliable data and sophisticated techniques that helpdiscover useful knowledge from the data.

A new generation of data mining tools

and applicationswork to unearth

hidden patterns inlarge volumes of crime data.

Network Analysisand Visualization

uB y J e n n i f e r X u a n d H s i n c h u n C h e n

Criminal

102 June 2005/Vol. 48, No. 6 COMMUNICATIONS OF THE ACM

The study of terrorist networks falls into the largercategory of criminal network analysis, which is oftenapplied to investigations of organized crimes (forexample, terrorism, narcotics trafficking, fraud, gang-related crimes, armed robbery, and so on). Unlikeother types of crimes often committed by single or afew offenders, organized crimes are carried out bymultiple, collaborating offenders, who may formgroups and teams and play different roles. In a nar-cotics network, for instance, different groups may beresponsible for handling the drug supply, distribu-tion, sales, smuggling, and money laundering. In eachgroup, there may be a leader who issues commandsand provides steering mechanisms to the group, aswell as gatekeepers who ensure that information anddrugs flow effectively to and from other groups.Criminal network analysis therefore requires the abil-ity to integrate information from multiple crime inci-dents or even multiple sources and discover regularpatterns about the structure, organization, operation,and information flow in criminal networks.

To untangle and disrupt criminal networks, bothreliable data and sophisticated techniques are indis-pensable. However, intelligence and law enforcementagencies are often faced with the dilemma of havingtoo much data, which in effect makes too little value.On one hand, they have large volumes of “raw data”collected from multiple sources: phone records, bankaccounts and transactions, vehicle sales and registra-tion records, and surveillance reports, to name a few[9, 10]. On the other hand, they lack sophisticatednetwork analysis tools and techniques to utilize thedata effectively and efficiently.

Today’s criminal network analysis is primarily amanual process that consumes much human timeand efforts, thus has limited applicability to crimeinvestigation. Our objective here is to provide a datamining perspective for criminal network analysis. Wediscuss the challenges in data processing, review exist-ing network analysis and visualization tools, and rec-ommend the Social Network Analysis (SNA)approaches. Although SNA is not traditionally con-sidered as a data mining technique, it is especiallysuitable for mining large volumes of association datato discover hidden structural patterns in criminal net-works [9, 10]. We also report some data mining pro-jects for criminal network analysis in the COPLINKresearch, which is the NIJ- and NSF- funded researchfor management of law enforcement knowledge [3].

CHALLENGES IN DATA PROCESSING

Like data mining applications in many otherdomains, mining law enforcement data involvesmany obstacles. First, incomplete, incorrect, or

inconsistent data can create problems. Moreover,these characteristics of criminal networks cause dif-ficulties not common in other data mining applica-tions:

• Incompleteness. Criminal networks are covert net-works that operate in secrecy and stealth [8].Criminals may minimize interactions to avoidattracting police attention and their interactionsare hidden behind various illicit activities. Thus,data about criminals and their interactions andassociations is inevitably incomplete, causingmissing nodes and links in networks [10].

• Incorrectness. Incorrect data regarding criminals’identities, physical characteristics, and addressesmay result either from unintentional data entryerrors or from intentional deception by criminals.Many criminals lie about their identity informa-tion when caught and investigated.

• Inconsistency. Information about a criminal whohas multiple police contacts may be entered intolaw enforcement databases multiple times. Theserecords are not necessarily consistent. Multipledata records could make a single criminal appearto be different individuals. When seemingly dif-ferent individuals are included in a networkunder study, misleading information may result.

Problems specific to criminal network analysis liein data transformation, fuzzy boundaries, and net-work dynamics:

• Data transformation. Network analysis requiresthat data be presented in a specific format, inwhich network members are represented bynodes, and their associations or interactions arerepresented by links. However, information aboutcriminal associations is usually not explicit in rawdata. The task of extracting criminal associationsfrom raw data and transforming them to therequired format can be fairly labor-intensive andtime-consuming.

• Fuzzy boundaries. Boundaries of criminal net-works are likely to be ambiguous. It can be quitedifficult for an analyst to decide whom to includeand whom to exclude from a network understudy [10].

• Network dynamics. Criminal networks are not sta-tic, but are subject to changes over time. Newdata and even new methods of data collectionmay be required to capture the dynamics of crim-inal networks [10].

Some techniques have been developed to address

these problems. For example, toimprove data correctness andconsistency, many heuristics areemployed in the FinCEN systemat the U.S. Department of theTreasury to disambiguate andconsolidate financial transactionsinto uniquely identified individ-uals in the system [5]. Otherapproaches like the concept spacemethod [3] can transform crimeincident data into a networkedformat [12].

CRIMINAL NETWORK ANALYSIS AND

VISUALIZATION TOOLS

Klerks [7] categorized existingcriminal network analysisapproaches and tools into threegenerations.

First generation: Manualapproach. Representative of thefirst generation is the AnacapaChart [6]. With this approach,an analyst must first construct anassociation matrix by identifyingcriminal associations from rawdata. A link chart for visualiza-tion purposes can then be drawnbased on the association matrix.For example, to map the terroristnetwork containing the 19hijackers in the September 11 attacks, Krebs [8] gath-ered data about the relationships among the hijackersfrom publicly released information reported in severalmajor newspapers. He then manually constructed anassociation matrix to integrate these relations [8] anddrew a network representation to analyze the struc-tural properties of the network (Figure 1).

Although such a manual approach for criminalnetwork analysis is helpful in crime investigation, itbecomes an extremely ineffective and inefficientmethod when data sets are very large.

Second generation: Graphic-based approach.These tools can automatically produce graphical rep-resentations of criminal networks. Most existing net-work analysis tools belong to this generation. Amongthem Analyst’s Notebook [7], Netmap [5], andXANALYS Link Explorer (previously called Watson)[1], are the most popular. For example, Analyst’sNotebook can automatically generate a link chartbased on relational data from a spreadsheet or text file(Figure 2a).

Recently, two second-generation network analysis

approaches have been developed by the COPLINKresearch and its co-developer, the Knowledge Com-puting Corporation. The first approach employs ahyperbolic tree metaphor to visualize crime relation-ships [3]. It is especially helpful for visualizing a largeamount of relationship data because it simultane-ously handles both focus and context (Figure 2b).The second approach uses a spring embedder algo-rithm [4] to adjust positions of nodes automaticallyto prevent a network display from being too clut-tered. In such a view, icons represent different typesof entities. A filtering function allows a user to selectonly those entity types of interest. (Figure 2c–e).

Although second-generation tools are capable ofusing various methods to visualize criminal networks,their sophistication level remains modest becausethey produce only graphical representations of crim-inal networks without much analytical functionality.They still rely on analysts to study the graphs withawareness to find structural properties of the net-work.

Third generation: SNA. This approach isexpected to provide more advanced analytical func-tionality to assist crime investigation. Sophisticatedstructural analysis tools are needed to go from merelydrawing networks to mining large volumes of data todiscover useful knowledge about the structure andorganization of criminal networks.

COMMUNICATIONS OF THE ACM June 2005/Vol. 48, No. 6 103

Figure 1. The terrorist network

containing the 19 hijackers on Sept. 11, 2001.

DATA MINING PERSPECTIVE

Intelligence and law enforcementagencies are often interested in find-ing structural properties of criminalnetworks [9]:

• What subgroups exist in the network?

• How do these subgroups interactwith each other?

• What is the overall structure of thenetwork?

• What are the roles (central/periph-eral) network members’ play?

A clear understanding of thesestructural properties in a criminal net-work may help analysts target criticalnetwork members for removal or sur-veillance, and locate network vulnera-bilities where disruptive actions can beeffective. Appropriate network analysis

techniques, therefore, are needed tomine criminal networks and gaininsight into these problems.

SNA. Because SNA techniques aredesigned to discover patterns of interac-tion between social actors in social net-works [11], they are especiallyappropriate for studying criminal net-works [9, 10].

Specifically, SNA is capable of detect-ing subgroups, discovering their pat-terns of interaction, identifying centralindividuals, and uncovering networkorganization and structure.

Subgroup detection. A criminalnetwork can often be partitioned intosubgroups consisting of individuals whoclosely interact with each other. Given anetwork, traditional data mining tech-niques such as cluster analysis may beemployed to detect underlying group-ings that are not otherwise apparent in

104 June 2005/Vol. 48, No. 6 COMMUNICATIONS OF THE ACM

Figure 2. Second-generation criminal network analysis and

visualization tools. (a) Analyst’s Notebook

(i2 Inc.; www.i2inc.com). The system can automatically

arrange network nodes and allowsone to drag nodes around for

easier interpretation. (b) The hyperbolic tree view of

relations among multiple criminalentities. (c) The initial layout of acriminal network produced by the

network view (COPLINK Knowl-edge Computing Corp.; www.

knowledgecc.com). (d) The network layout is adjusted

automatically. The system movesthe nodes having the largest

number of links to the center ofthe display. A user can actually

see movements of the nodes while their positions are

adjusted and can fix the display atany time during the adjustment.(e) A user may choose only the

entity type of interest (for example, person) and view textual explanations (for example, person

name, address, and relations).

a b

c d e

the data. Hierarchical clustering methods have beenproposed to partition a network into subgroups [11].Cliques whose members are fully or almost fully con-nected can also be detected based on clustering results.

Discovery of patterns of interaction. Patterns ofinteraction between subgroups can be discoveredusing an SNA approach called blockmodeling [11],which was originally designed to interpret and vali-date theories of social structures. When used in crim-inal network analysis, it can reveal patterns ofbetween-group interactions and associations and canhelp reveal the overall structure of criminal networksunder investigation. Given a partitioned network,blockmodel analysis determines the presence orabsence of an association between a pair of subgroupsbased on a link density measure. In a network withundirected links, for example, the link densitybetween two subgroups i and j can be calculated by dij

= ninj, where mij is the actual number of links between

subgroups i and j; ni and nj represent the number ofnodes within subgroups i and j, respectively. Whenthe density of the links between the two subgroups isgreater than a predefined threshold value, a between-group association is present, indicating the two sub-groups interact with each other constantly and thushave a strong association. By this means, blockmodel-

COMMUNICATIONS OF THE ACM June 2005/Vol. 48, No. 6 105

Figure 3. The interaction patterns (and overall structures of networks) discovered from two criminal networks. (a) The

network consists of 60 criminals dealing with narcotic drugs. It isdifficult to manually detect subgroups and interaction patterns

from this original network. (b) A chain structure becomes apparent using clustering and blockmodeling (see the red mark).

Circles represent groups, which are labeled by their leaders’names, and straight lines represent between-group relationships.

(c) The system can also show the inner structure of a selectedgroup, identify its central members (leaders by degree,

gatekeepers by betweenness, and outliers by closeness), andpresents the centrality rankings of the members in a table in a

separate window. (d) The network consisting of 57 gang members. (e) The star structure found in the gang network. (f)

The details of a selected group in the gang network.

a

d

e

f

b

c

ing summarizes individual interaction details intointeractions between groups so the overall structure ofthe network becomes more prominent.

Centrality deals with the roles of individuals in anetwork. Several centrality measures, such as degree,betweenness, and closeness can suggest the impor-tance of a node in a network [11]. The degree of aparticular node is its number of links; its betweennessis the number of geodesics (shortest paths betweenany two nodes) passing through it; and its closeness isthe sum of all the geodesics between the particularnode and every other node in the network. An indi-vidual’s having a high degree, for instance, may implyhis leadership; whereas an individual with a highbetweenness may be a gatekeeper in the network.Baker and Faulkner [2] employed these three mea-sures, especially degree, to find the central individualsin a price-fixing conspiracy network in the electrical

equipment industry. Krebs [8] found that in the net-work consisting of the 19 hijackers, Mohamed Attascored the highest on degree and closeness, but not onbetweenness.

Implications. Effective use of SNA techniques tomine criminal network data can have importantimplications for crime investigations. For example,clustering with blockmodeling can help show the hid-den structure of a criminal network. The knowledgegained may aid law enforcement agencies fightingcrime proactively, for example, allocating an appro-priate amount of police effort to prevent a crime tak-ing place, or ensuring a police presence when thecrime is carried out [9]. Sometimes, new structuresdiscovered may even modify investigators’ conven-tional views of certain crimes. For instance, Klerks [7]has found the stereotypical impression of hierarchicalorganizations within organized crime is beingreplaced by an image of more fluid and flattened net-works. Traditional police strategies targeting leadersof a hierarchical criminal organization may havebecome less effective in fighting organized crimestoday. The work by Krebs also demonstrates that the

network consisting of the 19 hijackers in the Septem-ber 11 attacks is fairly flat and dispersed [8]. Theadvantage of such a structure is an increase in the net-work’s resilience and an emphasis on minimizingdamage should some network members be capturedor compromised.

SNA may also help address the challenges of dataprocessing. Blockmodeling, for example, can easilydetect “structural holes” [7] in which the link densityis lower than a threshold density value. According toMcAndrew [9], structural holes may indicate incom-plete or missing data thereby drawing analysts’ atten-tion to further data collection and improvement.

THE COPLINK RESEARCH TEST BED

Several data mining projects in the COPLINKresearch have begun to employ these SNA tech-niques for criminal network analysis. The goal has

been to provide law enforcement and intelligenceagencies with third-generation network analysistechniques that not only produce graphical repre-sentations of criminal networks but also providestructural analysis functionality to facilitate crimeinvestigations. Prior to these data mining activities,several methods were employed to address the chal-lenges of data processing. For inconsistency andincorrectness problems, we used the record linkagealgorithm to relate multiple database records thatactually refer to a single individual. For data trans-formation, we used the concept space approach [3]to extracting criminal associations from crime inci-dent data.

The first stage of our network analysis develop-ment was intended to automatically identify thestrongest association paths, or geodesics, between twoor more network members using shortest-path algo-rithms. In practice, such a task often entails crimeanalysts to manually explore links and try to findassociation paths that might be useful for generatinginvestigative leads. In the user study, our domainexpert evaluated the paths identified automatically

106 June 2005/Vol. 48, No. 6 COMMUNICATIONS OF THE ACM

Effective use of SNA techniques to mine criminalnetwork data can have important implications for crime

investigations. The knowledge gained may aid lawenforcement agencies fighting crime proactively.

and those identified manually. He considered the for-mer to be useful around 70% of time and the latter tobe useful around only 30% of time.

Extending this attempt, a more sophisticated sys-tem for mining criminal network data has been devel-oped. In addition to the visualization functionality,the system is intended to help detect subgroups in anetwork, discover interaction patterns betweengroups, and identify central members in a network.

Based on the crime incident data provided by theTucson Police Department, several networks consist-ing of criminals involved in different types of crimeshave been created and analyzed. Several domainexperts validated the results of analysis (for example,subgroups, leaders, and gatekeepers). Moreover, inter-esting patterns of interactions between criminalgroups and network structures were revealed in thenetworks. For example, a network of criminals dealingwith narcotics and a network of gang members werefound to have different structures, which became evi-dent after cluster and blockmodel analysis (Figure 3).It appears the chain-structure network (for example,the narcotics network), may be disrupted by removingany part of the chain. Removing the central membersfrom a star-structure network (for example, the gangnetwork) might cause fatal damage to it.

To evaluate the system’s performance we conducteda laboratory experiment involving 30 (student) sub-jects who performed 14 investigative tasks under twoexperimental conditions: structural analysis plus visu-alization (characteristics of third-generation tools),and visualization only (characteristics of second-gen-eration tools). These 14 tasks were divided into twotypes: identifying interaction patterns between sub-groups and identifying central members within agiven subgroup. Our main performance metrics wereeffectiveness (defined as the total number of correctanswers a subject generated for a given type of tasks)and efficiency (defined as the average time a subjectspent to complete a given type of tasks). The resultsshowed the average time spent on interaction patternidentification tasks under condition (a) (7.13 seconds)was significantly shorter than that under condition (b)(12.10 seconds; t = 6.92, p < 0.001). The difference inefficiency was also significant for central memberidentification tasks (6.24 seconds vs. 26.93 seconds; t= 10.66, p < 0.001). Such a gain in efficiency hasimportant implications because time is of the essencefor law enforcement and intelligence agencies seekingto prevent or respond to terrorist attacks or other seri-ous crimes. No significant improvement in effective-ness was present (t = 1.80, p > 0.05), probably due tothe small sizes and relatively simple structures of thetesting networks [12].

CONCLUSION

It is believed that reliable data and sophisticated ana-lytical techniques are critical for law enforcementand intelligence agencies to understand and possiblydisrupt terrorist or criminal networks. Using auto-mated SNA and visualization techniques to revealvarious structures and interactions within a networkis a promising step forward. The continued advance-ment of criminal network analysis techniques willenable us finally to win the new netwar.

References1. Anderson, T., Arbetter, L., Benawides, A., and Longmore-Etheridge, A.

Security works. Security Management 38, 17, (1994), 17–20.2. Baker, W.E. and Faulkner, R.R. The social organization of conspiracy:

Illegal networks in the heavy electrical equipment industry. Amer. Soci-ological Rev. 58, 2 (1993), 837–860.

3. Chen, H., Zeng, D., Atabakhsh, H., Wyzga, W., and Schroeder, J.COPLINK: Managing law enforcement data and knowledge. Commun.ACM 46, 1 (Jan. 2003), 28–34.

4. Eades, P. A heuristic for graph drawing. Congressus Numerantium 42(1984), 149–160.

5. Goldberg, H.G., and Senator, T.E. Restructuring databases for knowl-edge discovery by consolidation and link formation. In Proceedings of1998 AAAI Fall Symposium on Artificial Intelligence and Link Analysis.AAAI Press (1998).

6. Harper, W.R., and Harris, D.H. The application of link analysis topolice intelligence. Human Factors 17, 2 (1975), 157–164.

7. Klerks, P. The network paradigm applied to criminal organizations:Theoretical nitpicking or a relevant doctrine for investigators? Recentdevelopments in the Netherlands. Connections 24, 3 (2001), 53–65.

8. Krebs, V. E. Mapping networks of terrorist cells. Connections 24, 3(2001), 43–52.

9. McAndrew, D. The structural analysis of criminal networks. The SocialPsychology of Crime: Groups, Teams, and Networks, Offender ProfilingSeries, III. D. Canter and L. Alison (Eds.). Aldershot, Dartmouth(1999).

10. Sparrow, M.K. The application of network analysis to criminal intelli-gence: An assessment of the prospects. Social Networks 13 (1991),251–274.

11. Wasserman, S., and Faust, K. Social Network Analysis: Methods andApplications. Cambridge University Press, Cambridge, MA, 1994.

12. Xu, J., and Chen, H. CrimeNet explorer: A framework for criminal net-work knowledge discovery. To appear in ACM Trans. on Info. Systems.

This work has primarily been funded by the National Science Foundation, DigitalGovernment Program COPLINK Center: Information and Knowledge Managementfor Law Enforcement, #9983304, July, 2000–June, 2003.

Jennifer Xu (jxu@eller.arizona.edu) is a doctoral candidate in management information systems at the University of Arizona, Tucson, AZ.Hsinchun Chen (hchen@eller.arizona.edu) is McClelland Professor of Management Information Systems and head of ArtificialIntelligence Lab at the MIS Department at the University of Arizona,Tucson, AZ.

Permission to make digital or hard copies of all or part of this work for personal orclassroom use is granted without fee provided that copies are not made or distributedfor profit or commercial advantage and that copies bear this notice and the full citationon the first page. To copy otherwise, to republish, to post on servers or to redistributeto lists, requires prior specific permission and/or a fee.

© 2005 ACM 0001-0782/05/0600 $5.00

c

COMMUNICATIONS OF THE ACM June 2005/Vol. 48, No. 6 107