Multidimensional Data Dissection Using Attribute Relationship GraphsChris Weaver∗

School of Computer Science and Center for Spatial AnalysisThe University of Oklahoma

ABSTRACT

Visual exploration and analysis is a process of discovering and dis-secting the abundant and complex attribute relationships that per-vade multidimensional data. Recent research has identified andcharacterized patterns of multiple coordinated views, such as cross-filtered views, in which rapid sequences of simple interactions canbe used to express queries on subsets of attribute values. In visual-izations designed around these patterns, for the most part, distinctviews serve to visually isolate each attribute from the others. Al-though the brush-and-click simplicity of visual isolation facilitatesdiscovery of many-to-many relationships between attributes, dis-secting these relationships into more fine-grained one-to-many re-lationships is interactively tedious and, worse, visually fragmentedover prolonged sequences of queries.

This paper describes: (1) a method for interactively dissect-ing multidimensional data by iteratively slicing and manipulating amultigraph representation of data values and value co-occurrences;and (2) design strategies for extending the construction of coordi-nated multiple view interfaces for dissection as well as discoveryof attribute relationships in multidimensional data sets. Using ex-amples from different domains, we describe how attribute relation-ship graphs can be combined with cross-filtered views, modularizedfor reuse across designs, and integrated into broader visual analysistools. The exploratory and analytic utility of these examples sug-gests that an attribute relationship graph would be a useful additionto a wide variety of visual analysis tools.

Index Terms: D.2.2 [Software Engineering]: Design Toolsand Techniques—[User Interfaces]; H.2.3 [Information Systems]:Database Management—[Languages]; H.5.2 [Information Sys-tems]: Information Interfaces and Presentation—[User Interfaces]

1 INTRODUCTION

Many data sets of interest in visual analytics [27] come in the formof one or more multidimensional relational tables. These data setsoften include a mix of geospatial, temporal, numerical, and categor-ical information. Nonetheless, they tend to be predominantly nom-inal in nature, consisting of multiple dimensions of people, places,events, institutions, things, ideas, and other kinds of named entitiesand groups that represent the who and the what of social, political,legal, educational, financial, and other complex human systems.

Graph visualization has become an extraordinarily popularmeans to explore and analyze the many relationships that exist be-tween the entities and groups in such systems. Our comprehen-sion of the world around us, from the material to the conceptual, ishighly associative. Consequently, the information representationsinto which we map our understanding of the world tend to be full ofrelationships, even if those relationships are not always explicit indata. Graph visualization is effective because it can explicitly rep-resent many-to-many relationships in perceptually immediate, cog-nitively direct, and interactively accessible ways. The visualization

∗e-mail: weaver@cs.ou.edu

community continues to develop many useful and usable techniquesfor displaying and interacting with objects and their relationships.The real challenge, however, lies in providing ways to deeply ex-plore and analyze the often tangled web of implicit relationshipsthat the complex human processes of naming and measurement andorganization infuse into every seemingly mundane data set.

With visualization as a discipline continuing to mature from craftto science, there is increasing interest in discovering patterns andpathways for synthesizing “visual-interactive compounds” that aredemonstrably useful for data exploration and analysis in both broadand targeted applications. This paper describes one such pattern,attribute relationship graphs, that allows analysts to dissect mul-tidimensional relationships by both drilling down into and slicingacross attributes in order to expose the many-to-many details of at-tribute value co-occurrences spread across one or more data sets.

Attribute relationship graphs extend the cross-filtered viewstechnique [33] by adding a node-edge-hyperedge diagram that sup-ports flexible exploration of attribute value co-occurrences acrosstwo or more columns of tabular data, using a multigraph representa-tion. This representation can be interactively filtered on a selectionof columns as well as on selections of arbitrary subsets of valuesin those columns—essentially a fully filterable mapping of the toptwo levels of an N-dimensional data cube into nodes, edges, andhyperedges (visually encoding the latter as node-containing packs).

The general structure of a typical attribute relationship graphdesign consists of well-known visualization components: dimen-sionally appropriate views for brushing the unique values of eachattribute; checkboxes for toggling filtering between any directedpair of these views; a graph view that shows values and value co-occurrences; and checkboxes that toggle filtering of nodes for eachattribute, edges for each undirected attribute pair, and hyperedgesfor each directed attribute pair. This structure supports a process inwhich trained analysts can follow complex lines of inquiry by us-ing sequences of simple, freely interleaved interactions to performspecific visual data exploration and analysis tasks.

We start by describing an example in which an attribute relation-ship graph is used for visual exploration of recent award-winningmovies in the Internet Movie Database. After summarizing relatedwork on multidimensional visual data analysis and graph visualiza-tion, we describe the attribute relationship graph technique and ageneral method for designing visualizations of tabular data sets us-ing it. Using additional examples from two other data domains, weproceed to describe how attribute relationship graphs can be flexiblycustomized with respect to data types across multiple dimensions,incorporated into more wide-ranging multiple view designs, andmodularized for reuse across designs. We conclude by consideringanalytic utility and usability in terms of the perceptual and inter-active characteristics of the graph view as an extension of cross-filtering, practical restrictions on query expressiveness, scalabilityand interactive performance, and future directions.

2 EXAMPLE: CINEGRAPH

The Internet Movie Database (IMDB) is a popular web site forbrowsing and searching information about movies, including theactors and crew members who make them, the genres they belongto, when they were released to theaters, the awards they have won,

75

IEEE Symposium on Visual Analytics Science and Technology October 24 - 28, Salt Lake City, Utah, USA 978-1-4244-9486-6/10/$26.00 ©2010 IEEE

DAA

G

J

K

OP

BI

C

LE

FN

HM

Figure 1: The Cinegraph visualization of recent movies in the Internet Movie Database. The attribute relationship graph shows correspondencesbetween awards and award-winning movies that took in less than $100M at the box office. The graph simultaneously shows that only one personinvolved with at least one of these movies has been both a director and a cinematographer, although not necessarily of the award winners.

and the ratings that online fans have given them. This large andcomplex mix of social, attitudinal, relational, and temporal data isstrongly analogous to many data sets of central interest in visual an-alytics. Moreover, popular interest in and familiarity with moviesmakes exploration and analysis of IMDB an exemplary target formore broadly accessible applications of new visualization methods.However, current tools typically allow users to ask at most a fewcanned questions about two or three dimensions of the data, typi-cally with interactive manipulation of only one of those dimensions.

Open-ended exploration also demands visual tools and tech-niques that can handle not only a broad range of data volumes, butalso an increasingly large number of interrelated data dimensionsthat involve a rich mix of data types with complex, domain-specificsemantics. The IMDB subset created for the InfoVis 2007 contest,for instance, describes only mainstream movies from 2001 to 2007,yet occupies four tables that contain 20204 records about movies,ratings, and release dates, 246755 records about people and roles,35384 records about genres, and 43 records about awards, respec-tively. Movies typically have ten or more people, many of whomfill multiple roles, and three or more genres. However, the distri-bution of data value occurrences and many-to-many relationshipsvaries extremely. Exploring data of this sort requires simultaneousagility, flexibility, expressiveness, and expansiveness of interaction.

We used Improvise [31] to develop Cinegraph [32], an interac-tive visual tool for exploring and analyzing the IMDB subset. Thetool maps nine attributes of the movies database—movies, genres,people, roles, awards, release dates, average ratings, number of rat-ings, and box office totals—into a collection of multiple coordi-nated views laid out in a single-window user interface (Figure 1).A scatter plot matrix displays the rating count, rating average, andbox office attributes together, each plotted against the other two, asa single ratings “dimension”. A calendar shows release dates. Theremaining five attributes populate table views. Cross-filtering con-nects these dimensions, allowing users to drill-down by turning onand off brushing of arbitrary sets of data values in those dimensions.

Figure 1 shows an example of one point in an exploration of

awards. After (A) selecting all awards, the user (B) filters genres,movies, people, and roles on awards to get a general sense of thenumber, distribution, and specific instances of “winners” in eachof these dimensions. Being particularly interested in cinematogra-phers and directors, the user (C) selects these two roles, then (D)further filters people on roles. Wondering which people have filledwhich roles, the user (E) turns on both kinds of nodes in the attributerelationship graph, then (F) the edges that connect them, revealingthat (G) only Steven Soderbergh has been both a director and a cin-ematographer in an award-winning movie. Turning on (H) packsaround people for each role shows this even more clearly. Nowcurious about relatively low-budget movies, the user (I) turns onfiltering of movies by rating, then (J) rubberbands to ultimately se-lect movies that made less than $100M at the box office; the moviestable view (K) dynamically updates during the navigation. Turningon (L) movie and award nodes, then (M) packs of movies for eachaward, the user can see which movies have some award in common,but not which one. Turning on (N) edges reveals the many-to-manyrelationships between movies and awards, including that (O) Mys-tic River won awards for both leading actor and supporting actor,but that (P) none of the selected movies has won best picture.

Using the attribute relationship graph, the user can drill into ar-bitrary subsets of movies, people, and their characteristics, thenslice across attributes to dissect many-to-many details about kindsof movies, groups of people, and even patterns of moviemakingover time. As a result, the user can explore local regions of com-plex associations between attribute values much more efficientlythan through exhaustive sequential construction of the one-to-manyvisual states that are frequently required using a cross-filtered de-sign alone. Furthermore, like cross-filtering, attribute relationshipgraphs generalize well to tabular data formats and thus appear suit-able for template-based rapid visualization design. Nevertheless,Cinegraph was designed and built by hand and applied to the In-foVis 2007 contest data set in under a week, with only two daysneeded for live design including implementation of both cross-filtering and the attribute relationship graph in seven dimensions.

76

3 RELATED WORK

Attributed relational graphs [5] have long been used for defining vi-sual languages [12]. Visualization of attributed relational data hasalso been noted as important [14], but has been largely overlookedfrom a systematic perspective, despite many examples of domain-specific visual tools that include some sort of interactive multi-graph representation. We make an important distinction betweenattributed relational graphs (also known as attributed graphs), inwhich an object is related to its attributes (characteristics) throughdirected edges, and an attribute relationship graph, in which the val-ues of database attributes (column values) are related through undi-rected edges that indicate co-occurrences in records. This distinc-tion is tricky when one interprets each data record to be a collectivedescription of some attributed object. We focus on co-occurrencesbetween attributes rather than between an entire record and its at-tributes. This difference has a profound impact on the data trans-formations, visual representations, and ultimately analytic utility ofattribute relationship graphs. (In this paper, we use “attribute rela-tionship graph” to denote both the design pattern and its instances.)

Well-known graph visualization toolkits, such as GraphViz [8]and JUNG [21], provide extensive options for reading, drawing,and navigating graphs in a scalable manner. These toolkits canbe used to define topology, encode data values, support spatialnavigation and selection, and perform dynamic layout. Interac-tive manipulation of graphs remains largely limited to interac-tion in graphs, including panning, zooming, rotation, brushing ofnodes, expand/collapse of subgraphs, and automatic and manuallayout [35]. The CGV system [28] provides a comprehensive setof such interaction and filtering capabilities, including basic coor-dination of attributed graphs with other views. The visual repre-sentation of graphs has also been considered in terms of a pipelineof data transformations [15, 16, 26]. The InfoVis Toolkit [6] andprefuse [10] used flattened tabular representations of graphs andtrees, thereby unifying the transformations used for graph viewsand other views under the data state model [3]. However, these ap-proaches provide at most a few fixed entry points for introducingprescribed kinds of coordinated interaction into the transformationpipeline, making them well-suited for visualizing graphs but less sofor general use in multiple coordinated view designs.

Multiple coordinated view visualizations draws on a variety ofdesign patterns [7], some general [18] and some specific to visual-ization [9]. A variety of classic and recent visual analysis tools usemultiple coordinated views to support drill-down into relationshipsin data, using multidimensional brushing in scatter plot matricesin XmdvTool [29], interactively filtering of a central view by se-lecting subranges of data attribute values in sliders in Spotfire [1],attribute aggregation polyarchies [4], elastic hierarchies [37], rank-ing in social network graphs [22], ontologically annotated semanticgraphs [36], a hybrid of node-edge diagrams and matrices in Node-Trix [13], and combinations of parallel coordinate plots with node-edge diagrams [24] and cross-linked space-filling hierarchies [23].In developing attribute relationships graphs, we are interested notonly in adding to the space of existing multidimensional visualanalysis tools, but also in identifying and formalizing the higher-order patterns of coordinated view composition that make thesetools useful, usable, and generalizable across application domains.

ShowMe [19] (based on Tableau) is an exploratory visualiza-tion builder based on drag-and-drop editing of attribute hierarchiesacross scales. The combination of cross-filtering and attribute rela-tionship graph is similar to multiscale visualization in ShowMe inhow it is based on rapid interactive composition of complementaryaspects of a central view, but differs in three ways. First, it providesa means to explore the details of many-to-many associations at theground level of individual records, rather than the patterns of asso-ciation that occur at higher levels of aggregation. Second, it allowsthe inclusion of dynamically derived attributes. Third, its data and

visual abstraction operations can be parameterized in terms of ad-ditional external interactions, allowing design variations in supportof special data types or analysis requirements.

Jigsaw [25] combines a variation of cross-filtering, in the formof value selection in cross-linked lists of entity types in documents,with a node-edge diagram specifically designed for exploring themany-to-many connections between those entities through the doc-uments in which they appear. As a kind of attributed relationalgraph, the Jigsaw graph allows expansion and collapse of the entitynodes surrounding any given document node. Consequently, thegraph provides an additional means of domain-specific drill-downinteraction, in contrast with the general slicing interaction providedby node/edge/pack toggling in attribute relationship graphs.

Visualization schemas [20] models coordinated interaction be-tween pairs of views in terms of one-to-one, one-to-many, andmany-to-many relationships between data attributes. Any twoviews can coordinate, using a compound join, to display associa-tions between data records in terms of their attributes. Cross-filteredviews by itself can be thought of as a visualization schema in-volving independent, switchable, many-to-many filtering on selec-tions between pairs of views. Attribute relationship graphs extendcross-filtering by making the one-to-one and one-to-many detailsof many-to-many relationships visually explicit within a centralizedtarget that reflects drilled subsets of values only within a sliced sub-set of dimensions. The approach generalizes across application do-mains but allows substantial customization of data transformations,visual representations, and interactions in response to the particulardata characteristics and analytic needs of those domains.

4 ATTRIBUTE RELATIONSHIP GRAPHS

In this section, we consider attribute relationship graphs from threedifferent perspectives: (1) as a method for interactively dissectingmany-to-many relationships in multidimensional data, by buildingand modifying a graph that shows selected values (in selected di-mensions) as nodes and selected classes of value co-occurrencesas edges and hyperedges; (2) as a general pattern for constructingan interdependent set of data transformation operations that sup-port the method; and (3) as an open-ended space of design varia-tions for instantiating the pattern in particular visual analysis ap-plications, including the encoding and filtering of data values andco-occurrences as graph elements. We describe each perspectivewith reference to examples of dissection in visualizations designedto support analysis in three different information domains.

4.1 Interaction As DissectionExploring data has often been compared with looking for needlesin a stack of needles. When exploring heterogeneously typed mul-tidimensional data, it is more like looking for hidden gems in alarge box full of miscellaneous buttons, baubles, screws, rubber-bands, and other objects of various shapes, sizes, and colors, un-tangling them from each other, and trying to understand how theycame to be in the same box. Looking, untangling, and understand-ing all rely on the analyst’s perception, knowledge, and experienceto drive choices about which potential relationships between ob-jects and their characteristics are worth considering over the courseof exploration and analysis. The corresponding goal of attribute re-lationship graphs is to facilitate the identification and detailed char-acterization of many-to-many relationships in multidimensional in-formation through visual interaction. As such, it extends the cross-filtered views method [33] by using visual interaction to ask evenmore detailed questions through dissection rather than mere obser-vation of bulk correspondences between data item characteristics.

Visualizations built around an attribute relationship graph add ashell of queries, views, and interactions to an existing cross-filtereddesign. This additional structure is made up of two essential ele-ments: a graph view, and a collection of switches for manipulat-

77

F

N

MB

C

DAA

K

L

E

HI

J

O

P

QG

a b c

d

Figure 2: Dissection of visualization-related CAREER awards from the NSF CISE directorate from 2000 to early 2010. The attribute relationshipgraph progressively shows: (a) all principal investigators (PIs) in visualization-related programs, (b) only the CAREER PIs, (c) all other programsunder which their CAREER awards fall, and finally (d) only these other programs. A yellow-to-red color scale encodes total award amounts.

ing its contents. The graph view displays a multigraph of attributevalue co-occurrences by mapping each unique value into a node,each set of common co-occurrences into an edge, and the set ofco-occurrence sets of any given value into a hyperedge. Nodes,edges, and hyperedges fall into equivalence classes according to theattributes, combination of attributes, and permutation of attributesthat define them, respectively. Each switch toggles the visibility ofa single equivalence class of graph element. Graph exploration isthus like dissection: examine data items for potential relationshipsby drilling into and slicing across dimensions to expose a specificcross-section. The example visualizations in Figures 1,2,5 use vari-ations of this structure to support exploration in different domains.

Dissection as a mechanical process is nothing more than a se-quence of freely interleaved brushing, switching, and graph layoutinteractions provoked by observation of visible values in the graphand the cross-filtering views. For example, Figure 2 shows dissec-tion of the cross-section of NSF CAREER awards related to visu-alization, built as follows. First, (A) select the visualization-relatedprograms, (B) filter PIs on programs, then (C) select all PIs. Turnon (D) Program and PI nodes, (E) Program-PI edges, and (F) PI-Program packs. (G) Apply manual and automatic layout as desired.(The generic force-directed algorithm alone does a reasonable job,but manual tweaking helps.) Figure 2a shows the graph at this point.

Next, (H) filter award IDs on programs. In the Awards view,(I) sort by title, then (J) range select all titles that start with “CA-REER”. (K) Filtering on award IDs shows how CAREER awardsdistribute over NSF organizations, managers, programs, states, in-stitutions, and PIs; CoPIs can be ignored because CAREER awardsdon’t have any. (L) Again select all PIs. The graph now shows onlyPIs who have visualization-related CAREER awards (Figure 2b).

Now, (M) filter programs on PIs. Only the Graphics & Visualiza-tion program remains of the three, meaning that every award is inthis program. (N) Again select all programs, then relayout the graph(Figure 2c). Knowing that all PIs are in Graphics & Visualization,(O) deselect it, thereby removing its node and associated links tomake the graph less dense. What remains is a graph showing onlythe “non-visualization” programs for each award (Figure 2d). Italso reveals that (P) only three awards are solely in Graphics & Vi-sualization, and others are in special programs like (Q) EPSCoR.

At this point, the analyst could, for instance, find out which statethe EPSCoR award is from by picking the award’s PI and filteringStates on PI to discover the answer, New Mexico. Alternately, theanalyst could select all States (already CAREER-filtered), turn onState nodes and State-PI edges, then look at the many-to-many rela-tionships to dissect nominal geographic relationships, also referringto the map to gain insight into spatial geographic relationships.

78

Packs

NodesEdges

SlicingCliquing

!"

!#

G $# G’## # N#

C $ ’#" #"

T’#

T’"

C#"

N

#"P P#"

"#P P"#

#"E E#"

%#T’#

id id# "

&N'N !N$NTinput

N Tgraph

N Tglyph

N Tview

N

&E'E !E$ETgraph

E Tglyph

E Tview

ETinput

E

&P'P !P$PTgraph

P Tglyph

P Tview

PTinput

P

(

1 2

4

5 6

7

8

9

?

?

?

?)!

3

$

$

$

$

Collecting Forming Encoding Filtering Brushing

Layout

*

*

*

Grouping

Drilling

cross-filteringmatrix

node, edge, packmatrices

Figure 3: Data transformation pipeline for attribute relationship graphs. Left: The analyst specifies graph elements by slicing off particular datadimensions and drilling into those dimensions to select grouped/cliqued data values. Right: The data values are collected and formed into agraph topology that is visually encoded and optionally filtered. The analyst can layout and brush elements in the resulting graph representation.

4.2 Queries

Attribute relationship graphs are built upon a general model ofdata abstraction, visual abstraction, and coordination for visualizinggraphs. The model is a specialization of the data state model [3] inwhich graph data structures and their visual representations are de-clared in terms of interactively driven tabular data transformations(Figure 3, right). Under the model, designers specify the topol-ogy, visual encoding, filtering, layout, and brushing of graph viewsusing well-known data abstraction, visual abstraction, and coordi-nation mechanisms to specify content, appearance, and behavior.

A series of transformation pipelines forms (ψ), encodes (π), fil-ters (φ ), and handles brushing (σ ) for each graph element. Inter-dependencies between pipelines enforce topological constraints; alledges require two anchor nodes (possibly the same), and all packsrequire at least one member node. In the node pipeline (N), an addi-tional transformation calculates the position of nodes as a functionof current visual state using a combination of interactive (λ ) and au-tomatic (Λ) layout. In the edge pipeline (E), the transformations oneach edge have access to the data and visual abstractions of its an-chor nodes. Similarly, the transformations on each pack in the packpipeline (P) have access to the abstractions of its member nodes.This allows encoding of edges and packs in terms of node encod-ings, such as to draw edges as arrows that connect node shapes ina visually appealing way, regardless of changing positions of nodesduring dynamic layout. This also allows encoding in terms of topol-ogy, such as to filter edges on in-degree or packs on member count.

Every transformation is a potential entry point for coordination;calculation of transformations (1−9), interactive brushing and lay-out (?), and automatic layout (∞) are all concurrent and update dy-namically during coordinated interaction to or from the graph. It isoften necessary to exploit only a few of the data transformation andinteraction entry points in the model to design a graph tailored to aparticular set of desirable exploration and analysis capabilities.

Collection of the three data sets for input into the three pipelinesis not specified by the model, and can consist of any transformationsthat the visualization designer chooses to use for preprocessing.This flexibility of graph specification makes it relatively straightfor-ward to adapt the results of cross-filtering for use as input into thegraph (Figure 3, left). In cross-filtered visualizations, data transfor-mation consists of: grouping records into sets (G) for each uniqueattribute value; filtering each set, keeping records whose attributevalues co-occur with those brushed in a subset of other views; vi-sually encoding each value and its filtered set; and brushing valuesin each view. Because values are mapped one-to-one into nodes,the filtered groups (G′) of values that have been drilled down to inthe cross-filtering views can be reused unchanged as input into the

node pipeline. The node input data (T Ninput ) is a concatenation of the

records in a subset of the filtered groups. The cross-filtering matrixdetermines which subsets of values; the node matrix determineswhich subset of groups (attributes).

Whereas nodes are first-class elements, independent of other el-ements, edges and packs are second-class elements defined in termsof a pair of anchor nodes (for edges) or a set of member nodes as-sociated with a common reference node (for packs). Both can becalculated by mapping a pair of values into two anchor nodes (foredges) or a member and a reference node (for packs). Just as thegroup transformation calculates a table of {value,set} records foruse in forming nodes, a clique transformation—essentially a group-ing equijoin—calculates a table (C) of {value,value,set} recordsfor use in forming edges and packs. Cliques are cross-filtered (C′)in the same way as groups. Because co-occurrence is symmetric,and hence edges are undirected, the edge input data (T E

input ) is a con-catenation of records from a combination of the filtered cliques; butbecause the co-occurrences for a given value can be both one-to-many and many-to-one, and hence packs are directed, the pack in-put data (T P

input ) is a concatenation of records from a permutation ofthe filtered cliques. Edge and pack matrices determine which sub-sets of cliques, respectively. Overall, construction of an attribute re-lationship graph requires calculation of m groups and m2 cliques foran m-dimensional data set. The results can be accumulated lazilyas previously unvisited checkboxes are toggled on in the matrices.

An attribute relationship graph can thus be incorporated into anexisting cross-filtered visualization with or without changes to ex-isting views or their coordinations. Moreover, visualizations com-bined in this way inherit much of the design flexibility of both thegeneral graph model and the cross-filtered views model.

4.3 Design VariationsThe query strategy used in the design of attribute relationshipgraphs has evolved through experience with several visualizations.The current data transformation pipeline is concrete and may bereused as a flexible template in new visualizations. Each visualiza-tion design involves customization of each transformation in orderto accommodate particular data sets and analysis tasks. Customiza-tion includes picking one or more data sets to load, choosing whichdimensions to include in dissection, then declaring m group andm2 clique transformations for those dimensions. Visual encodingin the cross-filtering views is essentially unconstrained and may bespecialized to suit the character and distribution of each data di-mension. Encoding of graph elements is constrained by space andtopology considerations, yet still affords a large space of practicalvariation. To elucidate attribute relationship graphs as a pattern for

79

Cinegraph VAST Wiki Editors NSF AwardsNominal movie, person author, word manager, program, institution, PI, co-PI

Node Categorical genre, award, role revision type, part of speech directorate, organization, application fieldNode Types Numerical box office, rating average, rating count - -Types Temporal theatrical release date edit date, time of day -

Spatial - - state

Pre-filtering sliders (ratings & roles thresholds) - directorate and year range (hidden)Per-Record Filtering - individual edits individual awards

Auxiliary Temporal Filtering cross-filtering on dates cross-filtering on dates, time of day start dateAuxiliary Effects Spatial Filtering - - map region brushing (United States)Effects Topological Filtering - - sliders (edge and pack counts)

Rich Node Encoding movie posters, person photos, genre icons coloring of nighttime/daytime, extreme words color scale for award amount totalsNode Actions all but roles & ratings link to imdb.com words link to dictionary.com links to award searches on nsf.gov

NodeNode TypesTTTypesTTNode

Types

AuxiliaryAuxiliary EffectsEffects

Auxiliary Effects

Figure 4: Variation in node data types and auxiliary graph effects (filtering, encoding, and activating) in the three example visualizations.

visual tool design, we briefly describe some of the more extensivestructural variations (Figure 4) used in the example visualizations.

4.3.1 Visual AbstractionWhereas each cross-filtered view can map values into graphicalattributes using nearly any visual encoding technique that allowsper-item brushing, the graph view is restricted to visual encod-ings that accommodate connectivity of nodes and edges, contain-ment/connection of nodes in packs, and dynamics of node positionin freeform or geometrically constrained layouts. Visual encodingof nodes can involve virtually any reasonably sized and reasonablyconvex shapes, but edges are best drawn as traditional lines or ar-rows, with packs as hulls or blobs (rather than, say, stars of thinlines that could be easily confused with edges). Perhaps surpris-ingly, brushing in the graph is not a requirement and is not a directpart of the dissection process. Nevertheless, an ability to highlightgraph elements by selecting them may increase the effectiveness ofexamining large graphs during dissection [30]. Locking [28] is ahigh priority for addition to the general graph model.

All graph elements can involve more than simple shape encod-ing to differentiate equivalence classes. For instance, nodes formovies, people, and awards in Figure 1 reflect differences in bothattribute types and specific values using clearly distinguishable im-ages. Edges can draw text describing connections (like in HaveGreen [36]), or show whether each node value is currently cross-filtered on the other using arrowheads on either or both ends. More-over, it is possible to encode the grouping or cliquing data set of co-occurrences associated with a node’s, edge’s, or pack’s value(s).Edge thickness and pack transparency are useful ways to depictco-occurrence counts, although large variation in these counts as afunction of data distribution requires careful design. Encoding theco-occurrence data sets themselves turns nodes into small multiplesof nested scatter plots, heatmaps, or histograms (although perfor-mance has been dismal when we have attempted to do so). Finally,a node can be triggered to perform an action as a function of itsvalue or its grouping set, such as to link to the IMDB web page ofan actor or to search the NSF web site for the awards of a PI.

4.3.2 Data AbstractionAll of the attribute relationship graph visualizations that we havebuilt so far have involved substantial customization of individualtransformations at all points in the pipeline, but without changingthe pipeline itself. Some of the more extensive structural variationsinvolve addition of auxiliary views and sliders that: (1) preprocessand prefilter raw data sets at the beginning of the pipeline; (2) pro-vide a detail view of all data records, optionally allowing dissec-tion on a per-record basis by adding a unique record identifier (orother primary key) as a filtering dimension; or (3) provide alternategeographic or temporal representations for additional encoding or

filtering of particular graph elements. The graph can also be filteredand/or encoded topologically. For example, the NSF visualization(Figure 2) varies in five major ways. First, we split the one-tableaward database into nine dimensions across four relational data ta-bles in a simple star schema with the award identifier as primarykey. Second, we mitigated interactive performance limitations dueto large data size by prefiltering out all but the CISE directorate.Third, we included the award identifier as a cross-filterable dimen-sion. Fourth, we allow dynamic filtering of the graph on navigationin time series and map views. Finally, sliders filter edges and packson minimum co-occurrence counts.

4.3.3 Attribute TypesThe data type of each dimension is a critical design factor. In gen-eral, graph views are unlike other views in that they depict bags ofobjects rather than true coordinate spaces ordered in one or moredimensions. Although many techniques exist for laying out graphsto account for such information, doing so would break the dimen-sional symmetry of graph appearance and behavior during dissec-tion, especially in the face of variation from zero to one to twoor more numerical dimensions. Data values are thus deliberatelytreated as effectively nominal or categorical at all points along thepipeline. Consequently, graph views depict numerical, temporal,and spatial values in an artificial way, stripped of the ordering anddistribution information implied by their parent dimensionality.

Designing the cross-filtering views to depict values in their nat-ural dimensional contexts may help to counterbalance this majorlimitation in using such a basic kind of graph view for dissection.For example, hours of the day in the wiki visualization (Figure 5)appear both as nodes and as selectable bars in a time series his-togram. Dates appear in a calendar as well. In a like manner, cross-filtering bestows attribute relationship graphs with qualified supportfor derived attributes, multiple attributes treated as one compositeattribute, and multiple attributes across spatial and temporal scales.

5 DISCUSSION

Evaluating complex visual tools is an ongoing challenge [2]. De-spite its demonstrated utility for expressing a wide variety of tasksin different application domains—we have also built attribute rela-tionship graph visualizations for CD collections, historic hotel reg-isters, and shape classification experiments—the very flexibility ofdesign that makes the technique so broadly useful also makes it anextraordinarily difficult target for evaluating usability. Evaluationof the cross-filtering technique on its own—see [34] for details—revealed two usability problems. First, a selection occlusion ef-fect arises when selected items both cross-filter other views and arethemselves cross-filtered out. We suspect that addition of the graphhas a mixed impact on this effect, because (1) it shows all selecteditems in a dimension (or at least allows them to be seen by panning)

80

Figure 5: Visualization of wiki edits from the VAST 2008 Wiki Editors Mini Challenge. The attribute relationship graph shows connections betweenprominent authors, frequent words, and revision types during early morning and afternoon activity at the end of two multi-day editing periods.Times (hours of the day) and dates are both treated as nominal data values; the corresponding nodes in the graph have been laid out by hand.

even when some of those items are invisible in their cross-filteredview, but (2) may create confusion due to this inconsistency. Thealternative, filtering out these invisible items in the graph, wouldeffectively reduce the graph to a simple drill-down view.

Second, an out of sight, out of mind effect occurs because usershave trouble remembering more than the most recent states in theanalysis process. We suspect that the graph alleviates this problemsomewhat by shifting memory demands from tedious one-to-manyinteraction sequences in the cross-filtering views to simpler many-to-many examinations in the graph. While we believe that everydayformative evaluation of particular attribute relationship graph visu-alizations would help us to identify usability problems like these,our growing conviction is that characterizing them in complex de-signs requires systematic access to analytic provenance [17, 11]—access that, in analysts’ hands, might address the problems directly.

The time to develop example visualizations has been about aweek; the most recent visualization (NSF) took about ten hours.Reusable templates, added to Improvise for declaring cross-filteringand attribute relationship transformations, are one reason for thisimprovement in development time. Attribute relationship graphsare nevertheless generalizable, and can be adapted to other visual-ization toolkits and systems that are based on the data state model.

When calculating a clique query, each pair of uniquely occurringvalues can accumulate the data records in which they co-occur. Theresulting small data set can be subjected to data transformations likeany other, but its primary purpose is for calculating simple aggre-gates over the pair of values, such as to count the records for en-coding as an edge width, or to sum amounts for encoding as a packfill color. Accumulating and transforming many small data sets isexpensive, and is not needed in any of the example visualizations.Consequently, clique queries currently calculate only counts ratherthan accumulate co-occurrence data subsets for each pair of values.On current desktop hardware, typical interactive response is below100 milliseconds in Cinegraph and the wiki edit visualization, butoften exceeds ten seconds in the NSF visualization.

For m dimensions and n data records, the full set of m2 cliquequeries has time complexity of O(m2n2). By lazily calculating each

clique, when the user first toggles on either the edge or one of thepacks for some pair of dimensions, clique calculation becomes a se-quence of amortized O(n2) operations interspersed throughout theanalysis process. This strategy is effective for dissection because itis common for only a fraction of all possible dimensional pairs tobe visited during any given analysis session—even when all dimen-sions are involved individually—but can cause impatience whilewaiting for new cliques to be calculated. Moreover, variability inthe number of unique values of each attribute is common, creat-ing imbalances in the cardinalities of graph element equivalenceclasses, causing unevenness in group and clique calculation speed,and posing a design challenge in mapping value distributions intoappropriate views for cross-filtering. Response time in the NSFvisualization is slow—especially when aggregating award totals—because the large and skewed interactive working set thrashes thein-memory data transformation caches implemented in Improvise.

Cross-filtered views supports analysis by selecting values of eachattribute separately, meaning that dissection under cross-filteringis unary. Attribute relationship graphs allow binary dissection bymaking it possible to examine values of attributes pairwise. Whileanalytic utility is substantially increased, it remains dimensionallylimited. In Figure 2, edge thickness reflects all awards to a PI undereach program, not just CAREER awards. Although each clique’sdata set could be filtered on the selected values of other attributes,the edge encoding would still only reflect a count; the multigraphinformation in each edge is not encoded at all, let alone to reflectfiltering beyond that of its two node anchor attributes. Attribute re-lationship graphs thus support visual dissection of relationships atone, but only one, order higher than cross-filtered views. One won-ders what visual techniques might support dissection of trinary andhigher-order relationships in multidimensional data.

6 CONCLUSION

Attribute relationship graphs are a method and visualization de-sign pattern for exposing and dissecting many-to-many relation-ships buried in heterogeneously typed multidimensional data sets.Several example visualizations demonstrate the generality and flex-

81

ibility of attribute relationship graphs, and provide insights into spe-cialization for different data sources and analysis needs. Throughsignificant extension of the cross-filtering method and pattern, wehave made further progress in identifying, characterizing, instanti-ating, and reusing effective multiple view compositions at a highlevel of design abstraction. Our ongoing goal is to provide analystswith increasingly flexible and precise yet usable means to exploreand analyze deep details and subtle patterns in complex data spaces.More broadly, we aim to help build a methodological foundationfor the development of classes and instances of synthetic pathwaysthat are demonstrably useful for designing visual-interactive com-pounds for effective exploration and analysis of data in both generaland specialized knowledge domains.

ACKNOWLEDGEMENTS

Thanks to the InfoVis 2007 Contest and VAST 2008 Challengeorganizers for preparation of the data used in this paper, and toCarsten Gorg and Anthony Robinson for many related discussions.

REFERENCES

[1] C. Ahlberg. Spotfire: An information exploration environment. SIG-MOD Record, 25(4):25–29, December 1996.

[2] S. Carpendale. Evaluating information visualizations. In InformationVisualization, volume 4950 of Lecture Notes in Computer Science,pages 19–45. Springer, 2008.

[3] E. H. Chi and J. T. Riedl. An operator interaction framework for vi-sualization systems. In Proceedings of the IEEE Symposium on Infor-mation Visualization (InfoVis), pages 63–70, Research Triangle Park,NC, October 1998.

[4] N. Conklin, S. Prabhakar, and C. North. Multiple foci drill-downthrough tuple and attribute aggregation polyarchies in tabular data.In Proceedings of the IEEE Symposium on Information Visualization(InfoVis), pages 131–134, Boston, MA, October 2002.

[5] H. Ehrig. Introduction to the algebraic theory of graph grammars (asurvey). In Proceedings of the International Workshop on Graph-Grammars and Their Application to Computer Science and Biology,pages 1–69, London, UK, 1979. Springer-Verlag.

[6] J.-D. Fekete. The InfoVis Toolkit. In Proceedings of the IEEE Sympo-sium on Information Visualization (InfoVis), pages 167–174, Austin,TX, October 2004.

[7] E. Gamma, R. Helm, R. Johnson, and J. Vlissides. Design Patterns:Elements of Reusable Object-Oriented Software. Addison-WesleyProfessional Computing Series. Addison Wesley, 1st edition, 1994.

[8] E. R. Gansner and S. C. North. An open graph visualization systemand its applications to software engineering. Software Practice andExperience, 30(11):1203–1233, 1999.

[9] J. Heer and M. Agrawala. Software design patterns for informa-tion visualization. IEEE Transactions on Visualization and ComputerGraphics, 12(5):853–860, September/October 2006.

[10] J. Heer, S. K. Card, and J. A. Landay. prefuse: A toolkit for interactiveinformation visualization. In Proceedings of the Conference on Hu-man Factors in Computing Systems (CHI), pages 421–430, Portland,OR, April 2005. ACM Press.

[11] J. Heer, J. D. Mackinlay, C. Stolte, and M. Agrawala. Graphical histo-ries for visualization: Supporting analysis, communication, and eval-uation. IEEE Transactions on Visualization and Computer Graphics,14(6):1189–1196, November/December 2008.

[12] R. Helm and K. Marriott. A declarative specification and semanticsfor visual languages. Journal of Visual Languages & Computing,2(4):311–331, 1991.

[13] N. Henry, J.-D. Fekete, and M. J. McGuffin. NodeTrix: A hybrid visu-alization of social networks. IEEE Transactions on Visualization andComputer Graphics, 13(6):1302–1309, November/December 2007.

[14] M. L. Huang. Information visualization of attributed relational data. InProceedings of the 2001 Asia-Pacific Symposium on Information Visu-alisation (APVis), pages 143–149, Darlinghurst, Australia, Australia,2001. Australian Computer Society, Inc.

[15] M. L. Huang, Q. V. Nguyen, and T. Hintz. Attributed graph visual-ization of collaborative workspaces. In International Conference on

Computer Graphics, Imaging and Vision: New Trends (CGIV), pages155–161, July 2005.

[16] X. Huang, P. Eades, and W. Lai. A framework of filtering, clusteringand dynamic layout graphs for visualization. In Proceedings of theAustralasian Conference on Computer Science (ACSC), pages 87–96,Newcastle, Australia, 2005. Australian Computer Society.

[17] T. J. Jankun-Kelly, M. Kwan-Liu, and M. Gertz. A model and frame-work for visualization exploration. IEEE Transactions on Visualiza-tion and Computer Graphics, 13(2):357–369, March-April 2007.

[18] G. E. Krasner and S. T. Pope. A cookbook for using the model-view-controller user interface paradigm in Smalltalk-80. Journal of Object-Oriented Programming, 1(3):26–49, August 1988.

[19] J. D. Mackinlay, P. Hanrahan, and C. Stolte. Show Me: Automatic pre-sentation for visual analysis. IEEE Transactions on Visualization andComputer Graphics, 13(6):1137–1144, November/December 2007.

[20] C. North, N. Conklin, K. Indukuri, and V. Saini. Visualizationschemas and a web-based architecture for custom multiple-view vi-sualization of multiple-table databases. Journal of Information Visu-alization, 1(3-4):211–228, December 2002.

[21] J. O’Madadhain, D. Fisher, S. White, and Y. B. Boey. The JUNG (Javauniversal network/graph) framework. Technical report, University ofCalifornia, Irvine, 2003.

[22] A. Perer and B. Shneiderman. Balancing systematic and flexible ex-ploration of social networks. IEEE Transactions on Visualization andComputer Graphics, 12(5):693–700, September/October 2006.

[23] A. J. Pretorius and J. J. van Wijk. Visual inspection of multivariategraphs. In Computer Graphics Forum, volume 27, pages 967–974.Eurographics Association, 2008.

[24] R. Shannon, T. Holland, and A. Quigley. Multivariate graph draw-ing using parallel coordinate visualisations. Technical Report 3020,University College Dublin, 2008.

[25] J. Stasko, C. Gorg, and Z. Liu. Jigsaw: Supporting investigativeanalysis through interactive visualization. Information Visualization,7(2):118–132, 2008.

[26] M. Streit. Interactive visualization of complex graphs. In Proceedingsof the Central European Seminar on Computer Graphics (CESCG),Slovakia, April 2007.

[27] J. J. Thomas and K. A. Cook, editors. Illuminating the Path: The Re-search and Development Agenda for Visual Analytics. IEEE ComputerSociety, August 2005.

[28] C. Tominski, J. Abello, and H. Schumann. CGV–an interactive graphvisualization system. Computers & Graphics, 33(6):660–678, 2009.

[29] M. O. Ward. XmdvTool: Integrating multiple methods for visualizingmultivariate data. In Proceedings of the IEEE Conference on Visual-ization (VIS), pages 326–333. IEEE Computer Society Press, 1994.

[30] C. Ware and R. Bobrow. Supporting visual queries on medium-sizednode-link diagrams. Information Visualization, 4(1):49–58, 2005.

[31] C. Weaver. Building highly-coordinated visualizations in Improvise.In Proceedings of the IEEE Symposium on Information Visualization(InfoVis), pages 159–166, Austin, TX, October 2004.

[32] C. Weaver. InfoVis 2007 contest entry: Cinegraph. In Proceedingsof the IEEE Symposium on Information Visualization (InfoVis) (Com-pendium), Sacramento, CA, October 2007.

[33] C. Weaver. Cross-filtered views for multidimensional visual anal-ysis. IEEE Transactions on Visualization and Computer Graphics,16(2):192–204, March-April 2010.

[34] C. Weaver, D. Fyfe, A. Robinson, D. W. Holdsworth, D. J. Peuquet,and A. M. MacEachren. Visual exploration and analysis of historichotel visits. Information Visualization, 6(1):89–103, February 2007.

[35] G. J. Wills. NicheWorks: Interactive visualization of very largegraphs. Journal of Computational and Graphical Statistics, 8(2):190–212, June 1999.

[36] P. C. Wong, G. Chin, Jr., H. Foote, P. Mackey, and J. Thomas. HaveGreen – a visual analytics framework for large semantic graphs. InProceedings of the IEEE Symposium on Visual Analytics Science andTechnology (VAST), pages 67–74, Baltimore, MD, October 2006.

[37] S. Zhao, M. J. McGuffin, and M. H. Chignell. Elastic hierarchies:Combining treemaps and node-link diagrams. In Proceedings of theIEEE Symposium on Information Visualization (InfoVis), pages 57–64,Minneapolis, MN, October 2005.

82