Live Re-orderable Accordion Drawing (LiveRAC)

Peter McLachlan, Tamara Munzner

Eleftherios Koutsofios, Stephen North

AT&T Research Symposium

August, 2007

2

Problem domain

• Managed hosting services, network operations centre staff

Motivation Problem Domain Monitored data Viz requirements

SWIFT

LiveRAC

Key additions

Field study

Conclusion

3

Monitored data

• Most data collected from monitored network devices is time-series data – any type of computer or appliance: servers,

routers …– time stamp and value

• Two types of time-series objects collected:– performance metrics

• [ 10 AUG 2006 9:52:37, CPU, 95% ]

– alarm data• [ 10 AUG 2006 9:52:37, MAJOR, “HIGH TEMP” ]

• Key difference for visualization:– performance metrics quantitative– alarms categorical

Motivation Problem Domain Monitored data Viz requirements

SWIFT

LiveRAC

Key additions

Field study

Conclusion

4

• Scale to large, dynamic datasets – thousands of devices– dozens of data channels

• Interact with previously gathered data

Visualization solution requirements

Active region Time scale # of items

Total database Days to years Billions

In memory Several seconds Millions

On screen Sub-second Thousands

DB (SWIFT)

LiveRAC

Motivation Problem Domain Monitored data Viz requirements

SWIFT

LiveRAC

Key additions

Field study

Conclusion

5

SWIFT

• SWIFT is a set of data storage, aggregation and visualization tools that integrate multiple data sources [Koutsofios, 1999]

• Developed at AT&T Labs, fully deployed in a production role

• Data sources include SNMP, intrusion detection systems, Windows system monitors, and custom written daemons

• Highly scalable• Optimized for streaming data

Motivation

SWIFTIntroductionVisualizations

LiveRAC

Key additions

Field study

Conclusion

6

SWIFT visualizations

Koutsofios, North, Truscott, and Keim. Visualizing large-scale telecommunication networks and services.IEEE Visualization 1999, 457-461

• Existing views:– geographic views– node-link diagrams– line charts– text tables

• Limitations: – difficult to compare

between large numbers of time-series objects

• LiveRAC: re-orderable matrix visualization for SWIFT

Motivation

SWIFTIntroductionVisualizations

LiveRAC

Key additions

Field study

Conclusion

7

Our solution: LiveRAC

• Interactive user-directed exploration of overview & detail

– live demo

Motivation

SWIFT

LiveRACDemoVisual encodingSemantic zoom

Key additions

Field study

Conclusion

8

LiveRAC visual encoding

QuickTime™ and aTIFF (LZW) decompressor

are needed to see this picture.

• High data density grid view– Rapidly explore

time-series data– Context

available at all times

• Re-orderable matrix supports side by side data comparison

Motivation

SWIFT

LiveRACDemoVisual encodingSemantic zoom

Key additions

Field study

Conclusion

9

Semantic zooming and aggregation

• compact representations in reduced areas

• large cells show time-series charts

• aggregate spatial representation shown in highly compressed regions

Motivation

SWIFT

LiveRACDemoVisual encodingSemantic zoom

Key additions

Field study

Conclusion

10

Project progress this year

• Last year: hardwired prototype• This year:

– robust, deployed cross-platform client– more scalable:

• 4000+ devices tested • designed to scale past 10000

– new architecture allows full visual configuration interface

– AT&T internal field study in progress

Motivation

SWIFT

LiveRAC

Key additionsProgress calloutConfigurationNew architecture

Field study

Conclusion

11

Configuration is critical in effective analytic tools• Exploratory data analysis requires

looking at data in different ways– different tasks require different views– even the same task may require changes

to the data representations

• Provide end user full and flexible control through a GUI – not pre-compiled settings

• Required complete data abstraction re-architecture

Motivation

SWIFT

LiveRAC

Key additionsProgress calloutConfigurationNew architecture

Field study

Conclusion

12

New architecture allows:• Data source groupings

– by business, customer, logical group, geography …

• Filter devices and channels of interest– pick ‘web’ devices and ‘cpu’

channels

• Manage marker thresholds– at 85%, turn box red

• Choose effective representations for data– use 2-line bar chart for

200x200 pixel cell

• Select data aggregation strategies – aggregate to daily max when

time window > 1 month

Motivation

SWIFT

LiveRAC

Key additionsProgress calloutConfigurationNew architecture

Field study

Conclusion

13

Field study in progress

• Deploying LiveRAC to AT&T internal users– tier 3 and tier 4 network professionals

– more exploratory than front line tier 1 ops staff

• Study objectives:– understand when and why users choose visualization in

their workflow

– specific benefits and limitations of LiveRAC in managed hosting environment

• Gather data on LiveRAC usage– logs

– structured interviews

– questionnaires

• Progress towards a system integrated into workflow of network operations analysts

Motivation

SWIFT

LiveRAC

Key additions

Field study

Conclusion

14

Conclusion

• Contributions: – robust system for interactive visualization

of large scale network monitoring data• thousands of devices, hundreds of

channels• months of data

– highly configurable– deployed inside AT&T– field study in progress

Motivation

SWIFT

LiveRAC

Key additions

Field study

Conclusion

15

Information visualization

• Human visual channel is highest-bandwidth perceptual system [Norretranders, 1999]

• Information visualization: field of study whose object is to aid cognition through the graphic representation of abstract data– displays relevant information graphically to

assist in memory tasks– supports data exploration through direct

interaction – assists in pattern finding through the display

of overview and detail, search, and user-directed reordering

Motivation

Related Work InfoVis Time-series Statistical Graphics Reorderable Vis Semantic Zoom Accordion Drawing SWIFT

LiveRAC

Discussion

16

Network devices

• Network device: any electronic device that connects to a computer network

• Most network devices can be monitored

• Network Operation Centre (NOC): facility for monitoring large numbers of network devices

Motivation Overview Problem Domain Devices & Data Domain Tools Requirements Our Solution

Related Work

LiveRAC

Discussion

25

Motivation

Related Work

LiveRAC Architecture Semantic Zoom Reordering Playback

DiscussionQuickTime™ and a

TIFF (LZW) decompressorare needed to see this picture.

QuickTime™ and aTIFF (LZW) decompressor

are needed to see this picture.

Data path Render path