Visual Correlation for Situational Awareness

Yarden Livnat∗

Scientific Computing andImaging InstituteUniversity of Utah

Jim Agutter,†

College ofArchitecture+Planning

University of Utah

Shaun Moon, ‡

College ofArchitecture+Planning

University of Utah

Stefano Foresti§

Center for HighPerformance Computing

University of Utah

Potomac

RiverRiver

dehyd

ration

stroke

unconcious

cardiac

dismemberment

severe hemorrhaging

spinal injury

severe head injury

difficulty breathing

labor / delivery

hypothermia

anaphylactic shock

bio-in

fection

burns

other 2

apartment f

ire

house fi

re

chemical fire

commercial fire

grass fire

auto fire

chemical spill

wildfire

search & rescue

other 1

inve

stig

atio

n

SWAT

hazm

at

pol

ice

pat

rol am

bula

nce

truc

ks

Kilarn

y

East

City Center M

emorial U

niversity Childrens M

ed.

Hig

hlan

d

bomb

terror activity

conspiracy

gang activity

drug traffic

other 1

human traffic

other 3

other 2

other 2

acci

dent

one

car

acci

den

t 2 c

ars

acci

den

t m

ult

.

wre

ckle

ss d

rivin

g

acci

dent

w/ p

ed.

racin

g DW

I

speeding

sppedin

g - sch

ool zone

failure to yield - emerg

failure to

stop

leaving the scene failure to yield - ped.

other 1

careless / unsafe

MEDIC

AL EMERGENCY

FIRE

EMERGENCY UNITS HOSPITALS

SUSPICIO

US

ACT

IVIT

Y

TRAFFIC INCIDENTS

6:00am

8:00am

4:00am

2:00am

12:00pm

10:00pm

(a) 911 Emergency Center

SNORT

ALERTS

FTPALERTS

HTTP

ALERTS

CHECKSUMWINDOWS EVENT ALERTS

-5

-0

-15

-25

-35

-45

(b) Network Intrusion Detection

vx

soman

tabun

sarin

typhus fever

Q fever

phosgene oxime

lewisite

sulfur mustard

nitrogen

mustard gas

strychnine

ricin

abrin

hant

aviru

s

Nip

ah v

irus viral e ncep hali t i s

staph

psittacosis

melioidosis

glanders

epsilon toxin

brucellosis

hemorrhagic fevers

tularemia

smallpox

plague

botu

lism

anth

rax

NERVE AGENTS

BLI

STER

ING

AN

GEN

TS

BIOTOXINS

BIO CAT C

BIO

CAT B

BIO CAT A

-2

0

-4

-6

-8

-10

(c) BioWatch

Figure 1: VisAware: A novel visualization paradigm for situational awareness.

Abstract

We present a novel visual correlation paradigm for situa-tional awareness (SA) and suggest its usage in a diverse setof applications that require a high level of SA. Our approachis based on a concise and scalable representation, which leadsto a flexible visualization tool that is both clear and intuitiveto use. Situational awareness is the continuous extractionof environmental information, its integration with previousknowledge to form a coherent mental picture, and the use ofthat picture in anticipating future events.

In this paper we build on our previous work on visual-ization for network intrusion detection and show how thatapproach can be generalized to encompass a much broaderclass of SA systems. We first propose a generalization thatis based on what we term, the w3premise, namely that eachevent must have have at least the What, When and Whereattributes. We also present a second generalization, whichincreases flexibility and facilitates complex visual correla-tions. Finally, we demonstrate the generality of our ap-proaches by applying our visualization paradigm in a col-lection of diverse SA areas.

CR Categories: H.5.1 [Information Interfaces and Pre-sentations]: User Interfaces—Graphical user interfaces;

Keywords: situation awareness, network intrusion, visu-alization

∗e-mail: yarden@sci.utah.edu†e-mail: agutterja@arch.utah.edu‡e-mail: moonsr@arch.edu.edu§e-mail: email@stefanoforesti.com

1 Introduction

Situational Awareness (SA) is the ability to identify, process,and comprehend the critical elements of information aboutwhat is happening. The term SA comes from the world ofmilitary pilots, where achieving high levels of SA was foundto be both critical and challenging [5]. The importance ofSA as a foundation of decision-making and performance spanmany fields such as air traffic controllers, driving, powerplant operations, maintenance, and military operations.

There is a growing body of research that validates therole of visualization as a means for solving complex dataproblems. Visualization elevates the comprehension of in-formation by fostering rapid correlation and perceived asso-ciations. To that end, the design of the display must supportthe decision making process: identifying problems, charac-terizing them, and determining appropriate responses. It isimperative that information be presented in a manner thatfacilitates the user’s ability to process the information andminimize any mental transformations that must be appliedto the data.

In this work we focus on developing a visualizationparadigm that takes advantage of human perceptive and cog-nitive facilities in order to enhance users’ situational aware-ness and support decision-making. We propose a novel vi-sual correlation paradigm for SA and suggest its usage in adiverse set of SA applications.

Recently, we proposed a new visualization paradigm fornetwork intrusion detection (VisAlert) [13] as seen in Fig-ure 1(b). The development of VisAlert involved traditionaluser-centric analysis, design and development cycles, butwas focused solely on network intrusion detection. In this pa-per, we revisit this earlier work and examine it from a moregeneral viewpoint, based on what we term the w3 premise.Using this premise, VisAlert can be seen as a special caseof a much broader class of SA systems, namely visual cor-

95

IEEE Symposium on Information Visualization 2005October 23-25, Minneapolis, MN, USA0-7803-9464-X/05/$20.00 ©2005 IEEE.

relation. We then propose a general framework for eventcorrelation and provide examples of SA applications alongwith prototype designs for their displays. We proceed byproposing a second generalization to this framework, whichintroduces the notion of mixing events and resources. Thisfinal framework is both flexible and scalable, yet clear andintuitive to use. As with the first framework, we provideexamples of SA application along with design prototypes oftheir displays.

This paper is structured as follows. Section 2 containsa brief introduction to situational awareness. Section 3 de-scribes our earlier work on visualization for network intru-sion detection and sets up the basis for its generalization.Section 4 presents two generalization frameworks and pro-vides a collection of examples of possible application of theseframeworks to various SA areas. We conclude and suggestfuture work in section 5.

2 Situational Awareness and Decision Making

Situational awareness was defined by Endsley [3] as “the per-ception of the elements in the environment with a volume oftime and space, the comprehension of their meaning, and theprojection of the their status in the near future”. The formaldefinition of SA breaks down into three separate levels [5]:

• Level 1 – perception of the elements in the environment.

• Level 2 – comprehension of the current situation.

• Level 3 – projection of future status.

The relationship between these SA levels, the mental model,and the decision-making is depicted in Figure 2.

PerceptionLevel 1

ComprehensionLevel 2

ProjectionLevel 3 Decision

Making

Situational Awareness

Environment

Mental Model

Figure 2: Situational Awareness

Level 1, perception of status, attributes and dynamics ofthe environment, may be gained by combination of visual,tactile, and auditory senses. Jones and Endsley [8] haveshown that 76% of SA errors in pilots were related to notperceiving the needed information.

The second level in SA involves the comprehension of whatthe received data means in relation to the relevant goalsand objectives. This may include integration of the data toform information, prioritizing and associating specific goal-related meanings and significance. In their study, Jones andEndsley [8] found that up to 19% of SA errors occured whenthe pilots received the necessary data (SA level 1) but werenot able to correctly understand its meaning.

Level 3 SA is achieved when one can predict how the envi-ronment will be affected in the future, based on the perceiveddata and its meaning. One must have a good understandingof the situation and the dynamics of the system in order toachieve level 3 SA.

The perception of time and the temporal dynamics of theenvironment are important factors in SA. The trends devel-oped over time can play a critical part as well. Time is astrong part of Level 2 SA and Level 3 [5].

2.1 Previous Work

The National Center for Supercomputing Applications(NCSA) has developed two applications for the detectionof network incidents: VisFlowConnect and NVisionIP. Vis-FlowConnect uses flexible animation capabilities to displaynetwork activity. These animations can be configured toshow an aggregation of network activity over time, or tomore accurately reflect traffic dynamics over time. Vis-FlowConnect also allows analysts to filter traffic based onattributes that may help reveal network misuse. [19] NVi-sionIP uses a visualization structure that represents networkactivity at three levels, from a global view of the entire net-work under study down to a single machine. This scalabilityincreases users’ understanding of the relationships of part towhole and vice versa [12].

The Automated Intrusion Detection Environment (AIDE)developed by Air Force Research Lab in Rome, New York,integrates data from various intrusion detection systems intoa standardized view. Using multiple data sets increases un-derstanding of network traffic and alert intricacies [10].

An anesthesia display developed by Michels organized 32variables by organ system, showing the absolute values ofvariables in relationship to a normal reference. Studies ofthis display showed that state changes were seen an averageof three minutes sooner than in traditional displays, provingthat organized structures and deviation displays can improvesituational awareness [7].

Quantum 3D developed a display for the Swedish AirForce that increases ground controllers’ situational aware-ness by generating three-dimensional views of the air space.This helped the controllers understand actual relationshipsbetween critical objects in real space. In this case, situ-ational awareness was enhanced by remote simulations ofreality more than traditional 2D radar displays [15].

3 A Visualization Paradigm for Network IntrusionDetection

Rather than describe a theoretical framework for a class ofSA systems without a concrete example, we first present areal system for which we have already developed a workingprototype. We then build on that work and develop thetheory leading to more general frameworks in Sections 4.2and 4.4. Finally, we provide concrete examples within thesegeneral frameworks and demonstrate possible designs oftheir displays. We wish to emphasize that the design processwe used in the development of VisAlert was based on a longand methodical user-centric design cycle. The process in-volved a psychology team; interviews with network experts;and many analysis, design and evaluation phase cycles. How-ever, in order to facilitate the generalization discussion whichis the main goal of this work, we present VisAlert with re-spect to a different framework. This framework is closelyrelated to the w3 framework that along with its generaliza-tion are the main topic of the next section.

3.1 Network Intrusion Detection

The spread of malicious network activities poses great risksto the operational integrity of many corporations, institu-tions, and organizations, in addition to imposing heavy eco-nomic burdens. Intrusion detection systems (IDS) analyzenetwork traffic and host-based processes, in an attempt todetect such malicious activities. The proliferation of differ-ent IDSs and the sophistication of attacks leads to a largenumber of alert types. This complexity is compounded by

96

the sheer number of alerts these systems generate and a highrate of false positives.

One approach to resolving these issues is to correlate var-ious alerts by common attributes. This approach is basedon the premise that while a false positive alert should notexhibit correlation to other alerts, a sustained attack willlikely raise several alerts. Furthermore, real attack activitieswill most likely generate multiple alerts of different types.There exists a large body of work aimed at correlating thesedisparate alert logs based upon clustering, probability, andsimilarity to predefined attacks [17, 1].

Visualization has only recently been applied to computersecurity data analysis, and these applications have typicallybeen limited in their capabilities. Historically, visualiza-tion has been applied extensively to network monitoring andanalysis, primarily for monitoring network health and per-formance [11, 6]. The few visualization techniques that havebeen developed for intrusion detection have been limited asto their applicability and effectiveness [16, 14, 18, 2].

3.2 VisAlert

Typically, alerts are correlated based on either their time ortype attributes. However, the real value of an alert is withrespect to the local node(s) it pertains to. It is, in fact, thepreservation of the nodes’ status and integrity that is themain focus of IDS to begin with.

We also distinguish between an alert’s definition and analert’s instances. An alert definition is static and describesthe preconditions and meaning of an alert. An alert in-stance, on the other hand, refers to a particular point in timewhen the alert preconditions were met. An alert instancemay also include detailed information about how and whattriggered a particular alert. Correlating alerts and nodes,therefore, means correlating alert instances with respect toa particular node.

A possible approach is to use a three-dimensional Carte-sian coordinate system and map the type, time, and nodeto the three axes. In this configuration, a network event isrepresented by a three-dimensional point. There are severalproblems with this approach. First, the points do not ex-hibit any obvious correlation; for example, two nearby pointsmay not share any attribute with each other. Second, thethree-dimensional view introduces visual obstacles such asocclusion and depth perception. Finally, the visual percep-tion will depend on the user’s specific viewpoint, which onlyadds another unnecessary degree of freedom to an alreadycomplicated situation.

In contrast, we base our approach on representing thenetwork alerts as connections between two domains. Thesetwo domains are a one dimensional domain representing thenode attribute, and a two-dimensional domain representingthe time and type attributes. Note that the node and typespaces are finite while the time space is semi-infinite. A net-work alert instance, in this scheme, is thus a straight linefrom a point in the type-time domain to a point in the nodedomain, as shown in Figure 3. We choose to separate thenode attribute from the type and time as nodes provide amore or less static set of objects that we can use as visual-ization anchors for the transient alert instances.

We can now expand the node domain into a two-dimensional domain, which enables to layout the nodes ina more flexible and meaningful way for the user, such as thenetwork topology map. The new design layout, as shown inFigure 4, maps the node domain onto a finite circle, whilethe type-time domain is wrapped around it in the shape ofa ring. We maintain the orthogonality of the type and time

Node

Tim

e

TypeNetwork Alerts

Figure 3: Domains: Representing network alerts as lines between twodomains. The design facilitates the correlation of alerts with respectto the same nodes.

spaces by mapping the type attribute to the angle aroundthe circle and the time attribute to the radial direction fromthe center of the circle.

networktopology

TypeTim

eFigure 4: Mapping: The finite two-dimensional node domain ismapped into a central circle containing the network topology map,while the time-type domain is mapped to a ring around it.

Finally, to reduce the possible cluttering when showingall the alerts simultaneously, we divide the time space intovarying intervals (rings) and show the alert instances forthe most recent history period (inner most ring). For alertsthat took place in the past, we display only the numberof alerts that occurred during a specific time period on itscorresponding ring.

3.2.1 Additional Visualization Cues

The design layout of the VisAlert display is shown in Fig-ure 1(b) and includes various host-based and network-basedalert logs. Figure 5 shows the display of our implementa-tion of VisAlert which includes other visual indicators thatencode additional information to increase the situationalawareness of the user. We have adopted a method of in-creasing the size of nodes experiencing unique alerts. Theassumption is that a resource/node on the topology that isexperiencing multiple unique alerts from various host- andnetwork-based sources has a higher probability of maliciousactivity than one experiencing only one alert. The size of thenode is a very clear indication and is easily distinguishable

97

from other machines in order to focus the user’s attentionso he or she can take action and correct the problem on thesuspect machine(s).

Figure 5: The VisAlert system

We employ an alert beam indicator to encode multipleinstances of the same alert type during the same historyperiod. The alert beam’s width represents the number ofinstances and thus shows the persistence of the problem.The color of the beam encodes the severity of the problem,such as the priority of a Snort alert.

Additional features that are not obvious in a static dia-gram include:

• Interactive information about alerts and nodes (notepopup windows in Figure 5).

• Pan and zoom operations over the topology map. Inessence the inner ring behaves as a circular windowthrough which the user see portion of the entire topol-ogy map. The user can then move the underlying mapand zoom in/out to a provide an interactive level ofdetail (LOD).

• Hierarchical representation of the alert types (LOD overtypes).

• The history periods, i.e., the rings, can each be be dy-namically configured to represent different time periods(LOD over the time domain).

• Animated interaction when opening or closing a groupof alert types. This is an important feature that allowsthe user to maintain awareness of where each alert typeis on the screen as their distribution around the ringchanges.

• Recording and playback of a session for forensic analy-sis.

More information and details about VisAlert can be foundin [13].

4 Visualization Paradigm for Situational Aware-ness

The VisAlert work, discussed in the previous section, tar-geted a very specific area within the domain of SA; namely,

that of network alerts and network topology. We describedVisAlert with respect to a framework that is based on thetime, node, and type attributes. We now revisit this frame-work and generalize it to pertain to a much larger class ofSA applications.

4.1 Events, Resources and the w3 Premise

Alert correlation systems, such as VisAlert, aim to establishthe validity of, or to generate a confidence measure for, theparticipating alerts. The main problem in correlating alertsfrom disparate logs is the seeming lack of mutual groundson which to base any kind of comparison between the alerts.VisAlert takes advantage of the fact that every alert has atype, a time, and a node attribute, which forms a consistentbase for comparison. However, this base is not specific tothe network alerts nor is it coincidental.

Consider a generic event in space-time. By its very natureit must possess what we term the w3 premise, namely, theWhat, When, and Where attributes. When refers to thepoint in time when the event happened. Where refers toits location. Finally, What refers to some global indicationof the type of the event. Network alerts are thus a genericevent with the mapping of (What, When, Where) to (type,time, node). Furthermore, the network node can be thoughtof representing a local resource which is of interest for oursituational awareness.

4.2 Generalization Part I

In VisAlert we used two fixed functions (hardcoded)

f : (type, time) → (angle, ring)

g : (node) → (x, y)

For each alert instance in the database, we first extractedonly the type, time and node attributes. We then used thef function to find out its location p0 = f(type, time) onthe rings. For alert instances that fall on the innermost ring(ring = 0), we used the function g to find its projection ontothe topology map p1 = g(node) and drew a line p0p1. If thep0 was outside the last ring we ignored the alert, otherwisewe incremented a counter num[p0].

Using the w3 premise, we can now generalize the displaydesign of VisAlert. We formalize this generalization by defin-ing an event instance as an n-tuple,

~e = (What, When, Where, . . .) (1)

That is, an event is an ordered list of attributes wherethe first three attributes are What, When, and Where. Therest of the attributes are not important within our w3 frame-work. We will return to this assumption and generalize it inSection 4.4

Next, we define three unknown functions:

θ : what → angle

ρ : when → ring (2)

χ : where → (x, y)

We can now combine equations 1 and 2 and define theprojection,

Γ(what, when, where) =

=

p0p1 ρ(when) = 0num[p0]++ 0 < ρ(when)

(3)

98

where,

p0 = (θ(what), ρ(when))

p1 = χ(where)

Effectively, we decouple the projection and renderingphases from the mapping phase using the three externalfunctions θ, ρ, and χ. The visualization process is thus madeof three separate phases.

1. Retrieval of the (type, time, resource) attributes fromthe database.

2. Mapping these values using three external functions.

3. Projection onto the display using a generic Γ function.

From an implementation point of view, the three map-ping functions can be supplied to the program via a pluginmechanism, while the resources map (the topology map inthe case of VisAlert) is nothing more than an image file.

Our new visualization paradigm, therefore, provides ageneral framework for correlating disparate events with re-spect to a collection of resources.

4.3 Event Correlation

The reader may have noticed that the presentation of theVisAlert visualization paradigm in Section 3 was in effecta special case of the event projections we formalized in theprevious section. In the case of network alerts, the Whereattribute (resources) consists of computers and switches,which in turn are represented by a network topology map.However, using the generalized mapping θ(type), ρ(time),and χ(resource) and the generic projection Γ, we can ap-ply our visualization paradigm to other areas where situa-tional awareness relies on correlations between events andresources. For each such case we need to

1. Identify the resource and type domains.

2. Create an appropriate resource map (image).

3. Define the θ(type), ρ(time) and χ(resource) plugins.

Our visualization engine can now be applied to a widevariety of SA areas without modifications to the engine itself.

In the following examples, we demonstrate the generalityof our visualization paradigm by applying it to various SAareas. For clarity, we refer to all these various visualizationsolutions under the same name, VisAware (thus tying it backto the original VisAlert work).

4.3.1 Example: Network Alert Incident Reporting

The first example relates to the original application ofVisAlert, namely network intrusion alerts. In very large or-ganizations there may be numerous operators that monitorthe network at the alert level. While VisAlert can fulfill theneeds of these operators, they are only responsible to reportany suspicious activity. It is the responsibility of higher levelanalysts to collect these reports and determine if any actionis warranted. In some cases there may even be a third levelof analysts who oversee the second tier.

It is clear that the second and third tier analysts cannotmonitor the entire local network infrastructure. Further-more, these analysts need to be able to correlate not onlynetwork alerts (as per the report they get from the lowerlevel operators) but they also need to correlate the actual

reports to each other. By correlating the reports, these an-alysts can detect broad range attacks such as simultaneousattack on distanced installations, or a set of attacks thatrepeats itself at different times or at different places.

To adapt VisAware to this scenario, we need to define theWhere (resources) and What (types) domains. The Wheredomain in this case includes all the various network instal-lations that a higher level analyst monitors, while the Whatdomain includes the various types of reports an operatormay submit. Figure 7 shows a possible design of a networkalert incident reporting application.

Org A

Org B

Org C

Org D

Org E

Org F

Org G

Org H

Network Status

Physical Securit

y

Main

tenance Status

Vu

lner

abili

ty S

tatu

s

Intrusion Attempts

Adm

inis

trat

ors

Data Exfiltration Successful Attempts

Figure 7: Network alert incident reporting.

4.3.2 Example: Biowatch

In order to protect US citizens from the threat of chemicaland biological attacks, the Department of Homeland Secu-rity has proposed a program called BioWatch to detect andreport the presence of harmful agents. One of the most sig-nificant obstacles BioWatch faces is how information aboutpossible attacks is quickly and effectively communicated tocontrol centers.

This information includes:

• Detected agent or agents,

• Probability of their existence,

• Where they are detected,

• How the probable presence of such agents changes overtime.

Understanding all of the dimensions of this information iscritical to developing response plans in order to protect peo-ple in danger, and to create defense strategies against futurepossible attacks.

99

vx

soman

tabun

sarin

typhus fever

Q fever

phosgene oxime

lewisite

sulfur mustard

nitrogen

mustard gas

strychnine

ricin

abrin

hant

aviru

s

Nip

ah v

irus viral e ncep hali t i s

staph

psittacosis

melioidosis

glanders

epsilon toxin

brucellosis

hemorrhagic fevers

tularemia

smallpox

plague

botu

lism

anth

rax

NERVE AGENTS

BLI

STER

ING

AN

GEN

TS

BIOTOXINS

BIO CAT C

BIO C

AT B

BIO CAT A

-2

0

-4

-6

-8

-10

(a) Incidents

vx

Nipah virus

abrin

hantavirus

strychnine

ricin

nitrogen mustard mustard gas

lewistie

sulfur mustard

sarin

phosgene oxime

tabun

soman

viral encephalitis

hem

orrh

agic

feve

rs

tula

rem

ia

epsilon to

xin

bruce

llosis

melioidosis

glanders

Q fever

psittacosis

staph enterotoxin

typhus fever

bot

ulis

m

anth

rax

smal

lpox

pla

gue

NERVE AGENTS

BLISTERIN

G AGENTS

BIO

TOXI

NS

BIO

CA

T C

BIO

CAT B

BIO CAT A SPREAD CONDITIONS

CON

TACT

SYM

PTO

MS

CONTACT TREATMEN

T

(b) Incidents, Symptoms, and Conditions

Figure 6: VisAware BioWatch showing (a) incidents of agent detection and (b) correlated contact symptoms, treatments, and spread conditions

BioWatch Data The most basic elements of chemicaland biological attacks that analysts need to understand are:

• What is the probability that it is an actual attack.

• What types of agents and specific chemical or biologicalagents are detected.

• Where is the incident.

Beyond this crucial information, other data can help analystsunderstand the nature of the threat, including:

• How the agents spread (e.g., airborne, water-borne,personal contact),

• Human symptoms of contact,

• Treatment strategies.

The variable relationships and details that different analystsmay or may not want to see require that the methods forcommunicating this information be simultaneously flexibleand precise.

VisAware for BioWatch The VisAware structure clas-sifies agents in colored sections around a ring. Figure6(a)shows the different categories of biological agents (as clas-sified by the Centers for Disease Control) and the differenttypes of chemical agents (i.e. blistering and nerve agents).The concentric rings in the circle represent sequential timesamples. This modified tree-ring shows how the presence ofagents has evolved over time. The inside of the ring struc-ture shows where sensors across the country are set up. Withthe map in the middle, it is easy to correlate the presenceof agents to the sensor that detected it. The correlating linehas a variable width that shows the probability of the agentunder analysis; the thicker the line the greater the proba-bility of an actual attack. Additional information can be

shown on the ring to support more complex analysis. Forexample, if an analyst wants to understand how symptomsand treatment correlate to the possible presence of an agent,two sections can be added to the ring to include this infor-mation as shown in Figure 6(b).

4.4 Generalization Part II

In Section 4.2 we propose viewing an event as an n-tuple,or a vector of attributes. That view was derived from theway information is stored in relational databases, namelytables where each row corresponds to an event, and eachcolumn represents a specific attribute (whether the eventsare actually stored in a single table or are recovered via a joinoperation of several tables is of no importance in our case).The fact that different events from different tables may havedifferent attributes was ignored, based on the w3 premise,and was hidden away by the use of a retrieval package thatselected from the database only the first three columns ofeach table, corresponding to the What, When, and Whereattributes.

We can generalize the n-tuple view of an event by in-troducing the notion of a property list. A property list isan unordered collection of (key, value) pairs. Using the w3

premise, we can represent an event as a special property listthat includes at least the three What, When, and Wherekeys:

~e = {(key, value) | ∃i, j, k

keyi =What, keyj =When, keyk =Where (4)

Let us also define a generic projection:

Πkey(~e) = ~e(key)

That is, for a given attribute key , Πkey(~e) returns the valueof ~e associated with that attribute.

100

The three mapping functions in equation 2 can now bereformulated as follows:

θ′(~e) = θ ◦ Πwhat(~e)

ρ′(~e) = ρ ◦ Πwhen(~e) (5)

χ′(~e) = χ ◦ Πwhere(~e)

It is clear from equation 5 that the What, When, andWhere are not intrinsic to our formulation. They are onlyleftover artifacts of our original VisAlert design and the w3

premise from Section 4.1. We can reformulate the threemapping again, this time taking out the last remnants ofthe w3 premise.

θα(~e) = θ ◦ Πα(~e)

ρβ(~e) = ρ ◦ Πβ(~e) (6)

χγ(~e) = χ ◦ Πγ(~e)

That is, given any three attributes α, β, γ, and any threemapping function θ, ρ, χ, we can define our generic projec-tion,

Γα,β,γ(~e) =

p0p1 ρ = 0num[p0]++ 0 < ρ

(7)

where,

θ = θα(~e) p0 = (θ, ρ)

ρ = ρβ(~e) p1 = χ

χ = χγ(~e)

Now that the mapping functions do not depend on anyspecific attribute, we can make the final step in our general-ization process, and extend them to operate on any collec-tion of attributes. More formally, we represent a collectionof attributes as:

~α = (α0, α1, . . . , αn)

and define the projection:

Π~α(~e) = (Πα0 , Πα1 , . . . , Παn)(~e)

= (Πα0(~e), Πα1(~e), . . . , Παn(~e))

and replace (α, β, γ) in equations 6 and 7 with (~α, ~β,~γ).That is, the three projections are each defined with respectto one or more attributes, and their value depends on somekind of combination of the values of these attributes for thegiven event, ~e.

Our final visualization paradigm, depicted in Figure 8,pro-vides a general framework for correlating disparate eventswith respect to any combination of attributes. We nowpresent a few examples of such a general visual correlationof events for situation awareness.

4.5 Example: Emergency Centers

Emergency forces such as the police, fire department, 911emergency centers, and recently the Department of Home-land Security are prime examples of areas where situationalawareness is a vital component of their everyday work. Forthe people in the field, SA basically means being aware of theenvironment they are embedded in. For these cases, our vi-sualization paradigm may not be the best solution. However,VisAware does fit well for the people at emergency centers

Figure 8: Generalized visual correlation of events for situationalawareness

who may need to be aware of large number of simultaneousevents of different emergency levels and at various locations.Furthermore, each event may include not only several simul-taneous emergencies, such as fire and wounded people, butmay require participation by many different units and forces.

Figure 9: Emergency awareness: Note that the ring includes bothevent types and resources such as hospital and emergency units (po-lice patrols, ambulances)

With respect to our generalized framework, the ρ(~e) func-tion is similar to the one in Examples 4.3.1 and 4.3.2. Thedomain of the θ(~e) on the other hand can now include re-sources, such police and other emergency units on or movingto the scene, as well as event types, such as fire and trafficoccident’s. Hospitals can also be listed on the ring to showwhere victims are being sent to. This can be especially im-portant in major incidents were the victims are sent to var-ious hospitals around the city. Finally, the the range of theχ function may be a map of a city, as depicted in Figure 9.

5 Conclusions and Future Work

In this paper we describe a flexible visualization frameworkfor increase situational awareness in various SA areas. Weprovide example of application of our framework to several

101

SA areas. We have not formally tested our framework, al-though VisAlert, the visualization for network intrusion de-tection, was alpha tested at Air Force Research Lab (AFRL)in Rome, New York, with very favorable initial reaction. Weare currently planning to install VisAlert in several beta sitesto systematically measure SA. We are also working on Vi-sAware, the second generation of VisAlert, that is based onthe generalized framework presented in this paper.

There are several techniques for measuring SA [4]. Thesetechniques can be loosely defined as objective measures,such as the Situation Awareness Global Assessment Tech-nique (SAGAT), subjective measures such as the SituationalAwareness Rating Technique (SART), and performance-based measures. The SAGAT asks probing questions aboutinformation displayed on the visualization during temporaryhalts to the user’s activity and blanking the screen. TheSART provides a subjective measure of SA by the users.The users rate their perception on a variety of scales such asdemand and understanding. Performance-based measuresprovide an outcome-based metric to assess the SA of theuser.

Each of these techniques has its advantages and disad-vantages. The SAGAT provides an objective measure of theuser’s ability to obtain information from the display; how-ever, the main disadvantage is the reliance upon the user’smemory and therefore the results may include a memorybias. The main advantage of SART is that it is very easyto implement; however, it may be influenced by the usersself-reporting bias. In addition to the SART and SAGATscores, performance data will also be collected to provide anaction-based assessment of the SA differences between usergroups. Although scores on the SART and SAGAT are of-ten not correlated [9], in this proposed investigation, SARTand SAGAT scores will be compared between groups of userswho use traditional tools and groups that use the VisAlertsystem.

We plan to use all three methods for assessing SA in asimulated network operational environment to probe users’ability to accurately understand the information that is pre-sented to them. We plan to simulate several attack scenar-ios and then measure how accurately and quickly they areable to extract this information from the display by stop-ping the simulation at predetermined intervals and askingquestions about the state of the network, how many alertsare associated with particular nodes, and what alerts are themost prevalent. Users will then take an appropriate actionthat will serve as the performance measure. In addition,at the end of the simulation we will use the SART methodto obtain indications of their perceived workload and per-formance. These numbers will be compared to the controlgroup that receives the same scenarios and questions butwho use the traditional tools.

Acknowledgments

We would like to thank ARFL’s Rome labs for hosting afield test of the VisAlert system. We owe special thanks toKirsten Whitley whose support of the VisAlert project wasinstrumental. Many thanks go to Geoff Draper, FernandoA. Iribarren and Xavier Tricoch for their valuable commentsand suggestions. This work was supported in part by a grantfrom the IC-ARDA.

References

[1] Herve Debar and Andreas Wespi. Aggregation and corre-lation of intrusion-detection alerts. In Recent Advances inIntrusion Detection, pages 85–103, 2001.

[2] Secure Decisions. http://www.securedecisions.com.[3] M. R. Endsley. Design and evaluation for situation aware-

ness enhancement. In Proceedings of the Human FactorsSociety 32nd Annual Meeting, pages 97–101, Santa Monica,CA, 1988. Human Factor Society.

[4] M. R. Endsley, R. Sollenberger, and E. Stein. Situationawareness: A comparison of measures. In Proceedings of theHuman Performance, Situation Awareness and Automation:User-Centered Design for the New Millennium, Savannah,GA, 2000. SA Technologies, Inc.

[5] Mica R. Endsley, Betty Bolte, and Debra G. Jones. De-signing For Situational Awareness, An Approach to User-Centered Design. Taylor & Francis, 2003.

[6] Deborah Estrin, Mark Handley, John Heidermann, StevenMcCanne, Ya Xu, and Haobo Yu. Network visualizationwith nam, the vint network animator. IEEE Computer,33(11):63–68, November 2000.

[7] Michels P. Gravenstein and D. Westenskow. Dr: An inte-grated graphic data display improves detection and identifi-cation of critical events during anesthesia. J. Clin. Monit,13:249–259, 1997.

[8] D. G. Jones and M. R. Endsley. Sources of situation aware-ness errors in aviation. Aviation, Space and EnvironmentalMedicine, 67(6):507–512, 1996.

[9] D. G. Jones and M. R. Endsley. Can real-time probes providea valid measure of situation awareness? In Proceedings of theHuman Performance, Situation Awareness and Automation:User-Centered Design for the New Millennium, Savannah,GA, 2000. SA Technologies, Inc.

[10] Clarence A. Robinson Jr. A powerful vision. Signal Maga-zine, August 2001. http://www.afcea.org/signal/articles.

[11] Stephen Eick Kenneth Cox and Taosong He. 3d geographicnetwork displays. ACM Sigmod Record, 25(4), 50 1996. De-cember.

[12] Kiran Lakkaraju and abd Adam J. Lee William Yurcik.Nvisionip: netflow visualizations of system state for secu-rity situational awareness. In Proceedings of CCS Workshopon Visualization and Data Mining for Computer Security,ACM Conference on Computer and Communications Secu-rity, October 29 2004.

[13] Yarden Livnat, Jim Agutter, Shaun Moon, Robert F. Er-bacher, and Stefano Foresti. A visualization paradigm fornetwork intrusion detection. In 6th IEEE Systems, Man andCybernetics Information Assurance Workshop, pages 92–99,West Point, NY, June 2005. IEEE.

[14] Jonathan McPherson, Kwan-Liu Ma, Paul Krys-tosekand Tony Bartoletti, and Marvin Christensen. Portvis:A tool for port-based detection of security events. In CCSWorkshop on Visualization and Data Mining for ComputerSecurity, October 2004.

[15] Quantum3D. http://www.quantum3d.com/stories/isd.htm.[16] K.L. Ma S.T. Teoh and S. F. Wu. Visual exploration process

for the analysis of internet routing data. In IEEE Conferenceon Visualization 2003, pages 523–530, 2003.

[17] Alfonso Valdes and Keith Skinner. Probabilistic alert cor-relation. In Recent Advances in Intrusion Detection, pages54–68, 2001.

[18] Alex Wood. Intrusion detection: Visualizing attacks in idsdata. Giac gcia practical, SANS Institute, February 2003.

[19] Xiaoxin Yin, William Yurcik, Michael Treaster, Yifan Li, andKiran Lakkaraju. Visflowconnect: netflow visualizations oflink relationships for security situational awareness. In Pro-ceedings of CCS Workshop on Visualization and Data Min-ing for Computer Security,ACM Conference on Computerand Communications Security, October 29 2004.

102