Design of an Intrusion Response System using Evolutionary...

Design of an Intrusion Response System usingEvolutionary Computation

Rohit [email protected]

Research Paper As Requirementfor

CS401 (Evolutionary Computation)Instructor: Dr. Daniel R. Tauritz

[email protected]

Computer Science DepartmentUniversity of Missouri-Rolla

December 5, 2003

1

Abstract

While the Internet epoch has made networks omnipresent, it hasalso rendered networked systems under unprecedented threat of net-work based intrusion.

There has been an increasing need for an automated response sys-tem that can handle these intrusions. However a major challengearises in building an automated response mechanism, because an au-tomated response may defend the system against an attack, but alsohas tendency to cause unwanted effects against legitimate users.

This paper presents an evolutionary computation method to tacklethis problem. A response model is presented that selects the bestresponse among other alternatives, such that the selected responsehas the least impact on the whole system.

1 Keywords

Computer Security, Intrusion Response, Automated Intrusion Response, En-tity, Dependency, Dependency Tree, Capability, Dependency Degree, FinalState of Network, Response History Agent, Effect of Response, Evaluatingthe Network State

2

Contents

1 Keywords 2

2 Introduction 5

3 Intrusion Response 63.1 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . 63.2 Ideal Response Mechanism . . . . . . . . . . . . . . . . . . . . 7

4 Challenges when building an Automated Intrusion ResponseSystem 7

5 Responses 8

6 Response Model 86.1 Network Elements . . . . . . . . . . . . . . . . . . . . . . . . . 96.2 Entity Dependencies and Dependency Trees . . . . . . . . . . 10

6.2.1 Dependency . . . . . . . . . . . . . . . . . . . . . . . . 106.2.2 Dependency vs. Availability . . . . . . . . . . . . . . . 116.2.3 Dependency Trees . . . . . . . . . . . . . . . . . . . . . 126.2.4 Capability . . . . . . . . . . . . . . . . . . . . . . . . . 126.2.5 Capability Calculation . . . . . . . . . . . . . . . . . . 126.2.6 Constructing Dependency Trees . . . . . . . . . . . . . 136.2.7 Dependency Degree . . . . . . . . . . . . . . . . . . . . 14

6.3 Evaluating the Network State . . . . . . . . . . . . . . . . . . 156.4 Response Mechanism and EC approach . . . . . . . . . . . . . 15

6.4.1 Representing Responses . . . . . . . . . . . . . . . . . 166.4.2 The EC approach . . . . . . . . . . . . . . . . . . . . . 186.4.3 Evaluation Function . . . . . . . . . . . . . . . . . . . 196.4.4 Response History Agent . . . . . . . . . . . . . . . . . 20

7 Experimental Model 207.1 Experimental Setup . . . . . . . . . . . . . . . . . . . . . . . . 207.2 Experimental Methodology . . . . . . . . . . . . . . . . . . . . 24

7.2.1 Selection . . . . . . . . . . . . . . . . . . . . . . . . . . 247.2.2 Recombination . . . . . . . . . . . . . . . . . . . . . . 257.2.3 Mutation . . . . . . . . . . . . . . . . . . . . . . . . . 257.2.4 Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . 257.2.5 Competition . . . . . . . . . . . . . . . . . . . . . . . . 25

7.3 The Parameter File . . . . . . . . . . . . . . . . . . . . . . . . 257.4 Tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

3

8 Conclusion 27

9 Appendix 30

4

2 Introduction

The number of computer attacks has been increasing each year. Not onlyare these attacks becoming more numerous, but they are also becoming moresophisticated [1].

Even with the most advanced protection using firewalls, effective securitypolicies and the use of identification and authentication, computer systemsare still not hundred percent secure. The complexity of information systemsand the difficulty in designing, configuring, and maintaining security compo-nents makes it virtually impossible to prevent network-based intrusions. Asa result, we must develop systems to discover (Intrusion Detection Systemsor IDS) and react (Intrusion Response Systems or IRS) to computer attacks.

While intrusion detection has been the focus of significant research overthe last few decades, the issue of response has received relatively little atten-tion [5]. This can be justified to the fact that during the last few decadesthe number of computers to supervise was considerably lesser. A manualresponse by the system administrator was enough to deter the intrusion.

The complexity and size of the information systems has considerablygrown, and has been growing. As pointed out earlier, there has also been arise in the number of attacks, the speed of attacks, and the sophisticationof attacks. Even with very effective intrusion detection systems, it is quiteunreasonable to expect a human system administrator to manually interveneto impede the attack, notice the magnitude of the damage done, and recoverand restore the system to a well known state. The solution is to developan automated response system that could generate a defensive or correctiveaction without waiting for the system administrator to react.

In this paper, we present a response model to evaluate the effect of intru-sion response mechanisms to the operation of network services, thus enablingthe IRS to choose the best alternative from a set of possible alternatives. Ourmodel takes into account the network topology and the dependencies betweendifferent entities to capture the consequences of responses more accurately.Based on this model, an evaluation function can estimate the impact of var-ious responses and select the one which has the least negative effect on thesystem.

The paper also presents an overview of the possible responses the sys-tem can generate, and explains the evolutionary computation based responsemechanism used. Finally the paper talks about the experimental system de-veloped for tests and results, which are also shown.

5

3 Intrusion Response

When an intrusion is detected, a series of actions are taken to counterattackthe attacker, prevent further attacks or restore the system to a normal state.These actions represent intrusion response.

Intrusion response today primarily remains a manual process. The IDSnotifies the system administrator that an intrusion has occurred or is occur-ring, and the system administrator must respond to the intrusion. Notifi-cation or alert to the administrator could be either by displaying a pop-upwindow, generating an email, pager or mobile phone message. Such systemscan be classified as passive systems, as against active systems1.

The obvious disadvantage of such a system is that they do nothing toimpede the intruder, and rely upon someone to manually respond at somelater point. Unfortunately, system administrators neither can keep up withthe pace that an IDS is delivering alerts, nor can they react upon these withinadequate time limits. In other words, there is a delay between detection ofa possible intrusion and response to that intrusion.

Response is a fundamental factor in whether or not an attack is successful.Research by Cohen [3] shows that if a skilled attacker is given 10 hoursbetween the intrusion and the response, they will succeed 80 percent of thetimes. If the delay in response is 20 hours, they will succeed 95 percent of thetimes. At 30 hours the attacker almost never fails. However, if the response isinstantaneous, the probability of a successful attack against a skilled systemadministrator is almost zero.

The above discussion shows the need for an automated intrusion responsesystem. Before we discuss our approach to solving the problem of automatedintrusion response, we explore existing work in this area.

3.1 Related Work

As reported by [2], response systems started of as being Notification Systems,adapted to become Manual Response Systems, and are moving to becomeAutomatic Response Systems.

A Notification System is one that generates a report, and waits for thesystem administrator to deal with the intrusion. A Manual Response Systemalso generates an alert to the system administrator. However, with a manualresponse system, the system administrator is given the capability to launcha response from a limited pre-programmed set of responses. Such systems

1Active response systems refer to automated response systems which take actions tocounter the incident that has occurred

6

support an administrator by offering ready-to-apply reconfiguration mech-anisms in order to quickly secure the system. Automated response systemsimmediately answer an attack with pre-programmed responses without hu-man intervention. They hence offer a quick response, reducing the windowof opportunity for the attacker.

Most current systems implementing automatic response mechanisms usesimple decision tables to determine how to react in the case of identifiedattacks. More sophisticated variants apply expert systems to perform thattask.

3.2 Ideal Response Mechanism

According to the NIST Interim Report–6416 [4] an ideal response to an in-trusion would be to gather evidence of the intrusive activity, remove theattackers access to the network, undo any damage if done, and reconfigurethe network parameters to resist against similar attacks. However this is nextto impossible since humans themselves have difficulty performing it. Never-theless it is possible to automate an approximation of the ideal response. Theapproximations are the set of pre-programmed responses that our automatedresponse system can generate. These are discussed in Section 5.

4 Challenges when building an Automated

Intrusion Response System

Countermeasures taken by an IRS during an intrusion, not only defend thesystem against the detected attack, but can also have negative effects onlegitimate users. This can lead to the possibility of a response action causingmore damage than the actual attack. This is a major challenge in buildingan automated response mechanism.

Another severe problem is false alarms. Intrusion Detection Systems arenot perfect and can generate false alarms or false positives. This has animpact on response as uncertainty is generated in formulating a response.

This paper does not currently look at the “false positives” issue, and con-centrates more on choosing a response mechanism that has the least impacton the system.

7

5 Responses

A response action is a set of operations that are performed when an intrusionis detected.

In our discussion, the following operations are available for use in a re-sponse action.

1. install/remove firewall rule

2. enable/disable user accounts

3. add/remove user permissions

4. enable/disable ports

5. backup system

6. shutdown host

The available operations are not limited to the ones listed, and can alwaysbe expanded. In Section 6.4.1 we will discuss about how an action can berepresented and used.

6 Response Model

The following section introduces the response model used to choosing anappropriate response action from among the possible alternatives. Each re-sponse action is evaluated, and the evaluation process uses ideas that wereintroduced by [6]. The ideas taken from [6] will be indicated as we comeacross them.

However before we get into the details of the response model, the basicelements of the model are presented.

The basic elements as described by [6] are:

- Resources (services provided by the hosts)

- System Users (users of the network)

- Network Topology (the underlying communication infrastructure)

- Firewall Rules

8

We will then talk about dependencies and entities and their representationcalled dependency trees. As will be explained, these trees will represent thestate of the network, and will help determine the effects of responses. Afterthat the evolutionary computation approach is presented that will elucidatethe process of choosing the response action.

6.1 Network Elements

This section describes the network elements as defined by [6]. Networks arecomplex structures that include many elements which are heavily related anddependent on each other.

In the following sections “system users” and “resources” are togetherreferred to as entities. The terms “entities” and “resources” or “entities”and “system users” are used interchangeably. Sometimes the word “service”is used interchangeably with “resource”.

For our model, the following elements are relevant

Resources: Resources are the basic building blocks of our response model.A resource is a network service offered by hosts. These services are pro-vided by a process running on the host. Typical examples of resourcesinclude (but are not limited to) DNS, NFS, NIS, HTTP and FTP.

It is also important to note that resources can depend on other re-sources to various degrees. As an example NFS can depend on DNSfor authorized file sharing.

A process provides resources to others by listening on a predefinedport. The users and other resources submit the requests to this port.Requests are processed and a reply is sent back to the originator of thequery.

In our model only resources that are used by other entities have to beincluded, and processes running at a host without providing services toexternal entities are not considered to be resources.

System Users: System users refer to the people that utilize the providedresources in the network. Users perform their daily tasks by utilizingthe provided resources. They can also assign different levels of impor-tance to resources.

Network Topology: The network topology is the underlying communi-cation infrastructure that helps the various entities to communicateamong each other. The network topology plays an important role whenwe evaluate the current state of the network as we will see later.

9

Firewall Rules Firewall rules are used to protect the network and effectthe availability of the entities in the network.

It is an important observation that two entities located in the samesubnet are not affected by changes in firewall rules. Changes in firewallrules affect entities in case the communication path from one entity toanother leads through the firewall, in which case its rules influence theavailability of the resource.

While the network topology is the glue between resources by providingcommunication paths, the firewall rules can be viewed as a method for im-posing constraints on these paths by allowing or disallowing certain traffic.

6.2 Entity Dependencies and Dependency Trees

This section again explains some concepts defined by [6].

6.2.1 Dependency

A dependency is a relation between two entities. An entity that needs aservice that is provided by another entity to be fully operational, is said tobe dependent on that entity.

A dependency between two entities can be of the following two types:

- Direct dependency

- Indirect dependency

A direct dependency represents a dependency of an entity on a service.An example would be a user ‘A’ (entity) that depends on a DNS server(service) to resolve DNS names.

An indirect dependency is a dependency that is formed due to the net-work topology and the firewall rules. The network topology and the firewallrules introduce new artificial dependencies between entities and their neededresources. This is caused by the fact that information exchange has to takeplace over routers and must be allowed by all firewall rules that are in effecton the communication path.

Consider an example for a better understanding of direct and indirectdependencies. In Figure 1, user ‘A’ accesses a DNS server in a differentsubnet for DNS name resolution. There is a direct dependency between user‘A’ and the DNS server. The figure also shows indirect dependencies betweenthe gateways that connect the different subnets (for the sake of simplicityrouting information as well as firewall rules are omitted).

10

User A GW

DNS Server GW

GW

Subnet 1

Subnet 3

Subnet 2

Direct Dependency

Indirect Dependency

Figure 1: Direct and Indirect Dependencies

An important note here is that indirect dependencies are a preconditionfor fulfilling a direct dependency. If an indirect dependency is not fulfilled,a direct dependency relying on it can never be fulfilled too.

6.2.2 Dependency vs. Availability

It is also important to note that availability of a resource is a key factor whenwe look at dependencies between two entities. An entity is considered to beavailable for a dependent one if:

1. communication between both is possible and

2. the entity providing the service is functional (i.e. the process providingthe service is running).

The communication between two entities is possible if:

1. there is a route provided by the underlying network topology betweenboth and

2. all hosts on the route permit the traffic between them.

11

6.2.3 Dependency Trees

We use dependency trees in our model to describe all the relationships ofan entity with other entities. An entity usually depends on several resources(services) in a network. Some entities need access to more than one service,which can be described by an COMBINE-dependency (and an COMBINE-node in the dependency tree). For others it is sufficient to have access to atleast one of a set of (similar) services which can be described by an CHOICE-dependency (and an CHOICE-node in the dependency tree). An exampleshowing the construction of a dependency tree will be shown in Section 6.2.6.

6.2.4 Capability

The capability c(r) of an entity r is a value ranging from 0.0 to 1.0 anddescribes in how far an entity ‘r’ can perform its work given the current net-work configuration. When all the resources the entity uses are available, thecapability is 1.0. When the capability value is determined for an entity, thecommunication paths to all the resources that it depends on are examined.Hence a given services’ unavailability for a given entity that uses that servicedecreases the value of c(r).

The calculation of capability is explained in the following section.

6.2.5 Capability Calculation

The types of nodes of the dependency tree determine which formulas are usedto calculate the capability values obtained from the subtrees below. In thefollowing discussion c(left) and c(right) will be used to denote the capabilityof the left and right link of a node respectively, and c describes the capabilityof any intermediate node (this value is calculated).

When examining the dependency tree we see the following kinds of nodes:

Leaf Node: The leaf node describes an entity that does not depend onother entities. The availability of the entity which determines whetherit provides service to other entities or not, determines its capability.

- If the entity provides service, the capability is set to 1.0.

- If the entity does not provide service, the capability is set to 0.0.

COMBINE-Node: The entity depends on the availability of both the en-tities defined by the COMBINE-node. The value c for the intermediatenode is calculated as follows:

12

- c = (c(left)+ c(right))/2

CHOICE-Node The entity depends on the availability of any one of thetwo entities defined by the CHOICE-node. The value c for the inter-mediate node is calculated as follows:

- c = Max (c(left), c(right))

6.2.6 Constructing Dependency Trees

Consider a user ‘A’ (entity) that uses the NFS server, the DNS server, andone of the two domain name servers DNS1 and DNS2, to accomplish all histasks.

The relationships can be expressed using a dependency tree as shown inFigure 2.

c=1.0

c=1.0

c=1.0

c(left)=1.0

c=1.0

HTTP

NFS

DNS1 DNS2

c=1.0

c=1.0

c(r)=1.0

r=User ’A’

c(right)=1.0

c(left)=1.0

c(left)=1.0

c(right)=1.0

c(right)=1.0

COMBINE

COMBINE

CHOICE

Figure 2: Ideal Dependency Tree for User ‘A’

In the figure the DNS, NFS, and HTTP servers are all available. Hencec=1.0 for all of them. To determine the value of c for an COMBINE-node or

13

an CHOICE-node, we use the relations defined in Section 6.2.5. Finally, thefigure also shows that the value calculated for c(r) is 1.0, where r representsuser ‘A’. It is also important to note that to calculate the value for c(r); weuse a depth first search on the dependency tree.

As another example consider the NFS server unavailable. The value ofc(r) now is 0.75, as shown in Figure 3

c=1.0

c=0.0

c=0.5

c(left)=1.0

c(r)=0.75

c(right)=1.0

c(left)=1.0

c(left)=0.5

c(right)=0.0

c(right)=1.0

c=1.0

HTTP

NFS

DNS1 DNS2

c=1.0

c=1.0

r=User ’A’

COMBINE

COMBINE

CHOICE

Figure 3: Dependency tree for User ‘A’ when NFS server is unavailable

6.2.7 Dependency Degree

The dependency degree describes in how far the operation of an entity isaffected if the resource, which it depends on, is no longer available. Theintroduction of a dependency degree can be best motivated by the followingexample. Consider a user that uses his machine mainly to surf the Internet.In our network model, the entity (representing this user) will depend a lot onthe availability of the DNS server and the HTTP server (dependencies withhigh degrees), but not on the NFS server. On the other hand, a user editing

14

files on the remote NFS machine will mainly need this service to accomplishhis work (dependency with a high degree).

Introducing the notion of dependency degree, we make the followingchanges to our capability calculation method described earlier in Section 6.2.5.The c(left)/c(right) now denotes capability of the left and right link of nodemultiplied with the dependency degree, when the left/right node is a leaf.i.e.

- c(left)=c(left)*dependency degree

- c(right)=c(right)*dependency degree

Consider the changes to Figure 2 when user ‘A’ mainly surfs the internet.In this case, the availability of the DNS servers and the HTTP server iscritical. We set a high degree of dependency, say 100 percent. The availabilityof the NFS server is not so critical and we set the dependency degree to say75 percent. The calculation of the capability of user ‘A’ now is 0.9375 asshown in Figure 4

6.3 Evaluating the Network State

So far we have considered only a single entity in the network, and looked atits relationship with other entities. However there are many entities heavilydependent on other entities in a network. In our response model we createdependency trees for every such entity.

To calculate the final state of the network we calculate the total capabilityof the network by averaging the capability values of all the dependency treescreated over all entities.

An issue in this approach is that in a network if a particular service isunavailable, it could also affect the availability of another service. This couldlead to cyclic dependencies. We can avoid cyclic dependencies by creatinganother dependency tree for the depending service.

6.4 Response Mechanism and EC approach

A response action is a set of operations that can be utilized to avert a certainthreat. There can be many types of response operations which were discussedin Section 5. Each response action might achieve the desired result. It is thetask of the IRS to choose the one with the least impact.

To do this, a response action is temporarily added to the network modeland the model is evaluated using the impact evaluation function. This func-tion evaluates the network state as discussed in Section 6.3 and returns thetotal capability of the network as a value between 0.0 and 1.0.

15

c=1.0

c=1.0

c=0.875

c(left)=1.0

c(r)=0.9375

c(right)=1.0

c(left)=1.0

c(left)=0.875

c(right)=1.0 x 3/4=0.75

c(right)=1.0

c=1.0

HTTP

NFS

DNS1 DNS2

c=1.0

c=1.0

r=User ’A’

COMBINE

COMBINE

CHOICE

Figure 4: Depencency Tree for User ‘A’ with a Dependency Degree for theNFS server

From among the response actions added to the model, the one with thehighest ‘total capability’ is chosen by the IRS. This response action is thenadded permanently to the network.

6.4.1 Representing Responses

As discussed in Section 5, a response is a set of operations performed whenan intrusion is detected. In our response model, each individual is a responseaction which is represented as a binary string of bits. Each bit is associatedwith an operation on a host that provides service.

Figure 5 illustrates the idea.Figure 5 shows the representation of a response action. As can be seen

in the figure, there are bits for installing/removing firewall rules, for en-abling/disabling user accounts, for adding/deleting user permissions, en-abling/disabling ports, for backing up a system, and for starting/shuttingdown hosts.

16

Eac

h bi

t ind

icat

es a

FIR

EW

ALL

RU

LEto

inst

all/r

emov

e

Eac

h bi

t ind

icat

es a

US

ER

AC

CO

UN

Tto

ena

ble/

disa

ble

Eac

h bi

t ind

icat

es a

US

ER

PE

RM

ISS

ION

to a

dd/d

elet

e

Eac

h bi

t ind

icat

es a

PO

RT

to e

nabl

e/di

sabl

e

Eac

h bi

t ind

icat

esth

e S

YS

TE

M to

BA

CK

UP

or

not

Eac

h bi

t ind

icat

es a

syst

em to

ST

AR

T/S

HU

TD

OW

N

1=In

stal

l0=

Rem

ove

1=E

nabl

e0=

Dis

able

1=A

dd

0=D

elet

e

1=E

nabl

e0=

Dis

able

1=B

acku

p0=

No

Bac

kup

1=S

tart

0=S

hutd

own

Figure 5: Response Representation17

During the evaluation phase of the model when a response action is addedto calculate its fitness, the following assumption is made. If the responseaction indicates an operation to be performed and the operation is alreadyin effect, it is ignored. Consider an example to elucidate this.

If a response action indicates that a particular firewall rule be installed(removed), and that rule is already installed (not installed), the responseaction ignores the rule

All individuals that are involved in the EC process use this representationscheme.

6.4.2 The EC approach

The objective of the project is to develop a response mechanism to counterthe intrusions that will have the least impact on the system.

Figure 6 shows our approach to the problem. A partial initial populationis created looking at the response history agent. This is analogous to choosinga “knowledge based” initial population of solutions. More members of thepopulation are initialized till a predetermined population size is reached. Thealgorithm then goes through the evolutionary cycle, and generates a solutionthat is then added to the Response History agent.

ResponseHistoryAgent

PartialInitialize

Initialize Evaluate

Select

ReproduceEvaluate

Compete

Respond

Figure 6: The EC approach

The algorithm below outlines the approach

18

1. Generate a partial initial population; with population members selectedfrom the Response History Agent. Select only those responses that havea similar intrusion (see Section 6.4.4).

2. Randomly initialize additional members of the population, till a definedpopulation size is reached.

3. Calculate the fitness of each individual in the population. This is fur-ther explained in Section 6.4.3

4. Select individuals from the population for reproduction.

5. Create children by recombining the individuals selected.

6. Mutate the children created, to create new Children

7. Calculate the fitness of each child.

8. Add all children to the population.

9. Use competition to eliminate individuals with low fitness (and till thepopulation size is the predetermined size)

10. If maximum number of generations is reached then goto step 4.

11. Select the best population member as the response action.

12. Record the intrusion specifics and the response into the Response His-tory Agent.

6.4.3 Evaluation Function

To calculate the fitness of each individual, we temporarily add the responseaction (defined by the individual) temporarily to the model. The total capa-bility of the network is then calculated as explained earlier in Section 6.3.

A penalty function is also associated with the evaluation process as fol-lows:

- For a mild attack, and a severe response, associate a penalty to thefitness.

- For a severe attack, and a mild response, associate a penalty to thefitness.

The mildness/severity of the attack can be determined from the IDSreport, which is an indication of the specifics of the attack.

19

6.4.4 Response History Agent

The Response History Agent stores information about the attack and theresponse to that attack.

Information about the attack is stored as “reports” generated by the IDS,and information about response is stored as a binary string that representsthe response action(s).

It was mentioned earlier that some of the population members are ini-tialized by selecting responses from the Response History Agent, that have“similar” intrusive patterns. The attack within the Response History Agentis considered “similar” to the current attack, if many of the variables withinthe report (that indicate the type of intrusion) are same.

If an exact attack had previously occurred, we can also have the optionto generate the response that was previously generated.

Also as new attacks are generated, information about the attack and theresponse action taken are stored in the Response History Agent.

7 Experimental Model

An experiment was conducted on the network represented in Figure 7 2.The objective of this experiment is to choose the best response from a

pool of response actions. Each response has an effect on the availability ofthe network services and the response that causes the least negative effect(this happens when more of the services are available) is chosen.

In the following sections the Experimental Setup and Results are dis-cussed.

7.1 Experimental Setup

Dependency trees were first constructed and hard coded into the program.The dependency trees used for this experiment are shown in Table 1.

The dependency degrees that were used for each service are shown inTable 2.

With the following setup, when all the services are available and all therouters allow communication, the total network capability was found to be0.873177. The experimental model will attempt to achieve the same value.

As discussed earlier, the response is a binary string of bits, with each bitrepresenting an operation. The experiment setup uses a 19 bit binary vector,which is shown in Figure 8.

2Constructed from a personal communication with Anthony D. Rich

20

HT

TP

D1

DN

S1

NF

S2

Alic

e

HT

TP

D2

Jane

t

HT

TP

D3

Har

ryIr

ina

MX

DN

S3

Geo

rge

NF

S1

Fra

nk

FT

PD

Bob

DN

S2

Cha

rlie

Deb

bie

Edd

ie

Ro

ute

r 0

Ro

ute

r 1

Ro

ute

r 2

Ro

ute

r 3

Ro

ute

r 4

Ro

ute

r 5

Ro

ute

r 6

Ro

ute

r 7

Ro

ute

r 8

Figure 7: Experimental Network

21

Eac

h B

it w

hich

is a

ssoc

iate

d w

ith a

par

ticul

ar S

ervi

cein

dica

tes

ST

AR

T/S

HU

TD

OW

N o

f tha

t Ser

vice

DN

S1

DN

S2

DN

S3

FT

PD

HT

TP

D1

HT

TP

D2

HT

TP

D3

MX

NF

S1

NF

S2

Rou

ter

0

Rou

ter

1R

oute

r2

Rou

ter

3R

oute

r4

Rou

ter

5

Rou

ter

6R

oute

r7

Rou

ter

8

Bit

0B

it 1

Bit

2B

it 3

Bit

4B

it 5

Bit

6B

it 7

Bit

8B

it 9

Bit

10B

it 11

Bit

12B

it 14

Bit

15B

it 16

Bit

17B

it 18

Bit

13

1=S

tart

0=S

hutd

own

Eac

h B

it w

hich

is a

ssoc

iate

d w

ith a

par

ticul

ar R

oute

r in

dica

tes

ALL

OW

ING

/DIS

ALL

OW

ING

of R

oute

r C

omm

unic

atio

n

1=A

llow

0=D

isal

low

Figure 8:22

Entity Services Used by Entity

Alice DNS1 and FTPD and HTTPD1 and HTTPD2and HTTPD3 and MX and NFS2

Bob DNS2 and FTPD and HTTPD1 and HTTPD2and HTTPD3 and MX and NFS2

Charlie DNS2 and FTPD and HTTPD1 and HTTPD2and HTTPD3 and MX and (NFS or NFS2)

Debbie (DNS2 or DNS3) and FTPD and HTTPD1and HTTPD2 and HTTPD3 and MX and NFS1

Eddie (DNS2 or DNS3) and FTPD and HTTPD1and HTTPD2 and HTTPD3 and MX and NFS1

Frank DNS3 and FTPD and HTTPD1 and HTTPD2and HTTPD3 and MX and NFS1

George DNS3 and FTPD and HTTPD1 and HTTPD2and HTTPD3 and MX and NFS1

Irina (DNS1 or DNS3) and HTTPD1 and HTTPD2and HTTPD3 and MX and NFS2

Harry (DNS1 or DNS3) and HTTPD1 and HTTPD2and HTTPD3 and MX and NFS2

Janet (DNS1 or DNS3) and HTTPD1 and HTTPD2and HTTPD3 and MX and NFS2

HTTPD1 DNS1 or DNS2 or DNS3HTTPD2 DNS1 or DNS2 or DNS3HTTPD3 DNS1 or DNS2 or DNS3FTPD DNS1 and DNS2 and DNS3MX DNS1 and DNS2 and DNS3

Table 1: Dependency Trees Used

The first ten bits of this vector indicates shutdown/start of services thatthe bits are assigned to, while the remaining nine bits are assigned for allow-ing/disallowing router communication; i.e. if the bit associated with a routeris 1, then that router will allow communication between the subnets that itis connected to, otherwise it will not (this emulates the concept of firewallrules).

While calculating the capability of a Dependency Tree for an entity, thecommunication of the entity with all the services the entity uses is checked.If the communication path cannot be established, the service is marked asunavailable. For an entity that uses a certain service, some routers have toallow communication, while others may or may not. Table 7 in the Appendix

23

Resource Dependency Degree (in percentage)

DNS1 100DNS2 100DNS3 100FTPD1 75HTTPD1 100HTTPD2 100HTTPD3 100MX 75NFS1 75NFS2 75

Table 2: Dependency Degrees Used

indicates which routers have to be turned on for a particular communicationbetween an entity and the service it uses.

These rules have also been hard coded into the program.

7.2 Experimental Methodology

The file “respExpt.cpp” is the central code for running the experiment. Thefile “ga.cpp” contains code for all the genetic algorithmic operations and thefile “eval.cpp” contains code necessary for evaluating the network state.

The program reads in the parameters from the file “par.dat” (discussedin Section 7.3 ) and then initializes members of the population. The numberof population members created are based on the parameter “populationSize”specified in the “par.dat” file.

Each member of the population stores information related to its fitness,as well as a 19 bit binary vector that is indicative to the response action to betaken. The binary vector is randomly initialized and based on this the fitnessof the individual is calculated and stored (as explained in Section 6.4.3).

After evaluating the members the population goes through the evolution-ary cycle that was discussed in Section 6.4.2 3.

7.2.1 Selection

Selection is rank based. A certain percentage (specified in the parameterfile) of the population members are chosen. Half of these members are then

3for simplicity, the concept of Response History Agent and Partial Initialization hasnot been implemented

24

selected for recombination.Each of these selected members is involved in the recombination process

only once, and recombines with another member of the whole population,that is randomly selected. (This could also cause the member to recombinewith itself albeit the chances of this is low)

7.2.2 Recombination

The members selected for recombination recombine two at a time. Thetwo members that recombine either use an n-Point Crossover or UniformCrossover. The type of crossover chosen is based on the “nPointCrossover”parameter specified in the “par.dat” file.

Each recombination of two individuals creates two new individuals, whichare both added to the child population.

7.2.3 Mutation

A certain percentage (specified in the parameter file) of the child populationis selected for mutation. Each member selected can produce a certain numberof children, which is also specified in the “par.dat” file.

Each child produced is added to the child population.

7.2.4 Evaluation

All members within the child population are evaluated as they are created.

7.2.5 Competition

In the competition phase, all the child population members are added to theparent population. The population is sorted according to the best memberfirst, and then the population size is dropped to the original population sizethat was specified in the “par.dat” file.

7.3 The Parameter File

Seed: This specifies the seed that will be used by the random number gen-erator.

populationSize: To indicate the population size to be used. The populationsize specifies the number of members that will exist in the population

maxGenerations: The evolutionary cycle will run the number of genera-tions specified here

25

printEveryPGenerations: To direct the program to print the populationdetails every “printEveryPGenerations” generations.

recombinePercent: specifies the percentage of the parent population thatis selected for recombination.

mutatePercent: specifies the percentage of the child population that isselected for mutation

nPointCrossover: If set to ‘0’, the algorithm performs “Uniform Crossover”as a recombination operator. To perform “n-Point Crossover”, we canset the value to any integer n > 0. If n ≥ 19, recombination is notperformed. This is because each response action is represented by a 19bit binary vector.

noOfMutatedChildren: Each child that is involved in the mutation pro-cess produces a certain number of children indicated by this parameter.

logFile: This is the file where all the output of the program will be stored.

7.4 Tests

From this basic genetic algorithm a number of different behaviors could beobserved. Numerous different parameter settings were tried and results fromthese trials can be seen in Table 3.

From the trials it can be seen that perhaps the most important parametersare the crossover and mutation percentages. This can be deduced by lookingat Trial 1 and Trial 2. Although Trial 2 converged to the solution in lessergenerations, it took more computational time.

It was also observed that a Uniform Crossover caused faster convergenceby comparing Trial 2 and Trial 3.

In Trial 4, a 15 Point Crossover was used. The solution achieved was notthe global maximum, and stagnation occurred at generation 10.

Trial 5 increased the crossover and mutation percentage of Trial 4, andit was observed that the algorithm converged at generation 3. Also an in-crease in mutation and crossover chance caused the solution to achieve globalmaximum.

Trial 6 can be compared to Trial 1, and it was observed that increasingthe number of children produced during mutation slowed the convergence.

Also increasing the number of parents involved in mutation further slowedthe convergence (as can be noted by comparing Trial 6 and Trial 7).

In Trial 8, the population size was drastically reduced. This preventedthe solution from achieving global maximum.

26

Trial Population Recombine Mutate n-Point No Of CommentsNo. Size Percent Percent Crossover Children

1 50 10 10 1 1 Peak Stagnation atgeneration 9



4 50 10 10 15 1 Stagnation atgeneration 10




8 5 10 75 1 1 Stagnation atgeneration 5


Table 3: GA Trials, Parameters and Results

In Trial 9, the population size was increased but was still kept low. GlobalMaximum was achieved but at generation 39.

Figures 9 through 17 are a plot of the behavior of the algorithm for thedifferent parameters sets defined by Trial 1 through 9. These figures illustratethe different convergence rates and whether the global optimal solution wasreached or not.

8 Conclusion

We have presented a network model together with an evaluation functionthat can be consulted by an intrusion response system to determine the re-sponse action which yields the minimal negative impact on deployed networkresources and their users.

We propose a network model that takes network topology, firewall rules,services and users into account and supports both, dependencies among en-tities within the network and those to and from outside users. This allows usto determine the costs of disabling crucial resources in a response function.

27

0.79

0.8

0.81

0.82

0.83

0.84

0.85

0.86

0.87

0.88

0 5 10 15 20 25 30

fitne

ss

generations

Trial 1

Figure 9: Trial 1 Results

The evaluation mechanism which determines the negative impact allows usto choose the best action among a set of possible alternatives.

To choose the best response action, an Evolutionary Computation modelwas discussed, and an experiment was conduced, to test the working of theprocess.

28

0.79

0.8

0.81

0.82

0.83

0.84

0.85

0.86

0.87

0.88

0 5 10 15 20 25 30

fitne

ss

generations

Trial 2


References

[1] Cert coordination center, 2002 annual report. 2002.

[2] Curtis A. Carver, John M. D. Hill, and Udo W. Pooch. Limiting uncer-tainty in intrusion response. In Proceedings of the 2nd IEEE InformationAssurance and Security Workshop, West Point, NY, USA, June 2001.

[3] Fred Cohen. Simulating cyber attacks, defenses, and consequences.March, 1999.

[4] W. Jansen, P. Mell, T. Karygiannis, and D. Marks. Applying mobileagents to intrusion detection and response. NIST Interim Report 6416,National Institute of Standards and Technology, October 1999.

[5] Maria Papadaki, Steven M. Furnell, S. J. Lee, Benn Lines, and Paul L.Reynolds. Enhancing response in intrusion detection systems. Journal ofInformation Warfare, 2(1):90–102, 2002.

[6] Thomas Toth and Christopher Kruegel. Evaluating the impact of auto-mated intrusion response mechanisms. In 18th Annual Computer Secu-

29

0.79

0.8

0.81

0.82

0.83

0.84

0.85

0.86

0.87

0.88

0 5 10 15 20 25 30

fitne

ss

generations

Trial 3


rity Applications Conference, Las Vegas, Nevada, USA, December 2002.ACM.

9 Appendix

30

0.79

0.8

0.81

0.82

0.83

0.84

0.85

0.86

0 5 10 15 20 25 30

fitne

ss

generations

Trial 4


31

0.79

0.8

0.81

0.82

0.83

0.84

0.85

0.86

0.87

0.88

0 5 10 15 20 25 30

fitne

ss

generations

Trial 5


32

0.79

0.8

0.81

0.82

0.83

0.84

0.85

0.86

0.87

0.88

0 5 10 15 20 25 30

fitne

ss

generations

Trial 6


33

0.79

0.8

0.81

0.82

0.83

0.84

0.85

0.86

0.87

0.88

0 5 10 15 20 25 30

fitne

ss

generations

Trial 7


34

0.785

0.79

0.795

0.8

0.805

0.81

0.815

0.82

0.825

0.83

0.835

0 10 20 30 40 50 60

fitne

ss

generations

Trial 8


35

0.78

0.79

0.8

0.81

0.82

0.83

0.84

0.85

0.86

0.87

0.88

0 10 20 30 40 50 60

fitne

ss

generations

Trial 9


36

User to Service 0 1 2 3 4 5 6 7 8

Alice to DNS1 * * * * * * * * *Alice to FTPD 1 1 * * * * * * *Alice to HTTPD1 1 * * * * * * * *Alice to HTTPD2 1 * * 1 * * 1 * *Alice to HTTPD3 1 * * 1 * * * * *Alice to MX 1 1 * * 1 * * * *

1 * * 1 1 * * * *Alice to NFS2 * * * * * * * * *Bob to DNS2 * 1 1 * * * * * *Bob to FTPD * 1 * * * * * * *Bob to HTTPD1 1 1 * * * * * * *Bob to HTTPD2 * 1 * 1 * * 1 * *Bob to HTTPD3 * 1 * 1 * * * * *Bob to MX * 1 * * 1 * * * *Bob to NFS2 1 1 * * * * * * *Charlie to DNS2 * * 1 * * * * * *Charlie to FTPD * 1 1 * * * * * *Charlie to HTTPD1 1 1 1 * * * * * *

1 * 1 1 1 1 * * *Charlie to HTTPD2 * 1 1 1 * * 1 * *

* * 1 1 1 1 1 * *Charlie to HTTPD3 * 1 1 1 * * * * *

* * 1 1 1 1 * * *Charlie to MX * * 1 * * 1 * * 1

* 1 1 * 1 * * 1 1Charlie to NFS2 1 1 1 * * * * * *

1 * 1 1 1 1 * * *Eddie to DNS2 * * 1 * * 1 * * *Eddie to DNS3 * * * * * 1 * 1 *Eddie to FTPD * 1 1 * * 1 * * *

* 1 * * 1 1 * * *Eddie to HTTPD1 1 1 1 * * * * * *

1 1 * * 1 1 * * *1 * * 1 1 1 * * *

Eddie to HTTPD2 * * * 1 1 1 1 * ** 1 1 1 * 1 1 * *

Eddie to HTTPD3 * * * 1 1 1 * * ** 1 1 1 * 1 * * *

Eddie to MX * * * * 1 1 * * *Eddie to NFS1 * * * * * 1 * * 1

Table 4: Router Configurations37

User to Service 0 1 2 3 4 5 6 7 8

Debbie to DNS2 * * 1 * * 1 * * *Debbie to DNS3 * * * * * 1 * 1 *Debbie to FTPD * 1 1 * * 1 * * *

* 1 * * 1 1 * * *Debbie to HTTPD1 1 1 1 * * * * * *

1 1 * * 1 1 * * *1 * * 1 1 1 * * *

Debbie to HTTPD2 * * * 1 1 1 1 * ** 1 1 1 * 1 1 * *

Debbie to HTTPD3 * * * 1 1 1 * * ** 1 1 1 * 1 * * *

Debbie to MX * * * * 1 1 * * *Debbie to NFS1 * * * * * 1 * * 1Frank to DNS3 * * * * * * * 1 1Frank to FTPD * 1 1 * * 1 * * 1

* 1 * * 1 * * 1 1* 1 * * 1 1 * * 1

Frank to HTTPD1 1 1 1 * * 1 * * 11 * * 1 1 1 * * 11 1 * * 1 1 * * 11 * * 1 1 * * 1 11 1 * * 1 * * 1 1

Frank to HTTPD2 * 1 1 1 * 1 1 * 1* * * 1 1 1 1 * 1* * * 1 1 * 1 1 1

Frank to HTTPD3 * 1 1 1 * 1 * * 1* * * 1 1 1 * * 1* * * 1 1 * * 1 1

Frank to MX * * * * 1 1 * * 1* * * * 1 * * 1 1

Frank to NFS1 * * * * * 1 * * *George to DNS3 * * * * * * * * *George to FTPD * 1 * * 1 * * 1 *

* 1 1 * * 1 * 1 *George to HTTPD1 1 1 * * 1 * * 1 *

1 1 1 * * 1 * 1 *1 * * 1 1 * * 1 *

George to HTTPD2 * * * 1 1 * 1 1 ** 1 1 1 * 1 1 1 *

George to HTTPD3 * * * 1 1 * * 1 ** 1 1 1 * 1 * 1 *

George to MX * * * * 1 * * 1 *George to NFS1 * * * * * * * 1 1

Table 5: Router Configurations

38

User to Service 0 1 2 3 4 5 6 7 8

Harry to DNS1 1 * * 1 * * * * *Harry to DNS3 * * * 1 1 * * 1 *

* 1 1 1 * 1 * 1 *Harry to HTTPD1 * 1 1 1 * * * * *

* * 1 1 1 1 * * *Harry to HTTPD2 * * * 1 * * 1 * *Harry to HTTPD3 * * * * * * * * *Harry to MX * * * 1 1 * * * *Harry to NFS2 1 * * 1 * * * * *Irina to DNS1 1 * * 1 * * * * *Irina to DNS3 * * * 1 1 * * 1 *

* 1 1 1 * 1 * 1 *Irina to HTTPD1 * 1 1 1 * * * * *

* * 1 1 1 1 * * *Irina to HTTPD2 * * * 1 * * 1 * *Irina to HTTPD3 * * * * * * * * *Irina to MX * * * 1 1 * * * *Irina to NFS2 1 * * 1 * * * * *Janet to DNS1 1 * * 1 * * 1 * *Janet to DNS3 * * * 1 1 * 1 1 *

* 1 1 1 * 1 1 1 *Janet to HTTPD1 1 * * 1 * * 1 * *Janet to HTTPD2 * * * * * * * * *Janet to HTTPD3 * * * 1 * * 1 * *Janet to MX * * * 1 1 * 1 * *Janet to NFS2 1 * * 1 * * 1 * *HTTPD1 to DNS1 1 * * * * * * * *HTTPD1 to DNS2 1 1 1 * * * * * *

1 * 1 1 1 1 * * *HTTPD1 to DNS3 1 1 * * 1 * * 1 *

1 1 1 * * 1 * 1 *1 * * 1 1 * * 1 *

HTTPD2 to DNS1 1 * * 1 * * 1 * *HTTPD2 to DNS2 * 1 1 1 * * 1 * *

* * 1 1 1 1 1 * *HTTPD2 to DNS3 * * * 1 1 * 1 1 *

* 1 1 1 * 1 1 1 *HTTPD3 to DNS1 1 * * 1 * * * * *HTTPD3 to DNS2 * 1 1 1 * * 1 * *

* * 1 1 1 1 1 * *HTTPD3 to DNS3 * * * 1 1 * 1 1 *

* 1 1 1 * 1 1 1 *


39

User to Service 0 1 2 3 4 5 6 7 8

FTPD to DNS1 1 1 * 1 * * * * *FTPD to DNS2 * 1 1 * * * * * *FTPD to DNS3 * * * 1 1 * 1 1 *

* 1 1 1 * 1 1 1 *MX to DNS1 1 1 * * 1 * * * *

1 * * 1 1 * * * *MX to DNS2 * * 1 * * 1 * * 1

* 1 1 * 1 * * 1 1MX to DNS3 * * * * 1 * * 1 *


40

Design of an Intrusion Response System using Evolutionary...

Documents

Transcript of Design of an Intrusion Response System using Evolutionary...