Scalable and Effective Test Generation for Access Control Systems

1

Scalable and Effective Test Generation for Access Control Systems

Ammar MasoodSchool of Electrical & Computer Engineering

Purdue University11th September, 2006

2

Outline

Introduction Problems and Contributions – Part A Details of Proposed Solutions – Part B Conclusion and Future Work

3

Motivation and Challenges Protection of information from unauthorized access or

modification and protection against denial of service to authorized users is an important security requirement

Access control is one of the key security service providing the support for secure information access

Desired access control objectives only achieved if the underlying implementation conforms to the policy, hence testing becomes essential

Key challenge: how to devise scalable and effective test generation techniques ?

4

Requirement for Testing

A number of vulnerabilities are related to design and/or coding flaws in access control modules of an application*

OSVDB reports 53 vulnerabilities related to access control NVD which records CVE and CERT advisories reports 859

vulnerabilities with impact “provides unauthorized access” and type “access validation error”, 1440 for any impact

Security Focus reports 80 vulnerabilities for the key word “access control”

Formal verification and static or dynamic program-analysis techniques only guarantee correctness of design

Testing is required to detect any faults in the implementation due to, for example, coding errors and incorrect configuration

*Data as of 8/30/06

5

Conformance and Functional Testing

6

Testing Context

7

Role Based Access Control (RBAC) and Temporal RBAC

RBAC is a promising approach for addressing diverse security needs of business organizations

Access control in organizations is based on “roles that individual users take on as part of the organization”

A role is “is a collection of permissions”

Constraints are applied to all the links

Role Hierarchies

Roles PermissionsUsers

Constraints

Role Hierarchies

Roles PermissionsUsers

Constraints

TRBAC extends RBAC by imposing duration constraints on user-role assignments/activations and permission-role assignments

8

Outline


9

Contributions

1. RBAC fault model

2. Test generation for RBAC Systems

3. A Probabilistic model for fault coverage

4. An empirical evaluation

5. Test generation for TRBAC Systems• Behavior modeling of TRBAC systems• TRBAC conformance testing

10

1. RBAC Fault Model

Required to study fault coverage of any test generation technique

Proposed fault model comprises Mutation-based (simple) faults Non-mutation (malicious) faults

Behavioral conformance used to study the fault model

11

2. Test Generation for RBAC Systems

Requirements :- Effectiveness – fault detection effectiveness measured

with respect to RBAC fault model Scalability – the cost of test generation and execution

Existing research – Chandarmouli and Blackburn functional testing technique for Discretionary Access Control Effectiveness not considered Not amenable for fault coverage analysis

12

Proposed Solution Set of conformance testing procedures with varying cost and

effectiveness Procedure A : Complete-FSM based Procedure B : Heuristics based Procedure C : Constrained Random Test Selection (CRTS) strategy

based

Procedure A is most effective – complete fault coverage for simple faults and a class of malicious faults – and most costly

Cost and effectiveness of Procedures B and C varies with the heuristic considered for test generation and the length of test cases in the CRTS suite

13

Proposed Solution (continued)

Functional Testing Required to ensure that ACUT conforms to all RBAC

policies Proposed methodology is based on policy meta test set White box coverage criteria used as a feed back

mechanism to establish correctness of ACUT functionality

The functional testing technique is generic in that it can be used for TRBAC systems

14

3. A Probabilistic Model for Fault Coverage

Requirement A mechanism for analytically comparing fault coverage

of heuristics and CRTS strategy based test generation techniques

Existing research Petrenko et. al. use mutation based approach to access

fault coverage of tests for FSM’s One-to-one relation between faults and structural mutants Not suitable for our analysis because of many-to-many

relation between RBAC/TRBAC faults and structural mutants

15

Proposed Approach

Coverage matrix used to model relation between FSM and RBAC faults

Faults exhibited randomly across the FSM transitions Fault coverage analytically studied for two general cases of

fault distribution (uniform and non-uniform) Simulation:- To study fault coverage of test generation

techniques for fault distributions achieved as mix of uniform and non-uniform distributions

High coverage of all techniques for uniform case Coverage drops as distribution limits to complete non-uniform case Coverage directly proportional to the number of transitions in the

test suite

16

4. Empirical Evaluation

To study cost and effectiveness of use of all the procedures in functional testing of an RBAC system

Based on X-GTRBAC prototype system X-GTRBAC consists of

Policy initializer Policy enforcer (ACUT)

Fault detection effectiveness measured through program mutation and manual injection of malicious faults

Program mutants manually associated with RBAC faults (simple faults)

Cost measured in terms of total number of state queries performed in the execution of a test suite

17

Results Procedure A most effective and most costly Heuristics and CRTS strategy perform equally well for

simple faults but heuristics lag CRTS strategy in detecting malicious faults

Effectiveness of CRTS increases as length of tests included in the suite increases, cost also increases but is significantly less then that of Procedure A

Reasons: Heuristics by design fail to consider a holistic view of the system Simple faults are exhibited across much higher number of transitions

as compared to malicious faults, thus easier to detect CRTS randomly select paths of fixed length from complete FSM,

thus as length of tests increases there are more chances of inclusion of higher length paths in the CRTS test suite

18

Recommendations to Practitioner

Although only Procedure A provides complete fault coverage it could be prohibitively expensive CRTS strategy provides the balance between cost and

effectiveness Reaffirmation of usefulness of white-box criteria to

enhance tests generated using black-box approach Malicious faults likely to be missed easily by the

heuristics As exhaustive testing not a viable option, functional

testing requires white-box criteria as a feed back mechanism to determine the stopping point

19

Comparison with Simulation Results

Fault coverage results for the case of uniform fault distribution in the simulation are close to case study results for simple faults

Given a test generation technique, the analytic result of fault coverage for uniform fault distribution may be used as a predictor of its effectiveness in detecting simple faults

Wide disparity between coverage results for the simulation and for the case study for malicious faults

Logical result as malicious faults are injected with malicious intent, thus can not be modeled with uniform distribution

20

5. Test Generation for TRBAC Systems Require effective and scalable test generation technique

How to measure effectiveness? TRBAC fault model (extensions in RBAC fault model)

Scalability ? Determined by the size of the test suite (size of model)

Why can’t existing approaches for test generation be directly used for TRBAC test generation?

Techniques for RBAC system not usable as simple FSM’s cannot capture real-time considerations

Solution – use Timed Input Output Automata (TIOA) to model TRBAC TIOA based test generation techniques

Symbolic clustering of states – scalable but effectiveness not measurable

State characterization set based (Timed-Wp method) – effective but not at all scalable

TIOA transformation to FSM (se-FSA based) – effective and scalable

21

Proposed Approach

22

Behavior modeling of TRBAC systems

Requirement Model correctly specify the behavior implied by the

TRBAC specification TRBAC model (TRBACM) is based on TIOA Two options in constructing TRBACM

Construct a single monolithic model Divide the system into parts – compositional

construction TRBACM= URM || PRM

TRBACM is proved to correctly model the TRBAC specification

23

TRBAC conformance testing Key steps

Transformation of TRBACM into se-FSA Constructing the test tree corresponding to the se-TRBACM

Use of an Integer Programming (IP) based approach to generate the conformance test suite

Fault detection effectiveness Provides complete fault coverage by virtue of correctness of

TRBACM and the correlation between TRBAC, TIOA and se-FSA faults

Heuristics can be used to reduce the model size and thus the size of the corresponding test suite

May result into reduced fault detection effectiveness, can be analytically studied for cases of fault distribution using the probabilistic model

24

Outline


25

Conformance Relation Based on behavioral

conformance Specified using the two

conditions, which informally imply that ACUT

assigns (deassigns) and activates (deactivates) a role only if such assignment (deassignment) and activation (deactivation) is allowable by the current policy in effect

assigns (deassigns) a set of permissions to (from) a role only if allowable by the current policy in effect

ignores ill-formed requests

26

RBAC Fault Model Conformance between ACUT and ACUT implies absence

of any faults in the ACUT i.e. faults in P Conformance testing of ACUT can thus be considered as verifying

that P does not belong to set of faulty policies RBAC fault model defines the set of faulty policies

Obtained using mutation based approach [Petrenko et.al.] Three types of operators used for mutating the elements of RBACP

Set mutation operators Element modification operators Rule mutation operators

27

Malicious Faults

Counter based A specific count of events leads to fault

I/O based Faults based on malformed requests

Sequence-based A specific sequence of events leads to fault

28

Conformance Testing Procedures

Behavior implied by a policy expressed as an FSM.

Heuristics applied to scale down the model.

Use the W-method, or its variant, to generate tests from the complete (Procedure A) or scaled down model (Procedure B) or randomly select paths of fixed length from the complete model (Procedure C)

29

Sample FSM

Two users, one role. Only one user can activate the role. Number of states≤32

.

AS11

0000

1000 0010

1100

1110

1010 0011

1011

AS21

AC11

AC21AS21

AS21 AS11AC21

AC11

AS11

DS11

DS21

DC11

DS21

DC11

DS11

DS21 DS11

DC21

DC21

DS21

DS11

DS11 DS21

AS: assign. DS: De-assign. AC: activate. DC: deactivate. Xij: do X for user i role j.

30

Heuristics

H1: Separate assignment and activation

H2: Use FSM for activation and single test sequence for assignment

H3: Use single test sequence for assignment and activation

H4: Use a separate FSM for each user

H5: Use a separate FSM for each role

H6: Create user groups for FSM modeling.

31

Reduced Models

AS11

00

10 01DS21DS11

11

AS21

DS11DS21

AC11

00

10 01

AC21

DC21DC11

AC21 AC11

Assignment Machine

Activation Machine

Heuristic 1

AS11

00

10 11

DS11 DS11

AC11

DC11

AC11

AS21

00

10 11

DS21 DS21

AC21

DC21

AC21

Heuristic 4

User u1 Machine User u2 Machine

32

Procedure C: CRTS Strategy

Constructs a pool RTi of n random tests. Lengths of all tests in the pool RTi is same, i.e. i which is

selected to be comparable with the length of longest test generated using Procedure A

The total number of tests n is selected based on comparison with the maximum number of tests generated using the heuristics (Procedure B)

Construct five test suites RTi1,…., RTi5 by randomly selecting fixed number p of tests from RTi p empirically chosen based on economical or statistical

criterion

33

Probabilistic Model for Fault Coverage State observability assumed Based on Coverage matrix Cx, x {H0, H1,…, RTi}

Visibility of faults among transitions is given by x=b. Cx where b is a identity row vector of length j

Fault Coverage (FCx) is computed as

where

34

Boundary Cases of Fault Distribution

1. |F|=j=|TH0|, such that one-to-one correspondence between faults and transitions, FCx= # of transitions in x/j

If x1 covers more transitions then x2 FCx1 > FCx2

2. Single fault f with equal probability of being exhibited across any transition t TH0

Fault coverage of x is now the probability of detecting f using x

35

General Cases of Fault Distribution Case A: The total number of transition across which each fault f is

exhibited is uniformly distributed

Case B: Total number of faults is more than 1, each fault f has equal probability of being exhibited across any transition t TH0

36

Simulation

Five cases of fault distribution Cases 0 and 4 – same as Cases A and B Cases 1, 2 and 3 – respectively correspond to cases in which 75%,

50% and 25% of faults are uniformly exhibited (as per Case 0) rest as per Case 4

Metrics used for comparison of testing generation techniques

Average fraction of faults detected Probability of detecting all faults p(F)

Setup 10,000 iterations 5 values of fault density 0.01, 0.05, 0.1, 0.2 and 0.5

37

Results : Average fraction of Faults Detected Common trend for all cases of fault distribution

Expected as faults are independently and identically exhibited

High coverage for all techniques for Case 0 As fault distribution limits to Case 4, coverage reduces dramatically for

techniques with less number of transitions in their test suitesFault Density : 0.01

Total Faults = 1

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Case 0 Case 1 Case 2 Case 3 Case 4

Fault Distribution

Av

era

ge

Fra

cti

on

of

Fa

ult

s D

ete

cte

d

H1

H2

H3

H4

RT2

RT4

RT6

RT8

38

Results : Probability of Detection of all Faults

p(F) reduces considerably with increase in fault density Expected as p(F) is the product of probabilities for

detection of individual faults As fault distribution limits to Case 4, the exponential

term in p(F) corresponding to Case 4 dominates No test generation technique other than the

complete FSM based, provides guarantee of detecting all faults Solution – use white box adequacy criteria for test

enhancement

39

Fault Density : 0.05Total Faults = 3

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1


Fault Distribution

Pro

bab

ility

of

Det

ecti

ng

All

Fau

lts

H1

H2

H3

H4

RT2

RT4

RT6

RT8


0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1


Fault Distribution

Pro

ba

bili

ty o

f D

ete

cti

ng

All

Fa

ult

s

H1

H2

H3

H4

RT2

RT4

RT6

RT8


0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Case0 Case1 Case2 Case3 Case4

Fault Distribution

Pro

ba

bili

ty o

f D

ete

cti

ng

All

Fa

ult

s

H1

H2

H3

H4

RT2

RT4

RT6

RT8

Fault Densiy : 0.5Total Faults = 32

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1


Fault Distribution

Pro

ba

bili

ty o

f D

ete

cti

ng

All

Fa

ult

s

H1

H2

H3

H4

RT2

RT4

RT6

RT8

40

Empirical Evaluation : Setup

Study carried out using the proposed functional testing methodology Stopping criterion – complete coverage of simple faults Policy meta set – comprises two policies Meta test sets – corresponding to the three procedures

Test generation techniques used H3, H4 and H5 heuristics RT4, RT6, RT10 and RT100 100 tests in each test suite RTij

41

Empirical Evaluation : Results

42

Empirical Evaluation and Simulation Results Comparison

43

TRBAC Fault Model

Conformance relation similar to the one for RBAC systems Addition of a condition to consider temporal conformance

RBAC fault model extended by changing the application of rule mutation operator, result is addition of three temporal faults

44

Timed Input Output Automata (TIOA)

45

TRBAC Modeling

TRBACM= URM || PRM

URM=URb1 ||ur URb2 ||ur, …,||ur URbk , three types of URb’s corresponding to user-role (UR) pairs with

1. Explicit assignment information

2. No explicit assignment and implicit activation

3. No explicit assignment but implicit activation

L0

L1

?AS(u1,r1,t1)x1:=0

x1=t1

!DS(u1,r1)

L0 URassign(u1,r1)=0, URactive(u1,r1)=0

L1 URassign(u1,r1)=1, URactive(u1,r1)=0

L2 URassign(u1,r1)=1, URassign(u1,r1)=1

?AC(u1,r1,t2)

L2

?AC(u1,r1,t2)x2:=0

x2=t2

!DC(u1,r1)

x1=t1

!DS(u1,r1)

46

TRBAC Modeling (continued) PRM=PRb1 ||pr PRb2 ||pr, …,||pr PRbk , two types of PRb’s corresponding to

permission-role (PR) pairs with1. Explicit assignment information2. No explicit and implicit assignment

Example: Three permissions p1, p2 and p3 , three roles r1, r2 and r3, r2 I r3

p2r1 , p3r1 and p1r2 explicit assignment

47

Sample TRBACM

Example policy with a user u1 two roles {r1, r2} Constraint: u1 cannot be simultaneously assigned to both roles No permissions considered thus TRBACM= URb(u1,r1) ||ur URb(u1,r2)

48

se-FSA Transformation [Khoumsi] Three types of events

Input events – input actions and/or clock resets Output events – output actions and/or clock expirations Complex events – mix of above two

L0

L1

?AS(u1,r1,t1)x1:=0

x1=t1

!DS(u1,r1)

?AC(u1,r1,t2)

L2

?AC(u1,r1,t2)x2:=0

x2=t2

!DC(u1,r1)

x1=t1

!DS(u1,r1)

t1=4 and t2=2

4<x1

x1<x2

-

l0

q2

0<x1

0<x2

-

l0

0<x1<42<x2

2<x1-x2<4

l1

q0

q3

0<x1<4x1<x2

-

l1

0<x1<40<x2<2

0<x1-x2<4

l2

4<x1

0<x2<2

2<x2-x1<4

l0

4<x1

2<x2

-

l0

q1

q5

q4

q6

?AS(u1,r1), Set(x1,4)

Exp(x1,4),!DS(u1,r1)

?AC(u1,r1), Set(x2,2)



Exp(x2,2),!DC(u1,r1)

Exp(x2,2)

?AS(u1,r1), Set(x1,4)

Exp(x1,4), Exp (x2,2) !DS(u1,r1)

Exp(x2,2),?AS(u1,r1), Set(x1,4)

se-FSA

49

Test Generation From se-TRBACM

se-TRBACM deterministic and finite state W-method can thus be used for test generation Assumed location observability – tests constructed from

test tree (Tr) Tr constructed so that all terminals correspond to

accepting states of se-TRBACM

Tr represents paths in se-TRBACM, Given a path pt in Tr, A test sequence is constructed by

associating all edges e pt with monotonically increasing time stamps

Temporal constraints determined by the Set and Exp events along edges of pt

50

How to Construct a Test Sequence? Corresponding to path pt1

The temporal constraints can be represented as

pt1

Formulate as an IP to control the minimum resolution dti

For k=0.1 the solution would be

Conformance Test Suite (CTS) constructed by finding feasible time stamps for all test sequences

51

How to Apply a Test Sequence ? Used the architecture proposed by Khoumsi Given a test sequence, following semantics considered for

time stamps associated with: Inputs – time at which Test-Controller should generate the

corresponding input for the ACUT and the Clock-Handler Outputs – ACUT will pass the given sequence if outputs by the

ACUT and Clock-Handler and the states match

input output

Set(c,k)

Exp(c,k)

Test SystemState query

State info

Test-Controller Clock-Handler

ACUT

52

Fault Coverage of CTS Determined using the relation between TRBAC, TIOA and se-FSA fault models

(FM)

TRBAC FM

TIOA FM

se-FSAFM

correlated with

correlated with

Output, Transfer, missing and extra location faults in TIOA FM have similar representation in se-FSA

Time constraint restriction/widening faults – output/transfer faults Clock reset faults not directly comparable – shown to be detectable by CTS Implies CTS detects all TRBAC Faults

53

Conclusion Proposed a unified framework for scalable and effective

conformance and functional testing of RBAC and TRBAC systems

Effectiveness studied using the proposed RBAC and TRBAC fault models

Scalability achieved using proposed conformance testing procedures with varying cost

Proposed a probabilistic model for fault coverage to analytically evaluate fault detection effectiveness of proposed conformance test generation techniques for various cases of fault distribution

Performed an empirical study to evaluate the cost and effectiveness of proposed procedures in functional testing of a prototype RBAC system

54

Future Work

Test generation for TRBAC systems Extending the temporal constraints in TRBAC

specification Extension of TRBC fault model Conducting an empirical evaluation

Validation of global meta-policy in collaborative environments

Regression testing techniques for access control systems

55

Backup Slides

56

Advantages of RBAC

Allows efficient security management through role hierarchy and administrative roles

Principle of least privilege allows minimizing damage due to misuse of privilege

Separation of duty constraints prevent fraud Role specific SoD constraint disallows conflicting roles

to be accessed by same user User specific SoD constraint disallows conflicting user to

access same role Encompasses traditional discretionary and

mandatory policies

57

Functional Testing Methodology

58

Many-to-Many Relation between RBAC and FSM faults

0000

0010

AS11 A transfer fault

UR1 and UR2 faults

AS11

0000

0010

AS21

AS11

DS21

f1: UR1 fault

transfer fault

transfer fault

59

Behavioral Conformance

60

RBAC Fault Model – Simple Faults

Relation between FSM and RBAC Fault Model

61

Fault Coverage of H4 for Boundary Case 1

AS11

0000

1000 0010

1100

1110

1010 0011

1011

AS21

AC11

AC21AS21

AS21 AS11AC21

AC11

AS11

DS11

DS21

DC11

DS21

DC11

DS11

DS21 DS11

DC21

DC21

DS21

DS11

DS11 DS21t1

t2

t3

t4

t5

t6

t7

t8

t9

t10

AS11

00

10 11

DS11 DS11

DC11

AC11

AS21

00

10 11

DS21 DS21

DC21

AC21

t1

t2

t3t4

t5

t6

t7

t8

t9

t10

FSM(P)

H4: Mu1 and Mu2

Scalable and Effective Test Generation for Access Control Systems

Documents

Transcript of Scalable and Effective Test Generation for Access Control Systems