Introduction to IRRIIS testing platform IRRIIS MIT Conference ROME 8 February 2007 Claudio...
-
Upload
bailey-hobdy -
Category
Documents
-
view
217 -
download
1
Transcript of Introduction to IRRIIS testing platform IRRIIS MIT Conference ROME 8 February 2007 Claudio...
IRRIIS- FP6-2005–IST-4
EC - LOGO
Introduction to IRRIIS testing platform
IRRIIS MIT Conference ROME 8 February 2007
Claudio Balducelli
IRRIIS
Summary
Design a testing environment for MIT Modelling and running attack and fault
behaviours Testing strategies for MIT components Proposed test-bed configuration Conclusions
IRRIIS
Target Infrastructures
Models
Vulnerabilities of the Target
Infrastructures
Fault/attack Scenarios Generation
Models of faults & attacksUse domain
knowledge
Considervulnerabilities
Design a testing environment for MIT
IRRIIS
Meaning of attacks and faults
Attacks: A disturbance of the LCCI generated by eventscoming from outside the LCCI
Faults: A disturbance of the LCCI generated by eventscoming from the components that are part of theLCCI
Definition of the meaning of attacks and faults
IRRIIS
Meaning of attacks and faults
Attacks:
Natural disaster (earthquake, flood, etc)
Premeditated terrorist attack
Cyber attacks (cyber-intrusion)
Operator errors
………….….
Faults:
Physical component failure (aging, stress, etc.)
Software component failure (bug, wrong istal. etc)
Wrong component activation
………….….
IRRIIS
Normal behavior & fault behavior in SimCIP
Activationevent
t1
Start Comp. 1
Comp. 1
Start Comp. 2
Comp. 2End
Start Comp. 3
t2 Comp. 3End
Activationevent
Normal behavior consists in an initial state and a sequence of events represented in form of a petri net oriented graph
IRRIIS
Initiatingevent
t1
Failure of Comp. 1
t2
Failure ofComp. 2
t3
RestartComp. 1
t4 t5
Loss ofservice 2
Normal behavior & fault behavior in SimCIP
Loss ofService 1
Fault behavior may be represented in a similar way
Fault eventsIn LCCI-1
Failure ofComp. 2
t6
Failure of Comp. 1
t7
Fault eventsIn LCCI-2
IRRIIS
For a certain LCCI normal behaviors are well known and their number is
limited the number and the combinations of fault behaviors are
very high and not always known in advance how to design fault behaviors? how to select fault behaviors? utilisation of a model based on attack/fault trees seem
useful to formalise and manage the knowledge needed to generate attack/fault behaviour
Normal behavior & fault behavior in SimCIP
IRRIIS
G0
A1 A2 A2
The root of the tree (G) represents an event that could significantly harm the infrastructure’s mission.
The terminal leafs (A) of the tree represent the actions to execute for reaching the high level goals
Every path in the attack tree represents a unique type of attack
Goal G0AND A1 A2 A3
Goal G0OR A1 A2 A3
The attack trees could be visualized also in textual form
G0
A1 A2 A3
Every node could be decomposed inside lower level nodes using <AND>, <XOR> and <OR> decomposition types
AND
OR
Modelling attack knowledge attack/fault trees
IRRIIS
G0
S1 A2 S2
A3 A4 A5 A6
The tree generate the following two attack patterns
<A3, A2, A5, A6>
<A4, A2, A5, A6>
The “terminal leafs” of the tree (A1..An) represent the actions steps needed to execute the attack
The “intermediate nodes” (S1..Sn) represent the steps in which a decision has to be taken
The attack tree generates attack patterns (attack behaviors), composed by sequences of actions.
Attack goal
Modelling attack knowledge attack/fault trees
IRRIIS
TE
S1 C2 S3
C11 C12 C31 C32
The tree generate the following two
fault patterns
<C11, C2, C31, C32>
<C12, C1, C31, C32>
The “terminal leafs” of the tree (C..) represent the elementary failures of the single components of LCCI.
The “intermediate nodes” (S…) represent failures of subsystems or services for which the components contribute
The fault tree generates fault patterns (fault behaviors), composed by sequences of elementary failures.
Top event Fault trees
Modelling attack knowledge attack/fault trees
IRRIIS
And gate
Or gateOR gate
AND gate
Example of attack tree to model an attack in a local area network (tree structure)
The reference model take in account the:Fault Tree Handbook ofUS Nuclear Regulatory Commission
IRRIIS
And gate
Or gateOR gate
AND gate
Example of attack tree to model an attack in a local area network (tree structure)
Verify theaccessibility to a subnet
IRRIIS
And gate
Or gateOR gate
AND gate
Example of attack tree to model an attack in a local area network (tree structure)
Discover the target locations & addresses
IRRIIS
And gate
Or gateOR gate
AND gate
Example of attack tree to model an attack in a local area network (tree structure)
Make sniffing activity or damages
IRRIIS
And gate
Or gateOR gate
AND gate
Example of attack tree to model an attack in a local area network (tree structure)
Generated behaviours table------------------------------------------------------------------------------------------------Attack behaviour 0 <A1, A2, A4, A5, A6, A7, A8 >
Attack behaviour 1 <A1, A2, A4, A5, A6, A7, A9>
Attack behaviour 2 <A1, A2, A4, A5, A6, A7, A10>
Attack behaviour 3 <A1, A2, A4, A5, A6, A7, A11>
Attack behaviour 4 <A1, A3, A4, A5, A6, A7, A8 >
Attack behaviour 5 <A1, A3, A4, A5, A6, A7, A9>
Attack behaviour 6 <A1, A3, A4, A5, A6, A7, A10>
Attack behaviour 7 <A1, A3, A4, A5, A6, A7, A11> ------------------------------------------------------------------------------------------------
IRRIIS
Example of attack tree to model an attack:associating difficulties to the actions
OR gate
AND gate
0.8
0.9 0.2
0.950.95
0.95
0.30.6
0.2
0.80.8
0.0 = maximum difficulty1.0 = minimum difficulty
Generated behaviours table ordered by action difficulties------------------------------------------------------------------------------------------------Attack behaviour 0 <A1, A2, A4, A5, A6, A7, A8 > with 0,39 of difficulty
Attack behaviour 2 <A1, A2, A4, A5, A6, A7, A10> with 0,24 of difficulty
Attack behaviour 1 <A1, A2, A4, A5, A6, A7, A9> with 0.12 of difficulty
Attack behaviour 3 <A1, A2, A4, A5, A6, A7, A11> with 0.08 of difficulty
Attack behaviour 4 <A1, A3, A4, A5, A6, A7, A8 > with 0.08 of difficulty
Attack behaviour 6 <A1, A3, A4, A5, A6, A7, A10> with 0.05 of difficulty
Attack behaviour 5 <A1, A3, A4, A5, A6, A7, A9> with 0.03 of difficulty
Attack behaviour 7 <A1, A3, A4, A5, A6, A7, A11> with 0.02 of difficulty------------------------------------------------------------------------------------------------
IRRIIS
Macro scenarios:
how to compose attack and fault trees
Attack tree
Fault tree
Attack tree
Wait for malfunction
IRRIIS
Composite attack and fault behavior
t1
BasicAction 0
t2
BasicAction 2
Final Action 0
t3 t4
Final Action 1
Networkmalfunction
BasicEvent 0
Attackbehavior
Attackbehavior
Faultbehavior
Attack escalation
IRRIIS
Testing MIT components (meaning)
REQUIREMENTS:
Risk Ass. (1) - The Risk estimator assessment of cascading and escalating effects shall be performed in near real-time.Risk Ass. (2) - The Risk estimator assessment of cascading and escalating effects shall be performed in a predictive way.Risk Ass. (3) - The Risk estimator shall estimate immediate risk to the LCCI.Risk Ass. (4) - The Risk estimator may estimate expected risk to the LCCI.Risk Ass. (5) - The Risk estimator shall estimate potential cascading effects.
Objective of the TEST: validate the requirements
Risk Ass. (1) - OK
Risk Ass. (2) - OK
Risk Ass. (3) - OK
Risk Ass. (4) - NOT OK
Risk Ass. (5) - NOT OK
IRRIIS
Testing MIT components (meaning)
One of the main objective of the MIT components test inside SimCIP simulated environment is the evaluation of the rate of false/true alarms.
The second is to evaluate how much the rate of false alarms may be acceptable for the LCCIs operators
IRRIIS
Detecting interdependency alarms
Real statesPredicted states
Alarm No Alarm
P(Alarm) A B P(No Alarm) C D
A = Number of alarm states correctly predicted
D = Number of no alarm states correctly predicted
B = Number of no alarm states predicted as true (FALSE POSITIVE)
C = Number of alarm states not predicted (FALSE NEGATIVE)
The goal is: max(A + D), min(B + C)
Evaluation Table
IRRIIS
Detecting interdependency alarms
Real statesPredicted states
Alarm No Alarm
P(Alarm) A B P(No Alarm) C D
Fn = C / ( C + D )Observed False Negative Ratio (FNR)
Fp = B / ( A + B )Observed False Positive Ratio (FPR)
IRRIIS
Be not afraid to discover false alarms during the tests. This is the tests objective!!
In many cases false alarms could be simply reduced tuning the “sensitivity” level of a MIT component.
To evaluate true/false alarms ratio is not sufficient a single attack/fault behavior. Many alternative behaviors are needed!!
Logging facilities are very important during experimentations, are the tests results must be archived and documented
Detecting interdependency alarms
IRRIIS
Proposed testing strategy
IRRIIS testing operator
Attack/Fault tree editor
Design or modify a scenario
tree
GA
S1 A2 S2
A3 A4 A5 A6
<A3, A2, A5, A6>
<A4, A2, A5, A6>
<A3, A2, A5, A6>
<A4, A2, A5, A6>
Fault behaviors
editor
Generate & modify fault behaviors,
insert timing information etc
Documentation console
View logsEdit test
documents
Logs
Test documen
ts
Fault behavior execution
Execute behavious,
sets monitors
Attacks/faultsexecution in
SimCIP
Test designentry point
Test designexit point
Testdesign
IRRIIS
Proposed testing strategy
IRRIIS testing operator
Attack/Fault tree editor
Design or modify a scenario
tree
GA
S1 A2 S2
A3 A4 A5 A6
<A3, A2, A5, A6>
<A4, A2, A5, A6>
<A3, A2, A5, A6>
<A4, A2, A5, A6>
Fault behaviors
editor
Generate & modify fault behaviors,
insert timing information etc
Documentation console
View logsEdit test
documents
Logs
Test documen
ts
Fault behavior execution
Execute behavious,
sets monitors
Attacks/faultsexecution in
SimCIP
Test execution entry point
Test execution exit point
Fasttesting
IRRIIS
Proposed testing strategy
IRRIIS testing operator
Attack/Fault tree editor
Design or modify a scenario
tree
GA
S1 A2 S2
A3 A4 A5 A6
<A3, A2, A5, A6>
<A4, A2, A5, A6>
<A3, A2, A5, A6>
<A4, A2, A5, A6>
Fault behaviors
editor
Generate & modify fault behaviors,
insert timing information etc
Documentation console
View logsEdit test
documents
Logs
Test documen
ts
Fault behavior execution
Execute behavious,
sets monitors
Attacks/faultsexecution in
SimCIP
Test entry point
Test exit point
Exhaustivetesting
IRRIIS
Physical TESTBED Configurations
LAMPSSys RTI
GUI Logger
To
ol 1
Electricity
SimulatorLCCI
Data
Com
Simulator
To
ol 2
Agent / Scenario
Behaviours
An
alysis 1
An
alysis 2Fault /
Attack
Tool
MITA
nalysis 3
SimCIP Architecture
IRRIIS
Physical TESTBED Configurations
GUILogger
LAMPSSys RTI
Agent / Scenario
Behaviours
Electricity
Simulator
Com
Simulator
LCCI Electricity
Data Base
Tool 1
Tool 2
Analysis 1, 2, 3 ..
LCCI Telecom
Data Base
Simple SimCIPconfiguration
IRRIIS
Physical TESTBED Configuration
LAMPSSys RTI
Agent / Scenario
Behaviours
Electricity
Simulator
Com
Simulator
LCCI Electricity
Data Base
Fault /Attack
Tool
Tool 1
Tool 2
Analysis 1, 2, 3 ..
LCCI Telecom
Data Base
SimCIPfor testing
attacks and faults without MIT
GUILogger
IRRIIS
Physical TESTBED Configuration
GUILogger
LAMPSSys RTI
Agent / Scenario
Behaviours
Electricity
Simulator
Com
Simulator
LCCI Electricity
Data Base
LCCI Telecom
Data Base
MT communicationElectricity Add-on Telecom Add-on
SimCIPfor testing MIT with normal
behaviors(detect false positive alarms)
IRRIIS
Physical TESTBED Configuration
GUILogger
LAMPSSys RTI
Agent / Scenario
Behaviours
Electricity
Simulator
Com
Simulator
LCCI Electricity
Data Base
LCCI Telecom
Data Base
MT communicationElectricity Add-on Telecom Add-on
SimCIP for testing MIT in presence of
attacks/faults (detect false negative alarms)
Fault /Attack
Tool
Tool 1
Tool 2
Analysis 1, 2, 3 ..
IRRIIS
Conclusions
Testing of MIT components will be a continuous and iterative process
It is necessary to distinguish between the fast tests of the more simple requirements and the exhaustive test process aimed to evaluate the MIT efficiency in detecting interdependency alarms
Test designing, reports logging/archiving in a standard way and with the support of a common tool, will help to have sets of comparable tests also if produced in different SimCIP installations.
The testing environment will be one of the major a research product of the project, where experimentation may continue also after the end of the project.
QUESTIONS?