Institut für Technische Informatik Fakultät für Informatik Universität der Bundeswehr München...

Institut für Technische Informatik

Fakultät für Informatik

Universität der Bundeswehr München

ITIS e.V.

Institut für Technik Intelligenter Systeme

an der Universität der Bundeswehr München

NATO Advanced Research Workshop

Velingrad, Bulgaria, Oct. 24, 2006

Email: [email protected]://www.unibw.de/inf4/personen/prof/al/index_html

Verification, Validation and AccreditationVerification, Validation and Accreditationof Simulation Models and Applicationsof Simulation Models and Applications

Axel Lehmann

Axel Lehmann NATO Advanced Research Workshop Velingrad, Bulgaria, Oct. 24, 2006

Technology Trends Technology Trends

Rapid technological innovations information and communication technologies computer-assisted tools, e.g. for

Design & analysis support Planing & decision support Education & training

Increasing systems complexity & lifetime embedded systems networked components / systems distributed systems

Increasing productivity & cost-benefit relations Decreasing cycle times for system’s innovations


Major ChallengesMajor Challenges

“Mastering” of system(s) complexity over lifetime w.r.t. multiple aspects/ goals? (safety, reliability, performance,…)

Approach: Goal-oriented, efficient, effective and dependable models!

“Mastering” of model(s) complexity !? STRATEGIC approach: “Devide and Conquer”! MODEL-ENGINEERING approach!!


M&S-Approach: “Divide and Conquer”M&S-Approach: “Divide and Conquer”

improvingEFFICENCY

increasingCREDIBILITY

Multi-Phase M&S process(es)(intermediate products)

verification and validation

correctness;suitability

reusability;distributed execution

M&S „components“


OutlineOutline

1. M&S: A Multiple-Phase Development & Application

2. Improving M&S-Efficiency by Component-Based M&S

3. Increasing Credibility through M&S-Verification & Validation

4. Conclusion


1. M&S – A Multiple-Phase Development and 1. M&S – A Multiple-Phase Development and Application ProcessApplication Process

Analyses goals e.g.: processing time per transaction client/server utilization queueing

How to approach this (complex) problem?

“Divide and Conquer”: A Multiple-Phase M&S Process

Example: Effectivness and efficiency of a „Booking System“(e.g. client-server architecture)


M&S – Development Team (Skills)M&S – Development Team (Skills)

ModelingExpertise

Experimental Design and

Analysis

DomainKnowledge

(HW-)SW-Expertise

UserKnowledge

ProblemDefinition

SystemAnalysis

Model Formalization

Implementation

ExperimentationProgrammer

Modeller

Domain Expert

UserCustomer

Project Manager(Contractor)


Phases & Products in the M&S-Phases & Products in the M&S-Development ProcessDevelopment Process

Model Input Data

Solution Techniques

Modeling Method

System Observations

Experimental req.&constr.

Technical req.&constr.

Formal req.&constr.

Conceptual req.&constr.

Examination Aim

Input:

ProblemDefinition

SystemAnalysis

Model Formalization

Implementation

Experimentation

Phase:

ModelDocumen-tation

Objekt

Objekt

Objekt

Objekt

Intera k tio n

Systemgrenze

“Umwelt-objekt“

“Hauptobjekt” mitModellattributen

CommunicativeConceptual Model

,Project

Objectives

StructuredProblem Description

FormalModel

ExecutableModel

ModelResults

Intermediate Products:


Example: „Booking System“Example: „Booking System“

I/O device 1

CPU

T 1

T n I/O device n

•••

•••

ServerClients

Conceptual Model:



Formal Model:

peripheral devices



Executable model: Performance measures Little´s law:

k = + t ;

with: response time t, queueing time w, service rate µ

t = w + ; µ1

1

mservice rate

arrival rate

State probability p(k):

k = k • p (k) Utilization ( m service stations):


2. Model “Component”2. Model “Component”-Based Approach-Based Approach::

Executable Model(s):e.g. analytic solution

simulation reusable SW- “components“

(e.g. regarding the example „Booking System“)

Problem Description: pragmatism/ goal specification

Conceptual Model: structural & functional description of “components” different levels of abstraction

Formal Model: formal specification of “components”

( selected modeling paradigm(s)) hierarchical modeling approach

( decomposition into submodels/ “components”)


Phases & Products in the M&S-Phases & Products in the M&S-Development ProcessDevelopment Process

t

M&S-„components“:

X

X XX

XModel Input Data

Solution Techniques

Modeling Method

System Observations

Experimental req.&constr.

Technical req.&constr.

Formal req.&constr.

Conceptual req.&constr.

Examination Aim

Input:

ModelDocumen-tationObjekt

Objekt

Objekt

Objekt

Intera kio n

Systemgrenze

“Umwelt-objekt“


CommunicativeConceptual Model

,

ProjectObjectives

StructuredProblem Description

FormalModel

ExecutableModel

ModelResults

Intermediate Products:

ProblemDefinition

SystemAnalysis

Model Formalization

Implementation

Experimentation

Phase:


Model („Component“) Specification LevelsModel („Component“) Specification Levels

Model repository

Model Federation Level:

„Black Boxes“

Library of submodels & communication

infrastructure

Model Level:

Autonomous, interoperable models

Library of objects/methods (for interaction)

Submodel/ Object Level:

Submodels/ Object structures

60und0mit

:0

0)(wenn:),()()(. 111

1

sni

sonst

tlttzttrftltls isiisiissiis

is

Program Library Function Level:

Coded basic functions/ algorithms


3. Increasing M&S-Credibility3. Increasing M&S-Credibility

M&S-Credibility

Perceived suitability

Perceived correctness

Correctness(specification Implementation)

Consistency Completeness

Verification

( Formalisms, specifications, transformation rules)

Suitability(requirements, constraints)

Capability Fidelity Accuracy

Validation


Influences:Influences: V&V FrameworkV&V Framework

Model knowledge

Risk

System knowledge

V&V

Credibility indicators

Results:


Multi-phase M&S-process Multi-phase M&S-process intermediate productsintermediate products

,

Structured problem description

ModelDocumen-tation

Objekt

Objekt

Objekt

Objekt

Intera k tio n

Systemgrenze

“Umwelt-objekt“


Conceptual model

Formal model

Executable model

Simulationresults

ProjectObjectives

Problem definition

System analysis

Formalization

Implementation

Experimentation

InterpretationInternal consistency& completenessConsistency of transformation

(pairwise comparison)


Our Approach: „The V&V Triangle“Our Approach: „The V&V Triangle“

1.1

Ok, we have got the following problem...

O k, w e h ave g ot t he fol l ow ing pr oblem...

?1.2

O bj ekt

O bj ekt

O bj ekt

O bj ekt

S y s t e m g r e n z e“Umwel t-

ob jekt“

“Ha uptob jekt” m it

Mod ell attrib uten?2 . 2

O k, w e h ave g ot t he fol l ow in g pr oblem ...

O bj ekt

O bj ekt

O bj ekt

O bj ekt

S y s t e m g r e n z e

“Umwel t-

ob jekt“



3.1

O bj ekt

O bj ekt

O bj ekt

O bj ekt


ob jekt“


Mod ell attrib uten

?3 . 2

?3.3


?3.4

4.1

?4.2

O bj ekt

O bj ekt

O bj ekt

O bj ekt


ob jekt“

“Ha uptob jekt” m itMod ell attrib uten

?4 . 3

?4.4


?4.5

5.1

?5.2

?5.3

O bj ekt

O bj ekt

O bj ekt

O bj ekt

S y s t e m g r e n z e“Umwel t-ob jekt“

“Ha uptob jekt” mitMod ell attrib uten

?5 . 4

?5.5


?5.6

2.1Objekt

Objekt

Objekt

Objekt

Interaktion

Systemgrenze“Umwelt-objekt“


Sponsor Needs

Structured Problem

Description

Formal Model

Executable Model

Model Results

“Umwelt-

Conceptual Model

V&V Phase

Intermediate Results


Model Development & V&VModel Development & V&V

I

t

Model Developm ent

Sponsor Needs

Structured Problem

Description

Formal Model

Executable Model

Model Results

Objekt

Objekt

Objekt

Objekt

nterakion

Systemgrenzeobjekt“

“Hauptobjekt” mitModellattributen“Umwelt-

Conceptual Model

ModelRequirements

Validation

StaticModel

StructureVerification &

Validation

StaticCode & HW

Verification &Validation

DynamicSimulation

Model Behavior


V&V Phase

Intermediate Results

FormalModel


P(EE)P(EI)P(EF)P(EA)P(EP)following problem...Ok, we have got the


V&V - Process „Tailoring“V&V - Process „Tailoring“

I

t

1.1

Ok, we have got the following problem...

O k, w e h ave g ot t he fol l ow ing pr oblem...

?1.2

O bj ekt

O bj ekt

O bj ekt

O bj ekt


ob jekt“



O k, w e h ave g ot t he fol l ow in g problem...

O bj ekt

O bj ekt

O bj ekt

O bj ekt

S y s t e m g r e n z e

“Umwel t-

ob jekt“


Mod ell attr ib uten?2 . 3

3.1

O bj ekt

O bj ekt

O bj ekt

O bj ekt


ob jekt“


Mod ell attr ib uten

?3 . 2

?3.3


?3.4

4.1

?4.2

O bj ekt

O bj ekt

O bj ekt

O bj ekt


ob jekt“

“Ha uptob jekt” m itMod ell attr ib uten

?4 . 3

?4.4


?4.5

5.1

?5.2

?5.3

O bj ekt

O bj ekt

O bj ekt

O bj ekt

S y s t e m g r e n z e“Umwel t-ob jekt“

“Ha uptob jekt” mitMod ell attr ib uten

?5 . 4

?5.5

O k, w e h ave g ot t he fol l ow in g problem...

?5.6

2.1Objekt

Objekt

Objekt

Objekt

nteakto

Systemgrenze“Umwelt-objekt“


Sponsor Needs

Structured Problem

Description

Executable Model

Model Results

“Umwelt-

Conceptual Model

Formal Model


Our Contribution to International Our Contribution to International Efforts on V&V:Efforts on V&V:

ITOP: International Test Operations Procedure

(Working Group of Experts WGE 7.2: V&V) NMSG: Nato Modeling and Simulation Group

(019 Task Group 016: “V&V of Federations”) SISO PDG: SISO Product Development Group:

(V&V of Federations)

V&V – CONVENTION

(October 20-24, 2004, Munich-Germany)

EU-COST 285 – Action

„ ..M&S in innovative telecommunications...“ ( 2003 – 2007)


4. Conclusions4. Conclusions

Strategic approach: „Divide and Conquer“ !

M&S as a Multi-Phase Engineering Process: Efficient M&S-design by component-based modeling Credibility assurance by risk-based V&V

Summary: How to improve M&S-credibility and efficiency for

increasingly complex simulation applications?


Focus of our research effortsFocus of our research efforts

Model engineering process: Re-useability of model components Tailored model documentation (Risk analysis) tailored V&V (&A)

M&S performance & reliability: Parallel and distributed M&S (Collaborative modeling)


I

t

Thank you very much

for your interest

and attention!


M&S – Source of Knowledge and ExpertiseM&S – Source of Knowledge and Expertise

ModelingExpertise

(HW-)SW-Expertise

Experimental

Design and Analysis

DomainKnowledge

User-Knowledge

SystemAnalysis

Model Formalization

Implementation

Experimentation

ProblemDefinition



Example: „Booking System“

Formal Model (Version 1): queuing net

Institut für Technische Informatik Fakultät für Informatik Universität der Bundeswehr München...

Documents

Transcript of Institut für Technische Informatik Fakultät für Informatik Universität der Bundeswehr München...