LIVERPOOL UNIVERSITY

MASTER OF RESEARCH

DECISION MAKING UNDER RISK AND UNCERTAINTY

Robust Probabilistic Risk/Safety Analysisof Complex Systems and Critical

Infrastructures

Author:

Roberto ROCCHETTA

Supervisors:

Dr. Edoardo PATELLI

Dr. Matteo BROGGI

Dr. Sven SCHEWE

September 21, 2015

Summary

This report summarize the research outputs carried out within the MRes in Decision Mak-

ing under Risk and Uncertainty. The supervisors Edoardo Patelli, Matteo Broggi and Sven

Schewe, involved in the project “Robust Probabilistic Risk/Safety Analysis of Complex Sys-

tems and Critical Infrastructures”, have considered of primary interest for the year to improve

my knowledge on advanced methodologies for risk analysis and uncertainty quantification.

Hence, classical uncertainty quantification approaches, generalized uncertainty quantification

methodologies and new applications for health management of mechanical systems have been

explored. It has been assumed that, in addition to a methodological review, best learning ap-

proach to be endorsed was a pragmatic one. Thus, real case studies and real applications have

been analysed, more specifically three practical applications have been faced so far.

First, a purely classical uncertainty quantification problem have been tackled, named the

“Uncertainties in GPS positioning” challenge. It is an academic challenge launched by French

Institute for Transportation Science and Technology Geolocalisation Team (IFSTTAR/ CoSys/

Geoloc), jointly with the French Federation of Mathematical Games. The challenge have been

accepted and faced in collaboration with three students of the risk institute. Although authors

and supervisors have not considered the completed work novel enough to be published, it has

been extremely beneficial for the MRes, especially thanks to the issues and problems tackled

during the study. The main challenges were to deal with imprecise real world data, minimize

computational time and combine classical uncertainty quantification techniques such as Monte

Carlo to algorithms for optimization. It is worth remark that this unpublished work has been

rewarded by the organizing groups and the Liverpool team won the second price in the inter-

national competition (first price between the groups). The interested reader is remanded to the

web link for further details:

http://scmsa.eu/archives/SCM FFJM Competitive Game 2014 2015 comments.pdf

The second case study consist of a challenge launched by the independent, not-for-profit

membership association “National Agency for Finite Element Methods and Standards” (NAFEMS).

This application has been retained good starting point to approach novel uncertainty quantifi-

1

cation methodologies, important tools to better cope with lack of information, imprecision and

not consistent data. The challenge has been faced by adopting classical but also generalized

(non-classical) uncertainty quantification approaches, therefore developing a more complex

framework. The computational framework can be seen as the original contribution of the work,

it allows to qualify uncertainty of the information regarding the model parameters, i.e. multi-

ple intervals, sampled values, imprecise intervals. The numerical implementation and solution

to the challenge have been summarised into a conference paper, recently presented during the

last NAFEMS World Congress (2015), which took place in San Diego, USA over four days in

June. The presenter M. Broggi has been awarded with the best presenter award. For the future,

possible journal targets have been already identified. Two possible targets are “Advances in En-

gineering Software” and “Computer Methods in Applied Mechanics and Engineering” which

are good quality journals treating computer science and engineering related problems. Indeed

the work is relevant for the future research project, which are going to be focused on novel

approaches for robust risk analysis of complex systems, therefore it will likely deal with robust

quantification of the uncertainty arising due to randomness and imprecision in the data.

The third application tackled within the Master of research year is a Bayesian model updat-

ing used to assess and manage the health state of a mechanical component, a suspension arm.

The main aims were to obtain an improved real-time crack detection framework accounting for

different sources of uncertainty i.e. unknown crack parameters, randomness in the device pa-

rameters and noisy measurements. The advancement of the work has been recently summarized

in a conference paper presented to the 25th edition of the conference, ESREL 2015, in Zrich.

Also in this case, uncertainty quantification is an important part of the work and Bayesian up-

dating framework have been tested under different prospectives. More specifically, in order to

meet computational time requirements which are strong constraints for real-time applications,

two surrogate model have been tested and selected. Furthermore, as known that effectiveness

of the procedure demand for accurate likelihood definition, three different ad-hoc likelihood

function have been applied and results compared and discussed. The framework resulted to be

a useful tool which require further analysis and tests. Journal publication has been considered,

possible targets are journals focused on prognostic health management and advanced Bayesian

model updating frameworks.

2

Highlights

This section are summed up in bullet points the main contributions and achievements of two of

the work proposed in the MRes academic year.

The NAFENS Challenge Problem

• Numerical rigorous framework to deal with incomplete, imprecise information.

• No unjustified hypothesis on data, no subjective and biased exemptions.

• Neglecting epistemic uncertainties might lead to severe over/under estimation of the re-

liability.

• Solution to the NAFEMS uncertainty quantification challenge problem shows effective-

ness of the framework.

The Bayesian Updating Framework for Real-Time Crack identification

• Bayesian Model Updating to estimate posterior distribution of crack parameters.

• Three empirical likelihood investigated and two surrogate model have been investigated

and discussed.

• Code parallelization strategy adopted and time saving framework thanks to surrogate

model obtained.

• Aleatory uncertainty in the Young’s modulus and measurement noises shows to be im-

portant factor in a good crack parameters updating.

3

Preliminary Conclusions and Future Directions

Risk is the potential of experiencing a loss when a system does not operate as expected due

to the occurrence of uncertain and difficult-to-predict events. The more general qualitative

definition of risk accounts for both uncertainty in the occurrence of hazardous events and the

consequences of these events. Hence, it seems intuitively correct to consider the risk concept

as pervasive and multidisciplinary by definition. In facts, discipline and sector of interest are

involved depending on the type of consequences addressed and on the events considered in the

study. Depending o the discipline, the Decision-Makers will have to face different risky deci-

sions, all affected by uncertainties in at least one of its forms. Point out the limitations of uncer-

tainty quantification approaches and improve the robustness of the decisions is hence essential

to better understand what can change and what can go wrong in achieving goals. In order to in-

crease the robustness of risk assessment several approaches and uncertainty quantification tech-

niques have been reviewed during the year. Indeed classical uncertainty analysis is an important

tool to obtain a representation of model predictions consistent with the state-of-knowledge and

available information. Within an imprecise-information scenario, combination of probabilistic

and non-probabilistic approaches are appealing tools to give different prospective to the results

without a demand of strong initial assumptions. The applicability of the different uncertainty

quantification techniques have been confirmed useful by the NAFEMS challenge case study

and the Bayesian Model Updating case study. The flexibility of the frameworks adopted in

both the works will be further improved and tested on different real-life problems.

For the future are expected beneficial collaboration with the project partners, firstly with the

industrial European Center for Soft Computing (ECSC) but also the academic partner profes-

sor Enrico Zio (Politecnico di Milano and/or Ecole Central de Paris). As example, the ECSC

research activity mainly focus on research units such as fuzzy evolutionary applications, in-

telligent data analysis (data mining, pattern recognition), cognitive computing (computational

systems able to provide linguistic descriptions of complex phenomena). Indeed their expertise

in advanced risk analysis techniques and competence in dealing with uncertainty and impreci-

sion are going to be uttermost important contribution to the final research output quality.

4

A Computational Framework for Classical and Generalized UncertaintyQuantification: Solution to the NAFEMS Challenge Problem

R. Rocchetta & M. Broggi & E. PatelliInstitute of Risk and Uncertainty, University of Liverpool, United Kingdom Tel.: 0044 (0) 151 7944079

ABSTRACT: Classical probabilistic approaches are well-established techniques often used to enhance ro-bustness of simulations by accounting for uncertainty from different sources. Application of classical proba-bilistic frameworks, especially in cases affected by lack of information, may require strong initial assumptionsoften hardly justifiable. The assumptions made to deal with uncertainty due to data unavailability can deeplyinfluence the final results and lead to severe risk misjudgement. Generalised probabilistic approaches have beenrecently introduced to better deal with scarce or limited information. The strengths and the shortcoming of clas-sic and generalised approaches have been pointed out in literature. However, to help acceptance and to betterunderstand their strengths, further comparisons seems to be needed, especially from an applicative prospective.In this paper, different probabilistic approaches are implemented in a common computational framework, whichhas been adopted to solve the NAFEMS uncertainty quantification challenge problem. Strength and weaknessof both the methodologies have been directly faced and presented. The results obtained have confirmed thatclassic probabilistic approaches when bearing strong artificial assumptions may lead to misleading conclusionwhich not fully representative the real available information quality. Generalised probabilistic approaches haveshown to be versatile and powerful tools.

1 INTRODUCTION

Nowadays, it is generally well understood that is nec-essary to include uncertainties in simulations, e.g. dueto variation in parameters and operational conditions,in order to achieve robust design of new products,execute model validation or ensure reliable operationfor the whole product-life. It is important that easyto follow and accurate guidance, best practices andcomputational tools are available to make uncertaintyquantification a standard techniques adopted by alarger public of engineering practitioners.

Classical probabilistic approaches are well es-tablished techniques which can be used to enhancerobustness of the desired results, accounting foruncertainties which can arise from different sources.However, in order to create probabilistic classicalframeworks and overcome lack of or impreciseinformation, strong initial assumption may be neededand are often hardly justifiable. Those assumptions,such as predefined probability distributions, can sub-stantially influence the final outcomes and especiallyin reliability analysis may led to misleading results(Beer, Ferson, & Kreinovich 2013).

Generalized approaches are powerful methodolo-

gies which could in some cases be coupled to thetraditional approaches in order to give a differentprospective on the results, enhancing the overallrobustness. Examples of such flexible frameworksare provided by e.g. (Ferson, Kreinovich, Ginzburg,Myers, & Sentz 2002) and (Patelli, Alvarez, Broggi,& de Angelis 2014). In the last years some effortshave been putted in comparing different generalizedand classical approaches, e.g. see (Zio & Pedroni2013), (Laura P. Swiler 2009), (Alex Diaz De La O2014). However, further confrontations of differentmethodology, especially applied to real study cases,are required to showcase and promote generalizedapproaches, as well as the use of dedicated softwaretools to automate the analysis.

In this paper, a computational tool useful toimplement generalized and classical uncertaintyquantification in a common framework is presented.The computational framework has been testedproviding solution of the uncertainty quantifica-tion challenge problem proposed by the NAFMSstochastic group (NAFEMS 2013). The problem istackled by adopting classical, but also generalisedprobabilistic approaches, which fits in the frameworkof imprecise probabilities. The results obtained bythese novel approaches, which account for both epis-

temic and aleatory source of uncertainty and requireweaker initial assumptions, will be compared with theprevious solution to take sides of the limitations andshortcomings of both the methodologies. This willconfirm the increased flexibility in the uncertaintytreatment found when also imprecise probabilitiesare employed, as well will provide guidance to im-plement classical and generalized approaches whendifferent type of imprecise information is provided.Increased flexibility can be achieved by improvingthe uncertainty models with provided information ofdifferent quality and by efficiently including evidencefrom wide range of sources.

At the last NAFEMS world congress, held in 2013Salzburg, the NAFEMS Stochastic Working Grouphas launched a challenge problem open to contribu-tors from Academia, Government and Industry. Thesystem subject to the analysis is a simple RLC serieselectric circuit with different levels of uncertaintyestimated around the model input parameters (i.e.,intervals from multiple sources, finite number ofsampled values, incomplete intervals). The scopeof the challenge was to evaluate the reliability ofthis circuit under the different input uncertaintiesas well as to quantify the value of information ineach case. The interested readers are reminded toreference (NAFEMS 2013) and (NAFEMS 2014)for further details about the problem definition. Thiscase study has been retained particularly suitable tobe solved using probabilistic and also generalizedapproaches and serve as platform for further compar-isons between different methodologies. The proposedcomputational framework has been embedded toOpenCOSSAN, see (Patelli, Broggi, Angelis, & Beer2014), open general purpose software for uncertaintyquantification which has been employed in all thesteps of the analysis.

This paper is structured as follows: Section 2presents theoretical background, a brief reviewof generalized approaches and an introduction toDempster-Shafer methodology. In Section 3 a syn-thetic overview the numerical framework is made.Section 4 shows the NAFEMS challenge problem,the reliability requirements and computational chal-lenges. In Section 5 the classical and generalized ap-proaches and results for the different tasks of the chal-lenge are presented. In Section 6 a brief discussion onthe limitations of the different approaches is presentedIn Section 7 conclusions are drawn.

2 THEORETICAL BACKGROUND

Uncertainties can generally be described in twogroups, the so called aleatory and epistemic uncer-tainties (Der Kiureghian and Ditlevsen 2009). Thealeatory, also indicated in literature as Type I or irre-ducible uncertainty, is related to stochastic behaviours

and randomness in events and variables. Hence, dueto its intrinsic random nature is normally regarded asnot reducible, that means the degree of uncertaintycannot be decreased even if the knowledge of thesystem or of the physical phenomena is improved.The epistemic, also called Type II or reducible un-certainty, is commonly related to lack of knowledgeabout a particular behaviour, imprecision in measure-ment and poorly designed models. It is considered asreducible since further data can reduce the level ofuncertainty, but this is not always practical or feasible.

The Monte Carlo is a broadly applied classical ap-proach often used to deal with uncertainties withoutdifferentiates between aleatory and epistemic types(Marseguerra and Zio 2002). It has been consideredin the study because is flexible and is one of themost well-established classical methodology to char-acterize and propagate uncertainty. As mentioned,the Monte Carlo (MC) approach has been appliedin several survey, e.g. Rocchetta et al. 2015 andChing and Chen 2007, no further details about thistechnique will be provided in this paper.

Nevertheless, it is well-understood that MC ap-proach have to face serious limitation, especiallyin the computational aspect, e.g. high number ofsimulations are needed in order to sample just onerare event. Moreover, MC classical approach cannotdifferentiates between aleatory and epistemic uncer-tainty, hence assumptions may be needed to deal withimprecise information. In the last decades, effortswere focused in the treatment of imprecise knowl-edge, non-consistent information and both epistemicand aleatory uncertainty by efficient approaches.The methodologies are discussed in literature bydifferent mathematical concepts: Dempster-ShaferEvidence theory (Dubois and Prade 1988, Shafer1976, interval probabilities (Augustin 2004), leveltwo probably approach (Helton et al. 2004, Pedroniet al. 2013), Fuzzy-based approaches (Blair et al.2001) and Bayesian updating approaches (Faber 2005Kiureghian 2008, Veneziano et al. 2009 Ching andChen 2007), are some of the most intensively appliedconcepts.

Considering all the reviewed methodologies, theDempster-Shafer approach based on the theory ofEvidence has been selected to tackle the challengeproblem through a generalized framework. The mainreasons for this selection are that Dempster-Shaferapproach requires little assumption and can be easilyimplemented and coupled to classical probabilisticframeworks; moreover it is generally applicableallowing solving all the tasks proposed in the chal-lenge problem. Kolmogorov-Smirnov test and KernelDensity Estimator have been also used to representthe sample uncertainty for extremely small samplesize, one of the challenge tasks (see Section 4) The

first is used combined to Dempster-Shafer approachin a Generalized framework; the latter is combinedto Monte Carlo simulation to provide a classicalsolution to the task.

2.1 Dempster-Shafer Structures and ProbabilityBoxes

One of the largely used frameworks of subjectiveprobability is the DempsterShafer theory which isa well-suited framework to represent both aleatoryand epistemic uncertainty and it can be seen asa generalization of Bayesian probability. In theDempster-Shafer theory, numerical measures ofuncertainty (a degree of belief also referred to asa mass) may be assigned to overlapping sets andsubsets of hypothesis, events or propositions as wellas individual hypothesis, see e.g. Beer et al. 2013.Probability values are assigned to sets of possibilitiesrather than single events. In the Shafer theory the setsare represented as intervals, bounded by two values,belief and plausibility.

In order to characterize both epistemic and aleatoryuncertainty, probability boxes (referred as P-boxes)are often used. P-boxes can be seen as a further gener-alization of the Dempster-Shafer structures where thesets are represented by distributions. The Dempster-Shafer structures are similar to discrete distributionbut rather than precise points, the locations where theprobability mass resides are set of real values, it canbe expressed as set of focal elements as presented byFerson et al. 2002:

(([x1, x1],m1), ([x2, x2],m2)..., ([xn, xn],mn)

)(1)

Where[xi, xi] is the ith focal element with upperbound xi and lower bound xi, mi is the probabilitymass associated with the ith focal element. P-boxesare set of cumulative distribution functions (CDFs)for which lower and upper bounds are assigned[FX , FX ]. Note that the probability distributionassociated to the random variable of interest canbe either defined or not. As summarized by (Patelliet al. 2014) the first are generally named distribu-tional P-boxes (or parametric P-boxes) the latter arecalled distribution-free P-Boxes (or non-parametricP-Boxes). Figure 1 shows an illustrative example ofdistributional P-Box, the parent distribution is thenormal distribution; the red dashed line is the lowerbound FX , black solid line is the upper bound FX .

The wider the distance between upper and lowerbound the higher epistemic uncertainty is associatedto the random variable. The bounds for the CDFsmean bounds on probabilities, upper and lower prob-ability bounds can be hence obtained as:

Figure 1: Illustrative example of distributional P-Box.

FX = P (X ≤ x) = 1− P (X ≤ x) (2)

FX = P (X ≤ x) (3)

In literature the lower bound on probability is some-time referred as plausibility Equation (4( and the up-per bound as belief, Equation (5). Cumulative Plausi-bility and Belief function can be computed as:

Pl(z) =∑xi≤z

mi (4)

Bel(z) =∑yi≤z

mi (5)

Dempster-Shafer structures can be always translatedinto a P-box but without forgetting that is not an infor-mation preserving procedure. These approaches arestraightforward to deal with some of the cases pro-posed in the NAFEMS challenge, such as multipleand overlapping intervals and inconsistent sources ofinformation. The drawback is that the propagation ofintervals and P-boxes through the system can resultscomputational expansive. Nevertheless, the quantifi-cation approaches are generally not-intrusive andhence applicably to any model. For further acknowl-edgement the reader is reminded to Laura P. Swiler2009. This is an interesting feature of the approachwhich is therefore suitable to be used in parallel toclassical uncertainty quantification method to givewider prospective on the analysis.

2.2 The Kernel Density estimator

It is in general difficult to get the true distributionfrom a small number of samples using parametricmethods. This is because there is no enough informa-tion to estimate the PDF when only few data pointsare available. Kernel density estimator is another non-parametric approach that can be used to estimatethe probability density function of a random variable

(Pradlwarter and Schuller 2008). The approach doesnot need any assumptions regarding the underlyingdistribution. A commonly used univariate parametrickernel is the Gaussian or normal Kernel:

f(x) =1

nσ√

2π

n∑i=1

(−(x− xi)2

2σ2

)(6)

where f(x) represents the estimated probabilitydensity function of n samples xi drawn from an un-known density function f . The variance (or band-width) σ2 is the only parameter that needs to be es-timated. The best bandwidth can be estimated usingfor instance the Silverman’s rule of thumb (Silverman1986) or in case of very small sample sizes the ap-proach proposed by Pradlwarter and Schuller 2008.

2.3 Kolmogorov-Smirnov Test

The so called Kolmogorov-Smirnov statistical test(Massey 1951) is one possible non-parametric ap-proach that can be used to characterize the uncer-tainty of a process starting from samples with areference probability distribution. The Kolmogorov-Smirnov (KS) test is a distribution-free statistical testbased on the maximum difference between an empiri-cal CDF and a hypothetical CDF. It returns upper andlower bound of CDFs assuming a predefined confi-dence level. The bounds can be computed by use ofEquation (7) as:

min (1,max(0,DF (x),D(α,n))) (7)

where DF (x) denotes the best estimate of the dis-tribution function and D(α,n) is the one-sample KScritical statistic for confidence level 100(1 − α)%where is the selected significance level and n the sam-ple size. The KS critical statistic can be therefore usedto obtain different confidence limits on the CDFs bychoosing different critical values of the statistic test.Different level of confidence lead to different confi-dence bounds on the CDFs which, when propagated,produce boarder or narrow bounds of the resulting P-boxes.

3 NUMERICAL IMPLEMENTATION

Generalized and classical probabilistic uncertaintyquantification methods are powerful techniques todeal with uncertainty and combine methodologies ina common computational framework is of high in-terest for the uncertainty analyst. OpenCossan is acollection of methods and tools under continuous de-velopment, coded exploiting the object-oriented Mat-lab programming environment. It allows defining spe-cialized solution sequences including any reliability

methods. Hence optimization algorithm, new reliabil-ity methods or uncertainty quantification and prop-agation techniques can be easily added. In the pre-sented work a computational tool useful to implementGeneralized and Classical probabilistic approaches ina common framework is presented. Schematic repre-sentation of the computational framework is shown inFigure 2.

Figure 2: Simplified representation of the computa-tional tool

Algorithm 1 Dempster-Shafer structures PropagationAlgorithm1: procedure DS PROPAGATION2: for each Parameterj do3: Assign mass mj

i∀ focal elements i ∈ Parameterj4: end for5: if

∑i

mji 6= 1 then

6: ERROR. Masses have to be normalized7: end if8: for each ω ∈ Ω which is the set of all possible interval

combinations do9: for K=1: last performance variable do

10: Compute Minimum and Maximum of K11: [Min(K),Max(K)]ω12: end for13: Compute mass for the combination ω

14: mω =∏

j mji

15: Save [Min(K),Max(K),m]ω16: end for17: Save the Propagated DS structure18: ([Min(K),Max(K),m]1, ..., [Min(K),Max(K),m]Ω)∀ variable K

19: Plot the DS structure translated as P-box20: end procedure

In the proposed framework, the parameters infor-mation is stored in UncertainParameter which con-tain parameters name, type of uncertainty and nu-merical information such as bound values or sam-ples values. The propriety Stype identify the un-certainty type affecting the parameter e.g. singleor multiple intervals, samples or imprecise intervalsfor which only upper or lower bound are well de-fined. Whereupon UncertainParameter is defined, pa-rameter uncertainty is further characterized by useof Kolmogrov-Smirnov test, Kernel Density estima-tion or by defining random variables from a set ofwell-known distribution (e.g. uniform). Depending onthe uncertainty type, only some characterization andpropagation methodology are available, as schemati-cally explained in Table 1. Among the different reli-ability and uncertainty propagation methods not justclassical methods are available, e.g. Monte Carlo,Latin Hypercube, but also some generalized methodssuch as Dempster-Shafer (DS) propagation and P-boxapproaches.

Table 1: Scheme of Methods for the uncertainty char-acterization propagation depending on the uncertaintytype

Type of un-certainty

Characterize Propagate

Single orMultipleIntervals

RandomVariable, DSstructure

Monte Carlo,DS propaga-tion

Sampled Val-ues

KernelDensity,KolmogrovSmirnov test

MonteCarlo,P-boxpropagation

Imprecise In-tervals

RandomVariable, DSstructure

Monte Carlo,Line Sam-pling, DSpropagation

Pseudo-code for the Dempster-Shafer structurepropagation is presented in Algorithm 1. Further de-tails on the propagation algorithm are provided in sec-tion 5.1.

4 THE NAFEMS CHALLENGE PROBLEM

The challenge problem, prepared by the stochasticgroup NAFEMS (2013), consist of four uncertaintyquantification and information qualification tasks,ideate to identify and promote to the industry bestpractices to deal with uncertainty. In the challenge,the analysts are asked to evaluate the reliability of anelectronic resistive, inductive, capacitive (RLC) seriescircuit shown in Figure 3, on meet performance re-quirements.

Four different cases (A, B, C and D) have beenproposed, each one having incomplete, scarce or im-precise information about the system parameters, as

Figure 3: RLC series circuit. Input signal is the stepvoltage source for a short duration. Output responseis the voltage at the capacitor.

Table 2: CASE-A CASE-B CASE-C and CASE-D,available information

CASE A R[Ω] L[mH] C[µF]Interval [40,1000] [1,10] [1,10]CASE B R[Ω] L[mH] C[µF]source 1 [40,1000] [1,10] [1,10]source 2 [600,1200] [10,100] [1,10]source 3 [10,1500] [4,8] [0.5,4]CASE-C R[Ω] L[mH] C[µF]SampledData

861, 87,430, 798,219, 152,64, 361,224, 61

4.1, 8.8,4.0, 7.6,0.7, 3.9,7.1, 5.9,8.2, 5.1

9.0, 5.2,3.8, 4.9,2.9, 8.3,7.7, 5.8,10, 0.7

CASE D R[Ω] L[mH] C[µF]Interval [40,RU1] [1,LU1] [CL1,10]Other info RU1 >650 LU1 >6 CL1 <7NominalValue

650 6 7

shown in Table 2. In the CASE-A single intervals,one upper bound and one lower bound for parameterR L and C are given. In the CASE-B, each parame-ter can lay within multiple intervals, i.e. three upperand lower bounds. In CASE-C small sample sizes often sampled points for each parameter are provided.Finally for the CASE-D, imprecise bounds are theonly available information; this last case is similarto CASE-A, but one bound is not precisely defined.Uncertainty is characterized using different classicaland generalized approaches according to the level ofknowledge available for the parameters R, L and C.

The equations governing the RLC circuit are pro-vided by the challengers. The transfer function of thesystem is defined as:

Vc(t)

V=

ω2

S2 + RLS + ω2

(8)

Depending on the values of R, L and C, the system

may be classified as under-damped critically dampedor over damped.Under damped (Z <1):

Vc(t) = V + (A1cos(ωt) +A2sin(ωt)) exp−αt (9)

Critically damped (Z =1):

Vc(t) = V + (A1 +A2t) exp−αt (10)

Over damped (Z >1):

Vc(t) = V + (A1 expS1t+A2 expS2t) (11)

where roots are computed as:

S1,2 = −α±√α2 − ω2 (12)

and damping factor Z, values of ω and α are deter-mined as follow:

α =R

2L;ω =

1√LC

;Z =α

ω(13)

Coefficients A1 and A2 are determine by assuminginitial voltage and voltage derivative equal zero,unitary step voltage function were considered.

In the challenge, the main goals consist in qualifythe value of information and evaluate the reliabilityof the system with respect to three requirements. Thefirst two are on the voltage at the capacitance Vc (sincethe second requirement on rise time can be translatedinto a voltage requirement as well). More specifically:

Vc(t = 10ms) > 0.9V (14)

Vc(t = tr) > 0.9V (15)where tr is the voltage rise time, the time from 0to 90% of the input voltage, and have to be lessthan 8 ms. The third requirement is on the under-damped system responses which have to be dis-charged (Z ≤1). It can be observed that the third re-quirement imply a monotonic behaviour for Vc, there-fore it is trivial rewrite the second requirement as inEquation (16):

Vc(t = 8ms) > 0.9V (16)

The cumulative distribution function (CDF) of a ran-dom variable is commonly used in reliability analysisto extract useful knowledge about the so called prob-ability of failure as follows:

FX(x) = P (X ≤ x) (17)

Specifically, Vc(10ms), Vc(8ms) and Z are con-sidered as random variable and Equation (17) canbe used to compute the probabilities that the sys-tem fail on meet the three requirements by estimatingFVc(10ms)(0.9), FVc(8ms)(0.9), and FZ(1)

If bounds on the CDFs are obtained, Equations (2)-(3) can be used to compute the bounds on probabil-ity of not meet the requirements, [PV c10fail , PV c10fail ],

[Ptrfail , Ptrfail ], and[PZfail, PZfail

].

5 CLASSICAL AND GENERALIZEDAPPROACHES FOR THE NAFEMS CASESSOLUTION

Within this section, classical and generalized ap-proaches are adopted to tackle the four cases of theNAFEMS challenge problem. Uncertainty character-ization, propagation are presented for each case, fur-thermore available information quality and system re-liability is discussed.

5.1 CASE A and CASE B

In CASE-A single interval information is providedfor the parameters (see, Table 2) while multiple in-terval information is available in CASE-B. Adoptingthe generalised probabilistic method and in particularthe Dempster-Shafer approach, the CASE-B degener-ate to CASE-A if probability mass equal one is as-signed to the first source of information. This becausein CASE-B intervals values for source 1 correspondto interval values in CASE-A. Due to the consider-ations made, the two cases are presented and solvedtogether.

Classical ApproachIn the Case-A, a Monte Carlo simulation-basedapproach has been implemented. Classical proba-bilistic approaches do not allow to model explicitlyintervals. A largely used approach is to model theparameters as random variables which may varyuniformly between the provided interval bounds,assumption made with respect to the Principe ofmaximum entropy. However, this implies that eachvalue inside the interval is equally probable and thisis a very strong assumption that might not be justifiedby the evidence from experimental data or expertjudgement. Uncertainty propagation and reliabilityassessment are performed by randomly sample RL and C from the associated uniform distributionsand evaluating if the system requirements are met(e.g. system is under failure with respect to the firstrequirements, if Equation (14) is not satisfied).

The solution of the Case-B follows the same proce-dure as the Case-A. Each interval is considered indi-vidually. Hence, three different uniform distributionsfor each R, L, and C variables are used to modelling,one for each source of information. The reliabilityanalyses have been performed to estimate compute 3probabilities of failure, one for each source of infor-mation. Furthermore, for comparative purpose, over-all probability of failure is obtained in CASE-B by av-eraging the results, this is done assuming each sourceof information as equally likely.

Table 3: Results CASE-A, comparison between Co-efficients of Variation when computing different sam-ples.

1000 samples 10000 samplesCASE-A Prob. Cov Prob. CovPV c10fail 0.284 0.050 0.268 0.017Ptrfail 0.346 0.044 0.364 0.013

Table 4: Results CASE-B: Monte Carlo resultingprobability of failure for the three sources, 10000samples.

CASE-B

Source1

Source2

Source3

MeanSources

PV c10fail 0.270 0.512 0.123 0.303Ptrfail 0.354 0.670 0.209 0.411

Classical ResultsThe Monte Carlo simulation has been performed inorder to compute failure probabilities and Covariancefor CASE-A and CASE-B. First, the failure probabili-ties has been estimated using 1000 samples, secondlynew analyses with 10000 samples has been carried tocompare the Coefficients of Variation (Cov) of the es-timated probabilities, as shown in Tables 3. It has beennoticed that, as in the CASE-A, also in CASE-B theprobability of failure slightly changes even after in-creasing the samples. The probability of fail to meetthe requirement one, as expected, is lower than theprobability of fail to meet the requirement two.

It is possible to see from Table 4 that the systemconsidering source 2 has the lower reliability. In 1000samples, approximately 512 will be out of the first re-quirements. The Source 3 has the lowest estimatedprobability of failure and the Source 1 shows an in-termediate Probability of failure. The averaged prob-ability of failure is about 0.3 and 0.41 for the first andsecond requirement, respectively.

Generalized ApproachThe intervals that define the possible values of the pa-rameters can be represented by means of the gener-alised probabilistic approach without making any as-sumption about the possible distribution of the values.Parameter uncertainty has been characterized by useof Dempster-Shafer structures. A general Dempster-Shafer structure for each parameter is defined byEquation (1). For CASE-A three Dempster-Shaferstructures composed by a single focal element havebeen defined as follows:

([R1,R1],m1

) ([L1,L1],m1

) ([C1,C1],m1

)(18)

where m1 is equal one. For CASE-B, three structureshave been considered as in Equation (19)-(21), eachcomposed by three focal elements:

(([R1,R1],m1), ([R2,R2],m2), ([R3,R3],mn)

)(19)

(([L1,L1],m1), ([L2,L2],m2), ([L3,L3],mn)

)(20)(

([C1,C1],m1), ([C2,C2],m2), ([C3,C3],mn))

(21)

m1 +m2 +m3 = 1 (22)

where [Ri,Ri] represents the ith interval bound forthe resistance, [Li,Li] is the ith bound for the induc-tance, [Ci,Ci] is the ith bound for the capacitanceand mi is the probability mass associated to the ithsource. The probability masses associated to the dif-ferent sources have been considered equal and nor-malized. The CASE-B degenerate to the CASE-A ifthe probability mass m2 and m3 are set equal to zero.It is not possible here to establish if some sources ofinformation are better, thus information derived fromdifferent sources is assumed as equally likely.

Dempster-Shafer structures PropagationThe following procedure was adopted in order topropagate the uncertainty characterized by Dempster-Shafer structures:

• First, probability mass is assigned to each focalelement (interval) of R L or C, e.g. for CASE-Bm1,m2,m3 are assigned to intervals of source 12 and 3, respectively.

• n ’Parameter cells’ are constructed by permuta-tion of the focal elements, i.e. in CASE-B n=3!.The first parameter cell is built by selecting thefirst interval of the parameter R and first inter-val of the parameter L and combining them withthe first interval of C. The second cell is built se-lecting the first interval of R and first of L andcombining them with the second interval of C,so on in a combinatorial manner .

• For each cell, Latin Hypercube Sampling (LHS)strategy is used to samples parameter values.Then the system responses are evaluated in termsof Vc and Z and the minimum and maximumof the outputs are calculated. An alternative ap-proach based on optimization technique can beused to identify the output bounds for intervalcells .

• The results are stored and n min-max intervalsof Vc(8ms), Vc(10ms) and n min-max interval ofZ are saved, each interval correspond to a fo-cal element for the output of interest. The Re-sults are used to create Dempster-Shafer struc-tures, see Equation (1,where probability masseshave been assigned depending on the probabil-ity masses of the focal elements composing thecells. As example Dempster-Shafer structure forZ and Vc(8ms) will result:

([Z1,Z1],mc1), ..., ([Zn,Zn],mcn)

([Vc(8)1, Vc(8)1],mc1), .., ([Vc(8)n, Vc(8)n],mcn)

Where the probability mass computed for nparameters cells are mc1 = m1m1m1, mc2 =m1m1m2, etc.

• Finally intervals the Dempster-Shafer structuresare used ,as explained in (Ferson et al. 2002),to create probability boxes of Vc(8ms), Vc(10ms)and Z. Equations (4)-(5) are used to compute theBelief and Plausibility bounds and failure proba-bility intervals.

Generalized ResultsApplying the procedure to the CASE-A, the resultingP-boxes gave no valuable information on the failureprobability for the three performance requirements.The probability of failure is in fact just bounded inthe interval [0,1] for all three the requirements.

In CASE-B equal probability mass have beenassigned to the three distinct sources of information,such that m1 = m2 = m3 = 1/3. The resultingP-boxes of V c and Z are shown in Figures 4-6.Resulting bounds are presented in Table 5, it can beseen that outputs have high uncertainty associated,but reduced if compared to the Case-A. Figure 5shows that Ptrfail lay within the interval [0,0.9]. Itcan be also observed that PZfail

upper and lowerbounds are [0,0.7] and that PV c10fail lay withinthe interval [0,1]. The CASE-B includes all theinformation available for the CASE-A (informationin source 1 coincide with information of CASE-A)plus two additional sources of information. In thisparticular case, the additional sources of informationcontribute on reducing the uncertainty on the systemperformance.

Comparison between CASE-B and A show that thefirst produces narrower intervals for the probability offail to meet requirement one and three. On the otherhand, failure probability for requirement two do notshow any uncertainty. It can be therefore argued that,with respect to the first and third requirements, thequality of the information given in CASE-B is highercompared to the quality of the information given inCASE-A. On the other hand, results have the samequality with respect to the second reliability require-ment.By comparing classical and generalized results, val-ues obtained for the failure probability (classicalapproach) lay within the bounds obtained usingthe Dempster-Shafer methodology (generalized ap-proach). Obtaining a solution for the generalized ap-proach point out that the uncertainty affecting the in-formation for both CASE-A and CASE-B seems tobe strongly underestimated by the classical approach.This is probably due to the assumption made on theparent distribution needed to apply classical method-ology.

Figure 4: P-box Vc(10ms) CASE-B

Figure 5: P-box Vc(8ms) CASE-B

Figure 6: P-box Damping coefficent CASE-B

Table 5: Results CASE-B, comparative table.CASE-B Source

1Source2

Source3

AllSources

PV c10fail [0,1] [0,1] [0,1] [0,0.9]Ptrfail [0,1] [0,1] [0,1] [0,1]PZfail

[0,1] [0,1] [0,1] [0,7]

5.2 CASE C

The available data consists of 10 sampled valuesfor each parameter of R, L and C, respectively. Ta-ble 3 summarize the parameter information given forCASE-C. No additional information about the param-eter such as, sampling procedure or parent probabilitydistribution functions has been provided.

Classical ApproachTwo classical approaches for the solution CASE-Chave been developed:

First, only finite number of combinations of theinput values is accounted. A Full Factorial Designof Experiments (DOE) was created to generate allthe possible combinations of the defined points anda failure analysis was performed for each of them.This approach is fast and rather easy to implement,but clearly have severe limitations, e.g. assume thatsampled values are the only allowed values for theparameters.

In the second classical approach, the Kernel Den-sity Estimator (KDE) is used to fit probability distri-bution to small sample data. Probability distributionfunctions for the parameter have been fitted using thesmall samples and the Gaussian Kernel Density esti-mation as explained in subsection 2.2. Then, a MonteCarlo simulation is carried out to sample parametersvalue from the fitted distributions, the system perfor-mance is evaluated and the reliability computed. Thissecond approach has been implemented assuming aGaussian Kernel and assuming samples as indepen-dent samples of a random variable.

Classical ResultsFirst attempt to solve classically CASE-C consideronly a finite number of parameter values and parame-ter combinations. Just 10 sampled points for each ele-ment R, L and C are available, which means 1000 pos-sible combinations. Within the 1000 possible combi-nations, 150 lead to system requirement failure be-cause Vc(10ms) was lower than 0,9V, 230 input com-binations produce rise times greater than 8ms and 71input combinations lead to under-damped responses.The Figure 7 shows combinations of input values thatproduce systems that satisfy or not each requirement.Note that the probabilities of failure are exact, not anestimation, since all the possible combinations wereanalysed. Red triangular marks correspond to failure

Figure 7: Design of experiment results

combinations, blue dots correspond systems whichhave sanctified the requirements.

In the second approach, the Kernel Density Estima-tor is used to fit probability distribution to small sam-ple data. The results obtained in the first classical ap-proach, which adopted Design of experiment to rep-resent the uncertainty are PV c10fail=0.15,Ptrfail=0.23and PZfail

=0.071. Kernel Density fitting and MonteCarlo sampling from the fitted distributions lead toprobability slightly major if compared to the DOE so-lution, PV c10fail=0.232, Ptrfail=0.292, PZfail

=0.121.This is probably due to the longer tail of the parame-ters fitted by KDE, which result having a very longright tails. Figures 8-10 shows the resulting CDFs,Z CDF has been zoomed around the value Z=1 forgraphical reasons.

Compared to the results obtained for CASE-A ap-plying classical methodology, the probability of fail-ure for CASE-C result generally lower. Same consid-eration can be done by comparison to the overall re-sults obtained for case CASE-B but not all the singlesources of information.

Generalized ApproachCASE-C has been solved also by using generalizedapproaches, first confidence bounds have been ob-tained by applying the KS test, see Section 2.3. Afterthis first characterization for the sample uncertaintythe obtained P-boxes for the parameter are propa-gated through the system and the outputs of inter-est computed. The uncertainty propagation procedureadopted is similar to the procedure used in CASE-Aand CASE-B. Three different confidence levels arebeen considered for the KS test, therefore three dif-ferent bounds have been propagated. To characterizethe uncertainty, Kolmogorov-Smirnov critical statis-tic is used as shown in Equation (7). The results arebounds on the empirical distribution function of thesampled parameters. Figures 11-13 display upper andlower bounds (dashed and solid black lines) for theempirical CDF (solid blue line) of the inductance L,R and C. The bounds in figure refer to a confidencelevel α=0.05. Some assumptions have to be made inorder to apply the KS test, the samples are regarded asindependent and identical distributed and maximum

Figure 8: CDF of Vc(10ms) CASE-C

Figure 9: CDF of Vc(8ms) CASE-C

Figure 10: CDF of Z

Table 6: Results CASE-C, probability Bounds for thethree requirements, three confidence levels .

CASE-C α=0.01 α=0.1 α=0.2PV c10fail [0,0.87] [0,0.7] [0,0.63]Ptrfail [0,0.92] [0,0.77] [0,0.7]PZfail

[0,0.83] [0,0.7] [0,0.64]

and minimum parameter values have been assumedin order to truncate the CDF bounds. Due too physicalconstraint, all the parameters must be positive and thiscondition allows to set then lower bound. The upperbound is in first instance assumed equal to the sam-ple mean plus three times the standard deviation ofthe samples. It is important to notice that very longtails if underestimated can lead to safety overconfi-dence especially in reliability studies. Hence, follow-ing the considerations made, it has been selected a rel-atively high upper bound to truncate the CDF. In orderto better investigate the goodness of the assumptions,analysis of the results (i.e. Failure probability) due tothe variation of the assumed upper bounds has beenperformed. An increasing of the upper bounds (e.g.6 time the standard deviation of the sample) slightlychange the resulting probability bounds, as displayedin Figure 14.

Generalized Results

Even for CASE-C, the failure probabilities obtainedby classical approach lay within the probability in-terval obtained by Generalized approaches. In Fig-ures 15-17 P-Boxes for the voltage at the 10th ms,8th ms and Z are presented, red blue and blackcolour lines refer to α=0.01, α=0.1 and α=0.2 respec-tively.Figure 17 has been zoomed around the valueZ=1 for improve the graphical output. Each confi-dence level lead to different probability bounds on theCDFs and resulting P-Boxes. The bounds on prob-ability of fail to meet the requirements are shownin the Table 6. It can be noticed that, on the firsthand probability intervals still wide, as already ob-served in CASE-A and CASE-B, which is typical ofcase denoted by high epistemic uncertainty. On thesecond hand, CASE-C shows an increased precisioncompared to the CASE-B and A. The resulting fail-ure probability bounds obtained applying Dempster-Shafer approach to CASE-C, appear less uncertaincompared to all the four cases. This increased pre-cision may find intuitive explanations in the as-sumptions made in order to apply the Kolmogorov-Smirnov approach or may be due to the higher qual-ity of the information provided in the challenge. Alsofor CASE-C, results show that uncertainty on the sys-tem reliability has been underestimated by using theclassical approaches.

Figure 11: P-box R parameter CASE-C

Figure 12: P-box L parameter CASE-C

Figure 13: P-box C parameter CASE-C

Figure 14: Qualitative variation in the probabilitybounds if maximum selected equal to mean plus 3 or6 standard deviations

5.3 CASE D

In this last task, bounds of the parameters areprovided, similarly to CASE-A. However, just onebound is precisely defined for each parameter. Moreprecisely, the imprecisely defined bounds are theupper bounds of R and L and the lower bounds of C.Information for CASE-C is provided in the challengeas shown in Table 5.The case-D, similarly to CASE-B, does not havea precise information about the parameters range.The lower bound of C could be any value betweenzero and 7 while the upper bound of R and L couldtake any value up to the infinite (even thought, inpractice, it is not possible). Lower bounds can befixed by physical consideration on the parameter C,which have to be positive. The upper bounds of Land R can be better defined by e.g. some physicalconsideration, but to do this further information haveto be assumed available. It is clear that in the realworld the parameter cannot be infinite. Thus, in orderto solve this case, the upper bound for the resistanceRU has been considered varying within the interval,[650,RUmax], the upper bound of the inductanceLU has been assumed varying within the interval[6x10−3,LUmax] and finally, the lower bound for thecapacitance CL has been assumed lying within theinterval [CLmin,7x10−6].

Classical ApproachWithin the different bounds the parameters are mod-elled as random variable having uniform parent dis-tribution, consistently with the maximum entropyPrincipe. Finally, all the combinations of upper andlower bounds have been analysed and failure proba-bilities computed via Monte Carlo simulation. In or-der to make this problem tractable using conventionalprobabilistic approaches, the bounds have been re-defined as m times its nominal value for the upper

Figure 15: P-box Vc(10ms), CASE-C

Figure 16: P-box Vc(8ms), CASE-C

Figure 17: P-box Z, CASE-C

bounds and m divided by its nominal value for thelower bound , with m=[1,10,100,1000]. Having re-duced the semi definite intervals to set of defined in-terval, it is now possible to estimate the reliability ofthe systems, adopting the same approach of case B.

Classical ResultsIt has been computed all the possible combinationsof the lower and upper bounds, in order to see howthe parameters interval combinations influence the re-sults. Some combinations show probability of failureequal to 0.104. But, on the other hand, other combina-tions present probability of failure that is equal 1. Thisis due to the low efficiency of the Monte Carlo methodwhich is not able to give a very accurate answer forsmall probability values. In order to accurately esti-mate the probability of failure in these cases, it wouldbe necessary to estimate the probability of failure us-ing either many more samples or by means of ad-vanced MC methods. However, applications of ad-vanced MC methods are out from the final purposeof this paper and have not been considered here.

Generalized ApproachThe uncertainty in the parameters has been charac-terized by partition of the three imprecise intervalsin n defined intervals, forming sets of n intervalsand given by the Equations (23)-(25). Finally, sameprobability mass function equal to 1/n has been as-signed to each interval (for normalization reasons)and Dempster-Shafter structures propagate similarlyto previous cases.

(([RL,RU1],

1

n), ..., ([RL,RUmax],

1

n)

)(23)(

([LL,LU1],1

n), ..., ([LL,LUmax],

1

n)

)(24)(

([CL1,CU ],1

n), ..., ([CLmin,CU ],

1

n)

)(25)

Where ith bound on the parameter (e.g. the resis-tance) is defined as:

RUi = RU1 + (i− 1)RUmax −RU1

n

Where RL=650Ω, LL=6mH and CU=7µF. Assumingdifferent values ofLUmax,RUmax andCLmin may leadto different results; hence the variations in the result-ing P-Boxes, due to variations the three threshold val-ues, have been investigated. The created Dempster-Shafer structures have been propagated through thesystem by intervals combination similarly to the othercases. Minimum and maximum values for Vc(8ms),Vc(10ms) and Z are computed and stored for each in-terval cell. The resulting Dempster-Shafer structuresare assembled and translated to Probability boxes forthe Voltages and damping factors.

Figure 18: P-box Vc(10ms), CASE-D

Figure 19: P-box Vc(8ms), CASE-D

Figure 20: P-box Z, CASE-D

Generalized ResultsFigures 18-20 show resulting P-Boxes forVc(8ms),Vc(10ms) and Z assuming RUmax=6500Ω,LUmax=60mH, CLmin=0.7µF and n=10. It can benoticed that probability of failure considering for thevoltage requirements lays within the interval [0,1]and for the third requirements on Z lay in the interval[0,0.99]. Thus, the uncertainty in the inputs definedin the CASE-D is too large to provide useful results.The probability bounds for CASE-D results slightlydifferent form the probability intervals for CASE-A(bounds equal to [0,1] for all the requirements), hencethe information compared to CASE B or CASE Ccan be seen having the highest amount of uncertaintyassociate and be of lower quality for the meetingrequirements prospective.Additionally, in order to investigate the effect of theassumptions on the results, the uncertainties havepropagated changing the values of RUmax, LUmax andCLmin. Results of this analysis show that RUmax isaffecting more the shape of the P-Boxes. On the otherhand, it has not highlighted any relevant changes inthe bounds of the failure probability.Comparing the results of probabilistic approaches andgeneralized approaches point out that the generalizedprobabilistic approaches can gave valuable prospec-tive on the final solutions providing evidence of thelow empirical quality of the available parameter data.

6 LIMITATION FACED IN APPLYING THEAPPROACHES

Classical probabilistic approaches required the defi-nition of distributions. The uniform distribution wasassumed to characterize parameter uncertainty inCASE-A, CASE-B and CASE-C, since it is a typicalassumption for model uncertainty if no informationbut bounds information is given. This assumption isoften regarded as a rational hypothesis that found the-oretical support in the Laplaces principle of indiffer-ence (or more in general principle of maximum en-tropy). In CASE-C first has been assumed a finitenumber of combination and a DOE, secondly it hasbeen considered just a pdf fitted using Gaussian Ker-nel Density estimation. Subjectively assuming proba-bility distribution functions can bear an underestima-tion of the uncertainty. As example consider the cu-mulative distribution function (CDF) of the uniformdistribution; the CDF will have a well-known linearlyincreasing shape between minimum and maximumvalues. If these minimum and maximum bounds val-ues are the only available data, the assumption canled to misleading results, especially in reliability as-sessments. Within an imprecise-information scenario(e.g. parent distribution not specified or unknown,or known but with vague parameters, conflicting andlimited knowledge, linguistic incomprehension, inter-vals etc.), it seems more conservative and robust forthe reliability assessment to take into account not just

one single CDF but all the plausible CDFs accord-ingly to the available data. Indeed, this approach iscomputationally expansive and will produce impre-cise results in reliability evaluation, but has the un-deniable advantage of not introducing artificial modelassumption.The analysed case study confirm that artificial modelassumptions might lead to uncertainty underestima-tion, hence reliability results that do not represent pre-cisely the real data’s quality. Another point to be high-lighted is the low efficiency of Monte Carlo methodto compute small probabilities of failure. For extremecases, where there are too few failures or too few suc-cesses, Monte Carlo is not efficient in producing accu-rate result. To have it more precise, it would be nec-essary to use another method of accuracy, e.g. LineSampling. In the proposed study a simple system hasbeen analysed, therefore computations were not timeexpensive. Drawback of the generalized approach isthat high computational effort may be required, espe-cially for the analysis of more complex systems.Nevertheless, results obtained by generalized ap-proaches provide valuable evidence about the qualityof the information available in all the tasks which, inthis case, result fairly low. The large epistemic uncer-tainty and lack of knowledge about the system param-eters may suggest to consider an investment on col-lecting more valuable empirical data rather than refinethe model for the reliability assessment. Overall out-comes of the study highlighted some of the positiveand negative aspects of embed generalized approachto classical uncertainty quantification methodologies.

7 CONCLUSIONS AND DISCUSSIONS

In this paper, the NAFEMS uncertainty quantificationchallenge problem has been tackled by use of clas-sical and generalized uncertainty quantification ap-proaches implemented in a common computationalframework. Dempster-Shafer structures, P-Boxes andKolmogorov-Smirnov tests are the tools used to tacklethe problem within the generalized approach. Resultshave been carried out using both the approaches andcomparison and discussion provided. Classical proba-bilistic methods provide results characterized by pre-cise estimation of the failure probabilities. Such pre-cise estimations are based on strong assumptions re-quired to characterise the uncertainties. By applyinggeneralized probabilistic methods, only bounds of thefailure probability can be obtained giving a lower pre-cision in the prediction of the system performance buton the concession for less restrictive assumptions inthe uncertainty characterization. Generalised proba-bilistic methods propagate the imprecise in the inputinto imprecision on the estimation, giving a clear in-dicator on the quality of the analysis. On the otherhands, classical probabilistic approaches are hidingsuch information providing a false sense of accuracy.

In fact, the results obtained using classical proba-

bilistic approaches are, as expected, included in thebounds obtained by generalized approaches. On theother hand, the values are not close to the upperbounds of the failure probability obtained by use ofgeneralized approach. Hence it can be stated that, inthe analysed case, the less reliable or worst case sce-nario reveals to be underestimated by classical ap-proach. Possible explanations are the strong initial as-sumptions used to define the underlying distributionfor the parameters (e.g. using uniform distribution).The results seem confirming that Dempster-Shafer ap-proach produces bounds that get narrower with betterempirical information. Moreover, Dampster-Shafferapproach seems particularly useful especially in reli-ability assessments where probabilistic characteriza-tions are desired and empirical information is limited.In conclusion, some of the limitation and shortcom-ing of the approaches have been highlighted, strengthof combine classical and generalized approaches havebeen outlined and the computational tool verified tobe flexible and effective. Generalized methods, al-though computational expensive, are generally appli-cable and they are able to provide essential informa-tion about the quality of the analysis, an unavoidabletool for the industry which may rely on more accurateinformation qualification approaches, understandingif the data is of high quality or poor quality aimingat designing safer and more reliable systems or com-ponents.

REFERENCES

Alex Diaz De La O, Edoardo Patelli, M. B. I. S. A. (2014). Un-certainty quantification and management through bayesianand/or imprecise probability perspective. In CST2014 TheTwelfth International Conference on Computational Struc-tures Technology CST2014-S08.

Augustin, T. (2004). Optimal decisions under complex uncer-tainty - basic notions and a general algorithm for data-baseddecision making with partial prior knowledge described byinterval probability. Special Issue of ZAMM - Zeitschrift frAngewandte Mathematik und Mechanik 84(10–11), 1–10.

Beer, M., S. Ferson, & V. Kreinovich (2013). Imprecise probabil-ities in engineering analyses. Mechanical Systems and SignalProcessing 37(1-2), 4–29.

Blair, A. N., B. M. Ayyub, & W. J. Bender (2001). Fuzzystochastic risk-based decision analysis with the mobile off-shore base as a case study. Marine Structures 14(12), 69 –88. Very Large Floating Structures (VLFS) Part II.

Ching, J. & Y.-C. Chen (2007). Transitional markov chain montecarlo method for bayesian model updating, model class se-lection, and model averaging. Journal of engineering me-chanics 133(7), 816–832.

Der Kiureghian, A. & O. Ditlevsen (2009). Aleatory or epis-temic? does it matter? Structural Safety 31(2), 105–112.

Dubois, D. & H. Prade (1988). Possibility Theory. New York:Plenum Press.

Faber, M. H. (2005). On the treatment of uncertainties and prob-abilities in engineering decision analysis. Journal of OffshoreMechanics and Arctic Engineering 127, 243–248.

Ferson, S., V. Kreinovich, L. Ginzburg, D. S. Myers, & K. Sentz(2002). Constructing probability boxes and Dempster-Shaferstructures, Volume 835. Sandia National Laboratories.

Helton, J., J. Johnson, & W. Oberkampf (2004). An explo-

ration of alternative approaches to the representation of un-certainty in model predictions. Reliability Engineering &System Safety 85(13), 39 – 71. Alternative Representationsof Epistemic Uncertainty.

Kiureghian, A. D. (2008). Analysis of structural reliability underparameter uncertainties. Probabilistic Engineering Mechan-ics 23(4), 351–358.

Laura P. Swiler, Thomas L. Paez, R. L. M. (2009). Epistemicuncertainty quantification tutorial. Proceedings of the IMAC-XXVII.

Marseguerra, M. & E. Zio (2002). Basics of the Monte CarloMethod with Application to System Reliability. Hagen, Ger-many: LiLoLe - Verlag GmbH (Publ. Co. Ltd.). ISBN 3-934447-06-6.

Massey, F. (1951). The kolmogorov-smirnov test for goodness offit. Journal of the American Statistical Association 46(253),6878.

NAFEMS (2013, June). Challenge problem retrieved from.World Congress Salzburg, Austria.

NAFEMS (2014). Bench mark challenge in uncertainty quan-tification. THE INTERNATIONAL MAGAZINE FOR ENGI-NEERING DESIGNERS & ANALYSTS FROM NAFEMS,40–43.

Patelli, E., D. A. Alvarez, M. Broggi, & M. de Angelis (2014).An integrated and efficient numerical framework for uncer-tainty quantification: application to the nasa langley multi-disciplinary uncertainty quantification challenge. 16th AIAANon-Deterministic Approaches Conference (SciTech 2014),2014–1501.

Patelli, E., M. Broggi, M. Angelis, & M. Beer (2014, June).Opencossan: An efficient open tool for dealing with epis-temic and aleatory uncertainties. In Vulnerability, Uncer-tainty, and Risk, pp. 2564–2573–. American Society of CivilEngineers.

Pedroni, N., E. Zio, E. Ferrario, A. Pasanisi, & M. Couplet(2013). Hierarchical propagation of probabilistic and non-probabilistic uncertainty in the parameters of a risk model.Computers & Structures 126(0), 199 – 213. UncertaintyQuantification in structural analysis and design: To com-memorate Professor Gerhart I. Schueller for his life-timecontribution in the area of computational stochastic mechan-ics.

Pradlwarter, H. & G. Schuller (2008). The use of kernel densi-ties and confidence intervals to cope with insufficient data invalidation experiments. Computer Methods in Applied Me-chanics and Engineering 197(2932), 2550 – 2560. ValidationChallenge Workshop.

Rocchetta, R., Y. Li, & E. Zio (2015). Risk assessment andrisk-cost optimization of distributed power generation sys-tems considering extreme weather conditions. Reliability En-gineering & System Safety 136(0), 47 – 61.

Shafer, G. (1976). A Mathematical Theory of Evidence. Prince-ton: Princeton University Press.

Silverman, B. W. (1986). Density estimation for statistics anddata analysis, Volume 26. CRC press.

Veneziano, D., A. Agarwal, & E. Karaca (2009). Decision mak-ing with epistemic uncertainty under safety constraints: Anapplication to seismic design. Probabilistic Engineering Me-chanics 24(3), 426 – 437.

Zio, E. & N. Pedroni (2013). Literature review of methods forrepresenting uncertainty, Volume 3. Foundation for an In-dustrial Safety Culture, Toulouse, France.

ON BAYESIAN APPROACHES FOR REAL-TIME CRACKDETECTION

R. Rocchetta & M. Broggi & E. Patelli & Quentin HuchetInstitute of Risk and Uncertainty, University of Liverpool, United Kingdom Tel.: 0044 (0) 151 7944079

ABSTRACT: Fatigue is the most dangerous failure mode for mechanical components subject to alternatingloads. Due to repeated loading and unloading, one or several cracks can be initiated and propagated through thecross section of the structure. Once a critical crack length is exceeded, the structure will catastrophically faileven for stress level much lower than the design stress limit. Non-destructive inspections may be performed atpredetermined time intervals in order to detect the cracks. Alternatively, a continuous monitoring of the dynamicresponse of the structure can allow real-time cracks detection and corrective maintenance procedures might betaken in case the monitoring procedure identifies a crack.In this paper, Bayesian model updating procedures is adopted for the detection of crack location and length on asuspension arm, normally used by automotive industry. Experimental data of the damaged structure (frequencyresponse function) are simulated using a high-fidelity numerical model of the arm. A second, coarse-modelrepresents the model to be updated where cracks of random locations and dimensions are introduced. The ideaunderlining the approach is to identify the most probable model consistent with the observations.The likelihood is the key mathematical formulation to include the experimental knowledge in the updating of theprobabilistic model. Different likelihood functions can be used based on different mathematical assumptions.In this work, the effect of different likelihood functions will be compared to verify the capability of Bayesianprocedure for system health monitoring. The different likelihoods will be categorized according to the accuracyof the results and the efficiency of the numerical procedure.

1 INTRODUCTION

Failure for mechanical components subject to alter-nating loads may occur in several different ways,and failures due to fatigue are one of the mostdangerous types. It is well known that cyclic loadscan initiate cracks which propagate through thecross section of the structures. Once a critical cracklength is exceeded, the structure will catastrophicallyand suddenly fail, even for stress level much lowerthan the design stress (Paris and Erdogan 1963). Inparticular, interactions may occur between the struc-tural responses and cracks in components subjectto high frequency dynamic excitations, leading tovibration-induced fatigue. Consequences may be apremature failure of the component or even worst theloss of the structures which rely on the componentintegrity.

Several strategies are accountable to prevent sud-den failures; for instance, non-destructive inspectionsmay be performed at predetermined time intervals,in order to detect the cracks (Faber et al. 1996);however failure can occur between inspections

(Beaurepaire et al. 2012). Alternatively, a continuousmonitoring of the dynamic response of the structurecan allow for real-time crack detection and for atimely intervention with maintenance procedures(Chang et al. 2003). Repair actions are taken in casethe monitoring procedure successfully identifies acrack which jeopardizes the structure.

In both cases, the procedure may fail in identifyinga crack, leading to fatigue failure. Thus, an efficientcrack detection procedure is required in order toavoid the loss of the structure. New emerging tech-niques are now available in the field of computationalmechanics, which can be employed to assist in themonitoring of the health of the structures. Thesetechniques modify some specific parameters in anumerical model to ensure a good agreement with thedata, a so-called inverse problem. A computationalframework well fitted for the solution of such inverseproblems is the model updating (Fritzen et al. 1988).

In previous work (Beaurepaire et al. 2013), someof the authors implemented an updating frameworkfor the detection of cracks in a suspension arm,

as normally used by automotive industry. The me-chanical behaviour of a device was characterized bycollecting Frequency Response Functions (FRF) dataat a specific location.

In this paper, the updating procedure for crackdetection has been further investigated and extended,aiming at better understand its limitations andstrengths. Different empirical likelihood expressionshave been proposed in order to fit the procedurewith different features of experimental evidence. Theempirical formulation are compared and discussedin two representative cases; first, the detection ofa single crack of known position and not-knownlength, secondly, the detection of a single crack ofnot-known position and not-known length.

Computational time is a an issue for real-time ap-plication of the procedure and it has been addressed.As a matter of fact, many model evaluations arerequired for the approach, thus a strategy for the par-allelization of the simulations is provided. Moreover,adopting surrogate models such as Artificial NeuralNetworks (ANN) and Poly-harmonic spline (PHS)the computational time has been further reduced.The general purpose software OpenCossan (Patelliet al. 2014) has been employed in all the stages of theprocedure.

The paper is structured as follows: Section 2 dealswith the modelling of fracture in a Finite Elementframework. Section 3 outlines the main concept ofBayesian model updating and the efficient simulationalgorithm employed in the particular case of structurewith cracks under dynamic excitation. A numericalexample and results are presented and discussed inSection 4, and likelihood expressions are comparedfor the two considered cases. Finally, conclusions andremarks are drawn in Section 7

2 MODELLING AND RELATEDBACKGROUND

Finite element (FE) analysis has become establishedas a powerful family of methods for the spatialapproximation of systems of partial differentialequations and variational problems. It has been usedin a multitude of areas in the engineering field, e.g. inthe analysis of mechanical components or structures.Nevertheless, the mechanical behaviour of structuresmay be altered if the elements are crossed by cracks.The cross section of the component is reduced, whichcauses a reduction of the stiffness. Moreover, thestress field is also modified in the vicinity of a crack.

Specific FE methods have been conceived tomodelling efficiently the mechanical behaviour ofstructures containing cracks. The extended finiteelements method (XFEM), first introduced by (Moes

et al. 1999), has received considerable attentionover the past few years. It consists of enriching theelements affected by a crack by introducing addi-tional shape functions, which increases the numberof degrees of freedom associated with the nodes. Thestress field in these elements is then expressed usinga combination of the standard and of the enrichmentshape functions.

In case an element is crossed by a crack, a Heav-iside function centred on the crack is introducedas an additional shape function. This step functionaccounts for the discontinuity of the displacementsbetween the two lips of the crack. In case an elementincludes the crack tip, the corresponding nodes of thefinite element model are enriched with specific shapefunctions. These functions correspond to the asymp-totic displacement field at the vicinity of a cracktip, which can be determined analytically. For moredetails about the enrichment of the tip elements see(Moes et al. 1999). This allows capturing efficientlythe displacement and strain fields near the crack tip,without excessive refinement of the mesh.

Details on the XFEM as implemented in theanalysis here presented may be found in the works Ziand Belytschko 2003 and Abdelaziz and Hamouine2008. However, mesh refinement in the vicinity of thecrack tip may be necessary when the extended finiteelements method is used, in spite of the enrichmentof the nodes at the crack tip (Geniaut 2011). Nev-ertheless, the mesh does not have to be compatiblewith the crack, which considerably simplifies there-meshing.

In case the behaviour of a cracked structure underdynamic excitation needs to be determined, the stiff-ness matrix may be computed using the XFEM, asstated above. The mass matrix is not modified by thepresence of cracks, and no special action needs to betaken. The problem is subsequently solved using thestandard procedure for linear dynamics: the modesand frequency of vibration are determined by solvingthe eigen-value problem associated with the mass andstiffness matrices; and the FRF associated with anynode of the finite element model are determined.

3 MODEL UPDATING PROCEDURE FORCRACK DETECTION

3.1 Bayesian updating of structural models

A Bayesian model updating procedure is based on thevery well-known Bayes theorem (Bayes 1763). Thegeneral formulation is the following:

P (θ|D,I) =P (D|θ, I)P (θ|I)

P (D|I)(1)

where θ represents any hypothesis to be tested, e.g.,the value of the model parameters, D is the availabledata or observations, and I is the background infor-mation. Main terms can be identified in the Bayes the-orem:

• P (D|θ, I) is the likelihood function of the dataD;

• P (θ|I) is the prior probability density function(PDF) of the parameters;

• P (θ|D,I) is the posterior PDF;

• P (D|I) is a normalization factor ensuring thatthe posterior PDF integrates to 1;

The theorem introduces a way to update somea-priori knowledge on the parameters θ, by usingdata or observations D and conditional to someavailable information or hypothesis I .

Bayes law has been successfully applied in theupdating of structural models see (Beck and Katafy-giotis 1998) and (Katafygiotis and Beck 1998); inparticular the Bayesian structural model updating hasbeen successfully used to update large finite elementmodels using experimental modal data (Goller et al.2011). In a structural model updating framework,the initial knowledge about the unknown adjustableparameters, e.g. from prior expertise, is expressedthrough the prior PDF. A proper prior distributioncan be a uniform distribution in the case when only alower and upper bound of the parameter is known, ora Gaussian distribution when the mean and a relativeerror of the parameter is known.

The likelihood function gives a measure of theagreement between the available experimental dataand the corresponding numerical model output.Particular care has to be taken in the definition of thelikelihood, and the choice of likelihood depends onthe type of data available, e.g. whether the data is ascalar or a vector quantity. Different likelihood leadsto different accuracy and efficiency in the results ofthe updating procedure and should be selected withcaution; as an example, the use of unsuitable likeli-hood function might cause that the model updatingdo not produce any relevant variation in the prior.

Finally, the posterior distribution expresses the up-dated knowledge about the parameters, providing in-formation on which parameter ranges are more proba-ble based on the initial knowledge and the experimen-tal data.

3.2 Transitional Markov-Chain Monte-Carlo

The Bayesian updating expressed in equation 1 needsa normalizing factor, that can be very complex to

obtain and computationally expensive. An effectivestochastic simulation algorithm, called TransitionalMarkov Chain Monte-Carlo (Ching and Chen 2007),has been used in this analysis. This algorithm allowsthe generation of samples from the complex shapedunknown posterior distribution through an iterativeapproach. In this algorithm, m intermediate distribu-tions Pi are introduced:

Pi ∝ P (D|θ, I)βi P (θ|I) (2)

where the contribution of the likelihoodis scaled down by an exponent βi, with0 = β0 < .. < βi < .. < βm = 1, thus the firstdistribution is the prior PDF, and the last is the poste-rior. The value of these exponents βi is automaticallyselected to ensure that the dispersion of the samplesat each step meet a prescribed target. For additionalinformation the reader is reminded to (Ching andChen 2007). These intermediate distributions showa more gradual change in the shape from one stepto the next when compared with the shape variationfrom the prior to the posterior.

In the first step, samples are generated from theprior PDF using direct Monte-Carlo. Then, samplefrom the Pi+1 distribution are generated using Markovchains with the Metropolis-Hasting algorithm (Hast-ings 1970), starting from selected samples taken fromthe Pi distribution, and βi is updated. This step is re-peated until the distribution characterized by βi = 1 isreached. By using the Metropolis-Hasting algorithm,samples are generated from the posterior PDF with-out the necessity of ever computing the normalizationconstant. By employing intermediate distributions, itis easier for the updating procedure to generate sam-ples also from posterior showing very complex dis-tribution, e.g., very peaked around a mean value orshowing a multi-modal behaviour.

3.3 Model updating for crack detection andlikelihood expression

In the FE model, the cracks are modelled usingXFEM. Experimental data from the reference struc-ture are taken into account in the form of FrequencyResponse Functions. Knowing that cracks willdevelop most probably in certain locations, charac-terized by high concentration of stresses, cracks areinserted in these specific positions assuming theirlengths are random parameters.

Within the model updating framework, the crackspresent in the damaged structure are seen as uncer-tain model properties. The prior will use uniformdistribution for the crack parameters, allowing thepossibility of crack in any stress concentrationpoint and with any possible physically acceptablelength, i.e. compatible with geometric constraints and

material proprieties.

The likelihood is the key mathematical componentof any Bayesian updating procedure. Within the pro-posed crack detection framework, synthetic experi-mental FRFs are compared with the numerical FRF ofthe numerical model. Within the case study, three em-pirical likelihood formulations are proposed and usedto compare the experimental data with the numericalinformation. Expressions are discussed on the basis ofthe accuracy of the results, i.e. the ability in detectingtrue cracks positions and lengths.The likelihoods have been be expressed as:

P (D|θ, I) =Ne∏k=1

P (xek;θ) (3)

or, equivalently, in the form of the log-likelihood:

P (D|θ, I) =Ne∑k=1

log(P (xek;θ)) (4)

where xek represent the kth experimental evidence,Ne is the number of available experimental data andθ is the vector of random crack lengths. The termP (xek;θ) which include the experimental evidencehave been assumed as exponentially distributed.Three heuristic formulations have been defined as fol-lows:

P (xek;θ) ∝ exp

(δk

MSEpks[h(θ)− hke ]

)(5)

P (xek;θ) ∝ exp

(δk

MSElow[h(θ)− hke ]

)(6)

P (xek;θ) ∝ exp

(δk

MSEall[h(θ)− hke ]

)(7)

where hke is the experimental FRF k, h(θ) repre-sents the numerical FRF, δk is the variance of theMeans Square Errors (MSE) for the experiment k.The Means Square Errors have been obtained consid-ering different frequency ranges, e.g. MSEall is com-puted between experimental FRF and numerical FRFover the entire frequency domain, MSElow is com-puted considering lower frequencies and MSEpks isobtained around the main resonance peaks. The pro-posed expressions and the selection of the frequencyranges were defined on empirical basis therefore de-tails will be discussed in the case study, Section 4.After the updating procedure, the posterior distribu-tions provide a qualitative indication of the cracklength and positions, i.e. it will concentrate around theunknown length and position of the crack with mostsimilar response if compared to the experimental ob-servations.

It worth remark that even if the procedure in gener-ally applicable to detect cracks, the component anal-ysed have a specific FRF, which therefore limit theconsideration on the validity of the defined likeli-hoods to the specific mechanical device in exam.

4 NUMERICAL EXAMPLES

This numerical example is similar to those used inthe automotive industry (Mrzyglod and Zielinski2006) and it has been recently applied (Beaurepaireet al. 2013) to detect cracks in a suspension arm,shown in Figure 1. It can freely rotate along the axisindicated by the dashed line; the suspension springand the wheel structure are connected at the locationindicated by “S”. The stress concentration points, andcandidate crack locations, are indicated in the figureby the numbers 1 to 6.

In this case study, “simulated” experimental dataare generated using a Finite Element (FE) model. Thesoftware used to construct the model and in the anal-ysis is Code Aster (Geniaut 2011). A crack with fixedlength is inserted in one of the candidate position, andthe reference FRF is computed at the position indi-cated by “O”. Both the FRF in direction X and Y areconsidered, while no FRF is obtained in the directionZ since the structure is not constrained in that direc-tion. Figure 2 display the FRFs in the directions X andY when a 5 mm length crack is considered. Just threeout of six possible position are shown for graphicalreasons.

The crack lengths are considered as uncertainparameters, and are modelled using uniformlydistributed random variables. Since the crack is phys-

Figure 1: The suspension arm FE model with indi-cated the six possible crack positions and the FRFmeasurement point.

Figure 2: Frequency response functions of the highfidelity FE model. A single crack of 5mm length isinserted in each of the stress concentration points.

ically constrained to not touch the flanges of the arm,a maximum crack length of 5 mm is assigned to thecracks in position 1 and 2, while the length is limitedto 10 mm for the cracks in positions 3 to 6. Thesampled values of the random variables are insertedinto the FE model by using the ASCII file injectionroutine provided by OpenCossan (Patelli et al. 2014),providing a deterministic FRF. Additionally, thesimulations run in parallel on a computer cluster andsurrogate model are used, allowing further reductionof the overall computational time.

The goodness of the procedure in detecting crackshas been tested for two updating cases:

• detection of single crack having known positionand unknown length

• detection of a single crack of unknown positionand length.

In both the cases analysis have been carried out byusing empirical likelihood expressions. Results com-pared and discussed point out positive and negativefeatures of the different formulations.

The likelihoods expressions and frequency rangeshave been selected after considerations on the com-putational inaccuracy affecting numerical FRFs,especially in the high frequencies domain. It canbe argued that likelihoods defined as in Equations5 - 6 might detect cracks with higher precision ifcompared to Equation 7 which computes MSEincluding FRF values in the high frequency range.MSElow in Equation 6 has been computed over thelow frequency range from 0 Hz to 1.7x104 Hz whileMSEall is computed over the entire domain (from0 Hz to 5x104 Hz). Furthermore, it has been noticedthat the FRFs around the main resonance peaks (e.g.around 2.4x104 Hz in Fig. 2) appear to be particularlyvariable with respect to the crack positions and

lengths. Hence, MSEpks has been computed aroundthe main resonance peaks (2.4x104 Hz) to verify ifit is a good indicator to distinguish different cracklengths and locations.

4.1 Surrogate model calibration and selection

The Bayesian model updating procedure is computa-tionally expensive, thus surrogate models have beenadopted to reduce the computational time. Amongthe different surrogate models, Artificial NeuralNetworks (ANN) and Poly-harmonic Spline (PHS)have been tested and the best model has been selected.

The classical architecture type for Artificial NeuralNetwork (Chojaczyk et al. 2015) consists of oneinput layer, one or more hidden layers and one outputlayer. Each layer employs several neurons and eachneuron in a layer is connected to the neurons in theadjacent layer with different weights. Poly-harmonicSpline is popular tool for model interpolation andhas been considered as alternative surrogate modelto be compared with the ANN due to their goodproprieties, see e.g. in (Madych and Nelson 1990).

The coefficient of determination R2 is often usedto evaluate the quality of the regression model andhave been adopted to choose the most suitable meta-model to use in the procedure. This coefficient can beexpressed as follows:

R2 = 1−∑

i(yi − yi)2∑i(yi − y)2

(8)

where yi is the ith mean square error, yi is the meansquare error predicted by the surrogate model, y isthe average of the mean square errors from the FEanalysis.It goes without saying that R2 values have to be fairlycompared between the two considered surrogates,therefore the coefficient have to be compared onthe validation set. Otherwise the R2 for the PHS,computed using the calibration set, will result bydefinition equal to 1.

Input data for the surrogate models is the vector θof simulated cracks, outputs are MSE between thesimulated FRFs and a preselected synthetic experi-mental FRF. Both direction X and Y are considered,hence MSE in X and Y directions are the outputs ofthe surrogate models. In order to reduce complexityand limit the number of outputs, the entire FRF hasnot been considered as output for the surrogatesmodel.The proposed ANN layout consists of hidden layerwith 11 nodes for the first two layers and 13 forthe last layer. The described architecture have been

(a) PHS (b) ANNFigure 3: Regression plots of meta-models for a single crack with fixed known position.

(a) PHS (b) ANNFigure 4: Regression plots of meta-models for a single crack of unknown position and length.

selected based on precedent results and in orderto capture the highly non-linear behaviours of theoutputs. Cubic PHS has been considered for modelthe behaviour of the MSE in x and y directions.Themodels have both been calibrated using 90% of thedata set and validated using the remaining 10%.Examples of results are proposed in Figures 3-4where mean square errors between experimental andsimulated FRFs in X direction are considered. Thetargets, in the X-axis, are MSE obtained using the FEmodel and the outputs, in the Y-axis, represent theoutput of the meta-model. The sub-plots in the top ofthe figures display results for the model calibration,while in the bottom are displayed results for thevalidation set. Experimental cracks is considered inposition 6 of length 8.07 mm.

In the first case 700 single crack lengths withknown position (crack in position 6 in Fig. 1) havebeen considered. Figures 3a-3b present linear regres-sion results for the first analysed case. Similarly, Fig-

ures 4a-4b shows linear regression plots and compar-isons for the second analysed case. In the second case,2800 single cracks of random length with not-knownposition are the input for the surrogate model.The R2 values for the validation data show that PHSis the best candidate surrogate model for both theconsidered detection cases. Hence, PHS appear tobe more suitable than ANN to mimic the MSEbehaviour. This can be explained if considered the“step” behaviour of the outputs along the cracklength. In order to reproduce this non-linear be-haviour, an ANN need to be built with several nodesand several hidden layers; on the other hand, a PHSworks well in capturing it because it is obliged to passthrough the support points and better predict non-linearly in-between the supports.

4.1.1 Single crack, known positionBayesian updating procedure has been used first todetect of single crack length which has a knownposition (position 6 in Fig. 1). The procedure starts

by first selecting prior distribution for the cracklength P (θ|I) ∼ U [0,10] and sample number equalto 1500. Number of samples has been selected basedon previous works, however future optimization ofthe initial setting might be considered to reduce thecomputational time. Three different synthetic exper-imental FRFs have been selected, corresponding tocracks of “short” length (2.4 mm) “medium” length(4.53 mm) and “long” length (8.07 mm). Updatingprocedure has been repeated to include the empiricallikelihoods defined in Section 3.3. Results have beenqualitatively ranked based on the accuracy of theposterior distributions; suitability of the likelihoodsis discussed.

Figure 5: Posterior distibution of “short” crack lengthin position 6, different likelihoods expressions.

In Figure 5 shows posterior distributions fordifferent likelihoods and for a crack lengths to bedetected equal to 2.4 mm. The posterior distributionsresult peaked around the true crack length. The cracklength is detected with high precision in this case;mean values for the distribution are almost equalto the true crack lengths 2.4 mm plotted in dottedline. Among the three analysed likelihoods, the onecomputed using Equation 5 result in a lower variancecompared to the others, as can be qualitativelyobserved in the top sub-plot of Figure 5. Similarbehaviour is observable for the other selected cracklengths.

Nevertheless, considered the final aim of the up-dating, the differences are of minor relevance. Morespecifically, in this stage of comparison which is in-tended to be mainly qualitative, detecting a crack withuncertainty ±0.06 mm or ±0.04 mm has been con-sidered a minor difference. The crack length is de-tected with high precision in case of single crackwith known position, and all the likelihood expres-sions seems well-suited to be used within the detec-tion framework.

4.1.2 Single crack, unknown positionThe procedure presented in 4.1.1 has been extendedfor detect both the length and position of a crack.The procedure, has been tested considering like-lihoods described in Section 3.3, and assuminguniform prior distributions for cracks in position 3to 6 (P (θ|I) ∼ U [0,10]) and in position 1 and 2(P (θ|I) ∼ U [0,5]). The synthetic experimental FRFis the one of a crack in position six of length 8.07mm.Figures 6, 7 and 8 display the posterior distributionsobtained by using likelihoods computed as in equa-tions 5, 6 and 7 respectively. Results indicate correctdetection of the crack within the true length rangeand the true position.

Computing likelihood considering main resonancefrequency, as in Figure 6 crack is detected within thelength interval [7-10 mm]. Similar results, displayedin Figures 7 and 8, are obtained by using differentlikelihood expressions. The results show higher un-certainty if compared to the results in Section 4.1.1.The posterior distributions for cracks in positionsfrom 1 to 5 result similar to the assumed prioruniform distributions. This mean that none of thesimulated cracks length in positions form 1 to 5can be fairly associated to the experimental evi-dence provided. However, results obtained by usingEquation 6 wrongly detect a crack in position 4 ofapproximate length of 5 mm. Explanation mightbe found considering the FRF behaviour in the lowfrequencies range. The updating procedure may haverevealed similarity between experimental data of“long” crack in position 6 and simulated FRF forcracks of mean length in position 4.

5 NOISE ANALYSIS AND UPDATING

In order to increase adherence to reality, noises havebeen added to the synthetic experimental FRF (simu-lated). Signal-to-noise ratio (SNR) can be defined asthe ratio between the power of the signal and power ofthe noise affecting the signal; input SNRs lower than20 dB are typically encountered at the resonance fre-quency of lightly damped vibrating mechanical struc-tures.The noise function has been added to the FRF as ex-plained in equations 9-10.

˙FRFx = FRFx +Nx (9)

˙FRFy = FRFy +Ny (10)

Where Nx and Ny are noises in the X and Yaxis directions respectively. General case of corre-lated noises in the X and Y directions has been con-sidered, consistently with the fact that noises in differ-ent directions are often generated by common sources

Figure 6: Likelihood computed with MSEpks.

Figure 7: Likelihood computed with MSElow

Figure 8: Likelihood computed with MSEall

(e.g. noise on the signal input, external environment,etc.). The goodness of the detection has been testedif three predefined level of SNR are added to the ref-erence FRFs. The SNR have been set equal to 100dB (almost negligible noise), 30 dB and 10 dB (fairlynoisy). An indicative plot of FRF in x direction addingdifferent noise levels is shown in figures 9-10.

Figure 9: ˙FRFx with added noise equal to 30 dBSNR.

Figure 10: ˙FRFx with added noise equal to 10 dBSNR.

5.1 Results of Updating including noises

Figure 11 displays Kernel-Density fitted posteriordistributions. The results have been obtained usingthree synthetic experimental crack lengths in position6 (short,medium and long lengths) and by addingnoise as previously indicated in section 6. Theposterior distributions obtained for the other possiblecrack locations (one to five) result approximativelyuniform, therefore not displayed for synthesis rea-sons.

As expected, the accuracy in the detection deterio-rates if SNR decrease (noise intensity increase). It canbe observed that for SNE equal to 100 dB, posteriordistribution is peaked around the true crack length,while increasing the noises to ratio to 30 dB lowerand upper bounds of the pdf result less narrow. Fi-nally, adding a fairly intense noise, SNR is set equalto 10dB, makes the updating procedure fail in identi-fying the crack (uniform distribution as posteriors).

6 ALEATORY UNCERTAINTY ON THEYOUNG’S MODULUS

In order to better understand the role played byaleatory uncertainty in the updating procedure intrin-sic randomness has been considered in one of the sus-pension arm parameters. The Youngs modulus E hasbeen considered as the aleatory parameter, assumednormally distributed around a known mean value (ini-tial set of the model) with a standard deviation equal

Figure 11: Posterior distribution results three noise level and three crack lengths in position 6.

to 3% of the mean value. Number 500 parallelizedMonte Carlo (MC) runs, coupled with the FE solverhave been performed and as many FRFs in directionsX and Y stored. The suspension arm has been consid-ered as a perfectly homogeneous with respect to E,hence each element of the FE model share the samesampled E within the single MC run. The FRF resultsfor sampled normal random E modulus are shown infigure 12.

Figure 12: Variablity of the frequency respnce fucn-tion with E.

As can be noticed, there is an high variability inthe FRFs profile due to uncertainty in the E modulus,even if some regularity are observable i.e. FRF shapesseems to be very similar. It is worth highlight that thespectrum of uncertainty associated to the aleatory un-certainty in the E modulus is fairly large and togetherto epistemic uncertainty due to lack of knowledgeabout the true crack parameters makes the updatingprocedure more challenging. Nevertheless, particularpattern in the FRF peaks and improved comparisonestimator between FRFs may be used to tune the up-

dating and improve the robustness of the detection ,which can be seen as future research direction.

7 REMARKS AND CONCLUSIONS

A Bayesian model updating procedure for cracks de-tection has been presented and it has been appliedto detect cracks in a suspension arm. Reference dy-namic data from vibration analysis was used as tar-get for the updating. The effects of different like-lihood expressions and different experimental dataon the crack detection strategy have been analysed.The procedure has been tested first to detect a sin-gle crack with unknown length but known position,the result comparison did not suggest major differ-ences between the likelihood formulas. Nevertheless,the second case point out the limitations of some ofthem. It is possibly due to similarity in the FRF fordifferent cracks or shortcoming in the computationalaccuracy. In both the analysed cases, the crack wasdetect correctly around the true length and position.Aleatory uncertainty have been added to the FRF un-der the form of noises of different intensity, Further-more, randomness in one of the suspension arm pa-rameter has been accounted. Preliminary results show,as expected, reduced precision of the detection if highnoises are accounted for. Moreover, uncertainty in theYoung’s modulus shows relatively high variability ofthe FRF results, which can be seen as a further is-sue for an effective model updating. Future develop-ment and additional research will be taken by usingreal experimental data to further validate and expandthe proposed approach, advanced noise filtering tech-niques to reduce rumours and more efficient indicatorfor the FRF comparison are going to be considered.

REFERENCES

Abdelaziz, Y. & A. Hamouine (2008). A survey of the extendedfinite element. Computers & Structures 86(11-12), 1141–

1151.Bayes, T. (1763). An essay towards solving a problem in the

doctrine of chances. Philosophical Transactions of the RoyalSociety of London 53, 370–418.

Beaurepaire, P., E. Patelli, & M. Broggi (2013). A bayesianmodel updating procedure for dynamic health monitoring. InCOMPDYN 2013 4th ECCOMAS Thematic Conference onComputational Methods in Structural Dynamics and Earth-quake Engineering.

Beaurepaire, P., M. Valdebenito, G. Schueller, & H. Jensen(2012). Reliability-based optimization of maintenancescheduling of mechanical components under fatigue.CMAME 221-222, 24–40.

Beck, J. L. & L. S. Katafygiotis (1998). Updating models andtheir uncertainties. i: Bayesian statistical framework. Journalof Engineering Mechanics, ASCE 124(4), 455–461.

Chang, P., A. Flatau, & S. Liu (2003). Review paper: healthmonitoring of civil infrastructure. Structural Health Moni-toring 3.

Ching, J. & Y.-C. Chen (2007). Transitional markov chain montecarlo method for bayesian model updating, model class se-lection, and model averaging. Journal of engineering me-chanics 133(7), 816–832.

Chojaczyk, A., A. Teixeira, L. Neves, J. Cardoso, & C. G. Soares(2015). Review and application of artificial neural networksmodels in reliability analysis of steel structures. StructuralSafety 52, Part A, 78 – 89.

Faber, M., I. Kroon, & J. Srensen (1996). Sensitivities in struc-tural maintenance planning. Reliability Engineering & Sys-tem Safety 51(3), 317 – 329. Maintenance and reliability.

Fritzen, C., D. Jennein, & T. Kiefer (1988). Damage detectionbased on model updating methods. Mechanical Systems andSignal Processing 12, 163 186.

Geniaut, S. (2011). Code aster, notice d’utilisation de la meth-ode x-fem (in french). technical report u2.05.02. Technicalreport, EDF-R&D.

Goller, B., M. Broggi, A. Calvi, & G. Schueller (2011). Astochastic model updating technique for complex aerospacestructures. Finite Elements in Analysis and Design 47(7),739–752.

Hastings, W. (1970). Monte carlo sampling methods usingmarkov chains and their applications. Biometrika 82, 711–732.

Katafygiotis, L. S. & J. L. Beck (1998). Updating models andtheir uncertainties ii: Model identifiability. Journal of Engi-neering Mechanics, ASCE 124(4), 463–467.

Madych, W. & S. Nelson (1990). Polyharmonic cardinal splines.Journal of Approximation Theory 60(2), 141 – 156.

Moes, N., J. Dolbow, & T. Belytschko (1999). A finite elementmethod for crack growth without remeshing. Int. J. Numer.Meth. Engng 46, 131–150.

Mrzyglod, M. & A. P. Zielinski (2006). Numerical implemen-tation of multiaxial high-cycle fatigue criterion to structuraloptimization. Journal of Theoretical and Applied Mechan-ics 44(3), 691–712.

Paris, P. & F. Erdogan (1963). A critical analysis of crack prop-agation laws. J. Basic Eng., Trans. ASME 85, 528–534.

Patelli, E., M. Broggi, M. de Angelis, & M. Beer (2014). Open-cossan: an efficient open tool for dealing with epistemic andaleatory uncertainties. Vulnerability, Uncertainty, and Risk:Analysis, Modeling, and Management, 2564–2573.

Zi, G. & T. Belytschko (2003). New crack-tip elements for xfemand applications to cohesive cracks. International Journal forNumerical Methods in Engineering 57(15), 2221–2240.