Struct Multidisc Optim (2011) 44:299–317DOI 10.1007/s00158-011-0659-2

RESEARCH PAPER

Sampling-based RBDO using the stochastic sensitivityanalysis and Dynamic Kriging method

Ikjin Lee · K. K. Choi · Liang Zhao

Received: 2 October 2010 / Revised: 4 April 2011 / Accepted: 11 April 2011 / Published online: 31 May 2011c© Springer-Verlag 2011

Abstract This paper presents a sampling-based RBDOmethod using surrogate models. The Dynamic Kriging(D-Kriging) method is used for surrogate models, anda stochastic sensitivity analysis is introduced to computethe sensitivities of probabilistic constraints with respectto independent or correlated random variables. For thesampling-based RBDO, which requires Monte Carlo sim-ulation (MCS) to evaluate the probabilistic constraints andstochastic sensitivities, this paper proposes new efficiencyand accuracy strategies such as a hyper-spherical local win-dow for surrogate model generation, sample reuse, localwindow enlargement, filtering of constraints, and an adap-tive initial point for the pattern search. To further improvecomputational efficiency of the sampling-based RBDOmethod for large-scale engineering problems, parallel com-puting is proposed as well. Once the D-Kriging accuratelyapproximates the responses, there is no further approxima-tion in the estimation of the probabilistic constraints andstochastic sensitivities, and thus the sampling-based RBDOcan yield very accurate optimum design. In addition, newlyproposed efficiency strategies as well as parallel computinghelp find the optimum design very efficiently. Numericalexamples verify that the proposed sampling-based RBDO

I. Lee · K. K. Choi (B) · L. ZhaoDepartment of Mechanical and Industrial Engineering,College of Engineering, The University of Iowa,Iowa City, IA 52242, USAe-mail: kkchoi@engineering.uiowa.edu

I. Leee-mail: ilee@engineering.uiowa.edu

L. Zhaoe-mail: liazhao@engineering.uiowa.edu

can find the optimum design more accurately than someexisting methods. Also, the proposed method can find theoptimum design more efficiently than some existing meth-ods for low dimensional problems, and as efficient as someexisting methods for high dimensional problems when theparallel computing is utilized.

Keywords Sampling-based RBDO · Surrogate model ·Stochastic sensitivity analyses · Score functions ·Monte Carlo simulation · Copula

Nomenclature

d Design vectorP[·] Probability measure�F Failure domainG(X) Performance or constraint functiony True function value at sample pointsF Matrix of basis function evaluated at sample pointsβ Regression coefficientse Stochastic process in Kriging modelR Covariance matrix in Kriging modelw0 Weight for predictor in Kriging modelθ Correlation parameterσ 2

p Variance of the surrogate model prediction at thepoint of interest

λ Lagrange multiplierd(x) Prediction interval bandwidth at point xy(x) Prediction of response at point xζ (θ) Negative logarithm of likelihood function �i

C , c Copula function and copula density functionu,v Marginal CDFs for Xi and X j , respectivelyβ t Target reliability index

300 I. Lee et al.

1 Introduction

Reliability-based design optimization (RBDO) has beenwidely applied to various engineering applications such asstamping (Yi et al. 2007; Youn et al. 2005a), vehicle designwith durability (Youn et al. 2004, 2005b), noise, vibra-tion, harshness (NVH) analysis (Dong et al. 2005, 2007),and structural analysis (Kim et al. 2006; Acar and Haftka2007; Lee et al. 2009; Yu and Du 2006), where accuratesensitivities of performance functions are available. If accu-rate sensitivities are available in a complex physical system,then the most probable point (MPP)-based reliability anal-ysis, which includes the First-Order Reliability Method(FORM) (Haldar and Mahadevan 2000; Madsen et al. 1986;Hasofer and Lind 1974; Ditlevsen and Madsen 1996), theSecond-Order Reliability Method (SORM) (Hohenbichlerand Rackwitz 1988; Breitung 1984), and the MPP-basedDimension Reduction Method (DRM) (Rahman and Wei2006; Wei 2006; Lee et al. 2008), can be used for approxi-mately assessing the reliability of the system, which is usedas a probabilistic constraint in RBDO.

However, for engineering applications where accuratesensitivities of performance functions are not readily avail-able, such as advanced and hybrid powertrain, robotic sys-tems, multi-physics and multi-scale problems, wind powersystems, micro- or nano-mechanics, and fluid-structureinteraction, etc., the MPP-based reliability analysis, whichuses the sensitivities of performance functions to find theMPP, cannot be readily used. Instead, surrogate models havebeen widely used to carry out design optimization for theengineering applications where sensitivities are unavailable(Youn and Choi 2004; Zhang et al. 2006; Kim and Choi2008; Simpson et al. 2001b; Queipo et al. 2005; Buranathitiet al. 2004; Gu et al. 2001). Once an accurate surrogatemodel is available for the design optimization, the MonteCarlo simulation (MCS) (Rubinstein 1981) can be directlyapplied to the accurate surrogate model to estimate the relia-bility of the system without intensive computational burden.To use the reliability of the system for the design optimiza-tion, its sensitivities are still required. Even if the surrogatemodel is accurate for the response value, the sensitivitiesobtained from the surrogate model and the finite differencemethod may not be accurate for a noisy response (Chanet al. 2007), and accordingly it is not a good idea to use themfor the design optimization. In addition, the FDM requiresdetermining appropriate perturbation size to accurately cal-culate the sensitivities (Chan et al. 2007); thus, the FDMmay yield wrong sensitivity estimation if the perturbationsize is not appropriate and the response is very noisy.

The main objective of this paper is to propose a newsampling-based RBDO, which uses the D-Kriging method(Zhao et al. 2010) to generate accurate surrogate models,and stochastic sensitivity analysis using the score function

(Lee et al. 2011) to derive the sensitivities of the reliabil-ity of the system with correlated random variables (Nohet al. 2009a, b, 2010). In addition, several numerical strate-gies are proposed in the paper to enhance the efficiencyand accuracy of the sampling-based RBDO; including ahyper-spherical local window for surrogate model genera-tion, sampling strategy, filtering of constraints, sample reuseand local window enlargement, and an adaptive initial pointof correlation parameter for the pattern search.

Section 2 briefly reviews the formulation of RBDO,D-Kriging, and stochastic sensitivity analysis using thescore function. Section 3 proposes numerical strategies foraccurate and efficient sampling-based RBDO. Section 4discusses the parallel computing for the sampling-basedRBDO to enhance the computational efficiency. Section 5uses numerical examples to illustrate how the proposedsampling-based RBDO works in terms of efficiency andaccuracy. Section 6 concludes the current study and dis-cusses future research directions.

2 Sampling-based RBDO

This section explains how to carry out RBDO of engineeringapplications where accurate sensitivities are not available.Before proposing the sampling-based RBDO, the formula-tion of RBDO is first briefly explained in Section 2.1. Then,Section 2.2 shows how to approximate true responses usingsurrogate models generated from the D-Kriging method,and Section 2.3 explains how to compute sensitivities ofprobabilistic constraints from the surrogate models gener-ated by the D-Kriging method without using sensitivities oftrue performance functions and even sensitivities of surro-gate models. Section 2.4 shows how to calculate probabilityof failure and its sensitivity by applying the MCS to the sur-rogate models.

2.1 RBDO Formulation

The mathematical formulation of a general component levelRBDO problem is expressed as

minimize Cost(d)

subject to P[G j (X) > 0

] ≤ PTarFj

, j = 1, · · · , nc

dL ≤ d ≤ dU, d ∈ Rnd and X ∈ Rnr (1)

where d = {di }T = μ(Xrv), i = 1 ∼ nd is the designvector, which is the mean value of the nd-dimensional ran-dom variable vector Xrv = {X1, X2, · · · , Xnd}T; X ={Xrv, Xrp}T, where Xrv and Xrp stand for the random vari-able and random parameter components of the random inputX, respectively; PTar

Fjis the target probability of failure

Sampling-based RBDO using the stochastic sensitivity analysis and Dynamic Kriging method 301

for the j th constraint; and nc, nd, and nr are the numberof probabilistic constraints, design variables, and randomvariables plus parameters, respectively.

A reliability analysis for both the component and sys-tem levels involves calculation of the probability of failure,denoted by PF , which is defined using a multi-dimensionalintegral as

PF (ψ) ≡ P[X ∈ �F ]=

∫Rnr

I�F (x) fX(x; ψ)dx = E[I�F (X)

](2)

where ψ is a vector of distribution parameters, which usu-ally includes the mean (μ) and/or standard deviation (σ )of the random input X = {X1, · · · , Xnr }T; P [·] repre-sents a probability measure; �F is the failure set; fX(x; ψ)

is a joint probability density function (PDF) of X; andE [·] represents the expectation operator. The failure set isdefined as �Fj ≡ {

x : G j (x) > 0}

for component reliabil-ity analysis of the j th constraint function G j (x), and �F ≡{

x : ⋃ncj=1 G j (x) > 0

}and �F ≡

{x : ⋂nc

j=1 G j (x) > 0}

for the series system and parallel system reliability analysisof nc performance functions, respectively (Rahman 2009;McDonald and Mahadevan 2008; Lee et al. 2010). I�F (x)

in Eq. (2) is called an indicator function and defined as

I�F (x) ≡{

1, x ∈ �F

0, otherwise(3)

In this paper, since the mean of X, μ = {μ1, · · · , μnd}T isused as a design vector, the vector of distribution parame-ters ψ is simply replaced with μ for the computation of theprobability of failure in Eq. (2).

To compute the probability of failure in Eq. (2), statis-tical sampling such as the MCS at a given design needsto be applied to true responses, which is computationallyvery expensive and almost prohibited. Hence, instead ofusing true responses, which are usually obtained from com-puter simulation, surrogate models need to be implementedfor the calculation of the probability of failure. To gen-erate accurate surrogate models, this paper introduces theD-Kriging method, which is known to be most accurateamong existing methods (Zhao et al. 2010). The method isexplained in detail in the next section.

2.2 Dynamic Kriging (D-Kriging) method

In the Kriging method, the outcomes are considered asa realization of a stochastic process. Consider n samplepoints, X = [x1, x2, ..., xn]T with xi ∈ Rnr, and n responsesy = [y(x1), y(x2), ..., y(xn)]T with y(xi ) ∈ R1. Then, the

response at the samples consists of a summation of twoparts as

y = Fβ + e. (4)

In Eq. (4), Fβ is called the mean structure of the responsewhere F = [ fk(xi )] , i = 1, ..., n, k = 1, ..., K is ann × K design matrix, and fk(x) represents user-definedbasis functions with K terms, which are usually in a sim-ple polynomial form, such as 1, x, x2, ...,; β = [β1, β2, ...,

βK ]T is the vector of regression coefficients; and e =[e(x1), e(x2), ..., e(xn)]T is a realization of the stochasticprocess e(x) that is assumed to have zero mean E[e(xi )] =0and covariance structure E[e(xi )e(x j )] = σ 2 R(θ, xi , x j ),where σ 2 is the process variance, θ is the correlation param-eter that has to be estimated by applying the maximumlikelihood estimator (MLE), and R(θ, xi , x j ) is the correla-tion function of the stochastic process (Chiles and Delfiner1999). Usually, the correlation function is set to Gaussianform in engineering applications and expressed as

R(θ, xi , x j ) =nr∏

l=1

exp(−θl(xi,l − x j,l)

2)

. (5)

where θ = (θ1, θ2, ..., θnr ) is the correlation parametervector and xi,l is the lth component of variable xi . Underthe decomposition of Eq. (4) and θ obtained to maximizethe MLE, the objective of the Kriging method is to predictthe noise-free unbiased response at a new point of interestdenoted by x0. This prediction of response at x0 is writtenas a linear predictor as

y(x0) = wT0 y (6)

where w0 = [w1(x0), w2(x0), ..., wn(x0)]T denotes then×1 weight vector for prediction at x0 and is obtained usingthe unbiased condition E[y(x0)] = E[y(x0)] as (Zhao et al.2010)

w0 = R−1(

r0 + 1

2σ 2Fλ

)(7)

where R is the symmetric correlation matrix with the i − j thcomponent Ri j = R(θ, xi , x j ), i, j = 1, ..., n; λ is the La-grange multiplier; and r0 =[R(θ, x1, x0), ..., R(θ, xn, x0)]T

is the correlation vector between x0 and samples xi , i =1, . . . , n. Hence, substituting Eq. (7) into Eq. (6) yields

y(x0) = wT0 y =

(r0 + 1

2σ 2Fλ

)T

R−1y

= fT0 β + rT

0 R−1(y − Fβ). (8)

where σ 2 = 1n (y−Fβ)TR−1(y−Fβ) and β = (

FTR−1F)−1

FTR−1y are obtained from the generalized least squareregression.

302 I. Lee et al.

Under the assumption of the Gaussian process, the 1−α

level prediction interval of the response is given by

y(x0) − Z1−α/2σp(x0) ≤ y(x0) ≤ y(x0) + Z1−α/2σp(x0)

(9)

where Z1−α/2 is the 1−α level quantile of the standardnormal distribution and σ 2

p(x0) is the predicted variance at

x0 given by σ 2p = σ 2(1 + wT

0 Rw0 − 2wT0 r0). Therefore, the

bandwidth of the prediction interval at a point of interestx0 is

d(x0) = 2Z1−α/2σp(x0) (10)

and this prediction interval will be used as an accuracymeasure to decide if the surrogate model is accurate or not.

Based on the basis functions fk(x) used in Eq. (4), theKriging method is called the ordinary Kriging method, first-order universal Kriging method, and second-order universalKriging method, where basis functions with constant termsonly, up to first-order polynomial terms, and up to second-order polynomial terms, respectively, are used. For both theordinary and universal Kriging methods, the basis functionsdo not change during the surrogate model generation pro-cess. However, it is obvious that in general higher-orderterms can predict nonlinear mean structure, and thus fixed-order basis functions may not be appropriate to describe thenonlinearity of the mean structure. On the other hand, it isalso pointed out that, in some cases, the accuracy of the sur-rogate model may not be improved by using higher-orderterms (Martin and Simpson 2005). Therefore, the problemis how to find the optimal set of polynomial basis functionssuch that the surrogate model would have the best accuracy.

The D-Kriging method dynamically selects the optimalbasis function set at each design point so that the generatedsurrogate model has the best accuracy. As explained above,the accuracy is measured using the prediction interval band-width in Eq. (10). To apply the D-Kriging method to findthe best basis function set, the highest-order P must first bedecided. With n samples given, Eq. (8) can be solved whenthe total number of basis functions is less than n, that is,

C Pnd+P ≤ n. (11)

By choosing the largest P that satisfies Eq. (11), we candecide the highest-order P . For example, if 10 samplesare given (n = 10) for a 2-D problem, the highest-orderP will be 3 to satisfy Eq. (11) so that we can have basisfunctions up to third-order polynomials as 1,x1,x2,x1x2,x2

1 ,x2

2 ,x21 x2,x1x2

2 ,x31 , and x3

2 . After deciding the highest-orderP , the genetic algorithm (GA) is applied to find the bestbasis function set efficiently. A detailed explanation of GAfor the D-Kriging method is provided in Zhao et al. (2010).

The D-Kriging method also proposes to use the patternsearch algorithm to find the optimal correlation parame-ter θ for the Kriging method based on MLE. The MLEmaximization problem for θ is equivalent to

find θ

min ζ(θ) = 1

2ln (|R|) + n

2ln

(σ 2

).

(12)

Since it is not a gradient-based optimization method andguarantees the global convergence, which is proven byLewis and Torczon (1999), the pattern search algorithm ispowerful enough to find the optimal θ, which minimizesEq. (12). How to find the optimal θ more efficiently isexplained in detail in Section 3.5.

Using the GA for the best basis function set and the pat-tern search for the optimal θ, the studied examples showthat the D-Kriging method outperforms existing surrogatemodel generation methods including the universal Krig-ing method, the polynomial response surface method, theradial basis function method, the support vector regressionmethod, and the blind Kriging method (Zhao et al. 2010).

2.3 Stochastic sensitivity analysis

Taking the partial derivative of probability of failure inEq. (2) with respect to the i th design variable μi yields

∂ PF (μ)

∂μi= ∂

∂μi

∫Rnr

I�F (x) fX(x; μ)dx (13)

and the differential and integral operators can be inter-changed using the Leibniz’s rule of differentiation (Rahman2009; Browder 1996), giving

∂ PF (μ)

∂μi=

∫Rnr

I�F (x)∂ fX(x; μ)

∂μidx

=∫

RnrI�F (x)

∂ ln fX(x; μ)

∂μifX(x; μ)dx

=[

I�F (x)∂ ln fX(x; μ)

∂μi

](14)

since I�F (x) is not a function of μi . The partial derivativeof the log function of the joint PDF in Eq. (14) with respectto μi is known as the first-order score function for μi and isdenoted as

s(1)μi

(x; μ) ≡ ∂ ln fX(x; μ)

∂μi. (15)

Hence, to further derive sensitivities of the probability offailure in Eq. (14), the derivation of the first-order scorefunction in Eq. (15) is required and can be performedusing two different inputs - independent and correlated inputrandom variables.

Sampling-based RBDO using the stochastic sensitivity analysis and Dynamic Kriging method 303

2.3.1 Independent input random variables

Consider a random input X = {X1, · · · , Xnr }T whosecomponents are statistically independent random variables.Then, the joint PDF of X is expressed as a multiplication ofits marginal PDFs as

fX(x; μ) =∏nr

i=1fXi (xi ; μi ) (16)

where fXi (xi ; μi ) is the marginal PDF corresponding to thei th random variable Xi . Therefore, for statistically indepen-dent random variables, the first-order score function for μi

is expressed as

s(1)μi

(x; μ) ≡ ∂ ln fX(x; μ)

∂μi= ∂ ln fXi (xi ; μi )

∂μi. (17)

Since the marginal PDF and the cumulative distributionfunction (CDF) are available analytically as listed in Table 1,where (·) and (·) are the standard normal CDF and PDF,respectively, given by

(x) =∫ x

−∞φ(ξ)dξ = 1√

2π

∫ x

−∞exp

(−1

2ξ2

)dξ, (18)

the derivation of the first-order score function for the statis-tically independent random input is very straightforward forthe distributions shown in Table 1.

For the uniform distribution, deriving the first-orderscore function is not straightforward since two integrationbounds are a function of μi so that the differential and inte-gral operators in Eq. (13) cannot be interchanged; thus, Eq.(14) cannot be used for the uniform distribution. However,using the Leibniz integral rule (Hijab 1997), the first-orderscore function for the uniform distribution can be derivedand is shown in detail in Lee et al. (2011).

2.3.2 Correlated input random variables

Consider a bivariate correlated random input X = {Xi ,

X j }T. Then, the joint PDF of X is expressed as (Noh et al.2009a; Noh et al. 2010)

fX(x; μ) = ∂2C(u, v; θ)

∂u∂vfXi (xi ; μi ) fX j (x j ; μ j )

= C,uv(u, v; θ) fXi (xi ; μi ) fX j (x j ; μ j ) (19)

where C is the copula function, u = FXi (xi ; μi ) and v =FX j (x j ; μ j ) are marginal CDFs for Xi and X j , respectively,and θ is the correlation coefficient between Xi and X j . Thepartial derivative of the copula function with respect to themarginal CDFs u and v in Eq. (19) is called the copuladensity function and denoted as

c(u, v; θ) ≡ ∂2C(u, v; θ)

∂u∂v= C,uv(u, v; θ). (20)

Accordingly, using Eq. (19), the first-order score functionsin Eq. (15) for a correlated bivariate input are expressed as

s(1)μi

(x;μ)≡∂ ln fX(x; μ)

∂μi= ∂ ln c(u, v; θ)

∂μi+∂ ln fXi (xi ; μi )

∂μi.

(21)

The derivation of the first term of the right-hand side ofEq. (21) is straightforward and is listed in Table 2, and thesecond term of the right-hand side of Eq. (21) is identical toEq. (17). One can see from Table 2 that Eq. (21) is identicalto Eq. (17) if the independent copula is used. This meanswe can generally use Eq. (21) for the first-order score func-tion and the joint PDF for independent random variables isa special case of Eq. (21). In Table 2, the partial deriva-tive of the marginal CDF with respect to μi , ∂μi can bestraightforwardly obtained from the analytic CDFs shownin Table 1. Even if several pairs of bivariate correlated ran-dom variables exist in X = {X1, · · · , Xnr }T, the first-orderscore function for μi is the same as Eq. (21).

Table 1 Marginal PDF, CDF, and parameters

Distribution PDF, f X (x) CDF, FX (x) Parameters

Normal1√

2πσe−0.5[ x−μ

σ]2

(x − μ

σ

)μ, σ

Log-normal1√

2πx σe−0.5[ ln x−μ

σ]2

(ln x − μ

σ

)σ 2 = ln

[1 +

(σ

μ

)2]

, μ = ln(μ) − 0.5σ 2

Gumbel αe−α(x−ν)−e−α(x−ν)

αe−e−α(x−ν)

μ = ν + 0.577

α, σ = π√

6α

Weibullk

ν

( x

ν

)k−1e−( x

v)k

1 − e−( xv)k

μ = v�

(1 + 1

k

), σ 2 = v2

[�

(1 + 2

k

)− �2

(1 + 1

k

)]

304 I. Lee et al.

Table 2 Log-derivative ofcopula density function Copula type

∂ ln c(u, v; θ)

∂μi

Clayton

(−1 + θ

u+ (2θ + 1)u−(1+θ)

u−θ + v−θ − 1

)∂u

∂μi

AMH

[θ2 (1 − v) + θ(v + 1)

1 − θ2 (1 − u) (1 − v) − θ (2 − u − v − uv)− 3θ(1 − v)

1 − θ (1 − u) (1 − v)

]∂u

∂μi

Frank θ

[2

(eθ(1+u) − eθ(u+v)

)eθ − eθ(1+u) − eθ(1+v) + eθ(u+v)

+ 1

]∂u

∂μi

FGM

[2θ(2v − 1)

1 + θ (1 − 2u) (1 − 2v)

]∂u

∂μi

Gaussian

[−1(u)

φ(−1(u))+ θ−1(v) − −1(u)

φ(−1(u))(1 − θ2)

]∂u

∂μi

Independent 0

2.4 Probability of failure and its sensitivity calculationusing MCS

Denote the surrogate model for the constraint functionG j (X) as G j (X). Then, by applying the MCS to the sur-rogate model G j (X), the probabilistic constraints in Eq. (1)can be approximated as

PFj ≡ P[G j (X) > 0] ∼= 1

M

M∑m=1

I�Fj

(x(m)) ≤ PTarFj

(22)

where M is the MCS sample size, x(m) is the mth realiza-tion of X, and the failure set �Fj for the surrogate model

is defined as �Fj ≡ {x : G j (x) > 0}. The sensitivity of theprobabilistic constraint in Eq. (1) is obtained as

∂ PFj

∂μi

∼= 1

M

M∑m=1

I�Fj

(x(m)) s(1)μi

(x(m); μ) (23)

where s(1)μi (x(m); μ) is obtained using Eq. (21). In this

calculation, only component probabilities of failure are con-sidered. However, if the failure set is appropriately definedfor the system reliability as explained in Section 2.1, thesame probability of failure and its sensitivity in Eqs. (22)and (23) can be also used for the system RBDO. Hence,in this paper, we will consider the component RBDO onlywithout loss of generality.

Accuracy of the MCS, denoted by εMC S , to computeEq. (22) can be measured using the 95% confidence intervalof the estimated probability of failure and given by (Haldarand Mahadevan 2000)

εMC S =√

(1 − PTarF )

M × PTarF

× 200%. (24)

For example, if the MCS sample size M is 500,000 andPTar

F = 2.275%, then εMC S is 1.85% of the target probabil-ity of failure, which means that there is 95% probability thatthe probability of failure estimated using the MCS will bebetween 2.233% and 2.317% with 500,000 samples. To esti-mate of the accuracy of Eq. (23), Eq. (23) can be rewrittenas

∂ PFj

∂μi= 1

M

M f∑m f =1

I�Fj

(x(m f )

)s(1)μi

(x(m f ); μ

)

= 1

M

M f∑m f =1

s(1)μi

(x(m f ); μ

) = M f

Mμs f = PFj μs f (25)

where M f is the number of failed samples and μs f is themean value of the score function values for the failed sam-ples. Hence, the accuracy for the MCS to compute Eq. (23)can be measured by εMC Sμs f . Since μs f changes basedon problems, the accuracy of Eq. (23) varies according toproblems or design points even for the same problem.

As shown in Eq. (23), the sensitivity calculation usingthe score function and MCS does not require the sensitiv-ity of the surrogate model and only uses the function valuesof the surrogate model for the sensitivity calculation. Fur-thermore, the computation of the sensitivity using the scorefunction does not include any approximation except the sta-tistical noise due to the MCS shown in Eq. (24), whichcan be avoided using a sufficiently large MCS sample size.In addition, this sensitivity analysis does not require thetransformation from the original design space to the stan-dard normal space, which is required for FORM or SORMand usually makes the performance function become highlynonlinear, especially when the random input follows non-Gaussian marginal distribution and is correlated. Therefore,the sensitivity analysis using the score function and MCS

Sampling-based RBDO using the stochastic sensitivity analysis and Dynamic Kriging method 305

will be very accurate and computationally efficient for engi-neering applications with correlated random input onceaccurate surrogate models are available. This is achieved inthis paper by using the D-Kriging method.

3 Practical use of sampling-based RBDO

This section explains five numerical strategies to accu-rately and efficiently carry out the sampling-based RBDOdescribed in Section 2, which are the local window for sur-rogate model generation, sampling strategy, filtering of con-straints, sample reuse and local window enlargement, andadaptive initial point for the pattern search. After explainingthe five strategies, this section concludes with showing theoverall algorithm flowchart of the sampling-based RBDO.

3.1 Hyper-spherical local window for surrogatemodel generation

Instead of using the global window, the local window con-cept is used for the generation of surrogate models as shownin Fig. 1. The radius R of the local window can be decided as

R = cRβt (26)

where cR is the coefficient, which is usually between 1.0and 2.0, and β t is the target reliability index of RBDO.However, Eq. (26) works only in the standard normal U-space where all random variables have the standard normaldistribution. Hence, for the generation of samples in the X-space, first samples are generated in the hyper-sphere withthe radius R given in Eq. (26) in the U-space. Then, the gen-erated samples are transformed back to the X-space usingthe Rosenblatt transformation (Rosenblatt 1952). How to

Fig. 1 Hyper-spherical local window for surrogate model

generate samples in the U-space is explained in Section 3.2.If β t is different for each constraint, then the maximum β t

can be used for Eq. (26). As shown in Eq. (26), the localwindow size is slightly larger than the hyper-sphere for theinverse reliability analysis, that is, the MPP search domain;so that it includes the limit state functions near the optimumdesign. When the target probability of failure is relativelylarge, the local window becomes smaller as shown in Eq.(26) which may make the reliability analysis inaccurate dueto the extrapolation outside the local window. However,once the limit state function, which will be located inside thelocal window is accurate, the extrapolation may not affectthe reliability analysis because the sampling-based reliabil-ity analysis does not require accurate function values awayfrom the limit state function; instead it requires accuratesign of function values as shown in Eq. (22).

If we have the hyper-spherical local window whose diam-eter is the same as the length of the hyper-cubic localwindow as shown in Fig. 2, we can count the number ofsamples located in the grey area in Fig. 2, which does notsignificantly affect the accuracy of the reliability analysisusing surrogate models.

Assuming Nc samples are uniformly distributed in thehyper-cubic local window, the number of samples located inthe grey area (Ng) and hyper-sphere (Ns) are approximatelygiven by

Ng = Nc − Ns ∼= Nc

(1 − Vs

Vc

)= Nc

(1 − CD RD

(2R)D

)(27)

where D is the dimension, Vs and Vc are the volume of thehyper-sphere and hyper-cube, respectively, and CD is theconstant of proportionality given by

CD = πD2

�( D2 + 1)

. (28)

Fig. 2 Hyper-sphere and hyper-cube

306 I. Lee et al.

Table 3 shows the number of samples Ns and Ng locatedin the hyper-sphere and grey area for different dimensionswhen 100 uniformly distributed samples (Nc) are located inthe hyper-cube. From the table, we can see that the num-ber of samples in the hyper-sphere which mostly affectsthe accuracy of the reliability analysis decreases dramat-ically as the dimension increases and even becomes zerowhen the dimension is 10 or higher. However, this resultdoes not mean that the hyper-cubic local window cannotaccurately approximate true responses since there is nosample in the hyper-sphere. It does mean that the hyper-cubic local window also requires accuracy of the grey areawhich may not affect the reliability analysis result since itis away from the limit state function. Consequently, it willbe computationally very inefficient to use the hyper-cubiclocal window for the reliability analysis. On the other hand,the hyper-spherical local window can improve the compu-tational efficiency while maintaining the similar accuracyespecially for high dimensional problems.

3.2 Sampling strategy

After deciding the radius of the hyper-spherical local win-dow for the surrogate model generation, Nr initial samplesneed to be generated in the local window. However, Nr uni-formly distributed samples cannot be directly generated inthe hyper-sphere using Hammersley sampling or Latin Cen-troidal Voronoi Tessellations (LCVT) (Burkardt et al. 2002;Simpson et al. 2001a). Instead, Nc samples which is givenby

Nc = NrVc

Vs= Nr

2D

CD(29)

are uniformly generated in the hyper-cubic local window,and then samples whose distances to the origin are less thanor equal to R are collected, which will be located in thehyper-sphere with the radius R. The minimum number ofthe initial samples is decided using Eq. (11). For example,for a 12-D example with first-order polynomial basis func-tions, the minimum number of Nr will be C1

12+1 = 13.However, for high-dimensional problems, the minimum

Table 3 Number of samplesin each region D Nc Ns Ng

2 100 79 21

4 100 31 69

6 100 8 92

8 100 2 98

10 100 0 100

12 100 0 100

number of initial samples may not be sufficient to gener-ate accurate surrogate models. In that case, users can definethe number of initial samples considering the dimension ofproblems as shown in Jin et al. (2001).

The accuracy of the surrogate model generated with theinitial samples Nr can be estimated using

mean(M SE(xi ))

V ar(y(x j )), for i = 1 ∼ S and j = 1 ∼ n (30)

where is the variance of n true responses and is used to nor-malize the accuracy measure, S is the total number of testingpoints, which are generated once using LCVT at the begin-ning and are reused during the design optimization to savethe computation time, and is the predicted mean square error(MSE) from the Kriging model written as

M SE(xi ) =(

d(xi )

2Z1−α/2

)2

(31)

where d(xi ) and are from Eq. (10). The physical mean-ing of the accuracy measure in Eq. (30) is related to thebandwidth of the Kriging model. Hence, the smaller thebandwidth is, the better the surrogate model is. If the accu-racy of a surrogate model is satisfactory, which means theaccuracy measure in Eq. (30) is less than the target accuracyεa , which is recommended to be 0.005 for low dimensionalproblems and 0.01 for high dimensional problems, respec-tively, then the surrogate model can be used for reliabilityand sensitivity analyses. However, if the accuracy does notsatisfy the target, more samples are sequentially insertedwithin the local window until the surrogate model satisfiesthe target accuracy condition. How to do the sequentialsample insertion is explained in detail in Zhao et al. (2010).

Equation (30) is the accuracy measure for cases whereonly one surrogate model is generated. When multiplesurrogate models need to be generated as in the RBDO pro-cedure in Eq. (1) where multiple constraint functions areused, then the accuracy measure in Eq. (30) needs to bemodified to reflect the effect of multiple modeling. In thispaper, the maximum value of accuracy measures for eachsurrogate model is used as the accuracy measure of mul-tiple surrogate models, and thus the accuracy criterion isgiven by

max

{mean(M SEk(xi ))

V ar(yk(x j ))

}≤ εa, for k = 1 ∼ nc (32)

where yk(x j ) and MSEk are the variance of n true responsesand the predicted MSE for the kth surrogate model, and ncis the number of surrogate models.

Sampling-based RBDO using the stochastic sensitivity analysis and Dynamic Kriging method 307

3.3 Filtering of constraints

After computer simulations such as durability or stress anal-ysis are carried out at generated sample points in the localwindow, true function values for each constraint functionare saved and are used to determine if constraints are fea-sible or not. If function values for a certain constraint arenegative at all sample points on the local window or lessthan a small negative feasibility constant ε f , which meansthat the constraint is feasible because we define a constraintas failing if G(X) > 0 in Eq. (1), then a surrogate modelfor the constraint is not generated because we can concludethat the probability of failure for the constraint will be zerowithout generating the surrogate model. Hence, if a con-straint is identified as very feasible, −1 for the constraintvalue and 0 for the sensitivity of the constraint are assignedwithout generating a surrogate model to save computationtime. −1 for the constraint value comes from the normal-ized constraint in Eq. (1), which is given by PF

PTarF

− 1 ≤ 0,

since PF will be zero for the constraint. Similarly, 0 forthe sensitivity of the constraint comes from Eq. (14) sinceI�F (x) will be zero for the constraint, too. Hence, the accu-racy measure in Eq. (32) does not include these very feasibleconstraints because surrogate models for the constraints arenot generated. In Section 5.3 using a 12-D M1A1 roadarmexample, it will be shown how the proposed filtering of con-straints works. As in the roadarm example, this filtering ofconstraints works very well for large dimension problemswhere a number of initial samples are required.

3.4 Sample reuse and local window enlargement

During the design iteration, the local window is scanned tocheck whether samples exist before generating Nr initialsamples. If there exist samples whose number is denotedas Ne in the local window, and if Ne is less than Nr , thenNr − Ne samples are generated in the local window insteadof generating Nr samples. In this case, the surrogate modelgenerated using Nr − Ne new samples may not be accuratecompared to the surrogate model generated using Nr newsamples. However, using the sequential sampling explainedin Section 3.2, the target accuracy can be achieved, and inmost cases the number of inserted samples is much less thanthe number of existing samples, which results in significantsave of the computation time. If the number of existingsamples Ne is larger than Nr , no samples are generated.

Moreover, if Ne > 0.9Nr , it means that design movementis very small or the current design moves back near the pre-viously existing design, which means the current design isvery close to the optimum design. In that case, the localwindow size is enlarged to include more existing samplesand to remove the extrapolation effect. For example, cR inEq. (26) is initially set up as 1.3, and if Ne > 0.9Nr , then

cR becomes 1.5 so that the local window size is enlarged.The enlargement of the local window may reduce the accu-racy of the surrogate model. However, inclusion of moresamples makes the surrogate model more accurate, and theaccuracy gain of the surrogate model due to the inclusionof more samples is more significant than the accuracy lossdue to the enlargement of the local window. Numbers usedin this section, such as 0.9, 1.3, and 1.5, can be decided byusers based on their experience.

3.5 Adaptive initial point for pattern search

When applying the sampling-based RBDO to complex engi-neering problems, the number of variables used for surro-gate modeling is usually large, i.e., 12 for the M1A1 tankroadarm, which will be shown in Section 5.3. In such cases,the pattern search algorithm to find the optimal correla-tion parameter θ in Eq. (12) may become computationallyexpensive. It is known that the computational time of thepattern search is strongly affected by the initial search point.If the pattern search starts from the neighboring area of thetrue optimum, it can find the optimum θ within remarkablyshorter time than if it starts from a point far away from theoptimum. Moreover, in RBDO, if the design movement is

Fig. 3 Flowchart of sampling-based RBDO

308 I. Lee et al.

very small, which means that the current design is near theoptimum design, then a surrogate model generated at thecurrent design will be very similar to the one generated atthe previous design. This means that two optimum θ will bevery similar. Therefore, we can adaptively use the optimal θ

obtained in the previous iteration as the initial point for thepattern search of the current iteration instead of using anyfixed initial point. This will save computation time for theD-Kriging method in particular when the design approachesthe optimum design.

Figure 3 shows the overall algorithm flowchart of thesampling-based RBDO using the D-Kriging method for thesurrogate model generation and the stochastic sensitivityanalysis by the score function.

4 Parallel computing for sampling-based RBDO

As explained in Section 2.2, the D-Kriging method usesthe pattern search and genetic algorithm for more accuratesurrogate model generation. Hence, as the dimension ofthe complex engineering system increases, the D-Krigingmethod and MCS for the reliability analysis using surro-gate models become computationally inefficient. Moreover,the number of samples required for finite element analysisand fatigue analysis increases. Therefore, a high perfor-mance computing strategy needs to be implemented into thesampling-based RBDO procedure to ensure it is applicablefor large-scale complex engineering applications.

For the sampling-based RBDO using the D-Krigingmethod, there exist three major places where the parallelcomputing can be applicable: parallelization of surrogatemodel generation for multiple constraints, parallelizationof MCS for multiple surrogate models, and parallelizationof computer simulations at samples. The Matlab paral-lel computing toolbox and the parallel computing platform(LSF-Platform) are utilized for the first two paralleliza-tions and for the last parallelization, respectively, of thesampling-based RBDO with the D-Kriging method.

Compared with the gradient-based or MPP-based RBDO,the sampling-based RBDO has inherent advantages in termsof the parallelization. In the MPP-based RBDO, the paral-lelization is limited to the number of constraints (nc), whichmeans that even if numerous client computers are avail-able it can only use nc client computers for the parallelizedcomputing. On the other hand, the sampling-based RBDOcan use as many client computers as the number of sam-ples, which is usually more than the number of constraintsfor high dimensional problems. In addition, the sampling-based RBDO has more places where the parallelizationcan be applicable. Hence, in terms of the parallel comput-ing, the sampling-based RBDO is more effective than theMPP-based RBDO.

4.1 Parallelization of surrogate models for multipleconstraints in RBDO

A typical RBDO problem contains more than one con-straint. Since the surrogate model from the D-Kriging meth-od is computationally expensive for high dimensional large-scale applications, it is desirable to carry out the surrogatemodeling for all constraints simultaneously, which leadsto the parallelization of surrogate modeling for multipleconstraints. It should be noted that one license of theMatlab parallel computing toolbox allows only eight coresworking simultaneously, therefore eight surrogate modelsfor constraints identified active using the feasibility checkexplained in Section 3.3 are generated at the same time.

4.2 Parallelization of MCS in reliability analysis

The MCS in the sampling-based RBDO is used to calcu-late failure probabilities of performance functions as wellas their sensitivities with respect to design variables. Usu-ally, MCS requires a large number of samples for accurateresults as shown in Eq. (24). Moreover, since the predic-tion from the D-Kriging at the MCS samples is implicit, alarge dimension matrix calculation is involved every timethe prediction is calculated at each MCS point. As the num-ber of the MCS samples increases, the total computationaltime for the reliability and sensitivity analysis increasesas well. Therefore, the parallelization of the MCS pro-cedure is also needed to reduce the large computationaltime. This parallelization is also conducted with eight coresworking simultaneously by using the Matlab parallel com-puting toolbox. Thus, using the parallelization explained inSections 4.1 and 4.2, the computation time is maximally 8times faster than the one without the parallelization. How-ever, the number of cores used for the parallelization canbe extended if parallel computing software other than theMatlab toolbox is utilized.

4.3 Parallelization of computer-aided engineering (CAE)at samples

To generate surrogate models of the performance functionsin RBDO by the D-Kriging method, it is required to eval-uate the performance functions at the sample points, whichis usually conducted by computer simulation. This proce-dure is computationally intensive for large-scale complexengineering applications. If the number of the computersimulations is large for a large-scale engineering applica-tion, apparently, the total computational time of conduct-ing the computer simulations for all samples may becomeunaffordable. Therefore, the parallelization is necessary forthis procedure. Usually the computer simulation is carried

Sampling-based RBDO using the stochastic sensitivity analysis and Dynamic Kriging method 309

Fig. 4 Workflow ofparallelization in RBDO

out by general CAE commercial software and the paral-lelization can be done by the parallel computing platform(LSF-Platform). Unlike the Matlab parallel computing tool-box which has the restriction of the maximum number ofcores, the number of parallel computing session controlledby the LSF-Platform is not restricted. Therefore, one can useas many client computers on the cluster network as possible.

4.4 Summary of parallelization in sampling-based RBDO

With all the discussion above, we can obtain the entire work-flow of the parallelization in the sampling-based RBDOshown in Fig. 4, where a “core” means a unit in a multiple-core desktop and a “node” means one client computer in acluster network.

5 Numerical examples

This section first uses one example to compare hyper-spherical and hyper-cubic local windows in terms of accuracyand efficiency, and then illustrates two design optimiza-tion examples—a 2-D mathematical example and a 12-DM1A1 Abrams tank roadarm—to see how the proposedsampling-based RBDO works for an RBDO problem. The2-D mathematical example is used to show the accuracy andefficiency of the proposed method since its analytic func-tions are given, and thus the MCS is applicable for the com-parison of the probability of failure calculation. The 12-DM1A1 Abrams tank roadarm is used to see how the pro-posed sampling-based RBDO works for a high-dimensionalengineering application in terms of accuracy and efficiency.In addition, using the roadarm example, the effectiveness

of the parallelization is also explained. For all exam-ples, one million testing points are used for the D-Krigingmethod, and 500,000 MCS samples are used for the relia-bility and sensitivity analysis and the MCS sample numberincreases to one million when constraints are identified asactive.

5.1 Comparison of hyper-spherical and hyper-cubiclocal window

To see how the hyper-spherical local window proposedin Section 3.1 works for high-dimensional problems incomparison to the hyper-cubic local window, consider a 9-D polynomial function, which is known as the extendedRosenbrock function (Viana et al. 2009) and modified forthe purpose of the probability of failure calculation,

G(X) =8∑

i=1

[(1 − Xi )

2 + 100(Xi+1 − X2i )2

]− 518,

− 5 ≤ Xi ≤ 10 for i = 1, · · · , 9 (33)

The properties of nine random variables in Eq. (33) areshown in Table 4.

The number of the initial samples (Nr ) is 150, thecoefficient of the local window size (cR) is 1.2, and the MSEnumber for the stopping criterion is 0.01. The probability offailure using the analytic function in Eq. (33) is 21.8268%.

Table 4 Properties of random variables

Random variables Distribution Mean Standard deviation

X1 ∼ X9 Normal 1.0 0.3

310 I. Lee et al.

Using the hyper-cubic local window, in addition to the ini-tial 150 samples, 202 more samples are inserted to satisfythe target accuracy and the probability of failure using thehyper-cubic local window is 22.3243%. On the other hand,using the hyper-spherical local window, 56 samples areinserted and the probability of failure is 21.4474%. Thisexample shows that the hyper-spherical local window cangenerate surrogate models much more efficiently than thehyper-cubic local window especially for high-dimensionalproblems.

5.2 RBDO of 2-D mathematical problem

Consider a 2-D mathematical RBDO problem, which isformulated to

minimize Cost(d)=− (d1 + d2−10)2

30− (d1 − d2 + 10)2

120

subject to P(G j (X(d))> 0) ≤ PTarFj

=2.275%, j =1∼3

dL ≤ d ≤ dU , d ∈ R2 and X ∈ R2 (34)

where three constraint functions are expressed as

G1(X) = 1 − X21 X2

20

G2(X) =−1 + (Y − 6)2+ (Y − 6)3 − 0.6 × (Y − 6)4 + Z

G3(X) = 1 − 80

X21 + 8X2 + 5

(35)

Fig. 5 Shape of constraint functions

Table 5 Properties of random variables

Random Distribution dL d0 dU Standard

variables deviation

X1 Normal 0.0 5.0 10.0 0.3

X2 Normal 0.0 5.0 10.0 0.3

where

{YZ

}=

[0.9063 0.42260.4226 − 0.9063

] {X1

X2

}, and are

drawn in Fig. 5. The properties of two random variables areshown in Table 5, and they are correlated with the Clay-ton copula (τ = 0.5). As shown in Eq. (34), the targetprobability of failure (PTar

F ) is 2.275% for all constraints.As shown in Fig. 5 and Table 5, the initial design is

d0 = [5,5)T . At the initial design, deterministic designoptimization (DDO) is first used to find the deterministicoptimum, which is usually close to the RBDO optimum, andthe sampling-based RBDO is launched at the determinis-tic optimum design. This approach is more computationallyefficient than launching the RBDO from the beginning(Youn et al. 2005c). As shown in Fig. 6, the DDO requires30 samples, which are marked as asterisks in the figure, forthe whole design iteration, and the deterministic optimumdesign is exactly identical to the optimum design obtainedusing analytic functions in Eq. (35). At the deterministicoptimum, the sampling-based RBDO is launched with thelocal window coefficient cR = 1.0. Near the deterministicoptimum, there exist samples used for the DDO, and thenumber of existing samples Ne is larger than the number

Fig. 6 Sample profile for DDO and RBDO

Sampling-based RBDO using the stochastic sensitivity analysis and Dynamic Kriging method 311

Table 6 Comparison of various RBDOs (PTarF = 2.275%)

Methods Cost Optimum design MCS (50 M) Number of function evaluations

PF1 , % PF2 , %

MPP-based RBDO FORM −1.8742 5.0026, 1.6165 2.3022 1.2835 52 + 52 × 2

DRM3 −1.8794 5.0315, 1.6050 2.2912 1.7496 128 + 106 × 2

DRM5 −1.8821 5.0454, 1.5988 2.2621 2.0183 146 + 102 × 2

Sampling-based RBDO D-Kriging −1.8845 5.0576, 1.5936 2.2716 2.2869 50

Analytic function −1.8853 5.0541, 1.5918 2.2912 2.2791 N.A.

of required initial samples Nr , which is 4 in this example.Hence, the local window coefficient cR increases to 1.3, anda total of 18 samples are initially used for the first itera-tion of the sampling-based RBDO. Twenty more samples,which are marked as dots in Fig. 6, are generated for thesampling-based RBDO whose result is shown in Table 6.

Table 6 compares the numerical results of five differentRBDO methods. The first three results are obtained fromthe MPP-based RBDO, which requires sensitivities of con-straint functions for the MPP search and design optimiza-tion. This MPP-based RBDO includes the FORM (Younet al. 2005c) and the DRM (Lee et al. 2008) with threeand five quadrature points, which are denoted in Table 6 asDRM3 and DRM5, respectively. The results of the last tworows are obtained from the sampling-based RBDO, whichuses the MCS for the estimation of the probability of failureand its sensitivity. The sampling-based RBDO using the D-Kriging method is the proposed method, and to compare theaccuracy of the proposed method, the result of the sampling-based RBDO using the analytic (true) functions given inEq. (35) is also shown in the table.

From the table, it can be seen that the probability offailure of the second constraint (1.2835%) estimated bythe MCS with 50 million samples at the optimum designobtained using the FORM is not close to the target probabil-ity of failure (2.275%). This is because the second constraintis highly nonlinear as shown in Fig. 6, and the FORM usesthe transformation from original X-space to standard nor-malized U-space, which makes the constraint even morehighly nonlinear due to the correlated nonlinear input. Forhighly nonlinear functions, the FORM cannot accuratelyestimate the probability of failure since it uses the linearapproximation of the nonlinear functions at the MPP in U-space. To improve the accuracy of the probability of failureat the optimum design, the MPP-based DRM with three orfive quadrature points can be used (Lee et al. 2008); Table 6shows that the MPP-based DRM indeed improves the accu-racy of the probability of failure at the optimum design withmore function evaluations. However, to obtain a more accu-rate optimum design, more quadrature points are required,such as the DRM7, etc. To obtain the optimum design, theFORM uses 52 function evaluations and 52 × 2 = 104

sensitivity calculations, whereas the MPP-based DRM withfive quadrature points uses 146 function evaluations and102 × 2 = 204 sensitivity calculations, and the numberof function evaluations for the MPP-based DRM will beincreased as the number of quadrature points increases (Leeet al. 2008).

On the other hand, the sampling-based RBDO showsvery accurate optimum design since the optimum designis very close to the optimum design obtained using theanalytic functions. However, it requires only 50 samples,which is even less than the FORM, for the accurate opti-mum design without requiring the sensitivity of the perfor-mance functions. The sampling-based RBDO can obtaina very accurate optimum design because it does not useany approximation on the calculation of the probability offailure, unlike the FORM and MPP-based DRM, and theD-Kriging method generates very accurate surrogate mod-els. In addition, it can be said that the proposed efficiencystrategies indeed work in this example. Therefore, once sur-rogate models for constraint functions are accurate enough,the proposed sampling-based RBDO could obtain a veryaccurate optimum design with good efficiency.

5.3 RBDO of M1A1 Abrams tank roadarm

5.3.1 Model description

The roadarm of the M1A1 Abrams tank (Lee et al. 2008)is used to compare two approaches: the MPP-based RBDO,which requires sensitivities of performance functions, andthe sampling-based RBDO, which does not require sensitiv-ities of performance functions, for the component RBDO.

Fig. 7 Finite element model of roadarm

312 I. Lee et al.

The roadarm is modeled using 1,572 eight-node isopara-metric finite elements (SOLID45) and four beam elements(BEAM44) of ANSYS (Swanson Analysis System Inc1989), as shown in Fig. 7, and is made of S4340 steel withYoung’s modulus E = 3.0 × 107 psi and Poisson’s ratioν = 0.3. The durability analysis of the roadarm is carriedout using Durability and Reliability Analysis Workspace(DRAW) (Center for Computer-Aided Design, College ofEngineering 1999a, b) to obtain the fatigue life contour. Thefatigue lives at the critical nodes are shown in Fig. 8, whichare chosen as the design constraints of the RBDO.

The shape design variables are shown in Fig. 9. Eightshape design variables characterize four cross-sectionalshapes of the roadarm. The widths (x1-direction) of thecross-sectional shapes are defined by the design variablesd1, d3, d5, and d7 at intersections 1, 2, 3, and 4, respectively,and the heights (x3-direction) of the cross-sectional shapesare defined using the remaining four design variables. Eightshape design variables are listed in Table 7 and are assumedto be independent random variables.

For the input fatigue material properties, since the statis-tical information on S4340 steel other than its nominal valueis not available, it is necessary to assume the statistical infor-mation on S4340 steel. The strain-life relationship is givenby the classical Coffin-Manson equation as (Meggiolaro andCastro 2004)

�ε

2= �εe

2+ �εp

2= σ ′

f

E(2N f )

bs + ε′f (2N f )

cd (36)

where σ ′f and bs are the fatigue strength coefficient and

exponent, respectively; ε′f and cd are the fatigue ductility

coefficient and exponent, respectively; N f is the fatigue

initiation life; and E is the Young’s modulus. It is knownthat σ ′

f and ε′f follow the lognormal distribution and bs and

cd follow the normal distribution. Furthermore, it is alsoknown that σ ′

f , bs, and ε′f , cd are highly negatively cor-

related (Noh et al. 2010; Annis 2004). For the correlatedfatigue material properties, it is assumed that σ ′

f and bs

follow the Gaussian copula with ρ = −0.828 and that ε′f

and cd follow the Frank copula with τ = −0.906 (Nohet al. 2010). For the standard deviations of S4340 steel, 3%coefficients of variation (COV) for fatigue material proper-ties are assumed as shown in Table 7. It is known that fatiguestrength parameters and fatigue ductility parameters are notcorrelated each other.

5.3.2 Sampling-based RBDO results

The RBDO for the M1A1 Abrams tank roadarm is formu-lated to

minimize Cost(d)

subject to P[G j (X) > 0] ≤ PTarFj

, j = 1, · · · , 13

dL ≤ d ≤ dU, d ∈ R8 and X ∈ R12 (37)

where

Cost(d) : Weight of Roadarm

G j (d) = 1 − L(d)

Lt, j = 1 ∼ 13

L(d) : Crack Initiation Fatigue Life,

Lt : Crack Initiation Target Fatigue Life (= 5 years)

PTarF = 2.275% (38)

Fig. 8 Fatigue life contour atcritical nodes of roadarm

Sampling-based RBDO using the stochastic sensitivity analysis and Dynamic Kriging method 313

Fig. 9 Shape design variablesfor roadarm

For the sampling-based DDO, 15 samples are used as theinitial number of samples in the local window whose size isdecided by the coefficient of cR = 0.3. Smaller local win-dow is used for the DDO since the accuracy of the surrogatemodel near a given design is required for sensitivity cal-culation and there is no need of reliability analysis. After

11 iterations, the sampling-based DDO converged to theoptimum design, using 135 samples. The optimum designobtained using the sampling-based DDO is almost identicalwith the optimum design obtained using the sensitivity-based DDO as shown in Table 8. The sensitivity-based DDOrequires 11 function and 11 sensitivity evaluations as shown

Table 7 Random variables andfatigue material properties Random variables Lower bound dL Initial design do Upper bound dU COV Distribution type

d1, in. 1.350 1.750 2.150 1% Normal

d2, in. 2.650 3.250 3.750

d3, in. 1.350 1.750 2.150

d4, in. 2.570 3.170 3.670

d5, in. 1.356 1.756 2.156

d6, in. 2.438 3.038 3.538

d7, in. 1.352 1.752 2.152

d8, in. 2.508 2.908 3.408

Fatigue material properties

Non-design uncertainties Mean COV Distribution type

Fatigue strength coefficient, σ ′f , psi 177000 3% Lognormal

Fatigue strength exponent, bs −0.0730 Normal

Fatigue ductility coefficient, ε′f 0.4100 Lognormal

Fatigue ductility exponent, cd −0.6000 Normal

314 I. Lee et al.

Table 8 Optimum designcomparison Design variable Initial Sensitivity-based Sampling-based

DDO RBDO DDO RBDO

d1 1.750 1.653 1.711 1.653 1.705

d2 3.250 2.650 2.650 2.650 2.650

d3 1.750 1.922 1.943 1.922 1.941

d4 3.170 2.570 2.570 2.570 2.570

d5 1.756 1.478 1.514 1.478 1.508

d6 3.038 3.287 3.348 3.287 3.352

d7 1.752 1.630 1.691 1.630 1.702

d8 2.908 2.508 2.508 2.508 2.508

# of F.E. 11 + 11 × 8 85 + 85 × 12 135 691

Active constraints 1, 3, 5, 8, 12 1, 3, 5, 8, 12 1, 3, 5, 8, 12 1, 3, 5, 8, 12

Cost 515.09 466.80 474.20 466.81 474.60

in Table 8, where F.E. stands for ‘function evaluation’. Onesensitivity evaluation includes sensitivity calculations for alldesign variables, so it requires 11 × 8 = 88 sensitivitycalculations in this example, whereas the sampling-basedDDO requires a total of 135 samples for the surrogate modelgeneration using the D-Kriging method.

The sampling-based RBDO is launched at the DDO. Inthis case, samples used for the DDO cannot be used forthe RBDO unlike the mathematical example in Section 5.2because the dimension of the DDO is 8, whereas the dimen-sion of the RBDO is 12. The coefficient for the localwindow size (cR) is starting as 1.3 and will be increased to1.5 when there are sufficient samples in the local window.The number of initial samples within the local window is200. It is found that four out of 13 performance functions arevery feasible at the deterministic optimum. Hence, surrogatemodels for those performance functions are not generatedto save the computation time. Table 8 also compares twoRBDO optimum designs obtained using the sensitivity-based and sampling-based RBDO. The FORM is used forthe sensitivity-based RBDO.

From the table, it can be seen that two optimum designsare very close to each other. At the optimum design obtainedusing the sampling-based RBDO, the MSE of the surro-gate model is 0.0062, which is much less than the targetMSE (0.01) and means that surrogate model at the optimumdesign is accurate enough. To obtain the optimum designusing the FORM, 85 function and 85 × 12 = 1020 sensi-tivity evaluations are used since there exist 8 random designvariables and 4 random parameters, whereas the sampling-based RBDO uses 691 samples, which means 691 functionevaluations, to find the optimum design. Table 9 also showshow 691 samples are generated in the local window at eachiteration step in the sampling-based RBDO. At the initialdesign, which is the deterministic optimum design, 200samples are generated in the local window since there are

no existing samples in it; the coefficient of the local win-dow size is also 1.3. At the second iteration, since thereare 8 samples in the local window, 192 samples are newlygenerated, resulting in spending 392 samples total. At thefifth iteration, since the number of existing samples Ne islarger than the number of required samples Nr , the localwindow size is enlarged and no samples are generated. Fromthe fifth iteration on, the sampling-based RBDO uses exist-ing samples only, which means each design is very close tothe optimum design, and thus the design movement is verysmall.

5.4 Efficiency of parallel computing

To demonstrate the improvement of the efficiency by apply-ing the parallelization to the sampling-based RBDO usingthe D-Kriging, the same M1A1 Abrams tank roadarm exam-ple in Section 5.3 is used. The comparison between the par-allel computing and the original serial computing is carriedout by running the reliability analysis at the deterministicoptimum design with 250 points in the local window. Nine

Table 9 Number of samples used for sampling-based RBDO

Iteration Number of samples Local window size

Existing Generated Total

1 0 200 200 1.3

2 8 192 392 1.3

3 35 165 557 1.3

4 66 134 691 1.3

5 202 0 691 1.5

6 214 0 691 1.5

7 223 0 691 1.5

8 222 0 691 1.5

Sampling-based RBDO using the stochastic sensitivity analysis and Dynamic Kriging method 315

Table 10 Comparison of computational time

Method No. of No. of Surrogate modeling, MCS,

samples constraints sec. sec.

Parallel 250 9 129.73 337.82

Serial 250 9 474.43 1034.79

constraints are identified as active or violated constraints atthe deterministic optimum design and thus surrogate modelsare generated for those nine constraints only.

As shown in Table 10, the parallel computing indeedreduces the computational time by 72.7% for surrogatemodeling and 67.4% for the MCS. The reason that thereduction of the computational time for the surrogate mod-eling and MCS is not exact 8 times compared with the serialone is that there are nine active or violated constraints usedto generate the surrogate models while eight cores are usedfor the parallelization. Therefore, it includes 2 iterationsfor the parallel computing, 1st iteration for the 1st ∼ 8thconstraints and 2nd iteration for the 9th constraint only.

In this example, the 3rd parallelization explained inSection 4.3, which is the parallelization of FEA, is not used;instead, only one client computer is used for the test. If50 client computers are used for this test, computer sim-ulation time for the FEA will be almost 50 times fastersince the data transmission for the parallelization is negligi-ble. Furthermore, the MPP-based RBDO can use maximally13 client computers only since there exist 13 constraintsfor the M1A1 Abrams tank roadarm. On the other hand,the sampling-based RBDO can use as many client com-puters as the number of samples used. The parallelizationof FEA is being tested on the U.S. ARMY TARDEC highperformance computing (HPC) system.

6 Summary

The numerical examples used in Section 5 show that the pro-posed sampling-based RBDO is more accurate and efficientthan the existing methods for low-dimensional problemsand is as accurate and efficient as the existing methods forhigh-dimensional problems when the parallel computing isutilized. However, the current sampling-based RBDO alsohas drawbacks. First, if the target probability of failure isvery small and/or the input COV is very large, the sampling-based RBDO becomes computationally very expensive andmay be inaccurate since the local window becomes verylarge which requires more samples to accurately approxi-mate true responses. Second, if the dimension of problemsis very high, then so called “curse of dimensionality” causesboth inaccuracy and inefficiency in generation of surrogatemodels.

7 Conclusion

Sampling-based RBDO using the D-Kriging method forsurrogate model generation and the score function forprobability of failure and its sensitivity analysis is pro-posed in this study. In addition, to improve the accuracyand efficiency of sampling-based RBDO, numerical strate-gies such as the hyper-spherical local window for surrogatemodel generation, sampling strategy, filtering of constraints,sample reuse and local window enlargement, and adaptiveinitial point for pattern search are proposed as well. Theproposed sampling-based RBDO does not use any approx-imation on the calculation of the probability of failure andits sensitivity except for statistical noise due to the MCS,which can be easily solved by increasing the MCS sam-ple set. Furthermore, the proposed method does not usethe transformation from the original X-space to the stan-dard normal U-space, which makes performance functionsbecome more highly nonlinear, especially when randominputs are correlated. Therefore, the proposed sampling-based RBDO is more accurate than the sensitivity-basedRBDO, which uses approximation and transformation forthe probability of failure estimation once surrogate modelsare sufficiently accurate. The accuracy issue of surrogatemodels is resolved in this paper by the use of the D-Krigingmethod. In addition, to further enhance the efficiency of theproposed method for high-dimensional problems, the par-allel computing is proposed to be used. Even though onlycomponent level RBDO examples are treated in this paper,the proposed sampling-based RBDO can be easily extendedto the system-level RBDO by using the failure set of eitherseries or parallel or mixed system. Numerical examples areillustrated to demonstrate how the proposed sampling-basedRBDO works compared with the sensitivity-based RBDO.The 2-D mathematical example shows that the proposedmethod is more accurate and even more efficient than thesensitivity-based RBDO, which means the proposed methodis very powerful when the dimension of problems is low.For high-dimensional problems such as the M1A1 Abramstank roadarm used in the paper, the sampling-based RBDOstill yields an accurate optimum design. However, it mayrequire more function evaluation in such cases which canbe resolved by parallelizing the computation procedure.

Acknowledgments Research is jointly supported by the AROProject W911NF-09-1-0250 and the Automotive Research Center,which is sponsored by the U.S. Army TARDEC. These supports aregreatly appreciated.

References

Acar E, Haftka RT (2007) Reliability-based aircraft structural designpays, even with limited statistical data. J Aircr 44(3):812–823

316 I. Lee et al.

Annis C (2004) Probabilistic life prediction isn’t as easy as it looks.J ASTM Int 1(2):3–14

Breitung K (1984) Asymptotic approximations for multinormal inte-grals. ASCE J Eng Mech 110(3):357–366

Browder A (1996) Mathematical analysis: an introduction. Springer,New York

Buranathiti T, Cao J, Chen W, Baghdasaryan L, Xia ZC (2004)Approaches for model validation: methodology and illustra-tion on a sheet metal flanging process. SME J Manuf Sci Eng126:2009–2013

Burkardt J, Gunzburger M, Peterson J, Brannon R (2002) User man-ual and supporting information for library of codes for CentroidalVoronoi placement and associated Zeroth, first, and secondmoment determination. Sandia National Laboratories TechnicalReport, vSAND2002–0099

Center for Computer-Aided Design, College of Engineering (1999a)DRAW concept manual. The University of Iowa, Iowa City

Center for Computer-Aided Design, College of Engineering (1999b)DRAW user reference. The University of Iowa, Iowa City

Chan KY, Skerlos SJ, Papalambros P (2007) An adaptive sequentiallinear programming algorithm for optimal design problems withprobabilistic constraints. J Mech Des 129(2):140–149

Chiles JP, Delfiner P (1999) Geostatistics:-modeling spatial uncer-tainty. Wiley, New York

Ditlevsen O, Madsen HO (1996) Structural reliability methods. Wiley,Chichester

Dong J, Choi KK, Wang A, Zhang W, Vlahopoulos N (2005) Para-metric design sensitivity analysis of high frequency structural-acoustic problems using energy finite element method. IntJ Numer Methods Eng 62:83–121

Dong J, Choi KK, Vlahopoulos N, Wang A, Zhang W (2007) Designsensitivity analysis and optimization of high frequency radia-tion problems using energy finite element and energy boundaryelement methods. AIAA J 45(6):1187–1198

Gu L, Yang RJ, Tho CH, Makowskit M, Faruquet O, Li Y (2001) Opti-mization and robustness for crashworthiness of side impact. IntJ Veh Des 26(4):348–360

Haldar A, Mahadevan S (2000) Probability, reliability and statisticalmethods in engineering design. Wiley, New York

Hasofer AM, Lind NC (1974) An exact and invariant first orderreliability format. ASCE J Eng Mech Div 100(1):111–121

Hijab O (1997) Introduction to calculus and classical analysis.Springer, New York

Hohenbichler M, Rackwitz R (1988) Improvement of second-orderreliability estimates by importance sampling. ASCE J Eng Mech114(12):2195–2199

Jin R, Chen W, Simpson TW (2001) Comparative studies of meta-modeling techniques under multiple modeling criteria. StructMultidiscpl Optim 23(1):1–13

Kim C, Choi KK (2008) Reliability-based design optimization usingresponse surface method with prediction interval estimation. JMech Des 130(12):1–12

Kim NH, Wang H, Queipo NV (2006) Adaptive reduction of designvariables using global sensitivity in reliability-based optimization.Int J Reliab Appl 1(1–2):102–119

Lee I, Choi KK, Du L, Gorsich D (2008) Inverse analysis method usingMPP-based dimension reduction for reliability-based design opti-mization of nonlinear and multi-dimensional systems. ComputMethods Appl Mech Eng 198(1):14–27

Lee I, Choi KK, Gorsich D (2010) System reliability-based designoptimization using the MPP-based dimension reduction method.J Struct Multidiscipl Optim 41(6):823–839

Lee I, Choi KK, Noh Y, Zhao L, Gorsich D (2011) Sampling-based stochastic sensitivity analysis using score functions for

RBDO problems with correlated random variables. J Mech Des133(2):21003

Lee SH, Chen W, Kwak BM (2009) Robust design with arbitrary distri-bution using gauss-type Quadrature formula. Struct MultidisciplOptim 39(3):227–243

Lewis RM, Torczon V (1999) Pattern search algorithms for boundconstrained minimization. SIAM J Optim 9(4):1082–1099

Madsen HO, Krenk S, Lind NC (1986) Methods of structural safety.Prentice-Hall, Englewood Cliffs

Martin JD, Simpson TW (2005) Use of kriging models to approximatedeterministic computer models. AIAA J 43(4):853–863

McDonald M, Mahadevan S (2008) Design optimization with system-level reliability constraints. J Mech Des 130(2):21403

Meggiolaro MA, Castro JTP (2004) Statistical evaluation of strain-life fatigue crack initiation predictions. Int J Fatigue 26(5):463–476

Noh Y, Choi KK, Du L (2009a) Reliability based design optimiza-tion of problems with correlated input variables using a GaussianCopula. Struct Multidiscipl Optim 38(1):1–16

Noh Y, Choi KK, Lee I (2009b) Reduction of ordering effect in RBDOusing dimension reduction method. AIAA J 47(4):994–1004

Noh Y, Choi KK, Lee I (2010) Identification of marginal and jointCDFs using Bayesian method for RBDO. Struct MultidisciplOptim 40(1):35–51

Queipo NV, Haftka RT, Shyy W, Goel T, Vaidyanathan R, Tucker PK(2005) Surrogate-based analysis and optimization. Prog AerospSci 41(1):1–28

Rahman S (2009) Stochastic sensitivity analysis by dimensionaldecomposition and score functions. Probab Eng Mech 24:278–287

Rahman S, Wei D (2006) A univariate approximation at most prob-able point for higer-order reliability analysis. Int J Solids Struct43:2820–2839

Rosenblatt M (1952) Remarks on a multivariate transformation. AnnMath Stat 23:470–472

Rubinstein RY (1981) Simulation and Monte Carlo method. Wiley,New York

Simpson T, Lin D, Chen W (2001a) Sampling strategies for computerexperiments: design and analysis. Int J Reliab Appl 2(3):209–240

Simpson TW, Mauery TM, Korte JJ, Mistree F (2001b) Kriging modelsfor global approximation in simulation-based multidisciplinarydesign optimization. AIAA J 39(12):2233–2241

Swanson Analysis System (1989) AnSYS engineering analysis systemuser’s manual, vol. I, II. Swanson Analysis System, Houston

Viana ACF, Haftka RT, Steffen V (2009) Multiple surrogates: howcross-validation errors can help us to obtain the best predictor.Struct Multidiscipl Optim 39(4):439–457

Wei D (2006) A univariate decomposition method for higher-orderreliability analysis and design optimization. Ph. D. Thesis, TheUniversity of Iowa

Yi K, Choi KK, Kim NH, Botkin ME (2007) Design sensitivity anal-ysis and optimization for minimizing springback of sheet-formedpart. Int J Numer Methods Eng 71:1483–1511

Youn BD, Choi KK (2004) A new response surface methodol-ogy for reliability-based design optimization. Comput Struct 82(2–3):241–256

Youn BD, Choi KK, Yang R-J, Gu L (2004) Reliability-based designoptimization for crashworthiness of vehicle side impact. StructMultidiscipl Optim 26(3–4):272–283

Youn BD, Choi KK, Yi K (2005a) Performance moment integra-tion (PMI) method for quality assessment in reliability-basedrobust optimization. Mech Based Des Struct Mach 33(2):185–213

Sampling-based RBDO using the stochastic sensitivity analysis and Dynamic Kriging method 317

Youn BD, Choi KK, Tang J (2005b) Structural durability design opti-mization and its reliability assessment. Int J Prod Dev 1(3/4):383–401

Youn BD, Choi KK, Du L (2005c) Enriched performance mea-sure approach (PMA+) for reliability-based design optimization.AIAA J 43(4):874–884

Yu X, Du X (2006) Reliability-based multidisciplinary optimizationfor aircraft wing design. Struct Infrastruct E 2(3/4):277–289

Zhang T, Choi KK, Rahman S, Cho K, Perry B, Shakil M, HeitkaD (2006) A response surface and pattern search based hybridoptimization method and application to microelectronics. StructMultidiscipl Optim 32(4):327–345

Zhao L, Choi KK, Lee I, Gorsich D (2010) A metamodel method usingdynamic Kriging and sequential sampling. 13th AIAA/ISSMOMultidisciplinary Analysis and Optimization Conference, FortWorth, Texas, September 13–15