Review of uncertainty-based multidisciplinary design optimization methods for aerospace vehicles

Progress in Aerospace Sciences 47 (2011) 450–479

Contents lists available at ScienceDirect

Progress in Aerospace Sciences

0376-04

doi:10.1

n Corr

E-m

journal homepage: www.elsevier.com/locate/paerosci

Review of uncertainty-based multidisciplinary design optimization methodsfor aerospace vehicles

Wen Yao a,n, Xiaoqian Chen a, Wencai Luo a, Michel van Tooren b, Jian Guo b

a College of Aerospace and Materials Engineering, National University of Defense Technology, Changsha 410073, Chinab Faculty of Aerospace Engineering, Delft University of Technology, Delft 2628KS, The Netherlands

a r t i c l e i n f o

Available online 19 July 2011

Keywords:

Uncertainty-based multidisciplinary design

optimization

Uncertainty analysis

Optimization under uncertainty

Reliability-based design optimization

Robust design optimization

UMDO procedure

21/$ - see front matter & 2011 Elsevier Ltd. A

016/j.paerosci.2011.05.001

esponding author. Tel.: þ86 731 84573188;

ail address: [email protected] (W. Yao).

a b s t r a c t

This paper presents a comprehensive review of Uncertainty-Based Multidisciplinary Design Optimiza-

tion (UMDO) theory and the state of the art in UMDO methods for aerospace vehicles. UMDO has been

widely acknowledged as an advanced methodology to address competing objectives of aerospace

vehicle design, such as performance, cost, reliability and robustness. However the major challenges

of UMDO, namely the computational complexity and organizational complexity caused by both

time-consuming disciplinary analysis models and UMDO algorithms, still greatly hamper its application

in aerospace engineering. In recent years there is a surge of research in this field aiming at solving these

problems. The purpose of this paper is to review these existing approaches systematically, highlight

research challenges and opportunities, and help guide future efforts. Firstly, the UMDO theory

preliminaries are introduced to clarify the basic UMDO concepts and mathematical formulations, as

well as provide a panoramic view of the general UMDO solving process. Then following the UMDO

solving process, research progress of each key step is separately surveyed and discussed, specifically

including uncertainty modeling, uncertainty propagation and analysis, optimization under uncertainty,

and UMDO procedure. Finally some conclusions are given, and future research trends and prospects are

discussed.

& 2011 Elsevier Ltd. All rights reserved.

Contents

1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 451

2. UMDO theory preliminaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 452

2.1. Basic concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 452

2.1.1. Uncertainty. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 452

2.1.2. Robustness . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 453

2.1.3. Reliability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 453

2.1.4. Deterministic design optimization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 453

2.1.5. Robust design optimization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 453

2.1.6. Reliability-based design optimization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 453

2.2. General UMDO process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 454

2.2.1. Uncertain system modeling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 454

2.2.2. UMDO procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 454

3. Uncertainty modeling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 454

3.1. Uncertainty classification. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 454

3.2. Uncertainty representation and modeling. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 455

3.2.1. Model input uncertainty and model parameter uncertainty . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 456

3.2.2. Model form uncertainty . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 458

3.2.3. Model error . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 458

3.2.4. Other uncertainties related to UMDO . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 458

3.3. Uncertainty sensitivity analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 459

ll rights reserved.

fax: þ86 731 84512301.

www.elsevier.com/locate/paerosci

dx.doi.org/10.1016/j.paerosci.2011.05.001

mailto:[email protected]

dx.doi.org/10.1016/j.paerosci.2011.05.001

W. Yao et al. / Progress in Aerospace Sciences 47 (2011) 450–479 451

4. Uncertainty propagation and analysis. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 459

4.1. Monte Carlo simulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 460

4.2. Taylor series approximation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 461

4.3. Reliability analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 461

4.4. Decomposition based uncertainty analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 463

5. Optimization under uncertainty . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 464

5.1. Reliability-based design optimization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 465

5.1.1. Worst case analysis method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 465

5.1.2. Corner space evaluation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 465

5.1.3. Variation patterns formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 465

5.2. Robust design optimization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 466

6. UMDO procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 467

6.1. Single level procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 467

6.2. Decomposition and coordination based procedure. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 470

6.2.1. CO-based UMDO procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 470

6.2.2. CSSO-based UMDO procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 471

6.2.3. ATC-based UMDO procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 471

7. Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 472

Acknowledgment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 473

References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 474

1. Introduction

With progress in science and technology, demands for aero-space vehicles are ever increasing to have better performance,higher reliability and robustness, and lower cost and risk. Toeffectively address these competing objectives, designers gener-ally take design and optimization methods with consideration ofall relevant aspects of the vehicle lifecycle from design, manu-facture, operation, to final disposal at the end of life. All throughthe lifecycle in a realistic world, there inherently exist a vastquantity of uncertainties arising from the aerospace vehiclesystem itself, as well as the environmental and operationalconditions it is involved in. Take structural design for example,uncertainties include prediction errors induced by design modelassumption and simplification, performance uncertainty arisingfrom material properties and manufacturing tolerance, and uncer-tainties of load conditions applied on the structure during opera-tion. These uncertainties may cause system performance tochange or fluctuate, or even cause severe deviation and result inunanticipated or even unprecedented function fault and missionfailure. Therefore it is important to take uncertainties intoaccount from the beginning of aerospace vehicle system design.

In traditional design, to account for uncertainties, constraintsimposed on the design are often reformulated with empirical orother predefined factors instead of the ideal ones based on themarginal design philosophy, so as to maintain redundancy of thesystem in face of uncertainties. For example, the ideal stressconstraint is often rewritten by multiplying the actual stress witha safety factor (larger than one) so as to represent the considera-tion of all the potential uncertainties in a lump [1]. The safetyfactor is defined mainly based on past experience and priorknowledge about the system, and up to now there is no straight-forward universal method to appropriately define it. With biggersafety factors, the design and optimization are prone to reachsolutions which are too conservative and over redundant thataccordingly result in weight and cost penalty, whereas withsmaller safety factors the reliability of the system cannot beguaranteed. Furthermore, the past experience in defining safetyfactor for existing structure forms may be inappropriate orobsolete for new structures and thus lead to potential danger.Hence the aforementioned traditional methods of implicitly androughly dealing with uncertainties are far from enough toeconomically improve system performance, robustness and relia-bility. So it is highly motivated to develop more advanced andaccurate analytical approaches based on uncertainty related

mathematical theory to tackle uncertainties systematically andrationally during design. These new approaches are usuallydescribed with the terms as Uncertainty-Based Design (UBD) [2],Non-Deterministic Approaches (NDA) [3], etc., which aim at solvingthe following two issues: (1) improve robustness of aerospacevehicle and decrease its sensitivity to variations, so as to maintainthe stability of performance under uncertainty; (2) enhancereliability of aerospace vehicle and decrease the chance of func-tion failure under potential critical conditions, so as to keep thesystem in normal state with required level of likelihood underextreme event. Corresponding to the two design aims, there arebasically two categories of uncertainty-based design methods,namely robust design optimization and reliability-based designoptimization as shown in Fig. 1. Take random uncertainty forexample, robust design is mainly concerned with the eventdistributed near the mean of the probability density function(small fluctuation around the normal status), whereas reliability-based design optimization is concerned with the event distrib-uted in the tails of the probability density function (extremeevents). These two non-deterministic approaches can also beformulated into one design problem to seek improvements ofthe system both in terms of robustness and reliability.

Uncertainty-based design optimization can be traced back tothe 1950s [4,5], and since then there is a surge of research in thisarea. Lots of research have been devoted to design and optimiza-tion algorithms under uncertainty [6,7] and successful applica-tions in wild fields have been observed, especially in the fields ofaerospace engineering and civil engineering which have stringentregulations on system reliability and robustness [8–11]. Theresearch of uncertainty-based design optimization in aerospaceengineering are mainly focused in disciplines such as structure[12–14], aerodynamics [15–17], and control [18,19], and thecurrent status and barriers are comprehensively surveyed in [2].As these disciplines are naturally close coupled and uncertaintyimpacts are cross propagated, it is much more desirable to take aholistic approach to solve the multidisciplinary design optimiza-tion (MDO) problem so as to enhance the system design byexploiting potential synergistic effect of the coupled disciplines.From this perspective, Uncertainty-Based Multidisciplinary Design

Optimization (UMDO) is introduced into academia.UMDO is a new trend of MDO [2,20]. It can greatly improve

design by benefiting from the synergistic effect of couplingdisciplinary collaboration optimization, and meanwhile enhancereliability and robustness. Being more close to the realistic

Nomenclature

x¼ ½x1,x2,. . .,xnX� random variable vector with nX dimensions

X original design variable spaceU standard normal spacep(x) multi-variate joint probability density function of xpi(xi) uni-variate probability density function of xi

g(x)¼0 limit state functionR reliabilitypf probability of failure:U: L2 norm or Euclidean normF(U) standard normal cumulative distribution functionf(U) standard normal probability density functionb reliability indexyi output vector of discipline i

yij the coupling state vector output from discipline i andinput into discipline j

D failure domainO the universe of uncertain variablesUMDO uncertainty-based multidisciplinary design

optimizationMDO multidisciplinary design optimizationRIA reliability index approachPMA performance measure approachGSE global sensitivity equationMPP most probable pointMCS Monte Carlo simulationMDA multidisciplinary analysisRBDO reliability-based design optimizationRDO robust design optimization

W. Yao et al. / Progress in Aerospace Sciences 47 (2011) 450–479452

systems engineering by systematically taking uncertainties intoconsideration in the design phase, UMDO has attracted wideresearch interest and is now under rapid development. In theNASA white paper which addresses the needs and opportunitiesfor uncertainty-based multidisciplinary design for aerospacevehicles [2], the term multidisciplinary is highlighted as one ofthe key phrases in uncertainty-based design research of NASALangley Research Center. On this fertile ground, rich literature hasbeen published which covers extensive topics including uncer-tainty classification and quantification, multidisciplinary uncer-tainty cross propagation and analysis, approximation methods forreducing calculation burden, optimization under uncertainty, andmultidisciplinary organization of UMDO problems. The successfulapplications of UMDO in aerospace engineering are also reportedwhich strongly demonstrate the efficacy of this new emergingmethodology [21–24].

The scope of this paper is to systematically introduce theUMDO theory and present a comprehensive review of the UMDOmethods. Only the fundamental theory and general UMDOapproaches for aerospace vehicles are covered, and the detailedissues related to algorithms and applications for specific disci-plines are excluded and referred to [2]. This paper is by no meansexhaustive, and we apologize to authors and readers for workthat could not be cited. The rest of the paper is structured asfollows. The UMDO theory preliminaries are firstly introduced.The basic concepts are clarified, and the general solving process ofUMDO problems is explained. Following the UMDO process,detailed literature reviews of all the key steps are expoundedseparately in corresponding sections, which include uncertaintymodeling, uncertainty propagation and analysis, optimizationunder uncertainty, and UMDO procedure. After these reviewsections, the paper is closed with some conclusion remarks,

Fig. 1. Two categories of uncertainty-based design [2]: (a) uncertainty-based design d

wherein future UMDO research challenges and opportunities arehighlighted.

2. UMDO theory preliminaries

2.1. Basic concepts

For convenience of discussion, we begin with the basic con-ceptions and definitions relevant to UMDO in this section.

2.1.1. Uncertainty

The term uncertainty has different definitions and taxonomiesin different research fields. In computational modeling andsimulation, uncertainty is regarded as a potential deficiency inany phase or activity of the modeling process that is due to a lackof knowledge [25]. In systems engineering, uncertainties arethings that are not known, or known only imprecisely [26]. Insome aerospace engineering literature, uncertainty is definedas the incompleteness in knowledge (either in information orcontext), that causes model-based predictions to differ fromreality in a manner described by some distribution function[27]. In another useful functional definition it is defined as theinformation/knowledge gap between what is known and whatneeds to be known for optimal decisions with minimal risks [22].From the perspective of systems engineering which takes thewhole lifecycle into account during the design phase, we give thedefinition of uncertainty as follows.

Definition 1. Uncertainty: The incompleteness in knowledge andthe inherent variability of the system and its environment.

omains and (b) robustness and reliability in terms of probability density function.

Fig. 2. Graphical illustration of RDO.


2.1.2. Robustness

In general, a system, organism or design may be said to berobust if it is capable of coping well with variations (sometimesunpredictable) in its operating environment with minimaldamage, alteration or loss of functionality. In IEEE guideline fornuclear power generating station, robustness is referred to as astatistical result that is not significantly affected by small changesin parameters, models, or assumptions [28]. In some non-deter-ministic design literature, a robust system is defined to berelatively insensitive to variations in both the system componentsand the environment, and the degree of tolerance to thesevariations is measured with robustness [22]. We use the defini-tion from Ref. [22] and state it as follows.

Definition 2. Robustness: The degree of tolerance of the system tobe insensitive to variations in both the system itself and theenvironment.

2.1.3. Reliability

The definition of reliability is quite consistent in differentresearch fields. It is generally defined as the likelihood that anitem will perform its intended function for a specified timeinterval under stated conditions [29,30]. We also use this defini-tion and state it as follows

Definition 3. Reliability: The likelihood that a component (or asystem) will perform its intended function without failure for aspecified period of time under stated operating conditions.

With different mathematical theory to model uncertainties,the likelihood can be quantified with different measures, e.g.probability in probability theory, and belief and plausibility inevidence theory.

2.1.4. Deterministic design optimization

The process of obtaining optimal designs is known as designoptimization. In traditional design optimization, most engineersassume that design variables in the optimization problem aredeterministic for simplification, and do not account for uncer-tainties that inherently exist in the design variables and para-meters, as well as simulation models [31].

Definition 4. Deterministic design optimization: The process ofobtaining optimal designs assuming that all the variables, para-meters, models, and simulations involved in the design problemare deterministic.

For a deterministic design optimization, the mathematicalproblem can be formulated as

find x

min f ðx,pÞ

s:t: gðx,pÞr0

xLrxrxU

8>>><>>>:

ð1Þ

where x is design variable vector, p is system constant parametervector, xL and xU are lower bound and upper bounds of x whichdefine the boundaries of design space, f(U) is the optimizationobjective function, and g(U) is unequal constraint vector.

2.1.5. Robust design optimization

Robust design optimization (RDO) is a systematic and efficientway to meet the challenge of design optimization for performanceand quality [32]. It is widely accepted that robust design is firstlyfounded by Japanese engineer Genichi Taguchi, who develops theTaguchi method to improve the quality of manufactured goodsand makes the product performance insensitive to variation invariables beyond the control of designers [33,34]. With referencefrom [34–36], the definition of RDO is stated as follows.

Definition 5. Robust design optimization: A methodology to opti-mize design which is insensitive to various variations.

The mathematical formulation for RDO with probability theory is

find x

min ~f ðx,pÞ ¼ Fðmf ðx,pÞ,sf ðx,pÞÞ

s:t: gðx,pÞr0

xLrxrxU

8>>><>>>:

ð2Þ

where both x and p could be uncertain, mf and sf are the mean andstandard deviation of the original optimization objective f(U), F(U) isthe reformulated optimization objective function with respect to mf

and sf. The simplest example of F(U) is the weighted sum of themean and standard deviation stated as kmf(x,p)/wmfþ(1�k)sf

(x,p)/wsf, where k is the weighting factor, and wmf and wsf are thescaling factors. By integrating sf into the objective function, mini-mization of system sensitivity to uncertainties can be achieved. Thegraphical illustration of RDO is shown in Fig. 2.

2.1.6. Reliability-based design optimization

Reliability-based design optimization (RBDO) is also referred toas Reliability-based optimization (RBO) [20], which deals withobtaining optimal design and meeting reliability constraints[31]. With reference from [2], we give the definition of RBDO asfollows.

Definition 6. Reliability-based design optimization: A methodologyto optimize design which is reliable with small chance of failureunder predefined acceptable level.

The mathematical formulation for RBDO with probabilitytheory is

find x

min ~f ðx,pÞ ¼ mf ðx,pÞ

s:t: Pfgðx,pÞr0gZR

xLrxrxU

8>>><>>>:

ð3Þ

where P{U} is the probability of the statement within the braces tobe true, and R is the reliability vector specified for the constraintvector. The graphical illustration of RBDO is shown in Fig. 3.

To improve system design in both robustness and reliability, RDOand RBDO can be combined and referred to as reliability-based robust

design optimization (RBRDO), which is formulated as [20,37]

find x

min ~f ðx,pÞ ¼ Fðmf ðx,pÞ,sf ðx,pÞÞ

s:t: Pfgðx,pÞr0gZR

xLrxrxU

8>>><>>>:

ð4Þ

Fig. 3. Graphical illustration of RBDO.

Fig. 4. General flowchart of UMDO.


2.2. General UMDO process

A panoramic view of the general UMDO solving process ispresent in this section to provide an overall understanding of thisnew methodology. UMDO is the methodology that solves theuncertainty-based design optimization of complex systems byfully considering coupling relationship and uncertainty propaga-tion between disciplines involved in the system. For a UMDOproblem, the general solving flowchart is shown in Fig. 4, and themain steps are explained as follows.

2.2.1. Uncertain system modeling

Uncertain system modeling is the first step to mathematicallydescribe the design optimization problem, which consists ofsystem modeling and uncertainty modeling.

2.2.1.1. System modeling. System modeling includes mathematicalmodeling of the system and disciplines, and mathematicalformulation of optimization problems, such as design variables,optimization objectives, constraints, design space, etc., which issimilar to the system modeling of deterministic optimization.

2.2.1.2. Uncertainty modeling. Uncertainty modeling is classifica-tion and quantification of uncertainties involved in the system

design. There are many mathematical theories and methods tomodel uncertainties [27,38], such as probability theory, possibi-lity theory, evidence theory, clouds theory [39], etc. Throughoutaerospace vehicle lifecycle, there exist a vast number of uncer-tainties, which inevitably lead to unacceptable calculationburden. Therefore, it is generally necessary to use sensitivityanalysis to screen out the factors which have no significantinfluence on system design.

2.2.2. UMDO procedure

UMDO procedure is the methodology about how to efficientlyorganize and realize UMDO by programming in computers [40].As shown in the flowchart, the key steps of UMDO proceduremainly include optimization under uncertainty and uncertaintyanalysis.

2.2.2.1. Optimization under uncertainty. This step is the designspace exploration under uncertainty. For large-scale, highlynonlinear, and non-convex problem, the deterministic globaloptimization is already very difficult and time-consuming, and itnaturally becomes even worse with additional efforts to deal withuncertainties, which may lead to prohibitive computation. Hencethe researches of optimization algorithms, as well as the specialtreatments of uncertain objectives and constraints, are essentialto enhance the overall optimization efficiency under uncertainty.

2.2.2.2. Uncertainty propagation and analysis. In this step, theuncertainty characteristics of the system output under impacts ofuncertainties propagated through the system inner mechanismare quantitatively analyzed, so as to further analyze reliability androbustness of the design. Especially for the complex aerospacevehicle system with multi-disciplines, the cross propagation ofuncertainties causes great difficulty to the uncertainty analysis,which is one of the hot issues in the UMDO research.

In the following sections, the aforementioned key steps, exceptsystem modeling which pertains to the specific research object,are thoroughly discussed and surveyed.

3. Uncertainty modeling

Appropriate uncertainty modeling is the premise of uncer-tainty-based design optimization, which includes adopting appro-priate taxonomy to comprehensively identify and classifyuncertainty sources, utilizing suitable mathematical tools torepresent and model these uncertainties, and using sensitivityanalysis approaches to screen out uncertainties with minoreffects on design so as to simplify UMDO problem. These issueswill be studied in this section.

3.1. Uncertainty classification

There are numerous taxonomies in literature to address uncer-tainty classification. The most popular uncertainty taxonomy isfirstly proposed in risk assessment, which classifies uncertaintyinto two general types: aleatory and epistemic. Aleatory uncer-

tainty describes the inherent variation of the physical system orthe environment under consideration. It is also known as varia-

bility, type A, or stochastic uncertainty, which is irreducible becauseit cannot be eliminated by collection of more information or data.Epistemic uncertainty is a potential inaccuracy in any phase oractivity of the modeling process that is due to a lack of knowledge.It is also known as subjective, type B, or cognitive uncertainty, whichis reducible because it can be eliminated with an increased state ofknowledge or collection of more data [41,42]. This taxonomy iswidely accepted and has been applied in lots of fields, including


decision analysis, inference, risk and policy analysis, scientificcomputing, and modeling and simulation [43,44]. The journal‘‘Reliability Engineering and System Safety’’ has special issues(vol. 54, nos. 2–3, 1996, and vol. 85, nos. 1–3, 2004) to discussabout these two kinds of uncertainties [3,45].

For complex system design, uncertainty taxonomy is specifi-cally studied. Thunnissen [46,47] proposed to classify uncertaintiesinto ambiguity, epistemic, aleatory, and interaction. Ambiguity(also called imprecision or vagueness) is about little precision ingeneral communication. Epistemic and aleatory uncertainties aredefined the same as those mentioned above. Interaction uncer-tainty arises from unanticipated interaction of many events ordisciplines, which might be or should have been foreseeable.Padmanabhan [20] defined the main types of uncertainty asvariations in parameters and design variable settings, uncertaintiesrelated to decision making or design problem formulations, andmodeling and numerical errors associated with analysis tools.DeLaurentis [19] and DeLaurentis and Mavris [27] establisheda framework to identify uncertainty types that cause ‘‘model-based predictions to differ from reality’’ in aerospace vehiclesynthesis and design. By analogy to the control model, uncertain-ties are classified into four types: input (the imprecise, ambiguous,or not clearly defined requirements), operational environment(due to unknown or uncontrollable external disturbances), modelparameter (error in mathematical models that attempt to repre-sent a physical system), and measurement (arise when theresponse of interest is not directly computable from the mathmodel). Walton [48] developed a holistic view of primary uncer-tainties over the space system lifecycle and categorized theminto development uncertainty, operational uncertainty, and modeluncertainty.

With computational simulation based design becoming themain tool in modern aerospace engineering, modeling and simu-lation uncertainties are also thoroughly studied. In general, uncer-tainties contributing to simulation prediction uncertainty can becategorized as external uncertainty and internal uncertaintyaccording to whether it is within or outside the system modelboundary [49–51]. The internal uncertainty is concerned with thesimulation model, which is further categorized as model structure

uncertainty and model parameter uncertainty. Model structureuncertainty, also mentioned as non-parametric uncertainty[13,38], is mainly due to assumptions underlying the model whichmay not capture the physics correctly [52]. Model parameteruncertainty, also called as parametric uncertainty, is mainly dueto limited information in estimating the model parameters for a

Fig. 5. Uncertainty sources in th

given fixed model form. Besides these two types of uncertainty,Oberkampf et al. [53,54] further proposed to use the term error tospecifically define the recognizable deficiency in any phase oractivity of modeling and simulation that is not due to a lack ofknowledge and is identifiable or knowable upon examination,such as programming error. In the NASA report of uncertainty-based multidisciplinary design methods for aerospace vehicles,two complementary categorizations of uncertainties are used toaddress computational uncertainties [2]. One categorizationdistinguishes between parameter uncertainties and model formuncertainties. The other one is based on the taxonomy proposedby Oberkampf which classifies the total computational uncer-tainty into variability, uncertainty, and error [53].

Since the UMDO problems of aerospace vehicles discussed inthis paper are mainly concerned with computational simulationbased design optimization which takes lifecycle uncertainties intoaccount, we define the uncertainty taxonomy as follows:

(1)

e sim

Uncertainties are generally categorized into two types: epis-temic and aleatory.

(2)
Sources of uncertainties throughout the aerospace vehiclelifecycle that have influence on UMDO in the design phase canbe classified as follows: (a) In mission analysis phase, uncer-tainties arise from the variability of mission needs andrequirements, government and related agency regulations,science and technology development, funding, missionschedule, political and cultural factors, to name a few. (b) Indesign phase, specifically the simulation-based computationaldesign, uncertainties mainly arise from computational simu-lations. There are three sources contributing to the totaluncertainty of computational simulation, namely model input
uncertainty (external uncertainty), model uncertainty (modelstructure uncertainty and model parameter uncertainty), andmodel error, as shown in Fig. 5. (c) In manufacturing phase,uncertainties arise from human operation error, manufactur-ing tolerance, etc. (d) In operation phase, uncertainties arisefrom operational conditions (environment).These uncertain-ties are shown in Fig. 6.

3.2. Uncertainty representation and modeling

Different approaches should be used to appropriately repre-sent and model uncertainty according to its specific character-istics and information available about it.

ulation-based design.

Fig. 6. Uncertainty sources relevant to UMDO throughout the aerospace vehicle

lifecycle.


3.2.1. Model input uncertainty and model parameter uncertainty

Model input and model parameter uncertainties have differentfeatures in different context. Numerous modeling approacheshave been studied, and the most popular ones include: prob-ability theory, evidence theory, possibility theory, interval analy-sis, and convex modeling. As probability theory has a long history,sound theoretical foundation, and deep root in the research ofnon-deterministic design, it is more prevalent or better known toengineers than other theories. Therefore, the terms non-probabil-

istic or imprecise probability approaches are used to cover allmathematical models which measure uncertainty without sharpnumerical probabilities [55,56].

3.2.1.1. Probability theory. Probability theory represents uncer-tainty as random variable or stochastic process (time-dependent).Here we mainly discuss about random variables. For discreterandom variable x, a sample space is firstly defined which relatesto the set of all possible outcomes denoted by O¼{x1,x2,...}. Eachelement of the sample space xAO is assigned a probability valuef(x) between 0 and 1, and the sum of the probabilities of all theelements in the sample space is equal to 1. The function f(x)mapping a point in the sample space to the ‘‘probability’’ value iscalled probability mass function (pmf). For continuous randomvariable X within the set of real numbers R, a function calledcumulative distribution function (CDF) F(x) exists, defined byF(x)¼P(Xrx), where x denotes a particular realization and P

denotes the probability. F(x) returns the probability of theoccurrence that X will be less than or equal to x. If F(x) is abso-lutely continuous, it can be differentiated with respect to X andyields probability density function (PDF) f(x). For a set EAR, theprobability of the random variable X being in E is

PðXAEÞ ¼

ZxAE

f ðxÞdx ð5Þ

The quantitative measures of random variables, e.g. mean,standard deviation, statistical moments, joint probability proper-ties, etc., can be defined with probability methods. Aleatoryuncertainty is generally modeled as random variable or stochasticprocess by probability theory if information is sufficient toestimate probability distribution. Firstly, certain distributionmodel (Gaussian, Poisson, log-normal, etc.) should be assumed,and then the parameters of the model can be estimated withsufficient data (or other kind of available information) to accu-rately fit its CDF or PDF function. The distribution model can beselected according to the uncertainty characteristics and the

context it is involved in based on past experiences, a prioriknowledge, or expert opinions [57,58]. The parameters of themodel can be estimated with parameter estimation methods, e.g.the method of moments, maximum likelihood method, etc. [59].If the data sample is small, the statistical distribution modelselection and model fitting can be defined by Bayesian inferenceof unbounded Johnson distribution [60]. Probability theory hasbeen widely used in solving non-deterministic design problemsin aerospace engineering [61–65]. But in practice, the applica-tion of this theory usually encounters the problem that therecannot always be sufficient information to develop the probabilitymodel. Especially in the conceptual design phase, there is verylimited available knowledge about the research object, andpast experience, expert opinions and empirical knowledge aregenerally quite subjective. In this situation uncertainties canbe considered to degenerate from aleatory to epistemic (subjec-tive) ones. To deal with epistemic uncertainty, the Bayesian

probability, a special interpretation of probability theory, hasbeen investigated. Bayesian probability interprets the concept ofprobability as a measure of a state of knowledge in contrastto interpreting it as a frequency or a physical property of asystem. It specifies some prior probability subjectively, andthen updates it in the light of new evidence or observationsby means of statistical inference approach, so called Bayesianinference. In this way it can combine pre-existing knowledge withsubsequent available information and update the prior knowl-edge with uncertainties. With the capability of dealing withboth aleotery and epistemic uncertainties, the Bayesian theoryhas been widely applied, especially in reliability engineering(Bayesian reliability analysis and Bayesian reliability-based optimiza-tion) [66–69].

3.2.1.2. Evidence theory. Evidence theory, also called the Dempster–

Shafer theory (D–S theory), is a mathematical theory whichmeasures uncertainty with belief and plausibility determinedfrom the known evidence (information) for a proposition. Thesemeasures can define lower and upper bounds (interval range) ofprobability with consistent evidence instead of assigning a preciseprobability for a proposition, which is very useful when poorknowledge is available about the uncertainties under study [70].The information or evidence to measure belief and plausibilitycomes from a wide range of sources, e.g. experimental data,theoretical evidence, experts’ opinions concerning belief in valueof a parameter or occurrence of an event, etc., and the evidencecan be combined with combination rules [71].

Let O be the universal set representing all possible statesof a system under consideration. The elements of the power set2O can be taken to represent propositions concerning the actualstate of the system. Evidence theory assigns a belief mass toeach element of the power set by a basic belief assignment(BBA) function m:2O-[0.1] which has the following two proper-ties: the mass of the empty set is zero m(|)¼0, and the massof all the member elements of the power set adds up to a totalof 1,

PAA2Om(A)¼1. The mass m(A) expresses the proportion of all

relevant and available evidence that supports the claim that theactual state belongs to A. The value of m(A) pertains onlyto A and makes no additional claims about any subsets of A, eachof which has its own mass. From the mass assignments,a probability interval can be defined which contains the preciseprobability (in the classical probabilistic sense), and the lowerand upper bound measures are belief (Bel) and plausibility(Pl) as Bel(A)rP(A)rPl(A). The belief Bel(A) is defined as thesum of mass of all the subsets of A, which represents the amountof all the evidence supporting that the actual state belongs toA, and the plausibility Pl(A) is the sum of mass of all the setsthat intersect with A, which represents the amount of all the

Fig. 8. Uncertainty representation with fuzzy set theory and possibility theory:

(a) fuzzy set vs. classical set and (b) plot of CPoF and CNF [72].


evidence that does not rule out that the actual state belongs to A:

BelðAÞ ¼X

B9BDA

mðBÞ, PlðAÞ ¼X

B9B\Aa|

mðBÞ ð6Þ

The two measures are related to each other as

PlðAÞ ¼ 1�BelðAÞ, BelðAÞþBelðAÞr1, PlðAÞþPlðAÞZ1 ð7Þ

where A is the complement of A.The evidence space is characterized with cumulative belief

function (CBF) and cumulative plausibility function (CPF) defined as

CBF ¼ f½x,BelðnxÞ� : xAOg, nx ¼ f ~x : ~xAO and ~xrxg

CPF ¼ f½x,PlðnxÞ� : xAOg, nx ¼ f ~x : ~xAO and ~xrxg ð8Þ

CBF and CPF are shown in Fig. 7 with CDF, which clearlyillustrates how belief and plausibility define a probability intervalas lower and upper bounds [72].

Detailed tutorials of evidence theory can be referred to [73].Evidence theory can handle both aleatory and epistemic uncer-tainties flexibly with its evidence combination rules to updateprobability measures. It is actually very close related to prob-ability theory, as an uncertainty representation with this theorycan approach an uncertainty representation with probabilitytheory as the amount of available information increases, whichis very appealing for application in industry [74,75]. However ithas limitations when dealing with highly inconsistent datasources, which may render the evidence combination rule unreli-able. Anyway, it has attracted great research interest and beenwidely applied in uncertainty-based information, risk assessment,decision making, and design optimization [74,76–78].

3.2.1.3. Possibility theory. Possibility theory is firstly introduced byZadeh in 1978 as an extension of his theory of fuzzy set and fuzzylogic, which can be used to model uncertainties when there is littleinformation or sparse data [79]. The term fuzzy set is used incontrast with the conventional set (crisp set) which has fixedboundaries. Let ~A be a fuzzy set, the degree of membership of asingle point x belonging to ~A is denoted using a membership func-

tion m ~A ðxÞ, also called the characteristic function. The comparisonbetween fuzzy set ~A and classical set A is shown in Fig. 8(a). It can

Fig. 7. Plot of CBF and CPF [72].

be seen that the degree of membership can vary between 0 and 1,while for the classical set the membership should either be 0 or 1,so crisp set can be seen as a special case of fuzzy set. Given thepossibility level of 0.4 (which is also referred to as a-cut), thepossible value of uncertain variable x lies between 2.5 and 6.5 as aninterval.

In possibility theory, the membership function is extended topossibility distribution which expresses the degree the analystconsiders that an event can occur. This subjective knowledge isnumerically modeled with a pair (w,r) to characterize uncertainvariable x, where w is the set of possible values for x, and r is afunction defined on w such that 0rr(x)r1 for xAw andsupfrðxÞ : xAwg ¼ 1. The function r provides a measure ofconfidence that is assigned to each element of w and is referredto as possibility distribution function for x. Possibility theoryprovides two measures of likelihood for subsets of w: possibility

and necessity. Specifically, possibility and necessity for a subset nof w are defined by

PosðnÞ ¼ supfrðxÞ : xAng, NecðnÞ ¼ 1�PosðncÞ ¼ 1�supfrðxÞ : xAncg

ð9Þ

nc denotes the complement of n. In consistency with the proper-ties of the possibility distribution function r, Pos(n) provides ameasure of the amount of information that does not refute theproposition that n contains the appropriate value for x, and Nec(n)provides a measure of the amount of uncontradicted informationthat supports the proposition that n contains the appropriatevalue for x. Relationships satisfied by possibility and necessity for


the possibility space (w,r) include

NecðnÞþPosðncÞ ¼ 1, NecðnÞrPosðnÞPosðnÞþPosðncÞZ1, NecðnÞþNecðncÞr1 ð10Þ

For any n, either Pos(n)¼1 or Nec(n)¼0.Similar to probability theory, the possibility space can be

characterized with cumulative necessity function (CNF) and cumu-

lative possibility function (CPoF), as shown in Fig. 8(b). CNF andCPoF are defined as [72]

CNF ¼ f½x,NecðnxÞ� : xAwg, nx ¼ f ~x : ~x Aw and ~xrxg

CPoF ¼ f½x,PosðnxÞ� : xAwg, nx ¼ f ~x : ~xAw and ~xrxg ð11Þ

Both the aleatory and epistemic (mainly vagueness) uncer-tainties can be represented by this theory. The detailed introduc-tion of possibility theory and fuzzy set can be referred to [79,80].Compared to probability theory, possibility theory can be moreconservative in terms of a confidence level. But given enoughinformation about uncertainties and accurate predictive models,probability theory is more advantageous [81–84]. The applicationof fuzzy set and possibility theory in engineering design optimi-zation and decision making can be referred to [85–88]. Possibilitytheory can also be applied along with probability theory con-sidering there may be both types of uncertainty within oneproblem, and the integrated or unified algorithms are studied in[89–91].

3.2.1.4. Interval analysis. Interval analysis is a method developedby mathematicians since the 1950s as an approach to puttingbounds on rounding errors and measurement errors in mathe-matical computation, and thus developing numerical methodsthat yield reliable results. In interval analysis the value of avariable is replaced by a pair of numbers representing the max-imum and minimum values that the variable is expected to take,which is the simplest form to represent uncertainties. Intervalarithmetic rules are used to perform mathematical operationswith the interval numbers, so as to propagate the interval boundsthrough the computational model and get the bounds on theoutput variables. A comprehensive introduction to this theory andits applications can be found in [92–95].

3.2.1.5. Convex modeling. Convex modeling is a more generalapproach proposed by Ben-Haim and Elishakoff in 1990 torepresent uncertainties with convex sets [96]. The convex modelsinclude energy-bound model, interval model, ellipsoid model,envelope-bound model, slope-bound model, Fourier-boundmodel, etc. One of the typical convex descriptions of uncertainparameter vector x¼ ½x1,x2,. . .,xnX

� is ellipsoid model defined byxTWxra, where W is a positive definite matrix and a is a positiveconstant. With this description, the uncertain object denoted by xis an ellipsoid rather than a hypercube defined by the lower andupper bounds on each component of the object. This is reasonableas it is unlikely that the uncertain components are independentwith each other and the bounds on the components of the objectare reached simultaneously. Therefore it is more general to usethe convex model with representation of correlations betweenuncertain components in realistic application. When the convexmodels are intervals, techniques in interval analysis can be used.Convex modeling and application, specifically in reliabilityanalysis and design, can be found in [96–98]. Based on convexmodeling, Info-Gap decision theory is further developed byBen-Haim as a methodology for robust decision making underconditions of severe uncertainty [99,100].

Besides the foregoing five theories, there are numerous otheralternative approaches to represent uncertainties, especially for

epistemic uncertainties, e.g. cloud theory mediating betweenfuzzy set theory and probability distributions [39,101,102], fuzzy

random theory and random fuzzy theory with characteristics ofboth fuzzy set theory and probability theory [103], etc., which arereviewed in [104–106]. There is a special issue of ReliabilityEngineering and System Safety (vol. 85, 2004) dedicated to thisresearch [45].

3.2.2. Model form uncertainty

Model form uncertainty can be characterized by Bayesianapproaches [44,107–109], or through model accuracy assessmentby comparison between simulation results and experimentalmeasurements [43,110]. This process is also called model valida-

tion which determines if the mathematical model of a physicalevent represents the actual physical event with sufficientreliability [111,112]. In uncertainty based design, uncertaintyrepresentation models per se also have model form uncertainties,especially probabilistic models whose distributions are assumedand fitted based on past experience, expert opinions, experimen-tal data, etc. Hence it is also necessary to measure the uncertaintyof the uncertainty model to validate the feasibility of the uncer-tainty representation. To assess whether a specific distribution issuitable to a data-set, the goodness of fit criteria, including thePearson w2 test, the Kolmogorov–Smirnov test, the Cramer–von-Mises criterion, the Anderson-Darling test, etc., can be applied[113–115]. If the data available to test the hypothesis aboutprobabilistic models are very scarce and do not allow definiteconclusions to choose or discard totally one model among others,Bayesian method can be used which is capable of combiningseveral competing probability distribution types together todescribe a random variable [116–118]. More generally, a com-plete Bayesian solution is proposed to average over all possiblemodels which can provide better predictive performance than anysingle model accounting for model uncertainties [119].

3.2.3. Model error

Great research efforts have been devoted to model errorestimation and control in computational simulations. Discretiza-tion errors can be evaluated by grid refinement and time steprefinement studies [110,120,121], discretization error transportequations [122,123], goal-oriented error estimation [124–126],and other priori information [127] and posteriori error estimationmethods [128,129]. Round-off errors can be tested and character-ized by comparing model calculations with advanced computerhardware results. Programming errors are simply mistakes andcan be detected by redundant procedures and double-checking inmodel verification [110]. In general, the discretization error,round-off error and programming error can be estimated bycomparison between numerical results of the programmed simu-lation model and analytical results of the exact mathematicalmodel. This process is also called model verification, whichdetermines if the computational simulation codes implementingthe theoretical model have sufficient accuracy [110].

To sum up, the model form uncertainty and model error can becharacterized by model verification and validation, and detailedstudies can be found in [130–134] and the references therein.

3.2.4. Other uncertainties related to UMDO

The uncertainties from other phases of aerospace vehiclelifecycle can be characterized with appropriate models accordingto their aleatory or epistemic features, specific context understudy, and available information about their quality and quantityof interest. The general clue underlying representation modelselection can be referred to the preceding discussion of thissection.


3.3. Uncertainty sensitivity analysis

Sensitivity analysis (SA) is the study of how the variation(uncertainty) in the model output can be apportioned, qualita-tively or quantitatively, to different sources of variations in themodel or model input [135]. By means of this technique, uncer-tainty factors can be systematically studied to measure theireffects on the system output, so as to filter out the uncertaintyfactors with negligible contributions and reduce UMDO complex-ity (Fig. 9). With this specific aim, sensitivity analysis in thiscontext is also termed Uncertainty Importance Analysis [136].

There are many approaches to address sensitivity analysisunder uncertainty, especially with probability theory. Probabil-istic sensitivity analysis methods mainly include differentialanalysis, response surface methodology, variance decomposition,Fourier amplitude sensitivity test (FAST), sampling-based method,etc. [137–140], which can well deal with aleatory uncertaintiesmodeled with probability theory. A thoroughly comparison studyof these approaches can be found in [141]. Among theseapproaches, sampling-based method is widely used for its flex-ibility and ease of implementation. With the sampling results,different measures and analysis methods can be used to quantifythe contribution of each uncertainty factor, e.g. scatterplots,correlation and partial correlation method, regression and non-parametric regression analysis, rank transformations, statisticaltests for patterns based on gridding, entropy tests for patternsbased on gridding, variance decomposition, etc. [136,137].Considering the calculation burden of global sampling of compu-tationally intensive models, it is desirable to find a balancebetween computational cost and accuracy. To address this pro-blem, Elementary Effects (EE) method is proposed by Morris [142]and improved by Campolongo et al. [143]. This method calculatesa number of incremental ratios for each uncertainty factor, calledElementary Effects (EE), from which basic statistics are computedto derive sensitivity information. For each factor, two sensitivitymeasures are computed: m which assesses the overall influenceof the factor on the output, and s which estimates the non-linear effects and interactions. This method can provide a good

Fig. 9. Sensitivity analysis for importance ranking and screening of uncertain

variables.

Fig. 10. Uncertaint

compromise between accuracy and efficiency, especially forsensitivity analysis of complex models. Little research has beendevoted for sensitivity analysis under epistemic uncertainty. Sofar the approaches are mainly based on sampling methods tostudy incremental effects of uncertain variables on complemen-tary cumulative belief functions and complementary cumulativeplausibility functions with evidence theory [144,145], or based ondifferential analysis methods to analytically derive sensitivity ofplausibility in evidence theory with respect to expert opinionsand uncertain parameters [146]. Guo and Du also proposed to useevidence theory to unify sensitivity analysis method forboth aleotary and epistemic uncertainties [147,148]. Uncertaintyanalysis methods are used to calculate belief and plausibilitymeasures, and the gap between these two measures is regardedas an indicator of the uncertainty effect on the model output. Theselection of appropriate sensitivity analysis method should relyon uncertainty types and the specific problem context. Forexample, sampling based method would be computationallyexpensive for complex system simulation models; but if approx-imation model is used in place of the high fidelity simulationmodel, sampling based method would be efficient as well.

4. Uncertainty propagation and analysis

Uncertainty analysis is concerned with quantifying uncer-tainty characteristics of system output resulted from model inputuncertainties and model uncertainties propagated through com-putational simulation (Fig. 10). Generally uncertainty analysisapproaches can be categorized into two types: intrusive and non-

intrusive [149]. The intrusive type is mainly related to the physics-based approaches which involve reformulation of governingequations and modification to the simulation codes so as toincorporate uncertainty directly into the system. Typical exampleof this type is Polynomial Chaos expansion based approaches,which represent a stochastic process with expansion of orthogo-nal polynomials. The coefficients of the expansion can be definedby substituting the stochastic process with its polynomial chaosexpansion in the original governing equations, which results in acoupled system of deterministic equations to be solved by editingthe existing analysis codes. Multi-dimensional Hermite ortho-gonal polynomials are firstly proposed to represent Gaussianstochastic process by Wiener [150], based on which a spectralstochastic finite element method is developed by Ghanem andSpanos [151] and widely used in various applications, includingstructural mechanics [151], fluid flow [152,153], etc. Xiu andKarniadakis further proposed to represent stochastic process withexpansions from the Askey family of orthogonal polynomials as ageneralization of the Wiener Hermite chaos expansion, which usedifferent subset of Askey family polynomials according to differ-ent property of random variables with different distributions,e.g. Laguerre polynomials for the Gamma distribution, Charlierpolynomials for the Poisson distribution, etc. [154]. In contrast tointrusive approaches, non-intrusive approaches treat computersimulation model as black-box and need no modification to theexisting deterministic simulation codes. So it can be developed for

y propagation.


general use and take the advantage of being applicable to legacycodes. With this merit, the preceding Polynomial Chaos expansionbased methods are also studied to be solved with non-intrusiveapproaches [155,156].

The scope of this section excludes the intrusive approach as itdepends on specific problems and disciplines. Interval and boundanalysis methods based on interval algebra or optimizationmethods to estimate the upper and lower bounds of systemoutputs are not discussed either. Herein we review some widelyused non-intrusive approaches, including Monte Carlo simulationmethod, Taylor series approximation method, and some methodsspecific for reliability analysis. In the end, considering the com-putational difficulty in application of the conventional uncer-tainty analysis methods to UMDO problems, decomposition basedmethods are introduced, which can treat uncertainty crosspropagation among complex coupling disciplines more efficientlyby decomposing the system uncertainty analysis problem intosubsystem or disciplinary level.

4.1. Monte Carlo simulation

Monte Carlo simulation (MCS) methods, also referred to assampling-based methods [157,158], are a class of computationalalgorithms that perform repeated sampling and simulation so asto compute the statistics of the response quantities of interest.Provided sufficient number of samples, MCS methods can givestatistic analysis results with arbitrary level of accuracy. HenceMCS is often used as a benchmark for evaluating the performanceof new uncertainty analysis techniques.

To begin with, the uncertainty analysis problem with prob-ability theory is firstly stated. Denote the computer simulationmodel as y¼ f(x) with y being the simulation output. For simpli-city, only one dimension output problem is discussed which canbe easily extended to multi-dimensional output problem. Assumethe joint probability distribution function of the vector x is p(x)and the universe of the random variables is O. For arbitraryfunction j(y), its expected value is

I¼ EðjðyÞÞ ¼ZOjðf ðxÞÞpðxÞdx ð12Þ

when j(y)¼yk, I is the estimate for the kth statistical moment;when j(y)¼y, I is the mean of y; when j(y)¼1 for yry0 andj(y)¼0 otherwise, I is an estimate of the quantile on thedistribution function of y associated with y0. It is worth notingthat (12) can be calculated with analytical methods in very rarecases in reality as both f(x) and p(x) can seldom explicitly defined,and the integration region is also generally complicated. Lots ofefforts have been devoted to develop approximation approachesto numerically evaluate this integral. Gauss quadratureapproaches [159] and other numerical quadrature and cubaturemethods [160–162] are proposed to approximate the multi-dimensional integral with weighted sum of the integrand valuesat a set of discrete integration points within the integrationregion. Laplace Approximation approach is proposed to approx-imate the integrand with second order Taylor series expansion atits minimum so as to derive the integral [163]. These approachesare comprehensively studied in [164]. Unfortunately, theseapproximate numerical integration approaches are generally onlyefficient and accurate for a special type of problems, e.g. quad-rature based method for polynomial response, and may be notapplicable especially for problems with high dimensional uncer-tainties and complex integrand which has no explicit formula andcan only be calculated with time-consuming simulation analysis,e.g. FEA (Finite Element Analysis) codes. These difficulties withthe traditional numerical integration approaches motivate thedevelopment of simulation based MCS integration methods which

can compute the statistics of the system response by simplyperforming repeated sampling and simulation. The basic MCSprocedure with probabilistic uncertainties includes three steps:

Step 1: In consistency with the assumed distributions, a set ofnS data points are randomly sampled. The random samplingmethods are discussed in [165].

Step 2: For each data point a deterministic simulation isexecuted to get the corresponding system output response andform nS sample pairs [x(i),y(i)].

Step 3: Analyze the samples. The integral in (12) can beapproximated as

I� ~j ¼ 1

nS

XnS

i ¼ 1

jðyðiÞÞ ð13Þ

And the standard deviation of j(y) can be approximated as

s2j �

1

nS�1

XnS

i ¼ 1

ðjðyðiÞÞ� ~jÞ2 ð14Þ

The accuracy of the estimation in (13) can be quantified withthe standard error defined as

err¼ sj=ffiffiffiffiffinSp

ð15Þ

The standard error can be used to assess the accuracy of MCS.From (15) it can be seen that the estimation accuracy is notrelated with the dimension of the problem, which is very appeal-ing for application in large scale uncertainty analysis problems.And it is proportional to 1=

ffiffiffiffiffinSp

, which means the improvement ofaccuracy by one order will result in the increase of samples by100 times. This becomes computational prohibitive especially forcomplex simulation models, and even worse for UMDO problemswhich need iterations of several coupled disciplinary simulationsto reach a consistent system response result. To address thisproblem, several improved MCS methods with different sampl-ing techniques have been developed and proved to be moreefficient than the random sampling method. Among these sam-pling methods, importance sampling, also referred to as ‘‘weightedsampling’’ [166], is pervasively studied as it is expected to reduceerr to zero with carefully selected importance sampling prob-ability density function [167]. The approaches for selection ofoptimum importance sampling function are discussed in[167,168], but generally theoretical optimum functions areunpractical in realistic engineering problems. A compromisemethod is Latin hypercube sampling approach which can improveMCS stability (reduce err) and meanwhile maintain the tractabil-ity of random sampling. It divides the range of each variable intonS disjoint intervals of equal probability and one value is selectedat random from each interval. Then the nS values of each variableare paired randomly (or with certain criterion, e.g. uniformdistributing) to form nS samples for further statistic analysis. Thismethod and its related operation techniques are thoroughlystudied in [141,157]. Cao et al. proposed to use the first-ordersensitivity information of the target response with respect to therandom variables to accelerate MCS estimation convergence as avariance reduction technique [169], and it is observed that thissensitivity enhanced method can improve accuracy by one orderof magnitude compared to err in (15). The aforementionedvariance reduction techniques are especially important whenMCS is applied to estimate small failure probability, which willbe discussed in detail in Section 4.3.

The MCS methods for other uncertainty types are referred to[170] for evidence theory, [90] for possibility theory, [171,172] forinterval analysis, and there is a comprehensive discussion of MCSmethods with different uncertainty theories in [72,173].

Fig. 11. Reliability analysis.


4.2. Taylor series approximation

Taylor series approximation methods can be employed toapproximate statistical moments of system output based onpartial derivatives of the output f with respect to the elementsof the random input vector x. The original simulation modelfunction y¼ f(x) can be approximated with the first-order Taylorseries as

yðxÞ � f ðx0ÞþXnX

i ¼ 1

@f ðx0Þ

@xiðxi�xi0Þ ð16Þ

where x0 is the base point vector at which the derivatives arecalculated. Based on (16) the output uncertainty resulting fromthe random input uncertainties can be determined with uncer-tainty propagation through this approximation formula, and themean and standard deviation of the output can be estimated as

my ¼ EðyÞ � f ðx0ÞþXnX

i ¼ 1

@f ðx0Þ

@xiEðxi�xi0Þ ¼ f ðx0Þ

sy ¼

ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiXnX

i ¼ 1

@f ðx0Þ

@xi

� �2

s2xiþ2

XnX

i ¼ 1

XnX

j ¼ iþ1

@f ðx0Þ

@xi

@f ðx0Þ

@xjCovðxi,xjÞ

vuut ð17Þ

where Covðxi,xjÞ is the covariance between the components of theinput vector. If the input vector components are uncorrelated, thestandard deviation can be simplified as

sy ¼

ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiXnX

i ¼ 1

@f ðx0Þ

@xi

� �2

s2xi

vuut ð18Þ

For UMDO problem, considering the coupling relationship ofdisciplines, Gu et al. [174] and Cao and Duan [175] proposed toemploy the first-order Taylor series approximation combinedwith global sensitivity equations (GSE) to analyze system outputuncertainty with cross uncertainty propagation between disci-plines, specifically for worst case uncertainties [174] and convexmodel uncertainties [175]. Consider a UMDO problem with nD

coupled disciplines. Denote the simulation model (contributinganalysis tool) of discipline i as Ti and the output vector ofdiscipline i as yi. The input of Ti includes both the design variablevector x and the coupled state variable vectors from otherdisciplinary outputs which are denoted as yUi¼{yj}(ja i). yi canbe calculated as yi¼Ti(x,yUi). Considering uncertainties in thedesign variable x with variation Dx and the analysis tool Ti withbias error DTi, the output uncertainty can be estimated as

Dy1

Dy2

^

DynD

8>>>><>>>>:

9>>>>=>>>>;¼

dy1dx

dy2dx

^dynD

dx

8>>>>><>>>>>:

9>>>>>=>>>>>;UDx

þ

I1 �@T1@y2� � � �

@T1@ynD

�@T2@y1

I2

^ & ^

�@TnD@y1

� � � InD

26666664

37777775

�1

U

DT1ðx,yU1Þ

DT2ðx,yU2Þ

^

DTnDðx,y

UnDÞ

8>>>><>>>>:

9>>>>=>>>>;ð19Þ

Du and Chen also proposed a system uncertainty analysismethod (SUAM) by means of Taylor approximations and sensi-tivity analysis to estimate the mean and variance of systemoutput subject to both parameter and model uncertaintiesin multidisciplinary systems [176]. The derived equation forvariance estimation of coupled state variables is essentially thesame with (19).

Taylor series approximation methods have several downsides:(1) its inherent local nature makes the estimation accuracy poor

as the coefficients of variation (defined as the standard deviationdivided by the mean) of the input random vector increase [149];(2) the increase of Taylor series expansion order leads to rapidincrease of estimation complexity as higher-order terms andcorrelations between the elements of x are involved [177]; and(3) the determination of partial derivatives could be very difficultfor complex system simulation models [141]. However, Taylorseries approximation methods have been widely used for therelative ease of understanding and implementation.

As Taylor series approximation methods only deal with thepropagation of first two moments rather than the exact distribu-tion of the randomness, it belongs to first-order, second-moment(FOSM) methods which are related to the class of problems onlyconcerning the means and variances and their propagation [178].This is a logical naming convention for the uncertainty propaga-tion techniques with a given choice of the order of approximationand the statistical moment to be used [179]. Besides Taylorseries approximation methods, there are also several other FOSMapproaches such as point-estimate-for-probability-moment(PEPM) methods [180], which are reviewed in [178].

4.3. Reliability analysis

Reliability analysis of constraint g(x)r0 at design A is todetermine the probability of failure pf with impact from uncer-tainties involved in the system and its operational environment,which is shown in Fig. 11 with two uncertain variables and alinear constraint.

The probability of failure can be calculated with the integralas [181]

pf ¼

ZD

pðxÞdx ð20Þ

The failure domain D is defined as g(x)40. Reliability R of thesystem is given by R¼1�pf. This integral is generally difficult tocalculate analytically as both the joint probability distributionfunction p(x) and the failure domain D are seldom accuratelydefined in an explicit analytical form, and the multidimensionalintegration can be computationally prohibitive especially for thecomplex system with time consuming analysis models. Hence, itis motivated to develop various approximation methods, includ-ing the preceding numerical integration methods mentioned inSection 4.1, as well as other integration approximation methodsspecific for reliability analysis, e.g. Laplace multidimensionalintegral method based asymptotic approximation [182], maindomain of failure coverage based integration [183], fast Fouriertransform (FFT) based method [184,185], tail modeling approach[186,187], dimension-reduction (DR) methodology [188–190],First Order Reliability Method (FORM) and Second Order

Fig. 12. MPP.

Fig. 13. FORM reliability analysis.


Reliability Method (SORM), etc. Among these approximationmethods, FORM and SORM are most prevailing and wide appliedin engineering problems. A full theory exists for FORM and SORMin standard space of independent standard normal variables andin original space [191], and several variants have been proposedto enhance the algorithm efficiency [192,193].

FORM and SORM methods generally include three steps.Firstly, the original non-Gaussian random variable vector x istransformed into an uncorrelated Gaussian random variablevector u with zero mean and unit variance in the standard normalspace U by Rosenblatt transformation [194]. Denote the transfor-mation as x¼T(u). The integral (20) is rewritten as

pf ¼

ZDu

fðuÞdu ð21Þ

where f(u) is the joint standard normal distribution densityfunction, and Du is the failure domain in the U space defined bylimit state function G(u)¼g(T(u))¼0.

Secondly, the Most Probable Point (MPP, most likely failurepoint, design point or check point) which is of maximum prob-ability density on the limit state function is searched, which is thekey step of FORM and SORM methods (Fig. 12). Generally MPPcalculation can be formulated as an optimization problem as

minu

:u:

s:t: GðuÞ ¼ 0

(ð22Þ

The optimum of (22) is denoted as u*. It can be solved byspecific iterative algorithms, e.g. HL-RF (Hasofer, Lind, Rackwitz,and Fiessler) method [195,196], or general constrained optimiza-tion algorithms, e.g. gradient based method, the augmentedLagrangian method, sequential quadratic programming, penaltymethod, etc. [197]. For non-convex limit state function, branchand bound strategy can be used to search MPP efficiently [198].

Yang et al. studies the convergence analysis of MPP searchiterations based on chaotic dynamics theory [199], which treatsthe iteration formulation as a multi-dimensional discrete dyna-mic equation. It is observed that complicated dynamics phenom-ena, such as periodic oscillation, bifurcation, and chaos, happen inthe test MPP search problems, which indicates that it is normal toencounter convergent failure with certain limit state functions.It is also concluded that there is no simple relationship betweenthe curvature value at the design point of the limit state functionand the convergence of FORM iteration, but the Lyapunov exponentof nonlinear map corresponding to the limit state function can beused to quantitatively describe the convergence feature of theiterative MPP search computations.

Thirdly, the limit state function is approximated with first- orsecond-order approximation at MPP, and the probability of failureis estimated using the approximate limit state function. FORM fitsa tangent hyperplane to the limit state hypersurface at MPP (first-order Taylor series expansion) as shown in Fig. 13, and probabilityof failure can be estimated as

Pf �Fð�bÞ ð23Þ

where b is the reliability index (safety index) defined as b¼ :un:,and F(U) is the standard normal cumulative distribution function.In reliability based optimization, by comparison betweenreliability R¼1�Pf and the desired reliability (target reliability)RT, the probabilistic constraint can be assessed as whether thereliability requirement has been achieved, and this method is calledreliability index approach (RIA). But this RIA method convergesslowly, or even fails to converge for a number of problems[200,201]. Furthermore, in reliability based optimization, it is notnecessary to calculate the exact reliability for each iteration pointduring optimization search, and only the judgment of whether thetarget reliability has been achieved is enough. Hence, an alternativeapproach, Performance Measure Approach (PMA), is proposed[202,203]. In PMA, with the constraint failure defined as g(x)40,the reliability analysis is formulated as the inverse of reliabilityanalysis in RIA, and stated as

maxu

GðuÞ

s:t: :u:¼ bT

8<: ð24Þ

where bT is the reliability index corresponding to the desiredreliability (target reliability) RT, and the optimum point on thetarget reliability surface is denoted as MPP un

b ¼ bT. If Gðun

b ¼ bT)40,

the reliability requirement is not satisfied. Unlike RIA, only thedirection vector of un

b ¼ bTneeds to be determined by exploring the

spherical equality constraint :u:¼ bT . Several methods can be used

to solve this optimization problem, such as advanced mean value(AMV) approach for convex performance functions, conjugate meanvalue (CMV) approach for concave performance functions, hybridmean value (HMV) and Enhanced HMV (HMVþ) methods for bothconvex and concave performance functions [203,204], steepestdecent direction and arc search based algorithm for general non-concave and non-convex functions [205], etc. The comparativestudies of RIA and PMA methods show that PMA has several majoradvantages over RIA in terms of numerical accuracy, simplicity, andstability [200,201,206]. Besides PMA which essentially employs theinverse reliability strategy [207,208], there are also other inversereliability measures introduced in recent years as alternate mea-sures of safety to improve computational efficiency of reliability-based design optimization. These measures and the correspondinganalysis approaches are surveyed in [209].

For highly nonlinear limit state function, the first-order approxi-mation is not sufficient to estimate accurately, so the limit statehypersurface is approximated by a quadratic hypersurface in the

Fig. 14. Shift of sampling distribution with importance sampling methods.


SORM method to obtain a more refined estimation, and the prob-ability of failure can be estimated as

Pf �Fð�bÞYnX�1

i ¼ 1

ð1�bkiÞ�1=2

ð25Þ

where ki is the main curvature of the hypersurface at MPP. Thereare also other alternative formulations to estimate Pf with SORMconsidering the balance of accuracy and computational efficiency[210,211].

Based on FORM and SORM methods, high-order reliabilitymethod (HORM) [212], second-order third-moment method[213], and higher moment methods [214] are developed toimprove accuracy. The method to generate cumulative distribu-tion function (CDF) of system output based on FORM is alsodiscussed in [215]. The accuracy and applicable ranges of FORMand SORM are comprehensively studied in [193,216].The asymp-totic behavior of FORM and SORM is thoroughly analyzed in [210],and the results show that only SORM gives an asymptoticapproximation for the integral in the sense of asymptotic analysis,whereas FORM produces an uncontrollable relative error; but ifthe generalized reliability index [217] is used, asymptotic approx-imation can also be obtained by FORM [210]. However, FORM ismore popular in application for its computational efficiency.

Besides the foregoing numerical approximation methods toestimate reliability, MCS method is another good choice as it iseasy to implement, and flexible for any type of distribution andany form of constraint function. Define the indicator functionI[g(x)40] such that I[U]¼1 if x is in D and zero otherwise. Then(20) can be rewritten as

pf ¼

ZO

I½gðxÞ40�pðxÞdx ð26Þ

An unbiased estimator of (26) by means of MCS with mutuallyindependent sample data pairs [x(i),y(i)] of size nS can be given as

pf � ~pf ¼1

nS

XnS

i ¼ 1

I½gðxðiÞÞ40� ð27Þ

Denote x(i)¼ I[g(x(i))40]. {x(i)} is a sequence of nS independent0/1 experiments, and x(i) is a single Bernoulli trial with occurrenceprobability of 1 as pf. The expectation of x(i) is m¼E(x(i))¼pf, andthe standard deviation is s¼

ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiVarðxðiÞÞ

q¼

ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffipf ð1�pf Þ

p. Denote

z1�u/2 as the 1�u/2 quantile of the standard normal distribution.Given the probability (also referred to as confidence level) 1�u,the error bound e of ~pf�pf

�� is

e¼ z1�u=2sffiffiffiffiffinSp ¼ z1�u=2

ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffipf ð1�pf Þ

nS

sð28Þ

and the percentage error ~e ¼ e=pf is

~e ¼ z1�u=2

ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffið1�pf Þ

nSpf

sð29Þ

From (29) it can be seen that given the required confidence leveland percentage error requirement, nS would be very large and resultin unaffordable computational burden when pf is very small. Tosolve this problem, several efficient sampling techniques are devel-oped, e.g. importance sampling [218–221] and its variant adaptivesampling [222], subset simulation [223], line sampling [224–226],directional simulation [227–229], etc. Among these sampling tech-niques, importance sampling (IS) is most frequently applied. Thebasic idea of IS is to extract more information from the samplepoints by taking samples in the vicinity of MPP and withinthe failure domain D, so the sampling distribution is shifted fromthe original one to MPP (the mean value shifted to MPP) by means ofimportance sampling probability density function, as shown inFig. 14. At MPP the probability of failure is about 50%. From (29)

we can deduce that given the same percentage estimation error andconfidence level, the ratio between the sampling number of theoriginal distribution and that of the shifted distribution is 100 forpf¼1% and 1000 for pf¼0.1%, which clearly indicates that impor-tance sampling approaches can reduce computational expensesefficiently [230]. The aforementioned sampling methods are thor-oughly studied and compared in [231], which concludes that MCSwith these variance reduction sampling techniques are more accu-rate and robust than FORM/SORM, and subset simulation and linesampling are especially advantageous in terms of efficiency andaccuracy in solving highly dimensional problems with multiplefailure domains and irregular limit state functions.

To further improve reliability analysis efficiency, responsesurface methodology (RSM) can be utilized to replace the com-putationally expensive accurate function so as to reduce calcula-tion burden [232–235]. The reliability analysis approachesdiscussed above are time-independent and concerned withrandom uncertainties. Reliability analysis with other uncertaintytheories are also studied, e.g. interval analysis [92,236], possibilitytheory [87,237–239], evidence theory [240], convex uncertainty[97,241]. Time-variant reliability analysis is referred to [242,243].Besides the methods to determine exact reliability, there are alsosome approaches dealing with reliability bounds [244–247].So far the methods introduced can be directly applied to singleconstraint or component reliability analysis. For complex systemconsisting of multiple components, constraints, and failure modes,matrix-based system reliability method [248–250], ComplementaryIntersection Method (CIM) [251,252], and other efficient compo-nents integration method [253–255] have been proposed tosystematically analyze both the component and system reliability.For more detailed reliability analysis methods, readers are referredto reviews in [256,257].

4.4. Decomposition based uncertainty analysis

For the complex system with close coupled disciplines, it isextremely time-consuming to run a multidisciplinary analysis(MDA) for one design as it generally involves several iterationsto converge to a consistent system output. Hence it becomescomputationally prohibitive for those uncertainty analysis meth-ods which entail lots of repeating MDA, e.g. Monte Carlo methods,FORM/SORM, etc. To solve this problem, decomposition strategiesare proposed to decompose the uncertainty analysis problemnested with MDA into several discipline or subsystem uncertaintyanalysis problems, so as to control each subproblem withinacceptable level and meanwhile take advantage of distributedparallel computing. In this field, Du and Gu have contributed a lotand proposed several efficient decomposition based uncertaintyanalysis methods. In [176,258,259], a concurrent subsystemuncertainty analysis method (CSSUA) for uncertainty propaga-tion is proposed. For a system with nD disciplines, the mean and


standard deviation of each subsystem output are firstly calculatedsimultaneously with each disciplinary contributing analysis,and denoted as mn

yiand sn

yifor discipline i. In each disciplinary

uncertainty analysis, the mean and standard deviation of coupledstate variables from other disciplines are passed down fromsystem level with presumed values and denoted as myj and syj

(ja i). Then based on the results of disciplinary uncertaintyanalysis, the compatibility of mean and standard deviation ofcoupled state variables is achieved by a system level optimizationstated as

find : myi,syiði¼ 1,. . .,nDÞ

min :XnD

i ¼ 1

½:myi�mn

yi:þ:syi

�sn

yi:�

ð30Þ

After system optimization, myi and syi are updated and passeddown to disciplinary uncertainty analysis again to obtain new mn

yi

and snyi

. The two steps iterate until convergence is achieved,and the mean and standard deviation of system outputs can beobtained. Based on CSSUA, a modified concurrent subsystemuncertainty analysis (MCSSUA) approach is further proposed in[260], which only use the strategy of concurrent subsystemanalysis to obtain mean values of coupled state variables, so thatthe design variables in the system level optimization can bereduced by half and the analysis efficiency can be improved. Afterthe mean values are obtained, the standard deviation of thesystem outputs can be calculated with analytical approach SUAM[176] introduced in Section 4.2.

Gu and Renaud [261] developed an Implicit UncertaintyPropagation (IUP) method to provide estimation of system outputuncertainties within the bilevel collaborative optimization (CO)framework. The variations of both the design variable vector andthe contributing analysis tools (bias errors) are considered, andthe coupling effect is treated by employing GSE. The varianceof the coupled state variables are estimated with first-orderTaylor series approximation method similar to that stated in(19), except that the matrix entries of GSE in IUP are calculated ineach subsystem simultaneously, which are not consistent untilconvergence is obtained. This is why it is call implicit uncertaintypropagation, in contrast to the direct/explicit calculationapproach. Based on IUP, a CO procedure accommodating uncer-tainties is developed and expounded in Section 6.2.1. The IUPapproach is also successfully applied to the simultaneous analysisand design (SAND) procedure with uncertainty [262]. The accu-racy of IUP is investigated in [262] and the results show that if thecurvature of the disciplinary design space is small, or the incon-sistency between subsystems at the current design point is small,IUP will provide a reliable estimation of propagated uncertainty.In other words, the IUP method should not be employed in theearly stage of uncertainty design optimization when the disci-plines are far from consistent, and it is recommended that atraditional deterministic formulation should be used in the earlyiterations until the disciplinary compatibility degree is goodenough, and then solve the UMDO problem with IUP.

For MPP based uncertainty analysis, considering that the MPPsearch procedure is essentially a double loop algorithm whichincludes a MPP search optimization in the outside loop and aMDA iteration procedure in the inner loop, several decompositionbased approaches have been proposed to improve the MPP searchefficiency. Du and Chen [263,264] proposed to utilize CO toorganize MPP search with concurrent operations on the disciplinelevel, and it is tested in FORM which can obtain a five-fold reduc-tion in computational expense [265]. The same idea is proposedby Padmanabhan et al. to employ Concurrent Subspace Optimiza-tion (CSSO) procedure to solve the MPP search optimizationproblem, so called MPP-CSSO, so as to greatly improve efficiency

with parallelization of disciplinary analysis and optimization[266]. Another idea to address the double loop problem isproposed by Ahn et al. to decompose MDA from the MPP searchand organize them sequentially as a recursive loop [267]. In thissequential approach to reliability analysis for multidisciplinarysystems (SARAM), concurrent subsystem analysis can be appliedin the separate MDA based on GSE to further alleviate computa-tional burden.

For simulation based reliability analysis, a Markov ChainMonte Carlo methodology named Gibbs sampling [268] is utilizedto decompose MDA into disciplinary subproblems as it does notenforce multidisciplinary system consistency at each run, but relyon the sampling process to produce compatibility within thedisciplines gradually. Without consistency constraint on MDA,only the number of disciplines times the disciplinary analysiscomputation are needed for each run of sample simulation, whichcan greatly reduce calculation cost compared to the traditionalsampling method that needs iterations of disciplinary analysis toobtain a consistent system response at each sample. The testindicates that this approach can result in a nine-fold improve-ment over traditional MCS in the demonstration case [265]. Formore detailed discussion on decomposition based uncertaintyanalysis, a survey can be found in [265].

5. Optimization under uncertainty

Being one of the most important issues in optimization,optimization under uncertainty has experienced rapid develop-ment in both theory and application. The first mathematicalformulation is stochastic linear programming which is firstlyintroduced in the middle of last century to address optimizationproblems with parameter randomness [4,269]. Later, to meetthe need for dealing with optimizations in a realistic world whichusually involve discrete integers and nonlinear nature, there isa surge in programming approaches to address these issues,e.g. stochastic integer programming [270,271], stochastic non-linear programming [272,273], robust stochastic programming[274,275], stochastic dynamic programming [276,277], etc.,which are generally grouped together and referred to as stochastic

programming [278,279], and the wide research bibliography inthis field can be found in [6]. To address non-probabilisticuncertainties in optimization, fuzzy programming is also quicklydeveloped [280–282], as well as the programming method forhybrid random and fuzzy uncertainties [283,284]. To unify thestochastic programming and fuzzy programming, Liu proposed touse the term uncertain programming to describe the general partof the optimization theory for uncertain systems [284]. Differentfrom the preceding uncertainty programming methods whichformulate the optimization constraints as reliability constraints(chance constraints), there is another type of approach, so calledrobust optimization, which formulates the optimization problemto search the robust feasible optimal solution which can satisfyconstraints for all possible realizations of uncertainties in thegiven uncertainty set [285–287].

To solve the different formulations of optimization underuncertainty introduced above, several search algorithms havebeen studied and can be generally categorized into two types:(1) gradient based, such as the Robbins–Monro algorithm, and(2) gradient free, such as finite difference based stochasticapproximation, random direction search, genetic algorithm, simu-lated annealing algorithm, etc. [288,289]. Specifically for simula-tion based optimization wherein the system output is calculatedby complex computer simulation rather than analytical equa-tions, the applicable simulation based algorithms are studied in[290,291].


Details of the mathematical theory and algorithms of optimi-zation under uncertainty are not covered here. The scope of thissection is to review two special formulations which are of greatinterest for both academia and industry, namely reliability-baseddesign optimization and robust design optimization.

5.1. Reliability-based design optimization

In reliability-based design optimization (RBDO), or reliability-based optimization (RBO), the central task is to optimize theobjective and meanwhile maintain the failure probability ofconstraints within an acceptable level. The reliability of eachconstraint can be analyzed with reliability analysis methodsdiscussed in Section 4.3. As the complexity of reliability analysisis prohibitive especially for large systems, there are alternativeapproximation methods which translate the constraints withuncertainty into quasi-deterministic constraints with simplifica-tions and assumptions, so as to obtain a balance betweencomputational cost and accuracy. Such methods are also widelyused in the robust design optimization where the feasibility ofconstraints should be maintained [230]. Herein some prevalentmethods are introduced.

Fig. 15. Corner space evaluation method.

5.1.1. Worst case analysis method

The worst case analysis method is firstly proposed by Parkin-son et al. [292], which presumes that all fluctuations may occursimultaneously in the worst possible combinations. For constraintg(x,p)rb, the effect of variations of x and p on the constraintfunction is estimated from the first-order Taylor’s series as

Dgðx,pÞ ¼Xn

i ¼ 1

@g

@xiDxi

��þXm

i ¼ 1

@g

@piDpi

�� ð31Þ

where Dxi and Dpi are the variation range (tolerance) of the ithcomponent of x and p, respectively. The total variation of theconstraint is

D¼Dgðx,pÞ�Db ð32Þ

To maintain the constraint value of the design within the saferegion, the constraint is reformulated as

gðx,pÞþDrb ð33Þ

It can also be viewed that the safe region has been reduced toaccommodate worst case variation.

For uncertainties defined with statistical characteristics, thesecond moment (standard deviation) is used to characterize theoutput performance of the constraint function as

sgðx,pÞ ¼

ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiXn

i ¼ 1

@g

@xisxi

� �2

þXm

i ¼ 1

@g

@pispi

� �2vuut ð34Þ

The total statistical standard deviation of the constraint is

s¼ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffis2

gðx,pÞ þs2b

qð35Þ

The constraint is reformulated as

gðx,pÞþksrb ð36Þ

where k is a constant chosen by the designer that reflects thedesired reliability level. For example, k¼3 means that the relia-bility is 99.87% with normal randomness.

This worst case formulation tends to yield conservative solu-tion as it is unlikely that the worst cases of variable or parameterdeviations will simultaneously occur. Besides, the accuracy ofTaylor series approximation is also very limited. However, it stillhas been applied widely for its simplification.

5.1.2. Corner space evaluation

The corner space evaluation method, proposed by Sundaresanet al. [293], is quite similar to the worst case analysis method. Forconstraint g(x,p)rb, assume the design variable vector x withnominal value xt and tolerance Dx, and the parameter vector pwith nominal value pt and tolerance Dp. The tolerance space T isdefined as a set of points close to the design point with nominalvalue wherein each point represents a possible combination dueto uncertainties in each variable:

Tðxt ,ptÞ ¼ fx : 9xt�x9rDx,p : 9pt�p9rDpg ð37Þ

Define the corner space W consists of the corner vertices of thetolerance space T as

Wðxt ,ptÞ ¼ fx : 9xt�x9¼Dx,p : 9pt�p9¼Dpg ð38Þ

To maintain the design solution within the safe region underany variation, the tolerance space should be kept within the saferegion by keeping the corner space always touching the originalconstraint boundary. Then the equivalent constraint can beformulated as

Maxfgðx,pÞ, 8x,pAWðxt ,ptÞgrb ð39Þ

This approach is shown in Fig. 15 for a two-dimensionalproblem. For the normal distributed random variables and para-meters, the tolerance can be chosen as three standard deviationsso as to obtain a reliability of 99.87%. The great advantage of thismethod compared to the worst case analysis method is that itdoes not require partial differential calculations of the constraintfunction.

5.1.3. Variation patterns formulation

Based on the corner space evaluation method, ManufacturingVariation Patterns (MVP) method is proposed with considerationof the coupled variation relationship [294], and is given a moregeneral name as variation patterns formulation in [230] as thismethod is not limited to the manufacturing related problems.

Define the Manufacturing Variation Pattern (MVP) under theconfidence level (1�a) as MVP(1�a). The shape of MVP(1�a) isdetermined by the distribution of random variables and para-meters, and the size of MVP(1�a) is determined by the con-fidence level. For example, for the problem with two normallydistributed dependant variables, the shape of the pattern is anellipsoid as shown in Fig. 16, and the equivalent constraint can bestated as

gðx,pÞrb 8x,pAMVPð1�aÞ ð40Þ

This method is more accurate than the corner space evaluationmethod, but it is quite complicated if the shape of the pattern isirregular.

Similar to the foregoing methods to convert reliability con-straints into equivalent deterministic constraints to maintain

Fig. 16. Variation patterns evaluation method.


designs within feasible region, Shan and Wang proposed theconcept of reliable design space (RDS) which is identified byreformulation of constraints as boundaries to clearly distinct thesafe region, and within RDS the RBDO can be solved as determi-nistic optimization problems. The constraints are reformulated tomaintain the estimated inverse MPP with predefined reliabilityindex located in the safe region, and the inverse MPP is estimatedwith direction cosine of the constraints at the design variablevector in the standard normal space [295].

The preceding approaches do not accurately calculate the relia-bility of constraint, but only formulate the constraint with uncer-tainty into its quasi-deterministic counterpart so as to approximatelymaintain optimization designs within the feasible region withrequired reliability. Therefore generally the optimization solutionswith these methods should be confirmed with reliability analysismethods in the end to ensure that reliability requirement can besatisfied. If reliability analysis is directly incorporated into optimiza-tion to calculate reliability of each constraint at each optimizationiteration point, RBDO becomes a double loop optimization problemwith optimization in the outer loop and iterative reliability analysisin the inner loop. To improve computational efficiency, severalapproaches have been proposed to convert this double loop algo-rithm into a single loop architecture, which will be discussed indetail in Section 6.1 where the single level optimization procedure isexpounded. To further alleviate computational burden, approxima-tion methods can be used to either replace the high accuracy simu-lation model or approximate the limit state function [296–299].These computational methods to solve RBDO problems are sur-veyed in detail in [7]. For non-normal probabilistic uncertaintiesand other types of uncertainties, e.g. evidence theory, possibilitytheory, etc., readers are referred to [66,300–303]. To sum up, withRBDO the reliability of the solution can be effectively improvedwhich is demonstrated in [304,305].

5.2. Robust design optimization

Robust Design Optimization (RDO) is the methodology con-cerned with optimization of system mean performance andminimization of its sensitivity to uncertainties. It has a deep rootin robust design and quality engineering which aim at developinglow-cost and high quality products and processes by makingproducts insensitive or ‘‘robust’’ to the effects of natural variationsin their production and operational environments. This methodwas firstly pioneered by Genichi Taguchi in the 1950s, which isalso referred to as Taguchi method, and has been widely used toimprove product quality [306]. The kernel part of the Taguchimethod is design of experiment (DOE) which evaluates differentdesigns to identify factors that affect product quality and tooptimize their nominal levels [307]. The experiment methodcarried out via computer simulators is studied in [308]. As this

method is DOE based rather than an automated optimizationprocedure, and design variables are defined in a discrete space, itis difficult to treat a wide range of design problems in a continuousspace with several design constraints. Hence RDO based onoptimization techniques are developed. The introduction ofTaguchi method to nonlinear programming was firstly proposedby Ramakrishnan and Rao [309]. Later researchers argued necessityof incorporating constraints in robust design [310] and Parkinsonproposed the concept of constraint feasibility under variation in1993 [292], which is also called feasibility robustness. Thus theoptimization problem is similar to reliability-based robust designoptimization problems (RBRDO) and the mathematical formula-tion is stated similarly with (4). In this situation, the goal in RDOis to locate a constrained optimum that is insensitive to variationsfor both the objective function and the constraints [230]. Theobjective function robustness makes the system performanceinsensitive to variations of design variables and parameters, andthe constraint function robustness ensures the optimum always liesin the feasible region under uncertainties. To address objectiverobustness, it is essentially a multi-objective optimization problem.Tradeoff method between mean and variance of objective hasbeen widely studied in literature. The general used approachesinclude weighted sum of the two objectives (aggregate objectivefunction) [275,311], preference based physical programmingmethod [312,313], compromise programming (CP) method[314,315], Normal-Boundary Intersection (NBI) method [316],and genetic and evolutionary optimization method [317,318].The visualization of the multi-objective RDO has also been studiedin [319]. For constraint robustness, the methods of reliability-baseddesign optimization discussed in Section 5.1 can be utilized.

Among the numerous approaches to accommodate objectiveand constraint robustness, the method Design for Six Sigma hasbeen specifically developed in engineering and has been widelyapplied in industry for product quality improvement [320,321],which is formulated as

find x

min ~f ðx,pÞ ¼ mf ðx,pÞþ6sf ðx,pÞ

s:t: mgðx,pÞþ6sgðx,pÞr0

xLrxrxU

8>>>><>>>>:

ð41Þ

Here we only consider about inequality constraints. For equal-ity constraints in RDO, readers are referred to [322,323]. Thisformulation ensures that the system performance with its sixstandard deviations can be minimized and the constraint with sixstandard deviations can be maintained within the feasible region.

To solve RDO problems, the mean and variance of objectivesand the feasibility robustness of constraints can be calculated withuncertainty analysis methods discussed in Section 4. In gradient-based optimization, calculating sensitivities of the objectivesand constraints are key issues to be addressed. Considering thecomputational cost of finite differencing techniques, automaticdifferentiation method can be used [324], or approximation ofthe original objectives and constraints by response surface modelscan be employed so that sensitivities of these response surfacemodels can be calculated analytically [325,326]. As realistic opti-mization problems usually feature multimodal characteristics, andgradient-based optimization approaches need sensitivity calcula-tions which are computationally expensive, genetic and evolu-tionary algorithms (gradient free) are quite popular [327] andimplementation of evolutionary algorithms for RDO is introducedin [149]. Besides the popular probabilistic RDO approaches, RDOwith other uncertainty types are also studied, such as cloud theory[102,328] and Info-Gap Decision Theory [99] for uncertainties withimprecise probabilities. For more detailed numerical implementa-tion of RDO, readers are referred to [7,35,36,329,330].

Fig. 18. The conventional double-loop UMDO procedure [349].


6. UMDO procedure

UMDO procedure is the organization of all the elementsinvolved in the uncertainty based design optimization, such assystem optimization, system analysis, disciplinary analysis,uncertainty analysis, to name a few. How to efficiently arrangethese elements into an execution sequence and implement it inthe computer or distributed parallel network is the key to realizeUMDO of large systems, as the complex coupling disciplinaryrelationship and computational intensive system analysis modelsmake UMDO very time consuming and computationally prohibi-tive. To illustrate the computational burden of UMDO consideringcoupling relationship, the general uncertainty optimizationformulation (4) is reformulated as

find x

min ~f ðx,p,yÞ ¼ Fðmf ðx,p,yÞ,sf ðx,p,yÞÞ

st: Pfgiðx,p,yÞr0gZRi, i¼ 1,2,. . .,ng

xLrxrxU

8>>><>>>:

ð42Þ

where y denotes the intermediate state variables of disciplinaryanalysis. Denote the output vector of discipline i as yi, thecoupling state vector output from discipline i and input intodiscipline j as yij, the complete set of output vectors fromdiscipline i coupled with other disciplines as yiU, and the completeset of coupling state vectors input into discipline i as yUi, then

y¼ fyi,i¼ 1,2,. . .,nDg, yUi ¼ fyji,j¼ 1,2,. . .,nD,ja ig, yiU¼yiU(xi,p,yUi),

and yiU ¼ fyij,j¼ 1,2,. . .,nD,ja ig. xi is the local design variable

vector of discipline i, and p is the system parameter vector. yiU

is a subset of yi, but for expression convenience and without lossof generality, we assume that yiU is equal to yi and denote equallyas yi in the following discussion. For a three-discipline system, thecoupling relationship with uncertainties is shown in Fig. 17.

The conventional approach to solve this UMDO problem is toemploy a double-loop strategy as shown in Fig. 18.

In the outer loop, optimization algorithm executes optimumsearch. At every iteration point, it calls the inter loop uncertaintyanalysis to evaluate the design and its uncertainty characteristics,which involves lots of sampling simulations based on MCS methodsor optimization (usually referred to as lower level or inner loopoptimization in this double loop procedure) based on FORM or SORMmethods. If it needs N iterations of optimization to converge to theoptimum and M times MDA to analyze the uncertainty characteristicsat each iteration point, the total UMDO problem computation size isat the scale of N�M times the single MDA calculation cost. Keep inmind that iterations are also needed to perform each MDA to obtain aconsistent output due to the coupling relationship of the disciplines.From this point of view, it is clear that the conventional UMDO proce-dure is very computationally expensive especially for aerospacevehicle design whose analysis or simulation tools are extremelytime consuming, e.g. FEA of structure dynamic analysis and CFD

Fig. 17. The coupling relationship of a three-discipline UMDO problem.

(Computational Fluid Dynamics) of aerodynamic analysis which mayuse up to several days to run a single simulation for a full scaleairplane with high fidelity models. To address this problem, sensitiv-ity analysis methods have been used to reduce design variable anduncertain parameter number, so as to control the UMDO problem toan acceptable level [331,332]. Approximation methods have also beenwidely used to replace the high fidelity disciplinary models withmetamodels and replace reliability constraints with simple approx-imation functions. However, it is found that for large number offactors, if more than 10 factors remain after screening, the computa-tional expenses associated with creating response surface approxima-tions can easily begin to outweigh the associated gains [332]. Withprogress in computer technology, advanced parallel computing toolscan be utilized to realize UMDO on distributed computer networkso as to reduce computational time. For example, the MDA of sampl-ing points for sensitivity analysis, approximation modeling, or MCSuncertainty analysis can be executed in parallel [331,333]. Anotherpromising alternative is to solve the problem from inside, i.e. toreformulate the organization of UMDO, including MDA, disciplinaryanalysis, uncertainty analysis, etc., so as to implement the wholeUMDO procedure more efficiently. Now much research has beendevoted to this field and several approaches have been proposed,which can be generally classified into the following two categories:(1) Single level procedure. The optimization loop and uncertaintyanalysis loop are either decoupled and executed in a sequentialway or merged into a single equivalent deterministic optimizationproblem. As such, in the equivalent deterministic MDO, the existingMDO methods can be utilized directly to enhance the efficiency.(2) Decomposition and coordination based procedure. It is expectedthat decomposition and coordination based procedures can bringcomputational efficiency for UMDO as they do in deterministic MDOproblems [2]. Hence, the existing decomposition based procedures fordeterministic MDO, such as IDF (individual discipline feasible), BLISS(bi-level integrated system synthesis), CSSO (concurrent subspaceoptimization), CO (collaborative optimization), ATC (analytical targetcascading), etc., can be used as reference to decompose the nestedoptimization and uncertainty analysis problem of UMDO into severaldisciplinary or subsystem level uncertainty optimization problems, sothat each sub-problem is within manageable control. Besides, thisdecomposition formulation can better take advantage of distribu-tion computation, which can further alleviate the time consumingproblem. These two categories are discussed, respectively, in detail inthe rest of this section.

6.1. Single level procedure

Considering the computational burden attributed to the con-ventional double loop procedure, several approaches have beenproposed to merge or decouple this nested formulation into asingle level procedure, also called single level approach (SLA).

To merge the double loop optimization into one single levelproblem, Agarwal et al. [334] proposed to replace the lower-level


reliability analysis with FORM PMA method by its correspondingfirst-order necessary Karush–Kuhn–Tucker (KKT) optimality con-ditions, and impose these KKT conditions on the outer optimiza-tion loop, so as to eliminate the inner loop uncertainty analysisand meanwhile satisfy reliability requirement. Assume that hardconstraints can be identified and only reliability analysis of thesehard constraints is needed. The merged optimization problem canbe formulated as

find x, u1,. . .,uNhard


s:t: Giðui,gÞr0 i¼ 1,. . .,Nhard

h1i � :ui::ruGiðui,gÞ:þuTi ruGiðui,gÞ

h2i � :ui:�bi

h1i ¼ 0, i¼ 1,. . .,Nhard

h2i ¼ 0, i¼ 1,. . .,Nhard

giðx,p,yÞr0, i¼ 1,. . .,Nsoft

xLrxrxU

8>>>>>>>>>>>>>>>>><>>>>>>>>>>>>>>>>>:

ð43Þ

where bi is reliability index corresponding to the reliabilityrequirement for the ith constraint, Gi(ui,g) is the limit statefunction of the ith hard constraint in the standard normal space,g is the transformation of p in the standard normal space U, ui isthe inverse MPP for the ith hard constraint, and gi(U) is the softconstraint which remains in the original X space. The initialoptimization search space is augmented which consists of boththe initial design variables and inverse MPPs of all the hardconstraints. For this reason, this approach is also called single-level-double-vector (SLDV) method, in contrast with the initialdouble-loop-single-vector approach (DLSV) [335]. For multidisci-plinary coupled problem, GSE is used to implicitly calculategradient of the limit state functions. This formulation is mathe-matically equivalent to the initial double loop optimizationproblem given that the KKT constraint qualifications are satisfied.But the major problem herein is that the design variables will begreatly increased if the number of hard constraints is large, i.e. fororiginal optimization with nX dimensional design variables, theaugmented design variable number will be nXU(1þNhard), whichmay increase optimization computational burden substantiallyand outweigh the gains in computational efficiency by integra-tion of double loop optimization into a single one. Furthermore,enforcing large number of equality constraints on the outer loopoptimization may also lead to poor numerical stability and con-vergence behavior. Besides, the KKT conditions are derived fromPMA first-order reliability analysis algorithm, the accuracy ofwhich is also doubtful for highly nonlinear uncertainty problems.

Chen et al. [335] proposed another method named single-loop-single-vector algorithm (SLSV) to approximately locate MPP ofeach active constraint with gradients of the limit state functionand the desired safety factor, so that the inner MPP searching loopcan be eliminated and the approximation of MPP can be directlyembedded in the outer loop optimization as equivalent determi-nistic constraints. The outer loop optimization problem (42) canbe reformulated as

find x


s:t: GiðzðkÞi ,gÞr0, i¼ 1,. . .,Nhard

zðkÞi ¼ mðkÞz þbia

nðk�1Þi , i¼ 1,. . .,Nhard

mðkÞz ¼ x=sx

anðk�1Þi ¼rzGiðz

ðk�1Þi ,gÞ=:rzGiðz

ðk�1Þi ,gÞ: i¼ 1,. . .,Nhard

giðx,p,yÞr0, i¼ 1,. . .,Nsoft

xLrxrxU

8>>>>>>>>>>>>>>>><>>>>>>>>>>>>>>>>:

ð44Þ

where zðkÞi denotes the approximate MPP of the ith hard constraintin the kth iteration of optimization, an

i is the vector of directioncosine of the ith hard constraint at zi. It is noteworthy that thelimit state function Gi, MPP vector zi, and transformed systemparameter g are in the uncorrelated normalized space. In thisalgorithm, the direction cosine of the hard constraint at theprevious approximate MPP is used to approximately locate MPPin the current optimization run, and after iterations zi canconverge to the accurate MPP of the ith hard constraint and theoptimum design can be obtained. Compared with SLDV, thismethod is simplified as the design vector remains the same asthe initial optimization problem. But the numerical demonstra-tion indicated that this method may be instable when the uncer-tainties feature constant coefficients of variation (COV), and theaccuracy is also compromised moderately for improvement ofcomputational efficiency. This method is further extended to non-normal distribution random variables with normal tail transfor-mation method in [300]. The thought of SLSV is also employedto address the problem of design variable augmentation in theSLDV approach, and the KKT conditions are reformulated to allowMPP vector to gradually converge to the accurate MPP afteriterations [336].

The other type of SLA is to decouple the inner uncertaintyanalysis and outer optimization with sequential cycles of uncer-tainty analysis and deterministic MDO. In each cycle, the relia-bility constraints are converted into equivalent deterministicconstraints based on the uncertainty analysis and then used inthe separate deterministic MDO to guide the optimum searchtowards the feasible region to meet the reliability requirements.The uncertainty analysis and deterministic optimization arearranged in a sequential manner and alternately executed, whichfinally converge to the optimum in compliance with reliabilityrequirements after several iterations. The key issue herein is howto convert the reliability constraint into the equivalent determi-nistic one with satisfaction of reliability requirement.

Sues and Cesare proposed to search MPPs firstly in reliabilityanalysis at initial design and approximate each limit state func-tion at its MPP with first-order linearized models, which are usedas equivalent deterministic constraints in the following MDO[337]. When the optimum of MDO is achieved, the reliabilityanalysis is conducted again at the optimum to update the MPPsand the corresponding approximate models of the limit statefunctions, which are further fed into the MDO of the next cycle.The steps of reliability analysis and deterministic MDO areiteratively conducted until convergence is achieved. The linear-ization of limit state functions is easy to implement and calculatein MDO, but the accuracy is also limited.

Wu et al. [338] developed a safety-factor based approach toconvert the reliability constraints into the equivalent determinis-tic constraints with safety factors. In uncertainty analysis, MPP xn

i

for random parameters (in the X space) and design shift factor si

of the ith hard constraint are defined as

Pfgiðxn

i ,dÞþsir0g ¼ Ri ð45Þ

where d is the deterministic design variable vector, Ri is therequired reliability of the constraint i, and si is the constant toshift the current constraint (without shape change) to exactlymeet the target reliability. Given d, MPP xn

i can be located withexisting MPP search approaches, and si can be consequentlycalculated with (45). Then the active reliability constraint i canbe converted as giðx

n

i ,dÞþsir0 and used in the deterministicoptimization, where the design variable vector d is updated withoptimization algorithm and fed into the uncertainty analysis ofthe next cycle to update the MPPs and the corresponding designshift factors. These two steps are alternately repeated until theyconverge to the optimum and the accurate MPPs. This method


firstly identifies active constraints in each cycle, and uncertaintyanalysis of only these constraints is treated, which can save a lotof computational cost.

Du and Chen [339] proposed a sequential optimization andreliability assessment (SORA) method to formulate the determi-nistic constraint by shifting the inverse MPP at the current designat least onto the deterministic boundary which ensures the con-straint feasibility in the next deterministic optimization satisfyingrequired reliability. Its schematic flowchart is shown in Fig. 19,and the graphical illustration of shifting constraint boundary isshown in Fig. 20. The deterministic optimization of the kth itera-tion is formulated as

find x


s:t: giððx�sðkÞi Þ,pðk�1ÞiMPP ,yÞr0, i¼ 1,2,. . .,ng

sðkÞi ¼ lðk�1Þx �xðk�1Þ

iMPP

xLrxrxU

8>>>>>>><>>>>>>>:

ð46Þ

When the optimum x* with mean value lðkÞx of this kthoptimization is achieved, reliability analysis is conducted at theoptimum, and the corresponding inverse MPP ðxðkÞiMPP ,pðkÞiMPPÞ andshift vector sðkþ1Þ

i of the ith constraint for the next cycle can becalculated, which are used to reformulate the deterministicoptimization in the next iteration. The computational efficiencyof SORA was testified by numerical examples compared withconventional double loop procedure, but the convergence of highnonlinear system with large number of design and randomvariables cannot be guaranteed with this approach. As the shiftvector in each deterministic optimization is based on the MPP ofthe preceding iteration, the estimated MPP with the shift vector incurrent optimization may be inaccurate as design variableschange, which may lead to pseudo-optimum solutions. To addressthis problem, Agarwal and Renaud proposed to use the first-orderTaylor series expansion of the MPP at the preceding MPP with itspostoptimal sensitivities with respect to the design variables, anduse this approximation to update MPPs for constraints during thedeterministic optimization [340]. Based on the SORA framework,Mixed Variables (random and fuzzy variables) MultidisciplinaryDesign Optimization (MVMDO) [341] and Random/Fuzzy Contin-uous/Discrete Variables Multidisciplinary Design Optimization(RFCDV-MDO) [342] procedures are further developed, whichutilize existing discrete-continuous optimization approaches inthe deterministic optimization, and deterministic MDO procedureIDF for each MDA in uncertainty analysis. SORA has beensuccessfully applied to a reusable launch vehicle design, whichintegrates the system design for minimum weight with aerody-namic constraints and the component design of a liquid-hydrogentank into a reliability-based MDO problem. It is observed thatthe decoupling approach of SORA can significantly reducecomputational burden for the UMDO problem, and it can be evenmore efficient if active constraints can be identified duringiterations [343].

Royset et al. [344] proposed to reformulate the reliabilityconstraint with general reliability index as its equivalent

Fig. 19. SORA pro

deterministic constraint by maintaining all the values of the limitstate function (in the standard normal space) within a ball ofspecified radius defined by the required reliability index in thefeasible region. For example, the constraint btrb(x) with saferegion denoted as g(u,x)r0 can be reformulated as

crðxÞr0

crðxÞ ¼maxfgðu,xÞ9uABrg

Br ¼ fuARm9:u:rrg

ð47Þ

where u is the random parameter vector in the standard normalspace, and x is the deterministic design variable vector. In thedeterministic optimization, the method to approximately calcu-late reliability index of affine limit state function with the form ofg(u,x)¼1þc(x)Tu in the standard normal space is

bðxÞ ¼rUgð0,xÞ

crðxÞþgð0,xÞð48Þ

After the deterministic optimization, reliability analysis isconducted to analyze the real reliability index at the currentoptimum, which is used to update the radius of the ball on theconstraint with (48) for the deterministic optimization in the nextcycle. This method is equivalent to the original optimizationproblem and can yield accurate solution when the limit statefunctions are affine in the standard normal space. But it is onlycorrect to first-order approximation for the nonaffine limit statefunctions.

Zou and Mahadevan [345] proposed to use the first-orderTaylor series approximation of the reliability index or failureprobability with respect to the mean of design variables at thecurrent optimum for all potentially active reliability constraintsas the deterministic constraints for the next cycle optimization,which is stated as

pkf þ

Xn

i ¼ 1

@pf

@mkni

ðmkþ1i �mkn

i Þrpft

bkf þ

Xn

i ¼ 1

@bf

@mkni

ðmkþ1i �mkn

i ÞZbft ð49Þ

where pkf and bk

f are failure probability and reliability index,respectively, for the optimum mkn

i in the deterministic optimiza-tion of the kth cycle, pft and bft are predefined target failureprobability and reliability index, and mkþ1

i is the mean value ofthe ith component of the design variable vector in the kþ1thoptimization. In the separate reliability analysis, pk

f and bkf can be

calculated with any existing reliability analysis method flexibly.If FORM, SORM, and MCS approaches are used, the sensitivity ofthe reliability index and failure probability with respect to thedesign variables can be calculated analytically as the by-productof reliability analysis without additional function evaluation.To further improve efficiency, the worst case failure probabilityestimated with (49) is used to identify potentially active relia-bility constraints, and reliability analysis only for the activeconstraints is conducted. This method is more flexible than theaforementioned approaches as the formulation of deterministicconstraint is not confined to specific reliability analysis approach

cedure [343].

Fig. 20. Graphical illustration of shifting constraint boundary in SORA [339].


such as MPP based method. But the first-order approximationaccuracy is limited when the difference between the currentdesign variables and the optimum of the previous iteration islarge, which may influence convergence.

For the separate deterministic MDO decoupled from uncer-tainty analysis, any formulations of MDO problems and existingMDO approaches can be used, such as AAO (all-at-once)[346,347], SAND [348], IDF, MDF [23,347,349], and BLISS [350],so as to enhance the deterministic optimization efficiency. Thepreceding SLA approaches, including SLSV, SFA and SORA, havebeen tested and compared with the conventional double loopapproach, and the results validate the computational efficiency ofthese methods [351,352]. It is observed that SLSV is mostpromising in terms of stable convergence and low calculationcost among these approaches, but it does not assess the accuratereliability of each design which should be confirmed for theoptimum in the end. SFA and SORA also have good convergencecapability, and their computational burden for reliability assess-ment in each cycle can be further reduced by identification ofactive constraints.

6.2. Decomposition and coordination based procedure

In deterministic MDO, decomposition and coordination basedprocedures are widely studied and their efficiency is verified byboth numerical and engineering demonstrations. This provides agood reference for UMDO to decompose the large scale computa-tional prohibitive problem into several manageable sub uncer-tainty optimization problems which are coordinated to achieve aconsistent optimum with certain strategy.

6.2.1. CO-based UMDO procedure

Collaborative Optimization (CO) is a bilevel optimization pro-cedure specifically for large scale distributed analysis applications[353]. To decompose the coupled disciplines and execute disci-plinary design optimization concurrently, auxiliary design vari-ables are introduced as additional design variables which representcoupling state variables. In the system level optimization, theshared design variables and auxiliary design variables are opti-mized to minimize objective with disciplinary compatibility con-straints which enforce the auxiliary design variables to be equal tothe real coupling state variables (passed up from the disciplinaryanalysis) by equality constraints. In each subsystem optimization,the local design variables are optimized to minimize the discre-pancy between the subsystem output and the auxiliary variablevalues passed down from the system, meanwhile satisfy the local

disciplinary constraints. After iterations the optimization can con-verge to a consistent system optimum design.

McAllister et al. firstly proposed to adapt CO to accommodateuncertainty to solve UMDO problems [354]. In the CO framework,both the system and subsystem level optimizations are formu-lated with multi-objective mathematical programming formula-tions, so called compromise Decision Support Problem (DSP)[355]. In the system DSP, both the mean and variance (robust-ness) of the objective are minimized with subsystem compat-ibility constraints. In the sub-system DSP, minimization of thediscrepancy between the targets specified by the system and thelocal output values is set to be the optimization goal with firstpriority. Once this objective is achieved, the local objectives withsecondary priority which the subsystem designers may concernbut not dealt with in the system level can be optimized, whichcan be realized based on the multi-objective optimization natureof compromise DSP. The uncertainty analysis for DSP in bothlevels employs first-order Taylor expansion method for mean andvariance estimation based on the assumptions that variations ofuncertainties are small and sources of uncertainties are indepen-dent. In the subsystem level DSP, the reliability constraints areapproximately converted to the equivalent deterministic constraintswith worst case analysis method introduced in Section 5.1, whichonly accounts for uncertainties of the local design and sharedvariables. It has been applied to an internal combustion enginemultidisciplinary robust design optimization problem [354], andthe results show that it needs significant iterations to achievesystem-level compatibility attributed to equality constraints inthe system level optimization, which would be worse for largescale problem with more shared variables.

Gu and Renaud [261] proposed a robust collaborative opti-mization (RCO) framework based on an implicit method forestimating system performance uncertainties with uncertaintypropagation through coupling disciplines. To address the conver-gence problem caused by the compatibility equality constraintsimposed on the system level optimization, an improved COframework proposed by DeMiguel and Murray [356] is utilizedto formulate system and sublevel optimization problems. In thesystem level, the mean of the original objective (robust objec-tive not considered) is optimized with subsystem compatibilityconstraints, and the implicit uncertainty propagation (IUP) isanalyzed based on the uncertainty analysis method for multi-disciplinary system proposed by Gu et al. [174] and stated in (19),which is reformulated for CO framework as

DðxauxÞ1

DðxauxÞ2

^

DðxauxÞnD

8>>>><>>>>:

9>>>>=>>>>;�

dy1dx

dy2

dx

^dynD

dx

8>>>>><>>>>>:

9>>>>>=>>>>>;Dx

þ

I1 �@T1

@ðxauxÞ21� � � �

@T1@ðxauxÞnD1

�@T2

@ðxauxÞ12I2

^ & ^

�@TnD

@ðxauxÞ1nD

� � � InD

26666664

37777775

�1

DT1

DT2

^

DTnD

8>>>><>>>>:

9>>>>=>>>>;ð50Þ

where xaux represents the coupling state variables and treated asdesign variables in CO, and (xaux)ij denotes the coupling state vari-able vector output from disciplinary i and input into disciplinary j.Here both the errors of the design variable vector x (with varia-tion Dx) and the contributing analysis tool Ti (with bias errors DTi)are considered, and the coupling effect is treated with employ-ment of GSE. The derivatives qTi/q(xaux)Ui are calculated withineach subsystem with corresponding analysis tool Ti, and fed


upward to the system level. After implicit uncertainty analysis inthe system level, variation characteristics of auxiliary variablesare obtained and passed down to subsystems, which are usedalong with the local design variable uncertainties and simulationmodel uncertainties to analyze subsystem output uncertaintycharacteristics. The block diagram of RCO procedure is shown inFig. 21, which takes a three-discipline UMDO problem for illus-tration. As CO does not enforce disciplinary consistency explicitly,it is notable that the sensitivities calculated from subsystemsare not consistent until the system level optimizer has satisfiedthe compatibility constraints, which may lead to inaccuracy ofthe uncertainty analysis and further influence optimizationconvergence.

Both the preceding CO based UMDO procedures adopt thefirst-order worst case uncertainty analysis method to deal withuncertainty propagation and reliability analysis, which still needaccuracy improvement.

6.2.2. CSSO-based UMDO procedure

Concurrent Subspace Optimization (CSSO) is a bilevel optimi-zation procedure which can decompose the coupled multidisci-plinary design optimization problem into several independentsubspace problems. These subspace optimizations are performedconcurrently by operating on the corresponding local designvariables and approximating non-local coupled state variableswith GSE or approximation models. Then the subspace optimiza-tion results are coordinated by certain coordination strategy toobtain system compatibility. CSSO can enable each disciplinarydesigner to design and optimize with their own tools inde-pendently, which complies with the organizational feature ofindustry, so it has been widely studied and improved for itsorganizational flexibility and computational efficiency [357,358].Based on CSSO procedure, UMDO problems can be decomposedinto several local (contributing analysis level) uncertainty-basedoptimization subproblems which can be organized concurrently.

Padmanabhan and Batill [359] proposed to realize reliabilitybased MDO in the CSSO procedure. In this procedure, a systemanalysis and a reliability analysis (with FORM method) are firstlyconducted to obtain outputs of objectives, intermediate statevariables, reliability constraints, as well as their sensitivities withrespect to the deterministic and random variables at the initialdesign point. Based on these outputs and sensitivities, the first-order Taylor series approximation models are employed to buildthe metamodels to be used in the subspace optimizations forestimation of non-local state variables and reliability constraints.Then the subspace optimizations (SSO) are executed concurrently,

Fig. 21. Block diagra

and the optimization results are further coordinated in theCoordination Procedure (CP) with approximation models builtfrom the data obtained during the optimization iterations withinSSOs. The design solution is updated after CP and fed into thesystem analysis and reliability analysis of next cycle, and theaforementioned steps are repeated until convergence is achieved.

Yao et al. [360] also proposed to integrate uncertainty analysisinto CSSO so as to account for MDO problems under uncertainty,and have successfully applied it to solve a small satellite con-ceptual system design problem [361]. In this procedure, sensitiv-ity analysis is firstly conducted to screen out the uncertain designvariables and uncertain system parameters which have negligibleinfluence on the design. Then approximation models of objectivesand coupled state variables are built with design of experimenttechniques. In SSOs, the mean and standard deviations of theobjective are optimized with local reliability constraints, and anyexisting uncertainty analysis methods mentioned in Section 4 canbe used to quantify uncertainty propagation within the subspace.In each SSO, only the accurate analysis tool of the local subspaceis used and non-local state variables are estimated with approx-imation models, so the computational burden can be balanced. Inthe coordination procedure, all the design variables are optimizedconsistently with approximation models. Then the approximationmodels are updated at the new optimum and to be used in theSSOs and CP of the next cycle. The preceding steps are repeateduntil convergence is achieved, and the reliability of the optimumis confirmed with Monte Carlo uncertainty analysis method. Thisprocedure is very flexible in selection of uncertainty analysisapproaches in system and subsystem level optimizations, but itignores the cross propagation effect of uncertaintes in the localSSOs, which results in loss of accuracy.

6.2.3. ATC-based UMDO procedure

Analytical target cascading (ATC) procedure is a very promis-ing approach to organize MDO of hierarchical systems [362].In this procedure, each element receives optimization target fromits parent, and cascades down sub targets to its children. Mean-while, each element passes its response up to its parent, so thatthe parent can adjust new targets according to the response. Afteriterations of targets cascading down from top to bottom andresponses passing up from bottom to top, the design can convergeto a consistent solution.

The probabilistic version of ATC is firstly proposed by Kokko-laras et al. [363] to solve MDO problems of hierarchical systemwith uncertainties. For the hierarchical system shown in Fig. 22,assuming that initial uncertainty information is available at the

m of RCO [261].

Fig. 22. The hierarchical system in ATC procedure.


bottom level of the hierarchy, the bottom-to-top coordinationstrategy is used to propagate uncertainty. The advanced meanvalue (AMV) method [364] is used to efficiently generate CDF ofeach element response with propagated random variables inuncertainty-based optimization of each element, and the meanand standard deviation of each element response are passedupwards to its parent. The elementary optimization problemsare solved upwards level by level until the top of the hierarchy isreached. Once the top-level optimization problem is finished, newtargets begin to be cascaded downwards level by level from thetop to the bottom. With updated parameters, elementary optimi-zations are conducted from bottom to top again. The precedingsteps are iterated until convergence is achieved.

In this approach, since only mean values of uncertainties arematched between different levels, convergence efficiency to achievea consistent optimum solution is low. To solve this problem, it isproposed to also match standard deviation [365] and covariance[366] between interrelated responses and linking variables. It isdemonstrated that the improved approach can obtain almost thesame result as that of Probabilistic All-In-One (AIO) optimizationmethod when the first two moments can sufficiently describethe probabilistic characteristics of random variables. Since eachsubproblem in each element of the hierarchy is linked only to thesubproblems directly above and below, it does not require exten-sive link among all the subsystems, so that the data relationship canbe simplified, which is very promising for ATC to be applied inUMDO problems of large scale hierarchical system.

7. Conclusions

In this paper, we have reviewed the fundamental issues ofUMDO theory and general UMDO approaches for aerospacevehicle design. The acknowledgement of importance of UMDO isspreading fast, but UMDO theory research is still in the earlystages of development. While fundamental terminology, philoso-phy, and general procedure are well established, detailed compu-tational algorithms and organizational procedures are still neededto solve the major challenges of UMDO, namely computationalcomplexity and organizational complexity. Hence the futureresearch should be directed towards increasing UMDO efficiencywith given level of cost and effort expended on UMDO activities.Besides, UMDO should be amenable to the realistic multidisci-plinary organization of aerospace vehicle design, especially con-cerning the trend of distributed and concurrent design, so as to beappealing to the industry for practical applications. Towards thesegoals, we provide some detailed recommendations as follows.

The premise of UMDO is to appropriately represent uncer-tainties, which essentially includes three parts: (1) List all theuncertainties exhaustively which should be considered inthe design optimization, as discussed in Section 3.1 wherein therelevant uncertainty sources for UMDO in the design phase areenumerated throughout the whole lifecycle of aerospace vehicle.

(2) Select appropriate mathematical models to represent uncer-tainties according to the different uncertainty types and availableinformation about uncertainties. Generally speaking, probabilitytheory is more advantageous with sufficient information, whilenon-probabilistic approaches are more feasible with insufficientinformation or imprecise data. Possibility theory is especiallysuitable for epistemic uncertainties concerning subjective vague-ness and sparse available information, but if the available infor-mation (evidence) is conflicting, evidence theory should be used.Possibility theory uses possibility and necessity to describe thelikelihood of a proposition, while evidence theory uses belief andplausibility to define lower and upper bounds of the probability.These measures are determined from the known informationwithout any assumptions beyond what is available, which canprovide designers or decision makers with more reliable choicethan the single probability obtained via probability approacheswith strong assumptions. But the problem with non-probabilisticapproaches is that generally more computational cost is needed,and the analysis result is not intuitively interpretable fordesigners, especially those who lack understanding of these newmathematical theories in contrast with probability theory whichhas long history. Therefore we strongly recommend that theprobabilistic and non-probabilistic approaches should be usedflexibly as complementary techniques to represent uncertaintiesaccording to the specific situation and make better use ofavailable information. (3) Screen uncertainties with sensitivityanalysis to reduce the uncertainty problem scale. There arenumerous sensitivity analysis approaches with probability theory,but very few for the non-probabilistic uncertainties. However,sampling-based method is a universal method for both thealeatory and epistemic uncertainties.

To quantify output uncertainty characteristics resulted fromuncertainty propagation through the system inner mechanism,we have covered a range of non-intrusive uncertainty analysisapproaches and have analyzed pros and cons of each methodalong the way. There is no method which is universally betterthan all others, and there are no accurate rules about the choice ofmethod to be used in a specific situation. But there is an under-lying clue that the suitability of each method mainly depends onits accuracy and computational cost in the specific problem, aswell as designers’ preference towards the balance of accuracyand computational efficiency. Generally MCS approaches aremore accurate and also more computationally expensive, butwith quickly development of variance reduction sampling tech-niques the efficiency can be greatly enhanced. The most compu-tational affordable and widely applied method is Taylor seriesapproximation approach for the relative ease of implementationand understanding. But this method can only estimate the firsttwo moments of the output rather than the exact distribution,and its accuracy is quite limited for highly nonlinear systems.Considering reliability as one of the dominating measures inUMDO, there is a specific type of methods dedicated to reliabilityanalysis. These approaches are specially devised to account forsmall failure probability calculation, which may lead to pooraccuracy or prohibitive computational cost with normal uncer-tainty analysis methods. Among these methods, FORM and SORMapproaches are most prevailing which have formed a full theorywith substantial mathematical foundation and been widelyapplied in practical reliability based optimization problems. Butthe first or second approximation of the limit state function at theMPP to estimate the failure probability integral may be insuffi-cient for highly nonlinear systems, and the algorithms to locateMPP especially with multi failure regions or multi optimums arealso problematic which may contribute to inaccuracy. For highlynonlinear problems, MPP based MCS method may be a goodchoice. Considering that aerospace vehicle design generally


involves several closely coupled disciplines, all of which usuallyuse complex analysis tools for disciplinary analysis, e.g. CFD andFEA, it would be computationally prohibitive to employ normaluncertainty analysis methods as each single MDA would be verytime consuming to obtain a consistent system output for a givenset of parameters. For these problems we strongly recommend thedecomposition based uncertainty analysis methods, which decom-pose the uncertainty analysis into several sub analysis problemswithin disciplinary level at a controllable scale and meanwhileaccount for the cross propagation through certain coordinationstrategies. Furthermore, the decomposition based method cantake advantage of distributed and concurrent computation, whichcan further alleviate computational burden. But so far theseapproaches mainly deal with the cross propagation of the firsttwo moments and the primary strategies to propagate uncertaintyeffect are based on first-order GSE, which should be investigatedmore thoroughly so that the exchange information betweendisciplines can be sufficient to enhance analysis accuracy.

The kernel of UMDO is to state the UMDO problem as itscorresponding mathematical optimization formulation and useappropriate algorithm to search the optimum or pareto set (inmulti-objective optimization problem) with predefined criterion.Numerous formulation models and a variety of search algorithmshave been proposed in literature for optimization under uncertaintyand used successfully in many applications. The most widely studiedand applied formulations are reliability based optimization androbust design optimization. These two formulations have deep rootsin industry, and have quickly developed as driven by urgent needfor robustness and reliability especially in the high risk and highcost aerospace field. To solve these optimization formulations withuncertainty, a straightforward method is to integrate uncertaintyanalysis directly into the search algorithm which analyzes theuncertainty characteristics of the system response at each optimiza-tion iteration point, but this leads to prohibitive computationalburden. Hence several approaches have been developed to trans-form the original UMDO problems into quasi-equivalent simplifiedformulations, such as Design for Six Sigma method to approximatethe robust design optimization, the approaches to convert reliabilityconstraints into quasi-deterministic ones, etc. But generally thesesimplification approaches tend to be too conservative resulted fromthe pervasively used worst case assumptions, and the accuracy ofthe result should also be confirmed by high fidelity uncertaintyanalysis so as to enhance confidence in optimization result. As forthe algorithms to search the optimum under uncertainty, there isrich literature to address this problem. Specifically for the simula-tion based design optimization, as the case of aerospace vehicledesign which usually resorts to computer simulation for systemanalysis rather than explicit analytical equations, the gradient freealgorithms are more appealing and have seen rich research activitiesin this area. Among the gradient free approaches, we firmly believethat genetic algorithms (GA) are very promising with their globaloptimization capability and multi-objective pareto set identificationcapability. And the population calculation of GA can take advantageof distributed computation to alleviate time cost. But the conver-gence of GA is largely affected by crossover and mutation strategies,which need further investigation to enhance efficiency.

With the technical blocks of UMDO as discussed above, theproblem of how to effectively organize them in an executivesequence so as to realize UMDO in the computer environment isof extraordinary importance, which is addressed by the UMDOprocedure research. Corresponding to the straightforward methodof optimization under uncertainty with direct integration of uncer-tainty analysis in the search algorithm, the most intuitive UMDOprocedure is to organize the optimization and uncertainty analysisblocks in this iterative way, which results in a double-loopoptimization problem with outer optimization loop and inner

uncertainty analysis loop. For multidisciplinary system, it is com-putational prohibitive as each MDA also needs iterations to obtaina consistent analysis result, which essentially makes UMDO intoa triple-loop optimization problem. Therefore many efforts aredevoted to this area to develop strategies to organize UMDO moreefficiently. In general there are two types, namely single levelprocedure and decomposition and coordination based procedure.Single level procedures either merge the two optimization loopsinto a single one, or decompose the two loops into two separatesteps executed in a sequential way, so that the uncertainty analysisand deterministic optimization can be programmed separately,which is easier to develop and can make use of legacy codes. It isworth noting that for multidisciplinary system optimization, theseparate deterministic optimization formulation is especially attr-active as it can make full use of the existing advanced approachesdeveloped in the traditional deterministic MDO to enhance opti-mization efficiency. But the problem with this single level approachis that convergence efficiency and result accuracy are not guaran-teed as the formulation of deterministic optimization is with strongsimplification based on the preceding uncertainty analysis result,which should be investigated more thoroughly. Specifically toaccount for the multidisciplinary feature, we strongly recommendthe decomposition and coordination based procedure, which candecompose large scale UMDO problems into manageable subuncertainty optimization problems within disciplinary scope andexecute these disciplinary optimizations simultaneously with cer-tain coordination strategy to converge to a consistent optimum.This type of procedure is very appealing not only for its advantagein efficiency as it can utilize distributed computation technology tosolve computational complexity problem, but also for its capabilityof allowing distributed concurrent disciplinary design optimizationwhich is very desirable in industry as to comply with the realisticdisciplinary organization structure and maintain disciplinary auton-omy. The research in this field is relatively new and up to now thedecomposition and coordination strategies are mainly based on themature deterministic MDO procedures to accommodate uncer-tainty, e.g. CO and CSSO, but we believe that with its promisingfeature it will be quickly developed.

Regarding computational efficiency, an additional area forUMDO that requires further research is approximation modelingmethods. As discussed in the review sections, it is desirable to useapproximation models as surrogate models of the high fidelitymodels in UMDO, as it can effectively reduce computational costin uncertainty analysis and optimization compared with the timeconsuming models. Furthermore, it can transform the implicitand discrete simulation based models into explicit and contin-uous models, so as to facilitate some UMDO technical blockswhich need utilization of the continuous function feature, such assensitivity analysis which needs derivative calculation and someoptimization algorithms which need gradient information.

To achieve the level of maturity of UMDO in aerospaceengineering, a much deeper understanding of mathematics, aero-space disciplines, computation, and their relationship is requiredthan it is reflected in current UMDO practice. And additionaltraining and understanding of UMDO theory is also needed forpractical application. The road to develop UMDO is long anddifficult, but we firmly believe that with strong demand fromindustry and quick development of science and technology,UMDO will hopefully become an advanced and powerful toolfor aerospace vehicle design in the near future.

Acknowledgment

The authors sincerely thank Dr. Yong Zhao, Dr. Weiwei Yang,Dr. Qi Ouyang, and Dr. Yuexing Wei for their helpful comments in


preparing the manuscript. This work was supported in part byNational Nature Science Foundation of China under Grant nos.50975280 and 61004094, Program for New Century ExcellentTalents in University of Ministry of Education of China underGrant no. NCET-08-0149, Fund of Innovation by Graduate Schoolof National University of Defense Technology under Grant no.B090102, and Fund of Innovation by Hunan Province, China.

References

[1] Elms DG. Structural safety-issues and progress. Progress in StructuralEngineering and Materials 2004;6:116–26, doi:10.1002/pse.176.

[2] Zang TA, Hemsch MJ, Hilburger MW, Kenny SP, Luckring JM, Maghami P,et al. Needs and opportunities for uncertainty-based multidisciplinarydesign methods for aerospace vehicle. NASA/TM-2002-211462. LangleyResearch Center; 2002.

[3] Oberkampf WL, Helton JC, Joslyn CA, Wojtkiewicz SF, Ferson S. Challengeproblems: uncertainty in system response given uncertain parameters.Reliability Engineering and System Safety 2004;85(1–3):11–9.

[4] Dantzig BG. Linear programming under uncertainty. Management Science1955;1(3–4):197–206.

[5] Freund RJ. The introduction of risk into a programming model. Econome-trica 1956;24(3):253–63.

[6] Sahinidis NV. Optimization under uncertainty: state-of-the-art and oppor-tunities. Computers and Chemical Engineering 2004;28(6–7):971–83.

[7] Schuellera GI, Jensen HA. Computational methods in optimization consider-ing uncertainties—an overview. Computer Methods in Applied Mechanicsand Engineering 2008;198(1):2–13.

[8] Dhillon BS, Belland JS. Bibliography of literature on reliability in civilengineering. Microelectronics and Reliability 1986;26(1):99–121.

[9] Tong YC. Literature review on aircraft structural risk and reliability analysis.DSTO-TR-1110. DSTO Aeronautical and Maritime Research Laboratory;2001.

[10] Frangopol DM, Maute K. Life-cycle reliability-based optimization of civiland aerospace structures. Computers and Structures 2003;81(7):397–410.

[11] Padula SL, Gumbert CR, Li W. Aerospace applications of optimization underuncertainty. Optimization and Engineering 2006;7(3):317–28.

[12] Long MW, Narciso JD. Probabilistic design methodology for compositeaircraft structures. DOT/FAA/AR-99/2. US Department of Transportation;1999.

[13] Uebelhart SA. Non-deterministic design and analysis of parameterizedoptical structures during conceptual design. PhD dissertation, Massachu-setts Institute of Technology, 2006.

[14] Li L. Structural design of composite rotor blades with consideration ofmanufacturability, durability, and manufacturing uncertainties. PhD dis-sertation, Georgia Institute of Technology, 2008.

[15] Li W, Huyse L, Padula S. Robust airfoil optimization to achieve consistentdrag reduction over a mach range. NASA/CR-2001-211042. LangleyResearch Center; 2001.

[16] Gumbert CR, Newman PA. Effect of random geometric uncertainty on thecomputational design of a 3-D flexible wing. In: Proceedings of the 20thAIAA applied aerodynamics conference, 2002.

[17] Lindsley NJ, Pettit CL, Beran PS. Nonlinear plate aeroelastic response withuncertain stiffness and boundary conditions. Structure and InfrastructureEngineering 2006;2(3–4):201–20.

[18] Wie B, Liu Q, Sunkel J. Robust stabilization of the space station in thepresence of inertia matrix uncertainty. In: The first IEEE regional conferenceon aerospace control systems proceedings, 1993.

[19] DeLaurentis DA. A probabilistic approach to aircraft design emphasizingguidance and stability and control uncertainties. PhD dissertation, GeorgiaInstitute of Technology, 1998.

[20] Padmanabhan D. Reliability-based optimization for multidisciplinary sys-tem design. PhD dissertation, University of Notre Dame, 2003.

[21] Sues RH, Oakley DR, Rhodes GS. MDO of aeropropulsion componentsconsidering uncertainty. In: AIAA/NASA/ISSMO symposium on multidisci-plinary analysis and optimization, Reston, VA, AIAA-96-4062-CP, 1996.

[22] Noor AK. Nondeterministic approaches and their potential for future aero-space systems. NASA/CP-2001-211050. Langley Research Center; 2001.

[23] Yu X, Du X. Reliability-based multidisciplinary optimization for aircraftwing design. Structure and Infrastructure Engineering 2006;2(3-4):277–89.

[24] Zeeshan Q, Yunfeng D, Rafique AF, Kamran A, Nisar K. Multidisciplinaryrobust design and optimization of multistage boost phase interceptor. In:51st AIAA/ASME/ASCE/AHS/ASC structures, structural dynamics, and mate-rials conference, Orlando, Florida, AIAA-2010-2920, 2010.

[25] Oberkampf WL, DeLand SM, Rutherford BM, Diegert KV, Alvin KF. A newmethodology for the estimation of total uncertainty in computationalsimulation. In: 40th AIAA/ASME/ASCE/AHS/ASC structures, structuraldynamics, and materials conference and exhibit, St. Louis, MO, AIAA-99-1612, 1999.

[26] Hastings D, McManus H. A framework for understanding uncertainty and itsmitigation and exploitation in complex systems. In: 2004 engineeringsystems symposium, Massachusetts Institute of Technology, 2004.

[27] DeLaurentis DA, Mavris DN. Uncertainty modeling and management inmultidisciplinary analysis and synthesis. In: 38th aerospace sciences meet-ing and exhibit, Reno, NV, AIAA-2000-0422, 2000.

[28] IEEE guide for the definition of reliability program plans for nuclear powergenerating stations. IEEE Std. 933-1999, 1999.

[29] Ingram-Cotton J.B., Hecht M.J., Duphily R.J., Zambrana M., Hiramoto T.,O’connor C. Reliability program requirements for space systems. AerospaceReport NO.TOR-2007(8583)-6889, US The Aerospace Corporation, 2007.

[30] MIL-STD-785REVB reliability program for systems and equipment devel-opment and production. MIL-STD-785B, US Department of Defense, 1980.

[31] Agarwal H. Reliability based design optimization formulations and meth-odologies. PhD dissertation, University of Notre Dame, 2004.

[32] Mohan NS. Robust design. PhD dissertation, Indian Institute of Technology,2002.

[33] Taguchi G, Elsayed EA, Hsiang TC. Quality engineering in productionsystems. New York: McGraw-Hill; 1989.

[34] Park SH. Robust design and analysis for quality engineering. London:Chapman and Hall; 1996.

[35] Park G, Lee T, Lee KH, Hwang K. Robust optimization: an overview.AIAA Journal 2006;44(1):181–91.

[36] Beyer H, Sendhoff B. Robust optimization: a comprehensive survey. Com-puter Methods in Applied Mechanics and Engineering 2007;196(33–34):3190–218.

[37] Mourelatos ZP, Jinghong L. A methodology for trading-off performance androbustness under uncertainty. Journal of Mechanical Design 2006;128(4):856–63.

[38] Uebelhart SA, Millery DW, Blaurock C. Uncertainty characterization inintegrated modeling. In: 46th AIAA/ASME/ASCE/AHS/ASC structures, struc-tural dynamics and materials, Austin, Texas, AIAA-2005-2142, 2005.

[39] Neumaier A, Fuchs M, Dolejsi E, Csendes T, Dombi J, Banhelyi B, et al.Application of clouds for modeling uncertainties in robust space systemdesign. ACT Ariadna Research ACT-RPT-05-5201. European Space Agency;2007.

[40] Wang Z, Chen X, Luo W, Zhang W. Research on theory and application ofmultidisciplinary design optimization of flight vehicles. Beijing: NationalDefense Industry Press; 2006.

[41] Hoffman FO, Hammonds JS. Propagation of uncertainty in risk assessments:the need to distinguish between uncertainty due to lack of knowledge anduncertainty due to variability. Risk Analysis 1994;14(5):707–12.

[42] Helton JC, Burmaster DE. Treatment of aleatory and epistemic uncertaintyin performance assessments for complex systems. Reliability Engineeringand System Safety 1996;54(2–3):91–4.

[43] Roy CJ, Oberkampf WL. A complete framework for verification, validation,and uncertainty quantification in scientific computing. In: 48th AIAAaerospace sciences meeting including the New Horizons forum and aero-space exposition, Orlando, Florida, AIAA-2010-124, 2010.

[44] Draper D. Assessment and propagation of model uncertainty. Journal of theRoyal Statistical Society: Series B (Statistical Methodology) 1995;57(1):45–97.

[45] Fersona S, Joslyn CA, Helton JC, Oberkampf WL, Sentz K. Summary from theepistemic uncertainty workshop: consensus amid diversity. ReliabilityEngineering and System Safety 2004;85(1–3):355–69.

[46] Thunnissen DP. Uncertainty classification for the design and developmentof complex systems. In: The third annual predictive methods conference,Newport Beach, CA, 2003.

[47] Thunnissen DP. Propagating and mitigating uncertainty in the design ofcomplex multidisciplinary systems. PhD dissertation, California Institute ofTechnology, 2005.

[48] Walton MA. Managing uncertainty in space systems conceptual designusing portfolio theory. PhD dissertation, Massachusetts Institute of Tech-nology, 2002.

[49] Batill SM, Renaud JE, Gu X. Modeling and simulation uncertainty in multi-disciplinary design optimization. In: Eighth AIAA/USAF/NASA/ISSMOsymposium on multidisciplinary analysis and optimization, Long Beach,CA, AIAA-2000-4803, 2000.

[50] Du X, Chen W. A methodology for managing the effect of uncertainty insimulation-based design. AIAA Journal 2000;38(8):1471–8.

[51] de Weck O. Claudia Eckert, Clarkson J. A classification of uncertainty forearly product and system design. In: International conference on engineer-ing design, Paris, France, ICED’07/No.480, 2007.

[52] McKay MD, Morrison JD, Upton SC. Evaluating prediction uncertainty insimulation models. Computer Physics Communications 1999;117(1-2):44–51.

[53] Oberkampf WL, Deland SM, Rutherford BM. Estimation for total uncertaintyin modeling and simulation. SAND2000-0824. Sandia National laboratories;2000.

[54] Oberkampf WL, Helton JC, Sentz K. Mathematical representation of uncer-tainty. In: Non-deterministic approaches forum, WA, Seattle, AIAA-2001-1645, 2001.

[55] Klir GJ. Uncertainty and information measures for imprecise probabilities:an overview. In: First international symposium on imprecise probabilitiesand their applications, Ghent, Belgium, 1999.

[56] Walley P. Towards a unified theory of imprecise probability. InternationalJournal of Approximate Reasoning 2000;24(2–3):125–48.

[57] Haimes YY, Barry T, Lambert JH. When and how can you specify a probabilitydistribution when you don’t know much? Risk Analysis 1994;14(5):661–706.

dx.doi.org/10.1002/pse.176


[58] Hattisl D, Burmaste DE. Assessment of variability and uncertainty distribu-tions for practical risk analyses. Risk Analysis 1994;14(5):713–30.

[59] Rice JA. Mathematical statistics and data analysis. 3rd ed. California:Duxbury Press; 2006.

[60] Marhadi K, Venkataraman S, Pai SS. Quantifying uncertainty in statisticaldistribution of small sample data using Bayesian inference of unboundedJohnson distribution. In: 49th AIAA/ASME/ASCE/AHS/ASC structures, struc-tural dynamics, and materials conference, Schaumburg, IL, AIAA-2008-1810, 2008.

[61] Ryan RS, Townsend JS. Application of probabilistic analysis design methodsin space programs—the approaches, the status, and the needs. In: 34thAlAA/ASME/ASCE/AHS/ASC structures, structural dynamics and materialsconference, AIAA/ASME Adaptive Structures Forum, La Jolla, CA, AIAA-93-1381, 1993.

[62] Smith N, Mahadevan S. Probabilistic methods for aerospace system con-ceptual design. Journal of Spacecraft and Rockets 2003;40(3):411–8.

[63] Nam T, Sobany DS, Mavris DN. A non-deterministic aircraft sizing methodunder probabilistic design constraints. In: 47th AIAA/ASME/ASCE/AHS/ASCstructures, structural dynamics, and materials conference, Newport, RhodeIsland, AIAA-2006-2062, 2006.

[64] Merrill RG, Andraschko M, Stromgren C, Cirillo B, Earle K, Goodliff K.A comparison of probabilistic and deterministic campaign analysis forhuman space exploration. In: AIAA SPACE 2008 conference & exposition,San Diego, California, AIAA-2008-7748, 2008.

[65] Hassan R, Crossley W. Spacecraft reliability-based design optimizationunder uncertainty including discrete variables. Journal of Spacecraft andRockets 2008;45(2):394–405.

[66] Youn BD, Wang P. Bayesian reliability based design optimization underboth aleatory and epistemic uncertainties. In: 11th AIAA/ISSMO multi-disciplinary analysis and optimization conference, Portsmouth, Virginia,AIAA-2006-6928, 2006.

[67] Wang P, Youn BD, Xi Z, Kloess A. Bayesian reliability analysis with evolving,insufficient, and subjective data sets. Journal of Mechanical Design2009;131(11):111008.

[68] Yu BH, Van Kuiken CJA, Telford DG. Managing uncertainty in reliabilityanalysis with Bayesian inference and uncertainty propagation. In: US AirForce T&E Days 2010, Nashville, Tennessee, AIAA-2010-2594, 2010.

[69] Wang P, Youn BD. Efficient Bayesian reliability analysis and design with auser-defined confidence level. In: 51st AIAA/ASME/ASCE/AHS/ASC struc-tures, structural dynamics, and materials conference, Orlando, Florida,AIAA-2010-2590, 2010.

[70] Yager R, Kacprzy KJ, Fedrizzi M. Advances in the Dempster—Shafer theoryof evidence. New York: John Wiley and Sons; 1994.

[71] Sentz K, Ferson S. Combination of evidence in Dempste–Shafer theory.SAND2002-0835. Sandia National Laboratories; 2002.

[72] Helton JC, Johnson JD, Oberkampf WL, Sallaberry CJ. Representation ofanalysis results involving aleatory and epistemic uncertainty. SAND2008-4379. Sandia National Laboratories; 2008.

[73] Shafer G. A mathematical theory of evidence. Princeton, New Jersey:Princeton University Press; 1976.

[74] Mourelatos Z, Zhou J. A design optimization method using evidence theory.Journal of Mechanical Design 2006;128(4):901–8.

[75] Heltona JC, Johnson JD, Oberkampf WL, Storlie CB. A sampling-basedcomputational strategy for the representation of epistemic uncertainty inmodel predictions with evidence theory. Computer Methods in AppliedMechanics and Engineering 2007;196(37–40):3980–98.

[76] Oberkampf WL, Helton JC. Investigation of evidence theory for engineeringapplications. In: Fourth non-deterministic approaches forum, Denver,Colorado, AIAA-2002-1569, 2002.

[77] Agarwal H, Renaud JE, Preston EL, Padmanabhan D. Uncertainty quantifica-tion using evidence theory in multidisciplinary design optimization. Relia-bility Engineering and System Safety 2004;85(1–3):281–94.

[78] Croisarda N, Vasilea M, Kembleb S, Radicea G. Preliminary space missiondesign under uncertainty. Acta Astronautica 2010;66(5–6):654–64.

[79] Zadeh LA. Fuzzy sets as a basis for a theory of possibility. Fuzzy Sets andSystems 1978;1(1):3–28.

[80] Zadeh LA. Fuzzy sets. Information and Control 1965;8(3):338–53.[81] Maglaras G, Nikolaidis E, Haftka RT, Cudney HH. Analytical-experimental

comparison of probabilistic methods and fuzzy set based methods fordesigning under uncertainty. Structural and Multidisciplinary Optimization1997;13(2–3):69–80.

[82] Ferrari P, Savoia M. Fuzzy number theory to obtain conservative resultswith respect to probability. Computer Methods in Applied Mechanics andEngineering 1998;160(3–4):205–22.

[83] Chen S. Comparing probabilistic and fuzzy set approaches for design in thepresence of uncertainty. PhD dissertation, Virginia Polytechnic Institute andState University, 2000.

[84] Nikolaidis E, Chen S, Cudney H, Haftka RT, Rosca R. Comparison ofprobability and possibility for design against catastrophic failure underuncertainty. Journal of mechanical design 2004;126(3):386–94.

[85] Braibant V, Oudshoorn A, Boyer C, Delcroix F. Non-deterministic possibilis-tic approaches for structural analysis and optimal design. AIAA Journal1999;37(10):1298–303.

[86] Youn B, Choi KK, Du L, Gorsich D. Integration of possibility-based optimiza-tion to robust design for epistemic uncertainty. In: Sixth world congresses ofstructural and multidisciplinary optimization, Rio de Janeiro, Brazil, 2005.

[87] Mourelatos ZP, Zhou J. Reliability estimation and design with insufficientdata based on possibility theory. AIAA Journal 2005;43(8):1696–705.

[88] He L, Huang H, Du L, Zhang X, Miao Q. A review of possibilistic approachesto reliability analysis and optimization in engineering design. In: Jacko J,editor. Human–computer interaction, part IV, HCII 2007, Lecture notes incomputer science, vol. 4553. Springer-Verlag; 2007. p. 1075–84.

[89] Langley RS. Unified approach to probabilistic and possibilistic analysis ofuncertain systems. Journal of Engineering Mechanics 2000;126(11):1163–72.

[90] Shin Y, Wu Y. A hybrid possibilistic–probabilistic analysis framework foruncertainty management. In: 44th AIAA/ASME/ASCE/AHS structures, struc-tural dynamics, and materials, Norfolk, Virginia, AIAA-2003-1573, 2003.

[91] Du L, Choi KK, Youn BD, Gorsich D. Possibility-based design optimizationmethod for design problems with both statistical and fuzzy input data. In:Sixth world congresses of structural and multidisciplinary optimization, Riode Janeiro, Brazil, 2005.

[92] Rao SS, Berke L. Analysis of uncertain structural systems using intervalanalysis. AIAA Journal 1997;35(4):727–35.

[93] Rao SS, Lingtao C. Optimum design of mechanical systems involvinginterval parameters. Journal of Mechanical Design 2002;124(3):465–72.

[94] Majumder L, Rao SS. Interval-based multi-objective optimization of aircraftwings under gust loads. AIAA Journal 2009;47(3):563–75.

[95] Moore RE, Kearfott RB, Cloud MJ. Introduction to interval analysis. Phila-delphia: SIAM Press; 2009.

[96] Ben-Haim Y, Elishakoff I. Convex models of uncertainty in appliedmechanics. Amsterdam: Elsevier; 1990.

[97] Ben-Haim Y. A non-probabilistic concept of reliability. Structural Safety1994;14(4):227–45.

[98] Ben-Haim Y. Convex models of uncertainty: applications and implications.Erkenntnis 1994;41(2):139–56.

[99] Ben-Haim Y. Info-gap decision theory: decisions under severe uncertainty.2nd ed. Amsterdam: Academic Press; 2006.

[100] Ben-Haim Y. Uncertainty, probability and information-gaps. ReliabilityEngineering and System Safety 2004;85(1–3):249–66.

[101] Fuchs M, Neumaier A. Uncertainty modeling with clouds in autonomousrobust design optimization. In: Proceedings of the third internationalworkshop reliable engineering computing, 2008.

[102] Fuchs M, Neumaier A. Handling uncertainty in higher dimensions withpotential clouds towards robust design optimization. Advances in Intelli-gent and Soft Computing 2008;48:376–82.

[103] Liu B. Uncertainty theory an introduction to its axiomatic foundations.Berlin: Springer-Verlag; 2004.

[104] Klir GJ, Smith RM. On measuring uncertainty and uncertainty-basedinformation: recent developments. Annals of Mathematics and ArtificialIntelligence 2001;32(1-4):1012–2443.

[105] Helton JC, Oberkampf WL. Alternative representations of epistemic uncer-tainty. Reliability Engineering and System Safety 2004;85(1–3):1–10.

[106] Helton JC, Johnsonb JD, Oberkampf WL. An exploration of alternativeapproaches to the representation of uncertainty in model predictions.Reliability Engineering and System Safety 2004;85(1–3):39–71.

[107] Laskey KB. Model uncertainty: theory and practical implications. IEEETransactions on System, Man, and Cybernetics—Part A: System and Human1996;26(3):340–8.

[108] Rebba R, Mahadevan S, Zhang R. Validation of uncertainty propagationmodels. In: 44th AIAA/ASME/ASCE/AHS structures, structural dynamics, andmaterials conference, Norfolk, Virginia, AIAA-2003-1913, 2003.

[109] Mahadevan S, Rebba R. Validation of reliability computational models usingBayes networks. Reliability Engineering and System Safety 2005;87(2):223–32.

[110] Faragher J. Probabilistic methods for the quantification of uncertainty anderror in computational fluid dynamics simulations. DSTO-TR-1633. DSTOPlatforms Sciences Laboratory; 2004.

[111] Oberkamp WL, Trucano TG. Verification and validation in computationalfluid dynamics. Progress in Aerospace Sciences 2002;38(3):209–72.

[112] Ivo B, Tinsley OJ. The reliability of computer predictions: can they be trusted?International Journal of Numerical Analysis and Modeling 2005;1(1):1–18.

[113] Anderson TW, Darling DA. Asymptotic theory of certain ’’goodness-of-fit’’criteria based on stochastic processes. Annals of Mathematical Statistics1952;23(2):193–212.

[114] Stephens MA. EDF statistics for goodness of fit and some comparisons.Journal of the American Statistical Association 1974;69(347):730–7.

[115] Anderson TW, You L. Adequacy of asymptotic theory for goodness-of-fitcriteria for specific distributions. Journal of Time Series Analysis 2008;17(6):533–52.

[116] Bichon BJ, McFarlandy JM, Mahadevan S. Using Bayesian inference andefficient global reliability analysis to explore distribution uncertainty. In:49th AIAA/ASME/ASCE/AHS/ASC structures, structural dynamics, and mate-rials conference, Schaumburg, IL, AIAA-2008-1712, 2008.

[117] McFarland JM, Bichon BJ. Bayesian model averaging for reliability analysiswith probability distribution model form uncertainty. In: 50th AIAA/ASME/ASCE/AHS/ASC structures, structural dynamics, and materials conference,Palm Springs, California, AIAA-2009-2231, 2009.

[118] Zhang R, Mahadevan S. Model uncertainty and Bayesian updating inreliability-based inspection. Structural Safety 2000;22(2):145–60.

[119] Adrian R, Madigan D, Hoeting J. Model selection and accounting for modeluncertainty in linear regression models. Journal of the American StatisticalAssociation 1993;92:179–91.


[120] Roache PJ. Verification and validation in computational science and engi-neering. New Mexico: Hermosa Publishers; 1998.

[121] Roy CJ. Review of discretization error estimators in scientific computing. In:48th AIAA aerospace sciences meeting, Orlando, FL, AIAA-2010-0126, 2010.

[122] Cavallo PA, Sinha N. Error quantification for computational aerodynamicsusing an error transport equation. Journal of Aircraft 2007;44(6):1954–63.

[123] Shih TI, Williams BR. Development and evaluation of an a posteriori methodfor estimating and correcting grid-induced errors in solutions of the Navier–Stokes equations. In: Proceedings of the 47th AIAA aerospace sciencesmeeting including the New Horizons forum and aerospace exposition, 2009.

[124] Oden JT, Prudhomme S. Goal-oriented error estimation and adaptivity forthe finite element method. Computers and Mathematics with Applications2001;41(5–6):735–56.

[125] Oden JT, Prudhomme S, Bauman P. On the extension of goal-oriented errorestimation and hierarchical modeling to discrete lattice methods. ComputerMethods in Applied Mechanics and Engineering 2005;194(34–35):3668–88.

[126] Fuentes D, Littlefield D, Oden JT, Prudhomme S. Error control in finiteelement approximations of nonlinear problems in mechanics. In: Proceed-ings of the second international conference on adaptive modeling andsimulation, 2005.

[127] Abdulle A. On a priori error analysis of fully discrete heterogeneous multi-scale FEM. Multiscale Modeling and Simulation 2005;4(2):447–59.

[128] Ainsworth M, Oden JT. A posteriori error estimation in finite elementanalysis. New York: Wiley Interscience; 2000.

[129] Zienkiewicz OC, Zhu JZ. The superconvergent patch recovery and a poster-iori error estimates, Part 2: error estimates and adaptivity. InternationalJournal for Numerical Methods in Engineering 1992;33(7):1365–82.

[130] Oden JT, Prudhomme S. Estimation of modeling error in computationalmechanics. Journal of Computational Physics 2002;182(2):496–515.

[131] Babuska I, Oden JT. Verification and validation in computational engineer-ing and science: basic concepts. Computer Methods in Applied Mechanicsand Engineering 2004;193(36–38):4057–66.

[132] Apley DW, Liu J, Chen W. Understanding the effects of model uncertainty inrobust design with computer experiments. Journal of Mechanical Design2006;128(4):945–58.

[133] Rebba R, Mahadevan S. Statistical methods for model validation underuncertainty. In: Proceedings of the 47th AIAA/ASME/ASCE/AHS/ASC struc-tures, structural dynamics, and materials conference, 2006.

[134] Rebbaa R, Mahadevan S, Huang S. Validation and error estimation of com-putational models. Reliability Engineering and System Safety 2006;91(10–11):1390–7.

[135] Saltelli A, Ratto M, Andres T, Campolongo F, Cariboni J, Gatelli D, et al.Global sensitivity analysis: the primer. Chichester: John Wiley and Sons;2008.

[136] Hofer E. Sensitivity analysis in the context of uncertainty analysis forcomputationally intensive models. Computer Physics Communications1999;117(1–2):21–34.

[137] Helton JC, Johnson JD, Salaberry CJ, Storlie CB. Survey of sampling-basedmethods for uncertainty and sensitivity analysis. Reliability Engineeringand System Safety 2006;91(10–11):1175–209.

[138] Iman RL, Helton JC. An investigation of uncertainty and sensitivity analysistechniques for computer models. Risk Analysis 1988;8(1):71–90.

[139] Cacuci DG, Ionescu-Bujor M. A comparative review of sensitivity anduncertainty analysis of large-scale systems—II: statistical methods. NuclearScience and Engineering 2004;147(3):204–17.

[140] Liu H, Chen W, Sudjianto A. Probabilistic sensitivity analysis methods fordesign under uncertainty. In: Proceedings of the 10th AIAA/ISSMO multi-disciplinary analysis and optimization conference, 2004.

[141] Helton JC, Davis FJ. Latin hypercube sampling and the propagation ofuncertainty in analyses of complex systems. Reliability Engineering andSystem Safety 2003;81(1):23–69.

[142] Morris MD. Factorial sampling plans for preliminary computational experi-ments. Technometrics 1991;33(2):161–74.

[143] Campolongo F, Cariboni J, Saltelli A. An effective screening design forsensitivity analysis of large models. Environmental Modelling and Software2007;22(10):1509–18.

[144] Helton JC, Johnson JD, Oberkampf WL, Sallaberry CJ. Sensitivity analysis inconjunction with evidence theory representations of epistemic uncertainty.Reliability Engineering and System Safety 2006;91(10–11):1414–34.

[145] Oberguggenberger M, King J, Schmelzer B. Classical and imprecise prob-ability methods for sensitivity analysis in engineering: a case study.International Journal of Approximate Reasoning 2009;50(4):680–93.

[146] Bae HR, Grandhi RV, Canfield RA. Sensitivity analysis of structural responseuncertainty propagation using evidence theory. Structural and Multidisci-plinary Optimization 2006;31(4):270–91.

[147] Guo J, Du X. Sensitivity analysis with mixture of epistemic and aleatoryuncertainties. AIAA Journal 2007;45(9):2337–49.

[148] Guo J. Uncertainty analysis and sensitivity analysis for multidisciplinarysystems design. PhD dissertation, Missouri University of Science andTechnology, 2008.

[149] Keane AJ, Nair PB. Computational approaches for aerospace design: thepursuit of excellence. Chichester, West Sussex: John Wiley and Sons; 2005.

[150] Wiener N. The homogeneous chaos. American Journal of Mathematics1938;60(4):897–936.

[151] Ghanem R, Spanos P. Stochastic finite elements: a spectral approach.New York: Springer-Verlag; 1991.

[152] Le Maitre OP, Knio OM, Najm HN, Ghanem RG. A stochastic projectionmethod for fluid flow I. Basic formulation. Journal of Computational Physics2001;173(2):481–511.

[153] Le Maitre OP, Reagan MT, Najm HN, Ghanem RG, Knio OM. A stochasticprojection method for fluid flow II. Random process. Journal of Computa-tional Physics 2002;181(1):9–44.

[154] Xiu D, Karniadakis GE. The Wiener-Askey polynomial chaos for stochasticdifferential equations. SIAM Journal of Scientific Computing 2002;24:619–44.

[155] Eldred MS, Burkardt J. Comparison of non-intrusive polynomial chaos andstochastic collocation methods for uncertainty quantification. In: Proceed-ings of the 47th AIAA aerospace sciences meeting including The NewHorizons forum and aerospace exposition, 2009.

[156] Eldred MS. Recent advances in non-intrusive polynomial chaos and sto-chastic collocation methods for uncertainty analysis and design. In: Pro-ceedings of the 50th AIAA/ASME/ASCE/AHS/ASC structures, structuraldynamics, and materials conference, 2009.

[157] Helton JC, Johnson JD, Sallaberry CJ, Storlie CB. Survey of sampling-basedmethods for uncertainty and sensitivity analysis. SAND2006-2901. SandiaNational Laboratories; 2006.

[158] Landau DP, Binder K. A guide to Monte Carlo simulations in statisticalphysics. 2nd ed. New York: Cambridge University Press; 2005.

[159] Davis PJ, Rabinowitz P. Methods of numerical integration. 2nd ed. Orlando:Academic Press; 1984.

[160] Li KS, Lumb P. Reliability analysis by numerical integration and curve fitting.Structural Safety 1985;3(1):29–36.

[161] Bernardo FP, Pistikopoulos EN, Saraiva PM. Integration and computationalissues in stochastic design and planning optimization problems. Industrialand Engineering Chemistry Research 1999;38(8):3056–68.

[162] Yiben L. A quadrature-based technique for robust design with computersimulations. PhD dissertation, Massachusetts Institute of Technology, 2007.

[163] Evans M, Swartz T. Methods for approximating integrals in statistics withspecial emphasis on Bayesian integration problems. Statistical Science1995;10(3):254–72.

[164] Monahan JF. Numerical methods of statistics. Cambridge: Cambridge Uni-versity Press; 2001.

[165] Barry T. Recommendations on the testing and use of pseudo-randomnumber generators used in Monte Carlo analysis for risk assessment. RiskAnalysis 1996;16(1):93–105.

[166] Christian R, George C. Monte Carlo statistical methods.2nd ed. US: Springer;2004.

[167] Ang GL, Tang WH. Optimal importance-sampling density estimator. Journalof Engineering Mechanics 1992;118(6):1146–63.

[168] Hinrichs A. Optimal importance sampling for the approximation of inte-grals. Journal of Complexity 2010;26(2):125–34.

[169] Cao Y, Hussaini MY, Zang TA. On the exploitation of sensitivity derivativesfor improving sampling methods. In: Proceedings of the 44th AIAA struc-tures, structural dynamics and mechanics conference, 2003.

[170] Heltona JC, Johnson JD, Oberkampf WL, Storlie CB. A sampling-basedcomputational strategy for the representation of epistemic uncertainty inmodel predictions with evidence theory. SAND2006-5557. Sandia NationalLaboratories; 2006.

[171] Kreinovich V, Ferson SA. A new Cauchy-based black-box technique foruncertainty in risk analysis. Reliability Engineering and System Safety2004;85(1–3):267–79.

[172] Swiler LP, Paez TL, Mayes RL. Epistemic uncertainty quantification tutorial.In: Proceedings of the IMAC-XXVII, 2009.

[173] Helton JC. Conceptual and computational basis for the quantification ofmargins and uncertainty. SAND2009-3055. Sandia National Laboratories;2009.

[174] Gu X, Renaud JE, Batill SM, Brach RM, Budhiraja AS. Worst case propagateduncertainty of multidisciplinary systems in robust design optimization.Journal of Structural and Multidisciplinary Optimization 2000;20(3):190–213.

[175] Cao H, Duan B. Uncertainty analysis for multidisciplinary systems based onconvex models. In: Proceedings of the 10th AIAA/ISSMO multidisciplinaryanalysis and optimization conference, 2004.

[176] Du X, Chen W. An efficient approach to probabilistic uncertainty analysis insimulation-based multidisciplinary design. In: Proceedings of the 38th AIAAaerospace sciences meeting and exhibit, 2000.

[177] Hahn G, Shapiro S. Statistical models in engineering. New York: Wiley;1967.

[178] Wong FS. First-order, second-moment methods. Computers and Structures1985;20(4):779–91.

[179] Green LL, Lin H, Khalessi MR. Probabilistic methods for uncertainty prop-agation applied to aircraft design. In: Proceedings of the 20th AIAA appliedaerodynamics conference, 2002.

[180] Rosenblueth E. Point estimates for probability moments. Proceedings of theNational Academy of Sciences of the United States of America 1975;72(10):3812–4.

[181] Melchers RE. Structural reliability analysis and prediction. Chichester:John Wiley and Sons; 1999.

[182] Breitung K. Asymptotic approximations for probability integrals. Probabil-istic Engineering Mechanics 1989;4(4):187–90.

[183] Song BF. A numerical integration method for computing structural systemreliability. Computers and Structures 1990;36(1):65–70.


[184] Sakamoto J, Mori Y, Sekioka T. Probability analysis method using fastFourier transform and its application. Structural Safety 1997;19(1):21–36.

[185] Penmetsa RC, Grandhi RV. Adaptation of fast Fourier transformations toestimate structural failure probability. Finite Elements in Analysis andDesign 2003;39(5–6):473–85.

[186] Chen X, Lind NC. Fast probability integration by three-parameter normal tailapproximation. Structural Safety 1982;1(4):269–76.

[187] Kim NH, Ramu P. Tail modeling in reliability-based design optimization forhighly safe structural systems. In: Proceedings of the 47th AIAA/ASME/ASCE/AHS/ASC structures, structural dynamics, and materials conference,2006.

[188] Acar E, Rais-Rohani M, Eamon CD. Reliability estimation using dimensionreduction and extended generalized lambda distribution. In: Proceedings ofthe 49th AIAA/ASME/ASCE/AHS/ASC structures, structural dynamics, andmaterials conference, 2008.

[189] Rahman S, Xu H. A univariate dimension-reduction method for multi-dimensional integration in stochastic mechanics. Probabilistic EngineeringMechanics 2004;19(4):393–408.

[190] Youn BD, Zhimin X, Wells L, Lamb D. Enhanced dimension-reduction (eDR)method for reliability-based robust design optimization. In: Proceedings ofthe 11th AIAA/ISSMO multidisciplinary analysis and optimization confer-ence, 2006.

[191] Rackwitz R. Reliability analysis—a review and some perspectives. StructuralSafety 2001;23:365–95.

[192] Hohenbichler M, Gollwitzer S, Kruse W, Rackwitz R. New light on first- andsecond-order reliability methods. Structural Safety 1987;4(4):267–84.

[193] Zhao Y, Ono T. A general procedure for first/second-order reliability method(FORM/SORM). Structural Safety 1999;21(2):95–112.

[194] Rosenblatt M. Remarks on a multivariate transformation. Annals of Math-ematical Statistics 1952;23(3):470–2.

[195] Hasofer AM, Lind NC. Exact and invariant second-moment code format.Journal of Engineering Mechanics 1974;100(1):111–21.

[196] Rackwitz R, Fiessler B. Structural reliability under combined random loadsequences. Computers and Structures 1978;9(5):489–94.

[197] Peiling L, Der KA. Optimization algorithms for structural reliability. Struc-tural Safety 1991;9(3):161–77.

[198] Sim~oes LMC. A branch and bound strategy for finding the reliability index withnon-convex performance functions. Structural Safety 1988;5(2):95–108.

[199] Yang D, Li G, Cheng G. Convergence analysis of first order reliability methodusing chaos theory. Computers and Structures 2006;84(8–9):563–71.

[200] Lee J, Yang Y, Ruy W. A comparative study on reliability-index and target-performance-based probabilistic structural design optimization. Computersand Structures 2002;80(3–4):257–69.

[201] Choi KK, Youn BD. On probabilistic approaches for reliability-based designoptimization (RBDO). In: Proceedings of the ninth AIAA/ISSMO symposiumon multidisciplinary analysis and optimization, 2002.

[202] Tu J, Choi KK, Young HP. A new study on reliability-based design optimiza-tion. Journal of Mechanical Design 1999;121(4):557–64.

[203] Youn BD, Choi KK, Park YH. Hybrid analysis method for reliability-baseddesign optimization. Journal of Mechanical Design 2001;125(2):221–32.

[204] Youn BD, Choi KK, Du L. Adaptive probability analysis using an enhancedhybrid mean value method. Journal of Structural and MultidisciplinaryOptimization 2005;29(2):134–48.

[205] Du X, Sudjianto A, Chen W. An integrated framework for optimization underuncertainty using inverse reliability strategy. In: Proceedings of the DETC’03ASME 2003 design engineering technical conferences and computers andinformation in engineering conference, 2004.

[206] Choi K, Youn B. An investigation of the nonlinearity of reliability-baseddesign optimization. In: Proceedings of the 28th ASME design automationconference, 2002.

[207] Der Kiureghian A, Zhang Y, Li C. Inverse reliability problem. ASCE Journal ofEngineering Mechanics 1994;120(5):1150–9.

[208] Li H, Foschi R. An inverse reliability method and its application. StructuralSafety 1998;20(3):257–70.

[209] Ramu P, Qu X, Youn BD, Choi KK. Safety factor and inverse reliabilitymeasures. In: Proceedings of the 45th AIAA/ASME/ASCE/AHS/ASC struc-tures, structural dynamics and materials conference, 2004.

[210] Breitung K. Asymptotic approximations for multinormal integrals. Journal ofEngineering Mechanics 1984;110(3):357–66.

[211] Cai GQ, Elishakoff I. Refined second-order reliability analysis. StructuralSafety 1994;14(4):267–76.

[212] Grandhi RV, Wang L. Higher-order failure probability calculation usingnonlinear approximations. Computer Methods in Applied Mechanics andEngineering 1999;168(1–4):185–206.

[213] Zhao Y, Ono T, Kato M. Second-order third-moment reliability method.Journal of Structural Engineering 2002;128(8):1087–90.

[214] Zhao Y, Ono T. Moment methods for structural reliability. Structural Safety2001;23(1):47–75.

[215] Du X, Chen W. A most probable point based method for uncertaintyanalysis. In: Proceedings of the DETC’00, ASME 2000 design engineeringtechnical conferences and computers and information in engineeringconference.

[216] Elishakoff I, Hasofer AM. Exact versus approximate determination ofstructural reliability. Applied Scientific Research 1987;44(3):303–12.

[217] Ditlevsen O. Generalized second moment reliability index. Structures andMachines 1979;7(4):435–51.

[218] Engelund S, Rackwit R. A benchmark study on importance samplingtechniques in structural reliability. Structural Safety 1993;12(4):255–76.

[219] Au SK, Beck JL. Important sampling in high dimensions. Structural Safety2003;25(2):139–63.

[220] Li F, Wu T. An importance sampling based approach for reliability analysis.In: Proceedings of the third annual IEEE conference on automation scienceand engineering, 2007.

[221] Patelli E, Pradlwarter HJ, Schueller GI. On multinormal integrals by impor-tance sampling for parallel system reliability. Structural Safety 2010,doi:10.1016/j.strusafe.2010.04.002.

[222] Bucher CG. Adaptive sampling—an iterative fast Monte Carlo method.Structural Safety 1988;5(2):119–26.

[223] Au S, Beck JL. Estimation of small failure probabilities in high dimensions bysubset simulation. Probabilistic Engineering Mechanics 2001;16(4):263–77.

[224] Koutsourelakis PS, Pradlwarter HJ, Schueller GI. Reliability of structures inhigh dimensions, Part I: algorithms and applications. Probabilistic Engineer-ing Mechanics 2004;19(4):409–17.

[225] Pradlwarter HJ, Pellissetti MF, Schenk CA, Schuellera GI, Kreis A, Fransen S,et al. Realistic and efficient reliability estimation for aerospace structures.Computer Methods in Applied Mechanics and Engineering 2005;194(12–16):1597–617.

[226] Pellissetti MF, Schueller GI, Pradlwarter HJ, Calvi A, Fransen S, Klein M.Reliability analysis of spacecraft structures under static and dynamicloading. Computers and Structures 2006;84(21):1313–25.

[227] Ditlevsen O, Melchers RE, Gluver H. General multi-dimensional probabilityintegration by directional simulation. Computers and Structures1990;36(2):355–68.

[228] Melchers RE. Structural system reliability assessment using directionalsimulation. Structural Safety 1994;16(1–2):23–37.

[229] Nie J, Ellingwood BR. Directional methods for structural reliability analysis.Structural Safety 2000;22(3):233–49.

[230] Du X, Chen W. Towards a better understanding of modeling feasibilityrobustness in engineering design. Journal of Mechanical Design 2000;122(4):385–94.

[231] Schueller GI, Pradlwarter HJ, Koutsourelakis PS. A critical appraisal ofreliability estimation procedures for high dimensions. Probabilistic Engi-neering Mechanics 2004;19(4):463–74.

[232] Rajashekhar MR, Ellingwood BR. A new look at the response surfaceapproach for reliability analysis. Structural Safety 1993;12(3):205–20.

[233] Zou T, Mahadevan S, Mourelatos ZP. Reliability analysis with adaptiveresponse surfaces. In: Proceedings of the 44th AIAA/ASME/ASCE/AHSstructures, structural dynamics, and materials conference, 2003.

[234] Buchera C, Most T. A comparison of approximate response functions instructural reliability analysis. Probabilistic Engineering Mechanics2008;23(2–3):154–63.

[235] Bichon BJ, McFarland JM, Mahadevan S. Applying EGRA to reliabilityanalysis of systems with multiple failure modes. In: Proceedings of the51st AIAA/ASME/ASCE/AHS/ASC structures, structural dynamics, and mate-rials conference, 2010.

[236] Guo J, Du X. Reliability analysis for multidisciplinary systems with randomand interval variables. AIAA Journal 2010;48(1):82–91.

[237] Braibant V, Oudshoorn A, Boyer C. Nondeterministic ’’possibilistic’’approaches for structural analysis and optimal design. AIAA Journal1999;37(10):1298–303.

[238] Du L, Choi K, Youn B. Inverse possibility analysis method for possibility-based design optimization. AIAA Journal 2006;44(11):2682–90.

[239] Adduri PR, Penmetsa RC. System reliability analysis for mixed uncertainvariables. Structural Safety 2009;31(5):375–82.

[240] Du X. Unified uncertainty analysis by the first order reliability method.Journal of Mechanical Design 2008;130(9):91401.

[241] Luo Y, Kang Z, Li A. Structural reliability assessment based on probabilityand convex set mixed model. Computers and Structures 2009;87(21–22):1408–15.

[242] Mori Y, Ellingwood BR. Time-dependent system reliability analysis byadaptive importance sampling. Structural Safety 1993;12(1):59–73.

[243] Kuschel N, Rackwitz R. Optimal design under time-variant reliabilityconstraints. Structural Safety 2000;22(2):113–27.

[244] Cui W, Blockley DI. On the bounds for structural system reliability.Structural Safety 1991;9(4):247–59.

[245] Ramachandran K. System reliability bounds: a new look with improve-ments. Civil Engineering and Environmental Systems 2004;21(4):265–78.

[246] Tonon F, Bae H, Grandhi RV, Pettit CL. Using random set theory to calculatereliability bounds for a wing structure. Structure and Infrastructure Engi-neering 2006;2(3–4):191–200.

[247] Adduri PR, Penmetsa RC. Bounds on structural system reliability in thepresence of interval variables. Computers and Structures 2007;85(5–6):320–9.

[248] Kang W, Song J, Gardoni P. Matrix-based system reliability method andapplications to bridge networks. Reliability Engineering and System Safety2008;93(11):1584–93.

[249] Song J, Kang W. System reliability and sensitivity under statistical depen-dence by matrix-based system reliability method. Structural Safety2009;31(2):148–56.

[250] Nguyen TH, Song J, Paulino GH. Single-loop system reliability-based designoptimization using matrix-based system reliability method theory andapplications. Journal of Mechanical Design 2010;132(1):11005–11.

dx.doi.org/10.1016/j.strusafe.2010.04.002


[251] Wang P, Youn BD, Hu C. A generalized complementary intersection method(CIM) for system reliability analysis. In: Proceedings of the 50th AIAA/ASME/ASCE/AHS/ASC structures, structural dynamics, and materials con-ference, 2009.

[252] Youn BD, Wang P. Complementary intersection method for system relia-bility analysis. Journal of Mechanical Design 2009;131(4):41004.

[253] Park S, Choi S, Sikorsky C, Stubbs N. Efficient method for calculation ofsystem reliability of a complex structure. International, Journal of Solids andStructures 2004;41(18-19):5035–50.

[254] Mahadevan S, Nagpal BSV, Venkataraman S, Pai SS. Probabilistic design andanalysis for system-level application. In: Proceedings of the 48th AIAA/ASME/ASCE/AHS/ASC structures, structural dynamics, and materials con-ference, 2007.

[255] McDonald M, Mahadevan S. Reliability based design optimization formula-tions for component and system reliability. In: Proceedings of the 49thAIAA/ASME/ASCE/AHS/ASC structures, structural dynamics, and materialsconference, 2008.

[256] Madsen H, Krenk S, Lind N. Methods of structural safety. EnglewoodCliffs.Prentice-Hall; 1986.

[257] Ditlevsen O, Madsen HO. Structural reliability methods. Chichester: JohnWiley and Sons; 1996.

[258] Du X, Chen W. Efficient uncertainty analysis methods for multidisciplinaryrobust design. AIAA Journal 2002;40(3):545–52.

[259] Du X, Wang Y, Chen W. Methods for robust multidisciplinary design. In:Proceedings of the 41st AIAA/ASME/ASCE/AHS/ASC structures, structuraldynamics, and materials conference and exhibit, 2000.

[260] Du X, Chen W. Concurrent subsystem uncertainty analysis in multidisci-plinary design. In: Proceedings of the eighth AIAA/USAF/NASA/ISSMOsymposium on multidisciplinary analysis and optimization, 2000.

[261] Gu X, Renaud JE. Implicit uncertainty propagation for robust collaborativeoptimization. In: Proceedings of the DETC’01 ASME 2001 design engineeringtechnical conferences and computers and information in engineeringconference, 2001.

[262] Gu X, Renaud J. Implementation study of implicit uncertainty propagation(IUP) in decomposition-based optimization. In: Proceedings of the ninthAIAA/USAF/NASA/ISSMO symposium on multidisciplinary analysis andoptimization, 2002.

[263] Du X, Chen W. Collaborative reliability analysis for multidisciplinarysystems design. In: Proceedings of the ninth AIAA/NASA/USAF/ISSMOsymposium on multidisciplinary analysis and optimization, 2002.

[264] Du X, Chen W. Collaborative reliability analysis under the framework ofmultidisciplinary systems design. Optimization and Engineering 2005;6:63–84.

[265] Mahadevan S, Smith NL. System risk assessment and allocation in con-ceptual design analysis. NASA/CR-2003-212162. Langley Research Center;2003.

[266] Padmanabhan D, Batill S. Decomposition strategies for reliability basedoptimization in multidisciplinary system design. In: Proceedings of theninth AIAA/USAF/NASA/ISSMO symposium on multidisciplinary analysisand optimization, 2002.

[267] Ahn J, Kwon JH. Sequential approach to reliability analysis of multidisci-plinary analysis systems. Journal of Structural and Multidisciplinary Opti-mization 2004;28(6):397–406.

[268] Casella G, George EI. Explaining the Gibbs sampler. American Statistician1992;46(3):167–74.

[269] Beale EML. On minimizing a convex function subject to linear inequalities.Journal of the Royal Statistical Society. Series B (Methodological)1955;17(2):173–84.

[270] Stougie L. Design and analysis of methods for stochastic integer program-ming. PhD dissertation, University of Amsterdam, 1985.

[271] Hene TS, Dua V, Pistikopoulos EN. A hybrid parametric/stochastic program-ming approach for mixed-integer nonlinear problems under uncertainty.Industrial and Engineering Chemistry Research 2002;41(1):67–77.

[272] Bastin F. Nonlinear stochastic programming. MS dissertation, University ofNamur, 2001.

[273] Bastin F. Trust-region algorithms for nonlinear stochastic programming andmixed logit models. PhD dissertation, University of Namur, 2004.

[274] Mulvey JM, Vanderbei RJ, Zenios SA. Robust optimization of large-scalesystems. Operations Research 1995;43(2):264–81.

[275] Chen X, Sim M, Sun P. A robust optimization perspective on stochasticprogramming. Operations Research 2007;55(6):1058–71.

[276] Zhang Y, Monder D, Forbes JF. Real-time optimization under parametricuncertainty: a probability constrained approach. Journal of Process Control2002;12(3):373–89.

[277] Kadam JV, Schlegel M, Srinivasan B, Bonvin D, Marquardt W. Dynamicoptimization in the presence of uncertainty from off-line nominal solutionto measurement-based implementation. Journal of Process Control2007;17(5):389–98.

[278] Ruszczynski A, Shapiro A. Handbooks in operations research and manage-ment science: stochastic programming. Amsterdam: Elsevier; 2006.

[279] Kall P, Wallace SW. Stochastic programming. Chichester: John Wiley andSons; 1994.

[280] Delgado M, Verdegay JL, Vila MA. A general model for fuzzy linearprogramming. Fuzzy Sets and Systems 1989;29(1):21–9.

[281] Guu S, Wu Y. Two-phase approach for solving the fuzzy linear programmingproblems. Fuzzy Sets and Systems 1999;107(2):191–5.

[282] Li X, Zhang B, Li H. Computing efficient solutions to fuzzy multiple objectivelinear programming problems. Fuzzy Sets and Systems 2006;157(10):1328–32.

[283] Shih CJ. Wangsawidjaja RAS. Mixed fuzzy-probabilistic programmingapproach for multiobjective engineering optimization with random vari-ables. Computers and Structures 1996;59(2):283–90.

[284] Liu B. Theory and practice of uncertain programming. Berlin: Springer-Verlag; 2009.

[285] Ben-Tal A, Nemirovski A. Robust convex optimization. Mathematics ofOperations Research 1998;23(4):769–805.

[286] Zhang Y. General robust-optimization formulation for nonlinear program-ming. Journal of Optimization Theory and Applications 2007;132(1):111–24.

[287] Boni O, Ben-Tal A, Nemirovski A. Robust solutions to conic quadraticproblems and their applications. Optimization Engineering 2008;9(1):1–18.

[288] Spall JC. Introduction to stochastic search and optimization. Hoboken, NewJersey: John Wiley and Sons; 2003.

[289] Ljung L, Pflug GC, Walk H. Stochastic approximation and optimization ofrandom systems. Basel: Birkhauser Verlag; 1992.

[290] Andradottir S. A review of simulation optimization techniques. In: Proceed-ings of the 1998 winter simulation conference, 1998.

[291] Rosen SL, Harmonosky CM, Traband MT. A simulation optimization methodthat considers uncertainty and multiple performance measures. EuropeanJournal of Operational Research 2007;181:315–30.

[292] Parkinson A, Sorensen C, Pouthassan N. A general approach for robustoptimal design. Transactions of the ASME 1993;115(1):74–80.

[293] Sundaresan S, Ishii K, Houser DR. A robust optimization procedure withvariations on design variables and constraints. Advances in Design Auto-mation 1993;69(1):379–86.

[294] Yu J, Ishii K. Design for robustness based on manufacturing variationpatterns. Transactions of the ASME 1998;120(2):196–202.

[295] Shan S, Wang GG. Reliable design space and complete single-loop relia-bility-based design optimization. Reliability Engineering and System Safety2008;93(8):1218–30.

[296] Youn BD, Choi KK. A new response surface methodology for reliability-based design optimization. Computers and Structures 2004;82(2–3):241–56.

[297] Cheng G, Xua L, Jiang L. A sequential approximate programming strategy forreliability-based structural optimization. Computers and Structures 2006;84(21):1353–67.

[298] Kuran B. Reliability based design optimization of a solid rocket motor usingsurrogate models. In: Proceedings of the 43rd AIAA/ASME/SAE/ASEE jointpropulsion conference and exhibit, 2007.

[299] Hyeon JB, Chai LB. Reliability-based design optimization using a momentmethod and a kriging metamodel. Engineering Optimization 2008;40(5):421–38.

[300] Wang L, Kodiyalam S. An efficient method for probabilistic and robustdesign with non-normal distributions. In: Proceedings of the 43rd AIAA/ASME/ASCE/AHS/ASC structures, structural dynamics, and materials con-ference, 2002.

[301] Bae H, Grandhi RV, Canfield RA. Reliability-based design optimization underimprecise uncertainty. In: Proceedings of the 46th AIAA/ASME/ASCE/AHS/ASC structures, structural dynamics and materials conference, 2005.

[302] Du X, Sudjianto A, Huang B. Reliability-based design with the mixture ofrandom and interval variables. Journal of Mechanical Design 2005;127(6):1068–76.

[303] Du L, Choi KK. An inverse analysis method for design optimization withboth statistical and fuzzy uncertainties. Structural and MultidisciplinaryOptimization 2008;37(2):107–19.

[304] Thanedar PB, Kodiyalam S. Structural optimization using probabilisticconstraints. Structural and Multidisciplinary Optimization 1992;4(3–4):236–40.

[305] Maglaras G, Ponslet E, Haftka RT, Nikolaidis E, Sensharma P, Cudney HH.Analytical and experimental comparison of probabilistic and deterministicoptimization. AIAA Journal 1996;34(7):1512–8.

[306] Phadke M. Quality engineering using robust design.Englewood Cliffs. NJ:Prentice-Hall; 1989.

[307] Steinberg DM. Robust design: experiments for improving quality. In: GhoshS, Rao CR, editors. Handbook of Statistics, vol. 13. Elsevier Science; 1996.p. 199–240.

[308] Bates RA, Kenett RS, Steinberg DM, Wynn HP. Robust design using computerexperiments. Progress in industrial mathematics at ECMI 2004. Springer;2004 pp. 564–8.

[309] Ramakrishnan B, Rao SS. A robust optimization approach using Taguchi’sloss function for solving nonlinear optimization problems. ASME Advancesin Design Automation 1991;DE-32(1):241–8.

[310] Otto JN, Antonsson EK. Extensions to the Taguchi method of product design.Journal of Mechanical Design 1993;115(1):5–13.

[311] Lee K, Park G. Robust optimization considering tolerances of designvariables. Computers and Structures 2001;79(1):77–86.

[312] Chen W, Sahai A, Messac A, Sundararaj GJ. Exploration of the effectivenessof physical programming in robust design. Journal of Mechanical Design2000;122(2):155–63.

[313] Messac A, Ismail-Yahaya A. Multiobjective robust design using physicalprogramming. Structural and Multidisciplinary Optimization 2002;23(5):357–71.


[314] Chen W, Wiecek MM, Zhang J. Quality utility: a compromise programmingapproach to robust design. Journal of Mechanical Design 1999;121(2):179–87.

[315] Govindaluri SM, Cho BR. Robust design modeling with correlated qualitycharacteristics using a multicriteria decision framework. Journal ofAdvanced Manufacturing Technology 2007;32(5–6):423–33.

[316] Das I. Robustness optimization for constrained nonlinear programmingproblems. Engineering Optimization 2000;32(5):585–618.

[317] Rai MM. Robust optimal design with differential evolution. In: Proceedingsof the 10th AIAA/ISSMO multidisciplinary analysis and optimization con-ference, 2004.

[318] Li M, Azarm S, Aute V. A multi-objective genetic algorithm for robust designoptimization. In: Proceedings of the 2005 conference on Genetic andevolutionary computation, 2005. p. 771–8.

[319] Rangavajhala S, Mullur AA, Messac A. Uncertainty visualization in multi-objective robust design optimization. In: Proceedings of the 47th AIAA/ASME/ASCE/AHS/ASC structures, structural dynamics, and materials con-ference, 2006.

[320] Koch PN. Probabilistic design: optimizing for six sigma quality. In: Proceed-ings of the 43rd AIAA/ASME/ASCE/AHS structures, structural dynamics, andmaterials conference, 2002.

[321] Pyzdek T. The six sigma handbook. 2nd ed. New York: McGraw-Hill; 2003.[322] Mattson C, Messac A. Handling equality constraints in robust design

optimization. In: Proceedings of the 44th AIAA/ASME/ASCE/AHS/ASC struc-tures, structural dynamics and materials conference, 2003.

[323] Rangavajhala S, Mullur A, Messac A. The challenge of equality constraints inrobust design optimization: examination and new approach. In: Proceed-ings of the 46th AIAA/ASME/ASCE/AHS/ASC structures, structural dynamics& materials conference, 2005.

[324] Su J, Renaud JE. Automatic differentiation in robust optimization. AIAAJournal 1997;35(6):1072–9.

[325] Wang H, Kim NH. Robust design using stochastic response surface andsensitivities. In: Proceedings of the first AIAA/ISSMO multidisciplinaryanalysis and optimization conference, 2006.

[326] Lee Y, Hong K, Choi D. An efficient robust optimal design method forengineering systems with numerical noise. In: Proceedings of the 10thAIAA/ISSMO multidisciplinary analysis and optimization conference, 2004.

[327] Hacker K, Lewis K. Robust design through the use of a hybrid geneticalgorithm. In: Proceedings of the DETC’02 ASME 2002 design engineeringtechnical conferences and computers and information in engineeringconference, 2002.

[328] Fuchs M, Neumaier A, Girimonte D. Uncertainty modeling in autonomousrobust spacecraft system design. Proceedings in Applied Mathematics andMechanics 2007;7(1):2060041–2.

[329] Zang C, Friswell MI, Mottershead JE. A review of robust optimal design andits application in dynamics. Computers and Structures 2005;83(4–5):315–26.

[330] Egorov I, Kretinin G, Leshchenko I. How to execute robust design optimiza-tion. In: Proceedings of the ninth AIAA/ISSMO symposium on multidisci-plinary analysis and optimization, 2002.

[331] Koch PN, Wujek B, Golovidov O. A multi-stage, parallel implementation ofprobabilistic design optimization in an MDO framework. In: Proceedingof the eighth AIAA/USAF/NASA/ISSMO symposium on multidisciplinaryanalysis and optimization, 2000.

[332] Koch PN, Simpson TW, Allen JK, Mistree F. Statistical approximations formultidisciplinary design optimization: the problem of size. Journal ofAircraft 1999;36(1):275–86.

[333] Sues RH, Rhodes GS. Portable parallel stochastic optimization for the designof aeropropulsion components. NASA CR-195312. Lewis Research Center;1994.

[334] Agarwal H, Renaud J, Lee J, Watson L. A unilevel method for reliability baseddesign optimization. In: Proceedings of the 45th AIAA/ASME/ASCE/AHSstructures, structural dynamics, and materials conference, 2004.

[335] Chen X, Hasselman TK, Neill DJ. Reliability based structural design optimi-zation for practical applications. In: Proceedings of the 38th AIAA/ASME/ASCE/AHS structures, structural dynamics, and materials conference, 1997.

[336] Liang J, Mourelatos ZP. A single-loop method for reliability-based designoptimisation. International Journal of Product Development 2008;5(1–2):76–92.

[337] Sues R, Cesare M. An innovative framework for reliability-based MDO. In:Proceedings of the 41st AIAA/ASME/ASCE/AHS/ASC structures, structuraldynamics, and materials conference and exhibit, 2000.

[338] Wu YT, Shin Y, Sues R, Cesare M. Safety-factor based approach for probability-based design optimization. In: 42nd AIAA/ASME/ASCE/AHS/ASC structures,structural dynamics and materials conference and exhibit, 2001.

[339] Du X., Chen W. Sequential optimization and reliability assessment methodfor efficient probabilistic design. In: Proceedings of the ASME 2002 designengineering technical conference and computers and information in engi-neering conference, 2002.

[340] Agarwal H, Renaud JE. New decoupled framework for reliability-baseddesign optimization. AIAA Journal 2006;44(7):1524–31.

[341] Zhang X, Huang H. Sequential optimization and reliability assessmentfor multidisciplinary design optimization under aleatory and epistemic

uncertainties. Journal of Structural and Multidisciplinary Optimization2010;40(1):165–75.

[342] Zhang X, Huang H, Xu H. Multidisciplinary design optimization withdiscrete and continuous variables of various uncertainties. Journal ofStructural and Multidisciplinary Optimization 2010, doi:10.1007/s00158-010-0513-y.

[343] Smith N, Mahadevan S. Integrating system-level and component-leveldesigns under uncertainty. Journal of Spacecraft and Rockets 2005;42(4):752–60.

[344] Royset JO, Der Kiureghian A, Polak E. Reliability-based optimal structuraldesign by the decoupling approach. Reliability Engineering and SystemSafety 2001;73(3):213–21.

[345] Zou T, Mahadevan S. A direct decoupling approach for efficient reliability-based design optimization. Journal of Structural and MultidisciplinaryOptimization 2006;31(3):190–200.

[346] McDonald M, Mahadevan S. All-at-once multidisciplinary optimization withsystem and component-level reliability constraints. In: Proceedings of the12th AIAA/ISSMO multidisciplinary analysis and optimization conference,2008.

[347] Chiralaksanakul A, Mahadevan S. Decoupled approach to multidisciplinarydesign optimization under uncertainty. Optimization and Engineering2007;8(1):21–42.

[348] Agarwal H, Renaud JE, Mack JD. A decomposition approach for reliability-based multidisciplinary design optimization. In: Proceedings of the 44thAIAA/ASME/ASCE/AHS structures, structural dynamics, and materials con-ference, 2003.

[349] Du X, Guo J, Beeram H. Sequential optimization and reliability assessmentfor multidisciplinary systems design. Journal of Structural and Multidisci-plinary Optimization 2008;35(2):117–30.

[350] Ahn J, Kwon J. An efficient strategy for reliability-based multidisciplinarydesign optimization using BLISS. Journal of Structural and MultidisciplinaryOptimization 2006;31(5):363–72.

[351] Yang RJ, Chuang C, Gu L, Li G. Numerical experiments of reliability-basedoptimization methods. In: 45th AIAA/ASME/ASCE/AHS/ASC structures,structural dynamics & materials conference, 2004.

[352] Yang RJ, Chuang C, Gu L, Li G. Experience with approximate reliability-basedoptimization methods II: an exhaust system problem. Journal of Structuraland Multidisciplinary Optimization 2005;29(6):488–97.

[353] Braun RD, Gage P, Kroo I, Sobieski I. Implementation and performanceissues in collaborative optimization. In: Proceedings of the sixth AIAA/NASA/USAF/ISSMO symposium on multidisciplinary analysis and optimiza-tion, 1996.

[354] McAllister CD, Simpson TW. Multidisciplinary robust design optimization ofan internal combustion engine. Journal of Mechanical Design 2003;125(1):124–30.

[355] Mistree F, Hughes OF, Bras BA. The compromise decision support problemand the adaptive linear programming algorithm. In: Kamat MP, editor.Structural optimization: status and promise. AIAA; 1993. p. 247–89.

[356] DeMiguel A, Murray W. An analysis of collaborative optimization methods.In: Proceedings of the eighth AIAA/USAF/NASA/ISSMO symposium onmultidisciplinary analysis and optimization, Long Beach, California, 2000.

[357] Sobieszczanski-Sobieski J. Optimization by decomposition: a step fromhierarchic to non-hierarchic systems. In: Second NASA/Air force symposiumon recent advances in multidisciplinary analysis and optimization, 1988.

[358] Sellar RS, Batill SM, Renaud JE. Response surface based, concurrent subspaceoptimization for multidisciplinary system design. In: Proceedings of the34th AIAA aerospace sciences meeting and exhibit, 1996.

[359] Padmanabhan D, Batill S. Reliability based optimization using approxima-tions with applications to multi-disciplinary system design. In: Proceedingsof the 40th aerospace sciences meeting and exhibit, 2002.

[360] Yao W, Chen X, Wei Y, Gao S. A game theory based composite subspaceuncertainty multidisciplinary design optimization procedure. In: Proceed-ings of the eighth world congress on structural and multidisciplinaryoptimization, 2009.

[361] Yao W, Guo J, Chen X, van Tooren M. Utilizing uncertainty multidisciplinarydesign optimization for conceptual design of space systems. In: Proceedingsof the eighth conference on systems engineering research, 2010.

[362] Kim HM, Michelena NF, Papalambros PY. Target cascading in optimalsystem design. Journal of Mechanical Design 2003;125(3):474–80.

[363] Kokkolaras M, Moulrlatos J, Papalambros PY. Design optimization ofhierarchically decomposed multilevel systems under uncertainty. In: Pro-ceedings of the ASME 2004 design engineering technical conference andcomputers and information in engineering conference, 2005.

[364] Wu YT, Millwater HR, Cruse TA. Advanced probabilistic structural analysismethod for implicit performance functions. AIAA Journal 1990;28(19):1663–9.

[365] Liu H, Chen W, Kokkolaras M, et al. Probabilistic analytical target cascading:a moment matching formulation for multilevel optimization under uncer-tainty. Journal of Mechanical Design 2006;128(4):503–8.

[366] Xiong F, Yin X, Chen W, Yang S. Enhanced probabilistic analytical targetcascading with application to multiscale design. In: Proceedings of theeighth world congress on structural and multidisciplinary optimization,2009.

dx.doi.org/10.1007/s00158-010-0513-y

dx.doi.org/10.1007/s00158-010-0513-y

Review of uncertainty-based multidisciplinary design optimization methods for aerospace vehicles

Documents

Transcript of Review of uncertainty-based multidisciplinary design optimization methods for aerospace vehicles