© United Technologies Corporation (2012) This document contains no technical data subject to the...

© United Technologies Corporation (2012)This document contains no technical data subject to the EAR or the ITAR.Reinman, Rev Date 6/19/2012 Slide 1 of 26

Design For Variation

UCM 2012

Sheffield, UK July 2-4, 2012Grant Reinman, Senior Fellow, Statistics and Design For Variation

Pratt & Whitney, East Hartford, CT

Design of Experiments – Gaussian Process Emulation

Monte Carlo Simulation – Bayesian Model Calibration

Un

certainty Q

uan

tification

Sen

siti

vity

An

alys

isIncrease Life

Improve Quality

Improve Producibility


Pratt & Whitney EngineeringA Passion for Innovation

PurePower® PW1000G Engine


Deterministic Design, Uncertain WorldTraditional Approach: Empirical Design Margins, Factors of Safety

▲ Manufacturing

Hole Diameter Value

Leading Edge Hole Diameter

▲ Usage

▲ Materials

X

Primary Creep

Secondary Tertiary

Time t

Str

ain

RuptureX

Primary Creep

Secondary Tertiary

Time t

Str

ain

Rupture

543210Hours per Flight

LCOLPR

MXANWARAMSIATCV

TWAUALUPS

AAL

UZBVIMA

AMTAMXCFGCSHDALETHFEAFIN

OPPW2000 Hours per Flight, by Operator

Stress

Tim

e to

Ruptu

re

Time To Creep/ Rupture

▲ Computational Models

0.00 0.02 0.04 0.06 0.08 0.10 0.12 0.14

-0.3

-0.2

-0.1

0.0

0.1

0.2

0.3

0.4

Concentrated Load, lbs

Dis

cre

pa

ncy

, in

.

0.00 0.02 0.04 0.06 0.08 0.10 0.12 0.14

-0.3

-0.2

-0.1

0.0

0.1

0.2

0.3

0.4

0.00 0.02 0.04 0.06 0.08 0.10 0.12 0.14

-0.3

-0.2

-0.1

0.0

0.1

0.2

0.3

0.4

(x)

(x) Discrepancy (bias) function

© United Technologies Corporation (2012)This document contains no technical data subject to the EAR or the ITAR.Reinman, Rev Date 6/19/2012

To Help Prevent Design Iterations due to a Model’s Meanline Miss, by using Bayesian model calibration

process Margin Miss, by replacing legacy margins with a

probabilistic model of uncertainty and variability

To Reduce Cost Focus on important features Relax requirements on unimportant features Use Robust Design to reduce sensitivity

To Maximize Stage Life (Time on Wing) Rotor life depends on max distress / min life airfoil Weakest-link structure pervasive in gas turbines Reducing variation increases rotor life

Probabilistic Design, Uncertain WorldWhy?

Total Effect of Leading Edge Parameters on Oxidation Life(Total effect is how much variation in life would be left if you knew precisely the values of all other

parameters.)

0

5

10

15

20

25

30

35

40

45

Parameter Name

To

tal E

ffect

(%)

Remove cost from low-impact features

Model Inputs

Age

Part-Part

Die/Config/Batch

Supplier

0.00 0.02 0.04 0.06 0.08 0.10 0.12 0.14

-0.3

-0.2

-0.1

0.0

0.1

0.2

0.3

0.4


Dis

cre

pa

ncy, in

.

0.00 0.02 0.04 0.06 0.08 0.10 0.12 0.14

-0.3

-0.2

-0.1

0.0

0.1

0.2

0.3

0.4

0.00 0.02 0.04 0.06 0.08 0.10 0.12 0.14

-0.3

-0.2

-0.1

0.0

0.1

0.2

0.3

0.4

Discrepancy (bias) function


Bia

s, in

.

CYCLES

DIS

TR

ES

S

Slide 4 of 26


To increase the speed of design parametric studies and optimization using engineering model emulators

iSight-FD, etcComputer Model

• Structural FEM• CFD model• Matlab code• Fortran code• Other models

Inputs Design Space

• Geometric dimensions• Loads• Temperatures• Material properties• Heat transfer

coefficients• Etc.

• Drive the DOE through the model

• Emulator• Sensitivit

y

Output in Design Space

• Stress• Deflection• Temperatur

e• Life• Performanc

e• Etc.

Maximin Latin Hypercube DOE

GEMSA, GPMSA, etc

Hours / Run

Seconds / Run

Probabilistic Design, Uncertain WorldWhy?

Slide 5 of 26


DFV Estimated Benefits

▲ Component-level Design For Variation has yielded an estimated 64%-88% return on internal investment. The savings resulted from:

Optimized inspection procedures and tolerances Reduced quality-related analysis and investigation time Reduced design iterations Improved reliability Improved on-time engine deliveries Improved root cause investigation process

▲ Based on Six Sigma history and internal trends, the return is expected to increase rapidly in subsequent years

▲ System-level Design For Variation is predicted to yield 40x return on investment due to

Achieving system-level performance and reliability goals earlier in the development cycle

Shorter development programs

Slide 6 of 26


Design For Variation (DFV) Strategic Plan

▲ Strategy☑ Identify Key Processes ☑ Define elements of a DFV-enabled modeling process☑ Provide Resources under Strategic Initiative

Fan & CompressorHFB ProducibilityParametric Airfoil

Compressor Aero DesignCompressor Tip Clearances

StructuresProbabilistic Rotor Lifing

Probabilistic Fracture MechanicsProbabilistic HCF

Parametric Geometry Simulation ModelEngine Dynamics and Loads

Combustor and AugmentorCombustor pattern factor

Combustor Liner TMFAugmentor Ignition Margin Audit

Mid Turbine Frame Robust Design

Mechanical Systems and ExternalsCarbon Seal Performance

Ball & Roller Bearing DesignFDGS Durability

Externals: Forced Response Analysis

TurbineTurbine Blade Durability

Turbine Vanes and BOAS DurabilityRotor Thermal Model

Airfoil LCF LifingHSE Combustor / Turbine DFV

Air SystemsThermal Management Model

Internal Air System ModelEngine Data Matching

Performance AnalysisPerformance Monte Carlo Risk Assessment

Engine Test Confidence, UncertaintyUncertainty in Engine System Predictions

Production Test Data Trending and AnalysisStatistical Data-match

System-Level Risk Communication and Decision Making

Validation TestingEngine Validation Planning

DFV Infrastructure (Statistics & Partners)

Emulation, Calibration SoftwareHigh Intensity Computing

Parametric ModelingOptimization

TrainingESW

CommunicationsInput Data

Tech Support

Vehicle SystemsProbabilistic Ambient

Temp Distribution

Vision: All Key Modeling Processes will be DFV-enabled


10 Elements of a DFV-Enabled Modeling Process Physics-Based Models

Model Preparation

1. A robust parametric physics-based model Model Input Variability and Uncertainty Quantification

2. Process for retrieving data needed to quantify variability and uncertainty in model inputs

3. Process for performing statistical analysis/developing statistical model of input dataa. Preserve correlations

Model Sensitivity Analysis

4. Process for generating a matrix of space-filling computer experiments (model runs) for emulator development

5. Process for running the computer code at the space-filling design points

6. Process for a. Building and validating the model emulator

b. Performing a variance-based sensitivity analysis

Model Calibration

7. Process for determining what experimental/field data are required for model calibration and measurement uncertainty (amount, characteristics to be measured, ..)

8. Process for performing Bayesian model calibration: calibrate and bias correct (if needed) and assess residual variation Uncertainty Analysis

9. Process for generating a Monte-Carlo sample and driving it through • Parametric model (if fast enough),• Model emulator, or • Bias corrected and calibrated model

Enable Practice

10. Update local ESW and local training. Put in place a process to ensure the model is capable over time.

Slide 8 of 26

ONE-TIM

E PROCESS



DEFINE Customer requirements (probabilistic)

ANALYZE Quantify model input variation / uncertainty, emulate

and calibrate model, perform sensitivity and uncertainty analyses

SOLVE Identify ‘optimum’ design that satisfies requirements

VERIFY/VALIDATE Variability/Uncertainty model

SUSTAIN Stable system of causes of performance variation

ANALYZE

SOLVE

VERIFYVALIDATE

DEFINE

SUSTAIN

Five Steps for Executing a DFV-Enabled Process

-3 -2 -1 0 1 2 3

0.0

0.1

0.2

0.3

0.4

y

Slide 9 of 26


▲ How do we define the allowable risk of not meeting a requirement?

-3 -2 -1 0 1 2 3

0.0

0.1

0.2

0.3

0.4

y

Requirement

Risk

DEFINE

Design For Variation (DFV): Five StepsDefine Customer Requirements

Slide 10 of 26

Explicit customer requirement Explicit customer requirement

Safety Impact: Follow Regulatory Requirements

Safety Impact: Follow Regulatory Requirements

System-Level Job Ticket Metric Impact: Follow flow-down or roll-up process

System-Level Job Ticket Metric Impact: Follow flow-down or roll-up process

Engine Certification Test ImpactEngine Certification Test Impact

None of the above• Previous acceptable experience or other business considerations• 6 Sigma Criteria • Solve for the probability or rate that minimizes expected total cost

None of the above• Previous acceptable experience or other business considerations• 6 Sigma Criteria • Solve for the probability or rate that minimizes expected total cost


Develop Model Emulator,

Sensitivity Analysis

Refine Distributions of Important Model Inputs

Run Real World Uncertainty

Analysis

Perform Bayesian

Model Calibration

Design Space Filling Experiment Over

Model Input Space

ANALYZE Quantify model input variation & uncertainty, emulate & calibrate model, perform sensitivity and uncertainty analyses


Inp

ut1

Inp

ut2

Inp

ut3

Inp

ut4

Inp

ut5

Inp

ut6

Inp

ut7

Inp

ut8

Inp

ut9

Inp

ut1

0

Inp

ut1

1

Inp

ut1

2

Inp

ut1

3

Inp

ut1

4

Inp

ut1

5

Inp

ut1

6

Inp

ut1

7

Inp

ut1

8

Inp

ut1

9

Inp

ut2

0

Ou

tpu

t1

Ou

tpu

t2

0.38 0.43 0.65 0.07 0.58 0.38 0.09 0.89 0.66 0.39 0.67 0.82 0.37 0.19 0.61 0.02 0.67 0.82 0.92 0.220.01 0.09 0.66 0.17 0.99 0.98 0.88 0.65 0.46 0.46 0.16 0.43 0.34 0.49 0.22 0.30 0.68 0.80 0.17 0.310.89 0.33 0.83 0.02 0.97 0.51 0.63 0.39 0.62 0.63 0.30 0.17 0.11 0.00 0.55 0.38 0.75 0.53 0.40 0.880.96 0.15 0.61 0.59 0.62 0.91 0.75 0.23 0.88 0.13 0.82 0.24 0.42 0.46 0.17 0.12 0.37 0.92 0.89 0.020.24 0.93 0.85 0.45 0.42 0.87 0.43 0.26 0.03 0.82 0.38 0.23 0.82 0.89 0.09 0.48 0.57 0.74 0.42 0.210.66 0.80 0.16 0.10 0.23 0.64 0.55 0.28 0.20 0.17 0.23 0.88 0.19 0.64 0.98 0.61 0.48 1.00 0.10 0.960.26 0.85 0.07 1.00 1.00 0.00 0.71 0.95 0.39 0.62 0.70 0.68 0.58 0.04 0.43 0.90 0.98 0.81 0.97 0.430.43 0.30 0.10 0.73 0.33 0.32 0.40 0.83 0.84 0.20 0.91 0.42 0.18 0.17 0.30 0.31 0.94 0.10 0.21 0.720.31 0.72 0.03 0.88 0.08 0.83 0.41 0.85 0.21 0.44 0.14 0.74 0.67 0.16 0.05 0.86 0.71 0.23 0.53 0.170.47 0.40 0.48 0.66 0.71 0.03 0.68 0.30 0.28 0.02 0.28 0.11 0.80 0.80 0.88 0.62 0.47 0.42 0.74 0.890.09 0.12 0.94 0.58 0.77 0.70 0.02 0.97 0.41 0.77 0.81 0.28 0.73 0.26 0.59 0.89 0.64 0.38 0.28 0.730.83 0.98 0.86 0.27 0.96 0.16 0.57 0.74 0.48 0.22 0.71 0.86 0.26 0.77 0.99 0.22 0.04 0.48 0.11 0.230.87 0.74 0.81 0.36 0.65 0.30 0.79 0.82 0.78 0.73 0.96 0.38 0.91 0.12 0.84 0.69 0.45 0.90 0.85 0.940.04 0.57 0.18 0.28 0.57 0.34 0.12 0.35 0.67 0.67 0.89 0.46 0.01 0.36 0.18 0.04 0.12 0.04 0.31 0.850.64 0.78 0.38 0.79 0.41 0.80 0.65 0.21 0.56 0.52 0.20 0.66 1.00 0.62 0.87 0.74 0.09 0.86 0.60 0.530.85 0.59 0.04 0.56 0.80 0.18 0.17 0.87 0.61 0.89 0.53 0.73 0.95 0.23 0.66 0.47 0.23 0.58 0.05 0.390.62 0.88 0.36 0.13 0.22 0.11 0.78 0.55 0.59 0.42 0.83 0.32 0.54 0.57 0.01 0.18 0.42 0.96 0.27 0.930.77 0.51 0.41 0.12 0.01 0.08 0.60 0.75 0.38 0.09 0.13 0.87 0.56 0.37 0.93 0.40 0.05 0.05 0.32 0.410.11 0.49 0.77 0.92 0.53 0.65 0.83 0.81 0.45 0.97 0.62 0.12 0.39 0.94 0.36 0.75 0.85 0.95 0.43 0.670.97 0.64 0.23 0.35 0.54 0.27 0.80 0.46 0.68 0.14 0.45 0.15 0.12 0.01 0.49 0.14 0.13 0.34 0.91 0.070.40 0.25 0.80 0.78 0.04 0.77 0.77 0.86 0.18 0.29 0.80 0.55 0.83 0.24 0.14 0.20 0.22 0.15 0.79 0.990.90 0.68 0.69 0.43 0.09 0.21 0.15 0.61 0.36 0.98 0.56 0.77 0.89 0.66 0.76 0.41 0.25 0.17 0.78 0.360.94 0.89 0.59 0.19 0.40 0.92 0.94 0.52 0.75 0.87 0.09 0.34 0.38 0.72 0.35 0.91 0.08 0.37 0.56 0.700.80 0.55 0.30 0.98 0.95 0.35 0.92 0.22 0.22 0.10 0.32 1.00 0.63 0.68 0.41 0.95 0.33 0.33 0.13 0.690.32 0.23 1.00 0.04 0.32 0.53 0.38 0.77 0.15 0.41 0.42 0.06 0.52 0.75 0.94 0.43 0.80 0.09 0.72 0.371.00 0.01 0.92 0.16 0.46 0.74 0.96 0.73 0.14 0.19 0.24 0.84 0.17 0.33 0.15 0.46 0.95 0.56 0.39 0.610.29 0.99 0.52 0.15 0.74 0.86 0.45 0.71 0.47 0.84 0.94 0.37 0.65 0.87 0.34 0.59 0.07 0.40 0.15 1.000.98 0.92 0.90 0.72 0.73 0.07 0.05 0.33 0.65 0.36 0.68 0.39 0.46 0.51 0.26 0.51 0.44 0.83 0.45 0.340.75 0.73 0.46 0.90 0.63 0.78 0.08 0.69 0.83 0.68 0.35 0.59 0.88 0.84 0.44 0.58 0.77 0.47 0.90 0.130.10 0.97 0.13 0.22 0.44 0.67 0.22 0.59 0.32 0.64 0.07 0.14 0.21 0.38 0.16 0.55 0.61 0.22 0.61 0.750.49 0.21 0.20 0.46 0.00 0.93 0.01 0.09 0.63 0.92 0.51 0.41 0.03 0.67 0.95 0.70 0.46 0.78 0.67 0.830.15 0.17 0.12 0.01 0.13 0.81 0.66 0.47 0.96 0.27 0.40 0.52 0.76 0.09 0.24 0.82 0.31 0.89 0.16 0.620.72 0.19 0.22 0.65 0.34 0.45 0.39 0.12 0.33 0.99 0.47 0.04 0.47 0.06 0.23 0.16 0.28 0.85 0.86 0.100.81 0.02 0.91 0.55 0.94 0.89 0.90 0.37 0.02 0.18 0.72 0.78 0.33 0.81 0.81 0.57 0.19 0.61 0.65 0.680.07 0.61 0.49 0.83 0.86 0.79 0.14 0.16 0.97 0.55 0.17 0.31 0.78 0.56 0.92 0.63 0.56 0.07 0.30 0.600.69 0.10 0.14 0.24 0.18 0.61 0.54 0.92 0.81 0.90 0.19 0.20 0.53 0.98 0.58 0.98 0.81 0.30 0.76 0.550.67 0.29 0.43 0.89 0.78 0.54 0.48 0.62 0.98 0.38 0.49 0.08 0.96 0.28 0.54 0.17 0.32 0.60 0.46 0.900.00 0.27 0.96 0.08 0.39 0.23 0.30 0.63 0.57 0.15 0.33 0.85 0.93 0.44 0.86 0.80 0.82 0.97 0.26 0.150.19 0.47 0.09 0.51 0.79 0.43 0.26 0.19 0.08 1.00 0.11 0.22 0.05 0.59 0.91 0.81 0.11 0.31 0.36 0.330.84 0.28 0.54 0.06 0.67 0.84 0.25 0.25 0.42 0.74 0.57 0.76 0.60 0.76 0.38 0.88 0.10 0.71 0.20 0.260.46 0.11 0.31 0.64 0.25 0.59 0.00 0.68 0.00 0.85 0.60 0.19 0.71 0.40 0.89 0.76 0.79 0.70 0.73 0.010.22 0.75 0.53 0.75 0.10 0.36 0.44 0.72 0.40 0.32 0.31 0.96 0.48 0.95 0.75 0.56 0.86 0.45 0.23 0.490.28 0.35 0.71 0.70 0.49 0.40 0.13 0.24 0.25 0.69 1.00 0.91 0.06 0.69 0.57 0.94 0.84 0.03 0.47 0.770.71 0.39 0.42 0.37 0.98 0.44 0.67 0.13 0.60 0.30 0.97 0.98 0.30 0.31 0.80 0.19 0.59 0.00 0.52 0.870.08 0.08 0.72 0.48 0.31 0.05 0.47 0.20 0.64 0.25 0.77 0.40 0.62 0.73 0.85 0.07 0.20 0.18 0.29 0.270.55 0.70 0.78 0.96 0.19 0.90 0.76 0.48 0.73 0.80 0.61 0.13 0.41 0.03 0.21 0.33 0.51 0.06 0.62 0.420.73 0.67 0.98 0.85 0.38 0.28 0.59 0.58 0.09 0.51 0.74 0.21 0.74 0.79 0.33 0.24 0.99 0.16 0.94 0.790.60 0.42 0.51 0.74 0.52 0.63 0.42 0.36 0.93 0.23 0.00 0.57 0.00 0.14 0.90 0.23 0.06 0.63 0.01 0.910.18 0.38 0.02 0.63 0.68 0.62 1.00 0.79 0.27 0.96 0.08 0.10 0.43 0.13 0.53 0.27 0.24 0.36 0.55 0.300.39 0.86 0.27 0.23 0.11 0.46 0.99 0.45 0.82 0.66 0.99 0.58 0.04 0.02 0.46 0.54 0.52 0.75 0.96 0.350.13 0.65 0.40 0.94 0.17 0.49 0.10 0.17 0.71 0.12 0.03 0.67 0.13 0.90 0.60 0.93 0.41 0.02 0.75 0.000.03 0.06 0.29 0.84 0.36 0.95 0.53 0.53 0.89 0.37 0.69 0.53 0.79 0.54 0.39 0.08 0.88 0.64 0.99 0.160.42 0.04 0.44 0.67 0.21 0.52 0.36 0.56 0.52 0.94 0.59 0.16 0.22 0.61 0.25 0.65 0.01 0.24 0.14 0.140.16 1.00 0.74 0.20 0.69 0.41 0.20 0.88 0.30 0.33 0.44 0.09 0.25 0.91 0.56 0.71 0.60 0.98 0.04 0.080.30 0.54 0.37 0.05 0.06 0.56 0.81 0.08 0.26 0.05 0.26 0.02 0.51 0.97 0.04 0.92 0.26 0.25 0.35 0.250.25 0.03 0.76 0.34 0.27 0.20 0.28 0.27 0.13 0.78 0.21 0.69 0.75 0.55 0.47 0.36 0.93 0.66 0.06 0.780.51 0.53 0.60 0.86 0.35 1.00 0.19 0.34 0.49 0.07 0.66 0.79 0.08 0.35 0.20 0.49 0.96 0.72 0.34 0.810.17 0.46 0.70 0.14 0.15 0.12 0.64 0.06 0.23 0.86 0.29 0.35 0.84 0.21 0.70 0.87 0.35 0.43 0.77 0.460.37 0.16 0.93 0.41 0.26 0.66 0.89 0.40 0.79 0.54 0.54 0.75 0.23 0.78 0.74 0.68 0.73 0.65 0.84 0.400.41 0.22 0.15 0.30 0.02 0.01 0.31 0.10 0.54 0.81 0.37 0.33 0.02 0.99 0.73 0.03 0.89 0.73 0.22 0.520.52 0.37 0.47 0.99 0.83 0.09 0.51 0.66 0.06 0.47 0.75 0.36 0.49 0.48 0.02 0.72 0.53 0.68 0.48 0.970.76 0.18 0.79 0.38 0.75 0.33 0.18 0.54 0.53 0.08 0.12 0.61 0.86 0.18 0.68 0.78 0.70 0.20 0.09 0.570.06 0.77 0.24 0.54 0.05 0.99 0.69 0.29 0.24 0.04 0.86 0.80 0.55 0.22 0.96 0.37 0.63 0.76 0.58 0.060.20 0.71 0.88 0.40 0.47 0.25 0.35 0.76 0.11 0.00 0.84 0.72 0.32 0.25 0.10 0.83 0.36 0.51 0.18 0.320.82 0.83 0.21 0.25 0.82 0.97 0.73 0.05 0.94 0.71 0.43 0.62 0.64 0.20 0.64 0.79 0.49 0.01 0.51 0.040.57 0.87 0.35 0.03 0.37 0.31 0.33 0.02 0.17 0.53 0.22 0.27 0.81 0.82 0.32 0.52 0.83 0.69 1.00 0.920.02 0.58 0.87 0.42 0.16 0.76 0.86 0.91 0.69 0.06 0.88 0.93 0.92 0.74 0.72 0.73 0.14 0.49 0.59 0.510.34 0.07 0.00 0.68 0.88 0.22 0.03 0.18 0.95 0.31 0.76 0.49 0.90 0.10 0.79 0.09 0.02 0.93 0.87 0.190.86 0.45 0.28 0.53 0.12 0.48 0.62 0.41 0.99 0.72 0.95 0.89 0.61 0.86 0.37 0.25 0.65 0.77 0.00 0.200.45 0.66 0.25 0.49 0.76 0.47 0.97 0.32 0.90 0.59 0.63 0.81 0.35 0.41 1.00 1.00 0.92 0.54 0.19 0.650.48 0.44 0.64 0.31 0.60 0.68 0.72 0.98 0.31 0.43 0.06 0.65 0.29 0.83 0.71 0.26 0.03 0.11 0.66 0.980.91 0.79 0.97 0.00 0.85 0.13 0.16 0.67 0.87 0.93 0.36 0.01 0.69 0.85 0.42 0.67 0.69 0.41 0.81 0.290.59 0.36 0.56 0.33 0.91 0.39 0.11 0.51 0.12 0.58 0.27 0.71 0.40 0.96 0.08 0.42 0.90 0.08 0.93 0.440.56 0.05 0.58 0.47 0.89 0.37 0.29 0.99 1.00 0.79 0.05 0.92 0.15 0.71 0.31 0.45 0.66 0.67 0.57 0.240.33 0.00 0.95 0.76 0.07 0.57 0.07 0.57 0.43 0.88 0.55 0.99 0.09 0.05 0.48 0.11 0.29 0.27 0.44 0.840.23 0.69 0.55 0.77 0.03 0.14 0.21 0.07 0.35 0.83 0.01 0.97 0.66 0.45 0.11 0.34 0.62 0.88 0.69 0.280.70 0.14 0.89 0.91 0.64 0.06 0.58 0.96 0.80 0.16 0.52 0.48 0.36 1.00 0.78 0.00 0.78 0.12 0.64 0.800.65 0.26 0.19 0.81 0.20 0.42 0.06 0.15 0.05 0.26 0.73 0.26 0.27 0.88 0.62 0.13 0.00 0.35 0.33 0.760.99 0.90 0.26 0.71 0.84 0.71 0.49 0.43 0.19 0.65 0.34 0.05 0.28 0.65 0.69 0.84 0.76 0.99 0.38 0.180.95 0.31 0.57 0.95 0.93 0.85 0.61 0.94 0.91 0.45 0.64 0.51 0.70 0.15 0.40 0.99 0.91 0.55 0.03 0.710.58 0.48 0.67 0.69 0.87 0.60 0.24 1.00 0.34 0.24 0.02 0.94 0.97 0.52 0.52 0.64 0.21 0.94 0.49 0.030.93 0.56 0.34 0.61 0.28 0.96 0.87 0.01 0.01 0.76 0.25 0.44 0.85 0.30 0.45 0.10 0.39 0.19 0.63 0.380.44 0.76 0.01 0.82 0.24 0.15 0.32 0.44 0.55 0.34 0.58 0.90 0.94 0.42 0.06 0.53 0.15 0.14 0.71 0.860.63 0.81 0.39 0.32 0.43 0.72 0.70 0.90 0.04 0.95 0.85 0.25 0.10 0.07 0.77 0.60 0.87 0.28 0.88 0.050.12 0.24 0.45 0.11 0.81 0.24 0.04 0.64 0.44 0.03 0.78 0.95 0.14 0.93 0.27 0.21 0.54 0.57 0.07 0.110.53 0.13 0.99 0.29 0.48 0.10 0.95 0.93 0.85 0.61 0.92 0.07 0.77 0.63 0.28 0.35 0.30 0.29 0.24 0.580.74 0.20 0.17 0.18 0.14 0.88 0.91 0.84 0.92 0.56 0.39 0.18 0.07 0.08 0.67 0.32 0.38 0.44 0.37 0.740.92 0.82 0.75 0.87 0.90 0.26 0.85 0.14 0.76 0.49 0.18 0.47 0.24 0.92 0.29 0.29 1.00 0.84 0.95 0.090.54 0.63 0.33 0.52 0.45 0.69 0.23 0.78 0.77 0.57 0.93 0.29 0.16 0.60 0.82 0.77 0.34 0.13 0.83 0.560.88 0.34 0.08 0.44 0.72 0.73 0.84 0.49 0.74 0.28 0.65 0.03 0.98 0.58 0.03 0.96 0.17 0.32 0.54 0.450.36 0.32 0.05 0.21 0.30 0.29 0.52 0.80 0.72 0.35 0.41 0.54 0.68 0.29 0.97 0.05 0.97 0.87 0.12 0.630.21 0.62 0.62 0.57 0.56 0.17 0.82 0.42 0.70 0.75 0.46 0.45 0.72 0.27 0.12 0.85 0.72 0.26 0.08 0.120.68 0.91 0.11 0.60 0.92 0.82 0.56 0.31 0.58 0.01 0.79 0.63 0.87 0.11 0.19 0.28 0.58 0.39 0.80 0.470.61 0.95 0.06 0.97 0.66 0.04 0.37 0.60 0.07 0.11 0.10 0.83 0.31 0.70 0.65 0.66 0.55 0.91 0.82 0.540.78 0.52 0.32 0.39 0.61 0.02 0.98 0.70 0.16 0.60 0.15 0.56 0.57 0.53 0.07 0.44 0.18 0.52 0.98 0.480.35 0.94 0.73 0.09 0.51 0.75 0.74 0.04 0.86 0.21 0.48 0.60 0.20 0.32 0.13 0.97 0.43 0.62 0.25 0.820.79 0.60 0.84 0.62 0.59 0.58 0.27 0.03 0.51 0.70 0.04 0.00 0.45 0.47 0.00 0.01 0.74 0.46 0.70 0.640.27 0.84 0.82 0.26 0.55 0.19 0.46 0.00 0.37 0.40 0.87 0.64 0.44 0.34 0.83 0.39 0.16 0.79 0.41 0.950.05 0.96 0.68 0.80 0.29 0.55 0.34 0.11 0.10 0.91 0.90 0.70 0.99 0.43 0.63 0.06 0.40 0.21 0.68 0.660.14 0.41 0.63 0.93 0.70 0.94 0.93 0.38 0.29 0.48 0.98 0.30 0.59 0.39 0.51 0.15 0.27 0.59 0.02 0.59

-1

0

1

2

3

4

5

6

7

8

9

0 1 2 3 4 5 6 7 8 9 10

x

y


parameters.)

0

5

10

15

20

25

30

35

40

45

Parameter Name

To

tal E

ffect

(%)

Model Inputs


parameters.)

0

5

10

15

20

25

30

35

40

45

Parameter Name

To

tal E

ffect

(%)

Model Inputs

Bayesian Model

Calibration

Real-World Validation Data

Bayesian Model

Calibration

Real-World Validation Data

EngineeringModel

• Parameter uncertainty update• Bias correction• Residual variation

0.00 0.20 0.40 0.60 0.80 1.00

0.00 0.20 0.40 0.60 0.80 1.00

0.00 0.20 0.40 0.60 0.80 1.00

Model Output

Run Experiment Through

Engineering Model

Accounting for uncertainty in • Model

input• Model

itself

Slide 11 of 26

http://www.science.psu.edu/alert/photos/miscphotos/Pleiades300.jpg


1. Latin Hypercube Experimental Designs

3. Variance-Based Sensitivity Analysis

2. Gaussian Process Emulators

4. Bayesian Model Calibration

ANALYZE : Key Technologies

-1

0

1

2

3

4

5

6

7

8

9

0 1 2 3 4 5 6 7 8 9 10

x

y

-1

0

1

2

3

4

5

6

7

8

9

0 1 2 3 4 5 6 7 8 9 10

x

y

-1

0

1

2

3

4

5

6

7

8

9

0 1 2 3 4 5 6 7 8 9 10

x

y

Simple function

f(x) = x + 3sin(x/2)

-1

0

1

2

3

4

5

6

7

8

9

0 1 2 3 4 5 6 7 8 9 10

x

y

-1

0

1

2

3

4

5

6

7

8

9

0 1 2 3 4 5 6 7 8 9 10

x

y

-1

0

1

2

3

4

5

6

7

8

9

0 1 2 3 4 5 6 7 8 9 10

x

y

Simple function

f(x) = x + 3sin(x/2)


parameters.)

0

5

10

15

20

25

30

35

40

45

Parameter Name

To

tal

Eff

ec

t (%

)

Model Inputs


parameters.)

0

5

10

15

20

25

30

35

40

45

Parameter Name

To

tal

Eff

ec

t (%

)

Model Inputs


0.0

0.2

0.4

0.6

0.8

1.0

0.0 0.2 0.4 0.6 0.8 1.0

Slide 12 of 26

0.00 0.02 0.04 0.06 0.08 0.10 0.12 0.140

24

68

10

12

14


De

fle

ctio

n, in

0.00 0.02 0.04 0.06 0.08 0.10 0.12 0.140

24

68

10

12

14

0.00 0.02 0.04 0.06 0.08 0.10 0.12 0.140

24

68

10

12

14

0.00 0.02 0.04 0.06 0.08 0.10 0.12 0.140

24

68

10

12

14

0.00 0.02 0.04 0.06 0.08 0.10 0.12 0.140

24

68

10

12

14

0.00 0.02 0.04 0.06 0.08 0.10 0.12 0.140

24

68

10

12

14

E (psi) x 10^7

2 3 4

0.00 0.02 0.04 0.06 0.08 0.10 0.12 0.14

02

46

810

12


Def

lect

ion,

in.

0.00 0.02 0.04 0.06 0.08 0.10 0.12 0.14

02

46

810

12

0.00 0.02 0.04 0.06 0.08 0.10 0.12 0.14

02

46

810

12

0.00 0.02 0.04 0.06 0.08 0.10 0.12 0.14

02

46

810

12

2.0 2.5 3.0 3.5 4.0

Modulus E (psix10^7)

0.00 0.02 0.04 0.06 0.08 0.10 0.12 0.14

-0.3

-0.2

-0.1

0.0

0.1

0.2

0.3

0.4


Dis

cre

pa

ncy, in

.

0.00 0.02 0.04 0.06 0.08 0.10 0.12 0.14

-0.3

-0.2

-0.1

0.0

0.1

0.2

0.3

0.4

0.00 0.02 0.04 0.06 0.08 0.10 0.12 0.14

-0.3

-0.2

-0.1

0.0

0.1

0.2

0.3

0.4


0.00 0.02 0.04 0.06 0.08 0.10 0.12 0.14

-0.3

-0.2

-0.1

0.0

0.1

0.2

0.3

0.4


Dis

cre

pa

ncy, in

.

0.00 0.02 0.04 0.06 0.08 0.10 0.12 0.14

-0.3

-0.2

-0.1

0.0

0.1

0.2

0.3

0.4

0.00 0.02 0.04 0.06 0.08 0.10 0.12 0.14

-0.3

-0.2

-0.1

0.0

0.1

0.2

0.3

0.4


w

y

F

Y

Xw


▲ Performance characteristic y = f (x1, x2, …, xp) depends on p inputs

▲ The variance of y can be approximated by

22

22

22

22211 pp xx

fxx

fxx

fy

SOLVE Identify optimum design that satisfies requirements


SOLVE

▲ We can reduce by

1. Reducing : the variance in the inputs x1, x2, …, xp

2. Reducing : the sensitivity of y to variation in x1, x2, ... , xp

2ix

ixf

2y

Slide 13 of 26


Design for Variation SOLVE: Robust Design Strategies

Noise Factors• Filter• Isolate• Reduce at source• Inoculate (anneal, heat treat)

Input Signal• Alter/smooth• Selectively block

Control Factors• Robust optimization• Material change• Create multiple operating modes

Output Response• Calibrate• Average

System

SOLVE

Slide 14 of 26

Adapted from: Jugulum, R. and Frey, D. (2007). Toward a taxonomy of concept designs for improved robustness, Journal of Engineering Design, 18:2, 139 - 156


▲ VERIFY/VALIDATE includes

– Data collection and analysis to validate model input probability distributions

Manufacturing process data

Material property data

Temperatures, pressures, rotor speeds, airflows

Flight characteristics (e.g. length, T2 at takeoff, taxi time, ..)

– Additional calibration of physics-based models

– Trending in-service parts (wear, performance, etc) where feasible to validate models and their inputs

VERIFY/VALIDATE Assumptions made in variability and uncertainty modeling


VAL/VER

Slide 15 of 26


▲ The SUSTAIN phase requires process control to ensure stable and consistent distributions over time

– Manufacturing

– Assembly

– Acceptance Testing

▲ Process Certification is vitally important– Sustaining capabilities to meet design requirements

– Identifying production & design improvement opportunities

▲ Design Sensitivity and Uncertainty Analyses indicate where process control resources should be focused

SUSTAIN Stable system of causes of performance variation


Date

Indiv

idual V

alu

e

1/31/20061/28/20061/25/20061/22/20061/19/20061/16/20061/13/20061/10/20061/7/20061/4/2006

35

30

25

20

15

_X=25.49

UCL=34.19

LCL=16.79

Date

Movin

g R

ange

1/31/20061/28/20061/25/20061/22/20061/19/20061/16/20061/13/20061/10/20061/7/20061/4/2006

10.0

7.5

5.0

2.5

0.0

__MR=3.27

UCL=10.69

LCL=0

I-MR Chart of Measured Value of a Key Characteristic

4036322824201612

LSL USLProcess Data

Sample N 30StDev(Within) 2.89922StDev(Overall) 3.30970

LSL 10.00000Target *USL 40.00000Sample Mean 25.48909

Potential (Within) Capability

CCpk 1.72

Overall Capability

Pp 1.51PPL 1.56PPU 1.46Ppk

Cp

1.46Cpm *

1.72CPL 1.78CPU 1.67Cpk 1.67

WithinOverall

Process Capability of Measured Value of a Key Characteristic

SUSTAIN

Slide 16 of 26



▲ Establish probabilistic design requirements

▲ Emulate, calibrate engineering models

▲ Solve for design that meets probabilistic requirements– Look for opportunities for making design less sensitive to variation

▲ Validate and sustain model

▲ Write Engineering Standard Work, develop local training

Systematic Process for Designing for and Managing Uncertainty and Variability

Slide 17 of 26



▲ Additional Training Courses Developed

▲ Automated Multi-physics Workflow

▲ System-Level Design

What’s New in 2012?

Slide 18 of 26


▲ Software– Emulation, Sensitivity Analysis, Model Calibration

– Statistical Analysis, Monte Carlo Simulation, Optimization

▲ High Performance Computing Resources

▲ Training– INTRODUCTION

– PRACTITIONERS I: SENSITIVITY ANALYSIS, EMULATION, AND DOE

– PRACTITIONERS II: ISIGHT-FD FOR SENSITIVITY AND UNCERTAINTY ANALYSIS

– PRACTITIONERS III: MODEL CALIBRATION AND UNCERTAINTY ANALYSIS

– MANAGERS: INTRODUCTION, REVIEW CHECKLIST

▲ Communication– Wiki, Website, Meetings

▲ Input Data Quality and Availability– Process Capability, Material Properties

– Systems Performance, Mission Analysis

▲ Engineering Standard Work

Infrastructure: Enabling Design For Variation

Design For Variation – What’s New?

Slide 19 of 26

http://www.science.psu.edu/alert/photos/miscphotos/Pleiades300.jpg


Design For Variation - What’s New?

Multi-discipline Automated Workflows• Link disciplines: Aero, Thermal, Structures, Materials,

Design• Link components• Enable probabilistic analyses, optimization

20

Slide 20 of 26


What’s New - PADME ProgramSystem Level Probabilistic Design & Validation of Engines

• PADME is a System-Level Extension of Design For Variation

• Quantify uncertainty/risk in system-level metrics

• Determine design drivers

• Determine optimum path to reduce risk

Design changes

Test changes

• PADME Goals

• Improve Mature vs. EIS Performance Gap by 33%

• Improve Mature vs. EIS Reliability Gap by 33%

• Reduce EVP Time by up to 50%

PADME: Probabilistic Analysis and Design of Materials and Engines

Page 12

Strategy☑ Identify Key Processes ☑Define elements of a DFV-enabled modeling process☑Provide Resources under Strategic Initiative

Leveraged Technologies: Design For Variation

Fan & CompressorHFB ProducibilityParametric Airfoil

Compressor Aero Design StructuresProbabilistic HCF

Parametric Geometry Simulation ModelEngine Dynamics and Loads

Combustor and AugmentorCombustor pattern factor

Combustor Liner TMFAugmentor Ignition Margin Audit

Mid Turbine Frame Robust Design

Mechanical Systems and ExternalsCarbon Seal Performance

Ball & Roller Bearing DesignFDGSDurability

Externals: Forced Response Analysis

TurbineTurbine Blade Durability

Turbine Vanes and BOAS DurabilityRotor Thermal Model

Airfoil LCF Lifing

Air SystemsThermal Management Model

Internal Air System ModelEngine Data Matching

Performance AnalysisPerformance Monte Carlo Risk Assessment

Engine Test Confidence, UncertaintyUncertainty in Engine System Predictions

Production Test Data Trending and Analysis

Validation TestingEngine Validation Planning

DFV Infrastructure (Statistics & Partners)

Sens / Uncert / Opt SoftwareHigh Perf Computing

TrainingESW

CommunicationsInput Data

Tech Support

Vehicle SystemsProbabilistic Ambient

Temp Distribution

EAR Export Classification: ECCN: EAR 99

21Slide 21 of 26

DesignValidation Service

Pro

bab

ility

Dis

trib

utio

n (

pdf

)

ConceptCustomer Use

PerformanceRequirement

Pro

bab

ility

Dis

trib

utio

n (

pdf

)

Design Prototype Production Maintenance &


Demo

Nominal Design

Performance

UncertaintyBounds on Design:Risk & Confidence


Pro

bab

ility

Dis

trib

utio

n (

pdf

)

ConceptCustomer Use


Pro

bab

ility

Dis

trib

utio

n (

pdf

)



Demo

Nominal Design

Performance


Go

od

Ba

d

Concept Design Test Service

Job

Tic

ket

Met

ric

Nominal/ExpectedConfidence bound

Requirement

Uncertainty bound


PADME VisionEntire Engine Life Cycle Governed By Uncertainty Quantification and Management

Rigorously Manage Uncertainty Throughout Life Cycle, Target Validation Testing to Address Largest Sources of Uncertainty

22Slide 22 of 26

Concept Design Test Service

Job

Tic

ket

Met

ric

Nominal/ExpectedConfidence bound

Requirement

Uncertainty boundDesignValidation Service

Pro

bab

ility

Dis

trib

utio

n (

pdf

)

ConceptCustomer Use


Pro

bab

ility

Dis

trib

utio

n (

pdf

)



Demo

Nominal Design

Performance



Pro

bab

ility

Dis

trib

utio

n (

pdf

)

ConceptCustomer Use


Pro

bab

ility

Dis

trib

utio

n (

pdf

)



Demo

Nominal Design

Performance


Go

od

Ba

d

Fuel ConsumptionDelay/Cancellation Rate

WeightCost


PADME: Manage Uncertainty Throughout Engine Life CycleQuantification of Uncertainty Enables Optimized Trades on System Level Metrics

23

= Needs DARPA Support = Supported by P&W or Prior Programs

EVP Optimizer Test

Sensitivity Analysis, Bayesian Model Calibration

Concept (CI/CO) Design (PD/DD) Validation (V&V) Service

Methods

Component & Sub-System Models

System Reliability Bayes Network

System Performance

Bayes Network

Re-optimize EVP

Bayesian Network Development and Updating

Large Scale Optimization Under Uncertainty

Robust Design, Real Options, Quantitative TRLs

PD/DD Emulators

Parametric Rel. Network

Parametric Perf. Network

EVP Optimizer Test

DFV-Enabled Design Models



EVP Optimizer Test







UBL / Prognosis

Bayesian Uncertainty Update




Pro

ba

bili

ty D

istr

ibu

tion

(p

df)

Concept Design Prototype Production Maintenance &Customer Use



Surprises;New Test Data

Nominal DesignPerformanceP

rob

ab

ility

Dis

trib

utio

n (

pd

f)





Nominal DesignPerformance P

rob

ab

ility

Dis

trib

utio

n (

pd

f)






rob

ab

ility

Dis

trib

utio

n (

pd

f)





Nominal DesignPerformance

Pro

ba

bili

ty D

istr

ibu

tion

(p

df)






rob

ab

ility

Dis

trib

utio

n (

pd

f)






rob

ab

ility

Dis

trib

utio

n (

pd

f)






rob

ab

ility

Dis

trib

utio

n (

pd

f)





Nominal DesignPerformance

Redesign Redesign Redesign Redesign

PADME Governed By System-Level NetworksPopulated By Calibrated Component-Level Emulators

Slide 23 of 26


▲ Uncertainty-Based Design Approach Relies on Calibration of Multivariate Aero-Thermal-Structural Models Using Highly Instrumented Engine

Deterministic Design

Engine Test Deterministic Redesign

Engine Test Deterministic Redesign

Engine Test

ProbabilisticDesign

R&D Rig/Engine Test Engine Endurance Test

crack

oxidation

RobustDesign

LegacyApproach

DFV-PADME Approach

EAR Export Classification: EAR 99

1st

Vane2nd Vane

1st

Blade2nd Blade

Exit Vane

Combustor

Gas Temps Gas Temps Gas TempsBlade Temps Blade Temps

PADME Strategy

Slide 24 of 26


Design For Variation – For More Information

▲ Statistical Engineering Issue

Slide 25 of 26


▲ Goal: quantify, understand, and control the risk of not meeting design criteria or exceeding thresholds

▲ “The revolutionary idea that defines the boundary between modern times and the past is the mastery of risk: the notion that the future is more than a whim of the gods and that men and women are not passive before nature.”– Peter Bernstein, “Against the Gods: The remarkable story of risk”

ModelPrediction

DesignCriteria

True Process Value


Slide 26 of 26

© United Technologies Corporation (2012) This document contains no technical data subject to the...

Documents

Transcript of © United Technologies Corporation (2012) This document contains no technical data subject to the...