New Directions in Model Based Data Assimilation Gregory J. McRae.

New Directions in Model New Directions in Model Based Data AssimilationBased Data AssimilationNew Directions in Model New Directions in Model Based Data AssimilationBased Data Assimilation

Gregory J. McRaeGregory J. McRaeGregory J. McRaeGregory J. McRae

Outline of PresentationOutline of Presentation

Introduction to data assimilation

Decision making in the presence of uncertainty

Challenges and new opportunities– Algorithms for uncertainty propagation– Data architectures and management– Standards for data exchange – Need for an interdisciplinary approach

Future directions and conclusions

There is a critical need for a new There is a critical need for a new approach to merging models / dataapproach to merging models / data

What is the Chemical Industry? What is the Chemical Industry?

Traditional ProcessesTraditional ProcessesTraditional ProcessesTraditional Processes

Semiconductor ManufacturingSemiconductor ManufacturingSemiconductor ManufacturingSemiconductor Manufacturing

“…“…Chemistry is the Chemistry is the importantimportant and and commoncommon ingredient…” ingredient…”

Models are Crucial in Chemical EngineeringModels are Crucial in Chemical Engineering

1.1. Technical Technical FeasibilityFeasibility

3.3. Performance Performance ImprovementsImprovements– Yield– Throughput– Quality

2.2. Operational Operational FeasibilityFeasibility

– Safety– Robustness– Consistency

– Synthesis routes– By-products– Separation issues

Models serve as the basic framework for knowledge management

Time

Modeling

Laboratory Studies

Pilot Plant(s)

Plant Startup

Operations

Time

Modeling

Laboratory Studies

Pilot Plant(s)

Plant Startup

Operations

Data Need: Model Verification / DiscriminationData Need: Model Verification / Discrimination

Variableof InterestVariableof Interest

Time, Space,...Time, Space,...

Meaningful comparisons requires estimates ofMeaningful comparisons requires estimates ofuncertainties in prediction uncertainties in prediction andand observations observationsMeaningful comparisons requires estimates ofMeaningful comparisons requires estimates ofuncertainties in prediction uncertainties in prediction andand observations observations

Predictions of Model- Uncertainties in Data & Structure

Measurements- Accuracy, Bias- Averaging Variance

AB

Other Dimensions of the Need for DataOther Dimensions of the Need for Data

Model discrimination – Model A vs. B

Hypothesis testing – Which parameter is “best”

Model verification – What are the “stopping” rules

Experimental design – Where to measure

Optimization objectives – Fail-Safe vs. Safe-Fail

Resource allocation – Where to spend the money

etc.

High Bandwidth Data Assimilation -- MixingHigh Bandwidth Data Assimilation -- Mixing

MotorMotor

DetectorsDetectors

Calibration RodCalibration Rod

Radioactive ParticleRadioactive Particle

MotorMotor

DetectorsDetectors

Calibration RodCalibration Rod

Radioactive ParticleRadioactive Particle

r (cm)

z (c

m)

Reynolds Shear StressReynolds Shear Stress

Counter rotating Counter rotating trailing Vorticestrailing Vortices

r (cm)

z (c

m)



r (cm)

z (c

m)



r (cm)

z (c

m)



X-Ray Tomography

CREL Washington University

Measurements are Vital – Historical ViewMeasurements are Vital – Historical View

“…When you can measure what you are speaking about, and express it in numbers, you know something about it; but when you cannot … your knowledge is of a meager and unsatisfactory kind: it may be the beginning of knowledge, but you have scarcely in your thoughts, advanced the state of science…”

William Thompson, Lord Kelvin

1. NEW ALGORITHMS THAT 1. NEW ALGORITHMS THAT CAN TACKLE NONLINEAR, CAN TACKLE NONLINEAR, COMPLEX AND LARGE-COMPLEX AND LARGE-SCALE PROBLEMSSCALE PROBLEMS

1. NEW ALGORITHMS THAT 1. NEW ALGORITHMS THAT CAN TACKLE NONLINEAR, CAN TACKLE NONLINEAR, COMPLEX AND LARGE-COMPLEX AND LARGE-SCALE PROBLEMSSCALE PROBLEMS

SUCCESS WILL DEPEND ON:

Opportunities for New Algorithms/TheoryOpportunities for New Algorithms/Theory

Statistics (Bayesian based approaches)– Model discrimination– Experimental design– Parameter and state estimation for ODE/PDE’s

Numerical Algorithms – Inverse problems for DAE/PDE’s – Data assimilation of n-dimensional data – Solution of large scale optimization problems– Uncertainty propagation– Multiscale integration

Data analysis and visualization– Immersive data mining and analysis

Overall Goal – From Bench to Plant ScalesOverall Goal – From Bench to Plant Scales

CHEMKIN model

Data from plant

Laboratory Experiment

Chemical Plant

Model baseddesign/analysisof experiments

A Simple Example – Linear Balance EquationsA Simple Example – Linear Balance Equations

Chemical PlantChemical Plant

Abstraction

(Mass & Energy Balances)

0 0 == Ax Ax ++ w w

Error in modeling

Measurementsy y == Cx Cx ++ e e

Error in measurement

Given A,C,w,e and y find best estimate of state x

Solution to State Estimation ProblemSolution to State Estimation Problem

11 Set up the constrained optimization problem (No model error)

0Ax

CxyWCxy 1

x

..

min

ts

)()( T

22 Solve for the estimates of the state

)constraintNo(OLS

yWCCWCx

xAACWCAACWCxx

0

00

111

11111

)(ˆ

ˆ])([)(ˆˆTT

TTTT

33 Determine the variance in the estimates1111111 )(])([)( CWCAACWCAACWCVV 0

TTTTT

The use of model-based constraints The use of model-based constraints reduces variance in estimates … BUTreduces variance in estimates … BUT

Assumptions underlying SolutionAssumptions underlying Solution

1. Model is linear

2. Normally distributed errors in data and solution

3. Data are uncorrelated in time

4. No errors in the model of the process!!

5. Etc.

There is a critical need for a more realistic approach that deals with model uncertainty

ModelModel

Sensitivity to parameter variationsSensitivity to parameter variations

SolutionSolution

dy tdt

k y t y y y t SiH t( )

( ) ; ( ) , ( ) [ ( )] 0 0 4

y t y e k t( ) 0

Sy tk

t y ek

k t

( )_ 0

But, what if k is uncertain?

Simple Problem: Kinetics of SiH4 (Si) + 2H2Simple Problem: Kinetics of SiH4 (Si) + 2H2

Solution in Presence of UncertaintySolution in Presence of Uncertainty

Kinetic ModelKinetic Model

k

P(k) P(y)

y(k)

Uncertain Rate Constant Uncertain Output

dy tdt

ky t( )

( )

P kk

e

k kk( )

1

21

12

01

2

P y

k tye

y y tk

tk( )

ln /

1

21

12

0 0

1

2

Normal distribution Normal distribution for rate constant (k)for rate constant (k)

Lognormal distributionLognormal distributionfor solution at each timefor solution at each time

But

Key Message – Outcomes are ImportantKey Message – Outcomes are Important

“... While there are always lots of uncertainties, the key challenge in engineering is to find those problem components that contribute most to uncertainties in outcomes...”

Example: Where to Allocate ResourcesExample: Where to Allocate Resources

Physical, chemical and decision process models

Uncertain Inputs- Literature- Experiments- Other models- etc.

Which parameterscontrol outcomes?

Process ModelProcess Model

y = f(k1,k2,…,km)

k1

km

P(km)

P(k1)

P(y)

y(k)

Uncertain Inputs Uncertain Output

How do Uncertain Inputs effect the Outputs?How do Uncertain Inputs effect the Outputs?

Process ModelProcess Model

y = f(k1,k2,…,km)

k1

km

P(km)

P(k1)

P(y)

y(k)

Uncertain Inputs Uncertain Output

E y k y k P y k dy k

y k P k dk dk

y

km

[ ( )] ( ) [ ( )] ( )

( ) [ ]

1

Measures of Uncertainty (Expected value, variance, pdf, etc.)

Multi-dimensional integrals are computationally very expensive

e.g.

Attributes of an Uncertainty Analysis SystemAttributes of an Uncertainty Analysis System

Compatible with existing modeling systems

At least four orders of magnitude faster than Monte Carlo

An ability to get the probability density function of outputs

Be able to identify the key contributions to uncertainties in outcomes

““...By rethinking conventional methods...By rethinking conventional methodsand directly embedding uncertainty intoand directly embedding uncertainty intothe modeling process itself...”the modeling process itself...”

How?

Incorporating Uncertainty at the BeginningIncorporating Uncertainty at the Beginning

f x a a x b xii

i i i( ) sin( ) cos( )

0

1

Fourier Series Representation of f(x)Fourier Series Representation of f(x)

What happens if x is a random variable?

Polynomial Chaos Representation of f(Polynomial Chaos Representation of f() (Wiener, 1947)) (Wiener, 1947)

f a Hii

i m( ) [ ( ), , ( )]

11

Functional (e.g. Hermite Polynomial)

Known probability distributions (e.g. unit Normal N[0,1])

Coefficients ofexpansion

Curse of DimensionalityCurse of Dimensionality

Measures of Uncertainty (expected value, variance, etc.)

DefinitionDefinition

fy()(y()) unknown

Statistically equivalentStatistically equivalentintegral definitionintegral definition

Multi-dimensional integrals arecomputationally very expensive

Projection into 1-D byProjection into 1-D byorthogonal polynomialsorthogonal polynomials

)())(()()}({ )( dyyfyyE y

n

n

ddfyyE 1)()()}({

))(( jj

iji Ha

nnnjj

jj dfydfycyEn

)()()()()}({ 111 1

Polynomial chaos expansion

New definition (one-dimensional integrals)New definition (one-dimensional integrals)

1

2

1 2

Example of Uncertainty AnalysisExample of Uncertainty Analysis

ModelModelModelModel

Expansion of y(t)Expansion of y(t)Expansion of y(t)Expansion of y(t)

ResidualResidualResidualResidual

Residual MinimizationResidual Minimization(Galerkin or Collocation)(Galerkin or Collocation)Residual MinimizationResidual Minimization(Galerkin or Collocation)(Galerkin or Collocation)

dy tdt

ky( )

y y H y t y t yi i [ ( )] ( ) ( ) ( ) 0 1 22 1

R tdy t

dtk y t( , )

( , )( ) ( , )

R t w d j nj( , ) ( ) ; , , ,

0 0 1

Uncertain coefficientsUncertain coefficients(Galerkin w(Galerkin wjj = = j j ))Uncertain coefficientsUncertain coefficients(Galerkin w(Galerkin wjj = = j j ))

Ady t

dtBy t

( )( ) 0

Simple Reaction SequenceSimple Reaction Sequence

100

80

60

40

20

0

Co

nc

en

tra

tio

n

1086420

Time

B

CA

B

Standard Deviation

A --> B k1 = 0.5 + 0.1

B --> C k2 = 2.0 + 0.5

CBA

Effect of Parameter Uncertainty on VarianceEffect of Parameter Uncertainty on Variance

16

14

12

10

8

6

4

2

0

Va

ria

nc

e o

f B

1086420

Time

(k2)

(k1)

A ---> B ---> CA ---> B ---> Ck1k1 k2

k2

AMAT Centura Chemical Vapor Deposition ReactorAMAT Centura Chemical Vapor Deposition Reactor

Operating ConditionsReactor Pressure 1 atmInlet Gas Temperature 698 KSurface Temperature 1173 KInlet Gas-Phase Velocity 46.6 cm/sec

SiCl3H HCl + SiCl2

SiCl2H2 SiCl2 + H2 SiCl2H2 HSiCl + HClH2ClSiSiCl3 SiCl4 + SiH2

H2ClSiSiCl3 SiCl3H + HSiClH2ClSiSiCl3 SiCl2H2 + SiCl2

Si2Cl5H SiCl4 + HSiClSi2Cl5H SiCl3H + SiCl2

Si2Cl6 SiCl4 + SiCl2

Gas Phase ReactionsGas Phase ReactionsSiCl3H + 4s Si(B) + sH + 3sClSiCl2H2 + 4s Si(B) + 2sH + 2sClSiCl4 + 4s Si(B) + 4sClHSiCl + 2s Si(B) + sH + sClSiCl2 + 2s Si(B) + 2sCl2sCl + Si(B) SiCl2 + 2sH2 + 2s 2sH2sH 2s + H2

HCl + 2s sH + sClsH + sCl 2s + HCl

Surface ReactionsSurface Reactions

TCS Lower Wall Deposition Rate (ANOVA)TCS Lower Wall Deposition Rate (ANOVA)

0.0

0.1

0.2

0.3

0.4

0.5

SICL3H+4s=>SI(B)+sH+3sCL

SICL2H2+4s=>SI(B)+2sH+2sCL

SICL4+4s=>SI(B)+4sCL

HSICL+2s=>SI(B)+sH+sCL

SICL2+2s=>SI(B)+2sCL

2sCL+SI(B)=>SICL2+2s

H2+2s=>2sH

2sH=>2s+H2

HCL+2s=>sH+sCL

sH+sCL=>2s+HCL

Adsorption/Desorption Processes are critical

Research Opportunities in UncertaintyResearch Opportunities in Uncertainty

Uncertainty analysis is a fertile and much needed area for inter-disciplinary research

There are many research opportunities

Database design for representation of uncertainties New algorithms for uncertainty propagation Decision making metrics in the presence of

uncertainties Treatment of structural uncertainties Valuation of cost of “safety factors”

Estimates of uncertainties in model inputs are desperately needed

Uncertainty Uncertainty Ignorance Ignorance

2. INTEGRATED APPROACHES 2. INTEGRATED APPROACHES FOR DATA MANAGEMENT, FOR DATA MANAGEMENT, MODELING, SOLUTION, MODELING, SOLUTION, ANALYSIS AND VISUALIZATIONANALYSIS AND VISUALIZATION

2. INTEGRATED APPROACHES 2. INTEGRATED APPROACHES FOR DATA MANAGEMENT, FOR DATA MANAGEMENT, MODELING, SOLUTION, MODELING, SOLUTION, ANALYSIS AND VISUALIZATIONANALYSIS AND VISUALIZATION

SUCCESS WILL DEPEND ON:

Opportunities in Computational SystemsOpportunities in Computational Systems

Data Modeling and Management– Data exchange standards– Enterprise integration– Risk management– Intellectual capital (e.g. corporate knowledge)

Computing Environments – Interoperability of commercial systems– Collaborative environments– Uncertainty propagation– Sensor technology

Visualization and Interpretation– Immersive data analysis

Role of Computing in Data AssimilationRole of Computing in Data Assimilation

IntegrationIntegration

Res

olu

tio

nR

eso

luti

on

1980’s

1990’s

2000+

Mobil

Visions for Software ArchitectureVisions for Software Architecture

An integrated web-based environment for chemical process modeling, control and optimization

Able to link molecular, reactor and plant scale models for whole plant simulation

Integration of corporate information repositories with experimental design

A Computational System that isA Computational System that is:A Computational System that isA Computational System that is:

Chemical Engineer’s WorkbenchChemical Engineer’s Workbench

Microscopic Computational Chemistry

Macroscopic Transport and Reaction Processes

UncertaintyAnalysis

Algorithmic Scale MatchingFile Transfer

Quantum ChemistryComputations ofThermochemistry

Experimental Data onMechanismsand Rates

Chemical MechanismFormulation

Transition State TheoryEstimates of Rates

Macroscopic ChemicalProcess Models

Momentum,Energy and Mass

Conservation Models

ExperimentalData & Validation

Reactor Design

SpecificationsSpecifications

Predicted

Performance

MolecularModeling

CFDSimulation

ProcessSimulation

UncertaintyAnalysis

ReactionModeling

Need for Multi-scale Models and their Integration Need for Multi-scale Models and their Integration

Molecular-level requirementsMolecular-level requirements

Reactor-level process controlsReactor-level process controlsWafer stack

Heaters

Support pedestal

Gas injector

Quartz

Insulation

Thermocouples

Low-Pressure Chemical Vapor Deposition Furnace

< 1 m

Filling a submicron trench by CVD

Macroscopic uniformity (at the wafer scale)

Microscopic features (at the device scale)

Atomic-scale features

Morphological requirements, crystal structure and doping

(at the surface scale)

Multi-Scale Integration of Software SystemsMulti-Scale Integration of Software Systems

XML As a standard for data exchange

[Ga*] [GaN*]

60

0

20

+ NH3

- CH4

32

0

55

15+ NH3- CH4

Close-Spaced RD reactor

Horizontal

Experimental Data Experimental Data & Quantum & Quantum ChemistryChemistry

(Gaussian, DFT)

CFD Model ofCFD Model ofReactor FlowReactor Flow

(STARCD, CFDRC,..)

Design Design OptimizationOptimization

(MINLP, Minos)

Distributed Computing ResourcesJensen - MIT

Data Structures and XML RepresentationsData Structures and XML Representations

<Thermodynamic_Data>

<Species>

<Chemical_Description>

<Formula> (CH2O)3 </Formula> <Atoms> C H O </Atoms> <Phase> G </Phase>

</Chemical_Description>

<Date_Entered_Database>

<Day> 5 </Day> <Month> 7 </Month> <Year> 1990 </Year>

</Date_Entered_Database>

<Polynomial_Coefficients>

<Low_T_Range>

<a1> 0.01913678E+03 </a1>

…

<a7> -0.08432507E+03 </a7>

</Low_T_Range>

</Polynomial_Coefficients>

</Species>

--- Additional species would be added in a similar manner---</Thermodynamic_Data>

Mechanism Library

NameCreation dateDocumentation

Mechanism

NameSpeciesReactionsStoichiometry

Mechanism


Mechanism


Species

IUPAC nameCAST numberFormulaMolecular weightThermodynamicsPropertiesStructureCharge

Reaction

TypeRate constantThird body effects

Mechanism


Rate constant

Pressure effectsArrhenius termsForward rateReverse

Mechanism


Thermodynamics

Species nameEntropyEnthalpySpecific heat

Structure

SpeciesSubstructuresSpectra

eXtended Markup Language (XML)

Data Structures

Reaction Mechanism ManagerReaction Mechanism Manager

Data Base StructureData Base Structure

Sample ReportsSample Reports

Benefits of Model Based Process ControllersBenefits of Model Based Process Controllers

* Controller design and implementation by Relman, Inc.* Cray test and evaluation by SEMATECH

Cost per deposition reduced by 25% Process cycle time cut by 20%. Stable operations 3 times faster 3 uniformity < 4% goal

Process modelProcess model

ControllerController

Embedded on-board computer

Estimated wafer temps. and oxidation rates

0

2

4

6

8

10

3-S

igm

a S

tan

d. D

ev. (

°C)

10 15 20 25 30 35 40 45 50

Model-Based Control

Standard Control

Stabilization time (min.)

Measured Process Improvement(SVG-Thermco AVP)

0

2

4

6

8

10

3-S

igm

a S

tan

d. D

ev. (

°C)

10 15 20 25 30 35 40 45 50

Model-Based Control

Standard Control

Stabilization time (min.)

Measured Process Improvement(SVG-Thermco AVP)

Chemkin Models Chemkin Models of Equipmentof Equipment - TWAFER- TWAFER - OVEND- OVEND

Economic BenefitsEconomic Benefits

Interacting with “Black Box” ModelsInteracting with “Black Box” Models

• Process Modeling Tools• Chemical Modeling Tools• Mathematical Modeling Tools

Problem Problem Specific Specific CompilersCompilers

ProblemProblemSpecific Specific OutputOutputParsersParsers

Uncertainty Description• Input distributions• Analysis variables• Computer environment• Input/output files

Uncertainty Analysis• Output density functions• Analysis of variance• Local sensitivity analysis• Key parameters

Integrated DatabaseManagement System

UNCERTAINTY ANALYSIS ENVIRONMENT UNCERTAINTY ANALYSIS ENVIRONMENT ( Standardized Interfaces )( Standardized Interfaces )

OLE, Corba OLE, CorbaBLACK BOXBLACK BOXSYSTEMSSYSTEMS

Example -- Aspen Process SimulationExample -- Aspen Process Simulation

0

20

40

60

80

100

Fractionalconversion

of reaction 1

Feed rateof toluene

Pressureof flash

Variance AnalysisVariance Analysis

Identification of key parameters for further work

Process Flow SheetProcess Flow SheetPurge

Toluene

Furnace

Cooler Flash 1

Flash 2

Toluene Recycle

Benzene Benzene

Reactor

Flue Gas

Diphenyl

H 2 Feed

H 2 Recycle

Uncertain ParametersUncertain Parameters-- Feed rateFeed rate-- Conversion in reactorConversion in reactor-- Flash operating pressureFlash operating pressure

Uncertainty Analysis

Hierarchical Information – Physical PropertiesHierarchical Information – Physical Properties

p -cresol

Alternative paths for producingthe data needed for simulation

ElectronDensity

CorrelationCorrelation

Model

GeometryOptimization

GroupIdentification

PhysicalProperty

Model

MolecularStructure

ChemicalPlantModel

Direct Measurement

Data Miningfor LiteratureInformation

200

250

300

350

400

450

500

550

600

200 250 300 350 400 450 500 550 600

Measured Boiling Point

Cal

cula

ted

Boi

ling

Poi

nt

Normal Boiling Point (K)Normal Boiling Point (K)

H2 O p -cresol

o -cresol

Environments to help Build ModelsEnvironments to help Build Models

Model of plant

Data from plant

Virtual Plant WalkthroughVirtual Plant Walkthrough

SGI, PSE, Adapco

Opportunities from Computational SystemsOpportunities from Computational Systems

Reducing the elapsed time for a “solution”Hierarchical / linked data bases– Community standards for data exchange– Corporate knowledge repositories

Model based approaches to: – Experimental design and optimization– Process controllers

Immersive data analysis and visualization for: – Data mining and analysis– Immersive graphical interfaces

ConclusionsConclusions

“...While it is hard to predict the future, creating it is much easier…”“...While it is hard to predict the future, creating it is much easier…”

We have an exciting opportunity to shape an integrated approach to merging models and data

John von NeumannJohn von Neumann

Contributions

- Algorithms - Software- Hardware architecture- Practical problems

B.S. Chemical EngineeringETH Zurich

New Directions in Model Based Data Assimilation Gregory J. McRae.

Documents

Transcript of New Directions in Model Based Data Assimilation Gregory J. McRae.