Statistical Model Calibration and Validation

© CAPE Centre, The University of Queensland Hungarian Academy of Sciences

PROCESS MODELLING AND MODEL ANALYSIS

Statistical Model Calibration and Validation

C12C12

2



Overview

Grey box models and model calibration Data analysis and preprocessing Model parameter and structure estimation:

linear-nonlinear static-dynamic

Model validation

3



A Systematic Modelling Procedure

Problem definition

Controlling factors

Problem data

Model construction

Model solution

Model verification

Model calibration &

validation

1

2

4 7

5

3 6

4



Grey-box Models

Process models

developed from first engineering principles (white box part)

part of their parameters and/or structure unknown (black box part)

are called grey-box models

5



Model CalibrationConceptual Problem Statement

Given:

grey-box model calibration data (measured data) measure of fit (loss function) Estimate: the parameter values and/or structural elements

6



Model CalibrationConceptual Steps of Solution

Analysis of model specification Sampling of continuous time dynamic models Data analysis and preprocessing Model parameter and structure estimation Evaluation of the quality of the estimate

7



Sampling of Continuous Time Dynamic Models

Equi-distant zero-order hold samplinginpu t

t0 t1 t2 t3t4 tim e...

u(0)

u(1)

u(2)

u(4)

Discrete time input signal: u : {u(k)=u(tk) | k=1,2,...}

output signal: y : {y(k)=y(tk) | k=1,2,...}

8



Sampling of Continuous Time Dynamic Models

Model parameters (1st order approximation))

Discrete time model equations:

model parameters: = I+Ah , = Bh ( ,,C,D)

Continuous time model equations:

model parameters: (A ,B,C,D)

)()()(

)()()1(

kDukCxky

kukxkx

)()()( tButAxtx

9



Data Analysis and Preprocessing -Data Screening-

gross errors outliers trends

Data visualization Outlier tests Trends, steady state tests Gross error detection

Check measured data for:

Methods to be used include:

10



Data Screening - Visualization

Gross errors

0 10 20 30 40 50 60 70 80 900

2

4

6

8

10

12

14

16

18

Time (minutes)

Ton

nage

rat

e

Instrument off-line

11



Data Screening - Visualization

Trends and jumps

0 20 40 60 80 100 1200

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Time (minutes)

Con

cent

ratio

n

Outliers

Instrument recalibration

12



Experimental Design for Parameter Estimation

Static modelsnumber of measurements test point spacing test point sequencing

Dynamic models (in addition) sampling time selectionexcitation

Pseudo Random Binary Signal (PRBS)

13



Model Parameter and Structure Estimation

Conceptual problem statement Least Squares parameter estimation

- estimation procedure - properties of the estimate - linear and nonlinear models

Parameter estimation for static models Parameter estimation for dynamic models

14



Problem Statement of Model Parameter Estimation

Given: System model: Set of measured data: Loss function:

Compute: an estimate such that)( of ˆ Mpp

min || || )( )( MyypL

),( )()( MM pxy

kiiyixkD ,...,1|)(),(],1[

)()( MyypL

15



Problem Statement of Model Structure Estimation

Given: System model:

(not parametrized) Set of measured data: Loss function: Compute: an estimate such that + “candidate structures” in

},...,1|)(,],1[ kiiyx(i)kD {

|| || )( )(MyypL

min )( pL

)()( xy M

of ˆ

DOM

16



Least Squares (LS) Parameter Estimation

Given: System model:

linear in p, single y(M)

Measured data: Loss function:

Compute: an estimate

such that

n

iii

TM pxpxy1

)(

m

j

Mij

Mm

iWSOS iyiyWiyiypL

1

)()(

1

))()(())()(( )(

min )( pL

)( of Mpp

miixixiyid n ,...,1 , ))(),...,();(()( 1

17



Properties of LS Parameter Estimation

Estimation:

with Gaussian measurement errors: unbiased: covariance matrix:

)(

...

)2(

)1(

,

)(...)()(

......

)2(...)2()2(

)1(...)1()1(

)(ˆor ˆ)(

21

21

21

1

my

y

y

y

mxmxmx

xxx

xxx

X

yXWXXpWyXpWXX

n

n

n

TTTT

ppE }ˆ{

})ˆ{,(~ˆ ppp COVN

1)(}ˆ{ WXXp TCOV

18



Assessing the Fit

Residuals are independent and

residual tests correlation coefficient measures

r r r

r

x

x x x

+

+

+

+

+

+ +

+

+

++

++

+++

+

+ ++ +

+

+

+

+

+

+

+

+

++

+

+

+

+

++

+

r andom

non r andom

),0(~ Nr

19



Confidence Regions and Intervals

Individual confidence intervals:

iippi pssmtpii

}]ˆ{[ , )2

11,(ˆ COV

p 1

p 2p 2(M)

A

B

E

p 2L p 2U

p 1L

p 1(M)

p 1U

20



LS Parameter Estimation for Nonlinear Models

Solution Transformation into linear form Solution by (numerical) optimization Properties

has lost its nice properties - non-normally distributed - confidence region and confidence intervals are not symmetric - unbiased

p̂

21



Confidence Interval for Nonlinear Parameter Estimation

Sum-of-squares as a function of a parameter2

2

)(

1)( )()( iyiyL M

k

ipSOS

22



Static Models Linear in Parameters

General form

n

i

Mii

MTM pxpxy1

)()()(

Examples

T2

0

)()( ] ... 1[ ,

xpyi

Mi

iM

TAA

M

TAA

RT

E-

A

] CRT

[x ry

][T Cx Ckr

1 1 ,)ln(

' ,e

)(

0

23



Identification: Model Parameter and Structure Estimation of Dynamic Models

Properties of the estimation problemvariables (y and x) are time dependent

orderedx : present and past inputs and outputs

measurement errors on both y and xSteps 1. sampling continuous time models

2. estimation

)}(),(),...,1(),1();({)( kukyuyux

},...,1|)(),({],1[ kxykD

24



Parameter Estimation of Dynamic Models Linear in Parameters

General form of the input-output model

LS parameter estimation with

N

i

MSOS iyiyL

1

2)( ))()((

},0;,..,1;,..,1|)(),({)(

)()()(1

)()()(

krkmjuyd

pdpdy

kji

n

i

Mii

MTM

25



Parameter Estimation of Nonlinear Dynamic Models

Lum ped dynam ic m odel(ODE or DAE withinitial conditions)

)0()( ˆˆ PP k

)0(ˆ

parameters

ofset Initial

P

)ˆ;(

estimates Generates)()( kM Pty

Solve DAE fort = [0,1 ..., T]

Estim ate LS O S

Min LS O S ?

Optim izer chooses)1(ˆ kP

)1()( ˆˆ kk PP

)(ˆ estimates

parameter with EndkP

26



Statistical Model Validation via Parameter EstimationConceptual Problem Statement

Given: a calibrated model validation data (measured data) measure of fit (loss function)Question: Is the calibrated model “good enough” for the purpose?(Does it reproduce the data well?)

27



A Systematic Modelling Procedure

Problem definition

Controlling factors

Problem data

Model construction

Model solution

Model verification

Model calibration &

validation

1

2

4 7

5

3 6

Statistical Model Calibration and Validation

Documents

Transcript of Statistical Model Calibration and Validation