Engineers (Chemical Industries) by Peter Englezos [Cap 11]

8/22/2019 Engineers (Chemical Industries) by Peter Englezos [Cap 11]

1/8

11Statistical Inferences

Once w e have est imated the u n k n o w n parameters that appear in an algebraico r O D E m o d e l , it is q u i t e impor tan t to per fo rm a few ad d i t i on a l c a l c u l a t i on s toestabl ish es t imates of the s tandard er ror in the param ete r s and in the expected re-sponse variables. These addi t iona l comput a t i ona l steps are very valuable as theyprovide us wi th a qu ant i ta tive measure of the qual i ty of the overal l f i t and informus how t rus two r thy t he parameter est imates are .

1 1 . 1 I N F E R E N C E S O N T H E P A R A M E T E R SWhen the Gauss-Newton met hod is used to est imate the u n k n o w n parame-ters, w e l inear ize t he mode l equa t i ons and a t each iteration we solve the corre-sponding l inear least squares prob lem. As a resul t , the est imated parameter valueshave l inear least squares propert ies. N a m e l y , the parameter est imates are normal lydis tr ibuted, unbiased (i.e., E(k*)=k) and the i r covar iance matrix is given by

COV(k*) = O g [A*]~ ' ( 1 1 . 1 )where A* is matr ix A evaluated at k*. It should be noted that fo r l inear leastsqu ares ma t r i x A is i n de p e n de n t of the p a ram e t e r s w h i l e th is is c learly not the casefo r no n l inea r least squares problems. The required est imate of the variance O g isobtained from

2 S(k*) S(k*) ( 1 1 . 2 )(d.f .) N m- p177


2/8

1 78 Chapter 1 1

where (d.f .)= N m p are the deg rees o f f reedom, n a m e l y th e total n u m b e r o f m eas-u r e m e n t s m i n u s t h e n u m b e r o f u n k n o w n p a ram e t e r s .T h e a b o v e express ions for the C(3F(k*) and o jr are val id , if the stat ist ically

correct choice of the weight ing matr ix Q ;, ( i= l , . . . ,N ) i s u s e d in the formula t ion o fthe prob lem. N a m e l y , if the errors in the response var iab les ( E J , i = l , . . . , N ) are nor-m a l l y distr ibuted with zero me an and covar iance matr ix ,

COF(]) = a;: M , (1L3)1 - 1w e shou ld u se [ M j ] ~ as the we igh t ing matr ix Q. w h e r e the matr ices M j ; i = l , . . . , N

a re known w h e r e a s th e sca l ing factor , a ^ , c o u l d b e u n k n o w n . Based on the s t ruc-tu re o f M J w e arrive at the var ious cases of least squares es t imat ion (Simple LS,Weighted LS or General ized LS) as descr ibed in detai l in Chapter 2 .A l t h o u g h the c omp uta t ion o f COK(k*) is a s imple extra step after conver-gence of the Gauss-Newton method, we are not obl iged to use the Gaus s -Ne wtonmethod for the search of the best parameter values . Once k* has been obtainedus ing any search me thod , one can proceed and c o m p u t e the sensitivity coefficientsby se t t ing up mat r ix A and t h u s q u a n t i fy t he uncer t a in ty in the es t imated p a ram e -te r values by es t imat ing COK(k*) .A p p r o x i m a t e in fe rence r e g i on s fo r n on l in e a r m o d e l s are def ined b y an a logyto the l inear mode l s . In par t icu lar , the (1 -a ) 1 0 0 % joint confidence region for theparameter vector k is described by the el l ipsoid,

o r[k-kf [A*r1[k-k*] = F N m P (1Mb)N m- p

whe r e a is the selected probabi l i ty level in Fisher's F-distr ibut ion and ^Nm-p isobtained from th e F-dist r ibut ion tables with V j = p and v 2=(N/w-/?) degrees o f free-d o m . Th e c o r re s p on d i n g ( I -a) 1 0 0 % marginal confidence interval fo r each pa-rameter , k j , i = l , 2 , . . . , p , is given b yk * - t * / 2 o k i < k j < k * +t* / 2o k i ( 1 1 . 5 )

whe r e t ^ 7 2 is obta ined from the tables of S tudent ' s T-dis tr ibution with v = ( N w - p )degrees o f f reedom. The s tandard error of parameter k j , ak. , is obta ined as the

Copyright 2001 by Taylor & Francis Group, LLC
http://dk5985_ch02.pdf/http://dk5985_ch02.pdf/http://dk5985_ch02.pdf/


3/8

Statistical Inferences 1 79

squ are r oo t of the c o r re s p on d i n g d i agon a l e l e m e n t of the inverse o f matr ix A *mult ip l ied b y 6g, i.e.,

kj =E

It is rem inded t ha t fo r v>30 th e appro x im at io n t^ /2 ~ z can be usedw h e r e zaj2 is obta ined from the standard normal dis t r ibu t ion tables . Simply put,w h e n w e h a v e m a n y data points , we can s imply t ake as the 95% confidence inter-val twice the standard error (recall that zom$=\ .96 whereas t0 0 2 5 =2.042).

T he l inear appro x im at io n em pl o yed b y the G a u s s - N e w t o n me thod in solv-ing th e n o n l i n e a r least squ ares pr o b le m enables us to obta in in fe rence reg io ns fo rthe p a r ame te r s very eas i ly . H o w e v e r , these regions are only approximate andpractice has shown that in m a n y cases they can be very m is l ead ing (Wat t s andBates, 1988). Nonetheless , even if the reg ions are not exact, w e obta in a very goodidea of the corre la t ion a m o n g the parameters .

W e c a n c o m p u t e th e exact (1 -a ) 100% joint p a r a m e t e r l ikel ihood regionu s ing th e equ a t i o n g ive n be low

( 11 .7 )(k) = S(k*)h+__N1Nw-_p

H H

The co m pu ta t i o n of the above surface in the parameter space is not t r ivia l .For t he t wo-paramet e r case (p=2), th e j o i n t conf idence region on the krk2 p lanecan be de te rmined by us ing any contour ing m e t h o d . The c on tou r l ine is approxi -mated from m a n y funct ion eva l u a t i o ns o f S(k) over a dense grid of (kb k 2 ) v a lue s .

11 . 2 I N F E R E N C E S O N T H E E X P E C T E D R E S P O N S E V A R I A B L E SHaving determined the uncertainty in the param eter es t imates , we c an pro-ceed and obtain confidence intervals for the expected mean response. Let us firstcons ider m odels descr ibed by a se t of nonl inear a lgebra ic equat ions , y=f(x,k) . T he

1 0 0 (1 -a )% confidence interval of the expected m e a n response of the var iable y. atx0 is g iven by

f( xo , k* ) -C 2


4/8

180 Chapter 1 1

w h e r e t^ / 2 i s obta ined f rom th e t ab les o f S tu den t ' s T-d i s t r ibu t ion w i th v=(Nw-p)degrees o f f reedom. Based on the linear approximation of the model equa t ions , w ehave

y0(k) = f ( x 0 , k * ) o f1

5k [k - k *] (11 .9)

with th e partial derivative (5 fT/ 5k) r evaluated a t x0 and k*. Taking variances fromboth sides w e have\T

5k dk (ll.lOa)

Subst i tu t ion of the expression fo r COK(k*) yields ,\T5fT T

5k 5k ( l l . l O b )

The standard prediction error o f y j o , oy . Q , is the squ are root of the j th diago-nal element of COF(y 0), name ly ,

5k 5k

Equat ion 11.8 represents th e confidence interval for the m e a n expected re-sponse ra ther t han a . future observation (future measurement ) of the re spo nse vari -able, y0 . In this case, bes ides the uncertainty in the est imated parameters, w e m u s tinc lude th e uncer ta in ty due to the m e a s u r e m e n t error ( E O ) . T he ( l~a)100% confi-dence interval of y J Q is

( I I . 1 2 )

where th e standard predict ion error of y J Q is given byCopyright 2001 by Taylor & Francis Group, LLC


5/8


y. ,o = - '5 f, 1okV ) T[A' * > \okV / ( 1 1 . 1 3 )N ext let us turn ou r at tention to m od els described b y a set o f ordinary differ-ential equat ions . We are interested in es tabl ishing confidence intervals fo r each o fth e response var iables y j , j = l , . . . , w at any t i m e t=t0. T h e l inear approximat ion o fth e out pu t vector at t ime to ,

y ( t 0 , k ) = C x ( t 0 , k * ) + CG(t0)[k-k*]yie lds the expression for COK(y(t0)) ,

CW(y0) = CG(t0)COF(k*)C TGT( t0)w h i c h can be rewr i t ten as

CW(y0) = d 2

( 1 1 . 1 4 )

( 1 1 . 1 5 )

( 1 1 . 1 6 )wi th the sensitivity coefficients matrix G(to) evaluated at k*. The est imated stan-dard prediction error o f y j ( t o ) is obta ined as the square root of the j* diagonal ele-me nt of C O F(y(t0)) .

A * ] - 1 C T G T ( t 0 ) J i j ( 1 1 . 1 7 )

Based on the lat ter , we can c o m p u t e th e (l-a)100% confidence interval ofth e expected m e a n response o f y, at t= t0,

y j ( t 0 , k * ) - t ^ / 2 d < u < y ( t 0 , k * ) +C26 ( 1 1 . 1 8 )If o n the o the r hand we w i s h to c o m p u t e lhe(l-a)100% confidence interval

of the response o f y , at t= t0, w e m u s t i n c l u d e th e error term ( E O ) in the calcula t ionof the standard error, namely w e h a v e

y j ( t 0 , k * ) - t v a / 2 o y j o < y j ( t 0 ) < ( 1 1 . 1 9 )with

Copyright 2001 by Taylor & Francis Group LLC


6/8

1 82 Chapter 1 1

+ {CG(t0)[A*]-1CTGT(t0)J ( 1 1 . 2 0 )

1 1 . 3 M O D E L A D E Q U A C Y TESTSThere is a ple thora of model adequacy tests that the user can employ to de-cide whether the assumed mathemat ica l m o d e l is indeed adequate. General lyspeaking these tests are based on the comparison of the experimental error vari-ance est imated by the model to that ob ta ined exper imenta l ly o r t h r ough othermeans .

1 1 3. 1 Single Response ModelsLet us first consider mode ls that have only o ne measured var iable ( m= \ ) .W e shall consider tw o cases. O ne where w e k n o w precisely the value of the ex-perimental error variance and the other when we have an est imate of i t . N a m e l y ,there is quan t i f i able uncertainty in our est imate of the experimental error variance.

CAS E 1 : < r is k n o w n precisely:I n this case we assume that we know preci se ly the va lue of the standard

experimental error in the measuremen t s (o e). U s i n g Equat ion 1 1.2 w e obtain anest imate of the experimental error variance under the assumpt ion tha t the model isadequate. Therefore, to test whe the r the model is adequate w e s imply need to testth e h y p o t h e s i sH: =: cm o d e l

at any desirable level of signif icance, e.g., a=0.05. Here with o , , w e denoteth e error variance est imated by the m o d e l equat ions (Equat ion 11 .2 ) ; nam el y ,

d g is an est imate o fSince og is known exactly (i.e., there is no uncertainty in its value, it is a

given n u m b e r ) the above hypothes i s test i s done throug h a x -test. N a m e l y ,I f X d a t a > X v = ( N m - p ) , l - a => Reject H0

where


7/8


and X v = ( N m - p ) , l - a 's obta ined from th e tables of the j f -dis t r ibu t ion with degreesof freedom v=(N/w-p) .

CASE 2 : o is known approximately:L et u s as s ume tha t o i s no t known exactly, however , we have per formed

n repeated measurements of the response variable . From this smal l sample o fm u l t i p l e measurements we can de te rmine the sample mean and sample variance. Ifsg is the s amp le est imate o f oe , est imated from the n repeated measuremen t s i tis given by

7 \ ^ jS E = / ( y j - y ) ( 1 1 . 2 2 )n-1where the sample m e a n is obtained from

( 1 1 . 2 3 )

A ga in , we test the hypothes i s at any desi rab le level of signif icance, for ex-amp le ce=0.05

Ho: < * model = a

In t h i s case, s ince og i s known only approximate ly , the above hy p o the s i sis tested using an F-test, i.e.,

i f r -^ i-v l=( N m~P)>v2=n~l ^ n *nIf Fdata > Fj_'aw'

2= > Reject Ho

w h e r eCopyright 2001 by Taylor & Francis Group LLC


8/8

1 84 Chapter 1 1

(H-24)

and Fv

j= d f > v

'2 = n

~ ' is obta ined from th e tables of the F-distr ibut ion.

11 .3 . 2 Multivariate ModelsLet us now consider models tha t h a v e o n ly m o r e tha n o n e measured va r i ab le( m> \) . T he previous ly desc r ibed mode l a d e q u a c y tes ts have mul t iva r i a te exten-sions tha t can be found in several advanced s tat is t ics textbooks. For example, theb o o k Introduction to Applied Multivariate Statistics by Sr ivas tava and Carter

(19 83) presents several tests on co variance m atrices.I n m a n y engineer ing applications, however , we can easi ly reduce the prob-lem to the univariate tests presented in the previous section b y assu m ing tha t thecovar iance matr ix of the errors can be wri t ten asCWfe) = c j g M , ; i = l , . . . , N ( 1 1 . 2 5 )

whe r e M ; a r e known matrices. Ac t ua l ly qui te often we c an further a s s u m e that thematr ices I M , , i = l , . . . , N are the same and equ a l to matr ix M .An independent estimate of COV(z) , E , tha t is required for the adequacytests can be obta ined by perfo rming N K repeated exper im ents as

or for the case o f univar ia te tests, s , the sample est imate o f O E in Equat ion11 .25 , can be o b ta ined from

NR( N R -i " '"" " J u " ' "' ( 1 1 . 2 7 )

Copyright 2001 by Taylor & Francis Group, LLC

Engineers (Chemical Industries) by Peter Englezos [Cap 11]

Documents

Transcript of Engineers (Chemical Industries) by Peter Englezos [Cap 11]