Chapter 14Chapter 14General Linear Squares andGeneral Linear Squares and

Nonlinear Nonlinear RegressionRegressiony = 20.5717 +3.6005xError Sr = 4201.3Correlation r = 0.4434x = [-2.5 3.0 1.7-4.90.6-0.5 4.0-2.2-4.3-0.2];y = [-20.1 -21.8 -6.0 -65.40.2 0.6 -41.3 -15.4 -56.1 0.5];Preferable to Preferable to fit a parabolafit a parabolaLarge error, Large error, poor correlationpoor correlationPolynomial RegressionPolynomial RegressionQuadratic Least Suaresyf!!"a"# a1! # a$!$ #ini$i%e total suare error n1 i$ $i $ i 1 " i $ 1 " r! a ! a a y a a a S " ! " & & !' n1 i$i $ i 1 " i$i$rn1 i$i $ i 1 " i i1rn1 i$i $ i 1 " i"r! a ! a a y ! $ "aS! a ! a a y ! $ "aS ! a ! a a y $ "aS %uadratic Least Squares%uadratic Least Squares'se C(oles)y deco$*osition to sol+e ,or t(e sy$$etric $atrixor use #-.L-/ ,unction % = -0r)')'1111111]1

n1 ii$in1 ii in1 ii$1"n1 i4in1 i&in1 i$in1 i&in1 i$in1 iin1 i$in1 iiy !y !yaaa! ! !! ! !! ! n Standard error ,or 2nd *olyno$ial re1ression/3ry xSsnwhere n observations2nd order polynomial (3 coefficients)(start off with n degrees of freedom, use up m+1 for mth-order polynomial) [x,y]=example2 z=Quadra!"#$%&x,y'xy&a0(a1)x(a2)x*2' &y-a0-a1)x-a2)x*2' -2.5000-20.1000-18.5529 -1.54713.0000-21.8000-22.08140.28141.7000 -6.0000 -6.37910.3791 -4.9000-65.4000-68.64393.24390.60000.2000 -0.28160.4816 -0.50000.6000 -0.77401.37404.0000-41.3000-40.4233 -0.8767 -2.2000-15.4000-14.4973 -0.9027 -4.3000-56.1000-53.1802 -2.9198 -0.20000.50000.01380.4862err = 25.6043%yx =1.9125r =0.9975z = 0.2668 0.7200 -2.7231y = 0.2662 + 0.7200 x 3 2.7231 x2 Correlation coe,,icient rStandard error o, t(e esti$ate+u,"!-, [x,y] = example2x = [ -2.5 3.01.7-4.9 0.6 -0.5 4.0-2.2-4.3 -0.2];y = [-20.1 -21.8 -6.0 -65.4 0.20.6 -41.3 -15.4 -56.10.5];Quadratic Least Suare4 y"'$(() # "'*$"" ! $'*$&1 !$Error Sr = 25.6043Correlation r = 0.5575Cubic Least SquaresCubic Least Squares)')'111111111]1

n1 ii&in1 ii$in1 ii in1 ii&$1"n1 i(in1 i+in1 i4in1 i&in1 i+in1 i4in1 i&in1 i$in1 i4in1 i&in1 i$in1 iin1 i&in1 i$in1 iiy !y !y !yaaaa! ! ! !! ! ! !! ! ! !! ! ! n + + + n1 i$ &i &$i $ i 1 " r&&$$ 1 "! a ! a ! a a y S! a ! a ! a a ! f" !" ! [x,y]=example2; z=.u/!"#$%&x,y'x yp&x'=a0(a1)x(a2)x*2(a3)x*3y-p&x' -2.5000-20.1000-19.9347 -0.16533.0000-21.8000-21.4751 -0.32491.7000 -6.0000 -5.0508 -0.9492 -4.9000-65.4000-67.43002.03000.60000.20000.5842 -0.3842 -0.50000.6000 -0.84041.44044.0000-41.3000-41.78280.4828 -2.2000-15.4000-15.79970.3997 -4.3000-56.1000-53.2914 -2.8086 -0.20000.50000.22060.2794err = 15.7361%yx =1.6195r =0.9985z =0.65131.5946 -2.8078 -0.0608y = 0.6513 + 1.5546x 6 2.2072x2 0.0602x3 Correlation coe,,icient r = 0.5525 [x,y]=example2; z1=$!,ear#$%&x,y'; z1z1 = -20.5717 3.6005 z2=Quadra!"#$%&x,y'; z2z2 =0.26680.7200 -2.7231 z3=.u/!"#$%&x,y'; z3z3 =0.65131.5946 -2.8078 -0.0608 x1=m!,&x'; x2=max&x'; xx=x10&x2-x1'11000x2; yy1=z1&1'(z1&2')xx; yy2=z2&1'(z2&2')xx(z2&3')xx.*2; yy3=z3&1'(z3&2')xx(z3&3')xx.*2(z3&4')xx.*3; 2=pl-&x,y,3r)3,xx,yy1,343,xx,yy2,3/3,xx,yy3,3m3'; xla/el&3x3'; yla/el&3y3'; 5e&2,3$!,e6!d73,3,38ar9er%!ze3,12'; pr!, -d:pe4075 re4re54.:p4Linear Least SuareQuadratic Least SuareCu7ic Least SuareLinear Least Suare4y, $"'+*1* # &'(""+!Quadratic4y"'$(() # "'*$"" ! $'*$&1!$Cu7ic4y"'(+1& # 1'+-4(! , $')"*)!$ "'"(")!&Standard error ,or *olyno$ial re1ression1 m nSsr! y+ 8where n observations m order polynomial(start off with n degrees of freedom, use up m+1 for mth-order polynomial)Dependence on more than one variableeg dependence of runoff volume on soil type and land cover,or dependence of aerodynamic drag on automobile shape and speed! " " 2 2! " " 2 2( )i iy a a x a xe y a a x a x + + + +.ultiple Linear Regression.ultiple Linear Regression.ultiple Linear Regression.ultiple Linear Regression#ith two independent variables, get a surface$ind the best-fit %plane& to the data$i $ $ i 1 1 " i r! a ! a a y S & &( )( )' i $ $ i 1 1 " i i $$ri $ $ i 1 1 " i i 11ri$ $ i 1 1 " i"r! a ! a a y ! $ "aS! a ! a a y ! $ "aS! a ! a a y $ "aS& & && & && &.ultiple Linear Regression.ultiple Linear Regression'uch li(e polynomial regression)um of s*uared residuals +earrange the e*uations,ery similar to polynomial regression)')'1111111]1

n1 ii$in1 ii in1 ii$1"n1 i4in1 i&in1 i$in1 i&in1 i$in1 iin1 i$in1 iiy !y !yaaa! ! !! ! !! ! n )')'1111111]1

n1 ii i $,n1 ii i 1,n1 ii$1"n1 i$i $,n1 ii $ i 1,n1 ii $,n1 ii $ i 1,n1 i$i 1,n1 ii 1,n1 ii $,n1 ii 1,y !y !yaaa! ! ! !! ! ! !! ! n &&.ultiple Linear Regression.ultiple Linear Regression-nce again, solve by any matri. method/holes(y decomposition is appropriate - symmetric and positive definite9ery use,ul ,or ,ittin1 *o:er euationm m $ $ 1 1 "ama$a1 "! a ! a ! a a y! ! ! a ym $ 1lo1 lo1 lo1 lo1 lo1 + + + + 0.ample1 )trength of concrete depends on cure time and cement/water ratio (or water content #//) cure time days W/C strength psi2 0.42 27704 0.55 26395 0.7 251916 0.53 34503 0.61 23157 0.67 25458 0.55 261327 0.66 369414 0.42 341420 0.58 3634 x1=[2 4 5 16 3 7 8 27 14 20]; x2=[0.42 0.55 0.7 0.53 0.61 0.67 0.55 0.66 0.42 0.58]; y=[2770 2639 2519 3450 2315 2545 2613 3694 3414 3634]; 2=pl-3&x1,x2,y,3r-3'; 4r!d -,; 5e&2,3$!,e6!d73,5'; 21=xla/el&3.ure ;!me &day5'3'; 5e&21,3,er 7e max!mum !era!-, ,um/er !max = 50, =21!er a0a1 da0 da11.00002.19775.06460.19772.06462.00001.02643.9349 -1.1713 -1.12963.00001.17574.36560.14940.43074.00001.10094.4054 -0.07480.03985.00001.10354.39690.0026 -0.00856.00001.10304.3973 -0.00050.00037.00001.10304.39720.00000.0000?au55-@e=-, me7-d 7a5 "-,Aer4eda =1.10304.3972" . cos!.! &-*$ 4 e ! f! 1"&" 1 C(oose initial a"$, a1&21 data *oints" . cos!.! &-*$ 4 e ! f! 1"&" 1 a"1'1"&", a14'&-*$