Ch17 Curve Fitting

8/12/2019 Ch17 Curve Fitting

1/44

Motilal Nehru National Institute of Technology

Civil Engineering Department

Computer Based Numerical Techniques

CE-401

Least Square Regression


2/44

CURVE FITTING

There are two general approaches for curve fitting: Least Squares regression:

Data exhibit a significant degree of scatter. The strategy isto derive a single curve that represents the general trend

of the data. Interpolation:

Data is very precise. The strategy is to pass a curve or aseries of curves through each of the points.


3/44

Introduction

In engineering, two types of applications areencountered:

Trend analysis . Predicting values of dependentvariable, may include extrapolation beyond datapoints or interpolation between data points.

Hypothesis testing . Comparing existingmathematical model with measured data.


4/44


5/44

Mathematical Background Arithmetic mean . The sum of the individual data

points (yi) divided by the number of points (n).

Standard deviation . The most common measure of aspread for a sample.

ni

n

y y i ,,1,

2)(,1

y yS n

S S it

t y


6/44

Mathematical Background (contd)

Variance . Representation of spread by the square ofthe standard deviation.

or

Coefficient of variation . Has the utility to quantify the

spread of data.

1 /22

2 n n y yS ii

y

1

)( 22 n

y yS i y

%100.. y

S vc y


7/44

Least Squares RegressionChapter 17

Linear RegressionFitting a straight line to a set of pairedobservations: (x 1, y 1 ), (x 2 , y 2 ),,(x n, y n ).y = a 0 + a 1 x + ea 1 - slopea 0 - intercepte - error, or residual, between the model andthe observations


8/44

Linear Regression: Residual


9/44

Linear Regression: Question

How to find a 0 and a 1 so that the error would beminimum?


10/44

Linear Regression: Criteria for a Best Fit

n

iii

n

ii xaa ye

110

1

)(min

e1 e2

e1= -e 2


11/44


n

iii

n

ii xaa ye

110

1

||||min


12/44


||||max min 10n

1i iii xaa ye


13/44

Linear Regression: Least Squares Fit

n

iii

n

iir xaa yeS

1

210

1

2 )(min

n

i

n

iiiii

n

iir xaa y y yeS

1 1

210

2

1

2 )()model,measured,(

Yields a unique line for a given set of data.


14/44

Linear Regression: Least Squares Fit

n

iii

n

iir xaa yeS

1

210

1

2 )(min

The coefficients a 0 and a 1 that minimize S r must satisfythe following conditions:

0

0

1

0

aS

aS

r

r


15/44

210

10

11

1

0

0

0)(2

0)(2

iiii

ii

iioi

r

ioio

r

xa xa x y

xaa y

x xaa ya

S

xaa yaS

Linear Regression:Determination of a o and a 1

210

10

00

iiii

ii

xa xa x y

ya xna

naa

2 equations with 2unknowns, can be solvedsimultaneously


16/44

Linear Regression:Determination of ao and a1

221

ii

iiii

x xn

y x y xna

xa ya 10


17/44


18/44

18


19/44

19


20/44

Error Quantification of Linear Regression

Total sum of the squares around the mean forthe dependent variable, y, is St

Sum of the squares of residuals around the

regression line is Sr

2it y yS )(

2n

1ii1oi

n

1i

2ir xaa yeS )(


21/44


St -Sr quantifies the improvement or errorreduction due to describing data in terms of astraight line rather than as an average value.

t

r t

S S S

r 2

r 2: coefficient of determinationr : correlation coefficient


22/44


For a perfect fit: Sr= 0 and r = r 2 =1, signifying that the line

explains 100 percent of the variability of thedata.

For r = r 2 = 0, Sr = St, the fit represents no

improvement.


23/44

Least Squares Fit of a Straight Line:Example

Fit a straight line to the x and y values in thefollowing Table:

5.119 ii y x

28 i x 0.24 i y

1402 i x

4728

x

428571.37

24 y

xi yi xiyi xi2

1 0.5 0.5 1

2 2.5 5 4

3 2 6 9

4 4 16 165 3.5 17.5 25

6 6 36 36

7 5.5 38.5 49

28 24 119.5 140


24/44

Least Squares Fit of a Straight Line: Example(contd)

07142857.048392857.0428571.3

8392857.0281407

24285.1197

)(

10

2

221

xa ya

x xn

y x y xna

ii

iiii

Y = 0.07142857 + 0.8392857 x


25/44

Least Squares Fit of a Straight Line: Example(Error Analysis)

9911.22

ir eS

932.0868.02 r r

xi y i 1 0.52 2.5

3 2.04 4.05 3.56 6.07 5.5

8.5765 0.16870.8622 0.5625

2.0408 0.34730.3265 0.32650.0051 0.58966.6122 0.79724.2908 0.1993

2^

22 )( y y e ) y(y iii

28 24.0 22.7143 2.9911

868.02

t

r t

S S S

r

7143.222 y yS it


26/44

Least Squares Fit of a Straight Line:Example (Error Analysis)

9457.117

7143.221n

S s t y

7735.0

27

9911.2

2/

n

S s r x y

y x y S S /

The standard deviation (quantifies the spread around the mean):

The standard error of estimate (quantifies the spread around theregression line)

Because , the linear regression model has good fitness


27/44

Algorithm for linear regression


28/44

Linearization of Nonlinear Relationships

The relationship between the dependent andindependent variables is linear. However, a few types of nonlinear functions

can be transformed into linear regressionproblems.The exponential equation.The power equation.The saturation-growth-rate equation.


29/44


30/44

Linearization of Nonlinear Relationships1. The exponential equation.

xbea y 11

xba y 11lnln

y* = a o + a 1 x


31/44

Linearization of Nonlinear Relationships2. The power equation

22

b xa y

xba y logloglog 22

y* = a o + a 1 x*


32/44

Linearization of Nonlinear Relationships3. The saturation-growth-rate equation

xb x

a y3

3

xa

ba y

111

3

3

3

y* = 1/yao = 1/a 3a1 = b3/a 3x* = 1/x


33/44

ExampleFit the following Equation:

22

b xa y

to the data in the following table:

xi y i 1 0.52 1.73 3.44 5.75 8.415 19.7

X*=log x i Y*=logy i 0 -0.3010.301 0.2260.477 0.5340.602 0.7530.699 0.9222.079 2.141

)log(log 22

b xa y

2120

**

log

logloglet

b , aaa

x, y, X Y xba y logloglog 22

*10* X aaY


34/44

Example

Xi Yi X*i=Log(X) Y*i=Log(Y) X*Y* X*^2

1 0.5 0.0000 -0.3010 0.0000 0.0000

2 1.7 0.3010 0.2304 0.0694 0.0906

3 3.4 0.4771 0.5315 0.2536 0.2276

4 5.7 0.6021 0.7559 0.4551 0.3625

5 8.4 0.6990 0.9243 0.6460 0.4886

Sum 15 19.700 2.079 2.141 1.424 1.169

1 2 22

0 1

5 1.424 2.079 2.141 1.755 1.169 2.079( )

0.4282 1.75 0.41584 0.334

i i i i

i i

n x y x yan x x

a y a x


35/44

Linearization of Nonlinear

Functions: Example

log y=-0.334+1.75log x

1.750.46 y x


36/44

Polynomial Regression

Some engineering data is poorly representedby a straight line.

For these cases a curve is better suited to fitthe data.

The least squares method can readily beextended to fit the data to higher orderpolynomials.


37/44

Polynomial Regression (contd)

A parabola is preferable


38/44


A 2nd order polynomial (quadratic) is defined by:

The residuals between the model and the data:

The sum of squares of the residual:

e xa xaa y o 2

21

221 iioii xa xaa ye

22212 iioiir xa xaa yeS


39/44


0 x xa xaa y2aS

0 x xa xaa y2aS

0 xa xaa y2aS

2i

2i2i1oi

2

r

i2i2i1oi

1

r

2i2i1oi

o

r

)(

)(

)(

4i2

3i1

2ioi

2i

3i2

2i1ioii

2i2i1oi

xa xa xa y x

xa xa xa y x

xa xaan y 3 linear equationswith 3 unknowns(a o,a 1,a 2), can besolved


40/44


A system of 3x3 equations needs to be solved to determinethe coefficients of the polynomial.

The standard error & the coefficient of determination

3/ nS

s r x yt

r t

S S S

r 2

ii

ii

i

iii

iii

ii

y x y x

y

aa

a

x x x x x x

x xn

22

1

0

432

32

2


41/44


General:The mth-order polynomial:

A system of (m+1)x(m+1) linear equations must be solved fordetermining the coefficients of the mth-order polynomial.

The standard error:

The coefficient of determination:

e xa xa xaa y mmo .....2

21

1/ mn S s r x y

t

r t

S S S

r 2


42/44

Polynomial Regression- Example

Fit a second order polynomial to data:

2253 i x

9794 i x

x i y i x i 2 x i

3 x i 4 x i y i x i

2 y i0 2.1 0 0 0 0 0

1 7.7 1 1 1 7.7 7.7

2 13.6 4 8 16 27.2 54.4

3 27.2 9 27 81 81.6 244.8

4 40.9 16 64 256 163.6 654.4

5 61.1 25 125 625 305.5 1527.5

15 152.6 55 225 979 585.6 2489

6.585 ii y x

15 i x

6.152 i y

552

i x

43.256

6.152 ,5.2

615

y x 8.24882 ii y x


43/44

Polynomial Regression- Example (contd)

The system of simultaneous linear equations:

2

210

86071.135929.247857.2

86071.1,35929.2,47857.2

x x y

aaa

8.2488

6.585

6.152

97922555

2255515

55156

2

1

0

a

a

a

74657.32

ir eS 39.25132 y yS it


44/44

Polynomial Regression- Example (contd) x i y i y model e i

2 (y i -y`) 2

0 2.1 2.4786 0.14332 544.42889

1 7.7 6.6986 1.00286 314.45929

2 13.6 14.64 1.08158 140.01989

3 27.2 26.303 0.80491 3.12229

4 40.9 41.687 0.61951 239.22809

5 61.1 60.793 0.09439 1272.13489

15 152.6 3.74657 2513.39333

The standard error of estimate:

The coefficient of determination:

12.136

74657.3/ x y s

99925.0,99851.074657.339.2513 22 r r r

Ch17 Curve Fitting

Documents

Transcript of Ch17 Curve Fitting