Multiple Regression (continued)& Polynomial Regression Theory Consider simplest form of multiple regression where y is the dependant variable and x 1 and x 2 independent variables y = b 0 + b 1 x 1 + b 2 x 2 + e Where e is a random error term Theory b 1 = [ (x 2 2 )( x 1 y)-( x 1 x 2 )( x 2 y)] [( x 1 2 )( x 2 2 )-( x 1 x 2 ) 2 ] [( x 1 2 )( x 2 2 )-( x 1 x 2 ) 2 ] b 2 = [ (x 1 2 )( x 2 y)-( x 1 x 2 )( x 1 y)] [( x 2 2 )( x 1 2 )-( x 1 x 2 ) 2 ] [( x 2 2 )( x 1 2 )-( x 1 x 2 ) 2 ] b 0 = mean(y)-b 1 mean(x 1 )-b 2 mean(x 2 ) Analysis of Variance Table Introduction to Matrixes 6b 1 + 3b 2 = 24 4b b 2 = 20 } Simultaneous equations 6 3 b b 2 20 x= } [[[]]] Matrix Form Matrix Formation y = b 0 + b 1 x 1 + b 2 x 2 + .. + b n x n e Y = X x b + e Matrix Formation Y = X x b Matrix Formation F = ee = YY - 2YXb + bb XX dF/db = 2XX b - 2YX = 0 XXb = YX Two Variable Example x 11 x 12 x 21 x 22 x 31 x 32 x 41 x 42 x 51 x 52 x 11 x 21 x 31 x 41 x 51 x 12 x 22 x 32 x 42 x 52 = x 1 2 x 1 x 2 x 2 x 1 x 2 2 x Matrix Formation XX = Two Variable Example = x1y x1yx2yx2y x1y x1yx2yx2y y1y1y2y2y1y1y2y2 x x 11 x 12 x 21 x 22 Matrix Formation = YX Two Variable Example x 1 2 x 1 x 2 b 1 x 1 y x 2 x 1 x 2 2 b 2 x 2 y x= [[[]]] XX x b = YX (XX) -1 XX x b = (XX) -1 YX b = (XX) -1 YX b = (XX) -1 YX Matrix Formation Find the inverse of XX Donated by (XX) -1 b = (XX) -1 YX Matrix Inverse with Two Variables A x A -1 = [U] A x A -1 = [U] Matrix Inverse with Two Variables a b c d d -b -c a 1ad-bc [[]] x [] = A x A -1 = [U] A x A -1 = [U] Matrix Inverse with Two Variables x 1 2 x 1 x 2 b 1 x 1 y x 2 x 1 x 2 2 b 2 x 2 y x= [[[]]] Matrix Inverse with Two Variables x 1 2 x 1 x 2 b 1 x 1 y x 2 x 1 x 2 2 b 2 x 2 y x= [[[]]] XX x b = XY Matrix Inverse with Two Variables x 1 2 x 1 x 2 b 1 x 1 y x 2 x 1 x 2 2 b 2 x 2 y x= [[[]]] XX x b = XY x 1 2 x 1 x 2 x x 1 x 2 x 2 x 1 x x 2 x 1 x 1 2 x [[]] 1 ad-bc = [U] XX x (XX) -1 = Unit Matrix Inverse with Two Variables x x 1 x 2 x 1 y b 1 - x 2 x 1 x 1 2 x 2 y b 2 x [[]] 1 ad-bc= (XX) -1 x Y = b [] 1 ad-bc = x 2 2 x [ x 2 x 1 ] 2 Compare Matrix with None b 1 = [ (x 2 2 )( x 1 y)-( x 1 x 2 )( x 2 y)] [( x 1 2 )( x 2 2 )-( x 1 x 2 ) 2 ] [( x 1 2 )( x 2 2 )-( x 1 x 2 ) 2 ] b 2 = [ (x 1 2 )( x 2 y)-( x 1 x 2 )( x 1 y)] [( x 2 2 )( x 1 2 )-( x 1 x 2 ) 2 ] [( x 2 2 )( x 1 2 )-( x 1 x 2 ) 2 ] Forward Step- Wise Regression Two Variable Multiple Regression Analysis of Variance Table y = x x 2 Two Variable Multiple Regression There is significant regression effects by regressing both independent variables onto the dependant variable The is significant linear relationship between height (x 1 ) and yield but no relationship between yield and tiller There is significant linear relationship between tiller (x 2 ) and yield and no relationship between yield and height Two Variable Multiple Regression Forward Step-wise Regression Backward Step-wise Regression We may have made the relationship too complex by including both variables. Two Variable Multiple Regression Analysis of Variance Table y = 10, Height (x 1 ) Analysis of Variance Table y = Height Tiller Analysis of Variance Table y = Height Tiller Analysis of Variance Table y = Height Tiller Analysis of Variance Table y = 10, Height (x 1 ) Forward Step-Wise Regression Example 2 20 Spring Canola Cultivars Average over 10 environments Seed yield; plant establishment; days to first flowering, days to end of flowering; plant height; and oil content CharacterEst.F.St.F.Fi.Ht.%OilYield Establish1.00 F.Start F.Finish Height %Oil Yield Example #2 CharacterEst.F.St.F.Fi.Ht.%OilYield Establish F.Start F.Finish Height %Oil Yield Example #2 Analysis of Variance Table y = 3, x F.Start CharacterEst.F.St.F.Fi.Ht.%OilYield Establish F.Start F.Finish Height %Oil Yield Example #2 A[i,j] = A[i,j]{A i,x x A x,j }/A x,x CharacterEst.F.St.F.Fi.Ht.%OilYield Establish F.Start F.Finish Height %Oil Yield Example #2 A[i,j] = A[i,j]{A i,x x A x,j }/A x,x CharacterEst.F.St.F.Fi.Ht.%OilYield Establish F.Start F.Finish Height %Oil Yield Example #2 A[i,j] = A[i,j]{A i,x x A x,j }/A x,x CharacterEst.F.St.F.Fi.Ht.%OilYield Establish F.Start F.Finish Height %Oil Yield Example #2 A[i,j] = A[i,j]{A i,x x A x,j }/A x,x CharacterEst.F.St.F.Fi.Ht.%OilYield Establish F.Start F.Finish Height %Oil Yield Example #2 A[i,j] = A[i,j]{A i,x x A x,j }/A x,x CharacterEst.F.St.F.Fi.Ht.%OilYield Establish F.Start F.Finish Height %Oil Yield Example #2 A[i,j] = A[i,j]{A i,x x A x,j }/A x,x Analysis of Variance Table y = F.Start %Oil CharacterEst.F.St.F.Fi.Ht.%OilYield Establish F.Start F.Finish Height %Oil Yield Example #2 A[i,j] = A[i,j]{A i,x x A x,j }/A x,x Analysis of Variance Table y = FS %Oil Height Analysis of Variance Table y = F.Start %Oil Forward Step-Wise Regression Enter the variable most associated with the dependant variable. Check to see if relationship is significant Adjust the relationship between the dependant variable and the other remaining variables, accounting for the relationship between the dependant variable and the entered variable(s) Forward Step- Wise Regression Enter most correlated variable Forward Step- Wise Regression Enter most correlated variable Check that entry is significant Forward Step- Wise Regression Enter most correlated variable Check that entry is significant Adjust correlation with other variables Forward Step- Wise Regression Enter most correlated variable Check that entry is significant Adjust correlation with other variables Forward Step- Wise Regression Polynomial Regression Analysis of Variance Table y = N N 2 Polynomial Regression dY/dN = Slope y = N N 2 Polynomial Regression y = N N 2 dy/dN = N N = n = 36.08 Multivariate Transformation