Orthogonal Decomp. Thm

Announcements

Ï Exam 2 on Thurs, Feb 25 in class.

Ï Practice Exam will be uploaded by 2 pm today.

Ï I will do some misc. topics (sec 5.5 and someapplications)tomorrow. These WILL NOT be covered on theexam but are useful for MA 3521.

Ï Review on Wednesday in class. I will have o�ce hours on Wedfrom 1-4 pm.

Ï Reminder that MA 3521 Di�. Eq starts next Monday sametime and place (Prof Todd King)

Ï Please collect your graded exams onFriday between 7 am and 6 pm in Fisher 214. I would like tosettle any grading issues and submit the grades o�cially onSaturday.

Last Week: Orthogonal Projection

The orthogonal projection of y onto any line L through u and 0 isgiven by

y= projLy=y �uu �u

The orthogonal projection is a vector (not a number).

The quantity ‖y− y‖ gives the distance between y and the line L.

Section 6.3 Orthogonal Projections

1. The above formula for orthogonal projection is for R2

2. We can extend this formula to any vector in Rn

Consider a vector y and a subspace W in Rn . We can �nd thefollowing

1. A unique vector y which is a linear combination of vectors inW.

2. The vector y− y which is orthogonal to W (orthogonal to eachvector in W). We can also say that y− y is in W⊥

3. This vector y is the unique vector in W closest to y

Decomposition of a vector

Let {u1,u2, . . . ,un} be an orthogonal basis in Rn . Then any vector yin Rn can be written as a sum of 2 vectors

y= z1 +z2,

where z1 is formed by a linear combination of few of these nvectors and z2 contains the rest of the vectors not contained in z1.

Example

Let {u1,u2,u3,u4,u5,u6} be an orthogonal basis in R6. Then anyvector y in R6 can be written as

y= c1u1 + c2u2 + c3u3 + c4u4︸︷︷︸z1

+c5u5 + c6u6︸︷︷︸z2

Here the subspace W=Span{u1,u2,u3,u4}, z1 is in W and z2 is inW⊥ (since the dot product of any vector in W with u5 or u6 is zero)

y= z1 +z2,

Example

y= c1u1 + c2u2 + c3u3 + c4u4︸︷︷︸z1

+c5u5 + c6u6︸︷︷︸z2

y= z1 +z2,

Example

y= c1u1 + c2u2 + c3u3 + c4u4︸︷︷︸z1

+c5u5 + c6u6︸︷︷︸z2

y= z1 +z2,

Example

y= c1u1 + c2u2 + c3u3 + c4u4︸︷︷︸z1

+c5u5 + c6u6︸︷︷︸z2

Example 2, section 6.3Let {u1,u2,u3,u4} be an orthogonal basis for R4.

−21−11

11−2−1

−111−2

45−33

Write v as the sum of 2 vectors, one in the Span{u1} and the otherin Span{u2,u3,u4}

Solution: The vector in the Span{u1} will be

z1 = c1u1 = v �u1

u1 �u1u1

v �u1 = 4+10−3+3 = 14, u1 �u1 = 1+4+1+1 = 7

z1 = 14

7u1 = 2u1

Example 2, section 6.3Let {u1,u2,u3,u4} be an orthogonal basis for R4.

−21−11

11−2−1

−111−2

45−33

Write v as the sum of 2 vectors, one in the Span{u1} and the otherin Span{u2,u3,u4}

Solution: The vector in the Span{u1} will be

z1 = c1u1 = v �u1

u1 �u1u1

v �u1 = 4+10−3+3 = 14, u1 �u1 = 1+4+1+1 = 7

z1 = 14

7u1 = 2u1

Example 2, section 6.3

The vector in the Span{u2,u3,u4} will be

z2 = c2u2 + c3u3 + c4u4 = v �u2

u2 �u2u2 + v �u3

u3 �u3u3 + v �u4

u4 �u4u4

v �u2 =−8+5+3+3 = 3, u2 �u2 = 4+1+1+1 = 7

v �u3 = 4+5+6−3 = 12, u3 �u3 = 1+1+4+1 = 7

v �u4 =−4+5−3−6 =−8, u4 �u4 = 4+1+1+1 = 7

z2 = 3

7u2 + 12

7u3 − 8

y= z1 +z2 = 2u1 + 3

7u2 + 12

7u3 − 8

z2 = c2u2 + c3u3 + c4u4 = v �u2

u2 �u2u2 + v �u3

u3 �u3u3 + v �u4

u4 �u4u4

v �u2 =−8+5+3+3 = 3, u2 �u2 = 4+1+1+1 = 7

v �u3 = 4+5+6−3 = 12, u3 �u3 = 1+1+4+1 = 7

v �u4 =−4+5−3−6 =−8, u4 �u4 = 4+1+1+1 = 7

z2 = 3

7u2 + 12

7u3 − 8

y= z1 +z2 = 2u1 + 3

7u2 + 12

7u3 − 8

z2 = c2u2 + c3u3 + c4u4 = v �u2

u2 �u2u2 + v �u3

u3 �u3u3 + v �u4

u4 �u4u4

v �u2 =−8+5+3+3 = 3, u2 �u2 = 4+1+1+1 = 7

v �u3 = 4+5+6−3 = 12, u3 �u3 = 1+1+4+1 = 7

v �u4 =−4+5−3−6 =−8, u4 �u4 = 4+1+1+1 = 7

z2 = 3

7u2 + 12

7u3 − 8

y= z1 +z2 = 2u1 + 3

7u2 + 12

7u3 − 8

z2 = c2u2 + c3u3 + c4u4 = v �u2

u2 �u2u2 + v �u3

u3 �u3u3 + v �u4

u4 �u4u4

v �u2 =−8+5+3+3 = 3, u2 �u2 = 4+1+1+1 = 7

v �u3 = 4+5+6−3 = 12, u3 �u3 = 1+1+4+1 = 7

v �u4 =−4+5−3−6 =−8, u4 �u4 = 4+1+1+1 = 7

z2 = 3

7u2 + 12

7u3 − 8

y= z1 +z2 = 2u1 + 3

7u2 + 12

7u3 − 8

z2 = c2u2 + c3u3 + c4u4 = v �u2

u2 �u2u2 + v �u3

u3 �u3u3 + v �u4

u4 �u4u4

v �u2 =−8+5+3+3 = 3, u2 �u2 = 4+1+1+1 = 7

v �u3 = 4+5+6−3 = 12, u3 �u3 = 1+1+4+1 = 7

v �u4 =−4+5−3−6 =−8, u4 �u4 = 4+1+1+1 = 7

z2 = 3

7u2 + 12

7u3 − 8

y= z1 +z2 = 2u1 + 3

7u2 + 12

7u3 − 8

z2 = c2u2 + c3u3 + c4u4 = v �u2

u2 �u2u2 + v �u3

u3 �u3u3 + v �u4

u4 �u4u4

v �u2 =−8+5+3+3 = 3, u2 �u2 = 4+1+1+1 = 7

v �u3 = 4+5+6−3 = 12, u3 �u3 = 1+1+4+1 = 7

v �u4 =−4+5−3−6 =−8, u4 �u4 = 4+1+1+1 = 7

z2 = 3

7u2 + 12

7u3 − 8

y= z1 +z2 = 2u1 + 3

7u2 + 12

7u3 − 8

Orthogonal Decomposition Theorem

TheoremLet W be a subspace of Rn . Then each y in Rn can be uniquely

written as

y= y+z,

where y is in W and z is in W⊥. If{u1,u2, . . . ,up

}is an orthogonal

basis for W,

y= y �u1

u1 �u1u1 + y �u2

u2 �u2u2 + . . .+ y �up

up �upup

z= y− y.

TheoremLet W be a subspace of Rn . Then each y in Rn can be uniquely

written as

y= y+z,

where y is in W and z is in W⊥. If{u1,u2, . . . ,up

}is an orthogonal

basis for W,

y= y �u1

u1 �u1u1 + y �u2

u2 �u2u2 + . . .+ y �up

up �upup

z= y− y.

1. As in the case of R2, the new vector y is the orthogonalprojection of y onto W. (In section 6.2, the orthogonalprojection was onto a line through u and the origin)

2. This is nothing but an extension of the orthogonal projectionformula we had for R2 in section 6.2 to accomodate theremaining vectors.

3. If W is one dimensional, we get the formula for y from sec 6.2

Example 4, section 6.3Let {u1,u2} be an orthogonal set.

,u1 = 3

,u2 = −4

Find the orthogonal projection of y onto Span{u1,u2}

Solution: The desired orthogonal projection is

y= c1u1 + c2u2 = y �u1

u1 �u1u1 + y �u2

u2 �u2u2

y �u1 = 18+12 = 30, u1 �u1 = 9+16 = 25

y �u2 =−24+9 =−15, u2 �u2 = 16+9 = 25

,u1 = 3

,u2 = −4

y= c1u1 + c2u2 = y �u1

u1 �u1u1 + y �u2

u2 �u2u2

y �u1 = 18+12 = 30, u1 �u1 = 9+16 = 25

y �u2 =−24+9 =−15, u2 �u2 = 16+9 = 25

,u1 = 3

,u2 = −4

y= c1u1 + c2u2 = y �u1

u1 �u1u1 + y �u2

u2 �u2u2

y �u1 = 18+12 = 30, u1 �u1 = 9+16 = 25

y �u2 =−24+9 =−15, u2 �u2 = 16+9 = 25

,u1 = 3

,u2 = −4

y= c1u1 + c2u2 = y �u1

u1 �u1u1 + y �u2

u2 �u2u2

y �u1 = 18+12 = 30, u1 �u1 = 9+16 = 25

y �u2 =−24+9 =−15, u2 �u2 = 16+9 = 25

25u1 − 15

−430

− −12

25u1 − 15

−430

− −12

Let W be the subspace spanned by the u's, then write y as the sumof a vector in W and a vector orthogonal to W.

y= −1

,u1 = 1

,u2 = −1

Solution: A vector in W is the orthogonal projection of y onto Wwhich is

y= c1u1 + c2u2 = y �u1

u1 �u1u1 + y �u2

u2 �u2u2

y �u1 =−1+4+3 = 6, u1 �u1 = 1+1+1 = 3

y �u2 = 1+12−6 = 7, u2 �u2 = 1+9+4 = 14

y= −1

,u1 = 1

,u2 = −1

y= c1u1 + c2u2 = y �u1

u1 �u1u1 + y �u2

u2 �u2u2

y �u1 =−1+4+3 = 6, u1 �u1 = 1+1+1 = 3

y �u2 = 1+12−6 = 7, u2 �u2 = 1+9+4 = 14

y= −1

,u1 = 1

,u2 = −1

y= c1u1 + c2u2 = y �u1

u1 �u1u1 + y �u2

u2 �u2u2

y �u1 =−1+4+3 = 6, u1 �u1 = 1+1+1 = 3

y �u2 = 1+12−6 = 7, u2 �u2 = 1+9+4 = 14

y= −1

,u1 = 1

,u2 = −1

y= c1u1 + c2u2 = y �u1

u1 �u1u1 + y �u2

u2 �u2u2

y �u1 =−1+4+3 = 6, u1 �u1 = 1+1+1 = 3

y �u2 = 1+12−6 = 7, u2 �u2 = 1+9+4 = 14

3u1 + 7

−13−2

+ −1

232−1

A vector orthogonal to W will be z= y− y which is −1

= −5

3u1 + 7

−13−2

+ −1

232−1

= −5

3u1 + 7

−13−2

+ −1

232−1

= −5

Example 10, section 6.3Let W be the subspace spanned by the u's, then write y as the sumof a vector in W and a vector orthogonal to W.

110−1

0−11−1

y= c1u1 + c2u2 + c3u3 = y �u1

u1 �u1u1 + y �u2

u2 �u2u2 + y �u3

u3 �u3u3

y �u1 = 3+4−6 = 1, u1 �u1 = 1+1+1 = 3

y �u2 = 3+5+6 = 14, u2 �u2 = 1+1+1 = 3

y �u3 =−4+5−6 =−5, u3 �u3 = 1+1+1 = 3

110−1

0−11−1

y= c1u1 + c2u2 + c3u3 = y �u1

u1 �u1u1 + y �u2

u2 �u2u2 + y �u3

u3 �u3u3

y �u1 = 3+4−6 = 1, u1 �u1 = 1+1+1 = 3

y �u2 = 3+5+6 = 14, u2 �u2 = 1+1+1 = 3

y �u3 =−4+5−6 =−5, u3 �u3 = 1+1+1 = 3

110−1

0−11−1

y= c1u1 + c2u2 + c3u3 = y �u1

u1 �u1u1 + y �u2

u2 �u2u2 + y �u3

u3 �u3u3

y �u1 = 3+4−6 = 1, u1 �u1 = 1+1+1 = 3

y �u2 = 3+5+6 = 14, u2 �u2 = 1+1+1 = 3

y �u3 =−4+5−6 =−5, u3 �u3 = 1+1+1 = 3

110−1

0−11−1

y= c1u1 + c2u2 + c3u3 = y �u1

u1 �u1u1 + y �u2

u2 �u2u2 + y �u3

u3 �u3u3

y �u1 = 3+4−6 = 1, u1 �u1 = 1+1+1 = 3

y �u2 = 3+5+6 = 14, u2 �u2 = 1+1+1 = 3

y �u3 =−4+5−6 =−5, u3 �u3 = 1+1+1 = 3

110−1

0−11−1

y= c1u1 + c2u2 + c3u3 = y �u1

u1 �u1u1 + y �u2

u2 �u2u2 + y �u3

u3 �u3u3

y �u1 = 3+4−6 = 1, u1 �u1 = 1+1+1 = 3

y �u2 = 3+5+6 = 14, u2 �u2 = 1+1+1 = 3

y �u3 =−4+5−6 =−5, u3 �u3 = 1+1+1 = 3

3u1 + 14

3u2 − 5

110−1

0−11−1

A vector orthogonal to W will be z= y− y which is

−2220

3u1 + 14

3u2 − 5

110−1

0−11−1

A vector orthogonal to W will be z= y− y which is3456

−2220

3u1 + 14

3u2 − 5

110−1

0−11−1

A vector orthogonal to W will be z= y− y which is

−2220

Applications

1. Let W = Span{u1,u2, . . . ,up

}(an orthogonal basis). Let y be

any vector in Rn . Then the orthogonal projection y gives theclosest point in W to y.

2. The orthogonal projection y is also the best approximation toy by the elements of W (this means we can replace y by avector in some �xed subspace W and this replacement comeswith the least error if we choose y).

Applications

1. Let W = Span{u1,u2, . . . ,up

}(an orthogonal basis). Let y be

any vector in Rn . Then the orthogonal projection y gives theclosest point in W to y.

2. The orthogonal projection y is also the best approximation toy by the elements of W (this means we can replace y by avector in some �xed subspace W and this replacement comeswith the least error if we choose y).

Example 12, section 6.3Find the closest point to y in the subspace spanned by v1 and v2.

3−11

1−2−12

−4103

Solution: The closest point to y is the orthogonal projection of yonto the span of v1 and v2 which is

y= c1v1 + c2v2 = y �v1

v1 �v1v1 + y �v2

v2 �v2v2

y �v1 = 3+2−1+26 = 30, v1 �v1 = 1+4+1+4 = 10y �v2 =−12−1+39 = 26, v2 �v2 = 16+1+9 = 26

10v1 + 26

26v2 = 3

1−2−12

−4103

−1−5−39

3−11

1−2−12

−4103

y= c1v1 + c2v2 = y �v1

v1 �v1v1 + y �v2

v2 �v2v2

y �v1 = 3+2−1+26 = 30, v1 �v1 = 1+4+1+4 = 10y �v2 =−12−1+39 = 26, v2 �v2 = 16+1+9 = 26

10v1 + 26

26v2 = 3

1−2−12

−4103

−1−5−39

3−11

1−2−12

−4103

y= c1v1 + c2v2 = y �v1

v1 �v1v1 + y �v2

v2 �v2v2

y �v1 = 3+2−1+26 = 30, v1 �v1 = 1+4+1+4 = 10y �v2 =−12−1+39 = 26, v2 �v2 = 16+1+9 = 26

10v1 + 26

26v2 = 3

1−2−12

−4103

−1−5−39

3−11

1−2−12

−4103

y= c1v1 + c2v2 = y �v1

v1 �v1v1 + y �v2

v2 �v2v2

y �v1 = 3+2−1+26 = 30, v1 �v1 = 1+4+1+4 = 10

y �v2 =−12−1+39 = 26, v2 �v2 = 16+1+9 = 26

10v1 + 26

26v2 = 3

1−2−12

−4103

−1−5−39

3−11

1−2−12

−4103

y= c1v1 + c2v2 = y �v1

v1 �v1v1 + y �v2

v2 �v2v2

y �v1 = 3+2−1+26 = 30, v1 �v1 = 1+4+1+4 = 10y �v2 =−12−1+39 = 26, v2 �v2 = 16+1+9 = 26

10v1 + 26

26v2 = 3

1−2−12

−4103

−1−5−39

3−11

1−2−12

−4103

y= c1v1 + c2v2 = y �v1

v1 �v1v1 + y �v2

v2 �v2v2

y �v1 = 3+2−1+26 = 30, v1 �v1 = 1+4+1+4 = 10y �v2 =−12−1+39 = 26, v2 �v2 = 16+1+9 = 26

10v1 + 26

26v2 = 3

1−2−12

−4103

−1−5−39

Example 14, section 6.3Find the best approximation z by v1 and v2 of the form c1v1 +c2v2.

240−1

20−1−3

5−242

Solution: The best approximation to z is the orthogonal projectionof z onto the span of v1 and v2 which is

z= c1v1 + c2v2 = z �v1

v1 �v1v1 + z �v2

v2 �v2v2

z �v1 = 4+3 = 7, v1 �v1 = 4+1+9 = 14z �v2 = 10−8−2 = 0, v2 �v2 = 25+4+16+4 = 49

14v1 + 0

49v2︸︷︷︸

20−1−3

−0.5−1.5

240−1

20−1−3

5−242

z= c1v1 + c2v2 = z �v1

v1 �v1v1 + z �v2

v2 �v2v2

z �v1 = 4+3 = 7, v1 �v1 = 4+1+9 = 14z �v2 = 10−8−2 = 0, v2 �v2 = 25+4+16+4 = 49

14v1 + 0

49v2︸︷︷︸

20−1−3

−0.5−1.5

240−1

20−1−3

5−242

z= c1v1 + c2v2 = z �v1

v1 �v1v1 + z �v2

v2 �v2v2

z �v1 = 4+3 = 7, v1 �v1 = 4+1+9 = 14z �v2 = 10−8−2 = 0, v2 �v2 = 25+4+16+4 = 49

14v1 + 0

49v2︸︷︷︸

20−1−3

−0.5−1.5

240−1

20−1−3

5−242

z= c1v1 + c2v2 = z �v1

v1 �v1v1 + z �v2

v2 �v2v2

z �v1 = 4+3 = 7, v1 �v1 = 4+1+9 = 14

z �v2 = 10−8−2 = 0, v2 �v2 = 25+4+16+4 = 49

14v1 + 0

49v2︸︷︷︸

20−1−3

−0.5−1.5

240−1

20−1−3

5−242

z= c1v1 + c2v2 = z �v1

v1 �v1v1 + z �v2

v2 �v2v2

z �v1 = 4+3 = 7, v1 �v1 = 4+1+9 = 14z �v2 = 10−8−2 = 0, v2 �v2 = 25+4+16+4 = 49

14v1 + 0

49v2︸︷︷︸

20−1−3

−0.5−1.5

240−1

20−1−3

5−242

z= c1v1 + c2v2 = z �v1

v1 �v1v1 + z �v2

v2 �v2v2

z �v1 = 4+3 = 7, v1 �v1 = 4+1+9 = 14z �v2 = 10−8−2 = 0, v2 �v2 = 25+4+16+4 = 49

14v1 + 0

49v2︸︷︷︸

20−1−3

−0.5−1.5

Topics to learn for Test 2

1. Evaluation of determinants, use of determinants in Cramer'srule, �nding adjugate of a matrix and �nding areas ofparallelograms.

2. Finding char. equation, eigenvalues, eigenvectors of a squarematrix (including triangular matrix) and use these indiagonalizing a matrix and computing higher exponents of thematrix. (you should know the condition under whichdiagonalization may fail).

3. Finding length of a vector, unit vector, orthogonal vectors andorthogonal basis, orthogonal projection onto a subspacespanned by any number of vectors, closest point/bestapproximation of a vector

Good luck. Feel free to email me if you have any questions. Pleasecome prepared for Wednesday's review. Bring any problems youwant me to go over.

Orthogonal Decomp. Thm

Education

Transcript of Orthogonal Decomp. Thm