STK 500 Pengantar Teori Statistika VectorPengantar Teori Statistika Vector x 2 2 2 x 1 p2 22 1 2...

STK 500

Pengantar Teori Statistika

Vector

x 2 2 2x 1 p2

2 21 2

length of L x x x

x x

x'x

Vectors have geometric properties of length and direction – for a vector

we have

x 1

2

xx

2

1 x1

x2

x 1 2x x

Why?

Geometry of Vectors

2 2ah b

2 21 2

length of hypotenuse L L L

x x

x'x

Geometry of Vectors

Recall the Pythagorean Theorem: in any right triangle, the lengths of the hypotenuse and the other two sides are related by the simple formula.

2

1

side a

side b

Geometry of Vectors

Vector addition – for the vectors

we have

,x y1 1

2 2

x yx y

2

1

x 1 2x x

x y 1 1

2 2

x y+ x y

y 1 2y y

q

1 1

2 2

x yx y

xx 2 2 2 2c 1 p2

2 2 21 2

length of c L c x x x

c x x

c x'x

Geometry of Vectors

Scalar multiplication changes only the vector length – for the vector

we have

x 1

2

xx

2

1

x 1 2c c x x

Geometry of Vectors

Vector multiplication have angles between them – for the vectors

we have

,x y1 1

2 2

x yx y

2

1

x 1 2x x

q q x y x y

xy xy xyarccos cos

L L L L x'x y'y y 1 2y y

q

A Little Trigonometry Review

The Unit Circle 2

1 q Cos q = 1

0 Cos q 1

Cos q = 0

-1 Cos q 0

0 Cos q 1 -1 Cos q 0

Cos q = 0

Cos q = -1

Suppose we rotate x any y so x lies on axis 1:

2

1

x 1 2x x

y 1 2y y

qxy


What does this imply about rxy? 2

1

qxy Cos q = 1

0 Cos q 1

Cos q = 0

-1 Cos q 0

0 Cos q 1 -1 Cos q 0

Cos q = 0

Cos q = -1


What is the correlation between the vectors x and y?

2

1

,1.0 -0.50.6 -0.3

x y

y

x

Plotting in the column space gives us

Geometry of Vectors

Rotating so x lies on axis 1 makes it easier to see:

2

1

Qxy=1800

Geometry of Vectors


,1.0 -0.50.6 -0.3

x y

q

cosL L

1.0 -0.5 + 0.6 -0.3

1.0 1.0 0.6 0.6 -0.5 -0.5 -0.3 -0.3

-0.68= -1.00

1.36 0.34

x y

xy xy

x'x y'y

Geometry of Vectors

Of course, we can see this by plotting the these values in the x,y (row) space:

,1.0 -0.50.6 -0.3

x y

Y

X

1.0, -0.5

0.6, -0.3

Geometry of Vectors


,x y1.0 -0.500.4 1.25

q

x y

xy xy

x'x y'ycos

L L

1.0 -0.50 + 0.4 1.25

1.0 1.0 0.4 0.4 -0.50 -0.50 1.25 1.25

0.00= 0.00

1.16 1.8125

Geometry of Vectors

Plotting in the column space gives us 2

1

x1.00.4

y-0.50 1.25

Qxy=900

Geometry of Vectors


2

1

Qxy=900

Geometry of Vectors

The space of all real m-tuples, with scalar multiplication and vector addition as we have defined, is called vector space.

The vector

is a linear combination of the vectors x1, x2,…, xk.

The set of all linear combinations of the vectors x1, x2,…, xk is called their linear span.

1 1 2 2 k ka a ay x x x

Geometry of Vectors

Here is the column space plot for some vectors x1 and x2:

1

2

3

1.000.400.20

1x

-0.50 0.50 1.50

2x

Geometry of Vectors

Here is the linear span for some vectors x1 and x2:

1

2

3

Geometry of Vectors

A set of vectors x1, x2,…, xk is said to be linearly dependent if there exist k numbers a1, a2,…, ak, at least one of which is nonzero, such that

Otherwise the set of vectors x1, x2,…, xk is said to be linearly independent

1 1 2 2 k ka a a 0x x x

Geometry of Vectors

Are the vectors

linearly independent?

Take a1 = 0.5 and a2 = 1.0. Then we have

The vectors x and y are dependent.

,1.0 -0.50.6 -0.3

x y

1 2a a 0.5 1.0x y

1.0 -0.5 0.00.6 -0.3 0.0

Geometry of Vectors

Geometrically x and y look like this: 2

1

1.00.6

x

-0.5-0.3

y

Geometry of Vectors

Rotating so x lies on axis 1 makes it easier to see: 2

1

Qxy=1800

Geometry of Vectors

Are the vectors

linearly independent?

There are no real values a1, a2 such that

so the vectors x and y are independent.

,x y1.0 -0.500.4 1.25

1 2 1 2a a a ax y

1.0 -0.50 0.00.4 1.25 0.0

Geometry of Vectors

Geometrically x and y look like this: 2

1

x1.00.4

y-0.50 1.25

Qxy=900

Geometry of Vectors


2

1

Qxy=900

Geometry of Vectors

Here x and y are called perpendicular (or orthogonal) – this is written x y.

Some properties of orthogonal vectors:

x’y = 0 x y

z is perpendicular to every vector iff z = 0

If z is perpendicular to each vector x1, x2, …, xk, then z is perpendicular their linear span.

Geometry of Vectors

Here vectors x1 and x2 (plotted in the column space) are orthogonal

1

2

3

1.000.400.20

1x

-0.50 0.50 1.50

2x

Geometry of Vectors

Recall that the linear span for vectors x1 and x2 is:

1

2

3

Geometry of Vectors

Vector z looks like this:

1

2

3

0.10-0.32 0.14

z

0.10-0.32 0.14

z

Geometry of Vectors

The vector z is perpendicular to the linear span for vectors x1 and x2:

1

2

3

Check each of the dot products!

0.10-0.32 0.14

z

Geometry of Vectors

Here vectors x1, x2, and z from our previous problem are orthogonal

1

2

3

1.000.400.20

1x

-0.50 0.50 1.50

2x

0.10-0.32 0.14

z

x1 and z are perpendicular

Geometry of Vectors


1

2

3

1.000.400.20

1x

-0.50 0.50 1.50

2x

0.10-0.32 0.14

z

x2 and z are perpendicular

Geometry of Vectors

Note we could rotate x1, x2, and z until they lied on our three axes!


1

2

3

1.000.400.20

1x

-0.50 0.50 1.50

2x

0.10-0.32 0.14

z

x1 and x2 are perpendicular

Geometry of Vectors

The projection (or shadow) of a vector x on a vector y is given by:

y

x'yy

2L

If y has unit length (i.e., Ly = 1), the projection (or shadow) of a vector x on a vector y simplifies to (x’y)y

Geometry of Vectors

For the vectors

the projection (or shadow) of x on y is:

,x y0.6 0.81.0 0.3

y

x'yy

2

0.80.6 1.0

0.3 0.780.8 0.8= =

0.3 0.30.8L 0.730.8 0.30.3

0.8 0.8548 = 1.0685 =

0.3 0.3205

Geometry of Vectors

Geometrically the projection of x on y looks like this:

2

1

x0.61.0

y0.80.3

projection of x on y

Perpendicular wrt y

Geometry of Vectors

Rotating so y lies on axis 1 makes it easier to see:

2

1

Geometry of Vectors

Note that we write the length of the projection of x on y like this:

x x xy

y x y

x'y x'y= L L cos

L L Lq

For our previous example, the length of the length of the projection of x on y is:

y

x'y0.8

0.6 1.00.3

= 0.912921L 0.8

0.8 0.30.3

Geometry of Vectors

The Gram-Schmidt (Orthogonalization) Process

For linearly independent vectors x1, x2,…, xk, there exist mutually perpendicular vectors u1, u2,…, uk with the same linear span. These may be constructed by setting:

1 1

'12

12 2 '1 1

''12 k k -1

1 1k k k -1' '1 1 k -1 k -1

u x

x uu x u

u u

x ux uu x u x u

u u u u

We can normalize (convert to vectors z of unit length) the vectors u by setting

j

j '

j j

uz

u u

Finally, note that we can project a vector xk onto the linear span of vectors x1, x2,…, xk-1:

k -1

'

k jj=1

x z


Here are vectors x1, x2, and z from our previous problem:

1

2

3

1.000.400.20

1x

-0.50 0.50 1.50

2x

0.10-0.32 0.14

z


Let’s construct mutually perpendicular vectors u1, u2, u3 with the same linear span – we’ll arbitrarily select the first axis as u1:

00.000.000.1

1u


Now we construct a vector u2 perpendicular with vector u1 (and in the linear span of x1, x2, z):

50.150.000.0

00.000.000.1

5.050.150.050.0

00.000.000.1

00.000.000.1

00.000.000.1

00.000.000.1

50.150.050.0

50.150.050.0

2u


Finally, we construct a vector u3 perpendicular with vectors u1 and u2 (and in the linear span of x1, x2, z):


1025.03325.0000.0

00.150.000.0

25.000.000.000.1

1.014.032.010.0

00.150.000.0

00.150.000.0

00.150.000.0

00.150.000.0

14.032.010.0

00.000.000.1

00.000.000.1

00.000.000.1

00.000.000.1

14.032.010.0

14.032.010.0

'

'

'

'2

22

21

11

13 u

uu

uzu

uu

uzzu


Here are our orthogonal vectors u1, u2, and u3:

1

2

3

1.000.000.00

1u

0.000.501.50

2u

0.0000-0.3325 0.1025

3u


If we normalized our vectors u1, u2, and u3, we get:

1.00000.00000.0000

1.00001.0000 0.0000 0.0000 0.0000

0.0000

1.0000 1.00001.00.0000 0.0000

1.0 0.0000 0.0000

11 '

1 1

uz

u u


and:

0.00000.50001.5000

0.00000.0000 0.5000 1.5000 0.5000

1.5000

0.0000 0.00001.00.5000 0.3162

2.5 1.5000 0.9487

22 '

2 2

uz

u u


and:

0.0000-0.3325 0.1025

0.00000.0000 -0.3325 0.1025 -0.3325

0.1025

0.0000 0.00001.0-0.3325 -0.9556

0.1211 0.1025 0.2946

33 '

3 3

uz

u u


The normalized vectors z1, z2, and z3 look like this:

1

2

3

1.000.000.00

1z

0.00000.31620.9487

2z

0.0000-0.9556 0.2946

3z


STK 500 Pengantar Teori Statistika VectorPengantar Teori Statistika Vector x 2 2 2 x 1 p2 22 1 2...

Documents

Transcript of STK 500 Pengantar Teori Statistika VectorPengantar Teori Statistika Vector x 2 2 2 x 1 p2 22 1 2...