Beyond Vectors - speech.ee.ntu.edu.tw
Transcript of Beyond Vectors - speech.ee.ntu.edu.tw
Introduction
โข Many things can be considered as โvectorsโ.
โข E.g. a function can be regarded as a vector
โข We can apply the concept we learned on those โvectorsโ.
โข Linear combination
โข Span
โข Basis
โข Orthogonal โฆโฆ
โข Reference: Chapter 6
Are they vectors?
โข A matrix
โข A linear transform
โข A polynomial
๐ด =1 23 4
๐ ๐ฅ = ๐0 + ๐1๐ฅ +โฏ+ ๐๐๐ฅ๐
1234
๐0๐1โฎ๐๐
Are they vectors?
โข Any function is a vector?
๐ ๐ก = ๐๐ก ๐ฃ =โฎ?
โฎ
๐ ๐ก = ๐ก2 โ 1
๐ฃ + ๐
๐ =โฎ?
โฎ
h ๐ก = ๐๐ก + ๐ก2 โ 1
What is the zero vector?
What is a vector?
โข If a set of objects V is a vector space, then the objects are โvectorsโ.
โข Vector space:
โข There are operations called โadditionโ and โscalar multiplicationโ.
โข u, v and w are in V, and a and b are scalars. u+v and au are unique elements of V
โข The following axioms hold:
โข u + v = v + u, (u +v) + w = u +(v + w)
โข There is a โzero vectorโ 0 in V such that u + 0 = u
โข There is โu in V such that u +(-u) = 0
โข 1u = u, (ab)u = a(bu), a(u+v) = au +av, (a+b)u = au +bu
unique
Rn is a vector space
Objects in Different Vector Spaces
๐ ๐ก = 1 ๐ ๐ก = ๐ก + 1 โ ๐ก = ๐ก2 + ๐ก + 1
All the polynomials with degree less than or equal to 2 as a vector space
All functions as a vector space
100
110
111
Vectors with infinite dimensions
Review: Subspace
โข A vector set V is called a subspace if it has the following three properties:
โข 1. The zero vector 0 belongs to V
โข 2. If u and w belong to V, then u+w belongs to V
โข 3. If u belongs to V, and c is a scalar, then cubelongs to V
Closed under (vector) addition
Closed under scalar multiplication
Are they subspaces?
โข All the functions pass 0 at t0
โข All the matrices whose trace equal to zero
โข All the matrices of the form
โข All the continuous functions
โข All the polynomials with degree n
โข All the polynomials with degree less than or equal to n
๐ ๐ + ๐๐ 0
P: all polynomials, Pn: all polynomials with degree less than or equal to n
Linear Combination and Span
โข Matrices
๐ =1 00 โ1
,0 10 0
,0 01 0
Linear combination with coefficient a, b, c
๐1 00 โ1
+ ๐0 10 0
+ ๐0 01 0
=๐ ๐๐ โ๐
What is Span S?
All 2x2 matrices whose trace equal to zero
Linear Combination and Span
โข Polynomials
๐ = 1, ๐ฅ, ๐ฅ2, ๐ฅ3
Is ๐ ๐ฅ = 2 + 3๐ฅ โ ๐ฅ2 linear combination of the โvectorsโ in S?
๐ ๐ฅ = 2 โ 1 + 3 โ ๐ฅ + โ1 โ ๐ฅ2
๐๐๐๐ 1, ๐ฅ, ๐ฅ2, ๐ฅ3
๐๐๐๐ 1, ๐ฅ,โฏ , ๐ฅ๐, โฏ
= ๐3
= ๐
Linear transformation
โข A mapping (function) T is called linear if for all โvectorsโ u, v and scalars c:
โข Preserving vector addition:
โข Preserving vector multiplication:
๐ ๐ข + ๐ฃ = ๐ ๐ข + ๐ ๐ฃ
๐ ๐๐ข = ๐๐ ๐ข
Is matrix transpose linear?
Input: m x n matrices, output: n x m matrices
Linear transformation
โข Derivative:
โข Integral from a to b
Derivativefunction f function fโe.g. x2 e.g. 2x
Integralfunction f
scalar
e.g. x2
e.g. 1
3๐3 โ ๐3
เถฑ๐
๐
๐ ๐ก ๐๐ก
(from a to b)
linear?
linear?
Null Space and Range
โข Null Space
โข The null space of T is the set of all vectors such that T(v)=0
โข What is the null space of matrix transpose?
โข Range
โข The range of T is the set of all images of T.
โข That is, the set of all vectors T(v) for all v in the domain
โข What is the range of matrix transpose?
One-to-one and Onto
โข U: MmnโMnm defined by U(A) = AT. โข Is U one-to-one?
โข Is U onto?
โข D: P3 โ P3 defined by D( f ) = f โข Is D one-to-one?
โข Is D onto?
yes
yes
no
no
Isomorphism
โข Let V and W be vector space.
โข A linear transformation T: VโW is called an isomorphism if it is one-to-one and onto
โข Invertible linear transform
โข W and V are isomorphic.
W V
Example 1: U: MmnโMnm defined by U(A) = AT.
Example 2: T: P2โ R3
๐ ๐ + ๐๐ฅ +๐
2๐ฅ2 =
๐๐๐
Independent
โข Example
โข Example
S = {x2 - 3x + 2, 3x2 โ 5x, 2x โ 3} is a subset of P2.
Is it linearly independent?
No
implies that a = b = c = 0
is a subset of 2x2 matrices.
Yes
Is it linearly independent?
Independent
โข Example
โข Example
icixi = 0 implies ci = 0 for all i.
The infinite vector set {1, x, x2, , xn, }
Is it linearly independent?
Yes
S = {et, e2t, e3t} Is it linearly independent?
aet + be2t + ce3t = 0 a + b + c = 0
aet + 2be2t + 3ce3t = 0
aet + 4be2t + 9ce3t = 0
a + 2b + 3c = 0
a + 4b + 9c = 0
Yes
If {v1, v2, โฆโฆ, vk} are L.I., and T is an isomorphism, {T(v1), T(v2), โฆโฆ, T(vk)} are L.I.
Basis
โข Example
โข Example
For the subspace of all 2 x 2 matrices,
S = {1, x, x2, , xn, } is a basis of P.
The basis is
Dim = 4
Dim = inf
Vector Representation of Object
โข Coordinate Transformation
basis
Pn: Basis: {1, x, x2, , xn}
๐ ๐ฅ = ๐0 + ๐1๐ฅ +โฏ+ ๐๐๐ฅ๐
๐0๐1โฎ๐๐
Matrix Representation of Linear Operatorโข Example:
โข D (derivative): P2 โ P2 Represent it as a matrix
polynomial polynomial
vector vector
Derivative
Multiply a matrix
2 โ 3๐ฅ + 5๐ฅ2 โ3 + 10๐ฅ
2โ35
โ3100
Matrix Representation of Linear Operatorโข Example:
โข D (derivative): P2 โ P2 Represent it as a matrix
polynomial polynomial
vector vector
Multiply a matrix
Derivative
100
010
001
1
๐ฅ
๐ฅ2
0
1
2๐ฅ
000
100
020
0 1 00 0 20 0 0
Matrix Representation of Linear Operatorโข Example:
โข D (derivative): P2 โ P2 Represent it as a matrix
polynomial polynomial
vector vector
Multiply a matrix
Derivative
0 1 00 0 20 0 0
5 โ 4๐ฅ + 3๐ฅ2 โ4 + 6๐ฅ
5โ43
โ460
0 1 00 0 20 0 0
5โ43
Not invertible
Matrix Representation of Linear Operatorโข Example:
โข D (derivative): Function set F โ Function set F
โข Basis of F is ๐๐ก cos ๐ก , ๐๐ก sin ๐ก
Function in F Function in F
vector vector
Multiply a matrix
Derivative
1 1โ1 1
10
01
1โ1
11
invertible
Matrix Representation of Linear Operator
Function in F Function in F
vector vector
Multiply a matrix
Derivative
1 1โ1 1
โ1/21/2
01
Basis of F is ๐๐ก cos ๐ก , ๐๐ก sin ๐ก
1/2 โ1/21/2 1/2
Antiderivative
Eigenvalue and Eigenvector
โข Consider derivative (linear transformation, input & output are functions)
โข Consider Transpose (also linear transformation, input & output are functions)
Is ๐ ๐ก = ๐๐๐ก an โeigenvectorโ? What is the โeigenvalueโ?
Every scalar is an eigenvalue of derivative.
Symmetric:
Skew-symmetric:
๐ด๐ = ๐ด
๐ด๐ = โ๐ด
Is ๐ = 1 an eigenvalue?
Symmetric matrices form the eigenspace
Skew-symmetric matrices form the eigenspace.
Is ๐ = โ1 an eigenvalue?
2x2 matrices 2x2 matrices
vector
transpose
Consider Transpose of 2x2 matrices
1000
1 00 0
1 00 0
1000
What are the eigenvalues?
0100
0010
0001
0010
0100
0001
vector
Eigenvalue and Eigenvector
โข Consider Transpose of 2x2 matrices
๐ ๐๐ ๐
0 ๐โ๐ 0
Matrix representation
of transpose
Characteristic polynomial
๐ก โ 1 3 ๐ก + 1
๐ = 1 ๐ = โ1
Symmetric matrices Skew-symmetric matrices
Dim=3 Dim=1
Inner Product
Inner Productโvectorโ v
โvectorโ uscalar
๐ข, ๐ฃ
For any vectors u, v and w, and any scalar a, the following axioms hold:
1. ๐ข, ๐ข > 0 if ๐ข โ 0
2. ๐ข, ๐ฃ = ๐ฃ, ๐ข 4. ๐๐ข, ๐ฃ = ๐ ๐ข, ๐ฃ
3. ๐ข + ๐ฃ,๐ค = ๐ข,๐ค + ๐ฃ,๐ค
Dot product is a special case of inner product
Can you define other inner product for normal vectors?
Norm (length):
Orthogonal: Inner product is zero
๐ฃ = ๐ฃ, ๐ฃ
Inner Product
โข Inner Product of Matrix
Frobeniusinner product
๐ด, ๐ต = ๐ก๐๐๐๐ ๐ด๐ต๐
= ๐ก๐๐๐๐ ๐ต๐ด๐
Element-wise multiplication
๐ด =1 23 4
๐ด = 12 + 22 + 32 + 42
Inner Product
โข Inner product for general functions
Is ๐ ๐ฅ = 1 and โ ๐ฅ = ๐ฅ orthogonal?
Can it be inner product for general functions?
1. ๐ข, ๐ข > 0 if ๐ข โ 0
2. ๐ข, ๐ฃ = ๐ฃ, ๐ข
4. ๐๐ข, ๐ฃ = ๐ ๐ข, ๐ฃ
3. ๐ข + ๐ฃ,๐ค = ๐ข,๐ค + ๐ฃ,๐ค
Orthogonal/Orthonormal Basis
โข Let u be any vector, and w is the orthogonal projection of u on subspace W.
โข Let ๐ = ๐ฃ1, ๐ฃ2, โฏ , ๐ฃ๐ be an orthogonal basis of W.
โข Let ๐ = ๐ฃ1, ๐ฃ2, โฏ , ๐ฃ๐ be an orthonormal basis of W.
๐ค = ๐1๐ฃ1 + ๐2๐ฃ2 +โฏ+ ๐๐๐ฃ๐
๐ข โ ๐ฃ1๐ฃ1
2
๐ข โ ๐ฃ2๐ฃ2
2
๐ข โ ๐ฃ๐๐ฃ๐
2
๐ค = ๐1๐ฃ1 + ๐2๐ฃ2 +โฏ+ ๐๐๐ฃ๐
๐ข โ ๐ฃ1 ๐ข โ ๐ฃ2 ๐ข โ ๐ฃ๐
Orthogonal Basis
Let ๐ข1, ๐ข2, โฏ , ๐ข๐ be a basis of a subspace V. How to transform ๐ข1, ๐ข2, โฏ , ๐ข๐ into an orthogonal basis ๐ฃ1, ๐ฃ2, โฏ , ๐ฃ๐ ?
Then ๐ฃ1, ๐ฃ2, โฏ , ๐ฃ๐ is an orthogonal basis for W
After normalization, you can get orthonormal basis.
Gram-Schmidt Process
Orthogonal/Orthonormal Basis
โข Find orthogonal/orthonormal basis for P2
โข Define an inner product of P2 by
โข Find a basis {1, x, x2}๐ข1, ๐ข2, ๐ข3
๐ฃ1, ๐ฃ2, ๐ฃ3
Orthogonal/Orthonormal Basis
โข Find orthogonal/orthonormal basis for P2
โข Define an inner product of P2 by
โข Get an orthogonal basis {1, x, x2-1/3}
Orthonormal Basis