Post on 15-Jun-2020
MatricesA brief introduction
Basilio Bona
DAUIN – Politecnico di Torino
Semester 1, 2016-17
B. Bona (DAUIN) Matrices Semester 1, 2016-17 1 / 41
Definitions
Definition
A matrix is a set of N real or complex numbers organized in m rows and ncolumns, with N = mn
A =
a11 a12 · · · a1na21 a22 · · · a2n· · · · · · aij · · ·am1 am2 · · · amn
≡ [aij] i = 1, . . . ,m j = 1, . . . ,n
A matrix is always written as a boldface capital letter, e.g., A.
To indicate matrix dimensions we use the following symbols
Am×n Am×n A ∈ Fm×n A ∈ Fm×n
where F = R for real elements and F = C for complex elements.
B. Bona (DAUIN) Matrices Semester 1, 2016-17 2 / 41
Transpose matrix
Given a matrix Am×n the transpose matrix is the matrix obtainedexchanging rows and columns
ATn×m =
a11 a21 · · · am1
a12 a22 · · · am2...
.... . .
...a1n a2n · · · amn
The following property holds (AT)T = A
B. Bona (DAUIN) Matrices Semester 1, 2016-17 3 / 41
Square matrix
A matrix is said to be square when m = n
A square n×n matrix is upper triangular when aij = 0, ∀i > j
An×n =
a11 a12 · · · a1n0 a22 · · · a2n...
.... . .
...0 0 · · · ann
If a square matrix is upper triangular its transpose is lower triangular andviceversa
ATn×n =
a11 0 · · · 0a12 a22 · · · 0
......
. . ....
a1n a2n · · · ann
B. Bona (DAUIN) Matrices Semester 1, 2016-17 4 / 41
Symmetric matrix
A real square matrix is said to be symmetric if A = AT, or
A−AT = O
In a real symmetric matrix there are at leastn(n+ 1)
2independent
elements.
If a matrix K has complex elements kij = aij + jbij (where j =√−1) its
conjugate is K with elements k ij = aij − jbij .
Given a complex matrix K, its adjoint matrix K∗ is the conjugate
transpose K∗ = KT
= KT
A complex matrix is called self-adjoint or hermitian when K = K∗. Sometextbooks indicate this matrix as K† or KH
B. Bona (DAUIN) Matrices Semester 1, 2016-17 5 / 41
Diagonal matrix
A square matrix is diagonal if aij = 0 for i 6= j
An×n = diag(ai ) =
a1 0 · · · 00 a2 · · · 0...
.... . .
...0 0 · · · an
A diagonal matrix is always symmetric.
B. Bona (DAUIN) Matrices Semester 1, 2016-17 6 / 41
Matrix Algebra
Matrices form an algebra, i.e., a vector space endowed with the productoperator. The main operations are: product by a scalar, sum, matrixproduct
Product by a scalar c
cA = c
a11 a12 · · · a1na21 a22 · · · a2n
......
. . ....
am1 am2 · · · amn
=
ca11 ca12 · · · ca1nca21 ca22 · · · ca2n
......
. . ....
cam1 cam2 · · · camn
Sum
A + B =
a11 +b11 a12 +b12 · · · a1n +b1na21 +b21 a22 +b22 · · · a2n +b2n
......
. . ....
am1 +bm1 am2 +bm2 · · · amn +bmn
B. Bona (DAUIN) Matrices Semester 1, 2016-17 7 / 41
Sum
Properties
A + O = A
A + B = B + A
(A + B) + C = A + (B + C)
(A + B)T = AT + BT
The neutral element O is called null or zero matrix. The matrixdifference is defined introducing the scalar α =−1:
A−B = A + (−1)B.
B. Bona (DAUIN) Matrices Semester 1, 2016-17 8 / 41
Matrix Product
Matrix product
The operation follows the rule “row by column”: the generic cij elementof the product matrix Cm×p = Am×n ·Bn×p is
cij =n
∑k=1
aikbkj
The following identity holds:
α(A ·B) = (αA) ·B = A · (αB)
B. Bona (DAUIN) Matrices Semester 1, 2016-17 9 / 41
Product
Properties
A ·B ·C = (A ·B) ·C = A · (B ·C)A · (B + C) = A ·B + A ·C(A + B) ·C = A ·C + B ·C(A ·B)T = BT ·AT
In general:
the matrix product is NOT commutative: A ·B 6= B ·A, exceptsome particular case;
A ·B = A ·C does not imply B = C, except some particular case;
A ·B = O does not imply A = O or B = O, except some particularcase.
B. Bona (DAUIN) Matrices Semester 1, 2016-17 10 / 41
Identity Matrix
The neutral element wrt the matrix product is called identity matrixusually written as In or I when there are no ambiguities on the dimension.
Identity matrix
I =
1 0 · · · 00 · · · · · · 0...
.... . .
...0 0 · · · 1
Given a rectangular matrix Am×n the following relations hold
Am×n = ImAm×n = Am×nIn
B. Bona (DAUIN) Matrices Semester 1, 2016-17 11 / 41
Matrix Power
Given a square matric A ∈ Rn×n, the k-th power is
Ak =k
∏`=1
A
One matrix is said to be idempotent iff
A2 = A→ Ak = A.
B. Bona (DAUIN) Matrices Semester 1, 2016-17 12 / 41
Matrix Trace
Trace
The trace of a square matrix An×n is the sum of its diagonal elements
tr (A) =n
∑k=1
akk
Trace satisfy the following properties
tr (aA +bB) = a tr (A) +b tr (B)tr (AB) = tr (BA)tr (A) = tr (AT)tr (A) = tr (T−1AT) for T non-singular (see below for explanation)
B. Bona (DAUIN) Matrices Semester 1, 2016-17 13 / 41
Row/column cancellation
Given the square matrix A ∈ Rn×n, we call A(ij) ∈ R(n−1)×(n−1) the matrixobtained deleting the la i-the row and the j-the columns of A.
Example: given
A =
1 −5 3 2
-6 4 9 -7
7 −4 -8 2
0 −9 -2 −3
deleting row 2, column 3 we obtain
A(23) =
1 −5 27 −4 20 −9 −3
B. Bona (DAUIN) Matrices Semester 1, 2016-17 14 / 41
Minors and Determinant
A minor of order p of a generic matrix Am×n is defined as thedeterminant Dp of a square sub-matrix obtained selecting any p rows andp columns of Am×n
There exist as many minors as the possible choices of p on m rows and pon n columns
The definition of determinant will be given soon.
Given a matrix Am×n its principal minors of order k are thedeterminants Dk , with k = 1, · · · ,min{m,n}, obtained selecting the first krows and k columns of Am×n.
B. Bona (DAUIN) Matrices Semester 1, 2016-17 15 / 41
Example
Given the 4×3 matrix
A =
1 −3 57 2 4−1 3 28 −1 6
we compute a generic minor D2, for example that obtained selecting thefirst and rows 1 and 3 and columns 1 and 2 (in red).
First we form the submatrix
D =
[1 −3−1 3
]and then we compute the determinant
D2 = det(D) = 3×1− (−3×−1) = 0
B. Bona (DAUIN) Matrices Semester 1, 2016-17 16 / 41
Example
Given the 4×3 matrix
A =
1 −3 57 2 4−1 3 28 −1 6
we compute the principal minors minors Dk ,k = 1,2,3,
D1 = 1
D2 = det
[1 −37 2
]= 23
D3 = det
1 −3 57 2 4−1 3 2
= 161
B. Bona (DAUIN) Matrices Semester 1, 2016-17 17 / 41
Complement
We call the complement Crc of a generic (r ,c) element of a squarematrix An×n the determinant of the matrix obtained deleting its r -the row
and the c-th column, i.e., detA(rc)
Drc = detA(rc).
The cofactor of the (r ,c) element of a square matrix An×n is the signedproduct
Crc = (−1)r+cDrc
B. Bona (DAUIN) Matrices Semester 1, 2016-17 18 / 41
Example
Given the 3×3 matrix
A =
1 2 34 5 67 8 9
some of the cofactors are
C11 = (−1)2(45−48) =−3
C12 = (−1)3(36−42) = 6
C31 = (−1)4(12−15) =−3
B. Bona (DAUIN) Matrices Semester 1, 2016-17 19 / 41
Adjugate/Adjunct/Adjoint
The cofactor matrix of A is the n×n matrix C whose (i , j) entry Cij isthe (i , j) cofactor of A
Cij = (−1)i+jDij
The adjugate or adjunct or adjoint of a square matrix A is the transposeof C, that is, the n×n matrix whose (i , j) entry is the (j , i) cofactor of A,
Aadjij = Cji = (−1)i+jDji
The adjoint matrix of A is the matrix X that satisfies the following equality
AX = XA = det(A)I
B. Bona (DAUIN) Matrices Semester 1, 2016-17 20 / 41
Example
Given the 3×3 matrix
A =
1 3 24 6 57 9 8
its adjoint is
Aadj =
3 −6 33 −6 3−6 12 −6
B. Bona (DAUIN) Matrices Semester 1, 2016-17 21 / 41
Determinant
The determinant of a square matrix Ax×n can be computed in differentways.
Choosing any row i , the definition “by row” is:
det(A) =n
∑k=1
aik(−1)i+k det(A(ik)) =n
∑k=1
aikAik
Choosing any column j , the definition “by column” is::
det(A) =n
∑k=1
akj(−1)k+j det(A(kj)) =n
∑k=1
akjAkj
Since these definitions are recursive, involving the determinants ofincreasingly smaller minors, we define the determinant of a 1×1 matrixA = a, simply as detA = a.
B. Bona (DAUIN) Matrices Semester 1, 2016-17 22 / 41
Properties
The determinant has the following properties:
det(A ·B) = det(A)det(B)
det(AT) = det(A)
det(kA) = kn det(A)
if one exchanges s rows or columns of A, obtaining As , thendet(As) = (−1)s det(A)
if A has two or more rows/columns equal or proportional, thendet(A) = 0
if A has a row/column that can be obtained as a linear combinationof other rows/columns, then det(A) = 0
if A is triangular, then det(A) = ∏ni=1 aii
if A is block-triangular, with p blocks Aii on the diagonal, thendet(A) = ∏
pi=1 detAii
B. Bona (DAUIN) Matrices Semester 1, 2016-17 23 / 41
Rank and Singularity
A matrix A is singular if det(A) = 0.
The rank (or characteristic) of a matrix Am×n is the number ρ(Am×n),defined as the largest integer p for which at least a minor Dp is non-zero.
The following properties hold:
ρ(A)≤min{m,n}if ρ(A) = min{m,n}, A is said to be full rank
if ρ(A) < min{m,n}, the rank of the matrix is said to drop
if An×n and detA < n the matrix is not full rank, or is rank deficient
ρ(A ·B)≤min{ρ(A),ρ(B)}ρ(A) = ρ(AT)
ρ(A ·AT) = ρ(AT ·A) = ρ(A)
B. Bona (DAUIN) Matrices Semester 1, 2016-17 24 / 41
Invertible Matrix
A square matrix An×n it is said to be invertible or non singular if theinverse A−1n×n exists, such that
An×nA−1n×n = A−1n×nAn×n = In
A square matrix A is invertible iff ρ(A) = n, i.e., it is full-rank; this isequivalent to have a non zero determinant det(A) 6= 0.
The inverse is computed as:
A−1 =1
det(A)Aadj
The following properties hold: (A−1)−1 = A; (AT)−1 = (A−1)T.
The inverse, if exists, allows to solve the following matrix equation
y = Ax
with respect to the unknown x, as
x = A−1y.
B. Bona (DAUIN) Matrices Semester 1, 2016-17 25 / 41
Matrix derivative
If a square matrix An×n(t) has elements function of a variable (e.g., thetime t) aij(t), then the matrix derivative is
ddt
A(t) = A(t) =
[ddt
aij(t)
]= [aij(t)]
If A(t) rank is full, ρ(A(t)) = n for every t, then the derivative of theinverse is
ddt
A(t)−1 =−A−1(t)A(t)A(t)−1
Notice that, since the inverse is a nonlinear function, the derivative ofthe inverse is in general different from the inverse of the derivative.[
dA(t)
dt
]−16= d
dt[A(t)−1
]
B. Bona (DAUIN) Matrices Semester 1, 2016-17 26 / 41
Example
Given the square matrix
A(t) =
[cosθ(t) −sinθ(t)sinθ(t) cosθ(t)
]we have
ddt
A(t) = A(t) =
[−sinθ(t) −cosθ(t)cosθ(t) −sinθ(t)
]θ(t)
The inverse of A is
A(t)−1 =
[cosθ(t) sinθ(t)−sinθ(t) cosθ(t)
]= A(t)T
and in this particular case the two inverses are equal[dA(t)
dt
]−1=
ddt[A(t)−1
]=
[cosθ(t) sinθ(t)−sinθ(t) cosθ(t)
]B. Bona (DAUIN) Matrices Semester 1, 2016-17 27 / 41
Matrix Decomposition
Given a real matrix A ∈ Rm×n, the following products give symmetricmatrices
ATA ∈ Rn×n
AAT ∈ Rm×m
Given a square matrix A, it is always possible to decompose it in a sum oftwo matrices
A = As + Ass
where
As =1
2(A + AT)
is symmetric, and
Ass =1
2(A−AT)
is skew-symmetric.
B. Bona (DAUIN) Matrices Semester 1, 2016-17 28 / 41
Similarity Transformation
Similarity transformation
Given a square matrix A ∈ Rn×n and a square nonsingular matrixT ∈ Rn×n, the matrix B ∈ Rn×n obtained as
B = T−1AT or B = TAT−1
is called similar to A, and the transformation T is called similaritytransformation.
B. Bona (DAUIN) Matrices Semester 1, 2016-17 29 / 41
Eigenvalues and Eigenvectors
If it is possible to find a nonsingular matrix U such that A is similar to thediagonal matrix Λ = diag(λi )
A = UΛU−1 → AU = UΛ
and if we call ui the i-th column of U,
U =[u1 u2 · · · un
]we have
Aui = λiui
This relation is the well known formula defining eigenvectors andeigenvalues of A.
The λi are the eigenvalues of A and the ui are the eigenvectors of A.
B. Bona (DAUIN) Matrices Semester 1, 2016-17 30 / 41
Eigenvalues and Eigenvectors
Eigenvalues and eigenvectors
Given a square matrix An×n, the matrix eigenvalues λi are the (real orcomplex) solutions of the characteristic equation
det(λ I−A) = 0
det(λ I−A) is a polynomial in λ , called the characteristic polynomial ofA.
If the eigenvalues are all distinct, we call eigenvectors the vectors ui thatsatisfy the following identity
Aui = λiui
If the eigenvalues are not all distinct, we obtain the generalizedeigenvectors, whose characterization goes beyond the scope of thispresentation.
B. Bona (DAUIN) Matrices Semester 1, 2016-17 31 / 41
Geometrical interpretation
From a geometrical point of view, the eigenvectors represent thoseparticular “directions” in the Rn space (i.e., the domain of the lineartransformation represented by A), that remain invariant under thetransformation, while the eigenvalues give the scaling constants alongthese same directions.
The set of the matrix eigenvalues is usually indicated as Λ(A) or {λi (A)};the set of the matrix eigenvectors is indicated as {ui (A)}. In general, theyare normalized, i.e., ‖{ui (A)}‖= 1
B. Bona (DAUIN) Matrices Semester 1, 2016-17 32 / 41
Eigenvalues Properties
Given a square matrix A and its eigenvalues, {λi (A)}, the followingproperties hold true
{λi (A + cI)}= {(λi (A) + c)}
{λi (cA)}= {(cλi (A)}
Given a triangular matrixa11 a12 · · · a1n0 a22 · · · a2n...
.... . .
...0 0 · · · ann
,a11 0 · · · 0a21 a22 · · · 0
......
. . ....
an1 an2 · · · ann
its eigenvalues are the elements on the main diagonal {λi (A)}= {aii}; thesame is true for a diagonal matrix.
B. Bona (DAUIN) Matrices Semester 1, 2016-17 33 / 41
Other properties
Given a square matrix An×n and its eigenvalues {λi (A)}, the followinghold true
det(A) =n
∏i=1
λi
and
tr (A) =n
∑i=1
λi
So, the determinant is the product of the eigenvalues, and the trace is thesum of the eigenvalues.
Given any invertible similarity transformation T,
B = T−1AT
the eigenvalues of A are invariant to it, i.e.,
{λi (B)}= {λi (A)}
B. Bona (DAUIN) Matrices Semester 1, 2016-17 34 / 41
Modal matrix
If we build a matrix M whose columns are the normalized eigenvectorsui (A)
M =[u1 · · · un
]then the similarity transformation with respect to M results in the diagonalmatrix
Λ =
λ1 0 · · · 00 λ2 · · · 0...
.... . .
...0 0 · · · λn
= M−1AM
M is the modal matrix.
If A is symmetric, all its eigenvalues are real and we have
Λ = MTAM
In this case M is orthonormal.
B. Bona (DAUIN) Matrices Semester 1, 2016-17 35 / 41
Singular Value decomposition (SVD)
Given a matrix A ∈ Rm×n, having rank r = ρ(A)≤ s, with s = min{m,n},it can be decomposed (factored) in the following way:
A = UΣVT =s
∑i=1
σiuivTi (1)
The decomposition is characterized by three elements:
σi
ui
vi
as follows.
B. Bona (DAUIN) Matrices Semester 1, 2016-17 36 / 41
SVD Characterization
σi (A)≥ 0 are the singular values and are equal to the non-negativesquare roots of the eigenvalues of the symmetric matrix ATA:
{σi (A)}= {√
λi (ATA)} σi ≥ 0
ordered in decreasing order
σ1 ≥ σ2 ≥ ·· · ≥ σs ≥ 0
if r < s there are r positive singular values; the remaining ones arezero
σ1 ≥ σ2 ≥ ·· · ≥ σr > 0; σr+1 = · · ·= σs = 0
U is a orthonormal (m×m) matrix
U =[u1 u2 · · · um
]containing the eigenvectors ui of AAT
B. Bona (DAUIN) Matrices Semester 1, 2016-17 37 / 41
SVD Characterization
V is a (n×n) orthonormal matrix
V =[v1 v2 · · · vn
]whose columns are the eigenvectors vi of the matrix ATA
Σ is a (m×n) matrix, with the following structure
if m < n Σ =[Σs O
]if m = n Σ = Σs
if m > n Σ =
[Σs
O
].
where Σs = diag(σi ) is diagonal with dimension s× s, having thesingular values on the diagonal.
B. Bona (DAUIN) Matrices Semester 1, 2016-17 38 / 41
Example
Given
A =
[1 3 24 6 5
]ρ(A) = 2
its SVD isA = UΣVT
where
U =
[−0.3863 −0.9224−0.9224 0.3863
]Σ =
[9.5080 0 0
0 0.7729 0
]
V =
−0.4287 0.8060 −0.4082−0.7039 −0.5812 −0.4082−0.5663 0.1124 0.8165
B. Bona (DAUIN) Matrices Semester 1, 2016-17 39 / 41
Alternative SVD Decomposition
Alternately, we can decompose the A matrix as follows:
A =[P P
]︸ ︷︷ ︸U
[Σr OO O
]︸ ︷︷ ︸
Σ
[QT
QT
]︸ ︷︷ ︸
VT
= PΣrQT (2)
where
P is an orthonormal m× r matrix,
P is an orthonormal m× (m− r) matrix;
Q is an orthonormal n× r matrix, QT
is an orthonormal n× (n− r)matrix;
Σr is an diagonal r × r matrix, whose diagonal elements are thepositive singular values σi > 0, i = 1, · · · , r .
B. Bona (DAUIN) Matrices Semester 1, 2016-17 40 / 41
Rank
The rank r = ρ(A) of A is equal to the number r ≤ s of nonzero singularvalues.
Given any matrix A ∈ Rm×n, both ATA and AAT are symmetric, withidentical positive singular values and differ only for the number of zerosingular values.
B. Bona (DAUIN) Matrices Semester 1, 2016-17 41 / 41