Matrices A brief introduction - polito.it€¦ · Matrices A brief introduction Basilio Bona DAUIN...

MatricesA brief introduction

Basilio Bona

DAUIN – Politecnico di Torino

Semester 1, 2016-17

B. Bona (DAUIN) Matrices Semester 1, 2016-17 1 / 41

Definitions

Definition

A matrix is a set of N real or complex numbers organized in m rows and ncolumns, with N = mn

A =

a11 a12 · · · a1na21 a22 · · · a2n· · · · · · aij · · ·am1 am2 · · · amn

≡ [aij] i = 1, . . . ,m j = 1, . . . ,n

A matrix is always written as a boldface capital letter, e.g., A.

To indicate matrix dimensions we use the following symbols

Am×n Am×n A ∈ Fm×n A ∈ Fm×n

where F = R for real elements and F = C for complex elements.


Transpose matrix

Given a matrix Am×n the transpose matrix is the matrix obtainedexchanging rows and columns

ATn×m =

a11 a21 · · · am1

a12 a22 · · · am2...

.... . .

...a1n a2n · · · amn

The following property holds (AT)T = A


Square matrix

A matrix is said to be square when m = n

A square n×n matrix is upper triangular when aij = 0, ∀i > j

An×n =

a11 a12 · · · a1n0 a22 · · · a2n...

.... . .

...0 0 · · · ann

If a square matrix is upper triangular its transpose is lower triangular andviceversa

ATn×n =

a11 0 · · · 0a12 a22 · · · 0

......

. . ....

a1n a2n · · · ann


Symmetric matrix

A real square matrix is said to be symmetric if A = AT, or

A−AT = O

In a real symmetric matrix there are at leastn(n+ 1)

2independent

elements.

If a matrix K has complex elements kij = aij + jbij (where j =√−1) its

conjugate is K with elements k ij = aij − jbij .

Given a complex matrix K, its adjoint matrix K∗ is the conjugate

transpose K∗ = KT

= KT

A complex matrix is called self-adjoint or hermitian when K = K∗. Sometextbooks indicate this matrix as K† or KH


Diagonal matrix

A square matrix is diagonal if aij = 0 for i 6= j

An×n = diag(ai ) =

a1 0 · · · 00 a2 · · · 0...

.... . .

...0 0 · · · an

A diagonal matrix is always symmetric.


Matrix Algebra

Matrices form an algebra, i.e., a vector space endowed with the productoperator. The main operations are: product by a scalar, sum, matrixproduct

Product by a scalar c

cA = c

a11 a12 · · · a1na21 a22 · · · a2n

......

. . ....

am1 am2 · · · amn

=

ca11 ca12 · · · ca1nca21 ca22 · · · ca2n

......

. . ....

cam1 cam2 · · · camn

Sum

A + B =

a11 +b11 a12 +b12 · · · a1n +b1na21 +b21 a22 +b22 · · · a2n +b2n

......

. . ....

am1 +bm1 am2 +bm2 · · · amn +bmn


Sum

Properties

A + O = A

A + B = B + A

(A + B) + C = A + (B + C)

(A + B)T = AT + BT

The neutral element O is called null or zero matrix. The matrixdifference is defined introducing the scalar α =−1:

A−B = A + (−1)B.


Matrix Product

Matrix product

The operation follows the rule “row by column”: the generic cij elementof the product matrix Cm×p = Am×n ·Bn×p is

cij =n

∑k=1

aikbkj

The following identity holds:

α(A ·B) = (αA) ·B = A · (αB)


Product

Properties

A ·B ·C = (A ·B) ·C = A · (B ·C)A · (B + C) = A ·B + A ·C(A + B) ·C = A ·C + B ·C(A ·B)T = BT ·AT

In general:

the matrix product is NOT commutative: A ·B 6= B ·A, exceptsome particular case;

A ·B = A ·C does not imply B = C, except some particular case;

A ·B = O does not imply A = O or B = O, except some particularcase.


Identity Matrix

The neutral element wrt the matrix product is called identity matrixusually written as In or I when there are no ambiguities on the dimension.

Identity matrix

I =

1 0 · · · 00 · · · · · · 0...

.... . .

...0 0 · · · 1

Given a rectangular matrix Am×n the following relations hold

Am×n = ImAm×n = Am×nIn


Matrix Power

Given a square matric A ∈ Rn×n, the k-th power is

Ak =k

∏`=1

A

One matrix is said to be idempotent iff

A2 = A→ Ak = A.


Matrix Trace

Trace

The trace of a square matrix An×n is the sum of its diagonal elements

tr (A) =n

∑k=1

akk

Trace satisfy the following properties

tr (aA +bB) = a tr (A) +b tr (B)tr (AB) = tr (BA)tr (A) = tr (AT)tr (A) = tr (T−1AT) for T non-singular (see below for explanation)


Row/column cancellation

Given the square matrix A ∈ Rn×n, we call A(ij) ∈ R(n−1)×(n−1) the matrixobtained deleting the la i-the row and the j-the columns of A.

Example: given

A =

1 −5 3 2

-6 4 9 -7

7 −4 -8 2

0 −9 -2 −3

deleting row 2, column 3 we obtain

A(23) =

1 −5 27 −4 20 −9 −3


Minors and Determinant

A minor of order p of a generic matrix Am×n is defined as thedeterminant Dp of a square sub-matrix obtained selecting any p rows andp columns of Am×n

There exist as many minors as the possible choices of p on m rows and pon n columns

The definition of determinant will be given soon.

Given a matrix Am×n its principal minors of order k are thedeterminants Dk , with k = 1, · · · ,min{m,n}, obtained selecting the first krows and k columns of Am×n.


Example

Given the 4×3 matrix

A =

1 −3 57 2 4−1 3 28 −1 6

we compute a generic minor D2, for example that obtained selecting thefirst and rows 1 and 3 and columns 1 and 2 (in red).

First we form the submatrix

D =

[1 −3−1 3

]and then we compute the determinant

D2 = det(D) = 3×1− (−3×−1) = 0


Example


A =

1 −3 57 2 4−1 3 28 −1 6

we compute the principal minors minors Dk ,k = 1,2,3,

D1 = 1

D2 = det

[1 −37 2

]= 23

D3 = det

1 −3 57 2 4−1 3 2

= 161


Complement

We call the complement Crc of a generic (r ,c) element of a squarematrix An×n the determinant of the matrix obtained deleting its r -the row

and the c-th column, i.e., detA(rc)

Drc = detA(rc).

The cofactor of the (r ,c) element of a square matrix An×n is the signedproduct

Crc = (−1)r+cDrc


Example


A =

1 2 34 5 67 8 9

some of the cofactors are

C11 = (−1)2(45−48) =−3

C12 = (−1)3(36−42) = 6

C31 = (−1)4(12−15) =−3


Adjugate/Adjunct/Adjoint

The cofactor matrix of A is the n×n matrix C whose (i , j) entry Cij isthe (i , j) cofactor of A

Cij = (−1)i+jDij

The adjugate or adjunct or adjoint of a square matrix A is the transposeof C, that is, the n×n matrix whose (i , j) entry is the (j , i) cofactor of A,

Aadjij = Cji = (−1)i+jDji

The adjoint matrix of A is the matrix X that satisfies the following equality

AX = XA = det(A)I


Example


A =

1 3 24 6 57 9 8

its adjoint is

Aadj =

3 −6 33 −6 3−6 12 −6


Determinant

The determinant of a square matrix Ax×n can be computed in differentways.

Choosing any row i , the definition “by row” is:

det(A) =n

∑k=1

aik(−1)i+k det(A(ik)) =n

∑k=1

aikAik

Choosing any column j , the definition “by column” is::

det(A) =n

∑k=1

akj(−1)k+j det(A(kj)) =n

∑k=1

akjAkj

Since these definitions are recursive, involving the determinants ofincreasingly smaller minors, we define the determinant of a 1×1 matrixA = a, simply as detA = a.


Properties

The determinant has the following properties:

det(A ·B) = det(A)det(B)

det(AT) = det(A)

det(kA) = kn det(A)

if one exchanges s rows or columns of A, obtaining As , thendet(As) = (−1)s det(A)

if A has two or more rows/columns equal or proportional, thendet(A) = 0

if A has a row/column that can be obtained as a linear combinationof other rows/columns, then det(A) = 0

if A is triangular, then det(A) = ∏ni=1 aii

if A is block-triangular, with p blocks Aii on the diagonal, thendet(A) = ∏

pi=1 detAii


Rank and Singularity

A matrix A is singular if det(A) = 0.

The rank (or characteristic) of a matrix Am×n is the number ρ(Am×n),defined as the largest integer p for which at least a minor Dp is non-zero.

The following properties hold:

ρ(A)≤min{m,n}if ρ(A) = min{m,n}, A is said to be full rank

if ρ(A) < min{m,n}, the rank of the matrix is said to drop

if An×n and detA < n the matrix is not full rank, or is rank deficient

ρ(A ·B)≤min{ρ(A),ρ(B)}ρ(A) = ρ(AT)

ρ(A ·AT) = ρ(AT ·A) = ρ(A)


Invertible Matrix

A square matrix An×n it is said to be invertible or non singular if theinverse A−1n×n exists, such that

An×nA−1n×n = A−1n×nAn×n = In

A square matrix A is invertible iff ρ(A) = n, i.e., it is full-rank; this isequivalent to have a non zero determinant det(A) 6= 0.

The inverse is computed as:

A−1 =1

det(A)Aadj

The following properties hold: (A−1)−1 = A; (AT)−1 = (A−1)T.

The inverse, if exists, allows to solve the following matrix equation

y = Ax

with respect to the unknown x, as

x = A−1y.


Matrix derivative

If a square matrix An×n(t) has elements function of a variable (e.g., thetime t) aij(t), then the matrix derivative is

ddt

A(t) = A(t) =

[ddt

aij(t)

]= [aij(t)]

If A(t) rank is full, ρ(A(t)) = n for every t, then the derivative of theinverse is

ddt

A(t)−1 =−A−1(t)A(t)A(t)−1

Notice that, since the inverse is a nonlinear function, the derivative ofthe inverse is in general different from the inverse of the derivative.[

dA(t)

dt

]−16= d

dt[A(t)−1

]


Example

Given the square matrix

A(t) =

[cosθ(t) −sinθ(t)sinθ(t) cosθ(t)

]we have

ddt

A(t) = A(t) =

[−sinθ(t) −cosθ(t)cosθ(t) −sinθ(t)

]θ(t)

The inverse of A is

A(t)−1 =

[cosθ(t) sinθ(t)−sinθ(t) cosθ(t)

]= A(t)T

and in this particular case the two inverses are equal[dA(t)

dt

]−1=

ddt[A(t)−1

]=

[cosθ(t) sinθ(t)−sinθ(t) cosθ(t)

]B. Bona (DAUIN) Matrices Semester 1, 2016-17 27 / 41

Matrix Decomposition

Given a real matrix A ∈ Rm×n, the following products give symmetricmatrices

ATA ∈ Rn×n

AAT ∈ Rm×m

Given a square matrix A, it is always possible to decompose it in a sum oftwo matrices

A = As + Ass

where

As =1

2(A + AT)

is symmetric, and

Ass =1

2(A−AT)

is skew-symmetric.


Similarity Transformation

Similarity transformation

Given a square matrix A ∈ Rn×n and a square nonsingular matrixT ∈ Rn×n, the matrix B ∈ Rn×n obtained as

B = T−1AT or B = TAT−1

is called similar to A, and the transformation T is called similaritytransformation.


Eigenvalues and Eigenvectors

If it is possible to find a nonsingular matrix U such that A is similar to thediagonal matrix Λ = diag(λi )

A = UΛU−1 → AU = UΛ

and if we call ui the i-th column of U,

U =[u1 u2 · · · un

]we have

Aui = λiui

This relation is the well known formula defining eigenvectors andeigenvalues of A.

The λi are the eigenvalues of A and the ui are the eigenvectors of A.


Eigenvalues and Eigenvectors

Eigenvalues and eigenvectors

Given a square matrix An×n, the matrix eigenvalues λi are the (real orcomplex) solutions of the characteristic equation

det(λ I−A) = 0

det(λ I−A) is a polynomial in λ , called the characteristic polynomial ofA.

If the eigenvalues are all distinct, we call eigenvectors the vectors ui thatsatisfy the following identity

Aui = λiui

If the eigenvalues are not all distinct, we obtain the generalizedeigenvectors, whose characterization goes beyond the scope of thispresentation.


Geometrical interpretation

From a geometrical point of view, the eigenvectors represent thoseparticular “directions” in the Rn space (i.e., the domain of the lineartransformation represented by A), that remain invariant under thetransformation, while the eigenvalues give the scaling constants alongthese same directions.

The set of the matrix eigenvalues is usually indicated as Λ(A) or {λi (A)};the set of the matrix eigenvectors is indicated as {ui (A)}. In general, theyare normalized, i.e., ‖{ui (A)}‖= 1


Eigenvalues Properties

Given a square matrix A and its eigenvalues, {λi (A)}, the followingproperties hold true

{λi (A + cI)}= {(λi (A) + c)}

{λi (cA)}= {(cλi (A)}

Given a triangular matrixa11 a12 · · · a1n0 a22 · · · a2n...

.... . .

...0 0 · · · ann

,a11 0 · · · 0a21 a22 · · · 0

......

. . ....

an1 an2 · · · ann

its eigenvalues are the elements on the main diagonal {λi (A)}= {aii}; thesame is true for a diagonal matrix.


Other properties

Given a square matrix An×n and its eigenvalues {λi (A)}, the followinghold true

det(A) =n

∏i=1

λi

and

tr (A) =n

∑i=1

λi

So, the determinant is the product of the eigenvalues, and the trace is thesum of the eigenvalues.

Given any invertible similarity transformation T,

B = T−1AT

the eigenvalues of A are invariant to it, i.e.,

{λi (B)}= {λi (A)}


Modal matrix

If we build a matrix M whose columns are the normalized eigenvectorsui (A)

M =[u1 · · · un

]then the similarity transformation with respect to M results in the diagonalmatrix

Λ =

λ1 0 · · · 00 λ2 · · · 0...

.... . .

...0 0 · · · λn

= M−1AM

M is the modal matrix.

If A is symmetric, all its eigenvalues are real and we have

Λ = MTAM

In this case M is orthonormal.


Singular Value decomposition (SVD)

Given a matrix A ∈ Rm×n, having rank r = ρ(A)≤ s, with s = min{m,n},it can be decomposed (factored) in the following way:

A = UΣVT =s

∑i=1

σiuivTi (1)

The decomposition is characterized by three elements:

σi

ui

vi

as follows.


SVD Characterization

σi (A)≥ 0 are the singular values and are equal to the non-negativesquare roots of the eigenvalues of the symmetric matrix ATA:

{σi (A)}= {√

λi (ATA)} σi ≥ 0

ordered in decreasing order

σ1 ≥ σ2 ≥ ·· · ≥ σs ≥ 0

if r < s there are r positive singular values; the remaining ones arezero

σ1 ≥ σ2 ≥ ·· · ≥ σr > 0; σr+1 = · · ·= σs = 0

U is a orthonormal (m×m) matrix

U =[u1 u2 · · · um

]containing the eigenvectors ui of AAT


SVD Characterization

V is a (n×n) orthonormal matrix

V =[v1 v2 · · · vn

]whose columns are the eigenvectors vi of the matrix ATA

Σ is a (m×n) matrix, with the following structure

if m < n Σ =[Σs O

]if m = n Σ = Σs

if m > n Σ =

[Σs

O

].

where Σs = diag(σi ) is diagonal with dimension s× s, having thesingular values on the diagonal.


Example

Given

A =

[1 3 24 6 5

]ρ(A) = 2

its SVD isA = UΣVT

where

U =

[−0.3863 −0.9224−0.9224 0.3863

]Σ =

[9.5080 0 0

0 0.7729 0

]

V =

−0.4287 0.8060 −0.4082−0.7039 −0.5812 −0.4082−0.5663 0.1124 0.8165


Alternative SVD Decomposition

Alternately, we can decompose the A matrix as follows:

A =[P P

]︸︷︷︸U

[Σr OO O

]︸︷︷︸

Σ

[QT

QT

]︸︷︷︸

VT

= PΣrQT (2)

where

P is an orthonormal m× r matrix,

P is an orthonormal m× (m− r) matrix;

Q is an orthonormal n× r matrix, QT

is an orthonormal n× (n− r)matrix;

Σr is an diagonal r × r matrix, whose diagonal elements are thepositive singular values σi > 0, i = 1, · · · , r .


Rank

The rank r = ρ(A) of A is equal to the number r ≤ s of nonzero singularvalues.

Given any matrix A ∈ Rm×n, both ATA and AAT are symmetric, withidentical positive singular values and differ only for the number of zerosingular values.


Matrices A brief introduction - polito.it€¦ · Matrices A brief introduction Basilio Bona DAUIN...

Documents

Transcript of Matrices A brief introduction - polito.it€¦ · Matrices A brief introduction Basilio Bona DAUIN...