Modified Gary Larson Far Side cartoon

Matlab

Linear Algebra Review

Matrices can represent sets of equations!

a11x1+a12x2+…+a1nxn=b1

a21x1+a22x2+…+a2nxn=b2

…

am1x1+am2x2+…+amnxn=bm

What’s the matrix representation?

€

A =

a11 a12 ... a1na21 a22 ... a2n

...am1 am2 ... amn

⎡

⎣

⎢ ⎢ ⎢ ⎢

⎤

⎦

⎥ ⎥ ⎥ ⎥

€

x =

x1x2.xn

⎡

⎣

⎢ ⎢ ⎢ ⎢

⎤

⎦

⎥ ⎥ ⎥ ⎥

€

b =

b1b2.bm

⎡

⎣

⎢ ⎢ ⎢ ⎢

⎤

⎦

⎥ ⎥ ⎥ ⎥

€

Ax = b

Vectors

Matrices

U = [umn] =

u11 u12 … u1n

u21 u22 … u2n

…um1 um2 …

umn

The general matrix consists of m rows and n columns. It is also known as an m x n (read m by n) array.

Each individual number, uij, of the array is called the element

Elements uij where i=j is called the principal diagonal

Transpose of a Matrix

Matrix & Vector Addition

Vector/Matrix addition is associative and commutative(A + B) + C = A + (B + C); A + B = B + A

Matrix and Vector Subtraction

Vector/Matrix subtraction is also associative and commutative(A - B) - C = A - (B - C); A - B = B - A

Matrix and Vector Scaling

X

ax

• For addition and subtraction, the size of the matrices must be the same

Amn + Bmn = Cmn

• For scalar multiplication, the size of Amn does not matter

• All three of these operations do not differ from their ordinary number counterparts

• The operators work element-by-element through the array, aij+bij=cij

Vector Multiplication

• The inner product or dot product

v

wa

€

v ⋅w = (x1,x2) ⋅(y1,y2) = x1y1 + x2y2

The inner product of vector multiplicationis a SCALAR

€

v ⋅w = (x1,x2) ⋅(y1,y2) =||v ||⋅ ||w || cosα

Inner product represents a row matrix multiplied by a column matrix. A row matrix can be multiplied by a column matrix, in that order, only if they each have

the same number of elements!In MATLAB, in order to properly calculate

the dot product of two vectors use

>>sum(a.*b)

element by element multiplication (.*)sum the results

A . prior to the * or / indicatesthat matlab should perform the array or element by element calculation rather

than linear algebra equivalent

or

>>a’*b

• The outer product

A column vector multiplied by a row vector.

In Matlab:

>>a*b’

ans =

24 6 30 8 2 10 -12 -3 -15

The outer product of vector multiplicationis a MATRIX

Two matrices can be multiplied together if and only if

the number of columns in the first equals the number of rows in the second.

Matrix Multiplication

pmmnpn BAC

m

kkjikij bac

1

The inner numbers have to match

a21 a22

In MATLAB, the * symbol represents matrix multiplication :

>>A=B*C

B C A

C*B #taken from previous slide ans =

19 18 17

16 24 32

32 24 16

nnnnnnnn ABBA

• Matrix multiplication is not commutative!

• Matrix multiplication is distributive and associative

A(B+C) = AB + BC

(AB)C = A(BC)

Revisit the vector example

>>a'*b

ans =

11

a1x3 * b3x1 = c1x1

>>a*b’

ans =

24 6 30 8 2 10 -12 -3 -15

a3x1 * b1x3 = c3x3

Dot products in Matlab(using this form – built in functions - don’t have to match dimensions of vectors – can mix

column and row vectors – although they have to be the same length)

>> a=[1 2 3];>> b=[4 5 6];

>> c=dot(a,b)c = 32

>> d=dot(a,b’)d = 32

Dot products using built-in functionFor matrices – does dot product of columns.

Essentially treating the columns like vectors.The matrices have to be the same size.

>> a=[1 2;3 4]a = 1 2 3 4

>> b=[5 6;7 8]b = 5 6 7 8

>> dot(a,b)ans = 26 44

Determinant of a Matrix

or follow the diagonals

det (A) = a11a22a33+ a12a23a31 + a13a21a32 − a11a23a32 − a12a21a33 − a13a22a31

Cross-product of two vectorsThe cross product a × b is defined as a vector c that is

perpendicular to both a and b, with a direction given by the right-hand rule and a magnitude equal to the area of the parallelogram that the vectors span.

€

a × b = a b sinθ

c = a2b3-a3b2 = 2*5 – (-3)*1 = 13

a3b2-a1b3 -3*4 – 6*5 -42

a1b2-a2b1 6*1 – 2*4 -2

Cross products in Matlab(using this form – built in functions - don’t have to match dimensions of vectors – can mix

column and row vectors – although they have to be the same length)

>> a=[1 2 3];>> b=[4 5 6];

>> e=cross(a,b)e = -3 6 -3

>> f=cross(a,b’)f = -3 6 -3

>> g=cross(b,a)g = 3 -6 3

For matrix – does cross product of columns. (one of the dimensions has to be 3 and takes other dimension as additional vectors)

>> a=[1 2;3 4;5 6]a = 1 2 3 4 5 6

>> b=[7 8;9 10;11 12]b = 7 8 9 10 11 12

>> cross(a,b)ans = -12 -12 24 24 -12 -12

Matrix Operators

• + Addition• - Subtraction• * Multiplication• / Division• \ Left division • ^ Power• ' Complex conjugate transpose• ( ) Specify evaluation order

Array Operators

• + Addition• - Subtraction• .* Element-by-element multiplication• ./ Element-by-element division.

• A./B: divides A by B by element

• .\ Element-by-element left division• A.\B divides B by A by element

• .^ Element-by-element power• .' Unconjugated array transpose

• the signs of imaginary numbers are not changed, unlike a regular matrix transpose

Operators as built-in commands

plus - Plus + uplus - Unary plus + minus - Minus - uminus - Unary minus - mtimes - Matrix multiply * times - Array multiply .* mpower - Matrix power ^ power - Array power .^ mldivide - Backslash or left matrix divide \ mrdivide - Slash or right matrix divide / ldivide - Left array divide .\ rdivide - Right array divide ./ cross - cross product

Multiplication in Matlab

>> x=[1 2];>> y=[3 4];

>> z=x*y’z = 11

>> w=x.*yw = 3 8

>> z=x'*yz = 3 4 6 8

Regular matrix multiplication – in this case with vectors 1x2 * 2x1 = 1x1 => dot productElement by element multiplication

Regular matrix multiplication – in this case with vectors 2x1 * 1x2 = 2x2 matrix

Division in MatlabIn ordinary math, division (a/b) can be thought of as a*1/b or a*b-1.

A unique inverse matrix of B, B-1, only potentially exists if B is square. And matrix multiplication is not communicative, unlike ordinary multiplication.

There really is no such thing as matrix division in any simple sense.

/ : B/A is roughly the same as B*inv(A). A and B must have the same number of columns for right division.

\ : If A is a square matrix, A\B is roughly the same as inv(A)*B, except it is computed in a different way. A and B must have the same number of rows for left division.

If A is an m-by-n matrix (not square) and B is a matrix of m rows, AX=B is solved by least squares.

The / and \ are related

B/A = (A'\B')’

Inverse of a Matrix

the determinant

the principal diagonal elements switch

the off diagonal elements change sign

• Square matrices with inverses are said to be nonsingular

• Not all square matrices have an inverse. These are said to be singular.

• Square matrices with determinants = 0 are also singular.

• Rectangular matrices are always singular.

Right- and Left- Inverse

If a matrix G exists such that GA = I, than G is a left-inverse of A

If a matrix H exists such that AH = I, than H is a right-inverse of A

Rectangular matrices may have right- or left- inverses, but they are still singular.

Some Special Matrices

• Square matrix: m (# rows) = n (# columns)

• Symmetric matrix: subset of square matrices where AT = A

• Diagonal matrix: subset of square matrices where elements off the principal diagonal are zero, aij = 0 if i ≠ j

• Identity or unit matrix: special diagonal matrix where all principal diagonal elements are 1

Linear Dependence

2a + 1b = c

a b c

Linear Independence

There is no simple, linear equation that can make these vectors related.

a b c

Rank of a matrix

Acknowledgement

• This lecture borrows heavily from online lectures/ppt files posted by

• David Jacobs at Univ. of Maryland• Tim Marks at UCSD• Joseph Bradley at Carnegie Mellon