Post on 25-Mar-2018
MATH10212 Linear Algebra
Lecture Notes
Last change: 23 Apr 2018
Textbook
Students are strongly advised to ac-quire a copy of the Textbook:
D. C. Lay. Linear Algebra and itsApplications. Pearson, 2006. ISBN0-521-28713-4.
Other editions can be used as well; the
book is easily available on Amazon (thisincludes some very cheap used copies)and in Blackwell’s bookshop on campus.
These lecture notes should betreated only as indication of thecontent of the course; the text-book contains detailed explana-tions, worked out examples, etc.
About Homework
The Undergraduate Student Hand-book 1 says:
As a general rule for each hour in classyou should spend two hours on indepen-dent study. This will include reading lec-ture notes and textbooks, working on ex-ercises and preparing for coursework andexaminations.
In respect of this course, MATH10212Linear Algebra B, this means that stu-dents are expected to spend 8 (eight!)hours a week in private study of LinearAlgebra.
Normally, homework assignments willconsist of some odd numbered exercisesfrom the sections of the Textbook cov-ered in the lectures up to Wednesday inparticular week. The Textbook containsanswers to most odd numbered exercises(but the numbering of exercises mightchange from one edition to another).
Homework N (already posted on thewebpage and BB9 space of the course)should be returned to SupervisionClasses teachers early in Week N, formarking and discussion at supervisionclasses on Tuesday and Wednesday.
1http://www.maths.manchester.ac.uk/study/undergraduate/information-for-current-students/
undergraduatestudenthandbook/teachingandlearning/timetabledclasses/.
2
Communication
The Course Webpage is
http://www.maths.manchester.ac.
uk/~avb/math10212-Linear-Algebra-B.
html
The Course Webpage page is updatedalmost daily, sometimes several times aday. Refresh it (and files it is linked to)in your browser, otherwise you may missthe changes.
Email: Feel free to write to me withquestions, etc., at the address
alexandre.borovik@manchester.ac.
uk
but only from your university e-mailaccount.
Emails from Gmail, Hotmail, etc. auto-matically go to spam.
What is Linear Algebra?
It is well-known that the total cost ofa purchase of amounts g1, g2, g3 of somegoods at prices p1, p2, p3, respectively, isan expression
p1g1 + p2g2 + p3g3 =3∑
i=1
pigi.
Expressions of this kind,
a1x1 + · · ·+ anxn
are called linear forms in variablesx1, . . . , xn with coefficients a1, . . . , an.
• Linear Algebra studies the mathe-matics of linear forms.
• Over the course, we shall developincreasingly compact notation foroperations of Linear Algebra. Inparticular, we shall discover that
p1g1 + p2g2 + p3g3
can be very conveniently written as
[p1 p2 p3
] g1g2g3
and then abbreviated
[p1 p2 p3
] g1g2g3
= P TG,
where
P =
p1p2p3
and G =
g1g2g3
,and T (transposition) turns rowvectors into column vectors, andvice versa:
P T =
p1p2p3
T
=[p1 p2 p3
],
and
[p1 p2 p3
]T=
p1p2p3
.• Physicists use even more short no-
tation and, instead of
p1g1 + p2g2 + p3g3 =3∑
i=1
pigi
write
p1g1 + p2g
2 + p3g3 = pig
i,
omitting the summation sign en-tirely. This particular trick wasinvented by Albert Einstein, of allpeople. I do not use “physics”tricks in my lectures, but amprepared to give a few addi-tional lectures to physics stu-dents.
3
• Warning: Increasingly compactnotation leads to increasingly com-pact and abstract language.
• Unlike, say, Calculus, Linear Alge-bra focuses more on the develop-ment of a special mathematics lan-guage rather than on procedures.
MATH10212 • Linear Algebra B • Lecture 1 • Linear systems • Last change: 23 Apr 2018 4
Lecture 1 Systems of linear equations [Lay 1.1]
A linear equation in the variablesx1, . . . , xn is an equation that can bewritten in the form
a1x1 + a2x2 + · · ·+ anxn = b
where b and the coefficients a1, . . . , anare real numbers. The subscript n canbe any natural number.
A system of simultaneous linearequations is a collection of one or morelinear equations involving the same vari-ables, say x1, . . . , xn. For example,
x1 + x2 = 3
x1 − x2 = 1
We shall abbreviate the words “a sys-tem of simultaneous linear equa-tions” just to “a linear system”.
A solution of the system is a list(s1, . . . , sn) of numbers that makes eachequation a true identity when the valuess1, . . . , sn are substituted for x1, . . . , xn,respectively. For example, in the systemabove (2, 1) is a solution.
The set of all possible solutions is calledthe solution set of the linear system.
Two linear systems are equivalent if thehave the same solution set.
We shall be use the following elemen-tary operations on systems od simul-taneous liner equations:
Replacement Replace one equation bythe sum of itself and a multiple ofanother equation.
Interchange Interchange two equa-tions.
Scaling Multiply all terms in a equa-tion by a nonzero constant.
Note: The elementary operations are re-versible.
Theorem: Elementary operationspreserve solutions.
If a system of simultaneous linear equa-tions is obtained from another system byelementary operations, then the two sys-tems have the same solution set.
We shall prove later in the course that asystem of linear equations has either
• no solution, or
• exactly one solution, or
• infinitely many solutions,
under the assumption that the coeffi-cients and solutions of the systems arereal or complex numbers.
A system of linear equations is said tobe consistent it if has solutions (eitherone or infinitely many), and a system ininconsistent if it has no solution.
Solving a linear system
The basic strategy is
to replace one system withan equivalent system (that is,with the same solution set)which is easier to solve.
Existence and uniquenessquestions
• Is the system consistent?
• If a solution exist, is it unique?
Equivalence of linear systems
• When are two linear systemsequivalent?
MATH10212 • Linear Algebra B • Lecture 1 • Linear systems • Last change: 23 Apr 2018 5
Checking solutions
Given a solution of the system of linear equations, how to check that it is correct?
MATH10212 • Linear Algebra B • Lecture 2 • Row reduction and echelon forms • Last change: 23 Apr 2018 6
Lecture 2 Row reduction and echelon forms [Lay 1.2]
Matrix notation
It is convenient to write coefficients of alinear system in the form of a matrix, arectangular table. For example, the sys-tem
x1 − 2x2 + 3x3 = 1
x1 + x2 = 2
x2 + x3 = 3
has the matrix of coefficients1 −2 31 1 00 1 1
and the augmented matrix1 −2 3 1
1 1 0 20 1 1 3
;
notice how the coefficients are alignedin columns, and how missing coefficientsare replaced by 0.
The augmented matrix in the exampleabove has 3 rows and 4 columns; we saythat it is a 3×4 matrix. Generally, a ma-trix with m rows and n columns is calledan m× n matrix.
Elementary row operations
Replacement Replace one row by thesum of itself and a multiple of an-other row.
Interchange Interchange two rows.
Scaling Multiply all entries in a row bya nonzero constant.
The two matrices are row equivalent ifthere is a sequence of elementary row op-erations that transforms one matrix intothe other.
Note:
• The row operations are reversible.
• Row equivalence of matrices is anequivalence relation on the set ofmatrices.
Theorem: Row Equivalence.
If the augmented matrices of two linearsystems are row equivalent, then the twosystems have the same solution set.
A nonzero row or column of a matrix isa row or column which contains at leastone nonzero entry.
We can now formulate a theorem (to beproven later).
Theorem: Equivalence of linearsystems.
Two linear systems are equivalent if andonly if the augmented matrix of one ofthem can be obtained from the aug-mented matrix of another system by frowoperations and insertion / deletion ofzero rows.
MATH10212 • Linear Algebra B • Lecture 3 • Solution of Linear Systems • Last change: 23 Apr 2018 7
Lecture 3 Solution of Linear Systems [Lay 1.2]
A nonzero row or column of a matrix isa row or column which contains at leastone nonzero entry. A leading entry ofa row is the leftmost nonzero entry (in anon-zero row).
Definition. A matrix is in echelonform (or row echelon form) if it hasthe following three properties:
1. All nonzero rows are above any rowof zeroes.
2. Each leading entry of a row is incolumn to the right of the leadingentry of the row above it.
3. All entries in a column below aleading entry are zeroes.
If, in addition, the following twoconditions are satisfied,
4. All leading entries are equal 1.
5. Each leading 1 is the only non-zeroentry in its column
then the matrix is in reduced echelonform.
An echelon matrix is a matrix in ech-elon form.
Any non-zero matrix can be row re-duced (that, transformed by elementaryrow operations) into a matrix in echelonform (but the same matrix can give riseto different echelon forms).
Examples. The following is a schematicpresentation of an echelon matrix:
� ∗ ∗ ∗ ∗0 � ∗ ∗ ∗0 0 0 � ∗
and this is a reduced echelon matrix:
1 0 ∗ 0 ∗0 1 ∗ 0 ∗0 0 0 1 ∗
Theorem 1.2.1: Uniqueness of thereduced echelon form.
Each matrix is row equivalent to one andonly one reduced echelon form.
Definition. A pivot position in a ma-trix A is a location in A that correspondsto a leading 1 in the reduced echelonform of A. A pivot column is a col-umn of A that contains a pivot position.
Example for solving in the lecture(The Row Reduction Algorithm):
0 2 2 2 21 1 1 1 11 1 1 3 31 1 1 2 2
A pivot is a nonzero number in a pivotposition which is used to create zeroes inthe column below it.
A rule for row reduction:
1. Pick the leftmost non-zero columnand in it the topmost nonzero en-try; it is a pivot.
2. Using scaling, make the pivotequal 1.
3. Using replacement row opera-tions, kill all non-zero entries in thecolumn below the pivot.
4. Mark the row and column contain-ing the pivot as pivoted.
5. Repeat the same with the matrixmade of not pivoted yet rows andcolumns.
6. When this is over, interchangethe rows making sure that the re-sulting matrix is in echelon form.
7. Using replacement row opera-tions, kill all non-zero entries in thecolumn above the pivot entries.
MATH10212 • Linear Algebra B • Lecture 3 • Solution of Linear Systems • Last change: 23 Apr 2018 8
Solution of Linear Systems
When we converted the augmented ma-trix of a linear system into its reducedrow echelon form, we can write out theentire solution set of the system.
Example. Let1 0 −5 10 1 1 40 0 0 0
be the augmented matrix of a a linearsystem; then the system is equivalent to
x1 − 5x3 = 1
x2 + x3 = 4
0 = 0
The variables x1 and x2 correspond topivot columns in the matrix and a recalled basic variables (also leading orpivot variables). The other variable,x3 is a free variable.
Free variables can be assigned arbitraryvalues and then leading variables ex-pressed in terms of free variables:
x1 = 1 + 5x3
x2 = 4− x3x3 is free
Theorem 1.2.2: Existence andUniqueness
A linear system is consistent if and onlyif the rightmost column of the aug-mented matrix is not a pivot column—that is, if and only if an echelon form of
the augmented matrix has no row of theform [
0 · · · 0 b]
with b nonzero
If a linear system is consistent, then thesolution set contains either
(i) a unique solution, when there areno free variables, or
(ii) infinitely many solutions, whenthere is at least one free variable.
Using row reduction to solve a lin-ear system
1. Write the augmented matrix of thesystem.
2. Use the row reduction algorithmto obtain an equivalent augmentedmatrix in echelon form. Decidewhether the system is consistent.
3. if the system is consistent, get thereduced echelon form.
4. Write the system of equations cor-responding to the matrix obtainedin Step 3.
5. Express each basic variable interms of any free variables appear-ing in the equation.
MATH10212 • Linear Algebra B • Lecture 4 • Vector equations • Last change: 23 Apr 2018 9
Lecture 4 Vector equations [Lay 1.3]
A matrix with only one column is calleda column vector, or simply a vector.
Rn is the set of all column vectors withn entries.
A row vector: a matric with one row.
Two vectors are equal if and only if theyhave
• the same shape,
• the same number of rows,
• and their corresponding entries areequal.
The set of al vectors with n entries isdenoted Rn.
The sum u+v of two vectors u and v inRn is obtained by adding correspondingentries in u and v. For example in R2[
12
]+
[−1−1
]=
[01
].
The scalar multiple cv of a vector vand a real number (“scalar”) c is thevector obtained by multiplying each en-try in v by c. For example in R3,
1.5
10−3
=
1.50−2
.The vector whose entries are all zeroes iscalled the zero vector and denoted 0:
0 =
00...0
.Operations with row vectors are definedin a similar way.
Algebraic properties of Rn
For all u,v,w ∈ Rn and all scalars c andd:
1. u + v = v + u
2. (u + v) + w = u + (v + w)
3. u + 0 = 0 + u = u
4. u + (−u) = −u + u = 0
5. c(u + v) = cu + cv
6. (c+ d)u = cu + du
7. c(du) = (cd)u
8. 1u = u
(Here −u denotes (−1)u.)
Linear combinations
Given vectors v1,v2, . . . ,vp in Rn andscalars c1, c2, . . . , cp, the vector
y = c1v1 + · · · cpvp
is called a linear combina-tion of v1,v2, . . . ,vp with weightsc1, c2, . . . , cp.
Rewriting a linear system asa vector equation
Consider an example: the linear system
x2 + x3 = 2
x1 + x2 + x3 = 3
x1 + x2 − x3 = 2
can be written as equality of two vectors: x2 + x3x1 + x2 + x3x1 + x2 − x3
=
232
which is the same as
x1
011
+ x2
111
+ x3
11−1
=
232
MATH10212 • Linear Algebra B • Lecture 4 • Vector equations • Last change: 23 Apr 2018 10
Let us write the matrix0 1 1 21 1 1 31 1 −1 2
in a way that calls attention to itscolumns: [
a1 a2 a3 b]
Denote
a1 =
011
, a2 =
111
, a3 =
11−1
and
b =
232
,then the vector equation can be writtenas
x1a1 + x2a2 + x3a3 = b.
Notice that to solve this equation is thesame as
express b as a linear combi-nation of a1, a2, a3, and findall such expressions.
Therefore
solving a linear system is the same asfinding an expression of the vector of theright part of the system as a linear com-bination of columns in its matrix of co-efficients.
A vector equation
x1a1 + x2a2 + · · ·+ xnan = b.
has the same solution set as the linearsystem whose augmented matrix is[
a1 a2 · · · an b]
In particular b can be generated by a lin-ear combination of a1, a2, . . . , an if andonly if there is a solution of the corre-sponding linear system.
Definition. If v1, . . . ,vp are in Rn,then the set of all linear combination ofv1, . . . ,vp is denoted by
Span{v1, . . . ,vp}
and is called the subset of Rn spanned(or generated) by v1, . . . ,vp; or thespan of vectors v1, . . . ,vp.
That is, Span{v1, . . . ,vp} is the collec-tion of all vectors which can be writtenin the form
c1v1 + c2v2 + · · ·+ cpvp
with c1, . . . , cp scalars.
We say that vectors v1, . . . ,vp span Rn
if
Span{v1, . . . ,vp} = Rn
We can reformulate the definition ofspan as follows: iv a1, . . . , ap ∈ Rn, then
Span{a1, . . . , ap} = {b ∈ Rn such that
[a1 · · · ap|b]
is the augmented matrix
of a consistent system
of linear equations},
or, in simpler words,
Span{a1, . . . , ap} = {b ∈ Rn such that
the system of equations
x1a1 + · · ·+ xpap = b
has a solution}.
MATH10212 • Linear Algebra B • Lecture 5 • The matrix equation Ax = b • Last change: 23 Apr 2018 11
Lecture 5: The matrix equation Ax = b [Lay 1.4]
Definition. If A is an m × n matrix,with columns a1, . . . , an, and if x is inRn, then the product of A and x, de-noted Ax, is the linear combination ofthe columns of A using the correspond-ing entries in x as weights:
Ax =[a1a2 · · · an
] x1...xn
= x1a1 + x2a2 + · · ·+ xnan
Example. The system
x2 + x3 = 2
x1 + x2 + x3 = 3
x1 + x2 − x3 = 2
was written as
x1a1 + x2a2 + x3a3 = b.
where
a1 =
011
, a2 =
111
, a3 =
11−1
and
b =
232
.In the matrix product notation it be-comes 0 1 1
1 1 11 1 −1
x1x2x3
=
232
or
Ax = b
where
A =[a1 a2 a3
], x =
x1x2x3
.
Theorem 1.4.3: If A is an m × n ma-trix, with columns a1, . . . , an, and if x isin Rn, the matrix equation
Ax = b
has the same solution set as the vectorequation
x1a1 + x2a2 + · · ·+ xnan = b
which has the same solution set as thesystem of linear equations whose aug-mented matrix is[
a1 a2 · · · an b].
Existence of solutions. The equationAx = b has a solution if and only if b isa linear combination of columns of A.
Theorem 1.4.4: Let A be an m×n ma-trix. Then the following statements areequivalent.
(a) For each b ∈ Rn, the equationAx = b has a solution.
(b) Each b ∈ Rn is a linear combina-tion of columns of A.
(c) The columns of A span Rn.
(d) A has a pivot position in every row.
Row-vector rule for computing Ax.If the product Ax is defined then the ithentry in Ax is the sum of products ofcorresponding entries from the row i ofA and from the vector x.
Theorem 1.4.5: Properties of thematrix-vector product Ax.
If A is an m× n matrix, u,v ∈ Rn, andc is a scalar, then
(a) A(u + v) = Au + Av;
(b) A(cu) = c(Au).
MATH10212 • Linear Algebra B • Lecture 5 • The matrix equation Ax = b • Last change: 23 Apr 2018 12
Homogeneous linear systems
A linear system is homogeneous if itcan be written as
Ax = 0.
A homogeneous system always has atleast one solution x = 0 (trivial solu-tion).
Therefore for homogeneous systems animportant question os existence of a
nontrivial solution, that is, a nonzerovector x which satisfies Ax = 0:
The homogeneous system Ax = b hasa nontrivial solution if and only if theequation has at least one free variable.
Example.
x1 + 2x2 − x3 = 0
x1 + 3x3 + x3 = 0
MATH10212 • Linear Algebra B • Lecture 6 • Solution sets of linear equations • Last change: 23 Apr 2018 13
Lecture 6: Solution sets of linear equations [Lay 1.5]
Nonhomogeneous systems
When a nonhomogeneous system hasmany solutions, the general solution canbe written in parametric vector form aone vector plus an arbitrary linea com-bination of vectors that satisfy the cor-responding homogeneous system.
Example.
x1 + 2x2 − x3 = 0
x1 + 3x3 + x3 = 5
Theorem 1.5.6: Suppose the equation
Ax = b
is consistent for some given b, and pbe a solution. Then the solution set ofAx = b is the set of all vectors of theform
w = p + vh,
where vh is any solution of the homoge-neous equation
Ax = 0.
MATH10212 • Linear Algebra B • Lecture 7 • Linear independence • Last change: 23 Apr 2018 14
Lecture 7: Linear independence [Lay 1.7]
Definition. An indexed set of vectors
{v1, . . . ,vp }
in Rn is linearly independent if thevector equation
x1v1 + · · ·+ xpvp = 0
has only trivial solution.
The set{v1, . . . ,vp }
in Rn is linearly dependent if there ex-ist weights c1, . . . , cp, not all zero, suchthat
c1v1 + · · ·+ cpvp = 0
Linear independence of matrixcolumns. The matrix equation
Ax = 0
where A is made of columns
A =[a1 · · · an
]can be written as
x1a1 + · · ·+ xnan = 0
Therefore the columns of matrix A arelinearly independent if and only if theequation
Ax = 0
has only the trivial solution.
A set of one vectors {v1} is linearly de-pendent if v1 = 0.
A set of two vectors {v1,v2 } is linearlydependent if at least one of the vectorsis a multiple of the other.
Theorem 1.7.7: Characterisation oflinearly dependent sets. An indexedset
S = {v1, . . . ,vp }
of two or more vectors is linearly depen-dent if and only if at least one of thevectors in S is a linear combination ofthe others.
Theorem 1.7.8: dependence of“big” sets. If a set contains more vec-tors than entries in each vector, then theset is linearly dependent. Thus, any set
{v1, . . . ,vp }
in Rn is linearly dependent if p > n.
MATH10212 • Linear Algebra B • Lecture 8 • Linear transformations • Last change: 23 Apr 2018 15
Lecture 8: Introduction to linear transformations [1.8]
Transformation. A transformationT from Rn to Rm is a rule that assignsto each vector x in Rn a vector T (x) inRm.
The set Rn is called the domain of T ,the set Rm is the codomain of T .
Matrix transformations. With everym × n matrix A we can associated thetransformation
T : Rn −→ Rm
x 7→ Ax.
In short:T (x) = Ax.
The range of a matrix transforma-tion. The range of T is the set of alllinear combinations of the columns of A.
Indeed, this can be immediately seenfrom the fact that each image T (x) hasthe form
T (x) = Ax = x1a1 + · · ·+ xnan
Definition: Linear transformations.A transformation
T : Rn −→ Rm
is linear if:
• T (u + v) = T (u) + T (v) for allvectors u,v ∈ Rn;
• T (cu) = cT (u) for all vectors uand all scalars c.
Properties of linear transforma-tions. If T is a linear transformationthen
T (0) = 0
and
T (cu + dv) = cT (u) + dT (v).
The identity matrix. An n × n ma-trix with 1’s on the diagonal and 0’s else-where is called the identity matrix In.For example,
I1 =[1], I2 =
[1 00 1
],
I3 =
1 0 00 1 00 0 1
, I4 =
1 0 0 00 1 0 00 0 1 00 0 0 1
.The columns of the identity matrix Inwill be denoted
e1, e2, . . . , en.
For example, in R3
e1 =
100
, e2 =
010
, e3 =
001
.Obsetrve that if x is an arbitrary vectorin Rn,
x =
x1...xn
,then
x =
x1x2x3...xn
= x1
100...0
+ x2
010...0
+ · · ·+ xn
000...1
= x1e1 + x2e2 + · · ·+ xnen.
The identity transformation. It iseasy to check that
Inx = x for all x ∈ Rn.
MATH10212 • Linear Algebra B • Lecture 8 • Linear transformations • Last change: 23 Apr 2018 16
Therefore the linear transformation as-sociated with the identity matrix is the
identity transformation of Rn:
Rn −→ Rn
x 7→ x
The matrix of a linear transformation [Lay 1.9]
Theorem 1.9.10: The matrix of alinear transformation. Let
T : Rn −→ Rm
be a linear transformation.
Then there exists a unique matrix A suchthat
T (x) = Ax for all x ∈ Rn.
In fact, A is the m×n matrix whose jthcolumn is the vector T (ej) where ej isthe jth column of the identity matrix inRn:
A =[T (e1) · · · T (en)
]
Proof. First we express x in terms ofe1, . . . , en:
x = x1e1 + x2e2 + · · ·+ xnen
and compute, using definition of lineartransformation
T (x) = T (x1e1 + x2e2 + · · ·+ xnen)
= T (x1e1) + · · ·+ T (xnen)
= x1T (e1) + · · ·+ xnT (en)
and then switch to matrix
notation:
=[T (e1) · · · T (en)
] x1...xn
= Ax.
The matrix A is called the standardmatrix for the linear transforma-tion T .
Definition. A transformation
T : Rn −→ Rm
is onto Rm if each b ∈ Rm is the imageof at least one x ∈ Rn.
A transformation T is one-to-one ifeach b ∈ Rm is the image of at mostone x ∈ Rn.
Theorem: One-to-one transforma-tions: a criterion. A linear transfor-mation
T : Rn −→ Rm
is one-to-one if and only if the equation
T (x) = 0
has only the trivial solution.
Theorem: “One-to-one” and“onto” in terms of matrices. Let
T : Rn −→ Rm
be linear transformation and let A be thestandard matrix for T . Then:
• T maps Rn onto Rm if and only ifthe columns of A span Rm.
• T is one-to-one if and only if thecolumns of A are linearly indepen-dent.
MATH10212 • Linear Algebra B • Lecture 9 • Matrix operations: Addition • Last change: 23 Apr 2018 17
Lecture 9: Matrix operations: Addition [Lay 2.1]
Labeling of matrix entries. Let A bean m× n matrix.
A =[a1 a2 · · · an
]
=
a11 · · · a1j · · · a1n...
......
ai1 · · · aij · · · ain...
......
am1 · · · amj · · · amn
Notice that in aij the first subscript idenotes the row number, the second sub-script j the column number of the entryaij. In particular, the column aj is
aj =
a1j...aij...amj
.
Diagonal matrices, zero matrices.The diagonal entries in A are
a11, a22, a33, . . .
For example, the diagonal entries of thematrix
A =
1 2 34 5 67 8 9
are 1, 5, and 9.
A square matrix is a matrix with equalnumbers of rows and columns.
A diagonal matrix is a square matrixwhose non-diagonal entries are zeroes.
Matrices
[1 00 2
],
1 0 00 0 00 0 2
,π 0 0
0√
2 00 0 3
are all diagonal. The identity matrices
[1],
[1 00 1
],
1 0 00 1 00 0 1
, . . .are diagonal.
Zero matrix. By definition, 0 is a m×nmatrix whose entries are all zero. For ex-ample, matrices
[0 0
],
[0 0 00 0 0
]are zero matrices. Notice that zerosquare matrices, like
[0 00 0
],
0 0 00 0 00 0 0
are diagonal!
Sums. If
A =[a1 a2 · · · an
]and
B =[b1 b2 · · · bn
]are m × n matrices then we define thesum A+B as
A+B =[a1 + b1 a2 + b2 · · · an + bn
]
=
a11 + b11 · · · a1j + b1j · · · a1n + b1n
......
...ai1 + bi1 · · · aij + bij · · · ain + bin
......
...am1 + bm1 · · · amj + bmj · · · amn + bmn
MATH10212 • Linear Algebra B • Lecture 9 • Matrix operations: Addition • Last change: 23 Apr 2018 18
Scalar multiple. If c is a a scalar thenwe define
cA =[ca1 ca2 · · · can
]
=
ca11 · · · ca1j · · · ca1n
......
...cai1 · · · caij · · · cain
......
...cam1 · · · camj · · · camn
Theorem 2.1.1: Properties of ma-trix addition. Let A, B, and C be ma-
trices of the same size and r and s bescalars.
1. A+B = B + A
2. (A+B) + C = A+ (B + C)
3. A+ 0 = A
4. r(A+B) = rA+ rB
5. (r + s)A = rA+ sA
6. r(sA) = (rs)A.
MATH10212 • Linear Algebra B • Lecture 10 • Matrix multiplication • Last change: 23 Apr 2018 19
Lecture 10: Matrix multiplication [Lay 2.1]
Composition of linear transforma-tions. Let B be an m × n matrix andA an p × m matrix. They define lineartransformations
T : Rn −→ Rm, x 7→ Bx
and
S : Rm −→ Rp, y 7→ Ay.
Their composition
(S ◦ T )(x) = S(T (x))
is a linear transformation
S ◦ T : Rn −→ Rp.
From the previous lecture, we know thatlinear transformations are given by ma-trices. What is the matrix of S ◦ T?
Multiplication of matrices. To an-swer the above question, we need to com-pute A(Bx) in matrix form.
Write x as
x =
x1...xn
and observe
Bx = x1b1 + · · ·+ xnbn.
Hence
A(Bx) = A(x1b1 + · · ·+ xnbn)
= A(x1b1) + · · ·+ A(xnbn)
= x1A(b1) + · · ·+ xnA(bn)
=[Ab1 Ab2 · · · Abn
]x
Therefore multiplication by the matrix
C =[Ab1 Ab2 · · · Abn
]transforms x into A(Bx). Hence C isthe matrix of the linear transformation
S ◦T and it will be natural to call C theproduct of A and B and denote
C = A ·B
(but the multiplication symbol “·” is fre-quently skipped).
Definition: Matrix multiplication.If A is an p × m matrix and B is anm× n matrix with columns
b1, . . . ,bn
then the product AB is the p×n matrixwhose columns are
Ab1, . . . , Abn :
AB = A[b1 · · · bn
]=
[Ab1 · · · Abn
].
Columns of AB. Each column Abj ofAB is a linear combination of columns ofA with weights taken from the jth col-umn of B:
Abj =[a1 · · · am
] b1j...bmj
= b1ja1 + · · ·+ bmjam
Mnemonic rules
[m× n matrix] · [n× p matrix] =[m× p matrix]
columnj(AB) = A · columnj(B)
rowi(AB) = rowi(A) ·B
Theorem 2.1.2: Properties of ma-trix multiplication. Let A be an m×nmatrix and let B and C be matrices forwhich indicated sums and products aredefined. Then the following identitiesare true:
MATH10212 • Linear Algebra B • Lecture 10 • Matrix multiplication • Last change: 23 Apr 2018 20
1. A(BC) = (AB)C
2. A(B + C) = AB + AC
3. (B + C)A = BA+ CA
4. r(AB) = (rA)B = A(rB) for anyscalar r
5. ImA = A = AIn
Powers of matrix. As it is usual inalgebra, we define, for a square matrixA,
Ak = A · · ·A (k times)
If A 6= 0 then we set
A0 = I
The transpose of a matrix. Thetranspose AT of an m × n matrix A isthe n×m matrix whose rows are formedfrom corresponding columns of A:
[1 2 34 5 6
]T=
1 42 53 6
Theorem 2.1.3: Properties of trans-pose. Let A and B denote matriceswhose sizes are appropriate for the fol-lowing sums and products. Then wehave:
1. (AT )T = A
2. (A+B)T = AT +BT
3. (rA)T = r(AT ) for any scalar r
4. (AB)T = BTAT
MATH10212 • Linear Algebra B • Lecture 11 • The inverse of a matrix • Last change: 23 Apr 2018 21
Lecture 11: The inverse
of a matrix [Lay 2.2]
Invertible matrices
An n×n matrix A is invertible if thereis an n× n matrix C such that
CA = I and AC = I
C is called the inverse of A.
The inverse of A, if exists, is unique (!)and is denoted A−1:
A−1A = I and AA−1 = I.
Singular matrices. A non-invertiblematrix is called a singular matrix.
An invertible matrix is nonsingular.
Theorem 2.2.4: Inverse of a 2 × 2matrix. Let
A =
[a bc d
]If ad− bc 6= 0 then A is invertible and[
a bc d
]−1=
1
ad− bc
[d −b−c a
]
The quantity ad − bc is called the de-terminant of A:
detA = ad− bc
Theorem 2.2.5: Solving matrixequations. If A is an invertible n × nmatrix, then for each b ∈ Rn, the equa-tion Ax = b has the unique solution
x = A−1b.
Theorem 2.2.6: Properties of in-vertible matrices.
(a) If A is an invertible matrix, thenA−1 is also invertible and
(A−1)−1 = A
(b) If A and B are n×n invertible ma-trices, then so is AB, and
(AB)−1 = B−1A−1
(c) If A is an invertible matrix, thenso is AT , and
(AT )−1 = (A−1)T
MATH10212 • Linear Algebra B • Lecture 12 • Characterizations of invertible matrices • Last change: 23 Apr 2018 22
Lecture 12: Characterizations of invertible matrices.
Definition: Elementary matrices.An elementary matrix is a matrix ob-tained by performing a single elementaryrow operation on an identity matrix.
If an elementary row transformation isperformed on an n × n matrix A, theresulting matrix can be written as EA,where the m×m matrix is made by thesame row operations on Im.
Each elementary matrix E is invertible.The inverse of E is the elementary ma-trix of the same type that transforms Eback into I.
Theorem 2.2.7: Invertible matrices.An n × n matrix A is invertible if andonly if A is row equivalent to In, andin this case, any sequence of elementaryrow operations that reduces A to In alsotransforms In into A−1.
Computation of inverses.
• Form the augmented matrix[A I
]and row reduce it.
• If A is row equivalent to I,then
[A I
]is row equivalent to[
I A−1].
• Otherwise A has no inverse.
The Invertible Matrix Theorem2.3.8: For an n × n matrix A, the fol-lowing are equivalent:
(a) A is invertible.
(b) A is row equivalent to In.
(c) A has n pivot positions.
(d) Ax = 0 has only the trivial solu-tion.
(e) The columns of A are linearly in-dependent.
(f) The linear transformation x 7→ Axis one-to-one.
(g) Ax = b has at least one solutionfor each b ∈ Rn.
(h) The columns of A span Rn.
(i) x 7→ Ax maps Rn onto Rn.
(j) There is an n × n matrix C suchthat CA = I.
(k) There is an n × n matrix D suchthat AD = I.
(l) AT is invertible.
One-sided inverse is the inverse. LetA and B be square matrices.
If AB = I then both A and B are in-vertible and
B = A−1 and A = B−1.
Theorem 2.3.9: Invertible lineartransformations. Let
T : Rn −→ Rn
be a linear transformation and A itsstandard matrix.
Then T is invertible if and only if A isan invertible matrix.
In that case, the linear transformation
S(x) = A−1x
is the only transformation satisfying
S(T (x)) = x for all x ∈ Rn
T (S(x)) = x for all x ∈ Rn
MATH10212 • Linear Algebra B • Lecture 13 • Partitioned matrices • Last change: 23 Apr 2018 23
Lecture 13 Partitioned matrices [Lay 2.4]
Example: Partitioned matrix
A =
1 2 a3 4 bp q z
=
[A11 A12
A21 A22
]
A is a 3× 3 matrix which can be viewedas a 2× 2 partitioned (or block) ma-trix with blocks
A11 =
[1 23 4
], A12 =
[ab
],
A21 =[p q
], A22 =
[z]
Addition of partitioned matrices. Ifmatrices A and B are of the same sizeand partitioned the same way, they canbe added block-by-block.
Similarly, partitioned matrices can bemultiplied by a scalar blockwise.
Multiplication of partitioned ma-trices. If the column partition of Amatches the row partition of B then
AB can be computed by the usual row-column rule, with blocks treated as ma-trix entries.
A =
1 2 a3 4 bp q z
, B =
α βγ δΓ ∆
Example
A =
1 2 a3 4 bp q z
=
[A11 A12
A21 A22
],
B =
α βγ δΓ ∆
=
[B1
B2
]
AB =
[A11 A12
A21 A22
] [B1
B2
]=
[A11B1 + A12B2
A21B1 + A22B2
]
=
[1 23 4
] [α βγ δ
]+
[ab
] [Γ ∆
][p q
] [α βγ δ
]+[z] [
Γ ∆]
Theorem 2.4.10: Column-Row Expansion of AB. If A is m × n and B is n × pthen
AB =[col1(A) col2(A) · · · coln(A)
]
row1(B)row2(B)
...rown(B)
= col1(A)row1(B) + · · ·+ coln(A)rown(B)
Example Partitioned matrices natu-rally arise in analysis of systems of si-multaneous linear equations. Consider a
sysetm of equations:
x1 + 2x2 + 3x3 + 4x4 = 1
x1 + 3x2 + 3x3 + 4x4 = 2
MATH10212 • Linear Algebra B • Lecture 13 • Partitioned matrices • Last change: 23 Apr 2018 24
In matrix form, it looks as
[1 2 3 41 3 3 4
]x1x2x3x4
=
[12
]
Denote
A =
[1 21 3
]C =
[3 43 4
]Y1 =
[x1x2
]Y2 =
[x3x4
]B =
[12
]then we can rewrite the matrices in-volved in the matrix equation ar parti-tioned matrices:[
A C] [Y1Y2
]= B
and, multiplying matrices as partitionedmatrices,
AY1 + CY2 = B.
Please notice that matrix A is invertible– at this stage it should be an easy men-tal calculation for you. Multiplying theboth parts of the last equation by A−1,we get
Y1 + A−1CY2 = A−1B
andY1 = A−1B − A−1CY2
The entries of
Y2 =
[x3x4
]can be taken for free parameters: x3 = t,x4 = s, and x1 and x2 expressed as[
x1x2
]= A−1B − A−1C
[ts
]
Observe that
A−1 =
[3 −2−1 1
]and
A−1C =
[3 −2−1 1
]·[3 43 4
]=
[3 40 0
]which gives us an answer:[x1x2
]=
[3 −2−1 1
] [12
]−[3 40 0
] [ts
]=
[−11
]−[3t+ 4s
0
][x3x4
]=
[ts
]We may wish to bring it in a more tra-ditional form:x1x2x3x4
=
−1100
+
−3t− 4s
0ts
=
−1100
+ t
−3010
+ s
−4001
Of course, when we solving a singles sys-tem of equations, the formulae of thekind
Y1 = A−1B − A−1CY2
do not save time and effort, but in manyapplications you have to solve millionsof systems of linear equations with thesame matrix of coefficients but differentright-hand parts, and in this situationshortcuts based on compound matricesbecome very useful.
MATH10212 • Linear Algebra B • Lecture 14 • Subspaces • Last change: 23 Apr 2018 25
Lecture 14: Subspaces of Rn [Lay 2.8]
Subspace. A subspace of Rn is any setH that has the properties
(a) 0 ∈ H.
(b) For each vectors u,v ∈ H, the sumu + v ∈ H.
(c) For each vector u ∈ H and eachscalar c, cu ∈ H.
Rn is a subspace of itself.
{0 } is a subspace, called the zero sub-space.
Span. Span{v1, . . . ,vp } is the sub-space spanned (or generated) byv1, . . . ,vp.
The column space of a matrix A is theset ColA of all linear combinations ofcolumns of A.
The null space of a matrix A is theset NulA of all solutions to the homo-geneous system
Ax = 0
Theorem 2.8.12: The null space.The null space of an m×n matrix A isa subspace of Rn.
Equivalently, the set of all solutions to asystem
Ax = 0
of m homogeneous linear equations in nunknowns is a subspace of Rn.
Basis of a subspace. A basis of a sub-space H of Rn is a linearly independentset in H that spans H.
Theorem 2.8.13: Pivot columns of amatrix A form a basis of ColA.
Dimension and rank [Lay 2.9]
Theorem: Given a basis b1, . . . ,bp in asubspace H of Rn, every vector u ∈ H isuniquely expressed as a linear combina-tion of b1, . . . ,bp.
Solving homogeneous systems.Finding a parametric solution of a sys-tem of homogeneous linear equations
Ax = 0
means to find a basis of NulA.
Coordinate system. Let
B = {b1, . . . ,bp }
be a basis for a subspace H.
For each x ∈ H, the coordinates of xrelative to the basis B are the weights
c1, . . . , cp
such that
x = c1b1 + · · ·+ cpbp.
The vector
[x]B =
c1...cp
is the coordinate vector of x.
Dimension. The dimension of anonzero subspace H, denoted by dimH,is the number of vectors in any basis ofH.
The dimension of the zero subspace {0}is defined to be 0.
Rank of a matrix. The rank of a ma-trix A, denoted by rankA, is the dimen-
MATH10212 • Linear Algebra B • Lecture 14 • Subspaces • Last change: 23 Apr 2018 26
sion of the column space ColA of A:
rankA = dim ColA
The Rank Theorem 2.9.14: If a ma-trix A has n columns,
rankA+ dim NulA = n.
The Basis Theorem 2.9.15: Let H bea p-dimensional subspace of Rn.
• Any linearly independent set of ex-actly p elements in H is automati-cally a basis for H.
• Also, any set of p elements of Hthat spansH is automatically a ba-sis of H.
For example, this means that the vectors100
,2
30
,4
56
form a basis of R3: there are three ofthem and they are linearly independent(the latter should be obvious to studentsat this stage).
The Invertible Matrix Theorem(continued) Let A be an n× n matrix.Then the following statements are eachequivalent to the statement that A is aninvertible matrix.
(m) The columns of A form a basis ofRn.
(n) ColA = Rn.
(o) dim ColA = n.
(p) rankA = n.
(q) NulA = {0}.
(r) dim NulA = 0.
MATH10212 • Linear Algebra B • Lecture 15 • Introduction to determinants • Last change: 23 Apr 2018 27
Lecture 15: Introduction to determinants [Lay 3.1]
The determinant detA of a square ma-trix A is a certain number assigned to thematrix; it is defined recursively, thatis, we define first determinants of matri-ces of sizes 1 × 1, 2 × 2, and 3 × 3, andthen supply a formula which expressesdeterminants of n× n matrices in termsof determinants of (n− 1)× (n− 1) ma-trices.
The determinant of a 1 × 1 matrixA =
[a11]
is defined simply as beingequal its only entry a11:
det[a11]
= a11.
The determinant of a 2 × 2 matrixis defined by the formula
det
[a11 a12a21 a22
]= a11a22 − a12a21.
The determinant of a 3× 3 matrix
A =
a11 a12 a13a21 a22 a23a31 a32 a33
is defined by the formula
det
a11 a12 a13a21 a22 a23a31 a32 a33
= a11 ·[a22 a23a32 a33
]− a12 ·
[a21 a23a31 a33
]+ a13 ·
[a21 a22a31 a32
]
Example. The determinant detA of thematrix
A =
2 0 70 1 01 0 4
equals
2 · det
[1 00 4
]+ 7 · det
[0 11 0
]which further simplifies as
2 · 4 + 7 · (−1) = 8− 7 = 1.
Submatrices. By definition, the sub-matrix Aij is obtained from the matrixA by crossing out row i and column j.
For example, if
A =
1 2 34 5 67 8 9
,then
A22 =
[1 37 9
], A31 =
[2 35 6
]
Recursive definition of determi-nant. For n > 3, the determinant ofan n × n matrix A is defined as the ex-pression
detA = a11 detA11 − a12 detA12 + · · ·+ (−1)1+na1n detA1n
=n∑
j=1
(−1)1+ja1j detA1j
MATH10212 • Linear Algebra B • Lecture 15 • Introduction to determinants • Last change: 23 Apr 2018 28
which involves the determinants ofsmaller (n−1)×(n−1) submatrices A1j,which, in their turn, can be evaluated bya similar formula wich reduces the cal-cualtions to the (n − 2) × (n − 2) case,and can be repeated all the way down todeterminants of size 2× 2.
Cofactors. The (i, j)-cofactor of A is
Cij = (−1)i+j detAij
Then
detA = a11C11 + a12C12 + · · ·+ a1nC1n
is cofactor expansion across the firstrow of A.
Theorem 3.1.1: Cofactor expan-sion. For any row i, the result of thecofactor expansion across the row i is thesame:
detA = ai1Ci1 + ai2Ci2 + · · ·+ ainCin
The chess board pattern of signs incofactor expansions. The signs
(−1)i+j
which appear in the formula for co-factors, form the easy-to-recognise andeasy-to-remember “chess board” pat-tern:
+ − + · · ·− + −+ − +...
. . .
Theorem 3.1.2: The determinantof a triangular matrix. If A is a tri-angular n × n matrix then detA is theproduct of the diagonal entries of A.
Proof: This proof contains more detailsthan the one given in the textbook. It
is based on induction on n, which, toavoid the use of “general” notation is il-lustrated by a simple example. Let
A =
a11 0 0 0 0a21 a22 0 0 0a31 a32 a33 0 0a41 a42 a43 a44 0a51 a52 a53 a54 a55
.All entries in the first row of A–with pos-sible exception of a11—are zeroes,
a12 = · · · = a15 = 0,
therefore in the formula
detA = a11 detA11 + · · ·+ a55 detA55
all summand with possible exception ofthe first one,
a11 detA11,
are zeroes, and therefore
detA = a11 detA11
= a11 det
a22 0 0 0a32 a33 0 0a42 a43 a44 0a52 a53 a54 a55
.But the smaller matrix A11 is also lowertriangle, and therefore we can concludeby induction that
det
a22 0 0 0a32 a33 0 0a42 a43 a44 0a52 a53 a54 a55
= a22 · · · a55
hence
detA = a11 · (a22 · · · a55)= a11a22 · · · a55
is the product of diagonal entries of A.
The basis of induction is the case n = 2;of course, in that case
det
[a11 0a21 a22
]= a11a22 − 0 · a21= a11a22
MATH10212 • Linear Algebra B • Lecture 15 • Introduction to determinants • Last change: 23 Apr 2018 29
is the product of the diagonal entries ofA.
Corollary. The determinant of a diag-onal matrix equals is the product of itsdiagonal elements:
det
d1 0 · · · 00 d2 0...
. . ....
0 · · · dn
= d1d2 · · · dn.
Corollary. The determinant of theidentity matrix equals 1:
det In = 1.
Corollary. The determinant of the zeromatrix equals 0:
det 0 = 0.
MATH10212 • Linear Algebra B • Lectures 15–16 • Properties of determinants • Last change: 23 Apr 2018 30
Lectures 15–16: Properties of determinants [Lay 3.2]
Theorem 3.2.3: Row Operations.Let A be a square matrix.
(a) If a multiple of one row of A isadded to another row to producea matrix B, then
detB = detA.
(b) It two rows of A are swapped toproduce B, then
detB = − detA.
(c) If one row of A is multiplied by kto produce B, then
detB = k detA.
Theorem 3.2.5: The determinant ofthe transposed matrix. If A is ann× n matrix then
detAT = detA.
Example. Let
A =
1 2 00 1 00 3 4
,then
AT =
1 0 02 1 30 0 4
and
detA = 1 · det
[1 03 4
]−2 · det
[0 00 4
]+0 · det
[0 10 3
]= 1 · 4− 2 · 0 + 0 · 0= 4.
Similarly
detAT = 1 · det
[1 30 4
]+ 0 + 0
= 1 · 4 = 4;
we got the same value.
Corollary: Cofactor expansionacross a column. For any column j,
detA = a1jC1j + a2jC2j + · · ·+ anjCnj
Example. We have already computedthe determinant of the matrix
A = det
2 0 70 1 01 0 4
by expansion across the first row. Nowwe do it by expansion across the thirdcolumn:
detA = 7 · (−1)1+3 · det
[0 11 0
]+0 · (−1)2+3 · det
[2 01 0
]+4 · (−1)3+3 · det
[2 00 1
]= −7− 0 + 8
= 1.
Proof: Columns of A are rows of AT
which has the same determinant as A.
Corollary: Column operations. LetA be a square matrix.
(a) If a multiple of one column of Ais added to another column to pro-duce a matrix B, then
detB = detA.
MATH10212 • Linear Algebra B • Lectures 15–16 • Properties of determinants • Last change: 23 Apr 2018 31
(b) It two columns of A are swappedto produce B, then
detB = − detA.
(c) If one column of A is multiplied byk to produce B, then
detB = k detA.
Proof: Column operations on A are rowoperation of AT which has the same de-terminant as A.
Computing determinants. In com-putations by hand, the quickest methodof computing determinants is to workwith both columns and rows:
• If a row or a column has a conve-nient scalar factor, take it out ofthe determinant.
• If convenient, swap two rows ortwo columns— but do not forgetto change the sign of the determi-nant.
• Adding to a row (column) scalarmultiples of other rows (columns),simplify the matrix to create a row(column) with just one nonzero en-try.
• Expand across this row (column).
• Repeat until you get the value ofthe determinant.
Example.
det
1 0 31 3 11 1 −1
C3−3C1= det
1 0 01 3 −21 1 −4
2 out of C3= 2 · det
1 0 01 3 −11 1 −2
−1 out of C3= −2 · det
1 0 01 3 11 1 2
expand
= −2 · 1 · (−1)1+1 · det
[3 11 2
]R1−3R2= −2 · det
[0 −51 2
]R1↔R2= −2 · (−1) · det
[1 20 −5
]= 2 · (−5)
= −10.
Example. Compute the determinant
∆ = det
1 1 1 1 01 1 1 0 11 1 0 1 11 0 1 1 10 1 1 1 1
Solution: Subtracting Row 1 fromRows 2, 3, and 4, we rearrange the de-
MATH10212 • Linear Algebra B • Lectures 15–16 • Properties of determinants • Last change: 23 Apr 2018 32
terminant as
∆ = det
1 1 1 1 00 0 0 −1 10 0 −1 0 10 −1 0 0 10 1 1 1 1
After expanding the determinant acrossthe first column, we get
∆ = 1 · (−1)1+1 · det
0 0 −1 10 −1 0 1−1 0 0 11 1 1 1
= det
0 0 −1 10 −1 0 1−1 0 0 11 1 1 1
R4+R3= det
0 0 −1 10 −1 0 1−1 0 0 10 1 1 2
After expansion across the 1st column
∆ = (−1) · (−1)3+1 · det
0 −1 1−1 0 11 1 2
= − det
0 −1 1−1 0 11 1 2
R3+R2= − det
0 −1 1−1 0 10 1 3
After expanding the determinant across
the first column we have
∆ = −(−1) · (−1)2+1 · det
[−1 11 3
]
= − det
[−1 11 3
]R3+R2= − det
[−1 10 4
]= −(−4)
= 4.
As an exercise, I leave you with this ques-tion: wouldn’t you agree that the deter-minant of the similarly constructed ma-trix of size n × n should be ±(n − 1)?But how can one determine the correctsign?
And one more exercise: you should nowbe able to instantly compute
det
1 2 3 4 01 2 3 0 51 2 0 4 51 0 3 4 50 2 3 4 5
.
Theorem 3.2.4: Invertible matri-ces. A square matrix A is invertible ifand only if
detA 6= 0.
Recall that this theorem has been al-ready known to us in the case of 2 × 2matrices A.
Example. The matrix
A =
2 0 70 1 01 0 4
has the determinant
detA = 2 · det
[1 00 4
]+ 7 · det
[0 11 0
]= 2 · 4 + 7 · (−1)
= 8− 7 = 1
MATH10212 • Linear Algebra B • Lectures 15–16 • Properties of determinants • Last change: 23 Apr 2018 33
hence A is invertible. Indeed,
A−1 =
4 0 −70 1 0−1 0 2
—check!
Theorem 3.2.6: MultiplicativityProperty. If A and B are n × n ma-trices then
detAB = (detA) · (detB)
Example. Take
A =
[1 03 2
]and B =
[1 50 1
],
then
detA = 2 and detB = 1.
But
AB =
[1 03 2
] [1 50 1
]=
[1 53 17
]
and
det(AB) = 1 · 17− 5 · 3 = 2,
which is exactly the value of
(detA) · (detB).
Corollary. If A is invertible,
detA−1 = (detA)−1.
Proof. Set B = A−1, then
AB = I
and Theorem 3.2.6 yields
detA · detB = det I = 1.
Hence
detB = (detA)−1.
Cramer’s Rule [Lay 3.3]
Cramer’s Rule is an explicit (closed) for-mula for solving systems of n linear equa-tions with n unknowns and nonsigular(invertible) matrix of coefficients. It hasimportant theoretical value, but is un-suitable for practical application.
For any n×n matrix A and any b ∈ Rn,denote by Ai(b) the matrix obtainedfrom A by replacing column i by the vec-tor b:
Ai(b) =[a1 · · · ai−1 b ai+1 · · · an
].
Theorem 3.3.7: Cramer’s Rule. LetA be an invertible n × n matrix. Forany b ∈ Rn, the unique solution x of thelinear system
Ax = b
is given by
xi =detAi(b)
detA, i = 1, 2, . . . , n.
For example, for a system
x1 + x2 = 3
x1 − x2 = 1
Cramer’s rule gives the answer
x1 =
det
[3 11 −1
]det
[1 11 −1
] =−4
−2= 2
x2 =
det
[1 31 1
]det
[1 11 −1
] =−2
−2= 1
MATH10212 • Linear Algebra B • Lectures 15–16 • Properties of determinants • Last change: 23 Apr 2018 34
Theorem 3.3.8: An Inverse For-mula. Let A be an invertible n× n ma-trix. Then
A−1 =1
detA
C11 C21 · · · Cn1
C12 C22 · · · Cn2...
......
C1n C2n · · · Cnn
where cofactors Cji are given by
Cji = (−1)j+i detAji
and Aji is the matrix obtained from Aby deleting row j and column i.
Notice that for a 2× 2 matrix
A =
[a bc d
]
the cofactors are
C11 = (−1)1+1 · d = d
C21 = (−1)2+1 · b = −bC12 = (−1)1+2 · c = −cC22 = (−1)2+2 · a = a,
and we get the familiar formula
A−1 =1
detA
[d −b−c a
]
=1
ad− bc
[d −b−c a
].
MATH10212 • Linear Algebra B • Lecture 17 • Eigenvalues and eigenvectors • Last change: 23 Apr 2018 35
Lecture 17: Eigenvalues and eigenvectors
Quiz
What is the value of this determinant?
∆ = det
1 2 30 2 31 2 6
Quiz
Is Γ positive, negative, or zero?
Γ = det
0 0 710 93 0
87 0 0
Eigenvectors. An eigenvector of ann × n matrix A is a nonzero vector xsuch that
Ax = λx
for some scalar λ.
Eigenvalues. A scalar λ is called aneigenvalue of A if there is a non-trivialsolution x of
Ax = λx;
x is called an eigenvector correspond-ing to λ.
Eigenvalue. A scalar λ is an eigenvalueof A iff the equation
(A− λI)x = 0
has a non-trivial solution.
Eigenspace. The set of all solutions of
(A− λI)x = 0
is a subspace of Rn called theeigenspace of A corresponding to theeigenvalue λ.
Theorem: Linear independence ofeigenvectors. If v1, . . . ,vp are eigen-vectors that correspond to distinct eigen-values of λ1, . . . , λp of an n×n matrix A,then the set
{v1, . . . ,vp }
is linearly independent.
MATH10212 • Linear Algebra B • Lecture 17 • The characteristic equation • Last change: 23 Apr 2018 36
Lecture 18: The characteristic equation
Characteristic polynomial
The polynomial
det(A− λI)
in variable λ is characteristic polyno-mial of A.
For example, in the 2× 2 case
det
[a− λ bc d− λ
]= λ2−(a+d)λ+(ad−bc).
det(A− λI) = 0
is the characteristic equation for A.
Characterisation of eigenvalues
Theorem. A scalar λ is an eigenvalueof an n × n matrix A if and only if λsatisfies the characteristic equation
det(A− λI) = 0
Zero as an eigenvalue. λ = 0 is aneigenvalue of A if and only if detA = 0.
The Invertible Matrix Theorem(continued). An n× n matrix A is in-vertible iff :
(s) The number 0 is not an eigenvalueof A.
(t) The determinant of A is not zero.
Theorem: Eigenvalues of a triangu-lar matrix. The eigenvalues of a trian-gular matrix are the entries on its maindiagonal.
Proof. This immediately follows fromthe fact the the determinant of a trian-gular matrices is the product of its diag-onal elements. Therefore, for example,for the 3× 3 triangular matrix
A =
d1 a b0 d2 c0 0 d3
its characteristic polynomial det(A−λI)equals
det
d1 − λ a b0 d2 − λ c0 0 d3 − λ
and therefore
det(A− λI) = (d1 − λ)(d2 − λ)(d3 − λ)
has roots (eigenvalues of A)
λ = d1, d2, d3.
MATH10212 • Linear Algebra B • Lectures 18–21 • Diagonalisation 37
Lectures 19–21: Diagonalisation • Last change: 23
Apr 2018
Similar matrices. A is similar to B ifthere exist an invertible matrix P suchthat
A = PBP−1.
Example:
Occasionally the similarity relation is de-noted by symbol ∼.
Similarity is an equivalence relation: itis
• reflexive: A ∼ A
• symmetric: A ∼ B implies B ∼ A
• transitive: A ∼ B and B ∼ C im-plies A ∼ C.
Conjugation. Operation
AP = P−1AP
is called conjugation of A by P .
Properties of conjugation
• IP = I
• (AB)P = APBP ; as a corollary:
• (A+B)P = AP +BP
• (c · A)P = c · AP
• (Ak)P = (AP )k
• (A−1)P = (AP )−1
• (AP )Q = APQ
Theorem: Similar matrices havethe same characteristic polynomial.If matrices A and B are similar then theyhave the same characteristic polynomialsand eigenvalues.
Diagonalisable matrices. A squarematrix A is diagonalisable if it is simi-lar to a diagonal matrix, that is,
A = PDP−1
for some diagonal matrix D and someinvertible matrix P .
Of course, the diagonal entries of D areeigenvalues of A.
The Diagonalisation Theorem. Ann × n matrix A is diagonalisable iff Ahas n linearly independent eigenvectors.In fact,
A = PDP−1
where D is diagonal iff the columns ofP are n linearly independent eigenvec-tors of A.
Theorem: Matrices with all eigen-values distinct. An n× n matrix withn distinct eigenvalues is diagonalisable.
Example: Non-diagonalisable ma-trices
A =
[2 10 2
]has repeated eigenvalues λ1 = λ2 = 2.
A is not diagonalisable. Why?
MATH10212 • Linear Algebra B • Lecture • Vector spaces • Last change: 23 Apr 2018 38
Lecture 22: Vector spaces
A vector space is a non-empty spaceof elements (vectors), with operations ofaddition and multiplication by a scalar.
By definition, the following holds for allu,v,w in a vector space V and for allscalars c and d.
1. The sum of u and v is in V .
2. u + v = v + u.
3. (u + v) + w = u + (v + w).
4. There is zero vector 0 such thatu + 0 = u.
5. For each u there exists −u suchthat u + (−u) = 0.
6. The scalar multiple cu is in V .
7. c(u + v) = cu + cv.
8. (c+ d)u = cu + du.
9. c(du) = (cd)u.
10. 1u = u.
Examples. Of course, the most impor-tant example of vector spaces is providedby standard spaces Rn of column vectorswith n components, with the usual op-erations of component-wise addition andmultiplication by scalar.
Another example is the set Pn of poly-nomials of degree at most n in variablex,
Pn = {a0 + a1x+ · · ·+ anxn}
with the usual operations of addition ofpolynomials multiplication by constant.
And here is another example: the setC[0, 1] of real-valued continuous func-tions on the segment [0, 1], with theusual operations of addition of functionsand multiplication by constant.
Definitions
Most concepts introduced in the previ-ous lectures for the vector spaces Rn canbe transferred to arbitrary vector spaces.Let V be a vector space.
Given vectors v1,v2, . . . ,vp in V andscalars c1, c2, . . . , cp, the vector
y = c1v1 + · · · cpvp
is called a linear combina-tion of v1,v2, . . . ,vp with weightsc1, c2, . . . , cp.
If v1, . . . ,vp are in V , then the set ofall linear combination of v1, . . . ,vp is de-noted by Span{v1, . . . ,vp} and is calledthe subset of V spanned (or gener-ated) by v1, . . . ,vp.
That is, Span{v1, . . . ,vp} is the collec-tion of all vectors which can be writtenin the form
c1v1 + c2v2 + · · ·+ cpvp
with c1, . . . , cp scalars.
Span{v1, . . . ,vp} is a vector subspaceof V , that is, it is closed with respect toaddition of vectors and multiplying vec-tors by scalars.
V is a subspace of itself.
{0 } is a subspace, called the zero sub-space.
An indexed set of vectors
{v1, . . . ,vp }
in V is linearly independent if the vec-tor equation
x1v1 + · · ·+ xpvp = 0
has only trivial solution.
MATH10212 • Linear Algebra B • Lecture • Vector spaces • Last change: 23 Apr 2018 39
The set{v1, . . . ,vp }
in V is linearly dependent if there ex-ist weights c1, . . . , cp, not all zero, suchthat
c1v1 + · · ·+ cpvp = 0
Theorem: Characterisation of lin-early dependent sets. An indexed set
S = {v1, . . . ,vp }
of two or more vectors is linearly depen-dent if and only if at least one of thevectors in S is a linear combination ofthe others.
A basis of V is a linearly independentset which spans V . A vector space Vis called finite dimensional if it hasa finite basis. Any two bases in a fi-nite dimensional vector space have thesame number of vectors; this number iscalled the dimension of V and is de-noted dimV .
If B = (b1, . . . ,bn) is a basis of V , everyvector x ∈ V can be written, and in aunique way, as a linear combination
x = x1b1 + · · ·xnbn;
the scalars x1, . . . , xn are called coordi-nates of x with respect to the basis B.
MATH10212 • Linear Algebra B • Lectures 23 and 24 • Linear transformations • Last change: 23 Apr 2018 40
Lectures 23–24: Linear transformations of abstract
vector spaces
A transformation T from a vectorspace V to a vector space W is a rulethat assigns to each vector x in V a vec-tor T (x) in W .
The set V is called the domain of T , theset W is the codomain of T .
A transformation
T : V −→ W
is linear if:
• T (u + v) = T (u) + T (v) for allvectors u,v ∈ V ;
• T (cu) = cT (u) for all vectors uand all scalars c.
Properties of linear transforma-tions. If T is a linear transformationthen
T (0) = 0
and
T (cu + dv) = cT (u) + dT (v).
The identity transformation of V is
V −→ V
x 7→ x.
The zero transformation of V is
V −→ V
x 7→ 0.
If
T : V −→ W
is a linear transformation, its kernel
KerT = {x ∈ V : T (x) = 0}
is a subspace of V , while its image
ImT = {y ∈ W : T (x) = y for some x ∈ V }
is a subspace of W .
MATH10212 • Linear Algebra B • Lecture 25 • Symmetric matrices and inner product • Last change: 23 Apr 2018 41
Lecture 25: Symmetric matrices and inner product
Symmetric matrices. A matrix A issymmetric if AT = A.
An example:
A =
1 2 32 3 43 4 5
.Notice that symmetric matrices are nec-essarily square.
Inner (or dot) product. For vectorsu,v ∈ Rn their inner product (alsocalled scalar product or dot product)u �v is defined as
u �v = uTv
=[u1 u2 · · ·un
]v1v2...vn
= u1v1 + u2v2 + · · ·+ unvn
Properties of dot product
Theorem 6.1.1. Let u,v,w ∈ Rn, andc be a scalar. Then
(a) u �v = v �u
(b) (u + v) �w = u �w + v �w
(c) (cu) �v = c(u �v)
(d) u �u > 0, and u �u = 0 if and onlyif u = 0
Properties (b) and (c) can be combinedas
(c1u1 + · · ·+ cpup) �w = c1(u1�w) + · · ·+ cp(up
�w)
The length of a vector v is defined as
‖v‖ =√
v �v =√v21 + v22 + · · ·+ v2n
In particular,
‖v‖2 = v �v.
We call v a unit vector if ‖v‖ = 1.
Orthogonal vectors. Vectors u and vare orthogonal to each other if
u �v = 0
Theorem. Vectors u and v are orthog-onal if and only if
‖u‖2 + ‖v‖2 = ‖u + v‖2.
That is, vectors u and v are orthogonalif and only if the Pythagoras Theoremholds for the triangle formed by u andv as two sides (so that its third side isu + v).
Proof is a straightforward computation:
‖u + v‖2 = (u + v) � (u + v)
= u �u + u �v + v �u + v �v
= u �u + u �v + u �v + v �v
= u �u + 2u �v + v �v
= ‖u‖2 + 2u �v + ‖v‖2.
MATH10212 • Linear Algebra B • Lecture 25 • Symmetric matrices and inner product • Last change: 23 Apr 2018 42
Hence
‖u + v‖2 = ‖u‖2 + ‖v‖2 + 2u �v,
and
‖u + v‖2 = ‖u‖2 + ‖v‖2 iff u �v = 0.
�
Orthogonal sets. A set of vectors
{u1, . . . ,up }
in Rn is orthogonal if
ui�uj = 0 whenever i 6= j.
Theorem 6.2.4. If
S = {u1, . . . ,up }
is an orthogonal set of non-zero vectorsin Rn then S is linearly independent.
Proof. Assume the contrary that Sis linearly dependent. This means thatthere exist scalars c1, . . . , cp, not all ofthem zeroes, such that
c1u1 + · · ·+ cpup = 0.
Forming the dot product of the left handside and the right hand side of this equa-tion with ui, we get
(c1u1 + · · ·+ cpup) �ui = 0 �ui = 0
After opening the brackets we have
c1u1�ui + · · ·+ ci−1ui−1 + ciui
�ui + ci+1ui+1�ui + · · ·+ cpup
�ui = 0.
In view of orthogonality of the set S, alldot products on the left hand side, withthe exception of ui
�u, equal 0. Hencewhat remains from the equality is
ciui�ui = 0
Since ui is non-zero,
ui�ui 6= 0
and therefore ci = 0. This argumentworks for every index i = 1, . . . , p. Weget a contradition with our assumptionthat one of ci is non-zero. �
An orthogonal basis for Rn is a basiswhich is also an orthogonal set.
For example, the two vectors[11
],
[1−1
]form an orthogonal basis of R2 (check!).
Theorem: Eigenvectors of symmet-ric matrices. Let
v1, . . . ,vp
be eigenvectors of a symmetric matrix Afor pairwise distinct eigenvalues
λ1, . . . , λp.
Then
{v1, . . . ,vp }
is an orthogonal set.
Proof. We need to prove the following:
If u and v are eigenvectorsfor A for eigenvalues λ 6= µ,then
u �v = 0.
For a proof, consider the following se-
MATH10212 • Linear Algebra B • Lecture 25 • Symmetric matrices and inner product • Last change: 23 Apr 2018 43
quence of equalities:
λu �v = (λu) �v
= (Au) �v
= (Au)Tv
= (uTAT )v
= (uTA)v
= uT (Av)
= uT (µv)
= µuTv
= µu �v.
Henceλu �v = µu �v
and(λ− µ)u �v = 0.
Since λ− µ 6= 0, we have u �v = 0. �
Corollary. If A is an n × n symmetricmatrix with n distinct eigenvalues
λ1, . . . , λn
then the corresponding eigenvectors
v1, . . . ,vn
form an orthogonal basis of Rn.
An orthonormal basis in Rn is an or-thogonal basis
u1, . . . ,un
made of unit vectors,
‖ui‖ = 1 for all i = 1, 2, . . . , n.
Notice that this definition can reformu-lated as
ui�uj =
{1, if i = j0, if i 6= j
.
Theorem: Coordinates with re-spect to an orthonormal basis. If
{u1, . . . ,un }
is an orthonormal basis and v a vectorin Rn, then
v = x1u1 + · · ·+ xnun
where
xi = v �ui, for all i = 1, 2, . . . , n.
Proof. Let
v = x1u1 + · · ·+ xnun,
then, for i = 1, 2, . . . , n,
v �ui = x1u1�ui + · · ·+ xnun
�ui
= xiui�ui
= xi.
Theorem: Eigenvectors of symmet-ric matrices. Let A be a symmetricn× n matrix with n distinct eigenvalues
λ1, . . . , λn.
Then Rn has an orthonormal basis madeof eigenvectors of A.
MATH10212 • Linear Algebra B • Lecture 26 • Orthogonal matrices • Last change: 23 Apr 2018 44
Lecture 26: Orthogonal matrices
A square matrix A is said to be orthog-onal if
AAT = I.
Some basic properties of orthogonal ma-trices:
If A is orthogonal then
• AT is also orthogonal.
• A is invertible and A−1 = AT .
• detA = ±1.
• The identity matrix is orthogonal.
An example of an orthogonal matrix:
A =
[√22
√22√
22−√22
]
Theorem: Characterisation of Or-thogonal Matrices. For a square n×nmatrix A, the following conditions areequivalent:
(a) A is orthogonal .
(b) Columns of A form an orthogonalbasis of Rn.
MATH10212 • Linear Algebra B • Lectures 25 and 26 • Inner product spaces • Last change: 23 Apr 2018 45
Lectures 25 and 26: Inner product spaces
Inner (or dot) product is a functionwhich associate, with each pair of vec-tors u and v in a vector space V a realnumber denoted
u �v,
subject to the following axioms:
1. u �v = v �u
2. (u + v) �w = u �w + v �w
3. (cu) �v = c(u �v)
4. u �u > 0 and u �u = 0 if and onlyif u = 0
Inner product space. A vector spacewith an inner product is called an innerproduct space.
Example: School Geometry. The or-dinary Euclidean plane of school geome-try is an inner product space:
• Vectors: directed segments start-ing at the origin O
• Addition: parallelogram rule
• Product-by-scalar: stretching thevector by factor of c.
• Dot product:
u �v = ‖u‖‖v‖ cos θ,
where θ is the angle between vec-tors u and v.
Example: C[0, 1]. The vector spaceC[0, 1] of real-valued continuous func-tions on the segment [0, 1] becomes aninner product space if we define innerproduct by the formula
f � g =
∫ 1
0
f(x)g(x)dx.
In this example, the tricky bit is to showthat the dot product on C[0, 1] satisfiesthe axiom:
u �u > 0 and u �u = 0 if andonly if u = 0,
that is, if f is a continuous function on[0, 1] and ∫ 1
0
f(x)2dx = 0
then f(x) = 0 for all x ∈ [0, 1]. This re-quires the use of properties of continuousfunctions; since we study linear algebra,not analysis, I am leaving it to the read-ers as an exercise.
Length. The length of the vector u isdefined as
‖u‖ =√
u �u.
Notice that
‖u‖2 = u �u,
and‖u‖ = 0 ⇔ u = 0.
Orthogonality. We say that vectors uand v are orthogonal if
u �v = 0.
The Pythagoras Theorem. Vectorsu and v are orthogonal if and only if
‖u‖2 + ‖v‖2 = ‖u + v‖2.
Proof is a straightforward computation,exactly the same as in Lectures 25–27:
‖u + v‖2 = (u + v) � (u + v)
= u �u + u �v + v �u + v �v
= u �u + u �v + u �v + v �v
= u �u + 2u �v + v �v
= ‖u‖2 + 2u �v + ‖v‖2.
MATH10212 • Linear Algebra B • Lectures 25 and 26 • Inner product spaces • Last change: 23 Apr 2018 46
Hence
‖u + v‖2 = ‖u‖2 + ‖v‖2 + 2u �v,
and
‖u + v‖2 = ‖u‖2 + ‖v‖2
if and only if
u �v = 0.
�
The Cauchy-Schwarz Inequality. Inthe Euclidean plane,
u �v = ‖u‖‖v‖ cos θ
hence|u �v| 6 ‖u‖‖v‖.
The following Theorem shows that thisis is a general property of inner productspaces.
Theorem: The Cauchy-Schwarz In-equality. In any inner product space,
|u �v| 6 ‖u‖‖v‖.
The Cauchy-Schwarz Inequality:an example. In the vector space Rn
with the dot product of
u =
u1...un
and v =
v1...vn
defined as
u �v = uTv = u1v1 + · · ·+ unvn,
the Cauchy-Schwarz Inequality becomes
|u1v1 + · · ·+ unvn| 6√u21 + · · ·+ u2n ·
√v21 + · · ·+ v2n.
or, which is its equivalent form,
(u1v1 + · · ·+ unvn)2 6 (u21 + · · ·+ u2n) · (v21 + · · ·+ v2n).
The Cauchy-Schwarz Inequality:one more example. In the inner prod-
uct space C[0, 1] the Cauchy-Schwarz In-equality takes the form
∣∣∣∣∫ 1
0
f(x)g(x)dx
∣∣∣∣ 6√∫ 1
0
f(x)2dx ·
√∫ 1
0
g(x)2dx,
or, equivalently, (∫ 1
0
f(x)g(x)dx
)2
6
(∫ 1
0
f(x)2dx
)·(∫ 1
0
g(x)2dx
).
Proof of the Cauchy-Schwarz In-equality given here is different from theone in the textbook by Lay. Our proof isbased on the following simple fact fromschool algebra:
Let
q(t) = at2 + bt+ c
be a quadratic function invariable t with the property
MATH10212 • Linear Algebra B • Lectures 25 and 26 • Inner product spaces • Last change: 23 Apr 2018 47
thatq(t) > 0
for all t. Then
b2 − 4ac 6 0.
Now consider the function q(t) in vari-able t defined as
q(t) = (u + tv) � (u + tv)
= u �u + 2tu �v + t2v �v
= ‖u‖2 + (2u �v)t+ (‖v‖2)t2.
Notice q(t) is a quadratic function in tand
q(t) > 0.
Hence
(2u �v)2 − 4‖u‖2‖v‖2 6 0
which can be simplified as
|u �v| 6 ‖u‖‖v‖.
�
Distance. We define distance betweenvectors u and v as
d(u,v) = ‖u− v‖.
It satisfies axioms of metric:
1. d(u,v) = d(v,u).
2. d(u,v) > 0.
3. d(u,v) = 0 iff u = v.
4. The Triangle Inequality:
d(u,v) + d(v,w) > d(u,w).
Axioms 1–3 immediately follow from theaxioms for dot product, but the TriangleInequality requires some attention.
Proof of the Triangle Inequality. Itsuffices to prove
‖u + v‖ 6 ‖u‖+ ‖v‖,
or‖u + v‖2 6 (‖u‖+ ‖v‖)2,
or
(u+v) � (u+v) 6 ‖u‖2+2‖u‖‖v‖+‖v‖2,
or
u �u+2u �v+v �v 6 u �u+2‖u‖‖v‖+v �v,
or2u �v 6 2‖u‖‖v‖
But this is the Cauchy-Schwarz Inequal-ity. Now observe that all rearrange-ments are reversible, hence the TriangleInequality holds. �
MATH10212 • Linear Algebra B • Lecture 27 •Orthogonalisation and the Gram-Schmidt Process • Last change: 23 Apr 201848
Lecture 27: Orthogonalisation and the Gram-Schmidt
Process
Orthogonal basis. A basis v1, . . . ,vn
in the inner product space V is calledorthogonal if vi
�vj = 0 for i 6= j.
Coordinates in respect to an or-thogonal basis:
u = c1v1 + · · ·+ cnvn
iff
ci =u �vi
vi�vi
Orthonormal basis. A basisv1, . . . ,vn in the inner product space V
is called orthonormal if it is orthogonaland ‖vi‖ = 1 for all i.
Equivalently,
vi�vj =
{1 if i = j0 if i 6= j
.
Coordinates in respect to an or-thonormal basis:
u = c1v1 + · · ·+ cnvn
iffci = u �vi
The Gram-Schmidt Orthogonalisation Process makes an orthogonal basis from abasis x1, . . . ,xp:
v1 = x1
v2 = x2 −x2�v1
v1�v1
v1
v3 = x3 −x3�v1
v1�v1
v1 −x3�v2
v2�v2
v2
...
vp = xp −xp�v1
v1�v1
v1 −xp�v2
v2�v2
v2 − · · · −xp�vp−1
vp−1 �vp−1vp−1
Main Theorem about inner prod-uct spaces. Every finite dimensionalinner product space has an orthonormalbasis and is isomorphic to on of Rn with
the standard dot product
u �v = uTv =[u1 · · · un
] v1...vn
.
MATH10212 • Linear Algebra B • Lectures 28–30 • Revision • Last change: 23 Apr 2018 49
Lectures 28–30: Revision, Systems of Linear Equa-
tions
Systems of linear equations
Here A is an m× n matrix.
We associate with A systems of simul-taneous linear equations
Ax = b,
where b ∈ Rm and
x =
x1...xn
is the column vector of unknowns.
Especially important is the homoge-neous system of simultaneous linearequations
Ax = 0.
The set
null A = {x ∈ Rn : Ax = 0}
is called the null space of A; it is a vec-tor subspace of Rn and coincides withthe solution space of the homogeneoussystem Ax = 0.
If a1, . . . , an are columns of A, so that
A =[a1 · · · an
],
then the column space of A is definedas
Col A = {c1a1 + · · ·+ cnan : ci ∈ R}= Span {a1, . . . , an}
and equals to the span of the system ofvectors a1, . . . , an, that is, the set of allpossible linear combinations of vec-tors a1, . . . , an.
The column space of A has another im-portant characterisation:
Col A = {b ∈ Rm : Ax = b has a solution}.
Assume now that A is a square n×nmatrix.
If we associate with A the linear trans-formation
TA : Rn −→ Rn
v 7→ Av
then the column space of A gets yet an-other interpretation: it is the image ofthe linear transformation TA:
Col A = Im TA
= {v ∈ Rn : ∃v ∈ Rn s.t. TA(w) = v},
and the null space of A is the kernel ofTA:
null A = kerTA
= {v ∈ R : TA(v) = 0}.
MATH10212 • Linear Algebra B • Lectures 28–30 • Revision • Last change: 23 Apr 2018 50
Linear dependence
If a1, . . . , ap are vectors in a vector spaceV , an expression
c1a1 + · · ·+ cpap, ci ∈ Ris called a linear combination of thevectors a1, . . . , ap, and coefficients ci arecalled weights in this linear combina-tion. If
c1a1 + · · ·+ cpap = 0
we say that weights c1, . . . , cp de-fine a linear dependency of vectorsa1, . . . , ap. We always have a trivial(or zero) dependency, in which allci = 0. We say that vectors a1, . . . , ap
are linearly dependent if the admit anon-zero (or non-trivial) linear depen-dency, that is, a dependency in which atleast one weight ci 6= 0.
Let us arrange the weights c1, . . . , cp in acolumn
c =
c1...cp
and form a matrix A using vectorsa1, . . . , ap as columns, then
c1a1 + · · ·+ cpap =[a1 · · · ap
] c1...cp
= Ac
and linear dependencies between vec-tors a1, . . . , ap are nothing else but solu-tions of the homogeneous system of lin-ear equation
Ax = 0.
Therefore vectors a1, . . . , ap are linearlydependent if and only if the system
Ax = 0
has a non-zero solution.
Elementary row transformation
Elementary row transformations
Ri ↔ Rj, i 6= j
Ri ← Ri + λRj, i 6= j
Ri ← λRi, λ 6= 0
performed on a matrix A amount to mul-tiplication of A on the left by corre-sponding elementary matrices,
A← EA,
and therefore do not change dependen-cies between columns:
Ac = 0⇔ (EA)c = 0.
Therefore if certain columns of A werelinearly dependent (respectively, inde-pendent) then the same columns of EAremain linearly dependent (respectively,independent)
Rank of a matrix
We took it without proof that if U is avector subspace of Rn, then the maximalsystems of linearly independent subsetsin U contain the same number of vectors.This number is called the dimension of
U and is denoted dimU .
A maximal system of linearly indepen-dent vectors in U is called a basis of U .
Each basis of U contains dimU vectors.
MATH10212 • Linear Algebra B • Lectures 28–30 • Revision • Last change: 23 Apr 2018 51
An important theorem:
dim null A + dim Col A = n
dim Col A is called the rank of A and isdenoted
rankA = dim Col A.
Therefore
dim null A + rank A = n
.
Theorem. If rankA = p then amongcolumns of A there are p linearly inde-pendent columns
Theorem. Since elementary row trans-formations of A do not change depen-dency of columns, e=elementary row op-eration do not change rankA.