7/28/2019 Definitions 060208

1/21

Denitions of Linear Algebra Terms

In order to learn and understand mathematics, it is necessary to understandthe meanings of the terms (vocabulary words) that are used. This documentcontains denitions of some of the important terms used in linear algebra.All of these denitions should be memorized (and not just memorized butunderstood). In addition to the denitions that are given here, we also includestatements of some of the most important results (theorems) of linear algebra.The proofs of these theorems are not included here but will be given in class.

There will be a question on each of the exams that asks you to statesome denitions. It will be required that you state these denitions veryprecisely in order to receive credit for the exam question. (Seemingly very

small things such as writing the word or instead of the word and, forexample, can render a denition incorrect.) The denitions given here areorganized according to the section number in which they appear in DavidLays textbook.

Here is a suggestion for how to study linear algebra (and other subjectsin higher level mathematics):

1. Carefully read a section in the textbook and also read your class notes.Pay attention to denitions of terms and examples and try to under-stand each concept along the way as you read it. You dont have tomemorize all of the denitions upon rst reading. You will memorize

them gradually as you work the homework problems.

2. Work on the homework problems. As you work on the homework prob-lems, you should nd that you will need to continuously refer back tothe denitions and to the examples given in the textbook. (This ishow you will naturally come to memorize the denitions.) You willnd that you have to spend a considerable amount of time working onhomework. When you are done, you will probably have read throughthe section and your course notes several times and you will probablyhave memorized the denitions. (There are some very good TrueFalsequestions among the homework problems that are good tests of your

understanding of the basic terms and concepts. Such TrueFalse ques-tions will also appear on the exams given in this course.)

3. Test yourself. After you are satised that you have a good under-standing of the concepts and that can apply those concepts in solving

1

7/28/2019 Definitions 060208

2/21

problems, try to write out the denitions without referring to your book

or notes. You might then also try to work some of the supplementaryproblems that appear at the end of each chapter of the textbook.

4. One other tip is that it is a good idea to read a section in the textbookand perhaps even get started on working on the homework problemsbefore we go over that section in class. This will better prepare you totake notes and ask questions when we do go over the material in class.It will alert you to certain aspects of the material that you might notquite understand upon rst reading and you can then ask questionsabout those particular aspects during class.

1.1: Systems of Linear Equations

Denition 1 A linear equation in the variablesx1; x2; : : : ; xn is an equa-tion of the form

a1x1 + a2x2 + : : : + anxn = b

wherea1; a2; : : : ; an and b are real or complex numbers (that are often givenin advance).

Example 24x + 4y 7z + 8w = 8

is a linear equation in the variables x;y;z; andw.

3x + 5y2 = 6

is not a linear equation (because of the appearance of y2).

Denition 3 Suppose that m and n are positive integers. A system oflinear equations in the variables x1; x2; : : : ; xn is a set of equations of the

form

a11x1 + a12x2 + : : : + a1nxn = b1

a21x1 + a22x2 + : : : + a2nxn = b2...

am1x1 + am2x2 + : : : + amnxn = bm

2

7/28/2019 Definitions 060208

3/21

whereaij (i = 1; : : : ;m, j = 1; : : : n) andbi (i = 1; : : : ;m) are real or complex

numbers (that are often given in advance). This system of linear equationsis said to consist ofm equations withn unknowns.

Example 4

2x1 + 2x2 5x3 + 4x4 2x5 = 4

4x1 + x2 5x4 = 1

3x1 2x2 + 4x4 2x5 = 4

is a system of three linear equations in ve unknowns. (The unknowns orvariables of the system arex1; x2; x3; x4; andx5.)

Denition 5 A solution of the linear system

a11x1 + a12x2 + : : : + a1nxn = b1

a21x1 + a22x2 + : : : + a2nxn = b2...

am1x1 + am2x2 + : : : + amnxn = bm

is an orderedntuple of numbers (s1; s2; : : : ; sn) such that all of the equationsin the system are true when the substitution x1 = s1; x2 = s2; : : : ; xn = sn ismade.

Example 6 The5tuple 0;1;125

; 0; 3 is a solution of the linear system2x1 + 2x2 5x3 + 4x4 2x5 = 4

4x1 + x2 5x4 = 1

3x1 2x2 + 4x4 2x5 = 4

because all equations in the system are true when we make the substitutions

x1 = 0

x2 = 1

x3 = 12

5

x4 = 0x5 = 3.

The 5tuple

1; 3; 35; 0; 1

2

is also a solution of the system (as the reader can

check).

3

7/28/2019 Definitions 060208

4/21

Denition 7 Thesolution set of a system of linear equations is the set of

all solutions of the system.

Denition 8 Two linear systems are said to be equivalent to each other ifthey have the same solution set.

Denition 9 A linear system is said to be consistent if it has at least onesolution and is otherwise said to be inconsistent.

Example 10 The linear system

2x1 + 2x2 5x3 + 4x4 2x5 = 4

4x1

+ x2

5x4

= 13x1 2x2 + 4x4 2x5 = 4

is consistent because it has at least one solution. (It in fact has innitelymany solutions.) However, the linear system

x + 2y = 5

x + 2y = 9

is obviously inconsistent.

Denition 11 Suppose thatm andn are positive integers. Anmn matrixis a rectangular array of numbers withm rows andn columns. Matrices areusually denoted by capital letters. If m = 1 or n = 1, then the matrix isreferred to as a vector. In particular, if m = 1, then the matrix is a rowvector of dimension n, and if n = 1 then the matrix is a column vector ofdimension m. Vectors are usually denoted by lower case boldface letters or(when handwritten) as lower case letters with arrows above them.

Example 12

A =

2

66664

8 10 6 85 7 6 6

0 5 10 11 3 10 54 8 1 10

3

77775

is a 5 4 matrix.r = !r =

2 3 1 9 8

4

7/28/2019 Definitions 060208

5/21

is a row vector of dimension 5.

c = !c =

24 45

10

35

is a column vector of dimension 3.

Remark 13 If A is an m n matrix whose columns are made up of thecolumn vectors c1; c2; : : : ;cn, thenA can be written as

A = [c1 c2 cn] .

Likewise, if A is made up of the row vectors r1; r2; : : : ; rm, then A can bewritten as

A =

26664

r1

r2...rm

37775 .

Example 14 The5 4 matrix

A =266664

8 10 6 85 7 6 60 5 10 11 3 10 54 8 1 10

377775

can be written asA = [c1 c2 c3 c4]

where

c1 =

2

66664

85

014

3

77775 , c2 =

2

66664

107

538

3

77775 , c3 =

2

66664

66

10101

3

77775 , c4 =

2

66664

86

1510

3

77775

5

7/28/2019 Definitions 060208

6/21

or can be written as

A =

266664

r1

r2

r3

r4

r5

377775

where

r1 =

8 10 6 8

r2 =5 7 6 6

r3 =

0 5 10 1

r4 =

1 3 10 5

r5 =

4 8 1 10

.

Denition 15 For a given linear system of equations

a11x1 + a12x2 + : : : + a1nxn = b1

a21x1 + a22x2 + : : : + a2nxn = b2...

am1x1 + am2x2 + : : : + amnxn = bm,

them n matrix

A =

26664

a11 a12 a1na21 a22 a2n

......

. . ....

am1 am2 amn

37775

is called the coecient matrix of the system and them (n + 1) matrix

[A b] =

2

6664

a11 a12 a1n b1a21 a22 a2n b2

..

.

..

.

. ..

..

.

..

.am1 am2 amn bm

3

7775

is called the augmented matrix of the system.

6

7/28/2019 Definitions 060208

7/21

Example 16 The coecient matrix of the linear system

2x1 + 2x2 5x3 + 4x4 2x5 = 4

4x1 + x2 5x4 = 1

3x1 2x2 + 4x4 2x5 = 4

is

A =

24 2 2 5 4 24 1 0 5 0

3 2 0 4 2

35

and the augmented matrix of this system is

[A b] =24 2 2 5 4 2 44 1 0 5 0 1

3 2 0 4 2 4

35 .

Denition 17 An elementary row operation is performed on a matrixby changing the matrix in one of the following ways:

1. Interchange: Interchange two rows of the matrix. If rows i andj areinterchanged, then we use the notationri ! rj.

2. Scaling: Multiply all entries in a single row by a nonzero constant,k. If theith row is multiplied byk, then we use the notationk ri ! ri.

3. Replacement: Replace a row with the sum of itself and a nonzeromultiple of another row. If the ith row is replaced with the sum of itselfandk times the jth row, then we use the notationri + k rj ! ri.

Example 18 Here are some examples of elementary row operations per-formed on the matrix

A =

24 2 5 7 77 4 8 2

8 7 8 2

35 .

1. Interchange rows 1 and 3:24 2 5 7 77 4 8 2

8 7 8 2

35 r1!r3!

24 8 7 8 27 4 8 22 5 7 7

35

7

7/28/2019 Definitions 060208

8/21

2. Scale row 1 by a factor of5.

24 2 5 7 77 4 8 2

8 7 8 2

35 5r1!r1!

24 10 25 35 357 4 8 2

8 7 8 2

35

3. Replace row 2 with the sum of itself and 2 times row 3.24 2 5 7 77 4 8 2

8 7 8 2

35 r2+2r3!r2!

24 2 5 7 79 18 8 2

8 7 8 2

35

Denition 19 Anmn matrix, A, is said to berow equivalent (or simply

equivalent) to anm n matrix, B, if A can be transformed into B by aseries of elementary row operations. IfA is equivalent to B, then we writeA B.

Example 20 As can be seen by studying the previous example24 2 5 7 77 4 8 2

8 7 8 2

35

24 8 7 8 27 4 8 22 5 7 7

35 ,

242 5 7 7

7 4 8 28 7 8 23524

10 25 35 35

7 4 8 28 7 8 235

,

and 24 2 5 7 77 4 8 2

8 7 8 2

35

24 2 5 7 79 18 8 2

8 7 8 2

35 .

Example 21 The sequence of row operations2

43 61 67 1

3

52r1!r1!

2

46 121 67 1

3

54r3!r3!

2

46 121 628 4

3

5r1!r3!

2

428 41 66 12

3

5shows that 2

4 3 61 67 1

35

24 28 41 66 12

35 .

8

7/28/2019 Definitions 060208

9/21

Remark 22 (optional) Row equivalence of matrices is an example of what

is called an equivalence relation. An equivalence relation on a nonemptyset of objects, X, is a relation, , that is reexive, symmetric, and transitive.The reexive property is the property that x x for all x 2 X. The symmetric property is the property that if x 2 X and y 2 X with x y,then y x. The transitive property is the property that if x 2 X, y 2 X,and z 2 X with x y and y z, thenx z. If we takeX to be the setof allm n matrices, then row equivalence is an equivalence relation on Xbecause

1. A A for allA 2 X.

2. IfA 2 X andB 2 X andA B, thenB A.3. IfA 2 X, B 2 X, andC2 X andA B andB C, thenA C.

1.2: Row Reduction and Echelon Forms

Denition 23 The leading entry in a row of a matrix is the rst (fromthe left) nonzero entry in that row.

Example 24 For the matrix

2666640 1 4 80 0 4 35 0 0 100 0 0 19 7 2 8

377775 ,

the leading entry in row 1 is 1, the leading entry in row 2 is4, the leadingentry in row 3 is 5, the leading entry in row 4 is1, and the leading entryin row 5 is 9. (Note that if a matrix happens to have a row consisting of allzeros, then that row does not have a leading entry.)

Denition 25 A zero row of a matrix is a row consisting entirely of zeros.A nonzero row is a row that is not a zero row.

Denition 26 Anechelon form matrix is a matrix for which

1. No zero row has a nonzero row below it.

9

7/28/2019 Definitions 060208

10/21

2. Each leading entry is in a column to the right of the leading entry in

the row above it.3. All entries in the same column and below any leading entry are zeros.

Denition 27 A reduced echelon form matrixis an echelon form matrixwhich also has the properties:

4. The leading entry in each nonzero row is 1.

5. Each leading 1 is the only nonzero entry in its column.

Example 28 The matrix

26646 4 101 2 104 4 20 10 1

3775

is not an echelon form matrix.The matrix

0 0 0 4 80 0 0 0 3

is an echelon form matrix but not a reduced echelon form matrix.The matrix 2

666640 0 1 00 0 0 10 0 0 00 0 0 00 0 0 0

377775

is a reduced echelon form matrix.

Theorem 29 (Uniqueness of the Reduced Echelon Form) Each ma-trix, A, is equivalent to one and only one reduced echelon form matrix. Thisunique matrix is denoted by rref(A).

Example 30 For

A =

26646 4 101 2 104 4 20 10 1

3775 ;

10

7/28/2019 Definitions 060208

11/21

we can see (after some work) that

rref(A) =

2664

1 0 00 1 00 0 10 0 0

3775 .

For

A =

0 0 0 4 80 0 0 0 3

,

we have

rref(A) = 0 0 0 1 0

0 0 0 0 1 .

For

A =

266664

0 0 1 00 0 0 10 0 0 00 0 0 00 0 0 0

377775 ,

we have

rref(A) =

2

66664

0 0 1 00 0 0 10 0 0 00 0 0 00 0 0 0

3

77775 .

Denition 31 A pivot position in a matrix, A, is a position in A thatcorresponds to a rowleading 1 in rref(A). A pivot column ofA is columnofA that contains a pivot position.

Example 32 By studying the preceding example, we can see that the pivotcolumns of

A =

26646 4 101 2 104 4 20 10 1

3775

are columns 1, 2, and 3. (Thus all columns ofA are pivot columns).

11

7/28/2019 Definitions 060208

12/21

The pivot columns of

A =

0 0 0 4 80 0 0 0 3

are columns 4 and 5.The pivot columns of

A =

266664

0 0 1 00 0 0 10 0 0 00 0 0 00 0 0 0

377775

are columns 3 and 4.

Denition 33 LetA be the coecient matrix of the linear system of equa-tions

a11x1 + a12x2 + : : : + a1nxn = b1

a21x1 + a22x2 + : : : + a2nxn = b2...

am1x1 + am2x2 + : : : + amnxn = bm.

Each variable that corresponds to a pivot column ofA is called abasic vari-

able and the other variables are called free variables.

Example 34 The coecient matrix of the linear system

2x1 + 2x2 5x3 + 4x4 2x5 = 4

4x1 + x2 5x4 = 1

3x1 2x2 + 4x4 2x5 = 4

is

A =

24

2 2 5 4 24 1 0 5 0

3 2 0 4 2

35 .

Since

rref(A) =

24 1 0 0

6

5

2

5

0 1 0 15

8

5

0 0 1 25

6

5

35 ,

12

7/28/2019 Definitions 060208

13/21

we see that the pivot columns ofA are columns 1, 2, and 3. Thus the basic

variables of this linear system are x1, x2; andx3. The free variables arex4andx5.

Theorem 35 (Existence and Uniqueness Theorem) A linear system ofequations is consistent if and only if the rightmost column of the augmentedmatrix for the system is not a pivot column. If a linear system is consistent,then the solution set of the linear system contains either a unique solution(when there are no free variables), or innitely many solutions (whenthere is at least one free variable).

Example 36 The augmented matrix of the system

2x1 + 2x2 5x3 + 4x4 2x5 = 4

4x1 + x2 5x4 = 1

3x1 2x2 + 4x4 2x5 = 4

is

[A b] =

24 2 2 5 4 2 44 1 0 5 0 1

3 2 0 4 2 4

35 .

Since

rref([A b]) =24 1 0 0

6

5

2

5

6

5

0 1 0 15

85

195

0 0 1 25

6

5

6

5

35 ,we see that the rightmost column of [A b] is not a pivot column. This meansthat the system is consistent. Furthermore, since the system does have freevariables, there are innitely many solutions to the system.

Here is a general procedure for determining whether a given linear sys-tem of equations is consistent or not and, if it is consistent, whether it hasinnitely many solutions or a unique solution:

1. Compute rref([A b]) to determine whether or not the rightmost col-umn of [A b] is a pivot column. If the rightmost of [A b] is a pivotcolumn, then the system is inconsistent.

13

7/28/2019 Definitions 060208

14/21

2. Assuming that the rightmost column of [A b] is not a pivot column,

take a look at rref(A). (This requires no extra computation.) If allcolumns ofA are pivot columns, then the linear system has a uniquesolution. If not all columns ofA are pivot columns, then the systemhas innitely many solutions.

1.3: Vector Equations

Denition 37 Suppose that

u =26664

u1u2...un

37775 and v =26664

v1v2...vn

37775

are column vectors of dimensionn. We dene the sumofu andv to be thevector

u + v =

26664

u1 + v1u2 + v2

...un + vn

37775 .

Denition 38 Suppose that

u =

26664

u1u2...un

37775

is a column vectors of dimensionn and suppose thatc is a real number (alsocalled a scalar). We dene the scalar multiple cu to be

cu =

26664cu1cu2

...cun

37775 .

14

7/28/2019 Definitions 060208

15/21

Denition 39 The symbolRn denotes the set of all orderedntuples of num-

bers. The symbol Vn denotes the set of all column vectors of dimension n.(The textbook author does not make this distinction. He usesRn to denoteboth things. However, there really is a dierence. Elements of Rn shouldbe regarded as points whereas elements of Vn are matrices with one column.)If x = (x1; x2; : : : ; xn) is a point inR

n, then the position vector of x is thevector

x =

26664

x1x2...xn

37775 .

Example 40 x = (1; 2) is a point inR2

. The position vector of this point(which is an element ofV2) is

x =

12

.

Denition 41 Suppose that u1;u2; : : : ;un are vectors in Vm. (Note thatthis is a set ofn vectors, each of which has dimensionm). Also suppose thatc1; c2; : : : ; cn are scalars. Then the vector

c1u1 + c2u2 + + cnun

is called a linear combination of the vectorsu1;u2; : : : ;un.

Example 42 Suppose that

u1 =

12

, u2 =

05

, u3 =

14

.

Then the vector

4u1 2u2 + 3u3 = 4

12

2

05

+ 3

14

=

7

30

is a linear combination ofu1,u2, andu3.

Denition 43 Suppose thatu1;u2; : : : ;un are vectors in Vm. The span of

this set of vectors, denoted by Spanfu1;u2; : : : ;ung, is the set of all vectorsthat are linear combinations of the vectorsu1;u2; : : : ;un. Formally,

Spanfu1;u2; : : : ;ung

= fv 2 Vm j there exist scalars c1; c2; : : : ; cn such thatv = c1u1 + c2u2 + + cnung .

15

7/28/2019 Definitions 060208

16/21

Denition 44 Thezero vector inVm is the vector

0m =

26664

00...0

37775 .

Remark 45 Ifu1;u2; : : : ;un isanyset of vectors inVm, then0m 2 Spanfu1;u2; : : : ;ungbecause

0m = 0u1 + 0u2 + + 0un.

Denition 46 A vector equation in the variablesx1, x2, . . . ,xn is an equa-

tion of the formx1a1 + x2a2 + + xnan = b

wherea1,a2, . . . ,an andb are vectors inVm (which are usually known before-hand). An orderedntuple (s1; s2; : : : ; sn) is called a solution of this vectorequation if the equation is satised when we setx1 = s1, x2 = s2, . . . , xn = sn.The solution set of the vector equation is the set of all of its solutions.

Example 47 An example of a vector equation is

x1 1

2 + x2

0

5 + x3

1

4 =

4

2 .

The ordered triple (x1; x2; x3) = (9; 4=5; 5) is a solution of this vector equa-tion because

9

12

+

4

5

05

+ 5

14

=

42

.

Remark 48 The vector equation

x1a1 + x2a2 + + xnan = b

is consistent if and only ifb 2 Spanfa1; a2; : : : ;ang.

Remark 49 The vector equation

x1a1 + x2a2 + + xnan = b

16

7/28/2019 Definitions 060208

17/21

where

a1 =

26664a11a21

...am1

37775 , a2 =26664

a12a22

...am2

37775 , . . . ,an =26664

a1na2n

...amn

37775and

b =

26664

b1b2...bm

37775

is equivalent to the system of linear equations

a11x1 + a12x2 + : : : + a1nxn = b1

a21x1 + a22x2 + : : : + a2nxn = b2...

am1x1 + am2x2 + : : : + amnxn = bm.

This means that the vector equation and the system have the same solutionsets. Each can be solved (or determined to be inconsistent) by rowreducingthe augmented matrix

a1 a2

an

b .

Example 50 Let us solve the vector equation

x1

12

+ x2

05

+ x3

14

=

42

.

This equation is equivalent to the linear system

x1 + x3 = 4

2x1 5x2 + 4x3 = 2.

Since

[A b] =

1 0 1 42 5 4 2

and

rref([A b]) =

1 0 1 40 1 2

56

5

,

17

7/28/2019 Definitions 060208

18/21

we see that the vector equation is consistent (because the rightmost column of

[Ab

] is not a pivot column) and that it in fact has innitely many solutions(because not all columns ofA are pivot columns). We also see that the generalsolution is

x1 = 4 x3

x2 = 6

5+

2

5x3

x3 = free.

Note that if we setx3 = 5, then we get the particular solution

x1 = 9x2 =

4

5x3 = 5

that was identied in an earlier example.

1.4: The Matrix Equation Ax = b

Denition 51 For a matrix of sizem n,

A =a1 a2 an

=

26664

a11 a12 a1na21 a22 a2n

......

. . ....

am1 am2 amn

37775 ,

and a vector of dimensionn,

x =

26664

x1x2...

xn

37775

,

we dene the product Ax to be

Ax = x1a1 + x2a2 + + xnan.

18

7/28/2019 Definitions 060208

19/21

Example 52 Suppose

A =

2 3 44 3 3

and

x =

24 41

0

35 .

Then

Ax = 4

24

1

33

+ 0

43

=

1119

.

Denition 53 Suppose that

A =a1 a2 an

=

26664

a11 a12 a1na21 a22 a2n

......

. . ....

am1 am2 amn

37775

is anm n matrix and that

b =

26664

b1b2

...bm

37775 2 Vm.

ThenAx = b

is called a matrix equation. Generally, A andb are known beforehand andthe problem is to solve for the unknown vectorx 2 Vn.

Remark 54 The system of linear equations

a11x1 + a12x2 + : : : + a1nxn = b1a21x1 + a22x2 + : : : + a2nxn = b2

...

am1x1 + am2x2 + : : : + amnxn = bm,

19

7/28/2019 Definitions 060208

20/21

the vector equation

x1a1 + x2a2 + + xnan = b,

and the matrix equationAx = b

are all equivalent to each other! The only subtle dierence is that solutionsof the system and of the vector equation are orderedntuples

(x1; x2; : : : ; xn) = (s1; s2; : : : ; sn) 2 Rn;

whereas solutions of the matrix equation are the corresponding vectors

26664x1

x2...xn

37775 =26664

s1

s2...sn

37775 2 Vn.

1.5: Solution Sets of Linear Systems

Denition 55 A homogeneous linear system is one of the form

Ax = 0m

whereA is anm n matrix.

A nonhomogeneous linear system is a linear system that is not ho-mogeneous.

Remark 56 Homogenous systems are consistent! Thats because they havethe solutionx = 0n.

Denition 57 The solution x = 0n is called the trivial solution of thehomogeneous systemAx = 0m. A nontrivial solutionof the homogeneoussystemAx = 0m is a solutionx 2 Vn such thatx 6= 0n.

Example 58 The system

5x + 2y 3z + w = 0

3x + y 5z + w = 0

x 3y 5z + 5w = 0

is homogeneous. A nontrivial solution of this system (as the reader cancheck) is (x;y;z;w) = (18; 34; 1; 25).

20

7/28/2019 Definitions 060208

21/21

Theorem 59 If every column ofA is a pivot column, then the homogeneous

system Ax

=0m has only the trivial solution. If not every column of A isa pivot column, then the homogeneous systemAx = 0m has innitely many

nontrivial solutions (in addition to the trivial solution).

Example 60 The system given in the preceding example has coecient ma-trix

A =

24 5 2 3 13 1 5 1

1 3 5 5

35 .

Without doing any computations, we can see that it is not possible that everycolumn of A can be a pivot column. (Why?) Therefore the homogeneous

systemAx = 0m has innitely many nontrivial solutions.

Theorem 61 Suppose thatA is anm n matrix and suppose thatb 2 Vm.Also suppose thatv 2 Vn is a solution of the homogeneous systemAx = 0mand thatw 2 Vn is a solution of the systemAx = b. Thenv + w is also asolution ofAx = b.

21