Module 1 - Gauss

8/11/2019 Module 1 - Gauss

1/23

Math 221a

GaussAnil Nerode 2007

We begin the course with the study of the basic Gauss algorithm for solutionof any system of simultaneous linear equations in many variables. Such systemsarise in all branches of applied mathematics as "linear approximations" to non-linear systems. They are the simplest to study and capture "local" propertiesof the corresponding "non-linear system". That is why we study them rst.Also understanding them will clarify many notions in the calculus of severalvariables, which is the normal course succeeding this course.

Gausss algorithm for solving systems of linear equations

We are interested in solving linear systems ofm equations in n unknowns ofthe form

a11x1 + : : : + a1nxn =b1: : : : :: : : : :: : : : :

am1x1 + :::+ amnxn =bm

by manipulating the matrix0

BBBB@

a11 : : a1n b1: : : : :: : : : :

: : : : :am1 : : amn bm

1

CCCCA

As we do so the matrices take on an inner life of their own!

The three row operations on equations or their matrices.

Type 1Interchange 2 rowsType 2Add c times one row to another.Type 3Multiply a row by a non-zero constant.

Note that each type of operation has an inverse (an operation which undoesit) of the same type, and that row operations leave solution sets unchanged.We give the algorithm before presenting a single example. The reason for this

is that the examples are easy to follow. After looking at them, the studentwill say "I can do that", and indeed this is true. At that point, the studentis likely to ignore reading the algorithm. But if you were to try to write acomputer program implementing the algorithm, the "I can do that" will nothelp a lot. The algorithm below is explicit enough as a program specication sothat someone who has read it with understanding can write a program executing

1


2/23

it easily. A huge number of programming errors in industrial settings are dueto not reading and understanding the program specication, ending up with a

program with unintended behavior. So learn to read mathematics accurately,then go to the examples, then come back and read the mathematics again.Acquiring the skill of puzzling through mathematical statements till you haveacquired full understanding.

Gausss Algorithm

STAGE 1

1) Find the leftmost column j with a non zero entry, and choose the upper-most non-zero entrya in that column.

2) Interchange the row of that entry a with the rst row.

3) Multiply the rst row by 1=a:

4) Add to each row, other than the rst, a suitable multiple of the rst rowto produce all zeros in columnj below the rst row.

STAGE 2

1) Find the leftmost column k entry with a non-zero entry below the rstrow, and choose the uppermost non-zero entry b:

2) Interchange the row of that entry b with the second row.

3) Multiple the second row by 1=b:

4) Add to each row below the second a suitable multiple of the rst row toproduce all zeros in column k below the second row..

LATER STAGES

Continue in the same manner till rows or columns are exhausted.

Note:At the end of stage 1 we stopped changing the rst row and column.At the end of stage 2 we stopped changing the second row and column...

.We end up with a so called echelon form for the original matrix, from

which solutions and their properties can be read o.Since row operations do not change the solution set, the echelon form has

the same solutions as the original system.

2


3/23

We have specied a specic order for the operations.This is to make gradingproblems easy, and to give a wholly deterministic algorithm, where there are

no choices as to what step to take next. Neither you nor I will stick to thisroutine, except for homework. We deviate from the Gauss algorithm by notmultiplying by 1

c to make the leading term1. That step my be omitted if the

only purpose is to nd a simple formula for the solutions. Afterwards, one canmultiply each row by the reciprocal of its leftmost term and we will have the1s on the diagonal of Gausss own echelon form.

Example Start out with equations

x2 x3 +x4 = 0x1 +x2 +x3 +x4 = 6

2x1 +4x2 +x3 2x4 = 1

3x1 +x2 2x3 +2x4 = 3

and the corresponding matrix

0BB@

0 1 1 1 01 1 1 1 62 4 1 2 13 1 2 2 3

1CCA

The leftmost column with a non-zero entry is the rst column. The highestrow with a non-zero entry in that column is the second. The entry in the rstcolumn and second row is 1. The rst operation is to interchange rows 1 and 2,

to bring this 1 to the top row.0BB@

1 1 1 1 60 1 1 1 02 4 1 2 13 1 2 2 3

1CCA

Next add(2)times the rst row to the third row to make the rst non-zeroelement of the third row zero.

0BB@

1 1 1 1 60 1 1 1 00 2 1 4 133 1 2 2 3

1CCA

Next add(3)times the rst row to the fourth row to make the rst elementof the fourth row zero0

BB@1 1 1 1 60 1 1 1 00 2 1 4 130 2 5 1 15

1CCA

3


4/23

If we write down the corresponding system of equations we have arranged itso that x1 occurs only in the rst equation.

x1+x2+x3+x4 = 6:Now, holding the rst row and the rst column xed (we have nished our

work there), repeat the same process starting with the second row and secondcolumn. Namely, nd the left most column with an entry below the rst rowthat is non-zero, and, if necessary, interchange rows so that it becomes thesecond row. It already IS the second row, so this step is not needed.

Add2times the second row to the third so that the second entry of the thirdrow is zero0

BB@

1 1 1 1 60 1 1 1 00 0 3 2 130 2 5 1 15

1CCA

Add(2)times the second row to the fourth row to make the second entryof the fourth row zero.0

BB@1 1 1 1 60 1 1 1 00 0 3 2 130 0 3 3 15

1CCA

At this point, in terms of the equation interpretation, x1 occurs only in therst equation, x2 occurs only in the rst equation and in the second equationx2 x3+x4 = 0

Now hold the rst two rows and the rst two columns xed, and repeat theprocess. Find the leftmost column with a non-zero entry below the rst tworows, and interchange rows so that it is in the third row. This is unnecessary

because the entry already there is non-zero.

Add(1)times the third row to the fourth to assure that the third elementof the fourth row is zero.0

BB@1 1 1 1 60 1 1 1 00 0 3 2 130 0 0 1 2

1CCA

,We now have thatx1 occurs only in the rst equation, x2 occurs only in the

rst two equations, x3 occurs only in the third equationx3 = 9, x ocurs only in the fourth equation x4 = 2:The corresponding equations are

x1 +x2 +x3 +x4 = 6x2 x3 +x4 = 0

3x3 2x4 = 13x4 = 2

Working from the bottom up,

4


5/23

0BB@

x1x2

x3x4

1CCA

=

0BB@

21

32

1CCA

This process is called back substitution.But solutions are not always unique. We discuss a couple of row reduced

systems without unique solutions.

Example241 4 20 1 3

0 0 1

35

We interpret this as representing three equations in two unknowns

x1 +4x2 = 2x2 = 30 = 1

:

If you row reduced a system and got this as the row reduced form, the lastequation shows there are no solutions whatever.

Example241 3 0 00 0 1 3

0 0 0 0

35

This is also row reduced. We interpret this as representing three equations

in three unknownsx1 +3x2 = 0

x3 = 30 = 0

If you row reduced a system and got this,the third equation is unneeded,the second species thatx3 = 3;the rst says that x1 = 3x2:

So we conclude that for any value of x2, we get one and only one solutionwithx3 = 3 and x1 = 3x2:

That is, the "general solution" is

0@3x2x2

3

1A :

Geometrically this says the solution consists of all points on a line in theplanex3 = 3:

5


6/23

If we set x2 = 1; we see that

0@31

3

1A lies on the line of solutions.

If we set x2 = 0, we see that

0@00

3

1A lies on that line.

A line is determined by two points. The set of all solutions is the set of all

points in Euclidean space on the line through

0@31

3

1A and

0@00

3

1A :.

This is a convenient time to introduce vector notation.

A 3-vector is a column of numbers

0@uv

w

1A :

The 3- vector

0@uv

w

1A is pictured, following Newton, as a directed line segment

stretching from the origin (0; 0; 0)to the point (u;v;w):

Scalar multiplication: If c is a particular number (scalar), the product

c

0@uv

w

1A is dened as

0@cucv

cw

1A :

For instance 5

0@67

8

1A =

0@3035

40

1A :

In the geometric interpretation, scalar multiplication by c stretches out theline segment from origin to (u; v; w) by a factorc in the direction indicated bythe sign of c.

VectorAddition:

0@uv

w

1A+

0@xy

z

1A =

0@u+xv+y

w+z

1A :

6


7/23

For instance;

0@

12

3

1A+

0@

45

6

1A =

0@

57

9

1A :

In the geometric interpretation, vector addition is interpreted by Newtonsparallelogram law.

One completes the parallelogram which has as two of its sides the line seg-ments from origin to (u;v;w)and (x; y; z) respectively.

The corner point opposite to the origin in this parallelogram is (u+x; v+

y; w+z); corresponding to the vector sum

0@

uv

w

1A+

0@

xy

z

1A :

For the problem just given, the general solution can be written0@3x2x2

3

1A =

0@00

3

1A+x2

0@31

0

1A

The general solution depends on one parameter x2:

That is, every solution is the vector sum of the specic solution

0@00

3

1A plus

some (any) scalar multiple of the xed vector0@3

101A

.

The solutions form the unique line in the plane containing the two points(0; 0; 3)and (3; 1; 3). (why?)

Example24 1 1 1 1 0 10 0 0 1 0 2

0 0 0 0 1 1

35

This is already row reduced. The corresponding equations are

x1 +x2 +x3 +x4 = 1x4 = 2

x5 = 1We see that the solutions are given by

7


8/23

x5 = 1;x4 = 2;

x1 = 1 x2 x3 x4 = 1 x2 x3,

wherex2; x3 can have any values. The general solution is0BBBB@

1 x2 x3x2x321

1CCCCA

This can be written

0BBBB@

1

0021

1CCCCA+0BBBB@

x2 x3

x2x300

1CCCCA

or as0BBBB@

10021

1CCCCA+x2

0BBBB@

11000

1CCCCA+x3

0BBBB@

10100

1CCCCA :

This solution depends on two independent parameters. x2; x3.

We say we are working in the5space of column vectors

0BBBB@

abcde

1CCCCA :In that space

the solutions form a plane through the three points0BBBB@

10021

1CCCCA

;0BBBB@

10021

1CCCCA+

0BBBB@

11000

1CCCCA ;

8


9/23

0BBBB@

10

021

1CCCCA+

0BBBB@

10

100

1CCCCA :

Gauss-Jordan Algorithm

Reduced Echelon Form

LetA be in row echelon form.1) Go to the leftmost non-zero column j ofA, in that column nd the largest

index i with aij = 1, subtract suitable multiples of that row from all previousrows to make the entries in that column above aij = 0 .

2) Repeat until every column with a leading 1 has zeros above and belowthat1:

Example

Start with

0@ 1 1 50 1 4

0 0 1

1A

:Add(1)times the second row to the rst row, getting

0@ 1 0 10 1 40 0 1

1A .

Add(4)times the third row to the second, getting0@ 1 0 10 1 0

0 0 1

1A

Add(1)times the third row to the rst, getting0@ 1 0 00 1 0

0 0 1

1A

This is in reduced row echelon form.

Example

Start with

9


10/23

0@

1 0 5 20 0 1 2

0 0 0 1

1A :

Add(5)times the second row to the rst, getting

0@ 1 0 0 80 0 1 2

0 0 0 1

1A :

Add(8) times the third row to the rst, getting

0@ 1 0 0 00 0 1 20 0 0 1

1AAdd(2)times the third row to the second, getting

0@ 1 0 0 00 0 1 0

0 0 0 1

1A :

Example.

No matter what values we choose for a, b, c, d, , the following matrices arealready in reduced row echelon form.

0@ 1 0 0 a0 1 0 b0 0 1 c

1A;

0@ 1 0 a 00 1 a 0

0 0 0 1

1A ;

0@ 1 0 a c0 1 b d

0 0 0 0

1A ;

0

@1 a 0 00 0 1 0

0 0 0 1

1

A

Matrix algebra

10


11/23

A matrix is an m by n array of numbers, m rows, n columns. We writesuch a matrix as A = (aij). The index i runs from 1 to m, indexing rows, and

the index j runs from 1 to n, indexing columns. We refer to this matrix as anm by n matrix, or a matrix of size (m; n). Here are two 3 by 2 matrices0

@a11 a12 a13 a14 a15 a16a21 a22 a23 a24 a25 a26a31 a32 a33 a34 a35 a36

1A

0@b11 b12 b13 b14 b15 b16b21 b22 b23 b24 b25 b26

b31 b32 b33 b34 b35 b36

1A

Two m by n matrices can be added.0@

a11 a12 a13 a14 a15 a16a21 a22 a23 a24 a25 a26a31 a32 a33 a34 a35 a36

1A+

0@

b11 b12 b13 b14 b15 b16b21 b22 b23 b24 b25 b26b31 b32 b33 b34 b35 b36

1A =

0@a11+b11 a12+b12 a13+b13 a14+b14 a15+b15 a16+b16a21+b21 a22+b22 a23+b23 a24+b24 a25+b25 a26+b26

a31+b31 a32+b32 a33+b33 a34+b34 a35+b35 a36+b36

1A

They can be scalar multiplied

c

0

@a11 a12 a13 a14 a15 a16a21 a22 a23 a24 a25 a26

a31 a32 a33 a34 a35 a36

1

A=

0@ca11 ca12 ca13 ca14 ca15 ca16ca21 ca22 ca23 ca24 ca25 ca26

ca31 ca32 ca33 ca34 ca35 ca36

1A

Row by Column Matrix Multiplication.

Given a row vector

a1 : : : an

of length n and a column vector0BBBB@

b1:::

bn

1CCCCA

of length n; we dene their product to be the number

a1b1+:::+anbn:Thus

1 2 30@45

6

1A = 14+25+36 = 4+10+18 = 32:

Matrix Product

11


12/23

If A; B are matrices, we only form the matrix product AB in case A is mby nand B is nby p, that is, only in case the rows of Aare the same length

as the columns ofB: In that case, the productAB is the m by pmatrix whoseij- th entry is the product of the i-th row of A by the j -th row ofB .

Examplea b cd e f

0@g hi jk l

1A =

ib+ag+ck ah+bj+cldg+f k+ie dh+f l+je

a bc d

e fg h

=

bg+ae af + bhdg+ce cf + dh

Square matrices of the same size can be multiplied in either order, but theresults are usually dierent.

e fg h

a bc d

=

cf+ ae df+ beag+ch bg+dh

Here is a numerical exampl.1 23 4

5 67 8

=

19 2243 50

5 67 8

1 23 4

=

23 3431 46

Thus matrices of the same size can be added and scalar multiplied(aij) + (bij) = (aij+bij)c(aij) = (acij):The setMmn of all m by n matrices is closed under these two operations.

We dene the productAB= (aij)(bjk)of two matricesA; B only in case A is m byn, andB is n byp;, that is, only

in case the number of columns ofA is equal to the number of rows ofB : Hereis the denition using summation notation .

.AB= (aij)(bjk) = (ai1b1k+:::+ainbnk) = (

nj=1aijbjk) = jaijbjk

Here are the laws that matrix multiplication obeys.A(B+C) = AB+AC(A+B)C= AC+BCc(AB) = (cA)B= A(cB)

The hardest law to verify is the

12


13/23

Associative law for matrix multiplication

A(BC) = (AB)C

Proof.LetA = (aij); B= (bjk); C= (ckl):The ik thentry ofAB is jaijbjk : The il thentry of(AB)C is

k(jaijbjk)ckl = k(jaijbjkckl)

The j l thentry ofB C iskbjkckl: The il thentry ofA(BC) isjaij(kbjkckl) = j( kaijbjkckl)But these are the same terms added in a dierent order . So due to the

commutative and associative laws for numbers,A(BC) = (AB)C:

Einstein Convention

The Einstein convention regards any repeated index as being summed over.He introduced writing

AB= aijbjk Physicists and engineers use this convention, most mathemati-cians use the summation sign:

Einstein would have written this proof as

(AB)C= (aijbjk)ckl = aij(bjkckl) = A(BC)

leaving the explicit use of associative and distibutive las for numbers to the

imagination.

The n by n identity matrix is the matrix In = (aij) with aij = 1 if i =j; aij = 0 otherwise.

Example

0@ 1 0 00 1 0

0 0 1

1A =I3

These matrices satisfy:InA= A ifA has n rows.

AIn= A ifA has n columns.Finally, the transpose At of A = (aij) is (aji);obtained by interchangingrows and columns of A.

The properties of transpose are(A+B)t =At +Bt

(aA)t =a(At)

13


14/23

(AB)t =BtAt

Elementary Row Matrices

If I is an identity matrix and is a row operation, then I is the rowmatrix corresponding to that row operation.

Example:If

I=

0@ 1 0 00 1 0

0 0 1

1A

and

is the row operation of adding 3 times the rst row to the second, then

I=

0@ 1 0 03 1 0

0 0 1

1A

is the corresponding row matrix.

Lemma: (AB) = (A)B:

Proof This is straightforward. Try a few examples.That is, a row operation applied to a product AB of matrices gives the

same result as applying the row operation to the rst matrix A and thenpremultiplying B by the result.

When A is the identity matrix this says:

To apply a row operation to a matrix premultiply by the corresponding row

matrix.

That is A= (I)A:

DenitionA square matrixA is calledinvertibleif there is a matrixB suchthatAB = BA= I :

Since each elementary row operation has an inverse row operation, it follows

that every elementary row matrix has an inverse matrix, which is the row matrixfor the inverse operation.

Corollary. A row reduction by a series of elementary row operations canbe carried out by premultiplying by the corresponding row matrices in the sameorder.

14


15/23

The product of invertible matrices is invertible too. The inverse is the prod-uct of the corresponding inverses in the opposite order.

(AB)1 =B1A1:

Combining all this we conclude that if a square matrix can be row reducedto the identity matrix, it has an inverse, the product of the row matrices corre-sponding to the row operations used and in the same order.

Lemma. Let Cbe the row echelon form of A: Let Pbe the invertible

matrix such thatP A= C. (ThisP is the product of the row matrices reducingA to C:) Then for any column vector B, the solutions X to AX = B are thesame as the solutions X to CX= P B:

Proof of Lemma: This is because the row operations applied to AX =B regarded as a set of non-homogeneous equations do not change the solutionset.

Main Theorem.LetA be ann byn matrix.Then the following two conditions are equivalent:1) For every column matrix B, there exists a column matrix X such that

AX= B:2) The row echelon form ofA is the identity matrix.

To prove this we have to show that 2) implies 1) and that 1) implies 2).

2) =)1)

Proof By the lemma above, if the row echelon form Cis the identity, thenCX= P B is X=P B: So AX= B has the same solutions asX= P B, namelyP B.

1) =)2)

Proof. This is the same as proving "(not 2) implies (not 1)". That is,suppose that the row echelon form C= P A ofA is not the identity matrix.

Then we must prove that there is a column vector Bsuch that the equation

AX= B has no solutionX: But this is the same, according to the lemma above,as showing there is a column vector B such that C X=P B has no solution X:

Since C is square and row echelon and not the identity, the row echelonprocess implies the last row is all zeros (why?)

Let D be any column vector with bottom entry non-zero, all other entrieszero.

15


16/23

The system C X=D has no solutions since in the corresponding system ofequations, the last equation has left side zero while the right side is non-zero.

We set P1D = B; and conclude that CX = P B has no solution X andtherefore thatAX=B has no solution X:

Corollary. Ifn by n matrix A has a right inverse B, then B is also a leftinverse ofA.

Proof.

B is called a right inverse ofA ifAB = I. We have to show that B is a leftinverse ofA, namely that B A= I

Now AB = I implies that for any column vector C, A(BC) = (AB)C =IC= C:

So AX= Calways has a solution X=BC:

Apply the theorem above, and conclude that A can be row reduced to theidentity matrix I :

Row reduction implies there is an invertible matrix P such that P A = I .Then

B = I B= (P A)B = P(AB) = P I= P:So B A= P A= Ias required.

Corollary Ifn byn matrix A has a left inverse B, then B is also a rightinverse ofA.

Proof IfB is a left inverse to A; then B A = I : Then B has A as a rightinverse: By the previous lemma, A is also a left inverse ofB :So AB= I : This makes B a right inverse ofA:

Proposition A matrix Ahas at most one inverse which is two sided (bothleft sided and right sided).

Proof This is an exercise in the associative law. Suppose that both B andCare two sided inverses of A:Then

AB= BA= I andAC= C A= I.ThenB = I B = (CA)B = C(AB) = C I= C

Theorem. Let A be an n by n matrix. Then A row reduces to reducedechelon form an identity matrix if and only ifA is invertible.

Proof:

16


17/23

=) If A has row echelon form the identity matrix, and Pis the productof the elementary row matrices used to reduce Ato I, thenP A= I. We have

shown above that If A has a left inverse P, then Pis also a right inverse. SoPis invertible.

(= Conversely, if A is invertible, then for any B , AX= B can be solvedbyX= A1B: So by a theorem above, Xrow reduces to the identity matrix.

CorollaryEvery invertible matrix A is a product of elementary row ma-trices.

Proof If the row echelon form of A is the identity matrix , then I =(kI):::(1I)A

using the indicated row operations. ThenAis invertible with inverse(kI):::(1I):AlsoA1 = (1I)1:::(kI)1

These are elementary row matrices too. So Ais the product of row matrices.

Comment. This says that the concept of an invertible matrix is not all thatcomplicated; applying an invertible matrix to a vector amounts to applying aseries of row matrices to that vector.

Dependence, Independence, Spanning, Bases, Dimension

Many fundamental theorems about Rn can be derived from the main theo-rem we established in the last section.

DenitionVector vis a linear combination of vectorv1;:::;vk if there exist

scalarsc1;:::;ck such that v = c1v1+:::+ckvkDenition Vectors v1;:::;vk span R

n if for every vector v in Rn, v is alinear combination of v1;:::;vk:

Denition Vectorsv1;:::;vk are independent (also called linearly indepen-dent) if for all scalars c1;:::;ck, ifc1v1+:::+ckvk= 0;thenc1 = ::: = ck = 0

DenitionVectorsv1;:::;vk are dependent (also called linearly dependent)if there exist scalars c1;:::;ck, not all zero, such that c1v1+:::+ckvk = 0.

DenitionVectorsv1;:::;vk are a basis for Rn if they are independent andspan Rn.

TheoremLetv1;:::;vnbe nvectors in Rn:Thenv1;:::;vnare independent

if and only ifv1;:::;vn span Rn:

Proof. Letv1 =

0BBBB@

a11:::

an1

1CCCCA ;:::;vn=

0BBBB@

a1n:::

ann

1CCCCA :

17


18/23

Thenv1;:::;vn span Rn if and only if for any vector B =

0BBBB@

b1:

::

bn

1CCCCA

there exist numbers x1;:::;xn such that

x1

0BBBB@

a11:::

an1

1CCCCA+:::+ xn

0BBBB@

a1n:::

ann

1CCCCA =

0BBBB@

b1:::

bn

1CCCCA

This is the same as

a11x1 + : : : + a1nxn = b1: : : : : : : : :: : : : : : : : :: : : : : : : : :

an1x1 + : : : + annxn = bn

IfA =

0BBBB@

a11 a12 : : : a1n: : : : : :: : : : : :: : : : : :

an1 : : : : ann

1CCCCA ;

we see that v1;:::;vn span Rn if and only if for every vector B, there is a

vector Xsuch that AX= B.

Now apply the "main theorem" of the previous section. Condition (1) ofthat theorem has just been veried here. So condition (2) also holds. It saysthatAX= 0implies X= 0. But AX= 0is equivalent to

a11x1 + : : : + a1nxn = 0: : : : : : : : :: : : : : : : : :: : : : : : : : :

an1x1 + : : : + annxn = 0which is equivalent to

x1

0BBBB@

a11:

::

an1

1CCCCA+:::+xn

0BBBB@

a1n:

::

ann

1CCCCA.=

0BBBB@

0:

::0

1CCCCA

Thus condition (2) has been decoded. It says that whenever

18


19/23

x1

0BBBB@

a11:

::

an1

1CCCCA+:::+xn

0BBBB@

a1n:

::

ann

1CCCCA.=

0BBBB@

0::

::0

1CCCCA,

then x1 = x2 = ::: = xn= 0:

This says that v1;:::;vn are independent.Corollary In Rn anyn + 1 vectors are dependent.

Proof We give a proof by contradiction. Suppose thatv1;:::;vn+1 wereindependent. Thenv1;:::;vnwould be independent. By the theorem just proved,v1;:::;vn span R

n. Therforevn+1 is a linear combination of v1:::;vn. That is,there exist scalars c1;:::cn such that vn+1 = c1v1+ :::+cnvn: So c1v1+ :::+

cnvn+ (1)vn+1 = 0: So v1;:::;vn+1 are dependent, a contradiction. So noindependent v1;:::;vn+1 exist. It follows that in Rn there is no sequence ofdistinct vectors of length greater than n that is independent. (Why?)

CorollaryNo sequencev1;:::;vn1 ofn1vectors from Rn can span Rn:

ProofWe give a proof by contradiction. Suppose thatv1;:::;vn1 spanRn:

Cross out from the list v1;:::;vn1 the rst entry that is a linear combinationof other entries. This leaves us with a spanning set (Why?). Do this repeat-edly until no element in the sequence is a linear combination of the rest. Theremaining sequence is independent. (Why?) It is still a spanning set. It is nowan independent spanning set. We have already shown that every independentspanning set has n elements. Thereforev1;:::;vn1 do not span R

n. Thus noset of less than n vectors spans Rn:

Corollary Every basis of Rn has precisely nmembers.Proof We have shown that all independent spanning sets in Rn have

exactly n elements, neither more nor less.

Denitionn is called the dimension (or linear dimension) of Rn:

Tests for Dependence, Independence, Spanning

Here are algorithms for testing for dependence, independence, and spanning.They all reduce to writing the problem as one about solutions of systems of linearequations. They are answered by row reduction.

Independence, Dependence

Let v1 =

0BBBB@

a11:::

an1

1CCCCA ;:::;vk =

0BBBB@

a1k:::

ank

1CCCCA We want to determine whether or not

19


20/23

v1;:::;vk are independent. This is the same as asking whether or not

x1

0BBBB@

a11

:::

a1n

1CCCCA :::+xk0BBBB@

a1k

:::

ank

1CCCCA.=0BBBB@

0:

:::0

1CCCCA

has only the one solution

0BBBB@

x1::::

xn

1CCCCA =

0BBBB@

0::::0

1CCCCA :

This is the same as asking whether or nota11x1 + : : : + a1kxk = 0

: : : : : : : : :: : : : : : : : :: : : : : : : : :

a1n + : : : + ankxk = 0

has only the solution x1 = :::= xk = 0: Row reduce and nd out.Spanning.

Let v1 =

0BBBB@

a11:::

an1

1CCCCA ;:::;vk=

0BBBB@

a1k:::

ank

1CCCCA.

We wish to determine whether or not v1;:::;vk span Rn , that is, whether

or not

for every column vector B =

0BBBB@

b1:::

bn

1CCCCA ;there exist scalarsx1;:::;xk such that

B = x1v1+:::+xkvk:

This is equivalent to asking for A =

0BBBB@

a11 a12 : : : a1k: : : : : :: : : : : :

: : : : : :an1 : : : : ank

1CCCCA

, whether

or not AX= B has a solution X: This is the same as asking whether or notthe system

20


21/23

a11x1 + : : : + a1kxk = b1: : : : : : : : :

: : : : : : : : :: : : : : : : : :

a1nx1 + : : : + ankxk = bk

has a solution. Row reduce and nd out.

Direct Algorithm for Matrix Inversion by Row Reduction

LetA be an n by n matrix, let Ibe ann by n identity matrix.

Form then by 2n matrix (A: I)with A simply placed to the left ofI.

If this matrix row reduces to a reduced row matrix of the form.(I :C);thenA is invertible with inverse C. Otherwise,A is not invertible.

Example0@ 4 0 5 1 0 00 1 6 0 1 0

3 0 4 0 0 1

1A

Multiply rst row by (1=4)0@ 1 0 5=4 1=4 0 00 1 6 0 1 0

3 0 4 0 0 1

1A

Add(3)times rst row to third0@ 1 0

5

4

1

4 0 0

0 1 6 0 1 00 0 1

4

3

4 0 1

1A

Multiply third row by 40@ 1 0

5

4

1

4 0 0

0 1 6 0 1 00 0 1 3 0 4

1A

Add(

5=4)third row to rst0@ 1 0 0 4 0 50 1 6 0 1 0

0 0 1 3 0 4

1A

Add6 times third row to second

21


22/23

0@ 1 0 0 4 0 50 1 0 18 0 24

0 0 1 3 0 4

1A

0@ 4 0 50 1 6

3 0 4

1A1

=

0@ 4 0 518 1 24

3 0 4

1A

Example

Suppose the reduced echelon matrix is

0@ 1 0 0 2 1 10 1 0 1 1 20 0 1 3 6 3

1A

with corresponding system of non-homogeneous equations

x1 +2x4 x5 = 1x2 x4 +x5 = 2

x3 +3x4 +6x5 = 3

The columns without a leading1 are the fourth and fth and sixth.

These correspond to the variables x4 ; x5, and the right constant column interms of which we express x1;x2;x3 in writing out the general solution

x1 = 2x4 +x5 +1x2 = x4 x5 +2

x3 = 3x4 6x5 +3meaning by this that any values of x4; x5 can be assigned and the corre-

spondingvalues ofx1; x2; x3give a solution.

In vector form, the general solution is0BBBB@

x1x2x3x4x5

1CCCCA

=

0BBBB@

1 2x4+x52 +x4 x53 3x4 6x5x4x5

1CCCCA

=x4

0BBBB@

21310

1CCCCA

+x5

0BBBB@

11601

1CCCCA

+

0BBBB@

12300

1CCCCA

Theorem. Suppose that AX=B is a matrix equation with xed A;B:Suppose X0 is a particular solution toAX=B.Then the solutions ofAX=B are all of the form X0+Y;whereY is a solution of the (homogeneous) systemAY = 0 .These are the only solutions.

22


23/23

Proof.IfY is a solution to AY = 0, then by the associative law for matrix multi-

plication,A(X0+Y) = AX0+AY =AX0+ 0 =AX0 = B: Conversely, suppose that

Xis any solution to AX= B.SetY =XX0: By the same associative law,AY =A(XX0) = AXAX0 = B B= 0:The example above bears this out.

The expression

x4

0BBBB@

2131

0

1CCCCA

+x5

0BBBB@

1160

1

1CCCCA

is the general solution to

0@ 1 0 0 2 10 1 0 1 1

0 0 1 3 6

1A0BBBB@

x1x2x3x4x5

1CCCCA =

0@ 00

0

1A :

The vector

0BBBB@

1230

0

1CCCCA

is a particular solution to

0@ 1 0 0 2 10 1 0 1 1

0 0 1 3 6

1A0BBBB@

x1x2x3x4x5

1CCCCA =

0@ 12

3

1A

23

Module 1 - Gauss

Documents

Transcript of Module 1 - Gauss