3333232131

2323222121

1313212111

bxaxaxa

bxaxaxa

bxaxaxa

3333232131

2323222121

1313212111

bxaxaxa

bxaxaxa

bxaxaxa

3

2

1

3

2

1

333231

232221

131211

b

b

b

x

x

x

aaa

aaa

aaa

3

2

1

3

2

1

333231

232221

131211

b

b

b

x

x

x

aaa

aaa

aaa

Linear SystemsSolve Ax=b, where A is an nn matrix and

b is an n1 column vectorCan also talk about non-square systems

whereA is mn, b is m1, and x is n1Overdetermined if m>n:

“more equations than unknowns”Underdetermined if n>m:

“more unknowns than equations”Can look for best solution using least squares

Singular SystemsA is singular if some row is

linear combination of other rowsSingular systems can be underdetermined:

or inconsistent: 1064

532

21

21

xx

xx

1064

532

21

21

xx

xx

1164

532

21

21

xx

xx

1164

532

21

21

xx

xx

Gauss-Jordan EliminationFundamental operations:

1. Replace one equation with linear combinationof other equations

2. Interchange two equations

3. Re-label two variablesCombine to reduce to trivial systemSimplest variant only uses #1 operations,

but get better stability by adding#2 (partial pivoting) or #2 and #3 (full pivoting)

Gauss-Jordan EliminationSolve:

Only care about numbers – form “tableau” or “augmented matrix”:

1354

732

21

21

xx

xx

1354

732

21

21

xx

xx

13

7

54

32

13

7

54

32

Gauss-Jordan EliminationGiven:

Goal: reduce this to trivial system

and read off answer from right column

13

7

54

32

13

7

54

32

?

?

10

01

?

?

10

01

Gauss-Jordan Elimination

Basic operation 1: replace any row bylinear combination with any other row

Here, replace row1 with 1/2 * row1 + 0 * row2

13

7

54

32

13

7

54

32

1354

1 27

23

1354

1 27

23

Gauss-Jordan Elimination

Replace row2 with row2 – 4 * row1

Negate row2

1354

1 27

23

1354

1 27

23

110

1 27

23

110

1 27

23

110

1 27

23

110

1 27

23

Gauss-Jordan Elimination

Replace row1 with row1 – 3/2 * row2

Read off solution: x1 = 2, x2 = 1

110

1 27

23

110

1 27

23

1

2

10

01

1

2

10

01

Gauss-Jordan EliminationFor each row i:

Multiply row i by 1/aii

For each other row j: Add –aji times row i to row j

At the end, left part of matrix is identity,answer in right part

Can solve any number of R.H.S. simultaneously

Recall that we'd like to use row operations on an augmented matrix to get it into the following form:

This is not always possible though. The following are matrices that cannot be put into this form.

1

2

3

1

1 0 0 0 0

0 1 0 0 0

0 0 1 0 0

0 0 0 1 0

0 0 0 0 1n

n

b

b

b

b

b

1 2 3 71 0 5 2

0 0 0 00 1 6 3

0 0 0 0

Recognize that if we can’t get our matrix to the desired form, then it won’t be as easy to see what the solution to the system of equations will be.

For example, this matrix has a solution that is easy to see, (1, 3, 5), because the matrix is in the final form that we want.

1 0 0 1

0 1 0 3

0 0 1 5

1 2 3 7

0 0 0 0

0 0 0 0

1 0 5 2

0 1 6 3

This matrix (on the right) has a solution but is not as clear what the solution is. What we can conclude about the solution, (x, y, z), is that the components x, y, and z must obey the equation x + 2y + 3z = 7.

This matrix (on the right) has a solution, but again it is not as clear what it is. What we can conclude about the solution, (x, y, z), is that the components x, y, and z must obey the two equations x + 5z = 2 and y + 6z = 3.

These last two matrices represent systems that do not have a unique solution. Whenever a matrix does not have a unique solution (if it has infinitely many solutions or no solution at all) we will not be able to get our augmented matrix into the form that we really want. When this happens, we want to at least get our matrix as close as possible to this form that we would really like it to be in. When it is as close as it can possibly get, we say it is in reduced row echelon form.

1

2

3

1

1 0 0 0 0

0 1 0 0 0

0 0 1 0 0

0 0 0 1 0

0 0 0 0 1n

n

b

b

b

b

b

PIVOTINGConsider this system:

Immediately run into problem:algorithm wants us to divide by zero!

More subtle version:

8

2

32

10

8

2

32

10

8

2

32

1001.0

8

2

32

1001.0

Partial Pivoting

Swap rows 1 and 2:

Now continue:

8

2

32

10

8

2

32

10

2

8

10

32

2

8

10

32

2

1

10

01

2

4

10

1 23

2

1

10

01

2

4

10

1 23

Full Pivoting

Swap largest element onto diagonal by swapping rows 1 and 2 and columns 1 and 2:

Critical: when swapping columns, must remember to swap results!

8

2

32

10

8

2

32

10

2

8

01

23

2

8

01

23

Full Pivoting

Full pivoting more stable, but only slightly

2

8

01

23

2

8

01

23

32

38

32

32

0

1

32

38

32

32

0

1

1

2

10

01

1

2

10

01