Boyce/D

iPrim

a 9th

ed, C

h 7.3: S

ystems of L

inear

Eq

uation

s, Lin

ear Ind

epen

den

ce, Eigen

values

Elem

entary Differential E

quations and Boundary V

alue Problem

s, 9thedition, by W

illiam E

. Boyce and R

ichard C. D

iPrim

a, ©2009 by John W

iley & S

ons, Inc.

A system

of nlinear equations in n

variables,

can be expressed as a matrix equation A

x=

b:

If b=

0, then system is h

omogen

eous; otherw

ise it is n

onh

omogen

eous.

nn

nn

nn

n n

b b b

x x x

aa

a

aa

a

aa

a

2 1

2 1

,2,

1,

,22,

21,

2

,12,

11,

1

,,

22,

11,

2,2

22,

21

1,2

1,1

22,

11

1,1

nn

nn

nn

nn

nn

bx

ax

ax

a

bx

ax

ax

a

bx

ax

ax

a

Nonsingular C

ase

If the coefficient matrix A

is nonsingular, then it is invertible and w

e can solve Ax

= b

as follows:

This solution is therefore unique. A

lso, if b=

0, it follows

that the unique solution to Ax

= 0

is x=

A-10

= 0.

Thus if A

is nonsingular, then the only solution to Ax

= 0

is the trivial solution x

= 0.

bA

xb

AIx

bA

Ax

Ab

Ax

11

11

Exam

ple 1: Nonsingular C

ase (1 of 3)

From

a previous example, w

e know that the m

atrix Abelow

is nonsingular w

ith inverse as given.

Using the definition of m

atrix multiplication, it follow

s that the only solution of A

x=

0is x

= 0:

4/1

4/3

4/1

4/1

4/7

4/5

4/1

4/5

4/3

,

11

2

21

1

32

11

AA

0 0 0

0 0 0

4/1

4/3

4/1

4/1

4/7

4/5

4/1

4/5

4/3

10A

x

Exam

ple 1: Nonsingular C

ase (2 of 3)

Now

let’s solve the nonhomogeneous linear system

Ax

= b

below using A

-1:

This system

of equations can be written as A

x=

b, where

Then

08

34

23

01

22

0

32

1

32

1

32

1

xx

x

xx

x

xx

x

1 1 2

4 5 7

4/1

4/3

4/1

4/1

4/7

4/5

4/1

4/5

4/3

1bA

x

4 5 7

,,

11

2

21

1

32

1

3 2 1

bx

A

x x x

Exam

ple 1: Nonsingular C

ase (3 of 3)

Alternatively, w

e could solve the nonhomogeneous linear

system A

x=

bbelow

using row reduction.

To do so, form

the augmented m

atrix (A|b

) and reduce, using elem

entary row operations.

1 1 2

1 2

73

2

11

00

21

10

73

21

44

00

21

10

73

21

107

30

21

10

73

21

107

30

21

10

73

21

41

12

52

11

73

21

3 32

32

1

x

bA

x xx

xx

x 42

52

73

2

32

1

32

1

32

1

xx

x

xx

x

xx

x

Singular C

ase

If the coefficient matrix A

is singular, then A-1

does not exist, and either a solution to A

x=

bdoes not exist, or there

is more than one solution (not unique).

Further, the hom

ogeneous system A

x=

0has m

ore than one solution. T

hat is, in addition to the trivial solution x=

0, there

are infinitely many nontrivial solutions.

The nonhom

ogeneous case Ax

= b

has no solution unless (b

, y) = 0, for all vectors y

satisfying A*y

= 0, w

here A*

is the adjoint of A

.

In this case, Ax

= b

has solutions (infinitely many), each of

the form x

= x

(0)+ , w

here x(0)is a particular solution of

Ax

= b

, and is any solution of A

x=

0.

Exam

ple 2: Singular C

ase (1 of 2)

Solve the nonhom

ogeneous linear system A

x=

bbelow

using row

reduction. Observe that the coefficients are nearly the sam

e as in the previous exam

ple

We w

ill form the augm

ented matrix (A

|b) and use som

e of the steps in E

xample 1 to transform

the matrix m

ore quickly

03

30

32

30

00

11

0

32

1

31

2

21

1

32

1

32

1

32

1

21

32

13

21

32

1

21

1

3 2 1

bb

b

bb

b

bb

xx

bx

xx

bb

b

bb

b

b b b

bA

33

21

23

21

13

21

32

2 32

bx

xx

bx

xx

bx

xx

Exam

ple 2: Singular C

ase (2 of 2)

From

the previous slide, if , there is no solution

to the system of equations

Requiring that

, assume, for exam

ple, that

Then the reduced augm

ented matrix (A

|b) becom

es:

It can be shown that the second term

in xis a solution of the

nonhomogeneous equation and that the first term

is the most

general solution of the homogeneous equation, letting

, w

here αis arbitrary

0 3 4

1 1 1

3 4

00

3 23

2

30

00

11

0

32

1

3

3

3 3

32

32

1

32

1

21

1

x

x

x x

xx

xx

x

bb

b

bb

b

xx 0

33

21

b

bb

5,1

,23

21

bb

b0

33

21

b

bb

33

21

23

21

13

21

32

2 32

bx

xx

bx

xx

bx

xx

3

x

Linear D

ependence and Independence

A set of vectors x

(1), x(2),…

, x(n)is lin

early dep

end

ent

if there exists scalars c

1 , c2 ,…

, cn , not all zero, such that

If the only solution of

is c1 =

c2

= …

= c

n=

0, then x(1), x

(2),…, x

(n)is linearly

ind

epen

den

t.

0x

xx

)(

)2(

2)1

(1

nn

cc

c

0x

xx

)(

)2(

2)1

(1

nn

cc

c

Exam

ple 3: Linear D

ependence (1 of 2)

Determ

ine whether the follow

ing vectors are linear dependent or linearly independent.

We need to solve

or

0 0 0

113

1

11

2

42

1

0 0 0

11 1 4

3 1 2

1 2 1

3 2 1

21

c c c

cc

c

0x

xx

)3

(3

)2(

2)1

(1

cc

c

11 1 4

,

3 1 2

,

1 2 1)3

()

2()1

(x

xx

Exam

ple 3: Linear D

ependence (2 of 2)

We can reduce the augm

ented matrix (A

|b), as before.

So, the vectors are linearly dependent:

Alternatively, w

e could show that the follow

ing determinant is zero:

number

any

becan

w

here

1 3 2

00

03

04

2

00

00

03

10

04

21

015

50

09

30

04

21

011

31

01

12

04

21

33

32

32

1

cc

cc

cc

c

c

bA

11 1 4

,

3 1 2

,

1 2 1)3

()

2()1

(x

xx

0

113

1

11

2

42

1

)det(

ijx

0x

xx

)3

()2

()1

(3

32,

1ifc

Linear Independence and Invertibility

Consider the previous tw

o examples:

The first m

atrix was know

n to be nonsingular, and its column vectors

were linearly independent.

The second m

atrix was know

n to be singular, and its column vectors

were linearly dependent.

This is true in general: the colum

ns (or rows) of A

are linearly independent iff A

is nonsingular iff A-1

exists.

Also, A

is nonsingular iff detA

0, hence columns (or row

s) of A

are linearly independent iff detA

0.

Further, if A

= B

C, then det(C

) = det(A

)det(B). T

hus if the colum

ns (or rows) of A

and Bare linearly independent, then

the columns (or row

s) of Care also.

Linear D

ependence & V

ector Functions

Now

consider vector functions x(1)(t), x

(2)(t),…, x

(n)(t), where

As before, x

(1)(t), x(2)(t),…

, x(n)(t) is lin

early dep

end

ent

on Iif

there exists scalars c1 , c

2 ,…, c

n , not all zero, such that

Otherw

ise x(1)(t), x

(2)(t),…, x

(n)(t) is linearly in

dep

end

ent

on I

See text for m

ore discussion on this.

,

,,

,2,1

,

)(

)(

)(

)(

)(

)(2

)(1

I

tn

k

tx

tx

tx

t

km k k

k

x

It

tc

tc

tc

nn

allfor

,)

()

()

()

()2

(2

)1(

10

xx

x

Eigenvalues and E

igenvectors

The eqn. A

x=

ycan be view

ed as a linear transformation

that maps (or transform

s) xinto a new

vector y.

Nonzero vectors x

that transform into m

ultiples of them

selves are important in m

any applications.

Thus w

e solve Ax

= x

or equivalently, (A-I)x

= 0.

This equation has a nonzero solution if w

e choose such

that det( A-I) =

0.

Such values of

are called eigenvalu

esof A

, and the nonzero solutions x

are called eigenvectors.

Exam

ple 4: Eigenvalues (1 of 3)

Find the eigenvalues and eigenvectors of the m

atrix A.

Solution: C

hoose such that det(A

-I) = 0, as follow

s.

24

13

A

1

,2

12

2

41

23

24

13

det

10

01

24

13

detdet

2

IA

Exam

ple 4: First E

igenvector (2 of 3)

To find the eigenvectors of the m

atrix A, w

e need to solve ( A

-I)x=

0 for =

2 and =

-1.

Eigenvector for

= 2: S

olve

and this implies that

. So

0 0

44

11

0 0

22

4

12

3

2 1

2 1

x x

x x0

xI

A

1 1

choose

arbitrary,

1 1)1

(

2 2)1

(x

xc

cx x

21

xx

Exam

ple 4: Second E

igenvector (3 of 3)

Eigenvector for

= -1: S

olve

and this implies that

. So

0 0

14

14

0 0

12

4

11

3

2 1

2 1

x x

x x0

xI

A

4 1

choosearbitrary

,4 1

4)2

(

1 1)2

(x

xc

cx x

12

4xx

Norm

alized Eigenvectors

From

the previous example, w

e see that eigenvectors are determ

ined up to a nonzero multiplicative constant.

If this constant is specified in some particular w

ay, then the eigenvector is said to be n

ormalized

.

For exam

ple, eigenvectors are sometim

es normalized by

choosing the constant so that || x|| = (x, x) ½

= 1.

Algebraic and G

eometric M

ultiplicity

In finding the eigenvalues of an n

x nm

atrix A, w

e solvedet( A

-I) = 0.

Since this involves finding the determ

inant of an nx n

matrix, the problem

reduces to finding roots of an nth degree polynom

ial.

Denote these roots, or eigenvalues, by

1 , 2 , …

, n .

If an eigenvalue is repeated mtim

es, then its algebraic

mu

ltiplicity

is m.

Each eigenvalue has at least one eigenvector, and a

eigenvalue of algebraic multiplicity m

may have q

linearly independent eigevectors, 1

q m

, and qis called the

geometric m

ultip

licityof the eigenvalue.

Eigenvectors and L

inear Independence

If an eigenvalue has algebraic m

ultiplicity 1, then it is said to be sim

ple, and the geom

etric multiplicity is 1 also.

If each eigenvalue of an nx n

matrix A

is simple, then A

has ndistinct eigenvalues. It can be show

n that the neigenvectors corresponding to these eigenvalues are linearly independent.

If an eigenvalue has one or more repeated eigenvalues, then

there may be few

er than nlinearly independent eigenvectors

since for each repeated eigenvalue, we m

ay have q < m

. T

his may lead to com

plications in solving systems of

differential equations.

Exam

ple 5: Eigenvalues (1 of 5)

Find the eigenvalues and eigenvectors of the m

atrix A.

Solution: C

hoose such that det(A

-I) = 0, as follow

s.

01

1

10

1

11

0

A

1,1

,2

)1)(

2(

23

11

11

11

detdet

22

1

2

3

IA

Exam

ple 5: First E

igenvector (2 of 5)

Eigenvector for

= 2: S

olve (A-I)x

= 0, as follow

s.

1 1 1

choose

arbitrary,

1 1 1

00

01

1

01

1

00

00

01

10

01

01

00

00

01

10

02

11

03

30

03

30

02

11

01

12

01

21

02

11

02

11

01

21

01

12

)1(

3 3 3)1

(

3 32

31

xx

cc

x x x

x xx

xx

Exam

ple 5: 2nd

and 3rd

Eigenvectors (3 of 5)

Eigenvector for

= -1: S

olve (A-I)x

= 0, as follow

s.

1 1 0

,

1 0 1

choose

arbitrary,

where

,

1 0 1

0 1 1

00

00

01

11

00

00

00

00

01

11

01

11

01

11

01

11

)3(

)2(

32

32

3 2

32

)2(

3

2

32

1

xx

xx

xx

x

x x

xx

x

x

xx

x

Exam

ple 5: Eigenvectors of A

(4 of 5)

Thus three eigenvectors of A

are

where x

(2), x(3)correspond to the double eigenvalue

= -

1.

It can be shown that x

(1), x(2), x

(3)arelinearly independent.

Hence A

is a 3 x 3 symm

etric matrix

(A=

AT

) with 3 real

eigenvalues and 3 linearly independent eigenvectors.

1 1 0

,

1 0 1

,

1 1 1)3

()2

()1

(x

xx

01

1

10

1

11

0

A

Exam

ple 5: Eigenvectors of A

(5 of 5)

Note that w

e could have we had chosen

Then the eigenvectors are orthogonal, since

Thus A

is a 3 x 3 symm

etric matrix

with 3 real eigenvalues

and 3 linearly independent orthogonal eigenvectors.

1 2 1

,

1 0 1

,

1 1 1)3

()2

()1

(x

xx

0

,,0

,,0

,)3

()2

()3

()1

()2

()1

(

xx

xx

xx

Herm

itian Matrices

A self-ad

joint, or H

ermitian

matrix, satisfies A

= A

*, w

here we recall that A

*=

AT

.

Thus for a H

ermitian m

atrix, aij =

aji .

Note that if A

has real entries and is symm

etric (see last exam

ple), then Ais H

ermitian.

An n

x nH

ermitian m

atrix Ahas the follow

ing properties:A

ll eigenvalues of Aare real.

There exists a full set of n

linearly independent eigenvectors of A.

If x(1)and x

(2)areeigenvectors that correspond to different

eigenvalues of A, then x

(1)and x(2)are

orthogonal.

Corresponding to an eigenvalue of algebraic m

ultiplicity m, it is

possible to choose mm

utually orthogonal eigenvectors, and hence A

has a full set of nlinearly independent orthogonal eigenvectors.