WEEK 7 SYSTEM OF EQUATIONS SYSTEM OF LINEAR EQUATIONS IN THREE VARIABLES.
System of Linear Equations
Transcript of System of Linear Equations
System of Linear EquationsDefinitions :
System of Linear Equations
1. Augmented Matrix : M = [A b]
2. Degenerate Equation : All coefficients aLj = 0 for equation “L”
Theorem :
Consider a system of linear equations that contains a degeneratey q g
equation L, say with constant b:
(i) If b = 0 => system has no solution (why ?)
(ii) If b = 0 => equation “L” can be deleted (why ?)
Equivalent Systems of EquationsEquivalent Systems of EquationsObservation : multiplying both sides from left by a matrix T ychanges A & b but x is not changed !
How to Choose T ?Theorem :
How to Choose T ?
Two systems of linear equations are said to be equivalent (have same solution) if and only if each equation in one system is obtained by applying elementary operations to the equations in the other systemapplying elementary operations to the equations in the other system (as in multiplying by matrix T from the left)
Definition :
Elementary operations include
(i) Interchanging 2 equations Ri Rj(ii) Replace equation by a non-zero multiple of other equation plus itself
jji RRKR
Elementary OperationsElementary Operations
MATLAB f(A) l f iMATLAB : rref(A) ; see also rrefmovie
Idea : Apply elementary operations to original system of linear
equations to transform it to another system whose solution can be
easily obtained by back substitution. This transformation
is called Gaussian Elimination
Example 1(Unique Solution)Example 1(Unique Solution)
2 428342623623
221
RRRzyzyxzyxzyx
313235863 331 RRRzyzyx
2314742
623
RRRzzyzyx
ofsystemoriginalinsolutionsubstitutealways(1;3;2
23 147 332
xyzRRRz
)!answer!your check toequationsofsystemoriginalin solution substitute always(
Example 2 (Infinite Solutions)
xxxxx522
152462 54321
system of equations in
echelon form
xxxxx
first)(except equation each in unknown Leading693
522
54
543
whileablespivot vari called are } x, x, x{ equation preceding in theunknown leading theofright the tois
)( pqg
431
byedparametrizsolutionsofnumber infinitean has equations of system thisHence, any value.
assumecan i.e. ; variablesfree called are } x,{x 52
tttxtxtxs
813/)96(225 ; 3/)96(numbers realany are t and s re whe ; x
by edparametriz solutions of
34
52
ststttx 3942/)6)81()3/)96(4(215()(;)(
1
34
Gaussian Elimination ProcedureGaussian Elimination Procedure
Find the first unknown in the system with a non-zero coefficient
(i) Arrange so that a11 is not equal to 0 (by interchanging equations)
(ii) Use a11 as the “pivot” to eliminate X1 from all equations except the first, i.e. multiply by m = -ai1/a11 ii RRmR 1p y y i1 11
to avoid fractions use iii
ii
RRaRa 1111
1
(i) Repeat elimination step for the other pivots to put system in echelon form where (1) the non-zero rows lie above any zero rows and (2) the first non-zero entry (pivot) in a non-zero row lies to the ( ) y (p )right of the first non-zero entry (pivot) in the row above it
Echelon Form
formthehasformechelon in equationsofsystemslinear A :Definition
qy
121222 2222
11212111
nnjjjj
nn
bxaxaxabxaxaxa
11
rnrnjrjjrj bxaxaxarrrr
ablespivot vari called are x, , x, x variablesThen r and zeronot are sa' theand j j 1 where
r2 jj1
r2
i blf)(tlbitiequations than unknowns more n r If (ii)
solution unique n r If (i)
variablesfreer)-(n tovaluesarbitrary assign
Rank of a MatrixRank of a Matrix
D fi iti Th k f t i A itt k(A) i l t thDefinition : The rank of a matrix A, written rank(A), is equal to the
number of pivots in an echelon form of A or equivalently the number of non-zero rows in an echelon form for A
Facts : 1) Reducing the augmented matrix M = [A b] to echelon form
solves the system of equations Ax = b by back-substitution
2) For an m x n matrix A;
nrank(A) and mrank(A) n)min(m,rank(A)0 Hence
Classifying Solutionsy g
• Theorem : Let M=[A b] be the m x (n+1)Theorem : Let M [A b] be the m x (n 1) augmented matrix of an m x n system of linear equations (S), then :q ( )
1) If rank (A) < rank (M) , then S has no solutions2) If rank (A)=rank (M)=n, S has unique solution2) If rank (A) rank (M) n, S has unique solution3) If rank (A)=rank (M)<n, than S has infinitely
many solutionsmany solutions
Remark : note that )()(0 MrankArank Remark : note that )()(0 MrankArank
Example 1
rank(M)rank(A) :1 Case] [
bAM
2,4
( )( )
nm
21
21
01
21
21
01
11
11
11
312
002
000
~
332
022
000
~
651
341
321
M
3rank(M)2rank(A)
300300633
solution No nt!Inconsiste 3rank(M)2rank(A)
Example 2
2n 4,mnrank(M)rank(A) :2 Case
001111
,
01
01
00
~41
41
21
M
000333
0x 1;x :solution Unique nrank(M)2rank(A)
12
12
Example 3
3n 3,m
nrank(M)rank(A) :3 Case
330011421
,
M 01
013
01
00~
31
65
125
52
n 32rank(M)rank(A)
We have 1 free variable
tt
n
303 x;131 xt; xby edparametriz solutions Infinite
32rank(M)rank(A)
123
;; 123
Homogeneous System of EquationsHomogeneous System of Equations
A 0 i i l f A b P t it i h l f thAx=0 is a special case of Ax=b. Put it in echelon form, then
1) If r = n (full rank), system has only the zero solution (unique sol.)) ( ), y y ( q )2) If r < n (rank-deficient), system has infinite solutions because
we have free variables
Question : Can a homogeneous system have no solution ? Why ?
Note that when a homogeneous system of equations has more unknowns than equations, then there are infinite solutions while qif the number of equations equals the number of unknowns, either we have infinite solutions or only the zero solution
Example 1p
More equations than unknowns (over-determined)q ( )
Question : for a homogeneous system, do we need to apply the elementary Operation to the “b” vector as well ? Why ?
Example 2
02.04.0 21 xx 02.04.0 21 xx
00
5.01
2.04.02.04.0
A
11
221
25 RR
RRR
variablefree 1r-n2;n1;rank(A)r002.04.0
2
numberrealaiswhere50;xby edparametriz solutions Infinite
ttxt numberreala is where5.0;x 12 ttxt
Application : Electrical Networks Analysis Kirchhoff’s Laws : (1) All currents flowing into a junction must flow out of it (2) Sum of (current x resistance) terms around a closed path equals total voltage
Application : Forces on a Trussnode 3
13F23F
3W
node 3
# Vector sum of forces at each node is equal to 0
# D fi f t b iti if th
1HF
node 1node 2
# Define forces to be positive if they- act to the right- act in an upward direction
1V 2V12F# acts from node i to node j
0i1
FVNode
20cos0sin
13121
131
NodeFFHFV
:FFFHVV
unknown
30cos0sin
2312
232
NodeFFFV 231312121 ,,,,, FFFHVV
180coscos
sinsin3
2313
32313
FFWFF
Node
Example : Forces on a Truss
00
0cos10100sin0001
1
1
HV
• Write in Ax=b form
000
cos01000sin00100
0cos1010
12
2
1
FVH
bAx
0coscos0000sinsin0000 3
23
13 WFF
• if :
002/10001 1V
100,3/,6/ 3 W
000
2/1010002/300100
02/31010
12
2
1
FVH
bAx
19
0100
0
2/12/300002/32/10000
2/101000
23
13
12
FFF
Example : Forces on a Truss• Let’s use MATLAB for two different values for W3
00
00
000
000
21 bandb
075
0100
0000
02/3101002/10001
2,11,1
2,11,1
HHVV
0000
2/1010002/300100
2,121,12
2,21,2
FFVV
bAx
20
0075100
2/12/300002/32/10000
2,231,23
2,131,13
FFFF
Example : Forces on a Trussx =% Applying Gaussian Eliminiation
function to Truss
sa = sin(pi/6); ca = cos(pi/6);
sb = sin(pi/3); cb = cos(pi/3);
x =
25.0000 18.750086.6025 64.951975.0000 56.2500sb = sin(pi/3); cb = cos(pi/3);
A = [ 1 0 0 0 sa 0
0 1 0 1 ca 0
0 0 1 0 0 sb
75.0000 56.2500-43.3013 -32.4760-50.0000 -37.5000-86.6025 -64.9519
0 0 1 0 0 sb
0 0 0 1 0 -cb
0 0 0 0 -sa -sb
0 0 0 0 -ca cb ];
>>
Q i h db = [ 0 0
0 0
0 0
0 0
RUNQuestion : what does a negative force value mean ?
100 75
0 0 ];
x = linsolve(A,b);
21display(x);
Matrix InversionDefinition : AA-1 = A-1A = I Unique MATLAB : inv(A)Definition : AA = A A = I Unique
proof (by contradiction): suppose both B and C are inverses of A
Consider : B(AC) = (BA)C BI = IC B = C
MATLAB : inv(A)
Consider : B(AC) (BA)C BI IC B C
1) Inverse only defined for square matrices nxn
2) Inverse exists iff elimination produces “n” pivots (rank = n)
3) Consider Ax = b, if A is invertible x = A-1b
Special Case : if b = 0 (homogenous) only zero solution4) Applying elimination to [A I] to reduce it to [T.A T.I]=[I A-1] gives inverse
5) Inverse of a
bd
bddba 1
1
Both forward and backward li i ti d d2x2 matrix
adbcb
adbcdababa
acbcaddc
00101f
elimination needed
adbca
adbcc
adbcadbc
cab
caddc 10
~10
~10
:Proof
Matrix Inverse (Cont’d)( )
6) Inverse of a diagonal matrix is diagonal : A = diag(di) A-1 = diag((di)-1)
(hint : start from fact that product of diagonal matrices is a diagonal matrix)
7) Inverse of lower (upper) triangular is lower (upper) triangular (why ?)
8) (AB)-1 = B-1A-1 prove it using definition of inverse !
9) (A-1)T = (AT)-1 prove it using definition of inverse !
10) AAAAp ....... (p times) matrix powerqpqp AAA pqqp AA )( TppT AA )()( AAA AA )( AA )()(
Example on Calculating Inverse
0121100 0 1 2 0 1
0103120 0 1 2 0 1
2212 RRR
1 0 4- 0 1 00 1 2- 1- 1- 0 ~
1 0 0 8 1 40 1 0 3 1- 2
3314 RRR
1-0401-02 2 11- 0 0 1
~012-1-1-00 0 1 2 0 1
232 RRR
1- 1- 6 1 0 0
1- 0 4 0 1- 0 ~1 1 6- 1- 0 00 1 2- 1- 1- 0
1132 RRR
1 0 4- 0 1 02 2 11- 0 0 1
~
(3 pivots Inverse exists) -R2 R2
1- 1- 6 1 0 0
A-1Always multiply AA-1 to check your answer!
Matrix Inversion Application : Decoding Digital Signals in Cell PhoneDigital Signals in Cell Phone
In cellular communications, digital data is transmitted in blocks. Each block received by your cell phone (after sampling and digitization) can be representedreceived by your cell phone (after sampling and digitization) can be representedmathematically in the form
HXY
Z
+where X is the transmitted blocks, H a matrix representing the effects of the multipath distortion of the channel (attenuation, reflections, scattering) on the p ( g)transmitted signal, and Z represents additive noise (e.g. due to cell phone analog front-end circuit imperfections, interference from other cell towers).
The transmitted signal can be decoded by the following matrix operation
Triangular/Cholesky/LU Factorizationg yMATLAB : [L,U] = lu(A)
Theorem: Suppose A is a non-singular (i.e. invertible with non-zero pivots) matrix that is reduced to upper-triangular form using Gaussian elimination, i.e.,
A ~ U TA=U A=inv(T).U then we can factor it as A = L U
where L is a lower triangular matrix with ones on the diagonal
U is upper triangular with the pivots on the main diagonal
Remark :
Elementary operations
(…..E32 E31 E21) A = U A = (E21-1 E31
-1 E32-1) U=LU
Elementary MatricesDefinition :
Let “e” be an elementary row operation. The elementary matrix E corresponding to “e” is constructed by applying “e” to thecorresponding to e is constructed by applying e to the
Identity matrix and is denoted E = e(I), where I is the identity matrix
Examples : 1 0 00 0 1
(i) R2 R3E = 0 0 1
0 1 0
E1 0 00 6 0
Remark : except for rows interchanges, all other elementary matrices are lower triangular
(ii) R2 -6R2
(iii)R3 -4R1 + R3
E = 0 -6 00 0 1
E =1 0 00 1 0( ) 3 1 3 E = 0 1 0-4 0 1
0150 0 1
0150 0 1 1
Remark : Inverse of elimination
1 0 0
0 1 51 0 0 0 1 5-Remark : Inverse of elimination
matrix is easy to compute !
LU Factorization (Cont’d)1) L contains the multipliers that take A to U in the elimination procedure
2) The elimination multipliers (with sign inversion) are below diagonal of L2) The elimination multipliers (with sign inversion) are below diagonal of L
3) If A is symmetric then U = LT (why ?)
4) All elementary matrices (except for interchange of 2 rows) are lower triangular and so are their inverses and products. Since we are considering invertible matrices (non-zero pivots), there is no need for row interchanges
Application : Solving System of Equations Using LU Factorization :
bLUxbAx cUx
bLUxbAx bLc
(1 System of Linear
(2 triangular systems)= Solve for c
firstEquations) first
Example on LU FactorizationExample on LU Factorization
321
ionfactorizatULfind
5121343A
700420321
~130420321
~A331
221
2
3
RRR
RRR
33223 RRR
700130
Exercise : write down the elementary matrices E1, E2, and E3
210321
020001
420321
;013001
UL
1007007001232
Always check by multiplying L U to get A
Application : Fitting Curvespp cat o tt g Cu es
equation The
011-n
1-nn
n
points m the through passes It curve. polynomial a defines yaxa...xaxa
1nn
mmm222111
: consistent is equations of system following the iff )y,(xP ..., , )y,(xP, )y,(xP :by given
20211-n
21-nn
2n
10111-n
11-nn
1n
yaxa...xaxa
yaxa...xaxa
m0m11-n
m1-nn
mn yaxa...xaxa
... ... ...
Application from : Fitting CurvesApplication from : Fitting Curves
:is 2))(nm size (of M matrix augmented The
y1xxxy1xxx
AM
:is 2))(nm size (of M matrix augmented The
221-n
2n2
111-n
1n1
hfhy1xxx
yy AM
mm1-n
mnm
2222
solution unique mrank(A) 1nm If ii)solutions infinite mrank(A) 1nm If i)
then j,i xx If :Theorem ji
solution unique 1nrank(A)solution no rank(M)rank(A)
1nm If iii)
solution unique mrank(A) 1nm If ii)
There are (n+1) unknowns and rank(A)=# of pivots
Fitting Curves ExampleFitting Curves Example
(1,2)P, (0,1)P 21
1n32nm distinct. values xparabola yaxaxap(x) 01
22
11
100011
~ 21
111100
M
t1avariable), (free ta
1,a
1
0
y 1txt)x(1 axaxap(x)
t1a
201
22
2
y 1txt)x(1
Application : Car Rentalspp cat o Ca e ta s
(D) Downtown (A), Airport :locations 3 has agency rentalcar A
is month each during cars of tionredistribu Thelocations. 3 the atcars 400 & 600,500 were there operation initial .At(S) Suburb &(D) Downtown (A), Airport :locations 3 has agency rentalcar A
:diagram following the by described
0.1A D
0.1
0.10.20.1 0.1
0.70.6
S
0.6
Application : Car RentalsApplication : Car Rentals
:tionInterpreta
been has 20% remaining TheS to returned & rented 10%another and D to returned & rented 10% A). to returned & rentedor
rented not(either A at still are A at cars of 60% end,-month At
service. from retired been has 20% remaining TheS. to returned & rented 10%another
500600
da
u & da
u
k. month each of beginning at cars of no. denote s &d,a Let
0
0
0k
k
k
kkk
400ss 0
00
k
kk
Application : Car RentalsApplication : Car Rentals
a0 20 10 6a
sda
0.60.10.10.10.70.10.20.10.6
sda
k
k
k
1k
1k
1k
A AA A k2
u
k
Au
1k
k1k
0 950.87
u e g
uAu uAAuu ,Auu 0k
k 02
1201
(why?) months 44after left cars no
0.690.95u e.g. 44
( y )
2 Equations in 2 Unknowns2 Equations in 2 Unknowns1212111
bXaXabXaXa
(Each
equation Solution Unique(i) Case2222121 bXaXa represents
a line)
22
12
21
11 SlopesDifferent
q( )
aa
aa
Solution21122211
S l tiN(ii)C
0 :Condition aaaa Solution
This is the determinant of matrix A !
2
1
22
12
21
11 :Condition
Solution No (ii) Case
bb
aa
aa
Parallel Lines
No Intersection
22221
coincide lines 2 Solutions ofNumber Infinite (iii) Caseinterceptsdifferent but Slope Same
2
1
22
12
21
11 :Condition bb
aa
aa
Exercise : Use Gaussian elimination to derive these 3 conditions
Review QuestionsReview Questions
1) If A is a diagonal matrix then An is also diagonal for any integer n (T)
True or False
1) If A is a diagonal matrix, then An is also diagonal for any integer n (T)
2) If A & B are invertible then AB is invertible. (T)
Answer the following short questionsAnswer the following short questions
1) Let A be a 4x4 matrix with , then the (2,3) element of A2 is equal to
2
|| jiaij
[1 0 1 2] = 2 + 2 = 42101
2) Let A & B be 4x4 matrices with trace(B)=2
Then trace(A 1BTA)
1
Then, trace(A-1BTA) = ……
Review Problem
32 3121
Gaussian elimination procedure applied directly to the augmented matrix
523452
32
zyxzyx
zyx
51-2-34- 1- 5 23 1 2 1
M523 zyx 5 1 2 3
103103 1 2 13313 RRR
103103 1 2 1
4- 4- 8- 010- 3- 1 0
2212 RRR
84- 28- 0 010- 3- 1 0
3328 RRR
110910338428
yyzyzz
233232 xxzyx