Otto Bretscher
Linear Algebra with ApplicationsFourth Edition
Otto BretscherColby College
Prentice Hallis an imprint of
Upper Saddle River, New Jersey 07458
Library of Congress Cataloging-in-Publication DataBretscher, Otto.
Linear algebra with applications / Otto Bretscher.—4th ed.p. cm.
Includes index.ISBN 978-0-13-600926-9
1. Algebras, Linear—Textbooks. I. Title.QA184.2.B73 2009512'.5-dc22 2008028930
Editor-in-Chief, Collegiate Mathematics: Deirdre LynchSenior Acquisitions Editor: William HoffmanAssociate Editor/Print Supplements: Caroline CelanoProject Manager: Raegan Keida HeeremaProduction Coordination/Composition: ICC Macmillan Inc.Associate Managing Editor: Bayani Mendoza deLeonSenior Managing Editor: Linda Mihatov BehrensSenior Operations Supervisor: Alan FischerOperations Specialist: Lisa McDowellExecutive Marketing Manager: Jeff WeidenaarMarketing Assistant: Jon ConnellyArt Director: Maureen EideInterior Designer: Dina CurroCover Designer: Daniel ConteAV Project Manager: Thomas BenfattiManager, Cover Visual Research & Permissions: Karen SanatarArt Studio: Laserwords Private LimitedCover Image Credit: Andy BurmfielaVIstockphoto.comThe cover is a close-up image of a guitar, showing four vibrating strings and the sound hole.Using Fourier analysis, we can measure the amplitudes of a plucked string's harmonics or evensynthesize the guitar's sound. See pages 239-244.
Prentice Hallis an imprint of
PEARSON© 2009, 2005, 2001, 1997 Pearson Education, Inc.Pearson Prentice HallPearson Education, Inc.Upper Saddle River, NJ 07458
All rights reserved. No part of this book maybe reproduced, in any form or by any means,without permission in writing from the publisher.
Pearson Prentice Hall® is a trademark of Pearson Education, Inc.
Printed in the United States of America1 0 9 8 7 6 5 4 3 7 I
I S B N - 1 0 : 0 - 1 3 - 6 0 0 9 2 6 - 3ISBN-13 : 978-0 -13-600926-9
Pearson Education Ltd., LondonPearson Education Singapore, Pte. LtdPearson Education Canada, Inc.Pearson Education-JapanPearson Education Australia PTY, Limited
Pearson Education North Asia, Ltd., Hong KongPearson Education de Mexico, S.A. de C.V.Pearson Education Malaysia, Pte. Ltd.Pearson Education Upper Saddle River, New Jersey
To my parentsOtto and Margrit Bretscher-Zwicky
with love and gratitude
ContentsText Features ix
Preface xi
1 L i n e a r E q u a t i o n s 11 . 1 I n t r o d u c t i o n t o L i n e a r S y s t e m s 11.2 Matrices, Vectors, and Gauss-Jordan Elimination 81.3 On the Solutions of Linear Systems; Matrix Algebra 25
2 L i n e a r T r a n s f o r m a t i o n s 4 02.1 Introduction to Linear Transformations and
T h e i r I n v e r s e s 4 02 . 2 L i n e a r T r a n s f o r m a t i o n s i n G e o m e t r y 5 42 . 3 M a t r i x P r o d u c t s 6 92.4 The Inverse of a Linear Transformation 79
3 Subspaces of R" and Their Dimensions 1013.1 Image and Kernel of a Linear Transformation 1013.2 Subspaces of R"; Bases and Linear Independence 1133.3 The Dimens ion o f a Subspace o f R" 1233 . 4 C o o r d i n a t e s 1 3 7
4 L i n e a r S p a c e s 1 5 34 . 1 I n t r o d u c t i o n t o L i n e a r S p a c e s 1 5 34.2 Linear Transformations and Isomorphisms 1654.3 The Matr ix of a Linear Transformation 172
5 Orthogonality and Least Squares 1875.1 Orthogonal Projections and Orthonormal Bases 1875.2 Gram-Schmidt Process and QR Factorization 2035.3 Orthogonal Transformations and Orthogonal Matrices 2105 . 4 L e a s t S q u a r e s a n d D a t a F i t t i n g 2 2 05 . 5 I n n e r P r o d u c t S p a c e s 2 3 3
vii i Contents
6 D e t e r m i n a n t s 2 4 96 . 1 I n t r o d u c t i o n t o D e t e r m i n a n t s 2 4 96 . 2 P r o p e r t i e s o f t h e D e t e r m i n a n t 2 6 16.3 Geometrical Interpretations of the Determinant;
C r a m e r ' s R u l e 2 7 7
7 E igenva lues and E igenvec tors 2947.1 Dynamical Systems and Eigenvectors:
A n I n t r o d u c t o r y E x a m p l e 2 9 47.2 F ind ing the E igenva lues o f a Mat r i x 3087.3 F inding the Eigenvectors of a Matr ix 3197 . 4 D i a g o n a l i z a t i o n 3 3 27 . 5 C o m p l e x E i g e n v a l u e s 3 4 37 . 6 S t a b i l i t y 3 5 7
8 Symmetric Matrices and Quadratic Forms 3678 . 1 S y m m e t r i c M a t r i c e s 3 6 78 . 2 Q u a d r a t i c F o r m s 3 7 68 . 3 S i n g u l a r V a l u e s 3 8 5
9 L i n e a r D i f f e r e n t i a l E q u a t i o n s 3 9 79.1 An Introduction to Continuous Dynamical Systems 3979 .2 The Complex Case : Eu le r ' s Formu la 4109.3 Linear Differential Operators and
L i n e a r D i f f e r e n t i a l E q u a t i o n s 4 2 3
Appendix A Vectors 437Answers to Odd-Numbered Exercises 446Subject Index 471Name Index 478
SUBJECT INDEX
Affine system, 329Algebraic multiplicity of an eigenvalue, 312
and complex eigenvalues, 352and geometric multiplicity, 326
Algorithm, 18Alternating property of determinant,
251,265Angle, 196
and orthogonal transformations, 211Approximations,
Fourier 232, 239least-squares 17, 220
Argument (of a complex number), 346of a product, 348
Associative law for matrix multiplication, 75Asymptotically stable equilibrium, 358
isee stable equilibrium)Augmented matrix, 11
BBasis, 116, 158
and coordinates, 139and dimension, 125, 159and unique representation, 120finding basis of a linear space, 161of an image, 116, 128, 131of a kernel, 127, 131ofRM32standard, 126
Binary digits, 112Block matrices, 75
determinant of, 257inverse of, 87multiplication of, 76
Bounded trajectory, 363
Carbon dating, 407Cardano's formula, 316, 319, 356Cartesian coordinates, 10, 137Cauchy-Schwarz inequality, 195, 237
and angles, 196and triangle inequality, 200
Cayley-Hamilton Theorem, 332Center of mass, 21Change of basis, 177, 179Characteristic polynomial, 311
and algebraic multiplicity, 312
and its derivative, 318of linear differential operator, 426of similar matrices, 326
Chebyshev polynomials, 246Cholesky factorization, 383Circulant matrix, 356Classical adjoint, 287Closed-formula solution
for discrete dynamical system, 303for inverse, 287for least-squares approximation, 224for linear system, 286
Coefficient matrixof a linear system, 11of a linear transformation, 41
Column of a matrix, 9Column space of a matrix, 105Column vector, 10Commuting matrices, 73Complements, 135Complex eigenvalues, 350
and determinant, 353and rotation-scaling matrices, 352,
359,416and stable equilibrium, 359and trace, 353
Complex numbers, 155, 345and rotation-scaling, 348in polar form, 347
Complex-valued functions, 411derivative of, 411exponential, 412
Component of a vector, 10Composite functions, 69Computational complexity, 89, 96Concatenation, 323Conies, 23, 134Consistent linear system, 25Continuous least-squares condition, 238Continuous linear dynamical
system, 400stability of, 415with complex eigenbasis, 416with real eigenbasis, 405
Contraction, 56Coordinates, 139, 158Coordinate transformation, 159, 167Coordinate vector, 139, 159Correlation (coefficient), 198Cramer's Rule, 286
471
472 Subject Index
Cross productand polar (QS) factorization, 394in R3, 52, 443, 445in R", 275rules for, 444
Cubic equation (Cardano's formula),316,356
Cubic splines, 19Cubics, 134Curve fitting
with conies, 23, 134with cubics, 134
Data compression, 393Data fitting, 225
multivariate, 228De Moivre's formula, 348Determinant, 85, 255
alternating property of, 251, 265and characteristic polynomial,
310,311and eigenvalues, 315, 353and Cramer's rule, 286and elementary row operations, 265and invertibility, 266and Laplace expansion, 268, 270and QR factorization, 279as area, 85, 278as expansion factor, 282as volume, 280is linear in rows and columns,
252, 262of inverse, 268of linear transformation, 272of orthogonal matrix, 278of permutation matrix, 260of product, 267of rotation matrix, 278of similar matrices, 267of 3 x 3 matrix, 249of transpose, 262of triangular matrix, 257of 2 x 2 matrix, 85patterns and, 255Vandermonde, 273Weierstrass definition of, 276
Diagonalizable matrices, 333and eigenbases, 333and powers, 336orthogonally, 367, 372simultaneously, 342
Diagonal matrix, 9
Diagonal of a matrix, 9Dilation, 56Dimension, 125, 159
and isomorphism, 168of image, 129, 165of kernel, 129,165of orthogonal complement, 193
Direction field, 401Discrete linear dynamical system, 303
and complex eigenvalues, 360, 362and stable equilibrium, 358, 359
Distance, 236Distributive Laws, 75Domain of a function, 43Dominant eigenvector, 405Dot product, 20, 28, 215, 441
and matrix product, 73, 215and product Ax, 29rules for, 442
Dynamical systemisee continuous, discrete linear
dynamical system)
Eigenbasis, 322and continuous dynamical system, 405and diagonalization, 333and discrete dynamical system, 303and distinct eigenvalues, 324and geometric multiplicity, 324
Eigenfunction, 337, 426Eigenspaces, 320
and geometric multiplicity, 322and principal axes, 381
Eigenvalue(s), 300algebraic multiplicity of, 312and characteristic polynomial, 311and determinant, 315, 353and positive (semi)definite
matrices, 379and QR factorization, 316and singular values, 387and stable equilibrium, 359, 415and trace, 315, 353complex, 350, 362geometric multiplicity of, 322of linear transformation, 337of orthogonal matrix, 302of rotation-scaling matrix, 351of similar matrices, 326of symmetric matrix, 370of triangular matrix, 310power method for finding, 355
Subject Index 473
Eigenvectors, 300and linear independence, 323dominant, 405of symmetric matrix, 369
Elementary matrix, 93Elementary row operations, 16
and determinant, 265and elementary matrices, 93
Ellipse, 69as image of the unit circle, 69, 372, 387as level curve of quadratic form, 381as trajectory, 362, 416
Equilibrium (state), 358Equivalence Relation, 146Error (in least-squares solution), 222Error-correcting codes, 112Euler identities, 240Euler's formula, 413Euler's theorem, 328Exotic operations (in a linear space), 172Expansion factor, 282Exponential functions, 398
complex-valued, 412
Factorizations,Cholesky, 383LDLT, 220, 383LDU, 93, 220LLT, 383LU, 93ßfl, 207, 208, 279, 316QS, 394SOS-1, 336SDST, 367[/EVT,390
Field, 350Finite dimensional linear space, 162Flow line, 401Fourier analysis, 239Function, 43Fundamental theorem of algebra, 349Fundamental theorem of linear
algebra, 129
Gaussian integration, 246Gauss-Jordan elimination, 17
and determinant, 265and inverse, 82
Geometric multiplicity of an eigenvalue, 322and algebraic multiplicity, 326and eigenbases, 324
Golden section, 331Gram-Schmidt process, 206
and determinant, 278and orthogonal diagonalization, 372and QR factorization, 207
HHarmonics, 241Hubert space £2,201,235
and quantum mechanics, 243Homogeneous linear system, 35Hyperbola, 381Hyperplane, 134
IIdentity matrix /„, 45Identity transformation, 45Image of a function, 101Image of a linear transformation,
105, 165and rank, 129is a subspace, 105, 165orthogonal complement of, 221written as a kernel, 111
Imageof the unit circle, 69, 372, 387
Imaginary part of a complex number, 346Implicit function theorem, 276Inconsistent linear system, 5, 17, 25
and least-squares, 222Indefinite matrix, 378Infinite dimensional linear space, 162Inner product (space), 234Input-output analysis, 5, 20, 90, 95, 150Intermediate value theorem, 68Intersection of subspaces, 123
dimension of, 135Invariant Subspace, 307Inverse (of a matrix), 80
and Cramer's rule, 287determinant of, 268of an orthogonal matrix, 215of a product, 83of a transpose, 216of a 2 x 2 matrix, 85of block matrix, 87
Inversion (in a pattern), 255Invertible function, 79Invertible matrix, 80
and determinant, 266and kernel, 109
Isomorphism, 167and dimension, 168
474 Subject Index
KKernel, 106, 165
and invertibility, 109and linear independence, 120and rank, 129dimension of, 129is a subspace, 108, 165
Kyle numbers, 131,320
Laplace expansion, 268, 270LDL1 factorization, 220, 383LDU factorization, 93, 220Leading one, 16Leading variable, 13Least-squares solutions, 17, 201, 222
and normal equation, 223minimal, 230
Left inverse, 122Legendre polynomials, 246Length of a vector, 187, 442
and orthogonal transformations, 210Linear combination, 30, 156
and span, 105Linear dependence, 116Linear differential equation, 161, 423
homogeneous, 423order of, 423solution of, 425, 433solving a first order, 431solving a second order, 428, 429
Linear differential operator, 423characteristic polynomial of, 426eigenfunctions of, 426image of, 432kernel of, 427
Linear independenceand dimension, 126and kernel, 120and relations, 118inR«, 116in a linear space, 158of eigenvectors, 323of orthonormal vectors, 189
Linearity of the determinant, 252, 262Linear relations, 118Linear space(s), 154
basis of, 158dimension of, 159finding basis of, 161finite dimensional, 162isomorphic, 167
Linear systemclosed-formula solution for, 286consistent, 25homogeneous, 35inconsistent, 5, 25least-squares solutions of, 222matrix form of, 32minimal solution of, 230number of solutions of, 25of differential equations: see continuous linear
dynamical systemunique solution of, 27vector form of, 32with fewer equations than unknowns, 27
Linear transformation, 41, 44, 48, 165image of, 105, 165kernel of, 106, 165matrix of, 41, 44, 47, 143, 173
Lower triangular matrix, 9LU factorization, 93
and principal submatrices, 94
MMain diagonal of a matrix, 9Mass-spring system, 422Matrix, 9
(for a composite entry such as "zero matrix"see zero)
Minimal solution of a linearsystem, 230
Minors of a matrix, 269Modulus (of a complex number), 346
of a product, 348Momentum, 21Multiplication (of matrices), 72
and determinant, 267column by column, 72entry by entry, 73is associative, 75is noncommutative, 73of block matrices, 76
NNegative feedback loop, 420Neutral element, 154Nilpotent matrix, 136, 375Norm
of a vector, 187in an inner product space, 236
Normal equation, 223Nullity, 129, 165Null space, 106
Subject Index 475
Orthogonal complement, 193of an image, 221
Orthogonally diagonalizable matrix, 367, 372Orthogonal matrix, 211, 216
determinant of, 278eigenvalues of, 302inverse of, 215transpose of, 215
Orthogonal projection, 58, 189, 191,225, 237
and Gram-Schmidt process, 203and reflection, 59as closest vector, 222matrix of, 217, 225
Orthogonal transformation, 210and orthonormal bases, 217preserves dot product, 218preserves right angles, 211
Orthogonal vectors, 187, 236, 443and Pythagorean theorem, 194
Orthonormal bases, 189, 192and Gram-Schmidt process, 203and orthogonal transformations, 212and symmetric matrices, 368
Orthonormal vectors, 188are linearly independent, 189
Oscillator, 184
Positive (semi)definite matrix, 378and eigenvalues, 379and principal submatrices, 379
Positively oriented basis, 290Power method for finding eigenvalues, 355Principal axes, 381Principal submatrices, 94
and positive definite matrices, 379Product Ax, 29
and dot product, 29and matrix multiplication, 72
Projection, 56, 67isee also orthogonal projection)
Pseudo-inverse, 230Pythagorean theorem, 194, 237
QR factorization, 207and Cholesky factorization, 383and determinant, 279and eigenvalues, 316is unique, 219
QS factorization, 394Quadratic forms, 377
indefinite, 378negative (semi)definite, 378positive (semi)definite, 378principal axes for, 381
Quaternions, 220, 356, 364form a skew field, 356
Parallelepiped, 280Parallel vectors, 440Parametrization (of a curve), 102Partitioned matrices, 76
isee block matrix)Pattern (in a matrix), 255
inversion in, 255signature of, 255
Permutation matrix, 89determinant of, 260
Perpendicular vectors, 187, 236, 443Phase portrait,
of continuous system, 400, 419of discrete system, 304, 419summary, 419
Piecewise continuous function, 239Pivot, 16Polar factorization, 394Polar form (of a complex number), 347
and powers, 348and products, 348with Euler's formula, 413
Rank (of a matrix), 26, 129and image, 129and kernel, 129and row space, 136and singular values, 389of similar matrices, 326of transpose, 216
Rank-nullity theorem, 129, 165Real part of a complex number, 346Reduced row-echelon form (rref), 16
and determinant, 265and inverse, 80, 82and rank, 26
Redundant vector(s), 116, 158image and, 116, 131kernel and, 131
Reflection, 59Regular transition matrix, 317, 354Relations (among vectors), 118Resonance, 184Riemann integral, 232, 234
476 Subject Index
Rotation matrix, 62, 278Rotation-scaling matrix, 63
and complex eigenvalues, 351and complex numbers, 348
Row of a matrix, 9Row space, 136Row vector, 10Rule of 69, 408
Sarrus's rule, 250Scalar, 12Scalar multiples of matrices, 28Scaling, 54Second derivative test, 383Secular equation, 308Separation of variables, 398, 407Shears, 64Signature of a pattern, 255Similar matrices, 145
and characteristic polynomial, 326Simultaneously diagonalizable
matrices, 342Singular value decomposition
(SVD), 390Singular values, 387
and ellipses, 387and rank, 389
Skew field, 356Skew-symmetric matrices, 214, 382
determinant of, 274Smooth function, 157Space of functions, 155Span
inR", 105in a linear space, 159
Spectral theorem, 368Spirals, 344, 349, 362, 413, 416, 419Square matrix, 9Square-summable sequences, 201, 235Stable equilibrium, 358
of continuous system, 415of discrete system, 359
Standard basis, 126, 160State vector, 295Subspace
invariant, 307of R", 113of a linear space, 157
Sudoku,75Sum of subspaces, 123
dimension of, 135Sums of matrices, 28
Symmetric matrices, 214are orthogonally diagonalizable, 368have real eigenvalues, 370
Target space (of a function), 43Tetrahedron, 92, 150, 289Theorem of Pythagoras, 194, 237Trace, 235, 310
and characteristic polynomial, 311and eigenvalues, 315, 353and inner product, 235of similar matrices, 326
Trajectory, 304, 400bounded, 363
Transpose (of a matrix), 214and determinant, 262and inverse, 216and rank, 216of an orthogonal matrix, 215of a product, 216
Triangle inequality, 200Triangular matrices, 9
and determinant, 257and eigenvalues, 310invertible, 89
Triangulizable matrix, 375Trigonometric polynomials, 240
uUnit vector, 187,443Upper triangular matrix, 9
Vandermonde determinant, 273Vector(s), 10
angles between, 196addition, 437column, 10coordinate, 139, 159cross product of, 52, 249, 443dot product of, 441geometric representation of, 439length of, 187,442orthogonal, 187, 443parallel, 440position, 439redundant, 116row, 10rules of vector algebra, 438scalar multiplication, 437standard representation of, 10, 439unit, 187,443
Subject Index 477
velocity, 400vs. points, 10, 439zero, 438
Vector field, 401Vector form of a linear system, 32Vector space, 10, 154
isee linear space)Velocity vector, 400
wWeierstrass definition of determinant, 276Wigner semicircle distribution, 246
XZero matrix, 9Zero vector, 438
NAME INDEXAbel, Niels Henrik, 316Alcuin of York, 24al-Kashi, Jamshid, 24al-Khowarizmi, Mohammed, 1, 18nal-Ma'mun, Caliph, 1Archimedes, 53, 331Argand, Jean Robert, 346n
Banach, Stefan, 163Berlinski, David, 17Bezout, Etienne, 272Binet, Jacques, 272Burke, Kyle, 131
Cajori, Florian, 272Cardano, Gerolamo, 316, 319Cauchy, Augustin-Louis, 195, 255n, 272, 309Cayley, Arthur, 272, 332Chebyshev, Pafnuty, 246Cholesky, Andre-Louis, 383Cramer, Gabriel, 135, 272, 286
d'Alembert, Le Rond, 125, 309Dantzig, Tobias, 345De Moivre, Abraham, 348Descartes, Rene, 118n, 137, 155De Witt, Johan, 378
Einstein, Albert, 356-357Euler, Leonhard Paul, 24, 135, 240, 243, 328,
345, 413
Fermat, Pierre de, 137Fibonacci (Leonardo of Pisa), 330-331Fourier, Jean Baptiste Joseph, 239n
Galois, Evariste, 316Gauss, Carl Friedrich, 17-18, 51, 246, 346n, 350Gram, Jörgen, 206Grassmann, Hermann Günther, 125
Hamilton, William Rowan, 220, 332, 356Hamming, Richard W., 112Harriot, Thomas, 118nHeisenberg, Werner, 201, 244Helmholtz, Hermann von, 309Hubert, David, 162,201,309
Jacobi, Carl Gustav Jacob, 255nJordan, Wilhelm, 17
Kepler, Johannes, 233Kronecker, Leopold, 255n
Lagrange, Joseph Louis, 309, 364Lanchester, Frederick William, 409nLaplace, Pierre-Simon Marquis de, 268n, 272Legendre, Adrien-Marie, 246Leibniz, Gottfried Wilhelm von, 272Leonardo da Vinci, 200Leontief, Wassily, 5, 150
Mahavira, 24Mas-Colell, Andreu, 201Mazur, Barry, 345Medawar, Peter. B., 64Mumford, David, 423
Newton, Isaac, 24
Olbers, Wilhelm, 17
Peano, Giuseppe, 154nPeirce, Benjamin, 413nPiazzi, Giuseppe, 17Pythagoras, 194, 243, 272
Riemann, Bernhard, 232, 234, 246Roosevelt, Franklin D., 90n
Samuelson, Paul E., 364Sarrus, Pierre Frederic, 250nSchläft, Ludwig, 126Schmidt, Erhard, 162, 206nSchrödinger, Erwin, 244Schwarz, Hermann Amandus, 195Seki, Köwa (or Takakazu), 255, 272Simpson, Thomas, 246Strang, Gilbert, 393Strassen, Völker, 96Sylvester, James Joseph, 9n
Thompson, d'Arcy, 64
Vandermonde, Alexandre-Theophile, 255n, 274
Weierstrass, Karl, 273, 276Wessel, Caspar, 346nWeyl, Hermann, 125Wigner, Eugene Paul, 246Wilson, Edward O., 285n
478
Top Related